Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Transformation Driven Visual Reasoning   This paper defines a new visual reasoning paradigm by introducing an
important factor, i.e.~transformation. The motivation comes from the fact that
most existing visual reasoning tasks, such as CLEVR in VQA, are solely defined
to test how well the machine understands the concepts and relations within
static settings, like one image. We argue that this kind of \textbf{state
driven visual reasoning} approach has limitations in reflecting whether the
machine has the ability to infer the dynamics between different states, which
has been shown as important as state-level reasoning for human cognition in
Piaget's theory. To tackle this problem, we propose a novel
\textbf{transformation driven visual reasoning} task. Given both the initial
and final states, the target is to infer the corresponding single-step or
multi-step transformation, represented as a triplet (object, attribute, value)
or a sequence of triplets, respectively. Following this definition, a new
dataset namely TRANCE is constructed on the basis of CLEVR, including three
levels of settings, i.e.~Basic (single-step transformation), Event (multi-step
transformation), and View (multi-step transformation with variant views).
Experimental results show that the state-of-the-art visual reasoning models
perform well on Basic, but are still far from human-level intelligence on Event
and View. We believe the proposed new paradigm will boost the development of
machine visual reasoning. More advanced methods and real data need to be
investigated in this direction. The resource of TVR is available at
https://hongxin2019.github.io/TVR.

3
The categories of this paper are: physics
24
The categories of this paper are: dg-ga
30
The categories of this paper are: q-alg
11
The categories of this paper are: hep-lat
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
13
The categories of this paper are: quant-ph
15
The categories of this paper are: nucl-ex
16
The categories of this paper are: stat
29
The categories of this paper are: funct-an
27
The categories of this paper are: comp-gas
2
The categories of this paper are: math
17
The categories of this paper are: q-fin
4
The categories of this paper are: cond-mat
33
The categories of this paper are: plasm-ph
14
The categories of this paper are: math-ph