Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Large-Scale Adversarial Training for Vision-and-Language Representation
  Learning   We present VILLA, the first known effort on large-scale adversarial training
for vision-and-language (V+L) representation learning. VILLA consists of two
training stages: (i) task-agnostic adversarial pre-training; followed by (ii)
task-specific adversarial finetuning. Instead of adding adversarial
perturbations on image pixels and textual tokens, we propose to perform
adversarial training in the embedding space of each modality. To enable
large-scale training, we adopt the "free" adversarial training strategy, and
combine it with KL-divergence-based regularization to promote higher invariance
in the embedding space. We apply VILLA to current best-performing V+L models,
and achieve new state of the art on a wide range of tasks, including Visual
Question Answering, Visual Commonsense Reasoning, Image-Text Retrieval,
Referring Expression Comprehension, Visual Entailment, and NLVR2.

34
The categories of this paper are: chem-ph
30
The categories of this paper are: q-alg
29
The categories of this paper are: funct-an
21
The categories of this paper are: adap-org
27
The categories of this paper are: comp-gas
2
The categories of this paper are: math
20
The categories of this paper are: acc-phys
15
The categories of this paper are: nucl-ex
17
The categories of this paper are: q-fin
7
The categories of this paper are: hep-th
28
The categories of this paper are: alg-geom
8
The categories of this paper are: hep-ex
12
The categories of this paper are: nucl-th
23
The categories of this paper are: solv-int
37
The categories of this paper are: supr-con
10
The categories of this paper are: q-bio