Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Adaptive Attention Span in Computer Vision   Recent developments in Transformers for language modeling have opened new
areas of research in computer vision. Results from late 2019 showed vast
performance increases in both object detection and recognition when
convolutions are replaced by local self-attention kernels. Models using local
self-attention kernels were also shown to have less parameters and FLOPS
compared to equivalent architectures that only use convolutions. In this work
we propose a novel method for learning the local self-attention kernel size. We
then compare its performance to fixed-size local attention and convolution
kernels. The code for all our experiments and models is available at
https://github.com/JoeRoussy/adaptive-attention-in-cv

22
The categories of this paper are: chao-dyn
16
The categories of this paper are: stat
28
The categories of this paper are: alg-geom
30
The categories of this paper are: q-alg
9
The categories of this paper are: nlin
32
The categories of this paper are: atom-ph
29
The categories of this paper are: funct-an
13
The categories of this paper are: quant-ph
34
The categories of this paper are: chem-ph
18
The categories of this paper are: econ
31
The categories of this paper are: ao-sci
36
The categories of this paper are: cmp-lg
8
The categories of this paper are: hep-ex
2
The categories of this paper are: math
23
The categories of this paper are: solv-int
6
The categories of this paper are: astro-ph