Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Dense Continuous-Time Optical Flow from Events and Frames   We present a method for estimating dense continuous-time optical flow.
Traditional dense optical flow methods compute the pixel displacement between
two images. Due to missing information, these approaches cannot recover the
pixel trajectories in the blind time between two images. In this work, we show
that it is possible to compute per-pixel, continuous-time optical flow by
additionally using events from an event camera. Events provide temporally
fine-grained information about movement in image space due to their
asynchronous nature and microsecond response time. We leverage these benefits
to predict pixel trajectories densely in continuous-time via parameterized
B\'ezier curves. To achieve this, we introduce multiple innovations to build a
neural network with strong inductive biases for this task: First, we build
multiple sequential correlation volumes in time using event data. Second, we
use B\'ezier curves to index these correlation volumes at multiple timestamps
along the trajectory. Third, we use the retrieved correlation to update the
B\'ezier curve representations iteratively. Our method can optionally include
image pairs to boost performance further. The proposed approach outperforms
existing image-based and event-based methods by 11.5 % lower EPE on DSEC-Flow.
Finally, we introduce a novel synthetic dataset MultiFlow for pixel trajectory
regression on which our method is currently the only successful approach.

5
The categories of this paper are: gr-qc
17
The categories of this paper are: q-fin
31
The categories of this paper are: ao-sci
28
The categories of this paper are: alg-geom
10
The categories of this paper are: q-bio
13
The categories of this paper are: quant-ph
19
The categories of this paper are: eess
34
The categories of this paper are: chem-ph
25
The categories of this paper are: patt-sol
36
The categories of this paper are: cmp-lg
30
The categories of this paper are: q-alg
3
The categories of this paper are: physics
7
The categories of this paper are: hep-th
15
The categories of this paper are: nucl-ex
32
The categories of this paper are: atom-ph
20
The categories of this paper are: acc-phys