Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Frame-To-Frame Consistent Semantic Segmentation   In this work, we aim for temporally consistent semantic segmentation
throughout frames in a video. Many semantic segmentation algorithms process
images individually which leads to an inconsistent scene interpretation due to
illumination changes, occlusions and other variations over time. To achieve a
temporally consistent prediction, we train a convolutional neural network (CNN)
which propagates features through consecutive frames in a video using a
convolutional long short term memory (ConvLSTM) cell. Besides the temporal
feature propagation, we penalize inconsistencies in our loss function. We show
in our experiments that the performance improves when utilizing video
information compared to single frame prediction. The mean intersection over
union (mIoU) metric on the Cityscapes validation set increases from 45.2 % for
the single frames to 57.9 % for video data after implementing the ConvLSTM to
propagate features trough time on the ESPNet. Most importantly, inconsistency
decreases from 4.5 % to 1.3 % which is a reduction by 71.1 %. Our results
indicate that the added temporal information produces a frame-to-frame
consistent and more accurate image understanding compared to single frame
processing. Code and videos are available at
https://github.com/mrebol/f2f-consistent-semantic-segmentation

7
The categories of this paper are: hep-th
26
The categories of this paper are: bayes-an
15
The categories of this paper are: nucl-ex
12
The categories of this paper are: nucl-th
21
The categories of this paper are: adap-org
23
The categories of this paper are: solv-int
3
The categories of this paper are: physics
2
The categories of this paper are: math
16
The categories of this paper are: stat
4
The categories of this paper are: cond-mat
25
The categories of this paper are: patt-sol
9
The categories of this paper are: nlin
37
The categories of this paper are: supr-con
30
The categories of this paper are: q-alg
22
The categories of this paper are: chao-dyn
17
The categories of this paper are: q-fin