Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Towards Visual-Prompt Temporal Answering Grounding in Medical
  Instructional Video   The temporal answering grounding in the video (TAGV) is a new task naturally
derived from temporal sentence grounding in the video (TSGV). Given an
untrimmed video and a text question, this task aims at locating the matching
span from the video that can semantically answer the question. Existing methods
tend to formulate the TAGV task with a visual span-based question answering
(QA) approach by matching the visual frame span queried by the text question.
However, due to the weak correlations and huge gaps of the semantic features
between the textual question and visual answer, existing methods adopting
visual span predictor perform poorly in the TAGV task. To bridge these gaps, we
propose a visual-prompt text span localizing (VPTSL) method, which introduces
the timestamped subtitles as a passage to perform the text span localization
for the input text question, and prompts the visual highlight features into the
pre-trained language model (PLM) for enhancing the joint semantic
representations. Specifically, the context query attention is utilized to
perform cross-modal interaction between the extracted textual and visual
features. Then, the highlight features are obtained through the video-text
highlighting for the visual prompt. To alleviate semantic differences between
textual and visual features, we design the text span predictor by encoding the
question, the subtitles, and the prompted visual highlight features with the
PLM. As a result, the TAGV task is formulated to predict the span of subtitles
matching the visual answer. Extensive experiments on the medical instructional
dataset, namely MedVidQA, show that the proposed VPTSL outperforms the
state-of-the-art (SOTA) method by 28.36% in terms of mIOU with a large margin,
which demonstrates the effectiveness of the proposed visual prompt and the text
span predictor.

17
The categories of this paper are: q-fin
30
The categories of this paper are: q-alg
34
The categories of this paper are: chem-ph
6
The categories of this paper are: astro-ph
3
The categories of this paper are: physics
28
The categories of this paper are: alg-geom
11
The categories of this paper are: hep-lat
20
The categories of this paper are: acc-phys
35
The categories of this paper are: mtrl-th
2
The categories of this paper are: math
33
The categories of this paper are: plasm-ph
14
The categories of this paper are: math-ph
22
The categories of this paper are: chao-dyn
16
The categories of this paper are: stat
26
The categories of this paper are: bayes-an
8
The categories of this paper are: hep-ex