Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: ESPnet-ST IWSLT 2021 Offline Speech Translation System   This paper describes the ESPnet-ST group's IWSLT 2021 submission in the
offline speech translation track. This year we made various efforts on training
data, architecture, and audio segmentation. On the data side, we investigated
sequence-level knowledge distillation (SeqKD) for end-to-end (E2E) speech
translation. Specifically, we used multi-referenced SeqKD from multiple
teachers trained on different amounts of bitext. On the architecture side, we
adopted the Conformer encoder and the Multi-Decoder architecture, which equips
dedicated decoders for speech recognition and translation tasks in a unified
encoder-decoder model and enables search in both source and target language
spaces during inference. We also significantly improved audio segmentation by
using the pyannote.audio toolkit and merging multiple short segments for long
context modeling. Experimental evaluations showed that each of them contributed
to large improvements in translation performance. Our best E2E system combined
all the above techniques with model ensembling and achieved 31.4 BLEU on the
2-ref of tst2021 and 21.2 BLEU and 19.3 BLEU on the two single references of
tst2021.

34
The categories of this paper are: chem-ph
3
The categories of this paper are: physics
12
The categories of this paper are: nucl-th
2
The categories of this paper are: math
16
The categories of this paper are: stat
29
The categories of this paper are: funct-an
21
The categories of this paper are: adap-org
6
The categories of this paper are: astro-ph
17
The categories of this paper are: q-fin
14
The categories of this paper are: math-ph
35
The categories of this paper are: mtrl-th
31
The categories of this paper are: ao-sci
33
The categories of this paper are: plasm-ph
27
The categories of this paper are: comp-gas
0
The categories of this paper are: hep-ph
23
The categories of this paper are: solv-int