Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Improving Streaming Automatic Speech Recognition With Non-Streaming
  Model Distillation On Unsupervised Data   Streaming end-to-end automatic speech recognition (ASR) models are widely
used on smart speakers and on-device applications. Since these models are
expected to transcribe speech with minimal latency, they are constrained to be
causal with no future context, compared to their non-streaming counterparts.
Consequently, streaming models usually perform worse than non-streaming models.
We propose a novel and effective learning method by leveraging a non-streaming
ASR model as a teacher to generate transcripts on an arbitrarily large data
set, which is then used to distill knowledge into streaming ASR models. This
way, we scale the training of streaming models to up to 3 million hours of
YouTube audio. Experiments show that our approach can significantly reduce the
word error rate (WER) of RNNT models not only on LibriSpeech but also on
YouTube data in four languages. For example, in French, we are able to reduce
the WER by 16.4% relatively to a baseline streaming model by leveraging a
non-streaming teacher model trained on the same amount of labeled data as the
baseline.

11
The categories of this paper are: hep-lat
16
The categories of this paper are: stat
24
The categories of this paper are: dg-ga
26
The categories of this paper are: bayes-an
37
The categories of this paper are: supr-con
13
The categories of this paper are: quant-ph
0
The categories of this paper are: hep-ph
20
The categories of this paper are: acc-phys
3
The categories of this paper are: physics
9
The categories of this paper are: nlin
31
The categories of this paper are: ao-sci
35
The categories of this paper are: mtrl-th
30
The categories of this paper are: q-alg
5
The categories of this paper are: gr-qc
29
The categories of this paper are: funct-an
25
The categories of this paper are: patt-sol