Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Why We Need New Evaluation Metrics for NLG   The majority of NLG evaluation relies on automatic metrics, such as BLEU . In
this paper, we motivate the need for novel, system- and data-independent
automatic evaluation methods: We investigate a wide range of metrics, including
state-of-the-art word-based and novel grammar-based ones, and demonstrate that
they only weakly reflect human judgements of system outputs as generated by
data-driven, end-to-end NLG. We also show that metric performance is data- and
system-specific. Nevertheless, our results also suggest that automatic metrics
perform reliably at system-level and can support system development by finding
cases where a system performs poorly.

28
The categories of this paper are: alg-geom
33
The categories of this paper are: plasm-ph
16
The categories of this paper are: stat
13
The categories of this paper are: quant-ph
34
The categories of this paper are: chem-ph
9
The categories of this paper are: nlin
15
The categories of this paper are: nucl-ex
22
The categories of this paper are: chao-dyn
24
The categories of this paper are: dg-ga
32
The categories of this paper are: atom-ph
27
The categories of this paper are: comp-gas
12
The categories of this paper are: nucl-th
20
The categories of this paper are: acc-phys
10
The categories of this paper are: q-bio
36
The categories of this paper are: cmp-lg
31
The categories of this paper are: ao-sci