Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Towards a Decomposable Metric for Explainable Evaluation of Text
  Generation from AMR   Systems that generate natural language text from abstract meaning
representations such as AMR are typically evaluated using automatic surface
matching metrics that compare the generated texts to reference texts from which
the input meaning representations were constructed. We show that besides
well-known issues from which such metrics suffer, an additional problem arises
when applying these metrics for AMR-to-text evaluation, since an abstract
meaning representation allows for numerous surface realizations. In this work
we aim to alleviate these issues by proposing $\mathcal{M}\mathcal{F}_\beta$, a
decomposable metric that builds on two pillars. The first is the principle of
meaning preservation $\mathcal{M}$: it measures to what extent a given AMR can
be reconstructed from the generated sentence using SOTA AMR parsers and
applying (fine-grained) AMR evaluation metrics to measure the distance between
the original and the reconstructed AMR. The second pillar builds on a principle
of (grammatical) form $\mathcal{F}$ that measures the linguistic quality of the
generated text, which we implement using SOTA language models. In two extensive
pilot studies we show that fulfillment of both principles offers benefits for
AMR-to-text evaluation, including explainability of scores. Since
$\mathcal{M}\mathcal{F}_\beta$ does not necessarily rely on gold AMRs, it may
extend to other text generation tasks.

15
The categories of this paper are: nucl-ex
4
The categories of this paper are: cond-mat
19
The categories of this paper are: eess
9
The categories of this paper are: nlin
26
The categories of this paper are: bayes-an
25
The categories of this paper are: patt-sol
5
The categories of this paper are: gr-qc
2
The categories of this paper are: math
28
The categories of this paper are: alg-geom
22
The categories of this paper are: chao-dyn
7
The categories of this paper are: hep-th
8
The categories of this paper are: hep-ex
17
The categories of this paper are: q-fin
20
The categories of this paper are: acc-phys
31
The categories of this paper are: ao-sci
13
The categories of this paper are: quant-ph