Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Deconstruct to Reconstruct a Configurable Evaluation Metric for
  Open-Domain Dialogue Systems   Many automatic evaluation metrics have been proposed to score the overall
quality of a response in open-domain dialogue. Generally, the overall quality
is comprised of various aspects, such as relevancy, specificity, and empathy,
and the importance of each aspect differs according to the task. For instance,
specificity is mandatory in a food-ordering dialogue task, whereas fluency is
preferred in a language-teaching dialogue system. However, existing metrics are
not designed to cope with such flexibility. For example, BLEU score
fundamentally relies only on word overlapping, whereas BERTScore relies on
semantic similarity between reference and candidate response. Thus, they are
not guaranteed to capture the required aspects, i.e., specificity. To design a
metric that is flexible to a task, we first propose making these qualities
manageable by grouping them into three groups: understandability, sensibleness,
and likability, where likability is a combination of qualities that are
essential for a task. We also propose a simple method to composite metrics of
each aspect to obtain a single metric called USL-H, which stands for
Understandability, Sensibleness, and Likability in Hierarchy. We demonstrated
that USL-H score achieves good correlations with human judgment and maintains
its configurability towards different aspects and metrics.

34
The categories of this paper are: chem-ph
10
The categories of this paper are: q-bio
4
The categories of this paper are: cond-mat
16
The categories of this paper are: stat
36
The categories of this paper are: cmp-lg
11
The categories of this paper are: hep-lat
6
The categories of this paper are: astro-ph
5
The categories of this paper are: gr-qc
19
The categories of this paper are: eess
25
The categories of this paper are: patt-sol
24
The categories of this paper are: dg-ga
17
The categories of this paper are: q-fin
33
The categories of this paper are: plasm-ph
0
The categories of this paper are: hep-ph
30
The categories of this paper are: q-alg
15
The categories of this paper are: nucl-ex