Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: SacreROUGE: An Open-Source Library for Using and Developing
  Summarization Evaluation Metrics   We present SacreROUGE, an open-source library for using and developing
summarization evaluation metrics. SacreROUGE removes many obstacles that
researchers face when using or developing metrics: (1) The library provides
Python wrappers around the official implementations of existing evaluation
metrics so they share a common, easy-to-use interface; (2) it provides
functionality to evaluate how well any metric implemented in the library
correlates to human-annotated judgments, so no additional code needs to be
written for a new evaluation metric; and (3) it includes scripts for loading
datasets that contain human judgments so they can easily be used for
evaluation. This work describes the design of the library, including the core
Metric interface, the command-line API for evaluating summarization models and
metrics, and the scripts to load and reformat publicly available datasets. The
development of SacreROUGE is ongoing and open to contributions from the
community.

8
The categories of this paper are: hep-ex
5
The categories of this paper are: gr-qc
16
The categories of this paper are: stat
22
The categories of this paper are: chao-dyn
25
The categories of this paper are: patt-sol
34
The categories of this paper are: chem-ph
24
The categories of this paper are: dg-ga
3
The categories of this paper are: physics
13
The categories of this paper are: quant-ph
29
The categories of this paper are: funct-an
17
The categories of this paper are: q-fin
30
The categories of this paper are: q-alg
18
The categories of this paper are: econ
7
The categories of this paper are: hep-th
26
The categories of this paper are: bayes-an
27
The categories of this paper are: comp-gas