Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Realistic Evaluation Principles for Cross-document Coreference
  Resolution   We point out that common evaluation practices for cross-document coreference
resolution have been unrealistically permissive in their assumed settings,
yielding inflated results. We propose addressing this issue via two evaluation
methodology principles. First, as in other tasks, models should be evaluated on
predicted mentions rather than on gold mentions. Doing this raises a subtle
issue regarding singleton coreference clusters, which we address by decoupling
the evaluation of mention detection from that of coreference linking. Second,
we argue that models should not exploit the synthetic topic structure of the
standard ECB+ dataset, forcing models to confront the lexical ambiguity
challenge, as intended by the dataset creators. We demonstrate empirically the
drastic impact of our more realistic evaluation principles on a competitive
model, yielding a score which is 33 F1 lower compared to evaluating by prior
lenient practices.

2
The categories of this paper are: math
37
The categories of this paper are: supr-con
20
The categories of this paper are: acc-phys
11
The categories of this paper are: hep-lat
15
The categories of this paper are: nucl-ex
35
The categories of this paper are: mtrl-th
25
The categories of this paper are: patt-sol
21
The categories of this paper are: adap-org
30
The categories of this paper are: q-alg
19
The categories of this paper are: eess
23
The categories of this paper are: solv-int
32
The categories of this paper are: atom-ph
12
The categories of this paper are: nucl-th
31
The categories of this paper are: ao-sci
13
The categories of this paper are: quant-ph
22
The categories of this paper are: chao-dyn