Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A Framework for Cluster and Classifier Evaluation in the Absence of
  Reference Labels   In some problem spaces, the high cost of obtaining ground truth labels
necessitates use of lower quality reference datasets. It is difficult to
benchmark model performance using these datasets, as evaluation results may be
biased. We propose a supplement to using reference labels, which we call an
approximate ground truth refinement (AGTR). Using an AGTR, we prove that bounds
on specific metrics used to evaluate clustering algorithms and multi-class
classifiers can be computed without reference labels. We also introduce a
procedure that uses an AGTR to identify inaccurate evaluation results produced
from datasets of dubious quality. Creating an AGTR requires domain knowledge,
and malware family classification is a task with robust domain knowledge
approaches that support the construction of an AGTR. We demonstrate our AGTR
evaluation framework by applying it to a popular malware labeling tool to
diagnose over-fitting in prior testing and evaluate changes whose impact could
not be meaningfully quantified under previous data.

6
The categories of this paper are: astro-ph
33
The categories of this paper are: plasm-ph
35
The categories of this paper are: mtrl-th
21
The categories of this paper are: adap-org
12
The categories of this paper are: nucl-th
30
The categories of this paper are: q-alg
31
The categories of this paper are: ao-sci
0
The categories of this paper are: hep-ph
19
The categories of this paper are: eess
36
The categories of this paper are: cmp-lg
5
The categories of this paper are: gr-qc
16
The categories of this paper are: stat
18
The categories of this paper are: econ
9
The categories of this paper are: nlin
13
The categories of this paper are: quant-ph
29
The categories of this paper are: funct-an