Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Annotators with Attitudes: How Annotator Beliefs And Identities Bias
  Toxic Language Detection   The perceived toxicity of language can vary based on someone's identity and
beliefs, but this variation is often ignored when collecting toxic language
datasets, resulting in dataset and model biases. We seek to understand the who,
why, and what behind biases in toxicity annotations. In two online studies with
demographically and politically diverse participants, we investigate the effect
of annotator identities (who) and beliefs (why), drawing from social psychology
research about hate speech, free speech, racist beliefs, political leaning, and
more. We disentangle what is annotated as toxic by considering posts with three
characteristics: anti-Black language, African American English (AAE) dialect,
and vulgarity. Our results show strong associations between annotator identity
and beliefs and their ratings of toxicity. Notably, more conservative
annotators and those who scored highly on our scale for racist beliefs were
less likely to rate anti-Black language as toxic, but more likely to rate AAE
as toxic. We additionally present a case study illustrating how a popular
toxicity detection system's ratings inherently reflect only specific beliefs
and perspectives. Our findings call for contextualizing toxicity labels in
social variables, which raises immense implications for toxic language
annotation and detection.

27
The categories of this paper are: comp-gas
11
The categories of this paper are: hep-lat
25
The categories of this paper are: patt-sol
20
The categories of this paper are: acc-phys
8
The categories of this paper are: hep-ex
34
The categories of this paper are: chem-ph
31
The categories of this paper are: ao-sci
32
The categories of this paper are: atom-ph
28
The categories of this paper are: alg-geom
10
The categories of this paper are: q-bio
12
The categories of this paper are: nucl-th
2
The categories of this paper are: math
7
The categories of this paper are: hep-th
14
The categories of this paper are: math-ph
21
The categories of this paper are: adap-org
23
The categories of this paper are: solv-int