Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: An External Stability Audit Framework to Test the Validity of
  Personality Prediction in AI Hiring   Automated hiring systems are among the fastest-developing of all high-stakes
AI systems. Among these are algorithmic personality tests that use insights
from psychometric testing, and promise to surface personality traits indicative
of future success based on job seekers' resumes or social media profiles. We
interrogate the validity of such systems using stability of the outputs they
produce, noting that reliability is a necessary, but not a sufficient,
condition for validity. Our approach is to (a) develop a methodology for an
external audit of stability of predictions made by algorithmic personality
tests, and (b) instantiate this methodology in an audit of two systems,
Humantic AI and Crystal. Crucially, rather than challenging or affirming the
assumptions made in psychometric testing -- that personality is a meaningful
and measurable construct, and that personality traits are indicative of future
success on the job -- we frame our methodology around testing the underlying
assumptions made by the vendors of the algorithmic personality tests
themselves.
  Our main contribution is the development of a socio-technical framework for
auditing the stability of algorithmic systems. This contribution is
supplemented with an open-source software library that implements the technical
components of the audit, and can be used to conduct similar stability audits of
algorithmic systems. We instantiate our framework with the audit of two
real-world personality prediction systems, namely Humantic AI and Crystal. The
application of our audit framework demonstrates that both these systems show
substantial instability with respect to key facets of measurement, and hence
cannot be considered valid testing instruments.

33
The categories of this paper are: plasm-ph
37
The categories of this paper are: supr-con
29
The categories of this paper are: funct-an
27
The categories of this paper are: comp-gas
16
The categories of this paper are: stat
24
The categories of this paper are: dg-ga
17
The categories of this paper are: q-fin
21
The categories of this paper are: adap-org
26
The categories of this paper are: bayes-an
13
The categories of this paper are: quant-ph
4
The categories of this paper are: cond-mat
15
The categories of this paper are: nucl-ex
20
The categories of this paper are: acc-phys
7
The categories of this paper are: hep-th
28
The categories of this paper are: alg-geom
8
The categories of this paper are: hep-ex