Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Do you really code? Designing and Evaluating Screening Questions for
  Online Surveys with Programmers   Recruiting professional programmers in sufficient numbers for research
studies can be challenging because they often cannot spare the time, or due to
their geographical distribution and potentially the cost involved. Online
platforms such as Clickworker or Qualtrics do provide options to recruit
participants with programming skill; however, misunderstandings and fraud can
be an issue. This can result in participants without programming skill taking
part in studies and surveys. If these participants are not detected, they can
cause detrimental noise in the survey data. In this paper, we develop screener
questions that are easy and quick to answer for people with programming skill
but difficult to answer correctly for those without. In order to evaluate our
questionnaire for efficacy and efficiency, we recruited several batches of
participants with and without programming skill and tested the questions. In
our batch 42% of Clickworkers stating that they have programming skill did not
meet our criteria and we would recommend filtering these from studies. We also
evaluated the questions in an adversarial setting. We conclude with a set of
recommended questions which researchers can use to recruit participants with
programming skill from online platforms.

8
The categories of this paper are: hep-ex
16
The categories of this paper are: stat
31
The categories of this paper are: ao-sci
2
The categories of this paper are: math
20
The categories of this paper are: acc-phys
35
The categories of this paper are: mtrl-th
37
The categories of this paper are: supr-con
22
The categories of this paper are: chao-dyn
29
The categories of this paper are: funct-an
33
The categories of this paper are: plasm-ph
11
The categories of this paper are: hep-lat
18
The categories of this paper are: econ
27
The categories of this paper are: comp-gas
14
The categories of this paper are: math-ph
7
The categories of this paper are: hep-th
3
The categories of this paper are: physics