Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A Framework for the Meta-Analysis of Randomized Experiments with
  Applications to Heavy-Tailed Response Data   A central obstacle in the objective assessment of treatment effect (TE)
estimators in randomized control trials (RCTs) is the lack of ground truth (or
validation set) to test their performance. In this paper, we provide a novel
cross-validation-like methodology to address this challenge. The key insight of
our procedure is that the noisy (but unbiased) difference-of-means estimate can
be used as a ground truth "label" on a portion of the RCT, to test the
performance of an estimator trained on the other portion. We combine this
insight with an aggregation scheme, which borrows statistical strength across a
large collection of RCTs, to present an end-to-end methodology for judging an
estimator's ability to recover the underlying treatment effect. We evaluate our
methodology across 709 RCTs implemented in the Amazon supply chain. In the
corpus of AB tests at Amazon, we highlight the unique difficulties associated
with recovering the treatment effect due to the heavy-tailed nature of the
response variables. In this heavy-tailed setting, our methodology suggests that
procedures that aggressively downweight or truncate large values, while
introducing bias, lower the variance enough to ensure that the treatment effect
is more accurately estimated.

29
The categories of this paper are: funct-an
23
The categories of this paper are: solv-int
35
The categories of this paper are: mtrl-th
8
The categories of this paper are: hep-ex
36
The categories of this paper are: cmp-lg
10
The categories of this paper are: q-bio
19
The categories of this paper are: eess
15
The categories of this paper are: nucl-ex
30
The categories of this paper are: q-alg
27
The categories of this paper are: comp-gas
11
The categories of this paper are: hep-lat
37
The categories of this paper are: supr-con
20
The categories of this paper are: acc-phys
24
The categories of this paper are: dg-ga
3
The categories of this paper are: physics
7
The categories of this paper are: hep-th