Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Forecast score distributions with imperfect observations   The classical paradigm of scoring rules is to discriminate between two
different forecasts by comparing them with observations. The probability
distribution of the observed record is assumed to be perfect as a verification
benchmark. In practice, however, observations are almost always tainted by
errors and uncertainties. If the yardstick used to compare forecasts is
imprecise, one can wonder whether such types of errors may or may not have a
strong influence on decisions based on classical scoring rules. We propose a
new scoring rule scheme in the context of models that incorporate errors of the
verification data. We rely on existing scoring rules and incorporate
uncertainty and error of the verification data through a hidden variable and
the conditional expectation of scores when they are viewed as a random
variable. The proposed scoring framework is compared to scores used in
practice, and is expressed in various setups, mainly an additive Gaussian noise
model and a multiplicative Gamma noise model. By considering scores as random
variables one can access the entire range of their distribution. In particular
we illustrate that the commonly used mean score can be a misleading
representative of the distribution when this latter is highly skewed or have
heavy tails. In a simulation study, through the power of a statistical test and
the computation of Wasserstein distances between scores distributions, we
demonstrate the ability of the newly proposed score to better discriminate
between forecasts when verification data are subject to uncertainty compared
with the scores used in practice. Finally, we illustrate the benefit of
accounting for the uncertainty of the verification data into the scoring
procedure on a dataset of surface wind speed from measurements and numerical
model outputs.

5
The categories of this paper are: gr-qc
30
The categories of this paper are: q-alg
11
The categories of this paper are: hep-lat
14
The categories of this paper are: math-ph
26
The categories of this paper are: bayes-an
19
The categories of this paper are: eess
15
The categories of this paper are: nucl-ex
3
The categories of this paper are: physics
8
The categories of this paper are: hep-ex
9
The categories of this paper are: nlin
22
The categories of this paper are: chao-dyn
25
The categories of this paper are: patt-sol
28
The categories of this paper are: alg-geom
17
The categories of this paper are: q-fin
4
The categories of this paper are: cond-mat
10
The categories of this paper are: q-bio