Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Quantifying Differences in Reward Functions   For many tasks, the reward function is inaccessible to introspection or too
complex to be specified procedurally, and must instead be learned from user
data. Prior work has evaluated learned reward functions by evaluating policies
optimized for the learned reward. However, this method cannot distinguish
between the learned reward function failing to reflect user preferences and the
policy optimization process failing to optimize the learned reward. Moreover,
this method can only tell us about behavior in the evaluation environment, but
the reward may incentivize very different behavior in even a slightly different
deployment environment. To address these problems, we introduce the
Equivalent-Policy Invariant Comparison (EPIC) distance to quantify the
difference between two reward functions directly, without a policy optimization
step. We prove EPIC is invariant on an equivalence class of reward functions
that always induce the same optimal policy. Furthermore, we find EPIC can be
efficiently approximated and is more robust than baselines to the choice of
coverage distribution. Finally, we show that EPIC distance bounds the regret of
optimal policies even under different transition dynamics, and we confirm
empirically that it predicts policy training success. Our source code is
available at https://github.com/HumanCompatibleAI/evaluating-rewards.

18
The categories of this paper are: econ
5
The categories of this paper are: gr-qc
21
The categories of this paper are: adap-org
4
The categories of this paper are: cond-mat
24
The categories of this paper are: dg-ga
32
The categories of this paper are: atom-ph
36
The categories of this paper are: cmp-lg
13
The categories of this paper are: quant-ph
25
The categories of this paper are: patt-sol
14
The categories of this paper are: math-ph
23
The categories of this paper are: solv-int
17
The categories of this paper are: q-fin
11
The categories of this paper are: hep-lat
30
The categories of this paper are: q-alg
37
The categories of this paper are: supr-con
20
The categories of this paper are: acc-phys