Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic
  Policies   Offline reinforcement learning, wherein one uses off-policy data logged by a
fixed behavior policy to evaluate and learn new policies, is crucial in
applications where experimentation is limited such as medicine. We study the
estimation of policy value and gradient of a deterministic policy from
off-policy data when actions are continuous. Targeting deterministic policies,
for which action is a deterministic function of state, is crucial since optimal
policies are always deterministic (up to ties). In this setting, standard
importance sampling and doubly robust estimators for policy value and gradient
fail because the density ratio does not exist. To circumvent this issue, we
propose several new doubly robust estimators based on different kernelization
approaches. We analyze the asymptotic mean-squared error of each of these under
mild rate conditions for nuisance estimators. Specifically, we demonstrate how
to obtain a rate that is independent of the horizon length.

22
The categories of this paper are: chao-dyn
31
The categories of this paper are: ao-sci
24
The categories of this paper are: dg-ga
37
The categories of this paper are: supr-con
9
The categories of this paper are: nlin
3
The categories of this paper are: physics
33
The categories of this paper are: plasm-ph
7
The categories of this paper are: hep-th
8
The categories of this paper are: hep-ex
4
The categories of this paper are: cond-mat
11
The categories of this paper are: hep-lat
12
The categories of this paper are: nucl-th
0
The categories of this paper are: hep-ph
29
The categories of this paper are: funct-an
6
The categories of this paper are: astro-ph
30
The categories of this paper are: q-alg