Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A Generalized Bootstrap Target for Value-Learning, Efficiently Combining
  Value and Feature Predictions   Estimating value functions is a core component of reinforcement learning
algorithms. Temporal difference (TD) learning algorithms use bootstrapping,
i.e. they update the value function toward a learning target using value
estimates at subsequent time-steps. Alternatively, the value function can be
updated toward a learning target constructed by separately predicting successor
features (SF)--a policy-dependent model--and linearly combining them with
instantaneous rewards. We focus on bootstrapping targets used when estimating
value functions, and propose a new backup target, the $\eta$-return mixture,
which implicitly combines value-predictive knowledge (used by TD methods) with
(successor) feature-predictive knowledge--with a parameter $\eta$ capturing how
much to rely on each. We illustrate that incorporating predictive knowledge
through an $\eta\gamma$-discounted SF model makes more efficient use of sampled
experience, compared to either extreme, i.e. bootstrapping entirely on the
value function estimate, or bootstrapping on the product of separately
estimated successor features and instantaneous reward models. We empirically
show this approach leads to faster policy evaluation and better control
performance, for tabular and nonlinear function approximations, indicating
scalability and generality.

31
The categories of this paper are: ao-sci
0
The categories of this paper are: hep-ph
30
The categories of this paper are: q-alg
10
The categories of this paper are: q-bio
32
The categories of this paper are: atom-ph
27
The categories of this paper are: comp-gas
20
The categories of this paper are: acc-phys
8
The categories of this paper are: hep-ex
14
The categories of this paper are: math-ph
3
The categories of this paper are: physics
18
The categories of this paper are: econ
36
The categories of this paper are: cmp-lg
24
The categories of this paper are: dg-ga
9
The categories of this paper are: nlin
37
The categories of this paper are: supr-con
6
The categories of this paper are: astro-ph