Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Optimal Reward Baseline for Gradient-Based Reinforcement Learning   There exist a number of reinforcement learning algorithms which learnby
climbing the gradient of expected reward. Their long-runconvergence has been
proved, even in partially observableenvironments with non-deterministic
actions, and without the need fora system model. However, the variance of the
gradient estimator hasbeen found to be a significant practical problem. Recent
approacheshave discounted future rewards, introducing a bias-variance
trade-offinto the gradient estimate. We incorporate a reward baseline into
thelearning system, and show that it affects variance without
introducingfurther bias. In particular, as we approach the
zero-bias,high-variance parameterization, the optimal (or variance
minimizing)constant reward baseline is equal to the long-term average
expectedreward. Modified policy-gradient algorithms are presented, and anumber
of experiments demonstrate their improvement over previous work.

13
The categories of this paper are: quant-ph
30
The categories of this paper are: q-alg
24
The categories of this paper are: dg-ga
36
The categories of this paper are: cmp-lg
28
The categories of this paper are: alg-geom
19
The categories of this paper are: eess
6
The categories of this paper are: astro-ph
9
The categories of this paper are: nlin
22
The categories of this paper are: chao-dyn
23
The categories of this paper are: solv-int
20
The categories of this paper are: acc-phys
29
The categories of this paper are: funct-an
17
The categories of this paper are: q-fin
32
The categories of this paper are: atom-ph
4
The categories of this paper are: cond-mat
26
The categories of this paper are: bayes-an