Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Emphatic Algorithms for Deep Reinforcement Learning   Off-policy learning allows us to learn about possible policies of behavior
from experience generated by a different behavior policy. Temporal difference
(TD) learning algorithms can become unstable when combined with function
approximation and off-policy sampling - this is known as the ''deadly triad''.
Emphatic temporal difference (ETD($\lambda$)) algorithm ensures convergence in
the linear case by appropriately weighting the TD($\lambda$) updates. In this
paper, we extend the use of emphatic methods to deep reinforcement learning
agents. We show that naively adapting ETD($\lambda$) to popular deep
reinforcement learning algorithms, which use forward view multi-step returns,
results in poor performance. We then derive new emphatic algorithms for use in
the context of such algorithms, and we demonstrate that they provide noticeable
benefits in small problems designed to highlight the instability of TD methods.
Finally, we observed improved performance when applying these algorithms at
scale on classic Atari games from the Arcade Learning Environment.

11
The categories of this paper are: hep-lat
13
The categories of this paper are: quant-ph
18
The categories of this paper are: econ
19
The categories of this paper are: eess
37
The categories of this paper are: supr-con
25
The categories of this paper are: patt-sol
28
The categories of this paper are: alg-geom
34
The categories of this paper are: chem-ph
4
The categories of this paper are: cond-mat
21
The categories of this paper are: adap-org
5
The categories of this paper are: gr-qc
0
The categories of this paper are: hep-ph
32
The categories of this paper are: atom-ph
20
The categories of this paper are: acc-phys
17
The categories of this paper are: q-fin
36
The categories of this paper are: cmp-lg