Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Modified Actor-Critics   Recent successful deep reinforcement learning algorithms, such as Trust
Region Policy Optimization (TRPO) or Proximal Policy Optimization (PPO), are
fundamentally variations of conservative policy iteration (CPI). These
algorithms iterate policy evaluation followed by a softened policy improvement
step. As so, they are naturally on-policy. In this paper, we propose to combine
(any kind of) soft greediness with Modified Policy Iteration (MPI). The
proposed abstract framework applies repeatedly: (i) a partial policy evaluation
step that allows off-policy learning and (ii) any softened greedy step. Our
contribution can be seen as a new generic tool for the deep reinforcement
learning toolbox. As a proof of concept, we instantiate this framework with the
PPO greediness. Comparison to the original PPO shows that our algorithm is much
more sample efficient. We also show that it is competitive with the
state-of-art off-policy algorithm Soft Actor Critic (SAC).

5
The categories of this paper are: gr-qc
33
The categories of this paper are: plasm-ph
3
The categories of this paper are: physics
20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin
24
The categories of this paper are: dg-ga
22
The categories of this paper are: chao-dyn
4
The categories of this paper are: cond-mat
27
The categories of this paper are: comp-gas
14
The categories of this paper are: math-ph
10
The categories of this paper are: q-bio
0
The categories of this paper are: hep-ph
37
The categories of this paper are: supr-con
17
The categories of this paper are: q-fin
25
The categories of this paper are: patt-sol
18
The categories of this paper are: econ