Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: On Multi-objective Policy Optimization as a Tool for Reinforcement
  Learning   Many advances that have improved the robustness and efficiency of deep
reinforcement learning (RL) algorithms can, in one way or another, be
understood as introducing additional objectives, or constraints, in the policy
optimization step. This includes ideas as far ranging as exploration bonuses,
entropy regularization, and regularization toward teachers or data priors when
learning from experts or in offline RL. Often, task reward and auxiliary
objectives are in conflict with each other and it is therefore natural to treat
these examples as instances of multi-objective (MO) optimization problems. We
study the principles underlying MORL and introduce a new algorithm,
Distillation of a Mixture of Experts (DiME), that is intuitive and
scale-invariant under some conditions. We highlight its strengths on standard
MO benchmark problems and consider case studies in which we recast offline RL
and learning from experts as MO problems. This leads to a natural algorithmic
formulation that sheds light on the connection between existing approaches. For
offline RL, we use the MO perspective to derive a simple algorithm, that
optimizes for the standard RL objective plus a behavioral cloning term. This
outperforms state-of-the-art on two established offline RL benchmarks.

5
The categories of this paper are: gr-qc
34
The categories of this paper are: chem-ph
3
The categories of this paper are: physics
4
The categories of this paper are: cond-mat
37
The categories of this paper are: supr-con
16
The categories of this paper are: stat
19
The categories of this paper are: eess
32
The categories of this paper are: atom-ph
6
The categories of this paper are: astro-ph
11
The categories of this paper are: hep-lat
27
The categories of this paper are: comp-gas
18
The categories of this paper are: econ
10
The categories of this paper are: q-bio
20
The categories of this paper are: acc-phys
7
The categories of this paper are: hep-th
2
The categories of this paper are: math