Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Robust Reinforcement Learning as a Stackelberg Game via
  Adaptively-Regularized Adversarial Training   Robust Reinforcement Learning (RL) focuses on improving performances under
model errors or adversarial attacks, which facilitates the real-life deployment
of RL agents. Robust Adversarial Reinforcement Learning (RARL) is one of the
most popular frameworks for robust RL. However, most of the existing literature
models RARL as a zero-sum simultaneous game with Nash equilibrium as the
solution concept, which could overlook the sequential nature of RL deployments,
produce overly conservative agents, and induce training instability. In this
paper, we introduce a novel hierarchical formulation of robust RL - a
general-sum Stackelberg game model called RRL-Stack - to formalize the
sequential nature and provide extra flexibility for robust training. We develop
the Stackelberg Policy Gradient algorithm to solve RRL-Stack, leveraging the
Stackelberg learning dynamics by considering the adversary's response. Our
method generates challenging yet solvable adversarial environments which
benefit RL agents' robust learning. Our algorithm demonstrates better training
stability and robustness against different testing conditions in the
single-agent robotics control and multi-agent highway merging tasks.

8
The categories of this paper are: hep-ex
21
The categories of this paper are: adap-org
17
The categories of this paper are: q-fin
26
The categories of this paper are: bayes-an
22
The categories of this paper are: chao-dyn
31
The categories of this paper are: ao-sci
37
The categories of this paper are: supr-con
0
The categories of this paper are: hep-ph
36
The categories of this paper are: cmp-lg
5
The categories of this paper are: gr-qc
10
The categories of this paper are: q-bio
12
The categories of this paper are: nucl-th
2
The categories of this paper are: math
27
The categories of this paper are: comp-gas
29
The categories of this paper are: funct-an
16
The categories of this paper are: stat