Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Challenging Common Assumptions in Convex Reinforcement Learning   The classic Reinforcement Learning (RL) formulation concerns the maximization
of a scalar reward function. More recently, convex RL has been introduced to
extend the RL formulation to all the objectives that are convex functions of
the state distribution induced by a policy. Notably, convex RL covers several
relevant applications that do not fall into the scalar formulation, including
imitation learning, risk-averse RL, and pure exploration. In classic RL, it is
common to optimize an infinite trials objective, which accounts for the state
distribution instead of the empirical state visitation frequencies, even though
the actual number of trajectories is always finite in practice. This is
theoretically sound since the infinite trials and finite trials objectives can
be proved to coincide and thus lead to the same optimal policy. In this paper,
we show that this hidden assumption does not hold in the convex RL setting. In
particular, we show that erroneously optimizing the infinite trials objective
in place of the actual finite trials one, as it is usually done, can lead to a
significant approximation error. Since the finite trials setting is the default
in both simulated and real-world RL, we believe shedding light on this issue
will lead to better approaches and methodologies for convex RL, impacting
relevant research areas such as imitation learning, risk-averse RL, and pure
exploration among others.

15
The categories of this paper are: nucl-ex
31
The categories of this paper are: ao-sci
28
The categories of this paper are: alg-geom
14
The categories of this paper are: math-ph
16
The categories of this paper are: stat
35
The categories of this paper are: mtrl-th
17
The categories of this paper are: q-fin
22
The categories of this paper are: chao-dyn
25
The categories of this paper are: patt-sol
20
The categories of this paper are: acc-phys
19
The categories of this paper are: eess
10
The categories of this paper are: q-bio
9
The categories of this paper are: nlin
36
The categories of this paper are: cmp-lg
33
The categories of this paper are: plasm-ph
2
The categories of this paper are: math