Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Reward is enough for convex MDPs   Maximising a cumulative reward function that is Markov and stationary, i.e.,
defined over state-action pairs and independent of time, is sufficient to
capture many kinds of goals in a Markov decision process (MDP). However, not
all goals can be captured in this manner. In this paper we study convex MDPs in
which goals are expressed as convex functions of the stationary distribution
and show that they cannot be formulated using stationary reward functions.
Convex MDPs generalize the standard reinforcement learning (RL) problem
formulation to a larger framework that includes many supervised and
unsupervised RL problems, such as apprenticeship learning, constrained MDPs,
and so-called `pure exploration'. Our approach is to reformulate the convex MDP
problem as a min-max game involving policy and cost (negative reward)
`players', using Fenchel duality. We propose a meta-algorithm for solving this
problem and show that it unifies many existing algorithms in the literature.

29
The categories of this paper are: funct-an
21
The categories of this paper are: adap-org
34
The categories of this paper are: chem-ph
12
The categories of this paper are: nucl-th
27
The categories of this paper are: comp-gas
4
The categories of this paper are: cond-mat
15
The categories of this paper are: nucl-ex
0
The categories of this paper are: hep-ph
36
The categories of this paper are: cmp-lg
7
The categories of this paper are: hep-th
18
The categories of this paper are: econ
26
The categories of this paper are: bayes-an
13
The categories of this paper are: quant-ph
17
The categories of this paper are: q-fin
10
The categories of this paper are: q-bio
37
The categories of this paper are: supr-con