Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Rethinking the Discount Factor in Reinforcement Learning: A Decision
  Theoretic Approach   Reinforcement learning (RL) agents have traditionally been tasked with
maximizing the value function of a Markov decision process (MDP), either in
continuous settings, with fixed discount factor $\gamma < 1$, or in episodic
settings, with $\gamma = 1$. While this has proven effective for specific tasks
with well-defined objectives (e.g., games), it has never been established that
fixed discounting is suitable for general purpose use (e.g., as a model of
human preferences). This paper characterizes rationality in sequential decision
making using a set of seven axioms and arrives at a form of discounting that
generalizes traditional fixed discounting. In particular, our framework admits
a state-action dependent "discount" factor that is not constrained to be less
than 1, so long as there is eventual long run discounting. Although this
broadens the range of possible preference structures in continuous settings, we
show that there exists a unique "optimizing MDP" with fixed $\gamma < 1$ whose
optimal value function matches the true utility of the optimal policy, and we
quantify the difference between value and utility for suboptimal policies. Our
work can be seen as providing a normative justification for (a slight
generalization of) Martha White's RL task formalism (2017) and other recent
departures from the traditional RL, and is relevant to task specification in
RL, inverse RL and preference-based RL.

7
The categories of this paper are: hep-th
8
The categories of this paper are: hep-ex
18
The categories of this paper are: econ
24
The categories of this paper are: dg-ga
36
The categories of this paper are: cmp-lg
26
The categories of this paper are: bayes-an
27
The categories of this paper are: comp-gas
31
The categories of this paper are: ao-sci
34
The categories of this paper are: chem-ph
6
The categories of this paper are: astro-ph
21
The categories of this paper are: adap-org
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
13
The categories of this paper are: quant-ph
35
The categories of this paper are: mtrl-th
19
The categories of this paper are: eess