Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Reward-Free RL is No Harder Than Reward-Aware RL in Linear Markov
  Decision Processes   Reward-free reinforcement learning (RL) considers the setting where the agent
does not have access to a reward function during exploration, but must propose
a near-optimal policy for an arbitrary reward function revealed only after
exploring. In the the tabular setting, it is well known that this is a more
difficult problem than PAC RL -- where the agent has access to the reward
function during exploration -- with optimal sample complexities in the two
settings differing by a factor of $|\mathcal{S}|$, the size of the state space.
We show that this separation does not exist in the setting of linear MDPs. We
first develop a computationally efficient algorithm for reward-free RL in a
$d$-dimensional linear MDP with sample complexity scaling as
$\mathcal{O}(d^2/\epsilon^2)$. We then show a matching lower bound of
$\Omega(d^2/\epsilon^2)$ on PAC RL. To our knowledge, our approach is the first
computationally efficient algorithm to achieve optimal $d$ dependence in linear
MDPs, even in the single-reward PAC setting. Our algorithm relies on a novel
procedure which efficiently traverses a linear MDP, collecting samples in any
given "feature direction", and enjoys a sample complexity scaling optimally in
the (linear MDP equivalent of the) maximal state visitation probability. We
show that this exploration procedure can also be applied to solve the problem
of obtaining "well-conditioned" covariates in linear MDPs.

29
The categories of this paper are: funct-an
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con
0
The categories of this paper are: hep-ph
14
The categories of this paper are: math-ph
28
The categories of this paper are: alg-geom
13
The categories of this paper are: quant-ph
25
The categories of this paper are: patt-sol
27
The categories of this paper are: comp-gas
33
The categories of this paper are: plasm-ph
5
The categories of this paper are: gr-qc
21
The categories of this paper are: adap-org
19
The categories of this paper are: eess
4
The categories of this paper are: cond-mat
24
The categories of this paper are: dg-ga
15
The categories of this paper are: nucl-ex