Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Reinforcement Learning in Non-Stationary Discrete-Time Linear-Quadratic
  Mean-Field Games   In this paper, we study large population multi-agent reinforcement learning
(RL) in the context of discrete-time linear-quadratic mean-field games
(LQ-MFGs). Our setting differs from most existing work on RL for MFGs, in that
we consider a non-stationary MFG over an infinite horizon. We propose an
actor-critic algorithm to iteratively compute the mean-field equilibrium (MFE)
of the LQ-MFG. There are two primary challenges: i) the non-stationarity of the
MFG induces a linear-quadratic tracking problem, which requires solving a
backwards-in-time (non-causal) equation that cannot be solved by standard
(causal) RL algorithms; ii) Many RL algorithms assume that the states are
sampled from the stationary distribution of a Markov chain (MC), that is, the
chain is already mixed, an assumption that is not satisfied for real data
sources. We first identify that the mean-field trajectory follows linear
dynamics, allowing the problem to be reformulated as a linear quadratic
Gaussian problem. Under this reformulation, we propose an actor-critic
algorithm that allows samples to be drawn from an unmixed MC. Finite-sample
convergence guarantees for the algorithm are then provided. To characterize the
performance of our algorithm in multi-agent RL, we have developed an error
bound with respect to the Nash equilibrium of the finite-population game.

10
The categories of this paper are: q-bio
23
The categories of this paper are: solv-int
11
The categories of this paper are: hep-lat
8
The categories of this paper are: hep-ex
15
The categories of this paper are: nucl-ex
35
The categories of this paper are: mtrl-th
16
The categories of this paper are: stat
14
The categories of this paper are: math-ph
29
The categories of this paper are: funct-an
9
The categories of this paper are: nlin
34
The categories of this paper are: chem-ph
3
The categories of this paper are: physics
5
The categories of this paper are: gr-qc
13
The categories of this paper are: quant-ph
0
The categories of this paper are: hep-ph
33
The categories of this paper are: plasm-ph