Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Correcting Experience Replay for Multi-Agent Communication   We consider the problem of learning to communicate using multi-agent
reinforcement learning (MARL). A common approach is to learn off-policy, using
data sampled from a replay buffer. However, messages received in the past may
not accurately reflect the current communication policy of each agent, and this
complicates learning. We therefore introduce a 'communication correction' which
accounts for the non-stationarity of observed communication induced by
multi-agent learning. It works by relabelling the received message to make it
likely under the communicator's current policy, and thus be a better reflection
of the receiver's current environment. To account for cases in which agents are
both senders and receivers, we introduce an ordered relabelling scheme. Our
correction is computationally efficient and can be integrated with a range of
off-policy algorithms. We find in our experiments that it substantially
improves the ability of communicating MARL systems to learn across a variety of
cooperative and competitive tasks.

33
The categories of this paper are: plasm-ph
8
The categories of this paper are: hep-ex
15
The categories of this paper are: nucl-ex
6
The categories of this paper are: astro-ph
11
The categories of this paper are: hep-lat
30
The categories of this paper are: q-alg
19
The categories of this paper are: eess
28
The categories of this paper are: alg-geom
13
The categories of this paper are: quant-ph
37
The categories of this paper are: supr-con
10
The categories of this paper are: q-bio
21
The categories of this paper are: adap-org
25
The categories of this paper are: patt-sol
17
The categories of this paper are: q-fin
23
The categories of this paper are: solv-int
18
The categories of this paper are: econ