Environments in Reinforcement Learning (RL) are usually only partially observable. To address this problem, a possible solution is to provide the agent with information about past  observations. While common methods represent this history using a Recurrent Neural Network (RNN), in this paper we propose an alternative representation which is based on the record of the past events observed in a given episode. Inspired by the human memory, these events describe only important changes in the environment and, in our approach, are automatically discovered using self-supervision.
  We evaluate our history representation method using two challenging RL benchmarks: some games of the Atari-57 suite and the 3D environment Obstacle Tower. Using these benchmarks we show the advantage of our solution with respect to common RNN-based approaches.
event discovery to represent the history for the agent in RL The authors study the problem of RL under partially observed settings, and propose a solution that uses a FFNN but provides a history representation, outperforming PPO. This paper proposes a new way to represent past history as input to an RL agent, showing to perform better than PPO and an RNN variant of PPO.