    Reinforcement learning (RL) has led to increasingly complex looking behavior in recent years. However, such complexity can be misleading and hides over-fitting. We find that visual representations may be a useful metric of complexity, and both correlates well objective optimization and causally effects reward optimization. We then propose curious representation learning (CRL) which allows us to use better visual representation learning algorithms to correspondingly increase visual representation in policy through an intrinsic objective on both simulated environments and transfer to real images. Finally, we show better visual representations induced by CRL allows us to obtain better performance on Atari without any reward than other curiosity objectives.
We present a formulation of curiosity as a visual representation learning problem and show that it allows good visual representations in agents. This paper formulates curiosity based RL training as learning a visual representation model, arguing that focusing on better LR and maximising model loss for novel scenes will get better overall performance.