Humans can learn task-agnostic priors from interactive experience and utilize the priors for novel tasks without any finetuning. In this paper, we propose Scoring-Aggregating-Planning (SAP), a framework that can learn task-agnostic semantics and dynamics priors from arbitrary quality interactions as well as the corresponding sparse rewards and then plan on unseen tasks in zero-shot condition. The framework finds a neural score function for local regional state and action pairs that can be aggregated to approximate the quality of a full trajectory; moreover, a dynamics model that is learned with self-supervision can be incorporated for planning. Many of previous works that leverage interactive data for policy learning either need massive on-policy environmental interactions or assume access to expert data while we can achieve a similar goal with pure off-policy imperfect data. Instantiating our framework results in a generalizable policy to unseen tasks. Experiments demonstrate that the proposed method can outperform baseline methods on a wide range of applications including gridworld, robotics tasks and video games.
We learn dense scores and dynamics model as priors from exploration data and use them to induce a good policy in new tasks in zero-shot condition. This paper discusses zero shot generalization into new environments, and proposes an approach with results on Grid-World, Super Mario Bros, and 3D Robotics. A method aiming to learn task-agnostic priors for zero-shot generalization, with the idea to employ a modeling approach on top of the model-based RL framework.