Decisions made by machine learning systems have increasing influence on the world. Yet it is common for machine learning algorithms to assume that no such influence exists. An example is the use of the i.i.d. assumption in online learning for applications such as content recommendation, where the (choice of) content displayed can change users' perceptions and preferences, or even drive them away, causing a shift in the distribution of users. Generally speaking, it is possible for an algorithm to change the distribution of its own inputs. We introduce the term self-induced distributional shift (SIDS) to describe this phenomenon. A large body of work in reinforcement learning and causal machine learning aims to deal with distributional shift caused by deploying learning systems previously trained offline. Our goal is similar, but distinct: we point out that changes to the learning algorithm, such as the introduction of meta-learning, can reveal hidden incentives for distributional shift (HIDS), and aim to diagnose and prevent problems associated with hidden incentives. We design a simple  environment as a "unit test" for HIDS, as well as a content recommendation environment which allows us to disentangle different types of SIDS.   We demonstrate the potential for HIDS to cause unexpected or undesirable behavior in these environments, and propose and test a mitigation strategy. 
Performance metrics are incomplete specifications; the ends don't always justify the means. The authors show how meta-learning reveals the hidden incentives for distributional shift and propose an approach based on swapping learners between environments to reduce self introduced distributional shift. The paper generalizes the inherent incentive for the learner to win by making the task easier in meta-learning to a larger class of problems.