Continual learning is a longstanding goal of artificial intelligence, but is often counfounded by catastrophic forgetting that prevents neural networks from learning tasks sequentially. Previous methods in continual learning have demonstrated how to mitigate catastrophic forgetting, and learn new tasks while retaining performance on the previous tasks. We analyze catastrophic forgetting from the perspective of change in classifier likelihood and propose a simple L1 minimization criterion which can be adapted to different use cases. We further investigate two ways to minimize forgetting as quantified by this criterion and propose strategies to achieve finer control over forgetting. Finally, we evaluate our strategies on 3 datasets of varying difficulty and demonstrate improvements over previously known L2 strategies for mitigating catastrophic forgetting.
Another perspective on catastrophic forgetting This paper introduces a framework for combatting catastrophic forgetting based upon changing the loss term to minimise changes in classifier likelihood, obtained via a Taylor series approximation. This paper tries to solve the continual learning prolem by focusing on regularization approaches, and it proposes a L_1 strategy to mitigate the problem.