Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: AlterSGD: Finding Flat Minima for Continual Learning by Alternative
  Training   Deep neural networks suffer from catastrophic forgetting when learning
multiple knowledge sequentially, and a growing number of approaches have been
proposed to mitigate this problem. Some of these methods achieved considerable
performance by associating the flat local minima with forgetting mitigation in
continual learning. However, they inevitably need (1) tedious hyperparameters
tuning, and (2) additional computational cost. To alleviate these problems, in
this paper, we propose a simple yet effective optimization method, called
AlterSGD, to search for a flat minima in the loss landscape. In AlterSGD, we
conduct gradient descent and ascent alternatively when the network tends to
converge at each session of learning new knowledge. Moreover, we theoretically
prove that such a strategy can encourage the optimization to converge to a flat
minima. We verify AlterSGD on continual learning benchmark for semantic
segmentation and the empirical results show that we can significantly mitigate
the forgetting and outperform the state-of-the-art methods with a large margin
under challenging continual learning protocols.

13
The categories of this paper are: quant-ph
5
The categories of this paper are: gr-qc
25
The categories of this paper are: patt-sol
26
The categories of this paper are: bayes-an
35
The categories of this paper are: mtrl-th
10
The categories of this paper are: q-bio
18
The categories of this paper are: econ
37
The categories of this paper are: supr-con
14
The categories of this paper are: math-ph
28
The categories of this paper are: alg-geom
23
The categories of this paper are: solv-int
12
The categories of this paper are: nucl-th
9
The categories of this paper are: nlin
36
The categories of this paper are: cmp-lg
27
The categories of this paper are: comp-gas
33
The categories of this paper are: plasm-ph