Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Class-Incremental Learning by Knowledge Distillation with Adaptive
  Feature Consolidation   We present a novel class incremental learning approach based on deep neural
networks, which continually learns new tasks with limited memory for storing
examples in the previous tasks. Our algorithm is based on knowledge
distillation and provides a principled way to maintain the representations of
old models while adjusting to new tasks effectively. The proposed method
estimates the relationship between the representation changes and the resulting
loss increases incurred by model updates. It minimizes the upper bound of the
loss increases using the representations, which exploits the estimated
importance of each feature map within a backbone model. Based on the
importance, the model restricts updates of important features for robustness
while allowing changes in less critical features for flexibility. This
optimization strategy effectively alleviates the notorious catastrophic
forgetting problem despite the limited accessibility of data in the previous
tasks. The experimental results show significant accuracy improvement of the
proposed algorithm over the existing methods on the standard datasets. Code is
available.

33
The categories of this paper are: plasm-ph
18
The categories of this paper are: econ
6
The categories of this paper are: astro-ph
16
The categories of this paper are: stat
0
The categories of this paper are: hep-ph
4
The categories of this paper are: cond-mat
26
The categories of this paper are: bayes-an
31
The categories of this paper are: ao-sci
17
The categories of this paper are: q-fin
20
The categories of this paper are: acc-phys
19
The categories of this paper are: eess
25
The categories of this paper are: patt-sol
9
The categories of this paper are: nlin
10
The categories of this paper are: q-bio
2
The categories of this paper are: math
28
The categories of this paper are: alg-geom