Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Scalable and Order-robust Continual Learning with Additive Parameter
  Decomposition   While recent continual learning methods largely alleviate the catastrophic
problem on toy-sized datasets, some issues remain to be tackled to apply them
to real-world problem domains. First, a continual learning model should
effectively handle catastrophic forgetting and be efficient to train even with
a large number of tasks. Secondly, it needs to tackle the problem of
order-sensitivity, where the performance of the tasks largely varies based on
the order of the task arrival sequence, as it may cause serious problems where
fairness plays a critical role (e.g. medical diagnosis). To tackle these
practical challenges, we propose a novel continual learning method that is
scalable as well as order-robust, which instead of learning a completely shared
set of weights, represents the parameters for each task as a sum of task-shared
and sparse task-adaptive parameters. With our Additive Parameter Decomposition
(APD), the task-adaptive parameters for earlier tasks remain mostly unaffected,
where we update them only to reflect the changes made to the task-shared
parameters. This decomposition of parameters effectively prevents catastrophic
forgetting and order-sensitivity, while being computation- and
memory-efficient. Further, we can achieve even better scalability with APD
using hierarchical knowledge consolidation, which clusters the task-adaptive
parameters to obtain hierarchically shared parameters. We validate our network
with APD, APD-Net, on multiple benchmark datasets against state-of-the-art
continual learning methods, which it largely outperforms in accuracy,
scalability, and order-robustness.

13
The categories of this paper are: quant-ph
3
The categories of this paper are: physics
22
The categories of this paper are: chao-dyn
20
The categories of this paper are: acc-phys
37
The categories of this paper are: supr-con
24
The categories of this paper are: dg-ga
28
The categories of this paper are: alg-geom
29
The categories of this paper are: funct-an
35
The categories of this paper are: mtrl-th
11
The categories of this paper are: hep-lat
23
The categories of this paper are: solv-int
10
The categories of this paper are: q-bio
4
The categories of this paper are: cond-mat
2
The categories of this paper are: math
8
The categories of this paper are: hep-ex
7
The categories of this paper are: hep-th