Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Optimizing Reusable Knowledge for Continual Learning via Metalearning   When learning tasks over time, artificial neural networks suffer from a
problem known as Catastrophic Forgetting (CF). This happens when the weights of
a network are overwritten during the training of a new task causing forgetting
of old information. To address this issue, we propose MetA Reusable Knowledge
or MARK, a new method that fosters weight reusability instead of overwriting
when learning a new task. Specifically, MARK keeps a set of shared weights
among tasks. We envision these shared weights as a common Knowledge Base (KB)
that is not only used to learn new tasks, but also enriched with new knowledge
as the model learns new tasks. Key components behind MARK are two-fold. On the
one hand, a metalearning approach provides the key mechanism to incrementally
enrich the KB with new knowledge and to foster weight reusability among tasks.
On the other hand, a set of trainable masks provides the key mechanism to
selectively choose from the KB relevant weights to solve each task. By using
MARK, we achieve state of the art results in several popular benchmarks,
surpassing the best performing methods in terms of average accuracy by over 10%
on the 20-Split-MiniImageNet dataset, while achieving almost zero forgetfulness
using 55% of the number of parameters. Furthermore, an ablation study provides
evidence that, indeed, MARK is learning reusable knowledge that is selectively
used by each task.

31
The categories of this paper are: ao-sci
13
The categories of this paper are: quant-ph
23
The categories of this paper are: solv-int
22
The categories of this paper are: chao-dyn
36
The categories of this paper are: cmp-lg
5
The categories of this paper are: gr-qc
0
The categories of this paper are: hep-ph
26
The categories of this paper are: bayes-an
16
The categories of this paper are: stat
28
The categories of this paper are: alg-geom
35
The categories of this paper are: mtrl-th
6
The categories of this paper are: astro-ph
27
The categories of this paper are: comp-gas
21
The categories of this paper are: adap-org
17
The categories of this paper are: q-fin
33
The categories of this paper are: plasm-ph