Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Frustratingly Easy Transferability Estimation   Transferability estimation has been an essential tool in selecting a
pre-trained model and the layers in it for transfer learning, to transfer, so
as to maximize the performance on a target task and prevent negative transfer.
Existing estimation algorithms either require intensive training on target
tasks or have difficulties in evaluating the transferability between layers. To
this end, we propose a simple, efficient, and effective transferability measure
named TransRate. Through a single pass over examples of a target task,
TransRate measures the transferability as the mutual information between
features of target examples extracted by a pre-trained model and labels of
them. We overcome the challenge of efficient mutual information estimation by
resorting to coding rate that serves as an effective alternative to entropy.
From the perspective of feature representation, the resulting TransRate
evaluates both completeness (whether features contain sufficient information of
a target task) and compactness (whether features of each class are compact
enough for good generalization) of pre-trained features. Theoretically, we have
analyzed the close connection of TransRate to the performance after transfer
learning. Despite its extraordinary simplicity in 10 lines of codes, TransRate
performs remarkably well in extensive evaluations on 26 pre-trained models and
16 downstream tasks.

6
The categories of this paper are: astro-ph
17
The categories of this paper are: q-fin
12
The categories of this paper are: nucl-th
11
The categories of this paper are: hep-lat
3
The categories of this paper are: physics
18
The categories of this paper are: econ
0
The categories of this paper are: hep-ph
15
The categories of this paper are: nucl-ex
10
The categories of this paper are: q-bio
31
The categories of this paper are: ao-sci
27
The categories of this paper are: comp-gas
32
The categories of this paper are: atom-ph
35
The categories of this paper are: mtrl-th
14
The categories of this paper are: math-ph
5
The categories of this paper are: gr-qc
16
The categories of this paper are: stat