Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Intrinisic Gradient Compression for Federated Learning   Federated learning is a rapidly-growing area of research which enables a
large number of clients to jointly train a machine learning model on
privately-held data. One of the largest barriers to wider adoption of federated
learning is the communication cost of sending model updates from and to the
clients, which is accentuated by the fact that many of these devices are
bandwidth-constrained. In this paper, we aim to address this issue by
optimizing networks within a subspace of their full parameter space, an idea
known as intrinsic dimension in the machine learning theory community. We use a
correspondence between the notion of intrinsic dimension and gradient
compressibility to derive a family of low-bandwidth optimization algorithms,
which we call intrinsic gradient compression algorithms. Specifically, we
present three algorithms in this family with different levels of upload and
download bandwidth for use in various federated settings, along with
theoretical guarantees on their performance. Finally, in large-scale federated
learning experiments with models containing up to 100M parameters, we show that
our algorithms perform extremely well compared to current state-of-the-art
gradient compression methods.

13
The categories of this paper are: quant-ph
29
The categories of this paper are: funct-an
32
The categories of this paper are: atom-ph
31
The categories of this paper are: ao-sci
6
The categories of this paper are: astro-ph
17
The categories of this paper are: q-fin
8
The categories of this paper are: hep-ex
3
The categories of this paper are: physics
7
The categories of this paper are: hep-th
26
The categories of this paper are: bayes-an
20
The categories of this paper are: acc-phys
30
The categories of this paper are: q-alg
4
The categories of this paper are: cond-mat
28
The categories of this paper are: alg-geom
11
The categories of this paper are: hep-lat
37
The categories of this paper are: supr-con