Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: D-Cliques: Compensating for Data Heterogeneity with Topology in
  Decentralized Federated Learning   The convergence speed of machine learning models trained with Federated
Learning is significantly affected by heterogeneous data partitions, even more
so in a fully decentralized setting without a central server. In this paper, we
show that the impact of label distribution skew, an important type of data
heterogeneity, can be significantly reduced by carefully designing the
underlying communication topology. We present D-Cliques, a novel topology that
reduces gradient bias by grouping nodes in sparsely interconnected cliques such
that the label distribution in a clique is representative of the global label
distribution. We also show how to adapt the updates of decentralized SGD to
obtain unbiased gradients and implement an effective momentum with D-Cliques.
Our extensive empirical evaluation on MNIST and CIFAR10 demonstrates that our
approach provides similar convergence speed as a fully-connected topology,
which provides the best convergence in a data heterogeneous setting, with a
significant reduction in the number of edges and messages. In a 1000-node
topology, D-Cliques require 98% less edges and 96% less total messages, with
further possible gains using a small-world topology across cliques.

6
The categories of this paper are: astro-ph
12
The categories of this paper are: nucl-th
10
The categories of this paper are: q-bio
16
The categories of this paper are: stat
24
The categories of this paper are: dg-ga
22
The categories of this paper are: chao-dyn
25
The categories of this paper are: patt-sol
14
The categories of this paper are: math-ph
35
The categories of this paper are: mtrl-th
9
The categories of this paper are: nlin
29
The categories of this paper are: funct-an
36
The categories of this paper are: cmp-lg
8
The categories of this paper are: hep-ex
21
The categories of this paper are: adap-org
11
The categories of this paper are: hep-lat
37
The categories of this paper are: supr-con