Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Elastic Gossip: Distributing Neural Network Training Using Gossip-like
  Protocols   Distributing Neural Network training is of particular interest for several
reasons including scaling using computing clusters, training at data sources
such as IOT devices and edge servers, utilizing underutilized resources across
heterogeneous environments, and so on. Most contemporary approaches primarily
address scaling using computing clusters and require high network bandwidth and
frequent communication. This thesis presents an overview of standard approaches
to distribute training and proposes a novel technique involving
pairwise-communication using Gossip-like protocols, called Elastic Gossip. This
approach builds upon an existing technique known as Elastic Averaging SGD
(EASGD), and is similar to another technique called Gossiping SGD which also
uses Gossip-like protocols. Elastic Gossip is empirically evaluated against
Gossiping SGD using the MNIST digit recognition and CIFAR-10 classification
tasks, using commonly used Neural Network architectures spanning Multi-Layer
Perceptrons (MLPs) and Convolutional Neural Networks (CNNs). It is found that
Elastic Gossip, Gossiping SGD, and All-reduce SGD perform quite comparably,
even though the latter entails a substantially higher communication cost. While
Elastic Gossip performs better than Gossiping SGD in these experiments, it is
possible that a more thorough search over hyper-parameter space, specific to a
given application, may yield configurations of Gossiping SGD that work better
than Elastic Gossip.

13
The categories of this paper are: quant-ph
8
The categories of this paper are: hep-ex
22
The categories of this paper are: chao-dyn
23
The categories of this paper are: solv-int
2
The categories of this paper are: math
31
The categories of this paper are: ao-sci
4
The categories of this paper are: cond-mat
28
The categories of this paper are: alg-geom
29
The categories of this paper are: funct-an
21
The categories of this paper are: adap-org
6
The categories of this paper are: astro-ph
34
The categories of this paper are: chem-ph
14
The categories of this paper are: math-ph
26
The categories of this paper are: bayes-an
20
The categories of this paper are: acc-phys
12
The categories of this paper are: nucl-th