Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Gradient Coding with Dynamic Clustering for Straggler-Tolerant
  Distributed Learning   Distributed implementations are crucial in speeding up large scale machine
learning applications. Distributed gradient descent (GD) is widely employed to
parallelize the learning task by distributing the dataset across multiple
workers. A significant performance bottleneck for the per-iteration completion
time in distributed synchronous GD is $straggling$ workers. Coded distributed
computation techniques have been introduced recently to mitigate stragglers and
to speed up GD iterations by assigning redundant computations to workers. In
this paper, we consider gradient coding (GC), and propose a novel dynamic GC
scheme, which assigns redundant data to workers to acquire the flexibility to
dynamically choose from among a set of possible codes depending on the past
straggling behavior. In particular, we consider GC with clustering, and
regulate the number of stragglers in each cluster by dynamically forming the
clusters at each iteration; hence, the proposed scheme is called $GC$ $with$
$dynamic$ $clustering$ (GC-DC). Under a time-correlated straggling behavior,
GC-DC gains from adapting to the straggling behavior over time such that, at
each iteration, GC-DC aims at distributing the stragglers across clusters as
uniformly as possible based on the past straggler behavior. For both
homogeneous and heterogeneous worker models, we numerically show that GC-DC
provides significant improvements in the average per-iteration completion time
without an increase in the communication load compared to the original GC
scheme.

17
The categories of this paper are: q-fin
27
The categories of this paper are: comp-gas
22
The categories of this paper are: chao-dyn
28
The categories of this paper are: alg-geom
11
The categories of this paper are: hep-lat
30
The categories of this paper are: q-alg
26
The categories of this paper are: bayes-an
9
The categories of this paper are: nlin
32
The categories of this paper are: atom-ph
14
The categories of this paper are: math-ph
18
The categories of this paper are: econ
10
The categories of this paper are: q-bio
6
The categories of this paper are: astro-ph
31
The categories of this paper are: ao-sci
25
The categories of this paper are: patt-sol
33
The categories of this paper are: plasm-ph