Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Towards Tight Communication Lower Bounds for Distributed Optimisation   We consider a standard distributed optimisation setting where $N$ machines,
each holding a $d$-dimensional function $f_i$, aim to jointly minimise the sum
of the functions $\sum_{i = 1}^N f_i (x)$. This problem arises naturally in
large-scale distributed optimisation, where a standard solution is to apply
variants of (stochastic) gradient descent. We focus on the communication
complexity of this problem: our main result provides the first fully
unconditional bounds on total number of bits which need to be sent and received
by the $N$ machines to solve this problem under point-to-point communication,
within a given error-tolerance. Specifically, we show that $\Omega( Nd \log d /
N\varepsilon)$ total bits need to be communicated between the machines to find
an additive $\epsilon$-approximation to the minimum of $\sum_{i = 1}^N f_i
(x)$. The result holds for both deterministic and randomised algorithms, and,
importantly, requires no assumptions on the algorithm structure. The lower
bound is tight under certain restrictions on parameter values, and is matched
within constant factors for quadratic objectives by a new variant of quantised
gradient descent, which we describe and analyse. Our results bring over tools
from communication complexity to distributed optimisation, which has potential
for further applications.

29
The categories of this paper are: funct-an
4
The categories of this paper are: cond-mat
37
The categories of this paper are: supr-con
14
The categories of this paper are: math-ph
20
The categories of this paper are: acc-phys
25
The categories of this paper are: patt-sol
22
The categories of this paper are: chao-dyn
9
The categories of this paper are: nlin
36
The categories of this paper are: cmp-lg
11
The categories of this paper are: hep-lat
17
The categories of this paper are: q-fin
31
The categories of this paper are: ao-sci
34
The categories of this paper are: chem-ph
18
The categories of this paper are: econ
21
The categories of this paper are: adap-org
5
The categories of this paper are: gr-qc