Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Scalability for Parallel Machine Learning Training Algorithm:
  Dataset Matters   To gain a better performance, many researchers put more computing resource
into an application. However, in the AI area, there is still a lack of a
successful large-scale machine learning training application: The scalability
and performance reproducibility of parallel machine learning training algorithm
are limited and there are a few pieces of research focusing on why these
indexes are limited but there are very few research efforts explaining the
reasons in essence. In this paper, we propose that the sample difference in
dataset plays a more prominent role in parallel machine learning algorithm
scalability. Dataset characters can measure sample difference. These characters
include the variance of the sample in a dataset, sparsity, sample diversity and
similarity in sampling sequence. To match our proposal, we choose four kinds of
parallel machine learning training algorithms as our research objects: (1)
Asynchronous parallel SGD algorithm (Hogwild! algorithm) (2) Parallel model
average SGD algorithm (Mini-batch SGD algorithm) (3) Decenterilization
optimization algorithm, (4) Dual Coordinate Optimization (DADM algorithm).
These algorithms cover different types of machine learning optimization
algorithms. We present the analysis of their convergence proof and design
experiments. Our results show that the characters datasets decide the
scalability of the machine learning algorithm. What is more, there is an upper
bound of parallel scalability for machine learning algorithms.

28
The categories of this paper are: alg-geom
23
The categories of this paper are: solv-int
9
The categories of this paper are: nlin
8
The categories of this paper are: hep-ex
30
The categories of this paper are: q-alg
17
The categories of this paper are: q-fin
34
The categories of this paper are: chem-ph
19
The categories of this paper are: eess
11
The categories of this paper are: hep-lat
12
The categories of this paper are: nucl-th
36
The categories of this paper are: cmp-lg
18
The categories of this paper are: econ
5
The categories of this paper are: gr-qc
35
The categories of this paper are: mtrl-th
37
The categories of this paper are: supr-con
7
The categories of this paper are: hep-th