Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Distributed Hierarchical GPU Parameter Server for Massive Scale Deep
  Learning Ads Systems   Neural networks of ads systems usually take input from multiple resources,
e.g., query-ad relevance, ad features and user portraits. These inputs are
encoded into one-hot or multi-hot binary features, with typically only a tiny
fraction of nonzero feature values per example. Deep learning models in online
advertising industries can have terabyte-scale parameters that do not fit in
the GPU memory nor the CPU main memory on a computing node. For example, a
sponsored online advertising system can contain more than $10^{11}$ sparse
features, making the neural network a massive model with around 10 TB
parameters. In this paper, we introduce a distributed GPU hierarchical
parameter server for massive scale deep learning ads systems. We propose a
hierarchical workflow that utilizes GPU High-Bandwidth Memory, CPU main memory
and SSD as 3-layer hierarchical storage. All the neural network training
computations are contained in GPUs. Extensive experiments on real-world data
confirm the effectiveness and the scalability of the proposed system. A 4-node
hierarchical GPU parameter server can train a model more than 2X faster than a
150-node in-memory distributed parameter server in an MPI cluster. In addition,
the price-performance ratio of our proposed system is 4-9 times better than an
MPI-cluster solution.

22
The categories of this paper are: chao-dyn
26
The categories of this paper are: bayes-an
9
The categories of this paper are: nlin
18
The categories of this paper are: econ
5
The categories of this paper are: gr-qc
7
The categories of this paper are: hep-th
33
The categories of this paper are: plasm-ph
19
The categories of this paper are: eess
31
The categories of this paper are: ao-sci
10
The categories of this paper are: q-bio
3
The categories of this paper are: physics
17
The categories of this paper are: q-fin
27
The categories of this paper are: comp-gas
0
The categories of this paper are: hep-ph
28
The categories of this paper are: alg-geom
14
The categories of this paper are: math-ph