Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: DistIR: An Intermediate Representation and Simulator for Efficient
  Neural Network Distribution   The rapidly growing size of deep neural network (DNN) models and datasets has
given rise to a variety of distribution strategies such as data, tensor-model,
pipeline parallelism, and hybrid combinations thereof. Each of these strategies
offers its own trade-offs and exhibits optimal performance across different
models and hardware topologies. Selecting the best set of strategies for a
given setup is challenging because the search space grows combinatorially, and
debugging and testing on clusters is expensive. In this work we propose DistIR,
an expressive intermediate representation for distributed DNN computation that
is tailored for efficient analyses, such as simulation. This enables
automatically identifying the top-performing strategies without having to
execute on physical hardware. Unlike prior work, DistIR can naturally express
many distribution strategies including pipeline parallelism with arbitrary
schedules. Our evaluation on MLP training and GPT-2 inference models
demonstrates how DistIR and its simulator enable fast grid searches over
complex distribution spaces spanning up to 1000+ configurations, reducing
optimization time by an order of magnitude for certain regimes.

30
The categories of this paper are: q-alg
10
The categories of this paper are: q-bio
33
The categories of this paper are: plasm-ph
34
The categories of this paper are: chem-ph
0
The categories of this paper are: hep-ph
19
The categories of this paper are: eess
7
The categories of this paper are: hep-th
15
The categories of this paper are: nucl-ex
16
The categories of this paper are: stat
35
The categories of this paper are: mtrl-th
5
The categories of this paper are: gr-qc
9
The categories of this paper are: nlin
21
The categories of this paper are: adap-org
26
The categories of this paper are: bayes-an
20
The categories of this paper are: acc-phys
11
The categories of this paper are: hep-lat