Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Stabilizing Equilibrium Models by Jacobian Regularization   Deep equilibrium networks (DEQs) are a new class of models that eschews
traditional depth in favor of finding the fixed point of a single nonlinear
layer. These models have been shown to achieve performance competitive with the
state-of-the-art deep networks while using significantly less memory. Yet they
are also slower, brittle to architectural choices, and introduce potential
instability to the model. In this paper, we propose a regularization scheme for
DEQ models that explicitly regularizes the Jacobian of the fixed-point update
equations to stabilize the learning of equilibrium models. We show that this
regularization adds only minimal computational cost, significantly stabilizes
the fixed-point convergence in both forward and backward passes, and scales
well to high-dimensional, realistic domains (e.g., WikiText-103 language
modeling and ImageNet classification). Using this method, we demonstrate, for
the first time, an implicit-depth model that runs with approximately the same
speed and level of performance as popular conventional deep networks such as
ResNet-101, while still maintaining the constant memory footprint and
architectural simplicity of DEQs. Code is available at
https://github.com/locuslab/deq .

14
The categories of this paper are: math-ph
0
The categories of this paper are: hep-ph
9
The categories of this paper are: nlin
20
The categories of this paper are: acc-phys
17
The categories of this paper are: q-fin
5
The categories of this paper are: gr-qc
13
The categories of this paper are: quant-ph
7
The categories of this paper are: hep-th
31
The categories of this paper are: ao-sci
10
The categories of this paper are: q-bio
34
The categories of this paper are: chem-ph
19
The categories of this paper are: eess
15
The categories of this paper are: nucl-ex
33
The categories of this paper are: plasm-ph
30
The categories of this paper are: q-alg
27
The categories of this paper are: comp-gas