Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: ODE Transformer: An Ordinary Differential Equation-Inspired Model for
  Sequence Generation   Residual networks are an Euler discretization of solutions to Ordinary
Differential Equations (ODE). This paper explores a deeper relationship between
Transformer and numerical ODE methods. We first show that a residual block of
layers in Transformer can be described as a higher-order solution to ODE.
Inspired by this, we design a new architecture, {\it ODE Transformer}, which is
analogous to the Runge-Kutta method that is well motivated in ODE. As a natural
extension to Transformer, ODE Transformer is easy to implement and efficient to
use. Experimental results on the large-scale machine translation, abstractive
summarization, and grammar error correction tasks demonstrate the high
genericity of ODE Transformer. It can gain large improvements in model
performance over strong baselines (e.g., 30.77 and 44.11 BLEU scores on the
WMT'14 English-German and English-French benchmarks) at a slight cost in
inference efficiency.

29
The categories of this paper are: funct-an
21
The categories of this paper are: adap-org
35
The categories of this paper are: mtrl-th
24
The categories of this paper are: dg-ga
31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
4
The categories of this paper are: cond-mat
23
The categories of this paper are: solv-int
9
The categories of this paper are: nlin
12
The categories of this paper are: nucl-th
15
The categories of this paper are: nucl-ex
37
The categories of this paper are: supr-con
27
The categories of this paper are: comp-gas
10
The categories of this paper are: q-bio
11
The categories of this paper are: hep-lat