\part{Algorithms \& Implementations}

\chapter{Hamiltonian Monte Carlo Sampling}\label{hmc.chapter}

\noindent
This part of the manual details the algorithm implementations used by
Stan and how to configure them. This chapter presents the Hamiltonian
Monte Carlo (HMC) algorithm and its adaptive variant the no-U-turn
sampler (NUTS) along with details of their implementation and
configuration in Stan; the next two chapters present Stan's optimizers
and diagnostics.

\section{Hamiltonian Monte Carlo}

Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC)
method that uses the derivatives of the density function being sampled
to generate efficient transitions spanning the posterior (see, e.g.,
\citep{Betancourt-Girolami:2013,Neal:2011} for more details). It uses
an approximate Hamiltonian dynamics simulation based on numerical
integration which is then corrected by performing a
Metropolis acceptance step.

This section translates the presentation of HMC by
\cite{Betancourt-Girolami:2013} into the notation of
\cite{GelmanEtAl:2013}.

\subsection{Target Density}

The goal of sampling is to draw from a density $p(\theta)$ for
parameters $\theta$.  This is typically a Bayesian posterior
$p(\theta|y)$ given data $y$, and in particular, a Bayesian posterior
coded as a Stan program.

\subsection{Auxiliary Momentum Variable}

HMC introduces auxiliary momentum variables $\rho$ and draws from a
joint density
%
\[
p(\rho,\theta) = p(\rho|\theta) p(\theta).
\]
%
In most applications of HMC, including Stan, the auxiliary density is
a multivariate normal that does not depend on the parameters $\theta$,
\[
\rho \sim \distro{MultiNormal}(0, \Sigma).
\]
The covariance matrix $\Sigma$ acts as a Euclidean metric to rotate
and scale the target distribution; see \citep{Betancourt-Stein:2011}
for details of the geometry.

In Stan, this matrix may be set to the identity matrix (i.e., unit
diagonal) or estimated from warmup draws and optionally restricted
to a diagonal matrix. The inverse $\Sigma^{-1}$ is known as the mass
matrix, and will be a unit, diagonal, or dense if $\Sigma$ is.

\subsection{The Hamiltonian}

The joint density $p(\rho,\theta)$ defines a Hamiltonian
%
\begin{eqnarray*}
H(\rho,\theta) & = & - \log p(\rho,\theta)
\\[3pt]
& = & - \log p(\rho|\theta) - \log p(\theta).
\\[3pt]
& = & T(\rho|\theta) + V(\theta),
\end{eqnarray*}
%
where the term
\[
T(\rho|\theta) = - \log p(\rho | \theta)
\]
is called the ``kinetic energy'' and the term
\[
V(\theta) = - \log p(\theta)
\]
is called the ``potential energy.''  The potential energy is specified
by the Stan program through its definition of a log density.

\subsection{Generating Transitions}

Starting from the current value of the parameters $\theta$, a
transition to a new state is generated in two stages before being
subjected to a Metropolis accept step.

First, a value for the momentum is drawn independently of the current
parameter values,
%
\[
\rho \sim \distro{MultiNormal}(0,\Sigma).
\]
%
Thus momentum does not persist across iterations.

Next, the joint system $(\theta,\rho)$ made up of the current
parameter values $\theta$ and new momentum $\rho$ is evolved
via Hamilton's equations,
%
\[
\begin{array}{rcccl}
\displaystyle
\frac{d\theta}{dt}
& = &
\displaystyle
+ \frac{\partial H}{\partial \rho}
& = &
\displaystyle
+ \frac{\partial T}{\partial \rho}
\\[12pt]
\displaystyle
\frac{d\rho}{dt}
& = &
\displaystyle
- \frac{\partial H}{\partial \theta }
& = &
\displaystyle
- \frac{\partial T}{\partial \theta}
- \frac{\partial V}{\partial \theta}.
\end{array}
\]
%
With the momentum density being independent of the target density,
i.e., $p(\rho|\theta) = p(\rho)$, the first term in the
momentum time derivative, ${\partial T} / {\partial \theta}$ is
zero, yielding the pair time derivatives
%
\begin{eqnarray*}
\frac{d \theta}{d t} & = & +\frac{\partial T}{\partial \rho}
\\[2pt]
\frac{d \rho}{d t} & = & -\frac{\partial V}{\partial \theta}.
\end{eqnarray*}

\subsection{Leapfrog Integrator}

The last section leaves a two-state differential equation to solve.
Stan, like most other HMC implementations, uses the leapfrog
integrator, which is a numerical integration algorithm that's
specifically adapted to provide stable results for Hamiltonian systems
of equations.

Like most numerical integrators, the leapfrog algorithm takes discrete
steps of some small time interval $\epsilon$. The leapfrog algorithm
begins by drawing a fresh momentum term independently of the parameter
values $\theta$ or previous momentum value.
%
\[
\rho \sim \distro{MultiNormal}(0,\Sigma).
\]
It then alternates half-step updates of the momentum and full-step
updates of the position.
%
\vspace*{-6pt}
\begin{eqnarray*}
\rho & \leftarrow
     & \rho \, - \, \frac{\epsilon}{2} \frac{\partial V}{\partial \theta}
\\[6pt]
\theta & \leftarrow
       & \theta \, + \, \epsilon \, \Sigma \, \rho
\\[6pt]
\rho & \leftarrow
     & \rho \, - \, \frac{\epsilon}{2} \frac{\partial V}{\partial \theta}.
\end{eqnarray*}
%
By applying $L$ leapfrog steps, a total of $L \, \epsilon$ time is
simulated. The resulting state at the end of the simulation ($L$
repetitions of the above three steps) will be denoted
$(\rho^{*},\theta^{*})$.

The leapfrog integrator's error is on the order of $\epsilon^3$ per
step and $\epsilon^2$ globally, where $\epsilon$ is the time interval
(also known as the step size);  \cite{LeimkuhlerReich:2004} provide a
detailed analysis of numerical integration for Hamiltonian systems,
including a derivation of the error bound for the leapfrog
integrator.

\subsection{Metropolis Accept Step}

If the leapfrog integrator were perfect numerically, there would no
need to do any more randomization per transition than generating a
random momentum vector. Instead, what is done in practice to account
for numerical errors during integration is to apply a Metropolis
acceptance step, where the probability of keeping the proposal
$(\rho^{*},\theta^{*})$ generated by transitioning from $(\rho,\theta)$ is
%
\[
\min\!\left(1, \ \exp\!\left( H(\rho,\theta) - H(\rho^{*},\theta^{*})\right)\right).
\]
%
If the proposal is not accepted, the previous parameter value is
returned for the next draw and used to initialize the next iteration.

\subsection{Algorithm Summary}

The Hamiltonian Monte Carlo algorithm starts at a specified initial
set of parameters $\theta$; in Stan, this value is either
user-specified or generated randomly. Then, for a given number of
iterations, a new momentum vector is sampled and the current value of
the parameter $\theta$ is updated using the leapfrog integrator with
discretization time $\epsilon$ and number of steps $L$ according to
the Hamiltonian dynamics. Then a Metropolis acceptance step is
applied, and a decision is made whether to update to the new state
$(\theta^{*},\rho^{*})$ or keep the existing state.

\section{HMC Algorithm Parameters}

The Hamiltonian Monte Carlo algorithm has three parameters which must
be set,
%
\begin{itemize}
\item discretization time $\epsilon$,
\item mass matrix $\Sigma^{-1}$, and
\item number of steps taken $L$.
\end{itemize}
%
In practice, sampling efficiency, both in terms of iteration speed and
iterations per effective sample, is highly sensitive to these three
tuning parameters \citep{Neal:2011,Hoffman-Gelman:2014}.

If $\epsilon$ is too large, the leapfrog integrator will be inaccurate
and too many proposals will be rejected. If $\epsilon$ is too small,
too many small steps will be taken by the leapfrog integrator leading
to long simulation times per interval. Thus the goal is to balance the
acceptance rate between these extremes.

If $L$ is too small, the trajectory traced out in each iteration will
be too short and sampling will devolve to a random walk.  If $L$ is
too large, the algorithm will do too much work on each iteration.

If the mass matrix $\Sigma$ is poorly suited to the covariance of the
posterior, the step size $\epsilon$ will have to be decreased to
maintain arithmetic precision while at the same time, the number of
steps $L$ is increased in order to maintain simulation time to ensure
statistical efficiency.

\subsection{Integration Time}

The actual integration time is $L \, \epsilon$, a function of number
of steps.  Some interfaces to Stan set an approximate integration time
$t$ and the discretization interval (step size) $\epsilon$.  In these
cases, the number of steps will be rounded down as
\[
L = \left\lfloor \frac{t}{\epsilon} \right\rfloor.
\]
and the actual integration time will still be $L \, \epsilon$.

\subsection{Automatic Parameter Tuning}

Stan is able to automatically optimize $\epsilon$ to match an
acceptance-rate target, able to estimate $\Sigma$ based on warmup
sample iterations, and able to dynamically adapt $L$ on the fly during
sampling (and during warmup) using the no-U-turn sampling (NUTS)
algorithm \citep{Hoffman-Gelman:2014}.

\begin{figure}
\setlength{\unitlength}{0.005in}
\centering
\begin{picture}(1000, 200)
%
\footnotesize
\put(25, 20) { \framebox(75, 200)[c]{I} }
\put(100, 20) { \framebox(25, 200)[c]{II} }
\put(125, 20) { \framebox(50, 200)[c]{II} }
\put(175, 20) { \framebox(100, 200)[c]{II} }
\put(275, 20) { \framebox(200, 200)[c]{II} }
\put(475, 20) { \framebox(400, 200)[c]{II} }
\put(875, 20) { \framebox(50, 200)[c]{III} }
\put(25, 20) { \vector(1, 0){950} }
\put(800, -10) { \makebox(200, 20)[l]{{\small Iteration}} }
%
\end{picture}
\caption{ \small\it Adaptation during warmup occurs in three stages:
  an initial fast adaptation interval (I),
  a series of expanding slow adaptation intervals (II),
  and a final fast adaptation interval (III).
  For HMC, both the fast and slow intervals are used for adapting the
  step size, while the slow intervals are used for learning the
  (co)variance necessitated by the metric.  Iteration numbering starts
  at 1 on the left side of the figure and increases to the right.}%
\label{adaptation.figure}
\end{figure}

When adaptation is engaged (it may be turned off by fixing a step size
and mass matrix), the warmup period is split into three stages, as
illustrated in \reffigure{adaptation}, with two \textit{fast}
intervals surrounding a series of growing \textit{slow} intervals.
Here fast and slow refer to parameters that adapt using local and
global information, respectively; the Hamiltonian Monte Carlo
samplers, for example, define the step size as a fast parameter and
the (co)variance as a slow parameter. The size of the the initial and
final fast intervals and the initial size of the slow interval are all
customizable, although user-specified values may be modified slightly
in order to ensure alignment with the warmup period.

The motivation behind this partitioning of the warmup period is to
allow for more robust adaptation.  The stages are as follows.

\begin{enumerate}
\item[I.]
In the initial fast interval the chain is allowed to converge
towards the typical set,%
%
\footnote{The typical set is a concept borrowed from information
  theory and refers to the neighborhood (or neighborhoods in
  multimodal models) of substantial posterior probability mass through
  which the Markov chain will travel in equilibrium.}
%
with only parameters that can learn from local information adapted.
\item[II.]
After this initial stage parameters that require global
information, for example (co)variances, are estimated in a series of
expanding, memoryless windows; often fast parameters will be adapted
here as well.
\item[III.]
Lastly, the fast parameters are allowed to adapt to the
final update of the slow parameters.
\end{enumerate}

These intervals may be controlled through the following
configuration parameters, all of which must be positive integers:
%
\begin{center}
\begin{tabular}{l|lc}
{\it parameter} & {\it description} & {\it default}
\\ \hline
{\it initial buffer} & width of initial fast adaptation interval
                     & 75
\\
{\it term buffer} & width of final fast adaptation interval
                  &  50
\\
{\it window}  & initial width of slow adaptation interval
              & 25
\end{tabular}
\end{center}

\subsection{Discretization-Interval Adaptation Parameters}

Stan's HMC algorithms utilize dual averaging \citep{Nesterov:2009} to
optimize the step size.%
%
\footnote{This optimization of step size during
adaptation of the sampler should not be confused with running Stan's
optimization method.}
%
This warmup optimization procedure is extremely flexible and for
completeness, Stan exposes each tuning option for dual averaging,
using the notation of \cite{Hoffman-Gelman:2014}. In practice, the
efficacy of the optimization is sensitive to the value of these
parameters, but we do not recommend changing the defaults without
experience with the dual-averaging algorithm. For more information,
see the discussion of dual averaging in \citep{Hoffman-Gelman:2011,
 Hoffman-Gelman:2014}.

The full set of dual-averaging parameters are
%
\begin{center}
\begin{tabular}{c|lcc}
{\it parameter} & {\it description} & {\it constraint} & {\it default}
\\ \hline
$\delta$ & target Metropolis acceptance rate
         & $\delta \in [0,1]$
         & 0.80
\\
$\gamma$ & adaptation regularization scale
         & $\gamma > 0$
         & 0.05
\\
$\kappa$ & adaptation relaxation exponent
         & $\kappa > 0$
         & 0.75
\\
$t_0$    & adaptation iteration offset
         & $t_0 > 0$
         & 10
\end{tabular}
\end{center}
%
By setting the target acceptance parameter $\delta$ to a value closer
to 1 (its value must be strictly less than 1 and its default value is
0.8), adaptation will be forced to use smaller step sizes. This can
improve sampling efficiency (effective sample size per iteration) at the
cost of increased iteration times. Raising the value of $\delta$ will
also allow some models that would otherwise get stuck to overcome
their blockages.

\subsection{Step-Size Jitter}

All implementations of HMC use numerical integrators requiring a step
size (equivalently, discretization time interval). Stan allows the
step size to be adapted or set explicitly. Stan also allows the step
size to be ``jittered'' randomly during sampling to avoid any poor
interactions with a fixed step size and regions of high curvature. The
jitter is a proportion that may be added or subtracted, so the maximum
amount of jitter is 1, which will cause step sizes to be selected in
the range of 0 to twice the adapted step size. The default value is 0,
producing no jitter.

Small step sizes can get HMC samplers unstuck that would otherwise get
stuck with higher step sizes. The downside is that jittering below the
adapted value will increase the number of leapfrog steps required and
thus slow down iterations, whereas jittering above the adapted value
can cause premature rejection due to simulation error in the
Hamiltonian dynamics calculation. See \citep{Neal:2011} for further
discussion of step-size jittering.

\subsection{Euclidean Metric}

All HMC implementations in Stan utilize quadratic kinetic energy
functions which are specified up to the choice of a symmetric,
positive-definite matrix known as a \textit{mass matrix} or, more
formally, a \textit{metric} \citep{Betancourt-Stein:2011}.

If the metric is constant then the resulting implementation is known
as \textit{Euclidean} HMC.  Stan allows for three Euclidean HMC
implementations,
%
\begin{itemize}
\item a unit metric (diagonal matrix of ones),
\item a diagonal metric (diagonal matrix with positive diagonal
  entries), and
\item a dense metric (a dense, symmetric positive definite matrix)
\end{itemize}
%
The user may configure the form of the metric.

If the mass matrix is specified to be diagonal, then regularized
variances are estimated based on the iterations in each slow-stage
block (labeled II in \reffigure{adaptation}).  Each of these estimates
is based only on the iterations in that block.  This allows early
estimates to be used to help guide warmup and then be forgotten later
so that they do not influence the final covariance estimate.

If the mass matrix is specified to be dense, then regularized
covariance estimates will be carried out, regularizing the estimate to
a diagonal matrix, which is itself regularized toward a unit matrix.

Variances or covariances are estimated using Welford accumulators
to avoid a loss of precision over many floating point operations.

\subsubsection{Warmup Times and Estimating the Mass Matrix}

The mass matrix can compensate for linear (i.e. global) correlations
in the posterior which can dramatically improve the performance of HMC
in some problems. This requires knowing the global correlations.

In complex models, the global correlations are usually difficult, if
not impossible, to derivate analytically; for example, nonlinear model
components convolve the scales of the data, so standardizing the data
does not always help.  Therefore, Stan estimates these correlations
online with an adaptive warmup.  In models with strong nonlinear
(i.e. local) correlations this learning can be slow, even with
regularization. This is ultimately why warmup in Stan often needs to
be so long, and why a sufficiently long warmup can yield such
substantial performance improvements.

\subsubsection{Nonlinearity}

The mass matrix compensates for only linear (equivalently global or
position-independent) correlations in the posterior. The hierarchical
parameterizations, on the other hand, affect some of the nasty
nonlinear (equivalently local or position-dependent) correlations
common in hierarchical models.%
%
\footnote{Only in Riemannian HMC does the metric, which can be thought
  of as a position-dependent mass matrix, start compensating for
  nonlinear correlations.}

One of the biggest difficulties with dense mass matrices is the
estimation of the mass matrix itself which introduces a bit of a
chicken-and-egg scenario;  in order to estimate an appropriate mass
matrix for sampling, convergence is required, and in order to
converge, an appropriate mass matrix is required.

\subsubsection{Dense vs.\ Diagonal Mass Matrices}

Statistical models for which sampling is problematic are not typically
dominated by linear correlations for which a dense mass matrix can
adjust.  Rather, they are governed by more complex nonlinear
correlations that are best tackled with better parameterizations or
more advanced algorithms, such as Riemannian HMC.

\subsubsection{Warmup Times and Curvature}

MCMC convergence time is roughly equivalent to the autocorrelation
time.  Because HMC (and NUTS) chains tend to be lowly autocorrelated
they also tend to converge quite rapidly.

This only applies when there is uniformity of curvature across the
posterior, an assumption which is violated in many complex models.
Quite often, the tails have large curvature while the bulk of the
posterior mass is relatively well-behaved; in other words, warmup is slow
not because the actual convergence time is slow but rather because the
cost of an HMC iteration is more expensive out in the tails.

Poor behavior in the tails is the kind of pathology that can be
uncovered by running only a few warmup iterations. By looking at the
acceptance probabilities and step sizes of the first few iterations
provides an idea of how bad the problem is and whether it must be
addressed with modeling efforts such as tighter priors or
reparameterizations.

\subsection{NUTS and its Configuration}

The no-U-turn sampler (NUTS) automatically selects an appropriate
number of leapfrog steps in each iteration in order to allow the
proposals to traverse the posterior without doing unnecessary work.
The motivation is to maximize the expected squared jump distance (see,
e.g., \citep{RobertsEtAl:1997}) at each step and avoid the random-walk behavior
that arises in random-walk Metropolis or Gibbs samplers when there is
correlation in the posterior. For a precise definition of the NUTS
algorithm and a proof of detailed balance, see
\citep{Hoffman-Gelman:2011,
  Hoffman-Gelman:2014}.

NUTS generates a proposal by starting at an initial position
determined by the parameters drawn in the last iteration. It then
generates an independent unit-normal random momentum vector. It then
evolves the initial system both forwards and backwards in time to form
a balanced binary tree. At each iteration of the NUTS algorithm the
tree depth is increased by one, doubling the number of leapfrog steps
and effectively doubles the computation time. The algorithm terminates
in one of two ways, either
%
\begin{itemize}
\item the NUTS criterion (i.e., a U-turn in Euclidean space on a
  subtree) is satisfied for a new subtree or the completed tree, or
\item the depth of the completed tree hits the maximum depth allowed.
\end{itemize}
%
Rather than using a standard Metropolis step, the final parameter
value is selected via multinomial sampling with a bias toward the
second half of the steps in the trajectory \citep{Betancourt:2016}.%
%
\footnote{Stan previously used slice sampling along the trajectory,
  following the original NUTS paper of \cite{Hoffman-Gelman:2014}.}

Configuring the no-U-turn sample involves putting a cap on the depth
of the trees that it evaluates during each iteration.   This is
controlled through a maximum depth parameter.   The number of leapfrog
steps taken is then bounded by 2 to the power of the maximum depth minus 1.

Both the tree depth and the actual number of leapfrog steps computed
are reported along with the parameters in the output as
\code{treedepth\_\_} and \code{n\_leapfrog\_\_}, respectively. Because
the final subtree may only be partially constructed, these two will
always satisfy
%
\[
2^{\mathrm{treedepth} - 1} - 1 < N_{\mathrm{leapfrog}} \le 2^{\mathrm{treedepth} } - 1.
\]

Tree depth is an important diagnostic tool for NUTS. For example, a
tree depth of zero occurs when the first leapfrog step is immediately
rejected and the initial state returned, indicating extreme curvature
and poorly-chosen step size (at least relative to the current
position). On the other hand, a tree depth equal to the maximum depth
indicates that NUTS is taking many leapfrog steps and being terminated
prematurely to avoid excessively long execution time. Taking very many
steps may be a sign of poor adaptation, may be due to targeting a very
high acceptance rate, or may simply indicate a difficult posterior
from which to sample. In the latter case, reparameterization may help
with efficiency. But in the rare cases where the model is correctly
specified and a large number of steps is necessary, the maximum depth
should be increased to ensure that that the NUTS tree can grow as
large as necessary.

\subsection{Sampling without Parameters}

In some situations, such as pure forward data simulation in a directed
graphical model (e.g., where you can work down generatively from known
hyperpriors to simulate parameters and data), there is no need to
declare any parameters in Stan, the model block will be empty, and all
output quantities will be produced in the generated quantities block.
For example, to generate a sequence of $N$ draws from a binomial with
trials $K$ and chance of success $\theta$, the following program suffices.
%
\begin{stancode}
data {
  real<lower=0,upper=1> theta;
  int<lower=0> K;
  int<lower=0> N;
}
model {
}
generated quantities {
  int<lower=0,upper=K> y[N];
  for (n in 1:N)
    y[n] = binomial_rng(K, theta);
}
\end{stancode}
%
This program includes an empty model block because every Stan program
must have a model block, even if it's empty.  For this model, the
sampler must be configured to use the fixed-parameters setting because
there are no parameters.  Without parameter sampling there is no need
for adaptation and the number of warmup iterations should be set to
zero.

Most models that are written to be sampled without parameters will not
declare any parameters, instead putting anything parameter-like in the
data block.  Nevertheless, it is possible to include parameters for
fixed-parameters sampling and initialize them in any of the usual ways
(randomly, fixed to zero on the unconstrained scale, or with
user-specified values).  For example, \code{theta} in the example
above could be declared as a parameter and initialized as a parameter.

\section{General Configuration Options}\label{general-config.section}

Stan's interfaces provide a number of configuration options that are
shared among the MCMC algorithms (this chapter), the optimization
algorithms (\refchapter{optimization-algorithms}), and the diagnostics
(\refchapter{diagnostic-algorithms}).

\subsection{Random Number Generator}

The random-number generator's behavior is fully determined by the
unsigned seed (positive integer) it is started with. If a seed is not
specified, or a seed of 0 or less is specified, the system time is
used to generate a seed. The seed is recorded and included with Stan's
output regardless of whether it was specified or generated randomly
from the system time.

Stan also allows a chain identifier to be specified, which is useful
when running multiple Markov chains for sampling. The chain identifier
is used to advance the random number generator a very large number of
random variates so that two chains with different identifiers draw
from non-overlapping subsequences of the random-number sequence
determined by the seed.  When running multiple chains from a single
command, Stan's interfaces will manage the chain identifiers.

\subsubsection{Replication}

Together, the seed and chain identifier determine the behavior of the
underlying random number generator. See \refchapter{reproducibility} for a
list of requirements for replication.

\subsection{Initialization}

The initial parameter values for Stan's algorithms (MCMC,
optimization, or diagnostic) may be either specified by the user or
generated randomly. If user-specified values are provided, all
parameters must be given initial values or Stan will abort with an
error message.

\subsubsection{User-Defined Initialization}

If the user specifies initial values, they must satisfy the
constraints declared in the model (i.e., they are on the constrained
scale).

\subsubsection{System Constant Zero Initialization}

It is also possible to provide an initialization of 0, which causes
all variables to be initialized with zero values on the unconstrained
scale. The transforms are arranged in such a way that zero
initialization provides reasonable variable initializations for most
parameters, such as 0 for unconstrained parameters, 1 for parameters
constrained to be positive, 0.5 for variables to constrained to lie
between 0 and 1, a symmetric (uniform) vector for simplexes, unit
matrices for both correlation and covariance matrices, and so on. See
\refchapter{variable-transforms} for full details of the
transformations.

\subsubsection{System Random Initialization}

Random initialization by default initializes the parameter values with
values drawn at random from a $\distro{Uniform}(-2,2)$ distribution.
Alternatively, a value other than 2 may be specified for the absolute
bounds. These values are on the unconstrained scale, and are inverse
transformed back to satisfy the constraints declared for parameters.
See \refchapter{variable-transforms} for a complete description of the
transforms used.

Because zero is chosen to be a reasonable default initial value for
most parameters, the interval around zero provides a fairly diffuse
starting point. For instance, unconstrained variables are initialized
randomly in $(-2,2)$, variables constrained to be positive are
initialized roughly in $(0.14,7.4)$, variables constrained to fall
between 0 and 1 are initialized with values roughly in $(0.12,0.88)$.

\section{Divergent Transitions}

The Hamiltonian Monte Carlo algorithms (HMC and NUTS) simulate the
trajectory of a fictitious particle representing parameter values when
subject to a potential energy field, the value of which at a point is
the negative log posterior density (up to a constant that does not
depend on location).  Random momentum is imparted independently in
each direction, by drawing from a standard normal distribution.  The
Hamiltonian is defined to be the sum of the potential energy and
kinetic energy of the system. The key feature of the Hamiltonian is
that it is conserved along the trajectory the particle moves.

In Stan, we use the leapfrog algorithm to simulate the path of a
particle along the trajectory defined by the initial random momentum
and the potential energy field.  This is done by alternating updates
of the position based on the momentum and the momentum based on the
position.  The momentum updates involve the potential energy and are
applied along the gradient.  This is essentially a stepwise
(discretized) first-order approximation of the trajectory.
\cite{LeimkuhlerReich:2004} provide details and error analysis for the
leapfrog algorithm.

A divergence arises when the simulated Hamiltonian trajectory departs
from the true trajectory as measured by departure of the Hamiltonian
value from its initial value.  When this divergence is too high,%
%
\footnote{The current default threshold is a factor of $10^3$, whereas
  when the leapfrog integrator is working properly, the divergences
  will be around $10^{-7}$ or so and do not compound due to the
  symplectic nature of the leapfrog integrator.}
%
the simulation has gone off the rails and cannot be trusted.  The
positions along the simulated trajectory after the Hamiltonian
diverges will never be selected as the next draw of the MCMC
algorithm, potentially reducing Hamiltonian Monte Carlo to a simple
random walk and biasing estimates by not being able to thoroughly
explore the posterior distribution.  \cite{Betancourt:2016b} provides
details of the theory, computation, and practical implications of
divergent transitions in Hamiltonian Monte Carlo.

The Stan interfaces report divergences as warnings and provide ways to
access which iterations encountered divergences.  ShinyStan provides
visualizations that highlight the starting point of divergent
transitions to diagnose where the divergences arise in parameter
space.  A common location is in the neck of the funnel in a centered
parameterization (as in the funnel example discussed in
\refsection{reparameterization}).

If the posterior is highly curved, very small step sizes are required
for this gradient-based simulation of the Hamiltonian to be accurate.
When the step size is too large (relative to the curvature), the
simulation diverges from the true Hamiltonian.  This definition is
imprecise in the same way that stiffness for a differential equation
is imprecise; both are defined by the way they cause traditional
stepwise algorithms to diverge from where they should be.

The primary cause of divergent transitions in Euclidean HMC (other
than bugs in the code) is highly varying posterior curvature, for
which small step sizes are too inefficient in some regions and diverge
in other regions.  If the step size is too small, the sampler becomes
inefficient and halts before making a U-turn (hits the maximum tree
depth in NUTS); if the step size is too large, the Hamiltonian
simulation diverges.

\subsection{Diagnosing and Eliminating Divergences}

In some cases, simply lowering the initial step size and increasing
the target acceptance rate will keep the step size small enough that
sampling can proceed.  In other cases, a reparameterization is
required so that the posterior curvature is more manageable; see the
funnel example in \refsection{reparameterization} for an example.
Before reparameterization, it may be helpful to plot the posterior
draws, highlighting the divergent transitions to see where they arise.
This is marked as a divergent transition in the interfaces; for
example, ShinyStan and RStan have special plotting facilities to
highlight where divergent transitions arise.

\chapter{Transformations of Constrained Variables}\label{variable-transforms.chapter}

\noindent
To avoid having to deal with constraints while simulating the
Hamiltonian dynamics during sampling, every (multivariate) parameter
in a Stan model is transformed to an unconstrained variable behind
the scenes by the model compiler.  The transform is based on the
constraints, if any, in the parameter's definition.  Scalars or the
scalar values in vectors, row vectors or matrices may be constrained
with lower and/or upper bounds.  Vectors may alternatively be
constrained to be ordered, positive ordered, or simplexes.  Matrices
may be constrained to be correlation matrices or covariance matrices.
This chapter provides a definition of the transforms used for each
type of variable.

Stan converts models to \Cpp classes which define probability
functions with support on all of $\reals^K$, where $K$ is the number
of unconstrained parameters needed to define the constrained
parameters defined in the program.  The \Cpp classes also include
code to transform the parameters from unconstrained to constrained and
apply the appropriate Jacobians.

\section{Changes of Variables}\label{change-of-variables.section}

The support of a random variable $X$ with density $p_X(x)$ is that
subset of values for which it has non-zero density,
%
\[
\mbox{supp}(X) = \{ x | p_X(x) > 0 \}.
\]

If $f$ is a total function defined on the support of $X$, then $Y =
f(X)$ is a new random variable.  This section shows how to compute the
probability density function of $Y$ for well-behaved transforms $f$.
The rest of the chapter details the transforms used by Stan.

\subsection{Univariate Changes of Variables}

Suppose $X$ is one dimensional and $f: \mbox{supp}(X) \rightarrow
\reals$ is a one-to-one, monotonic function with a differentiable
inverse $f^{-1}$.  Then the density of $Y$ is given by
%
\[
p_Y(y) = p_X(f^{-1}(y))
         \,
         \left| \, \frac{d}{dy} f^{-1}(y)\, \right|.
\]
The absolute derivative of the inverse transform measures how the
scale of the transformed variable changes with respect to the
underlying variable.

\subsection{Multivariate Changes of Variables}

The multivariate generalization of an absolute derivative is a
Jacobian, or more fully the absolute value of the determinant of the
Jacobian matrix of the transform.  The Jacobian matrix measures the change of
each output variable relative to every input variable and the absolute
determinant uses that to determine the differential change in volume
at a given point in the parameter space.

Suppose $X$ is a $K$-dimensional random variable with probability
density function $p_X(x)$.  A new random variable $Y = f(X)$ may be
defined by transforming $X$ with a suitably well-behaved function $f$.
It suffices for what follows to note that if $f$ is one-to-one
and its inverse $f^{-1}$ has a well-defined Jacobian, then the
density of $Y$ is
%
\[
p_Y(y) = p_X(f^{-1}(y)) \, \left| \, \det \, J_{f^{-1}}(y) \, \right|,
\]
%
where $\det{}$ is the matrix determinant operation and $J_{f^{-1}}(y)$
is the Jacobian matrix of $f^{-1}$ evaluated at $y$.  Taking $x =
f^{-1}(y)$, the Jacobian matrix is defined by
\[
J_{f^{-1}}(y) =
\left[
\begin{array}{ccc}\displaystyle
\frac{\partial x_1}{\partial y_1}
& \cdots
& \displaystyle \frac{\partial x_1}{\partial y_{K}}
\\[6pt]
\vdots & \vdots & \vdots
\\
\displaystyle\frac{\partial x_{K}}{\partial y_1}
& \cdots
& \displaystyle\frac{\partial x_{K}}{\partial y_{K}}
\end{array}
\right].
\]
%
If the Jacobian matrix is triangular, the determinant reduces to the
product of the diagonal entries,
%
\[
\det \, J_{f^{-1}}(y)
= \prod_{k=1}^K \frac{\partial x_k}{\partial y_k}.
\]
%
Triangular matrices naturally arise in situations where the variables
are ordered, for instance by dimension, and each variable's
transformed value depends on the previous variable's transformed
values.  Diagonal matrices, a simple form of triangular matrix, arise
if each transformed variable only depends on a single untransformed
variable.

\section{Lower Bounded Scalar}\label{lower-bound-transform.section}

Stan uses a logarithmic transform for lower and upper bounds.

\subsection{Lower Bound Transform}

If a variable $X$ is declared to have lower bound $a$, it is
transformed to an unbounded variable $Y$, where
%
\[
Y = \log(X - a).
\]

\subsection{Lower Bound Inverse Transform}
%
The inverse of the lower-bound transform maps an unbounded
variable $Y$ to a variable $X$ that is bounded below by $a$ by
%
\[
X = \exp(Y) + a.
\]

\subsection{Absolute Derivative of the Lower Bound Inverse Transform}

The absolute derivative of the inverse transform is
\[
\left| \,
\frac{d}{dy} \left( \exp(y) + a \right)
\, \right|
= \exp(y).
\]
Therefore, given the density $p_X$ of $X$, the density of $Y$ is
%
\[
p_Y(y)
= p_X\!\left( \exp(y) + a \right) \cdot \exp(y).
\]

\section{Upper Bounded Scalar}

Stan uses a negated logarithmic transform for upper bounds.

\subsection{Upper Bound Transform}

If a variable $X$ is declared to have an upper bound $b$, it is
transformed to the unbounded variable $Y$ by
%
\[
Y = \log(b - X).
\]

\subsection{Upper Bound Inverse Transform}
%
The inverse of the upper bound transform converts the unbounded
variable $Y$ to the variable $X$ bounded above by $b$ through
%
\[
X = b - \exp(Y).
\]

\subsection{Absolute Derivative of the Upper Bound Inverse Transform}

The absolute derivative of the inverse of the upper bound transform is
\[
\left| \,
\frac{d}{dy} \left( b - \exp(y) \right)
\, \right|
= \exp(y).
\]
%
Therefore, the density of the unconstrained variable $Y$ is defined in
terms of the density of the variable $X$ with an upper bound of $b$ by
%
\[
p_Y(y)
 =   p_X \!\left( b - \exp(y) \right) \cdot \exp(y).
\]

\section{Lower and Upper Bounded Scalar}\label{logit-transform-jacobian.section}

For lower and upper-bounded variables, Stan uses a scaled and
translated log-odds transform.

\subsection{Log Odds and the Logistic Sigmoid}

The log-odds function is defined for $u \in (0,1)$ by
%
\[
\mbox{logit}(u) = \log \frac{u}{1 - u}.
\]
%
The inverse of the log odds function is the logistic sigmoid, defined
for $v \in (-\infty,\infty)$ by
%
\[
\mbox{logit}^{-1}(v) = \frac{1}{1 + \exp(-v)}.
\]
%
The derivative of the logistic sigmoid is
%
\[
\frac{d}{dy} \mbox{logit}^{-1}(y)
= \mbox{logit}^{-1}(y) \cdot \left( 1 - \mbox{logit}^{-1}(y) \right).
\]

\subsection{Lower and Upper Bounds Transform}

For variables constrained to be in the open interval $(a,b)$, Stan
uses a scaled and translated log-odds transform.  If variable $X$ is
declared to have lower bound $a$ and upper bound $b$, then it is
transformed to a new variable $Y$, where
%
\[
Y = \mbox{logit} \left( \frac{X - a}{b - a} \right).
\]
%

\subsection{Lower and Upper Bounds Inverse Transform}

The inverse of this transform is
%
\[
X = a + (b - a) \cdot \mbox{logit}^{-1}(Y).
\]
%

\subsection{Absolute Derivative of the Lower and Upper Bounds Inverse
  Transform}

The absolute derivative of the inverse transform is given by
\[
\left|
  \frac{d}{dy}
    \left(
      a + (b - a) \cdot \mbox{logit}^{-1}(y)
    \right)
  \right|
= (b - a)
    \cdot \mbox{logit}^{-1}(y)
    \cdot \left( 1 - \mbox{logit}^{-1}(y) \right).
\]
Therefore, the density of the transformed variable $Y$ is
%
\[
p_Y(y)
=
 p_X \! \left( a + (b - a) \cdot \mbox{logit}^{-1}(y) \right)
    \cdot (b - a)
    \cdot \mbox{logit}^{-1}(y)
    \cdot \left( 1 - \mbox{logit}^{-1}(y) \right).
\]
%
Despite the apparent complexity of this expression, most of the terms
are repeated and thus only need to be evaluated once.  Most
importantly, $\mbox{logit}^{-1}(y)$ only needs to be evaluated once,
so there is only one call to $\exp(-y)$.

\section{Ordered Vector}

For some modeling tasks, a vector-valued random variable $X$ is
required with support on ordered sequences.  One example is the set of
cut points in ordered logistic regression (see \refsection{ordered-logistic}).

In constraint terms, an ordered $K$-vector $x \in \reals^K$ satisfies
\[
x_k < x_{k+1}
\]
%
for $k \in \setlist{1,\ldots,K-1}$.

\subsection{Ordered Transform}

Stan's transform follows the constraint directly.  It maps an
increasing vector $x \in \reals^{K}$ to an unconstrained vector $y \in
\reals^K$ by setting
%
\[
y_k
=
\left\{
\begin{array}{ll}
x_1 & \mbox{if } k = 1, \mbox{ and}
\\[4pt]
\log \left( x_{k} - x_{k-1} \right) & \mbox{if } 1 < k \leq K.
\end{array}
\right.
\]

\subsection{Ordered Inverse Transform}

The inverse transform for an unconstrained $y \in \reals^K$ to an
ordered sequence $x \in \reals^K$ is defined by the recursion
%
\[
x_k
=
\left\{
\begin{array}{ll}
y_1 & \mbox{if } k = 1, \mbox{ and}
\\[4pt]
x_{k-1} + \exp(y_k) & \mbox{if } 1 < k \leq K.
\end{array}
\right.
\]
%
$x_k$ can also be expressed iteratively as
\[
x_k = y_1 + \sum_{k'=2}^k \exp(y_{k'}).
\]

\subsection{Absolute Jacobian Determinant of the Ordered
  Inverse Transform}

The Jacobian of the inverse transform $f^{-1}$ is lower triangular,
with diagonal elements for $1 \leq k \leq K$ of
\[
J_{k,k} =
\left\{
\begin{array}{ll}
1 & \mbox{if } k = 1, \mbox{ and}
\\[4pt]
\exp(y_k) & \mbox{if } 1 < k \leq K.
\end{array}
\right.
\]
%
Because $J$ is triangular, the absolute Jacobian determinant is
%
\[
\left| \, \det \, J \, \right|
\ = \
\left| \, \prod_{k=1}^K J_{k,k} \, \right|
\ = \
\prod_{k=2}^K \exp(y_k).
\]

Putting this all together, if $p_X$ is the density of $X$, then the
transformed variable $Y$ has density $p_Y$ given by
%
\[
p_Y(y)
= p_X(f^{-1}(y))
\
\prod_{k=2}^K \exp(y_k).
\]

\section{Unit Simplex}\label{simplex-transform.section}

Variables constrained to the unit simplex show up in multivariate
discrete models as both parameters (categorical and multinomial) and
as variates generated by their priors (Dirichlet and multivariate
logistic).

The unit $K$-simplex is the set of points $x \in \reals^K$ such that
for $1 \leq k \leq K$,
\[
x_k > 0,
\]
and
\[
\sum_{k=1}^K x_k = 1.
\]
%
An alternative definition is to take the convex closure of the
vertices.  For instance, in 2-dimensions, the simplex vertices are the
extreme values $(0,1)$, and $(1,0)$ and the unit 2-simplex is the line
connecting these two points; values such as $(0.3,0.7)$ and
$(0.99,0.01)$ lie on the line.  In 3-dimensions, the basis is
$(0,0,1)$, $(0,1,0)$ and $(1,0,0)$ and the unit 3-simplex is the
boundary and interior of the triangle with these vertices.  Points in
the 3-simplex include $(0.5,0.5,0)$, $(0.2,0.7,0.1)$ and all other
triplets of non-negative values summing to 1.

As these examples illustrate, the simplex always picks out a subspace
of $K-1$ dimensions from $\reals^K$.  Therefore a point $x$ in the
$K$-simplex is fully determined by its first $K-1$ elements $x_1, x_2,
\ldots, x_{K-1}$, with
%
\[
x_K = 1 - \sum_{k=1}^{K-1} x_k.
\]
%

\subsection{Unit Simplex Inverse Transform}

Stan's unit simplex inverse transform may be understood using the
following stick-breaking metaphor.%
%
\footnote{For an alternative derivation of the same transform using
  hyperspherical coordinates, see \citep{Betancourt:2010}.}
%
\begin{quote}
  Take a stick of unit length (i.e., length 1).  Break a piece off and
  label it as $x_1$, and set it aside.  Next, break a piece off what's
  left, label it $x_2$, and set it aside.  Continue doing this until
  you have broken off $K-1$ pieces labeled $(x_1,\ldots,x_{K-1})$.
  Label what's left of the original stick $x_K$.  The vector $x =
  (x_1,\ldots,x_{K})$ is obviously a unit simplex because each piece
  has non-negative length and the sum of their lengths is 1.
\end{quote}
%
This full inverse mapping requires the breaks to be represented as the
fraction in $(0,1)$ of the original stick that is broken off.  These
break ratios are themselves derived from unconstrained values in
$(-\infty,\infty)$ using the inverse logit transform as described
above for unidimensional variables with lower and upper bounds.

More formally, an intermediate vector $z \in \reals^{K-1}$, whose
coordinates $z_k$ represent the proportion of the stick broken off in
step $k$, is defined elementwise for $1 \leq k < K$ by
%
\[
z_k = \mbox{logit}^{-1} \left( y_k
                             + \log \left( \frac{1}{K - k}
                                            \right)
                       \right).
\]
%
The logit term
$\log\left(\frac{1}{K-k}\right) (i.e., \mbox{logit}\left(\frac{1}{K-k+1}\right)$) in
the above definition adjusts the transform so that a
zero vector $y$ is mapped to the simplex $x = (1/K,\ldots,1/K)$.  For instance, if
$y_1 = 0$, then $z_1 = 1/K$; if $y_2 = 0$, then $z_2 = 1/(K-1)$; and
if $y_{K-1} = 0$, then $z_{K-1} = 1/2$.

The break proportions $z$ are applied to determine the stick sizes and
resulting value of $x_k$ for $1 \leq k < K$ by
%
\[
x_k =
\left( 1 - \sum_{k'=1}^{k-1} x_{k'} \right) z_k.
\]
%
The summation term represents the length of the original stick left at
stage $k$.  This is multiplied by the break proportion $z_k$ to yield
$x_k$.  Only $K-1$ unconstrained parameters are required, with
the last dimension's value $x_K$ set to the length of the remaining
piece of the original stick,
\[
x_K = 1 - \sum_{k=1}^{K-1} x_k.
\]

\subsection{Absolute Jacobian Determinant of the Unit-Simplex
  Inverse Transform}

The Jacobian $J$ of the inverse transform $f^{-1}$ is
lower-triangular, with diagonal entries
\[
J_{k,k}
=
\frac{\partial x_k}{\partial y_k}
=
\frac{\partial x_k}{\partial z_k} \,
\frac{\partial z_k}{\partial y_k},
\]
%
where
\[
\frac{\partial z_k}{\partial y_k}
= \frac{\partial}{\partial y_k}
   \mbox{logit}^{-1} \left(
                       y_k + \log \left( \frac{1}{K-k}
                                          \right)
                    \right)
= z_k (1 - z_k),
\]
%
and
%
\[
\frac{\partial x_k}{\partial z_k}
=
\left(
  1 - \sum_{k' = 1}^{k-1} x_{k'}
   \right)
.
\]
%
This definition is recursive, defining $x_k$ in terms of
$x_{1},\ldots,x_{k-1}$.

Because the Jacobian $J$ of $f^{-1}$ is lower triangular and positive, its
absolute determinant reduces to
%
\[
\left| \, \det J \, \right|
\ = \
\prod_{k=1}^{K-1} J_{k,k}
\ = \
\prod_{k=1}^{K-1}
z_k
\,
(1 - z_k)
\
\left(
1 - \sum_{k'=1}^{k-1} x_{k'}
\right)
.
\]
%
Thus the transformed variable $Y = f(X)$ has a density given by
%
\[
p_Y(y)
= p_X(f^{-1}(y))
\,
\prod_{k=1}^{K-1}
z_k
\,
(1 - z_k)
\
\left(
1 - \sum_{k'=1}^{k-1} x_{k'}
\right)
.
\]
%
Even though it is expressed in terms of intermediate values $z_k$,
this expression still looks more complex than it is. The exponential
function need only be evaluated once for each unconstrained parameter
$y_k$; everything else is just basic arithmetic that can be computed
incrementally along with the transform.

\subsection{Unit Simplex Transform}

The transform $Y = f(X)$ can be derived by reversing the stages of the
inverse transform.  Working backwards, given the break proportions
$z$, $y$ is defined elementwise by
%
\[
y_k
= \mbox{logit}(z_k)
- \mbox{log}\left(
   \frac{1}{K-k}
   \right)
.
\]
%
The break proportions $z_k$ are defined to be the ratio of $x_k$ to
the length of stick left after the first $k-1$ pieces have been broken
off,
%
\[
z_k
= \frac{x_k}
       {1 - \sum_{k' = 1}^{k-1} x_{k'}}
.
\]

\section{Unit Vector}\label{unit-vector.section}

An $n$-dimensional vector $x \in \reals^n$ is said to be a unit vector
if it has unit Euclidean length, so that
%
\[
\Vert x \Vert
\ = \ \sqrt{x^{\top}\,x}
\ = \ \sqrt{x_1^2 + x_2^2 + \cdots + x_n^2}
\ = \ 1\ .
\]
%

\subsection{Unit Vector Inverse Transform}

Stan divides an unconstrained vector $y \in \reals^{n}$ by its norm,
$\Vert y \Vert = \sqrt{y^\top y}$, to obtain a unit vector $x$,
%
\[
x = \frac{y}{\Vert y \Vert}.
\]
%

To generate a unit vector, Stan generates points at random in
$\reals^n$ with independent unit normal distributions, which are then
standardized by dividing by their Euclidean length.
\cite{Marsaglia:1972} showed this generates points uniformly at random
on $S^{n-1}$.  That is, if we draw $y_n \sim \distro{Normal}(0, 1)$
for $n \in 1{:}n$, then $x = \frac{y}{\Vert y \Vert}$ has a uniform
distribution over $S^{n-1}$.  This allows us to use an $n$-dimensional
basis for $S^{n-1}$ that preserves local neighborhoods in that points
that are close to each other in $\reals^n$ map to points near each
other in $S^{n-1}$.  The mapping is not perfectly distance preserving,
because there are points arbitrarily far away from each other in
$\reals^n$ that map to identical points in $S^{n-1}$.

\subsubsection{Warning: undefined at zero!}

The above mapping from $\reals^n$ to $S^n$ is not defined at zero.
While this point outcome has measure zero during sampling, and may
thus be ignored, it is the default initialization point and thus unit
vector parameters cannot be initialized at zero.  A simple workaround
is to initialize from a very small interval around zero, which is an
option built into all of the Stan interfaces.

\subsection{Absolute Jacobian Determinant of the Unit Vector Inverse
  Transform}

The Jacobian matrix relating the input vector $y$ to the output vector
$x$ is singular because $x^\top x = 1$ for any non-zero input vector
$y$. Thus, there technically is no unique transformation from $x$ to
$y$. To circumvent this issue, let $r = \sqrt{y^\top y}$ so that $y = r
x$. The transformation from $\left(r, x_{-n}\right)$ to $y$ is
well-defined but $r$ is arbitrary, so we set $r = 1$. In this case,
the determinant of the Jacobian is proportional to $-\frac{1}{2} y^\top y$,
which is the kernel of a standard multivariate normal distribution
with $n$ independent dimensions.

\section{Correlation Matrices}\label{correlation-matrix-transform.section}

A $K \times K$ correlation matrix $x$ must be is a symmetric, so that
%
\[
x_{k,k'} = x_{k',k}
\]
for all $k,k' \in \setlist{1,\ldots,K}$, it must have a unit diagonal,
so that
\[
x_{k,k} = 1
\]
for all $k \in \setlist{1,\ldots,K}$, and it must be positive
definite, so that for every non-zero $K$-vector $a$,
\[
a^{\top} x a > 0.
\]
To deal with this rather complicated constraint, Stan implements the
transform of \cite{LewandowskiKurowickaJoe:2009}.  The number of free
parameters required to specify a $K \times K$ correlation matrix is $\binom{K}{2}$.

\subsection{Correlation Matrix Inverse Transform}

It is easiest to specify the inverse, going from its $\binom{K}{2}$
parameter basis to a correlation matrix.  The basis will actually be
broken down into two steps.  To start, suppose $y$ is a vector
containing $\binom{K}{2}$ unconstrained values.  These are first
transformed via the bijective function $\tanh : \reals \rightarrow
(0,1)$
%
\[
\tanh x = \frac{\exp(2x) - 1}{\exp(2x) + 1}.
\]
%
Then, define a $K \times K$ matrix $z$, the upper triangular values of
which are filled by row with the transformed values.  For example, in
the $4 \times 4$ case, there are $\binom{4}{2}$ values arranged as
%
\[
z
=
\left[
\begin{array}{cccc}
0 & \tanh y_1 & \tanh y_2 & \tanh y_4
\\
0 & 0 & \tanh y_3 & \tanh y_5
\\
0 & 0 & 0 & \tanh y_6
\\
0 & 0 & 0 & 0
\end{array}
\right]
.
\]
%
Lewandowski et al.\ show how to bijectively map the array $z$ to a correlation
matrix $x$.  The entry $z_{i,j}$ for $i < j$ is interpreted as the
canonical partial correlation (\CPC) between $i$ and $j$, which is the
correlation between $i$'s residuals and $j$'s residuals when both $i$
and $j$ are regressed on all variables $i'$ such that $i'< i$.
In the case of $i=1$, there are no earlier variables,
so $z_{1,j}$ is just the Pearson correlation between $i$ and $j$.

In Stan, the \LKJ transform is reformulated in terms of a Cholesky factor $w$
of the final correlation matrix, defined for $1 \leq i,j \leq K$ by
%
\[
w_{i,j} =
\left\{
\begin{array}{cl}
%
0 & \mbox{if } i > j,
\\[4pt]
1 & \mbox{if } 1 = i = j,
\\[12pt]
\prod_{i'=1}^{i - 1} \left( 1 - z_{i'\!,\,j}^2 \right)^{1/2}
& \mbox{if } 1 < i = j,
\\[12pt]
z_{i,j} & \mbox{if } 1 = i < j, \mbox{ and}
\\[12pt]
z_{i,j} \, \prod_{i'=1}^{i-1} \left( 1 - z_{i'\!,\,j}^2 \right)^{1/2}
& \mbox{ if } 1 < i < j.
%
\end{array}
\right.
\]
%
This does not require as much computation per matrix entry as it may appear;
calculating the rows in terms of earlier rows yields the more
manageable expression
%
\[
w_{i,j} =
\left\{
\begin{array}{cl}
%
0 & \mbox{if } i > j,
\\[4pt]
1 & \mbox{if } 1 = i = j,
\\[8pt]
z_{i,j} & \mbox{if } 1 = i < j, \mbox{ and}
\\[8pt]
z_{i,j} \ w_{i-1,j} \left( 1 - z_{i-1,j}^2 \right)^{1/2}
& \mbox{ if } 1 < i \leq j.
%
\end{array}
\right.
\]
Given the upper-triangular Cholesky factor $w$, the final correlation
matrix is
\[
x = w^{\top} w.
\]

Lewandowski et al.\ show that the determinant of the correlation
matrix can be defined in terms of the canonical partial correlations
as
%
\[
\mbox{det} \, x = \prod_{i=1}^{K-1} \ \prod_{j=i+1}^K \ (1 - z_{i,j}^2)
 = \prod_{1 \leq i < j \leq K} (1 - z_{i,j}^2),
\]

\subsection{Absolute Jacobian Determinant of the Correlation
  Matrix Inverse Transform}
From the inverse of equation 11 in \cite{LewandowskiKurowickaJoe:2009},
the absolute Jacobian determinant is
%
\[
\sqrt{\prod_{i=1}^{K-1}\prod_{j=i+1}^K \left(1-z_{i,j}^2\right)^{K-i-1}} \
\times \prod_{i=1}^{K-1}\prod_{j=i+1}^K
\frac{\partial \tanh z_{i,j}}{\partial y_{i,j}}
\]
\subsection{Correlation Matrix Transform}

The correlation transform is defined by reversing the steps of the
inverse transform defined in the previous section.

Starting with a correlation matrix $x$, the first step is to find the
unique upper triangular $w$ such that $x = w w^{\top}$.  Because $x$
is positive definite, this can be done by applying the Cholesky
decomposition,
\[
w = \mbox{chol}(x).
\]

The next step from the Cholesky factor $w$ back to the array $z$ of
{\CPC}s is simplified by the ordering of the elements in the
definition of $w$, which when inverted yields
%
\[
z_{i,j} =
\left\{
\begin{array}{cl}
0 & \mbox{if } i \leq j,
\\[8pt]
w_{i,j} & \mbox{if } 1 = i < j, \mbox{ and}
\\[8pt]
{w_{i,j}}
\
\prod_{i'=1}^{i-1} \left( 1 - z_{i'\!,j}^2 \right)^{-2}
& \mbox{if } 1 < i < j.
\end{array}
\right.
\]
The final stage of the transform reverses the hyperbolic tangent
transform, which is defined by
\[
\tanh^{-1} v = \frac{1}{2} \log \left( \frac{1 + v}{1 - v} \right).
\]
The inverse hyperbolic tangent function, $\tanh^{-1}$, is also called
the Fisher transformation.

\section{Covariance Matrices}

A $K \times K$ matrix is a covariance matrix if it is symmetric and
positive definite (see the previous section for definitions).  It
requires $K + \binom{K}{2}$ free parameters to specify a $K \times K$
covariance matrix.

\subsection{Covariance Matrix Transform}

Stan's covariance transform is based on a Cholesky decomposition
composed with a log transform of the positive-constrained diagonal
elements.%
%
\footnote{An alternative to the transform in this section, which can
  be coded directly in Stan, is to parameterize a covariance matrix
  as a scaled correlation matrix.  An arbitrary $K \times K$
  covariance matrix $\Sigma$ can be expressed in terms of a $K$-vector
  $\sigma$ and correlation matrix $\Omega$ as
  \[
  \Sigma = \mbox{diag}(\sigma) \times \Omega \times \mbox{diag}(\sigma),
  \]
  so that each entry is just a deviation-scaled correlation,
  \[
  \Sigma_{m,n} = \sigma_m \times \sigma_n \times \Omega_{m,n}.
  \]
}

If $x$ is a covariance matrix (i.e., a symmetric, positive definite
matrix), then there is a unique lower-triangular matrix $z =
\mbox{chol}(x)$ with positive diagonal entries, called a Cholesky
factor, such that
\[
x = z \, z^{\top}.
\]
The off-diagonal entries of the Cholesky factor $z$ are unconstrained,
but the diagonal entries $z_{k,k}$ must be positive for $1 \leq k
\leq K$.

To complete the transform, the diagonal is log-transformed to produce
a fully unconstrained lower-triangular matrix $y$ defined by
\[
y_{m,n} =
\left\{
\begin{array}{cl}
0 & \mbox{if } m < n,
\\[4pt]
\log z_{m,m} & \mbox{if } m = n, \mbox{ and}
\\[4pt]
z_{m,n} & \mbox{if } m > n.
\end{array}
\right.
\]

\subsection{Covariance Matrix Inverse Transform}

The inverse transform reverses the two steps of the transform.
Given an unconstrained lower-triangular $K \times K$ matrix $y$, the
first step is to recover the intermediate matrix $z$ by reversing the
log transform,
\[
z_{m,n} =
\left\{
\begin{array}{cl}
0 & \mbox{if } m < n,
\\[4pt]
\exp(y_{m,m}) & \mbox{if } m = n, \mbox{ and}
\\[4pt]
y_{m,n} & \mbox{if } m > n.
\end{array}
\right.
\]
%
The covariance matrix $x$ is recovered from its Cholesky factor $z$ by
taking
%
\[
x = z \, z^{\top}.
\]

\subsection{Absolute Jacobian Determinant of the Covariance
  Matrix Inverse Transform}

The Jacobian is the product of the Jacobians of the exponential
transform from the unconstrained lower-triangular matrix $y$ to matrix
$z$ with positive diagonals and the product transform from the
Cholesky factor $z$ to $x$.

The transform from unconstrained $y$ to Cholesky factor $z$ has a
diagonal Jacobian matrix, the absolute determinant of which is thus
%
\[
\prod_{k=1}^K  \frac{\partial}{\partial_{y_{k,k}}} \, \exp(y_{k,k})
\ = \
\prod_{k=1}^K \exp(y_{k,k})
\ = \
\prod_{k=1}^K z_{k,k}.
\]

The Jacobian matrix of the second transform from the Cholesky factor $z$ to
the covariance matrix $x$ is also triangular, with diagonal entries
corresponding to pairs $(m,n)$ with $m \geq n$, defined by
\[
\frac{\partial}{\partial z_{m,n}}
\left( z \, z^{\top} \right)_{m,n}
\ = \
\frac{\partial}{\partial z_{m,n}}
\left( \sum_{k=1}^K z_{m,k} \, z_{n,k} \right)
\ = \
\left\{
\begin{array}{cl}
2 \, z_{n,n} & \mbox{if } m = n \mbox{ and }
\\[4pt]
z_{n,n} & \mbox{if } m > n.
\end{array}
\right.
\]
%
The absolute Jacobian determinant of the second transform is thus
\[
2^{K} \ \prod_{m = 1}^{K} \ \prod_{n=1}^{m} z_{n,n}
\ = \
\prod_{n=1}^K \ \prod_{m=n}^K z_{n,n}
\ = \
2^{K} \ \prod_{k=1}^K z_{k,k}^{K - k + 1}.
\]
Finally, the full absolute Jacobian determinant of the inverse
of the covariance matrix transform from the unconstrained lower-triangular
$y$ to a symmetric, positive definite matrix $x$ is the product of the
Jacobian determinants of the exponentiation and product transforms,
\[
\left( \prod_{k=1}^K z_{k,k} \right)
\left(
2^{K} \ \prod_{k=1}^K z_{k,k}^{K - k + 1}
\right)
\ = \
2^K
\, \prod_{k=1}^K z_{k,k}^{K-k+2}.
\]

Let $f^{-1}$ be the inverse transform from a $K + \binom{K}{2}$-vector
$y$ to the $K \times K$ covariance matrix $x$.  A density function
$p_X(x)$ defined on $K \times K$ covariance matrices is transformed to
the density $p_Y(y)$ over $K + \binom{K}{2}$ vectors $y$ by
\[
p_Y(y) = p_X(f^{-1}(y)) \ 2^K \ \prod_{k=1}^K z_{k,k}^{K-k+2}.
\]

\section{Cholesky Factors of Covariance Matrices}

An $M \times M$ covariance matrix $\Sigma$ can be Cholesky factored to
a lower triangular matrix $L$ such that $L\,L^{\top} = \Sigma$.  If
$\Sigma$ is positive definite, then $L$ will be $M \times M$.  If
$\Sigma$ is only positive semi-definite, then $L$ will be $M \times N$,
with $N < M$.

A matrix is a Cholesky factor for a covariance matrix if and only if
it is lower triangular, the diagonal entries are positive, and $M \geq
N$.  A matrix satisfying these conditions ensures that $L \,
L^{\top}$ is positive semi-definite if $M > N$ and positive definite
if $M = N$.

A Cholesky factor of a covariance matrix requires $N + \binom{N}{2} +
(M - N)N$ unconstrained parameters.

\subsection{Cholesky Factor of Covariance Matrix Transform}

Stan's Cholesky factor transform only requires the first step of the
covariance matrix transform, namely log transforming the positive
diagonal elements.  Suppose $x$ is an $M \times N$ Cholesky factor.
The above-diagonal entries are zero, the diagonal entries are
positive, and the below-diagonal entries are unconstrained.  The
transform required is thus
%
\[
y_{m,n} =
\left\{
\begin{array}{cl}
0 & \mbox{if } m < n,
\\[4pt]
\log x_{m,m} & \mbox{if } m = n, \mbox{ and}
\\[4pt]
x_{m,n} & \mbox{if } m > n.
\end{array}
\right.
\]

\subsection{Cholesky Factor of Covariance Matrix Inverse Transform}

The inverse transform need only invert the logarithm with an
exponentiation.  If $y$ is the unconstrained matrix representation,
then the elements of the constrained matrix $x$ is defined by
\[
x_{m,n} =
\left\{
\begin{array}{cl}
0 & \mbox{if } m < n,
\\[4pt]
\exp(y_{m,m}) & \mbox{if } m = n, \mbox{ and}
\\[4pt]
y_{m,n} & \mbox{if } m > n.
\end{array}
\right.
\]

\subsection{Absolute Jacobian Determinant of Cholesky Factor Inverse Transform}

The transform has a diagonal Jacobian matrix, the absolute determinant
of which is
%
\[
\prod_{n=1}^N  \frac{\partial}{\partial_{y_{n,n}}} \, \exp(y_{n,n})
\ = \
\prod_{n=1}^N \exp(y_{n,n})
\ = \
\prod_{n=1}^N x_{n,n}.
\]

Let $x = f^{-1}(y)$ be the inverse transform from a $N + \binom{N}{2}
+ (M - N)N$ vector to an $M \times N$ Cholesky factor for a covariance
matrix $x$ defined in the previous section.  A density function
$p_X(x)$ defined on $M \times N$ Cholesky factors of covariance
matrices is transformed to the density $p_Y(y)$ over $N + \binom{N}{2}
+ (M - N)N$ vectors $y$ by
%
\[
p_Y(y) = p_X(f^{-1}(y)) \prod_{N=1}^N x_{n,n}.
\]

\section{Cholesky Factors of Correlation Matrices}

A $K \times K$ correlation matrix $\Omega$ is positive definite and
has a unit diagonal.  Because it is positive definite, it can be
Cholesky factored to a $K \times K$ lower-triangular matrix $L$ with
positive diagonal elements such that $\Omega = L\,L^{\top}$.  Because
the correlation matrix has a unit diagonal,
\[
\Omega_{k,k} = L_k\,L_k^{\top} = 1,
\]
each row vector $L_k$ of the Cholesky factor is of unit length.  The
length and positivity constraint allow the diagonal elements of $L$ to
be calculated from the off-diagonal elements, so that a Cholesky
factor for a $K \times K$ correlation matrix requires only
$\binom{K}{2}$ unconstrained parameters.

\subsection{Cholesky Factor of Correlation Matrix Inverse Transform}

It is easiest to start with the inverse transform from the
$\binom{K}{2}$ unconstrained parameters $y$ to the $K \times K$
lower-triangular Cholesky factor $x$.  The inverse transform is based
on the hyperbolic tangent function, $\tanh$, which satisfies $\tanh(x)
\in (-1,1)$.  Here it will function like an inverse logit with a sign
to pick out the direction of an underlying canonical partial
correlation (see \refsection{correlation-matrix-transform} for more
information on the relation between canonical partial correlations and
the Cholesky factors of correlation matrices).

Suppose $y$ is a vector of $\binom{K}{2}$ unconstrained values.  Let
$z$ be a lower-triangular matrix with zero diagonal and below
diagonal entries filled by row.  For example, in the $3 \times 3$
case,
\[
z =
\left[
\begin{array}{ccc}
0 & 0 & 0
\\
\tanh y_1 & 0 & 0
\\
\tanh y_2 & \tanh y_3 & 0
\end{array}
\right]
\]
%
The matrix $z$, with entries in the range $(-1,1)$, is then
transformed to the Cholesky factor $x$, by taking%
%
\footnote{For convenience, a summation with no terms, such as
  $\sum_{j' < 1} x_{i,j'}$, is defined to be 0.  This implies
  $x_{1,1} = 1$ and that $x_{i,1} = z_{i,1}$ for $i > 1$.}
%
\[
x_{i,j}
=
\left\{
\begin{array}{lll}
0 & \mbox{ if } i < j & \mbox{ [above diagonal]}
\\[12pt]
\sqrt{1 - \sum_{j' < j} x_{i,j'}^2}
  & \mbox{ if } i = j & \mbox{ [on diagonal]}
\\[12pt]
z_{i,j} \ \sqrt{1 - \sum_{j' < j} x_{i,j'}^2}
  & \mbox{ if } i > j & \mbox{ [below diagonal]}
\end{array}
\right.
\]
%
In the $3 \times 3$ case, this yields
\[
x =
\left[
\begin{array}{ccc}
1 & 0 & 0
\\[6pt]
z_{2,1} & \sqrt{1 - x_{2,1}^2} & 0
\\[6pt]
z_{3,1} & z_{3,2} \sqrt{1 - x_{3,1}^2}
        & \sqrt{1 - (x_{3,1}^2 + x_{3,2}^2)}
\end{array}
\right],
\]
where the $z_{i,j} \in (-1,1)$ are the $\tanh$-transformed $y$.

The approach is a signed stick-breaking process on the quadratic
(Euclidean length) scale.  Starting from length 1 at $j=1$, each
below-diagonal entry $x_{i,j}$ is determined by the (signed) fraction
$z_{i,j}$ of the remaining length for the row that it consumes. The
diagonal entries $x_{i,i}$ get any leftover length from earlier
entries in their row.  The above-diagonal entries are zero.

\subsection{Cholesky Factor of Correlation Matrix Transform}

Suppose $x$ is a $K \times K$ Cholesky factor for some correlation
matrix.  The first step of the transform reconstructs the intermediate
values $z$ from $x$,
\[
z_{i,j} = \frac{x_{i,j}}{\sqrt{1 - \sum_{j' < j}x_{i,j'}^2}}.
\]
%
The mapping from the resulting $z$ to $y$ inverts
$\tanh$,
\[
y
\ = \
\tanh^{-1} z
\ = \
\frac{1}{2} \left( \log (1 + z) - \log (1 - z) \right).
\]
%

\subsection{Absolute Jacobian Determinant of Inverse Transform}

The Jacobian of the full transform is the product of the Jacobians of
its component transforms.

First, for the inverse transform $z = \tanh y$, the derivative is
%
\[
\frac{d}{dy} \tanh y = \frac{1}{(\cosh y)^2}.
\]
%

Second, for the inverse transform of $z$ to $x$, the resulting
Jacobian matrix $J$ is of dimension $\binom{K}{2} \times
\binom{K}{2}$, with indexes $(i,j)$ for $(i > j)$.  The Jacobian
matrix is lower triangular, so that its determinant is the product of
its diagonal entries, of which there is one for each $(i,j)$ pair,
%
\[
\left| \, \mbox{det} \, J \, \right|
  \ = \ \mathlarger{\prod}_{i > j} \left| \frac{d}{dz_{i,j}} x_{i,j} \right|,
\]
%
where
%
\[
\frac{d}{dz_{i,j}} x_{i,j}
= \sqrt{1 - \sum_{j' < j} x^2_{i,j'}}.
\]
%
So the combined density for unconstrained $y$ is
\[
p_Y(y)
= p_X(f^{-1}(y))
  \ \
  \mathlarger{\prod}_{n < \binom{K}{2}} \frac{1}{(\cosh y)^2}
  \ \
  \mathlarger{\prod}_{i > j} \left( 1 - \sum_{j' < j} x_{i,j'}^2
  \right)^{1/2},
\]
%
where $x = f^{-1}(y)$ is used for notational convenience.  The log
Jacobian determinant of the complete inverse transform $x = f^{-1}(y)$
is given by
\[
\log \left| \, \det J \, \right|
=
-2 \sum_{n \leq \binom{K}{2}}
\log \cosh y
\
+
\
\frac{1}{2} \
\sum_{i > j}
\log \left( 1 - \sum_{j' < j} x_{i,j'}^2 \right)
.
\]

\chapter{Optimization  Algorithms}%
\label{optimization-algorithms.chapter}

\noindent
Stan provides optimization algorithms which find modes of the density
specified by a Stan program. Such modes may be used as parameter
estimates or as the basis of approximations to a Bayesian posterior;
see \refchapter{mle} for background on point estimation.

Stan provides three different optimizers, a Newton optimizer, and two
related quasi-Newton algorithms, BFGS and L-BFGS; see
\citep{NocedalWright:2006} for thorough description and analysis of
all of these algorithms. The L-BFGS algorithm is the default
optimizer. Newton's method is the least efficient of the three, but
has the advantage of setting its own stepsize.

\section{General Configuration}

All of the optimizers are iterative and allow the maximum number of
iterations to be specified;  the default maximum number of iterations
is 2000.

All of the optimizers are able to stream intermediate output reporting
on their progress.  Whether or not to save the intermediate iterations
and stream progress is configurable.

\section{BFGS and L-BFGS Configuration}

\subsection{Convergence Monitoring}

Convergence monitoring in (L-)BFGS is controlled by a number of
tolerance values, any one of which being satisfied causes the
algorithm to terminate with a solution. Any of the convergence tests
can be disabled by setting its corresponding tolerance parameter to
zero.  The tests for convergence are as follows.

\subsubsection{Parameter Convergence}

The parameters $\theta_i$ in iteration $i$ are considered to have
converged with respect to tolerance \code{tol\_param} if
%
\[
|| \theta_{i} - \theta_{i-1} || < \mbox{\code{tol\_param}}.
\]

\subsubsection{Density Convergence}
%
The (unnormalized) log density
$\log p(\theta_{i}|y)$ for the parameters $\theta_i$ in iteration $i$
given data $y$ is considered to have converged with
respect to tolerance \code{tol\_obj} if
%
\[
\left| \log p(\theta_{i}|y) - \log p(\theta_{i-1}|y) \right| <
\mbox{\code{tol\_obj}}.
\]
%
The log density is considered to have converged to within
relative tolerance \code{tol\_rel\_obj} if
%
\[
\frac{\left| \log p(\theta_{i}|y) - \log p(\theta_{i-1}|y) \right|}{\
  \max\left(\left| \log p(\theta_{i}|y)\right|,\left| \log
      p(\theta_{i-1}|y)\right|,1.0\right)}
 < \mbox{\code{tol\_rel\_obj}} * \epsilon.
\]
%

\subsubsection{Gradient Convergence}

The gradient is considered to have converged to 0 relative to a
specified tolerance \code{tol\_grad} if
%
\[
|| g_{i} || < \mbox{\code{tol\_grad}},
\]
where $\nabla_{\theta}$ is the gradient operator with respect to
$\theta$ and $g_{i} = \nabla_{\theta} \log p(\theta_{i}|y)$ is the gradient at
iteration $i$.

The gradient is considered to have converged to 0 relative to a
specified relative tolerance
\code{tol\_rel\_grad} if
%
\[
\frac{g_{i}^T \hat{H}_{i}^{-1} g_{i} }{ \max\left(\left|\log p(\theta_{i}|y)\right|,1.0\right) } < \mbox{\code{tol\_rel\_grad}} * \epsilon,
\]
%
where $\hat{H}_{i}$ is the estimate of the Hessian at iteration $i$,
$|u|$ is the absolute value (L1 norm) of $u$, $||u||$ is the vector
length (L2 norm) of $u$, and $\epsilon \approx 2e-16$ is machine
precision.

\subsection{Initial Step Size}

The initial step size parameter $\alpha$ for BFGS-style optimizers may
be specified. If the first iteration takes a long time (and requires a
lot of function evaluations) initialize $\alpha$ to be the roughly
equal to the $\alpha$ used in that first iteration. The default value
is intentionally small, 0.001, which is reasonable for many problems
but might be too large or too small depending on the objective
function and initialization. Being too big or too small just means
that the first iteration will take longer (i.e., require more gradient
evaluations) before the line search finds a good step length. It's not
a critical parameter, but for optimizing the same model multiple times
(as you tweak things or with different data), being able to tune
$\alpha$ can save some real time.

\subsection{L-BFGS History Size}

L-BFGS has a command-line argument which controls the size of the
history it uses to approximate the Hessian. The value should be less than
the dimensionality of the parameter space and, in general, relatively
small values (5--10) are sufficient; the default value is 5.

If L-BFGS performs poorly but BFGS performs well, consider increasing
the history size. Increasing history size will increase the
memory usage, although this is unlikely to be an issue for typical
Stan models.

\section{General Configuration Options}

The general configuration options for optimization are the same as
those for MCMC;  see \refsection{general-config} for details.

\section{Writing Models for Optimization}

\subsection{Constrained vs.\ Unconstrained Parameters}

For constrained optimization problems, for instance, with a standard
deviation parameter $\sigma$ constrained so that $\sigma > 0$, it can
be much more efficient to declare a parameter \code{sigma} with no
constraints.  This allows the optimizer to easily get close to 0
without having to tend toward $-\infty$ on the $\log \sigma$ scale.

The Jacobian adjustment is not an issue for posterior modes, because
Stan turns off the built-in Jacobian adjustments for optimization.

With unconstrained parameterizations of parameters with constrained
support, it is important to provide a custom initialization that is
within the support.  For example, declaring a vector
%
\begin{quote}
\begin{Verbatim}
vector[M] sigma;
\end{Verbatim}
\end{quote}
%
and using the default random initialization which is
$\distro{Uniform}(-2,2)$ on the unconstrained scale, means that there
is only a $2^{-M}$ chance that the initialization will be within
support.

For any given optimization problem, it is probably worthwhile trying
the program both ways, with and without the constraint, to see which
one is more efficient.

\chapter{Variational Inference}\label{vi-algorithms.chapter}

\noindent
Stan implements an automatic variational inference algorithm, called
Automatic Differentiation Variational Inference (ADVI)
\citep{Kucukelbir:2015}. In this chapter, we describe the specifics of
how ADVI maximizes the variational objective. For a high-level
description, please see \refchapter{vi-advanced}.

\section{Stochastic Gradient Ascent}

ADVI optimizes the ELBO in the real-coordinate space using stochastic
gradient ascent. We obtain noisy (yet unbiased) gradients of the
variational objective using automatic differentiation and Monte Carlo
integration. The algorithm ascends these gradients using an adaptive
stepsize sequence. We evaluate the ELBO also using Monte Carlo
integration and measure convergence similar to the relative tolerance
scheme in Stan's optimization feature.

\subsection{Monte Carlo Approximation of the ELBO}

ADVI uses Monte Carlo integration to approximate the variational
objective function, the ELBO. The number of draws used to
approximate the ELBO is denoted by \texttt{elbo\_samples}. We
recommend a default value of $100$, as we only evaluate the ELBO every
\texttt{eval\_elbo} iterations, which also defaults to $100$.

\subsection{Monte Carlo Approximation of the Gradients}

ADVI uses Monte Carlo integration to approximate the gradients of the
ELBO. The number of draws used to approximate the gradients is
denoted by \texttt{grad\_samples}. We recommend a default value of
$1$, as this is the most efficient. It also a very noisy estimate of
the gradient, but stochastic gradient ascent is capable of following
such gradients.

\subsection{Adaptive Stepsize Sequence}

ADVI uses a finite-memory version of adaGrad \citep{Duchi:2011}. This
has a single parameter that we expose, denoted \texttt{eta}. We now
have a warmup adaptation phase that selects a good value for
\texttt{eta}. The procedure does a heuristic search over \texttt{eta}
values that span 5 orders of magnitude.

\subsection{Assessing Convergence}

ADVI tracks the progression of the ELBO through the stochastic
optimization.  Specifically, ADVI heuristically determines a rolling
window over which it computes the average and the median change of the
ELBO. Should either number fall below a threshold, denoted by
\texttt{tol\_rel\_obj}, we consider the algorithm to have
converged. The change in ELBO is calculated the same way as in Stan's
optimization module.

\chapter{Diagnostic Mode}\label{diagnostic-algorithms.chapter}

\noindent
Stan's diagnostic mode runs a Stan program with data, initializing
parameters either randomly or with user-specified initial values, and
then evaluates the log probability and its gradients. The gradients
computed by the Stan program are compared to values calculated by
finite differences.

Diagnostic mode may be configured with two parameters.
%
\begin{center}
\begin{tabular}{c|lcc}
{\it parameter} & {\it description} & {\it constraints} & {\it
  default}
\\ \hline
{\it $\epsilon$} & finite difference size & $\epsilon > 0$ & 1e--6
\\
{\it error} & error threshold for matching & $\mbox{error} > 0$ & 1e--6
\end{tabular}
\end{center}
%
If the difference between the Stan program's gradient value and that
calculated by finite difference is higher than the specified
threshold, the argument will be flagged.

\section{Output}

Diagnostic mode prints the log posterior density (up to a proportion)
calculated by the Stan program for the specified initial values. For
each parameter, it prints the gradient at the initial parameter values
calculated by Stan's program and by finite differences over Stan's
program for the log probability.

\subsection{Unconstrained Scale}

The output is for the variable values and their gradients are on the
unconstrained scale, which means each variable is a vector of size
corresponding to the number of unconstrained variables required to
define it. For example, an $N \times N$ correlation matrix, requires
$\binom{N}{2}$ unconstrained parameters. The
transformations from constrained to unconstrained parameters are based
on the constraints in the parameter declarations and described in
detail in \refchapter{variable-transforms}.

\subsection{Includes Jacobian}

The log density includes the Jacobian adjustment implied by the
constraints declared on variables; see
\refchapter{variable-transforms} for full details. The Jacobian
adjustment will be turned off if optimization is used in practice, but
there is as of yet no way to turn it off in diagnostic mode.

\section{Configuration Options}

The general configuration options for diagnostics are the same as
those for MCMC; see \refsection{general-config} for details. Initial
values may be specified, or they may be drawn at random. Setting the
random number generator will only have an effect if a random
initialization is specified.

\section{Speed Warning and Data Trimming}

Due to the application of finite differences, the computation time
grows linearly with the number of parameters. This can be require a
very long time, especially in models with latent parameters that grow
with the data size. It can be helpful to diagnose a model with smaller
data sizes in such cases.