\section{Totality}\label{sec:totality}
%% OK \NV{DONE intro (R1)
%% OK Sections 3 and 4 cover "totality" and "termination" respectively. It would be helpful to explain these terms at the start of section 3, so that the two are distinguished appropriately.
%% OK }
Well typed Haskell code can go very wrong:
%
\begin{code}
  *** Exception: Prelude.head: empty list
\end{code}
%
As our first application, let us see how to use
\toolname to statically guarantee the absence
of such exceptions, \ie, to prove various
functions \emph{total}.

\subsection{Specifying Totality}

First, let us see how to specify the notion of
totality inside \toolname. Consider the source of
the above exception:
%
\begin{code}
  head :: [a] -> a
  head (x:_) = x
\end{code}
%
Most of the work towards totality checking is done by
the translation to GHC's Core, in which every function
\emph{is} total, but may explicitly call an \emph{error}
function that takes as input a string that describes the
source of the pattern-match failure and throws an exception.
%
For example @head@ is translated into
%
\begin{code}
  head d = case d of
             x:xs -> x
             []   -> patError "head"
\end{code}

Since every core function is total, but may explicitly
call error functions, to prove that the source function is
total, it suffices to prove that @patError@
will \emph{never} be called.
%
We can specify this requirement by giving the error
functions a @false@ pre-condition:
%
\begin{code}
  patError :: {v:String | False } -> a
\end{code}
%
The pre-condition states that the input type is \emph{uninhabited}
and so an expression containing a call to @patError@ will only type
check if the call is \emph{dead code}.

\subsection{Verifying Totality}

The (core) definition of @head@ does not typecheck
as is; but requires a pre-condition that states that the function
is only called with non-empty lists. Formally, we do so by
defining the alias
%
\begin{code}
  predicate NonEmp X = 0 < len X
\end{code}
%
and then stipulating that
%
\begin{code}
  head :: {v : [a] | NonEmp v} -> a
\end{code}
%
To verify the (core) definition of @head@, \toolname uses the above signature
to check the body in an environment
%
\begin{code}
  d :: {0 < len d}
\end{code}
%
When @d@ is matched with @[]@, the environment is
strengthened with the corresponding refinement from
the definition of @len@, \ie,
%
\begin{code}
  d :: {0 < (len d) && (len d) = 0}
\end{code}
%
Since the formula above is a contradiction, \toolname concludes that the
call to @patError@ is dead code, and thereby verifies the totality
of @head@. Of course, now we have pushed the burden of proof onto clients
of @head@ -- at each such site, \toolname will check that the argument
passed in is indeed a @NonEmp@ list, and if it successfully does so, then
we, at any uses of @head@, can rest assured that @head@ will never throw an
exception.

\mypara{Refinements and Totality}
While the @head@ example is quite simple, in general, refinements make
it easy to prove totality in complex situations, where we must track
dependencies between inputs and outputs. For example, consider the @risers@
function from \cite{catch}:
%
\begin{code}
  risers []       = []
  risers [x]      = [[x]]
  risers (x:y:zs)
    | x <= y      = (x:s) : ss
    | otherwise   = [x] : (s:ss)
    where
      s:ss    = risers (y:etc)
\end{code}
%
The pattern match on the last line is partial; its core translation is
%
\begin{code}
  let (s, ss) = case risers (y:etc) of
                  s:ss -> (s, ss)
                  []   -> patError "..."
\end{code}
%
What if @risers@ returns an empty list?
Indeed, @risers@ \emph{does}, on occasion, return an empty list per its
first equation. However, on close inspection, it turns out that
\emph{if} the input is non-empty, \emph{then} the output is also
non-empty. Happily, we can specify this as:
%
\begin{code}
  risers :: l:_ -> {v:_ | NonEmp l => NonEmp v}
\end{code}

\toolname verifies that @risers@ meets the above specification,
and hence that the @patError@ is dead code as at that
site, the scrutinee is obtained from calling @risers@ with a
@NonEmp@ list.

\mypara{Non-Emptiness via Measures}
Instead of describing non-emptiness indirectly using @len@, a
user could a special measure:
%
\begin{code}
  measure nonEmp  :: [a] -> Prop
    nonEmp (x:xs)   = True
    nonEmp []       = False

  predicate NonEmp X = nonEmp X
\end{code}
%
After which, verification would proceed analagous to the above.

\mypara{Total Totality Checking}
@patError@ is one of many possible errors thrown by non-total functions.
@Control.Exception.Base@ has several others including @recSelError@, @irrefutPatError@, \etc which serve the purpose of making
core translations total.
%
Rather than hunt down and specify @False@ preconditions one
by one, the user may automatically turn on totality checking
by invoking \toolname with the \cmdtotality command line option,
at which point the tool systematically checks that all the above
functions are indeed dead code, and hence, that all definitions are total.

\subsection{Case Studies}

We verified totality of two libraries: \lbhscolour and \lbmap, earlier versions
of which had previously been proven total by \texttt{catch}~\citep{catch}.

\mypara{\lbmap}
is a widely used library for (immutable) key-value maps, implemented
as balanced binary search trees.
Totality verification of \lbmap was quite straightforward.
We had already verified termination and the crucial
binary search invariant~\ref{chapter:abstractrefinements}. To verify
totality it sufficed to simply re-run verification with
the \cmdtotality argument.
%
All the important specifications were already captured by the types,
and no additional changes were needed to prove totality.
%
%% \RJ{was it trivially total? i.e. is it total if you strip out all refinements
%% from specs?}
%% \NV{No, it fails in 6 functions all of which can trivially be reasoned to be total}
%% \NV{hedgeUnion, hedgeDiff, hedgeMerge, submap', join, merge}
%% \NV{The interesting story is that during verification \emph{we accidentally modified}
%% turn a function to partial, see my commit 041f1f0fea4d34ee41f50dbf7ce43e3c084c2743}
%

This case study illustrates an advantage of \toolname over specialized provers
(\eg, \texttt{catch}~\citep{catch}): it can be used to prove totality, termination and
functional correctness at the same time, facilitating a nice reuse of
specifications for multiple tasks.

%% DONE \NV{(R3)
%% DONE Before discussing HsColour, I'd give a brief explanation of what it is.
%% DONE }
\mypara{\lbhscolour} is a library for generating syntax-highlighted LATEX and HTML from
Haskell source files.
Checking \lbhscolour was not so easy, as in some cases assumptions are used about the
structure of the input data:
%
For example, @ACSS.splitSrcAndAnnos@ handles an
input list of @String@s and assumes that whenever
a specific @String@ (say @breakS@) appears then
at least two @String@s (call them @mname@ and @annots@)
follow it in the list.
Thus, for a list @ls@ that starts with @breakS@
the irrefutable pattern  @(_:mname:annots)@ @=@ @ls@
should be total.
%
Though possible, it is currently it is somewhat cumbersome to specify such
properties.
%
As an easy and practical solution,
to prove totality, we added a dynamic check that
validates that the length of the input @ls@ exceeds @2@.

%% measure follows a b c = \case
%%   []   -> true
%%   x:xs -> if x == a then first2 b c xs else follows a b c xs
%%
%% measure first2 b c = \case
%%   []   -> false
%%   x:xs -> x == b && first1 c xs
%%
%% measure first1 c = \case
%%   []   -> false
%%   x:xs -> x == c
%%
%% Worse, \toolname has no way to express such an invariant:
%% \toolname naturally describes invariants that recursively
%% hold for every list element and
%% reaches its limitations when reasoning about non-recursive
%% properties.

In other cases assertions were imposed via monadic checks, \eg @HsColour.hs@ reads the input arguments and
checks their well-formedness using
%
\begin{code}
  when (length f > 1) $ errorOut "..."
\end{code} %$
%
Currently \toolname does not support monadic reasoning that
allows assuming that @(length f <= 1)@
holds when executing the action \emph{following} the @when@ check.
%
Finally, code modifications were required to capture properties
that are cumbersome to express with \toolname.
%
For example, @trimContext@ checks if there is an element that
satisfies @p@ in the list @xs@; if so it defines
%
@ys = dropWhile (not . p) xs@
%
and computes @tail ys@.
%
By the check we know that @ys@ has at least one element, the
one that satisfies @p@.
%
Due to the complexity of this property, we preferred to rewrite the specific code
in a more verification friendly version.

%%% \mynote{Bug}
%%% %
%%% \RJ{WHY? Seems like a simple GHC CHECK?}
%%% %
%%% On the positive side, totality verification revealed a subtle bug:
%%% %
%%% The instance @Enum@ of @Highlight@ does not define the @toEnum@
%%% method. In core, this reduces to a call to the error function
%%% @noMethodBinding@.
%%% %
%%% Even though this totality bug can be tracked by GHC compilation,
%%% it exposes the strengths of our totality checker.

On the whole, while proving totality can be cumbersome
(as in \lbhscolour) it is a nice side benefit of refinement
type checking and can sometimes be a fully automatic corollary
of establishing more interesting safety properties (as in \lbmap).


\chapter{Some more one dimensional problems}\label{c5}
In the previous chapter we solved the Schr\"{o}dinger equation for two extreme
situations, the free particle and a particle in an infinite potential well. In
this chapter, we shall restrict ourselves to one dimensional problems so that
Schr\"{o}dinger's time independent equation remains an ordinary differential
equation. We will further consider only those problems where it is possible to
obtain an exact, analytical solution.

\section{Particle in a finite potential well}\label{c5s1}
This problem is a minor, but an interesting, modification of the problem we
considered in section \ref{c4s2}. The only change in the problem is the
potential,
\begin{equation}\label{c5s1e1}
V(x) = \begin{cases}
0 & \text{ if } -L/2 \le x \le L/2 \\
V & \text{ otherwise.}
\end{cases}
\end{equation}
Schr\"{o}dinger's time independent equation in one dimension is
\begin{equation}\label{c5s1e2}
-\frac{\hslash^2}{2m}\frac{d^2\psi}{dx^2} + V(x)\psi = E\psi.
\end{equation}
In the region $-L/2 \le x \le L/2$, which is in the interior of the well,
the equation becomes
\begin{equation}\label{c5s1e3}
-\frac{\hslash^2}{2m}\frac{d^2\psi}{dx^2} = E\psi
\end{equation}
while outside the well it is
\begin{equation}\label{c5s1e4}
-\frac{\hslash^2}{2m}\frac{d^2\psi}{dx^2} + V\psi = E\psi.
\end{equation}
We can rearrange \eqref{c5s1e3} as
\begin{equation}\label{c5s1e5}
\frac{d^2\psi}{dx^2} + k^2\psi = 0,
\end{equation}
where
\begin{equation}\label{c5s1e6}
k^2 = \frac{2mE}{\hslash^2}.
\end{equation}
The general solution of this equation is,
\begin{equation}\label{c5s1e7}
\psi(x) = \alpha e^{-ikx} + \beta e^{ikx},
\end{equation}
where $\alpha$ and $\beta$ are constants of integration. Unlike the case of
an infinitely deep well, we do not have the boundary conditions like
\eqref{c4s2e24} and \eqref{c4s2e25}. In order to get an appropriate boundary
condition we must solve the Schr\"{o}dinger equation \eqref{c5s1e4} in the
region outside the well. It can be rearranged as
\begin{equation}\label{c5s1e8}
\frac{d^2\psi}{dx^2} + \frac{2m(E - V)}{\hslash^2}\psi = 0.
\end{equation}
Let us consider the case $E < V$ in which the particle does not have enough
energy to cross the potential well. Define
\begin{equation}\label{c5s1e9}
k_o^2 = \frac{2m(V - E)}{\hslash^2} > 0
\end{equation}
so that we can write \eqref{c5s1e8} as
\begin{equation}\label{c5s1e10}
\frac{d^2\psi}{dx^2} - k_o^2\psi = 0.
\end{equation}
The general solution of this equation is
\begin{equation}\label{c5s1e11}
\psi_o(x) = \alpha_0 e^{-k_ox} + \beta_o e^{k_ox}
\end{equation}
where $\alpha_o$ and $\beta_o$ are constants of integration. This solution
indicates that the wavefunction of the particle is \emph{not zero} outside the
potential well. It means that there is a non-zero probability of finding the
particle outside the well in spite of the fact that the particle lacks the
energy to cross the potential barrier. This is one example of `quantum
tunnelling', a phenonemon in which a particle appears in a position where no
classical particle would.

We would want the wave function of the particle to be continuous all
throughtout. Therefore, we insist that
\begin{eqnarray}
\psi_o\left(-\frac{L}{2}\right) &=& \psi\left(-\frac{L}{2}\right)
  \label{c5s1e12} \\
\psi_o\left(\frac{L}{2}\right) &=& \psi\left(\frac{L}{2}\right)
  \label{c5s1e13}
\end{eqnarray}
Further, we also want the solution to converge to zero as $x \rightarrow \pm
\infty$. Therefore, the solution outside the box must be
\begin{equation}\label{c5s1e14}
\psi_o(x) = \begin{cases}
\beta_o e^{k_ox} & \text{ if } x \le -L/2 \\
\alpha_o e^{-k_ox} & \text{ if } x \ge L/2.
\end{cases}
\end{equation}
From equations \eqref{c5s1e12}, \eqref{c5s1e13} and \eqref{c5s1e14} we get
\begin{eqnarray}
\beta_o e^{-k_oL/2} &=& \alpha e^{ikL/2} + \beta e^{-ikL/2} \label{c5s1e15} \\
\alpha_o e^{-k_oL/2} &=& \alpha e^{-ikL/2} + \beta e^{ikL/2} \label{c5s1e16}
\end{eqnarray}
Equation \eqref{c5s1e2} is a second order equation. Therefore, we also insist
on the continuity of the first derivative of $\psi$. We then get the pair of
equations
\begin{eqnarray}
k_o\beta_o e^{-k_oL/2} &=& -ik\alpha e^{ikL/2} + ik\beta e^{-ikL/2}
   \label{c5s1e17} \\
-k_o\alpha_o e^{-k_oL/2} &=& -ik\alpha e^{-ikL/2} + ik\beta e^{ikL/2}.
   \label{c5s1e18}
\end{eqnarray}
Adding equations \eqref{c5s1e15} and \eqref{c5s1e16} we get
\begin{equation}\label{c5s1e19}
(\alpha_o + \beta_o)e^{-k_oL/2} = 2\cos\left(\frac{kL}{2}\right)(\alpha + \beta)
\end{equation}
while subtracting \eqref{c5s1e18} from \eqref{c5s1e17} gives
\begin{equation}\label{c5s1e20}
k_o(\alpha_o + \beta_o)e^{-k_oL/2} = -2\sin\left(\frac{kL}{2}\right)(\alpha +
\beta).
\end{equation}
From equations \eqref{c5s1e19} and \eqref{c5s1e20} we get
\begin{equation}\label{c5s1e21}
k_o + \tan\left(\frac{kL}{2}\right) = 0.
\end{equation}
From equations \eqref{c5s1e6} and \eqref{c5s1e9} it is clear that the above
equation is really a condition on the energy $E$, since the `depth' $V$ of the
potential is a given quantity. Once again we observe that $E$ cannot take
arbitrary values but only those permitted by equation \eqref{c5s1e21}.

\subsection{Problem set - 1}
\begin{enumerate}
\item Equation \eqref{c5s1e21} does not have analytical solutions. Can you
think of a graphical way to solve it?
\item While defining $k_o$ in equation \eqref{c5s1e9} we assumed that $E<V$.
What happens if $E > V$? Without solving the problem mathematically, can you
guess the solution using only physical considerations?
\item Suppose that we define the potential function as
\begin{equation}
V(x) = \begin{cases}
V & \text{ if } -L/2 \le x \le L/2 \\
0 & \text{ otherwise.}
\end{cases}
\end{equation}
so that we instead have the case of a particle incident upon a finite potential
barrier. Consider the case when the particle's energy $E$ is less than $V$.
Will it cross the barrier? If it will then this is a more stark example of
`quantum tunnelling'.
\end{enumerate}

\section{The harmonic oscillator}\label{c5s2}
The harmonic oscillator appears as a toy example in classical mechanics. It is
far more serious in quantum physics. Sidney Coleman, an American theoretical
physicist, once remarked that ``Quantum field theory is harmonic motion taken
to increasing levels of abstraction". We will not have a chance to look at the
simple harmonic oscillator in too many guises in this course. We will use it
to illustrate how to solve a quantum mechanical problem whose classical analogue
is known.

In classical physics, a simple harmonic oscillator is a body of mass $m$
subjected to a force proportional to its displacement from a certain point and
in a direction opposite to the displacement. In one dimension, where the
vector notation is superfluous, we can express the force as
\begin{equation}\label{c5s2e1}
F = -kx,
\end{equation}
where the number $k$ is a constant. Newton's second law for a particle subjected
to such a force is
\begin{equation}\label{c5s2e2}
m\ddot{x} = -kx,
\end{equation}
where the a dot overhead signifies a derivative with respect to time. We
rearrange this equation as
\begin{equation}\label{c5s2e3}
\ddot{x} + \omega^2 x = 0,
\end{equation}
where
\begin{equation}\label{c5s2e4}
\omega = \sqrt{\frac{k}{m}}.
\end{equation}
A general solution of \eqref{c5s2e3} is
\begin{equation}\label{c5s2e5}
x(t) = \alpha e^{-i\omega t} + \beta e^{i\omega t},
\end{equation}
where $\alpha$ and $\beta$ are constants of integration. From this equation it
is evident that the constant $\omega$ is the angular frequency of the harmonic
oscillator. It can be easily show that the particle has a potential energy
\begin{equation}\label{c5s2e6}
V = \frac{1}{2}kx^2
\end{equation}
so that its total energy is
\begin{equation}\label{c5s2e7}
H = T + V = \frac{p^2}{2m} + \frac{1}{2}kx^2.
\end{equation}
The quantity $H$ is called the Hamiltonian and it is the starting point of
our quantum mechanical analysis.

Given a classical Hamiltonian function we write the corresponding quantum
mechanical Hamiltonian operator by replacing the dynamical variables with their
operator representation. Thus, a harmonic oscillator's Hamiltonian is
\begin{equation}\label{c5s2e8}
\hat{H} = \frac{\hat{p}^2}{2m} + \frac{1}{2}m\omega^2\hat{x}^2.
\end{equation}
From equations \eqref{c4s3e1} and \eqref{c4s3e2} we have
\begin{eqnarray}
\hat{x} &=& x \label{c5s2e9} \\
\hat{p} &=& -i\hslash\frac{d}{dx} \label{c5s2e10}
\end{eqnarray}
so that
\begin{equation}\label{c5s2e11}
\hat{H} = -\frac{\hslash^2}{2m}\frac{d^2}{dx^2} + V(x)
\end{equation}
and hence the Schr\"{o}dinger's time independent equation $\hat{H}\psi = E\psi$
is
\begin{equation}\label{c5s2e12}
-\frac{\hslash^2}{2m}\frac{d^2\psi}{dx^2} + \frac{1}{2}m\omega^2x^2\psi = E\psi.
\end{equation}
We can rearrange it as
\begin{equation}\label{c5s2e13}
\frac{d^2\psi}{dx^2} + \left(\frac{2mE}{\hslash^2} - \frac{m^2\omega^2x^2}{\hslash^2}\right)\psi = 0.
\end{equation}
Introduce a dimensionless variable
\begin{equation}\label{c5s2e14}
\xi = \sqrt{\frac{m\omega}{\hslash}}x
\end{equation}
so that
\[
\frac{d\psi}{dx} = \frac{d\psi}{d\xi}\frac{d\xi}{dx} = \sqrt{\frac{m\omega}{\hslash}}\frac{d\psi}{d\xi}
\]
and
\[
\frac{d^2\psi}{dx^2} = \frac{m\omega}{\hslash}\frac{d^2\psi}{d\xi^2}
\]
so that equation \eqref{c5s2e13} becomes
\begin{equation}\label{c5s2e15}
\frac{d^2\psi}{d\xi^2} + \left(\frac{2E}{\hslash\omega} - \xi^2\right)\psi = 0.
\end{equation}
Let
\begin{equation}\label{c5s2e16}
\alpha = \frac{2E}{\hslash\omega}
\end{equation}
so that equation \eqref{c5s2e15} becomes
\begin{equation}\label{c5s2e17}
\frac{d^2\psi}{d\xi^2} + (\alpha - \xi^2)\psi = 0.
\end{equation}
A standard way to solve equations like \eqref{c5s2e17} is to use Frobenius
method. It consists in expressing $\psi$ as a power series and finding a
recurrence relation among the series' coefficients. If we use it for
\eqref{c5s2e17} we get a three term recurrence relation, which is hard to solve.
Instead we first try to guess the asymptotic form of the solution, that is, the
form of the solution as $\xi \rightarrow \pm\infty$. In this limit, the equation
\eqref{c5s2e17} can be approximated to
\[
\frac{d^2\psi}{d\xi^2} - \xi^2\psi = 0
\]
or, more conveniently,
\[
\frac{d^2\psi}{d\xi^2} - (\xi^2 - 1)\psi = 0.
\]
One can readily verify that the solution of this equation is $\psi(\xi) =
e^{-\xi^2/2}$. Therefore, we express the solution of the original equation as
\begin{equation}\label{c5s2e18}
\psi(\xi) = e^{-\xi^2/2}H(\xi),
\end{equation}
where the function $H$ satisfies the equation
\begin{equation}\label{c5s2e19}
\frac{d^2H}{d\xi^2} - 2\xi\frac{dH}{d\xi} + (\alpha - 1)H(\xi) = 0.
\end{equation}
This equation is called as Hermite's differential equation and its solutions are
called as Hermite polynomials. We will solve \eqref{c5s2e19} using Frobenius
method. Let
\begin{equation}\label{c5s2e20}
H(\xi) = \sum_{n \ge 0}a_n\xi^n
\end{equation}
so that
\begin{eqnarray}
H^\prime(\xi) &=& \sum_{n \ge 1}na_n \xi^{n-1} \label{c5s2e21} \\
H^{\prime\prime}(\xi) &=& \sum_{n \ge 2}n(n-1)a_n \xi^{n-2} \label{c5s2e22}
\end{eqnarray}
Substituting \eqref{c5s2e20} to \eqref{c5s2e22} in \eqref{c5s2e19} we get
\begin{equation}\label{c5s2e23}
\sum_{n \ge 2}n(n-1)a_n \xi^{n-2} - 2\sum_{n \ge 1}na_n \xi^n + (\alpha - 1)
\sum_{n \ge 0}a_n\xi^n = 0
\end{equation}
Writing all sums so that their indices start from $0$,
\begin{equation}\label{c5s2e24}
\sum_{n \ge 0}(n+2)(n+1)a_{n+2} \xi^n - 2\sum_{n \ge 0}(n+1)a_{n+1} \xi^{n+1} +
(\alpha - 1)\sum_{n \ge 0}a_n\xi^n = 0
\end{equation}
or
\begin{equation}\label{c5s2e25}
\sum_{n \ge 0}\left[(n+2)(n+1)a_{n+2} + (\alpha - 1)a_n\right]\xi^n -
2\sum_{n \ge 0}(n+1)a_{n+1} \xi^{n+1} = 0.
\end{equation}
If this equation has to be valid for all values of $\xi$ then the coefficient of
every power of $\xi$ must vanish. In particular,
\begin{equation}\label{c5s2e26}
(n+2)(n+1)a_{n+2} + (\alpha - 1)a_n - 2na_n = 0
\end{equation}
or
\begin{equation}\label{c5s2e27}
a_{n+2} = \frac{2n - \alpha + 1}{(n+2)(n+1)}a_n.
\end{equation}
The recurrence relation terminates when
\begin{equation}\label{c5s2e28}
\alpha = 2n + 1
\end{equation}
or, using equation \eqref{c5s2e16}
\begin{equation}\label{c5s2e29}
E = \left(n + \frac{1}{2}\right)\hslash\omega.
\end{equation}
The only solutions that are physically permissible are the ones for which the
recurrence terminates. For in this case, the series of \eqref{c5s2e20} becomes
a polynomial and the complete solution \eqref{c5s2e18} does indeed assume the
asymptotic form of $\psi(\xi) = e^{-\xi^2/2}$. The energy of the physically
permissible solutions is restricted to the discrete set of values in equation
\eqref{c5s2e29}. For a particular $n$, we distingush the energy eigenvalue by
denoting it as $E_n$ and the corresponding polynomial as $H_n$. Thus, the
eigenfunctions of the Hamiltonian \eqref{c5s2e11} are
\begin{equation}\label{c5s2e30}
\psi_n(\xi) = e^{-\xi^2/2}H_n(\xi)
\end{equation}
and the corresponding eigenvalues are
\begin{equation}\label{c5s3e31}
E_n = \left(n + \frac{1}{2}\right)\hslash\omega,
\end{equation}
where $n = 0, 1, 2, \ldots$. The lowest energy of a harmonic oscillator is
\begin{equation}\label{c5s3e32}
E_0 = \frac{1}{2}\hslash\omega.
\end{equation}
It is \emph{not} zero. Thus, a quantum harmonic oscillator is never at rest. If
it were then it would have violated Heisenberg's uncertainty relation. We also
note that
\begin{equation}\label{c5s2e33}
E_{n+1} - E_n = \hslash\omega.
\end{equation}
Thus, when a harmonic oscillator goes from the ground state of $n = 0$ to the
first excited state of $n = 1$, it absorbs an energy $\hslash\omega$. When it
goes to the second exsited state, it would have absorbed two quanta, each one
of energy $\hslash\omega$. In general, when the harmonic oscillator is in the
$m$th excited state, it would have absorbed $m$ quanta of energy $\hslash
\omega$.

It is possible to consider a crystal as an ensemble (the French word for a
collection) of harmonic oscillators of various frequencies. An individual
harmonic oscillator corresponds to energy quanta of a specific frequency. In
the case of a vibrating crystal, the energy quanta are called \emph{phonons}.
A crystal has $m_1$ phonons of frequency $\omega_1$ if the harmonic oscillator
with frequency $\omega_1$ is in the energy state $E_{m_1} = (m_1 + 1/2)\hslash
\omega_1$.

The harmonic oscillator we considered so far was a mass $m$ that was subject to
a force $-kx$ and we showed that its Hamiltonian is given by equation
\eqref{c5s2e7}
\[
H = \frac{p^2}{2m} + \frac{1}{2}kx^2.
\]
If we choose our units such that $m = 1$ and $k = 1$ and we start calling the
coordinate $x$ as $q$ then the Hamiltonian can be written as
\begin{equation}\label{c5s2e34}
H = \frac{p^2}{2} + \frac{q^2}{2}.
\end{equation}
Any system whose Hamiltonian can be written in this form is called a harmonic
oscillator. The quantities $p$ and $q$ are called generalised momentum and
generalised coordinate. It is possible to write the Hamiltonian of
electromagnetic radiation in a black body as a sum of terms of the form
\eqref{c5s2e34}. Analogous to a vibrating crystal, the electromagnetic radiation
in a black body can also be considered to be an ensemble of harmonic oscillators
of various frequencies. The energy quanta of these oscillators are called
\emph{photons}. If an oscillator of frequency $\omega_1$ is in the $m_1$th
excited state then we say that there are $m_1$ photons of frequency $\omega_1$
in the black body.

When one utters the word  `photon' or a `phonon' we should interpret it in terms
of the excited state of an ensemble of oscillators. We should not visualise them
as `particles' of light or sound.


\input{../header_class}

%---------- start document ---------- %
 \section{vector -- vector object and arithmetic}\linkedzero{vector}
 \begin{itemize}
   \item {\bf Classes}
   \begin{itemize}
     \item \linkingone{vector}{Vector}
   \end{itemize}
   \item {\bf Functions}
     \begin{itemize}
       \item \linkingone{vector}{innerProduct}
     \end{itemize}
 \end{itemize}

This module provides an exception class.
\begin{description}
  \item[VectorSizeError]:\ Report vector size is invalid. (Mainly for operations with two vectors.)
\end{description}

\C

 \subsection{Vector -- vector class}\linkedone{vector}{Vector}
 Vector is a class for vector.
  \initialize
  \func{Vector}{\hiki{compo}{list}}{\out{Vector}}\\
  \spacing
  % document of basic document
  \quad Create Vector object from \param{compo}.
  % added document
  %
  % \spacing
  % input, output document
  \param{compo} must be a list of elements which are an integer or an instance of \linkingone{ring}{RingElement}.
  \begin{at}
    \item[compo]\linkedtwo{vector}{Vector}{compo}:\\ It expresses component of vector.
  \end{at}
  \begin{op}
    \verb|u+v| & Vector sum.\\
    \verb|u-v| & Vector subtraction.\\
    \verb|A*v| & Multiplication vector with matrix\\
    \verb|a*v| & or scalar multiplication.\\
    \verb|v//a| & Scalar division.\\
    \verb|v%n| & Reduction each elements of \linkingtwo{vector}{Vector}{compo}\\
    \verb|-v| & element negation.\\
    \verb|u==v| & equality.\\
    \verb|u!=v| & inequality.\\
    \verb+v[i]+ & Return the coefficient of i-th element of Vector.\\
    \verb+v[i] = c+ & Replace the coefficient of i-th element of Vector by c.\\
    \verb|len(v)| & return length of \linkingtwo{vector}{Vector}{compo}.\\
    \verb|repr(v)| & return representation string.\\
    \verb|str(v)| & return string of \linkingtwo{vector}{Vector}{compo}.\\
  \end{op}
  Note that index is 1-origin, which is standard in mathematics field.
\begin{ex}
>>> A = vector.Vector([1, 2])
>>> A
Vector([1, 2])
>>> A.compo
[1, 2]
>>> B = vector.Vector([2, 1])
>>> A + B
Vector([3, 3])
>>> A % 2
Vector([1, 0])
>>> A[1]
1
>>> len(B)
2
\end{ex}%Don't indent!
  \method
  \subsubsection{copy -- copy itself}\linkedtwo{vector}{Vector}{copy}
   \func{copy}{\param{self}}{\out{Vector}}\\
   \spacing
   % document of basic document
   \quad Return copy of \param{self}.\\
   \spacing
   % added document
   %\quad \negok Note that this function returns integer only.\\
   %\spacing
   % input, output document
   %\quad \param{a} must be int, long or rational.Integer.\\
  \subsubsection{set -- set other compo}\linkedtwo{vector}{Vector}{set}
   \func{set}{\param{self},\ \hiki{compo}{list}}{(None)}\\
   \spacing
   % document of basic document
   \quad Substitute \linkingtwo{vector}{Vector}{compo} with \param{compo}.\\
   \spacing
   % added document
   %\quad \negok Note that this function returns integer only.\\
   %\spacing
   % input, output document
   %\quad \param{a} must be int, long or rational.Integer.\\
  \subsubsection{indexOfNoneZero -- first non-zero coordinate}\linkedtwo{vector}{Vector}{indexOfNoneZero}
   \func{indexOfNoneZero}{\param{self}}{integer}\\
   \spacing
   % document of basic document
   \quad Return the first index of non-zero element of \param{self}.\linkingtwo{vector}{Vector}{compo}.\\
   \spacing
   % added document
   \quad \negok Raise ValueError if all elements of \linkingtwo{vector}{Vector}{compo} are zero.\\
   \spacing
   % input, output document
   %\quad \param{a} must be int, long or rational.Integer.\\
  \subsubsection{toMatrix -- convert to Matrix object}\linkedtwo{vector}{Vector}{toMatrix}
   \func{toMatrix}{\param{self},\ \hikiopt{as\_column}{bool}{False}}{\out{Matrix}}\\
   \spacing
   % document of basic document
   \quad Return \linkingone{matrix}{Matrix} object using \linkingone{matrix}{createMatrix} function.\\
   \spacing
   % added document
   %\quad \negok Note that this function returns integer only.\\
   %\spacing
   % input, output document
   \quad If \param{as\_column} is True, create the column matrix with \param{self}.
   Otherwise, create the row matrix.\\
\begin{ex}
>>> A = vector.Vector([0, 4, 5])
>>> A.indexOfNoneZero()
2
>>> print A.toMatrix()
0 4 5
>>> print A.toMatrix()
0
4
5
\end{ex}%Don't indent!
\C
  \subsection{innerProduct(function) -- inner product}\linkedone{vector}{innerProduct}
  \func{innerProduct}{\hiki{bra}{Vector}, \ \hiki{ket}{Vector}}{\out{RingElement}}\\
   \spacing
   % document of basic document
   \quad Return the inner product of \param{bra} and \param{ket}.\\
   \spacing
   % added document
   \quad The function supports Hermitian inner product for elements in the complex number field.\\
   \spacing
   % input, output document
   \quad \negok Note that the returned value depends on type of elements.\\
\begin{ex}
>>> A = vector.Vector([1, 2, 3])
>>> B = vector.Vector([2, 1, 0])
>>> vector.innerProduct(A, B)
4
>>> C = vector.Vector([1+1j, 2+2j, 3+3j])
>>> vector.innerProduct(C, C)
(28+0j)
\end{ex}%Don't indent!
\C

%---------- end document ---------- %

\input{../footer}


\section{Tricks}
\begin{enumerate}

\item
The Eigenmath result field can be copied to the pasteboard by
click, drag, then release.

\item
The last result is stored in the symbol {\tt last}.

\item
In a script, setting {\tt trace=1}
causes each line to be printed just before it is evaluated.
Useful for debugging.

\item
Use {\tt contract(A)} to get the mathematical trace of matrix $A$.

\item
Calculations in a script can span multiple lines.
The trick is to arrange things so the parser will keep going.
For example, if a calculation ends with a plus sign, the parser will go to the next line to get another term.
Also, the parser will keep going when it expects a close parenthesis.

\item
Normally a function body is not evaluated when a function is defined.
However, in some cases it is required that the function body be the result of something.
The trick is to use eval.
For example, the following code causes the function body to be a sixth order Taylor series expansion of $cos(x)$.

\begin{Verbatim}[formatcom=\color{blue}]
f(x) = eval(taylor(cos(x),x,6))
\end{Verbatim}

\item
Use {\tt binding} to see the unevaluated binding of a symbol.

\begin{Verbatim}[formatcom=\color{blue}]
binding(f)
\end{Verbatim}

\item
This is how to clear a symbol.

\begin{Verbatim}[formatcom=\color{blue}]
f = quote(f)
\end{Verbatim}

\end{enumerate}


\chapter{Preconditioning}
\label{chap:precond}

The linear system \eqref{eq:mhd_saddle} is typically sparse and of large dimensions, hence to efficiently solve for it we use a preconditioned iterative approach as proposed in \cite{li2010numerical}. We start by reviewing some preconditioning strategies for the incompressible Navier-Stokes and Maxwell subproblems in isolation. From these techniques we will then introduce and  numerically test  preconditioners for the full MHD system in linearised form as in \eqref{eq:mhd_saddle}.

Let us introduce a number of matrices in addition to the ones already defined in Chapter~2: the velocity mass matrix $Q$, the pressure mass matrix $Q_p$, the pressure convection diffusion matrix  $F_p$, the pressure Laplacian matrix $A_p$, the scalar Laplacian matrix $L$ and the mass matrix~$X$ by
\begin{subequations}
\label{eq:PrecondMatrices}
\begin{eqnarray}
Q_{i,j}&=&\int_\Omega\, \uu{\psi_j}\cdot \uu{\psi_i} \,d\uu{x}, \quad 1\leq i,j \leq n_u,\\
(Q_p)_{i,j}&=& \int_\Omega\, \alpha_j \alpha_i \,dx, \ \ 1\leq i,j \leq m_u, \\
(F_{p})_{i,j}&=& \nu \int_\Omega\, \grad \alpha_j \cdot \grad \alpha_i +(\uu{w} \cdot \grad \alpha_j)\alpha_i\,dx,\ \ 1\leq i,j \leq m_u, \\
(A_{p})_{i,j}&=&  \int_\Omega\, \grad \alpha_j \cdot \grad \alpha_i \,dx, \ \ 1\leq i,j \leq m_u, \\
L_{i,j}&=&\int_\Omega\,\nabla\beta_j\cdot\nabla\beta_i\,d\uu{x},  \ \ 1\leq i,j \leq m_b\\
X_{i,j}&=&\int_\Omega\, \uu{\phi}_j\cdot\uu{\phi}_i\,d\uu{x},  \ \ 1\leq i,j \leq n_b,
\end{eqnarray}
\end{subequations}
where $\uu{w}$ is the finite element velocity from the current iteration. The matrices $F_p$ and $A_p$ are well defined as we use continuous elements for the pressure finite element space $Q_h$. The matrices incorporate homogeneous Neumann boundary conditions.

\section{Preconditioning the incompressible Navier-Stokes equations}
\label{sec:NSprecond}

Consider the steady state incompressible Navier-Stokes equations in isolation. Let
\begin{equation}
\label{eq:ns_coeff}
\mathcal{K}_{\rm NS}=
\begin{pmatrix}
F & B^T \\
B & 0
\end{pmatrix},
\end{equation}
be the discretised and linearised Navier-Stokes (or Oseen) subproblem where $F~=~A~+~O$ has been defined in \eqref{eq:forms}. Due to the convection term, $O$, this  system is non-symmetric and we will use GMRES to iteratively solve this subproblem; \cite{saad1986gmres}. A common approach for solving a saddle point system, as in \eqref{eq:ns_coeff}, is to use a block diagonal or block triangular preconditioner of the form
\begin{equation}
\label{eq:ns_pc_upper}
\mathcal{M}_{\rm tNS} =
\begin{pmatrix}
F & B^T \\
0 & -S
\end{pmatrix} \quad \mbox{or} \quad
\mathcal{M}_{\rm dNS} =
\begin{pmatrix}
F & 0 \\
0 & S
\end{pmatrix},
\end{equation}
where $S\approx B F^{-1} B^T$ is an approximation to the Schur complement. If $S = B F^{-1} B^T$ is {\em precisely} the Schur complement, then it has been proved in~\cite{murphy2000note} that the preconditioned matrix has exactly $2$ eigenvalues when using the block triangular preconditioner  (i.e., $\mathcal{M}_{\rm tNS}^{-1}\mathcal{K}_{\rm NS}$) or $3$ distinct eigenvalues in the block diagonal case  (i.e., $\mathcal{M}_{\rm dNS}^{-1}\mathcal{K}_{\rm NS}$). Both are diagonalisable and hence GMRES will converge in exactly 2 or 3 iterations in the absence of round-off errors.

In practice it is often too expensive to form and solve for the exact Schur complement $B F^{-1} B^T$, hence, a good approximation is needed. Two well-known preconditioners for the incompressible Navier-Stokes equations are the Least Squares Commutator (LSC) and the Pressure Convection-Diffusion (PCD) preconditioners. A description of both can be found in \cite{elman2005finite} and we will just outline the procedure how these can be applied on the discrete level. For the Navier-Stokes preconditioner we will consider block triangular preconditioners of the same form as $\mathcal{M}_{\rm tNS}$.

% Both methods (LSC and PCD)  start with the convection-diffusion operator associated with the velocity space $\uu{V}_h$ given by
% $$\mathcal{L} = -\nu \Delta +\uu{w} \cdot \nabla\, .$$
% As before $\uu{w}$ is the discrete velocity calculated at the previous non-linear iteration. Suppose that there is a corresponding operator defined in the pressure space
% $$\mathcal{L}_p = (-\nu \Delta +\uu{w} \cdot \nabla)_p\, .$$
% Consider the commutator of the convection-diffusion operator associated with the gradient operator
% \begin{equation} \label{eq:ContCommutator}
% \epsilon = (-\nu \Delta +\uu{w} \cdot \nabla)\nabla - \nabla (-\nu \Delta +\uu{w} \cdot \nabla)_p
% \end{equation}
% to be small. In fact, if $\uu{w}$ was constant then $\epsilon = 0$. We will be using \eqref{eq:ContCommutator} to derive both LSC and PCD.

\subsection{Pressure Convection-Diffusion (PCD)}
\label{sec:PCD_outline}

In \cite[Chap. 8]{elman2005finite} the discrete commutator of the convection-diffusion operator associated with the gradient operation is introduced and given by
\begin{equation} \label{eq:DisCommutator}
    \epsilon_h = (Q^{-1}F)(Q^{-1}B^T)-(Q^{-1}B^T)(Q_p^{-1}F_p).
\end{equation}
where the matrices $Q$, $Q_p$ and $F_p$ are defined in  \eqref{eq:PrecondMatrices}. Assuming that the commutator is small then pre- and post-multiplying \eqref{eq:DisCommutator} by $B F^{-1} Q$ and $F_p^{-1}Q_p$, respectively, let us separate out the Schur complement to give
\begin{equation} \label{eq:SchurApprox}
    BF^{-1}B^T \approx B Q^{-1}B^T F_p^{-1} Q_p.
\end{equation}

Our discretisation is inf-sup stable which implies that there is spectral equivalence between $BQ^{-1}B^T$ and the pressure Laplacian matrix, $A_p$; see \cite[Section 5.5.1]{elman2005finite}. Hence, the Schur complement can be approximated by:
$$S_{\rm PCD} =A_p F_p^{-1}Q_p.$$
Applying the PCD preconditioner (i.e., taking $S = S_{\rm PCD}$) to the linearised Navier-Stokes system involves solving the system
\begin{equation} \nonumber
% \label{eq:matrix-system}
\left(
\begin{array}{cc}
F & B^T \\
0 & -A_p F_p^{-1}Q_p
\end{array}
\right)
\,
\left(
\begin{array}{c}
x \\
y
\end{array}
\right) =
\left(
\begin{array}{c}a\\b
\end{array}
\right)
\end{equation}
at each Krylov iteration. This can be solved  efficiently by splitting it into the following two steps
\begin{itemize} \label{it:PCDsolve}
    \item[1.] Solve for $y$: $y = -Q_p^{-1}F_p A_p^{-1}b;$
    \item[2.] Solve for $x$: $x = F^{-1}(a-B^Ty).$
\end{itemize}
This means that we have one pressure Poisson solve ($A_p^{-1}$), one mass matrix solve ($Q_p^{-1}$), one convection-diffusion solve ($F^{-1}$) and multiplications with $F_p$ and $B^T$ at each Krylov iteration. It is possible to solve these iteratively using multigrid and/or iterative methods. However, to test the preconditioner we will use direct solver in this thesis.

\subsection{Least-Squares Commutator (LSC)}
\label{sec:LSC_outline}

One disadvantage with using the PCD preconditioner is that it requires the construction of the matrices $A_p$, $Q_p$ and $F_p$ in \eqref{eq:PrecondMatrices}. A second approach to approximate the Schur complement is the LSC preconditioner \cite[Chap. 8]{elman2005finite} which primarily uses the available matrix coefficients in \eqref{eq:ns_coeff} and the construction of $Q$ to form the preconditioner (without explicitly forming $A_p$, $Q_p$ and $F_p$).

As for the derivation of the PCD preconditioner we start off with the discrete commutator of the convection-diffusion operator
\begin{equation} \nonumber
    \epsilon_h = (Q^{-1}F)(Q^{-1}B^T)-(Q^{-1}B^T)(Q_p^{-1}F_p).
\end{equation}
Suppose that the $Q$-norm is defined by $\|v\|_{Q} = (Qv,v)^{\nicefrac{1}{2}}$. Then this time we minimise $\epsilon_h$ over the $j$th column of $F_p$ (that is $[F_p]_j$) in the $Q$-norm to try to find an approximation for $F_p$. As shown in \cite{elman2005finite}, the minimisation is given by
$$\min \|[Q^{-1}FQ^{-1}B^T]_j-Q^{-1}B^TQ_p^{-1}[F_p]_j) \|_Q.$$
Solving this optimisation problem, as shown in \cite{elman2005finite}, is done by solving the  normal equations
$$Q_p^{-1}BQ^{-1}B^TQ_p^{-1}F_p = Q_p^{-1}BQ^{-1} FQ^{-1}B^T.$$
This yields the an approximation to $F_p$ as
$$F_p \approx Q_p(BQ^{-1}B^T)^{-1}(BQ^{-1} FQ^{-1}B^T).$$
By substituting this into expression \eqref{eq:SchurApprox} we obtain the LSC approximation to the Schur complement:
\begin{equation} \nonumber
    S \approx BF^{-1}B^T \approx S_{\rm LSC} = (B Q^{-1} B^T)(BQ^{-1}FQ^{-1}B^T)^{-1}(B Q^{-1} B^T).
\end{equation}
Therefore, applying the LSC preconditioner to the Oseen system $\mathcal{K}_{\rm NS}$ in \eqref{eq:ns_coeff} involves solving for the matrix
\begin{equation} \nonumber
\left(
\begin{array}{cc}
F & B^T \\
0 & -S_{\rm LSC}
\end{array}
\right)
\,
\left(
\begin{array}{c}
x \\
y
\end{array}
\right) =
\left(
\begin{array}{c}a\\b
\end{array}
\right)
\end{equation}
at each Krylov iteration. Again, this can be efficiently split up into the following two steps:
\begin{itemize}
    \item[1.] Solve for $y$: $y = -(B Q^{-1} B^T)^{-1}(BQ^{-1}FQ^{-1}B^T)(B Q^{-1} B^T)^{-1}b;$
    \item[2.] Solve for $x$: $x = F^{-1}(a-B^Ty).$
\end{itemize}
Hence, we have two pressure Poisson solves ($(B Q^{-1} B^T)^{-1}$) and one convection-diffusion solve ($F^{-1}$) at each Krylov iteration as well as matrix multiplications. In practice, we take the diagonal or lumped diagonal of $Q$ to form $B Q^{-1} B^T$. These solves, as  with the PCD preconditioner, will be done directly in this thesis.

\subsection{PCD versus LSC}

The main advantage of solving the commutator using the least-squares approach is that the matrices that define the preconditioner are available from the original system $\mathcal{K}_{\rm NS}$ in \eqref{eq:ns_coeff} and the construction of $Q$ . However, for PCD we require the construction of the matrices $A_p$, $Q_p$ and $F_p$. Therefore, LSC is slightly more computationally efficient to form. On the other hand, to apply the LSC preconditioner we require two pressure Poisson solves, whereas PCD only requires one.

We will consider experiments with both preconditioners for the incompressible Navier-Stokes problem in isolation to determine which seems more effective. This preconditioner  will then be applied within the solver for the linearised MHD system.

\section{Preconditioning Maxwell's equations}
\label{sec:MaxwellPrecond}

Next, consider the Maxwell subproblem
\begin{equation}
\label{eq:m_coeff}
\mathcal{K}_{\rm MX}=
\begin{pmatrix}
M & D^T \\
D & 0
\end{pmatrix},
\end{equation}
appearing in a linearised MHD system \eqref{eq:mhd_saddle}.
% , but without coupling terms.
As we did for the Navier-Stokes subproblem in Section \ref{sec:NSprecond}, we apply a block preconditioning strategy for $\mathcal{K}_{\rm MX}$ in  \eqref{eq:m_coeff}. Notice that, $\mathcal{K}_{\rm MX}$ in \eqref{eq:m_coeff} is symmetric and hence we will focus on SPD block diagonal preconditioners.

\subsection{An ideal preconditioner}

The $(1,1)$ block of $\mathcal{K}_{\rm MX}$ is the curl-curl operator, hence, the matrix $M$ is singular, where the null space is of dimension $m_b$ and consists of the discrete gradients. Therefore the usual Schur complement does not exist as the matrix $M$ cannot be inverted. To overcome this difficulty, we employ the approach in  \cite{golub2003solving,greif2006preconditioners} based on  augmentation and use preconditioners of the form:
\begin{equation} \nonumber
% \label{eq:maxwell_pc_ideal}
\begin{pmatrix}
M+D^T W^{-1} D & 0 \\
0 & W
\end{pmatrix},
\end{equation}
where $W$ is a symmetric positive definite matrix.

% . More precisely, we consider replacing $M$ by $M+D^T\mathcal{W}^{-1}D$ where $\mathcal{W}\in {\mathbb R}^{m_b\times m_b}$ is a symmetric positive definite matrix just to form the preconditioner, see \cite{golub2003solving,greif2006preconditioners} for more details. The addition of the matrix ($D^T\mathcal{W}^{-1}D$) removes the singularity of the $(1,1)$ block of $\mathcal{K}_{\rm MX}$ without changing the solution (since $Db = 0$).  In practice, we don't actually replace $M$ by $M+D^T\mathcal{W}^{-1}D$ but just consider preconditioners with $M+D^T\mathcal{W}^{-1}D$ in the $(1,1)$ block which then are applied to $\mathcal{K}_{\rm MX}$. For the Maxwell subproblem the appropriate choice of $\mathcal{W}$ is the scalar Laplacian on $S_h$ defined as $L=(L_{i,j})_{i,j=1}^{m_b} \in{\mathbb R}^{m_b \times m_b}$ with
% \begin{equation}
% \label{eq:scalar_laplace}
% L_{i,j}=\int_\Omega\,\nabla\beta_j\cdot\nabla\beta_i\,d\uu{x},
% \end{equation}
% see  \cite{greif2007preconditioners}. Therefore, the augmented system is given by:
% \begin{equation}
% \label{eq:AugmentMaxwell}
% \bar{\mathcal{K}}_{\rm MX}=
% \begin{pmatrix}
% M + D^TL^{-1}D & D^T \\
% D & 0
% \end{pmatrix}.
% \end{equation}

It has been shown in \cite{greif2007preconditioners} that an appropriate choice of $W$ is the scalar Laplacian, $L$, defined in \eqref{eq:PrecondMatrices}. This leads to the ideal preconditioner:
\begin{equation}
\label{eq:maxwell_pc_ideal}
\mathcal{M}_{\rm iMX} =
\begin{pmatrix}
M+D^T L^{-1} D & 0 \\
0 & L
\end{pmatrix}.
\end{equation}
Applying \eqref{eq:maxwell_pc_ideal} as the preconditioner yields exactly two eigenvalues, $1$ and $-1$ of algebraic multiplicities of $n_b$ and $m_b$, respectively. Therefore using this matrix as a preconditioner means that MINRES will converge in two iterations, in the absence of roundoff errors  \cite{paige1975solution}. However, forming the matrix $M+D^T L^{-1} D$ is costly, hence, $\mathcal{M}_{\rm iMX}$  is  impractical  for large systems.

\subsection{A practical preconditioner}

A good approximation for $M+D^T L^{-1} D$ is required to make the ideal preconditioner, $\mathcal{M}_{\rm iMX}$, suitable in practice. It has been shown in \cite{greif2007preconditioners}  that $M+D^T L^{-1} D$ is spectrally equivalent to $M+X$  where $X$ is the vector mass matrix on the magnetic space defined in \eqref{eq:PrecondMatrices}. Using this approximation yields the practical preconditioner
\begin{equation}
\label{eq:maxwell_pc_X}
\mathcal{M}_{\rm MX} =
\begin{pmatrix}
N& 0 \\
0 & L
\end{pmatrix},
\end{equation}
where $N = M+X$ is a shifted curl-curl operator. A scalable multigrid solver for $N$ has been developed in \cite{hiptmair2007nodal} which involves one shifted Laplacian solves on the vector space and one scalar Laplacian solve. However, the construction of this multigrid solver is involved and hence we will consider direct solves for this preconditioner and leave the multigrid implementation as a possible area of future work.

\section{Preconditioning the MHD equations}
\label{sec:MHDp}
In Section \ref{sec:nonlinear} and \ref{sec:FEMdecouple} we introduced three iteration schemes, namely Picard iteration (P), Magnetic Decoupling (MD) and Complete Decoupling (CD). Using the results from Sections \ref{sec:MaxwellPrecond} and \ref{sec:NSprecond} we will discuss the preconditioning approaches that we apply for these non-linear iteration schemes.

\subsection{Picard iteration}
\label{sec:MHDprecond}

% In Sections \ref{sec:NSprecond} and \ref{sec:MaxwellPrecond} we looked briefly at the preconditioning strategies for the Navier-Stokes and Maxwell's equations.

Using the techniques of Sections \ref{sec:NSprecond} and \ref{sec:MaxwellPrecond} for the incompressible Navier-Stokes and Maxwell problems, respectively, we will look at possible scalable preconditioners for the linearised MHD problem
\begin{equation} \label{eq:K}
    {\mathcal K}_{\rm MH} = \left(
\begin{array}{cccc}
A+O & B^T & C^T & 0\\
B & 0 & 0 & 0\\
-C & 0 & M & D^T \\
0 & 0 & D & 0
\end{array}
\right).
\end{equation}
% Using the Navier-Stokes and Maxwell subproblem preconditioners \eqref{eq:ns_pc_upper} and \eqref{eq:maxwell_pc_X} respectively, then w
We propose the following preconditioner for ${\mathcal K}_{\rm MH}$
\begin{equation}
\label{eq:mhd_pc_ls}
\mathcal{M}_{\rm MH} =
\left(
\begin{array}{cccc}
F & B^T & C^T & 0\\
0 & -{S} & 0 & 0 \\
-C & 0 & N & 0\\
0 & 0 & 0 & L
\end{array}
\right),
\end{equation}
where ${S}$ is now either the LSC or PCD approximation to the fluid flow Schur complement. The preconditioned matrix, $\mathcal{M}_{\rm MH}^{-1}{\mathcal K}_{\rm MH}$, has eigenvalues $\lambda = 1$ with algebraic multiplicity of at least $n_u + n_b$ and the eigenvalue $\lambda = -1$ with algebraic multiplicity of at least $m_b$ see \cite[Theorem~8]{li2010numerical}. Due to the coupling terms, $C$, the application of this preconditioner is computationally expensive. To overcome this, we propose to invert  $\mathcal{M}_{\rm MH}$ by means of an inner preconditioner. The inner preconditioner is taken as
\begin{equation}
\label{eq:mhd_pc_inner}
\mathcal{M}_{\rm innerMH} =
\left(
\begin{array}{cccc}
F & B^T & 0 & 0\\
0 & -{S} & 0 & 0 \\
0 & 0 & N & 0\\
0 & 0 & 0 & L
\end{array}
\right).
\end{equation}
Here the preconditioned matrix, $\mathcal{M}_{\rm innerMH}^{-1}{\mathcal M}_{\rm MH}$,  has an eigenvalue $\lambda = −1$ with algebraic multiplicity of at least $m_u +n_u+3m_b -n_b$, see \cite[Theorem~10]{li2010numerical}.

\subsection{Magnetic Decoupling}
\label{sec:MDprecond}

From Section \ref{sec:FEMmd} the  matrix to be preconditioned for (MD) is
\begin{equation}
\label{eq:Kmd}
   \mathcal{K}_{\rm MD} =
    \left(
    \begin{array}{cc|cc}
    F& B^T & 0 & 0\\
    B & 0 & 0 & 0 \\
    \hline
    0 & 0 & M & D^T\\
    0 & 0 & D & 0
    \end{array}
    \right).
\end{equation}
Recall that removing the coupling terms completely decouples the system. This therefore enables us to use the preconditioners for each of the subproblems separately and in parallel. Using the subproblem preconditioners \eqref{eq:ns_pc_upper} and \eqref{eq:maxwell_pc_X} then we propose the following preconditioner for $\mathcal{K}_{\rm MD}$:
\begin{equation}
\label{eq:mhd_pc_explicit}
\mathcal{M}_{\rm MD} =
\left(
\begin{array}{cc|cc}
F & B^T & 0 & 0\\
0 & -{S} & 0 & 0 \\
\hline
0 & 0 & N & 0\\
0 & 0 & 0 & L
\end{array}
\right),
\end{equation}
where $S$ is the LSC or PCD approximation and $N$ is the shifted curl-curl matrix.

\subsection{Complete Decoupling}
\label{sec:CDprecond}

From Section \ref{sec:FEMcd} the matrix to be preconditioned for (CD) is
\begin{equation}
\label{eq:Kcd}
%\mathcal{K} x \equiv
 \mathcal{K}_{\rm CD} = \left(
\begin{array}{cc|cc}
A & B^T & 0 & 0\\
B & 0 & 0 & 0 \\
\hline
0 & 0 & M & D^T\\
0 & 0 & D & 0
\end{array}
\right).
\end{equation}
Note that the matrix $\mathcal{K}_{\rm CD}$ is now symmetric. First we consider how to deal with the upper $(2,2)$ block matrix which corresponds to the discrete Stokes equations
\begin{equation}\nonumber
   \mathcal{K}_{\rm S} =
    \left(
    \begin{array}{cc}
    A& B^T \\
    B & 0
    \end{array}
    \right).
\end{equation}
As with the incompressible Navier-Stokes subproblem the idea for the Stokes preconditioner is again to approximate the Schur complement
$$S_{\rm S} =  BA^{-1}B^T.$$
Recall that the matrix $A$ is defined with the viscosity $\nu$ in Section \ref{sec:variation}. It was shown in \cite{silvester1993fast,silvester1994fast} that the scaled pressure mass matrix, $\mbox{\small \(\frac{1}{\nu}\)} W$ defined in \eqref{eq:PrecondMatrices}, is spectrally equivalent to the Schur complement $S_S$ (which is also a consequence of the inf-sup stability from our mixed discretisation). Therefore a possible scalable Stokes preconditioner is
\begin{equation}
\label{eq:mhd_pc_explicit2_1}
\begin{pmatrix}
A & 0 \\
0 & \mbox{\small \(\frac{1}{\nu}\)} W
\end{pmatrix}.
\end{equation}
Using \eqref{eq:mhd_pc_explicit2_1} together with the Maxwell subproblem preconditioner  \eqref{eq:maxwell_pc_X} gives the preconditioner
\begin{equation}
\label{eq:mhd_pc_explicit2}
\mathcal{M}_{\rm CD} =
\left(
\begin{array}{cc|cc}
A & 0 & 0 & 0\\
0 & \mbox{\small \(\frac{1}{\nu}\)} W & 0 & 0 \\
\hline
0 & 0 & N & 0\\
0 & 0 & 0 & L
\end{array}
\right).
\end{equation}

As the matrix system $\mathcal{K}_{\rm CD}$ is symmetric, then the appropriate choice for the Krylov subspace method is MINRES for each subproblem. The main advantage of using MINRES over GMRES is that we do not need to store a new vector at each iteration. Therefore, in terms of computational memory MINRES is more efficient.

\subsection{Summary}

In summary, we outlined the three preconditioning approaches for the linearised systems arising in the non-linear iteration schemes proposed in Chapter \ref{sec:discretization}. Table \ref{tab:SummaryTable} references the coefficient matrices together the associated preconditioner.
\begin{table}[h!]
\begin{center}
\begin{tabular}{|c|c|c|}
\hline
  Iteration & Coefficient & (Outer) \\
  scheme & matrix & preconditioner\\
\hline
\rule{0pt}{12pt}(P) & $\mathcal{K}_{\rm MH}$ in \eqref{eq:K}& $\mathcal{M}_{\rm MH}$ in \eqref{eq:mhd_pc_ls}  \\[0.1cm]
\hline
\rule{0pt}{12pt}(MD) & $\mathcal{K}_{\rm MD}$ in \eqref{eq:Kmd}& $\mathcal{M}_{\rm MD}$ in \eqref{eq:mhd_pc_explicit}  \\[0.1cm]
\hline
\rule{0pt}{12pt}(CD) & $\mathcal{K}_{\rm CD}$ in \eqref{eq:Kcd}&$\mathcal{M}_{\rm CD}$ in \eqref{eq:mhd_pc_explicit2}\\[0.1cm]
\hline
\end{tabular}
\caption{Summary of coefficient matrices and corresponding preconditioners for each iteration scheme}
\label{tab:SummaryTable}
\end{center}
\end{table}

\noindent Note that for the Picard iteration (P), we employ the inner preconditioner $\mathcal{M}_{\rm innerMH}$ in \eqref{eq:mhd_pc_inner} to solve systems corresponding to \eqref{eq:mhd_pc_ls}.

% \begin{table}[h!] \small
% \begin{center}
% \begin{tabular}{|c|c|c|}
% \hline
%   Iteration & Coefficient & (Outer) \\
%   scheme & matrix & preconditioner\\
% \hline
% (P) & $\left(
% \begin{array}{cccc}
% A+O & B^T & C^T & 0\\
% B & 0 & 0 & 0\\
% -C & 0 & M & D^T \\
% 0 & 0 & D & 0
% \end{array}
% \right)$&$\left(
% \begin{array}{cccc}
% F & B^T & C^T & 0\\
% 0 & -{S} & 0 & 0 \\
% -C & 0 & N & 0\\
% 0 & 0 & 0 & L
% \end{array}
% \right)$  \\
% \hline
% (MD) & $\left(
%     \begin{array}{cccc}
%     F& B^T & 0 & 0\\
%     B & 0 & 0 & 0 \\
%     0 & 0 & M & D^T\\
%     0 & 0 & D & 0
%     \end{array}
%     \right)$&$\left(
% \begin{array}{cccc}
% F & B^T & 0 & 0\\
% 0 & -{S} & 0 & 0 \\
% 0 & 0 & N & 0\\
% 0 & 0 & 0 & L
% \end{array}
% \right)$  \\
% \hline
% (CD) & $\left(
% \begin{array}{cccc}
% A & B^T & 0 & 0\\
% B & 0 & 0 & 0 \\
% 0 & 0 & M & D^T\\
% 0 & 0 & D & 0
% \end{array}
% \right)$& $\left(
% \begin{array}{cccc}
% A & 0 & 0 & 0\\
% 0 & \mbox{\small \(\frac{1}{\nu}\)} W & 0 & 0 \\
% 0 & 0 & N & 0\\
% 0 & 0 & 0 & L
% \end{array}
% \right)$\\
% \hline
% \end{tabular}
% \caption{Summary of coefficient and corresponding preconditioners for each iteration scheme}
% \label{tab:SummaryTable}
% \end{center}
% \end{table}

% \textit{Generate a new subSection that lays out a summary of the systems and the preconditioners. This generates a redundancy, but it will be very helpful for the reader, who at this point may be lost trying to track all preconditioners and systems. This section should also include a summary of multiplicities (briefly, no need to state theorems), as per Dan's thesis and/or the paper that is currently being written.}

% \paragraph{Picard iteration (P)} ~\\

% The coefficient matrix corresponding to the Picard iteration is
% \begin{equation}\nonumber
% {\mathcal K}_{\rm MH} = \left(
% \begin{array}{cccc}
% A+O & B^T & C^T & 0\\
% B & 0 & 0 & 0\\
% -C & 0 & M & D^T \\
% 0 & 0 & D & 0
% \end{array}
% \right).
% \end{equation}
% We consider the  preconditioner
% \begin{equation} \nonumber
% \mathcal{M}_{\rm MH} =
% \left(
% \begin{array}{cccc}
% F & B^T & C^T & 0\\
% 0 & -{S} & 0 & 0 \\
% -C & 0 & N & 0\\
% 0 & 0 & 0 & L
% \end{array}
% \right),
% \end{equation}
% The preconditionered matrix, $\mathcal{M}_{\rm MH}^{-1}{\mathcal K}_{\rm MH}$, has eigenvalues $\lambda = 1$ with algebraic multiplicity of at least $n_u + n_b$ and and an eigenvalue $\lambda = −1$ with algebraic multiplicity of at least $m_b$ see \cite{li2010numerical}. To apply this preconditioner, we solve linear systems associated with $\mathcal{M}_{\rm MH}$. The proposed way of doing this is to use GMRES with the inner preconditioner:
% \begin{equation} \nonumber
% \mathcal{M}_{\rm innerMH} =
% \left(
% \begin{array}{cccc}
% F & B^T & 0 & 0\\
% 0 & -S & 0 & 0 \\
% 0 & 0 & N & 0\\
% 0 & 0 & 0 & L
% \end{array}
% \right).
% \end{equation}
% Here the preconditioner matrix, $\mathcal{M}_{\rm innerMH}^{-1}{\mathcal M}_{\rm MH}$,  has an eigenvalue $\lambda = −1$ with algebraic multiplicity of at least $m_u +n_u+3m_b -n_b$ see \cite{li2010numerical}.

% \paragraph{Magnetic Decoupling (MD)} ~\\
% The coefficient matrix corresponding to the (MD) iteration is
% \begin{equation}\nonumber
% {\mathcal K}_{\rm MH} = \left(
% \begin{array}{cc|cc}
% F& B^T & 0 & 0\\
% B & 0 & 0 & 0\\
% \hline
% 0& 0 & M & D^T \\
% 0 & 0 & D & 0
% \end{array}
% \right).
% \end{equation}
% This complete decouples the system to enable use to use the individual Navier-Stokes and Maxwell's equation preconditioners seperately. The proposed preconditioner is:
% \begin{equation}
% \nonumber
% \mathcal{M}_{\rm MD} =
% \left(
% \begin{array}{cc|cc}
% F & B^T & 0 & 0\\
% 0 & -{S} & 0 & 0 \\
% \hline
% 0 & 0 & N & 0\\
% 0 & 0 & 0 & L
% \end{array}
% \right).
% \end{equation}

% \paragraph{Complete Decoupling (CD)} ~\\

% The coefficient matrix corresponding to the (CD) iteration is
% \begin{equation} \nonumber
%  {\mathcal K}_{\rm MH} = \left(
% \begin{array}{cc|cc}
% A & B^T & 0 & 0\\
% B & 0 & 0 & 0\\
% \hline
% 0& 0 & M & D^T \\
% 0 & 0 & D & 0
% \end{array}
% \right).
% \end{equation}
% This complete decouples the system to enable use to use the individual Stokes and Maxwell's equation preconditioners seperately. The proposed preconditioner is:
% \begin{equation}
% \nonumber
% \mathcal{M}_{\rm CD} =
% \left(
% \begin{array}{cc|cc}
% A & 0 & 0 & 0\\
% 0 & \mbox{\small \(\frac{1}{\nu}\)} W & 0 & 0 \\
% \hline
% 0 & 0 & N & 0\\
% 0 & 0 & 0 & L
% \end{array}
% \right).
% \end{equation}


\chapter{Evolution of the simulator}
	\section{Solutions for 2 cars}\label{sec:base2car}
		Equation~\eqref{eq:numerical_idm} can be solved with the Explicit Euler method. To test the model a basic setup was implemented in \textsc{Matlab} \cite{matlab}. The visual representation of the setup can be seen in Figure~\ref{fig:basic2car}.
		\begin{figure}
			\centering
			\includegraphics[width=.95\textwidth]{common/basic_2_car}
			\caption{Basic setup with 2 cars}
			\label{fig:basic2car}
		\end{figure}
		The leading car has a constant 100 km/h velocity. The following car, which is modeled with IDM, has an initial velocity of 100 km/h as well. The second car is 100 meters behind the other car. The parameters of the following car can be seen in Table~\ref{tab:idm_params}. The parameters have been chosen based on literature \cite{opensource}.
		\begin{table}
			\begin{center}
				\begin{tabular}{ |c|c|c| }
					\hline
					$a_{\rm max}$ & $1.5$ & $\rm m/s^2$ \\
					$b_{\rm max}$ & $1.67$ & $\rm m/s^2$ \\
					$v_{\rm d}$ & $130$ & km/h \\
					$T$ & $1.8$ & s \\
					$h_0$ & $2$ & m \\
					$\rm \delta$ & $4$ & - \\
					$L$ & $4.5$ & m \\
					\hline
				\end{tabular}
			\end{center}
			\caption{Intelligent Driver Model parameters}
			\label{tab:idm_params}
		\end{table}
		The simulation was run until 100 seconds with a time step of 0.01s. Figure \ref{fig:basic2car_case_1} shows the result. The initial gap between the cars is greater than the second car's desired headway, consequently the vehicle will accelerate. The gap between the vehicles starts to decrease. At a certain time (around $t =$ 10s) the second car starts to decelerate slowly based on the velocity difference and the decreasing headway. It will reach the desired safe gap at some point and will have exactly the same speed as the car before. It will maintain its desired headway.
		\begin{figure}
			\centering
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{ee/basic_2_car_headaway_case_1_2}
			\end{minipage}\hfill
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{ee/basic_2_car_velocity_case_1_2}
			\end{minipage}
			\caption{Following car's headway and velocity in Setup 1}
			\label{fig:basic2car_case_1}
		\end{figure}

		Another simulation was run with the same configuration except that the first car's front is set to be at 40 meters instead of 100. Figure \ref{fig:basic2car_case_2} shows the result. In this case the gap between cars is less than the second vehicle's desired safety headway. So it will decelerate first than accelerate to reach the desired gap. Fundamentally the same happened but in the opposite direction.
		\begin{figure}
			\centering
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{ee/basic_2_car_headaway_case_2_2}
			\end{minipage}\hfill
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{ee/basic_2_car_velocity_case_2_2}
			\end{minipage}
			\caption{Following car's headway and velocity in Setup 2}
			\label{fig:basic2car_case_2}
		\end{figure}
		Both examples showed that the stationary state does not depend on the initial gap. After a little bit of time the second car has reached the desired safety gap (which is around 64.5 meters) and the 100 km/h speed in both cases.
	\section{Model behavior verification}
		In Section \ref{sec:base2car} two examples were shown where the model produced the same stationary state for different initial conditions. In this case stationary state means that the vehicle's acceleration is zero. So Eq.~\eqref{eq:aidm} modifies in a stationary state when $\dot{x}(t)\equiv \dot{x}_{\rm lead}(t)\equiv v_{\rm stac}$:
		\begin{equation}
		1 - \left ( \frac{v_{\rm stac}}{v_{\rm d}} \right )^{\rm \delta} - \left ( \frac{h_0+v_{\rm stac}\cdot T}{h_{\rm stac}} \right )^2=0\,.
		\label{eq:aidm_stac1}
		\end{equation}
		Of course all this means that the initial headway should be chosen properly, namely $h(t=0)=h_{\rm stac}$.
		Equation~\eqref{eq:aidm_stac1} shows that $v_{\rm stac}$ only depends on $h_{\rm stac}$. Everything else in the equations is a parameter of the model or initial condition. Consequently every stationary gap has a corresponding stationary speed value. Using \textsc{Matlab}'s \textit{fsolve} method Eq.~\eqref{eq:aidm_stac1} can be solved. Plot of the result can be seen in Figure \ref{fig:aidm_stac}.
		\begin{figure}
			\centering
			\includegraphics{check/check_stationary_states}
			\caption{Stationary velocity based on stationary headway}
			\label{fig:aidm_stac}
		\end{figure}

		In the examples of Section \ref{sec:base2car} the stationary gap between the vehicles was around 64.5 meters and the velocity was 100 km/h. Figure \ref{fig:aidm_stac} shows exactly the same values. It can be stated that the model is working as expected. Note that the bumper to bumper or traffic jam distance ($h_0$) appears on the horizontal axis and it has a zero velocity value.
	\section{Solver verification}
		A custom EE solver was implemented to solve Eq.~\eqref{eq:numerical_idm} numerically. However \textsc{Matlab} provides built-in solvers for differential equation systems like IDM. It has a numerical solver called ode45. It is a 4th order numerical solver which is capable of changing some of its parameters (e.g.: time step) based on the system's behavior. So theoretically it is a better choice to solve differential equation systems over a custom implemented Explicit Euler. Calculating the solution with this solver also provides a verification to the custom implemented EE solver.
		\begin{figure}
			\centering
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{ode/basic_2_car_headaway_case_3_2}
			\end{minipage}\hfill
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{ode/basic_2_car_velocity_case_3_2}
			\end{minipage}
			\caption{Following car's headway and velocity in Setup 1 with ode45 and EE}
			\label{fig:basic2car_case_1_ode}
		\end{figure}
		\begin{figure}
			\centering
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{ode/basic_2_car_headaway_case_4_2}
			\end{minipage}\hfill
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{ode/basic_2_car_velocity_case_4_2}
			\end{minipage}
			\caption{Following car's headway and velocity in Setup 2 with ode45 and EE}
			\label{fig:basic2car_case_2_ode}
		\end{figure}

		The same two examples were implemented with ode45 as before in Section \ref{sec:base2car}. The time step is not given this time since the solver can adjust it to the system's current behavior. But it is recommended the provide a relative error tolerance value to ensure a fine solution. For this case it was set to $10^{-4}$s.
		It can be seen in Figure \ref{fig:basic2car_case_1_ode} and \ref{fig:basic2car_case_2_ode} that the solutions are overlapping exactly, consequently the custom made solver is working as expected.
	\section{Solver for $n$ cars}
		The simulator works as expected for 2 cars simulations. It is time to implement an $n$-car simulation as well. The simulator still uses the built-in ode45 function to solve the differential equation system so the only task is to produce $\dot{\textbf{y}}$. The visual representation of the setup can be seen in Figure \ref{fig:basic_n_car}. The dimension of the system grows from 4 to 2n. The first car will have a constant velocity of 100 km/h so $\dot{y}_1 = 100$ and $\dot{y}_2 = 0$, since there is no acceleration.
		\begin{figure}
			\centering
			\includegraphics[width=\textwidth]{common/basic_n_car}
			\caption{n-car setup}
			\label{fig:basic_n_car}
		\end{figure}
		Unfortunately the previously used implementation is not good enough to produce $\dot{\textbf{y}}$. The problem is that to be able to calculate vehicle n's position and velocity the solver needs to have the position and velocity of vehicle n-1. To make it easy to read let us use function $f$ which represents Eq.~\eqref{eq:numerical_idm}. Than $\dot{\textbf{y}}$ looks like the following:
		\begin{equation}
			\dot{\textbf{y}}=
			\begin{pmatrix}
				100\\
				0\\
				f_1(v_{2})\\
				f_2(x_{2}, v_{2},x_{1}, v_{1})\\
				f_1(v_{3})\\
				f_2(x_{3}, v_{3},x_{2}, v_{2})\\
				\vdots\\
				f_1(v_{\rm n-1})\\
				f_2(x_{n-1}, v_{n-1},x_{n-2}, v_{n-2})\\
				f_1(v_{\rm n})\\
				f_2(x_{n}, v_{n},x_{n-1}, v_{n-1})
			\end{pmatrix}\,.
			\label{eq:n_ode_math}
		\end{equation}
		\section{Solutions for n cars}
		A couple of setups were made with n=5 to test the new n-car solver. The vehicles' IDM parameters were unchanged, it can be found in Table~\ref{tab:idm_params}.
		\subsection*{Case 1}
		The initial positions and velocities of the cars can be seen on Table~\ref{tab:node_case1}.
		\begin{table}
			\centering
			\begin{tabular}{ |c|c|c| }
				\hline
				n [-] & $x$ [m] & $v$ [km/h]\\
				\hline
				1 & 400 & 100 \\
				2 & 300 & 100 \\
				3 & 200 & 100 \\
				4 & 100 & 100 \\
				5 & 0 & 100 \\
				\hline
			\end{tabular}
			\caption{Case 1}
			\label{tab:node_case1}
		\end{table}
		The result of the simulation is in Figure~\ref{fig:node_case1_headways} and \ref{fig:node_case1_velocities}.

		Every car's leader is a bit further than their desired safety headway so every car is accelerating then they try to maintain their desired headway so they decelerate to the first car's speed.
		\begin{figure}
			\centering
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{node/n_car_headway_case1}
				\caption{Case 1: headways}
				\label{fig:node_case1_headways}
			\end{minipage}\hfill
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{node/n_car_velocity_case1}
				\caption{Case 1: velocities}
				\label{fig:node_case1_velocities}
			\end{minipage}
		\end{figure}
		\subsection*{Case 2}
		The initial positions and velocities of the cars can be seen on Table~\ref{tab:node_case2}.
		\begin{table}
			\centering
			\begin{tabular}{ |c|c|c| }
				\hline
				n [-] & $x$ [m] & $v$ [km/h]\\
				\hline
				1 & 160 & 100 \\
				2 & 120 & 100 \\
				3 & 80 & 100 \\
				4 & 40 & 100 \\
				5 & 0 & 100 \\
				\hline
			\end{tabular}
			\caption{Case 2}
			\label{tab:node_case2}
		\end{table}
		The result of the simulation is in Figure~\ref{fig:node_case2_headways} and \ref{fig:node_case2_velocities}.

		Every car's leader is a bit closer than their desired safety headway so every car is decelerating first, then they try to maintain their desired headway so the cars accelerate until they reach the first car's speed.
		\begin{figure}
			\centering
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{node/n_car_headway_case2}
				\caption{Case 2: headways}
				\label{fig:node_case2_headways}
			\end{minipage}\hfill
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{node/n_car_velocity_case2}
				\caption{Case 2: velocities}
				\label{fig:node_case2_velocities}
			\end{minipage}
		\end{figure}
		\subsection*{Case 3}
		The initial positions and velocities of the cars can be seen on Table~\ref{tab:node_case3}.
		\begin{table}
			\centering
			\begin{tabular}{ |c|c|c| }
				\hline
				n [-] & $x$ [m] & $v$ [km/h]\\
				\hline
				1 & 280 & 100 \\
				2 & 240 & 100 \\
				3 & 140 & 100 \\
				4 & 100 & 100 \\
				5 & 0 & 100 \\
				\hline
			\end{tabular}
			\caption{Case 3}
			\label{tab:node_case3}
		\end{table}
		The result of the simulation is in Figure~\ref{fig:node_case3_headways} and \ref{fig:node_case3_velocities}.

		Some of the cars are closer to their leader than their desired safety gap and the others have more headway than their safety gap. As a result the former ones will decelerate, the latter ones will accelerate to reach the desired headway. At the end all of them have the same speed as the leader car.
		\begin{figure}
			\centering
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{node/n_car_headway_case3}
				\caption{Case 3: headways}
				\label{fig:node_case3_headways}
			\end{minipage}\hfill
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{node/n_car_velocity_case3}
				\caption{Case 3: velocities}
				\label{fig:node_case3_velocities}
			\end{minipage}
		\end{figure}
		\subsection*{Case 4}
		The initial positions and velocities of the cars can be seen on Table~\ref{tab:node_case4}.
		\begin{table}
			\centering
			\begin{tabular}{ |c|c|c| }
				\hline
				n [-] & $x$ [m] & $v$ [km/h]\\
				\hline
				1 & 290 & 100 \\
				2 & 240 & 80 \\
				3 & 150 & 120 \\
				4 & 110& 115 \\
				5 & 0 & 70 \\
				\hline
			\end{tabular}
			\caption{Case 4}
			\label{tab:node_case4}
		\end{table}
		The result of the simulation is in Figure~\ref{fig:node_case4_headways} and \ref{fig:node_case4_velocities}.

		In this case the initial positions and the initial velocities were varied. As expected the result has changed significantly. Now both the speed and the headway must be changed by the drivers in order to get to the stationary point.
		\section*{Case summary}
		The same stationary point have been reached in every cases as expected. That stationary point was where all of the car had the same speed as the first car and they maintained their desired safety gap. In some cases it took longer to get to the stationary state then in others.
		\begin{figure}
			\centering
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{node/n_car_headway_case4}
				\caption{Case 4: headways}
				\label{fig:node_case4_headways}
			\end{minipage}\hfill
			\begin{minipage}{.5\textwidth}
				\centering
				\includegraphics{node/n_car_velocity_case4}
				\caption{Case 4: velocities}
				\label{fig:node_case4_velocities}
			\end{minipage}
		\end{figure}


\documentclass[12pt]{article}
\usepackage{graphicx}
\usepackage{enumerate}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage[mathscr]{euscript}
\usepackage{hyperref}
\usepackage{bbm}

\hypersetup{
    colorlinks=true,
    urlcolor=blue
}

\setlength{\oddsidemargin}{.1in}
\setlength{\textwidth}{6.3in}
\setlength{\textheight}{8.9in}
\setlength{\topmargin}{-.5in}

\title{Solving the inverted pendulum as a Markov decision process}
\date{}

\begin{document}

\maketitle

\section{Markov decision processes}

A Markov decision process (MDP) is defined by a 5-tuple $(\mathscr{S}, \mathscr{A}, P_\cdot(\cdot, \cdot), R_\cdot(\cdot, \cdot), \gamma)$ where

\begin{enumerate}[(i)]
\item $\mathscr{S}$ is a finite set of states;
\item $\mathscr{A}$ is a finite set of actions;
\item $P_a(s, s^\prime)$ is the probability that action $a$ in state $s$ at time $t$ will lead to state $s^\prime$ at time $t + 1$;
\item $R_a(s, s^\prime)$ is the immediate reward received after transitioning from state $s$ to state $s^\prime$ due to action $a$; and
\item $\gamma \in [0, 1]$ is a discount factor which represents the difference in importance between future rewards and present rewards.
\end{enumerate}

The problem is to find a policy for a decision-making agent that navigates the states of this world via actions. The policy is a function $\pi$ that specifies the action $\pi(s)$ that the agent will choose when in state $s$. Our goal is to choose a policy that will maximize a cumulative function of the random rewards; in particular, we wish to maximize the discounted sum of expected rewards over a potentially infinite horizon:

\begin{equation}
\sum_{t = 0}^{\infty} \gamma^t \mathbb{E}[R_{a_t}(s_t, s_{t+1})] \quad \mbox{where } a_t = \pi(s_t).
\end{equation}

\subsection{Policies and value functions}

Imagine that we have some magical function $V(\cdot)$ which tells us the inherent value of a state. It accepts a state as its argument, and its output value tells us \emph{how characteristically good} it is to be in that state for the purposes of achieving our goal (that is, maximizing reward). A bit philosophical, eh?

If we had this function, then choosing a policy would be trivial: the optimal policy is one which always selects the action that maximizes the expectation of $V(s^\prime)$ based on the present state $s$. The expectation of $V(s^\prime)$ under a given action $a$ in state $s$ is a sum of $V(s^\prime)$ over all possible $s^\prime$, with each term weighted by the probability of transitioning from $s$ to $s^\prime$ due to action $a$:

\begin{equation}
\mathbb{E}[V(s^\prime)] = \sum_{s^\prime \in \mathscr{S}} P_a(s, s^\prime) V(s^\prime).
\end{equation}

So the optimal policy is

\begin{equation}
\pi^*(s) = \underset{a}{\arg\max} \, \mathbb{E}[V(s^\prime)] = \underset{a}{\arg\max} \, \sum_{s^\prime \in \mathscr{S}} P_a(s, s^\prime) V(s^\prime).
\end{equation}

But, the inherent value of a state $V(\cdot)$ depends on what policy we are following. In particular, it is the sum of the expected reward over all future times if we act according to a policy $\pi$. Intuitively, there are two factors which influence the value of state $s$:

\begin{enumerate}[(i)]
\item{the immediate expected reward of acting on our policy in state $s$} and
\item{the expected value of the state which follows from acting on our policy in state $s$.}
\end{enumerate}

Note that the second of these is defined in terms of the expected value of the state which follows. Thus, our description of the value of state $s$ includes the value of state $s^\prime$, and in this way recursively captures something about the value of all future states.

Let's work out each of the two factors above, starting with the first piece. The expected reward of taking action $a$ in a given state $s$ is a sum of $R_a(s, s^\prime)$ over all possible $s^\prime$, with each term weighted by the probability of transitioning from $s$ to $s^\prime$ due to action $a$:

\begin{equation}
\mathbb{E}[R_a(s)] = \sum_{s^\prime \in \mathscr{S}} P_a(s, s^\prime) R_a(s, s^\prime).
\end{equation}

The action $a$ is selected by policy $\pi(s)$, so we can rewrite our equation above as the expected reward of acting under policy $\pi$ in a given state $s$:

\begin{equation}
\label{eq:reward} \mathbb{E}[R_{\pi(s)}(s)] = \sum_{s^\prime \in \mathscr{S}} P_{\pi(s)}(s, s^\prime) R_{\pi(s)}(s, s^\prime).
\end{equation}

This is our expression for the immediate expected reward of acting on our policy in state $s$ (the first of the two factors mentioned above). But the value of state $s$ depends not only on this immediately available reward; it also depends on the expected value of the state which follows from acting on our policy (the second of the two factors above):

\begin{equation}
\label{eq:future_value} \mathbb{E}[V(s^\prime)] = \sum_{s^\prime \in \mathscr{S}} P_{\pi(s)}(s, s^\prime) V(s^\prime).
\end{equation}

The full value of state $s$ is a sum of equations (\ref{eq:reward}) and (\ref{eq:future_value}), with the second weighted by our discount factor to reflect the fact that this is the value of a state ($s^\prime$) which is one step ahead in the future from the state ($s$) whose value we are presently considering. So, to get the value of state $s$ under policy $\pi$, we sum the expectation of the immediate reward for acting on our policy from state $s$ and the future-discounted expectation of the value of the state $s^\prime$ which follows from acting on our policy in state $s$:

\begin{align}
V(s) &= \mathbb{E}[R_{\pi(s)}(s)] + \gamma \mathbb{E}[V(s^\prime)] \\[5pt]
&= \sum_{s^\prime \in \mathscr{S}} P_{\pi(s)}(s, s^\prime) R_{\pi(s)}(s, s^\prime) + \gamma \sum_{s^\prime \in \mathscr{S}} P_{\pi(s)}(s, s^\prime) V(s^\prime) \\[5pt]
&= \sum_{s^\prime \in \mathscr{S}} P_{\pi(s)}(s, s^\prime) \Big(R_{\pi(s)}(s, s^\prime) + \gamma V(s^\prime)\Big)
\end{align}

Let's take stock. We have an expression for the optimal policy with respect to any given value function, and we have an expression for a value function under a given policy. To reiterate:

\begin{align}
\pi^*(s) &= \underset{a}{\arg\max} \, \mathbb{E}[V(s^\prime)] = \underset{a}{\arg\max} \, \sum_{s^\prime \in \mathscr{S}} P_a(s, s^\prime) V(s^\prime) \\[5pt]
V(s) &= \sum_{s^\prime \in \mathscr{S}} P_{\pi(s)}(s, s^\prime) \Big(R_{\pi(s)}(s, s^\prime) + \gamma V(s^\prime)\Big)
\end{align}

\subsection{The Bellman optimality equations}

But now we have a bit of a conundrum. We know the optimal policy with respect to a given value function, but our value function itself depends on what policy we're acting under! How do we find, at once, the optimal value function and its optimal policy?

There exists a large space of possible value functions due to all possible policies we could use. However, consider for a moment that we are handed from on high the optimal policy for our task. Under this policy, the output of the value function will be greater than or equal to the output of the value function under any other policy. This is because under the optimal policy, the value of any given state is always maximized. So, the maximum of all possible value functions for a state coincides with the value function we have when acting under an optimal policy.

With this in mind, we can readily define a notion of the optimal value function. The optimal value function for a state must equal the value function for that state under the best possible action for that state:

\begin{equation}
V^*(s) = \underset{a}{\max} \, V(s).
\end{equation}

Note that in this case, the action $a$ is the optimal policy $\pi^*(s)$. This enables us to write out $V(s)$ with no reference to $\pi(s)$ at all, instead substituting $\pi(s) = a$:

\begin{equation}
V^*(s) = \underset{a}{\max} \, \sum_{s^\prime \in \mathscr{S}} P_{\pi(s)}(s, s^\prime) \Big(R_{\pi(s)}(s, s^\prime) + \gamma V^*(s^\prime)\Big).
\end{equation}

This is a set of $|\mathscr{S}|$ nonlinear recurrence relations in $|\mathscr{S}|$ variables\footnotemark, each one requiring a maximum over $|\mathscr{A}|$ expressions ($|\mathscr{A}|$ and $|\mathscr{S}|$ are the number of elements in the sets $\mathscr{A}$ and $\mathscr{S}$, respectively). These are called the \emph{Bellman equations} for the optimal value function. Intuitively, solving each one amounts to finding the value of trying each available action in a state, and then selecting the value which corresponds to the best action in that state (the value of that state under an optimal policy). Solving them all simultaneously amounts to finding the value of all states at once when acting under an optimal policy, yielding the optimal value function.

\footnotetext{Such systems generally cannot be characterized analytically and lack a closed-form solution. In fact, solving nonlinear recurrence relations is often viewed as a hopeless endeavor: to solve \emph{every} nonlinear recurrence relation would imply that one could solve the Halting problem, as one could encode a program as initial states and the workings of the Turing machine as the recurrence relations (see \href{https://math.stackexchange.com/users/11513/rex-kerr}{this Stack Exchange discussion} by Rex Kerr). But these equations can be solved numerically (at least in principle), and with sufficiently small action and state spaces this computation is tractable. This where we're headed.}

Once we have $V^*(s)$, we can substitute this back into our expression for $\pi^*(s)$:

\begin{equation}
\pi^*(s) = \underset{a}{\arg\max} \, \sum_{s^\prime \in \mathscr{S}} P_a(s, s^\prime) V^*(s^\prime).
\end{equation}

This is an optimal policy, and our solution to the MDP.

\section{The inverted pendulum problem as an MDP}

To set up the inverted pendulum problem as an MDP, we need to define for our agent

\begin{enumerate}[(i)]
\item{a world of available states,}
\item{a set of actions it can take,}
\item{the transition probabilities which describe the physical dynamics of this world,}
\item{a reward function which incentivizes remaining balanced upright, and}
\item{a discount factor which reflects how long- or short-sighted our agent is.}
\end{enumerate}

\subsection{The agent's world}

First we will build the world of our agent by choosing (i) and (ii): the states it can visit and the actions it can take. The computational complexity of solving the Bellman equations explodes rapidly as the combinations of possible states and actions grow. So, our choices of (i) and (ii) ultimately depend on how long we are willing to wait, and unfortunately it doesn't take much to have to wait an astronomically period time. Since we're not willing to wait the age of the universe, our state and action spaces will have to be pretty small indeed.

Let's begin with a state space for our agent. The state of the pendulum-cart system can be fully characterized by four variables: angular position of the pendulum, angular velocity of the pendulum, position of the cart, and velocity of the cart. In order to avoid our state space becoming overly large, we will have to discretize each of these continuous variables into distressingly course-grained substates.

First let's consider the pendulum. For the angular position subspace $\mathscr{S}^\theta$, we choose the four substates

\begin{equation}
s^\theta = \left\{
\begin{array}{lr}
s_1^\theta \quad : -\pi \leq \theta < -\frac{\pi}{2} \\
s_2^\theta \quad : -\frac{\pi}{2} \leq \theta < 0 \\
s_3^\theta \quad : 0 \leq \theta < \frac{\pi}{2} \\
s_4^\theta \quad : \frac{\pi}{2} \leq \theta < \pi
\end{array}
\right.
.
\end{equation}

\begin{figure}
\center
\includegraphics[width=0.9\linewidth]{substates.png}
\caption{Visualization of the substate space for angular position.}
\label{fig:substates}
\end{figure}

For the angular velocity subspace $\mathscr{S}^\omega$, we similarly choose four substates

\begin{equation}
s^\omega = \left\{
\begin{array}{lr}
s_1^\omega \quad : \omega < -\alpha \\
s_2^\omega \quad : -\alpha \leq \omega < 0 \\
s_3^\omega \quad : 0 \leq \omega < \alpha \\
s_4^\omega \quad : \alpha \leq \omega
\end{array}
\right.
\end{equation}

where $\alpha$ is a parameter deciding the `width' of our state bins.

Now on to the cart. For its position subspace $\mathscr{S}^x$, we choose the four substates

\begin{equation}
s^x = \left\{
\begin{array}{lr}
s_1^x \quad : x < -\beta \\
s_2^x \quad : -\beta \leq x < 0 \\
s_3^x \quad : 0 \leq x < \beta \\
s_4^x \quad : \beta \leq x
\end{array}
\right.
\end{equation}

and for its velocity subspace $\mathscr{S}^v$ we choose four substates

\begin{equation}
s^v = \left\{
\begin{array}{lr}
s_1^v \quad : v < -\eta \\
s_2^v \quad : -\eta \leq v < 0 \\
s_3^v \quad : 0 \leq v < \eta \\
s_4^v \quad : \eta \leq v
\end{array}
\right.
\end{equation}

where $\beta$ and $\eta$ are again parameters deciding the `width' of the state bins.

The full state space $\mathscr{S}$ is $\mathscr{S}^\theta \times \mathscr{S}^\omega \times \mathscr{S}^x \times \mathscr{S}^v$, or the set of all ordered pairs $(s^\theta, s^\omega, s^x, s^v)$. By our choices above, this space has 256 states.

Our agent acts by applying horizontal forces to the cart. We'll allow our agent to choose among five actions, each applying a different, discrete-valued force to the cart:

\begin{equation}
\mathscr{A} = \left\{a_1, a_2, a_3, a_4, a_5\right\} \mbox{ where }\left\{
\begin{array}{lr}
a_1 \quad : f = -\kappa_2 \\
a_2 \quad : f = -\kappa_1 \\
a_3 \quad : f = 0 \\
a_4 \quad : f = \kappa_1 \\
a_5 \quad : f = \kappa_2
\end{array}
\right.
.
\end{equation}

$\kappa_1$ and $\kappa_2$ are parameters deciding the force of the action.

\subsection{How the world works}

Now our agent needs a model of the physical dynamics of its world and how its actions affect those dynamics. This model comes in the form of a set of transition probabilities $P_a(s, s^\prime)$---one for every possible action $a$---describing the probability of transitioning from state $s$ to state $s^\prime$ due to that action.

In our case the transition probabilities represent the physics of the pendulum-cart system. However, forming these transition probabilities is not totally straightforward: the states which describe our system each contain a wide, continuous range of actual physical configurations, leaving a great range of physical variability over which state will proceed from which. So, we will produce the transition probabilities via a Monte Carlo estimate of randomly simulated dynamics. For each state $a$ and state $s$, we'll randomly sample an angular position and velocity uniformly from within the state $s$, and run the simulation one step forward to end up in state $s_{next}$. We'll repeat this a large number of times $N$ and estimate the transition probabilities as

\begin{equation}
P_a(s, s^\prime) = \frac{1}{N}\sum_{i = 1}^N \mathbbm{1}(s_{next} = s^\prime).
\end{equation}

Since the most extreme states for velocity and angular velocity cover an unbounded range of physical possibilities (off to velocities of positive and negative $\infty$), we'll just have to choose physically realistic boundaries for the sake of sampling.

This is going to be an inherently flawed estimate. We're estimating the dynamics by sampling uniformly from the continuum of available positions and velocities within each state; yet in reality, there's no reason the physical system should visit all possible (continuous) positions and velocities within a state with equal probability. Nonetheless, let's consider this satisfactory, reasoning that our estimate of the transition probabilities should encode no finer-grained statistical assumptions about our system than our agent's representation of its state allows.\footnotemark

\footnotetext{Note that if we \emph{could} choose arbitrarily fine-grained states, this issue would go away: as the state ``slices" grew thinner, the physical dynamics of the system would become far less variable within each one, and our estimate of their transition probabilities would increasingly-well approximate the dynamics of the physical system. Furthermore, these transition probabilities would become increasingly sparse, reflecting the fact that in a causally deterministic system each state may transition to only one other state with unity probability. But alas, our agent must content itself to inhabit a simple---and unsettlingly noisy---world.}

\subsection{What the agent wants}

Next we need to construct a reward function that incentivizes our agent to do the right thing. We would like our agent to balance the pendulum and remain near the center of the track. This means there are two things we wish to avoid:

\begin{enumerate}[(i)]
\item{allowing the pendulum to fall below the level of the track and}
\item{moving the cart too close to the ends of the track.}
\end{enumerate}

So, we will let the reward function equal -1 for any transition into a state that satisfies either of these conditions, and 0 otherwise:

\begin{equation}
R_a(s, s^\prime) = \left\{
\begin{array}{lr}
-1 \quad : s^{\theta\prime} \in \left\{s_1^\theta, s_4^{\theta\prime} \right\}\mbox{ or }s^x \in \left\{s_1^{x\prime}, s_4^{x\prime} \right\} \\
0 \quad : \mbox{otherwise}
\end{array}
\right.
.
\end{equation}

Finally, we make sure to choose a discount factor $\gamma$ as well. Having fleshed out $\mathscr{S}$, $\mathscr{A}$, $P_\cdot(\cdot, \cdot)$, $R_\cdot(\cdot, \cdot)$, and $\gamma$, we have fully set up our inverted pendulum problem as an MDP. The rest is up to the Bellman equations.

\end{document}