\section{Massive Dirac Fermion Model}
The dispersion relation $E(k_x, k_y)$ looks like a cone.

New band touching,
we get Dirac fermion
\begin{align}
    H &=
    \int d^2 x\,
    \Psi^\dagger\left(
        -i\sigma^x v_x D_x
        - i\sigma^y D_y
        + m\sigma^z
    \right)\Psi
\end{align}
where
\begin{align}
    D_j &= \partial_j - i A_j
\end{align}
We can rescale space to get isotropic velocity
\begin{align}
    y \to \frac{v_y}{v_x}y
\end{align}

In the Lagrangian formulation
\begin{align}
    S &= \int d^3x\, \mathcal{L}
\end{align}
the Lagrangian density is
\begin{align}
    \mthcal{L} &=
    \overline{\Psi} \left( i \slash{D} - m \right) \Psi
\end{align}
where we use Feynman slash notation
\begin{align}
    \slash{D} = \gamma^{\mu}D_{\mu}
\end{align}
and
\begin{align}
    \overline{\Psi} &= \Psi^\dagger \gamma^0
\end{align}
and the Dirac matrices here are
\begin{align}
    \gamma^0 &= \sigma^z\\
    \gamma^1 &= i \sigma^x\\
    \gamma^2 &= i\sigma^y.
\end{align}
In continuum,
can also define on curved space-time a metric $g_{\mu\nu}$
with action
\begin{align}
    S &= \int d^3x\,
    \sqrt{|g|} \mathcal{L}
\end{align}
where you can diagonalize the metric with orthonormal frame fields
\begin{align}
    g_{\mu\nu}^a \eta_{\alpha\beta} e_{\nu}^b
\end{align}
and we use the Minkowski metric
\begin{align}
    \eta &=
    \begin{pmatrix}
        1 & &\\
        & -1 &\\
        & & -1
    \end{pmatrix}.
\end{align}
The group of transformations which leaves the metric invariant is
$SO(2, 1)$, which means there is a redundancy in this description,
as transformations like
\begin{align}
    e_{\mu}^a \to \sum_{b} W_{ab} e_\mu^b
\end{align}
where $W\in SO(2,1)$ describe the same thing.
This gauge symmetry leads to a $SO(2, 1)$
non-Abelian gauge field
\begin{align}
    W_\mu^{ab}
\end{align}
which is also called the ``spin connection''.
The holomony of this gauge filed will tell you how you wind up rotated as you go
in a closed loop.
It tells you how to do parallel transport in your space to relate frame fields to
each other in different points in space.

So the field strength of the spin connection will then give you the curvature of
spacetime.

The way you couple Dirac fermions to space-time curvature is to couple them to
this non-Abelian gauge field,
the spin connection.

\begin{question}
    What was the redundancy?
    If $W$ is just a redundancy,
    how can it define a field?
\end{question}
When you have $U(1)$ gauge symmetry,
you have
\begin{align}
    \psi \to e^{i\theta} \psi
\end{align}
Then we gauge the global symmetry.

\begin{question}
    Although it's called spin connection,
    it seems only related to the $SO(2, 1)$ spacetime,
    but not the field that lives on the spacetime?
\end{question}
What you're alluding to will come up next,
how to couple the field to the Dirac spinor,
during which we must pick a representation,
and we have to pick the spinor representation.

\begin{question}
    How will we be gauging this?
    These are local transformations to begin with,
    do we need to gauge them?
\end{question}
It's like turning on gravity.
When you turn on $g$,
you have curved spacetime,
and to describe it,
it's turning on a gauge field associated with $SO(2, 1)$.
It gives a way of thinking of curvature as an ordinary gauge term,
of $SO(2,1)$ in this case.

\begin{question}
    If you want your theory to be invariant,
    there should be some cancellation of fermionic field,s
    and since the field is present,
    it can demonstrate the curvature of spacetime.c
    But sine the $W$ is just a local Lorentz transformation,
    it doesn't carry.
\end{question}
I'm not spelling this out,
I'm just sketching the general thing.
Just to see what fields are involved in the description.
So fermions should couple to this gauge field $W_\mu^{ab}$.
So the covariant derivative $D_\mu \psi_\alpha$.

The spinor should be written as
\begin{align}
    \Psi =
    \begin{pmatrix}
        \psi_1\\
        \psi_2
    \end{pmatrix}
\end{align}
and then we want to couple to the gauge field the following way.
\begin{align}
    D_{\mu}\psi_\alpha &=
    \left( \partial_\mu - iA_\mu \right)\psi_\alpha
    + \frac{1}{2} W_\mu^{ab}\left( S_{ab} \right)_{\alpha\beta}\psi_\beta
\end{align}
where the $S_{ab}$ is the spinor representation of the Lorentz group,
can be written in terms of Gamma matrices.

So now we can consider integrating out the fermions.

By integrating out the fermions,
I mean we do something like this.
\begin{align}
    e^{iS_{\mathrm{eff}}[A,W]} &=
    \int \mathcal{D}\overline{\Psi}\,\mathcal{D}\Psi\,
    e^{iS[\Psi, A, W]}.
\end{align}
It will be homework,
but the point  is,
to lowest order,
you get Chern-Simons terms.
\begin{align}
    S_{\mathrm{eff}} &=
    \frac{1}{4\pi} \frac{\sgn(m)}{2}
    \int \epsilon^{\mu\nu\lambda} A_\mu \partial_\nu A_\lambda
    + \frac{\sgn(m)}{2} S_{\mathrm{CS,grav}}
\end{align}
where the second term is the gravitational Chern-Simons term for the non-Abelian
gauge field.

And this term is
\begin{align}
    S_{\mathrm{CS,grav}} \propto
    \int\Tr\left( w\wedge dw + \frac{2}{3} w\wedge w \wedge w \right).
\end{align}
Changing the sign of the mass gives
\begin{align}
    \Delta S &= \frac{1}{4\pi} \int A\, dA
    + S_{\mathrm{CS,grav}}
\end{align}
The point of this was to see what the effective action is from the point of view
of the Dirac fermions and see how we can reproduce the
$A_\mu \partial_\nu A_\lambda$
Chern-Simons term and that we also get this over gravitational term.

Every time we describe topological phases with free Fermions,
there's always some way to also describe it in terms of Majorana fermions
or Dirac fermion.

\begin{question}
    Why is there a 1/2 factor in
    \begin{align}
        \frac{1}{2}W_\mu^{ab} \left( S_{ab} \right)_{\alpha\beta}\psi_\beta.
    \end{align}
\end{question}

Now we have a Dirac fermion of 2+1D and we can chase the phase we're in by
changing the mass.
The same thing happens in other dimensions,
and by changing the mass term,
you can go through different topological phases.

\subsection{Chiral fermion on domain wall in mass}
If you have a system with boundaries,
you can have chiral fermion modes.
Because changing the sign of the mass changes the Chern number,
because it changes the effective action by one unit of the CS term.
It turns out,
at the domain wall,
we're going to have exponentially localized to the domain wall,
that's gapless and chiral.
Let $m(x)$ is negative for negative $x$ and positive for positive $x$.
It reduces to the Jakiw-Rabbi soliton in 1D.
The Hamiltonian is
\begin{align}
    H &=
    \int dx\, dy\,
    \Psi^\dagger\left(
        -i\sigma^x \partial_x - i\sigma^y \partial_y + m(x) \sigma^z
    \right)
\end{align}
where I set the velocities to 1.

For a plane wave
\begin{align}
    \Psi(x, y) &= \Psi(x) e^{ik_x y},
\end{align}
the eigenvalue equation is
\begin{align}
    \left( -i\sigma^x \partial_x + m(x) \sigma^z \right)\psi
    &=
    \left( E_{k_y} - k_y \sigma^y \right)\psi.
\end{align}
If you remember what we did in the Jakiv-rabbi Soliton,
there was a zero-energy solution where
$E_{k_y} + k_y \sigma^x$ was the solution
and it was exponentially localized to the domain wall.

Recall in the J-R soliton,
there is a $z$ mode solution $\psi_0$
with
\begin{align}
    \sigma^x \psi_0 = \lambda \psi_0
\end{align}
where $\lambda = \pm 1$.
This zero-energy solution means that we get a solution localized to the domain
wall with an energy
\begin{align}
    E_{k_y} &= -\lambda k_y.
\end{align}
If we had a domain wall where $m(x)<0$ on one side
but $m(x)>0$ on the other side,
we have a propagating fermion exponentially confined to the domain wall.

We have a chiral anomaly and a gravitational anomaly,
which is partially responsible for a quantized thermal quantum hall effect.

The $(1+1)D$ chiral fermion has a
$U(1)$ chiral anomaly,
which is a t' Hooft anomaly.
There is also a gravitational anomaly.
Both of these anomalies can be cancelled by the
$(2+1)D$ bulk,
which is the Chern insulator.
This is an example
of how you have anomalies in one dimension,
but are cancelled in higher dimensions.
Related to this is the fact that the chiral fermion is gives a quantized thermal
hall conductance.

The gravitational anomaly and the quantized thermal Hall conductance
defines a chiral central charge $C_{-}$.
It is the same as the Chern number for free fermion systems,
but that breaks down for interacting systems.

If you view them as boundaries of a Chern insulator,
there's no longer anything wrong with the anomalies.

U(1) chiral anomaly

The effective action for Dirac fermions is
\begin{align}
    S_{\mathrm{eff}} &=
    \int A\, dA
    + S_{\textrm{CS,grav}}
\end{align}

Then if we turn on the electric field
\begin{align}
    \frac{dp}{dt} &= E
\end{align}
and suppose we increase the momentum by $\Delta t= \int E\, dt$.
Imagine putting it on a  ring with periodic boundary conditions.
$p$ is quantized in units $2\pi/L$.
Then the charge is
\begin{align}
    \Delta Q &=
    \frac{\Delta p}{2\pi /L}\\
    &= \frac{L}{2\pi} \int E\, dt\\
    &= \frac{1}{2\pi} \int E\, dy dt\, j
\end{align}
and also nothing that the charge is just
\begin{align}
    \int \partial_t n \, dt\, dy
\end{align}
where $n$I s the charge density,
you get
\begin{align}
    \partial_t n &= \frac{1}{2\pi E}.
\end{align}
The covariant versions of this is
\begin{align}
    \partial_\mu j^\mu &= \frac{1}{2\pi}\epsilon_{\mu\nu}F^{\mu\nu}.
\end{align}
The key is that some conservation law breaks down when you turn on the field.

Suppose you have a cylinder with boundaries
The bulk cancels the anomaly because it tells you exactly where the charge comes
from.
If you have both edges at the same time,

Include both edges.
\begin{align}
    \partial_\mu j_{\nu}^{\mathrm{axial}} &=
    \frac{1}{2\pi} \epsilon_{\mu\nu} F_{\mu\nu}..
\end{align}
Sometimes axial current is called chiral current.

Earlier in the course I said you can have 't Hooft anomalies
which allow you to couple to the background gauge do
Trying to $A_\mu$
\begin{align}
    S_{\mathrm{coupling}} &=
    \int A_\mu j^\mj
    = \to \int_{} \left( A_\mu - \partial_\mu f \right) \gamma^\mu\\
    \delta S &= \int f \partial_\mu j^\mu &= 0
\end{align}

\subsection{Gravitational Anomaly}
To couple a field theory to gravity,
that means we want to couple it to some spacetime metric.
If we want to couple to $g_{\mu\nu}$,
we need a conserved stress energy tensor $T_{m\nu}$
which is conserved meaning
\begin{align}
    \Delta_\mu T^{\mu\nu}.
\end{align}
And the gravitational anomaly is a breakdown of this equation.
For example,
for the chiral fermion,
\begin{align}
    \nabla_\mu T^{\mu\nu} \propto C_{-}\epsilon^{\nu\sigma}\partial_{\sigma}R
\end{align}
where $C_{-}$ is the chiral central charge
and $R$ is the space-time curvature.

The next thing I want to describe is the last property,
which is the quantized thermal Hall conductance.

\subsection{Quantized Thermal Hall Conductance}
Suppose we have a $(2+1)$D Chern insulator region
where $C=1$,
which is coupled to a thermal bath at temperature $T_1$ with $C=0$
and the other side is coupled to a thermal bath at temperature $T_2$.
So we have a \emph{thermal current}
\begin{align}
    \vec{j}_Q = -\kappa \vec{\nabla} T
\end{align}
But here there is a quantized thermal hall conductance
\begin{align}
    \kappa_H &=
    \frac{\pi^2}{3} \frac{k_B^2}{h}T
\end{align}
per chiral edge mode
so the quantized thermal current is
\begin{align}
    j_{Q,X} &= -\kappa_H \Delta T.
\end{align}

The intuition is as follows.
If you couple the top to some temperature,
you're creating a thermal density of excitations,
where the average energy is $k_B T_1$
and same for the other side.
But because it's chiral,
you have a flow of energy along that boundary,
and a flow of energy along the other boundary in the opposite direction,
and if the temperatures and thus energies different,
there is a net flow of energy.

Consider a single chiral edge mode at temperature $T$.
The average energy density
\begin{align}
    n_Q &=
    \int \frac{dk}{2\pi}
    \left( \varepsilon_k - \mu \right)\left[
    \frac{1}{e^{\beta\left( \varepsilon_k - \mu \right)} + 1}
    - n_{FD;k}(T = 0)
    \right]
\end{align}
where the energy is $\varepsilon_k = \hbar v\cdot k$ and the thermal current is
\begin{align}
    j_Q = v\cdot n_Q = \frac{\pi^2}{6} \frac{k_B^2}{h}T^2.
\end{align}
For two edge modes,
one going to the right with current
\begin{align}
    j_{Q,1} &=
    \frac{\pi^2}{6} \frac{k_B^2}{h} T_1^2
\end{align}
and one going to the left
\begin{align}
    j_{Q,2} &=
    \frac{\pi^2}{6} \frac{k_B^2}{h} T_2^2
\end{align}
so the total current is their difference
\begin{align}
    j_{Q,\mathrm{tot}} &=
    \frac{\pi^2}{6} \frac{k_B^2}{h}
    \left( T_1^2 - T_2^2 \right)\\
    &=
    \underbrace{\frac{\pi^2}{6} \frac{k_B^2}{h} 2T_1 \Delta T}_{\kappa_H}
    + \mathcal{O}\left( \Delta T^2 \right)
\end{align}
where $T_2 = T_1 + \Delta T$.
In general,
\begin{align}
    \kappa_H &=
    C_{-} \frac{\pi^2}{3} \frac{k_B^2}{h}T
\end{align}
and here the chiral central charge $C_{-}$ is the number of right-moving chiral
fermion modes minus the number of left-moving chiral fermion modes on a given
edge.

The reason we introduce this is because
Majorana fermions are half a complex fermion,
so it has a $\frac{1}{2}$ chiral central charge.
In fact,
it's much more general,
and has to do with a conformal field theory.
More complicated topological quantum phases of matter
with boundaries not described by chiral fermions,
but there is still a chiral central charge.

\begin{question}
    About the $U(1)$ anomaly there's a way to see the bulk.
    Is there a way to see how the gravitational anomaly comes from the bulk?
\end{question}
The gravitational anomaly,
you can do calculations and put your system on a curved spacetime,
you see extra energy,
and if you write the extra CS term,
it will account for the non-conservation of the stress energy tensor.
But you won't be able to relate it to the quantized thermal Hall effect,
because there is no boundary.
The boundaries are supposed to be moving as we adiabatically insert flux.
You can't really relate it to the thermal hall effect in a clear way.

\begin{question}
    Is the only way you can have these chiral fermions is in relation to the
    bulk?
\end{question}

\begin{question}
    Is the gravitational anomaly what leads to the quantized hall conductance?
\end{question}
I don't have a good explanation except that $C_-$ appears in both cases.
Just like how we derive conductivity by turning on an electric field,
you can turn on thermal conductivity by turning on a gravitational field.
In the context of the quantum Hall effect,
there have been many studies,
but I don't think it led to a direct connection between the thermal Hall effect
and the gravitational response.
From my perspective,
there's central charges appearing in both places.
There might e ways of introducing thermal conductivity on the edge
by introducing gravitational potential.
There's a fundamental mismatch,
because the bulk can never have thermal quantum hall effect.
It's possible other people might have a better answer.

\begin{question}
    A cylinder with a gradient around the cylinder?
\end{question}
That doesn't even make sense.
You can't really put a temperature gradient on the boundary of this thing
because it's chiral.
I didn't put a gradient along the edge,
I don't think it's even something you can do.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                                                                 %
%   PAW   - Reference Manual -- LaTeX Source                      %
%                                                                 %
%   Chapter 6: SIGMA                                              %
%                                                                 %
%   EPS file      : sigexa1.eps                                   %
%                                                                 %
%   Editor: Michel Goossens / CN-AS                               %
%   Last Mod.:  8 Feb 1994 20:15 mg                               %
%                                                                 %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{SIGMA}
\label{chap-sigma}
\index{SIGMA|(}
\def\PAWchap{SIGMA}

\section{Access to SIGMA}
\label{sec:H2SIGMA}
\index{SIGMA!access}

The SIGMA array manipulation package can be
accessed in three different ways in PAW:

\subsection*{Precede the statement by the prefix SIGMA}
\index{SIGMA!prefix SIGMA}
\index{prefix SIGMA}

\subsection*{Example}
\begin{alltt}
  PAW > \Ucom{SIGMA xvec=array(100,-pi#pi*2)}
  PAW > \Ucom{SIGMA y=sin(xvec)*xvec}
\end{alltt}
Note the use of the predefined constant \texttt{PI}
in SIGMA with the obvious value.

\subsection*{The PAW command: APPLication SIGMA}
\index{SIGMA!APPLication SIGMA}
\index{application SIGMA}

All commands typed in after this command will be directly processed by
SIGMA. The command \Ucom{EXIT} will return control to PAW, e.g.

\begin{alltt}
  PAW > \Ucom{APPLication SIGMA}
  SIGMA > \Ucom{xvec=array(100,-pi#pi*2)}
  SIGMA > \Ucom{sinus=sin(xvec)*xvec}
  SIGMA > \Ucom{cosinus=cos(xvec)*xvec}
  SIGMA > \Ucom{exit}
  PAW > \Ucom{vector/list}
   Vector Name                    Type    Length    Dim-1    Dim-2    Dim-3

   XVEC                              R       100      100
   SINUS                             R       100      100
   COSINUS                           R       100      100

   Total of  3 Vector(s)
\end{alltt}

\subsection*{The PAW system function \dollar SIGMA}
\label{sec:H3SIGVE}
\index{SIGMA!\protect\dollar SIGMA}
\index{\protect\dollar SIGMA}

The expression to be evaluated must be
enclosed in parentheses. The function will return the numerical
value of the expression (if the result is a scalar) or the
name of a temporary vector (if the result is a vector).

Assuming that the computation of the function \texttt{sin(x)*x}
in the above
example would be only for the purpose of producing a graph, (i.e.
the result is not needed for further calculations), then one could
just have typed the following commands:

\begin{alltt}
  PAW > \Ucom{SIGMA xvec=array(100,-pi#pi*2)}
  PAW > \Ucom{GRAph 100 xvec $SIGMA(SIN(XVEC)*XVEC)}
\end{alltt}

\section{Vector arithmetic operations using SIGMA}
\index{array!in SIGMA}
\index{vector!operations}
\index{vector!in SIGMA}
\index{vector!arithmetic}
\index{SIGMA!vector}
\index{SIGMA!array}

A complete and convenient mechanism for the mathematical
manipulation of vectors is provided by SIGMA. In the following,
we use the words ``array'' and ``vector'' as synonyms.
In both
cases, we refer to PAW vectors, in the sense that SIGMA offers
an alternative way to generate and to manipulate PAW vectors
(see section \ref{sec:H1PVECT} on page~\pageref{sec:H1PVECT}).
The notation of SIGMA is similar to that of FORTRAN, in the sense
that is based upon formulae and assignment statements.

The special operator \texttt{ARRAY} is used to generate vectors:

\PAWfdef{ARRAY}{vname = ARRAY (arg1,arg2)}

\begin{DLtt}{12345}
\item[vname] Name of the vector (array) being created.
\item[arg1]  Defines the array {\bf structure},
             i.e. the Number of COmponents (\texttt{NCO}) of the array.
\item[arg2]  Provides the {\bf numerical values} filling the array
             row-wise.\\
             If \texttt{arg2} is absent (or does not provide enough
             values) the array is filled with 1.
\end{DLtt}
\index{array!filling}
\index{SIGMA!array!structure}
\index{SIGMA!array!filling}
\index{operator in SIGMA}

\subsection{Basic operators}
\index{SIGMA!basic operator}
\index{basic operator in SIGMA}

\begin{DLtt}{123}
\item[+]  Add
\item[-]  Subtract
\item[*]  Multiply
\item[/]  Divide
\item[**] Exponentiation
\item[\&] Concatenation
\end{DLtt}

\fbox{\parbox{.97\textwidth}{%
Note that ill defined operations will give \texttt{0.} as result. For
instance: a division by zero gives zero as result.}}

\subsection{Logical operators}
\index{SIGMA!logical operator}
\index{SIGMA!boolean value}
\index{logical operator in SIGMA}
\index{boolean value in SIGMA}

Logical operators act on entities that have
{\bf Boolean} values \texttt{1} (true) or \texttt{0} (false).
The result is Boolean.

\begin{DLtt}{123}
\item[AND] Logical operation AND
\item[NOT] Logical operation NOT
\item[OR]  Logical operation OR
\item[EQ]  EQual to
\item[GE]  Greater or Equal to
\item[GT]  Greater Than
\item[LE]  Less or Equal to
\item[LT]  Less Than
\item[NE]  Not Equal
\end{DLtt}

\subsection{Control operators}
\index{SIGMA!control operator}
\index{control operator in SIGMA}

\begin{DLtt}{12345678}
\item[!PRINT]   Provides the automatic printing of
                every newly created array or scalar.
\item[!NOPRINT] Suppresses the automatic printing of
                every newly created array or scalar.
\end{DLtt}

\subsection*{Examples}
\begin{alltt}
\Ucom{A=ARRAY (6,1#6)}             1  2  3  4  5  6
\Ucom{A=ARRAY (4)}                 1  1  1  1
\Ucom{A=ARRAY (5,2&3&-1&2&1.2)}    2  3  -1  2  1.2
\Ucom{A=ARRAY (3)*PI}              3.1415927  3.1415927  3.1415927
\Ucom{A=ARRAY (1,123E4)}           1230000.0
\end{alltt}

\section{SIGMA functions}
\index{SIGMA!function}
\index{function!in SIGMA}

SIGMA provides some functions which perform a task on a whole array. These
functions have no analogues in FORTRAN because all FORTRAN functions operate on
one or more single numbers. Presently available SIGMA functions
are listed in table \ref{tab:SIGFUN} below.

\begin{table}[t]
\begin{center}
\begin{tabular}{|llp{.75\textwidth}|}
\hline
\rm\bf Name            & \rm\bf Result                    &
\rm\bf Explanation                                        \\
\hline
\PAWfind{ANY}          & Scalar                           &
The result is a Boolean scalar of value \texttt{1} (true)
if at least one component of the argument
is true and \texttt{0} (false) otherwise.                    \\
\PAWfind{DEL}          & Vector                           &
Analog to the {\bf Dirac-DELta Function}.
\texttt{V1=DEL(V)} sets each element of \texttt{V1} to \texttt{0.0}
(if corresponding element in \texttt{V}
is {\bf non-zero}) or to \texttt{1.0}
(if corresponding element is {\bf zero}).                 \\
\PAWfind{DIFF}         & Vector                           &
\texttt{V2=DIFF(V)} {\bf forward difference} of \texttt{V}.
The rightmost value in \texttt{V1} is obtained by quadratic
extrapolation over the last three elements of \texttt{V}.    \\
\PAWfind{LS}           & Vector                           &
\texttt{V1=LS(V,N)} {\bf shifts} index of \texttt{V} to the left
by \texttt{N} steps (cyclic).                                \\
\PAWfind{LVMAX}        & Scalar                           &
\texttt{S1=LVMAX(V1)} sets \texttt{S1} equal to the {\bf index}
(location) of the {\bf maximum} value in vector \texttt{V1}. \\
\PAWfind{LVMIM}        & Scalar                           &
\texttt{S1=LVMIN(V1)} sets \texttt{S1} equal to the {\bf index}
(location) of the {\bf minimum} value in vector \texttt{V1}. \\
\PAWfind{MAX}          & Vector                           &
\texttt{V3=MAX(V1,V2)}
sets each element of \texttt{V3} equal to the {\bf maximum}
of the corresponding elements in \texttt{V1} and \texttt{V2}.   \\
\PAWfind{MAXV}         & Vector                           &
\texttt{V1=MAXV(V)} sets each element of \texttt{V1} equal
to the {\bf maximum} value in \texttt{V}.                    \\
\PAWfind{MIN}          & Vector                           &
\texttt{V3=MIN(V1,V2)}
sets each element of \texttt{V3} equal to the {\bf minumum}
of the corresponding elements in \texttt{V1} and \texttt{V2}.   \\
\PAWfind{MINV}         & Vector                           &
\texttt{V1=MINV(V)} sets each element of \texttt{V1} equal
to the {\bf minimum} value in \texttt{V}.                    \\
\PAWfind{NCO}          & Scalar                           &
\texttt{V1=NCO(V)} {\bf Number of COmponents}
of vector of \texttt{V}.                                     \\
\PAWfind{ORDER}        & Vector                           &
\texttt{V1=ORDER(V,V2)} finds a permutation that brings
\texttt{V2} in a non-descending order
and applies it to \texttt{V} to generate \texttt{V1}.           \\
\PAWfind{PROD}         & Vector                           &
\texttt{V1=PROD(V)} \texttt{V1} is the {\bf running product}
of \texttt{V}.                                               \\
\PAWfind{QUAD}         & Vector                           &
\texttt{V2=QUAD(V1,H)}
The quadrature function \texttt{QUAD} numerically integrates
each row of \texttt{V1} with respect
to the scalar step size \texttt{H}.                          \\
\PAWfind{SUMV}         & Vector                           &
\texttt{V2=SUMV(V1)} {\bf running sum} of \texttt{V}.           \\
\PAWfind{VMAX}         & Scalar                           &
\texttt{S1=VMAX(V1)} sets \texttt{S1} equal to the
{\bf maximum} value in vector \texttt{V1}.                   \\
\PAWfind{VMIN}         & Scalar                           &
\texttt{S1=VMIN(V1)} sets \texttt{S1} equal to the
{\bf minimum} value in vector \texttt{V1}.                   \\
\PAWfind{VSUM}         & Scalar                           &
\texttt{S1=VSUM(V)} {\bf sum} of all components of \texttt{V}.  \\
\hline
\end{tabular}
\end{center}
\caption{SIGMA functions}
\label{tab:SIGFUN}
\end{table}

\subsection{SIGMA functions - A detailed description.}

In the following description of the SIGMA functions, the letter \texttt{R} always
denotes the {\bf result} and \texttt{arg} denotes one or more {\bf arguments}.
Any argument may itself be an expression.
In that case \texttt{arg} means the result of this expression.
Let \texttt{OP} denote any of the above array functions, then the statement:

\PAWfdef{OP}{R=OP(arg1,arg2,...)}

produces \texttt{R} without doing anything to the
contents stored under the names
appearing in \texttt{arg1,arg2,...}.
Thus, although in the description we may say
``...\texttt{OP} does such and such to \texttt{arg} ...'',
in reality it leaves \texttt{arg} intact and works on
the argument to produce \texttt{R}.

\PAWfdef{ANY}{R = ANY(arg)}

The function \texttt{ANY} considers the result
of the argument expression as a Boolean
array. SIGMA represents ``true'' by \texttt{1} and ``false'' by \texttt{0}.
Thus the components of \texttt{arg}
must be either \texttt{0} or \texttt{1}, otherwise an error is generated.

If at least one component of the result of the argument expression is \texttt{1},
than \texttt{ANY} returns the scalar \texttt{1}. If all components of the result of the argument
expression are \texttt{0} then \texttt{ANY} returns the scalar \texttt{0}.
If \texttt{arg} is a Boolean scalar, \texttt{R = arg}.

\subsection*{Example of the \texttt{ANY} command}
\begin{alltt}
  PAW > \Ucom{APPL SIGMA}
  SIGMA > \Ucom{!PRINT}                            | Print newly created vectors and scalars
  SIGMA > \Ucom{W=(-2)**ARRAY(10,1#10)}
  NCO(W       )=   10
  W       =
   -2.000      4.000     -8.000      16.00     -32.00      64.00
   -128.0      256.0     -512.0      1024.
  SIGMA > \Ucom{X=W GT 0}
  NCO(X       )=   10
  X       =
   0.0000      1.000     0.0000      1.000     0.0000      1.000
   0.0000      1.000     0.0000      1.000
  SIGMA > \Ucom{R=ANY(X)}
  NCO(R       )=    1
  R         1.000
\end{alltt}

\PAWfdef{DEL}{R = DEL(arg)}
\index{delta function}

\texttt{DEL} is a discrete analogue of a {\bf Dirac delta function}.
\texttt{DEL} works independently
on each row of the argument array.
If the elements of any row of the argument are denoted by
\(X_1,\:X_2,\:\dots,\:X_i,\:\ldots,\:X_n\)
then the corresponding row of the result of the delta function
operation will be
\(Z_1,\:Z_2,\:\dots,\:Z_i,\:\ldots,\:Z_n\)
where all \(Z_i = 0\) except in three cases,
in which \(Z_i = 1\), namely:

\begin{OL}
\item When the component \( X_i \) is itself zero.
\item When \( X_{i-1}, \:X_i \) are of opposite sign and
      \( \left| X_i \right| < \left| X_{i-1} \right| \)
      If \( i = 1 \) then linear extrapolation to the left is used.
\item When \( X_i, \:X_{i+1} \) are of opposite sign and
      \( \left| X_i \right| \leq \left| X_{i+1} \right| \)
      If \( i = 1 \) then linear extrapolation to the right is used.
\end{OL}

If \texttt{arg} is a scalar, the value of \texttt{DEL(arg)} will be \texttt{1}
if \texttt{arg} is zero, and \texttt{0} otherwise.

\subsection*{Example of the \texttt{del} command}
\begin{alltt}
  SIGMA > \Ucom{W=array(11,-1#1)}
  NCO(W       )=   11
  W       =
   -1.000    -0.8000    -0.6000    -0.4000    -0.2000    -0.2980E-07
   0.2000     0.4000     0.6000     0.8000      1.000

  SIGMA > \Ucom{X= (W+1.01)*W*(W-.35)*(W-.42)}
  NCO(X       )=   11
  X       =
  -0.1917E-01 -0.2357    -0.2384    -0.1501    -0.5524E-01-0.4425E-08
   0.7986E-02 -0.5640E-03 0.4347E-01 0.2476     0.7578

  SIGMA > \Ucom{R=del(x)}
  NCO(R       )=   11
  R       =
    1.000     0.0000     0.0000     0.0000     0.0000      1.000
   0.0000      1.000     0.0000     0.0000     0.0000
\end{alltt}

\PAWfdef{DIFF}{R = DIFF(arg)}
\index{DIFF}

The \texttt{DIFF} function generates the {\bf forward difference}
of each row of the argument array, say
\(X_1,\:X_2,\:\ldots,\)
\(X_i,\:\ldots,\:X_n\)
and creates an array with components equal to the forward difference of
\(X\): \(X_2-X_1,\:X_3-X_2,\)
\(\ldots,\:X_n-X_{n-1},X_0\)
where the rightmost value \(X_0\)
is obtained by quadratic extrapolation over the last
three elements of the result of \texttt{arg}.
Applied to a scalar \texttt{DIFF} gives a zero result.

\subsection*{Example of the \texttt{DIFF} command}
\begin{alltt}
  SIGMA > \Ucom{x=array(6,5#0)}
  NCO(X       )=    6
  X       =
    5.000      4.000      3.000      2.000      1.000     0.0000
  SIGMA > \Ucom{Y=x*x}
  NCO(Y       )=    6
  Y       =
    25.00      16.00      9.000      4.000      1.000     0.0000
  SIGMA > \Ucom{Z=Diff(Y)}
  NCO(Z       )=    6
  Z       =
   -9.000     -7.000     -5.000     -3.000     -1.000      1.000
\end{alltt}

\PAWfdef{LS}{R=LS(arg1,arg2)}
\index{LS}

The \texttt{LS} rearrangement function performs a {\bf left shift}.
\texttt{arg1} is the array to be shifted;
\texttt{arg2} must be a scalar value (rounded
if necessary by the system),
interpreted as the number of places the array has to
be shifted to the left.
The scalar \texttt{arg2} can be negative, in which case \texttt{LS} shifts
to the right a number of places equal to the absolute value of \texttt{arg2}.

It should be noted the the shift is performed circularly \texttt{modulo N},
where \texttt{N}
is the number of components in the rows of the array to be shifted.
Hence, \texttt{LS(X,N+l)}
shifts the \texttt{N} component rows of \texttt{X} by \texttt{1} to the left,
and \texttt{LS(X,-l)} shifts the rows by
\texttt{N-1} to the left (or by \texttt{1} to the right).
If \texttt{arg1} is a scalar, \texttt{R = arg1}.

\subsection*{Example of the left shift command}
\begin{alltt}
 SIGMA > \Ucom{X=array(4&5,array(20,1#20))}
 NCO(X       )=    4    5
 X       =
   1.000      2.000      3.000      4.000
   5.000      6.000      7.000      8.000
   9.000      10.00      11.00      12.00
   13.00      14.00      15.00      16.00
   17.00      18.00      19.00      20.00

 SIGMA > \Ucom{y=ls(x,1)}

 NCO(Y       )=    4    5
 Y       =
   2.000      3.000      4.000      1.000
   6.000      7.000      8.000      5.000
   10.00      11.00      12.00      9.000
   14.00      15.00      16.00      13.00
   18.00      19.00      20.00      17.00

 SIGMA > \Ucom{y=ls(x,-3)}
 NCO(Y       )=    4    5
 Y       =
   2.000      3.000      4.000      1.000
   6.000      7.000      8.000      5.000
   10.00      11.00      12.00      9.000
   14.00      15.00      16.00      13.00
   18.00      19.00      20.00      17.00

 SIGMA > \Ucom{X=array(5,1#5)}
 NCO(X       )=    5
 X         1.000      2.000      3.000      4.000      5.000
 SIGMA > \Ucom{z=ls(x,3)}
 NCO(Z       )=    5
 Z         4.000      5.000      1.000      2.000      3.000
 SIGMA > \Ucom{z1=ls(x,-4)}
 NCO(Z1      )=    5
 Z1        2.000      3.000      4.000      5.000      1.000
\end{alltt}

\PAWfdefii{LVMAX}{R=LVMAX(arg1)}{LVMIN}{R=LVMIN(arg1)}

The functions \texttt{LVMAX} and \texttt{LVMIN}
returns as a scalar result the index (position) of
the largest or smallest element, respectively,
in the argument array.

\subsection*{Example of using the \texttt{LVMAX} and \texttt{LVMIN}
  commands}
\begin{alltt}
 SIGMA >\Ucom{  x=sin(array(10,1#10))}
 NCO(X       )=   10
 X       =
  0.841     0.909     0.141    -0.757    -0.959    -0.279     0.657
  0.989     0.412    -0.544

 SIGMA >\Ucom{ r=lvmax(x)}
 NCO(R       )=    1
 R         8.00
\end{alltt}

\PAWfdefii{MAX}{R=MAX(arg1,arg2)}{MIN}{R=MIN(arg1,arg2)}

The functions \texttt{MAX} and \texttt{MIN} work
independently on each element of their arguments.
\texttt{arg2} can be a scalar.
The result has the same dimension as the argument array
\texttt{arg1} and each element of
the result is set equal to the largest or smallest element, respectively,
of the corresponding element of the argument arrays.

\subsection*{Example of using the \texttt{MAX} and \texttt{MIN}
  commands}
\begin{alltt}
 SIGMA >\Ucom{  x=sin(array(10,1#10))}
 NCO(X       )=   10
 X       =
  0.841     0.909     0.141    -0.757    -0.959    -0.279     0.657
  0.989     0.412    -0.544

 SIGMA >\Ucom{   y=cos(array(10,1#10))}
 NCO(Y       )=   10
 Y       =
  0.540    -0.416    -0.990    -0.654     0.284     0.960     0.754
 -0.146    -0.911    -0.839

 SIGMA >\Ucom{ z=min(x,y)}
 NCO(Z       )=   10
 Z       =
  0.540    -0.416    -0.990    -0.757    -0.959    -0.279     0.657
 -0.146    -0.911    -0.839
\end{alltt}

\PAWfdefii{MAXV}{R=MAXV(arg)}{MINV}{R=MINV(arg)}

The extrema functions \texttt{MAXV} and \texttt{MINV} work
on each element of their argument and
the result has the same dimension as the argument array \texttt{arg1}.
Each element of
of the result is set equal to the largest or smallest element, respectively,
of the corresponding row of the argument array.

All these functions, if applied to a scalar argument, yield \texttt{R=arg}.

\subsection*{Example of using the \texttt{MAX} and \texttt{MIN}
  commands}
\begin{alltt}
  SIGMA > \Ucom{x=array(10,0#10)}
  NCO(X       )=   10
  X       =
   0.0000      1.111      2.222      3.333      4.444      5.556
    6.667      7.778      8.889      10.00

  SIGMA > \Ucom{s=sin(x)*x}
  NCO(S       )=   10
  S       =
   0.0000     0.9958      1.767    -0.6352     -4.286     -3.695
    2.494      7.755      4.539     -5.440

  SIGMA > \Ucom{ x=minv(s)}
  NCO(X       )=   10
  X       =
   -5.440     -5.440     -5.440     -5.440     -5.440     -5.440
   -5.440     -5.440     -5.440     -5.440
\end{alltt}

\PAWfdef{NCO}{R = NCO(arg)}

The ``Number of COmponents'' (\texttt{NCO}) control
function obtains the \texttt{NCO} vector of the \texttt{arg}.
The \texttt{NCO} vector of a scalar is the scalar \texttt{1}.
For any argument the \texttt{NCO(NCO(arg))} gives the number
of dimensions of the \texttt{arg}.

\subsection*{Using the \texttt{NCO} command}
\begin{alltt}
 SIGMA > \Ucom{ x=array(4&3&2,array(24,2#48))}
 NCO(X       )=    4    3    2
 X       =
   2.000      4.000      6.000      8.000
   10.00      12.00      14.00      16.00
   18.00      20.00      22.00      24.00

   26.00      28.00      30.00      32.00
   34.00      36.00      38.00      40.00
   42.00      44.00      46.00      48.00

 SIGMA > \Ucom{ r=nco(x)}
 NCO(R       )=    3
 R         4.000      3.000      2.000
 SIGMA > \Ucom{ ndim=nco(nco(x))}
 NCO(NDIM    )=    1
 NDIM      3.000
\end{alltt}

\PAWfdef{ORDER}{R=ORDER(arg1,arg2)}

The ordering function \texttt{ORDER} acts independently on each row
of \texttt{arg1}. \texttt{arg2} must have the same row length as \texttt{arg1}.

\texttt{ORDER} finds the permutation that
brings \texttt{arg2} into a non-descending sequence
(row-wise) and constructs the result by applying
this permutation to \texttt{arg1}.
It may in some cases be expanded to that structure by
using the techniques of the topological arithmetic.
This is particularly useful if \texttt{arg2} is
a single vector with the length of the rows of \texttt{arg1}.

\subsection*{Using the \texttt{ORDER} command}
\begin{alltt}
 SIGMA > \Ucom{X = 1&1&2&4&-3&1&3}
 NCO(X       )=    7
 X       =
   1.00      1.00      2.00      4.00     -3.00      1.00      3.00
 SIGMA > \Ucom{P = ORDER(X,X)}
 NCO(P       )=    7
 P       =
  -3.00      1.00      1.00      1.00      2.00      3.00      4.00
 SIGMA > \Ucom{P = ORDER(X,-X)}
 NCO(P       )=    7
 P       =
   4.00      3.00      2.00      1.00      1.00      1.00     -3.00
 SIGMA > \Ucom{Y = ARRAY(7,1# 7)}
 NCO(Y       )=    7
 Y       =
   1.00      2.00      3.00      4.00      5.00      6.00      7.00
 SIGMA > \Ucom{P = ORDER(Y,X)}
 NCO(P       )=    7
 P       =
   5.00      1.00      2.00      6.00      3.00      7.00      4.00
\end{alltt}

\PAWfdef{PROF}{R=PROD(arg)}

The \texttt{PROD} function generates the {\bf running product}
of each row of the argument
array, say \( X_1,\: X_2,\ldots,\:X_n \)
and creates an array with components equal to the
running product of the component of the argument:
\( X_1,\: X_2,\ldots,\:X_n \)
\(X_1 ,X_1 \times X_2 ,\: \ldots, \: X_1 \times X_2 \times \ldots X_n \)

\subsection*{Using the \texttt{TIMES} command}
\begin{alltt}
  SIGMA > \Ucom{x=array(6&4,array(24,1#24))}
  NCO(X       )=    6    4
  X       =
    1.000      2.000      3.000      4.000      5.000      6.000
    7.000      8.000      9.000      10.00      11.00      12.00
    13.00      14.00      15.00      16.00      17.00      18.00
    19.00      20.00      21.00      22.00      23.00      24.00

  SIGMA > \Ucom{y=prod(x)}
  NCO(Y       )=    6    4
  Y       =
    1.000      2.000      6.000      24.00      120.0      720.0
    7.000      56.00      504.0      5040.     0.5544E+05 0.6653E+06
    13.00      182.0      2730.     0.4368E+05 0.7426E+06 0.1337E+08
    19.00      380.0      7980.     0.1756E+06 0.4038E+07 0.9691E+08
\end{alltt}

\PAWfdef{QUAD}{R=QUAD(arg1,arg2)}

The {\bf quadrature function} \texttt{QUAD} numerically integrates each row of
\texttt{arg1} with respect to the scalar step size \texttt{h} defined by \texttt{arg2}.

The result \texttt{R} has the same dimension as \texttt{arg1}
and the integration constant is
fixed by choosing the first point of the result to be zero.

The method uses a four-point forward and backward one-strip-formula based
on Lagrange interpolation. We have for the first point of the result:
\[
R_1 = \int_{ x_1 }^{ x_1 } ( arg1 ) \mathrm{d}x = 0
\]

for the second and third points
\[
R_{i+1} = R_i + \frac{h}{24} ( 9 f_i + 19 f_{i+1} - 5 f_{i+2} + f_{i+3} )
\]

and for all subsequent points
\[
R_i = R_{i-1} + \frac{h}{24} ( f_{i-3} - 5 f_{i-2} + 19 f_{i-1} + 9 f_i )
\]

where the \( f_i \) are elements of \texttt{arg1} and are assumed to be values of
some functions evaluated at equidistant intervals
with interval width equal to \texttt{h} (\texttt{h} being
equal to the value of \texttt{arg2}).

\begin{figure}
\begin{minipage}{.33\textwidth}
\begin{alltt}
SIGMA > *********************
SIGMA > * SIGMA application *
SIGMA > *  showing use of   *
SIGMA > *   QUAD numeric    *
SIGMA > *   integration     *
SIGMA > *********************
SIGMA > \Ucom{x=array(101,0#2*pi)}
SIGMA > * Function value array
SIGMA > \Ucom{y=sin(x)}
SIGMA > * Step size
SIGMA > \Ucom{dx=0.6283186E-01}
SIGMA > \Ucom{print dx}
 NCO(DX      )=    1
 DX       0.6283186E-01
SIGMA > * Integration of SIN(X)
SIGMA > * in interval 0<=X<+2*PI
SIGMA > \Ucom{f=quad(y,dx)}
SIGMA > * Analytical result
SIGMA > * is   1-COS(X)
SIGMA > \Ucom{g=1-cos(x)}
SIGMA > * Compute the difference
SIGMA > \Ucom{erro=(g-f)*10**6}
SIGMA > * Plot the difference
SIGMA > *  in units of \(10\sp{-6}\)
SIGMA > \Ucom{exit}
PAW > \Ucom{opt GRID}
PAW > \Ucom{gra 101 x erro}
\end{alltt}
\end{minipage}
\hfill
\begin{minipage}{.66\textwidth}
\begin{center}\mbox{\epsfig{file=sigexa1.eps,width=.95\linewidth}}\end{center}
\end{minipage}

\caption{Using numerical integration with SIGMA}
\label{fig:SIGEXA1}
\end{figure}

\PAWfdef{SUMV}{R=SUMV(arg)}

The \texttt{SUMV} function generates the {\bf running summation}
of each row of the argument
array, say
\(X_1,\:X_2,\:\ldots,\)
\(X_i,\:\ldots,\:X_n\)
and creates an array with components equal to the running sum of the
\( X_i \) namely:
\( X_1,\: X_1 + X_2 ,\)
\(\ldots, \: X_1 + X_2 + \ldots X_i, \:
                             \ldots, \: X_1 + X_2 + \ldots X_n \).

\subsection*{Using the \texttt{SUM} function}
\begin{alltt}
  SIGMA > \Ucom{ x=array(6&4,array(24,1#24))}
  NCO(X       )=    6    4
  X       =
    1.000      2.000      3.000      4.000      5.000      6.000
    7.000      8.000      9.000      10.00      11.00      12.00
    13.00      14.00      15.00      16.00      17.00      18.00
    19.00      20.00      21.00      22.00      23.00      24.00

  SIGMA > \Ucom{y=sumv(x)}
  NCO(Y       )=    6    4
  Y       =
    1.000      3.000      6.000      10.00      15.00      21.00
    7.000      15.00      24.00      34.00      45.00      57.00
    13.00      27.00      42.00      58.00      75.00      93.00
    19.00      39.00      60.00      82.00      105.0      129.0
\end{alltt}

\PAWfdefii{VMAX}{R=VMAX(arg)}{VMIN}{R=VMIN(arg)}

The functions \texttt{VMAX} and \texttt{VMIN}
return a scalar equal to the largest or smallest element of the array \texttt{arg}.

\PAWfdef{VSUM}{R=VSUM(arg1)}

The \texttt{VSUM} function generates the {\bf  sum}
of each element of the argument
array, say
\( X_1,\:X_2,\:\ldots,\:X_i,\:\ldots,\:X_n \)
and creates a scalar whose value is equal to
the sum of all the components of \( X \) namely:
\( X_1 + X_2 + X_3,\:\ldots,\:X_n \)

\subsection*{Using the \texttt{VSUM} function}
\begin{alltt}
 SIGMA >\Ucom{ x=array(10)}
 NCO(X       )=   10
 X       =
   1.00      1.00      1.00      1.00      1.00      1.00      1.00
   1.00      1.00      1.00

 SIGMA >\Ucom{ r=vsum(x)}
 NCO(R       )=    1
 R         10.0
\end{alltt}

\section{Available library functions}
\index{library functions in SIGMA}
\index{SIGMA!library functions}

The library functions available under SIGMA are
listed below. All these functions have a single
argument, unless otherwise indicated.
The number indicated between parentheses corresponds to the
number of the same function in the CERN program library.

{\small
\begin{DLttc}{1234567}
\item[ABS]     ABSolute value
\item[ACOS]    ArCOSine
\item[ALOGAM]  LOGarithm of the GAMma Function (C341)
\item[ASIN]    ArcSINe
\item[ATAN]    ArcTANgent
\item[ATAN2]   ArcTANgent2 (2 arguments)
\item[BESI0]   Mod. Bessel Function I0 (C313)
\item[BESI1]   Mod. Bessel Function I1 (C313)
\item[BESJ0]   Bessel Function J0 (C312)
\item[BESJ1]   Bessel Function J1 (C312)
\item[BESK0]   Mod. Bessel Function K0 (C313)
\item[BESK1]   Mod. Bessel Function K1 (C313)
\item[BESY0]   Bessel Function Y0 (C312)
\item[BESY1]   Bessel Function Y1 (C312)
\item[COS]     COSine
\item[COSH]    Hyperbolic COSine
\item[COSINT]  COSine INTegral (C336)
\item[DILOG]   DILOGarithm Function (C304)
\item[EBESI0]  \(\exp(-\left|x\right|)I_0(x)\) (C313)
\item[EBESI1]  \(\exp(-\left|x\right|)I_1(x)\) (C313)
\item[EBESK0]  \(\exp(x)K_0(x)\) (C313)
\item[EBESK1]  \(\exp(x)K_1(x)\) (C313)
\item[ELLICK]  Complete Elliptic Integral K (C308)
\item[ELLICE]  Complete Elliptic Integral E (C308)
\item[ERF]     Error Function ERF (C300)
\item[ERFC]    Error Function ERFC (C300)
\item[EXP]     EXPonential
\item[EXPINT]  EXPonential INTegral (C337)
\item[FREQ]    Normal Frequency Function FREQ (C300)
\item[GAMMA]   GAMMA Function (C305)
\item[INT]     Takes INTegral part of decimal number
\item[LOG]     Natural LOGarithm
\item[LOG10]   Common LOGarithm
\item[MOD]     Remaindering
\item[RNDM]    Random Number Generator: \texttt{V1=RNDM(V)}, with
               \texttt{NCO(V1)=NCO(V)} generates random numbers
               between \texttt{0} and \texttt{1}.
\item[SIGN]    Transfer of SIGN: \texttt{V2=SIGN(V,V1)},
               \texttt{V2=|V|*V1/|V1|}
\item[SIN]     SINe Function
\item[SINH]    Hyperbolic SINe
\item[SININT]  SINe INTegral (C336)
\item[SQRT]    SQuare RooT
\item[TAN]     TANgent
\item[TANH]    Hyperbolic Tangent
\end{DLttc}
}%%% end of \small
\fbox{Ill defined functions will return \texttt{0.} as result.
(e.g. \texttt{SQRT} of a negative number is taken as \texttt{0}).}
\index{SIGMA|)}


\documentclass[letterpaper]{article}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{commath}
\usepackage{enumerate}
\usepackage{lmodern}
\usepackage{microtype}
\usepackage{fullpage}
\usepackage{graphicx}

\title{STAT 527: Assignment \#1}
\author{Philip Pham}
\date{\today}

\DeclareMathOperator{\E}{E}
\DeclareMathOperator{\Prob}{P}

\begin{document}
\maketitle

\section*{Problem 1}

\begin{enumerate}[(a)]
\item
  \begin{proof}
    This follows from the definition of variance after some algebra.
    \begin{align*}
      &\E\left[
      \left(\hat{f}(x) - f(x)\right)^2
        \right]
        = \E\left[
        \left[\left(\hat{f}(x) - \E\left[\hat{f}(x)\right]\right)
        +
        \left(\E\left[\hat{f}(x)\right] - f(x)\right)
        \right]^2
        \right] \\
      &=
        \E\left[\left(\hat{f}(x) - \E\left[\hat{f}(x)\right]\right)^2\right]
        +
        2\E\left[
        \left(\hat{f}(x) - \E\left[\hat{f}(x)\right]\right)
        \left(\E\left[\hat{f}(x)\right] - f(x)\right)
        \right]
        +
        \E\left[\left(\E\left[\hat{f}(x)\right] - f(x)\right)^2\right]
      \\
      &= \operatorname{var}\left(\hat{f}(x)\right)
	+ \left(\E\left[\hat{f}(x)\right] - f(x)\right)^2
        + 2\left(\E\left[\hat{f}(x)\right] - f(x)\right)\E\left[
        \left(\hat{f}(x) - \E\left[\hat{f}(x)\right]\right)
        \right] \\
      &= \left(\E\left[\hat{f}(x)\right] - f(x)\right)^2 +
        \operatorname{var}\left(\hat{f}(x)\right).
    \end{align*}
  \end{proof}
\item
  \begin{proof}
    The prediction error is
    \begin{align*}
      &\E\left[
      \left(y_{\text{new}} - \hat{f}\left(x_{\text{new}}\right)\right)^2
      \right]
      = \E\left[
        \left(f\left(x_{\text{new}}\right) + \epsilon_\text{new} -
        \hat{f}\left(x_{\text{new}}\right)\right)^2
        \right] \\
      &= \E\left[
        \left(\epsilon_\text{new} +
        \left(f\left(x_{\text{new}}\right) -
        \hat{f}\left(x_{\text{new}}\right)\right)\right)^2
        \right] \\
      &= \E\left[\epsilon_\text{new}^2\right]
        + 2\E\left[\epsilon_\text{new}\left(f\left(x_{\text{new}}\right) -
        \hat{f}\left(x_{\text{new}}\right)\right)\right]
        + \left(\E\left[\hat{f}(x_\text{new})\right] -
        f(x_\text{new})\right)^2 +
        \operatorname{var}\left(\hat{f}\left(x_\text{new}\right)\right) \\
      &= \sigma^2 +
        \left(\E\left[\hat{f}(x_\text{new})\right] -
        f(x_\text{new})\right)^2 +
        \operatorname{var}\left(\hat{f}\left(x_\text{new}\right)\right).
    \end{align*}

    There is an additional $\sigma^2$ term to account for the variance of our
    new observation.
  \end{proof}
\item
  \begin{proof}
    \begin{align*}
      a \geq \E\left[L\right]
      &= \int_0^\infty t \,\dif L(t) \\
      &= \int_0^{a/\epsilon} t \,\dif L(t) + \int_{a/\epsilon}^\infty t \,\dif L(t) \\
      &\geq \int_0^{a/\epsilon} t \,\dif L(t) + \frac{a}{\epsilon}\int_{a/\epsilon}^\infty \,\dif L(t) \\
      &\geq \frac{a}{\epsilon}\int_{a/\epsilon}^\infty \,\dif L(t) = \frac{a}{\epsilon}\Prob\left(L > \frac{a}{\epsilon}\right).
    \end{align*}

    Thus, we have that
    \begin{equation*}
      \frac{a}{\epsilon}\Prob\left(L > \frac{a}{\epsilon}\right) \leq a
      \Leftrightarrow
      \Prob\left(L > \frac{a}{\epsilon}\right) \leq \epsilon
      \Leftrightarrow
      \Prob\left(\frac{L}{a} > \frac{1}{\epsilon}\right) \leq \epsilon.
    \end{equation*}
  \end{proof}
\item
  \begin{proof}
    Note that $y = X\beta + \epsilon$, where
    $\epsilon \sim \operatorname{N}\left(0, \sigma^2I\right)$, so
    \begin{align*}
      \hat{\beta}
      &= \left(X^\top X\right)^{-1}X^\top y_i \\
      &= \left(X^\top X\right)^{-1}X^\top \left(X\beta + \epsilon\right) \\
      &= \beta + \left(X^\top X\right)^{-1}X^\top\epsilon \\
      \hat{\beta} - \beta &= \left(X^\top X\right)^{-1}X^\top\epsilon.
    \end{align*}

    Thus, we have that
    \begin{align*}
      \E\left[\hat{\beta} - \beta\right]
      &= \E\left[\left(X^\top X\right)^{-1}X^\top\epsilon\right]
        = \E\left[\left(X^\top X\right)^{-1}X^\top\right]
        \E\left[\epsilon\right] = 0 \\
      \operatorname{var}\left(\hat{\beta} - \beta\right)
      &=
        \E\left[\left(X^\top X\right)^{-1}X^\top\epsilon\epsilon^\top X
        \left(X^\top X\right)^{-1}\right]
        = \sigma^2\E\left[\left(X^\top X\right)^{-1}\right].
    \end{align*}

    If the design was fixed ($\left\{x_i\right\} \subset \mathbb{R}^p$ were
    deterministic), then we simply have
    $\operatorname{var}\left(\hat{\beta} - \beta\right) = \sigma^2\left(X^\top
      X\right)^{-1}$, so
    \begin{equation*}
      \hat{\beta} - \beta \sim \operatorname{N}\left(0, \sigma^2\left(X^\top X\right)^{-1}\right)
    \end{equation*}

    With a random design, we appeal to Slutsky's theorem.
    $X^\top X = \sum_{i=1}^n x_i^\top x_i$, so by strong law of large numbers
    $\left(X^\top X\right)/n \xrightarrow{\text{a.s.}} \Sigma$, where $\Sigma$
    is a constant.

    $X^\top\epsilon = \sum_{i=1}^n x_i\epsilon_i$, where the $x_i\epsilon_i$ are
    i.i.d. and $\operatorname{var}\left(x_i\epsilon_i\right) = \sigma^2\Sigma$.

    By CLT, we have that
    \begin{equation*}
      \frac{X^\top\epsilon}{\sqrt{n}}
        \xrightarrow{d} \operatorname{N}\left(0, \sigma^2\Sigma\right)
    \end{equation*}

    So, we can now apply Slutsky's theorem to show that
    \begin{align*}
      \sqrt{n}\left(\hat{\beta} - \beta\right)
      =
      \left(\frac{X^\top X}{n}\right)^{-1}\frac{X^\top\epsilon}{\sqrt{n}}
      \xrightarrow{d} \Sigma^{-1}\operatorname{N}\left(0, \sigma^2\Sigma\right)
      &= \operatorname{N}\left(0, \sigma^2\Sigma^{-1}\Sigma\Sigma^{-1}\right) \\
      &= \operatorname{N}\left(0, \sigma^2\Sigma^{-1}\right)
    \end{align*}
    as desired.
  \end{proof}
\end{enumerate}

\section*{Problem 2}

\begin{enumerate}[(a)]
\item $f(x) = 2x$
  \begin{figure}[h]
    \centering
    \includegraphics{mse_linear.pdf}
  \end{figure}
\item $f(x) = \sin(\pi x)$
    \begin{figure}[h]
      \centering
      \includegraphics{mse_sin.pdf}
    \end{figure}
    \pagebreak
  \item $f(x) = 2x + x^3 - 6x^4$
    \begin{figure}[h!]
      \centering
      \includegraphics{mse_polynomial.pdf}
    \end{figure}

  \item $f(x) = 1/\left(1 + (5x)^2\right)$
    \begin{figure}[h!]
      \centering
      \includegraphics{mse_inverse.pdf}
    \end{figure}
  \end{enumerate}

  $n = 1000$ and 100 simulations were run for each case. Almost all the
  estimators achieve the optimal MSE of 1 when $f$ is linear.

  When $f$ is sinusoidal, the linear and degree 2 polynomial can no longer model
  the data.

  When $f$ is a higher-degree polynomial, the linear estimator does
  terribly. Unsuprisingly, the degree 4 and degree 5 polynomials do the best.

  When $f$ is an inverse polynomial function, increasing the degree of the
  polynomial estimator helps, but only the non-parametric models are able to
  achieve the theoretical best error of 1.
\end{document}


\subsection{BinaryFunction Class}
\label{sec:binaryFunction}

A \code{BinaryFunction} object represents a function that can combine two \code{Expression}s
and produce another \code{ValueComputation}.

For the purposes of representing a single operand of an instruction, the
\code{BinaryFunction}s of interest are addition and multiplication of integer values;
this allows an \code{Expression} to represent all addressing modes on the architectures
currently supported by the Instruction API.

\begin{apient}
BinaryFunction(Expression::Ptr arg1,
               Expression::Ptr arg2,
               Result_Type result_type,
               funcT:Ptr func)
\end{apient}
\apidesc{
The constructor for a \code{BinaryFunction} may take a reference-\/counted pointer or a plain C++ pointer to each of the child \code{Expression}s that represent its arguments. Since the reference-\/counted implementation requires explicit construction, we provide overloads for all four combinations of plain and reference-\/counted pointers. Note that regardless of which constructor is used, the pointers \code{arg1} and \code{arg2} become owned by the \code{BinaryFunction} being constructed, and should not be deleted. They will be cleaned up when the \code{BinaryFunction} object is destroyed.

The \code{func} parameter is a binary functor on two \code{Result}s. It should be derived from \code{funcT}. \code{addResult} and \code{multResult}, which respectively add and multiply two \code{Result}s, are provided as part of the InstructionAPI, as they are necessary for representing address calculations. Other \code{funcTs} may be implemented by the user if desired. \code{funcTs} have names associated with them for output and debugging purposes. The addition and multiplication functors provided with the Instruction API are named "+" and "*", respectively.
}

\begin{apient}
const Result & eval () const
\end{apient}
\apidesc{
The \code{BinaryFunction} version of \code{eval} allows the \code{eval} mechanism to handle complex addressing
modes. Like all of the \code{ValueComputation} implementations, a \code{BinaryFunction}'s \code{eval} will return the
result of evaluating the expression it represents if possible, or an empty \code{Result} otherwise. A \code{BinaryFunction} may have arguments that can be evaluated, or arguments that cannot. Additionally, it
may have a real function pointer, or it may have a null function pointer. If the arguments can be
evaluated and the function pointer is real, a result other than an empty \code{Result} is guaranteed to
be returned. This result is cached after its initial calculation; the caching mechanism also allows
outside information to override the results of the \code{BinaryFunction}'s internal computation. If the
cached result exists, it is guaranteed to be returned even if the arguments or the function are not
evaluable.
}

\begin{apient}
void getChildren (vector< InstructionAST::Ptr > & children) const
\end{apient}
\apidesc{
The children of a \code{BinaryFunction} are its two arguments.
Appends the children of this BinaryFunction to \code{children}.
}

\begin{apient}
void getUses (set< InstructionAST::Ptr > & uses)
\end{apient}
\apidesc{
The use set of a \code{BinaryFunction} is the union of the use sets of its children.
Appends the use set of this \code{BinaryFunction} to \code{uses}.
}

\begin{apient}
bool isUsed (InstructionAST::Ptr findMe) const
\end{apient}
\apidesc{
\code{isUsed} returns \code{true} if \code{findMe} is an argument of this \code{BinaryFunction}, or if it is in the use set of either argument.
}


\section{Derivatives of the solutions of IVP with respect to data}
\subsection{Derivatives with respect to the initial values}
\begin{intro}
  In order to understand BVP, we have to introduce the notion of the
  derivative of the solution to an IVP with respect to its initial
  values. Our interest lies in the change of $u$ if the initial value
  is changed. We will thus denote in these cases $u=u(t;v)$, where $t$ is
  the usual ``time'' variable and $v$ the initial value, which now is
  a variable as well. Thus, $u(t;v)$ is the solution to the IVP
  \begin{gather}
    \label{eq:derivatives:1}
    \begin{split}
      u'(t;v) = \tfrac{\partial}{\partial t} u(t;v) &= f\bigl(t,u(t;v)\bigr) \\
      u(t_0;v) &= v.
    \end{split}
  \end{gather}
  The purpose of this section is the study of the derivative
  \begin{gather*}
    \tfrac{\partial}{\partial v} u(t;v),
  \end{gather*}
  which is fundamentally different from $\nicefrac{\partial}{\partial t}
  u(t;v)$.
  It can be obtained by solving the variational equation of the
  original IVP, defined as follows.
\end{intro}

\begin{Definition}{gateaux-derivative}
  Let $F(v)$ be a function defined on some function space. Then, the Gâteaux
  derivative of $F$ at a point $v$ in direction $w$ with $v$ and $w$
  in this function space is defined as
  \begin{gather}
    \label{eq:derivatives:6}
    \frac{\partial}{\partial w} F(v)
    = \lim_{\epsilon\to 0} \frac{F(v+\epsilon w) - F(v)}{\epsilon},
  \end{gather}
  if this limit exists.

  Here, we have used the notation for directional derivatives in
  $\R^n$, since this is indeed the character of the Gâteaux
  derivative.
  % Let $f$ be a function depending on another function $v$. Then, the
  % \define{Gateaux derivative} of $f$ at $v$ in direction $w$ is
  % defined as

\end{Definition}
% \begin{remark}
%   In terms of linear \putindex{perturbation theory} this section deals
%   with the derivative of the solution $u(t)$ of the
%   IVP~\eqref{eq:IVP}, but not with respect to the argument $t$, but
%   rather with respect to the initial values. That is why we will write
%   the solution $u = u(t;s)$ of the IVP
%   \begin{gather}
%     \label{eq:derivatives:1}
%     \begin{split}
%       u'(t;s) = \tfrac{\partial}{\partial t} u(t;s) &= f\bigl(t,u(t;s)\bigr) \\
%       u(t_0;s) &= s_0
%     \end{split}
%   \end{gather}
%   with two arguments, the actual time variable $t$ and the initial value $s$.
% \end{remark}

\begin{Definition}{variational-equation}
  The \define{variational equation} to the first order system of
  ODE
  \begin{gather*}
    u' = f,
  \end{gather*}
  of dimension $d$ is the linear matrix-valued system of ODE
  \begin{subequations}
  %  \label{eq:derivatives:1}
  \begin{gather}
    \label{eq:derivatives:2}
      \fundam' = \nabla_u f\bigl(t,u(t)\bigr) \fundam
  \end{gather}
  for $d\times d$ matrices $\fundam$. Here is $u$ a solution of the
  equation~\eqref{eq:IVP:ode} and
  \begin{gather*}
    \nabla_u f(t,u) =
    \begin{pmatrix}
      \tfrac{\partial f_1}{\partial u_1} & \cdots
      & \tfrac{\partial f_1}{\partial u_d} \\
      \vdots & & \vdots \\
      \tfrac{\partial f_d}{\partial u_1} & \cdots
      &  \tfrac{\partial f_d}{\partial u_d}
    \end{pmatrix}
  \end{gather*}
  is the matrix of the derivatives of $f$ with respect to the
  components of $u$. The \define{fundamental matrix}
  $\fundamental(t;t_0)$\index{Y@$\fundamental(t;t_0)$} is solution of
  the IVP to this equation with
  \begin{gather}
    \label{eq:derivatives:3}
    \fundam(t_0) = \identity.
  \end{gather}
  \end{subequations}
\end{Definition}

\begin{remark}
  The fundamental matrix $\fundam$ can also be read column
  by column.  Then each column is a vector-valued function
  $\phi^{(i)}(t)$ and solves the IVP
  \begin{gather*}
    \begin{split}
      \tfrac{d}{dt}\phi^{(i)}(t) &= \nabla_u
      f\bigl(t,u(t)\bigr)\phi^{(i)}(t),
      \\
      \phi^{(i)}(t_0) &= e_i.
    \end{split}
  \end{gather*}
\end{remark}

\begin{remark}
  The definition of the fundamental matrix here is consistent with the
  one in Definition~\ref{Definition:fundamental-system} for linear
  equations. Namely, for $f(u) = Au$, we have $\nabla_u f(u) = A$.
\end{remark}

\input{theorems/fundamental}

\begin{proof}
  In order to prove the first equation, denote by $V(\tau;s)$ the
  solution to the IVP
  \begin{gather*}
    V'(\tau;s) = \nabla_u f(\tau,u(\tau)) V(\tau;s),
    \qquad V(s;s) = \fundamental(s;t).
  \end{gather*}
  Because of uniqueness, we must have
  $V(\tau;s) = \fundamental(\tau;t)$ for any $\tau$ between $s$ and
  $t$, in particular for $r=t$, such that $V(t;s) = \fundamental(t;t) = \identity$. On the
  other hand, by linearity, we have
  $V(\tau;s) = \fundamental(\tau;s)\fundamental(s;t)$, and thus the
  equation is proven by
  \begin{gather*}
    \identity = V(t;s) = \fundamental(t;s)\fundamental(s;t).
  \end{gather*}

  Now, assume without loss of generality that $s$ is between $r$ and
  $t$. Indeed, if for instance $t$ is between $r$ and $s$, multiply
  equation~\eqref{eq:derivatives:4} from the left by $\fundamental(s;t)$ and
  prove the equation for
  \begin{gather*}
    \fundamental(s;t)\fundamental(t;r)
    = \fundamental(s;t)\fundamental(t;s)U(s;r)
    = \fundamental(s;r).
  \end{gather*}
  Take the auxiliary function $V(\tau;s)$ as defined above. By
  uniqueness, it is equal to $\fundamental(\tau;t)$ for all $\tau$. But, on the
  other hand, we have by linearity $V(\tau;s) = \fundamental(\tau;s)\fundamental(s;r)$, in
  particular for $\tau=t$.

  The statement follows from the definition as a solution of an IVP
  and the fact that solutions of linear IVP are linear combinable.
\end{proof}

\input{theorems/derivative}

\begin{proof}
  We write the IVP in its full dependence on $v$ as
  \begin{align*}
    \frac{\partial u(t;v)}{\partial t} &= f\bigl(t,u(t;v)\bigr) \\
    u(t_0;v) &= v.
  \end{align*}
  From the second equation, we immediately obtain
  \begin{gather*}
    \frac{\partial u(t_0;v)}{\partial v} = \identity.
  \end{gather*}
Assuming differentiability of $f$ with respect to $u$, the first
equation yields
\begin{gather*}
  \frac{\partial}{\partial v}\frac{\partial u(t;v)}{\partial t}
  = \frac{\partial f\bigl(t,u(t;v)\bigr)}{\partial v}
  = \nabla_u f(t,u(t;v)) \frac{\partial u(t;v)}{\partial v}.
\end{gather*}
Thus, $\tfrac{\partial}{\partial v} u$ solves the
IVP~\eqref{eq:derivatives:1}
\begin{gather*}
  \left( \frac{\partial}{\partial v} u \right)'
  = \nabla_u f(t,u(t;v)) \frac{\partial u(t;v)}{\partial v}.
\end{gather*}
$\tfrac{\partial}{\partial v} u$ solves the same IVP as the
fundamental matrix and thus, they coincide.
\end{proof}

\subsection{Derivatives with respect to the right hand side function}
\begin{intro}
  We close this section by studying the differential dependence of the
  solution $u(t)$ of an ODE at time $t$ on the function $f(t,u)$, that
  is, the derivative of a value with respect to a function. In order
  to keep things simple, we reduce this question to a regular
  derivative of a function with respect to a real variable by using
  the Gâteaux derivative.
  Back to differential equations, our task is now to compute the
  derivative of $u(t)$ with respect to changes in $f$, denoted as
  \begin{gather}
    \label{eq:derivatives:7}
    \frac{\partial}{\partial g} u(t)
    = \lim_{\epsilon\to 0} \frac{u_\epsilon-u}{\epsilon}
    = \left.\frac{d}{d\epsilon} u_\epsilon(t)\right|_{\epsilon=0},
  \end{gather}
  where $u$ and $u_\epsilon$ respectively solve the IVPs
  \begin{xalignat*}{2}
    u'&=f(t,u) & u(t_0)& = u_0 \\
    u_\epsilon'&=f(t,u_\epsilon)+\epsilon g(t,u_\epsilon)
    & u_\epsilon(t_0)& = u_0.
  \end{xalignat*}
  For this derivative, we have the following theorem.
\end{intro}

\input{theorems/derivative-f.tex}

\begin{proof}
   We set out by devising a differential equation for the Gâteaux
  derivative $\mathcal U(t) := \left.\frac{d}{d\epsilon} u_\epsilon(t)\right|_{\epsilon=0}$. The differential equation for $u$ yields
  \begin{align*}
    \mathcal U'(t) =& \left( \left.\frac{d}{d\epsilon} u_\epsilon(t)\right|_{\epsilon=0} \right)' \\
    =& \left.\frac{d}{d\epsilon} u_\epsilon ' (t)\right|_{\epsilon=0} \\
    =& \left.\frac{d}{d\epsilon}
    \Bigl(f\bigl(t,u_\epsilon(t)\bigr)
    + \epsilon g\bigl(t,u_\epsilon(t)\bigr)\Bigr) \right|_{\epsilon=0} \\
    =& \left.\nabla_u f(t,u_\epsilon) \frac{d}{d\epsilon} u_\epsilon(t)
     + \epsilon \nabla_u g \bigl(u_\epsilon (t) \bigr) \frac{d}{d\epsilon} u_\epsilon(t)
    + g\bigl(t,u_\epsilon(t)\bigr) \right|_{\epsilon=0}
  \\
    =& \left.\nabla_u f(t,u_\epsilon) \mathcal U(t)
    + g\bigl(t,u_\epsilon(t)\bigr) \right|_{\epsilon=0}
  \end{align*}
  Furthermore, we have
  \begin{gather*}
     \mathcal U(t_0) = \frac{d}{d\epsilon} u_\epsilon(t_0) = 0.
  \end{gather*}

  According to Lemma~\ref{Lemma:linear-representation}, the solution
  of this initial value problem can be represented with the
  integrating factor $M(t)$ as
  \begin{gather*}
    \mathcal U(t) = M^{-1}(t) \int_{t_0}^t M(s) g\bigl(s,u(s)\bigr)
    \, \ds.
  \end{gather*}
  Noticing that $M(\tau)^{-1} = \fundamental(\tau;t_0)$, we obtain
  \begin{gather*}
    u(t) = \int_{t_0}^t \fundamental(t;s) g\bigl(s,u(s)\bigr) \, \ds
\end{gather*}
\end{proof}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "notes"
%%% End:


\section{Methodology}
\subsection{Strategy}
It's easy to notice that the nested $i$ and $j$ for-loops in \textbf{Algorithm \ref*{alg:fw1}} can be parallizable, but one of the \emph{intermediate edges} for a process
could be the \emph{edge under analysis} for another process. For example process $P^1$ is writing in a cell while process $P^2$ is reading from the same cell, leading to a unpredictable final result
if $P^1$'s writing comes before or after $P^2$'s reading.

It seems that enabling parallelization requires some sort of blocking mechanism between processes, like a semaphore, but we prove that no data race can occur as in the previous example as far as the value of $k$
is shared between processes.

Let's assume we have 2 processes, namely $P^1$ and $P^2$, and they have \emph{under analisys} $M_{i,j}$ and $M_{x,y}$ respectively, with $x \neq i$ so that each process partitions the matrix horizontally.
At any point the following system of preconditions exists (the two inequalities are taken from \textbf{Algorithm \ref*{alg:fw1}} at line 10):

\begin{flalign}\label{eq:sys1}
 &&  \left\{\begin{matrix}
P^{1}_{i,j} & > & P^{1}_{i,k} & + & P^{1}_{k,j} \\
\\
P^{2}_{x,y} & > & P^{2}_{x,k} & + & P^{2}_{k,y}
\end{matrix}\right. &&
\end{flalign}

\emph{i.e.} $P^1$ is reading values of $M_{i,k}$ and $M_{k,j}$ so it can decide if $M_{i,j}$ should be overwritten;
the same does $P^2$ with $M_{x,k}$, $M_{k,y}$ and $M_{x,y}$ respectively. \\
In order to have a data race, the following statement must be true as well

\[(x = i \wedge k=j) \vee (k = i \wedge y = j)\]

\emph{i.e.} $P^2$ is using as one of its \emph{intermediate edges} the \emph{edge under analysis} of $P^1$. Without losing
of generality we do not analyze the opposite case because the proof develops symmetrically.\\
Because $x \neq i$, only the following must be verified

\begin{flalign}\label{eq:cond1}
 &&  k = i \wedge y = j &&
\end{flalign}
and by applying (\ref*{eq:cond1}) to (\ref*{eq:sys1}) we have

\begin{flalign}\label{eq:sys2}
 &&  P^{1}_{i,j} > P^{1}_{k,k} + P^{1}_{i,j} &&
\end{flalign}
but (\ref*{eq:sys2}) is clearly false, because $M$ is a hollow matrix \emph{i.e.} the  diagonal elements are all equal to $0$, leaving the
following inequality:
\begin{flalign}\label{eq:sys3}
 &&  P^{1}_{i,j} > P^{1}_{i,j} &&
\end{flalign}
Clearly no number can be greater than itself and this means that at this point the comparison made by $P^1$ is irrilevant
and is always evaluated to false. If $P^2$ writes after or before $P^1$'s evaluation does not really matter and thus
there is no data race as far as $k$ is the same for all the running processes.

This is really important because it assures that there is no need for locking mechanism on $M$ when it comes to
parallelize the nested $i$ and $j$ for-loops and having no blocks guarantees better performance.

Now the strategy relies on dividing the matrix by rows and assigning an equal number of rows to each process.
Once $k$ is set and it's shared among all processes, each process puts \emph{under analysis} every cell
of its sub-matrix and selects its \emph{intermediate edges} depending on $k$.

In this work we propose 3 parallel architectures implemented with 3 different parallel programming environments:
\emph{Distributed computing} with MPI, \emph{Shared-memory Mutithreading} with OpenMP and \emph{GPGPU} with CUDA.

The document analyzes each implementation and gives an overview of timings, implementations and pros and cons for each case.

\subsection{Distributed with MPI}

Message Passing Interface (MPI) is a standardized and portable message-passing standard designed by a group of researchers from academia and industry to function on a wide variety of parallel computing architectures. The standard defines the syntax and semantics of a core of library routines useful to a wide range of users writing portable message-passing programs in C, C++, and Fortran. There are several well-tested and efficient implementations of MPI, many of which are open-source or in the public domain. These fostered the development of a parallel software industry, and encouraged development of portable and scalable large-scale parallel applications. \par

The main strategy is based on scattering horizontally the whole matrix among all the process, so that
each process can read a portion of the matrix of size $\frac{n^2}{p}$; then a \emph{process of competence}
is chosen: as $k$ is in common (but never transmitted) to all the processes, there's always a cell in the $k^{th}$
row representing one of the two intermediate vertices for any process and there's always one process which this row was assigned to.
The value $k$ is always "up to date" among processes because each for-loop involving $k$ starts with a collective communication
which implies a synchronization point among processes.

\begin{figure}[h!]
\centering
\includegraphics[width=3.5in]{diagrams/mpi-scatter}
\captionsetup{justification=centering}
\caption{View of the data each thread can reach}
\label{fig:threads}
\end{figure}
Everytime $k$ changes, the $k^{th}$ row is broadcasted by the \emph{process of competence} (the process that "owns" the row) to the other processes;
a total of $k$ \texttt{MPI\_Bcast} operations are required. \par
Once the $k^{th}$ row has been received, each process acts like the original FW in the $i$ and $j$ for-loops; obviously
they each write values in their own local matrix. \par

At the end of the $k$ for-loop, all the local matrices are gathered to the root process.
\textbf{Algorithm \ref*{alg:mpi}} shows an high level pseudo-code of the strategy exposed above. For the concrete implementation see \textbf{section \ref{mpiimpl}}.\par

\begin{algorithm}[h!]

\SetAlgoLined

broadcast($n$, $ROOT$) \\
scatter($M$, $p$)

\For{$k = 1 \rightarrow n$}{
	\If{$rank =$ \textnormal{findOwnerOfK$^{th}$Row}($k$, $rank$)}{
		$K \leftarrow $ getRow($M$, $k$) \\
	}
	broadcast($K$) \\

	\For{$i = 1 \rightarrow \frac{n}{p}$}{
		\For{$j = 1 \rightarrow n$}{
			\If{$M_{i,j} < M_{i,k} + K_j$}{
				$M_{i,j} \leftarrow M_{i,k} + K_j$
			}
		}
	}
}
gather($M$, $p$);

\caption{Distributed version of FW}\label{alg:mpi}
\end{algorithm}

We can list all the communication required:
\begin{itemize}
\item{1 \texttt{MPI\_Bcast} for communicating the value of $n$}
\item{1 \texttt{MPI\_Scatter} for the assignment of the local sub-matrix}
\item{$n$ \texttt{MPI\_Bcast} for communicating the $k^{th}$ row}
\item{1 \texttt{MPI\_Gather} for the collection of the local sub-matrix}
\end{itemize}

\textbf{Table \ref*{tab:comm}} approximates how many bytes are involved in the communications, assuming that
the implementation uses 4 bytes to store one \texttt{int} and omitting the overhead of the communication protocol.
\begin{table}[h!]
\centering
\begin{tabular}{|l|l|l|}
\hline
\rowcolor[HTML]{F56B00}
{\color[HTML]{FFFFFF} \textbf{Count}} & {\color[HTML]{FFFFFF} \textbf{Type}} & {\color[HTML]{FFFFFF} \textbf{Size}} \\ \hline
1                                     &  \texttt{MPI\_Bcast}                 &  $4(p-1)$ bytes                      \\ \hline
1                                     &  \texttt{MPI\_Scatter}               &  $4\frac{n^2(p-1)}{p}$ bytes         \\ \hline
$n$                               &  \texttt{MPI\_Bcast}                 &  $4n^2(p-1)$ bytes                    \\ \hline
1                                     &  \texttt{MPI\_Gather}                &  $4\frac{n^2(p-1)}{p}$ bytes         \\ \hline
\end{tabular}
\caption{Approximation of the size of each communication for \emph{MPI} FW}
\label{tab:comm}
\end{table}
\par

Total communications can be expressed with the following formula, that can help to calculate the bandwidth of the network required
for this implementation:
\[W_{comm} = 4(p-1)(1 + n^2(1 + \frac{2}{p})) \text{ bytes}\]
This formula is important because the time spent on communications is time taken from the calculation and a network with an inadequate
bandwith is the main source of bottlenecks. For example having 8 processes consuming a $12500 \times 12500$ matrix implies a total of
$\textasciitilde 5.46$GB transferred  over the entire network; this means that each processor would theorically spend $5468$ms (43 seconds in total) on communication over a 1Gbps network.

\begin{figure}[h!]
\centering
\includegraphics[width=3.5in]{diagrams/mpi-time}
\captionsetup{justification=centering}
\caption{Calculation and MPI instantiation + communication time of \emph{MPI} FW consuming a $5040\times5040$ matrix over a 1Gbps network depending on the number of processes}
\label{fig:mpi-time}
\end{figure}

\textbf{Figure \ref*{fig:mpi-time}} shows the amount of time taken to initialize the MPI cluster and communicate during the execution of the \emph{MPI} FW. In this case a dense
$5040\times5040$ matrix is used as input and the MPI cluster was located in a completely-connected network with 1Gbps bandwith.

The bandwidth required has a quadratic and linear growth with respect to the number of vertices and the number of processes respectively;
so with a medium-small matrix the time used in communication can exceed the 10\% of the total computational time.
But by increasing the size of the matrix to $12600\times12600$ the time used in communication drops to 4\%, with a total of 50 seconds. Thus the implementation fits well for huge matrixes, rather than medium ones.

We can also easily calculate the time required by the algorithm with this formula:
\[T = t_c \frac{n^3}{p} + log(p) \cdot(t_s(3+n) + t_w(1 + 2 \frac{n^2}{p} + n))\]
where $t_c$ is the computation time, $t_s$ the time required to start a communication and $t_w$ the time to communicate a single word (in this case a \texttt{int}); the formula
assumes that the collective communications use a tree-based hierarchical pattern that takes $log(p)$ steps rather than $p$ steps for $p$ processes.

For this reason the speedup is considerable but far from the ideal for a $5040\times5040$ matrix. \textbf{Figure \ref*{fig:mpi-speedup}} shows the trend of the speedup in relation to the
sequential version.

\begin{figure}[h!]
\centering
\includegraphics[width=3.5in]{diagrams/mpi-speedup}
\captionsetup{justification=centering}
\caption{Speedup of \emph{MPI} FW}
\label{fig:mpi-speedup}
\end{figure}

With a consequent decrease in efficiency as shown in \textbf{Figure \ref*{fig:mpi-efficiency}}
\begin{figure}[h!]
\centering
\includegraphics[width=3.5in]{diagrams/mpi-efficiency}
\captionsetup{justification=centering}
\caption{Efficiency of \emph{MPI} FW consuming a medium matrix}
\label{fig:mpi-efficiency}
\end{figure}

\subsection{Shared-memory multiprocessing with OpenMP}
OpenMP (Open Multi-Processing) is an application programming
interface (API) for parallel programming intended to work on shared-memory architectures. More specifically, it is a set of compiler
directives, library routines and environmental variables, which influence runtime behavior. OpenMP enables parallel programming in
various languages, such as C, C++ and FORTRAN and runs on mostoperating systems. \par
The OpenMP API uses the fork-join model of parallel execution.
Multiple threads perform tasks defined implicitly or explicitly by OpenMP directives. All OpenMP applications begin as a single thread
of execution, called the initial thread. The initial thread executes sequentially until it encounters a parallel construct. At that point,
this thread creates a group of itself and zero or more additional threads and becomes the master thread of the new group. Each thread
executes the commands included in the parallel region, and their execution may be differentiated, according to additional directives
provided by the programmer. At the end of the parallel region, all threads are synchronized. \par
The runtime environment is responsible for effectively scheduling threads. Each thread, receives a unique id, which differentiates it
during execution. Scheduling is performed according to memory usage, machine load and other factors and may be adjusted by altering
environmental variables. In terms of memory usage, most variables in OpenMP code are visible to all threads by default.
However, OpenMP provides a variety of options for data management, such as a thread-private memory and private variables, as well as multiple ways of
passing values between sequential and parallel regions. Additionally, recent OpenMP implementations introduced the concept of tasks,
as a solution for parallelizing applications that produce dynamic workloads.
Thus, OpenMP is enriched with a flexible model for irregular parallelism, providing parallel while loops and recursive data structures. \par
The main advantage on using OpenMP is the ease of developing parallelismswith simple constructs that (often) do not differ too much from the original implementation.

The snippet at \textbf{Algorithm \ref*{alg:omp}} shows the implementation used in this work: the matrix containing the
distances between vertices is shared among all the threads, while the two nested for-loops are executed
by each thread indipendentely. The $k^{th}$ row is extracted by a single thread and shared among the other threads: this
guarantees a sequential access to the $k^{th}$ row faster than accessing the same row directly on $M$, resulting in a speedup of up to $\textasciitilde31\%$.

\begin{algorithm}[h!]

\SetAlgoLined

\texttt{\#pragma omp parallel num\_threads(t) shared(M, K)} private(k) \\
\For{$k = 1 \rightarrow n$}{
	\texttt{\#pragma omp master} \\
	$K \leftarrow k^{th}Row(M, k)$ \\
	\texttt{\#pragma omp for private(i,j) schedule(dynamic)} \\
	\For{$i = 1 \rightarrow n$}{
		\For{$j = 1 \rightarrow n$}{
			\If{$M_{i,j} < M_{i,k} + K_j$)}{
				$M_{i,j} \leftarrow M_{i,k} + K_j$
			}
		}
	}
}

\caption{Multithreaded FW with \emph{OpenMP}}\label{alg:omp}
\end{algorithm}

Despite the overhead that the dynamic scheduler entails, in this case it works better than a static one because the
content of the sub-matrices varies from thread to thread so a group of threads may do more write operations on $M$ and a
static scheduler would make wait the others until they finish. We benchmarked a loss in performance from $\textasciitilde7\%$ to $\textasciitilde48\%$ when choosing a static scheduler over
a dynamic one, depending on the size of the matrix.
 \\
Also note that the solution does not implement a \texttt{collapse} directive because when we collapse multiple loops, OpenMP turns them into a single loop: there is a single
index that is constructed from \texttt{i} and \texttt{j} using division and modulo operations. This can
have huge impacts on the performance because of this overhead, especially if the matrix is really wide.

For the concrete implementation see \textbf{section \ref{ompimpl}}.

\textbf{Figure \ref*{fig:threads}} shows how 2 threads interact inside the matrix: the red and orange zones highlight the cells where
Thread 1 and Thread 2 can write respectively; the blue and turquise cells represent the intermediate vertices that are compared
with the vertice under analysis for Thread 1 and Thread 2 respectively.

\begin{figure}[h!]
\centering
\includegraphics[width=3.5in]{diagrams/openmp-threads}
\captionsetup{justification=centering,margin=2cm}
\caption{View of the data each thread can reach}
\label{fig:threads}
\end{figure}

We already proved that no race condition may appear in this case.
No race condition can occur if $k$ is the same among the threads and threrfore there's no need to verify atomicity of the write operation;
the lack of OpenMP directive like \texttt{atomic} or \texttt{critical} plays in favor of performance. \par

The speedup of the solution is sligthy worse than the ideal speedup (see \textbf{Figure \ref*{fig:omp-speedup}})

\begin{figure}[h!]
\centering
\includegraphics[width=3.5in]{diagrams/openmp-speedup}
\captionsetup{justification=centering,margin=2cm}
\caption{Speedup of \emph{OpenMP} FW on a octacore CPU}
\label{fig:omp-speedup}
\end{figure}
When scaling from 7 to 8 threads, we notice a slight deviation from the previous (almost) linear trend. That's because the measurement
is taken from a 8-core/8-thread CPU, namely Intel Core i7-9700K, and because no other cores were free to manage the OS and its subprocesses, the scheduler
divided this task among all the threads. So we interpolated the speedup, not counting the fluctuations due to the management of the OS. \par

The efficiency, which stays always above $90\%$, is shown in \textbf{Figure \ref*{fig:omp-efficiency}} alongside with its theoretical counterpart.

\begin{figure}[h!]
\centering
\includegraphics[width=3.5in]{diagrams/openmp-efficiency}
\captionsetup{justification=centering,margin=2cm}
\caption{Efficiency of \emph{OpenMP} FW on a octacore CPU}
\label{fig:omp-efficiency}
\end{figure}
An overview of the timings collected can be found in \textbf{Table \ref*{tab:omp-time}}.

\subsection{GPGPU with CUDA}
In recent years, many scientific and numeric GPGPU applications found success due to graphics hardware’s streaming data-parallel organizational model.

The GPU serves, to an extent, as a coprocessor to the CPU programmed through the CUDA API. A single program known as kernel is compiled to operate on the GPU
device to exploit the massive data parallelism inherit on Single Instruction, Multiple Data (SIMD) architecture.
Groups of threads then execute the kernel on the GPU. Threads are organized into blocks which allow efficient sharing of
data through a high-speed shared memory region accessible to the programmer directly through CUDA.
Shared-memory is shared among threads in a block, facilitating higher bandwidth and overall performance gains.
Therefore, algorithms must intelligently manage this fast shared memory cache effectively.
This will fully utilize the data parallelism capabilities of graphics
hardware and alleviates any memory latency that data intensive algorithms suffer from on the GPU.

Unlike the other two implementations, \emph{CUDA} FW can rely on its \emph{grid-of-blocks} system and thus the kernel can do no loops: $k$ is still shared among threads
so that no data race occurs but each thread instead of covering one or more rows, is assigned to a single cell.

\begin{figure}[h!]
\centering
\includegraphics[width=2.5in]{diagrams/cuda-threads}
\captionsetup{justification=centering}
\caption{Mapping of CUDA threads inside $M$}
\label{fig:cuda-threads}
\end{figure}

Depending on the number of threads per block, each row  of matrix $M$ is covered by 1 or more blocks, that
have always height equal to 1 thread. This mapping allows threads of the same block share some data that can be reused. \\
In this case the shared variable stores the value $M_{i,k}$ that is in common with the entire row. Because the
shared memory offers lower latency and higher bandwidth than the global memory, it is worth storing this value
for other threads.

\begin{algorithm}[h!]

\SetAlgoLined
\DontPrintSemicolon

\For{$k = 0 \rightarrow n$}{
	\textnormal{kernel<{}<{}<$\frac{n+b-1}{b} \times n$, $b$>{}>{}>($M$, $n$, $k$)}
}

\;

\SetKwFunction{FMain}{kernel}
  \SetKwProg{Fn}{Function}{:}{}
  \Fn{\FMain{$M$, $n$, $k$}}{

$j \leftarrow \left|B\right|_x \times B_x + T_x$ \\
$i \leftarrow \left|B\right|_y$ \\

\If{$j < n$}{

	\If{$T_x = 0 $}{
		$K_{\textnormal{shared}} \leftarrow M_{i,k};$
	}

	$\textnormal{syncthreads()}$

	\If{$M_{i,j} < K_{\textnormal{shared}} + M_{k,j}$}{
		$M_{i,j} \leftarrow K_{\textnormal{shared}} + M_{k,j}$
	}
}
}

\caption{Kernel of the \emph{CUDA-FW} on a pre-Volta architecture}\label{alg:cuda}
\end{algorithm}

\textbf{Algorithm \ref*{alg:cuda}} shows the kernel function for this implementation. \\
$B_x$ is the coordinate $x$ of the block, $\left|B\right|_x$ is the horizontal size of the block, $T_x$ the
coordinate $x$ of the thread; $\left|B\right|_y$ is the coordinate $y$ of the block and $K_{\textnormal{shared}}$ is the variable
shared among all the threads of the same block.

One thread (namely $T_{0,0}$) reads from $M_{i,k}$ and stores the value in the shared memory. From here threads encounter a synchronization point which
assures that $T_{0,0}$ had written in the shared memory and the block can actually use the correct value.

The state-of-the-art L1 cache in Volta and Turing offers lower latency, higher bandwidth, and higher capacity compared to the earlier architectures. Like Volta, Turing's L1 can cache write operations (write-through). The result is that for many applications Volta and Turing narrow the performance gap between explicitly managed shared memory and direct access to device memory \cite{nvidia}. \par
For this reason we benchmarked \textbf{Algorithm \ref*{alg:cuda2}} which relies only on global memory and we do not experienced any performance degradation on the Turing architecture.
This mechanism is present in the newer Ampere architecture as well.

For the concrete implementation see \textbf{section \ref{cudaimpl}}.

\begin{algorithm}[h!]

\SetAlgoLined

\DontPrintSemicolon
  \SetKwFunction{FMain}{kernel}
  \SetKwProg{Fn}{Function}{:}{}
  \Fn{\FMain{$M$, $n$, $k$}}{

$j \leftarrow \left|B\right|_x \times B_x + T_x$ \\
$i \leftarrow \left|B\right|_y$ \\

\If{$j < n$}{

	\If{$M_{i,j} < M_{i,k} + M_{k,j}$}{
		$M_{i,j} \leftarrow M_{i,k} + M_{k,j}$
	}
}
}

\caption{Kernel of the \emph{CUDA-FW} on a Volta and post-Volta architectures}\label{alg:cuda2}
\end{algorithm}

Because each thread does little computation and uses high memory bandwidth, timings are really low if compared
to \emph{serial-FW} and \emph{OMP} FW with 8 cores. \\
In fact from \textbf{Figure \ref*{fig:cuda-time}} we see a speedup up to $87.8$ times compared to \emph{serial} FW
and $12.34$ times compared to \emph{OMP} FW with 8 cores. We also notice that the maximum speedup is reached when the number of threads per block is equal to 128;
that's because the occupancy of the Streaming Multiprocessors is at its peak and there are less wasted threads.
\begin{figure}[h!]
\centering
\includegraphics[width=3.5in]{diagrams/cuda-speedup}
\captionsetup{justification=centering}
\caption{Speedup of \emph{CUDA} FW (128 threads per block) consuming a medium matrix compared to \emph{sequential} and \emph{OMP} FW}
\label{fig:cuda-speedup}
\end{figure}

Because the CUDA APIs don't allow the user to control the number of Streaming Multiprocessors or how many CUDA cores
can be involved in the computation, the speedup is calculated based on the number of threads per block.

\begin{figure}[h!]
\centering
\includegraphics[width=3.5in]{diagrams/cuda-time}
\captionsetup{justification=centering}
\caption{Efficiency of \emph{MPI} FW consuming a medium matrix}
\label{fig:cuda-time}
\end{figure}

\subsection{Hybrid: MPI + OpenMP}
In order to make the most of \emph{MPI FW}, it can be combined with \emph{OpenMP}; this
makes sense because otherwise each process would run the $i$ and $j$ for loops on one thread and nowadays
is more than rare to have a cluster composed of single thread CPUs.

\textbf{Algorithm \ref*{alg:mpi+omp}} shows how easily \emph{MPI FW} can be modified
so that each process can benefit from multithreading.
\begin{algorithm}[h!]

\SetAlgoLined

broadcast($n$, $ROOT$) \\
scatter($M$, $processes$)

\texttt{\#pragma omp parallel num\_threads(t) private(k) shared(M, K, p)} \\
\For{$k = 1 \rightarrow n$}{
	\texttt{\#pragma omp master} \\
	\If{$rank =$ \textnormal{findOwnerOfKthRow}($k$, $rank$)}{
		$K \leftarrow $ getRow($M$, $k$) \\
	}
	broadcast($K$) \\

	\texttt{\#pragma omp for private(i,j) schedule(dynamic)} \\
	\For{$i = 1 \rightarrow \frac{n}{p}$}{
		\For{$j = 1 \rightarrow n$}{
			\If{$M_{i,j} < M_{i,k} + K_j$}{
				$M_{i,j} \leftarrow M_{i,k} + K_j$
			}
		}
	}
}
gather($M$, $processes$);

\caption{\emph{MPI+OpenMP} FW}\label{alg:mpi+omp}
\end{algorithm}

For the concrete implementation see \textbf{section \ref{hybimpl}}.

\subsection{Hybrid: MPI + CUDA}
Another way to improve the efficiency of the cluster is to
use GPGPU on each node.

\textbf{Algorithm \ref*{alg:mpi+cuda}} shows how this solution
can be implemented.
\begin{algorithm}[h!]

\SetAlgoLined
\DontPrintSemicolon
broadcast($n$, $ROOT$) \\
scatter($M$, $p$)

\For{$k = 1 \rightarrow n$}{
	\If{$rank =$ \textnormal{findOwnerOfKthRow}($k$, $rank$)}{
		$K \leftarrow $ getRow($M$, $k$) \\
	}
  	broadcast($K$) \\

	\textnormal{kernel<{}<{}<$\frac{n+b-1}{b\cdot p}\times n$, $b$>{}>{}>($M$, $n$, $k$)}

}
gather($M$, $p$);
\;
\;
  \SetKwFunction{FMain}{kernel}
  \SetKwProg{Fn}{Function}{:}{}
  \Fn{\FMain{$M$, $n$, $k$}}{

$j \leftarrow \left|B\right|_x \times B_x + T_x$ \\
$i \leftarrow \left|B\right|_y$ \\

\If{$j < n$}{

	\If{$M_{i,j} < M_{i,k} + M_{k,j}$}{
		$M_{i,j} \leftarrow M_{i,k} + M_{k,j}$
	}
}
}

\caption{\emph{MPI+CUDA} FW}\label{alg:mpi+cuda}
\end{algorithm}

Although the increase of performance, this solution implies
high costs because each node must be equipped with a GPU; moreover the GPUs
(that must be produced by NVIDIA) must be compatibile with the version of the CUDA Toolkit
for which the program was compiled for.


\documentclass{article}
\title{Chapter 08}
\author{Newton Ni}

\usepackage{bussproofs}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}

% Set cardinality: |#1|
\newcommand{\cardinality}[1]{\lvert#1\rvert}

% Set: { #1 }
\newcommand{\set}[1]{\{\ #1\ \}}

% Set comprehension: { #1 | #2 }
\newcommand{\comp}[2]{\set{#1\ \mid\ #2}}

% t₁, t₂, t₃, ...
\newcommand{\term}[1]{\texttt{t\textsubscript{#1}}}

% v₁, v₂, v₃, ...
\newcommand{\val}[1]{\texttt{v\textsubscript{#1}}}

\renewcommand{\ss}[2]{#1 \longrightarrow #2}
\renewcommand{\bs}[2]{#1 \Downarrow #2}

% Monospace
\newcommand{\ms}[1]{\texttt{#1}}

\newcommand{\LabelR}[1]{\RightLabel{\textsc{#1}}}

\theoremstyle{remark}
\newtheorem*{case}{Case}

\begin{document}
\maketitle

\section{8.2.3}

    \textit{Prove that every subterm of a well-typed term is well typed.}

    \begin{proof}
        By induction on the typing derivation of term \term{}. Our inductive hypothesis is:
        $$H(\term{}): \term{} \textit{ is well-typed} \implies \textit{all subterms of } \term{} \textit{ are well-typed}$$

        We proceed by case analysis on the final step of the derivation.
        \begin{case}[\ms{true}, \ms{false}, 0]
            These terms are well-typed by the inversion lemma, and have no subterms.
        \end{case}
        \begin{case}[\ms{if \term{1} then \term{2} else \term{3}}]
            By the inversion lemma, this term is well-typed, and we have three well-typed subterms:
            \begin{itemize}
                \item $\term{1} : \ms{Bool}$
                \item $\term{2} : \ms{R}$
                \item $\term{3} : \ms{R}$
            \end{itemize}
            We can apply the inductive hypothesis to each.
        \end{case}
        \begin{case}[\ms{succ \term{1}}, \ms{pred \term{1}}, \ms{iszero \term{1}}]
            These cases are analagous to the previous case, except they
            only have one subterm each.
        \end{case}
    \end{proof}

\section{8.3.4}

    \textit{Restructure} [the \textsc{Preservation}] \textit{proof so that it goes by}
    \textit{induction on evaluation derivations rather than typing derivations.}

    \begin{proof}
        By induction on the derivation of $\ss{\term{}}{\term{}'}$. Our inductive hypothesis is:
        $$H(\term{}): \term{} : T\ \land\ \ss{\term{}}{\term{}'} \implies \term{}' : T$$
        We proceed by case analysis on the last step of the derivation. In all cases,
        we have that $\term{} : T$.

        \begin{case}[\textsc{E-IfTrue}]
            Here, $\ss{\term{}}{\term{}'}$ is:
            $$\ss{\ms{if true then \term{2} else \term{3}}}{\term{2}}$$
            By the inversion lemma, $\term{} : T \implies \term{2} : T$ as desired.
        \end{case}
        \begin{case}[\textsc{E-IfFalse}]
            This case is analagous to the previous one.
        \end{case}

        \begin{case}[\textsc{E-If}]
            Here, we have:
            $$\ss{\ms{if \term{1} then \term{2} else \term{3}}}{\ms{if \term{1}' then \term{2} else \term{3}}}$$
            $$\ss{\term{1}}{\term{1}'}$$
            By the inversion lemma, we have:
            \begin{itemize}
                \item $\term{1} : \ms{Bool}$
                \item $\term{2} : \ms{T}$
                \item $\term{3} : \ms{T}$
            \end{itemize}
            By the inductive hypothesis, we also have $\term{1}' : \ms{Bool}$.
            We conclude $\term{}' : \ms{T}$ by \textsc{T-If}.
        \end{case}

        \begin{case}[\textsc{E-Succ}]
            Here, we have:
            $$\ss{\ms{succ \term{1}}}{\ms{succ \term{1}'}}$$
            $$\ss{\term{1}}{\term{1}'}$$
            By the inversion lemma, we have $\term{1} : \ms{Nat}$.
            By the inductive hypothesis, we have $\term{1}' : \ms{Nat}$.
            We conclude that $\ms{succ \term{1}'} : \ms{Nat}$ by \textsc{T-Succ}.
        \end{case}
        \begin{case}[\textsc{E-Pred}, \textsc{E-IsZero}]
            These cases are analagous to the previous case.
        \end{case}

        \begin{case}[\textsc{E-PredZero}, \textsc{E-IsZeroZero}]
            These cases follow directly from the inversion lemma and typing
            rules \textsc{T-Zero} and \textsc{T-True}, respectively.
        \end{case}

        \begin{case}[\textsc{E-PredSucc}]
            Here, we have:
            $$\ss{\ms{pred (succ \term{1})}}{\term{1}}$$
            By two applications of the inversion lemma, we have:
            \begin{itemize}
                \item $\ms{succ \term{1}} : \ms{Nat}$
                \item $\ms{\term{1}} : \ms{Nat}$
            \end{itemize}
        \end{case}
        \begin{case}[\textsc{E-IsZeroSucc}]
            This case is analagous to the previous case.
        \end{case}

    \end{proof}

\section{8.3.5}

    \textit{The evaluation rule \textsc{E-PredZero} (Figure 3--2) is a bit counterintuitive:}
    \textit{we might feel that it makes more sense for the predecessor of zero to be undefined,}
    \textit{rather than being defined to be zero. Can we achieve this simply by removing the}
    \textit{rule from the definition of single-step evaluation?} \\

    No, because the term $\ms{pred 0}$ would be well-typed by \textsc{T-Zero} and \textsc{T-Pred},
    but would be stuck, violating the progress property.

\section{8.3.6}

    \textit{Having seen the subject reduction property, it is reasonable to wonder whether}
    \textit{the opoosite property---\text{subject expansion}---also holds. Is it always the}
    \textit{case that, if} $\ss{\term{}}{\term{}'}$ \textit{and} $\term{}' : \ms{T}$ \textit{, then}
    $\term{} : \ms{T}$? \textit{If so, prove it. If not, give a counterexample.} \\

    No, since ill-typed terms can also take a step. For example:
    $$\ss{\ms{if true then 0 else true}}{0}$$

\section{8.3.7}

    \textit{Suppose our evaluation relation is defined in the big-step style, as in exercise}
    \textit{3.5.17. How should the intuitive property of type safety be formalized?} \\

    Type preservation remains the same, but progress is modified to state that all well-typed terms
    evaluate to a value.

\section{8.3.8}

    \textit{Suppose our evaluation relation is augmented with rules for reducing nonsensical}
    \textit{terms to an explicit} \ms{wrong} \textit{state, as in Exercise 3.5.16. Now how}
    \textit{should type safety be formalized?} \\

    Type preservation remains the same, but progress is no longer useful, as terms can always
    take a step (sometimes to \ms{wrong}).

\end{document}