%In what follows, we use the notation $(x_1,y_1)$ to represent a point in
%the $(x,y)$ coordinate system, also called the $(x,y)$-plane.
%Previously, we used $(a,b)$ to represent
%an open interval. Notation often gets reused and abused in mathematics,
%but thankfully, it is usually clear from the context what we mean.
%
%In the $(x,y)$ coordinate system we normally write the $x$-axis
%horizontally, with positive numbers to the right of the origin, and
%the $y$-axis vertically, with positive numbers above the origin.  That
%is, unless stated otherwise, we take ``rightward'' to be the positive
%$x$-direction and ``upward'' to be the positive $y$-direction.  In a
%purely mathematical situation, we normally choose the same scale for
%the $x$- and $y$-axes.  For example, the line joining the origin to
%the point $(a,a)$ makes an angle of 45${}^\circ$ with the $x$-axis
%(and also with the $y$-axis).
%
%In applications, often letters other than $x$ and $y$ are used, and
%often different scales are chosen in the horizontal and vertical
%directions.  \\
%
%\begin{example}{Data Plot}{DataPlot}
%Suppose you drop a coin from a window, and you want to study how
%its height above the ground changes from second to second.
%It is natural to let the letter $t$ denote the time
%(the number of seconds since the object was released) and to let the
%letter $h$ denote the height.  For each $t$ (say, at one-second
%intervals) you have a corresponding height $h$.  This information can
%be tabulated, and then plotted on the $(t,h)$ coordinate plane, as
%shown in figure~\ref{fig:data plot}.
%\end{example}
%
%\figure[!ht]
%\vbox{
%$$\begin{array}{|r|ccccc|}
%\hline
%seconds & 0 & 1 & 2 & 3 & 4\\
%\hline
%meters & 80 & 75.1 & 60.4 & 35.9 & 1.6\\
%\hline
%\end{array}$$
%
%\centerline{\vbox{\beginpicture
%\normalgraphs
%%\ninepoint
%\setcoordinatesystem units <2truecm,0.5truemm>
%\setplotarea x from 0 to 4.2, y from 0 to 90
%\axis left shiftedto x=0 ticks numbered from 20 to 80 by 20 /
%\axis bottom shiftedto y=0 ticks numbered from 0 to 4 by 1 /
%\put {$t$} [l] <3pt,0pt> at 4.2 0
%\put {$h$} [l] <3pt,0pt> at 0 90
%\setquadratic
%\plot 0 80 1 75.1 2 60.4 3 35.9 4 1.6 /
%\multiput {$\bullet$} at 0 80 1 75.1 2 60.4 3 35.9 4 1.6 /
%\endpicture}}}
%\caption{A data plot, height versus time. \label{fig:data plot}}
%\endfigure
%
%We use the word ``quadrant'' for each of the four regions into which
%the plane is divided by the axes: the first quadrant is where points
%have both coordinates positive, or the ``northeast'' portion of the
%plot, and the second, third, and fourth quadrants are counted off
%counterclockwise, so the second quadrant is the northwest, the third
%is the southwest, and the fourth is the southeast.
%
%Suppose we have two points $A$ and $B$ in the $(x,y)$-plane.
%We often want to know the change in $x$-coordinate (also called the
%``horizontal distance'') in going from $A$ to $B$.  This is often
%written $\Delta x$, where the meaning of $\Delta$ (a capital delta in
%the Greek alphabet) is ``change in''. Thus, $\Delta x$ can be read as
%``change in $x$'' although it usually is read as ``delta $x$''. The
%point is that $\Delta x$ denotes a single number, and should not be
%interpreted as ``delta times $x$''. Similarly, the ``change in $y$'' is written $\Delta y$
%and represents the difference between the $y$-coordinates of the
%two points. It is the vertical distance you have to move in going from $A$ to $B$. \\
%
%\begin{example}{Change in $x$ and $y$}{ChangeIn}\label{ChangeIn}
%If $A=(2,1)$ and $B=(3,3)$ the change in $x$ is
%$$\Delta x=3-2=1$$
%while the change in $y$ is
%$$\Delta y= 3-1=2.$$
%\vspace{-0.5cm}
%\end{example}
%
%The general formulas for the change in $x$ and the change in $y$
%between a point $(x_1,y_1)$ and a point $(x_2,y_2)$ are:
%$$
%\Delta x=x_2-x_1,\qquad\qquad\Delta y=y_2-y_1.
%$$
%Note that either or both of these might be negative.

%%%%%%%%%%%%%%%%%%
\subsection{Lines}\label{sec:lines}
If we have two \ifont{distinct} points $A(x_1,y_1)$ and $B(x_2,y_2)$, then we can draw one
and only one straight line through both points.  By the \dfont{slope} of this line
we mean the ratio of $\Delta y$ to $\Delta x$.  The slope is often denoted by the letter $m$. \\

\begin{formulabox}[Slope Formula]
The slope of the line joining the points $(x_1,y_1)$ and $(x_2,y_2)$ is:
$$m=\frac{\Delta y}{\Delta x}=\frac{y_2-y_1}{x_2-x_1}=\frac{\mbox{rise}}{\mbox{run}}$$
\end{formulabox}

\bigskip

\begin{example}{Slope of a Line Joining Two Points}{SlopeofLine}
The line joining the two points $(1,-2)$ and $(3,5)$ has slope
$\ds m=\frac{5-(-2)}{3-1}=\frac{7}{2}$
%\vspace{-0.8cm}
\end{example}

The most familiar form of the equation of a straight line is:
$$y=mx+b.$$
Here $m$ is the slope of the line: if you increase $x$ by
1, the equation tells you that you have to increase $y$ by $m$; and if
you increase $x$ by $\Delta x$, then $y$ increases by $\Delta
y=m\Delta x$.  The number $b$ is called the \dfont{y-intercept}, because
it is where the line crosses the $y$-axis (when $x=0$).  If you know two points on
a line, the formula $m=(y_2-y_1)/(x_2-x_1)$ gives you the slope.
Once you know a point and the slope, then the $y$-intercept can be
found by substituting the coordinates of either point in the equation:
$y_1=mx_1+b$, i.e., $b=y_1-mx_1$.  Alternatively, one can use the
\dfont{``point-slope'' form} of the equation of a straight line: start with
$(y-y_1)/(x-x_1)=m$ and then multiply to get
$$(y-y_1)=m(x-x_1),$$ the
point-slope form. Of course, this may be further manipulated to get
$y=mx-mx_1+y_1$, which is essentially the \dfont{``$y=mx+b$'' form}.

It is possible to find the equation of a line between two points directly
from the relation $m=(y-y_1)/(x-x_1)=(y_2-y_1)/(x_2-x_1)$, which says ``the
slope measured between the point $(x_1,y_1)$ and the point $(x_2,y_2)$ is
the same as the slope measured between the point $(x_1,y_1)$ and any other
point $(x,y)$ on the line.''  For example, if we want to find the equation
of the line joining points $A(2,1)$ and $B(3,3)$, we can use
this formula:
$$
m=\frac{y-1}{x-2}=\frac{3-1}{3-2}=2,\qquad\hbox{so~that}\qquad
y-1=2(x-2),\qquad\hbox{i.e.,}\qquad y=2x-3.
$$
Of course, this is really just the point-slope formula, except that we
are not computing $m$ in a separate step.
We summarize the three common forms of writing a straight line below:\\

\begin{formulabox}[Slope-Intercept Form of a Straight Line]
An equation of a line with slope $m$ and $y\, $-intercept $b$ is:
$$y=mx+b.$$
\end{formulabox}

\bigskip

\begin{formulabox}[Point-Slope Form of a Straight Line]
An equation of a line passing through $(x_1,y_1)$ and having slope $m$ is:
$$y-y_1=m(x-x_1).$$
\end{formulabox}

\bigskip

\begin{formulabox}[General Form (Standard Form) of a Straight Line]
Any line can be written in the form
$$Ax+By+C=0,$$
where $A,B,C$ are constants and $A,B$ are not both $0$.
\end{formulabox}

The slope $m$ of a line in the form $y=mx+b$ tells us the
direction in which the line is pointing.  If $m$ is positive, the line goes
into the 1st quadrant as you go from left to right.   If $m$ is large and
positive, it has a steep incline, while if $m$ is small and positive,
then the line has a small angle of inclination.  If $m$ is negative, the
line goes into the 4th quadrant as you go from left to right.  If $m$ is
a large negative number (large in absolute value), then the line points
steeply downward. If $m$ is negative but small in absolute value, then it points
only a little downward.

If $m=0$, then the line is horizontal and its equation is simply $y=b$.

All of these possibilities are illustrated below.
%figure~\ref{fig:graphs of lines}.

$$\includegraphics[width=6.4in]{images/slopes}$$

%\figure[!ht]
%\vbox{
%\centerline{\vbox{\beginpicture
%\normalgraphs
%%\sevenpoint
%\setcoordinatesystem units <3truemm,3truemm>
%\setplotarea x from -5 to 5, y from -5 to 5
%\axis left shiftedto x=0 /
%\axis bottom shiftedto y=0 /
%\setlinear
%\plot -1 -5 2.33 5 /
%\linethickness 0.1pt
%\axis left ticks in andacross from -5 to 5 by 1 /
%\axis left ticks in andacross numbered from -4 to 4 by 2 /
%\axis bottom ticks in andacross from -5 to 5 by 1 /
%\axis bottom ticks in andacross numbered from -4 to 4 by 2 /
%\endpicture}\qquad\vbox{\beginpicture
%\normalgraphs
%%\sevenpoint
%\setcoordinatesystem units <3truemm,3truemm>
%\setplotarea x from -5 to 5, y from -5 to 5
%\axis left shiftedto x=0 /
%\axis bottom shiftedto y=0 /
%\setlinear
%\plot -5 1 5 2 /
%\linethickness 0.1pt
%\axis left ticks in andacross from -5 to 5 by 1 /
%\axis left ticks in andacross numbered from -4 to 4 by 2 /
%\axis bottom ticks in andacross from -5 to 5 by 1 /
%\axis bottom ticks in andacross numbered from -4 to 4 by 2 /
%\endpicture}\qquad\vbox{\beginpicture
%\normalgraphs
%%\sevenpoint
%\setcoordinatesystem units <3truemm,3truemm>
%\setplotarea x from -5 to 5, y from -5 to 5
%\axis left shiftedto x=0 /
%\axis bottom shiftedto y=0 /
%\setlinear
%\plot -0.5 5 2 -5 /
%\linethickness 0.1pt
%\axis left ticks in andacross from -5 to 5 by 1 /
%\axis left ticks in andacross numbered from -4 to 4 by 2 /
%\axis bottom ticks in andacross from -5 to 5 by 1 /
%\axis bottom ticks in andacross numbered from -4 to 4 by 2 /
%\endpicture}\qquad\vbox{\beginpicture
%\normalgraphs
%%\sevenpoint
%\setcoordinatesystem units <3truemm,3truemm>
%\setplotarea x from -5 to 5, y from -5 to 5
%\axis left shiftedto x=0 /
%\axis bottom shiftedto y=0 /
%\setlinear
%\plot -5 2 5 1 /
%\linethickness 0.1pt
%\axis left ticks in andacross from -5 to 5 by 1 /
%\axis left ticks in andacross numbered from -4 to 4 by 2 /
%\axis bottom ticks in andacross from -5 to 5 by 1 /
%\axis bottom ticks in andacross numbered from -4 to 4 by 2 /
%\endpicture}}}
%\caption{Lines with slopes 3, $0.1$, $-4$, and $-0.1$.\label{fig:graphs of lines}}
%\endfigure

There is one type of line that cannot be written in the form $y=mx+b$,
namely, vertical lines.  A vertical line has an equation of the form $x=a$.
Sometimes one says that a vertical line has an ``infinite'' slope.

It is often useful to find the $x$-intercept of a line $y=mx+b$.  This is
the $x$-value when $y=0$.  Setting $mx+b$ equal to 0 and solving for
$x$ gives: $x=-b/m$.  \\

\begin{example}{Finding $x$-intercepts}{xintercepts}
To find $x$-intercept of the line $y=2x-3$, we set $y=0$ and solve for $x$:
$$0 = 2x-3\qquad\to\qquad x = \frac{3}{2}.$$
Thus, the line has an $x$-intercept of $3/2$.
\end{example}

It is often necessary to know if two lines are parallel or perpendicular.
Let $m_1$ and $m_2$ be the slopes of the nonvertical lines $L_1$ and $L_2$.
Then:
\begin{itemize}
\item $L_1$ and $L_2$ are \dfont{parallel} if and only if $m_1=m_2$.
\item $L_1$ and $L_2$ are \dfont{perpendicular} if and only if $\ds{m_2=\frac{-1}{m_1}}$.
\end{itemize}
In the case of perpendicular lines, we say their slopes are \ifont{negative reciprocals}.
Below is a visual representation of a pair of parallel lines and a pair of perpendicular lines.
$$\includegraphics[width=3.7in]{images/lines-1}$$

\begin{example}{Equation of a Line}{EquationLine}
For each part below, find an equation of a line satisfying the requirements:
\begin{enumerate}
\item[(a)] Through the two points $(0,3)$ and $(-2,4)$.
\item[(b)] With slope $7$ and through point $(1,-2)$.
\item[(c)] With slope $2$ and $y$-intercept $4$.
\item[(d)] With $x$-intercept $8$ and $y$-intercept $-3$.
\item[(e)] Through point $(5,3)$ and parallel to the line $2x+4y+2=0$.
\item[(f)] With $y$-intercept $4$ and perpendicular to the line $\ds{y=-\frac{2}{3}x+3}$.
\end{enumerate}
\end{example}

\begin{solution}
\noindent(a) We use the \ifont{slope formula} on $(x_1,y_1)=(0,3)$ and $(x_2,y_2)=(-2,4)$ to find m:
$$m=\frac{(4)-(3)}{(-2)-(0)}=\frac{1}{-2}=-\frac{1}{2}.$$
Now using the \ifont{point-slope formula} we get an equation to be:
$$y-3=-\frac{1}{2}\left(x-0\right)\quad\to\quad y=-\frac{1}{2}x+3.$$

\noindent(b) Using the \ifont{point-slope formula} with $m=7$ and $(x_1,y_1)=(1,-2)$ gives:
$$y-(-2)=7(x-1)\quad\to\quad y = 7x-9.$$

\noindent(c) Using the \ifont{slope-intercept formula} with $m=2$ and $b=4$ we get $y=2x+4$.

\noindent(d) Note that the intercepts give us two points: $(x_1,y_1)=(8,0)$ and $(x_2,y_2)=(0,-3)$.
Now follow the steps in part (a):
$$m=\frac{-3-0}{0-8}=\frac{3}{8}$$
Using the \ifont{point-slope formula} we get an equation to be:
$$y-(-3)=\frac{3}{8}\left(x-0\right)\quad\to\quad y=\frac{3}{8}x-3$$

\noindent(e)  The line $2x+4y+2=0$ can be written as:
$$4y = -2x - 2 \quad\to\quad y=-\frac{1}{2}x-\frac{1}{2}$$
This line has slope $-1/2$.
Since our line is \ifont{parallel} to it, we have $m=-1/2$.
Now we have a point $(x_1,y_1)=(5,3)$ and slope $m=-1/2$, thus, the \ifont{point-slope formula} gives:
$$y-3=-\frac{1}{2}\left(x-5\right).$$

\noindent(f) The line $\ds{y=-\frac{2}{3}x+3}$ has slope $m=-2/3$.
Since our line is perpendicular to it, the slope of our line is the \ifont{negative reciprocal}, hence, $m=3/2$.
Now we have $b=4$ and $m=3/2$, thus by the \ifont{slope-intercept formula}, an equation of the line is
$$y=\frac{3}{2}x+4.$$
\vspace{-0.5cm}
\end{solution}

\begin{example}{Parallel and Perpendicular Lines}{ParallelPerpendicularLines}
Are the two lines $7x+2y+3=0$ and $6x-4y+2=0$ perpendicular? Are they parallel? If they are not parallel, what is their point of intersection?
\end{example}

\begin{solution}
The first line $L_{1}$ re-expressed in slope-intercept form is:
$$7x+2y+3=0\quad\to\quad 2y=-7x-3\quad\to\quad y=-\frac{7}{2}x-\frac{3}{2}.$$
It has slope $m_1=-7/2$.
The second line $L_{2}$ in slope-intercept form is:
$$6x-4y+2=0\quad\to\quad -4y=-6x-2\quad\to\quad y=\frac{3}{2}x+\frac{1}{2}.$$
It has slope $m_2=3/2$. Since $m_1\neq m_2$ the lines are not parallel. The lines are also not perpendicular since $m_1\cdot m_2\neq -1$ (they are not negative reciprocals). Therefore, the lines must intersect.\\
We find points of intersection by setting the $y$ -values of $L_{1} = L_{2}$, and then solve for $x$.

\begin{minipage}{0.6\textwidth}
	In particular, we have

	$$\begin{array}{rcl}
	L_{1} & = & L_{2} \\
	\ds{-\frac{7}{2}x-\frac{3}{2}} & = & \ds{\frac{3}{2}x+\frac{1}{2}} \\
	\end{array} $$

Solving for $x$ gives $x=-2/5$. \\

Then substituting this
into either $L_{1}$ or $L_{2}$ \\
gives $y=-1/10$.\\

Therefore, the lines intersect at the point $(-2/5,-1/10)$.

\vspace{2cm}
\end{minipage}
\begin{minipage}{0.35\textwidth}

\vspace{3mm}
\hspace{3mm}\includegraphics[width=2.2in]{images/ExampleLines}
\end{minipage}
\end{solution}


\subsection{Renewal arrays theory}
\label{sec:back:to:the:basics:rogers}

\citeauthor{rogers:1977}, in \cite{rogers:1977}, proposes a theory
as a solution to the question left open by \citeauthor{shapiro:1976},
introducing the concept of \emph{renewal arrays}.

In order to understand his work, we formalize first the concept of $k$-fold
convolution of a sequence $\lbrace s_{i}\rbrace_{i\in\mathbb{N}}$
with itself and, second, the concept of \emph{renewal arrays}.
\\\\
Let $\lbrace \alpha_{i}\rbrace_{i\in\mathbb{N}}$ and
$\lbrace \beta_{i}\rbrace_{i\in\mathbb{N}}$ be two sequences
\marginpar{to not clutter notations, we discard subscript in
notation $\lbrace \alpha_{i} \rbrace$ for sequences}. Denote
with $\cdot$ the \emph{convolution} operator which consumes a pair
of sequences and produces a new sequence, written in \emph{infix}
notation:
\begin{displaymath}
    \lbrace \alpha_{i}\rbrace_{i\in\mathbb{N}}\cdot
    \lbrace \beta_{i}\rbrace_{i\in\mathbb{N}} =
    \lbrace \theta_{i}\rbrace_{i\in\mathbb{N}}
\end{displaymath}
where sequence  $\lbrace \theta_{i}\rbrace_{i\in\mathbb{N}}$ satisfies:
\begin{displaymath}
    \theta_{n}=\sum_{j=0}^{n}{\alpha_{j}\beta_{n-j}}
\end{displaymath}

The $k$-fold convolution of a sequence $\lbrace
\theta_{i}\rbrace_{i\in\mathbb{N}}$ \marginpar{$k$-fold convolution, precisely}
with itself, denoted with  $\left\lbrace
\theta_{i}^{(k)}\right\rbrace_{i\in\mathbb{N}}$, is defined recursively as:
\begin{displaymath}
    \left\lbrace\theta_{n}^{(k+1)}\right\rbrace= \left\lbrace\theta_{n}^{(k)}\right\rbrace\cdot
        \lbrace\theta_{n}\rbrace
\end{displaymath}
with boundary condition $\left\lbrace\theta_{n}^{(0)}\right\rbrace=(1,\underbrace{0,0,\ldots}_{\text{infinitely many zeros}})$ so as:
\begin{displaymath}
    \left\lbrace\theta_{n}^{(1)}\right\rbrace= \left\lbrace\theta_{n}^{(0)}\right\rbrace\cdot
        \lbrace\theta_{n}\rbrace=
        \lbrace\theta_{n}+0\theta_{n-1}+\ldots+0\theta_{0}\rbrace=
        \lbrace\theta_{n}\rbrace
\end{displaymath}
\\\\
Let $\vect{b}=\lbrace b_{i}\rbrace_{i\in\mathbb{N}}$ be a sequence.
The \emph{renewal array} \marginpar{renewal arrays, precisely}
$\mathcal{B}=\lbrace b_{nm}\rbrace_{n,m\in\mathbb{N}}$
generated by sequence $\vect{b}$ is defined as follows:
\begin{displaymath}
    b_{nm}=b_{n-m}^{(m+1)} \quad\wedge\quad n < m\rightarrow b_{nm}=0
\end{displaymath}
For the sake of clarity, let $n\in\mathbb{N}$, then
% the following set should be helpful if subscript index change `n-m'
% hasn't be done in the definition of Renewal array
%let $S_{m}=\mathbb{N}\setminus\lbrace0,\ldots,m-1\rbrace$, then
the following is the sequence of the very first column, where $m=0$:
\begin{displaymath}
    \lbrace b_{n0}\rbrace
        =\lbrace b_{n}^{(1)}\rbrace
        =\lbrace b_{n}\rbrace
\end{displaymath}
the following is the sequence of the second column, where $m=1$:
\begin{displaymath}
    \lbrace b_{n1}\rbrace
        =\lbrace b_{n-1}^{(2)}\rbrace
        =\left\lbrace \sum_{j=0}^{n-1}{b_{j}b_{n-1-j}}\right\rbrace
\end{displaymath}
the following is the sequence of the third column, where $m=2$:
\begin{displaymath}
    \lbrace b_{n2}\rbrace
        =\lbrace b_{n-2}^{(3)}\rbrace
        =\left\lbrace \sum_{i+j+l=n-2}{b_{i}b_{j}b_{l}}\right\rbrace
\end{displaymath}
and, in general, the following is the sequence of column $m\in\mathbb{N}$:
\begin{displaymath}
    \lbrace b_{nm}\rbrace
        =\lbrace b_{n-m}^{(m+1)}\rbrace
        =\left\lbrace \sum_{i_{1}+i_{2}+\ldots+i_{m+1} =n-m}
            {b_{i_{1}}b_{i_{2}}\cdots b_{i_{m+1}}}\right\rbrace
\end{displaymath}
Choose a column index $m\in\mathbb{N}$ and let $B$ a \ac{fps} over sequence $\vect{b}$.
Denote with $B^{(m)}$ a \ac{fps} over the \emph{renewal array}
$\lbrace b_{nm} \rbrace_{n,m\in\mathbb{N}}$, defined by sequence $\vect{b}$,
such that:
\begin{displaymath}
    B^{(m)}(t) = \sum_{n\geq m}{b_{nm}t^{n-m}}
\end{displaymath}
then following relation exists between functions $B$ and $B^{(m)}$:
\begin{displaymath}
    B^{(m)}(t) = B(t)^{m+1}
\end{displaymath}
Consider a sequence\marginpar{$A$-sequence, informally}
$\vect{a}=\lbrace a_{n} \rbrace_{n\in\mathbb{n}}$ and let $A$ be
a \ac{fps} over it:
\begin{displaymath}
    A\left(t\,B(t)\right)
        = \sum_{r\geq 0}{a_{r}\left(t\,B(t)\right)^{r}}
        = \sum_{r\geq 0}{a_{r}B(t)^{r}t^{r}}
        = \sum_{r\geq 0}{a_{r}B^{(r-1)}(t)t^{r}}
\end{displaymath}
by definition of $B^{(r-1)}$ rewrite:
\begin{displaymath}
    \begin{split}
        \sum_{r\geq 0}{a_{r}B^{(r-1)}(t)t^{r}}
            &=\sum_{r\geq 0}{a_{r}\left(\sum_{n\geq r-1}{b_{n,r-1}t^{n-r+1}}\right)t^{r}}\\
            &=\sum_{r\geq 0}{\sum_{n\geq r-1}{a_{r}b_{n,r-1}t^{n+1}}}
    \end{split}
\end{displaymath}
now we tabulate sums respect to index $r$:
\begin{displaymath}
    \begin{split}
        r=0 &\rightarrow \sum_{n\geq -1}{a_{0}b_{n,-1}t^{n+1}}\\
        r=1 &\rightarrow \sum_{n\geq 0}{a_{1}b_{n,0}t^{n+1}}\\
        r=2 &\rightarrow \sum_{n\geq 1}{a_{2}b_{n,1}t^{n+1}}\\
        &\ldots\\
        r=k+1 &\rightarrow \sum_{n\geq k}{a_{k+1}b_{n,k}t^{n+1}}\\
        &\ldots\\
    \end{split}
\end{displaymath}
therefore extracting coefficient of $t^{n+1}$ yields:
\begin{displaymath}
    [t^{n+1}]A(t\,B(t))=\sum_{k\geq 0}{a_{k}\,b_{n,k-1}}
\end{displaymath}
and if \marginpar{$A$-sequence, precisely}function $A$ satisfies
$B(t)=A(t\,B(t))$, which is the same to say:
\begin{equation}
    b_{n+1}=\sum_{k\geq 0}{a_{k}\,b_{n,k-1}}
    \label{eq:A:sequence:for:Renewal:arrays}
\end{equation}
then, sequence $\vect{a}$ is the $A$-sequence for the \emph{renewal array}
$\lbrace b_{nk}\rbrace_{n,k\in\mathbb{N}}$, generated by sequence $\vect{b}$.
A subtle point underlying all this theory is shown in
\autoref{eq:A:sequence:for:Renewal:arrays}: it says that coeffient
$b_{n+1}$, in sequence $\vect{b}$, is a combination
of all coefficients lying on row $n$ of the \emph{Renewal array}
$\lbrace b_{nm}\rbrace_{n,m\in\mathbb{N}}$,\marginpar{a recursion trick}
which is generated by sequence $\vect{b}$ itself!
\\\\
Allow us to generalize a little further. Choose
$s\in\mathbb{N}\setminus\lbrace0\rbrace$ and consider the following:
\begin{displaymath}
    (t\,B(t))^{s-1}\,A\left(t\,B(t)\right)
        = \sum_{r\geq 0}{a_{r}B(t)^{r+s-1}t^{r+s-1}}
        = \sum_{r\geq 0}{a_{r}B^{(r+s-2)}(t)t^{r+s-1}}
\end{displaymath}
by definition of $B^{(r+s-2)}$ rewrite:
\begin{displaymath}
    \sum_{r\geq 0}{a_{r}B^{(r+s-2)}(t)t^{r+s-1}}
        =\sum_{r\geq 0}{a_{r}\sum_{n\geq r+s-2}{b_{n,r+s-2}t^{n+1}}}
\end{displaymath}
finally:
\begin{displaymath}
    =\sum_{r\geq 0}{\sum_{n\geq r+s-2}{a_{r}b_{n,r+s-2}t^{n+1}}}
\end{displaymath}
Now we tabulate sums respect to index $r$, as done in the previous derivation:
\begin{displaymath}
    \begin{split}
        r=0 &\rightarrow \sum_{n\geq s-2}{a_{0}b_{n,s-2}t^{n+1}}\\
        r=1 &\rightarrow \sum_{n\geq s-1}{a_{1}b_{n,s-1}t^{n+1}}\\
        r=2 &\rightarrow \sum_{n\geq s}{a_{2}b_{n,s}t^{n+1}}\\
        r=3 &\rightarrow \sum_{n\geq s+1}{a_{3}b_{n,s+1}t^{n+1}}\\
        &\ldots\\
        r=k+2 &\rightarrow \sum_{n\geq s+k}{a_{k+2}b_{n,s+k}t^{n+1}}\\
        &\ldots\\
    \end{split}
\end{displaymath}
therefore extracting coefficient of $t^{n+1}$ yields:
\begin{displaymath}
    [t^{n+1}](t\,B(t))^{s-1}A(t\,B(t))=\sum_{k\geq 0}{a_{k}\,b_{n,s+k-2}}
\end{displaymath}
and if \marginpar{$A$-sequence, generally}function $A$ satisfies
$B(t)^{s}=(t\,B(t))^{s-1}A(t\,B(t))$, which is the same to say:
\begin{equation}
    \sum_{i_{1}+\ldots+i_{s}=n+1}{b_{i_{1}}\cdots b_{i_{s}}}
        =b_{n+1}^{(s)}
        =b_{n+s,s-1}
        =\sum_{k\geq 0}{a_{k}\,b_{n,s+k-2}}
    \label{eq:generalized:A:sequence:for:Renewal:arrays}
\end{equation}
then, sequence $\vect{a}$ is the (\emph{generalized}) $A$-sequence for the
\emph{renewal array} $\lbrace b_{nk}\rbrace_{n,k\in\mathbb{N}}$, generated by
sequence $\vect{b}$ \marginpar{a dependency within array $\lbrace
b_{nm}\rbrace_{n,m\in\mathbb{N}}$}.

Fixing $s=1$, we go back to result shown in
\autoref{eq:A:sequence:for:Renewal:arrays}.


\chapter{Other}

\section{Greek Alphabet}

\begin{tabbing}
	\hspace{2em} \= \hspace{3em} \= $N$ = Yet another statement \= \kill

	A           \>  $\alpha$                 \>  alpha    \\
	B           \>  $\beta$                  \>  beta     \\
	$\Gamma$    \>  $\gamma$                 \>  gamma    \\
	$\Delta$    \>  $\delta$                 \>  delta    \\
	E           \>  $\epsilon \ \varepsilon$ \>  epsilon  \\
	Z           \>  $\zeta$                  \>  zeta     \\
	H           \>  $\eta$                   \>  eta      \\
	$\Theta$    \>  $\theta \ \vartheta $    \>  theta    \\
	I           \>  $\iota$                  \>  iota     \\
	K           \>  $\kappa$                 \>  kappa    \\
	$\Lambda$   \>  $\lambda$                \>  lambda   \\
	M           \>  $\mu$                    \>  mu       \\
	N           \>  $\nu$                    \>  nu       \\
	$\Xi$       \>  $\xi$                    \>  xi       \\
	O           \>  o                        \>  omicron  \\
	$\Pi$       \>  $\pi \ \varpi $          \>  pi       \\
	P           \>  $\rho$                   \>  rho      \\
	$\Sigma$    \>  $\sigma \ \varsigma $    \>  sigma    \\
	T           \>  $\tau$                   \>  tau      \\
	$\Upsilon$  \>  $\upsilon$               \>  upsilon  \\
	$\Phi$      \>  $\phi \ \varphi $        \>  phi      \\
	X           \>  x                        \>  chi      \\
	$\Psi$      \>  $\psi$                   \>  psi      \\
	$\Omega$    \>  $\omega$                 \>  omega    \\

\end{tabbing}


\documentclass{article}
    \usepackage{subcaption}
    \usepackage{amsmath, amssymb}
    \usepackage{graphicx, float}
    \usepackage[hidelinks]{hyperref}
    \usepackage[bottom]{footmisc}
    \usepackage[margin=.8in, tmargin=.8in]{geometry}
    \usepackage{esint} % for double line integrals
    \usepackage{xcolor} % for color macros
    \usepackage{tikz}
    \usepackage{algorithm, algpseudocode}
    \newcommand{\citep}[2][]{(\cite[#1]{#2})}

    \usetikzlibrary{arrows,decorations.markings}
    \usepackage[
        backend=biber,
        % style=alphabetic,
        citestyle=authoryear,
        sorting=ynt,
        bibencoding=utf8
    ]{biblatex}

    \bibliography{bibliography}

    \renewcommand{\baselinestretch}{1.2}
    \newcommand{\eps}{\epsilon}
    \newcommand{\der}{\partial}
    \newcommand{\del}{\nabla}
    \newcommand{\bm}[1]{\mathbf{#1}}
    % \newcommand{\vf}[1]{\vec{\mathbf{#1}}}
    \newcommand{\vf}[1]{\mathbf{#1}}
    \newcommand{\norm}{\hat{\bm{n}}}
    \newcommand{\normal}{\mathcal{N}}
    \newcommand{\var}{\text{var}}
    \newcommand{\bx}{\vf{x}}
    \newcommand{\by}{\vf{y}}
    \newcommand{\bz}{\vf{z}}
    \newcommand{\bw}{\vf{w}}
    \newcommand{\bfu}{\vf{f}}
    \newcommand{\giv}{\ |\ }
    \newcommand{\bmu}{\pmb{\mu}}
    \newcommand{\gauss}{\mathcal{N}}
    \newcommand{\data}{\mathcal{D}}
    \newcommand{\model}{\mathcal{M}}
    \DeclareMathOperator*{\argmax}{arg\,max}
    \DeclareMathOperator*{\argmin}{arg\,min}

    \newcommand{\id}{\mathbb{I}}
    \newcommand{\de}{\text{d}}
    \newcommand{\tran}{\text{T}}
    \newcommand{\expect}{\mathbb{E}}
    \newcommand{\kl}{D_{KL}}
    \newcommand{\post}{P(\bw \giv \data)}

    \newcommand{\red}[1]{\textcolor{red}{#1}}
    \newcommand{\blue}[1]{\textcolor{blue}{#1}}

    \setlength\parindent{0pt}
    \captionsetup{justification=centering}

    \title{Machine Learning and Pattern Recognition}
    \date{\today}
    \author{Traiko Dinev \textless traiko.dinev@gmail.com\textgreater}

\begin{document}

\maketitle
\textit{NOTE: This partially follows Machine Learning and Pattern Recognition, a masters level course at the University of Edinburgh.}

\textit{NOTE: Note this "summary" is NOT a reproduction of the course materials nor is it copied from the corresponding courses. It was entirely written and typeset from scratch.}

\textit{License: Creative Commons public license; See README.md of repository}

\section{Linear Regression}
Consider the vector of features $\bm{x} = \langle x_1, x_2, \dots, x_D \rangle$, where $D$ is the number of dimensions. Then we want to learn this transformation (i.e. learn $\bm{w}$):

\begin{equation}
    \bm{f} = X \bm{w} + b
\end{equation}

where X is a $N\times D$ matrix, containing $N$ examples of $D$ dimensions. We have $N$ targets, our training vector $y$ of size $N\times1$. This corresponds to computing $\bm{w}^T \bm{x}^{(i)}$, i.e. the dot product between the weights and training examples for each $i$. We can use mean squared error:

\begin{align*}
    \min_{\bm{w}} \quad& \sum_i (y^{(i)} - f^{(i)})^2
        = (\bm{y} - \bm{f})^T (\bm{y} - \bm{f})
\end{align*}

This has a linear solution of the form $\bm{w} = \underbrace{(X^TX)^{-1}X^T}_\text{\blue{pseudo-inverse}} \bm{y}$. We can also introduce basis functions $phi$:
\vskip 0.1in

\begin{align*}
    f(\bm{x}) &= \sum_k w_k\ \phi_k(\bm{x}) \\
    \phi(\bm{x}) &= \exp(-(\bm{x} - \bm{c})^T (\bm{x} - \bm{c}) / h^2)
        && \text{\blue{Radial Basis Function}} \\
    \sigma(\bm{x}) &= \frac{1}{\exp(-\bm{v}^T \bm{x} - b)}
        && \text{\blue{Sigmoid}}
\end{align*}

\subsection{Regularization}
We can add L2 regularization, which penalizes big weights.

\begin{equation}
    \hat{E}(\bm{w}) = E(\bm{w}) + \lambda \bm{w}^T\bm{w}
\end{equation}

which we can solve analytically to get something similar:

\begin{equation}
    \bm{w}^* = (X^TX + \lambda \id)^{-1}X^T \bm{y}
\end{equation}

Alternatively, we can augment the $\bm{y}$ vector and $X$ matrix to obtain the same results:

\begin{align*}
    \bm{y}' =
        \begin{pmatrix}
            \bm{y} \\
            \bm{0}_k
        \end{pmatrix} \\
    \Phi' =
        \begin{pmatrix}
            \Phi \\
            \sqrt{\lambda} \id_k
        \end{pmatrix}
\end{align*}

\section{Error Bars}
We normally train on a \textit{training set}, find best hyperparameters (like the L2 coefficients) on a \textit{validation set} and judge only once on a \textit{test set}.

Now let's say we compute some error measure or loss $L(y, f(x))$, like the mean squared error above. If we have a test set, we can compute the mean of this error:

\begin{equation}
    L_\text{test} = \frac{1}{M} \sum_{m=1}^M L(y_m, f(x_m)) = \frac{1}{M} \sum_m L_m
\end{equation}

This is a sum of test errors, which means that we can \textit{most of the time} apply the \textbf{Central Limit Theorem} and say that $L_\text{test}$ is approximately Gaussian. We need to have a finite variance and all the $x_m$'s need to be independent, but we as things in ML often are, we assume things work until proven otherwise.

\vskip 0.1in
The $x$'s actually come from an \textbf{input distribution}, which we don't know. That doesn't stop us from computing its mean and variance from independent samples (different than the sets above):

\begin{align*}
    \mu &\approx \bar{x} = \frac{1}{N} \sum_n x_n \\
    \sigma^2 &\approx \bar{\sigma}^2 = \frac{1}{N - 1} \sum_n (x_n - \bar{x})^2
\end{align*}

Now we can also compute the variance of the mean. If we collected a lot of sets, we can compute the variance:

\begin{align}
    \var[\bar{x}] &= \frac{1}{N^2} \sum_i \var[x_n] = \frac{\sigma^2}{N}
        \approx \frac{\bar{\sigma}^2}{N}
\end{align}

Using the rules of variance (google variance of a sum). We can apply the same reasoning to the errors, where we replace $x_n$ above by $L(x_n, f(x_n))$. This combined with the central theorem reasoning gives us a "confidence interval" like so:

\begin{equation}
    \mu \pm \hat{\sigma} / \sqrt{N}
\end{equation}

TL;DR: Report standard error bars.

\section{Normal Distributions}
A random variable is Gaussian (normal) if it has the following pdf:

\begin{equation} \label{eq:normal1d}
    p(y) = \mathcal{N}(y; \mu, 1) = \frac{1}{\sqrt{2 \pi}} e^{-\frac{1}{2}(y - \mu) ^ 2}
\end{equation}

Standard normals have a mean of 0. We can scale and shift a standard normal like so:

\begin{align*}
    z &= \sigma x + \mu \\
    x &= \frac{z - \mu}{\sigma}
\end{align*}

We then substitute the above in \autoref{eq:normal1d} and scale the PDF to be normalized:

\begin{equation}
    p(z) = \mathcal{N}(z; \mu, \sigma) = \frac{1}{\sigma \sqrt{2 \pi}} e^{-\frac{1}{2\sigma^2}(z - \mu) ^ 2}
\end{equation}

\subsection{Multivariate Gaussians}
The joint probability density of $D$ univariate Gaussians will give us a multivariate one:

\begin{equation}
    p(\vf{x}) = \prod_d p(x_d) = \frac{1}{(2\pi)^{D/2}} e^{-\frac{1}{2} \vf{x}^T \vf{x}}
\end{equation}

This has a diagonal covariance matrix ($\vf{x}^T \vf{x}$). If we transform a variable as the one above:

\begin{equation}
    \vf{y} = A \vf{x}
\end{equation}

Then we can show that $\text{cov}[\vf{y}] = \Sigma = AA^T$. Also, assuming A is invertible, $\vf{x} = A^{-1} \vf{y}$. Substituting and normalizing, we obtain:

\begin{equation}
    p(\vf{y}) = {|2\pi\Sigma}|^{-1/2} e^{-\frac{1}{2} \vf{y}^T \Sigma^{-1} \vf{y}}
\end{equation}

Shifting the distribution ($\vf{z} = \vf{y} + \vf{\mu}$), we obtain:

\begin{equation}
    p(\vf{z}) = {|2\pi\Sigma}|^{-1/2} e^{-\frac{1}{2} (\vf{z} - \vf{\mu})^T \Sigma^{-1} (\vf{z} - \vf{\mu})}
\end{equation}

Covariance matrices need to be \textbf{positive semi-definite}. This ensures that the above matrix is invertible and the hence the transformation doesn't "reduce" dimensionality:

\begin{equation}
    \vf{z}^T \Sigma \vf{z} \geq 0, \text{for all real real}\ \vf{z}
\end{equation}

\section{Simple Classifiers}
In classification targets $y^{(n)}$ are either $\{0, 1\}$ or $\{-1, +1\}$. Always 1-hot encode categorical variables: $\vf{x} = [0\ 1\ 0\ 0\ \dots]^T$.

\subsection{Generative Normal Models}
We can assume that features come from a normal distribution for each class:

\begin{equation*}
    p(\bx | y=k) = \gauss(\bx, \bmu_k, \Sigma_k)
\end{equation*}

We can fit the mean and covariance to the mean and covariance of the vectors. Using Bayes' rule, we can get a probability for a label belonging to class $k$:

\begin{align*}
    P(y=k | \bx) &\propto p(\bx | y = k) P(y = k) \\
                 &\propto \gauss(\bx; \mu_k, \Sigma_k) \pi_k
\end{align*}

To get priors, we can just count:

\begin{equation*}
    P(y = k) = \pi_k \approx \frac
        {\Sigma_n I(y^{(n)} = k)}
        {N}
\end{equation*}

At test time we just compute probabilities for each class, normalize and hey-presto, classifier:

\begin{equation*}
    P(y = k | \bx^{(\text{test})}, \theta) = \frac
        {s_k}
        {\sum_{k'} s_{k'}}
\end{equation*}

where $\theta = \{\bmu_k, \Sigma_k, \pi_k\}$, the learned parameters and $s_k = P(y = k | \bx)$ as above.

\subsection{Naive Bayes}
If the features are discrete, we can assume they are independent (big, very wrong assumption) and fit a model like this:

\begin{equation*}
    P(\bx | y = k, \theta) = \prod_d P(x_d | y = k; \theta)
        = \prod_d \theta_{d, k}^{x_d} (1 - \theta_{d, k})^{1 - x_d}
\end{equation*}

Here $\theta_{d,k} = P(x_d = 1| y = k)$.

\vskip 0.1in
We can also make the Normal Model above "naive" by making the covariance matrices diagonal, ignoring dependence (covariances) between features.

\begin{figure}
    \centering
    \begin{subfigure}{0.4\textwidth}
        \centering
        \includegraphics[width=\textwidth]{figures/normal_class}
        \label{fig:udp_header}
        \caption{A normal classifier}
    \end{subfigure}
    %
    \centering
    \begin{subfigure}{0.4\textwidth}
        \centering
        \includegraphics[width=\textwidth]{figures/normal_class_naive}
        \label{fig:tcp_header}
        \caption{A "naive" normal classifier. The multivariate Gaussian has no diagonal covariance, hence it can only stretch, but not rotate.}
    \end{subfigure}
\end{figure}

\section{Logistic Regression}
\subsection{Gradient Descent}
From the chain rule, we have $\frac{dw}{dt} = w_x \frac{dx}{dt} + w_y \frac{dy}{dt} + w_z \frac{dz}{dt}$. In vector notation, this becomes $\frac{dw}{dt} = \nabla w \cdot \frac{\vf{r}}{dt}$, where $\vf{r}$ is a curve in 3 dimensions. Thus:

\begin{equation*}
    \nabla w = \langle w_x, w_y, w_z \rangle
\end{equation*}

The gradient vector's values depend on the point where we evaluate it. It is also perpendicular to a level surface $w = c$ by the following proof:
\vskip 0.1in

Proof: For any curve $\vec{r} = \vec{r}(t)$ that is inside the level surface $w = c$, the velocity is in the tangent plane. This follows from the fact that derivatives are tangents to the function curve from 1-D calculus. Then by the chain rule above $\frac{dw}{dt} = \nabla w \cdot \frac{\vec{r}}{dt} = \frac{c}{dt} = 0$. Hence the gradient is perpendicular to the surface.
\vskip 0.1in

\textbf{Directional derivatives}. These are derivatives in the direction of some vector $\hat{\mathbf{u}}$. Treat $\frac{\partial w}{\partial x}$ as the derivative along the $x$-axis. In other words, this is a slice of the function as given by the $x$-plane. Now let's have a curve $\vec{r}(t)$, such that it has a \textit{normal} velocity $\hat{\mathbf{u}} = \langle a, b \rangle$. We can assume $|\hat{\mathbf{u}}| = 1$, since we are only interested in the direction. In other words:

\begin{equation*}
    \begin{cases}
        x(s) &= x_0 + as \\
        y(s) &= y_0 + bs
    \end{cases}
\end{equation*}

Now $s$ is the change in the direction of the curve. We can ask then what $\frac{\partial w}{\partial s}$ is. Well, by the chain rule:

\begin{equation*}
    \frac{\partial w}{\partial s} = \nabla w \cdot \frac{d\vf{r}}{ds} = \nabla w \cdot \hat{\mathbf{u}}
\end{equation*}

But from algebra we know:

\begin{equation*}
    \frac{\partial w}{\partial s} = \nabla w \cdot \hat{\mathbf{u}} = |\nabla w| |\hat{\mathbf{u}}| \cos{\theta} = |\nabla w| \cos{\theta}
\end{equation*}

This is maximized when $\cos{\theta} = 1$, i.e. when the gradient $\hat{\mathbf{u}}$ is in the direction of $\nabla w$. Hence the gradient points to the direction of steepest increase.

\subsection{So what?}
Patience. This can be applied to machine learning with the following idea. We normally define a loss or cost function $L$ (or $J$ as often seen in statistics). This is a function of our weights/ parameters. We usually aim to minimize said cost function. Well, if we know the direction of its steepest increase, $\nabla_{\vf{w}} L$, we know the direction of its steepest decrease - $-\nabla_{\vf{w}} L$. Hence we can take small steps in that direction:

\begin{equation*}
    \vf{w} \leftarrow \vf{w} - \eta \nabla_{\vf{w}}
\end{equation*}

\subsection{Logistic Regression.. finally}
\begin{figure}
    \centering
    \includegraphics[width=0.4\textwidth]{figures/600px-Logistic-curve.png}
    \caption{Logistic Sigmoid (from wikipedia)}
    \label{fig:sigmoid}
\end{figure}

We want to classify things, which we can interpret as computing the probability $P(y = 1 | \bx, \bw)$. Enter the sigmoid:

\begin{equation*}
    f(\vf{x}; \vf{w}) = \sigma(\vf{w}^T \vf{x}) = \frac{1}{1 + e^{\vf{w}^T \vf{x}}}
\end{equation*}

See \autoref{fig:sigmoid}. We also need to define our loss function. We define another \textbf{super-duper important} concept in ML - the likelihood of the data given the weights:

\begin{align*}
    \text{Likelihood} &= P(\mathcal{D} | \vf{w}) =
        P(\vf{x}^{(1)}, y^{(1)}, \dots, \vf{x}^{(N)}, y^{(N)}\ |\ \vf{w}) \\
    &= \underbrace{\prod_i P(\vf{x}^{(i)}, y^{(i)} |\ \vf{w})}_{\text{\blue{independent x's}}} = \prod_i P(y^{(i)} |\ \vf{x}^{(i)}, \vf{w}) P(\vf{x}^{(i)}\ |\ \vf{w}) \\
    &= \prod_i P(y^{(i)} |\ \vf{x}^{(i)}, \vf{w}) \underbrace{P(\vf{x}^{(i)})}_{\text{\blue{ignore weights}}} = \prod_i P(y^{(i)} |\ \vf{x}^{(i)}, \vf{w})
\end{align*}

We \blue{ignore the weights}, since our model doesn't tell us anything about the input distribution. I.e. no matter the weights, the probability of having $\bx$ will remain $1$. Another way of saying this is that our model is \textbf{discriminative} as opposed to \textbf{generative}. We don't "generate" anything, everything has a probability $1$ of being generated. \footnote{\textit{I know this is sloppy but I've looked and I haven't found a better (mathematically rigorous) explanation for this. Suggestions are welcome.}}
\vskip 0.1in

We often minimize the negative \textbf{log-likelihood}, in this case primarily because it transforms the product into a sum. With some careful math, we can calculate it and fit logistic regression models.

\subsection{Softmax Regression}
If there are $K$ classes, we can one-hot encode them as:

\begin{equation*}
    \vf{y} = [0\ 0\ 0\ \dots\ 0\ 1\ 0\ \dots\ 0]
\end{equation*}

where we have a $1$ for the target class. We fit $K$ weights, $\vf{w}^{(k)}$, such that our output contains $K$ probabilities, for each class:

\begin{equation*}
    f_k = \frac{
        {e^{(\vf{w}^{(k)})}}^T \vf{x}
    }{
        \sum_{k'} {e^{(\vf{w}^{(k')})}}^T \vf{x}
    }
\end{equation*}

where we normalize by the sum of all outputs so that we get a proper probability distribution. We can use batch gradient descent to minimize the negative log-likelihood. The likelihood would be the following then:

\begin{align*}
    \nabla_{\vf{w}}^{(k)}\ \text{LL} &= \nabla_{\vf{w}^{(k)}} \log \prod P(y_c = 1\ |\ \vf{x}, W) \\
            &= \sum_n \nabla_{\vf{w}^{(k)}} \log P(y_c = 1\ |\ \vf{x}, W) \\
            &= \sum_n (\vf{y}_k^{(n)} - f_k(\vf{x}^{(n)})) \vf{x}^{(n)}
\end{align*}

\subsection{Robust Regression}
Sometimes we get outliers in the data we cannot filter in the pre-processing stage. We assume that for each observation there is a probability it isn't part of the data. Or, in math terms:

\begin{equation*}
    P(m)\ =\ \text{Bernoulli}(m; 1 - \epsilon) =
        \begin{cases}
            1 - \epsilon & \text{m = 1} \\
            \epsilon     & \text{m = 0}
        \end{cases}
\end{equation*}

then we pick a random label when $m = 0$:

\begin{equation}
    P(y = 1\ |\ \vf{x}, \vf{w}, m) =
        \begin{cases}
            \sigma(\vf{w}^T\vf{x}) & m = 1 \\
            \frac{1}{2}              & m = 0
        \end{cases}
\end{equation}

The likelihood for a single example becomes:

\begin{align*}
    P(y = 1\ |\ \vf{x}, \vf{w}) &= \sum_m
        P(y = 1, m\ |\ \vf{x}, \vf{w}) \\
    &= \sum_m P(y = 1\ |\ \vf{x}, \vf{w}, m) P(m) \\
    &= (1 - \epsilon) \sigma(\vf{w}^T\vf{x}) + \frac{\epsilon}{2}
\end{align*}

We can then derive the gradient for this and maximize the likelihood. We can treat $\epsilon$ as a hyperparameter and optimize over a validation set or set $\epsilon = \sigma(b)$ and optimize $b = \text{logit}(\epsilon) = \log \frac{\epsilon}{1 - \epsilon}$.

\section{Neural Networks}
\label{sec:nn_intro}
\textit{This section is taken from by Bachelor's thesis, so it might not read very well in a summary kind of way.}
\vskip 0.1in

Neural networks are universal function approximators inspired by biological neurons (e.g. \cite[Chapter~6]{goodfellow_deep_2016}). The main building blocks of neural networks are called \textit{neurons} and the networks are often drawn as directed graphs, as in figure \ref{fig:neural_net}. This graph illustrates the \textit{flow} of the network, from the \textit{inputs} $x_1, x_2, x_3$ to the \textit{output} $y$. Each circle in the figure is called a neuron and each of the three vertical slices is called a layer. The layers that are not the input or output layers are called \textit{hidden} layers.

\begin{figure}
    \centering
    \begin{tikzpicture}
        \tikzset{myptr/.style={decoration={markings,mark=at position 1 with %
        {\arrow[scale=2,>=stealth]{>}}},postaction={decorate}}}

        % left row
        \draw (0, 0) circle (1) node {$x_3$};
        \draw (0, 2.5) circle (1) node {$x_2$};
        \draw (0, 5) circle (1) node {$x_1$};

        % middle row
        \draw (3, 1) circle (1) node {$h_2$};
        \draw (3, 4) circle (1) node {$h_1$};

        % right row
        \draw (6, 2.5) circle (1) node {$y$};

        % lines
        % to h_2
        \draw[myptr] (1, 0) -- (2, 1);
        \draw[myptr] (1, 2.5) -- (2, 1);
        \draw[myptr] (1, 5) -- (2, 1);
        % to h_1
        \draw[myptr] (1, 0) -- (2, 4);
        \draw[myptr] (1, 2.5) -- (2, 4);
        \draw[myptr] (1, 5) -- (2, 4);
        % to y
        \draw[myptr] (4, 1) -- (5, 2.5);
        \draw[myptr] (4, 4) -- (5, 2.5);
    \end{tikzpicture}

    \caption[Neural network example]{Neural Network illustration. The inputs are $x_1, x_2, x_3$, the hidden layers are $h_1, h_2$ and $y$ is the output.}
    \label{fig:neural_net}
\end{figure}

In what is called the \textit{supervised learning} scenario, the neural network is trained to predict the outputs $y$ given the inputs $\mathbf{x}$ (see next section). If $y$ is real-valued, then the task is called \textit{regression} and is what we will attempt to do in the context of ABC.
\vskip 0.1in

We can define the transformations equivalently from the inputs to the hidden layer and from the hidden units to output(s) via a system of equations:
%
\begin{align}
    \mathbf{h} &= W^{(1)}\ \mathbf{x} \\
    y &= W^{(2)}\ \mathbf{h}
\end{align}

Thus we can increase or decrease the size of the input vector in subsequent layers of the network depending on the shapes of the  matrices $W^{(1)}$ and $W^{(2)}$.
\vskip 0.1in

We apply a piecewise non-linearity called the \textit{activation function} after each matrix multiplication, i.e. at each neuron; without it, the model could be collapsed to a single layer:

\begin{align}
    \mathbf{h} &= h^{(1)}(W^{(1)}\ \mathbf{x}) \\
    y &= h^{(2)} (W^{(2)}\ \mathbf{h})
\end{align}

Choices for activation functions $h(\mathbf{x})$ include hyperbolic tangent ($\tanh$), sigmoids ($\sigma(x)$) and rectified linear units (relu):

\begin{align}
    \text{tanh}(x) &= \frac{e^x - e^{-x}}{e^x + e^{-x}} \\
    \sigma(x) &= \frac{1}{1 + e^{-x}} \\
    \text{relu}(x) &= \text{max}(0, x)
\end{align}

Additionally, parametric rectifier units allow the activation to be non-zero (and hence, the derivative as well):

\begin{equation}
    \text{prelu}(x) = \begin{cases}
        x       & \text{if}\ x > 0 \\
        a\ x    & \text{otherwise}
    \end{cases}
\end{equation}

The parameter $a$ is learned akin to the rest of the neural network parameters.
\vskip 0.1in

Rectified (and parametric) linear units generally learn faster than $\tanh$ and sigmoid units and are preferred in state-of-the-art applications \citep{glorot_deep_2011}. One improvement we will make is to replace the previously used $\tanh$ activations with relu units.
\vskip 0.1in

We can think of the hidden layers of the network as \textit{learned} feature transformations. If the outputs $y$ are real-valued and unbounded, i.e.\ in a regression task, we normally use a linear activation for the final layer, so we can think of the outputs from the previous hidden layer as features or \textit{basis} functions for the linear output neuron.

\begin{equation}
    y = W^{{K - 1}}\ \mathbf{h}^{(K - 1)}
\end{equation}

where we have K layers, including the output.

\subsection{Training Neural Networks}
\label{sec:nn_train}

To train the neural network, we use \textit{backpropagation} \citep{rumelhart_learning_1985}. More specifically, we use more optimized versions of backpropagation, namely \textit{Adam} \citep{kingma_adam:_2014} and \textit{RMSProp} \citep{tieleman_t._and_hinton_g._e._lecture_2012}.
\vskip 0.1in

Before we train, we specify a \textit{loss function}. The loss function gives us a measure of "how bad" it is to differ from the true output $y$, where $\hat{y}$ is the output of the neural network. We then train on a set of examples $\{\mathbf{x}^{(i)}, \mathbf{y}^{(i)}\}_{i = 0}^{N}$ via backpropagation. For a regression task, we use the mean-squared error loss function:

\begin{equation}
    \mathcal{L}(y, \hat{y}) = \frac{1}{N} \sum_{i = 1}^{N} (y^{(i)} - \hat{y}^{(i)})^2
\end{equation}

Backpropagation computes the derivative of the weight matrices $W^{(i)}$ with respect to the loss function for a \textit{batch} of training examples from the training set. It then moves the weight in the direction of the negative gradient, thus minimizing the loss function. The backpropagation algorithm takes as a parameter a \textit{learning rate}, which can be optimized. There are also more advanced versions, which often perform better, such as Adam \citep{kingma_adam:_2014} and RMSProp \citep{tieleman_t._and_hinton_g._e._lecture_2012}.

\begin{algorithm}
    \caption{Basic Backpropagation Algorithm}
    \begin{algorithmic}[1]
        \State initialize learning rate $\eta$

        \While{Error is still high (see Early Stopping)}
            \State $\mathbf{w} \gets \mathbf{w} - \eta \nabla \mathbf{w}$
            \Comment Update weights
        \EndWhile
    \end{algorithmic}
\end{algorithm}

\subsection{Modern Deep Neural Networks}
\label{subsec:nn_modern}

Using the simple backpropagation algorithm and neural networks introduced so far does not scale well to larger models and datasets \citep[Chapter 7]{goodfellow_deep_2016}. We can employ a number of techniques to improve the learning procedure. We describe here Batch Normalization \citep{ioffe_batch_2015}, Early Stopping \citep{prechelt_early_2012} and Dropout \citep{srivastava_dropout:_2014}, as well as \textit{regularization} in general.
\vskip 0.1in

A problem with larger networks is \textit{overfitting}. Normally, we train the network on a training set and then evaluate the performance on a separate \textit{validation set}, unseen during training. If the network learned the underlying structure of the data, then we say it \textit{generalizes} and we expect the loss on the validation set to be comparable to that on the training set. Networks with more free parameters (weights) as compared to the number of training examples can, however, learn to "remember" quirks in the training instances instead of learning such an underlying structure. In this thesis, in the context of ABC, we have the benefit of procedurally generating the data, so we do not have issues with the number of examples; we can have as many as needed.
\vskip 0.1in

\textbf{Early Stopping} is a relatively easy way to prevent overfitting \citep{prechelt_early_2012}. Consider figure \ref{fig:overfitting}. The training error normally will approach a very small number if the model has enough capacity. The validation error, however, usually starts increasing after it reaches a (sometimes local) minimum. If we continue to train, we no longer learn useful information, but rather quirks in the training set. Thus early stopping monitors the validation error at each epoch and stops training if the validation error continues to \textit{increase} for a number of epochs. We then revert to the best epoch so far.
\vskip 0.1in

\textbf{Batch Normalization} forces the activations of different layers in the network to be normalized \citep{ioffe_batch_2015}. This has been shown to help with poor initialization strategies and large input values by forcing outputs of layers to take on the same distribution, thus helping learning. Using batch normalization after each (fully-connected) layer helps the network by eliminating the need to learn the different output distributions of each layer. The following transformation is used after applying the activation functions on each layer:

\begin{align}
    \hat{z}_i &= \frac{h_i - \mu_i}{\sqrt{\sigma_i^2 + \epsilon}} \\
    h_i &= \gamma_i\ \hat{z}_i + \beta_i
\end{align}

Here $h_i$ are the outputs of each hidden unit and $\mu_i$ and $\sigma_i$ are minibatch mean and standard deviation, computed over training examples. $\gamma_i$ and $\beta_i$ are \textit{learned} scaling parameters and $\epsilon$ is a small value to prevent division by 0.
\vskip 0.1in

\textbf{Dropout} is another simple regularization technique \citep{srivastava_dropout:_2014}, that can prevent overfitting. During training we disconnect each neuron with a certain probability. This forces the other neurons to learn when some of the features are not present, thus making the network more flexible. At test time all the neurons are used and the outputs are scaled accordingly.
\vskip 0.1in

\textbf{L2 Regularization} is another way to prevent overfitting. We add a penalty term to the loss function for the entire neural network consisting of the squared absolute value of the weights for each layer. This forces the weights of the network to be small, thus ensuring the gradient updates are also small. It also prevents the activations from \textit{exploding}.

\begin{equation}
    \mathcal{\hat{L}}(y, \hat{y}, \{W^{(i)}\}_{i = 1}^{K}) = \frac{1}{N} \sum_{i = 1}^{N} (y^{i} - \hat{y}^{i})^2 + \sum_{i,j,k} \lambda (W_{i,j}^{(k)})^2
\end{equation}

where $\lambda$ is the regularization strength.

\begin{figure}
    \centering
    \includegraphics[width=0.7\textwidth]{figures/overfitting_early_stopping}
    \caption[Overfitting illustration]{Neural Network overfitting illustration. The training error almost reaches $0$, but the validation error starts to increase as the networks begins to learn structure present \text{only} in the training set.}
    \label{fig:overfitting}
\end{figure}

\textbf{Note} for all neural network models used in this thesis both the input data and the targets were standardized. This helps learning by avoiding large values for any neuron and therefore large gradients that become unstable:

\begin{equation}
    \hat{x}\ =\ \frac{x - \langle x \rangle}{\sqrt{\text{Var}[x]}}
\end{equation}

\section{Autoencoders and PCA}
\subsection{Autoencoders}
Autoencoders learn transformations or representations of data. The basic idea is to create a bottleneck somewhere in the middle of the network so that networks learn to represent the data in a smaller number of dimensions. Thus autoencoders are functions that return the input approximately:

\begin{equation}
    f(\bx) \approx \bx
\end{equation}

We can use a neural network with a hidden size less than the input, like so:

\begin{align*}
    \vf{h} &= g^{(1)}(W^{(1)} \bx + \vf{b}^{(1)}) \\
    \vf{f} &= g^{(2)}(W^{(2)} \vf{h} + \vf{b}^{(1)}) \\
\end{align*}

where $W^{(1)}$ is $K \times D$ and $K << D$. We can additionally impose a regularization penalty to further constrain the network. Sparse autoencoders, for instance, impose an L1 regularization. More about why that drives weights to zero below.
\vskip 0.1in

Denoising autoencoders learn to reconstruct the data when some of the inputs are missing or swapped or with added noise. Those are useful for removing noise from real-world images.

\subsection{Principal Component Analysis (PCA)}
We compute the covariance matrix of the data $\Sigma = \frac{1}{N} X^T X$. We want to reduce dimensionality (from $D$ to $K$) in a way that maximizes the variance along the $K$ new axis. This can be done by computing the eigenvectors of the covariance matrix and then projecting the data along those axis. These eigenvectors are also orthogonal to each other; if we keep all dimensions (i.e. project onto all eigenvectors), no information is lost. This process is also equivalent to a linear autoencoder with a square loss function (TODO: proof).
\vskip 0.1in

Often we standardize the data (i.e. $\frac{x - \mu}{\sigma}$) which means we can omit the ${1}{N}$ in the calculation above.

\subsection{Singular Value Decomposition}
Or SVD decomposes an $N\times D$ matrix into a product of a $N\times K$ matrix $U$, a diagonal $K\times K$ matrix $S$ and $K\times D$ matrix $V^T$:

\begin{equation*}
    X \approx USV^T
\end{equation*}

The columns of $U$ contain the eigenvectors of $X X^T$ and the rows of $U$ and the columns of $V^T$ give $K$-dimensional embeddings of the rows and columns of $X$, respectively.

\subsection{Probabilistic PCA}
We want to embed the data $X = \{\vf{x}^{(n)}\}$ into a $K$-dimensional space, such that $K < D$. Let's assume that each point is embedded into a latent variable $\vf{z}^{(n)}$. We assume that the latent variables are normally distributed:

\begin{equation*}
    \vf{z}^{(n)} \sim \mathcal{N}(\vf{0}, \mathbb{I})
\end{equation*}

We also assume that the original data is generated via a projection:

\begin{equation}
    \vf{x}^{(n)}\ |\ \vf{z}^{(n)} \sim \mathcal{N}(W\vf{z}^{(n)}, \sigma^2 \mathbb{I})
\end{equation}

where the matrix $W$ is $D\times K$. $\sigma$ is known as the noise term. We are interested in estimating $W$ and $\sigma$. Marginalizing out the $\vf{z}^{(n)}$, we arrive at:

\begin{equation}
    \vf{x}^{(n)} \sim \mathcal{N}(\vf{0}, WW^T + \sigma^2 \mathbb{I})
\end{equation}

This is then fitted via e.g. maximum likelihood.

\section{Bayesian Modeling}
We can adopt a probabilistic model for regression. Our training data consists of pairs $(\bx, y)$. We can assume a simple model:

\begin{equation*}
    P(y\ |\ \bx, \bw) = \mathcal{N}(y; f(\bx, \bw), \sigma_y^2)
\end{equation*}

where $f(\bx, \bw)$ is any function. A quadratic regression would have a quadratic expansion, for example. We can then fit using maximum likelihood or in practice by minimizing the negative log-likelihood:

\begin{align*}
    - \log P(\by \giv X, \bw) &= - \sum_n logP(y^{(n)} \giv \bx^{(n)}, \bw) \\
        &= \frac{1}{2 \sigma_y^2} \sum_n \big[  (y^{(n)} - f(\bx^{(n)}, \bw)) \big] + \frac{N}{2} \log(2 \pi \sigma_y^2)
\end{align*}

which is equivalent to fitting by least squares. Neat.

\section{Bayesian Regression}
\begin{figure}
    \centering
    \includegraphics[width=0.8\textwidth]{figures/bayes_regression}
    \caption{Example Bayesian Regression with basis functions. The left graph has fits sampled from the prior. The shaded region in the right graph is centered at the posterior mean with one standard deviation plotted. The red line is the mean fit. The bars on each datapoint represent the noise $\sigma_y$.}
    \label{fig:bayes_regression}
\end{figure}

We need to set-up two things - a probabilistic model, like the one above, and a prior belief for our model. A simple way to do this would be as follows:

\begin{align*}
    P(\bw) &= \mathcal{N}(\bw; \vf{0}, \sigma_w^2 \mathbb{I}) \\
    P(y\ |\ \bx, \bw) &= \mathcal{N}(y; \bw^T \phi(\bx^{(n)}), \sigma_y^2)
\end{align*}

where this is a linear regression and $\phi(\bx^{(n)})$ represents any transformation we can apply. We can thus compute our posterior beliefs about the weights $\bw$ given our training examples (using Bayes' theorem, of course):

\begin{align*}
    P(\bw \giv \mathcal{D}) &\propto P(\mathcal{D} \giv \bw) P(\bw) \\
        &\propto \mathcal{N}(\bw; \vf{0}, \sigma_w^2 \mathbb{I})
            \prod_n \mathcal{N}(y^{(n)}; \bw^T \phi(\bx^{(n)}), \sigma_y^2) \\
        &\propto \mathcal{N}(\bw; \vf{0}, \sigma_w^2 \mathbb{I})\
            \mathcal{N}(\by; \Phi \bw, \sigma_y^2 \mathbb{I})
\end{align*}

This is also a Gaussian and can be derived (Murphy). If $\bw_0 = \bf{0}$ for the prior and $V_0 = \sigma_w^2 \mathbb{I}$, then:

\begin{align*}
    V_n &= \sigma_y^2(\sigma_y^2 V_0^{-1} + \Phi^T \Phi)^{-1} \\
    \bw_n &= V_n V_0^{-1} \bw_0 + \frac{1}{\sigma_y^2} V_N \Phi^T \by
\end{align*}

where $\bw_N$ is the posterior mean and $V_n$ - the posterior covariance.

\subsection{Test Predictions}
So we have our posterior $\bw$. Now we want, conditioned on the training data $\mathcal{D} = \{\bx^{(n)}, y^{(n)}\}$ to give predictions for a new test point $\bx^*$ with output $y^*$. We introduce our weights using the sum rule:

\begin{equation*}
    P(y^* \giv \bx^*, \mathcal{D}) = \int P(y^*, \bw \giv \bx^*, \mathcal{D}) \ \de\bw
\end{equation*}

Now we split the above using the product rule:
\begin{align*}
    P(y^* \giv \bx^*, \mathcal{D}) &= \int P(y^* \giv \bw, \bx^*, \mathcal{D})
        \ P(\bw \giv \bx^*, \mathcal{D}) \ \de\bw \\
    &= \int P(y^* \giv \bw, \bx^*)\
        P(\bw \giv \mathcal{D}) \ \de\bw
\end{align*}

where $\bw$ does not depend on the test input $\bx^*$. The test prediction also doesn't depend on the training data if we know $\bw$, the parameters. Now, we know the posterior for $\bw$:

\begin{equation*}
    P(\bw \giv \mathcal{D}) = \mathcal{N}(\bw; \bw_N, V_N)
\end{equation*}

For linear models the above integral is Gaussian. Expecting that, we can write:

\begin{align*}
    y^* &= f(\bx^*) + \nu = \bw^T \bx^* + \nu \\
    \nu &\sim \mathcal{N}(0, \sigma_y^2)
\end{align*}

This matches what we previously descibed as:

\begin{equation*}
    P(y^* \giv \bw) = \mathcal{N}(y^*; \bw^T \bx^*, \sigma_y^2)
\end{equation*}

We can thus show that:

\begin{equation*}
    P(y^* \giv \mathcal{D}, \bx^*) =
        \mathcal{N}(y^*; \bw_N^T \bx^*, (\bx^*)^T V_n \bx^* + \sigma_y^2)
\end{equation*}

\subsection{Decision Making}
The above answer is a distribution of $y^*$. Sometimes we need to ouput a single prediction, a guess. Thus we need to write a loss function $L(y, y^*)$, which says how bad it is to guess $y^*$ if the actual output is $y$. We can compute the expected loss:

\begin{equation*}
    c = \mathbb{E}_{P(y \giv \mathcal{D})}[L(y, y^*)] =
        \int L(y, y^*)\ P(y \giv \mathcal{D})\ \de y
\end{equation*}

For square loss, $L(y, y^*) = (y - y^*)^2$, we can differentiate the cost $c$ with respect to our guess:

\begin{equation*}
    \frac{\partial c}{\partial y^*} =
        \mathbb{E}_{P(y \giv \mathcal{D})} \frac{\partial L(y, y^*)}{\partial y^*} =
        \mathbb{E}_{P(y \giv \mathcal{D})} [2(y - y^*)] =
        2 (\mathbb{E}_{P(y \giv \mathcal{D})} [y] - y^*)
\end{equation*}

Setting this to $0$, the optimal guess is the posterior mean, $y^* = \mathbb{E}_{P(y \giv \mathcal{D})}[y]$. Thus if we only need the guess, using Bayesian regression does not change much at all. In fact this can be shown to correspond to a L2-regularized standard linear regression.

\section{Bayesian Logistic Regression}
For logistic regression, the likelihood function (as above) is defined as:

\begin{align*}
    P(\mathcal{D} \giv \model) &=  P(\data \giv \bw) =
        P(\vf{x}^{(1)}, y^{(1)}, \dots, \vf{x}^{(N)}, y^{(N)}\ |\ \vf{w}) \\
    &= \underbrace{\prod_i P(\vf{x}^{(i)}, y^{(i)} |\ \vf{w})}_{\text{\blue{independent x's}}} = \prod_i P(y^{(i)} |\ \vf{x}^{(i)}, \vf{w})\ P(\vf{x}^{(i)}\ |\ \vf{w}) \\
    &= \prod_i P(y^{(i)} |\ \vf{x}^{(i)}, \vf{w}) \underbrace{P(\vf{x}^{(i)})}_{\text{\blue{ignore weights}}} = \prod_i P(y^{(i)} |\ \vf{x}^{(i)}, \vf{w})
\end{align*}

When fitting via maximum likelihood, we do something like:

\begin{equation*}
    \bw^* = \argmax_{\bw} [ \log\ P(\by \giv X, \bw) -  \bw^T \bw ]
\end{equation*}

or, as most optimizers are optimized for minimizing functions, equivalently minimize the negative log-likelihood. Bayesian logistic regression begins the same way as Bayesian linear regression:

\begin{equation*}
    P(\bw \giv \data) = \frac{P(\data \giv \bw) P(\bw)}{P(\data)} \propto P(\data \giv \bw) P(\bw)
\end{equation*}

To make predictions (for a datapoint $y^*, \bx^*$), we can use probability theory:

\begin{align*}
    P(y^* \giv \bx^*, \data) &= \int P(y^*, \bw \giv \bx^*, \data) \de\bw \\
        &= \int P(y^* \giv \bx^*, \bw) P(\bw \giv \data) \de\bw
\end{align*}

The problem, of course, is that we can't really compute the posterior $P(\bw \giv \data)$. It never really is Gaussian as it were in the linear regression case. Thus we must be clever. In reality, if we assume a Gaussian prior, then posterior is this:

\begin{align*}
    P(\bw) &= \normal(\bw; \mathbf{0}, \sigma_w^2 \mathbb{I}) \\
    P(\bw \giv \data) &= \normal(\bw; \mathbf{0}, \sigma_w^2 \mathbb{I})
        \prod_n \sigma(\bw^T \bx^{(n)} z^{(n)})
\end{align*}

where $z^{n} = \{ +1, -1 \}$ instead of $\{0, 1\}$. If we notice that $\sigma(-a) = 1 - \sigma(a)$, then the above follows. Either way, this product of sigmoids and a Gaussian is nasty and very slow to integrate numerically above. We can do importance sampling to predict like so:

\subsection{Importance Sampling}
\begin{figure}
    \centering
    \includegraphics[width=0.6\textwidth]{figures/bayes_logistic}
    \caption{Bayesian Logistic Regression via Importance Sampling}
    \label{fig:bayes_log_importance}
\end{figure}

\begin{align*}
    P(y^* \giv \bx^*, \data) &= \mathbb{E}_{P(\bw \giv \data)}[P(y^* \giv \bx^*, \bw)] \\
        &\approx \frac{1}{S} \sum_{s = 1}^{S} P(y \giv, \bx^*, \bw^{(s)})
\end{align*}

where $\bw^{(s)} \sim P(\bw \giv \data)$. Sampling from the posterior $P(\bw \giv \data)$ is hard. We can do many things, like MCMC, but perhaps the easiest is importance sampling:

\begin{align*}
    P(y^* \giv \bx^*, \data) &= \int P(y^* \giv \bx^*, \bw) P(\bw \giv \data)\ \de\bw \\
        &= \int P(y^* \giv \bx^*, \bw) \frac{P(\data \giv \bw) P(\bw)}{P(\data)}\  \de\bw \\
    &= \mathbb{E}_{P(\bw)} [P(y^* \giv \bx^*, \bw) \frac{P(\data \giv \bw)}{P(\data)}] \\
    &= \mathbb{E}_{P(\bw)} [P(y^* \giv \bx^*, \bw) \frac{P(\data \giv \bw)}{\int P(\data \giv \bw) P(\bw)\ \de\bw}] \\
    &= \mathbb{E}_{P(\bw)} [P(y^* \giv \bx^*, \bw) \frac{P(\data \giv \bw)}{\mathbb{E}_{P(\bw)}[P(\data \giv \bw)]}]
\end{align*}

Now we can do importane sampling in the denominator and in the entire expression, perhaps even storing samples from $P(\bw)$, the prior. The end result looks like this:

\begin{align*}
    P(y^* \giv \bx^*, \data) &\approx
        \frac{1}{S} \sum_s P(y^* \giv \bx^*, \bw^{(s)}) \frac{
            P(\data \giv \bw^{(s)})
        }{
            \frac{1}{S'} \sum_{s'} P(\data \giv \bw^{(s')})
        }
\end{align*}

where $w^{(s)} \sim P(\bw)$. The bias of this, of course, is humongous. \autoref{fig:bayes_log_importance} shows an example result of this procedure.
\vskip 0.1in

Another thing we can do is:

\subsection{Maximum a Posteriori}
which is just this:

\begin{equation*}
    \bw^* = \argmax_\bw[\log P(\bw \giv \data)] =
        \argmax_\bw[\log P(\data \giv \bw) P(\bw)]
\end{equation*}

c.f. maximum likelihood. Really we only added the prior here. Onwards to:

\subsection{Laplace Approximation}
Let's assume that the posterior $p(\bw \giv \data)$ is Gaussian. That's not a very good assumption, putting it mildly. We can try matching the best Gaussian to it. We first find the maximum likelihood, the most probable value of $\bw$, which we label $\bw^*$, as usual:

\begin{align*}
    \bw^* &= \argmax_\bw P(\bw \giv \data) \\
        &= \argmax_\bw\ \log \frac{P(\data, \bw)}{P(\data)} \\
        &= \argmax_\bw\ \log P(\data | \bw) P(\bw)
\end{align*}

The last line follows from the fact that $P(\data)$ is independent of $\bw$, i.e. a constant in the optimization problem. We also can't really evaluate $P(\data)$, since it's a nasty integral. So far, so good. We can define the energy as follows:

\begin{align*}
    E(\bw) &= -\log P(\bw, \data) \\
    \bw^* &= \argmin_\bw E(\bw)
\end{align*}

Then we compute the Hessian, the matrix of second derivatives. This gives us the curvature of the function:

\begin{equation*}
    H_{i, j} = \frac{
        \partial^2 E(\bw^*)
    }{
        \partial w_i\ \partial w_j
    }
\end{equation*}

The energy of a Gaussian distribution would be, up to a constant:

\begin{equation*}
    E_\gauss (w) = \frac{(w - \mu)^2}{2\sigma^2}
\end{equation*}

The minimum is at $w^* = \mu$ and the second derivative - $H = 1 / \sigma^2$. Generalizing to a higher dimension, we get:

\begin{equation*}
    E_\normal(\bw) = \frac{1}{2} (\bw - \mathbf{\mu})^T \Sigma^{-1} (\bw - \mathbf{\mu})
\end{equation*}

where $\bw^* = \mathbf{\mu}$ and $H = \Sigma^{-1}$ or $\Sigma = H^{-1}$. Thus matching energies, we get:

\begin{equation*}
    P(\bw \giv \data) \approx \normal(\bw; \bw^*, H^{-1})
\end{equation*}

To approximate $P(\data)$, we can do:

\begin{align*}
    P(\bw \giv \data) &= \frac{P(\bw, \data)}{P(\data)}
        \approx \normal(\bw; \bw^*, H^{-1}) =
        \frac{|H|^{1/2}}{(2\pi)^{D/2}} \exp(-\frac{1}{2}
            (\bw - \bw^*)^{\text{T}} H (\bw - \bw^*))
\end{align*}

This is true for any $\bw$, so let's evaluate it for $\bw^*$, giving us:

\begin{align*}
    P(\bw \giv \data) &= \frac{P(\bw, \data)}{P(\data)}
        \approx \normal(\bw^*; \bw^*, H^{-1}) =
        \frac{|H|^{1/2}}{(2\pi)^{D/2}}
\end{align*}

which gives us:
\begin{equation*}
    P(\data) \approx P(\bw^*, \data)\ |2 \pi H^{-1}|^{1/2}
\end{equation*}

\subsection{Test Predictions}
For a test point $y^*, \bx^*$, we need to compute:

\begin{align*}
    P(y^* \giv \bx^*, \data) &= \int P(y^*, \bw \giv \bx^*, \data)\ \de\bw \\
    &= \int P(y^* \giv \bx^*, \bw)\ P(\bw \giv \data)\ \de\bw
\end{align*}

Using the Laplace approximation above, we know that $P(\bw \giv \data) \approx \normal(\bw; \bw^*, H)$, we have the following integral:

\begin{align*}
    P(y^* = 1 \giv \bx, \data) &\approx
        \int \sigma(\bw \bx^\text{T}) \normal(\bw; \bw^*, H^{-1})\ \de\bw \\
    &= \mathbb{E}_{\normal(\bw; \bw^*, H^{-1})} [\sigma(\bw \bx^\text{T})]
\end{align*}

Calling $a = \bw^* \bx^\text{T}$ and changing variables:

\begin{align*}
    P(a) &= \normal(a; \bw^* \bx^\tran, \bx^\tran H^{-1} \bx)\\
    P(y = 1 \giv \bx, \data) &=
        \int \sigma(a)\ \normal(a; \bw^* \bx^\tran, \bx^\tran H^{-1} \bx) \ \de a
\end{align*}

which we can then compute numerically. From Murphy Section 8.4.4.2, we can derive another approximation:

\begin{align*}
    P(y = 1 \giv \bx, \data) &\approx \sigma(\kappa \bw^* \bx) \\
    \kappa &= \frac{
        1
    }{
        \sqrt{
            1 + \frac{\pi}{8} \bx^\tran V \bx
        }
    }
\end{align*}

\section{Optimizing Hyperparameters in a Bayesian Setting}
We normally opt to choose simpler models, that is models that assign a high probability to the training data, over models that have an high probability of generating many datasets. We can thus specify the probability:

\begin{align*}
    P(\data \giv \model) &= P(X, \by \giv \model) =
        P(\by \giv X, \model) \underbrace{P(X \giv \model)}_\text{\blue{1}} \\
    &= P(\by \giv X, \model) = \int P(\by, \bw \giv X, \model)\ \de\bw =
        \int P(\by \giv X, \bw, \model)\ P(\bw \giv \model)\ \de\bw
\end{align*}

The probability of the data given the model, i.e. $P(\data \giv \model)$ is the \textbf{marginal likelihood} of the model. We can use it to select between different models $\model_k$.
\vskip 0.1in

(\textit{Yet another example from MLPR}) If we have a linear model with:
\begin{align*}
    P(\bw \giv \sigma_w) &= \normal(\bw; \mathbf{0}, \sigma_w^2 \mathbb{I}) \\
    P(y \giv \bw, \sigma_y) &= \normal(y; \bw^\tran \bx, \sigma_y^2)
\end{align*}

We then fit $\sigma_w$ and $\sigma_y$ by maximizing the marginal likelihood:

\begin{equation*}
    P(\by \giv X, \sigma_w, \sigma_y) =
        \int P(\by \giv X, \bw, \sigma_y)\ P(\bw \giv \sigma_w)\ \de\bw
\end{equation*}

\section{Gaussian Processes}
\begin{figure}
    \centering
    \includegraphics[width=0.6\textwidth]{figures/gp}
    \caption{Gaussian Process posterior}
    \label{fig:gp}
\end{figure}

Gaussian processes are just big Gaussians. Let's say we have a training set of values $\by$ and inputs $X$ and a test set of values (unknown) $\bfu_*$ and test inputs $X_*$. If we assume that everything comes from a Gaussian, i.e.:

\begin{equation*}
    P\bigg(\begin{bmatrix}
        \by \\
        \bfu_*
    \end{bmatrix}\bigg) = \normal\bigg(
        \begin{bmatrix}
            \by \\
            \bfu_*
        \end{bmatrix};
        \mathbf{0};
        \begin{bmatrix}
            K(X, X) + \sigma_y^2 \mathbb{I} & K(X, X_*) \\
            K(X_*, X) & K(X_*, X_*)
        \end{bmatrix}
    \bigg)
\end{equation*}

This really is the essence of the thing. The $K$ functions are the kernels that represent the covariance between datapoints. This is enough to describe any dataset; in fact a non-zero mean Gaussian process is always equivalent to a zero mean one (TODO: proof). The kernels are defined as:

\begin{equation*}
    K(X, Y)_{i, j} = k(\bx^{(i)}, \bz^{(j)})
\end{equation*}

for two test inputs $\bx$ and $\bz$. For training inputs we also have noise variance $\sigma_y^2$. Let's call the noise:

\begin{equation*}
    \nu_i \sim \normal(0, \sigma_y^2)
\end{equation*}

then for the cross-covariance:

\begin{align*}
    \text{cov}(y_i, f_{*, j}) &= \expect[y_i f_{*, j}] -
        \expect[y_i] \expect[f_{*,j}] \\
        &= \expect[(f_i + \nu_i) f_{*, j}] \\
        &= \expect[f_i f_{*,j}] + \underbrace{\expect[\nu_i]}_0\expect[f_{*, j}] \\
        &= \expect[f_i f_{*,j}] = k(\bx^{(i)}, \bx^{*, j})
\end{align*}

Once we specify the kernel $k$, we can extract the distribution of the test datapoints by simply marginalizing the big Gaussian:

\begin{align*}
    P(\bfu_* \giv \data, \model) =
        \normal&(\bfu_*; \\
        &K(X_*, X)(K(X, X) + \sigma_y^2 \mathbb{I})^{-1} \by, \\
        &K(X_*, X_*) - K(X_*, X)(K(X, X) + \sigma_y^2 \mathbb{I})^{-1}K(X, X_*) \\
        &)
\end{align*}

which is the posterior over test datapoints.

\subsection{Gaussian Kernel}
A common choice for a kernel is the Gaussian kernel:

\begin{equation*}
    k(\bx^{i}, \bx^{j}) =
        \sigma_f^2 \exp\bigg(
            -\frac{1}{2} \sum_{d=1}^D (x_d^{(i)} - x_d^{j})^2 /l_d^2
        \bigg)
\end{equation*}

We can fit $\sigma_f$ and $\l_d$ by maximizing the marginal likelihood.

\section{Variational Inference}
Not going into details here. The idea is to minimize the Kullback-Leibler Divergence between two distributions. We can use it instead of the Laplace Approximation for Bayesian Logistic Regression above.

\begin{equation*}
    D_{KL}(p || q) = \int p(\bx) \log \frac{p(\bx)}{q(\bx)}\ \de\bx
\end{equation*}

We can minimize the KL divergence $D_{KL}(q(\bw; \alpha)\ ||\ P(\bw \giv \data))$ in the logistic regression case. We are trying to match a parametric function $p$ with some parameters captured in $\alpha$ to the posterior distribution of the weights $P(\bw \giv \data)$:

\begin{align*}
    \kl(q(\bw; \alpha)\ ||\ \post) &= \int q(\bw; \alpha) \log \frac{
        q(\bw; \alpha)
    }{
        \post
    }
    \ \de\bw \\
    &= -\int q(\bw; \alpha) \log \post\ \de\bw +
    \underbrace{
        \int q(\bw; \alpha) \log q(\bw; \alpha)\ \de\bw
    }_{\text{negative entropy,} -H(q)}
\end{align*}

The first term is large if we put probability on regions where the posterior is small. The second term encourages more spread-out distributions with high entropy.
\vskip 0.1in

Using Bayes' theroem in the KL divergence formula:

\begin{equation*}
    \kl(q\ ||\ p) = \underbrace{
        \expect_q[\log q(\bw)] -
        \expect_q[\log P(\data \giv \bw)] -
        \expect_q[\log P(\bw)]
    }_{J(q)} + \log P(\data)
\end{equation*}

We thus minimize $J(q)$, also known as ELBO (Evidence Lower Bound). We can get a bound on the $P(\data)$ term, the marginal likelihood, by observing the KL divergence is non-negative:

\begin{equation*}
    \log P(\data) \geq -J(q)
\end{equation*}

\subsection{Minimizing the KL-divergence for a Gaussian function}
Let $\alpha = \{ \bm{m}, V \}$, the mean and co-variance of a Gaussian distribution. We will be (numerically) minimizing the $J(q)$ with respect to $\alpha$. Thus:

\begin{equation*}
    q(\bw; \alpha) = \normal(\bw; \bm{m}, V)
\end{equation*}

We first Cholesky decompose $V$ to get $V = LL^\tran$, $L$ being a lower-triangular matrix with positive diagonal entries. We also optimize $\log \sigma_w$; otherwise we might make $\sigma_w$ negative.

\begin{equation*}
    L = \begin{cases}
        \log L_{i, j} & i = j \\
        L_{i, j}      & i \neq j
    \end{cases}
\end{equation*}

We assume a Gaussian prior:

\begin{equation*}
    P(\bw) = \normal(\bw; \bf{0}, \sigma_w^2 \mathbb{I})
\end{equation*}

We can write two of the terms of $J(q)$ in closed form:

\begin{align*}
    \expect_{\normal(\bw; \bm{m}, V)}[\normal(\bw; \bmu, \Sigma)] &=
        \expect_{\normal(\bw; \bm{m}, V)}\bigg[
            -\frac{1}{2}(\bw - \bmu)^\tran \Sigma^{-1} (\bw - \bmu)
        \bigg] - \frac{1}{2} \log |2 \pi \Sigma] \\
    %
    \expect_q[\log q(\bw)] &=
        \expect_{\normal(\bw; \bm{m}, V)}[\log \normal(\bw; \bm{m}, V)] = -\frac{D}{2} - \frac{1}{2}\log|2 \pi V| \\
    %
    \expect_q[\log P(\bw)] &=
        \frac{1}{2\sigma_w^2}\bigg[ \text{Tr}(V) + \bm{m}^\tran \bm{m} \bigg] + \frac{D}{2} \log(2\pi\sigma_w^2)
\end{align*}

Using this trick:

\begin{align*}
    \expect_{\normal(\bz; 0, V)}[\log \normal(\bz; 0, V)] &=
        \expect[-\frac{1}{2} \text{Tr}(\bz^\tran V^{-1} \bz)]
    = -\frac{1}{2} \text{Tr}(\expect[\bz^\tran V^{-1} \bz])
    = -\frac{1}{2} \text{Tr}(\expect[\bz^\tran \bz] V^{-1}) \\
    &= -\frac{1}{2} \text{Tr}(VV^{-1}) = -\frac{1}{2} \text{Tr}(\mathbb{I}_D) = -\frac{D}{2}
\end{align*}

where $\expect[\bz^\tran \bz] = V$ by the definition of covariance for a $0$-mean random variable. We also know that $\bz^\tran V \bz$ is a number and hence equal to its trace, a.k.a. the "trace trick".
\vskip 0.1in

From the Cholesky decomposition:
\begin{align*}
    \frac{1}{2} \log|V| &= \sum_i \log L_{i,i} \\
    \text{Trace}(V) &= \sum_{i,j} L_{i,j}^2
\end{align*}

hence the above reparametrization.

\subsection{Log-likelihood term}
The nastiest of the three terms is the log-likelihood:

\begin{equation*}
    -\expect_{\normal(\bw; \bm{m}, V)}[\log P(\data \giv \bw)] =
        -\expect_{\normal(\bw; \bm{m}, V)}\bigg[
            \sum_{n=1}^N \log p(y^{(n)} \giv \bx^{(n)}, \bw)
        \bigg]
\end{equation*}

We cannot compute this in closed form. Instead we do a Monter Carlo estimate:

\begin{equation*}
    -\expect_{\normal(\bw; \bm{m}, V)}[\log P(\data \giv \bw)] \approx
        - \sum_{n = 1}^{N} \log p(y^{(n)}, \bx^{n}, \bw)
\end{equation*}

where $\bw \sim \normal{\bm{m}, V}$.

\subsection{Gradients of the log-likelihood}
Here we do a "reparametrization trick" (gotta love those). To sample a random $\bw$, we sample a vector of normal variables $\nu \sim \normal(0, \mathbb{I})$ and then transform it (see multivariate Gaussians above): $\bw = \bm{m} + L \nu$. We can rewrite the expectation:

\begin{equation*}
    \expect_{\normal(\bw; \bm{m}, V)}[f(\bw)] = \expect_{\normal(\pmb{\nu}; \bm{0}, \mathbb{I})}[f(\bm{m} + L \pmb{\nu})]
\end{equation*}

Now differentiating:
\begin{align*}
    \nabla_\bm{m} \expect_{\normal(\bw; \bm{m}, V)}[f(\bw)] &=
    \expect_{\normal(\pmb{\nu}; \bm{0}, \mathbb{I})}[\nabla_\bm{m} f(\bm{m} + L \pmb{\nu})] \\
    &\approx \nabla_\bm{m} f(\bm{m} + L \pmb{\nu}) \\
    %
    \nabla_L \expect_{\normal(\bw; \bm{m}, V)}[f(\bw)] &=
    \expect_{\normal(\pmb{\nu}; \bm{0}, \mathbb{I})}[\nabla_L f(\bm{m} + L \pmb{\nu})] \\
    &\approx [\nabla_L f(\bw)]\pmb{\nu}^\tran
\end{align*}

\section{Gaussian Mixture Models}
This is a clustering algorithm. We have $K$ classes and each datapoint has a class: $z^{(n)} = \{1, \dots, K\}$. These are unobserved and drawn from a prior. $z^{(n)} \sim \pi$. The points come from the Gaussian mixtures:

\begin{equation*}
    P(\bx^{(n)} \giv z^{n} = k, \theta) = \normal(\bx^{(n)}; \pmb{\mu}^{(k)}, \Sigma^{(k)})
\end{equation*}

where $\theta = \{ \pi, \{ \bmu^{(k)}, \Sigma^{(k)} \}\}$. We can fit it via maximum likelihood by maximizing:

\begin{equation*}
    \log P(\data \giv \theta) = \sum_n \log P(\bx^{(n)} \giv \theta)
\end{equation*}

where:

\begin{align*}
    P(\bx^{(n)} \giv \theta) &=
        \sum_k P(\bx^{(n)}, z^{(n)} = k \giv \theta) \\
    &= \sum_k P(\bx^{(n)} \giv z^{(n)} = k, \theta) P(z^{(n)} = k \giv \theta) \\
    &= \sum_k \pi_k \normal(\bx^{(n)}; \bmu^{(k)}, \Sigma)
\end{align*}

We can't solve this analytically, but we can use gradient-based optimizers. We decompose $\Sigma = LL^\tran$ (Cholesky) and set:

\begin{equation*}
    L = \begin{cases}
        \log L_{i, j} & i = j \\
        L_{i, j}      & i \neq j
    \end{cases}
\end{equation*}

The $\pmb{\pi}$ vector must represent probabilities. We can use a softmax transformation.

\subsection{Expectation Maximization (EM)}
This consists of two steps, the $\bm{E}$ and $\bm{M}$ step:
\vskip 0.1in

$\bm{E}$-step:
\begin{equation}
    r_k^{(n)} = P(z^{(n)} = k \giv \bx^{(n)}, \theta) = \frac{
        \pi_k \normal(\bx^{(n)}; \bmu^{(k)}, \Sigma^{(k)})
    }{
        \sum_l \pi_l \normal(\bx^{(n)}; \bmu^{(l)}, \Sigma^{(l)})
    }
\end{equation}

$\bm{M}$-step:
\begin{align*}
    r_k = \sum_n r_k^{(n)}
    \pi_k &= \frac{r_k}{N} \\
    \bmu^{(k)} &= \frac{1}{r_k} \sum_n r_k^{(n)} \bx^{(n)} \\
    \Sigma^{(k)} &= \frac{1}{r_k} \sum_n r_k^{(n)} \bx^{(n)} \bx^{(n)\tran}
        - \bmu^{(k)} \bmu^{(k)\tran}
\end{align*}

% \section{Optimization}
% \subsection{Regularization in Depth}
% \subsection{Expectation Maximization}
% \subsection{Proximal Methods}

% \section{Ensembles}
% \subsection{Averaging}
% \subsection{Bagging}
% \subsection{Mixture of Experts}

\printbibliography

\end{document}


% Question  ##################################################################################################################
\section{Question 3}\label{ssec:pt2q3}
\textbf{Hidden Markov Models (HMM) are an important unsupervised learning method used to analyse sequence data and identifying underlying regimes that govern the data.}

\noindent
\textbf{This question relates to detecting regime changes in volatility (risk).}
% END Question  ##############################################################################################################

% Question (i) ###############################################################################################################
\subsection{Q2 (i)}\label{sssec:pt2q3i}
\textbf{Download and calculate the last 10 year daily returns of the S\&P500 index (till 31/12/2017).}

\noindent
Code to download the closing prices for S\&P500 in this question can be found in ‘Question 3 (i)’ in the python notebook. The data was downloaded from Yahoo Finance and saved as .csv format in the following path \textit{‘src/data/part\_2’}. The file was saved as \textit{‘GSPC.csv’}. The data was then reloaded from the file and converted into pandas dataframe and the daily log returns were computed.
% END Question (i) ###########################################################################################################

% Question (ii) ##############################################################################################################
\subsection{Q2 (ii)}\label{sssec:pt2q3ii}
\textbf{Create a new series of volatility based on 10 day rolling time window (for each day apply standard deviation on the previous 10 day returns).}

\noindent
To calculate the 10 day rolling volatility a function called 'rolling\_vol' was created in the 'fintech' library as shown in Fig.~\ref{fig:rollvol}, and this was called from the notebook in 'Question 3 (ii)'.

\begin{figure}[H]
\centering
  \includegraphics[scale = .58]{imgs/rolling_vol.png}
  \caption{Function to calculate rolling volatility.}
  \label{fig:rollvol}
\end{figure}
% END Question (ii) ##########################################################################################################

% Question (iii) ############################################################################################################

\subsection{Q2 (iii)}\label{sssec:pt2q3iii}
\textbf{Utilize this new series to train two Gaussian HMM: one using 2 latent states and the other with 3 latent states (components).  These latent states indicate the different volatility regimes.}

\noindent
The code to train the two models can be found in 'Question 3 (iii)'. Both models were trained using 1000 iteration using a diagonal covariance matrix. Also both models were trained using one feature, which is the series created in (ii). A third party library called 'hmmlearn' \cite{python:hmmlearn} was utilised for this task.

% END Question (iii) #########################################################################################################

% Question (iv) ##############################################################################################################

\subsection{Q2 (iv)}\label{sssec:pt2q3iv}
\textbf{For each of the two models above, present a visualization (one or more plots) which clearly shows the identified volatility regimes over the 10-year period being investigated.}

\noindent
Fig.~\ref{fig:regime1} shows the volatility regime over the 10-period when using 2 latent states while Fig.~\ref{fig:regime2} shows the volatility regimes when using 3 latent states. The code for these plots can be found in the notebook in 'Question 3 (iv)'.

\begin{figure}[H]
\centering
  \includegraphics[scale = .60]{imgs/regime1.png}
  \caption{Volatility regime over the 10-period when using 2 latent states.}
  \label{fig:regime1}
\end{figure}

\begin{figure}[H]
\centering
  \includegraphics[scale = .65]{imgs/regime2.png}
  \caption{Volatility regime over the 10-period when using 3 latent states.}
  \label{fig:regime2}
\end{figure}

% END Question (iv) ##########################################################################################################

% Question (v) ###############################################################################################################

\subsection{Q2 (v)}\label{sssec:pt2q3v}
\textbf{Comment on the findings and discuss the difference in outcomes between the two models.}

\noindent
Using the models in the previous question (iii) the Hidden Markov model enables us to investigate underlying latent states (must specify the number of components in this unsupervised learning model) and the probability transitions between them even though they are not directly observable.

\noindent
Finding volatility regimes can help an investor to manage risk more effectively by identifying states where there is high volatility. This model has the potential of identifying incorrect identification of a “trend”.

\noindent
As seen from both the plots the regime detection captures highly volatile and “trending” periods. In the first model (two latent states) the majority is captured in the 2nd state but most of 2008-2009 and late 2011 is captured in the first state. In the second model the majority was captured in the 2nd state, while most of 2008-2009 was captured in the first state together with the late 2011 period. Utilizing these regimes one can apply different investing/risk management methods according to the state the asset is in, in terms of volatility. The states which were captured show a low volatility state and a high volatility state/s.

\noindent
All the states in the two models have a positive mean. The variance of the 2nd state in both models indicate low variance (low volatility periods), but the first state has a variance which is significantly higher in both models (high volatility periods). This indicates a state with high volatility. The third state in the second model also indicate high volatility but it must be noted that this state captures relatively low information.

\noindent
It is also important to note the states during the 2008 period (market crash), as you can see both models captured high volatility during that period (both models capture this in the first state). The measurements discussed above (mean and var) are shown in Fig.~\ref{fig:model_a_regime} and Fig.~\ref{fig:model_b_regime}.

\begin{figure}[H]
     \centering
     \begin{subfigure}[b]{0.45\textwidth}
         \centering
         \includegraphics[width=\textwidth]{imgs/model_a_regime.JPG}
         \caption{HMM measurements for Model A (two latent states).}
         \label{fig:model_a_regime}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.45\textwidth}
         \centering
                 \includegraphics[width=\textwidth]{imgs/model_b_regime.JPG}
        \caption{HMM measurements for Model B (three latent states).}
         \label{fig:model_b_regime}
     \end{subfigure}
\end{figure}

% END Question (v) ###########################################################################################################


% !TEX root = as_grf_sopt.tex

% \begin{appendices}
\section{Active Search Regret Bound}
\label{appendix:proof_as_regret}
We start by stating the following result.% by \cite{srinivas2012information}.
\begin{theorem}[Theorem 6, {\cite{srinivas2012information}}]
	\label{thm:dev}
	Let $\delta \in (0,1)$. Assume the observation noises are uniformly bounded by $\sigma_n$ and $f$ has RKHS norm $B$ with kernel $C_0$, which is equivalent to $\bff^\top\widetilde{\cLap}_0\bff\leq B^2$. Define
	% \begin{equation*}
	$
		\alpha_t = \sqrt{2 B^2 + 300 \gamma_t \log(t/\delta)^3},
		$
	% \end{equation*}
	% where $\Vert \cdot \Vert_K$ denotes the RKHS norm associated with the kernel $K$. Then
	then
	\begin{equation*}
		\mbox{Pr}\left(\forall t, \forall v \in V, \;\; |\mu_t(v) - f(v)| \leq \alpha_{t+1} \sigma_t(v) \right) \geq 1 - \delta.
	\end{equation*}
\end{theorem}
We use this result to bound our instantaneous regrets.
\begin{lemma}
	Conditioned on the high-probability event in Theorem \ref{thm:dev}, the following bound holds:
	\begin{equation*}
		\forall t, \;\; r_t := f(v^*_t) - f(v_t) \leq 2 \alpha_t k \sigma_{t-1}(v_t),
	\end{equation*}
	where $v^*_t$ is the node with the $t$-th globally largest function value and $v_t$ is node selected at round $t$.
\end{lemma}
%\begin{proof}
\emph{Proof}.
At round $t$ there are two possible situations. If
$v^*_t$ was picked at some earlier round, the definition of $v^*_t$ implies that there exists some $t' < t$ such that $v^*_{t'}$
has not been picked yet. According to our selection rule, the fact that $s_t(v) \geq \sigma_t(v)$,
and Theorem \ref{thm:dev}, the following holds:
\begin{align*}
	&\mu_{t-1}(v_t) + \alpha_t s_{t-1}(v_t) \geq \mu_{t-1}(v^*_{t'}) + \alpha_t s_{t-1}(v^*_{t'}) \\
		&\qquad \geq \mu_{t-1}(v^*_{t'}) + \alpha_t \sigma_{t-1}(v^*_{t'})
	 \geq f(v^*_{t'}) \geq f(v^*_t).
\end{align*}
If $v^*_t$ has not been picked yet, a similar argument gives
\begin{equation*}
	\mu_{t-1}(v_t) + \alpha_t s_{t-1}(v_t) \geq \mu_{t-1}(v^*_{t}) + \alpha_t s_{t-1}(v^*_{t})
		%&\geq& \mu_{t-1}(v^*_{t}) + \alpha_t \sigma_{t-1}(v^*_{t})\\
	\geq f(v^*_t).
\end{equation*}
Thus we always have
\begin{align*}
f(v^*_t) &\leq \mu_{t-1}(v_t) + \alpha_t s_{t-1}(v_t) \\
&\leq f(v_t) + \alpha_t \sigma_{t-1}(v-t) +\alpha_t s_{t-1}(v_t)\\
%&\leq& f(v_t) + 2 \alpha_t s_{t-1}(v_t) \\
&\leq f(v_t) + 2 \alpha_t k \sigma_{t-1}(v_t).
\end{align*}
%\end{proof}
%\begin{lemma}[Lemma 5.3, {\cite{srinivas2012information}}]
%	The information gain achieved by the selected nodes can be expressed in terms of the predictive variances.
%	Let $\bv_t = (v_1, v_2, \ldots, v_t)$ denote the sequence of selected nodes. Then
%	\begin{equation*}
%		\mathcal{I}(\by_{\bv_t};f_{\bv_t}) = \frac{1}{2} \sum_{i=1}^t \log(1 + \sigma^{-2}\sigma_{i-1}(v_i)^2).
%	\end{equation*}
%\end{lemma}
%Next we bound the sum of squared instantaneous regrets in terms of the maximum information gain.
\begin{lemma}[Lemma 5.4, {\cite{srinivas2012information}}]
	Let $\alpha_t$ be defined as in Theorem \ref{thm:dev} and $c_1$ be defined as in Theorem \ref{thm:as_regret}.
	Conditioned on the high probability event of Theorem \ref{thm:dev}, the following holds:
	\begin{equation*}
	\forall T\geq 1, \;\;\sum_{t=1}^T r_t^2 \leq \alpha_T k^2 c_1 \mathcal{I}(\by_{\bv_T};f_{\bv_T}) \leq \alpha_T k^2 c_1 \gamma_T.
	\end{equation*}
\end{lemma}
Finally, the Cauchy-Schwarz inequality gives
$R_T \leq \sqrt{T \sum_{t=1}^T r_t^2} \leq k \sqrt{T c_1 \alpha_T \gamma_T}.$
% \end{appendices}


% Copyright 2018 Melvin Eloy Irizarry-Gelpí
\chapter{Bi-Perplex Numbers}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
A split quaternion has the form
\begin{equation}
    a_{0} + a_{1} A + a_{2} S + a_{3} AS
\end{equation}
These follow from a hyperbolic Cayley-Dickson construct on the binions.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Arithmetic Operations}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
...
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Multiplication}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
...
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Conjugate Operations}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
...
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Asterisk Conjugation}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
...
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Quadrance}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
...
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Cloak Conjugation}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
...
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Dagger Conjugation}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
...
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Hodge Star}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
...
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Differential Operators}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
...


\section{National Camp 2019 Problem Set}

	\prob{https://math.stackexchange.com/questions/675646/if-there-are-5-points-on-a-sphere-then-4-of-them-belong-to-a-half-sphere}{		}{E}{Given any five points on a sphere, show that some four of them must lie on a closed hemisphere.}

	\prob{}{}{}{Alice and Bob play a game. There is a threshold $ n $, Initially, the game starts with the number $ 1 $. In a move the player replaced the number $ m $ with either $ m+1 $ or $ 2m $. Assuming optimal play, count the number of threshold under $ 2019 $ where Alice wins.}

	\prob{}{}{}{For an integer $ n $, let $ a_1, a_2, \dots a_{\phi(n)} $ be the integers less than and coprime to $ n $. Determine all possible values of \[\prod_{i=1}^{\phi(n)} a_i\ (\text{mod}\ n)\] and for which values of $ n $ they appear.}

	\prob{}{}{}{Let $ O $ and $ N $ be the circumcenter and nine-point-center of $ \triangle ABC $ respectively. Let $ I_b $ and $ I_c $ be the $ B $ and $ C $ excenters of $ \triangle ABC $ respectively. Prove that \[ \angle I_bOI_c = 180^\circ - \frac 1 2 \angle I_bNI_c \]}

	\prob{https://artofproblemsolving.com/community/q3h448003p2521640}{Putnam 2001 A5}{E}{Prove that there are unique positive integers $ a, n $ such that \[a^{n+1} - (a+1)^n = 2001\]}

	\prob{}{}{}{Find all polynomials $ P(x) $ with real coefficients such that for all real numbers $ x+ y+ z= 0 $, then the determinant of the following matrix is $ 0 $
	\[
	\begin{bmatrix}
		1 & x & P(x)\\
		1 & y & P(y)\\
		1 & z & P(z)
	\end{bmatrix}
	\]
	}

	\prob{https://artofproblemsolving.com/community/c6h60830p366681}{IMO 1971 P5}{M}{Prove that for every positive integer $m$ we can find a finite set $S$ of points in the plane, such that given any point $A$ of $S$, there are exactly $m$ points in $S$ at unit distance from $A$.}

	\prob{https://artofproblemsolving.com/community/q2h368177p2026501}{ISL 1971}{M}{Consider a sequence of polynomials $P_0(x), P_1(x), P_2(x), \ldots, P_n(x), \ldots$, where $P_0(x) = 2, P_1(x) = x$ and for every $n \geq 1$ the following equality holds:
		\[P_{n+1}(x) + P_{n-1}(x) = xP_n(x).\]
		Prove that there exist three real numbers $a, b, c$ such that for all $n \geq 1,$
		\[(x^2 - 4)[P_n^2(x) - 4] = [aP_{n+1}(x) + bP_n(x) + cP_{n-1}(x)]^2.\]}

	\prob{https://artofproblemsolving.com/community/q2h58616p359102}{IMO 1974 P3}{}{Prove that for any $ n $ natural, the number \[\sum_{k=0}^{n} \binom{2n+1}{2k+1}2^3k\] cannot be divided by $ 5 $.}

	\prob{https://artofproblemsolving.com/community/c7h513439p2882561}{Putnam 1999 B6}{M}{Let $S$ be a finite set of integers, each greater than $1$. Suppose that for each integer $n$ there is some $s\in S$ such that $\gcd(s,n)=1$ or $\gcd(s,n)=s$. Show that there exist $s,t\in S$ such that $\gcd(s,t)$ is prime.}

	\prob{https://artofproblemsolving.com/community/c6h279890p1511554}{Sharygin 2009 P4}{E}{Let $ P$ and $ Q$ be the common points of two circles. The ray with origin $ Q$ reflects from the first circle in points $ A_1$, $ A_2$,$ \ldots$ according to the rule ``the angle of incidence is equal to the angle of reflection''. Another ray with origin $ Q$ reflects from the second circle in the points $ B_1$, $ B_2$,$ \ldots$ in the same manner. Points $ A_1$, $ B_1$ and $ P$ occurred to be collinear. Prove that all lines $ A_iB_i$ pass through $ P $.}

	\prob{https://artofproblemsolving.com/community/q2h127832p725339}{Greece}{M}{Find all surjective functions $ f:\N \to \N $ such that for all $ m, n\in \N $\[m|n \Longleftrightarrow f(m)|f(n)\]}

	\prob{https://artofproblemsolving.com/community/c6h358783p1960094}{USA TST 2010 P9}{H}{Determine whether or not there exists a positive integer $k$ such that $p = 6k+1$ is a prime and
		\[\binom{3k}{k} \equiv 1 \pmod{p}\]}

	\prob{https://artofproblemsolving.com/community/c6h420428p2374811}{USA TST 2011 P6}{H}{A polynomial $P(x)$ is called nice if $P(0) = 1$ and the nonzero coefficients of $P(x)$ alternate between $1$ and $-1$ when written in order. Suppose that $P(x)$ is nice, and let $m$ and $n$ be two relatively prime positive integers. Show that
		\[Q(x) = P(x^n) \cdot \frac{(x^{mn} - 1)(x-1)}{(x^m-1)(x^n-1)}\]
		is nice as well.}

	\prob{}{}{}{Find all pairs of positive integers $ (m,n)$ such that $ mn - 1$ divides $ (n^2 - n + 1)^2$.}

	\prob{}{}{}{In isosceles $\triangle ABC$, $AB=AC$, points $D,E,F$ lie on segments $BC,AC,AB$ such that $DE\parallel AB$, $DF\parallel AC$. The circumcircle of $\triangle ABC$ $\omega_1$ and the circumcircle of $\triangle AEF$ $\omega_2$ intersect at $A,G$. Let $DE$ meet $\omega_2$ at $K\neq E$. Points $L,M$ lie on $\omega_1,\omega_2$ respectively such that $LG\perp KG, MG\perp CG$. Let $P,Q$ be the circumcenters of $\triangle DGL$ and $\triangle DGM$ respectively. Prove that $A,G,P,Q$ are concyclic.}

		\figdf{.3}{nat_pset_16}{}

	\prob{}{}{}{Given a polynomial $f(x)$ with rational coefficients, of degree $d \ge 2$, we define the sequence of sets $f^0(\mathbb{Q}), f^1(\mathbb{Q}), \ldots$ as $f^0(\mathbb{Q})=\mathbb{Q}$, $f^{n+1}(\mathbb{Q})=f(f^{n}(\mathbb{Q}))$ for $n\ge 0$. (Given a set $S$, we write $f(S)$ for the set $\{f(x)\mid x\in S\})$.
		Let $f^{\omega}(\mathbb{Q})=\bigcap_{n=0}^{\infty} f^n(\mathbb{Q})$ be the set of numbers that are in all of the sets $f^n(\mathbb{Q})$, $n\geq 0$. Prove that $f^{\omega}(\mathbb{Q})$ is a finite set.}

	\prob{}{}{}{Define the polymonial sequence $\left \{ f_n\left ( x \right ) \right \}_{n\ge 1}$ with $f_1\left ( x \right )=1$, $$f_{2n}\left ( x \right )=xf_n\left ( x \right ), \; f_{2n+1}\left ( x \right ) = f_n\left ( x \right )+ f_{n+1} \left ( x \right ), \; n\ge 1.$$Look for all the rational number $a$ which is a root of certain $f_n\left ( x \right ).$}

	\prob{https://artofproblemsolving.com/community/q1h547874p3176078}{Sharygin 2013 Final Round 10.8}{}{Two fixed circles are given on the plane, one of them lies inside the other one. From a point $ C $ moving arbitrarily on the external circle, draw two chords $ CA, CB $ of the larger circle such that they tangent to the smaller one. Find the locus of the incenter of triangle $ ABC $.}

	\prob{https://artofproblemsolving.com/community/q1h1279375p6724144}{IMC 2014 P3}{}{Let $n$ be a positive integer. Show that there are positive real numbers $a_0, a_1, \dots, a_n$ such that for each choice of signs the polynomial
		\[\pm a_nx^n\pm a_{n-1}x^{n-1} \pm \dots \pm a_1x \pm a_0\]
	has $n$ distinct real roots.}

	\prob{https://artofproblemsolving.com/community/c260h1442608p8732088}{Dunno}{}{Suppose that the real numbers $a_0,a_1,...,a_n$ and $x$, with $0<x<1$, satisfy
		\[\frac{a_0}{1-x}+\frac{a_1}{1-x^2}+...+\frac{a_n}{1-x^{n+1}}=0\]
	Prove that there exist a real number $y$ with $0<y<1$ such that
		\[a_0+a_1y+a_2y^2+...+a_ny^n=0\]}

	\prob{https://artofproblemsolving.com/community/q2h523187p2952670}{RMM 2013 P6}{}{A token is placed at each vertex of a regular $2n$-gon. A move consists in choosing an edge of the $2n$-gon and swapping the two tokens placed at the endpoints of that edge. After a finite number of moves have been performed, it turns out that every two tokens have been swapped exactly once. Prove that some edge has never been chosen.}

	\prob{https://artofproblemsolving.com/community/c6h1683076p10734638}{Sharygin 2018 Final Round 8.4}{}{Find all sets of six points in the plane, no three collinear, such that if we partition the set into two sets, then the obtained triangles are congruent.}