\chapter{Error analysis}

\section{Introduction}

Numerical analysis is mathematics brought down to earth.  In the world
of mathematics   numbers   are exact:   $\pi$ is  the   ratio  of  the
circumference to its radius and it is known exactly, each single digit
after  the other.  Numerical analysis teaches  humbleness: it tells us
that in  the real world  we cannot know numbers  exactly, we  can only
know 2, 20, a million of the digits that form the number $\pi$, but in
no way we can know any non-rational number exactly.  This may not seem
a serious problem: does it really matter that we cannot represent more
than 10 of the  first digits of  $\pi$?   The answer to  this specific
question is that generally it does not matter.  However, if we were to
rephrase the question as: ``Does it  matter that we cannot represent a
number with infinite  precision?'' then the answer  would have to  be:
``Yes, it does matter''.  Small approximations caused  by the fact the
we  represent numbers  using  ``only'' a finite set  of  digits can be
amplified in the process of solving a given mathematical problem up to
a point where they make the entire result meaningless.

It is in this sense that numerical analysis  teaches us humbleness: it
forces us to  realise our limitations  in representing  numbers and to
ask ourselves the question of whether the results obtained by applying
exact mathematical procedures to  these inexact representations can be
considered accurate.

This chapter is an introduction to numerical methods.   Its aims are
to highlight the consequences of finite precision representation of
numbers on numerical calculations.

\section{Floating point representation}

Real    numbers are represented     in   digital  computers   using  a
\textit{normalised scientific   notation} or   \textit{floating  point
notation}: the  decimal point is shifted  and appropriate powers of 10
are  supplied so that  all the digits are to  the right of the decimal
point and the first digit displayed is not zero.  For example,
%
\begin{align*}
  8491.2913 & = 0.84912913 \times 10^4 , \\
  -12.32 & = -0.1232 \times 10^2 , \\
  0.00321 & = 0.321 \times 10^{-2} .
\end{align*}
%
The general notation for a floating point number (in base $10$) is
%
\begin{equation*}
 \pm \, 0.a_1 a_2 a_3 a_4 \ldots a_m \times 10^c ,
\end{equation*}

%
where $0.a_1 a_2 a_3 a_4 \ldots a_m$ is the \textit{mantissa}, the
integer $c$ is the \textit{exponent} and the integer $10$ is
called the \textit{base} of this representation.  The number $m$ of
digits in the mantissa is a measure of the precision of the
representation: the bigger $m$ the more digits are stored in the
computer and the more accurately the real number is approximated.  The
digits $a_i$ are called ``significant digits'': $a_1$ is the most
significant digit and $a_m$ is the least significant digit.  The size
of the mantissa is also referred to as ``the number of significant
digits''.

\section{Absolute and relative errors}

Before discussing how the errors introduced in the representations of
numbers affect numerical calculations we must define ways of measuring
the size of the error.  There are two standard definitions.    We define
the \textit{absolute error} in representing the number $x$ with a
finite precision representation $\bar{x}$ as the quantity
%
\begin{equation*}
  E_a = x - \bar x.
\end{equation*}

%
We define the \textit{relative error} as the quantity
%
\begin{equation*}
  E_r = \frac{x - \bar x}{x} .
\end{equation*}
%
The absolute error is independent of the size of the number $x$, while
the relative error is essentially a measure of the size of the error
in comparison with the quantity measured.

\medskip

\noindent {\bf Example}

\smallskip

Suppose that $x=1322$ and $\bar x = 1000$.  The absolute and relative
errors are respectively:
%
\begin{align*}
  E_a &= 1322 - 1000 = 322, \\
  E_r &= \dfrac{1322 - 1000}{1322} = 0.24 .
\end{align*}
%
Now suppose that $x=1000322$ and $\bar x = 1000000$.   The two errors
are:
%
\begin{align*}
  E_a &= 1000322 - 1000000 = 322, \\
  E_r &= \dfrac{1000322 - 1000000}{1000322} = 0.0032 = 0.32
  \times 10^{-2} .
\end{align*}
%
The absolute error is the same in both cases, but the relative errors
are completely different.    The choice between the two errors is
problem dependent.

\medskip

One of the aims of numerical analysis is to control the growth of the
errors that occur naturally while solving a given mathematical
problem.     In other words, it is the \textit{total error from all
sources} which is of most interest.   Normally, we try, when examining
a method, to prove a theorem of the form

\begin{quotation}
  \textit{If we apply method X to solve problem Y then $ | x - \bar x | \le \varepsilon \quad \forall x$.}
\end{quotation}

The crux of numerical analysis is that some problems are such that
even a small amount of rounding or initial error causes a large
inaccuracy in the answer: these are known as \textit{ill-conditioned
problems}.

\section{Errors and simple mathematical operations}

Before studying algorithms to solve typical numerical problems, like
finding the roots of a nonlinear equation, we must understand how
numerical errors affect and are affected by simple operations.

\subsection{Addition}

The error in computing the addition of two real numbers by adding up
two floating point numbers is given by
%
\begin{equation*}
  x + y = \overline{x} + \overline{y} + e(x) + e(y),
\end{equation*}
%
where $e(x)$ and $e(y)$ are respectively the errors in the floating
point representation of $x$ and $y$.

The addition of two floating point numbers loses most of the
properties of the standard addition.   In the examples that follow
assume that the size of the mantissa is $m=3$:

\begin{enumerate}
  %
\item In general, the sum of two floating point numbers is not the
  floating point representation of the sum:
  %
  \begin{equation*}
    \overline{x+y} \ne \bar x + \bar y.
  \end{equation*}

  %
  For example,
  %
  \begin{align*}
    x = 1.007, \, y = 2.005 &\implies \bar x = 1.00, \,
    \bar y = 2.00, \\
    \intertext{whilst}
    \overline{x+y} = \overline{3.012} = 3.01 & \ne \bar x
    + \bar y = 3.00
  \end{align*}
  %
\item The sum of two numbers may be too big to be represented in the
  computer, i.e.\ the result of the addition is a numerical overflow.

\item Order matters, i.e.\ the addition of floating point numbers is
  not associative:
  %
  \begin{equation*}
    \overline{\overline{(\overline{x} + \overline{y})} + \overline{z}} \ne
    \overline{\overline{x} + \overline{(\overline{y} + \overline{z})}}.
  \end{equation*}
  %
  As a matter of fact, it is always better to add in ascending order
  of magnitude.
  %
  \begin{equation*}
    \left\{
      \begin{aligned}
        \overline{x} &= 1.00, \\
        \overline{y} &= 0.007, \\
        \overline{z} &= 0.006, \\
      \end{aligned}
    \right. \implies
    \left\{
      \begin{aligned}
        \overline{\overline{(\overline{x} + \overline{y})} +
          \overline{z}} = \overline{\overline{(1.00+0.007)} +
          0.006} =
        \overline{1.00 + 0.006} = 1.00, \\
        \overline{\overline{x} + \overline{( \overline{y} +
            \overline{z} )}} = \overline{ 1.00 +
          \overline{(0.007+0.006)}} = \overline{1.00 + 0.013} =
        1.01
      \end{aligned}
    \right.
  \end{equation*}
  %
  The second result is more accurate than the first.  If each small
  number is added separately to a big number then they may be all
  ``chopped off''.  However, if all the small numbers are added
  together their sum may be sufficiently big not to suffer such fate.

\end{enumerate}

\subsection{Subtraction}

The error in computing the difference of two real numbers by subtracting
two floating point numbers is given by
%
\begin{equation*}
  x - y = \overline{x} - \overline{y} + e(x) - e(y),
\end{equation*}

%
where $e(x)$ and $e(y)$ are respectively the errors in the floating
point representation of $x$ and $y$.

The subtraction of two floating point numbers is a very delicate
operation that can lead to an outstanding loss of significant digits
especially when the numbers to be subtracted are nearly equal.
If, for example, $x = 42.345$ and $y = 42.287$ then we have:
%
\begin{align*}
  x - y &= 0.058 , \\
  \overline{x} - \overline{y} &= 42.3 - 42.2 = 0.100
\end{align*}

\subsection{Multiplication}

The error in computing the product of two real numbers by multiplying
two floating point numbers is given by
%
\begin{equation*}
  x \, y = \overline{x} \, \overline{y} + \overline{y} e(x) +
  \overline{x} e(y) + e(x) e(y),
\end{equation*}
%
where $e(x)$ and $e(y)$ are respectively the errors in the floating
point representation of $x$ and $y$.   The last term in this
expression is usually negligible.

Note that on an $m$ digit computer,  $\overline{x} \, \overline{y}$ is at
most $2 m$ digit long.  This is the main reason why most computers use
$2 m$ digits for calculations on $m$-digit numbers.  The result is cut
back to $m$ digits once the calculation is completed.

\subsection{Division}

The error in computing the ratio of two real numbers by dividing
two floating point numbers is given by
%
\begin{equation*}
  \frac{x}{y} =
  \frac{\overline{x} + e(x)}{\overline{y} + e(y)} =
  \frac{\overline{x}}{\overline{y}} + \frac{e(x)}{\overline{y}} -
  \frac{\overline{x} e(y)}{\overline{y}^2} + O [e(x) e(y)],
\end{equation*}
%
where $e(x)$ and $e(y)$ are respectively the errors in the floating
point representation of $x$ and $y$.   The last term in this
expression is usually negligible.   Clearly the error increases
dramatically if $y \simeq 0$.

% The symbol $O(\varepsilon)$ [big oh of  $\varepsilon$] is a short hand
% notation for ``terms that decrease to zero as fast as $\varepsilon$''.
% More formally, a function  $f(\varepsilon)$ is $O(\varepsilon)$ if the
% ratio  $f(\varepsilon)/\varepsilon$ is  finite  and bounded  away from
% zero as $\varepsilon$ tends to zero:
% %
% \Be
%   0 < \left | \frac{f(\varepsilon)}{\varepsilon} \right | < \infty ,
%   \qquad \varepsilon \to 0 .
% \Ee
% %
% For example, $\sin(x) = O(x)$ because
% %
% \Be \lim_{x \to 0} \frac{\sin(x)}{x} = 1 . \Ee
%
% There is a similar symbol, $o(\varepsilon)$ [little oh of
% $\varepsilon$]: it is a short hand notation for ``terms that decrease
% to zero faster than $\varepsilon$''.  More formally a function
% $f(\varepsilon)$ is $o(\varepsilon)$ if
% %
% \Be
%    \lim_{\varepsilon \to 0} \frac{f(\varepsilon)}{\varepsilon} = 0.
% \Ee
% %
% For example, $\sinh^2(x) = o(x)$ because
% %
% \Be \lim_{x \to 0} \frac{\sinh^2(x)}{x} = 0 . \Ee

\noindent
\textbf{Remark} - The errors introduced by the basic arithmetical
operations can be combined when studying more complicated operations.
For example, one can show that in the dot product of two vectors the
error is roughly $n \varepsilon$, where $n$ is the size of the two vectors and
$\varepsilon$ is an upper bound on the error of the representation of each
component of the vectors.

\section{Stable and unstable computations}

The procedure to solve numerically a mathematical problem involves a
series of operations that, quite often, must be repeated over and over
again.    It may happen that a small error gets amplified in the
course of the iteration procedure until it dominates the numerical
solution and makes it totally unreliable.

A numerical process is \textit{unstable} if small errors made at one
stage of the process are magnified in subsequent stages and seriously
degrade the accuracy of the overall solution.

Consider for example the sequence of real numbers defined by
%
\begin{equation}
  \label{unstable}
  \left\{
    \begin{aligned}
      x_0 & = 1, \\
      x_1 & = \dfrac{1}{3}, \\
      x_{n+1} & = \dfrac{13}{3} x_n - \dfrac{4}{3} x_{n-1}, \quad n \ge 1 .
    \end{aligned}
  \right.
\end{equation}
%
One can show by induction that this recurrence relation generates the
sequence
%
\begin{equation*}
  x_n = \left ( \frac{1}{3} \right )^n .
\end{equation*}
%
However, if we compute~(\ref{unstable}) numerically using 8
significant digits we obtain the results shown in Table~\ref{unst1}.
The absolute and relative error increase with the iterations until
after the seventh iteration the result is completely unreliable.
The algorithm is therefore \textit{unstable}.

\begin{table}
 \begin{center}
  \begin{tabular}{|c|c|c|c|} \hline
   $n$ & $x_n$ & Abs. err. & Rel. err. \\ \hline \hline
    0 & +1.0000000e+00 & +0.0000000e-01 & +0.0000000e-01 \\ \hline
    1 & +3.3333333e-01 & +0.0000000e-01 & +0.0000000e-01 \\ \hline
    2 & +1.1111110e-01 & +1.0000000e-08 & +9.0000000e-08 \\ \hline
    3 & +3.7036990e-02 & +4.7000000e-08 & +1.2690000e-06 \\ \hline
    4 & +1.2345490e-02 & +1.8900000e-07 & +1.5309000e-05 \\ \hline
    5 & +4.1144710e-03 & +7.5530000e-07 & +1.8353790e-04 \\ \hline
    6 & +1.3687210e-03 & +3.0211000e-06 & +2.2023819e-03 \\ \hline
    7 & +4.4516310e-04 & +1.2084270e-05 & +2.6428298e-02 \\ \hline
    8 & +1.0407880e-04 & +4.8336990e-05 & +3.1713899e-01 \\ \hline
    9 & -1.4254266e-04 & +1.9334792e-04 & +3.8056671e+00 \\ \hline
   10 & -7.5645659e-04 & +7.7339168e-04 & +4.5668005e+01 \\ \hline
   11 & -3.0879216e-03 & +3.0935666e-03 & +5.4801604e+02 \\ \hline
   12 & -1.2372384e-02 & +1.2374266e-02 & +6.5761923e+03 \\ \hline
   13 & -4.9496435e-02 & +4.9497062e-02 & +7.8914304e+04 \\ \hline
   14 & -1.9798804e-01 & +1.9798825e-01 & +9.4697166e+05 \\ \hline
   15 & -7.9195293e-01 & +7.9195300e-01 & +1.1363660e+07 \\ \hline
  \end{tabular}
 \end{center}
 \caption{\label{unst1} \it Iterations of the unstable
   map~(\ref{unstable}) with initial conditions $x_0=1$ and
   $x_1=1/3$.   The correct value of the $n$-th iterate is $x_n=(1/3)^n$.}
\end{table}

The instability of this algorithm is caused by the fact that an error
present in $x_n$ is multiplied by 13/3 in computing $x_{n+1}$ (the
proof of instability is more complicated than this, but this statement
is essentially correct).  Hence there is a possibility that the error
in $x_1$ will be propagated into $x_{15}$, for example, with a factor
$(13/3)^{14}$.    Since the absolute error in $x_1$ is approximately
$10^{-8}$ and since $(13/4)^{14}$ is roughly $10^9$, the error in
$x_{15}$ due solely to the error in $x_1$ could be as much as 10.

Whether a process is numerically stable or unstable should be decided
on the basis of \textit{relative} errors.  Thus if there are large
errors in the computations, that situation maybe quite acceptable if
the answers are large.  In the preceding example, let us start with
initial values $x_0=1$ and $x_1=4$.  The recurrence
relation~(\ref{unstable}) is unchanged and therefore errors will still
be propagated and magnified as before.  But the correct solution is
now $x_n=4^n$ and the results of the computation are correct to seven
significant figures (see Table~\ref{unst2}).  In this case the correct
values are large enough to overwhelm the errors.  The absolute errors
are undoubtedly large (as before), but they are relatively negligible.

\begin{table}
  \begin{center}
    \begin{tabular}{|c|c|c|c|} \hline
      $n$ & $x_n$ & Abs. err. & Rel. err. \\ \hline \hline
      0 & +1.0000000e+00 & +0.0000000e-01 & +0.0000000e-01 \\ \hline
      1 & +4.0000000e+00 & +0.0000000e-01 & +0.0000000e-01 \\ \hline
      2 & +1.6000000e+01 & +0.0000000e-01 & +0.0000000e-01 \\ \hline
      3 & +6.4000000e+01 & +0.0000000e-01 & +0.0000000e-01 \\ \hline
      4 & +2.5600000e+02 & +0.0000000e-01 & +0.0000000e-01 \\ \hline
      \vdots & \vdots & \vdots & \vdots \\ \hline
      15 & +1.0737418e+09 & +0.0000000e-01 & +0.0000000e-01 \\ \hline
      16 & +4.2949672e+09 & +1.0000000e+02 & +2.3283064e-08 \\ \hline
      17 & +1.7179868e+10 & +1.0000000e+03 & +5.8207661e-08 \\ \hline
      18 & +6.8719471e+10 & +6.0000000e+03 & +8.7311490e-08 \\ \hline
      19 & +2.7487788e+11 & +3.0000000e+04 & +1.0913936e-07 \\ \hline
      20 & +1.0995115e+12 & +1.0000000e+05 & +9.0949470e-08 \\ \hline
    \end{tabular}
  \end{center}
  \caption{\label{unst2} \it Iterations of the unstable
    map~(\ref{unstable}) with initial conditions $x_0=1$ and
    $x_1=4$.   The correct value of the $n$-th iterate is $x_n=4^n$.}
\end{table}

\smallskip

\noindent
\textbf{Remark} - The errors we have discussed until now are
independent of the mathematical problem we wish to solve and are
inherent to the numerical procedure that we are using.   However,
there are mathematical problems that are intrinsically ``difficult'',
in the sense that a small alteration of one of the parameters of the
problem (for example, one of the coefficients of a system of linear
equations) alters dramatically the value of the solution.   Such
problems are called \textit{ill conditioned} and we will discuss them
as we encounter them.


%%%%%%%%%%%%%%%%%%%%%%%%
%
%   Thesis template by Youssif Al-Nashif
%
%   May 2020
%
%%%%%%%%%%%%%%%%%%%%%%%%

\section{Graph Kernels for Similarity}

\hspace*{0.3cm} Graph kernels can be used to compare the similarity of two graphs. The development of these methods arose out of the need to determine if graphs were isomorphic in a faster way. The solution was a graph kernel which produces a scalar value for how similar two, or more, graphs are. The result of a graph kernel is a matrix where the similarity of graphs $i$ and $j$ is in the kernel's $i$-th row and $j$-th column entry. This resulting matrix can be used in a variety of ways, but it will be used as a distance matrix in applications here. \\

Amongst graph kernels which consider edge or vertex labels in their computation, various surveys and studies have found edge label histogram kernels or vertex label histograms to be the most efficient. They may not out perform the other methods, like a Weisfeiler-Lehman or other subgraph methods, but they are computationally cheap. The datasets being studied here are both large: one contains many smaller graphs, and one contains 48 very large graphs. So for this study, computation efficiency was prioritized. \\

The edge label histogram kernel is the graph kernel that was chosen for these datasets, and it can be computed using either a linear kernel or a Guassian radial basis function (RBF) kernel between the edge label histograms. \\
An edge label histogram is defined as $\vec{g} = (g_1,g_2, ... g_i)$ such that $g_i = | \{ (u,v) \in E | \phi(u,v) = i \} |$ for each $i$ \cite{sugiyama2015halting}. Where $g_i$ is a histogram bin for a unique edge label's magnitude, $E$ is the set of edges, and $\phi$ is a function that maps each label to a scalar value in the range of unique values. The edge label histograms are then passed through a kernel, either a linear kernel or a Gaussian RBF kernel.\\
Computation using a linear kernel takes two graphs, $G$ and $G'$, and uses their edge label histograms $\vec{g}$ and $\vec{g'}$. The kernel is computed as:

\begin{equation}
K(\vec{g},\vec{g'}) = \vec{g}^{T}\vec{g'}
\end{equation}

The resulting value is stored in the graph kernel matrix as the measure of similarity between the two graphs in the corresponding row and column for the pair \cite{sugiyama2015halting}.

Alternatively, the Gaussian RBF kernel takes the edge label histograms of $G$ and $G'$, that we call $\vec{g}$ and $\vec{g'}$, and the kernel is computed as:

\begin{equation}
K(\vec{g},\vec{g'}) = e^{- \left( \frac{||\vec{g}- \vec{g'}||^2}{2 \sigma^2} \right) }
\end{equation}

The resulting value is stored in the graph kernel matrix as the measure of similarity between the two graphs in the corresponding row and column for the pair. Through either of these kernels, we obtain a kernel of dimensions $n \times n$ for a list of $n$ graphs. This kernel can then be used for clustering methods.

% https://papers.nips.cc/paper/2015/file/31b3b31a1c2f8a370206f111127c0dbd-Paper.pdf


\chapter{Regularized Regression: Lasso, Ridge, and Elastic Net \label{chapter:lassoridge}}

In Chapter~\ref{chapter:featureselection}, we discussed embedded feature selection methods. These methods incorporate feature selection into the process of model training. Decision trees and tree ensembles (boosted trees, random forests) naturally perform feature selection in the process of model training, and will simply ignore irrelevant features.

Regression methods, however, have a harder time. When people began to want to apply regression methods to supervised learning problems with large numbers of predictors -- particularly when the number of predictors was greater than the number of samples, or when the predictors were highly correlated -- they faced serious problems of overfitting and model instability. This led to the development of \textbf{regularized regression} (a.k.a. \textbf{penalized regression}) methods, which introduce penalty terms into the objective functions being optimized in the model fitting process to try to control the size of the coefficients assigned to various predictors and/or set some of them to zero. The goal of these methods is to allow researchers to stick with the machinery of regression models while reducing the possibility of overfitting.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Linear Regression}

Linear regression (Chapters~\ref{chapter:regression}, \ref{chapter:linreg}, and \ref{chapter:glms}) uses the mean-squared loss:
$$ \text{loss} = \sum_{i=1}^N (y^{(i)} - \beta^T x^{(i)})^2 $$
where $\beta^T x^{(i)} = \beta_0 + \beta_1 x_1^{(i)} + \dots + \beta_p x_p^{(i)}$ is the model's prediction for the $i$th training example. To minimize the loss, we adjust the $\beta$s to make the model's predictions close to the true $y^{(i)}$ values. This is equivalent to maximizing the likelihood (see Section~\ref{section:mleglms}).

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Logistic Regression}

Logistic regression (Chapters~\ref{chapter:classification}, \ref{chapter:logreg}, and \ref{chapter:glms}) uses the negative binomial log-likelihood as its loss (compare the expression below to the one in Chapter~\ref{chapter:glms}:
$$ \text{loss} = -\sum_{i=1}^n \left[ y^{(i)} \beta^T x^{(i)} - \log \left(1 + \exp(\beta^T x^{(i)}) \right) \right]. $$
The expression has a negative sign in front because high log-likelihood is a good thing; we want low values of the log-likelihood to correspond to high values of the loss.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Lasso, Ridge, and Elastic Net Penalties}

In situations where there is a risk of overfitting and model instability (e.g., highly correlated predictors, more predictors than training examples), one can apply a penalty term to the loss function to prevent the model coefficients from taking on unrealistically large or small values.

\subsection{Lasso Penalty}

The \textbf{Lasso}, or \textbf{L1}, penalty is related to the sum of the absolute values of the coefficients:

$$ \text{penalty}_\text{Lasso} = \lambda \sum_{j=1}^p \vert \beta_j \vert $$

This penalty will tend to cause the model to perform feature selection, setting some of the $\beta$s to zero. The values of the others may shrink, or they may be unaffected.

\subsection{Ridge Penalty}

The \textbf{ridge}, or \textbf{L2}, penalty is related to the sum of the squared values of the coefficients:

$$ \text{penalty}_\text{ridge} = \lambda \sum_{j=1}^p \beta_j^2 $$

This penalty will tend to shrink the values of all of the model coefficients without setting any to zero.

\subsection{Elastic Net Penalty}

The \textbf{elastic net} penalty is just a weighted sum of the Lasso and ridge penalties:

$$ \text{penalty}_\text{EN} = \lambda \left[ \alpha \sum_{j=1}^p \vert \beta_j \vert + (1-\alpha)  \sum_{j=1}^p \beta_j^2 \right] $$

The parameter $\alpha$ governs the relative weights of the two penalties. In practice, $\lambda$ and $\alpha$ are set using cross-validation. The value of $\lambda$ is unconstrained, but $\alpha$ must lie between 0 and 1.

\begin{question}{}
This picture, from \emph{Elements of Statistical Learning} (Figure 6.7) is a geometric picture of what happens to the coefficients under the Lasso and ridge penalties.
\begin{center}
\includegraphics[width=0.8\textwidth]{img/esl-lasso-ridge-6.7.png}
\end{center}
Which picture is which? What are the red ellipses? What are the blue shapes? Do you see why Lasso is more likely than ridge to set some of the coefficients to zero?
\end{question}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Example: Predicting Blood Pressure}

Imagine we have collected blood pressure data on $10$ patients. In addition, we have information on the patients' sex, age, and obesity status.

{\small
\begin{center}
\begin{tabular}{cccccc}
  \toprule
 & sexf & sexm & age & obesity & blood\_pressure \\
  \midrule
  1 & 0 & 1 & 54 & 1 & 123 \\
  2 & 1 & 0 & 66 & 0 & 111 \\
  3 & 1 & 0 & 23 & 0 & 98 \\
  4 & 0 & 1 & 59 & 1 & 154 \\
  5 & 0 & 1 & 76 & 1 & 199 \\
  6 & 1 & 0 & 33 & 0 & 101 \\
  7 & 0 & 1 & 35 & 1 & 91 \\
  8 & 1 & 0 & 54 & 0 & 133 \\
  9 & 1 & 0 & 21 & 0 & 116 \\
  10 & 0 & 1 & 26 & 0 & 121 \\
  \bottomrule
\end{tabular}
\end{center}
}

Note: These data are from an example in the paper ``Predictive analytics with gradient boosting in clinical medicine'', by Zhang et al, published in \emph{Annals of Translational Medicine} in 2019.

\vspace{2mm}
\begin{question}{}
Which two predictors in this model are clearly correlated?
\end{question}

We will now try many different values of $\lambda$ and $\alpha$ on these data and see what happens to the coefficients. To do this, we use the \texttt{glmnet} package in R. Here are the results:

{\small
\begin{center}
\begin{tabular}{rrrrrrr}
  \toprule
 & lambda & alpha & beta\_sexf & beta\_sexm & beta\_age & beta\_obesity \\
  \midrule
1 & 0.0 & 0.0 & -24.61 & -0.00 & 1.15 & -13.78 \\
  2 & 1.0 & 0.0 & -11.19 & 10.71 & 1.09 & -9.97 \\
  3 & 5.0 & 0.0 & -8.25 & 8.24 & 0.91 & -2.11 \\
  4 & 10.0 & 0.0 & -6.97 & 6.99 & 0.77 & 1.66 \\
  5 & 0.0 & 0.2 & -24.61 & -0.00 & 1.15 & -13.78 \\
  6 & 1.0 & 0.2 & -10.35 & 9.73 & 1.07 & -7.84 \\
  7 & 5.0 & 0.2 & -6.81 & 6.72 & 0.88 & 0.00 \\
  8 & 10.0 & 0.2 & -6.10 & 6.07 & 0.76 & 0.00 \\
  9 & 0.0 & 0.5 & -24.61 & -0.00 & 1.15 & -13.78 \\
  10 & 1.0 & 0.5 & -9.09 & 8.04 & 1.04 & -4.38 \\
  11 & 5.0 & 0.5 & -5.65 & 5.50 & 0.87 & 0.00 \\
  12 & 10.0 & 0.5 & -3.84 & 3.77 & 0.71 & 0.00 \\
  13 & 0.0 & 1.0 & -24.61 & -0.00 & 1.15 & -13.78 \\
  14 & 1.0 & 1.0 & -13.15 & 0.00 & 1.00 & 0.00 \\
  15 & 5.0 & 1.0 & -6.93 & 0.00 & 0.84 & 0.00 \\
  16 & 10.0 & 1.0 & 0.00 & 0.00 & 0.62 & 0.00 \\
   \bottomrule
\end{tabular}
\end{center}
}

\begin{question}{}
Which values for $\lambda$ and $\alpha$ correspond to: (a) unregularized linear regression, (b) pure Lasso, (c) pure ridge, (d) elastic net with an even combination of Lasso and ridge penalties? What happens to the values of the coefficients in each case?
\end{question}


\chapter{Graph Games}
\marginurl{%
    Graph Games:\\\noindent
    Introduction to Combinatorial Game Theory \#4
}{youtu.be/71fSkXp4DJc}

This section gives an alternative definition of a combinatorial game.
This definition allows us to study general combinatorial games.
\begin{definition}
  A \emph{directed graph} $G$ is a pair $(V, N)$ such that $V$ is a non-empty
  set and $N : V \to 2^V$.\footnote{%
    We are going to have a more in-depth
    discussion of graphs in \Cref{part:graph-theory}.
  }

  We say that a game on $G$ is the impartial game where elements of $V$ are
  positions and each player can move from $x \in V$ to any $y \in N(x)$.
  (Elements of $N(x)$ are called followers of $x$.)
\end{definition}

\begin{remark}
  It is also easy to see that any impartial game can be transformed into a graph
  $G$ such that the game on $G$ and the impartial game are equivalent.
\end{remark}

For example, the take-away game from \Cref{chapter:combinatorial-games}
can be considered as a graph on a graph $G = (\N_0, N)$,
where $N(0) = \emptyset$, $N(1) = \set{0}$, $N(2) = \set{0, 1}$, and
$N(n + 3) = \set{n, n + 1, n + 2}$ for any $n \in \N_0$.

The key ingredient for the analysis of games based on graphs was proposed
by Sprague and Grundy. They proposed to consider the following function:
\begin{definition}
  Let $G = (V, N)$ be a directed graph. A function $g : V \to \N$ is a
  \emph{Sprague--Grundy function} for $G$ iff
  $g(x) = \mex\set[y \in N(x)]{g(y)}$, where
  $\mex S = \min \set[n \notin S]{n \in \N_0}$.
\end{definition}

Consider the following graph (arrows depict possible moves).
\begin{center}
\begin{tikzpicture}[thick]
  \node[circle, draw, minimum size=6pt] (v1) at (0,0) {$1$};
  \node[circle, draw, minimum size=6pt] (v2) at (1,1) {$2$};
  \node[circle, draw, minimum size=6pt] (v3) at (-1,1) {$3$};
  \node[circle, draw, minimum size=6pt] (v4) at (0,2) {$4$};

  \draw[->] (v2) -- (v3);
  \draw[->] (v1) -- (v3);
  \draw[->] (v4) -- (v3);
  \draw[->] (v4) -- (v2);
\end{tikzpicture}
\end{center}
Let us assume that $g$ is a Sprague--Grundy function for this graph.
Note that $3$ is a terminal position so $g(3) = \mex \emptyset = 0$.
Since from $1$ and $2$ there are only moves to $3$, it is clear that
$g(1) = g(2) = \mex \set{0} = 1$. Finally, $g(4) = \mex \set{0, 1} = 2$.

Note that the Sprague--Grundy function is recursively defined so it may
not exist or not to be unique if graph violates the ending condition.
For example, the graph depicted on \Cref{figure:sprague-grundy-not-exists}
does not have a Sprague--Grundy function.
Indeed, assume that such a function $g$ exists. Consider two following cases.
\begin{itemize}
    \item First case is when $g(3) = 0$. Note that $g(2) = \mex \set{0} = 1$.
        Hence, $g(1) = \mex \set{1} = 0$ which contradicts the assumption
        that $g(3) = 0$ since $g(3) = \mex \set{g(0)} = 1$.
    \item Second case is when $g(3) \neq 0$. Note that $g(2) = \mex \set{g(3)} = 0$.
        Therefore $g(1) = \mex \set{0} = 1$ and $g(3) = \mex \set{1} = 0$ which
        is a contradiction.
\end{itemize}

Note that the graph depicted on \Cref{figure:sprague-grundy-not-unique} has
several Sprague--Grundy functions. We may consider functions $g_1$ and $g_2$
such that $g_1(1) = g_1(3) = g_2(2) = g_2(4) = 0$ and
$g_2(1) = g_2(3) = g_1(2) = g_1(4) = 1$. It is clear that they are
Sprague--Grundy functions for the graph from
\Cref{figure:sprague-grundy-not-unique}.

\begin{figure}
    \centering
    \subfloat[A graph without a Sprague--Grundy function
              \label{figure:sprague-grundy-not-exists}]{
      \begin{tikzpicture}[thick]
        \node[circle, draw, minimum size=6pt] (v1) at (0,0) {$1$};
        \node[circle, draw, minimum size=6pt] (v2) at (1,1) {$2$};
        \node[circle, draw, minimum size=6pt] (v3) at (-1,1) {$3$};
        \node[] (dummy) at (2.25,1) {};
        \node[] (dummy) at (-2.25,1) {};

        \draw[->] (v1) -- (v2);
        \draw[->] (v2) -- (v3);
        \draw[->] (v3) -- (v1);
      \end{tikzpicture}
    }
    \qquad
    \subfloat[A graph with several Sprague--Grundy functions
              \label{figure:sprague-grundy-not-unique}] {
      \begin{tikzpicture}[thick]
        \node[circle, draw, minimum size=6pt] (v1) at (1,1) {$1$};
        \node[circle, draw, minimum size=6pt] (v2) at (0,0) {$2$};
        \node[circle, draw, minimum size=6pt] (v3) at (-1,1) {$3$};
        \node[circle, draw, minimum size=6pt] (v4) at (0,2) {$4$};
        \node[] (dummy) at (2.25,1) {};
        \node[] (dummy) at (-2.25,1) {};

        \draw[->] (v1) -- (v2);
        \draw[->] (v2) -- (v3);
        \draw[->] (v3) -- (v4);
        \draw[->] (v4) -- (v1);
      \end{tikzpicture}
    }
    \caption{Graphs where Sprague--Grundy function is either not unique or does
      not exist.}
    \vskip 10pt
\end{figure}

Unfortunately, even if there are no cycles, a graph my not have a
Sprague--Grundy function or have several Sprague--Grundy functions. Indeed,
consider the graph $G = (\Z, N)$ such that $N(x) = \set{x - 1}$. It is clear
that the functions $g_1$ and $g_2$ such that
\begin{gather*}
    g_1(x) =
    \begin{cases}
        0 & \text{if } x \text{ is even} \\
        1 & \text{if } x \text{ is odd}
    \end{cases} \\
    \text{and} \\
    g_2(x) =
    \begin{cases}
        0 & \text{if } x \text{ is even} \\
        1 & \text{if } x \text{ is odd}
    \end{cases}
\end{gather*}
are Sprague--Grundy functions for $G$.

\begin{exercise}
    Let $G = (\N \cup \set{\infty}, N)$ such that
    $N(x) = \set[y < x]{y \in \N_0}$, and $N(\infty) = \N$.
    Show that $G$ does not have a Sprague--Grundy function.
\end{exercise}

However, for all the combinatorial games we are going to consider
the Sprague--Grundy function exists and is unique.
\begin{theorem}
\label{theorem:ending-condition-sg-function}
  Let $G = (V, N)$ be a graph such that $N(v)$ is finite and $G$ satisfies the
  ending condition. Then Sprague--Grundy function for $G$ exists and it is
  unique.
\end{theorem}

Before we prove this theorem, let us illustrate by proving that the
Sprague--Grundy function for \Cref{game:take-away-21-3-2-1} is unique.
Assume that a Sprague--Grundy function $g$ for \Cref{game:take-away-21-3-2-1}
exists. We are going to show that it is unique.
Note that if $x$ is a terminal position, then $g(x) = 0$. Hence, $g(0) = 0$.
There is only one move from $1$ so $g(1) = \mex \set{0} = 1$. Similarly
there are two moves from $2$: one to $1$ and one to $0$ so
$g(2) = \mex \set{0, 1} = 2$. In the same way $g(3) = \mex \set{0, 1, 2} = 3$
and $g(4) = \mex \set{1, 2, 3} = 0$.
One may notice that there is a pattern and conjecture that
\[
  g(x) =
  \begin{cases}
      0 & \text{if } x \equiv 0 \pmod{4} \\
      1 & \text{if } x \equiv 1 \pmod{4} \\
      2 & \text{if } x \equiv 2 \pmod{4} \\
      3 & \text{if } x \equiv 3 \pmod{4}
  \end{cases}.
\]
We already proved the base case, let us now prove the induction step.
Assume the equality is true for all $y < x$ and consider the following cases.
\begin{itemize}
    \item If $x \equiv 0 \pmod{4}$, then $x - 1 \equiv 3 \pmod{4}$,
        $x - 2 \equiv 2 \pmod{4}$, and $x - 3 \equiv 1 \pmod{4}$.
        Hence, $g(x) = \mex \set{1, 2, 3} = 0$.
    \item If $x \equiv 1 \pmod{4}$, similarly $g(x) = \mex \set{2, 3, 0} = 1$.
    \item If $x \equiv 2 \pmod{4}$, $g(x) = \mex \set{3, 0, 1} = 2$.
    \item If $x \equiv 3 \pmod{4}$, $g(x) = \mex \set{0, 1, 2} = 3$.
\end{itemize}
It is also clear that the constructed function is indeed a Sprague--Grundy
function for \Cref{game:take-away-21-3-2-1}. Therefore, we proved its
existence and uniqueness.
\begin{table}
  \centering
  \begin{tabular}{l l l l l l l l l}
      \toprule
      0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 \\
      \midrule
      0 & 1 & 2 & 3 & 0 & 1 & 2 & 3 & 0 \\
      \bottomrule
  \end{tabular}
  \caption{The Sprague--Grundy function for \Cref{game:take-away-21-3-2-1}}
  \label{table:take-away-21-3-2-1-grundy}
  \vskip 10pt
\end{table}

Note that in this proof we used a procedure very similar to the one we used to
find P- and N-positions.
\begin{template}
  \textbf{Steps necessary to find a Sprague--Grundy function} \\

  Assume we are trying to construct a a Sprague--Grundy function $g$.
  \begin{enumerate}
    \item Set $g(x) = 0$ for all terminal positions $x$.
    \item If $g(x)$ is not defined but $g(y)$ is defined for all $y \in N(x)$,
      then set $g(x) = \mex \set[y \in N(x)]{g(y)}$.
    \item If $g(x)$ is not defined for some $x$, go to Step~2.
  \end{enumerate}
\end{template}

Let $V_0$ be the set of terminal positions, and let $V_i$ be the set of $x$ such
that we define $g(x)$ on the $i$th iteration of the procedure.
The following lemma essentially says that this procedure defines $g$ everywhere.
\begin{lemma}[K\H{o}nig]
\label{lemma:konig}
  Let $G = (V, N)$ be a graph such that $N(v)$ is finite for all $v \in V$ and
  $G$ satisfies the ending condition.
  Let $V_0$ be the set of terminal positions, and let $V_{n + 1} =
  \set[N(v) \subseteq V_n]{v \in V}$. Then $V = \bigcup_{n \in \N_0} V_n$.
\end{lemma}

\begin{proof}[Proof of \Cref{theorem:ending-condition-sg-function}]
  Let $V_0$ be the set of terminal positions, and let $V_{n + 1} =
  \set[N(v) \subseteq V_n]{v \in V}$.
  \Cref{lemma:konig} claims that $V = \bigcup_{n \in \N_0} V_n$.

  For each $n \in \N_0$, we define $g_n : V_n \to \N_0$ such that
  $
    g_{n + 1}(v) = \mex \set[u \in N(v)]{g_n(u)}
  $
  for $v \in V_{n + 1}$ and $g_0(v) = 0$ for $v \in V_0$. It is easy to see
  that the function $g : V \to \N_0$ such that $g(v) = g_n(v)$ for $v \in V_n$
  is a Sprague--Grundy function for $G$.

  To finish the proof we need to prove that there are no other Sprague--Grundy
  functions for $G$. Assume that $g'$ is a Sprague--Grundy function for
  $G$; we prove using induction by $n \in \N_0$ that $g(v) = g'(v)$ for
  $v \in V_n$. The base case for $v = 0$ is clear since $N(v) = \emptyset$ for
  $v \in V_0$. Let us consider the induction step from $n$ to $n + 1$. By the
  induction hypothesis, $g(u) = g'(u)$ for $u \in V_n$. Let us consider some $v
  \in V_{n + 1}$. Note that
  \[
    g'(v) = \mex \set[u \in N(v)]{g(u)} = \mex \set[u \in N(v)]{g(u)} = g(v).
  \]
  Therefore $g(v) = g'(v)$ for $v \in V_{n + 1}$. As a result, $g(v) = g'(v)$
  for $v \in V$; i.e., $g = g'$.
\end{proof}

One may note that in \Cref{game:take-away-21-3-2-1} P-positions are the positions
where the Sprague--Grundy function is zero. In fact, this is not a coincidence.
\begin{theorem}
\label{theorem:grundy-to-np}
  Let $G = (V, N)$ be a graph such that $N(v)$ is finite for all $v \in V$ and
  $G$ satisfies the ending condition. Then all the vertices of $G$ can be
  labeled as either P- or N-positions. Moreover, $v \in V$ is a P-position iff
  $g(v) = 0$, where $g$ is the Sprague--Grundy function for $G$.
\end{theorem}

\begin{chapterendexercises}
    \exercise Prove \Cref{theorem:grundy-to-np}.
    \exercise Show that there are only two Sprague--Grundy functions for the
      graph depicted on \Cref{figure:sprague-grundy-not-unique}.
    \exercise Prove that there is unique Sprague--Grundy function for the one pile
        Nim game.
    \exercise Prove that there is unique Sprague--Grundy function for the
        subtraction game where players may subtract $2$ and $3$ chips on their
        turn.
    \exercise
       Prove that there is unique Sprague--Grundy function for the subtraction
       game where players may subtract $1$, $2$, or $5$ chips
       on their turn.
\end{chapterendexercises}


\chapter{Additional Pipeline Information}
\section{Derivations of the Flow Variance}
\label{sec:derivation_flow_var}
Flow estimation can be noisy for various reasons and the problem is that the nois is often proportional to the magnitude of the absolud flow. thus using a simple flow difference as ameasure will constantly underestimate the simmilarity between heigh magnitude flow. Hence, let us assume
\begin{equation}
\begin{aligned}
\colvec{u_1}{v_1} = f + \colvec{x_1}{y_1} \\
\colvec{u_2}{v_2} = f + \colvec{x_2}{y_2}
\end{aligned}
\label{eq:def_flow_tracking}
\end{equation}

where f is the true flow and
\begin{equation}
	\colvec{x_1}{y_1} \sim \colvec{x_2}{y_2}
\end{equation}

Let us assume, that the random variables $x_1$ and $x_2$ are i.i.d. distributed having a zero mean and a variance $\sigma_x^2$, $x_1$ and $x_2$ respectively, i.e.

\begin{equation}
\begin{aligned}
x_1 \sim x_2 (0, \sigma_x^2) \\
y_1 \sim y_2 (0, \sigma_y^2
\end{aligned}
\label{eq:def_flow_tracking}
\end{equation}

\begin{equation}
\begin{aligned}
\mathbf{E} \left[ \norm{\colvec{u_1}{v_1} - \colvec{u_2}{v_2}} \right]
& = \mathbf{E} \left[ (u_1 - u_2)^2 \right] + \mathbf{E} \left[ (v_1 - v_2)^2 \right] \\
& = \mathbf{E} \left[ u_1^2 - 2 u_1 u_2 + u_2^2 \right] + \mathbf{E} \left[ v_1^2 - 2 v_1 v_2 + v_2^2 \right] \\
& = \mathbf{E} \left[ x_1^2 \right] + \mathbf{E} \left[ x_2^2 \right] + \mathbf{E} \left[ y_1^2 \right] + \mathbf{E} \left[ y_2^2 \right] \\
& = 2 \left( \sigma_x^2 + \sigma_y^2 \right)
\end{aligned}
\label{eq:flow_variance_formula}
\end{equation}
In the last step of Equation $\ref{eq:flow_variance_formula}$ we used the abbreviation $\sigma_x^2$ which denotes the variance of the optical flow along its $x$ direction. In this derivation we exploited the fact that $x_1$ and $x_2$ are independent random variables, $y_1$ and $y_2$ respectively. Hence, the term $\mathbf{E} \left[ x_1 x_2\right]$ yields the value zero.

\chapter{Additional Theoretical Background}
\section{On Filtering Images}
\subsection{Gaussian Filter}
A Gaussian filter (\textbf{GF}) is a linear operator that reduces noise by smoothing the image. Applying GF corresponds to a low-pass filtering.
At each position it estimates a local average of the intensities as defined in Equation $\ref{eq:gaussian_filtering_def}$.
\begin{equation}
	\mathcal{F}^{\text{gf}} \{I \} (p) = \sum_{q \in \Omega_p} G_{\sigma} (\norm{p - q}) I_q
\label{eq:gaussian_filtering_def}
\end{equation}
where $I$ defines the input image and $\Omega_p$ contains all neighboring points within the window that are centered at the image point $p$. Moreover, $G_{\sigma}$ denotes the two dimensional Gaussian kernel, which is defined in Equation $\ref{eq:def_g_weight}$.
\begin{equation}
	G_{\sigma} (x) = \frac{1}{2 \pi \sigma^2} e^{-\frac{x^2}{2 \sigma^2}}
\label{eq:def_g_weight}
\end{equation}
The Gaussian filtering is the weighted average of the intensity of the adjacent positions with a weight decreasing with the spatial distance to the center position. This distance is defined by $G_{\sigma} (\norm{p - q})$ where $\sigma$ is a parameter that defines the size of the neighborhood. The resulting image is a blurred version of the input image.

\subsection{Bilateral Filter}
\label{sec:bilateral_filter}
A Bilateral filter (\textbf{BF}) is a non-linear operator that reduces noise by smoothing the image but at the same time preserves its edges. \\ \\
The rational of this filter is that two pixels are close to each other if not only if their spacial distance is small but also if they are similar regarding their intensity range.
\begin{figure}[H]
\begin{center}
\subfigure[Raw Image]{
   \includegraphics[width=0.47\linewidth] {background/filtering/l_raw}
}
\subfigure[Filtered Image]{
   \includegraphics[width=0.47\linewidth] {background/filtering/l_filtered}
}
\end{center}
\caption[Example Bilateral Filter]{A bilateral filtering example: On the left side a noisy input image on the right its bilateral filtered version.}
\label{fig:bilat_filtering_eg}
\end{figure}
The filter replaces the intensity values at each pixel in an image by a weighted average of intensity values from nearby pixels. In our formulation we rely on the Gaussian distributions $G$ for defining the weights. Crucially, the weights depend not only on distances between pixels, but also on the intensity difference. That is why, when iterating through each pixel and adjusting weights to the adjacent pixels accordingly, Sharp edges are preserved. A bilateral filtering example$\footnote{All illustrated results were generated by using our own Bilateral filter implementation. For further information please visit our repository at \url{https://github.com/simplay/master_thesis}}$ is shown in Figure $\ref{fig:bilat_filtering_eg}$.\\ \\
The mathematical definition of this filter is given in Equation $\ref{eq:def_bilateral_filter}$. For a given Image $I$ we want to compute its bilateral filtered version by applying the following definition:
\begin{equation}
\begin{aligned}
&\mathcal{F}^{\text{bf}} \{I \} (p) = \frac{1}{W_p} \sum_{q \in \Omega_p} \underbrace{G_{\sigma_s} (\norm{p-q})G_{\sigma_r} (I_p - I_q)}_{w_p} I_q \\
& \text{where } W_p = \sum_{q \in \Omega_p} G_{\sigma_s} (\norm{p-q})G_{\sigma_r} (I_p - I_q)
\end{aligned}
\label{eq:def_bilateral_filter}
\end{equation}
The set $\Omega_p$ contains all neighboring points within the window that are centered at the image point $p$. The scalar $W_p$ denotes the normalization factor of the filter and $I_p$ represents the image intensity at the pixel position $p$. Please notice that the definition of the bilateral filter is given per pixel, i.e. gives an representation of the filtered pixel intensity. \\ \\
The bilateral filter is controlled by the two parameters $\sigma_s$ and $\sigma_r$. The range variance increases $\sigma_r$, the BF becomes closer to the Gaussian blur filter. In other words, the larger $\sigma_r$ the more weight a pixel with a large intensity deviation gets. However, when increasing the spatial variance $\sigma_s$ results in smoothing larger features, i.e more distant pixels get a larger weight and thus influence the result more. Figure $\ref{fig:bfilter_influence_sigmas}$ demonstrates the influence of these parameters.
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.6\linewidth] {background/filtering/bilat_filter_sigmas}
\end{center}
\caption[Influence of $\sigma_s$ and $\sigma_r$]{Different filtering results produced by our bilateral filter implementation for varying $\sigma_s$ and $\sigma_r$ values.}
\label{fig:bfilter_influence_sigmas}
\end{figure}

\subsection{Harris Corner Detector}
\label{sec:harris_corner_detector}
In this section we explain the principles of the Harris corner detector $\cite{Harris88acombined}$. Let us, therefore, consider a grayscale image $I$. We are going to sweep a window $w(x,y)$ (with displacements u in the x direction and v in the right direction) I and will calculate the variation of intensity. Since we are looking for windows with corners, we are looking for windows with a large variation in intensity. Hence, we have to maximize the equation above, specifically the term:
\begin{equation}
	E \left( u, v \right) = \sum_{x,y} w \left( x,y \right) \left[ I(x + u, y + v) - I(x, y) \right]^2
\label{eq:var_intensitiy_def}
\end{equation}
Next, the term $I(x + u, y + v)$ is expressed by the first order taylor series expansion as the follows:
\begin{equation}
	I(x + u, y + v) = I(x,y) + u I_x (x, y) + v I_y (x,y) + \text{h.o.t}
\label{eq:taylor_exp_intensity}
\end{equation}
Next, we put the first order approximation of Equation $\ref{eq:taylor_exp_intensity}$ into Equation $\ref{eq:var_intensitiy_def}$ to simplify the definition of $E$.
\begin{equation}
\begin{aligned}
E \left( u, v \right)
&= \sum_{x,y} w \left( x,y \right) \left[ I(x + u, y + v) - I(x, y) \right]^2 \\
&\approx \sum_{x,y} w \left( x,y \right) \left[ I(x,y) + u I_x (x, y) + v I_y (x,y) - I(x, y) \right]^2 \\
&= \sum_{x,y} w \left( x,y \right) \left[ u I_x (x, y) + v I_y (x,y) \right]^2 \\
&= \sum_{x,y} w \left( x,y \right) u^2 I_x^2 + 2 u v I_x I_y v^2 I_y^2 \\
&= \left( u,v \right) \left( \sum_{x,y} w (x,y)
\begin{pmatrix}
I_x^2 & I_x I_y \\
I_x I_y & I_y^2 \\
\end{pmatrix}
\right) \colvec{u}{v}
\end{aligned}
\label{eq:var_intensitiy_developed}
\end{equation}
Let us define the following substitution
\begin{equation}
M = \sum_{x,y} w  (x,y)
\begin{pmatrix}
I_x^2 & I_x I_y \\
I_y^2 & I_x I_y \\
\end{pmatrix}
\label{eq:var_intensity_sub}
\end{equation}
Putting the substitution from Equation $\ref{eq:var_intensity_sub}$ into the final form of Equation $\ref{eq:var_intensitiy_developed}$ we obtain the final form
\begin{equation}
	E \left( u, v \right) \approx \left( u,v \right) M \colvec{u}{v}
\end{equation}.
A score is calculated for each window, to determine if it can possibly contain a corner:
\begin{equation}
\begin{aligned}
& R = \det(M) - \kappa \left(\text{trace}(M)\right)^2 \\
&\text{where } \det(M) = \lambda_1 \lambda_2 \text{ and } \text{trace}(M) = \lambda_1 + \lambda_2
\end{aligned}
\label{eq:harris_response}
\end{equation}

\section{On Statistics}
\label{sec:on_statistics_bg}

\subsection{Conditional Probability}
Given two events $A$ and $B$ with the probability $P(B) > 0$. The conidtional probability of $A$ given $B$ is defined as
\begin{equation}
	P(A|B) = \frac{P(A \cap B)}{P(B)}
\label{eq:conditional_prob}
\end{equation}
Furthermore, Equation $\ref{eq:conditional_prob}$ gives us an alternative interpretation of the probability of an intersection
\begin{equation}
P(A \cap B) = P(A|B)P(B)
\end{equation}
The probability that two events happen the same time is the same as the Probability of the event A given B times the probability of event B. Sticking to this definition, we can visualize all possible outcomes of two events happening the same time as shown in Figure $\ref{fig:prob_tree_diagram}$.
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.45\linewidth] {background/statistics/probability_tree_diagram}
\end{center}
\caption[Probability Tree Diagram]{Tree liked representation$\footnotemark$ of the four possible outcomes for a conditional probability.}
\label{fig:prob_tree_diagram}
\end{figure}
\footnotetext{The visualized graphic has been taken from: \url{https://en.wikipedia.org/wiki/File:Probability_tree_diagram.svg}}

\subsection{Evaluating Binary Classifiers}
In this section we explain the mathematical framework we use in our evaluation.

The purpose of a binary classifier is to classify given elements into two groups according to a specified classification rule. Think about a binary random variable modelling a certain event. The actual observations of the random variable are then either equals true or false. A classification$\footnote{I.e. the produced output after applying the classifier on the given data.}$ of a given dataset yields two numbers: the number of the positives and negatives, which add up to the size of the set. \\ \\
In order to evaluate the quality of binary classifier its prediction is compared against a standard reference method or, if existing, against a ground truth assignment and then cross tabulates the data into a $2 \times 2$ contingency table as shown in Figure $\ref{tab:prediction_sensitivity}$.
\begin{table}[H]
\centering
\begin{tabular}{c|c|c|}
\cline{2-3}
 & \begin{tabular}[c]{@{}l@{}}Prediction\\ Positive\end{tabular} & \begin{tabular}[c]{@{}l@{}}Prediction\\ Negative\end{tabular} \\ \hline
\multicolumn{1}{|l|}{\begin{tabular}[c]{@{}l@{}}Condition\\ Positive\end{tabular}} & \cellcolor[HTML]{34FF34}{\color[HTML]{000000} $\bf{TP}$ } & \cellcolor[HTML]{CB0000} $\bf{FN}$ \\ \hline
\multicolumn{1}{|l|}{\begin{tabular}[c]{@{}l@{}}Condiation\\ Negative\end{tabular}} & \cellcolor[HTML]{CB0000}{\color[HTML]{000000} $\bf{FP}$ } & \cellcolor[HTML]{34FF34} $\bf{TN}$ \\ \hline
\end{tabular}
\caption[Conditional Probability]{The four possible outcomes of a conditional probability}
\label{tab:prediction_sensitivity}
\end{table}
There are four possible outcomes: the classifier prediction, which has either are positive or negative condition, was actually correct or incorrect. \\ \\
Let us consider the following example where we test some people for the presence of a disease. Some of these people have the disease, and the test correctly detects them. These findings are called true positives (\textbf{TP}). Some have the disease, but the test incorrectly claims that they do not have it. These results are called false negatives (\textbf{FN}). Some people do not suffer from the disease and the test classifies correctly as health. These results are called true negatives (\textbf{TN}). And Lastly, some healthy people who are incorrectly classified as infected. These are the so called false positives (\textbf{FP}) results. \\ \\
There are many metrics that can be used to measure the performance of a classifier. In the following a listing of some prominent metrics
%
\begin{itemize}
\item \textbf{Precision}: Tells us what proportion of patients we diagnosed as having the disease actually had that disease. In other words, proportion of TP in the set of positive disease diagnoses. This is given by the rightmost column in the confusion matrix.
\begin{equation}
	\text{precision} = \frac{\text{TP}}{\text{TP} + \text{FP}}
\label{eq:def_precision}
\end{equation}
\item \textbf{Recall}: Tells us what proportion of people that actually had the disease were diagnosed by the test as having the disease. In other words, proportion of TP in the set of true disease states. This is given by the bottom row in the confusion matrix.
\begin{equation}
	\text{recall} = \frac{\text{TP}}{\text{TP} + \text{FN}}
\label{eq:def_recall}
\end{equation}
\end{itemize}
A visualization of these measures is given in Figure $\ref{fig:eval_concept_recall_precc}$.
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.6\linewidth] {evaluation/prec_recall}
\end{center}
\caption[Concept Recall and Precision]{This figure illustrates graphically the concept of Recall (\textbf{R}) and Precision (\textbf{P}). When estimating the sampling of a binary random variable, there are basically four possible outcomes: The we predicted it being true and it is true (\textbf{TP}), we predicted it true but it is false (\textbf{FP}), we predicted it false and it was actually negative (\textbf{FN}) or we predicted it to be true but it was actually false (\textbf{TN}). }
\label{fig:eval_concept_recall_precc}
\end{figure}
An alternative measure is the $F_1$ score. This measure is often used in determining the performance of classification tasks in machine learning. To compute its score, this measure takes into account both, the precision and the recall of the test. The exact definition of the F1 measure is given in Equation $\ref{eq:f1_score}$.
\begin{equation}
F_1 = 2 \left( \frac{\text{precision} \times \text{recall}}{\text{precision} +\text{recall}} \right)
\label{eq:f1_score}
\end{equation}
The F1 score can be interpreted as a normalized weighted average of the precision and recall measures. The best possible F1 score is equals 1 and its worst value is 0.

\section{Camera Model}
\subsection{Pinhole Camera}
\label{sec:pinhole_camera}
A camera projects a 3d scene in a 2d image. A simple mathematical model to describe such a mapping from the 3d scene space to the 2d image space is the pinhole camera model. This model is conceptually illustrated in Figure $\ref{fig:pinhole_camera_model}$.
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.8\linewidth] {background/camera_model/pinhole_camera}
\end{center}
\caption[Pinhole Camera Model]{Visualizing the pinhole camera model. The camera center $C$ represents an infinitely small aperture hole and assumed to be located at $(0,0,0)$, $f$ is the focal length in pixel units, $p$ is the principal point in images coordinates, $\textbf{A} = (X, Y, Z)$ a 3d point in the scene and $\textbf{a} = (f \frac{X}{Z}, f \frac{Y}{Z})$ its projected pixel version mapped onto the image plane.}
\label{fig:pinhole_camera_model}
\end{figure}
The aperture is an infinitely small hole and there is no lens used to focus light. Therefore, lens distorting or depth of field effects are ignored. The intersection point where all rays meet is the center of the perspective projection and usually referred by \textit{camera center}. The principal axis is formed by the line perpendicular to the image plane, which passes though the camera center. Its intersection point with the image plane is called principal point. The distance between the camera center and the principal point is the so called focal length.

\subsection{Camera Parameters}
\paragraph{Intrinsic Parameters} Let $\textbf{X} = (X,Y,Z,1)^T$ denote a point in a 3D scene and $\textbf{x} = (x,y,1)^T$ its projected version in the image plane. Both points are written in their homogeneous form. The mapping from a scene point to the image plane is defined as
\begin{equation}
\textbf{x} =
\begin{pmatrix}
f & s & p_x & 0 \\
0 & f & p_y & 0 \\
0 & 0 & 1 & 0
\end{pmatrix}
\textbf{X}
\end{equation}
where $f$ denotes the focal length and $p = (p_x, p_y)^T$ the offset to the origin in image coordinates with respect to the principal point. In modern cameras, the skew $s$ is usually equals zero. The parameters $f$, $p_x$, $p_y$ and $s$ are called \textit{intrinsic camera parameters}. Notice, that $\textbf{X}$ is expressed in terms of camera coordinates and similarly, $\textbf{x}$ in pixel coordinates.

\paragraph{Extrinsic Parameters} In general, a camera is not located at the origin of the world coordinate system and can have an arbitrary orientation. Let $\textbf{X}_w$ denote a point, living in an arbitrary camera coordinate system. To express this point in the regular camera coordinate system, we have to apply a certain transformation. In a rigid object model, the points $\textbf{X}$ and $\textbf{X}_w$ can be transformed to one another by applying a linear transformation that consists of a rotational matrix $R$ and a translation $t$. The actual mapping is defined as
\begin{equation}
	X = \left[ R | t \right] X_w
\end{equation}
The parameters $t$ and $R$ are called \textit{extrinsic camera parameters}.


\subsection{Projection algorithm}
\paragraph{}
Point projection algorithm is to find the nearest point in parameter $(u)$ on the NURBS curve of the test point.
In the proposed method, all points on the NURBS curve is generated based on the approximated polylines and hence are not exactly on the boundaries.
Although there exists a closed form solution for point projection, it require the order of the NURBS curve must be less than $4$ \citep{Pie1997}.
As a consequence, a projection algorithm \citep{MA200379} using Newton-Raphson method is introduced to tackle this problem.

\paragraph{}
For a given point $P=(x,y)$, its projection on the curve $C(u$ so that the distance $|P-C(u)|$ is minimum is targeted.
However, in the proposed method, the existence of the large number of the possible curves increase the computational cost significantly.
The projection point for the test point $P$ need to be determined for every existing curves and the one with the smallest minimum distance will be selected.
One possible improvement could be limit the possible curves to only a few by utilizing the fact that the NURBS curves has been divided into multiple sub-curves without interior knot by knot insertion introduced in Sec.~\ref{lr_sec:nurbs_knot_ins}
Another property that can be utilized is that most of the test point $P$ is expected to be extremely close to its projection on the curve $C(u)$.

\paragraph{}
As a consequence, the strong convex hull property can be adopted to limit the number of possible curves to less than $2$.
The building of the convex hull is explained in detail in Sec.~\ref{qdt_sc:convex_hull}.
The signed distance of the test point to all curves' convex hull is calculated and only the curves with negative signed distance which indicate that the point is in the convex hull will be selected as candidates.
If no negative distance is detected, a few number (taken as $3$ in the proposed method) of curves with minimum signed distance will be selected.

\paragraph{}
In order to find the projection of the test point $P$ on the curve $C$, target function $f$ can be expressed as
\begin{equation}
    f(u) = \mathbf{C}^\prime (u) \cdot (\mathbf{C}(u) - \mathbf{P})
\end{equation}
%
When $f(u)$ gives $0$, the point either located on the curve or the distance $|\mathbf{C}(u) - \mathbf{P}|$ is minimal.
and two scalars $f$ and $g$ are defined as
%
The iteration can be concluded as
\begin{equation}
    u_{i+1} = u_i -
    \frac{ f(u_i) }{ f^\prime(u_i) }
\label{qdt_eq:projection_iteration}
\end{equation}
After one iteration is finished, the following criteria are checked in sequence.
\paragraph{1}
Is the point coincide with $C(u_i)$
\begin{equation*}
    |\mathbf{C} (u_i) - \mathbf{P}| \leq \epsilon_1
\end{equation*}
%
where $\epsilon_1$ stands for the tolerance for distance in Euclidean space.
\paragraph{2}
Is the cosine zero
\begin{equation*}
    \frac{
        |\mathbf{C}^\prime (u) \cdot (\mathbf{C}(u) - \mathbf{P})|
    }{
        |\mathbf{C}^\prime (u)|
        |\mathbf{C}(u) - \mathbf{P}|
    } \leq \epsilon_2
\end{equation*}
%
where $\epsilon_2$ stands for the tolerance for cosine.
If either of these conditions are met, the iteration is terminated.
Otherwise Eq.~\ref{qdt_eq:projection_iteration} is performed to find the parameter $u_{i+1}$ for next iteration.
\paragraph{3}
Make sure $u$ and $v$ are within there domains
\begin{equation*}
    u_{i+1} \in [a,b]
\end{equation*}
%
where $a$ and $b$ are the lower and upper bounds for the knot vector of curve $C$.
If the curve is open
\begin{equation}
    \left\{
        \begin{array}{rl}
            u_{i+1} = a & u_{i+1} < a \\
            u_{i+1} = b & u_{i+1} > b
        \end{array}
    \right.
\end{equation}
%
If the curve is closed
\begin{equation}
    \left\{
        \begin{array}{rl}
            u_{i+1} = b - ( a - u_{i+1} ) & u_{i+1} < a \\
            u_{i+1} = a + ( u_{i+1} - b ) & u_{i+1} > b
        \end{array}
    \right.
\end{equation}
%
\paragraph{4}
The difference between the new parameter $u_{i+1}$ and the old one $u_i$ is insignificant
\begin{equation*}
    |
        (u_{i+1} - u_i)\mathbf{C}^\prime(u_i)
    |
    \leq \epsilon_1
\end{equation*}
The iteration will be terminated if this condition is meet.

\subsection{Convex hull in 2D}
\label{qdt_sc:convex_hull}
\paragraph{}
The convex hull property of the NURBS curve indicates that all points on the curve must be contained within the convex hull constructed by its control points \citep{SELIMOVIC2009772}
There are great number of algorithm that can be used including gift wrapping \citep{Cormen:2009:IAT:1614191}, graham scan \citep{ANDERSON197853}, quick hull \citep{Barber:1996:QAC:235815.235821}, Chan's algorithm \citep{Chan1996} and so on \citep{doi:10.1137/0215021, ANDREW1979216}.
The quick hull is adopted in the proposed as it provides a computationally efficient and stable algorithm.
The algorithm utilize the idea of `` divide and conquer'' to build the convex hull with an expected time complexity of $O(nlog(n))$ and  $O(n^2)$ for the worst case.
Generally speaking, it works as expected in most of the situation except for the case of high symmetry or most of the points located at the circumference of a circle.
The algorithm can be implemented with following steps:
\begin{enumerate}
    \item Find the most left and right points (points with minimal and maximum $x$) since they are proved to be part of the convex hull.
    \item Connect these two points and use the line to separate other points into two group.
    \item Find the point with maximum distance to the line in step 2 in any group.
    \item Construct a triangle with two points in step 2 and the point in step 3.
    \item Eliminate all points contained by these two subsets in step 4.
    \item Repeat the previous three steps and the distance calculated in step 2 is determined as the point to the triangle instead of the line in step 1.
    \item Terminate the iteration when no points are left
\end{enumerate}


\section*{Neural Networks}
\subsection*{Learning features}
Parameterize the feature maps and optimize over the parameters:\\
$w^* = \underset{w, \theta}{\operatorname{argmin}} \sum_{i=1}^n l(y_i; \sum_{j=1}^m w_j \phi(x_i, \theta_j))$\\
One possibility: $\phi(x,\theta) = \varphi(\theta^T x) = \varphi(z)$

\subsection*{Activation functions}
Sigmoid: $\varphi(z) = \frac{1}{1+exp(-z)}$\\
Tanh: $\varphi(z) = tanh(z) = \frac{exp(z)-exp(-z)}{exp(z)+exp(-z)}$\\
ReLu:  $\varphi(z) = max(z,0)$

\subsection*{Forward propagation}
For each unit $j$ on input layer, set value $v_j=x_j$\\
For each layer $l=1:L-1$: For each unit $j$ on layer $l$ set its value $v_j = \varphi(\sum_{i\in Layer_{l-1}} w_{j,i}v_i)$ For each unit $j$ on output layer, set its value $f_j = \sum_{i\in Layer_{L-1}} w_{j,i}v_i$\\
Predict $y_j = f_j$ for reg. / $y_j = sign(f_j)$ for class.

\subsection*{Backpropagation}
For each unit $j$ on the output layer:\\
- Compute error signal: $\delta_j = \ell_j'(f_j)$\\
- For each unit $i$ on layer $L$: $\frac{\partial}{\partial w_{j,i}} = \delta_j v_i$

For each unit $j$ on hidden layer $l=\{L-1,...,1\}$:\\
- Error signal: $\delta_j = \varphi'(z_j) \sum_{i\in Layer_{l+1}} w_{i,j}\delta_i$\\
- For each unit $i$ on layer $l-1$: $\frac{\partial}{\partial w_{j,i}} = \delta_j v_i$

\subsection*{Learning with momentum}
$a \leftarrow m \cdot a + \eta_t \nabla_W l(W;y,x)$; $W \leftarrow W - a$


\documentclass{article}

\input{../../FrequentlyUsed/latex/mydefs}
\input{../../FrequentlyUsed/latex/mypackages}

\title{Yun Family Math Problems}
\author{Sunghee Yun}

\begin{document}
\maketitle

\section{Jisun's pizza problem (21-MAR-2020)}

While we were chatting over \href{https://zoom.us/}{Zoom} on 21-MAR-2020,
Jisun brought up the below problem:
\begin{quote}
Suppose that you have a round pizza and an analog watch or clock.
How can you divide it into 11 (exactly) equal pieces?
\end{quote}
Though this problem looked really interesting,
I myself could not figure it out.
Taehoon said you can erase the digit $12$,
which was a very creative idea.
However, nobody could give the correct question to it.

On 23-MAR-2020, Jisun asked again, ``Has anybody solved the pizza problem?'' over KakaoTalk.
I googled it and found the following answer.

\begin{center}
\includegraphics[width=.7\linewidth]{figures/pizze_cut_prob_answer_by_googling}
\end{center}

And Jisun confirmed that that is the right answer.

Then my tendency to want to prove things rigorously
made me have to spend my time on writing the below.
However, I'm pretty sure that this will be helpful for my nephews (and niece),
so this would definitely not waste of time. :)

Here I want to write two different ways to see this, the first of which is kind of repeat what is written above.

\begin{proof}
\begin{enumerate}
\item
One way to prove this can go as follows.

If you position the hour hand (HH) and the minute hand (MH) at noon
and let the watch operator as it usually does,
then the two hands do not meet until 1PM (the left clock in \figurename~\ref{fig:1})
because the MH goes faster than the HH.
Now if the time passes a bit more,
there comes a moment that the two hands meets exactly (the right clock in \figurename~\ref{fig:1}).
Now mark this position (or angle) on the pizza.
Then wait until the two hands meet again and mark that position, too.
You repeat this until the two meets again at midnight.

\begin{figure}
\begin{center}
\includegraphics[width=.7\linewidth]{figures/pizza_HH_MH}
\end{center}
\caption{The locations of hour and minute hands at 1PM (left) and those when it meets the next time (right).}
\label{fig:1}
\end{figure}

Now ask how many times the two hands have met after noon, \ie,
exclude noon.
Since they met a bit after 1PM, then a more bit after 2PM, \ldots,
and after 11PM. When they met after 11PM is at midnight.
Thus, \emph{they have met 11 times}.

Because you can insist that the time between two consecutive \emph{rendezvious} of the two hands is the same for every time,
this divides $360$ degrees into 11 equal angles.

\item
Now let us examine this quantitatively.
Let $x$ be the angle between HH at midnight and HH when it meets MH next time as shown in \figurename~\ref{fig:2}.
Because HH goes to the position of $1$
when MH does one rotation,
\ie, the speed of MH is $12$ times faster then HH,
the angle $y$ in \figurename~\ref{fig:2} is $12$ times smaller than $x$,
or equivalently, $x$ is $12$ times larger than $y$.
Then we have
\begin{equation}
\label{eq:1}
x = 12 \times y.
\end{equation}

\begin{figure}
\begin{center}
\includegraphics[width=.4\linewidth]{figures/pizza_HH_MH_angles}
\end{center}
\caption{Pizza problem: $x$ refers to the angle between $12$ o'clock and where the minute hand meets the hour hand,
and $y$ refers to the angle between $1$ o'clock and where the minute hand meets the hour hand.}
\label{fig:2}
\end{figure}

Now we have two variables, $x$ and $y$, but only one equation. In this case, (in general) we cannot decide
unique values for the variables.
However, we have one more equation
since $x$ is equal to $y$ plus $360/12$.
Therefore we have
\begin{equation}
\label{eq:2}
x = y + \frac{360}{12}
\Leftrightarrow
12 \times x = 12 \times y + 360
\end{equation}
We obtained the right side equality
by multiplying $12$ on both sides of the left equality in (\ref{eq:2}).
Now if you replace $12\times y$ with $x$ in (\ref{eq:1}),
we obtain
\begin{equation}
12 \times x = x + 360
\Leftrightarrow
11 \times x = 360
\Leftrightarrow
x = 360/11.
\end{equation}
Therefore our method \emph{indeed} divides the pizza into $11$ equal-sized pieces.
\qed

An interesting observation is that this number $11$ comes from the calculation of $12 - 1$.
Thus, if half day were $13$ hours, the technique would give us $12$ pizza pieces,
and, \eg, if half day were $18$ hours, the technique would give us $17$ pizza pieces.
In general, if half day were $n$ hours, the technique would give us $n-1$ pizza pieces!
This type of generalization is a good practice to improve your math skills (\emph{I} believe).

\end{enumerate}

\end{proof}

\end{document}


\section{Theory}

The effect of any invariant linear system (LTI) on an arbitrary input  signal is
obtained by convolution of the input signal  with  the system's impulse response
function. In a LTI system, the  output  of the system $y(t)$ for an input $x(t)$
can be obtained by the convolution integral:

\begin{equation}
    y(t)=g(t)*x(t)=\int_0^t g(t-\tau)x(\tau)\,d\tau
    \label{eq:conv_int}
\end{equation}

where $g(t)$ is the \textit{impulse response}  of the system. That is, $g(t)$ is
the output of the system with an input $x(t)$ = $\delta(t)$,  where  $\delta(t)$
is the Dirac delta. The impulse response completely  characterizes  the  dynamic
behaviour of the system.

Applying   the   Laplace   transform  to  the  convolution  integral   (equation
\ref{eq:conv_int}) we obtain equation \ref{eq:laplace}:

\begin{equation}
    \laplace[y(t)] = \laplace[g(t) ∗ x(t)] = \laplace[g(t)] \laplace[x(t)]
\end{equation}

or in simple expression:

\begin{equation}
    Y(s) = G(s)X(s)
    \label{eq:laplace}
\end{equation}

where $Y(s)$, $G(s)$ and $X(s)$ are the Laplace transforms of $y(t)$, $g(t)$ and
$x(t)$ respectively. A Transfer Function (TF) is the mathematical representation
of  the  relation between the input and output of a system. In a LTI system,  TF
can be  expressed  as  the  ratio of the Laplace transform of the output and the
input, and  corresponds to the Laplace transform of the impulse response $G(s)$.

\begin{equation}
    G(s) = \frac{Y(s)}{X(s)}
\end{equation}

In order  to  obtain  $G(s)$ from an unknown system, a signal in the form of the
Dirac delta function  must  be applied to the input of the system and its output
must  be  measured.  Unfortunately, it is very hard to do this in most practical
cases, due to:

\begin{enumerate}
    \item Difficulty of generating a Dirac delta function (infinite amplitude, zero time).
    \item Any finite approximation of the Dirac delta function will cause an extremely small and hard to measure response signal.
\end{enumerate}

A far more practical method is to measure the  \textit{step response} instead. A
step function is easier to create in the physical world.  The  derivative of the
resulting measured signal will  very  closely  resemble  its theoretical impulse
response.

\input{sections/theory/model}
\input{sections/theory/characterisation_of_step_responses}
\input{sections/theory/approximating_step_response}
\input{sections/theory/least_squares}


\documentclass{article}

\usepackage{mathrsfs,amssymb,yfonts}

\begin{document}

\section{Let $A$ be a commutative ring. Let $M$ be a module, and $N$ a submodule. Let $N=Q_1 \cap ... \cap Q_r$ be a primary decomposition of $N$. Let $\bar{Q}_i = Q_i/N$. Show that $0 = \bar{Q}_1 \cap ... \bar{Q}_r$ is a primary decomposition of $0$ in $M/N$. State and prove the converse.}

1. Each $\bar{Q_i}$  is primary

Given $a \in A$, $a_{M/Q_i}$ is either injective or nilpotent, we must show that, given $a \in A$, $a_{ (M/N) / ( Q_i / N)}$ is either injective or nilpotent.

The function $a_{M/Q_i}$ is a particular function from $M/Q_i$ to itself. Via the isomorphism $\sigma: M/Q_i \mapsto (M/N) / ( Q_i / N)$, define $\hat{a}$ as a function from  $(M/N) / ( Q_i / N)$ to itself (Is just quoting the isomorphism theorem sufficient here?)

Since they are both multiplication by $a$, $a_{ (M/N) / ( Q_i / N)}$ and $\hat{a}$ are the same function on  $(M/N) / ( Q_i / N)$. Thus if $a_{M/Q_i}$ is injective (resp. nilpotent) then  $a_{ (M/N) / ( Q_i / N)}$ is injective (resp. nilpotent).

\noindent
2. Their intersection is $0 = \bar{Q}_1 \cap ... \cap \bar{Q}_r$

Assume this is false.

Take element $a \ne (0) \in \bar{Q}_1 \cap ... \cap \bar{Q}_r$. (Better way to write this?) The pre-image of $a$ under the canonical homomorphism $M \mapsto M/N$ is also not in $N$, since $N$ is exactly the kernel of this homomorphism.

However it $a$ *is* in each $\bar{Q}_i$ so its preimage has to be in $Q_i$, so has to be in $N$, a contradiction.

\noindent
3. If 0 is primary decom, then N is primary decomp

Showing that $Q_i$ is primary given that $\bar{Q}_i$ is primary is identical to part 1 by following the isomorphism $ M / Q_i \cong (M/N)/(Q_i/N)$ the other direction.

The proof that the intersection is N is also directly analogous to part 2, shown here:

Assume it is false.

Take some $x$ not in $N$ but in $N=Q_1 \cap ... \cap Q_r$. Under the canonical homomorphism the image of $x$ is in each of $\bar{Q}$, thus is $0$. However this is a contradiction since the image of $x$ is non-zero.

\section{Let $\mathfrak{p}$ be a prime ideal and $\mathfrak{a}$, $\mathfrak{b}$ be ideals of $A$. If $\mathfrak{a b} \subset \mathfrak{p}$, show that $\mathfrak{a} \subset \mathfrak{p}$ or $\mathfrak{b} \subset \mathfrak{p}.$}

If $\mathfrak{a} \not\subset \mathfrak{p}$ and $\mathfrak{b} \not\subset \mathfrak{p}$, prove that $\mathfrak{ab} \not\subset \mathfrak{p}$

Pick $a$ in $\mathfrak{a}$ and $b$ in $\mathfrak{b}$ but not in $\mathfrak{p}$. Since $\mathfrak{p}$ is prime, $ab$ cannot be in $\mathfrak{p}$.

\section{Let $\mathfrak{q}$ be a primary ideal. Let $\mathfrak{a}$, $\mathfrak{b}$ be ideals, and assume $\mathfrak{ab} \subset \mathfrak{q}$. Assume that $\mathfrak{b}$ is finitely generated. Show that $\mathfrak{a} \subset  \mathfrak{q}$ or there exists some positive integer $n$ such that $\mathfrak{b}^n \subset \mathfrak{q}$.}

Assume that $\mathfrak{a} \not\subset \mathfrak{q}$. We will show that this implies there there exists some positive integer $n$ such that $\mathfrak{b}^n \subset \mathfrak{q}$. Take $a_0$ to be in $\mathfrak{a}$ but not in $\mathfrak{q}$.

An arbitrary element of $\mathfrak{b}$, being finitely generated, looks like $k_1 b_1 + k_2 b_2 + ... + k_r b_r$ (where $k_i$ are positive integers and $r$ and $n$ are unrelated). For each $k_i b_i$, since $a_0 (k_i b_i)$ in $\mathfrak{q}$ but $a_0$ is not and $\mathfrak{q}$ is primary, $(k_i b_i)^{n_i} \in \mathfrak{q}$ for some finite $n_i$. Our goal is to find some $n$ large enough such that every term of $(k_1 b_1 + k_2 b_2 + ... + k_r b_r)^n$ is in $\mathfrak{q}$, since, being additively closed, that would imply $b^n$ itself were in $\mathfrak{q}$.

Take $n=\prod_{i=1}^{r} n_i$. Each term of $(k_1 b_1 + k_2 b_2 + ... + k_r b_r)^n$ is a homogenous monomial of degree $n$, thus in general looks like $\prod_{i=1}^{r} (k_i b_i)^{m_i}$ where the $m_i$'s sum to $n$.

If any $m_i \ge n_i$, then the term itself will be in $\mathfrak{q}$, since $(k_i b_i)^{n_i}$ is in $\mathfrak{q}$ as are its subsequent powers and anything multiplied by it, since ideals absorb multiplication.

It is impossible for all $m_i < n_i$, since then they would not fully sum to $n$, so therefore at least one is and the term is in $\mathfrak{q}$. Since $r$ is finite, so too is $n$, thus proving the statement.

\section{Let $A$ be Noetherian and let $\mathfrak{q}$ be a $\mathfrak{p}-$primary ideal. Show that there exists some $n \ge 1$ such that $\mathfrak{p}^n \subset \mathfrak{q}$}

$\mathfrak{q}$ being $\mathfrak{p}$-primary means that the radical of $\mathfrak{q}$ is $\mathfrak{p}$. Hence $\sqrt{\mathfrak{q}} = \{r \in A | r^n \in \mathfrak{q}\} = \mathfrak{p}$

Since $A$ is Noetherian, $\mathfrak{p}$ is finitely generated, so an arbitrarily element of $p$ is of the form $k_1 p1 + k_2 p_2 + ... + k_r p_r$, where $k_i$ and $r$ are positive integers.

For every $k_i p_i$, there's an $n_i$ such that $(k_i p_i)^{n_i}$ is in $\mathfrak{q}$. For the same reasoning as 3) above, $n=\prod_{i=1}^{r} n_i$ is such that this arbitrary $p$ is in $\mathfrak{q}$.

\break
\break

8. Let $A$ be a local ring. Show that any idempotent $\ne 0$ in $A$ is necessary the unit element.

We have to show that $e * e = e \Rightarrow e = 1$

If $e$ is a unit, we're done (Left multiplication by $e^{-1}$ shows $e = 1$).

So assume $e$ is not a unit. Then it must be in the maximal ideal $\mathfrak{m}$ (otherwise it would generate a proper ideal not contained in $\mathfrak{m}$). Not quite sure how to finish...

\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[portrait, margin=1in]{geometry}
\usepackage{cancel}
\usepackage{amsmath}
\usepackage{graphicx}
\begin{document}

\section{Introduction and simplification.}

This is the story of an infinite product integral that seems intimidating at first but is actually easier than it looks. Let's take a look:

\begin{equation}
    \int{ \frac{1}{x} \prod_{n=1}^{\infty} \left( 1-\tan^2{\frac{x}{2^n}} \right) dx}
\end{equation}

Recall some trig identities:

\begin{center}
    $1-\tan^2{(a)}=2-\sec^2{(a)}=\left(2\cos^2{(a)}-1\right)\sec^2{(a)}= \cos{(2a)}\sec^2{(a)}$
\end{center}

This simplifies our integral to:

\begin{equation}
\rightarrow \int{ \frac{1}{x} \prod_{n=1}^{\infty} \left( \cos{\left( \frac{x}{2^{n-1}}\right)} \sec^2{\left( \frac{x}{2^n} \right)}  \right) dx}
\end{equation}

Use this property of products: $\prod_{n}{a_{n}^k}=\left [ \: \prod_n{a_n} \: \right]^k$ to get:

\begin{equation}
    \rightarrow \int{ \frac{1}{x} \prod_{n=1}^{\infty}  \cos{\left( \frac{x}{2^{n-1}}\right)} \left[ \prod_{n=1}^{\infty}\sec{\left( \frac{x}{2^n} \right)}  \right]^2  dx}
\end{equation}

\section{Finding a way to simplify the infinite product.}
We seek to find the convergence of $\cos{ \left( \frac{x}{2^{n-1}} \right) }$ with respect to n:

\begin{equation}
    \lim_{k\rightarrow \infty} {\prod_{n=1}^{k}} \cos{\left( \frac{x}{2^{n-1}}\right)}=\cos{\left(x\right)}\lim_{k\rightarrow \infty} \prod_{n=1}^{k} \cos{\left( \frac{x}{2^n}\right)}
\end{equation}

Now we need to recall this simple trig identity:

\begin{center}
        $ \sin{(2a)}=2\sin{(a)}\cos{(a)}\rightarrow \cos{(a)}=\frac{\sin{(2a)}}{2\sin{(a)}}$
\end{center}

Redefine the limit to get:

\begin{equation}
    \Rightarrow\cos{\left(x\right)}\lim_{k\rightarrow \infty} \left [ \frac{\sin{\left(x\right)}}{2\sin\left(\frac{x}{2}\right)} \cdot  \frac{\sin{\left(\frac{x}{2}\right)}}{2\sin\left(\frac{x}{2^2}\right)} \cdot \frac{\sin{\left(\frac{x}{2^2}\right)}}{2\sin\left(\frac{x}{2^3}\right)} \; ... \; \frac{\sin{\left(\frac{x}{2^{k-1}}\right)}}{2\sin\left(\frac{x}{2^k}\right)} \right]
\end{equation}

Now this is interesting, because we are looking at a telescoping series of product:

\begin{equation}
    \Rightarrow\cos{\left(x\right)}\lim_{k\rightarrow \infty} \left [ \frac{\sin{\left(x\right)}}{2\; \cancel{\sin\left(\frac{x}{2}\right)}} \cdot  \frac{\cancel{\sin{\left(\frac{x}{2}\right)}}}{2\;\cancel{\sin\left(\frac{x}{2^2}\right)}} \cdot \frac{\cancel{\sin{\left(\frac{x}{2^2}\right)}}}{2\; \cancel{\sin\left(\frac{x}{2^3}\right)}} \; ... \; \frac{\cancel{\sin{\left(\frac{x}{2^{k-1}}\right)}}}{2\sin\left(\frac{x}{2^k}\right)} \right]
\end{equation}

So our product series ends with this:
\begin{equation}
    \Rightarrow\cos{\left(x\right)}\lim_{k\rightarrow \infty} \frac{\sin{(x)}}{2^k\sin{\frac{x}{2^k}}}
\end{equation}

Using L'Hospital's rule to evaluate the limit:

\label{s}
\begin{equation}
    \Rightarrow\cos{(x)}\sin{(x)}\lim_{k\rightarrow \infty} \frac{\frac{d}{dk}2^{-k}}{\frac{d}{dk}\sin{\frac{x}{2^k}}}=\cos{(x)}\sin{(x)}\lim_{k\rightarrow \infty} \frac{\cancel{-2^{-k}\ln{(2)}}}{\cancel{(-2^{-k}\ln{2})}x \cos{\frac{x}{2^k}}}=\boxed{\frac{\cos{(x)}\sin{(x)}}{x}}
\end{equation}

\newpage

\section{The final stretch.}

Recall our integral from before:

\begin{equation}
    \int{ \frac{1}{x} \prod_{n=1}^{\infty}  \cos{\left( \frac{x}{2^{n-1}}\right)} \left[ \prod_{n=1}^{\infty}\sec{\left( \frac{x}{2^n} \right)}  \right]^2  dx}
\end{equation}

Use the substitutions from before:
\begin{center}
        $ \prod_{n=1}^{\infty}  \cos{\left( \frac{x}{2^{n-1}}\right)} = \frac{\sin{(x)}\cos{(x)}}{x}$
\end{center}

And similarly:

\begin{center}
    $\left[ \prod_{n=1}^{\infty}\sec{\left( \frac{x}{2^n} \right)}  \right]^2 = \frac{x^2}{\sin^2{(x)}} $
\end{center}

Now it can be rewritten as:

\begin{equation}
        \int \frac{1}{\cancel{x}} \left[ \frac{\cos{(x)}\;\cancel{\sin{(x)}}}{\cancel{x}}\cdot\frac{\cancel{x^{2}}}{\sin^{\cancel{2}}{x}} \right] dx
        =\int{\cot{(x)}}\;dx = \ln{\left| \sin{x}\right|} + \textsc{c}
\end{equation}
\newline

Plot for $c = 0$:

\begin{center}
    \includegraphics{boi.jpg}
\end{center}
\end{document}


\graphicspath{{MDP/fig/}}

\chapter{Markov Decision Processes}
\label{chap:MDP}
\section{Introduction}

In order to describe reinforcement learning techniques, we first have to have a way of mathematically defining reinforcement learning problems. In this chapter, we do so by formally introduce the concept of Markov decision processes (MDPs). This chapter introduces the key concepts required to address the reinforcement learning problem. These concepts includes, state and action space, returns, value functions and Bellman equations.
The majority of the theory discussed in this chapter is based on the work by Sutton and Barto \cite{sutton_barto}.
\section{Definition of a MDP}
A Markov decision process is a stochastic, discrete, time controlled process. What this means is that a MDP is a partially controlled and partially random process \cite{sutton_barto}.
In general RL problems can be described using MDPs \cite{David_Silver}.
Paul Weng states that, MDP's are described by the following: state-space S, action-space A, state transition probability $P^{a}_{ss'}$, reward function $R^{a}_{s}$ and discount factor $\gamma$\cite{MDP_Paul_Weng}. These variables can be denoted in a 5-tuple as
\begin{equation}
	(S,A,P^{a}_{ss'},R^{a}_{s},\gamma)
\end{equation}
 where,
\begin{itemize}
	\item S is a finite set of states with state s $\in$ S
	\item A is a finite set of actions with action a $\in$ A
	\item $P^{a}_{ss'}$ is a matrix of probabilities which predicts the next state s', where\\ $P^{a}_{ss'} = P(S_{t+1} = s' | S_t = s,A_t = a) $
	\item $R^{a}_{s}$ is the immediate reward after transitioning from state s to s' using action a, where $R^{a}_{s} = E[R_{t+1}|S_t =s, A_t =a]$
	\item $\gamma$ $\in$ [0,1] is the discount factor applied to the reward
\end{itemize}
In an MDP, the assumption is that all the states have the Markov property \cite{sutton_barto}. This means that
\begin{equation}
	P[S_{t+1}|S_{t}] = P[S_{t+1}|S_{1},...,S_{t}].
\end{equation}
In other words, the probability of transitioning to state $S_{t+1}$ given state $S_t$, must be equal to the probability of transitioning to state $S_{t+1}$ given all states $S_1$ to $S_t$. This means that the current state is required to contain all the information of the previous states.

\section{Value function and policies}
The return $G_t$ is the total sum of the reward at each time step, multiplied by a constant diminishing factor $\gamma$.
The discount $\gamma$ $\in$ [0,1] determines the present value of future reward $R_{t}$.

\begin{align}
	G_t &= R_{t+1}+\gamma R_{t+2} +\gamma^2 R_{t+3}+ ... \nonumber\\
	&= \sum_{k=0}^{\infty}\gamma^{k} R_{t+k+1}
	\label{eq:G_t}
\end{align}

For $\gamma=0$, the return $G_t$ only depends on the single reward that can be obtained in the next step, namely $R_{t+1}$. When $\gamma=0$ the return $G_t$ is 'short-sighted'. For $\gamma=1$ on the other hand, the return depends on all the rewards that are projected to be obtained until the process terminates. $G_t$ is thus "far-sighted" for $\gamma=1$ \cite{sutton_barto}.
What can also note is that using a discount factor between 0 and 1 makes the return finite because $R_{t+k+1}$ is finite and
\begin{equation}
	\sum_{k=0}^{\infty}\gamma^{k}=\frac{1}{1-\gamma}\hspace{5pt},
\end{equation}
which is finite for $\gamma$ $\in$ [0,1]. \\
The probability that the agent will choose a specific action given the state is called its policy $\pi(a|s)$. Mathematically this is defined as
\begin{equation}
	\pi(a|s) = P[A_t = a, S_t = s].
\end{equation}
Another important definition is the value function v(s) defined as
\begin{equation}
	v(s) = E[G_t | S_t = s].
\end{equation}
This equation shows that the value function v(s) is the expected return given that v(s) is evaluated in state s at time-step t.
\section{Bellman Expectation Equation}
Sutton and Barto \cite{sutton_barto} decomposes v(s) so that it becomes a recursive function as follows
\begin{align}
	v(s) &= E[G_t | S_t = s]\\
	&= E[R_{t+1} + \gamma R_{t+2} + \gamma^{2} R_{t+3} + ...|S_t = s]\\
	&= E[R_{t+1} + \gamma (R_{t+2} + \gamma R_{t+3} + ...)|S_t = s]\\
	&= E[R_{t+1} + \gamma G_{t+1}|S_t = s]\\
	&= E[R_{t+1} + \gamma v(S_{t+1})|S_t = s].
	\label{bellmanv1}
\end{align}
This means that v(s) only depends on $R_{t+1}$ (the reward that can be obtained in the next step) and $\gamma v(S_{t+1})$, the discounted value function at time t+1.
What has been defined in equation \ref{bellmanv1} is known as the Bellman equation.\cite{sutton_barto}
Sutton and Barto then defines the value function given that the agent follows the policy $\pi$ as
\begin{equation}
	v_{\pi}(s) = E_{\pi}[G_t | S_t = s].
\end{equation}
Additionally, they define the action-value function (q-function),
\begin{equation}
	q_{\pi}(s,a) = E_{\pi}[G_t | S_t = s,A_t = a].
\end{equation}
The q-function is defined as the expected return of taking action a from state s and thereafter following policy $\pi$. What can be noted is that v(s) is a prediction of what the value for being in a certain state is, which is related to planning. While q(s,a) is the value for being in a certain state and taking an action, which relates to control.

Once again, Sutton and Barto \cite{sutton_barto} decomposes the state-value and action-value functions to be in the form of the Bellman Equation in equation \ref{bellmanv1}. This results in the Bellman \textit{expectation} equations
\begin{align}
	v_{\pi}(s)	&= E_{\pi}[R_{t+1} + \gamma v_{\pi}(S_{t+1})|S_t = s]\\
	&= \sum_{a'\in A}\pi(a|s)(R^{a}_s+\gamma\sum_{s'\in S}P^{a}_{ss'}v_\pi(s'))
	\label{bellmanv2}
\end{align}
and
\begin{align}
	q_{\pi}(s,a)	&= E_{\pi}[R_{t+1} + \gamma q_{\pi}(S_{t+1},A_{t+1})|S_t = s,A_t = a]\\
	&= R^{a}_s +\gamma \sum_{s'\in S}P^{a}_{ss'}\sum_{a'\in A}\pi(a'|s')q_\pi(s',a').
	\label{bellmanq}
\end{align}
Equation \ref{bellmanv2} can be placed into vector form as:
\begin{equation}
	v_\pi = R^{\pi} + \gamma P^{\pi}v_\pi
\end{equation}
Which has the solution:
\begin{equation}
	v_\pi = (I - \gamma P^{\pi})^{-1}R^{\pi}
	\label{bellman_matrix_form}
\end{equation}
Equation \ref{bellman_matrix_form} is a concise formula which is easily implemented in code.

\section{Optimal Policies and Bellman Optimality Equations}
Sutton and Barto now defines the optimal policy, which only has value 1 if the action the agent takes maximizes $q_*(s,a)$ \cite{sutton_barto}.
This is mathematically defined as:
\begin{align}
	\pi_{*}(a|s)=\begin{cases}
		1, & \text{if a = $\argmax\limits_{a \in A}q_* (s,a)$}\\
		0, & \text{otherwise}
	\end{cases}
	\label{eq:pi_*}
\end{align}
There always exists an optimal deterministic policy for any MDP \cite{sutton_barto}. We only need to know $q_* (s,a)$ to know the optimal policy. This is important because the optimal policy describes which actions must be taken to maximize rewards, which is precisely the goal of RL.
Sutton and Barto \cite{sutton_barto} then defines what is know as the Bellman \textit{optimality} equations.
\begin{equation}
	v_*(s) = \max\limits_{a}(R^{a}_s+\gamma\sum_{s'\in S}P^{a}_{ss'}v_*(s'))
	\label{bellmanv*}
\end{equation}
and
\begin{equation}
	q_*(s,a) = R^{a}_s +\gamma \sum_{s'\in S}P^{a}_{ss'}\max\limits_{a'}q_*(s',a').
	\label{bellmanq*}
\end{equation}
What equation \ref{bellmanv*} represents is the maximum return that can be obtained from state s, when following the optimal policy $\pi_*$. Equation \ref{bellmanq*} represents the maximum return that can be obtained by taking action a state s and thereafter following the policy.
At this point, we have outlined a mathematical background for reinforcement learning problems.
In the next two chapters we will discuss methods of solving the MDP and obtaining the optimal state-value and action-value functions.