\subsection{Spinning cell}
\label{spinning_cell}

The next problem which needed to be solved was a photodecomposition of samples
by incoming laser light.
Resonance Raman spectroscopy uses excitation light with frequency inside the
electronic absorption band.
It means that the investigated molecules accept a significant part of the power
from the incoming laser beam, and this excess energy can destroy the samples.
It also locally
increases temperature and causes the thermal lens effect, which distorts laser
focus.
We had to decide between several so far developed and successfully employed
approaches, as described in
\cref{introduction_sample_placement},
to minimize these effects.

As we wanted to study RNAs on our apparatus, which are highly sensitive to
contamination by ubiquitous RNase H and therefore cleaning of the sample cells
was a factor of major importance for us, we decided to utilize the idea of a
spinning cell to minimize photodecomposition of samples.
We designed the cell to be also usable with small volumes of the samples, the
inner radius of 4\,mm and height of 5\,mm.
It means that the maximal volume could be $\sim250$\,\g{m}L, but due to
centrifugal force, the smaller amount of samples could be measured;
for example, 50\,\g{m}L of sample results in an $\sim 4$\g{m}m thick layer
attached to the cell wall if the cell is rotated sufficiently fast.

To estimate the required rotation speed, we modeled the behavior of the water
inside the cell.
We neglected all the complicated effects like surface tension and currents
inside the liquid.
We can use energy conservation, where the sum of kinetic energy $E_\text{k}$
and gravitation potential energy $U_\text{g}$ is constant total energy $E$.
If we consider a closed system, the change in potential energy can go only to
kinetic energy and vice versa, so we can write
\begin{equation*}
	E_\text{k} = \frac{mr^2\omega^2}{2} = -U_\text{g} = mg(z - z_0),
\end{equation*}
where $m$ stands for mass, $\omega$ for angular speed, $z$ for the height of
the equipotential surface, $z_0$ for reference height for potential energy and
$g = 9.80665$\, Nkg$^{-1}$ as gravitation acceleration constant.
Solving the equation results into
\begin{equation}
	z = z_0 + \frac{r^2\omega^2}{2g}.
	\label{\eqnlabel{spinning_cell:liquid_surface}}
\end{equation}

The value for $z_0$ can be calculated from the sample volume
\begin{equation*}
	V = 2\text{\g{p}}h(R^2 - R_2^2)
		+ 2\text{\g{p}}\int_{R_1}^{R_2}{rz(r)\text{d}r},
\end{equation*}
where $R$ is the internal radius of the cell, $h$ is the height of the cell,
$R_1$ is the radius where the surface of the liquid touches the floor of the
cell, and $R_2$ is the radius where the surface of the liquid meets the ceiling
of the cell, so it means that $R \geq R2 > R1 \geq 0$.
The values of $R_1$ and $R_2$ can be estimated from
\eqnref{spinning_cell:liquid_surface}.

We can estimate the sufficient rotation speed of about 4000\,rpm from sample
model results for 50\,\g{m}L of a sample displayed in
\figref{spinning_cell:model}.

\begin{figure}
	\centering
	\begin{subfigure}[b]{0.49\columnwidth}
		\centering
		\includegraphics[width=\columnwidth]%
			{results_and_discussion/assets/spinning_cell_model/model_500rpm}
		\caption{500 rpm}
		\label{\figlabel{spinning_cell:model_500rpm}}
	\end{subfigure}
	\begin{subfigure}[b]{0.49\columnwidth}
		\centering
		\includegraphics[width=\columnwidth]%
			{results_and_discussion/assets/spinning_cell_model/model_1000rpm}
		\caption{1000 rpm}
		\label{\figlabel{spinning_cell:model_1000rpm}}
	\end{subfigure}
	\begin{subfigure}[b]{0.49\columnwidth}
		\centering
		\includegraphics[width=\columnwidth]%
			{results_and_discussion/assets/spinning_cell_model/model_2000rpm}
		\caption{2000 rpm}
		\label{\figlabel{spinning_cell:model_2000rpm}}
	\end{subfigure}
	\begin{subfigure}[b]{0.49\columnwidth}
		\centering
		\includegraphics[width=\columnwidth]%
			{results_and_discussion/assets/spinning_cell_model/model_4000rpm}
		\caption{4000 rpm}
		\label{\figlabel{spinning_cell:model_4000rpm}}
	\end{subfigure}
	\caption[%
		Model of the spinning cell.%
	]{%
		\captiontitle{%
			Model of the spinning cell following
			\eqnref{spinning_cell:liquid_surface}
			with different rotation speeds using 4\,mm as internal diameter, 5\,mm
			as height, and 50\,\g{m}L of liquid.%
		}
	}
	\label{\figlabel{spinning_cell:model}}
\end{figure}

The sample holder was inspired by the work of
\textcite{Shriver1974},
but we used FPM o-rings with an inner diameter of 11\,mm and 1\,mm diameter of
the rubber as displayed in
\figref{spinning_cell:drawing}
instead of the nylon cone.
The knurled aluminum nut compressed the o-ring, effectively centering and
securing the spinning cell to the holder chuck.
The cell holder was secured to the driving motor shaft by M2 screws.

\begin{figure}
	\centering
	\input{results_and_discussion/assets/spinning_cell_drawing}
	\caption[%
		Spinning cell holder.%
	]{%
		\captiontitle{%
			Spinning cell holder.%
		}
		A Teflon plug seals the cell (in blue), the knurled nut (light grey)
		secures the cell to the holder chuck (dark grey) by compressing the o-ring
		(black).
		The cell is attached to the driving motor shaft by M2 screws.
	}
	\label{\figlabel{spinning_cell:drawing}}
\end{figure}

As a driving motor, we used a DC motor (Maxon A-max 110119) controlled by a
homemade power source that could produce from 0\,V to 9\,V at the output.
The power supply was equipped with a digital voltmeter for user convenience.
The capabilities of the motor were measured with an attached fully filled cell.
A tiny dot was stuck to the cell, and a photodiode with an attached
oscillometer was then used for rotation frequency measurement.
The results of the measurement are shown in
\figref{spinning_cell:rotation}.
The values of rotation speed $\omega$ as a function of input voltage $U$ were
fitted by linear dependence
\begin{gather*}
	\omega = a_1U + a_0,\\
	a_1 = (1071 \pm 4) \text{V}^{-1}, a_0 = 2 \pm 20.
\end{gather*}

\begin{figure}
	\centering
	\input{results_and_discussion/assets/spinning_cell_rotation/rotation}
	\caption[%
		Spinning cell rotation evaluation.%
	]{%
		\captiontitle{%
			Spinning cell rotation evaluation.%
		}
		The dependence of rotation speed $\omega$ on the input voltage for the
		driving motor.
	}
	\label{\figlabel{spinning_cell:rotation}}
\end{figure}

The dependence has not got any significant constant factor $a_0$, and with the
maximal power of 9\,V accepted by the motor, about 9600\,rpm was achieved,
which was satisfactory for the spinning cell to centrifuge the samples to its
walls completely.


\documentclass[a4paper,12pt]{scrartcl}

\usepackage{bm,amsmath,url,graphicx}
\usepackage{palatino}
\usepackage{color, xcolor}
\usepackage{listings}

\newcommand{\n}{\mathbf{ n}}
\newcommand{\h}{\mathbf{ h}}
\newcommand{\x}{\mathbf{ x}}
\newcommand{\y}{\mathbf{ y}}
\newcommand{\w}{\mathbf{ w}}
\newcommand{\HH}{\mathbf{ H}}
\newcommand{\R}{\mathbf{ R}}
\newcommand{\C}{\mathbf{ C}}
\newcommand{\thb}{{\boldsymbol{\theta}}}
\newcommand{\mub}{{\boldsymbol{\mu}}}
\newcommand{\python}{{\fbox{\texttt{\bfseries python}}\quad}}
\newcommand{\pen}{{\fbox{\texttt{\bfseries pen\&paper}}\quad}}

\renewcommand{\familydefault}{\rmdefault}

\begin{document}
\section*{SGN-41007 Pattern Recognition and Machine Learning}
\emph{Exercise Set 5: February 8--February 10, 2017}
\bigskip
\sloppy

\lstdefinestyle{mystyle}{
  belowcaptionskip=1\baselineskip,
  breaklines=true,
  frame=single,
  xleftmargin=\parindent,
  language=Python,
  showstringspaces=false,
  basicstyle=\ttfamily,
  keywordstyle=\bfseries\color{green!40!black},
  commentstyle=\itshape\color{purple!40!black},
  identifierstyle=\color{blue},
  stringstyle=\color{orange},
  moredelim=**[is][\color{red}]{@}{@},
}

\lstset{language=Python,style=mystyle}

\noindent
Exercises consist of both pen\&paper and computer assignments.
Pen\&paper questions are solved at home before exercises, while
computer assignments are solved during exercise hours. The
computer assignments are marked by  \python and
Pen\&paper questions by  \pen

\begin{enumerate}

\item \pen \emph{Derive the explicit mapping corresponding to a kernel trick.}

In the lectures we saw that the kernel trick $\kappa(\x, \y) = (\x \cdot \y)^2$
for $\x = (x_1,x_2)$ and $\y = (y_1,y_2)$ corresponds to the mapping
\[
\begin{pmatrix}
u \\ v
\end{pmatrix}
\mapsto
\begin{pmatrix}
u^2\\
v^2\\
\sqrt{2}u v
\end{pmatrix}
\]
Find the explicit mapping corresponding to the inhomogeneous kernel
$\kappa(\x, \y) = (\x \cdot \y + 1)^2$ with $\x, \y\in\R^2$.

\emph{Hint:} Expand the kernel formula as far as you can.
At that point, reformulate the result into a dot product of
two 5-dimensional vectors; one composed of coordinates of $\x$ only
and the other of coordinates of $\y$ only.

\item \pen \emph{Compute the gradient of the log-loss.}

In the lectures we defined the \emph{logistic loss function}:
\begin{equation}
\ell(\w) = \sum_{n = 0}^{N-1} \ln(1 + \exp(-y_n\w^T\x_n)).
\label{eq:grad}
\end{equation}
Note that there was a typo in the slides with minus sign missing from the $\exp()$ function.

\begin{enumerate}
	\item Compute the formula for its gradient $\frac{\partial \ell(\w) }{\partial \w}$.
	\item There are two alternative strategies for using the gradient.
	\begin{itemize}
		\item \textbf{Batch gradient:} Compute the gradient from all samples and then apply the gradient descent rule
		$\w \leftarrow \w - \eta \frac{\partial \ell(\w) }{\partial \w}$.
		\item \textbf{Stochastic gradient:} Compute the gradient from one sample and then apply the gradient descent rule.
		In other words, pretend $N = 1$ in formula \ref{eq:grad}.
	\end{itemize}
	In the latter case, compute the next estimate for $\w$ when $\x_n = [-0.3, -1.7]^T$ and
	$y[n] = 1$ and $\w =[  0.9,  0.9]^T$.
\end{enumerate}

\item \python \emph{Implement gradient descent for log-loss.}

\begin{enumerate}
\item Implement a log-loss minimization algorithm. You may use the template provided by the
teaching assistant.
\item Apply the code for the data downloaded from

\url{https://github.com/mahehu/SGN-41007/tree/master/exercises/Ex5/log_loss_data.zip}

The data is in CSV format. Load  \verb+X+ and \verb+y+ using \verb+numpy.loadtxt+.

\item Plot the path of $\w$ over 100 iterations and check the accuracy (see plots below).

\end{enumerate}

\centerline{\includegraphics[width=0.5\textwidth]{log_loss_minimization.pdf}}

\item \python \emph{Select appropriate hyperparameters for the GTSRB data.}

Last week we trained classifiers for the German Traffic Sign
Recognition Benchmark (GTSRB) dataset. It turned out that the SVM was
really poor with default arguments, but changing the kernel pushed it
to the top. In this exercise, we use brute force to find good hyperparameters
for the classifiers (kernel, C, number of trees, etc.).

Consider the following two classifiers
	\begin{verbatim}
clf_list = [LogisticRegression(), SVC()]
clf_name = ['LR', 'SVC']
	\end{verbatim}
	Most important hyperparameters are the \emph{regularization strength} \verb+C+ and
	the penalty type parameter \verb+penalty+, which can have values "\verb+l1+" and "\verb+l2+".

	In order to use the same range for the two methods, you need to scale the data to
	zero mean and unit variance using \verb+sklearn.preprocessing.Normalizer+.

	Implement
	a grid search over these two parameters along the following lines:
	\begin{verbatim}
for clf,name in zip(clf_list, clf_name):
    for C in C_range:
		    for penalty in ["l1", "l2"]:
            clf.C = C
            clf.penalty = penalty
            clf.fit(X_train, y_train)
            y_pred = clf.predict(X_test)
            score = accuracy_score(y_test, y_pred)
\end{verbatim}
A reasonable range for \verb+C+ is $C\in\{10^{-5},...,10^{0}\}$.

\item \python \emph{Train ensemble methods with the GTSRB data.}

\begin{enumerate}
\item Train a 100-tree Random Forest classifier with the GTSRB and compute the accuracy on the test set.
\item Train a 100-tree Extremely Randomized Trees classifier with the GTSRB and compute the accuracy on the test set.
\item Train a 100-tree AdaBoost classifier with the GTSRB and compute the accuracy on the test set.
\item Train a 100-tree Gradient Boosted Tree classifier with the GTSRB and compute the accuracy on the test set.
\end{enumerate}

\end{enumerate}

\end{document}


\subsection{Money is liquid and information efficient}

\subsubsection{Liquidity of money}

Liquidity (don’t always want a goat)

Alice fishes and Bob hunts, and they trade meat and fish with each other. This much fish for that much meat. This all works perfectly well until Carol, Dan and Eve come along, who wants to trade berries, wood and furs. Now agreeing how much of this for that becomes more complicated, for every good there’s a ratio relative to every other good.

So they agree a new system, they collect all of the shells on the island and use these to trade with each other. Now they only have to remember a price for each good.

We want divisibility of money.

\subsubsection{Information requirements of money}

Information requirements \((n-1)+(n-2)\) etc v \((n-1)\)

\(\sum_{i=1}^n n-i\)

\(n^2-\sum_{i=1}^n i\)

\(n^2-\dfrac{n(n+1)}{2}\)

\(\dfrac{n(n-1)}{2}\)

Compared to \(n-1\) when money is used.

\subsubsection{Stability}

Don't want to be used for other purposes, demand could fluctuate more, more wasteful.


\documentclass[12pt,a4paper]{article}
\usepackage{algorithm, algpseudocode, amsmath, amssymb, caption, csquotes, empheq, geometry, graphicx, hyperref, listings, multirow, physics, siunitx, subcaption, upgreek}
\usepackage[section]{placeins}

\title{Computational Physics\\Problem Set 1}
\author{Saleh Shamloo Ahmadi\\Student Number: 98100872}
\date{September 27, 2021}

\hypersetup{colorlinks=true}

\newcommand*{\defeq}{\mathrel{\vcenter{\baselineskip0.5ex \lineskiplimit0pt
			\hbox{\scriptsize.}\hbox{\scriptsize.}}}
			=}

\newcommand{\figpath}{../fig}

\begin{document}
	\maketitle
    \section{Fractal Generation}
    Fractals are self-similar shapes, so they can be generated by repeatedly applying mappings to a set of points.
    The fractals in this problem set can all be created with linear iterated function systems (IFS).
    The transformations for these fractals can be described with affine transformations:
    \begin{gather}
        \vb{v}_{i+1, j} = T_j\vb{v}_{i, k} + \vb{c}_j\\
        \vb{v}_{i, j} = \mqty(x_{i,j} \\ y_{i,j}) \\
        T_j = \mqty(a_j & b_j \\ c_j & d_j),\quad \vb{c}_j = \mqty(e_j \\ f_j)
    \end{gather}
    There are two ways to define the mappings: relative and absolute. Also, the fractals can be generated randomly
    with the absolute mappings.
    \subsection{Relative Mapping}
    This type of mapping is suitable for use with line-based fractals (i.e. the fractal consists of a single curve).
    In this method, the transformations are applied locally on each line; The origin of the coordinate system is set
    to on end of the line and the point on the other end of the line is transformed relative to that.
    Algorithm \ref{alg:rel} is the implementation of this method.
    \begin{algorithm}
        \caption{Fractal Generation by Relative Mapping}
        \label{alg:rel}
        \begin{algorithmic}[1]
            \Function{fractal}{$x_{init}, y_{init}, T, \vb{c}, steps$}
            \Comment{$T$ and $\vb{c}$ represent the matrix and vector for every transformation}
                \State $x \gets x_{init}$
                \State $y \gets y_{init}$
                \For{$step \gets 1,steps$}
                    \State $i \gets 1$
                    \While{$i < N(x)$}
                    \Comment{N(x) is the number of x coordinates (which is the same as the number of points)}
                        \State $\mqty(x_{rel} \\ y_{rel}) \gets \mqty(x_{i+1} - x_i \\ y_{i+1} - y_i)$
                        \ForAll{$T_j$ and $\vb{c}_j$}
                            \State $\mqty(x_{new} \\ y_{new}) \gets T_j\mqty(x_{rel} \\ y_{rel}) + \vb{c}$
                            \State insert $x_{new}$ into $x$ at index $i$
                            \State insert $y_{new}$ into $y$ at index $i$
                            \State $i \gets i+1$
                        \EndFor
                    \EndWhile
                \EndFor
                \State \textbf{return} $x$ and $y$
            \EndFunction
        \end{algorithmic}
    \end{algorithm}
    \subsection{Absolute Mapping}
    This type of mapping is suitable for non-line-based fractals (the fractal isn't only a single curve).
    In this method, the same transformations are applied to every point in each step to generate new points.
    The origin of the coordinate system is always set at the first point of the fractal. By repeatedly applying
    the transformations to the fractal in each step, the self-similar shape is created.
    Algorithm \ref{alg:abs} is the implementation of this method.
    \begin{algorithm}
        \caption{Fractal Generation by Absolute Mapping}
        \label{alg:abs}
        \begin{algorithmic}[1]
            \Function{fractal}{$x_{init}, y_{init}, T, \vb{c}, steps$}
            \Comment{$T$ and $\vb{c}$ represent the matrix and vector for every transformation}
                \State $x \gets x_{init}$
                \State $y \gets y_{init}$
                \For{$step \gets 1,steps$}
                    \State define the $x_{new}$ empty array
                    \State define the $y_{new}$ empty array
                    \ForAll{$x_i \in x$ and $y_i \in y$}
                        \ForAll{$T_j$ and $\vb{c}_j$}
                            \State $\mqty(x_{gen} \\ y_{gen}) \gets T_j\mqty(x_{rel} \\ y_{rel}) + \vb{c}$
                            \State append $x_{gen}$ to $x_{new}$
                            \State append $y_{gen}$ to $y_{new}$
                        \EndFor
                    \EndFor
                    \State $x \gets x_{new}$
                    \State $y \gets y_{new}$
                \EndFor
                \State \textbf{return} $x$ and $y$
            \EndFunction
        \end{algorithmic}
    \end{algorithm}
    \subsection{Random Fractals}
    Some fractals can be generated by repeatedly applying random transformations from the fractal. If this is done with enough
    sample points, and enough steps so that the smalles unit of the shape is smaller than a pixel of the display, the fractal
    will be perfect. Algorithm \ref{alg:rand} is the implementation of this method.
    \begin{algorithm}
        \caption{Fractal Generation by Random Mapping}
        \label{alg:rand}
        \begin{algorithmic}[1]
            \Function{fractal}{$range_x, range_y, T, \vb{c}, steps, samples$}
            \Comment{$T$ and $\vb{c}$ represent the matrix and vector for every transformation}
                \State generate random $x$ values in $range_x$ ($N(x)=samples$)
                \State generate random $y$ values in $range_y$ ($N(y)=samples$)
                \ForAll{$x_i \in x$ and $y_i \in y$}
                    \For{$step \gets 1,steps$}
                        \State choose random $T_j$ and $\vb{c}_j$
                        \State $\mqty(x_i \\ y_i)\gets T_j\mqty(x_i \\ y_i) + \vb{c}_j$
                    \EndFor
                \EndFor
                \State \textbf{return} $x$ and $y$
            \EndFunction
        \end{algorithmic}
    \end{algorithm}
    \subsection{Representation of the fractals}
    The fractals are defined by functions $\{f_1, f_2, \dots, f_n\}$. Each of the functions are
    \begin{equation}
        f_i(\vb{v}) = T_i\vb{v} + \vb{c}_i,
    \end{equation}
    where
    \begin{gather}
        T = \mqty(a & b \\ c & d),\quad \vb{c} = \mqty(e \\ f).
    \end{gather}
    So we can represent each fractal by defining the values $a$, $b$, $c$, $d$, $e$.
    \section{Koch Snowflake}
    \begin{table}
        \centering
        \begin{tabular}{|c|c|c|c|c|c|c|}
            \hline
            transformation & $a$ & $b$ & $c$ & $d$ & $e$ & $f$ \\
            \hline
            $f_1$ & $\flatfrac{1}{3}$ & 0 & 0 & $\flatfrac{1}{3}$ & 0 & 0 \\
            \hline
            $f_2$ & $\flatfrac{1}{6}$ & $-\flatfrac{\sqrt{3}}{6}$ & $\flatfrac{\sqrt{3}}{6}$ & $\flatfrac{1}{6}$ & $\flatfrac{1}{3}$ & 0 \\
            \hline
            $f_3$ & $\flatfrac{1}{6}$ & $-\flatfrac{\sqrt{3}}{6}$ & $\flatfrac{\sqrt{3}}{6}$ & $\flatfrac{1}{6}$ & $\flatfrac{1}{2}$ & $\flatfrac{\sqrt{3}}{6}$ \\
            \hline
            $f_4$ & $\flatfrac{1}{3}$ & 0 & 0 & $\flatfrac{1}{3}$ & $\flatfrac{2}{3}$ & 0 \\
            \hline
        \end{tabular}
    \end{table}
    \begin{figure}
        \centering
        \makebox[\linewidth][c]{\includegraphics{\figpath/koch}}
        \makebox[\linewidth][c]{\includegraphics{\figpath/koch-large}}
        \makebox[\linewidth][c]{\includegraphics{\figpath/koch-color.pdf}}
        \caption{Koch Curve}
    \end{figure}
    \begin{figure}
        \centering
        \makebox[\linewidth][c]{\includegraphics{\figpath/koch-tri}}
        \caption{Koch Snowflake}
    \end{figure}
    \begin{figure}
        \centering
        \makebox[\linewidth][c]{\includegraphics{\figpath/koch-tri-color}}
        \caption{Colored Koch Snowflake}
    \end{figure}
    \section{Heighway Dragon}
    \begin{table}
        \centering
        \begin{tabular}{|c|c|c|c|c|c|c|}
            \hline
            transformation & $a$ & $b$ & $c$ & $d$ & $e$ & $f$ \\
            \hline
            $f_1$ & $\flatfrac{1}{2}$ & $\flatfrac{1}{2}$ & $-\flatfrac{1}{2}$ & $\flatfrac{1}{2}$ & 0 & 0 \\
            \hline
            $f_2$ & $-\flatfrac{1}{2}$ & $\flatfrac{1}{2}$ & $-\flatfrac{1}{2}$ & $-\flatfrac{1}{2}$ & $\flatfrac{1}{2}$ & $-\flatfrac{1}{2}$ \\
            \hline
        \end{tabular}
    \end{table}
    \begin{figure}
        \centering
        \includegraphics[width=\linewidth]{\figpath/heighway}
    \end{figure}
    \begin{figure}
        \centering
        \makebox[\linewidth][c]{\includegraphics{\figpath/heighway-large}}
    \end{figure}
    \section{Sierpiński Triangle}
    \begin{figure}[htb!]
        \centering
        \makebox[\linewidth][c]{\includegraphics{\figpath/sierpinski}}
    \end{figure}
    \begin{table}
        \centering
        \begin{tabular}{|c|c|c|c|c|c|c|}
            \hline
            transformation & $a$ & $b$ & $c$ & $d$ & $e$ & $f$ \\
            \hline
            $f_1$ & $\flatfrac{1}{2}$ & 0 & 0 & $\flatfrac{1}{2}$ & 0 & 0 \\
            \hline
            $f_2$ & $\flatfrac{1}{2}$ & 0 & 0 & $\flatfrac{1}{2}$ & $\flatfrac{1}{4}$ & $\flatfrac{\sqrt{3}}{4}$ \\
            \hline
            $f_3$ & $\flatfrac{1}{2}$ & 0 & 0 & $\flatfrac{1}{2}$ & $\flatfrac{1}{2}$ & 0 \\
            \hline
        \end{tabular}
    \end{table}
    \begin{figure}
        \centering
        \centering
        \makebox[\linewidth][c]{\includegraphics{\figpath/sierpinski-large}}
        \caption{Step 10 Sierpiński triangle}
    \end{figure}
    \newgeometry{top=0.4in}
    \thispagestyle{empty}
    \begin{figure}
        \centering
        \begin{subfigure}{\linewidth}
            \centering
            \includegraphics[width=\linewidth]{\figpath/sierpinski-random}
        \end{subfigure}
        \begin{subfigure}{\linewidth}
            \centering
            \includegraphics[width=\linewidth]{\figpath/sierpinski-random-large}
        \end{subfigure}
        \caption{Randomly Generating the Sierpiński Triangle}
    \end{figure}
    \FloatBarrier
    \restoregeometry
    \subsection{The Sierpiński Triangle Inside Pascal's Triangle}
    \newgeometry{left=0in,right=0in}
    \begin{figure}
        \centering
        \begin{subfigure}{0.45\linewidth}
            \centering
            \makebox[0.45\linewidth][c]{\includegraphics{\figpath/pascal-triangle}}
            \caption{Pascal's Triangle}
        \end{subfigure}
        \begin{subfigure}{0.45\linewidth}
            \centering
            \makebox[0.45\linewidth][c]{\includegraphics{\figpath/pascal-triangle-fractal}}
            \caption{Connecting the odd numbers generates the Sierpiński triangle}
        \end{subfigure}
        \begin{subfigure}{0.45\linewidth}
            \centering
            \makebox[0.45\linewidth][c]{\includegraphics{\figpath/pascal-fractal}}
            \caption{The Sierpiński triangle inside a larger Pascal's triangle}
        \end{subfigure}
        \begin{subfigure}{0.45\linewidth}
            \centering
            \makebox[0.45\linewidth][c]{\includegraphics{\figpath/pascal-fractal-large}}
            \caption{Painted on Pascal's triangle of size 256}
        \end{subfigure}
    \end{figure}
    \FloatBarrier
    \restoregeometry
    \section{Barnsley Fern}
    To create this fractal, we need to apply random functions with different probabilities.
    \begin{table}
        \centering
        \begin{tabular}{|c|c|c|c|c|c|c|c|}
            \hline
            transformation & $a$ & $b$ & $c$ & $d$ & $e$ & $f$ & probability \\
            \hline
            $f_1$ & 0 & 0 & 0 & 0.16 & 0 & 0 & 1\% \\
            \hline
            $f_2$ & 0.85 & 0.04 & -0.04 & 0.85 & 0 & 1.6 & 85\% \\
            \hline
            $f_3$ & 0.20 & -0.26 & 0.23 & 0.22 & 0 & 1.6 & 7\% \\
            \hline
            $f_4$ & -0.15 & 0.28 & -0.26 & 0.24 & 0 & 0.44 & 7\% \\
            \hline
        \end{tabular}
    \end{table}
    \thispagestyle{empty}
    \begin{figure}
        \centering
        \makebox[\linewidth][c]{\includegraphics{\figpath/barnsley-fern}}
        \caption{Barnsley fern with 200000 random samples and 25 transformation steps}
    \end{figure}
\end{document}


\section{Universal coefficient theorem, and products in $ H^\ast$}
We talked about cohomology $ H^\ast(X,A;N)$, which was contravariant in $(X,A)$. We can repeat some of the arguments for homology in the cohomological context. We can also relate cohomology to homology. This is the purpose of the universal coefficient theorem for cohomology. I won't actually prove this here, because I've put up notes on the website that covers this.
\begin{theorem}[Mixed variance UCT]
Let $R$ be a PID, let $N$ be a $R$-module, and let $C_\bullet$ be a chain complex of free $R$-modules. We've decided that $\Hom_R(C_\bullet,N)$ is a cochain complex. I'm always a little confused on how to write the homology of a cochain complex? Should I write $ H^n$ or $ H_n$? Maybe this is a personal problem, and I should keep it personal? We'll just write $ H^n$. (Some ridiculous notation with the $n$ sitting on the line in $H$ was suggested, but this'd be \emph{impossible} to TeX!)

Anyway, we had a map $ H^n\Hom_R(C_\bullet,N)\to\Hom_R( H_n(C_\bullet),N)$. The theorem is that this is surjective, which has kernel $\Ext^1_R( H_{n-1}(C_\bullet),N)$. I.e, there is a natural sexseq:
\begin{equation*}
0\to\Ext^1_R( H_{n-1}(C_\bullet),N)\to H^n\Hom_R(C_\bullet,N)\to\Hom_R( H_n(C_\bullet),N)\to 0
\end{equation*}
that splits, but not naturally.
\end{theorem}
\begin{proof}[Strategy of the proof]
I have a sexseq $0\to Z_n(C_\bullet)\to C_n\to C_n/Z_n(C_\bullet)\to 0$ where $Z_n(C_\bullet)=\ker d_n$ (where $C_\bullet$ is a chain complex). But $C_n/Z_n(C_\bullet)=B_{n-1}\hookrightarrow C_{n-1}$ where $B_n(C_\bullet)=\img d_{n-1}$ (this last thing probably has wrong indexing). We assumed $C_{n-1}$ is a free $R$-module, and that $R$ is a PID, so that $B_{n-1}$ is also free. Thus the sexseq $0\to Z_n(C_\bullet)\to C_n\to B_{n-1}\to 0$ splits.

Another sexseq that's important is $0\to Z_n(C_\bullet)/B_n(C_\bullet)\to C_n/B_n(C_\bullet)\to C_n/Z_n(C_\bullet)\to 0$. If you like, you can think of this as follows:
\begin{equation*}
\xymatrix{
	& 0 & 0 & 0 & \\
	0\ar[r] & Z_n(C_\bullet)/B_n(C_\bullet)\ar[r]\ar[u] & C_n/B_n(C_\bullet)\ar[r]\ar[u] & C_n/Z_n(C_\bullet)\ar[r]\ar[u] & 0\\
	0\ar[r] & Z_n(C_\bullet)\ar[r]\ar[u] & C_n\ar[r]\ar[u] & C_n/Z_n(C_\bullet)\ar[r]\ar[u] & 0\\
	0\ar[r] & B_n(C_\bullet)\ar[r]\ar[u] & B_n(C_\bullet)\ar[r]\ar[u] & 0\ar[r]\ar[u] & 0\\
	& 0\ar[u] & 0\ar[u] & 0\ar[u] &
}
\end{equation*}
Thus $0\to Z_n(C_\bullet)/B_n(C_\bullet)\to C_n/B_n(C_\bullet)\to C_n/Z_n(C_\bullet)\to 0$ splits. But $Z_n(C_\bullet)/B_n(C_\bullet)= H_{n-1}$, so this gives: $0\to H_{n-1}\to C_n/B_n(C_\bullet)\to B_{n-1}\to 0$.

Now, we have a sexseq $0\to B^n\Hom_R(C_\bullet,N)\to Z^n\Hom_R(C_\bullet,N)\to H^n\Hom_R(C_\bullet,N)\to 0$. We want to compare this to $\Hom_R( H_n(C_\bullet),N)$. But, now, $0\to H_{n-1}\to C_n/B_n(C_\bullet)\to B_{n-1}\to 0$ splits, so we get a sexseq $0\to \Hom(B_{n-1},N)\to \Hom(C_n/B_n(C_\bullet),N)\to\Hom( H_{n-1}(C_\bullet),N)\to 0$. Let me write this out:
\begin{equation*}
\xymatrix{
	0\ar[r] & B^n\Hom_R(C_\bullet,N) \ar[r]\ar@{-->}[d] & Z^n\Hom_R(C_\bullet,N) \ar[r]\ar@{-->}[d] & H^n\Hom_R(C_\bullet,N) \ar[r]\ar[d] & 0\\
	0 \ar[r] & \Hom(B_{n-1},N) \ar[r] & \Hom(C_n/B_n(C_\bullet),N)\ar[r] & \Hom( H_{n-1}(C_\bullet),N) \ar[r] & 0
}
\end{equation*}
An element of $Z^n\Hom_R(C_\bullet,N)$ is a map $f:C_n\to N$ such that $f\circ d=0$. So this map $f$ factors as $C_n/B_n(C_\bullet)\to N$. Thus we have the middle dotted map, and it's actually an isomorphism. You can then check compatibility, to get the left dotted map.

Is the map $ H^n\Hom_R(C_\bullet,N)\to\Hom( H_{n-1}(C_\bullet),N)$ surjective? Well, use the snake lemma. I can then use diagram chasing to see that the map is indeed surjective, with kernel given by the kernel of $B^n\Hom_R(C_\bullet,N)\to \Hom(B_{n-1},N)$. And the rest of the proof amounts to showing that this kernel is $\Ext^1_R( H_{n-1}(C_\bullet),N)$. And for splitting we can construct a splitting map, see the notes.
\end{proof}
\begin{remark}
Miguel: Why is $\Ext$ called Ext?

Miller: It deals with extensions. Let $R$ be a commutative ring, and let $M,N$ be two $R$-modules. I can think about extensions $0\to N\to L\to M\to 0$. Well, for example, I have two extensions $0\to\Z/2\Z\to\Z/2\Z\oplus\Z/2\Z\to\Z/2\Z\to 0$, and $0\to \Z/2\Z\to\Z/4\Z\to\Z/2\Z\to 0$. Say that two extensions are equivalent if there's a map of sexseqs between them is the identity on $N$ and $M$. The two extensions above aren't equivalent, for example.

Another definition of $\Ext^1_R(M,N)$ is the set of extensions like this modulo this notion of equivalence. The zero in the group is the split extension.

Also, $\Ext$ is contravariant in the first variable, but not in the second variable. If you want to find the Ext groups, you can use an injective resolution of the second variable, or a projective resolution of the first variable. These are what are known as derived functors. $\Tor$ is a left derived functor because it uses a projective resolution that goes off to the left, but $\Ext$ is a right derived functor because it uses an injective resolution that goes off to the right.
\end{remark}
\subsection{Products}
We'll talk about the cohomology cross product.
\begin{construction}
Define $S^p(X)\otimes S^q(Y)\xrightarrow{\times}S^{p+q}(X\times Y)$ as follows. Let $\sigma$ be a $(p+q)$-simplex in $X\times Y$. Let $f\otimes g\in S^p(X)\otimes S^q(Y)$. We'll define $f\times g\in S^{p+q}(X\times Y)$. Then $f:S_p(X)\to R$ and $g:S_q(Y)\to R$. I can write $\sigma=\begin{pmatrix}\sigma_1 \\ \sigma_2\end{pmatrix}$ where $\sigma_1:\Delta^{p+q}\to X$ and $\sigma_2:\Delta^{p+q}\to Y$. Define:
\begin{equation*}
(f\times g)(\sigma)=f(\sigma_1\circ\alpha_p)g(\sigma_2\circ\omega_q)
\end{equation*}
where $\alpha_p:\Delta^p\to\Delta^{p+q}$ takes $k\mapsto k$ where $k\in[p]$. And $\omega_q:\Delta^q\to\Delta^{p+q}$ sends $\ell\mapsto \ell+p$ where $\ell\in[q]$. It is an extremely (idiotic I think was the word used) construction that I have never gotten used to.
\begin{remark}
It is \emph{incredibly} stupid.
\end{remark}
We get a map on cochains. We need to check that this is a chain map $S^\ast(X)\otimes S^\ast(Y)\to S^\ast(X\times Y)$. This is your homework! What this means is that you get a map $ H^\ast(S^\ast(X)\otimes S^\ast(Y))\to H^\ast(X\times Y)$. I guess I have coefficients in $R$. I'm not quite done, but I have a map $ H^\ast(X)\otimes H^\ast(Y)\to H^\ast(S^\ast(X)\otimes S^\ast(Y))$. The composition $ H^\ast(X)\otimes H^\ast(Y)\to H^\ast(S^\ast(X)\otimes S^\ast(Y))\to H^\ast(X\times Y)$ is the cross product.
\end{construction}
It's not very easy to do computations with this. It's just hard to deal with. Let me make some points about this construction, though.
\begin{definition}
Now take $X=Y$. Then I get $ H^p(X)\otimes H^q(X)\xrightarrow{\times} H^{p+q}(X\times X)\xrightarrow{\Delta^\ast} H^{p+q}(X)$ where $\Delta:X\to X\times X$ is the diagonal. This composition is called the cup product. Some people write $\smile$, and others write $\cup$. (I'll TeX it as $\cup$.)
\end{definition}
Some properties of the cup product are the following. I claim that $ H^0(X)\cong\Map(\pi_0(X),R)$ as rings. This $\alpha$ and $\omega$ stuff collapses if $p=q=0$. There's nothing to do ... they're both the identity maps. So this isomorphism is clear. Also, inside $ H^0(X)$, we pick the element that maps to $(c\mapsto 1)\in\Map(\pi_0(X),R)$. This is the identity for the cup product. This comes out because when $p=0$ in our above story, then $\alpha_0$ is just including the $0$-simplex, and $\omega$ is the identity, so this is completely clear from that description.
\begin{prop}
If $f\in S^p(X)$ and $g\in S^q(Y)$ and $h\in S^r(Z)$, then $((f\times g)\times h)(\sigma)=(f\times(g\times h))(\sigma)$ where $\sigma:\Delta^{p+q+r}\to X\times Y\times Z$.
\end{prop}
\begin{proof}
Well, $((f\times g)\times h)(\sigma)=(f\times g)(\sigma_{12}\circ\alpha_{p+q})h(\sigma_3\circ\omega_r)$ where $\sigma_{12}:\Delta^{p+q+r}\to X\times Y$ and $\sigma_3:\Delta^{p+q+r}\to Z$. But $(f\times g)(\sigma_{12}\circ\alpha_{p+q})=f(\sigma_1\circ\alpha_p)g(\sigma_2\circ\mu_q)$ where $\mu_q$ is the ``middle portion'' that sends $\ell\mapsto \ell+p$ where $\ell\in[q]$. In other words, $((f\times g)\times h)(\sigma)=f(\sigma_1\circ\alpha_p)g(\sigma_2\circ\mu_q)h(\sigma_3\circ\omega_r)$. I've used associativity of the ring. But this thing is exactly the same as $(f\times(g\times h))(\sigma)$, so this is associative.
\end{proof}
Therefore, $(\alpha\cup\beta)\cup\gamma=\alpha\cup(\beta\cup\gamma)$.

But this product is obviously not commutative. It treats the two maps completely differently. But we have ways of dealing with this. This comes from the story of acyclic models, where I'll show that $\alpha\cup\beta=(-1)^{|\alpha|\cdot|\beta|}\beta\cup\alpha$. Thus $ H^\ast(X)$ forms a \emph{graded commutative ring}.


%% Digital Systems
%% Continuous System Equivalence
\def\FileDate{98/12/02}
\def\FileVersion{1.0}
% ----------------------------------------------------------------
% Notes pages *********************************************************
% ----------------------------------------------------------------

\ifslidesonly
\begin{slide}
\heading{Introduction}
\input{chunk00}
\end{slide}
\fi
\input{chunk00}

Before we can present such a system, it is necessary to establish the
relationship between digital operations, such as the shift, and
continuous operations.

\ifslidesonly
\begin{slide}
\heading{Agenda}
In this pre-lecture presentation we will start by discussing the relationship of $s$ to $z$.
We will then present four ways to convert a transfer function $H(s)$ into its
digital equivalent $H(z)$. These are:
\begin{itemize}
\item The zero-order hold equivalent
\item The Tustin bilinear transform equivalent
\item Matched pole zero equivalent
\item Modified matched pole-zero equivalent
\end{itemize}
\end{slide}
\fi

\section*{Equivalence of $s$ and $z$}

\ifslidesonly
\begin{slide}\label{slide:l11s1}
  \heading{Delaying a Sampled Signal}
\input{chunka}
\end{slide}
\fi
\input{chunka}

\ifslidesonly
\begin{slide}\label{slide:l11s2}
  \heading{Sampling a Delayed Signal}
  \input{chunkb}
\end{slide}
\fi
\input{chunkb}

\ifslidesonly
\begin{slide}\label{zisexpmst}
  \input{chunk01}
\end{slide}
\fi
\input{chunk01}

This is the fundamental relationship of equivalence. Before using it,
we must see how a continuous signal is reconstructed from a digital
signal. This is accomplished by means of a ``\emph{Digital-to-Analogue
Converter}''

\section*{Digital-to-Analogue Converter}

\ifslidesonly
\begin{slide}\label{dac}
\heading{Modelling a DAC with a Zero-Order Hold}
\input{chunk02}
\end{slide}
\fi
\input{chunk02}

\ifslidesonly
\begin{slide}\label{dac}
\heading{Operaton of the Zero-order Hold}
\input{chunk03}
\end{slide}
\fi
\input{chunk03}

\begin{slide}\label{slide:l11s3}
  \heading{Step-wise continuous signal}
  This represents the output of the zero-order hold $v(t) = v_k$ for
  $kT \le t < (k+1)T$.\\
  \begin{center}
\resizebox{300pt}{!}{\includegraphics{pictures/steps.pdf}}
  \end{center}
\end{slide}

The signal may be considered as an infinite number of pulses of which
  the $k$-th is that shown in \sref{slide:l11s4}. To model such a
  signal we use the so-called ``\emph{gating}'' property of the
  time-delayed unit-step function $\epsilon(t)$ illustrated in \sref{slide:l11s5}.

\ifslidesonly
\begin{slide}
\heading{Modelling the ZOH Mathematically}
\end{slide}
\fi
\begin{slide}\label{slide:l11s4}
  \heading{The $k$-th ``Pulse''}
  \begin{center}
\resizebox{300pt}{!}{\includegraphics{pictures/pulse.pdf}}
  \end{center}
\end{slide}

\begin{slide}\label{slide:l11s5}
  \heading{The Gating Function (1)}
  \begin{center}
\resizebox{300pt}{!}{\includegraphics{pictures/gate.pdf}}
  \end{center}
\end{slide}

The opening ``gate'' is given by $v_k \epsilon (t-kT)$, a step of height
$v_k$, which is activated at $t=kT$ seconds. The gate is ``closed'' by
a negative going unit step, also of height $v_k$, which is activated
at $t=\{k+1\}T$ seconds.

\ifslidesonly
\begin{slide}
\heading{The Gating Function (2)}
\input{chunk04}
\end{slide}
\fi
\input{chunk04}

\ifslidesonly
\begin{slide}
\heading{The Gating Function (3)}
\input{chunk05}
\end{slide}
\fi
\input{chunk05}

\ifslidesonly
\begin{slide}
\heading{ZOH as mixed transfer function}
\input{chunk06}
\end{slide}
\fi
\input{chunk06}

\begin{slide}\label{slide:l11s10}
  \heading{Hold-Equivalent Digital System}
  \resizebox{300pt}{!}{\includegraphics{pictures/holdequiv.pdf}}
\end{slide}

We can now design a `hold-equivalent' digital system.

\ifslidesonly
\begin{slide}
\heading{Hold-equivalent transform}
\input{chunk07}
\end{slide}
\fi
\input{chunk07}

\begin{slide}\label{slide:l11s11}
  \heading{Example 1}
  If \[H(s) = \frac{a}{s+a}\] then find the Zero-Order Hold Equivalent $H(z)$
\end{slide}

\subsubsection*{Solution}
\begin{eqnarray*}
  H(z) &=& \frac{z-1}{z}\mathcal{Z} \frac{a}{s(s+a)}\\
       &=& \frac{z-1}{z} \frac{z(1-e^{-aT})}{(z-1)(z-e^{-aT})}\\
       &=& \frac{1-e^{-aT}}{z-e^{-aT}}.
\end{eqnarray*}

\textbf{Malab note}: the zero-order-hold equivalent is the default
system used for continuous system equivalence in \emph{Matlab}. To
convert a continous system in any \emph{lti} format\footnote{%
The LTI functions are \emph{tf}, \emph{ss}, and \emph{zpk}. For more
information, type \emph{help lti} inside \emph{Matlab}.} use:
\begin{verbatim}
lti_d = c2d(lti, Ts); % You must provide a sampling time or use -1 if undefined
\end{verbatim}

\section*{Other Continuous System Equivalences}

\subsection*{Approximation based on numerical integration}

\ifslidesonly
\begin{slide}
\heading{Approximation based on numerical integration}
\input{chunk08}
\end{slide}
\fi
\input{chunk08}

\begin{slide}\label{slide:l11s20a}
  \heading{Approximation by Numerical Integration}
  \begin{itemize}
  \item We wish to find a transfer function $T(z)$ that is equivalent to $T(s)=s$.
  \item Let us instead seek a transfer function $D(z)$ that is equivalent to $D(s) = 1/s$.
  \end{itemize}
  \resizebox{300pt}{!}{\includegraphics{pictures/s-to-z.pdf}}
\end{slide}

Thus, the transfer function we are seeking will in fact be an approximation of
the integral $y(t)=\int u(t) dt$. We can illustrate this as shown in SLIDE~\ref{slide:l11s20b}.
If we sample the curve $u(t)$ and consider the situation at the $n$-th sampling
instant, we will have $$y(nT) = \int_{0}^{nT} u(t) dt$$ We can rewrite this as
$$y(nT)= \int_{0}^{nT-T} u(t) dt + \int_{nT-T}^{nT} u(t) dt$$
where the second integral term is the shaded area shown in SLIDE~\ref{slide:l11s20c}.
Now if we assume that the first integral term was approximated by the digital
integrator in the previous sampling instant, then
$y(nT-T) = \int_{0}^{nT-T} u(t) dt$ is known, and consequently we have
$$y(nT) = y(nT-T) + \int_{nT-T}^{nT} u(t) dt$$
and we only need to determine the area of the shaded region to approximate $y(nT)$.

\begin{slide}\label{slide:l11s20b}
\heading{Model of Integration}
\begin{center}
	\resizebox{220pt}{!}{\includegraphics{pictures/integral1.pdf}}
\end{center}
\end{slide}

\begin{slide}\label{slide:l11s20c}
\heading{Sampled Model of Integration}
\begin{center}
	\resizebox{220pt}{!}{\includegraphics{pictures/integral2.pdf}}
\end{center}
\end{slide}

The most obvious approximations are illustrated in \sref{slide:l11s20d} to
\sref{slide:l11s20f}. It is clear that if we are to disallow any further
subdivision of the shaded area, the \emph{trapezoidal approximation}
will provide the most accurate result.

\begin{slide}\label{slide:l11s20d}
\heading{Forward Rectangular Approximation}
\begin{center}
	\resizebox{220pt}{!}{\includegraphics{pictures/forward-rect.pdf}}
\end{center}
\end{slide}

\begin{slide}\label{slide:l11s20e}
\heading{Backward Rectangular Approximation}
\begin{center}
	\resizebox{220pt}{!}{\includegraphics{pictures/backward-rect.pdf}}
\end{center}
\end{slide}

\begin{slide}\label{slide:l11s20f}
\heading{Trapezoidal Approximation}
\begin{center}
	\resizebox{220pt}{!}{\includegraphics{pictures/trapezoidal.pdf}}
\end{center}
\end{slide}

\ifslidesonly
\begin{slide}
\heading{$z$-transform of trapezoidal approximation}
\input{chunk09}
\end{slide}
\fi
\input{chunk09}

\ifslidesonly
\begin{slide}
\heading{Approximation of $s$ by numerical integration}
\input{chunk10}
\end{slide}
\fi
\input{chunk10}

\begin{slide}\label{slide:l11s22}
  \heading{Trapezoidal Approximation}

  \begin{eqnarray*}
     s &=& \frac{2}{T}\, \frac{z-1}{z+1} \\
     z &=& \frac{1 + 1/2 sT}{1 - 1/2 sT}
  \end{eqnarray*}
  compare expansion $$z=(1+ (1/2) sT)(1- (1/2) sT)^{-1} = 1 + sT + (1/2)
 s^2T^2 + (1/4) s^3T^3
 + \cdots$$  with $$z = e^{sT} = 1 + sT + (1/2) s^2T^2 + (1/6) s^3T^3 + \cdots$$
\end{slide}

\begin{slide}\label{slide:l11s20}
  \heading{Forward Rectangular Approximation}

  \begin{eqnarray*}
     s &=& \frac{z-1}{T} \\
     z &=& 1 + sT
  \end{eqnarray*}
  compare with $$z = e^{sT} = 1 + sT + (1/2) s^2T^2 + (1/6) s^3T^3 + \cdots$$
\end{slide}

\begin{slide}\label{slide:l11s21}
  \heading{Backward Rectangular Approximation}

  \begin{eqnarray*}
     s &=& \frac{z-1}{Tz} \\
     z &=& \frac{1}{1 + sT}
  \end{eqnarray*}
  compare expansion $$z=(1-sT)^{-1} = 1 + sT + s^2T^2 + s^3T^3
 + \cdots$$  with $$z = e^{sT} = 1 + sT + (1/2) s^2T^2 + (1/6) s^3T^3 + \cdots$$
\end{slide}

The approximation shown in SLIDE\sref{slide:l11s22}) is
is known as ``\emph{Tustin's Bilinear Transformation}''.

\begin{slide}\label{slide:l11s23}
  \heading{Example 2}
  If \[H(s) = \frac{a}{s+a},\] determine the equivalent $H(z)$ using Tustin's
  bilinear transformation.
\end{slide}

\subsubsection{Solution}
Tustin's bilinear transformation gives a
 digital system with transfer function
 \begin{eqnarray*}
   H(z)&=& \frac{a}{\frac{2}{T}\,\frac{z-1}{z+1}+a} \\
       &=& \frac{\left(\frac{(aT)/2}{1+(aT)/2}\right)(z+1)}{z-\left(\frac{1-(aT)/2}{1+(aT)/2}\right)}.
 \end{eqnarray*}

\textbf{Malab note}: the bilinear transform equivalent is obtained by passing the argument \verb|'tustin'| to the \emph{c2d} method:
\begin{verbatim}
lti_d = c2d(lti, Ts, 'tustin');
\end{verbatim}

\subsection*{Matched pole-zero approximations}

Another way to obtain a digital approximation of a continuous transfer function
is to use the relationship $z = e^{sT}$ to map poles and zeros of the continuous
transfer function into poles and zeros in the z-transfer function.

Since continuous transfer functions often have more poles than zeros,
that is $n-m$ zeros at infinity, zeros at infinity are replaced in the z-transfer
function by zeros at $z = -1$ which is equivalent to half the sampling frequency
$\omega_s/2$ (i.e. the highest frequency possible in the z-domain).
Thus if $n-m =1$ we add $(1+z^{-1})$, if $n-m=2$, $(1+z^{-1})^2$, etc.

\begin{slide}\label{slides:l11s24}
	\heading{Matched-Pole Zero method}
	Idea is that all poles and zeros of continuous transfer function $D(s)$ can
  become poles and zeros of digital transfer function $D(z)$ if the mapping
  $z=e^{sT}$ is used.

	Method:
	\begin{enumerate}
		\item Map finite poles and zeros of $D(s)$ to poles and zeros of $D(z)$ according to $z=e^{sT}$.
		\item Add zeros at $z=-1$ for each infinite zero in $D(s)$.
		\item Match DC or low-frequency gain.
	\end{enumerate}
	See examples in the notes.
\end{slide}

\begin{slide}
\heading{Example 3}
Use the Matched-Pole-Zero (MPZ) approximation to give the z-transfer function equivalent to
$$D(s)=\frac{s+a}{s+b}.$$
\end{slide}
\subsubsection*{Solution}
Order of numerator and denominator are equal so there are no infinite zeros and
by matching poles and zeros
$$D(z)=k\frac{(1-e^{-aT}z^{-1})}{(1 - e^{-bT}z^{-1})}.$$
For $D(s)$ the DC gain is $D(s)|_{s=0} = a/b$.
The DC gain for $D(z)$ is $D(z)|_{z=1}$ (from final value theorem) that is
$$k\frac{(1-e^{-aT})}{(1-e^{-bT})} = \frac{a}{b}.$$
We choose $k$ so that the DC gains match, i.e.
$$k=\frac{a(1-e^{-bT})}{b(1-e^{-aT})}.$$

\begin{slide}
\heading{Example 4}

Use the Matched-Pole-Zero (MPZ) approximation to give the z-transfer function equivalent to
$$D(s)=\frac{s+a}{s(s+b)}.$$
\end{slide}

\subsubsection*{Solution}
Here, the order of the denominator is one greater than the numerator so there is
$n-m = 2 - 1 = 1$ infinite zero. Placing this zero at $z = -1$ makes the MPZ transfer
function
$$D(z)=k\frac{(1+z^{-1})(1-e^{-aT}z^{-1})}{(1-z^{-1})(1 - e^{-bT}z^{-1})}.$$
As $D(s)$ is type 1, we can't use the value of $D(0)$ to compute the DC gain.
Instead, let us compute the gain at $s = -1$:
\begin{eqnarray*}
  D( - 1) &=& {\left. {\frac{{s + a}}{{s(s + b)}}} \right|_{s =  - 1}} \\
   &=& \frac{{-1 + a}}{{( - 1)(-1 + b)}} = \frac{{a - 1}}{{(1 - b)}} \\
\end{eqnarray*}
The equivalent value $s=-1$ in the $z$-plane is $z=e^{-T}$.
\begin{eqnarray*}
  k{\left. {\frac{{(1 + {z^{ - 1}})(1 - {e^{ - aT}}{z^{ - 1}})}}{{(1 - {e^{ - bT}}{z^{ - 1}})}}} \right|_{z = {e^{ - T}}}} &=& k\frac{{(1 + {e^T})(1 - {e^{ - aT}}{e^T})}}{{(1 - {e^{ - bT}}{e^T})}} \hfill \\
   &=& k\frac{{\left( {1 + {e^T}} \right)\left( {1 - {e^{ - (1 - a)T}}} \right)}}{{\left( {1 - {e^{ - (1 - b)T}}} \right)}} \hfill \\
\end{eqnarray*}

Again, we choose $k$ so that the equivalent gains match, i.e.
$$k =  \left( {\frac{{1 - b}}{{a - 1}}} \right)\frac{{\left( {1 - {e^{ - (1 - b)T}}} \right)}}{{\left( {1 + {e^T}} \right)\left( {1 - {e^{ - (1 - a)T}}} \right)}}.$$

We would implement $D(z)$ as a \emph{difference equation} defined in terms of the
sampled inputs and outputs as shown in SLIDE~\ref{slides:l11s25} for Example 3.
\begin{slide}\label{slides:l11s25}
	\heading{Implementation of an MPZ Approximation}
	If $$\frac{Y(z)}{U(z)}=D(z)=k\frac{1-\alpha z^{-1}}{1-\beta z^{-1}}$$ then digital implementation will be
	$$y(n)=\beta y(n-1)+k(u(n)-\alpha u(n-1))$$
	$y(n)$ is the current calculated value, $y(n-1)$ is previous calculated value, $u(n)$ is current sample, and $u(n-1)$ is previous sample.

	Implementation only works if computation rate $<<$ significant dynamics of the sampled system and a small fraction of sampling time $T$.
\end{slide}

In the implementation, you should notice that it is actually physically impossible
to sample $u(t)$, compute $y(n)$ and output $y(n)$ all at the same instant of time.
Hence the equation is actually impossible to implement. However, if the
computation is sufficiently fast, the delay between sampling $u(t)$ and outputting
$y(t)$ will be small and can often be neglected.
A rule of thumb has computation delay $< t_r/20$ of the dominant pole.
It should certainly be a small fraction of the sampling period $T$.

\begin{slide}\label{slides:l11s26}
	\heading{Modified MPZ}
	\begin{itemize}
	\item
	Used if constraint on computation time cannot be met\footnote{Which these days is unlikely}.

	\item Allow a zero at infinity in $D(z)$ so that order of numerator is one
  less than denominator.

	\item This ensures that only past values of $u$ and $y$ appear in the implementation
  equation and a whole sample period is available for computation

\end{itemize}
\end{slide}

\begin{slide}\heading{Example 5}

Re-implement Example 3 using the Modified MPZ method.
\end{slide}
\subsubsection*{Solution}
If we allow an infinite zero
$$D(s)=\frac{s+a}{s(s+b)}$$ becomes
$$D(z)=k\frac{(1-e^{-aT}z^{-1})}{(1-z^{-1})(1-e^{-bT}z^{-1})}$$ and
$$k = \left( {\frac{{1 - b}}{{a - 1}}} \right)\frac{{\left( {1 - {e^{ - (1 - b)T}}} \right)}}{{\left( {1 - {e^{ - (1 - a)T}}} \right)}}.$$

The implementation is now

$$y(n) = (1 - e^{-bT})y(n-1)-e^{-bT}y(n-2)+k(u(n-1)-e^{-aT}u(n-2))$$

and now the computation for $y(n)$ is performed only on past values of $u(n)$ and
$y(n)$ and a whole sample period is available for computation.

\textbf{Matlab note}: the matched-pole zero equivalent of an LTI system is obtained by passing \verb|'matched'| as the third argument to \emph{c2d}:
\begin{verbatim}
lti_d = c2d(lti, Ts, 'matched')
\end{verbatim}
There isn't a built-in method that returns the \emph{modified}-matched-pole zero equivalent.

%----------------------------------------------------------------
% The end of notes
% ----------------------------------------------------------------
\endinput

% Local Variables:
% TeX-master: "lecture03"
% End:


\section{Overview}

%%%%%%%%%%
\begin{frame}{Dynamic Programming}
  Q: What is DP?
  \begin{itemize}
    \item A: Smart scheduling of subproblems.
  \end{itemize}

  \vspace{0.20cm}
  Q: What does DP look like?
  \begin{enumerate}
    \item Define subproblems (\textcolor{red}{types})
    \item Set the goal: what is the solution to the original problem
    \item Define recurrence: (\textcolor{red}{ask the right questions} $\Rightarrow$ reduce to subproblems)
      \begin{itemize}
	\item larger problem $\Leftarrow$ a \textcolor{blue}{number} of ``smaller'' subproblems
      \end{itemize}
    \item Write pseudo-code (\textcolor{red}{fill the array/table/matrix} in order)
    \item Analyze time complexity
    \item Extract optimal solutions
  \end{enumerate}
\end{frame}
%%%%%%%%%%
\begin{frame}{Common subproblems}
  \begin{enumerate}
    \item 1-D subproblems
      \begin{itemize}
	\item input: $x_{1}, x_{2}, \cdots, x_{n}$ (array, sequence, string)
	\item subproblems: $x_{1}, x_{2}, \cdots, x_{i}$ (prefix\textcolor{gray}{/postfix})
	\item $\# = O(n)$
	\item examples: weighted interval scheduling, max-sum subarray, breaking into lines
      \end{itemize}
    \item 2-D subproblems
      \begin{enumerate}
	\item input: $x_{1}, x_{2}, \cdots, x_{m}; y_{1}, y_{2}, \cdots, y_{n}$
	\begin{itemize}
	  \item subproblems: $x_{1}, x_{2}, \cdots, x_{i}; y_{1}, y_{2}, \cdots, y_{j}$
	  \item $\# = O(mn)$
	  \item examples: edit distance, LCS
	\end{itemize}
	\item input: $x_{1}, x_{2}, \cdots, x_{n}$
	\begin{itemize}
	  \item subproblems: $x_{i}, \cdots, x_{j}$
	  \item $\# = (n^{2})$
	  \item examples: multiplying a sequence of matrices, optimal binary search tree
	\end{itemize}
      \end{enumerate}
  \end{enumerate}
\end{frame}
%%%%%%%%%%
\begin{frame}{Common subproblems}
  \begin{enumerate}
    \setcounter{enumi}{2}
    \item 3-D subproblems:
      \begin{itemize}
	\item example: Floyd-Warshall algorithm, Bellman-Ford algorithm
      \end{itemize}
    \item DP on graphs (tree, DAG $\ldots$)
      \begin{itemize}
	\item input: rooted tree
	\item subproblems: rooted subtree
      \end{itemize}
    \item knapsack problem
      \begin{itemize}
	\item example: changing coins
      \end{itemize}
    \item \textcolor{red}{others $\ldots$}
  \end{enumerate}
\end{frame}
%%%%%%%%%%


\chapter{TD Prediction}

\section{Summary}

\subsection{TD prediction}

The basic formula for monte carlo prediction is $V(S_t)=V(S_t)+\alpha [G_t - V(S_t)]$. $G_t$ is the final result, this means that the update only can happen at the end of the simulation. By replacing $G_t$ with $R_{t+1} V(S_{t+1})$ we get the TD method $V(S_t) := V(S_t)+\alpha [R_{t+1} + \gamma V(S_{t+1}) - V(S_t)]$.

The update of the TD method is called the \textbf{TD error} $\delta = G_t - R_{t+1} V(S_{t+1})$. An equivalent entity exists with Monte-Carlo methods, and is called the Monte-Carlo error. The monte carlo error can be written as a sum of TD errors, illustrated by equation \ref{eq:monte carlo error is a sum of td errors}. (proof on page 121)

\begin{equation}
G_t - V(S_t) = \sum_{k=t}^{T-1} \gamma^{k-t} \delta_k
\label{eq:monte carlo error is a sum of td errors}
\end{equation}

\subsection{TD Advantages}

\begin{enumerate}
	\item No model of the behavior is required
	\item Naturally online/incremental algorithm (useful with long episodes)
	\item Learns from experimental choices (monte carlo need to discard them)
	\item In practice faster then monte carlo methods
\end{enumerate}

\subsection{Optimality of TD(0)}
When using batch learning, as in only changing the value function everytime a whole batch is processes. TD(0) and Monte Carlo do not converge to the same solution. Monte Carlo methods finds the solution that minimized the error on the dataset. TD(0) finds the parameters that most like would cause a markov process to result in the dataset. This is called the certainty-equivalence estimate.

\subsection{SARSA}
SARSA stands for $S_t,A_t,R_{t+1},S_{t+1}, A_{t+1}$. It uses an policy to generate $A_{t}$ and $A_{t+1}$. Updates the Q value, applies $A_{t+1}$ and then finds the next input $A_{t+2}$.

\begin{equation}
Q(S_t, A_t) := Q(S_t, A_t) + \alpha [R_{t+1} + \gamma Q(S_{t+1}, A_{t+1})-Q(S_t, A_t)]
\end{equation}

\subsection{Q-Learning}
Q-learning acts greedily in when predicting, but acts according to it's policy when finding an input to apply to the system. So in contrast to SARSA it won't reuse $A_{t+1}$ it generated when predicting.

\begin{equation}
Q(S_t, A_t) := Q(S_t, A_t) + \alpha [R_{t+1} + \gamma \max_a Q(S_{t+1},a) - Q(S_t, A_t)]
\label{eq:Q learning update}
\end{equation}

\subsection{Difference between SARSA and Q-Learning}
SARSA will act a bit more carefull, as it's prediction is not greedy. And it takes into account that the next action might not be the best one. Q-Learning will take the more risky route, as it uses the best(according to Q(S, A)) possible action in it's prediction.

\subsection{Expected Sarsa}
Expected SARSA uses the expected value of all possible actions $A_{t+1}$ given the policy. Then it uses a greedy policy to act, just like with Q-learning. Expected Sarsa will work with $\alpha=1$, which would not work very will with classical SARSA. This makes the short term behavior much better. But is more computational expensive.

\begin{equation}
Q(S_t, A_t) := Q(S_t, A_t) + \alpha [R_{t+1} + \gamma \EX[Q(A_{t+1}, S_{t+1})|S_{t+1}] - Q(s_t, A_t)]
\label{eq:expected sarsa update rule}
\end{equation}

\subsection{Double learning}
Equation~\ref{eq:Q learning update} uses an argmax to estimate the value of Q. If one of these estimates is over-estimated, it will result in bad behavior(bias). Double learning reduces the odds of this happening by using two $Q(A,S)$ estimates. One to find the maximum action, and one to estimate it's value.(equation~\ref{eq:estimation double learning}) It's less like that the overestimate will happen this way.

\begin{equation}
A = Q_2(\argmax_a Q1(S,a))
\label{eq:estimation double learning}
\end{equation}

 It's good practice to swap $Q_1$ and $Q_2$ in equation~\ref{eq:estimation double learning} constantly. For example at random with odds 50/50.

\section{Exercises}

\subsection{Exercise 6.1}

\begin{equation}
V_{t+1}(s_{t}) = \alpha [R_{t+1} + \gamma V_t(s_{t+1})-V_t(s_t)] + V_t(s_t)
\label{eq:difference value function update}
\end{equation}

The difference between the value function at time t and t+1 is defined by equation~\ref{eq:difference value function update}.

The equality $G_t = R_{t+1} + \gamma G_{t+1}$ still holds. However the monte carlo error is slightly different in every iteration. $G_t - V_t(s_t)$ becomes $G_{t+1} - V_{t+1}(s_{t+1})$ in the next iteration. As the value function now changes at iteration t, with a difference of  $d_t = \alpha [R_{t+1} + \gamma V_t(s_{t+1})-V_t(s_t)]$.

\begin{equation}
G_{t+1} - V_t(S_{t+1}) = G_{t+1} - V_{t+1}(S_{t+1})-d_{t+1}
\label{eq:single iteration difference}
\end{equation}

\begin{equation}
error = -\sum_{k=t+1}^{T-1} \gamma^{k-t} d_{k-1}
\label{eq:ex_6_1_difference}
\end{equation}

In conclusion the different factor is equation~\ref{eq:ex_6_1_difference}.

\subsection{Exercise 6.2}
If (as explained in the example of the hint) a part of the statespace is already well estimated. Then the TD prediction will be very good as you enter those states and if your path ends on one of those states. So you only have lesser predictions while in an unexplored part.

The Monte Carlo approach would still need to evaluate through the already well estimated part. Which is rather slow.

\subsection{Exercise 6.3}
The change on a value function is defined by:  $\alpha [R_{t+1} + \gamma V_t(s_{t+1})-V_t(s_t)] = 0.1[0 + 0 - 0.5]=-0.05$ if $V_t(s_{+1}) = 0$ so it ends on the left terminal state. And $\alpha=0.1$ and $V_t(A)=0.5$.

\subsection{Exercise 6.4}
The TD algo is over-fitting when $\alpha>0.05$ we could try to make it a bit smaller. But at $\alpha=0.05$ it seems to flatten out nicely, so I would not expect better results.

A similar story with the MC method, this time at $\alpha0.02$ we get a nice flat tail. It's not as clear as with the TD method, but that's due the larger variance on the MC method.

So no, I would not expect any changes in results if more samples were ran with different values for $\alpha$.

\subsection{Exercise 6.5}
Overfitting, the step is too large so TD cannot find the optimal values. But keeps over/under estimating every time it runs through an episode.

\subsection{Exercise 6.6}
You setup the bellman optionality equation, and the pick a method to solve it. As this is a rather simple example, you could just manually solve the equation.

\begin{equation}
\begin{split}
V(A) = 0.5 V(B)\\
V(B) = 0.5 V(A) + 0.5 V(C)\\
V(C) = 0.5 V(B) + 0.5 V(D)\\
V(D) = 0.5 V(C) + 0.5 V(E)\\
V(E) = 0.5 V(D) + 0.5\\
\end{split}
\end{equation}

This seems like the simplest way to do it, as it's small.

\subsection{Exercise 6.7}
The normal on-policy TD(0) update looks like $V(s_t) = V(S_t) + \alpha[R_{t+1}+\gamma V(S_{t+1}) - V(S_t)]$. I would expect that $\alpha=\frac{\rho}{\sum_t \rho_t }$ as it becomes a weighted average due too the importance sampling.

\subsection{Exercise 6.8}
todo, not hard, but a bit of bookkeeping to be done.

\subsection{Exercise 6.11}
In Q-learning the actions that are applied to the system are learning through a $\epsilon$-greedy policy(behavior policy) are not used for the prediction(Q). This is by definition an off-policy control.

\subsection{Exercise 6.12}
It would be nearly the same, SARSA selects the next action before updating Q and Q-learning selects it after. So the update of Q might make a difference in some cases.

\subsection{Exercise 6.13}
todo

\subsection{Exercise 6.14}
todo


%2multibyte Version: 5.50.0.2960 CodePage: 1252

\documentclass[11pt]{article}

\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{lineno}
\usepackage{fancybox}
\usepackage{fancyhdr}
\usepackage{rotating}
\usepackage{sectsty}

\setcounter{MaxMatrixCols}{10}

\begin{document}

\section{Appendix}

Letting $\eta _{t}^{s}$ , $\eta _{t}^{k}$ and $\lambda _{t}$ be the
Lagrangian multipliers on equations (\ref{3}), (\ref{4}) and (\ref{6})
respectively, the first order conditions are given by:

\begin{equation}
c_{t}:[c_{t}-\varkappa c_{t-1}]^{-\gamma }-\varkappa \beta (1+\mathfrak{g}%
)[c_{t+1}-\varkappa c_{t}]^{-\gamma }=\lambda _{t},  \label{A1}
\end{equation}

\begin{equation}
g_{t}:[g_{t}-\varkappa g_{t-1}]^{-\gamma }-\varkappa \beta (1+\mathfrak{g}%
)[g_{t+1}-\varkappa g_{t}]^{-\gamma }=\lambda _{t},
\end{equation}

\begin{equation}
i_{t}^{k}:e_{k}\eta _{t}^{k}=\lambda _{t},  \label{A3}
\end{equation}

\begin{equation}
i_{t}^{s}:e_{s}\eta _{t}^{s}=\lambda _{t},  \label{A4}
\end{equation}

\begin{equation}
d_{t}:\lambda _{t}=\beta \lambda _{t+1}\left\{
\begin{array}{c}
1+r^{\ast }+\frac{\pi }{\rho _{1}^{2}}\left[ \exp \left[ \rho _{1}(d_{t}-%
\bar{d}-\psi V_{t})\right] -\rho _{2}(d_{t}-\overline{d}-\psi V_{t})-\rho
_{3}\right] \\
+\frac{\pi }{\rho _{1}^{2}}\left[ \rho _{1}\exp \left[ \rho _{1}(d_{t}-\bar{d%
}-\psi V_{t})\right] -\rho _{2}\right] d_{t}%
\end{array}%
\right\} ,  \label{A5}
\end{equation}

\begin{equation}
k_{t}:\lambda _{t}=\beta (1+\mathfrak{g})\lambda _{t+1}\frac{\left[
e_{k}\theta _{k}\frac{y_{t+1}^{n}}{k_{t}}+(1-\delta _{k})-e_{k}\frac{\phi
_{k}}{2}\left( \frac{k_{t+1}}{k_{t}}-1\right) ^{2}+e_{k}\phi _{k}\left(
\frac{k_{t+1}}{k_{t}}-1\right) \frac{k_{t+1}}{k_{t}}\right] }{\left[ (1+%
\mathfrak{g})+e_{k}\phi _{k}\left( \frac{k_{t}}{k_{t-1}}-1\right) \right] },
\label{A6}
\end{equation}

\begin{equation}
s_{t}:\lambda _{t}=\beta (1+\mathfrak{g})\lambda _{t+1}\frac{\left[
e_{s}\theta _{s}\frac{y_{t+1}^{n}}{s_{t}}+(1-\delta _{s})-e_{s}\frac{\phi
_{s}}{2}\left( \frac{s_{t+1}}{s_{t}}-1\right) ^{2}+e_{s}\phi _{s}\left(
\frac{s_{t+1}}{s_{t}}-1\right) \frac{s_{t+1}}{s_{t}}\right] }{\left[ (1+%
\mathfrak{g})+e_{s}\phi _{s}\left( \frac{s_{t}}{s_{t-1}}-1\right) \right] },
\label{A7}
\end{equation}

\begin{equation}
\eta _{t}^{s}:(1+\mathfrak{g})s_{t+1}=e_{s}i_{t}^{s}+(1-\delta _{s})s_{t},
\label{A8}
\end{equation}

\begin{equation}
\eta _{t}^{k}:(1+\mathfrak{g})k_{t+1}=e_{k}i_{t}^{k}+(1-\delta _{k})k_{t},
\label{A9}
\end{equation}

\begin{equation}
\lambda _{t}:(1+\mathfrak{g}%
)d_{t}=(1+r_{t-1})d_{t-1}+c_{t}+i_{t}^{k}+AC_{t}^{k}+g_{t}+i_{t}^{s}+AC_{t}^{s}-y_{t}^{n}-y_{t}^{o}-T_{t}.
\label{A10}
\end{equation}

At steady-state, we find that:

\begin{equation}
\lbrack c(1-\varkappa )]^{-\gamma }[1-\varkappa \beta (1+\mathfrak{g}%
)]=\lambda ,  \label{SS1}
\end{equation}

\begin{equation}
\kappa \lbrack g(1-\varkappa )]^{-\gamma }[1-\varkappa \beta (1+\mathfrak{g}%
)]=\lambda ,  \label{SS2}
\end{equation}

\begin{equation}
e_{k}\eta ^{k}=\lambda ,  \label{SS3}
\end{equation}

\begin{equation}
e_{s}\eta ^{s}=\lambda ,  \label{SS4}
\end{equation}

\begin{equation}
1=\beta \left[ 1+r^{\ast }+\frac{\pi }{\rho _{1}^{2}}[1-\rho _{3}+(\rho
_{1}-\rho _{2})d]\right] ,  \label{SS5}
\end{equation}

\begin{equation}
1=\beta \left[ e_{k}\theta _{k}\frac{y^{n}}{k}+(1-\delta _{k})\right] ,
\label{SS6}
\end{equation}

\begin{equation}
1=\beta \left[ e_{s}\theta _{s}\frac{y^{n}}{s}+(1-\delta _{s})\right] ,
\label{SS7}
\end{equation}

\begin{equation}
s(\mathfrak{g}+\delta _{s})=e_{s}i^{s},  \label{SS8}
\end{equation}

\begin{equation}
k(\mathfrak{g}+\delta _{k})=e_{k}i^{k},  \label{SS9}
\end{equation}

\begin{equation}
(\mathfrak{g}-r)d+y^{n}+y^{o}+T=c+i^{k}+g+i^{s}.  \label{SS10}
\end{equation}

Combining (\ref{SS1}) and (\ref{SS2}):

\begin{equation}
\kappa =\left( \frac{g}{c}\right) ^{\gamma }  \label{SS11}
\end{equation}

\end{document}


\section{An Enumerative Synthesizer} \label{sec:optimizations}
%
To test our ideas in practice we implemented a simple
enumerative synthesizer.
%
In the remainder of this section we detail some of the
challenges and optimizations we implemented.

\subsection{Beta-Normal Form Enumeration}
%
The synthesizer enumerates terms in beta-normal form.
%
This is probably the most important optimization as it
largely reduces the search space.
%
More specifically, our implementation makes a bottom up
enumeration of the expressions in the following grammar:

\begin{align*}
  e & ::= \lambda x. e ~|~ \lambda x. t \\
  t & ::= x ~|~ t\;t ~|~ t\;e
\end{align*}

\subsection{Number of Function Arguments}
One minor optimization that we were able to implement was to require
  functions to be enumerated in a form such that the first $n$ non-terminals
  were lambdas where $n$ is the number of arguments to the function.
For example, the function $\f{add}$ which takes two arguments must have the
  form $\lambda x. \lambda y. E$ where $E$ is the rest of the expression.
Although this is a minor optimization, it allowed us to effectively decrease
  the depth of our search space by the number of arguments to a function, and
  we would not have been able to synthesize $\f{add}$ without it.

\subsection{Enumerating Closed Terms}
%
One additional challenge in generating lambda expressions
lies in generating only closed terms; terms with no free
variables.
%
A nice trick we found makes this easier is to generate
variables in a separate step.
%
That is, we first enumerate lambda expressions were we leave
a ``hole'' in the place a variable should go and then fill
those holes with variables in scope which is easier once the
rest of the term is fixed.

We also found it easier to work with De Bruijn indices.
%
With explicit names, it becomes necessary for the
synthesizer to have some sort of variable context that it
passes around when generating variable terms, both so that
it knows what names to gives new bindings, as well as to
know which bindings it has available.
%
Passing through such a context introduces a lot of
complexity to the enumeration itself.
%
Using De Bruijn indices has the added benefit of removing
alpha renaming and alpha equivalence from our expression
evaluation.

  % In order to deal with this challenge, we exclusively use De
  % Bruijn indices to reference our variables.
  % This simplifies our enumeration greatly, as we no longer need to pass through
  % a context to know which variable names are bound, instead we can just keep
  % track of the depth of the expression.


\subsection{The function \stackpush}
\Label{sec:stackpush}
\Label{sec:stackpushwd}

The following listing shows the specification of the function \stackpush.
In accordance with Axiom~\eqref{eq:stack-size-push}, \stackpush is supposed
to increase the number of elements of a non-full stack by one.
The specification also demands that the value that is pushed on a
non-full stack becomes the top element of the resulting stack (see
Axiom~\eqref{eq:stack-top-push}).

\input{Listings/stack_push.h.tex}

The implementation of \stackpush is shown in the next listing.
It checks whether its argument is a non-full stack in which case it
increases the field \inl{size} by one but only after it has assigned
the function argument to the element \inl{obj[size]}.

\input{Listings/stack_push.c.tex}

\vfill

The following listing shows our formalization of the well-definition for \stackpush.
%
The function \stackpush does not return a value but rather modifies its argument.
For the well-definition of \stackpush we therefore check whether it
turns equal stacks into equal stacks.

\input{Listings/stack_push_wd.c.tex}

However, equality of the stack arguments is not sufficient for a proof
that \stackpush is well-defined.
We must also ensure that there is no \emph{aliasing} between the two stacks.
Otherwise modifying one stack could modify the other stack in unexpected ways.
In order to express that there is no aliasing between two stacks,
we use the predicate \logicref{StackSeparated}.

In order to achieve an automatic verification of \implref{stackpushwd} we
have added the assertions \inl{top} and \equal and introduced the lemma
\logicref{StackPushEqual} in the following listing.

\input{Listings/StackLemmas.acsl.tex}


\subsection{Input layer normalisation}

We normalise the input layer.

This speeds up training, and makes regularisation saner.


\begin{comment}

\chapter{Reliability of Supervision Tree}

\section{Single Node Analysis}

Case 0: Single Failure, No restart
\begin{equation}
R(t) = \int_{t}^{\infty} f(x)dx
\end{equation}
\begin{equation}
A(t) = \frac{\int_{0}^{t} xf(x)dx + t \cdot R(t) }{t}
\end{equation}
Case I: Every $T$ time, spend $t_R$ time to restart

\begin{equation}
A(t) = \frac{ n(\int_{0}^{T} x f(x) dx + T \cdot R(T)) + \int_{0}^{t_{tail}} x
f(x) dx}{t}
\end{equation}
\hspace{7 cm} where  $n = \left \lfloor \frac{t}{T+t_R} \right \rfloor$ and
$t_{tail} = t-nT$

Case II: When failed, spend $t_R$ time to restart

\begin{equation}
A(t) = ?
\end{equation}

\hspace{7 cm} where  $W(t) = \int_{0}^{t} xf(x)dx$

Case III: When failed or running for $T$ time, spend $t_R$ time to restart

$A(t) = ? $

\end{comment}