\section{Co-occurrence Data Embedding}
\label{sec:code}

The general strategy we follow for unsupervised syntactic category
acquisition is to combine features of the context with the identity
and features of the target word.  Our preliminary experiments
indicated that using the context information alone (e.g. clustering
substitute vectors) without the target word identity and features had
limited success.\footnote{A 10-nearest-neighbor supervised baseline
  using cosine distance between substitute vectors gives .7213
  accuracy.  Clustering substitute vectors using various distance
  metrics and dimensionality reduction methods give results inferior
  to this upper bound.} It is the co-occurrence of a target word with a
particular type of context that best predicts the syntactic category.
In this section we review the unsupervised methods we used to model
co-occurrence statistics: the Co-occurrence Data Embedding (CODE)
method \cite{globerson2007euclidean} and its spherical extension
(S-CODE) introduced by \cite{maron2010sphere}.

Let $X$ and $Y$ be two categorical variables with finite cardinalities
$|X|$ and $|Y|$.  We observe a set of pairs $\{x_i, y_i\}_{i=1}^n$
drawn IID from the joint distribution of $X$ and $Y$.  The basic idea
behind CODE and related methods is to represent (embed) each value of
$X$ and each value of $Y$ as points in a common low dimensional
Euclidean space $\mathbf{R}^d$ such that values that frequently
co-occur lie close to each other.  There are several ways to formalize
the relationship between the distances and co-occurrence statistics, in
this paper we use the following:
\begin{equation} \label{eq:probability}
p(x,y) = \frac{1}{Z} \bar{p}(x) \bar{p}(y) e^{-d^2_{x,y}}
\end{equation}
\noindent where $d^2_{x,y}$ is the squared distance between the
embeddings of $x$ and $y$, $\bar{p}(x)$ and $\bar{p}(y)$ are empirical
probabilities, and $Z=\sum_{x,y} \bar{p}(x) \bar{p}(y) e^{-d^2_{x,y}}$ is
a normalization term.  If we use the notation $\phi_x$ for the
point corresponding to $x$ and $\psi_y$ for the point corresponding
to $y$ then $d^2_{x,y} = \|\phi_x-\psi_y\|^2$.  The log-likelihood
of a given embedding $\ell(\phi, \psi)$ can be expressed as:
\begin{eqnarray}
&&\ell(\phi, \psi) = \sum_{x,y} \bar{p}(x,y) \log p(x,y) \label{eq:likelihood} \\
&&= \sum_{x,y} \bar{p}(x,y) (-\log Z + \log \bar{p}(x)\bar{p}(y) - d^2_{x,y}) \nonumber \\
&&= -\log Z + \mathit{const} - \sum_{x,y} \bar{p}(x,y) d^2_{x,y} \nonumber
\end{eqnarray}
The likelihood is not convex in $\phi$ and $\psi$.  We use gradient
ascent to find an approximate solution for a set of $\phi_x$, $\psi_y$
that maximize the likelihood.  The gradient of the $d^2_{x,y}$ term
pulls neighbors closer in proportion to the empirical joint
probability:
\begin{equation}
\frac{\partial}{\partial\phi_x} \sum_{x,y} -\bar{p}(x,y) d^2_{x,y} =
\sum_y 2 \bar{p}(x,y) (\psi_y - \phi_x) \label{eq:attract}
\end{equation}
The gradient of the $Z$ term pushes neighbors apart in proportion to the
estimated joint probability:
\begin{equation}
\frac{\partial}{\partial\phi_x} (-\log Z) = \sum_y 2 p(x,y) (\phi_x -
\psi_y) \label{eq:repulse}
\end{equation}
Thus the net effect is to pull pairs together if their estimated
probability is less than the empirical probability and to push them
apart otherwise.  The gradients with respect to $\psi_y$ are
similar.

S-CODE \cite{maron2010sphere} additionally restricts all $\phi_x$ and
$\psi_y$ to lie on the unit sphere.  With this restriction, $Z$ stays
around a fixed value during gradient ascent.  This allows S-CODE to
substitute an approximate constant $\tilde{Z}$ in gradient
calculations for the real $Z$ for computational efficiency.  In our
experiments, we used S-CODE with its sampling based stochastic
gradient ascent algorithm and smoothly decreasing learning
rate.


\section{Oscillations on a paraboloid basin}
This test simulates water oscillations on a paraboloid basin. The analytical solution was derived by Thacker~\cite{Thacker1981}, and is periodic. At any instant in time, the free surface elevation is paraboloid, and the velocity is linear. The scenario includes regular wetting and drying, as the flow oscillates up and down in the basin. As well as testing the ability of the code to do wetting and drying, it will highlight any numerical energy loss or gain, and manifest as an increase or decrease in the magnitude of the flow oscillations over long time periods (compared with the analytical solution). This test was also implemented by Yoon and Cho~\cite{YC2001} to investigate the performance of their numerical method.

Consider the topography in two dimensions
\begin{equation}
z(x,y) = -D_0\left[1 -\left(\frac{r}{L}\right)^2\right]
\end{equation}
where $r=\sqrt{x^2 + y^2}$. Here $D_0$ is the largest depth when water is still and $L$ is the distance between the centre of water surface and the shore when water is still.
The analytical solution is
\begin{equation}
u(x,y,t) = \frac{\omega r A \sin{(\omega t)}}{ 2 \left[1 -A \cos(\omega t)\right] },
\end{equation}
\begin{equation}
w(x,y,t) = D_0 \left[\frac{\sqrt(1-A^2)}{1-A\cos(\omega t)}  -1
-\left( \frac{r}{L}\right)^2 \frac{1-A^2}{[(1-A\cos(\omega t))^2]-1} \right].
\end{equation}
Here $\omega=\frac{2\sqrt{2 g D_0}}{L}$ and $A = \frac{L^4 - R_0^4}{L^4 + R_0^4}$ and $R_0$ is the horizontal distance between the centre of water surface and the shore at the initial condition.
The initial condition is set by taking $t=0$ in the analytical solution.

\subsection{Results}
For our test, we consider $D_0=1000$, $L=2500$, and $R_0=2000$.
After running the simulation for some time, we have Figures~\ref{fig:cs_stage}--\ref{fig:cs_xvel} showing the stage, $x$-momentum, and $x$-velocity respectively. There should be a good agreement between numerical and analytical solutions, although wet-dry artefacts may appear in the 'nearly-dry' areas, where the numerical method can cause the water to drain too slowly (similar to that reported in \cite{KESSERWANIA14} using a finite volume scheme with similarities to discontinuous elevation algorithms in \anuga{}).

As time goes on, some small deviations may appear. These are shown in Figures~\ref{fig:w_centre}--\ref{fig:u_centre}, which illustrate the stage, $x$-momentum, and $x$-velocity at the centroid of the domain.

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{cross_section_stage.png}
\caption{Stage on a cross section of the basin at time $t=50$\,.}
\label{fig:cs_stage}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{cross_section_xmom.png}
\caption{Xmomentum on a cross section of the basin at time $t=50$\,.}
\label{fig:cs_xmom}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{cross_section_xvel.png}
\caption{Xvelocity on a cross section of the basin at time $t=50$\,.}
\label{fig:cs_xvel}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{Stage_origin.png}
\caption{Stage over time in the centre of the paraboloid basin.}
\label{fig:w_centre}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{Xmom_origin.png}
\caption{Xmomentum over time in the centre of the paraboloid basin.}
\label{fig:p_centre}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{Xvel_origin.png}
\caption{Xvelocity over time in the centre of the paraboloid basin.}
\label{fig:u_centre}
\end{center}
\end{figure}

\endinput


The kinematic tree is an abstract representation of a collection of transforms.
Unlike the rigid collection (section \ref{sec: rigid}), the kinematic tree is able to represent transforms in multiple datum frames.
The tree can then use the network of frames to "lookup" a transform between any two connected nodes, as hinted at in figure \ref{fig: poses}.
This concept is widely used in robotics and therefore has a place in the prototyping package.
There are several algorithms this class implements:
\begin{enumerate}
	\item Representation: Construct a tree representation given only the edges
	\item Lookup: apply breadth first graph search to find a path between any two nodes on the tree
	\item Root: use depth first search to express all frames in the base link frame or another specified frame on the tree
\end{enumerate}

\subsection{Representation}
Frames in a kinematic tree must have a child frame, a parent frame, or both.
Frame $T$ is described as:
\begin{equation}
	T_i^{p(i)} = \begin{bmatrix}
		R_i^{p(i)} & t_i^{p(i)} \\
		\bf{0} & 1
	\end{bmatrix}
\end{equation}
where $i$ is the child frame, and $p(i)$ is the parent frame.
The kinematic tree has a root, which we can also call the "base link".
The challenge with the kinematic tree is that we are only given the transforms, which are the edges in the graph.
So we need to create a tree representation from just the transforms.
Transforms contain the parent and child of the edge, and are therefore directed.
Thus we want a representation that is easily able to express the directed nature of the graph.

The importance of the edges in the kinematic tree suggests that an incidence list is the best representation for this situation.
For example, say we have a tree as shown in figure \ref{fig: tree}.

\begin{figure}
	\centering
	\includegraphics[width=0.5\textwidth]{images/example_tree.png}
	\caption{Example tree}
	\label{fig: tree}
\end{figure}

This takes the form:
\begin{align}
	base &: \big[ T^{base}_1, T^{base}_2\big] \\
	1 &: \big[ T^1_3, T^1_{base} \big]\\
	2 &: \big[ T^2_{base}\big]\\
	3 &: \big[ T_4^3, T_5^3 \big] \\
	4 &: \big[ T_3^4\big] \\
	5 &: \big[ T_3^5\big] \\
\end{align}
We can improve memory storage by using a dictionary (hash map) in order to only save the relevant edges.
The construction algorithm is shown in algorithm \ref{alg: inc_list}.

\begin{algorithm}
	\DontPrintSemicolon
	\KwIn{Unordered edge set $T$}
	\KwOut{Hash map where frames are the keys and the transforms they are incident with are values}
	tree $\gets \{\}$

	\tcp{hash.insert(key : value)}
	tree.insert(root : $\varnothing$)

	\For{$t \in T$}{$p \gets t.parent$

		$c \gets t.child$

		\tcp{process parents first}
		\If{$p \neq NULL$}{
			\eIf{$p \in$ tree}{
				\tcp{we store list of transforms touching $p$}
				tree[$p$].append($t$)
			}{
				tree.insert($p : \varnothing$)
			}
		}

		\tcp{process children}
		\If{$c \neq NULL$}{
			\eIf{$c \in $ tree}{
				\tcp{store whether the edge is forwards (used later)}
				($t^{-1}).forwards \gets false$

				tree[$c$].append($t^{-1}$)
			}{
				tree.insert($c : \varnothing$)
			}

		}
	}

	\Return{tree}
	\caption{Incidence list representation for the Kinematic Tree}
	\label{alg: inc_list}
\end{algorithm}

Once the incidence list is constructed, then we can use it to query the tree to find paths between frames.
The incidence list can keep track of the direction by assigning a flag to each edge specifying whether it is backwards or forwards.
This will be useful for the rooting algorithm, shown later.

An important thing to note is that in the context of the kinematic tree, an edge transform $T^i_j$ contains a child $j$ and a parent $i$ similar to a node.
However an edge can only have one parent and one child, whereas a node can have only one parent, but many children.

\subsection{Lookup path between frames}

Now say we wanted to know the transform between two frames $i, j$ on the tree that aren't connected by an existing edge.
We can find this edge, $T_i^j$ by finding the path between the two frames, and then multiplying the transforms we pass through, as shown in figure \ref{fig: lookup}.

\begin{figure}
	\centering
	\includegraphics[width=\textwidth]{images/lookup_path.png}
	\caption{Lookup path between frames}
	\label{fig: lookup}
\end{figure}

This can be done by executing a breadth first search on the tree in an attempt to find a path between two frames.
The algorithm is given in alorithm \ref{alg: lookup}.
The algorithm shows how the algorithm is modified to deal with the need to store edges.
Once we have a path as a sequence of edges, we can multiply these transforms together to obtain the transform between frames $i, j$.

\begin{algorithm}
	\KwIn{start frame, end frame}
	\KwOut{backpointers dictionary}
	\DontPrintSemicolon
	\SetKwFunction{expand}{expand}
	\SetKwFunction{bfs}{search\_for\_path}
	\SetKwProg{Fn}{def}{:}{}	$O \gets \varnothing$

	\tcp{the main search function}
	\Fn{\bfs{$x_{start}, x_{goal}$}}{

		$C \gets \varnothing$

		\tcp{the backpointers dictionary stores edges we've passed through}
		backpointers = \{\}

		$O$.insert(start)

		backpointers.insert(start:$NULL$)

		\While{$O \notin \varnothing$}{
			$x \gets O$.pop()

			$C$.insert($x$)

			\If{\expand{x}}{
				\Return{Success, backpointers}
			}

	}

	\Return{Failure, backpointers}
	}

	\tcp{The node expansion function}
	\Fn{\expand{x}}{
		\tcp{get the ordered set of transforms linked to $x$}
		$T \gets x.edges$

		\For{
			$t \in T$
		}{

			\uIf{$t$.child = end}{
				backpointers.insert($t.child : t$)

				\Return{Success}
				}
			\ElseIf{$t.child \notin O \textrm{ and } t.child \notin C$}{

				backpointers.insert($t.child : t$)

				O.insert($t.child$)
			}

			\Return{Failure}

		}
	}

	\caption{Lookup between frames using breadth first search. The backpointers can then be used to extract the path of $n$ transforms between the frames: $\prod_i^n T_i^{p(i)}T^i_{c(i)}$}
	\label{alg: lookup}
\end{algorithm}

\subsection{Root function}
The other main algorithm used by the kinematic tree is the root function.
The purpose of this is simply to put every frame in the tree into the base frame.
From here, it is trivial to place the entire tree in coordinates of any frame in the tree.
If we use figure \ref{fig: lookup} for reference, then we want the frames $T^{base}_1,T^{base}_2, T^{base}_3, T^{base}_4, T^{base}_5$.
We can obtain this through a depth first search that visits nodes in order and extracts the transform during the traversal.
Using the tree in figure \ref{fig: lookup}, we would visit the nodes from left to right: 4, 3, 5, 1, 2.

\begin{algorithm}
	\SetKwFunction{expand}{expand}
	\SetKwProg{Fn}{def}{:}{}	$O \gets \varnothing$

	\KwIn{Root of the tree}
	\KwOut{Dictionary of transforms $T_i^{base}$ for each frame $i$ in the tree}

	frames = \{\}

	$O \gets \varnothing$

	$C \gets \varnothing$

	$O$.insert(root)

	\While{$O \neq \varnothing$}{
		$x \gets O$.pop

		$C$.insert($x$)

		\expand{x}

	}

	\Return{frames}

	\tcp{depth first search expansion}
	\Fn{\expand{$x$}}{
		\tcp{traverse the ordered set $T$ of neighboring edges in reverse order}

		$R \gets reverse(T)$

		\ForAll{$t \in R$}{
			\If{$t.forward()$}{
				$x \gets t.child$

				$p \gets t.parent$

				\If{$x \notin O and x \notin C$}{
					$O$.insert($x$)

					$t^p_x \gets $ frames[$p$]

					frames.insert($x$ : $t^{p}_x$ * $t$)
				}
			}
		}
	}

	\caption{The depth first traversal used to derive the set of transforms $\mathcal{T} =\{ T^{base}_i \forall i : i \in K\}$ with frames $i$ in kinematic tree $K$.}
	\label{alg: root}
\end{algorithm}

This is shown in algorithm \ref{alg: root}.
The root function can then be used to graph the entire tree in the base frame or any other frame in the tree.


\chapter{Spectral approximation properties of ReLU DNN}
Here we will talk about how to use ReLU DNN to approximate polynomials
and then achieve the ``spectral" accuracy for analytical functions.
The main results can be found in \cite{yarotsky2017error,
  wang2018exponential}. Some related works are
\cite{liang2016why,lu2017expressive}

First, we will introduce the following notation
\begin{itemize}
\item Colon notation for subscript: let $\{x_{m:n}\} = \{x_i:i =
  m,m+1,...,n\}$ and $\{x_{m_1:n_1,m_2:n_2}\}= \{x_{i,j}:i =
  m_1,...,n_1,j = m_2,...,n_2\}.$
\item Linear combination: denote $y\in \mathcal L(x_1,...,x_n)$ if
  there exist $\beta_i\in \mathbb{R},i=1,...,n,$ such that $y =
  \beta_0+\beta_1x_1+\cdots+\beta_n x_n$.
\item Linear combination with ReLU activation: denote $\tilde{y}\in
  \tilde{\mathcal L}(x_1,...,x_n)$ if there exists $y\in
  \mathcal{L}(x_1,...,x_n)$ and $\tilde{y} = \mbox{ReLU}(y) =
  \max(y,0).$
\end{itemize}
\begin{definition}
Given a function $f(x_1,...,x_d)$, if there exist variables $\{y_{1:L,1:M}\}$ such that
\begin{equation}
y_{1,m}\in \tilde{\mathcal L}(x_{1:d}),\quad y_{l+1,m} \in \tilde{\mathcal L}(x_{1:d},y_{l,1:M}),\quad f\in\mathcal{L}(x_{1:d},y_{1:L,1:M}),
\end{equation}\label{def:netclass}
where $m=1,...,M,l=1,...,L-1$, then $f$ is said to be in the neural nets class $\mathcal{F}_{L,M}(\mathbb{R}^d)$, and $\{y_{1:L,1:M}\}$ is called a set of hidden variables of $f$.
\end{definition}
\begin{properties}
A function $f\in \mathcal F_{L,M}(\mathbb R^d)$ can be represented by a ReLU network with depth $L+1$ and width $M+d+1$.
\end{properties}
\begin{proof}
Let $\{y_{1:L}\}$ be the hidden variables of $f$ that satisfies (\ref{def:netclass}), where
$$
f = \alpha_0 + \sum_{i=1}^d \alpha_1x_1+\sum_{l=1}^L\sum_{m=1}^M \beta_{l,m}y_{l,m}.
$$
Consider the following variables $\{h_{1:L,1:M}\}$:
$$
h_{l,1:M} = y_{l,1:M}, \quad h_{l,M+1:M+d} = x_{1:d}
$$
for $l = 1,...,L,$ and
$$
h_{1,M+d+1} = \alpha_0 + \sum_{i=1}^d\alpha_ix_i,\quad h_{l+1,M+d+1} =h_{l,M+d+1} + \sum_{m=1}^M\beta_{l,m}h_{l,m}
$$
for $l=1,...,L-1$. One can see that $h_{1,m}\in \tilde{\mathcal L}(x_{1:d}),h_{l+1,m}\in \tilde{L}(h_{l,1:M+d+1}),m = 1,...,M+d+1,l = 1,...,L-1,$ and $f\in \mathcal{L}(h_{L,1:M+d+1}),$ which is a representation of a standard neural net.
\end{proof}

\begin{properties}\label{prop:net class}
(Addition and composition of neural net class $\mathcal F_{L,M}$)
\begin{itemize}
\item[1]
$$
\mathcal F_{L_1,M} + \mathcal{F}_{L_2,M} \subseteq \mathcal{F}_{L_1+L_2,M},
$$
i.e. if $f_1\in \mathcal F_{L_1,M}(\mathbb{R}^d)$ and $f_2\in \mathcal{F}_{L_2,M}(\mathbb{R}^d),$ then $f_1+f_2\in\mathcal{F}_{L_1+L_2,M}(\mathbb{R}^d)$.
\item[2]
$$
\mathcal F_{L_2,M}\circ \mathcal F_{L_1,M+1} \subseteq \mathcal F_{L_1+L_2,M+1}.
$$
i.e. if $f_1(x_1,...,x_d)\in \mathcal{F}_{L_1,M+1}(\mathbb{R}^d)$ and $f_2(y,x_1,...,x_d)\in \mathcal{F}_{L_2,M}(\mathbb{R}^{d+1}),$ then
$$
f_2(f_1(x_1,...,x_d),x_1,...,x_d)\in \mathcal{F}_{L_1+L_2,M+1}(\mathbb{R}^d).
$$
\end{itemize}
\end{properties}

\begin{proof}
For the addition property, denote the hidden variables of $f_1$ and $f_2$ as $\{y_{1:L_1,1:M}^{(1)}\}$ and $\{y_{1:L_2,1:M}^{(2)}\}$. Let
$$
y_{1:L_1,1:M} = y_{1:L_1,1:M}^{(1)},\quad y_{L_1+1,L_1+L_2,1:M} = y_{1:L_2,1:M}^{(2)}.
$$
By definition, $\{y_{1:L_1+L_2,1:M}\}$ is a set of hidden variables of $f_1+f_2$. Thus $f_1+f_2\in \mathcal F_{L_1+L_2,M}.$

For the composition property, let the hidden variables of $f_1$ and $f_1$ as $\{y_{1:L_1,1:M+1}^{(1)}\}$ and $\{y_{1:L_2,1:M}^{(2)}\}$. Let
$$
y_{1:L_1,1:M+1} = y_{1:L_1,1:M+1}^{(1)},\quad y_{L_1+1:L_1+L_2,1:M} = y_{1:L_2,1:M}^{(2)},
$$
$$
y_{L_1+1,M+1} =\cdots= y_{L_1+L_2} = f_1(x_1,...,x_d).
$$
One can see that $\{y_{1:L_1+L_2,1:M+1}\}$ is a set of hidden variables of $f_2(f_1(\bm x),\bm x)$, thus the composition property holds.
\end{proof}

\begin{definition}
Given a continuous function $\varphi(\bm{x}),\bm{x} \in [-1,1]^d$ and a continuous function class $\mathcal F([-1,1]^d),$ define the $L^\infty$ distance
$$
\mbox{dist} (\varphi,\mathcal F) = \inf_{f\in \mathcal F} \max_{\bm x \in [-1,1]^d} |\varphi(\bm{x}) - f(\bm{x})|.
$$
\end{definition}

\begin{properties}\label{prop:dis}
(Addition and composition properties for distance function)
\begin{itemize}
\item[1] Let $\varphi_1$ and $\varphi_2$ be continuous functions. Let $\mathcal F_{1}$ and $\mathcal F_2$ be two continuous function classes, then
$$
\mbox{dist}(\alpha_{1}\varphi_1+\alpha_{2}\varphi_2,\mathcal{F}_1+\mathcal F_2)\le |\alpha_1|\mbox{dist}(\varphi_1,\mathcal F_1) + |\alpha_2|\mbox{dist}(\varphi_2,\mathcal F_2),
$$
where $\alpha_1$ and $\alpha_2$ are two real numbers.
\item[2] Assume that $\varphi_1(\bm x) = \varphi_1(x_1,...,x_d),\varphi_2(y,\bm x) = \varphi_2(y,x_1,...,x_d)$ satisfy $\varphi_1([-1,1]^d)\subseteq[-1,1]$. Let $\mathcal F_1([-1,1]^d),\mathcal F_2([-1,1]^{d+1})$ be two continuous function classes, then
$$
\mbox{dist}(\varphi_2(\varphi_1(\bm x),\bm x),\mathcal F_2\circ \mathcal F_1)\le L_{\varphi_2}\mbox{dist}(\varphi_1,\mathcal F_1) +\mbox{dist}(\varphi_2,\mathcal F_2)
$$
where $L_{\varphi_2}$ is the Lipschitz norm of $\varphi_2$ with respect to $y$.
\end{itemize}
\end{properties}
\begin{proof}
The additional property obviously holds. Now we prove the composition property. For any $f_1\in\mathcal F_1,f_2\in \mathcal F_2$, one has
\[
\begin{split}
|\varphi_2(\varphi_1(\bm x),\bm x) - f_2(f_1(\bm x),\bm x)|&\le |\varphi_2(\varphi_1(\bm x),\bm x) - \varphi_2(f_1(\bm x),\bm x)|+|\varphi_2(f_1(\bm x),\bm x)-f_2(f_1(\bm x),\bm x)|\\
& \le L_{\varphi_2} ||\varphi_1(\bm x) -f_1(\bm x)||_\infty +||\varphi_2(y,\bm x) - f_2(y,\bm x)||_\infty
\end{split}
\]
Take $f_1^* = \argmin_f ||\varphi_1(\bm x) - f(\bm x)||_\infty$ and $f_2^* = \argmin_f ||\varphi_2(y,\bm x) - f(y,\bm x)||_\infty$, then it is proved.
\end{proof}

\begin{lemma}\label{lem:xsquare}
The function $\varphi(x) = x^2,x\in [-1,1]$ can be approximated by deep neural nets with an exponential convergence rate:
$$
\mbox{dist}(x^2,\mathcal{F}_{L,2}([-1,1]))\le 2^{-2L}.
$$
\end{lemma}
\begin{proof}
Consider the function
$$
g(y) = \left\{\begin{split}
	&2y,&\quad 0\le y<1/2,\\
	&2(1-y),&\quad 1/2\le y\le 1,
	\end{split}\right.
$$
then
\begin{equation}\label{func:g}
g(y) = 2y -4\mbox{ReLU}(y-1/2)
\end{equation}
in $[0,1]$. Define the hidden variables $\{y_{1:L,1:2}\}$ as follows:
\[
\begin{split}
 y_{1,1} = \mbox{ReLU}(x),&\quad y_{1,2}=\mbox{ReLU}(-x),\\
 y_{2,1} = \mbox{ReLU}(y_{1,1}+y_{1,2}),&\quad y_{2,2} = \mbox{ReLU}(y_{1,1}+y_{1,2}-1/2),\\
 y_{l+1,1} = \mbox{ReLU}(2y_{l,1}-4y_{l,2}),&\quad y_{l+1,2} = \mbox{ReLU}(2y_{l,1}-4y_{l,2}-1/2)\\
\end{split}
\]
for $l = 2,3,...,L-1$. Using induction, one can see that $|x| = y_{1,1}+y_{1,2}$ and
\begin{equation}\label{func:glinduction}
g_l(|x|)=\underbrace{g\circ g\circ \cdots \circ g}_{l}(|x|) = 2y_{l+1,1}-4y_{l+1,2},\quad l=1,...,L-1,
\end{equation}
for $x\in [-1,1]$. i.e.
\[
\begin{split}
g_l(|x|) &= g\left(g_{l-1}(|x|)\right)\\
         &= g(2y_{l,1}-4y_{l,2})\qquad (\mbox{Eq~(\ref{func:g})})\\
         &= 2(2y_{l,1}-4y_{l,2}) - 4\mbox{ReLU}(2y_{l,1}-4y_{l,2}-1/2)\qquad (g_{l-1}(|x|)\ge 0~\mbox{if}~x\in[0,1])\\
         &= 2\mbox{ReLU}(2y_{l,1}-4y_{l,2})- 4\mbox{ReLU}(2y_{l,1}-4y_{l,2}-1/2)\\
         & = 2y_{l+1,1} - 4y_{l+1,2}\\
\end{split}
\]
by induction, Eq (\ref{func:glinduction}) holds.
\begin{figure}[h]
	\centering
	\includegraphics[width=0.8\textwidth]{dl_approx_analytic_gl}
	\caption{The figure of $g_l(x)$}
	\label{fig:gl}
\end{figure}

Let $f_m$ be the piecewise linear interpolation of $f$ with $2^m+1$ uniformly distributed breakpoints $\frac{k}{2^m},k = 0,...,2^m:$
$$
f_{m}(\frac{k}{2^m}) = (\frac{k}{2^m})^2,\quad k = 0,...,2^m.
$$
Note that refining the interpolation from $f_{m-1}$ to $f_m$ amounts to adjusting it by a function proportional to a sawtooth function:
$$
f_{m-1}(x) - f_m(x) = \frac{g_m(x)}{2^{2m}}.
$$
Hence
$$
f_{L-1}(|x|) = |x| - \sum_{l=1}^{L-1}\frac{g_l(|x|)}{2^{2l}} .
$$
then $f_{L-1}\in \mathcal{F}_{L,2}$, and
\[
\begin{split}
||x^2-f_{L-1}(x)||_{\infty} & = \max_{k} \max_{x\in [\frac{k}{2^{L-1}},\frac{k+1}{2^{L-1}}]} |x^2 - f_{L-1}(x)|\\
  & = \max_{k} |(\frac{1}{2}(\frac{k}{2^{L-1}}+\frac{k+1}{2^{L-1}}))^2 - f_{L-1}(\frac{1}{2}(\frac{k}{2^{L-1}}+\frac{k+1}{2^{L-1}}))|\\
  & = \max_k |(\frac{1}{2}(\frac{k}{2^{L-1}}+\frac{k+1}{2^{L-1}}))^2 - \frac{1}{2}((\frac{k}{2^{L-1}})^2+(\frac{k+1}{2^{L-1}})^2))|\\
  &=\frac{1}{4^L}.
\end{split}
\]
$|x^2-f_{L-1}(x)|\le 2^{-2L}$ for $x\in[-1,1].$
\end{proof}

\begin{lemma}\label{lem:xy}
For multiplication function $\varphi(x,y) = xy$, we have
$$
\mbox{dist}(xy,\mathcal{F}_{3L,2}([-1,1]^2))\le 3\cdot 2^{-2L}.
$$
\begin{proof}
Notice that
$$
\varphi = xy = 2\left(\frac{x+y}{2} \right)^2-\frac{1}{2}x^2-\frac{1}{2}y^2.
$$
By properties \ref{prop:net class},\ref{prop:dis} and lemma \ref{lem:xsquare}, lemma \ref{lem:xy} is proved.
\end{proof}
\end{lemma}

\begin{lemma}
For a monomial $M_p(\bm x)$ of $d$ variables with degree $p$, we have
$$
\mbox{dist}(M_p,\mathcal F_{3(p-1)L,3})\le 3(p-1)\cdot 2^{-2L}.
$$
\end{lemma}

\begin{proof}
Let $M_p(\bm x)=x_{i_1}x_{i_2}\cdots x_{i_p},i_1,...,i_p\in\{1,...,d\}.$ Using induction, assume that the lemma holds for the degree-$p$ monomial $M_p$, consider a degree-$(p+1)$ monomial $M_{p+1}(\bm{x}) = M_p(\bm{x})\cdot x_{i_{p+1}}$. Let $\varphi(y,x) = yx,$ then $M_{p+1}(\bm x) = \varphi(M_p(\bm{x}),x_{i_{p+1}})$. We have
\[
\begin{split}
\mbox{dist}(M_{p+1},\mathcal{F}_{3pL,3})&\le \mbox{dist}(\varphi(M_p(\bm x),x_{i_{p+1}}),\mathcal F_{3L,2}\circ \mathcal F_{3(p-1)L,3})\\
& \le L_{\varphi}\mbox{dist}(M_p,\mathcal{F}_{3(p-1)L,3})+\mbox{dist}(\varphi,\mathcal F_{3L,2})\le 3p\cdot 2^{-2L}.
\end{split}
\]
Note that the Lipschitz norm $L_{\varphi}=1$ since $x_{i_{p+1}}\in [-1,1].$
\end{proof}

\begin{lemma}
For a degree-$p$ polynomial $P_p(\bm{x}) = \sum_{|\bm{k}|\le p}a_{\bm k}\bm x^{\bm k},\bm x\in [-1,1]^d,\bm k = (k_1,...,k_d)\in\mathbb{N}^d,$ we have
\[
\mbox{dist}\left(P_p,\mathcal F_{\binom{p+d}{d}(p-1)L,3}\right)<3(p-1)\cdot 2^{-2L}\sum_{|\bm k|\le p}|a_{\bm k}|
\]
\end{lemma}\label{lem:poly}

\begin{proof}
This lemma can be proved by the properties \ref{prop:net class}, \ref{prop:dis} and lemma \ref{lem:poly}.
Note that the number of monomials of $d$ variables with degree less or equal to $p$ is $\binom{p+d}{d}$.
$$
k_1 + k_2 + \cdots + k_p \le p,
$$
Add a variables $k_{p+1}$, we have
$$
k_1 + k_2 + \cdots + k_p + k_{p+1} = p.
$$
the number of the non-negative solution is $\binom{p+d}{d}$.
\end{proof}

\begin{theorem}
Let $f$ be an analytic function over $(-1,1)^d$. Assume that the power series $f(\bm x) = \sum_{\bm k\in \mathbb N^d}a_{\bm k}\bm x^{\bm k}$ is absolutely convergent in $[-1,1]^d.$ Then for any $\delta>0,$ there exists a function $\hat f$ that can be represented by a deep ReLU network with depth $L$ and width $d+4$, such that
\[
|f(\bm x) - \hat f(\bm x)|<2\sum_{\bm k\in \mathbb N^d}|a_{\bm k}|\cdot \exp\left(-d\delta\left(e^{-1}L^{1/2d}-1\right)\right)
\]
for all $\bm x\in [-1+\delta,1-\delta]^d.$
\end{theorem}
\begin{proof}
Let $\epsilon = \exp(-d\delta(e^{-1}L^{1/2d}-1)),$ then $L = [e(\frac{1}{d\delta}\log\frac{1}{\epsilon}+1)]^{2d}$. Without loss of generality, assume $\sum_{\bm k}|a_{\bm k}|=1.$ We will show that there exists $\hat f\in \mathcal F_{L,3}$ such that $||f-\hat f||_{\infty}<2\epsilon.$
Denote
$$
f(\bm x) = P_p(\bm x) + R(\bm x): = \sum_{|\bm k|\le p}a_{\bm k}\bm x^{\bm k} +  \sum_{|\bm k|> p}a_{\bm k}\bm x^{\bm k}.
$$
For $\bm x\in [-1+\delta,1-\delta]^d$, we have $|R(\bm x)|<(1-\delta)^p$, thus truncation to $p = \frac{1}{\delta}\log\frac{1}{\epsilon}$ with ensure $|R(\bm x)|<2\epsilon$. From lemma \ref{lem:poly}, we have dist$(P_p,\mathcal F_{L,3})<3(p-1)\cdot 2^{-2L'}$, where
\[
\begin{split}
L' &= L\binom{p+d}{d}^{-1}(p-1)^{-1}\\
   &\ge L(\frac{(p+d)!}{p!d!})^{-1}p^{-1}\quad (d! \thicksim \sqrt{2\pi d}(d/e)^d)\\
   &\ge L(\frac{(p+d)^d}{(d/e)^d})^{-1}p^{-1}\\
   &= L[e(\frac{1}{d\delta}\log \frac{1}{\epsilon}+1)]^{-d}(\frac{1}{\delta}\log\frac{1}{\epsilon})^{-1}\\
   & = [e(\frac{1}{d\delta}\log \frac{1}{\epsilon}+1)]^{d}(\frac{1}{\delta}\log\frac{1}{\epsilon})^{-1}\\
   & \ge e^d (\frac{1}{\delta}\log\frac{1}{\epsilon}) \\
   &\gg\log\frac{1}{\delta}+ \log\frac{1}{\epsilon}
\end{split}
\]

for $d\ge 2$ and $\epsilon<<1$, then dist$(P_p,\mathcal F_{L,3})<<\epsilon$. Thus there exists $\hat f\in \mathcal F_{L,3}$ such that $||P_p - \hat f||_{\infty}<\epsilon$, and $||f-\hat f||_{\infty}\le ||R(x)||_\infty + ||P_p - \hat f||_\infty<3\epsilon$.
\end{proof}
for $d\ge 2$ and $\epsilon\ll 1$, then dist$(P_p,\mathcal F_{L,3})<3(p-1)\cdot 2^{-2L'} = 3(p-1)(\epsilon^2+\delta^2)\ll\epsilon$. Thus there exists $\hat f\in \mathcal F_{L,3}$ such that $||P_p - \hat f||_{\infty}<\epsilon$, and $||f-\hat f||_{\infty}\le ||f-P_p||_\infty + ||P_p - \hat f||_\infty<3\epsilon$.


% This is part of the TFTB Reference Manual.
% Copyright (C) 1996 CNRS (France) and Rice University (US).
% See the file refguide.tex for copying conditions.

\markright{ifmt}
\section*{\hspace*{-1.6cm} ifmt}

\vspace*{-.4cm}
\hspace*{-1.6cm}\rule[0in]{16.5cm}{.02cm}
\vspace*{.2cm}

{\bf \large \sf Purpose}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
Inverse fast Mellin transform.
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Synopsis}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
x = ifmt(mellin,beta)
x = ifmt(mellin,beta,M)
\end{verbatim}
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Description}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
        {\ty ifmt} computes the inverse fast Mellin transform of {\ty
        mellin}.\\ {\it Warning} : the inverse of the Mellin transform is
        correct only if the Mellin transform has been computed from {\ty
        fmin} to 0.5 Hz, and if the original signal is analytic.\\

\hspace*{-.5cm}\begin{tabular*}{14cm}{p{1.5cm} p{8.5cm} c}
Name & Description & Default value\\
\hline
        {\ty mellin} & Mellin transform to be inverted. {\ty mellin} must have been
         obtained from {\ty fmt} with frequency running from {\ty fmin} to 0.5 Hz\\
        {\ty beta} & Mellin variable issued from {\ty fmt}\\
        {\ty M} & number of points of the inverse Mellin transform
                                        & {\ty length(mellin)}\\
  \hline {\ty x} & inverse Mellin transform with {\ty M} points in time\\

\hline
\end{tabular*}

\end{minipage}
\vspace*{1cm}

{\bf \large \sf Example}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
To check the perfect reconstruction property of the inverse Mellin
transform, we consider an analytic signal, compute its fast Mellin
transform with an upper frequency bound of 0.5, and apply on the output
vector the {\ty ifmt} algorithm\,:
\begin{verbatim}
         sig=atoms(128,[64,0.25,32,1]); clf;
         [mellin,beta]=fmt(sig,0.08,0.5,128);
         x=ifmt(mellin,beta,128); plot(abs(x-sig));
\end{verbatim}
We can observe the almost perfect equality between {\ty x} and {\ty sig}.
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf See Also}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
fmt, fft, ifft.
\end{verbatim}
\end{minipage}


\chapter{Calibration}

Our goal is to create a system which does not need any prior information about the
position of cameras. Since we do not know how far away the cameras are, nor
angles between them, we firstly use calibration to obtain this information.

By calibrating cameras with a well describable pattern, we can obtain
information about the single camera, such as what distortion its lens cause, or
how the world points are projected to an image plane of the camera.

After discovering these parameters for both cameras, we will continue on stereo
calibration. Stereo calibration will provide us information about their
position in space relative to each other.

For these calibration processes, we use algorithms implemented in OpenCV.
Therefore, we provide only a short overview of the process, and we describe
only results obtained from calibration essential to us.

\section{Intrinsic parameters}

Each camera type is different. Moreover, each camera of a specific type is
different due to a manufacturing process. Therefore we introduce intrinsic
camera parameters, which help us to model the camera precisely.

Intrinsic camera parameters define a transformation between the world coordinates and
the coordinates within an image plane (in pixels). Physical attributes of the
camera influence this transformation.

These parameters include focal length, a position of the principal point and
distortion coefficients. All these parameters are needed to get a correct
transformation between the point in the space and the point at the image plane.
Sometimes even more parameters are used for a better description of the model
of the camera.

\subsection{Camera matrix}

Camera matrix will provide us a transformation from
world coordinates (with the origin in the camera) to image plane coordinates
(seen in the image taken from the camera). If the coordinates in 3D are
available, we can obtain 2D coordinates by simple multiplication by camera
matrix. After multiplication we obtain 2D homogeneous coordinates.

As the next step, we describe the camera matrix obtained by OpenCV calibration
procedure. In the OpenCV implementation the camera matrix is $3\times3$ matrix of the
following format:

\[
\begin{pmatrix}
	f_x 	& 0 	& c_x \\
	0	& f_y	& c_y \\
	0	& 0	& 1
\end{pmatrix}
\]

Where $f_x$, $f_y$ denote focal lengths expressed in pixel units. It is usual
to introduce two of them, separately for both axes, since pixels usually are
not perfect squares but rather rectangles. Therefore the same focal length, has
different length in pixel units over a given axis.

We refer the ray of the view of the camera as the principal ray. The point
where this ray intersects the image plane is called principal
point\footnote{For more information visit \url{https://en.wikipedia.org/wiki/Pinhole\_camera\_model\#The\_geometry\_and\_mathematics\_of\_the\_pinhole\_camera}}.
Parameters $c_x$ and $c_y$ define coordinates of the principal point in the
image plane.  It should be the center of the image, but assembling process of
the camera might cause a small displacement.

\subsection{Distortion coefficients}

Cameras are equipped with lenses to obtain sharp image instead of blurry. The
lens may causes various distortions. Fish-Eye lenses are known for their
distortion.  Even web camera lenses have distortion, but not as visible as the
camera with a fish-eye lens. It is important to correct these distortions.

Two distortion types cause a significant effect on the image. The first one is
the radial distortion, creating barrel effect and the second one tangential distortion.

The radial distortion is caused by the camera lens (the effect of radial
distortion is displayed in the image \ref{fig:distortion}). It can usually be
described by three parameters. Highly distorted images (like from fish-eye)
often need more parameters. Since our system use web cameras, we will use only
three parameters.

In the ideal camera, the lens would be placed parallel to the chip. Since such
precision is not possible, due to an assembling process, tangential distortion
arises. For this distortion, we use two parameters.

We describe both distortion effects by five parameters. More about the meaning
of the parameters could be found in the book by \citet*{bradski2008learning}.

\begin{figure}
	\begin{subfigure}[b]{0.48\linewidth}
		\includegraphics[width=\linewidth]{img/chessboard/7x8chessboard-positivedistortion}
		\caption{Effect of positive radial distortion}
	\end{subfigure}
	\begin{subfigure}[b]{0.48\linewidth}
		\includegraphics[width=\linewidth]{img/chessboard/7x8chessboard-negativedistortion}
		\caption{Effect of negative radial distortion}
	\end{subfigure}
	\caption{Effects of lens distortion on the image of the chessboard}
	\label{fig:distortion}
\end{figure}

These nine parameters (four for camera matrix and five for distortion
coefficients) describe the camera model. They may be used for example for
projecting a point from 3D space to image coordinates in the camera (OpenCV
function \verb+projectPoints+). We will use them to get better results for
stereo calibration and for localization, since they provide us a more accurate
model of the camera.

%We can now describe the transformation process in one camera by multiplication by
%the camera matrix and then correcting the results by distortion coefficients. With
%these parameters, we can continue on stereo calibration.

%\todo[inline]{Ale co se pronasobi tou matici?}
%\todo[inline]{Napis sem ten vzorecek}

\section{Stereo calibration}

After performing a calibration of both cameras separately, we also need
information about their relative position to each other. By the relative
position, we understand translation from the first camera to the second and the
rotation of the camera (among three axes). Using this information, we can
transform a view of the one camera, by moving it by translation vector and
rotating accordingly. Therefore, the position of the second camera can be
described by six parameters, three as an angle around each axis to rotate and
three for translation vector in respect to the first camera.

Stereo calibration routine in OpenCV can also perform mono calibration for both
cameras, but we pre-calibrate cameras on itself for better precision and better
convergence of the algorithm.

Stereo calibration routine will provide more information. For the goals of this
thesis, only rotation matrix and translation vector are important.

\section{Calibration process}

In previous sections we shortly discussed theoretical background behind the
calibration. In this section we will focus on the implementation.

In OpenCV a chessboard is commonly used, since it is a planar object (easy to
reproduce by printing) and well described. OpenCV also provides methods for
automatic finding of the chessboard in the picture, so no human intervention is
needed to select the chessboard corners from the image (an example of the
chessboard is provided in a Figure \ref{fig:chessboard}). Therefore, we also
use the chessboard pattern for calibration.

\begin{figure}
	\centering
	\includegraphics[width=0.8\linewidth]{img/chessboard/7x8chessboard}
	\caption{Example of $7\times8$ chessboard used for the calibration}
	\label{fig:chessboard}
\end{figure}

\subsection{How many images?}

Now we know that, we can use a chessboard to calibrate the cameras. However,
the question of how many images are needed for the calibration remains. We
discuss only the images where the full chessboard is found. For enough
information from each image, we need a chessboard with at least $3\times3$
inner corners. It is better to have a chessboard with even, and odd dimension
(for example 7$\times$ 8 inner corners) since it has only one symmetry axis,
and the pose of the object can be detected correctly. It is important that the
chessboard indeed has squares, not rectangles (so it is better to check it
after printing). A bigger chessboard is easier to recognize, so we recommend at
least format A4.

We need only one image for computing the distortion coefficients. However, for
the camera matrix, we need at least two images. For stereo calibration, we need at
least one pair of images of the chessboard.

We know the minimum number of images needed for calibration. On the other hand,
we use more images for the robustness of the algorithm. We need a ``rich'' set of
views. Therefore it is important to move the chessboard between snapshots. With
more provided information to the algorithm, it compensates the errors in the
measurements (like a wrongly detected chessboard in one of the images).
Therefore also the computation time increases. Since it is enough for a given
setup of the cameras to perform a calibration only once and then use results
from it again, we recommend to do the calibration on more images and wait a
little bit longer for the results.


% declare document class and geometry
\documentclass[12pt]{article} % use larger type; default would be 10pt
\usepackage[margin=1in]{geometry} % handle page geometry

% import packages and commands
\input{../header2.tex}

\title{Phys 220A -- Classical Mechanics -- HW07}
\author{UCLA, Fall 2014}
\date{\formatdate{21}{11}{2014}} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed

\begin{document}
\maketitle

\section*{Problem 1 (25 pts)}
\textit{
Consider a canonical transformation with a generating function $F_2(q_i,P_i)$ where the $p_i, Q_i$ are determined by
\begin{eqn}
p_i = \pd{F_2}{q_i}, \qquad
Q_i = \pd{F_2}{P_i}, \qquad
H' = H + \pd{F_2}{t}
\end{eqn}
and $F_2$ has the form
\begin{eqn}
F_2 = \sum_i q_i P_i + X.
\end{eqn}
}

\begin{enumproblem}

% part A
\item \textit{
Find the relation of $q_i, p_i$ and $Q_i, P_i$ if $X = \delta \v a \cdot \v P$ where $\delta \v a$ is an infinitesimal vector.
}

% part B
\item \textit{
Find the relation of $q_i, p_i$ and $Q_i, P_i$ if $X = \delta \v \phi \cdot (\v q \times \v P)$ where $\delta \v \phi$ is an infinitesimal vector.
}

% part C
\item \textit{
Find the relation of $q_i, p_i$ and $Q_i, P_i$ if $X = \delta \delta \tau \, H$ where $\delta \tau$ is an infinitesimal paramter and $H$ is the Hamiltonian.
}

% part D
\item \textit{
Find the relation of $q_i, p_i$ and $Q_i, P_i$ if $X = -e \chi(\v q, t)$.
}

% part E
\item \textit{
For the Hamiltonian
\begin{eqn}
H = \frac{1}{2m} (\v p - e \v A(\v q,t))^2 + e \Phi(\v q,t)
\end{eqn}
where
\begin{eqn}
\v E = -\v \nabla \Phi - \pd{\v A}{t}, \qquad
\v B = \v \nabla \times \v A.
\end{eqn}
Calculate the new Hamiltonian $H'(Q,P)$ under the canonical transformations defined in parts (a)--(d).
}

\end{enumproblem}

\section*{Problem 2 (20 pts)}
\textit{
Consider the Hamiltonian
\begin{eqn}
H = \frac{1}{2m} p^2 q^4 + \frac{k}{2q^2}
\end{eqn}
and the canonical transformation given by the generating function
\begin{eqn}
F_1(q,Q) = -\sqrt{mk} \, \frac{Q}{q}.
\end{eqn}
}

\begin{enumproblem}

% part A
\item \textit{
Find the canonical transformations $Q=Q(q,p)$ and $P=P(q,p)$.
}

% part B
\item \textit{
Find the new Hamiltonian $H'(Q,P)$.
}

% part C
\item \textit{
Find the general solution for $Q(t), P(t)$.
}

% part D
\item \textit{
Solve for $q(t), p(t)$ with the initial condition $q(0) = q_0$ and $p(0) = 0$.
}

\end{enumproblem}

\section*{Problem 3 (15 pts)}
\textit{
Use Hamilton-Jacobi theory to solve for the dynamics of a particle with Cartesian coordinates $(x,y,z)$ moving in the gravitational potential of two fixed masses $m_+, m_-$ located at the points $(\pm c,0,0)$. Restrict motion to the plane $z=0$ for the sake of simplicity. This system has Lagrangian (we set $\mu_\pm = G m_\pm m > 0$ and $r_\pm^2 = (x \mp c)^2 + y^2$ for brevity)
\begin{eqn}
L = \frac{1}{2} m (\dot x^2 + \dot y^2) + \frac{\mu_+}{r_+} + \frac{\mu_-}{r_-}.
\end{eqn}
\textbf{Hint:} Change variables to elliptical coordinates, $x = c \cosh q_1 \cos q_2$ and $y = c \sinh q_1 \sin q_2$.
}

\section*{Problem 4 (10 pts)}
\textit{
Consider an infinitesimal canonical transformation with generating function
\begin{eqn}
F_2(q,P) = P q + \epsilon S(P,q).
\end{eqn}
}

\begin{enumproblem}

% part A
\item \textit{
Show that to first order in $\epsilon$ one has
\begin{eqn}
p = P + \epsilon \frac{S}{q}, \qquad
Q = q + \epsilon \pd{S}{P}.
\end{eqn}
}

% part B
\item \textit{
From this show that the infinitesimal canonical transformation satisfies Hamilton equations with $\epsilon$ viewed as ``time''
\begin{eqn}
\eval{\od{P}{\epsilon}}_{\epsilon=0} = -\pd{H}{q}, \qquad
\eval{\od{Q}{\epsilon}}_{\epsilon=0} = -\pd{Q}{p}
\end{eqn}
where $H(q,p) = S(q,p)$. [Should the second one not have a plus instead of minus sign?]
}

\end{enumproblem}

\end{document}


\chapter{Conclusions and future developments}%
\label{ch:conclusions}
In this thesis we developed a multi-cellular model accounting for a spatially-extended intra-cellular system for ROPs pattern formation. In the proposed framework, we solved ROPs pattern formation in a system composed by multiple cells, together with a transport model for hormone auxin. The RD model explains the auxin-mediated action of ROPs in an Arabidopsis root hair cell leading to formation of the localized patches of activated ROPs.

Our simulations support conclusions reached in other works regarding ROPs dynamics under a-priori defined auxin distribution \cite{phdthesis:victor, intra1_R, intra2}. We have analyzed various scenarios such that a stripe-like patch forms where auxin concentration is higher. Then, instability of stripe into spot-like states occurs and multiple spots align with auxin gradient or travel towards auxin minimum. Several results confirm that, for a transversally independent gradient, lateral stripes become unstable states.

Successively as a new contribution, we extended the intra-cellular dynamics for root-hair initiation model to a multi-cellular system, developing a new model taking into account communication between cells. In order to do so, we have defined a boundary value problem which assumes new boundary conditions between neighboring cells. Subsequently, we develop an iterative procedure to solve it. We take as reference scheme a Robin-Robin domain decomposition method. We impose fluxes of ROPs betweeen neighboring cells, depending on the difference of the ROPs concentrations, through localised open channels. Such connections are tuned in order to visualize considerably different results with respect to a configuration characterized by stagnant cells. In addition,we aim at preserving all previous analyses over important parameters characterizing the system. We numerically assessed the robustness of the proposed model in cooperating with auxin distribution in influencing ROPs pattern formation.

Having defined a reliable multi-cellular model, we have taken into account also auxin concentration dynamics. Hormone auxin is regulated by carriers PIN following a non-linear ODEs system. We implement a semi-implicit method to solve such as a system on a two cells setting. The simulations we carried out show oscillating values of auxin concentration for specific sets of parameters, as expected from previous studies. A further confirmation of the robustness of pattern formation according to the proposed new multi-cellular model, when considering channel communication between cells, is given in Chapter \ref{cap:4}. It is shown that even if the system is under steady homogeneous auxin concentrations, the model robustly forms hotspots when considering open channels under a sufficiently high overall auxin level.

The results show two ways spots of active ROPs can be generated. The first factor is the auxin gradient, which still guarantees and influence stripe to spot evolution. The second factor is the structural coupling between cells. Even if not characterized by variation in space but with a sufficiently high value of the auxin concentrations, the multi-cellular model leads the system to multiple spots.

In conclusion, this thesis provides a first attempt in modeling communication between root-hair cells in pattern formation. Future developments include to study the structural model or to deeply analyze channels characterization. Moreover, the iterative procedure implemented could be applied to a multi-cellular system composed by more than four cells and eventually try to increase the computational performance through parallel computing. Another possible road to follow is to approach the problem through homogenization techniques. The cited method may increase the efficiency in solving the model, particularly when the number of cells involved becomes large. Finally, one could assume other auxin transport models, less simplified or accounting for spatial dependence inside the cells.


After testing analog sorting on a prototype analog computer chip and simulating the process using an ODE solver, we can evaluate its time and space complexity.

Compared to the other sorting algorithms mentioned at the beginning of the paper, analog sorting is competitive in terms of time complexity. However, due to the nature of circuits, it is not as space efficient as most other popular sorting algorithms.

\subsection{Functionality validation of analog sorting}

\begin{figure}[h]
\centering
\includegraphics[width=\columnwidth]{graphics/10d_toda_7.png}
\caption{Simulated sorting a 10 element vector.}
\end{figure}

We see in Figure 1, generated from the ODE solver, that the sorting system has periods of swapping, interspersed in quiet periods of little change. The ODE solver was used on a 10 element vector.

The output of the analog chip, run on two variables and over multiple loops, is shown in Figure 2.

\begin{figure}[h]
\centering
\includegraphics[width=\columnwidth]{graphics/2d_analog.jpg}
\caption{Sorted 2 elements using analog computer chip.}
\end{figure}

This confirms that the values on the diagonals will converge towards a final value.

\subsection{Time cost of the discrete QR algorithm}
Since the analog sorting method is the continuous version of the QR algorithm, there might be a correlation between their time complexities as well.

In essence, our analog sorter takes a tridiagonal matrix $H$ and sorts the eigenvalues along the diagonal in descending order. Deift explains that the QR algorithm takes $O(N)$ time for a tridiagonal $N\times N$ matrix.

The assumption is that the time complexity of analog sorting will also be in that range.

%Time cost of overall QR loop:
	%how many iterations of qr til convergence?
	% time to convergence of qr w.r.t.. problem size
	%Since we know the ODE is analogous to QR algorithm, they should take the same amount of time.
%Time cost of QR step:
	%Numerical Recipes 3rd Edition p585 says the QR algorithm takes $O(N)$ time for symmetric tridiagonal matrices.

\subsection{Time cost of analog sorting}

In terms of time, the sorter takes at least $O(N)$ time because of the time it takes just for signals to propagate across the circuit.
Another issue is the time it takes for the ODE to settle to its final value.
This shows preliminary data showing the time to convergence of analog sorting, plotted against problem size.

\begin{figure}[h]
\centering
\includegraphics[width=\columnwidth]{graphics/ode_time_vs_problem_size.pdf}
\caption{Preliminary time to convergence vs. problem size.}
\end{figure}

The axes are the time to solution, plotted against $N$, the number of real numbers we are sorting.
The data points come from multiple random trials at each $N$.
The set of real numbers is randomly generated from a chosen dynamic range.
It appears that as the $N$ size increases, the average time grows linearly with respect to $N$.
Furthermore, the variance of the solution time, measured as the standard deviation, also grows.

\begin{figure}[h]
\centering
\includegraphics[width=\columnwidth]{graphics/range_vs_solution.png}
\caption{Preliminary time to convergence vs. dynamic range.}
\end{figure}

Another possible factor for solution time to consider is the dynamic range of the real numbers to sort. However, running the simulation on a constant number of elements with different dynamic ranges did not show any significant trends.

\subsection{Hardware cost of analog sorting}

Analog sorting on the chip relies on the hardware components needed to build the circuit.
We need a constant amount of components for each of the $N$ ODEs.
Thus, the analog sorter takes up $O(N)$ amount of circuit components to sort $N$ elements.

\subsection{Limitations}

As mentioned before, the choice of initial $y$ values affects the sorting mechanism. Changing the initial values of the off-diagonals impacts whether or not the algorithm ever reaches a correctly sorted state of the diagonal entries.

Setting the initial values for the off-diagonals to $0$ will prevent the algorithm from starting in the first place. Initial $y$ values that are too small may stop the algorithm mid-way and result in an approximate sorting with two elements being out of place. After the diagonal entries converge to their final state, the $y$ values are nonzero, but in the magnitude of less than $10^{30}$. How big the initial off-diagonal entries have to be might depend on the ratio of the numbers to sort.

Brockett mentions briefly that $H_\infty$ will be sorted in almost all cases, but does not elaborate on the exceptions~\cite{brockett}.

% \begin{figure}
% \centering
% \includegraphics[height=1in, width=1in]{fly}
% \caption{A sample black and white graphic
% that has been resized with the \texttt{includegraphics} command.}
% \end{figure}

% \begin{figure*}
% \centering
% \includegraphics{flies}
% \caption{A sample black and white graphic
% that needs to span two columns of text.}
% \end{figure*}

% \begin{figure}
% \centering
% \includegraphics[height=1in, width=1in]{rosette}
% \caption{A sample black and white graphic that has
% been resized with the \texttt{includegraphics} command.}
% \vskip -6pt
% \end{figure}


\documentclass{article}

\usepackage[colorlinks=true]{hyperref}
\usepackage[cmex10]{amsmath}
\usepackage{bbm}
\usepackage{graphicx}
\usepackage{subfig}
\usepackage{algorithm}
\usepackage{algorithmic}
\usepackage{comment}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{multirow}

\DeclareMathOperator*{\argmin}{\mathrm{argmin}}
\DeclareMathOperator*{\pro}{\mathcal P_{\Omega}}
\DeclareMathOperator*{\pron}{\mathcal P_{\bar{\Omega}}}
\DeclareMathOperator*{\proe}{\mathcal P_{H}}
\newcommand{\BigO}[1]{\ensuremath{\operatorname{O}\left(#1\right)}}

\begin{document}

\title{MC-Kit Manual}
\author{Stephen Tierney}
\maketitle

\section{Introduction}
\subsection{Classic Matrix Completion}

The problem of Matrix Completion (MC) is to recover a matrix $\mathbf A$ from only a small sample of its entries. Let $\mathbf A \in \mathbb R^{m \times n}$  be the matrix we would like to know as precisely as possible while only observing a subset of its entries $(i, j) \in \Omega$. It is assumed that observed entries in $\Omega$ are uniform randomly sampled. Low-Rank Matrix Completion is a variant that assumes that $\mathbf A$ is low-rank. The tolerance for ``low'' is dependant upon the size of $\mathbf A$ and the number of sampled entries.

For further explanation let us define the sampling operator $\pro : \mathbb R^{m \times n} \rightarrow \mathbb R^{m \times n}$ as
\begin{align}
[ \pro  ( X ) ]_{ij} = \left\{
\begin{array}{ll}
X_{ij}, &(i, j) \in \Omega,\\
0, &\text{otherwise}.
\end{array}
\right.
\end{align}
We also define the opposite operator $\pron$, which keeps those outside $\Omega$ unchanged and sets values inside $\Omega$ (i.e. $\bar{\Omega}$) to $0$. Since we assume that the matrix to recover is low-rank, one could recover the unknown matrix by solving
\begin{align}
\min_{\mathbf A} \; \text{rank}(\mathbf A)\\
\text{s.t.} \; \pro  (\mathbf A) = \pro (\mathbf M) \nonumber
\end{align}
where matrix $\mathbf M$ is the partially observed matrix. In practice this problem is intractable therefore we use the closest convex relaxation i.e. the nuclear norm
\begin{align}
\min_{\mathbf A} \; \tau \| \mathbf A \|_* \\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) \nonumber
\label{classic_objective}
\end{align}
We also consider the case where our observed entries may contain a limited amount of noise. Our corresponding objective is the following
\begin{align}
\min_{\mathbf A} \; \tau \| \mathbf A \|_* + \frac{\lambda}{2} \| \pro (\mathbf E) \|_F^2\\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) + \pro (\mathbf E) \nonumber
\end{align}

\section{Singular Value Shrinkage Operator}

Central to this work is the {\bf{singular value shrinkage operator}}. Consider the singular value decomposition (SVD) of a matrix $\mathbf Y \in \mathbb R^{m \times n}$ with rank $r$
\begin{align}
\mathbf Y = \mathbf{U \Sigma V^T}, \;\; \mathbf \Sigma = \text{diag}(\{\sigma_i\}_{i=1}^r).
\end{align}
Then we define the singular value shrinkage operator for any $\tau \geq 0$ as
\begin{align}
\mathcal D_{\tau}(\mathbf Y) = \mathbf U S_{\tau}(\mathbf \Sigma) \mathbf V^T, \;\; S_{\rho}(\mathbf \Sigma) = \text{diag}(\{\text{max}(\sigma_i - \tau, 0)\}).
\end{align}
It has been shown \cite{cai2010singular} that the operator $\mathcal D_{\tau}(\mathbf Y)$ is the solution to the proximal nuclear norm problem i.e.
\begin{align}
\mathcal D_{\tau}(\mathbf Y) = \argmin_{\mathbf X} \; \tau \| \mathbf X \|_* + \frac{1}{2} \| \mathbf{X - Y} \|_F^2
\end{align}
The implementation of the singular value singular shrinkage operator is implemented by the function $[ \mathbf X, \mathbf s ] = \text{nn\_prox}( \mathbf Y, \tau )$, where $\mathbf s$ is the vector of the singular values of $\mathbf X$.

\newpage
\section{Function Listing}

\begin{table}[!h]
{\small{
\centering

\begin{tabular}{c | c | c}
\hline
Problem & Function & Section \\
\hline

$\begin{array}{c} \min_{\mathbf A} \; \tau \| \mathbf A \|_* +  \frac{1}{2} \| \mathbf{ A } \|_F^2 \\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) \end{array}$ & solve\_svt	& 4.1.1  \\
\hline

\multirow{2}{*}{$\begin{array}{c} \min_{\mathbf A} \; \tau \| \mathbf A \|_* \\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) \end{array}$} & solve\_ialm	& 4.1.2 \\
		& solve\_lin	& 4.1.3 \\

\hline
\multirow{3}{*}{$\begin{array}{c} \tau \| \mathbf A \|_*  +  \frac{\lambda}{2} \| \mathbf{ \pro (A) - \pro (M)  } \|^2_F  \end{array}$}
	& solve\_e\_lin & 4.2 \\
	& solve\_e\_lin\_ext & \\
	& solve\_e\_lin\_acc & \\
\hline

$\begin{array}{c} \min_{\mathbf{A,E}} \; \tau \| \mathbf A \|_* + \frac{\lambda}{2} \| \pro (\mathbf E) \|_F^2\\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) + \pro (\mathbf E) \end{array}$ & solve\_e\_exact	& 4.3  \\
\hline

\end{tabular}
}}
\end{table}

\newpage
\section{Classic Implementations}
\subsection{Noise Free Data}
\subsubsection{SVT}

The function
\begin{align}
[ \mathbf A, \mathbf{f\_values}, \mathbf{stop\_vals} ] = \text{solve\_svt}( \mathbf M, \Omega, \tau, \mu, iterations, tol )\notag
\end{align}
solves the following
\begin{align}
\min_{\mathbf A} \; \tau \| \mathbf A \|_* +  \frac{1}{2} \| \mathbf{ A } \|_F^2 \\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) \nonumber
\end{align}\
as proposed by the authors of \cite{cai2010singular}.

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\Omega$ - vector of constrained matrix indices
\item $\tau$ - regularisation (optional)
\item $\mu$ - step size (optional)
\item $iterations$ - maximum number of iterations (optional)
\item $tol$ - stopping criteria tolerance (optional)
\end{itemize}

\subsubsection{Inexact ALM}

The function
\begin{align}
[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{solve\_ialm}( \mathbf M, \Omega, \tau, \mu, iterations, tol )\notag
\end{align}
solves the following
\begin{align}
\min_{\mathbf A} \; \tau \| \mathbf A \|_* \\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) \nonumber
\end{align}
as proposed by the authors of \cite{lin2010augmented}.

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\Omega$ - vector of constrained matrix indices
\item $\tau$ - regularisation (optional)
\item $\mu$ - step size (optional)
\item $iterations$ - maximum number of iterations (optional)
\item $tol$ - stopping criteria tolerance (optional)
\end{itemize}

\subsubsection{Linearised ALM}

The function
\begin{align}
[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{solve\_lin}( \mathbf M, \Omega, \tau, \mu, \rho, iterations, tol ) \notag
\end{align}
solves the following
\begin{align}
\min_{\mathbf A} \; \tau \| \mathbf A \|_* \\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) \nonumber
\end{align}

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\Omega$ - vector of constrained matrix indices
\item $\tau$ - regularisation (optional)
\item $\mu$ - step size (optional)
\item $\rho$ - linearisation step size (optional)
\item $iterations$ - maximum number of iterations (optional)
\item $tol$ - stopping criteria tolerance (optional)
\end{itemize}

\subsection{Noisey Data Relaxation}

The functions
\begin{align}
[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{solve\_e\_lin}( \mathbf M, \Omega, \tau, \lambda, \rho, iterations, tol ) \notag \\
[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{solve\_e\_lin\_ext}( \mathbf M, \Omega, \tau, \lambda, \rho, iterations, tol ) \notag \\
[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{solve\_e\_lin\_acc}( \mathbf M, \Omega, \tau, \lambda, \rho, iterations, tol )\notag
\end{align}
solve the following
\begin{align}
\min_{\mathbf A} \; \tau \| \mathbf A \|_*  +  \frac{\lambda}{2} \| \mathbf{ \pro (A) - \pro (M)  } \|^2_F
\end{align}
with increasing convergence speed based on \cite{ji2009accelerated}.

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\Omega$ - vector of constrained matrix indices
\item $\tau$ - regularisation (optional)
\item $\lambda$ - regularisation (optional)
\item $\rho$ - linearisation step size (optional)
\item $iterations$ - maximum number of iterations (optional)
\item $tol$ - stopping criteria tolerance (optional)
\end{itemize}

\subsection{Noisey Data Exact}

The function
\begin{align}
[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{solve\_e\_exact}( \mathbf M, \Omega, \tau, \lambda, \rho, iterations, tol ) \notag
\end{align}
solve the following
\begin{align}
\min_{\mathbf A} \; \tau \| \mathbf A \|_* + \frac{\lambda}{2} \| \pro (\mathbf E) \|_F^2\\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) + \pro (\mathbf E) \nonumber
\end{align}

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\Omega$ - vector of constrained matrix indices
\item $\tau$ - regularisation (optional)
\item $\lambda$ - regularisation (optional)
\item $\rho$ - linearisation step size (optional)
\item $iterations$ - maximum number of iterations (optional)
\item $tol$ - stopping criteria tolerance (optional)
\end{itemize}

\newpage
\bibliographystyle{plain}
\bibliography{references}

\end{document}


\section{Introduction}\label{introduct}
A well-studied phenomenon in material science concerns the coarsening of microstructure found in metals, porcelains, and other common materials.  Typically, these materials have polycrystalline structure, where each grain has a particular orientation.  Annealing induces an evolution on grain boundaries which reduces their total length.  During this process, grains are deleted (and not created), causing coarsening of microstructure through an increase of average grain area (see Fig. \ref{samplegrains}).  Microstructure can determine material properties. For instance, the Hall-Petch effect gives a relation between yield stress and average grain size (see \cite{anderson2005fracture}, section 5.2.3).     More scientific and  mathematical properties of grain networks can be found in \cite{fradkov1994two,thompson2001grain}.

Grain coarsening in two dimensions  can be modeled by a geometric network, or planar graph $G(t) \subset \mathbb{R}^2$, $t\in (0,T)$, with edges  that evolve in time. Faces of $G$ are called \textbf{grains}, and edges are referred to as \textbf{boundaries}. An idealization of the coarsening process assumes that grain boundaries are isotropic, or that grain boundary energies are identical along edges of the network. Under this idealization, we  define a \textbf{grain network with existence interval} $(0,T)$  as a geometric finite planar graph $G(t)$ which satisfies the
following restrictions for all $t\in(0,T)$:

\begin{enumerate}
\item (\textbf{Herring's Condition})   $G(t)$ is trivalent, and angles between two edges at a vertex are fixed at 120 degrees.
This restriction follows from a force balance law on edges  emanating from a vertex (see \cite{herring1999surface}).\item (\textbf{Mean Curvature Flow}) Boundaries are smooth, and satisfy mean curvature flow.  This means that the time evolution of a smooth boundary $\gamma(x,t)\subset \mathbb R^2$ is also smooth, and satisfies
 \begin{equation} \label{curveflow}
\partial_t \gamma(x,t) = M\sigma \kappa(x,t)  \vec n(x,t), \quad t\in(0,T), x \in \mathbb{R}^2,
\end{equation}
with $M$ and $\sigma$ denoting grain mobility and surface energy constants, $\vec n(x,t)$ denoting the inward unit normal, and $\kappa(x,t)$ denoting mean curvature. This type of motion was observed experimentally by Beck for crystallites in metals \cite{bec52}.
\end{enumerate}

Note that grain networks retain their topological structure during their existence time.  The parabolic PDE (\ref{curveflow})  is referred to as curve-shortening flow, and it can in fact be shown that the total edge length of a network decreases in time \cite{barmak2011entropy}. A theorem of curve shortening flow on grain networks, due to von Neumann and Mullins \cite{mul56,von1952discussion} gives an elegant relation between a topological quantity (sides of a grain) and a geometrical quantity (area of a grain).

\begin{figure}
\includegraphics[width=\textwidth]{samplegrains.png}
\caption{\textbf{Coarsening of a grain network.} Grains in material microstructure delete through annealing, causing the average grain size to increase.}\label{samplegrains}
\end{figure}

\begin{theorem}(\textbf{The von Neumann-Mullins $n-6$ rule}). For a grain with area $A$ and $n$ sides,
\begin{equation}
\frac{dA}{dt} = M\sigma\frac \pi 3 (n-6).
\end{equation}
\end{theorem}
One consequence of the $n-6$ rule  is  that grains with less than six sides may shrink to a point in finite time. Such deletions, along with possible edge deletions, result in networks with vertices that violate Herring's condition.  This raises the question: What types of solutions exist with initial conditions that are not grain networks?  Specifically, the \textbf{continuation problem for a planar network $G$} asks the following: If we are given a planar network $G$ with vertices that may not satisfy Herring's conditions,  is there an  $\varepsilon>0$ where $G(t)$ is a grain network for $t\in (0, \varepsilon)$, and $G(t)\rightarrow G$ in the Hausdorff distance as $t\rightarrow  0^+$? Another interesting question is whether the flow is unique, or if it can become unique under additional assumptions.

Currently, there are no comprehensive answers to the above questions.  Viewed as a problem in parabolic PDE theory, the current focus is local, fixing on simplified models with initial conditions of $k$ rays meeting at a vertex. Schnurer and Shulze found a unique self-similar solution with initial conditions of three rays meeting at arbitrary angles \cite{sch07}.  Mazzeo and Saez  generalized with the $k$-ray initial conditions, and discovered a bijection between self-similar solutions   and $k$-Steiner trees of a hyperbolic metric \cite{maz07}.  On the computational front, techniques of flowing through a graph not satisfying the grain network conditions include level set \cite{elsey2009diffusion}  and variational \cite{kin06} methods.

\section{Mean-field models of grain growth}\label{fradkov}
Another approach to studying grain boundary coarsening involves using mean field assumptions to gather bulk statistics of grain behavior.  A well known kinetic model is due to Fradkov \cite{fra882,fra881}, who exploits  the $n-6$ rule, along with an additional deterministic rule for side redistribution, to derive kinetic equations for densities of grains with a specific number of sides.   Fradkov makes the following mean field assumptions about the redistribution of sides when either a grain or edge deletes:
\begin{enumerate}
\item If a grain or side deletes, topological changes of grains will be selected in proportion to the number of sides of grains, and independent of their areas.  For instance, if a singular event causes a grain to lose a side, the probability that a particular grain with $j$ sides will drop to one with $j-1$ sides is
\begin{equation}
p_j=\frac{j}{\sum_{k>1} kN_k},
\end{equation}
where $N_k$ is the number of grains with $k$ sides.
  This eliminates the notion of grains having neighbors.
\item A free parameter $\beta$ is introduced which is defined as the constant ratio between side deletions and grain deletions.  Such an assumption comes from the lack of a topological rule for edge growth.
\end{enumerate}

For a number density $f_n(a,t)$ of $n $ sided grains with area $a$ at time $t$, the Fradkov equations take the form of a transport equation with a topological source term:
\begin{equation}
\partial_t f_n(a,t)+(n-6)\partial_a f_n(a,t) = \Gamma(f(t))(Jf)_n(a,t) \quad (a,t) \in (0,\infty)^2, n \ge 2 ,
\end{equation}
with a collision operator $J$ and coupling weight $\Gamma$ determined from conservation of area and polyhedral defect (networks retain an average of six sides). Well-posedness and self-similar solutions for the Fradkov model can be found in \cite{henseler2008kinetic, her12}.

\section{PDMPs and grain coarsening}\label{pdmpgraincoarse}
The main goal of this thesis is to understand grain  statistics as a hydrodynamic limit of finitely many grains respecting deterministic drift and random topological transitions.  In particular, we will  construct limiting kinetic equations describing the evolution of grain area densities for the class of grains with a fixed number of sides.   Toward this goal, the main tool we use is the theory of piecewise deterministic Markov process (PDMP).  The construction and theory of PDMPs will be discussed in Chapter 2.  However, not much is lost to think of a PDMP  as a generalization of a jump process for particles that drift on vector fields.

For the problem of grain coarsening, we wish to build a PDMP $X(t)$ that tracks a finite set of grains $(g_1,\dots, g_n) =  ((a_1,s_1), \dots, (a_n,s_n))$ with areas $a_i>0$ and side numbers $s_i \in \mathbb N.$
Note that the dimension of $X(t)$ can change with time, as grains delete from coarsening. Grain areas change at a constant rate, following the $n-6$ rule until a grain shrinks to a point, or an edge deletes (we will make an assumption on side deletion rates based on total particle number).  At that time, we impose mean field assumptions to change topologies of grains.

\begin{figure}
\begin{centering}
 \includegraphics[width = .4\textwidth]{graintops.png}
 \caption{\textbf{Grain growth via a PDMP}. Particles drift according to their number of sides. Each tier denotes particles of different side numbers. In this instance, a vanishing grain in tier three triggers a  random reassignment  of the number of sides of grains in tiers 4,7, and 8.}
\end{centering}
\end{figure}

\section{Overview}

In Chapter 2, we describe a PDMP in generality. We then state its infinitesimal generator $\mathcal A$, and the class of functionals $\mathcal D(\mathcal A)$ that give rise to martingales through the Dynkin formula.

In Chapter 3, we  study a simplified version of the PDMP described in Section \ref{pdmpgraincoarse}. Here, particles drift on the positive half-line until reaching the origin, where they are reassigned according to a probability density $p(x)$.     We first prove that for empirical densities approaching nonatomic initial conditions, the densities $u(x,t)$ of particles converge to a weak form of the limiting kinetic equation
\begin{alignat}{2}\label{pde1}
\partial_tu(x,t) - \partial_x u(x,t) = p(x)u(0,t) \quad t, x \in \mathbb{R}^+, \\
u(x,0) = u_0(x) . \nonumber
\end{alignat}
We then focus on constructing measure valued weak solutions of (\ref{pde1}) with mild restrictions on initial data. Finally, we use a popular result of renewal theory to show the existence of an attractor under certain assumptions.

  Chapter 4 is a generalization of PDMPs on grain networks described in Section \ref{pdmpgraincoarse}.  Here, we show the existence of fluid limits of PDMPs that allow generalized rules and probability distributions for particle jumps between different species.  As in Chapter 3, we will show the existence of a weak limit for densities $u_k(x,t)$ on $L_k$, that satisfy the advection-reaction equations
\begin{eqnarray} \label{pde2}
  \lefteqn{\partial_t u_k(x,t)+\partial_{x}(v_k(x)u_{k}(x,t))=  \phantom{u_j(x,t)\mathbf{1}_{R^{(l)}_{ij}}
-K^{(l)}W_k^{(l)}(t)u_k(x,t)}}\label{smoooth}\\
 && \sum_{l = 1}^{M_-}u_l(0,t)v_{l}(0)\left[\sum_{i = 1}^{M}\sum_{j= 1}^{K^{(l)}} W_i^{(l)}(t)u_i(x,t)\mathbf{1}_{R^{(l)}_{ij}=k}
-K^{(l)}W_k^{(l)}(t)u_k(x,t)\right]\nonumber\\
&&+\frac{\beta G(t)}{G^{int}(t)}\left( \sum_{i = 1}^{M}\sum_{j= 1}^{K^{int}} w^{int} _iu_{i}(x,t)\mathbf{1}_{R^{int}_{ij} = k}-K^{int}w^{int}_ku_{k}(x,t)
 \right), \nonumber
 \end{eqnarray}
 with initial conditions
 \begin{equation}
u_k(x,0) = u_{k}^0(x), \quad k = 1,\dots, M. \nonumber
\end{equation}
The solutions of (\ref{pde2}) will be shown to be unique in the class of $L^1\cap L^\infty(\mathbb R^+)$ functions.

Finally, in Chapter 5, we return to the problem of grain coarsening. First, we examine the topological behavior of grain networks before and after a grain or side vanishes. After finding a finite set of rules that dictate topological changes,    we then define the various free parameters in the $k$-species model to align with a mean field model for grain coarsening.  Next, we prove some basic properties of the limiting kinetic equation, such as conservation of mass and polyhedral defect.  Finally, we end the chapter with a computational investigation of grain growth, and raise several conjectures about grain statistics.


\subsection{Weak duality theorem}

The duality gap (\(p^*-d^*\) is non-negative.


\section{A few more things about coefficients, cohomology}
Problem Set 5 now due Friday! Recall our example where we let $\cI=(\Z_{>0},|)$. You can consider $\cI\to\mathbf{Ab}$, say assigning to each $i$ the integers $\Z$, and $f_{ij}:\Z\xrightarrow{j/i}\Z$. The colimit is $\QQ$, where you send $X_n\to\QQ$ via $1\mapsto \frac{1}{n}$. It's rather natural. But it's not the simplest way to define $\QQ$. You just could have defined $\QQ=\varinjlim(\Z\xrightarrow{2}\Z\xrightarrow{3}\Z\xrightarrow{4}\Z\to\cdots)$.

We saw that $\varinjlim$ is exact, that it commutes with tensor products, and that homology commutes with direct limits. So if I had a chain complex $C_\bullet$ of abelian groups, then the process of computing homology involves exact sequences. If I want to compute:
\begin{equation*}
 H(C_\bullet\otimes\QQ)= H(C_\bullet\otimes\varinjlim \Z)= H(\varinjlim(C_\bullet\otimes\Z))= H(\varinjlim C_\bullet)=\varinjlim H(C_\bullet)= H(C_\bullet)\otimes\varinjlim \Z= H(C_\bullet)\otimes\QQ
\end{equation*}
Thus $-\otimes\QQ$ is exact, i.e., $\QQ$ is \emph{flat} over $\Z$, so $\Tor^\ZZ_1(M,\QQ)=0$. By UCT, this means that $ H_\ast(X)\otimes \QQ\simeq H_\ast(X;\QQ)$.
\begin{definition}
The $n$th Betti number of $X$ is defined to be $\beta_n:=\dim_\QQ H_n(X;\QQ)$.
\end{definition}
The Euler characteristic can just as well have been defined as $\chi(X)=\sum^\infty_{n=0}(-1)^n\beta_n$.

Oh, let me ask if 11 am works for those of you who are going to take 18.906. OK? It seems like it's too early.

Anyway, every space has a diagonal map $X\xrightarrow{\Delta}X\times X$. This induces a map $ H_\ast(X;R)\to H_\ast(X\times X;R)$, where $R$ is a PID. But now, we have a K\"{u}nneth map $\alpha: H_\ast(X;R)\otimes_R H_\ast(X;R)\to H_\ast(X\times X;R)$. We can get some kind of comultiplication if $\alpha$ is an isomorphism. And, well, the K\"{u}nneth theorem says that $\Tor^R_1( H_\ast(X;R), H_\ast(X;R))=0$ if and only if $\alpha$ is an isomorphism. This condition is satisfied, for example, if each $ H_n(X;R)$ is flat over $R$ for all $n$ (for example, free over $R$). A special case is when $R$ is a field. So you get a comultiplication $\Delta: H_\ast(X;R)\to H_\ast(X;R)\otimes_R H_\ast(X;R)$ if this condition is satisfied.

This diagonal map has many properties.
\begin{definition}
Let $R$ be a ring. A (graded, bounded below) coalgebra over $R$ is a (graded) $R$-module $M$ with a multiplication $\Delta:M\to M\otimes_R M$ and an augmentation map $\varepsilon:M\to R$ such that all of the following diagrams commute:
\begin{equation*}
\xymatrix{ & M\ar[d]\ar[dr]\ar[dl] & \\
R\otimes_R M & M\otimes_R M\ar[l]^{\varepsilon\otimes 1}\ar[r]^{1\otimes\varepsilon} & M\otimes_R R}
\end{equation*}
Where the diagonal maps are the canonical isomorphisms. And you have coassociativity:
\begin{equation*}
\xymatrix{
	M\ar[r]^{\Delta}\ar[d]^{\Delta} & M\otimes_R M\ar[d]^{\Delta\otimes 1}\\
	M\otimes_R M\ar[r]_{1\otimes\Delta} & M\otimes_R M\otimes_R M
}
\end{equation*}
And it's cocommutative (he'll just say commutative because there's no need to say ``co'' if we know we're working with coalgebras, but I want to write it anyway -- I'll probably abuse it though) if:
\begin{equation*}
\xymatrix{
	 & M\ar[dl]^\Delta\ar[dr]^\Delta &\\
	M\otimes_R M\ar[rr]^{\tau} & & M\otimes_R M
}
\end{equation*}
Where $\tau(x\otimes y)=(-1)^{|x|\cdot|y|}y\otimes x$ is the twisting map.
\end{definition}
\begin{example}
The K\"{u}nneth map is coassociative and cocommutative.
\end{example}
Consider:
\begin{equation*}
\xymatrix{S_\ast(X)\otimes S_\ast(Y)\ar[r]^{\tau}\ar[d]^{\times} & S_\ast(Y)\otimes S_\ast(X)\ar[d]^{\times}\\
S_\ast(X\times Y)\ar[r]^{\tau_\ast} & S_\ast(Y\times X)}
\end{equation*}
Where $\tau$ is as defined above on the tensor product and $\tau$ is also used to denote the twisting map $X\times Y\to Y\times X$. And this diagram commutes up to chain homotopy.
\begin{corollary}
Let $k$ be a field. Then $ H_\ast(X;k)$ has the natural structure of a cocommutative graded coalgebraic structure.
\end{corollary}
I could just talk about coalgebras. But one of my friends told me that nobody in France knew what coalgebras were. So we're going to talk about cohomology, and get an algebra structure. Some say that cohomology is better because you have algebras, but that's more of a sociological statement than a mathematical one.
\begin{slogan}
Cohomology gives algebras. It's a contravariant functor on spaces.
\end{slogan}
A better reason for looking a cohomology is that there are many geometric constructions that pull back. For example, if I have some covering space $\widetilde{X}\to X$, and I have a map $f:Y\to X$, I get a pullback covering space $f^{\ast}\widetilde{X}$. A better example is vector bundles (that we'll talk about in 18.906) -- they don't push out, they pullback. This'll give the theory of \emph{characteristic classes}. Another even better reason is that cohomology is the target of the Poincar\'{e} duality map.
\begin{definition}
Let $N$ be an abelian group. A singular $n$-cochain on $X$ with coefficients in $N$ is a function $\Sin_n(X)\to N$.
\end{definition}
If $N$ is an $R$-module, then I can extend linearly to get a map $S_n(X;R)\to N$.
\begin{notation}
Write $S^n(X;N):=\Map(\Sin_n(X);N)=\Hom_R(S_n(X;R);N)$.
\end{notation}
This is going to give me something contravariant, that's for sure. But I want to get a cochain complex $(S^\ast(X;N),d)$. Cochain means that the differential increases the degree.

I have a map $(-,-):S^n(X;N)\otimes S_n(X;R)\to N$ given by evaluation. I want $d$ on $S^\ast(X;N)$ that makes the map $S^n(X;N)\otimes S_n(X;R)\to N$ is a chain map where we regard $N$ as a chain complex with zeros everywhere except in dimension $0$. But the way I've said it doesn't quite make sense because $S^\ast(X;N)$ isn't a chain complex! So let $S^\ast(X;N)$ be a chain complex with $S^n(X;N)$ in dimension $(-n)$. But now it's not bounded below. So, to be honest, I should say that we're going to make a decision. If $C_\bullet,D_\bullet$ aren't necessarily bounded below chain complex, then $(C_\bullet\otimes D_\bullet)_n:=\bigoplus_{i+j=n}C_i\otimes D_j$.

Let $f\in S^n(X;R)$ and $\sigma\in S_n(X;R)$. This is a chain map if $d (f,\sigma)=0$. And $d(f,\sigma)=(df,\sigma)+(-1)^{|f|}(f,d\sigma)$. Let me erase this. This doesn't make sense. Scratch that. Let's start over.

Let $f\in S^{n+1}(X;R)$ and $\sigma\in S_n(X;R)$. Then $(df,\sigma)+(-1)^{|f|}(f,d\sigma)=d(f,\sigma)=0$. So $(df)(\sigma)=(-1)^{|f|+1}f(d\sigma)$. That's what the differential is. This makes $S^\ast(X;N)$ a cochain complex. So you can consider its homology.
\begin{definition}
The $n$th cohomology of $X$ with coefficients in $N$ is $ H^n(X;N):= H^n(S^\ast(X;N))$. This is a contravariant functor on $\mathbf{Top}$.
\end{definition}
If you fix $X$, I get a covariant functor $ H^n(X;-)$ on $\mathbf{Mod}_R$.
\begin{construction}
I have a chain map $(,):S^\ast(X;N)\otimes_R S_\ast(X;R)\to N$. I can apply homology to this, to get $\alpha: H^\ast(X;N)\otimes_R H_\ast(X;R)\to H(S^\ast(X;N)\otimes_R S_\ast(X;R))$. In particular, you get a natural pairing $(,): H^\ast(X;N)\otimes_R H_\ast(X;R)\xrightarrow{\alpha} H(S^\ast(X;N)\otimes_R S_\ast(X;R))\xrightarrow{(,)}N$. This ``evaluation map'' is called the Kronecker pairing.
\end{construction}
\begin{warning}
$S^n(X;\Z)=\Map(\Sin_n(X);\Z)=\prod_{\Sin_n(X)}\Z$, which is probably an uncountable product. An awkward fact is that this is never free abelian.
\end{warning}
\begin{construction}
If $A\subseteq X$, there is a restriction map $S^n(X;N)\to S^n(A;N)$. There is an injection $\Sin_n(A)\hookrightarrow \Sin_n(X)$. And as long as $A$ is empty, I can split this. So any function $\Sin_n(A)\to N$ extends to $\Sin_n(X)\to N$. This means that $S^n(X;N)\to S^n(A;N)$ is surjective. You can define the kernel as $S^n(X,A;N)$, which sits in $0\to S^n(X,A;N)\to S^n(X;N)\to S^n(A;N)\to 0$. This gives the \emph{relative cochains}.
\end{construction}
I can take the homology of this sexseq to get a lexseq:
\begin{equation*}
\xymatrix{
	\cdots & & \\
	 H^1(X,A;N)\ar[r] & H^1(X;N)\ar[r] & H^1(A)\ar[ull]^{\delta}\\
	 H^0(X,A;N)\ar[r] & H^0(X;N)\ar[r] & H^0(A)\ar[ull]^{\delta}
}
\end{equation*}
And $ H^0(X;N)$ sits in the cokernel of $\Map(\Sin_0(X),N)\to \Map(\Sin_1(X),N)$, so that $ H^0(X;N)=\Map(\pi_0(X),N)$.


Having described how to operate with basic sets, we consider a more fundamental representation problem.
%
We have seen that there exist classes of sets.
Some of them, such as polyhedra, are closed under various set operations. That property is convenient because the type of the result is known in advance.
%
The whole class of convex sets is also closed under the operations described in the upper part of Table~\ref{tab:operations}, but we can typically not represent the result with the limited amount of set types in LazySets. This is not a shortcoming of LazySets: you would need infinitely many set representations for all possible combinations! However, we can resort to a simple yet powerful trick to effectively represent the result of the set operations: \emph{lazy representation}.
%
Going a step further, by making both basic set types and operations between sets live in the same abstraction layer (namely always subtyping \code{LazySet} irrespective whether it is a concrete set or the result of an operation) allows to easily \emph{compose} set computations.

\subsection{Type composition} \label{sec:composition}

To give an example, the Minkowski sum of a square and a disc (two-dimensional balls in the infinity norm and Euclidean norm) is not representable with a basic type in LazySets. Hence \code{minkowski\_sum} will yield an error. But we can apply the \emph{lazy} operation, which is called \code{MinkowskiSum}. \code{MinkowskiSum} itself subtypes \code{LazySet}. This choice allows for ease of composition.

We wrap the operands in a new object that, by definition, represents the result of the operation, but without actually performing the computation. (This also motivates the name of LazySets).

\smallskip

As an illustrative example, suppose that we are interested in the formula $\Omega_0 = \CH(\X_0, \Phi \X_0 \oplus Y)$, where $\CH$ and $\oplus$ are defined in Table~\ref{tab:operations}, $X_0$ and $Y$ are sets, and $\Phi$ is a matrix defined in Appendix~\ref{sec:omega0}. Such formulas are prevalent in reachability analysis of linear initial-value problems, or nonlinear ones after some form of conservative linearization; see for example \cite{althoff2020set,ForetsS21} and references therein. Given the sets $\X_0$, $Y$, and the matrix $\Phi$, we can write:

\begin{minipage}{\linewidth}
\begin{lstlisting}
julia> Ω₀ = CH(X₀, Φ*X₀ ⊕ Y)
\end{lstlisting}
\end{minipage}

\begin{minipage}{0.69\linewidth}
Then, $\Omega_0$ is a (nested) \emph{lazy} representation of the  operation just as a normal \code{LazySet}. As such, it can be used for further operations (conversion, approximation, evaluation). The structure of the nested operations is internally represented in the form of a tree, which can be visualized with \href{https://github.com/JuliaTeX/TreeView.jl}{TreeView.jl} as shown in the diagram (right).
\end{minipage}
\hfill
\begin{minipage}{0.2\linewidth}
	\begin{tikzpicture}[level/.style={sibling distance=8mm,level distance=8mm}]
		\node {\texttt{CH}}
		child {
			node {\texttt{X}}
		}
		child {
			node {$\oplus$}
			child {
				node {\texttt{*}}
				child {
					node {$\Phi$}
				}
				child {
					node {\texttt{X}}
				}
			}
			child {
				node {\texttt{Y}}
			}
		};
	\end{tikzpicture}
\end{minipage}

\smallskip

Lazy operations can be efficiently evaluated, as we describe next.

\subsection{Support-function calculus} \label{sec:supfun}

A standard approach to operate with compact and convex sets $\X \in \R^n$ is to use the \emph{support function} \cite{LeGuernic09}.
%
The support function along direction $d \in \R^n$, denoted $\rho(d, \X)$, is the maximum of $d^T x$ over all $x \in \X$, i.e., the support function describes the (signed) distance of the supporting hyperplane in direction $d$ from the origin.
%
It can be used to efficiently find the boundary of a set in a given direction.
%
The maximizers are called \emph{support vectors} $\sigma(d, \X)$. Intuitively, the support vectors are the extreme points of $\X$ in direction $d$.
%
Basic properties of the support function are given in Appendix~\ref{sec:supfunc_properties}.

For various set representations, the support function is known analytically and can be efficiently evaluated numerically. Such cases include hyperrectangular sets and zonotopic sets.
%
For sets with less structure, e.g., if $\X$ is a polytope in half-space representation, its support function can be computed by solving a linear program, for which fast and robust solvers exist.
%
But the main advantage of using the support function in LazySets lies in the extensive use of \emph{composition rules}, as we describe later.

\smallskip

LazySets offers \code{$\uprho$(d, X)} (or \code{support\_function(d, X)}) to compute the support function $\rho(d,\X)$, and \code{$\upsigma$(d, X)} (or \code{support\_vector(d, X)}) to compute (some) support vector $\sigma(d,\X)$. Fig.~\ref{fig:supfunc} (left) illustrates the evaluation of the support function over the polygon $\X$ (orange) along direction $(-1, 1)^T$.

\begin{minipage}{\linewidth}
\vspace{-\abovedisplayskip}
\begin{lstlisting}
julia> d = [-1, 1]

julia> ρ(d, X) # or support_function(d, X)
3.6

julia> σ(d, X) # or support_vector(d, X)
[-3.0, 0.6]
\end{lstlisting}
\end{minipage}

\begin{figure}
	\centering
	\includegraphics[width=0.49\linewidth, keepaspectratio]{img/supfunc}
	\includegraphics[width=0.49\linewidth, keepaspectratio]{img/supfunc_oct}
	\vspace*{1mm}
	\caption{Left: The supporting hyperplane of the set $\X$ along direction $d$.
	In red we plot the distance of the hyperplane to the origin, which is given by $\rho(d', \X)$ where $d' = d / \Vert d \Vert$.
	Right: An outer approximation of $\X$ using the eight directions of a regular octagon.}
	\label{fig:supfunc}
\end{figure}

Consider again the set $\Omega_0$ from the previous section. Suppose that we are interested in the support value of $\Omega_0$ along a given direction $d \in \R^2$.
%
Since the support function distributes over the Minkowski sum, $\rho(d, \X \oplus \Y) = \rho(d, \X) + \rho(d, \Y)$ for any pair of sets $\X, \Y \subseteq \R^n$, and since it holds that $\rho(d, M \X) = \rho(M^T d, \X)$ for any matrix $M \in \R^{n \times n}$, we can propagate the computation through the operation tree until a concrete set is found, and in many cases, an analytic formula is available.
%
That is precisely what LazySets does, automatically, when we make queries to the lazy set such as asking for its support function along $d = (-1, 1)^T$ from before.

\begin{minipage}{\linewidth}
	\vspace{-\abovedisplayskip}
	\begin{lstlisting}
julia> @btime ρ($d, $Ω₀)
117.236 ns (2 allocations: 192 bytes)
-0.8

julia> @btime ρ($d, concretize($Ω₀))
  23.700 μs (203 allocations: 16.03 KiB)
-0.8
	\end{lstlisting}
\end{minipage}

In the first benchmark we evaluate the support function on the lazy set.\footnote{The Julia macro \code{@btime} provided by the \href{https://github.com/JuliaCI/BenchmarkTools.jl}{BenchmarkTools.jl} package evaluates the given command multiple times and returns the smallest record.}
In the second benchmark we first convert to a concrete set representation (in this case a polygon in vertex representation) and then evaluate the support function.
The first case is two orders of magnitude faster.
This exemplifies that set computations can be implemented efficiently using the support function, which becomes more prominent in higher dimensions.

We note that in the above computation we have only obtained a bound in direction $d$, not in other directions. For many applications it is sufficient to evaluate the support function in only a few directions. For example, in the helicopter model presented in Section~\ref{sec:reachability} we are only interested in the vertical velocity, so we only need to evaluate the support function twice to obtain the upper and lower bound in that dimension.
Another example is to enclose $\Omega_0$ with a bounded set, for which we can pick a list of \emph{template} directions (see Section~\ref{sec:approximation}).


\documentclass{memoir}
\usepackage{notestemplate}

% \begin{figure}[ht]
%     \centering
%     \incfig{riemmans-theorem}
%     \caption{Riemmans theorem}
%     \label{fig:riemmans-theorem}
% \end{figure}

\begin{document}
\subsection{Direct Product of Groups}
\begin{defn}[Direct Product]
	The \textbf{direct product} \(G_1\times G_2\) of groups \(G_1,G_2\) is the group of all ordered pairs \((g_1,g_2)\) where \(g_i \in G_i\), with the usual definition of multiplication:
	\begin{align*}
		(g_1,g_2)(h_1,h_2) = (g_1h_1,g_2h_2).
	\end{align*}
\end{defn}
	It is clearly a group, and the projection subsets \(G_1^{*}= \left\{(g_1,e_2) \mid g_1 \in G_1 \right\} \) and \(G_2^{*}=\left\{(e_1,g_2) \mid g_2 \in G_2 \right\} \) are isomorphic to their respective groups. This in turn tells us that \(\left| G_1\times G_2 \right| = \left| G_1 \right| \left| G_2 \right| \).\\
	\begin{prop}
\(G_1^{*},G_2^{*} \triangleleft G_1\times G_2\) are normal subgroups, and every \(u \in G_1\times G_2\) can be decomposed as \(u = u_1u_2\), \(u_i \in G_i^{*}\).\\

The converse holds as well; if \(N,M\) are normal subgroups of a group \(G\), and every \(g \in G\) can be written as \(g = nm\), \(n \in N, m \in M\), then \(G \cong N\times M\).
	\end{prop}
Of course, \((G_1\times G_2) / G_1^{*} \cong G_2\) and vice versa.\\

We can even take the direct product of more than two groups:
\begin{align*}
	g = (g_1,g_2,\ldots,g_n) \in G_1\times G_2\times \ldots \times G_n\\
	(g_1,g_2,\ldots,g_n)(h_1,h_2,\ldots,h_n) = (g_1h_1,g_2h_2,\ldots,g_nh_n).
\end{align*}

We will later see that direct products provide us a means of characterizing all finitely generated abelian groups.
\end{document}

Abstract Algebra - Introductory/Algebra I/Notes/source/2020-04-07-GroupProd.tex


%!TEX root = TTK4215-Summary.tex
\section{Adaptive pole placement control (APPC)}
APPC is pretty cool, because it does not require a plant of minimum phase (as opposed to MRAC).

\subsection{Indirect PPC}
Given the system
\begin{equation}
	y_p = \frac{Z_p(s)}{R_p(s)} u_p
\end{equation}
where
\begin{itemize}
	\item $R_p$ is monic, of known degree $n$,
	\item $Z_p, R_p$ are coprime,
	\item $G_p$ strictly proper ($\deg{Z_p} < n$).
\end{itemize}

The objective is to choose $u_p$ so that the closed-loop poles are the roots of a polynomial $A^*(s)$. Can be done by choosing $P$ (degree $q+n-1$) and $L$ (monic, degree $n-1$) such that
\begin{equation}
	L Q_m R_p + P Z_p = A^*
\end{equation}
is satisfied for $A^*$ of degree $2n+q-1$. $Q_m$ is chosen so that
\begin{equation}
	Q_m y_p = 0,
\end{equation}
and the control input is given by
\begin{equation}
	Q_m L u_p = -P (y_p - y_m)
	.
\end{equation}

TTK4215 System identification and adaptive control/sec-appc.tex


\section{Introduction}
We will begin with a brief introduction of the basic terminology of algebraic geometry. This script will follow standard notation as in Bourbaki cfg. \cite{bou} A integral domain (ID) is a unital commutative ring, with no zero divisors. For this, we define a specialised functor
$$\bao{rrcl}
\mrm{Ann}_R(\cdot) &:\modr& \longrightarrow& \modr(R)\\
&&&\\
&M &\longmapsto&\{r \in R : r m = 0\; \forall m \in M\}\\
\ea$$
In particular, $\mrm{Ann}_R(R)  = \{r \in R : r' \in R \backslash\{0\},\; r r' = 0\}$.
\subsection{Affine and projective schemes}
For a unital commutative ring $R$ we define a prime ideal to be an ideal $\prm \subsetneq R$ such that for all $r, s \in R$ we have
$$r s \in \prm \ \Rightarrow r \in \prm \vee s \in \prm.$$
Furthermore, for $R$ as above we define
$$\specr := \{\prm \in \mathrm{Mod}_R : \prm \ \mathrm{prime}\}$$
to be the spectrum of $R$. Thereby, $\mathrm{Mod}_R$ is the category of $R$-modules. We call an ideal $\mx \in \mathrm{Mod}_R$ maximal if for all ideals $\mathfrak{n} \in \mathrm{Mod}_R$
$$\mx \subset \mathfrak{n} \Leftrightarrow \mx = \mathfrak{n} \ \mathrm{or}\ R = \mathfrak{n}.$$
We denote the class of all maximal ideals with $\mathrm{Max}(R)$. If $R$ is unital as assumed we get that $\mathrm{Max}(R) \subset \specr$. Let $R$ be as above and $\prm \in \specr$ the localisation, $S_\prm^{-1} R$, of $R$ wrt. $\prm$ is defined to be
$$R_\prm := S_{\prm}^{-1}R = S_\prm \times R/\sim, \ \mathrm{with}\ \sim =\left\{\left((s,r),(s',r')\right) : \exists t \in S_\prm, \;\mrm{s.t.}\; t(s' r - s r') \in \prm\right\},$$
where $S_\prm = \{r \in R : r \notin \prm\} = R \backslash \prm$ and with addition and multiplication:
$$\begin{array}{rcl}
m_+ &=& \left[\left([s,r],[s',r']\right) \longmapsto [s s', s' r + s r']\right],\\
&&\\
m_\cdot &=& \left[\left([s,r],[s',r']\right) \longmapsto [s s', r r']\right],\\
\end{array}$$
respectively. We remark that $S_\prm^{-1}R$ is a local ring i.e. it contains only one maximal ideal. Given a non-zero divisor and non-unit $r \in R$, we define
$$S_r := R \backslash \bigcup_{r \notin \prm} \prm.$$
This is set is in general not an ideal. However, it is a multiplicative monoid: if $a$ is not in $\prm$ and $b$ is not in $\prm$ then neither is $a b$. This holds for all $\prm$ not containing $r$. Now, given a topological space $(X, \tau)$ and a ring $R$ we get the
$$S := \mathrm{map}(X,R) = R^X$$
the ring of $R$-valued functions.
\subsubsection{Affine case}
\newcommand{\calf}{\mathcal{F}}
Given a finite set of elements $F = \{f \in R\}$, we define its ideal to be
$$I(F) := \left\{\sum_{f \in F} p_f f : p_f \in R,\ \forall f \in F\right\} = \sum_{f \in F} R. f.$$
Conversely, for any ideal $I \in \modr$, the set of prime ideals containing $I$ is defined to be
$$V(I) := \{\prm \in \specr : I \subset \prm\},$$
the so-called zero set (or more specifically: common set of zeros). For two ideals $I, J \in \modr$, the sums and products are:
$$\bao{rcl}
V(I \cdot J) &=& \{\prm : I J \subset \prm\}\\
&&\\
&=& \{\prm : I \cdot J \subset I \cap J \subset I, J \subset \prm\}\\
&&\\
&=& \{\prm : I \cdot J \subset I \cap J \subset I \subset \prm \vee I \cdot J \subset I \cap J \subset J \subset \prm\}\\
&&\\
&=& \{\prm : I \cdot J \subset I \cap J \subset I, J \subset \prm\} \cup \{\prm:I \cdot J \subset I \cap J \subset J \subset \prm\}\\
&&\\
&=&V(I) \cup V(J)\\

\ea$$
The argument goes as follows: $I J \subset \prm$ implies $I \subset \prm \vee J \subset \prm$.
$$\bao{rcl}
V(I + J) &=& \{\prm : I + J \subset \prm\}\\
&&\\
&=& \{\prm : I + J \subset \prm \Leftrightarrow I \subset \prm \vee J \subset \prm\}\\
&&\\
&=& \{\prm : I \subset \prm\} \cap \{\prm : J \subset \prm\}\\
&&\\
&=& V(I) \cap V(J)\\
\ea$$
We remark that in general unital commutative rings $R$, there need not to be a unique (up to sequence and association) factorisation. Therefore, $I \cdot J \subsetneq I \cap J$. One promiment example:
\begin{defi} Some important definitions:
\bd
\item[Associatedness] Two elements $r, r' \in R$ are called associated, if there exists an $c \in R^\times$ s.t.
$$r = c r'.$$
\item[Irreducibility] An element $r \in R \backslash (R^\times \cup \{0\})$ is called irreducible iff each factorization is associated:
$$c r' = r \Rightarrow c \in R^\times,$$
in particular, $r'$ is its associated irreducible.1
\item[UFD] An ID $R$ is called a unique factorization domain (UFD) if for each $r \in R\backslash R^\times \cup \{0\}$, there exists irreducible elements $p_i$ s.t. the factorization
$$r = \prod_i p_i^{s_i}, \sum_i s_i \geq 1$$
is unique up to associatedness and permutation.
\item[PID] An UFD $R$ is called a principle ideal domain, if each ideal $I$ is generated by one element.
\item[EUCL] A PID $R$ is called euclidean, if there are a multiplicative map $\phi : R \backslash\{0\} \longrightarrow \zz_{>0}, $
\ed
\end{defi}
\ex Following rings are all examples of UFDs:
$$\zz,\ \qz,\ \fz{p},\ \mrm{where}\ p\ \mrm{prime}.$$
\ex The ring $R = \zz[\sqrt{-3}] = \zz \oplus \zz.\sqrt{-3}$: The element $2 \in R$ is irreducible as any divisor $d \in R$ is either a unit $d \in R^\times = \mrm{Gl}_1(R) = \{\pm 1\}$, or associated to $2$.\\
\indent Similarly, $1 + \sqrt{-3}$ is irreducible by the same argument and the consideration that:
$$R \simeq_{\mrm{Rng}} \zz.I_2 \oplus \zz.\left(\bao{cc}0&-3\\1&0\\\ea\right), \det_\zz : R \longrightarrow \zz, r \longmapsto \det \left(\bao{cc}r & -3 r'\\r'&r\\\ea\right) = r^2 + 3 r'^2 .$$
Therefore, $\det_\zz (1 + \sqrt{-3}) = 1 + 3 = 4 = \det_\zz 2$. We arrive immediately at
$$2 \mid 4 \wedge 1 + \sqrt{-3} \mid 4 \Leftrightarrow 4 \in \left<2\right> \wedge 4 \in \left<1 + \sqrt{-3}\right>$$
without any of the two to be associated. Moreover, one can show quickly that
$$\left<2\right> \cdot \left<1 + \sqrt{-3}\right> = \left<2 + 2 \sqrt{-3}\right> \subsetneq \left<2\right> \cap \left<1 + \sqrt{-3}\right> = \left\{\left(\bao{cc}r&-3 r'\\r'&r\\\ea\right) : r + r' \in 2 \zz\right\}.$$
An interesting result - the intersection is strictly greater than the product of two (principal) ideals. This is one of the prominent examples of a non-UFD. Moreover, $4$ has no unique decomposition wrt to irreducibility:
$$(1 + \sqrt{-3}) (1 - \sqrt{-3}) = 1 - (-3) = 4 = 2^2.$$
as well as
$$2 - (1 + \sqrt{-3}) = 1 - \sqrt{-3} = \frac{4}{1 + \sqrt{-3}}.$$
This example will be revisited in one of the following sections.
\begin{theo}
If $R$ is a UFD then so is its polynomial ring $R[X]$.
\end{theo}
This theorem has important implications - we can propose the existence of a decomposition of each finitely generate module and a given endomorphism:
\begin{coro}
Let $M \in \mrm{Mod}_R$ finitely generated, $\varphi \in \mrm{End}_R(M)$, $\mathbb{P}$ a representation of all irreducible polynomials in $R[X]$ and $p_\varphi \in R[X]$ its characteristic polynomial then there exists a unique monic polynomial $m_\varphi$ of smallest degree with decomposition:
$$m_\varphi = \prod_{\substack{p \in \mathbb{P}\\p \mid p_\varphi\\}} p^{s_p},\ s_p \geq 1 \forall p$$
such that $m_\varphi(\varphi) = 0 \in R[\varphi]$. Then there exists a sequence of integers $t_p \leq s_p$ and $u_{t_p}\geq 1$ such that
$$R[\varphi] \simeq \sum_{\substack{p \in \mathbb{P}\\p \mid p_\varphi}}\left(R[X]/\left<p^{t_p}\right>\right)^{u_{t_p}}$$
and $\sum_p u_{t_p} t_p = deg p_\varphi$.
\end{coro}