\chapter{Properties of metric spaces}
At the end of the last chapter on metric spaces,
we introduced two adjectives ``open'' and ``closed''.
These are important because they'll grow up
to be the \emph{definition} for a general topological space,
once we graduate from metric spaces.

To move forward, we provide a couple niceness adjectives
that applies to \emph{entire metric spaces},
rather than just a set relative to a parent space.
They are ``(totally) bounded'' and ``complete''.
These adjectives are specific to metric spaces,
but will grow up to become the notion of \emph{compactness},
which is, in the words of \cite{ref:pugh},
``the single most important concept in real analysis''.
At the end of the chapter,
we will know enough to realize that something is amiss
with our definition of homeomorphism,
and this will serve as the starting point for the next chapter,
when we define fully general topological spaces.

\section{Boundedness}
\prototype{$[0,1]$ is bounded but $\RR$ is not.}
Here is one notion of how to prevent a metric space
from being a bit too large.

\begin{definition}
	A metric space $M$ is \vocab{bounded}
	if there is a constant $D$ such that
	$d(p,q) \le D$ for all $p,q \in M$.
\end{definition}
You can change the order of the quantifiers:
\begin{proposition}
	[Boundedness with radii instead of diameters]
	A metric space $M$ is bounded if and only if
	for every point $p \in M$, there is a radius $R$
	(possibly depending on $p$) such that $d(p,q) \le R$ for all $q \in M$.
\end{proposition}
\begin{exercise}
	Use the triangle inequality to show these are equivalent.
	(The names ``radius'' and ``diameter'' are a big hint!)
\end{exercise}

\begin{example}
	[Examples of bounded spaces]
	\listhack
	\begin{enumerate}[(a)]
		\ii Finite intervals like $[0,1]$ and $(a,b)$ are bounded.
		\ii The unit square $[0,1]^2$ is bounded.
		\ii $\RR^n$ is not bounded for any $n \ge 1$.
		\ii A discrete space on an infinite set is bounded.
		\ii $\NN$ is not bounded, despite being
		homeomorphic to the discrete space!
	\end{enumerate}
\end{example}

The fact that a discrete space on an infinite set
is ``bounded'' might be upsetting to you, so
here is a somewhat stronger condition you can use:

\begin{definition}
	A metric space is \vocab{totally bounded}
	if for any $\eps > 0$,
	we can cover $M$ with finitely many $\eps$-neighborhoods.
\end{definition}
For example, if $\eps = 1/2$, you can cover $[0,1]^2$
by $\eps$-neighborhoods.
\begin{center}
	\begin{asy}
		size(4cm);
		draw(shift( (-0.5,-0.5) )*unitsquare, black+1);
		real d = 0.4;
		real r = 0.5;
		draw(CR(dir( 45)*d, r), dotted);
		draw(CR(dir(135)*d, r), dotted);
		draw(CR(dir(225)*d, r), dotted);
		draw(CR(dir(315)*d, r), dotted);
	\end{asy}
\end{center}
\begin{exercise}
	Show that ``totally bounded'' implies ``bounded''.
\end{exercise}
\begin{example}
	[Examples of totally bounded spaces]
	\listhack
	\begin{enumerate}[(a)]
		\ii A subset of $\RR^n$ is bounded if and only if
		it is totally bounded.

		This is for Euclidean geometry reasons:
		for example in $\RR^2$ if I can cover a set
		by a single disk of radius $2$,
		then I can certainly cover it by finitely many
		disks of radius $1/2$.
		(We won't prove this rigorously.)

		\ii So for example $[0,1]$ or $[0,2] \times [0,3]$
		is totally bounded.

		\ii In contrast, a discrete space on
		an infinite set is not totally bounded.
	\end{enumerate}
\end{example}

\section{Completeness}
\prototype{$\RR$ is complete, but $\QQ$ and $(0,1)$ are not.}

So far we can only talk about sequences converging if they have a limit.
But consider the sequence
\[ x_1 = 1, \; x_2 = 1.4, \; x_3 = 1.41, \; x_4 = 1.414, \dots. \]
It converges to $\sqrt 2$ in $\RR$, of course.
But it fails to converge in $\QQ$;
there is no \emph{rational} number this sequence converges to.
And so somehow, if we didn't know about the existence of $\RR$, we would
have \emph{no idea} that the sequence $(x_n)$ is ``approaching'' something.

That seems to be a shame.
Let's set up a new definition to describe these sequences whose terms
\textbf{get close to each other},
even if they don't approach any particular point in the space.
Thus, we only want to mention the given points in the definition.

\begin{definition}
	Let $x_1, x_2, \dots$ be a sequence which lives in a metric space $M = (M,d_M)$.
	We say the sequence is \vocab{Cauchy} if for any $\eps > 0$, we have
	\[ d_M(x_m, x_n) < \eps \]
	for all sufficiently large $m$ and $n$.
\end{definition}

\begin{ques}
	Show that a sequence which converges is automatically Cauchy.
	(Draw a picture.)
\end{ques}

Now we can define:
\begin{definition}
	A metric space $M$ is \vocab{complete} if every
	Cauchy sequence converges.
\end{definition}

\begin{example}
	[Examples of complete spaces]
	\listhack
	\begin{enumerate}[(a)]
		\ii $\RR$ is complete.
		(Depending on your definition of $\RR$,
		this either follows by definition, or requires some work.
		We won't go through this here.)
		\ii The discrete space is complete,
		as the only Cauchy sequences are eventually constant.
		\ii The closed interval $[0,1]$ is complete.
		\ii $\RR^n$ is complete as well.
		(You're welcome to prove this by induction on $n$.)
	\end{enumerate}
\end{example}

\begin{example}
	[Non-examples of complete spaces]
	\listhack
	\begin{enumerate}[(a)]
		\ii The rationals $\QQ$ are not complete.
		\ii The open interval $(0,1)$ is not complete,
		as the sequence $0.9$, $0.99$, $0.999$, $0.9999$, \dots
		is Cauchy but does not converge.
	\end{enumerate}
\end{example}

So, metric spaces need not be complete, like $\QQ$.
But we certainly would like them to be complete,
and in light of the following theorem this is not unreasonable.
\begin{theorem}
	[Completion]
	Every metric space can be ``completed'',
	i.e.\ made into a complete space by adding in some points.
\end{theorem}
We won't need this construction at all,
so it's left as \Cref{prob:completion}.
\begin{example}
	[$\QQ$ completes to $\RR$]
	The completion of $\QQ$ is $\RR$.
\end{example}
(In fact, by using a modified definition of completion
not depending on the real numbers,
other authors often use this as the definition of $\RR$.)

\section{Let the buyer beware}
There is something suspicious about both these notions:
neither are preserved under homeomorphism!

\begin{example}
	[Something fishy is going on here]
	\label{ex:fishy}
	Let $M = (0,1)$ and $N = \RR$.
	As we saw much earlier $M$ and $N$ are homeomorphic.
	However:
	\begin{itemize}
		\ii $(0,1)$ is totally bounded, but not complete.
		\ii $\RR$ is complete, but not bounded.
	\end{itemize}
\end{example}

This is the first hint of something going awry with the metric.
As we progress further into our study of topology,
we will see that in fact \emph{open and closed sets}
(which we motivated by using the metric)
are the notion that will really shine later on.
I insist on introducing the metric first so that
the standard pictures of open and closed sets make sense,
but eventually it becomes time to remove the training wheels.

%It's a theorem that $\RR$ is complete.
%To prove this I'd have to define $\RR$ rigorously, which I won't do here.
%In fact, there are some competing definitions of $\RR$.
%It is sometimes defined as the completion of the space $\QQ$.
%Other times it is defined using something called \emph{Dedekind cuts}.
%For now, let's just accept that $\RR$ behaves as we expect and is complete.

%And thus, I can now tell you exactly what $\RR$ is.
%You might notice that it's actually not that easy to define:
%we can define $\QQ = \left\{ a/b : a,b \in \NN \right\}$, but
%what do we say for $\RR$?
%Here's the answer:
%\begin{definition}
%	$\RR$ is the completion of the metric space $\QQ$.
%\end{definition}

\section{Subspaces, and (inb4) a confusing linguistic point}
\prototype{A circle is obtained as a subspace of $\RR^2$.}

As we've already been doing implicitly in examples, we'll now say:
\begin{definition}
	Every subset $S \subseteq M$ is a metric space in its own right,
	by re-using the distance function on $M$.
	We say that $S$ is a \vocab{subspace} of $M$.
\end{definition}
For example, we saw that the circle $S^1$
is just a subspace of $\RR^2$.

It thus becomes important to distinguish between
\begin{enumerate}[(i)]
	\ii \textbf{``absolute'' adjectives} like ``complete'' or ``bounded'',
	which can be applied to both spaces,
	and hence even to subsets of spaces (by taking a subspace),
	and
	\ii \textbf{``relative'' adjectives}
	like ``open (in $M$)'' and ``closed (in $M$)'',
	which make sense only relative to a space,
	even though people are often sloppy and omit them.
\end{enumerate}
So ``$[0,1]$ is complete'' makes sense,
as does ``$[0,1]$ is a complete subset of $\RR$'',
which we take to mean ``$[0,1]$ is a complete as a subspace of $\RR$''.
This is since ``complete'' is an absolute adjective.

But here are some examples of ways in which relative adjectives
require a little more care:
\begin{itemize}
	\ii Consider the sequence $1$, $1.4$, $1.41$, $1.414$, \dots.
	Viewed as a sequence in $\RR$, it converges to $\sqrt 2$.
	But if viewed as a sequence in $\QQ$,
	this sequence does \emph{not} converge!
	Similarly, the sequence $0.9$, $0.99$, $0.999$, $0.9999$
	does not converge in the space $(0,1)$,
	although it does converge in $[0,1]$.

	The fact that these sequences fail to converge
	even though they ``ought to'' is weird and bad,
	and was why we defined complete spaces to begin with.

	\ii In general, it makes no sense to ask a question like ``is $[0,1]$ open?''.
	The questions ``is $[0,1]$ open in $\RR$?''
	and ``is $[0,1]$ open in $[0,1]$?'' do make sense, however.
	The answer to the first question is ``no''
	but the answer to the second question is ``yes'';
	indeed, every space is open in itself.
	Similarly, $[0, \half)$ is an open set % chktex 9
	in the space $M = [0,1]$
	because it is the ball \emph{in $M$}
	of radius $\half$ centered at $0$.

	\ii Dually, it doesn't make sense to ask ``is $[0,1]$ closed''?
	It is closed \emph{in $\RR$} and \emph{in itself}
	(but every space is closed in itself, anyways).
\end{itemize}

%Thus to list out all four cases:
%\begin{itemize}
%	\ii The statement ``$[0,1]$ is complete'' makes sense (and is true);
%	it says that $[0,1]$ is a complete metric space.
%	\ii The statement ``$[0,1]$ is a complete subset of $\RR$'' is valid;
%	it says that the subspace $[0,1]$ of $\RR$ is a complete metric space.
%	\ii The statement ``$[0,1]$ is a closed subset of $\RR$'' makes sense;
%	it says that the set of points $[0,1]$ form a closed subset of a parent space $\RR$.
%	\ii The statement ``$[0,1]$ is closed'' does \emph{not} make sense officially.
%	Closed sets are only defined relative to parent spaces.
%\end{itemize}

To make sure you understand the above,
here are two exercises to help you practice relative adjectives.

\begin{exercise}
	Let $M$ be a complete metric space and let $S \subseteq M$.
	Prove that $S$ is complete if and only if it is closed in $M$.
	% (This is obvious once you figure out what the question is asking.)
	In particular, $[0,1]$ is complete.
\end{exercise}
\begin{exercise}
	Let $M = [0,1] \cup (2,3)$.
	Show that $[0,1]$ and $(2,3)$ are both open and closed in $M$.
\end{exercise}

This illustrates a third point:
a nontrivial set can be both open and closed\footnote{Which
	always gets made fun of.}
As we'll see in \Cref{ch:top_more}, this implies the space is disconnected;
i.e.\ the only examples look quite like the one we've given above.

\section{\problemhead}
\begin{dproblem}[Banach fixed point theorem]
	Let $M = (M,d)$ be a complete metric space.
	Suppose $T \colon M \to M$ is a continuous map
	such that for any $p \neq q \in M$,
	\[ d\left( T(p), T(q) \right) < 0.999 d(p,q). \]
	(We call $T$ a \vocab{contraction}.)
	Show that $T$ has a unique fixed point.
\end{dproblem}

\begin{problem}
	[Henning Makholm,
	on \href{https://math.stackexchange.com/a/3051746/229197}{math.SE}]
	We let $M$ and $N$ denote the metric spaces obtained
	by equipping $\RR$ with the following two metrics:
	\begin{align*}
		d_M(x,y) &= \min \left\{ 1, \left\lvert x-y \right\rvert \right\} \\
		d_N(x,y) &= \left\lvert e^x - e^y \right\rvert.
	\end{align*}
	\begin{enumerate}[(a)]
		\ii Fill in the following $2 \times 3$ table
		with ``yes'' or ``no'' for each cell.
		\begin{center}
		\begin{tabular}[h]{l|ccc}
			& Complete? & Bounded? & Totally bounded? \\ \hline
			$M$ \\
			$N$ \\
		\end{tabular}
		\end{center}
		\ii Are $M$ and $N$ homeomorphic?
	\end{enumerate}
	\begin{hint}
		(a): $M$ is complete and bounded
		but not totally bounded. $N$ is all no.
		For (b) show that $M \cong \RR \cong N$.
	\end{hint}
	\begin{sol}
		Part (a) is essentially by definition.
		The space $M$ is bounded since no distances exceed $1$,
		but not totally bounded since we can't cover $M$
		with finitely many $\half$-neighborhoods.
		The space $M$ is complete since a sequence of real
		numbers converges in $M$ if it converges in the usual sense.
		As for $N$, the sequence $-1$, $-2$, \dots is Cauchy
		but fails to converge; and it is obviously not bounded.

		To show (b), the identity map (!) is an homeomorphism $M \cong \RR$
		and $\RR \cong N$, since it is continuous.

		This illustrates that $M \cong N$ despite the fact
		that $M$ is both complete and bounded
		but $N$ is neither complete nor bounded.
		On the other hand, we will later see that complete
		and totally bounded implies \emph{compact},
		which is a very strong property preserved under homeomorphism.
	\end{sol}
\end{problem}

\begin{dproblem}[Completion of a metric space]
	\yod
	\label{prob:completion}
	Let $M$ be a metric space.
	Construct a complete metric space $\ol M$
	such that $M$ is a subspace of $\ol M$,
	and every open set of $\ol M$ contains a point of $M$
	(meaning $M$ is \vocab{dense} in $\ol M$).
	\begin{hint}
		As a set, we let $\ol M$ be the sequence of Cauchy
		sequences $(x_n)$ in $M$, modulo the relation that
		$(x_n) \sim (y_n)$ if $\lim_n d(x_n, y_n) = 0$.
	\end{hint}
\end{dproblem}

\begin{problem}
	Show that a metric space is totally bounded
	if and only if any sequence has a Cauchy subsequence.
	\begin{sol}
		See \url{https://math.stackexchange.com/q/556150/229197}.
	\end{sol}
\end{problem}

\begin{problem}
	\kurumi
	Prove that $\QQ$ is not homeomorphic to any complete metric space.
	\begin{hint}
		The standard solution seems to be
		via the so-called ``Baire category theorem''.
	\end{hint}
\end{problem}


\section{Additional Exercises}\label{sec:MoreFunctionExercises}

\Opensolutionfile{solutions}[ex]
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{enumialphparenastyle}

%%%%%%%%%%
\begin{ex}
Find the derivatives of the following functions
from definition.
\begin{enumerate}
	\item	$f(x)=(2x+3)^2$
	\item	$g(x)=x^{3/2}$
\end{enumerate}
\begin{sol}
\begin{enumerate}
	\item	$4(2x+3)$
	\item	$\frac{3}{2}x^{1/2}$
\end{enumerate}
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Let $f(x)=\left\{
\begin{array}{cc}
x^{3} & \text{if }x\leq 1 \\
5x-x^{2} & \text{if }x>1%
\end{array}%
\right. .$
Use the definition of the derivative to find $f^{\prime}(1)$.
\begin{sol}
	3
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Differentiate the following functions.
\begin{enumerate}
	\item	$y=7x^4-7\pi^4+\dfrac{1}{\pi\sqrt[3]{x}}$
	\item	$f(x)=\dfrac{1-\sqrt{x}}{1+\sqrt{x}}$
	\item	$f(x)=\vert x-1\vert+\vert x+2\vert$
	\item	$f(x)=x^2\sin x\cos x$
	\item	$y=\dfrac{x\sin x}{1+\sin x}$
	\item	$g(x)=\sqrt{2+\dfrac{3}{\sqrt{x}}}$
	\item	$y=\sqrt[3]{x^4+x^2+1}+\dfrac{1}{(x^3-x+4)^5}$
	\item	$y=\sin^3 x-\sin(x^3)$
	\item	$F(x)=\sec^4 x+\tan^4 x$
	\item	$y=\cos^2\left(\dfrac{1-x}{1+x}\right)$
	\item	$y=\tan(\sin(x^2+\sec^2 x))$
	\item	$y=\dfrac{1}{2+\sin\frac{\pi}{x}}$
\end{enumerate}
\begin{sol}
\begin{enumerate}
	\item	$28x^3-\dfrac{1}{3\pi x^{4/3}}$
	\item	$-\dfrac{1}{\sqrt{x}(1+\sqrt{x})^2}$
	\item	$f^{\prime}(x)=\left\{
	\begin{array}{ll}
	-2 & \text{if }x<-2 \\
	0 & \text{if }-2<x<1 \\
	2 & \text{if }x>1%
	\end{array}%
	\right. $
	\item	$2x\sin x\cos x+x^2\cos^2 x-x^2\sin^2 x$
	\item	$\dfrac{(\sin x+x\cos x)(1+\sin x)-x\sin x\cos x}{(1+\sin x)^2}$
	\item	$-\dfrac{3}{4x^{3/2}}\left(2+\dfrac{3}{\sqrt{x}}\right)^{-1/2}$
	\item	$\frac{1}{3}(x^4+x^2+1)^{-2/3}(4x^3+2x)-\dfrac{5(3x^2-1)}{(x^3-x+4)^6}$
	\item	$3\sin^2 x\cos x-3x^2\cos(x^3)$
	\item	$4\sec^4 x\tan x + 4\tan^3 x\sec^2 x$
	\item	$\dfrac{4}{(1+x)^2}\cos\left(\dfrac{1-x}{1+x}\right)\sin\left(\dfrac{1-x}{1+x}\right)$
	\item	$(2x+2\sec^2 x\tan x)\sec^2(\sin(x^2+\sec^2 x))\cos(x^2+\sec^2 x)$
	\item	$-\dfrac{\pi\cos\frac{\pi}{x}}{x^2(2+\sin\frac{\pi}{x})^2}$
\end{enumerate}
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Differentiate the following functions.
\begin{enumerate}
	\item	$y=e^{3x}+e^{-x}+e^2$
	\item	$y=e^{2x}\cos 3x$
	\item	$f(x)=\tan(x+e^x)$
	\item	$g(x)=\dfrac{e^x}{e^x+2}$
	\item	$y=\ln(2+\sin x)-\sin(2+\ln x)$
	\item	$f(x)=e^{x^{\pi}}+x^{\pi^{e}}+\pi^{e^x}$
	\item	$y=\log _{a}(b^x)+b^{\log_a x}$, where $a$ and $b$ are positive real numbers and $a\neq 1$
	\item	$y=(x^2+1)^{x^3+1}$
	\item	$y=(x^2+e^x)^{1/\ln x}$
	\item	$y=\dfrac{x\sqrt{x^2+x+1}}{(2+\sin x)^4(3x+5)^7}$
\end{enumerate}
\begin{sol}
\begin{enumerate}
	\item	$3e^{3x}-e^{-x}$
	\item	$2e^{2x}\cos 3x-3e^{2x}\sin 3x$
	\item	$(1+e^x)\sec^2(x+e^x)$
	\item	$2e^x/(e^x+2)^2$
	\item	$\dfrac{\cos x}{2+\sin x}-\dfrac{\cos(2+\ln	x)}{x}$
	\item	$e^{x^{\pi}}\cdot\pi x^{\pi-1}+\pi^e x^{\pi^e-1}+\pi^{e^x}\ln\pi\cdot e^x$
	\item	$\log_{a}b+(\log_{a}b) x^{(\log_{a}b)-1}$
	\item	$(x^2+1)^{x^3+1}\left(3x^2\ln(x^2+1)+\dfrac{2x(x^3+1)}{x^2+1}\right)$
	\item	$\dfrac{(x^2+e^x)^{1/\ln x}}{(\ln x)^2}\left(\dfrac{2x+e^x}{x^2+e^x}\ln	x-\dfrac{x^2+e^x}{x}\right)$
	\item	$\dfrac{x\sqrt{x^2+x+1}}{(2+\sin x)^4 (3x+5)^7} \left(\dfrac{1}{x}+\dfrac{2x+1}{2(x^2+x+1)}-\dfrac{4\cos x}{2+\sin x}-\dfrac{21}{3x+5}\right)$
\end{enumerate}
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Find $\dfrac{dy}{dx}$ if $y$ is a differentiable
function that satisfy the given equation.
\begin{enumerate}
	\item	$x^2+xy+y^2=7$
	\item	$x^2+y^2=(2x^2+2y^2-x)^2$
	\item	$x^2\sin y+y^3=\cos x$
	\item	$x^2+xe^y=2y+e^x$
\end{enumerate}
\begin{sol}
\begin{enumerate}
	\item	$-(2x+y)/(x+2y)$
	\item	$\dfrac{x-(2x^2+2y^2-x) (4x-1)}{4y(2x^2+2y^2-x)-y}$
	\item	$-\dfrac{\sin x+2x\sin y}{x^2\cos y+3y^2}$
	\item	$\dfrac{2x+e^y-e^x}{2-xe^y}$
\end{enumerate}
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Differentiate the following functions.
\begin{enumerate}
	\item	$y=x\sin^{-1}x$
	\item	$f(x)=\dfrac{\sin^{-1}x}{\cos^{-1}x}$
	\item	$g(x)=\tan^{-1}\left(\dfrac{x}{a}\right)$, where $a>0$
	\item	$y=x\tan^{-1}x-\dfrac{1}{2}\ln(x^2+1)$
\end{enumerate}
\begin{sol}
\begin{enumerate}
	\item	$\sin^{-1}x+x/\sqrt{1-x^2}$
	\item	$\dfrac{\cos^{-1}x+\sin^{-1}x}{(\cos^{-1}x)^2 \sqrt{1-x^2}}$
	\item	$a/(x^2+a^2)$
	\item	$\tan^{-1}x$
\end{enumerate}
\end{sol}
\end{ex}

\end{enumialphparenastyle}


\section*{Method}

Our design imposes additional structure on top of existing machine learning
approaches, enforcing properties on the reconstructed values. For functions \[ f(p, t)\to\mathbb{R}^n, t\in[0,1] \] which is modelled through
a learned approach, we decompose it into a function: \[ f(p)\to\mathbb{R}^{n\times O}, B_O(f(p), t)\to\mathbb{R}^n
\] Where
$O$ is the order of the Bezier spline, $f(p)$ is the learned control points, and $B_O(f(p), \cdot)$ is the evaluation
of the $O$th order Bezier spline with control points defined by $f(p)$.

\subsection*{Architecture}

Specifically for dynamic NeRF, $f(p)$ is defined as
$\text{MLP}(x,y,z)\to\mathbb{R}^{3\times O}$. We ray march from a camera with known position and
view direction through the scene, and at every point we compute the set of control points for a Bezier curve. We then evaluate the Bezier curve at a given time, and deform the ray by the result, producing some $\Delta_0(x)$. The number of spline points for the Bezier curve is hyperparameter, but we are able
to accurately reconstruct movement with as low as 4 spline points, but experiment with 16 spline points. In order to evaluate the Bezier curve in a numerically stable way, we use De Casteljau's
algorithm. In a future extension, we plan on extending it to handle rational Bezier splines, which would allow for even greater control of movement.

De Casteljau's algorithm evaluated at time $t$ is characterized by the recurrence relation:
\[ \beta_i^{(0)} = \beta_i, \]
\[ \beta_i^{j} = (1-t)\beta_i^{(j-1)} + t\beta_{i-1}^{(j-1)}, \]
which can be thought of as linearly interpolating between adjacent control points until there is only a single point remaining. This takes $O(n^2)$ operations to evaluate, where $n$ is the number of control points.

We are also interested in constructing a reasonable canonical NeRF, and arbitrarily select $t = 0$ to be canonical. Thus, we are interested in Bezier Curves where $B_O(0) = \overrightarrow{0}$. This can be achieved in two different ways, either by assigning $p_0 = \overrightarrow{0}$, and only computing the other control points: $f(p) = p_{1\cdots O-1}$. Then, we can use the Bezier spline with the control points as the concatenation of $\overrightarrow{0}$ with the other control points: $[\overrightarrow{0}, p_{1\cdots O-1}]$. Alternatively, we can compute $p_{0\cdots O-1}$ and use the Bezier spline with control points but subtract the first predicted point from all of them: $[p_{0\cdots O-1}]-p_0$, and the final change in position is $\Delta_0(x) = B_O(f(p)-f(p)_0,t)+p_0$. While both formulations are in theory equivalent and explicitly concatenating $\overrightarrow{0}$ leads to fewer degrees of freedom, we find the second approach leads to better convergence near $t=0$, so we use that.

Following NR-NeRF, we also learn how rigid each ray is, which in theory should allow more efficient learning of which regions are space are fixed. This rigidity $\in[\epsilon, 1]$)is computed as a function of position:
\[ \text{Rigidity} =\sigma_{\epsilon^\uparrow}(\text{MLP}(x)), \]
\[ \sigma_{\epsilon^\uparrow}(v) = \sigma(v)(1-\epsilon) + \epsilon, \]
where $\sigma$ is defined as the sigmoid function $\frac{1}{1+e^{-x}}$. Rigidity rescales the
difficulty of learning movement, making it more easy to represent and classify static scene
objects. The final change in position is defined as $\Delta(x) = \text{Rigidity}\times
\Delta_0(x)$.

In order to reconstruct RGB values, we also diverge from the original NeRF, and only allow for
positional dependent colors:
\[ \text{RGB} = \sigma_{\epsilon-\text{wide}}(\text{MLP}(x)) \]
\[ \sigma_{\epsilon-\text{wide}}(v) = \sigma(v)(1+2\epsilon) - \epsilon \]
using the activation function as described in MIP-NeRF~\cite{barron2021mipnerf} in order to prevent vanishing gradients on colors. Because of the low number of
samples for a moving object at a given view, it is more difficult to learn specular
reflection, thus it leads to better convergence to model the diffuse color of an object. This is in line with NR-NeRF and D-NeRF, which also model the diffuse component.

\iffalse
\subsection*{Loss}

While training, we introduce another loss term, as the $\ell_2$ loss may have
difficulty when there is a large pixel-wise gap in movement between the learned and predicted
component. $\ell_2$ may also miss large structural differences between two images if they are close in color, so we add a loss function which is high for incorrect structure of an image.

The new loss term is \[
    L_\text{FFT} = \text{MSE}(\text{FFT}(I_\text{GT}), \text{FFT}(I_\text{predicted}))
\],
where the FFT is the 2D fast Fourier transform of an image.
MSE is the mean squared error in the complex domain as the FFT has both a real and imaginary component, and is defined as \[ \text{MSE}(a\in\mathbb{C}^k,b\in\mathbb{C}^k) \]
\[ = \frac{1}{k}\Sigma_{i=0}^k|a_i-b_i| \]
\[ |z\in\mathbb{C}| = (\Re(z)+\Im(z))(\Re(z)-\Im(z)) \]
We introduce this term as a cheap replacement to structural similarity (SSIM), since the
FFT captures information about the structure of the whole image, which is useful in a dynamic setting due to the disjointedness of where a predicted object may be as opposed to its final position\footnote{In a published iteration, we would likely perform an ablation of this loss function.}.

Our final loss formulation is:
\[ \ell_2(I_\text{predicted}, I_\text{GT}) + L_\text{FFT}(I_\text{predicted}, I_\text{GT}) \]
without any additional regularization terms.

Our formulation is simpler compared to NR-NeRF since there is no need to regularize
temporal consistency, making the optimization process simpler. Our training process also does not require adding in additional frames over time for short sequences, so we randomly sample from all available frames. This is contrast to D-NeRF, which requires adding frames in during training since it has to enforce that at $t=0$ the predicted movement is 0.

\subsection*{Training}

For training, we sample random crops of random frames, computing the loss defined previously and back-propagating through the whole network. We use autograd to optimize control points and the canonical NeRF jointly, but note that there are also classical approaches to solving spline points which may lead to faster optimization in the future. We use the Adam optimizer~\cite{Kingma2015AdamAM} with cosine simulated annealing~\cite{loshchilov2017sgdr} to go from $\num{2e-4}$ to $\num{5e-5}$ over the course of a day.

For some scenes, we are able to have higher learning rates, but for much darker scenes it's necessary to lower the learning rate to converge.

\subsection*{Long Duration $C^0$ Continuity}

While the above structure is sufficient to model short-term $C^0$ continuity, it runs into
numerical instability and higher cost the longer the sequence is, due to the $O(n^2)$
evaluation cost of De Casteljau's algorithm. To deal with this, we design an additional architecture which
composes the previous architecture for many small $C^0$ curves. Concatenating many small splines permits for reconstruction of infinitely long sequences with guaranteed $C^0$-continuity with only a few additional models.

The architecture is based off of poly-splines, or a composition of many small Bezier splines.
For a known-length sequence, we divide it into $K$ segments, and assume that $K$ evenly divides the total number of frames. We can then treat $t$ as is
in the range $[0, K]$, and $t$ can be decomposed into a segment number and fractional component:
\[ k\in\mathbb{Z}_K=\lfloor t\rfloor, t'\in[0,1)=t-k \]
Then we define an embedding $\text{Emb} = \mathbb{R}^{Z\times(K+1)}$, where $Z$ is a latent
dimension size. We then create an "anchor" network:
\[
    \textit{anchor}(x\in\mathbb{R}^3,z\in\mathbb{R}^Z) = \text{MLP}(x,z)\to(\mathbb{R}^3,\mathbb{R}^{Z'})
\]
We call it an "anchor" network because it computes
endpoints which anchors each of the curves, and also computes a representative latent vector. Between these two
endpoints, we are interested in computing $O-2$ intermediate spline points. We define the
control point estimation network as
\[
\textit{control}(x\in\mathbb{R}^3, z_1, z_2\in\mathbb{R}^{Z'}) = \text{MLP}(x,z_1,z_2)\to\mathbb{R}^{O\times\mathbb{R}^3}
\]
Where $z_1,z_2$ are the
representative latent vectors from the anchor network, and it outputs $O-2$ spline points in
$R^3$. We can also include a per-segment rigidity value, similar to above, but lose
some guarantees of continuity by doing so.

In order to evaluate the network at a given time $t$ and other inputs $x$, we first compute $k, t'$. Then, we compute
the anchor points at
\[ p_0,z_1=\textit{anchor}(x,\text{Emb}[k]) \]
\[ p_\text{end},z_2=\textit{anchor}(x,\text{Emb}[k+1]) \]
and the intermediate control points as \[ p_{1\cdots O-2} = \text{control}(x,z_1,z_2) \]
The final set of control points we use
is the concatenation of the anchors with the intermediate points:
$[p_0, p_{1\cdots O-2}, p_\text{end}]$. Using this set of control points, we evaluate the Bezier
curve spline at $t'$ using De Casteljau's algorithm:
\[ \text{De Casteljau}([p_0, p_{1\cdots O-2}, p_\text{end}], t) = \Delta(x) \]
Because we are evaluating adjacent embeddings ($k, k+1$) for each anchor, we are guaranteed that the endpoints between each spline segment are identical. Information is also carried over between segments through the anchor's
representative latent vector, allowing the network to maintain velocity between curves if necessary.

This architecture enforces a guarantee of consistency, regardless of how long a reconstructed signal is. It also naturally allows movement at one time to be distinct from another,
as while the endpoints of the spline are fixed, movement defined by the intermediate control points is allowed to be fully independent of other
segments. While we construct this architecture for long dynamic sequence reconstruction, we argue that it works more generally for generating $C^0$-continuous functions in any domain.

One obvious question is why a more complex architecture is necessary as compared to predicting all spline points and using only some of them. In the case of extremely long sequences, the number of spline points may increase linearly with the length of the function, and for dynamic NeRF we would have memory cost of $O(H\times W\times D\times t)$ which is prohibitively expensive. With the proposed architecture, we are able to sample sparsely, maintaining a constant memory usage with respect to time.

We do yet not demonstrate the effectiveness of the architecture, due to time constraints, and the author's incompetence\footnote{If this were to be published, I would demonstrate this architecture, but cannot due to time constraints.}.
\fi


\documentclass{memoir}
\usepackage{linalg}
\usepackage{marginnote}
\begin{document}
% \section{}
\section{First Consequences of Axioms of a Vector Space}
Let $(V,F,+,\cdot)$ be a vector space.
\begin{prop}
The additive identity of $V$ is unique
\end{prop}
\begin{proof}
	Suppose $V$ has two additive identities $\overline{0}, \overline{0'}$. Then $\overline{0} = \overline{0}+ \overline{0'} = \overline{0'}$.
\end{proof}

\begin{prop}
	Additive inverses are unique
\end{prop}

\begin{proof}
	Let $v\in V$ be any vector. Suppose that $w, w'$ are additive inverses of v.
	Then
	\begin{align*}
		w &= w + \overline{0}\\
		  &= w + (v+w')\\
		  &= (w+v) + w'\\
		  &= \overline{0} + w'\\
		  &= w'
	\end{align*}
\end{proof}

\begin{prop}
	For all $v\in V, 0\cdot v = \overline{0}$
\end{prop}

\begin{proof}
	\begin{align*}
		0 \cdot v &= (0+0) \cdot v\\
			  &= 0\cdot v + 0\cdot v
	\end{align*}
	Add $-0\cdot v$ to both sides:
\begin{align*}
	-(0\cdot v) + (0\cdot v) &= -(0\cdot v) + (0\cdot v) + (0\cdot v)\\
				 &\implies \overline{0} = \overline{0} + (0\cdot v)\\
				 &\implies \overline{0} = 0\cdot v
\end{align*}
\end{proof}

\begin{hw}
	Prove that $-v = (-1)*v$.
\end{hw}
\end{document}


\RequirePackage[l2tabu, orthodox]{nag} % Complaints if syntax isn't right.
\documentclass[10pt]{extarticle}

\usepackage[utf8]{inputenc}
\usepackage{extsizes} % Allow for more sizes such as 14pt or 17pt on document class.
\usepackage{mathtools} % Allows conditional math expressions, etc.
\usepackage{amsfonts} % Allows \mathbb{R}, which is the real numbers symbol.
\usepackage[export]{adjustbox} % Wraps \includegraphics with more keys (options).

% For writing pseudocode snippets: http://ctan.mackichan.com/macros/latex/contrib/algorithmicx/algorithmicx.pdf
\usepackage{algorithm}
\usepackage{algpseudocode}

\usepackage[top=3.5cm, bottom=3.5cm, left=3.5cm, right=3.5cm]{geometry}

\usepackage{microtype} % Makes document more readable by optimizing space between letters.

\usepackage{units} % Adds \nicefrac{1}{2}: (½), \unit[5]{m}{s}: 5 m/s.

\usepackage[colorlinks=false, pdfborder={0 0 0}]{hyperref} % Allow clicking references and the table of contents in pdfs.
\usepackage{cleveref} % Adds fig/formula etc. before references (use \cref).

%%% New commands.

% Don't output references in case they're empty - http://tex.stackexchange.com/questions/74476/how-to-avoid-empty-thebibliography-environment-bibtex-if-there-are-no-refere

\let\myBib\thebibliography
\let\endmyBib\endthebibliography

\renewcommand\thebibliography[1]{\ifx\relax#1\relax\else\myBib{#1}\fi}

% Norm -> ||something||
\newcommand{\norm}[1]{\left\lVert#1\right\rVert}

% Stdfig -> Used as \stdfig{width}{label_name}{caption}
% Requires: image called 'caption' in img folder.
% Output: A figure with the given width, labeled as 'fig:label_name'

\newcommand{\stdfig}[3]{
    \begin{figure}
    \centering
    \includegraphics[width = #1]{img/#2.eps}
    \caption{#3}
    \label{fig:#2}
    \end{figure}
}

\begin{document}

%% Front page.
\title{09.02 - Recommender Systems}

    \date{}

    \maketitle

\newpage
%% Abstract page.

%% Table of contents page.

%% Body start.
\section{Problem Formulation}\label{problem-formulation}

Example: Predicting movie ratings.

Imagine a company where users rate movies using zero to five stars.

Notation:

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  $n_u$ = number of users
\item
  $n_m$ = number of movies
\item
  $r(i, j)$ = 1 if user $j$ has rated movie $i$
\item
  $y^{(i,j)}$ = rating given by user $j$ to movie $i$ (defined only if
  $r(i, j)$ = 1)
\end{itemize}

The problem is: given this dataset, to look through all the missing
ratings on the dataset and fill them (\cref{fig:movie_ratings_table}).

\stdfig{9cm}{movie_ratings_table}{Movie ratings table}

\section{Content Based
Recommendations}\label{content-based-recommendations}

How do we predict the ``?'' in the movies table
(\cref{fig:movie_ratings_table})?

Let's guess that we have two additional features in the table: one that
measures the amount of romance a movie has and another one that measures
the amount of action (\cref{fig:movie_ratings_table_with_features}).

\stdfig{11cm}{movie_ratings_table_with_features}{Movie ratings table with features}

If we then add the interceptor term $x_0 = 1$, we have a feature vector
for each movie ($x^{(1)} = [1, \, 0.9, \,  0]^T$, for example). \bigskip

The idea is to use linear regression for each different user and predict
the missing parameters with that. More formally: for each user $j$,
learn a parameter $\theta^{(j)} \in \mathbb{R}^3$. Predict user $j$ as
rating movie $i$ with $(\theta^{(j)})^Tx^{(i)}$ stars. \smallskip

For example, in \cref{fig:movie_ratings_table_with_features}, if we
wanted to predict ``Cute pupies of love'' for ``Alice'', and we already
got $\theta$, we would do: $(\theta^{(1)})^Tx^{(3)}$. \smallskip

\subsection{Optimization Objective}\label{optimization-objective}

We can describe this intention (learn $\theta$ for the user $j$) with
the formula:

\[ \min_{\theta^{(j)}}  \frac{1}{2} \sum_{i:r(i, j)=1} ((\theta^{(j)})^Tx^{(i)} - y^{(i,j)})^2 + \frac{\lambda}{2} \sum_{k=1}^n(\theta_k^{(j)})^2 \]

Where $\sum_{i:r(i, j)=1} $ is a for loop with all the rated movies by
that user. \smallskip

To learn all the $\theta$, we can do:

\[ \min_{\theta^{(1)}, \dots, \theta^{(n_u)}}  \frac{1}{2} \sum_{j=1}^{n_u} \sum_{i:r(i, j)=1} ((\theta^{(j)})^Tx^{(i)} - y^{(i,j)})^2 + \frac{\lambda}{2} \sum_{j=1}^{n_u} \sum_{k=1}^n(\theta_k^{(j)})^2 \]

Which is very, very similar to the standard linear regression equation
(only filtering the unknows votes out).

\subsubsection{Gradient descent update}\label{gradient-descent-update}

\[ \theta_k^{(j)} := \theta_k^{(j)} - \alpha \sum_{i:r(i, j)=1} \left( (\theta^{(j)})^Tx^{(i)} - y^{(i,j)}\right) x_k^{(i)} \,\, \text{(for $k$ = 0)} \]

\[ \theta_k^{(j)} := \theta_k^{(j)} - \alpha \left( \sum_{i:r(i, j)=1} \left( (\theta^{(j)})^Tx^{(i)} - y^{(i,j)}\right) x_k^{(i)} + \lambda\theta_k^{(j)} \right) \,\, \text{(for $k \neq$ 0)} \]

\section{Collaborative Filtering}\label{collaborative-filtering}

This algorithm has the characteristic of learning the features it needs
by himself. \smallskip

In the last example (\cref{fig:movie_ratings_table_with_features}) we
had some features. Extracting those features is quite time consuming,
though. \smallskip

In this new algorithm, we won't have the features
(\cref{fig:movie_ratings_table_unknown_features}). Let's imagine that
the users tells us whether they like romance and action movies (the
$\theta$). Then, it's possible to infer the values of $x_1$ and $x_2$.
\smallskip

\stdfig{12cm}{movie_ratings_table_unknown_features}{Move ratings with unknown features}

We can exploit that we know the ratings of the users and their
preferences to know which value to assign to each movie feature. For
example, if a user says that he likes action movies and he rates a movie
very high, we can assume that that movie will have action.

\subsection{Optimization Algorithm}\label{optimization-algorithm}

Given $\theta^{(1)}, \dots, \theta^{(n_u)}$, to learn $x^{(i)}$:

\[ \min_{x^{(i)}}  \frac{1}{2} \sum_{i:r(i, j)=1} ((\theta^{(j)})^Tx^{(i)} - y^{(i,j)})^2 + \frac{\lambda}{2} \sum_{k=1}^n(x_k^{(i)})^2 \]

Which reads as: choosing the features so that the predicted value will
be similar to the actual value that we observe on the rating of the
users. \smallskip

We can then compute this for all the movies. Given
$\theta^{(1)}, \dots, \theta^{(n_u)}$, to learn
$x^{(1)}, \dots, x^{(n_m)}$:

\[ \min_{x^{(1)}, \dots, x^{(n_m)}}  \frac{1}{2} \sum_{i=1}^{n_m} \sum_{j:r(i, j)=1} ((\theta^{(j)})^Tx^{(i)} - y^{(i,j)})^2 + \frac{\lambda}{2} \sum_{i=1}^{n_m} \sum_{k=1}^n(x_k^{(i)})^2 \]

\subsubsection{Gradient descend update}\label{gradient-descend-update}

\[ x_k^{(i)} := x_k^{(i)} - \alpha \sum_{j:r(i, j)=1} \left( (\theta^{(j)})^Tx^{(i)} - y^{(i,j)}\right) \theta_k^{(j)} \,\, \text{(for $k$ = 0)} \]

\[ x_k^{(i)} := x_k^{(i)} - \alpha \left( \sum_{j:r(i, j)=1} \left( (\theta^{(j)})^Tx^{(i)} - y^{(i,j)}\right) \theta_k^{(j)} + \lambda x_k^{(i)} \right) \,\, \text{(for $k \neq$ 0)} \]

\subsection{How to continue}\label{how-to-continue}

We've seen in content based filtering that given
$x^{(1)}, \dots, x^{(n_m)}$ we can estimate
$\theta^{(1)}, \dots, \theta^{(n_u)}$. \smallskip

In collaborative filtering, given $\theta^{(1)}, \dots, \theta^{(n_u)}$,
we can estimate $x^{(1)}, \dots, x^{(n_m)}$. \bigskip

So, this is the egg-chicken problem. What we can do, therefore, is
randomly guess $\theta$ to learn $x$ (features). We can then use the $x$
to learn $\theta$, etc.

\section{Collaborative Filtering
Algorithm}\label{collaborative-filtering-algorithm}

We've talked about the ideas of how if you're given features for movies,
we can learn parameters $\theta$ for users. We've also seen how if we're
given parameters for the users we can use that to extract features for
the movies. We'll now find an algorithm to solve that problem. \bigskip

We could to the chicken-egg back and forth optimization algorithm, but
there is an algorithm that optimizes both $x$ and $\theta$ at the same
time, so it's better. \smallskip

The cost function is:

\[ J(x^{(1)}, \dots, x^{(n_m)}, \theta^{(1)}, \dots, \theta^{(n_u)}) = \frac{1}{2} \sum_{(i,j):r(i, j)=1} \left( (\theta^{(j)})^Tx^{(i)} - y^{(i,j)}\right)^2 + \frac{\lambda}{2} \sum_{i=1}^{n_m} \sum_{k=1}^n(x_k^{(i)})^2  + \frac{\lambda}{2} \sum_{j=1}^{n_u} \sum_{k=1}^n(\theta_k^{(j)})^2\]

Which is basically merging both cost functions (collaborative and
content based filtering) into one, taking advantage that the less
squares sumation is the same in both and then adding the respective
regularization term.

We also suppress the interceptor term $x_1 = 1$, as the algorithm will
learn it by himself if it's really needed.

\subsection{Gradient descent}\label{gradient-descent}

The same as before for the $\theta$ and $x$ respectively.

\subsection{Detailed algorithm}\label{detailed-algorithm}

We can find a more detailed explanation in
\cref{alg:collaborative_filtering}.

\begin{algorithm}
\caption{Collaborative filtering algorithm} \label{alg:collaborative_filtering}
\begin{algorithmic}[1]
\State Initialize $x^{(1)}, \dots, x^{(n_m)}, \theta^{(1)}, \dots, \theta^{(n_u)}$ to small random values.
\State Minimize $J(x^{(1)}, \dots, x^{(n_m)}, \theta^{(1)}, \dots, \theta^{(n_u)})$ using gradient descent (or an advanced optimization algorithm).

For every $j = 1, \dots, n_u, i = 1, \dots, n_m$:

$ x_k^{(i)} := x_k^{(i)} - \alpha \left( \sum_{j:r(i, j)=1} \left( (\theta^{(j)})^Tx^{(i)} - y^{(i,j)}\right) \theta_k^{(j)} + \lambda x_k^{(i)} \right) $

$ \theta_k^{(j)} := \theta_k^{(j)} - \alpha \left( \sum_{i:r(i, j)=1} \left( (\theta^{(j)})^Tx^{(i)} - y^{(i,j)}\right) x_k^{(i)} + \lambda\theta_k^{(j)} \right) $

\State For a user with parameters $\theta$ and a movie with (learned) features $x$, predict a star rating of $\theta^Tx$.

\end{algorithmic}
\end{algorithm}

\section{Vectorization: Low Rank Matrix
Factorization}\label{vectorization-low-rank-matrix-factorization}

We'll see an alternative algorithm for solving the collaborative
filtering problem. \bigskip

Take the dataset we have (\cref{fig:movie_ratings_table}), and put it
directly onto a matrix called $Y$. Then, we can stack the $x$ vectors
and the $\theta$ vectors in two matrices, $X$ and $\Theta$. Afterwards,
we can compute the prediction of the ratings simply by doing
$X\Theta^T$. All this can be seen in
\cref{fig:low_rank_matrix_factorization}

\stdfig{10cm}{low_rank_matrix_factorization}{Low rank matrix factorization}

\subsection{Finding related movies}\label{finding-related-movies}

For each product $i$, we learn a feature vector
$x^{(i)} \in \mathbb{R}^n$.

How to find 5 movies $j$ most related to movie $i$?

Find the 5 movies $j$ with the smallest $\norm{x^{(i)} - x^{(j)}}$.

%% Body end.

%% Bibliography.

\clearpage

    \nocite{*}

\bibliographystyle{plain}
\bibliography{references}

\end{document}

theory/09.02 - Recommender Systems/09.02 - Recommender Systems.tex


\section{Appendix}

\newcommand{\frule}[3]{
  \begin{prooftree}
  \hypo{#1}
  \infer1[(#3)]{#2}
  \end{prooftree}
}

\begin{figure}[H]
  \begin{framed}
  $$\arraycolsep=0.75cm\def\arraystretch{2.75}\begin{array}{c c}
  % Hypothesis
  \frule{}{{?a} \vdash {?a}}{H.} & \\
  % And
  \frule{{?a}, {?b} \vdash}{{?a} \land {?b} \vdash}{I.L.$\land$} & \frule{\vdash {?a} \quad \vdash {?b}}{\vdash {?a} \land {?b}}{I.R.$\land$} \\
  \frule{{?a} \land {?b} \vdash}{{?a}, {?b} \vdash}{E.L.$\land$} & \frule{\vdash {?a} \land {?b}}{\vdash {?a}}{E.R.$\land_1$} \\ % cont.
  & \frule{\vdash {?a} \land {?b}}{\vdash {?b}}{E.R.$\land_2$} \\
  % Or
  \frule{{?a} \vdash \quad {?b} \vdash}{{?a} \lor {?b} \vdash}{I.L.$\lor$} & \frule{\vdash {?a}, {?b}}{\vdash {?a} \lor {?b}}{I.R.$\lor$} \\
  \frule{{?a} \lor {?b} \vdash}{{?a} \vdash}{E.L.$\lor_1$} & \frule{\vdash {?a} \lor {?b}}{\vdash {?a}, {?b}}{E.R.$\lor$} \\ % cont.
  \frule{{?a} \lor {?b} \vdash}{{?b} \vdash}{E.L.$\lor_2$} & \\
  % Implies
  \frule{\vdash {?a} \quad {?b} \vdash}{{?a} \Rightarrow {?b} \vdash}{I.L.$\Rightarrow$} & \frule{{?a} \vdash {?b}}{\vdash {?a} \Rightarrow {?b}}{I.R.$\Rightarrow$} \\
  \frule{{?a} \Rightarrow {?b} \vdash}{\vdash {?a}}{E.L.$\Rightarrow_1$} & \frule{\vdash {?a} \Rightarrow {?b}}{{?a} \vdash {?b}}{E.R.$\Rightarrow$} \\ % cont.
  \frule{{?a} \Rightarrow {?b} \vdash}{{?b} \vdash}{E.L.$\Rightarrow_2$} & \\
  % Iff
  \frule{{?a} \Rightarrow {?b}, {?b} \Rightarrow {?a} \vdash}{{?a} \Leftrightarrow {?b} \vdash}{I.L.$\Leftrightarrow$} & \frule{\vdash {?a} \Rightarrow {?b} \quad \vdash {?b} \Rightarrow {?a}}{\vdash {?a} \Leftrightarrow {?b}}{I.R.$\Leftrightarrow$} \\
  \frule{{?a} \Leftrightarrow {?b} \vdash}{{?a} \Rightarrow {?b}, {?b} \Rightarrow {?a} \vdash}{E.L.$\Leftrightarrow$} & \frule{\vdash {?a} \Leftrightarrow {?b}}{\vdash {?a} \Rightarrow {?b}}{E.R.$\Leftrightarrow_1$} \\ % cont.
  & \frule{\vdash {?a} \Leftrightarrow {?b}}{\vdash {?b} \Rightarrow {?a}}{E.R.$\Leftrightarrow_2$}
  \end{array}$$
  \end{framed}
  \caption[Rules (1)]{Predefined rules provided in the front. For clarity, contexts have been omitted: the intended semantic is the one described in \autoref{sec:proof-framework-rules}. ``H'', ``I'', ``E'', ``L'' and ``R'' respectively stand for ``Hypothesis'', ``Introduction'', ``Elimination'', ``Left'' and ``Right''. In the source, shorthand for these names are used, for instance ``I.L.$\land$'' would be identified by \code{introLAnd}. Most introduction rules map to a single kernel proof step. Elimination rules on the other hand rely on the cut rule.}
  \label{fig:rules-list-1}
\end{figure}

\begin{figure}[H]
\begin{framed}
  $$\arraycolsep=0.75cm\def\arraystretch{3}\begin{array}{c c}
  % Not
  \frule{\vdash {?a}}{\neg {?a} \vdash}{I.L.$\neg$} & \frule{{?a} \vdash}{\vdash \neg {?a}}{I.R.$\neg$} \\
  \frule{\neg {?a} \vdash}{\vdash {?a}}{E.L.$\neg$} & \frule{\vdash \neg {?a}}{{?a} \vdash}{E.R.$\neg$} \\
  % Eq. refl.
  \frule{{?s} = {?s} \vdash}{\vdash}{E.L.$=$} & \frule{}{\vdash {?s} = {?s}}{I.R.$=$} \\
  % Forall
  \frule{{?p}({?t}) \vdash}{\forall x. {?p}(x) \vdash}{I.L.$\forall$} & \frule{\vdash {?p}(\overline{?t})}{\vdash \forall x. {?p}(x)}{I.R.$\forall$} \\
  \frule{\forall x. {?p}(x) \vdash}{{?p}({?t}) \vdash}{E.L.$\forall$} & \frule{\vdash \forall x. {?p}(x)}{\vdash {?p}(\overline{?t})}{E.R.$\forall$} \\
  % Exists
  \frule{{?p}(\overline{?t}) \vdash}{\exists x. {?p}(x) \vdash}{I.L.$\exists$} & \frule{\vdash {?p}({?t})}{\vdash \exists x. {?p}(x)}{I.R.$\exists$} \\
  \frule{\exists x. {?p}(x) \vdash}{{?p}(\overline{?t}) \vdash}{E.L.$\exists$} & \frule{\vdash \exists x. {?p}(x)}{\vdash {?p}({?t})}{E.R.$\exists$} \\
  % Iff subst.
  \frule{{?f}({?a}) \vdash}{{?a} \Leftrightarrow {?b}, {?f}({?b}) \vdash}{I.L.S.$\Leftrightarrow$} & \frule{\vdash {?f}({?a}) \quad}{{?a} \Leftrightarrow {?b} \vdash {?f}({?b})}{I.R.S.$\Leftrightarrow$} \\
  \frule{{?f}({?a}) \vdash \quad \vdash {?a} \Leftrightarrow {?b}}{{?f}({?b}) \vdash}{E.L.S.$\Leftrightarrow$} & \frule{\vdash {?f}({?a}) \quad \vdash {?a} \Leftrightarrow {?b}}{\vdash {?f}({?b})}{E.R.S.$\Leftrightarrow$} \\
  % Eq. subst.
  \frule{{?p}({?s}) \vdash \quad}{{?s} = {?t}, {?p}({?t}) \vdash}{I.L.S.$=$} & \frule{\vdash {?p}({?s})}{{?s} = {?t} \vdash {?p}({?t})}{I.R.S.$=$} \\
  \frule{{?p}({?s}) \vdash \quad \vdash {?s} = {?t}}{{?p}({?t}) \vdash}{E.L.S.$=$} & \frule{\vdash {?p}({?s}) \quad \vdash {?s} = {?t}}{\vdash {?p}({?t})}{E.R.S.$=$}
  \end{array}$$
  \end{framed}
  \caption[Rules (2)]{Continuation of \autoref{fig:rules-list-1}. Schemas marked with an overline are constrained to resolve to fresh schemas; namely schemas that do not appear elsewhere in the sequent, apart from this binding.}
  \label{fig:rules-list-2}
\end{figure}

\input{figures/proof-sample.scala}

\begin{figure}[H]
  \centering
  \input{figures/proof-sample.lisa.tex}
  \caption[Sample proof (kernel)]{Example of a generated kernel proof following its description in the front (\autoref{fig:scala-proof-sample}). This proof is independent of any theory but depends on the axiom of powerset, the definition of the subset relation and the axioms of extensionality and union (imports 1-4). Such a kernel proof is objectively difficult to be produced by hand; in contrast it is relatively straightforward to be achieved in the front. Note that the latex code used in this figure is also part of the output.}
  \label{fig:lisa-proof-sample}
\end{figure}


\documentclass[10pt,a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{makeidx}
\usepackage{graphicx}
\usepackage[all]{xy}
\newcommand{\zz}{\mathbb{Z}}
\author{moi}
\begin{document}
\section{Intro}
We are briefly discussing bialgebras and their module algebras. Appropriate knowledge in the theory of bialgebras and category theroy is recommanded. An interesting example will be a central part of this paper.
\subsection{Algebras and coalgebras}
We say ring when we are talking about unital commutative rings, rings with
$$a b = b a,\ \forall a, b \in R \ \mathrm{and}\ \exists! 1_R \in R: 1_R a = a 1_R = a, \forall a \in R$$
\subsubsection{Algebras}
An $R$-algebra is a two-sided $R$-module $A$ with an $R$-linear map $\mu_A : A \otimes A \longrightarrow A$, called the multiplication. We call $A$ associative if
$$\xymatrix{
A^{\otimes 3} \ar[rr]^{\mu_A \otimes id_A} \ar[d]_{id_A \otimes \mu_A} && A^{\otimes 2}\ar[d]^{\mu_A}\\
A^{\otimes 2} \ar[rr]_{\mu_A} && A\\
}$$
commutes. We call $A$ unital if there is an $R$-linear map $\eta_A : R \longrightarrow A$ such that
$$\xymatrix{
R \otimes A \ar[rr]^{\eta_A \otimes id_A} \ar[rrd]_\simeq & & A \otimes A\ar[d]^{\mu_A} && A \otimes R \ar[ll]_{id_A \otimes \eta_A}\ar[lld]^\simeq\\
&&A&&\\
}$$
commutes. The triple $(A, \mu_A, \eta_A)$ denotes a associative unital algebra. If no ambiguity can arise we will omit the subscripts. Furthermore, we will omit the structure maps when clear from context.
\paragraph{Commutative algebras}
The $R$-linear map
$$\tau_{A\otimes B} : A \otimes B \longrightarrow B \otimes A, a \otimes b \longmapsto b \otimes a$$
is called the flip-isomorphism for two $R$-modules $A, B$. We say an $R$-algebra is commutative if
$$\xymatrix{
A \otimes A \ar[rr]^{\tau_{A\otimes A}} \ar[rrd]_{\mu}&& A \otimes A\ar[d]^\mu\\
&&A\\
}$$
commutes.
\paragraph{Examples} Some examples:
\begin{description}
\item[General unital rings] Clearly, all rings in theirselves are commutative associative unital algebras. In particular, $(\zz, \mu_{\zz}, \eta_{\zz})$ is an $\zz$-algebra and for any ring $(R, + , \cdot, 1)$ the
map
$$\varphi : \zz \longrightarrow R, n \longmapsto \begin{cases}
\sum_{n\ \mathrm{times}} 1_R & n \geq 0\\
\sum_{n\ \mathrm{times}} -1_R & n < 0\\
\end{cases}$$
is an $\zz$-algebra homomorphism. This summed up in the proposition: the class of unital associative algebra forms a category, with $\zz$-algebra homomorphisms as morphisms and has $\zz$ as inital object.
\item[Lie algebras] We call an algebra a Lie algebra if
$$\ker (id_{A \otimes A} - \tau_{A \otimes A}) \subset \ker \mu_A$$
and
$$\ker \left[((id \otimes \mu) \mu)(1 + \zeta_3 + \zeta_3^2)\right] \subset A.$$
The first condition is equivalent in demanding $A$ is anticommutative, i.e.
$$\tau \mu = - \mu$$
the second is called Jacobian identity. Any given associative $R$-algebra $A$, has a Lie algebras structure:
$$(\mathrm{End}_R(M), \mu = [\_,\_]),\ \mu = [a \otimes b \longmapsto a b - b a].$$
It is denoted by $\mathfrak{g}(A)$ and its multiplication is called commutator. For a given $R$-module $M$, the endomorphism algebra $\mathrm{End}_R(M)$ is a Lie algebra - denoted by $\mathfrak{gl}_(M)$. The matrix ring $M_n(R)$ has the general Lie algebra $\mathfrak{gl}_n(R)$ as Lie algebra. Its elements are all matrices and its multiplication is the commutator. We remark that these $R$-algebras are in general not associative or commutative. Some other Lie algebras are:
\begin{description}
\item[$\mathfrak{d}$] the Lie algebra of all upper diagonal matrices - a sub Lie algebra if $\mathfrak{gl}_n(R)$.
\item[$\mathfrak{n}$] the Lie algebra of all strictly upper diagonal matrices, a sub Lie algebra of $\mathfrak{d}$.
\item[$\mathfrak{sl}_n(R)$] the Lie algebra of all trace zero matrices.
\end{description}
\item[Ring extensions] If $R$ is a ring and $S/R$ is a ring extension, i.e. $S$ is an $R$-algebra,
\end{description}
\newpage
\section{Example}
Let $R$ be a ring with $2$ not dividing its characteristic. We want to study the an interesting example of a module algebra.
\subsection{Ring of Laurent polynomials}
Let $R$ be as above and $R[z, z^{-1}]$ be the ring of Laurent polynomials. We may wish to identify $R_1 := R[z,z^{-1}]$ with $R_2 := R[x,y]/\left<x y - 1 \right>$ with standard $R$-derivations:
$$\partial_x = \frac{\partial}{\partial x},\ \partial_y = \frac{\partial}{\partial y},$$
making $\left(R_1, \left\{\partial_x,\partial_y\right\}\right)$ an differential $R$-algebra. Since the commutator $\left[\partial_x,\partial_y\right]$ is trivial it is a partial differential algebra.
\subsubsection{The $\mathfrak{sl}_2(R)$-module}
We want to introduce two new $R$-linear maps $\partial_X, \partial_Y \in \mathrm{End}_R(R_1)$:
$$\partial_X := \partial_x - y^2 \partial_y,\ \partial_Y := \partial_y - x^2 \partial_x.$$
\paragraph{Claim}
We are proposing:
\begin{enumerate}
\item\label{claim01} $\left(R_1, \left\{\partial_X,\partial_Y\right\}\right)$ is a differential $R$-algebra or equivalently, $\partial_X, \partial_Y$ are $R$-derivations.
\item\label{claim02} $R_1$ is an $\mathfrak{sl}_2(R)$-module or equivalently, $$\mathfrak{g}_R(\{\partial_X, \partial_Y\}) := \left<\partial_X, \partial_Y\right>_{\mathrm{Lie-alg}} \simeq \mathfrak{sl}_2(R).$$
\item\label{claim03} $R_1$ is an $U(\mathfrak{sl}_2(R))$-module algebra.
\end{enumerate}
We are proving our claims step-by-step.
\begin{description}
\item[ad \ref{claim01}] We only need to show the Leibniz-rule as $R$-linearity is aready assumed. Let $r, s \in R_1$:
$$\begin{array}{rcl}
\partial_X(r s) &=& (\partial_x - y^2 \partial_y)(r s)\\
&&\\
 &=& \partial_x(r s) - y^2 \partial(r s)\\
 &&\\
 &=& \partial_x(r) s + r \partial_x(s) - y^2(\partial_y(r) s + r \partial_y(s))\\
 &&\\
 &=& \underbrace{\partial_x(r) s - y^2 \partial_y(r) s}_{\partial_X(r) s} +\underbrace{r \partial_x(s) - r y^2 \partial_y(s)}_{r \partial_X(s)}\\
 &&\\
 &=& \partial_X(r) s + r d\partial_X(s)\\
 \end{array}$$
By the symmetry of definition, we see that both maps are indeed $R$-derivations.
\item[ad \ref{claim02}] Let us commute the commutator of $\partial_X$ and $\partial_Y$:
$$\begin{array}{rcl}
[\partial_X, \partial_Y] &=& (\partial_x - y^2 \partial_y)(\partial_y - x^2 \partial_x) - (\partial_y - x^2 \partial_x)(\partial_x - y^2 \partial_y)\\
&&\\
&=& \partial_x \partial_y - 2 x \partial_x - x^2 \partial_x^2 - y^2 \partial_y^2 + x^2 y^2 \partial_y \partial_x \\
&&\\
&& -(\partial_y \partial_x - 2 y \partial_y - y^2 \partial_y^2 - x^2 \partial_x^2 + x^2 y^2 \partial_x \partial_y)\\
&&\\
&=& 2 (y \partial_y - x \partial_x)\\
\end{array}$$
This endomorphism is clearly an $R$-derivation. Setting $\partial_H := [\partial_X,\partial_Y]$ we will continue:
$$\begin{array}{rcl}
[\partial_H, \partial_X] &=& 2 [(y \partial_y - x \partial_x)(\partial_x - y^2 \partial_y) - (\partial_x - y^2 \partial_y)(y \partial_y - x \partial_x)]\\
&&\\
&=& 2 \left[y \partial_y \partial_x - 2 y^2 \partial_y - y^3 \partial_y^2 - x \partial_x^2 + x y^2 \partial_x \partial_y\right]\\
&&\\
&& - 2\left[y \partial_x \partial_y - \partial_x - x \partial_x^2 - y^2 \partial_y - y^3 \partial_y ^2 + x y^2 \partial_y \partial_x\right]\\
&&\\
&=& 2 \left(\partial_x - y^2 \partial_y\right) = 2 \partial_X\\
\end{array}$$
and
$$\begin{array}{rcl}
[\partial_H, \partial_Y] &=& 2 [(y \partial_y - x \partial_x)(\partial_y - x^2 \partial_x) - (\partial_y - x^2 \partial_x)(y \partial_y - x \partial_x)]\\
&&\\
&=& 2 \left[y \partial_y^ 2 - x^2 y \partial_y \partial_x - x \partial_x \partial_y + 2 x^2 \partial_x + x^3 \partial_x^2\right]\\
&&\\
&& - 2\left[\partial_y + y \partial_y^2 - x \partial_y \partial_x - x^2 y \partial_x \partial_y + x^2 \partial_x + x^3 \partial_x^2\right]\\
&&\\
&=& -2 \left(\partial_y - x^2 \partial_x\right) = -2 \partial_Y\\
\end{array}$$
which proves our second claim. We remark that this proof is applicable to $\left(R[x,y], \left\{\partial_X,\partial_Y\right\}\right)$.
\item[ad \ref{claim03}] Let %$i, j, k, \in \mathbb{Z}_{\geq 0}$ and
$m, n \in \mathbb{Z}$ and $\Psi_{R_1} := \left[\partial_Z \otimes r \longmapsto ev_r(\partial_Z) := \partial_Z(r)\right]$ for $Z = H, X, Y$:
$$\begin{array}{rcl}
\partial_H \partial_X \partial_Y \otimes z^m \otimes z^n &\stackrel{\Delta \otimes id_{R_1^{\otimes 2}}}{\longmapsto}& [(1 \otimes \partial_H + \partial_H \otimes 1) (1 \otimes \partial_X + \partial_X \otimes 1)\\
&& (1 \otimes \partial_Y + \partial_Y \otimes 1)] \otimes z^m \otimes z^n\\
&&\\
&=& %\sum_{i_0, j_0, k_0 \in \{0,1\}} %\substack{0 \leq i_0 \leq i\\0 \leq j_0 \leq j\\0 \leq k_0 \leq k\\}}\left(
%\begin{array}{c}
%i\\i_0\\
%\end{array}\right)\left(
%\begin{array}{c}
%j\\j_0\\
%\end{array}\right)\left(
%\begin{array}{c}
%k\\k_0\\
%\end{array}\right)
\sum_{i_0, j_0, k_0 \in \{0,1\}}%\substack{0 \leq i_0 \leq 1\\0 \leq j_0 \leq 1\\0 \leq k_0 \leq 1\\}}
\partial_H^{1 - i0} \partial_X^{1 - j_0} \partial_Y^{1 - k_0} \\
&&\\&&\otimes \partial_H^{i0} \partial_X^{j_0} \partial_Y^{k_0} \otimes z^m \otimes z^n\\
&&\\
&\stackrel{f}{\longmapsto}& \sum_{i_0, j_0, k_0 \in \{0,1\}}%\substack{0 \leq i_0 \leq 1\\0 \leq j_0 \leq 1\\0 \leq k_0 \leq 1\\}}%\left(
%\begin{array}{c}
%i\\i_0\\
%\end{array}\right)\left(
%\begin{array}{c}
%j\\j_0\\
%\end{array}\right)\left(
%\begin{array}{c}
%k\\k_0\\
%\end{array}\right)
\partial_H^{1 - i0} \partial_X^{1 - j_0} \partial_Y^{1 - k_0}(z^m) \\
&&\\&&\otimes \partial_H^{i0} \partial_X^{j_0} \partial_Y^{k_0}(z^n)\\
&&\\
&=& \partial_H(\partial_X(\partial_Y(z^m))) \otimes z^n + \partial_H(\partial_X(z^m)) \otimes \partial_Y(z^n)\\
&& + \partial_H(\partial_X(z^m)) \otimes \partial_X(z^n) + \partial_X(\partial_Y(z^m)) \otimes \partial_H(z^n)\\
&& + \partial_Y(z^m) \otimes \partial_H(\partial_X(z^n)) + \partial_X(z^m) \otimes \partial_H(\partial_X(z^n)) \\
&& + \partial_H(z^m) \otimes \partial_X(\partial_Y(z^n)) + z^m \otimes \partial_H(\partial_X(\partial_Y(z^n)))\\
\end{array}$$
where $f = (\Psi_{R_1} \otimes \Psi_{R_1}) (id_{U(\mathfrak{sl}_2(R))} \otimes \tau_{U(\mathfrak{sl}_2(R)) \otimes R_1}\otimes id_{R_1})$.
$$\begin{array}{rcl}
\partial_H\partial_X\partial_Y \otimes z^m \otimes z^n &\stackrel{id \otimes \mu_{R_1}}{\longmapsto}&
\partial_H \partial_X \partial_Y \otimes z^{m + n}\\
&&\\
&\stackrel{\Psi_{R_1}}{\longmapsto}& \partial_H(\partial_X(\partial_Y(z^{m + n})))\\
&&\\
&=& \partial_H(\partial_X(\partial_Y(z^m) z^{n})) + \partial_H(\partial_X(z^m \partial_Y(z^n)))\\
&&\\
&=& \partial_H(\partial_X(\partial_Y(z^m)) z^{n}) + \partial_H(\partial_Y(z^m) \partial_X(z^{n}))\\
&&\\
&& + \partial_H(\partial_X(z^m) \partial_Y(z^n)) + \partial_H(z^m \partial_X(\partial_Y(z^n)))\\
&&\\
&=& \partial_H(\partial_X(\partial_Y(z^m))) z^{n} + \partial_X(\partial_Y(z^m)) \partial_H(z^n)\\
&&\\
&& + \partial_H(\partial_Y(z^m)) \partial_X(z^{n}) + \partial_Y(z^m) \partial_H(\partial_X(z^n))\\
&&\\
&& + \partial_H(\partial_X(z^m)) \partial_Y(z^n) + \partial_X(z^m) \partial_H(\partial_Y(z^n))\\
&&\\
&& + \partial_H(z^m) \partial_X(\partial_Y(z^n)) + z^m \partial_H(\partial_X(\partial_Y(z^n)))\\
&&\\
&=& \mu_{R_1}(f(\Delta \otimes id_{R_1^{\otimes 2}}(\partial_H \partial_X \partial_Y \otimes z^m \otimes z^n)))\\%\sum_{i_0, j_0, k_0 \in \{0,1\}} \mu_{R_1} \left(\partial_H^{1 - i_0} (\partial_X^{1 - j_0} (\partial_Y^{1 - k_0}(z^m))) \otimes \partial^{i_0} (\partial_X^{j_0}(\partial_Y^{k_0}(z^n)))\right)\\
\end{array}$$
Replacing $\partial_H \partial_X \partial_Y$ with powers of derivations $\partial_H^i \partial_X^j \partial_Y^k$ follows inductively wrt. binomial formula for each power. Furthermore for $d = \sum_{i,j,k} d_{i,j,k} \partial_H^i \partial_X^j \partial_Y^k$
$$\Psi_{R_1} (d \otimes 1_{R_1}) = \begin{cases}
d_{0,0,0} & d_{i,j,k} = 0\ \forall i + j + k \geq 1\\
0 & \mathrm{else}\\
\end{cases}$$
The two commutative diagrams defining module algebras hold and we proved the last claim.
\end{description}
As we stated earlier, the proof for each claim can be easily translated to $\left(R[x,y], \{\partial_X,\partial_Y\}\right)$. This is as $R[x,y]$ is itself a $U(\mathfrak{sl}_2(R))$-module algebra and the ideal
$$I = \left<x y - 1 \right>$$
is $U(\mathfrak{sl}_2(R))$-stable:
$$\begin{array}{rcl}
\partial_X(f (x y - 1)) &=& \partial_X(f) (x y - 1) + f \partial_X(x y - 1)\\
&&\\
&=& \underbrace{\partial_X(f) (x y - 1)}_{\in I} + f (\partial_X(x) y + x \partial_X(y))\\
&&\\
&\equiv& f (y - x y^2) \mod I \equiv -f y (x y - 1) \mod I \in I\\
&&\\
\partial_Y(f (x y - 1)) &=& \partial_Y(f) (x y - 1) + f \partial_Y(x y - 1)\\
&&\\
&=& \underbrace{\partial_Y(f) (x y - 1)}_{\in I} + f (\partial_Y(x) y + x \partial_Y(y))\\
&&\\
&\equiv& f (x - x^2 y) \mod I \equiv -f x (x y - 1) \mod I \in I\\
\end{array}$$
Proving $\partial_Z(I) \subset I$ for $Z = X, Y$. Furthermore, $\partial_X \circ \partial_Y (I) \subset I$ and $\partial_Y \circ \partial_X(I) \subset I$ implying $\partial_H(I) \subset I$. Therefore, $I$ is $D = U(\mathfrak{sl}_2(R))$-stable. In particular, we have that the projection
$$\pi : R[x,y] \longrightarrow R[x,y]/I$$
is a homomorphism of $D$-module algebras. Now, we would like to know if $R_1$ is simple as $D$-module algebra. We assume that there is an ideal $J \subset R_1$ being $D$-stable. Let $F = \{f_i \in R_1\backslash R_1^\times : i \in \mathbb{Z}_{>0}\}$ be a finite set of generators of $J$. Then
$$\begin{array}{rcl}
\partial_Z\left(\sum_{i=0}^n g_i f_i\right) &=& \sum_{i=0}^n \left(\underbrace{\partial_Z(g_i) f_i}_{\in J} + g_i \partial_Z(f_i)\right)\\
&&\\
&\equiv& \sum_{i=0}^n g_i \partial_Z(f_i) \mod J \equiv 0 \mod J\\
&\Leftrightarrow&\\
\sum_{i=0}^n g_i \partial_Z(f_i) &=& \sum_{j=0}^m h_i f_j,\\
\end{array}$$
for $Z = X, Y, H$ and $h_i \in R_1$. Now, we are going to use induction on $n$. Firstly, let $n = 1$ then
$$F = \{f\}\ \mathrm{and}\ g \partial_X(f) = h f$$
for appropriate $g, h \in R_1$. A non-trivial solution is clearly $f = g, h = \partial(f)$. If $f$ is not square-free, i.e. there is a $h': h'^2\mid f$ and $h'^3\nmid f$ we may consider $f_h = f/h'^2$ as ideal generator. If $f = \sum_{i=-m}^n f_i x^i$ we may write $f$ as
$$f = x^{-m} \underbrace{\sum_{i=0}^{n+m} f_{i-m} x^i}_{\hat{f}}$$
i.e. $f$ is associated to some element $\hat{f} \in R[x]$. Hence, if $\partial_Z(f) \in R_1.f \Leftrightarrow \partial_Z(x^{-m}) \hat{f} + x^{-m} \partial_Z(\hat{f})$.
$$\begin{array}{rcl}
f = \sum_{i=-m}^n f_i x^i &\stackrel{\partial_X}{\longmapsto}& \sum_{i=0}^{n-1} (i + 1) f_{i+1} x^i - \sum_{i=2}^{m + 1} (i - 1) f_{1 - i} x^{-i}\\
&&\\
n f - x \partial_X(f) &=& \sum_{i=0}^{n-1} (n - i) f_i x^i + \sum_{i=1}^m (n + i) f_{-i} x^{-i}\\
&&\\
\frac{f_{n-1}}{f_n} \partial_X(f) - (n - 1) (n f + x \partial_X(f)) &=&
\end{array}$$
\end{document}


\section{Gradient-based Length-scale Optimization}\label{sec:ls}

In the previous sections, we defined preference learning models that
incorporate GP priors over the latent functions.
The covariances of these GPs are defined by a kernel function $k$,
often of the following form:
\begin{flalign}
k_{\bs \theta}(\bs x, \bs x') = \prod_{d=1}^D k_d\left(\frac{|x_d - x_d'|}{l_d}, \bs\theta_d \right),
\label{eq:kernel}
\end{flalign}
where $D$ is the number of features,
$l_d$ is a length-scale hyper-parameter,
and $\bs \theta_d$ are additional hyper-parameters for an individual
feature kernel, $k_d$.
%Each $k_d$ is a function of the distance between the $d$th feature values in
%feature vectors $\bs x$ and  and $\bs x'$.
%The product over features in $k$ means that data points have
%high covariance only if the kernel functions, $k_d$, for all features are high
%(a soft AND operator).
It is possible to replace the product with a sum, allowing covariance to increase
for every feature that is similar (analogous to OR),
rather than only being high if all dimensions are similar (AND).
%or other combinations of the individual feature kernels.
%The choice of combination over features is therefore an additional hyper-parameter.
% citations?
The length-scale controls the smoothness of $k_d$
across the feature space.
%and the contribution of each feature to the model.
If a feature has a large length-scale,
its values have less effect on $k_{\bs\theta}(\bs x, \bs x') $
than if it has a shorter length-scale.
Hence, it is important to set $l_d$ to correctly capture feature relevance.
A computationally frugal option is a median heuristic, which effectively normalizes
the feature but allows extreme values to remain outliers:
\begin{flalign}
 l_{d,MH} = D \mathrm{median}( \{ |x_{i,d} - x_{j,d}| \forall i=1,..,N, \forall j=1,...,N\} ).
\end{flalign}
%The motivation is that the median will normalize the feature, so that features
%are equally weighted regardless of their scaling. By using a median to perform this
%normalization,
Multiplying the median by the number of features, $D$,
prevents  the average covariance $k_{\bs \theta}(\bs x, \bs x')$ between items
from increasing as we add more features using the
product kernel in Equation \ref{eq:kernel}.
This heuristic has been shown to work reasonably well for the task of
comparing distributions~\citep{gretton2012optimal}, but has %is a simple heursitic with
 no guarantees of optimality.

An alternative for setting $l_d$ is Bayesian model selection using
\emph{type II maximum likelihood},
which chooses the value of $l_d$ that
maximizes the approximation to the log marginal likelihood, %, $p(\bs y | \bs \theta)$,
%Since the marginal likelihoods for our models are intractable, we maximize
%the value of the variational lower bound,
$\mathcal{L}$.
% after convergence of the
%inference algorithm
(Equation \ref{eq:lowerbound} for GPPL and and Equation \ref{eq:lowerbound_crowd} for crowdGPPL).
Optimizing kernel length-scales in this manner is known as automatic relevance determination (ARD)~\citep{rasmussen_gaussian_2006}, since the optimal
value of $l_d$ depends on the relevance of feature $d$.
% Removing irrelevant features could improve performance,
% since it reduces the dimensionality of the space of the preference function.
%A problem when using text data is that large vocabulary sizes and additional linguistic features
%lead to a large number of dimensions, $D$.
%The standard maximum likelihood II optimisation requires
%$\mathcal{O}(D)$ operations to tune each length-scale.
%To perform ARD on feature $d$,
%we only need to be able to evaluate $\mathcal{L}$
%after variational inference has converged with any given value of $l_d$.
%However,
Computing derivatives of $\mathcal{L}$
with respect to $l_d$ enables the use of
efficient gradient-based optimization methods
such as L-BFGS-B~\citep{zhu1997algorithm}.
%which perform iterative optimization,
%using gradients to guide changes for all $D$ length-scales simultaneously.
For the single user GPPL, the required gradient
with respect to the $d$th length-scale, $l_d$, is as follows:
%Following the derivations in Appendix \ref{sec:vb_eqns}, Equation \ref{eq:gradient_ls},
\begin{flalign}
& \nabla_{l_{\! d}} \mathcal{L} =
\frac{\partial \mathcal{L}}{\partial \hat{\bs f}_m} \frac{\partial \hat{\bs f}_m}{\partial l_d}
+ \frac{\partial \mathcal{L}}{\partial \bs S^{-1}} \frac{\partial \bs S^{-1}}{\partial l_d}
+ \frac{\partial \mathcal{L}}{\partial a} \frac{\partial a}{\partial l_d}
+ \frac{\partial \mathcal{L}}{\partial b} \frac{\partial b}{\partial l_d}
+ \frac{\partial \mathcal{L}}{\partial \bs K}\frac{\partial \bs K}{\partial l_d}. &
\end{flalign}
The partial derivatives with respect to the variational parameters
$\hat{\bs f}_m$, $\bs S$, $a$ and $b$
%arise because they
depend indirectly on the length-scale through the expectations
in the variational $q$ factors.
However, when the variational inference algorithm has converged,
$\mathcal{L}$ is at a maximum, %$\frac{\partial \mathcal{L}}{\partial \hat{\bs f}_m}$,
%$\frac{\partial \mathcal{L}}{\partial \bs S^{-1}}$,
%$\frac{\partial \mathcal{L}}{\partial a}$ and
%$\frac{\partial \mathcal{L}}{\partial b}$
these terms % partial derivatives of $\mathcal{L}$ with respect to $\hat{\bs f}_m$, $\bs S$, $a$ and $b$
are zero, and the derivative is relatively simple to compute (see Appendix \ref{sec:gradients} for the full equations).

% is defined by Equation \ref{eq:kernel_der}.
% Since we cannot compute $\bs K$ in high dimensions, in practice we substitute $\bs K_{mm}$ for $\bs K$,
% $\bs S$ for $\bs C$, $\hat{\bs f}_{m}$ for $\hat{\bs f}$ and $\bs\mu_{m}$ for $\bs\mu$ so that


\documentclass[letterpaper,10pt]{article}
\usepackage[margin=2cm]{geometry}

\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage[colorlinks]{hyperref}

\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\argmax}{arg\,max}

\newcommand{\pandochline}{\vspace{2em}\href{./document.html}{\textbf{Back to Top}}
	\vspace{-2em}\begin{center}\rule{\textwidth}{1pt}\end{center}}

\setlength{\parindent}{0em}
\setlength{\parskip}{1em}

\title{\textbf{18794 Pattern Recognition Theory\\Prof. Marios Savvides}}
\author{HMW-Alexander}

\begin{document}

\maketitle

\tableofcontents
\newpage

\section{Introduction}

\pandochline
\section{Decision Theory}

\subsection{Terms}

\begin{itemize}
	\item Feature Space:
	\begin{itemize}
		\item \textbf{Feature}: a distinctive characteristic or quality of the object
		\item \textbf{Feature vector}: combine more than one feature as a vector
		\item \textbf{Feature space}: The space defined by the feature vectors
	\end{itemize}
	\item Classifiers:
	\begin{itemize}
		\item \textbf{Decision regions}: a classifier partitions the feature space into class-corresponding decision regions.
		\item \textbf{Decision boundaries}: the borders between the decision regions.
	\end{itemize}
\end{itemize}

\subsection{Bayes Rule}

\begin{equation}
P(w_i|x)=\frac{P(x,w_i)}{P(x)}=\frac{P(x|w_i)P(w_i)}{\sum_{k=1}^{C}{P(x|w_k)P(w_k)}}
\end{equation}
\begin{itemize}
	\item \textbf{Posterior Probability} $P(w_i|x)$: the conditional probability of correct class being $w_i$ given that feature value $x$ has been observed.
	\item \textbf{Evidence} $P(x)$: the total probability of observing the feature value of $x$.
	\item \textbf{Likelihood} $P(x|w_i)$: the conditional probability of observing a feature value of $x$ given that the correct class is $w_i$.
	\item \textbf{Prior Probability} $P(w_i)$: the probability of class $w_i$, $\sum_{k=1}^{C}{P(w_k)}=1$.
	\item \textbf{Bayes Classifiers} decide on the class that has the \textbf{largest posterior probability} ($\max_{w_i}{P(w_i|x)}$). They are statistically the best classifiers i.e. they are minimum error classifiers (optimal).
\end{itemize}

\subsection{Minimum Probability of Error}

\begin{itemize}
	\item $\epsilon=P(error|class)$: probability of assigning $x$ to the wrong class $w$.
	\item $P_e=\sum_{k=1}^{C}{P(w_k)\epsilon_k}$: total probability of error.
\end{itemize}
\begin{figure}[!ht]
	\centering
	\includegraphics[width=10cm]{./img/minimum_probability_of_error.png}
\end{figure}
For the two class case shown above, we want to minimize $P_e$ as below:
\begin{equation}
\begin{array}{rcl}
P_e & = & P(w_1)\epsilon_1 + P(w_2)\epsilon_2 \\
	& = & P(w_1)\int_{R_2}{f(x|w_1)dx} + P(w_2)\int_{R_1}{f(x|w_2)dx} \\
	& = & P(w_1)(1-\int_{R_1}{f(x|w_1)dx}) + P(w_2)\int_{R_1}{f(x|w_2)dx} \\
	& = & P(w_1) + \int_{R_1}{(P(w_2)f(x|w_2) - P(w_1)f(x|w_1))dx} \\
\end{array}
\end{equation}
To minimize $P_e$, we want $P(w_2)f(x|w_2) - P(w_1)f(x|w_1)$ to be always negative$(<0)$ in the region $R_1$:
\begin{equation}
\begin{array}{rcl}
P(w_1)f(x|w_1) - P(w_2)f(x|w_2) >0 & \Rightarrow & w_1 \\
P(w_1)f(x|w_1) - P(w_2)f(x|w_2) <0 & \Rightarrow & w_2 \\
\end{array}
\end{equation}

\subsection{Likelihood Ratio}

\begin{itemize}
	\item \textbf{Likelihood ratio}: $l(x)=\frac{f(x|w_1)}{f(x|w_2)}$
	\item \textbf{Log likelihood ratio}: $\ln(l(x))=\ln(\frac{f(x|w_1)}{f(x|w_2)})=\ln(f(x|w_1))-\ln(f(x|w_2))$
	\item \textbf{Ratio of a priori probabilities}: $T=\frac{P(w_2)}{P(w_1)}$
	\item \textbf{Log ratio of a priori probabilities}: $\ln(T)=\ln(\frac{P(w_2)}{P(w_1)})=\ln(P(w_2))-\ln(P(w_1))$
\end{itemize}

\begin{equation}
\begin{array}{rcl}
\ln(l(x)) = \ln(\frac{f(x|w_1)}{f(x|w_2)}) > \ln(\frac{P(w_2)}{P(w_1)}) = \ln(T) & \Rightarrow & w_1 \\
\ln(l(x)) = \ln(\frac{f(x|w_1)}{f(x|w_2)}) < \ln(\frac{P(w_2)}{P(w_1)}) = \ln(T) & \Rightarrow & w_2 \\
\end{array}
\end{equation}

\subsubsection{Likelihood as Gaussian distribution}
Assume likelihood $f(x|w_i)$ are Gaussian distributions with mean $\mu_i$ and variance $\sigma_i^2$.
\begin{equation}
f(x|w_i)=\frac{1}{\sqrt{2\pi\sigma_i^2}}\exp\left(-\frac{(x-\mu_i)^2}{2\sigma_i^2}\right)
\end{equation}
\begin{figure}[!ht]
	\centering
	\includegraphics[width=6cm]{./img/gaussian_distribution_likelihood.png}
\end{figure}
The log likelihood ratio:
\begin{equation}
\begin{array}{rcl}
\ln(l(x)) & = & \ln\left(\frac{\frac{1}{\sqrt{2\pi\sigma_1^2}}\exp\left(-\frac{(x-\mu_1)^2}{2\sigma_1^2}\right)}{\frac{1}{\sqrt{2\pi\sigma_2^2}}\exp\left(-\frac{(x-\mu_2)^2}{2\sigma_2^2}\right)}\right) \\
         & = & \ln(\frac{\sigma_2}{\sigma_1})+\frac{(x-\mu_2)^2}{2\sigma_2^2}-\frac{(x-\mu_1)^2}{2\sigma_1^2}
\end{array}
\end{equation}

Case: $\sigma_1=\sigma_2=\sigma$
\begin{equation}
\begin{array}{rcl}
\ln(l(x)) & = & \frac{2x(\mu_1-\mu_2)-(\mu_1^2-\mu_2^2)}{2\sigma^2}
\end{array}
\end{equation}

\begin{equation}
\begin{array}{rcl}
x(\mu_1-\mu_2) - \frac{\mu_1^2-\mu_2^2}{2} > \sigma^2 \ln(\frac{P(w_2)}{P(w_1)}) & \Rightarrow & w_1 \\
x(\mu_1-\mu_2) - \frac{\mu_1^2-\mu_2^2}{2} < \sigma^2 \ln(\frac{P(w_2)}{P(w_1)}) & \Rightarrow & w_2 \\
\end{array}
\end{equation}

If $P(w_1)=P(w_2)$
\begin{equation}
x = \frac{\mu_1+\mu_2}{2}
\end{equation}

\begin{figure}[!ht]
	\centering
	\includegraphics[width=8cm]{./img/gaussian_linear_classifier.png}
\end{figure}

Case: $\sigma_1\neq\sigma_2$

\begin{figure}[!ht]
	\centering
	\includegraphics[width=8cm]{./img/guassian_quadratic_classifier.png}
\end{figure}

\pandochline
\section{Parametric \& Non-Parametric Density Estimation}

Learning \& Classification
\begin{itemize}
	\item Supervised:
	\begin{itemize}
		\item Bayes Density:
		\begin{itemize}
			\item Parametric: assumes a particular form of a PDF (e.g. Gaussian) is known $\rightarrow$ determine the parameters (e.g. mean and variance)
			\begin{itemize}
				\item Maximum Likelihood ($f(x|w_i)$) Estimation (MLE)
				\item Maximum A Posteriori (Bayesian $p(w_i|x)$) Estimation (MAPE)
			\end{itemize}
			\item Non-Parametric: no assumption about the density
			\begin{itemize}
				\item Parzen Windows
				\item Kernel Density Estimation
				\item K-Nearest Neighbor Rule
			\end{itemize}
		\end{itemize}
		\item Discriminant Analysis:
		\begin{itemize}
			\item Linear
			\item Non-Linear
		\end{itemize}
	\end{itemize}
	\item Unsupervised:
	\begin{itemize}
		\item Clustering
	\end{itemize}
\end{itemize}

\subsection{ML Estimation (MLE)}
Steps:
\begin{itemize}
	\item Assume $P(x|w)$ has a known parametric form uniquely determined by the parameter vector $\theta$
	\item The parameters are assumed to be fixed (i.e. non random) but unknown
	\item Suppose we have a dataset $D$ with the samples in $D$ having been drawn \textbf{independently} according to the probability law $P(x|w)$
	\item The MLE is the value of $\theta$ that best explains the data and once we know this value, we know $P(x|w)$
	\begin{equation}
	\hat{\theta}=\argmax_\theta\{P(D|\theta)\}=\argmax_\theta\{\prod_{k=1}^{N}{P(x_k|\theta)}\}=\argmax_\theta\{\sum_{k=1}^{N}{\log(P(x_k|\theta))}\}
	\end{equation}
	Choose the value of $\theta$ that is the most likely to give rise to the data we observe.
\end{itemize}

Method:
\begin{itemize}
	\item Assume $\theta=[\theta_1,\theta_2,\dots,\theta_p]^T$
	\item Assume gradient operator $\nabla_\theta=[\frac{\partial}{\partial\theta_1},\frac{\partial}{\partial\theta_2},\dots,\frac{\partial}{\partial\theta_p}]^T$
	\item Assume the log-likelihood of the objective function $l(\theta)=\sum_{k=1}^{N}{\log(P(x_k|\theta))}$
	\item Therefore: $\nabla_\theta l(\theta)=\sum_{k=1}^{N}{\nabla_\theta \log(P(x_k|\theta))}=0$
\end{itemize}

\subsubsection{E.g. Univariate Gaussian}

\begin{equation}
P(x|\mu,\sigma^2)=\frac{1}{\sqrt{2\pi\sigma^2}}\exp(-\frac{(x-\mu)^2}{2\sigma^2})
\end{equation}

\begin{equation}
\theta=\left[\begin{array}{c}
\theta_1=\mu \\
\theta_2=\sigma^2 \\
\end{array}\right]
\end{equation}

\begin{equation}
\begin{array}{rcl}
\log(P(x_k|\theta)) & = & \log(\frac{1}{\sqrt{2\pi\theta_2}}\exp(-\frac{(x_k-\theta_1)^2}{2\theta_2})) \\
                   & = & -\frac{1}{2}\log(2\pi\theta_2)-\frac{(x_k-\theta_1)^2}{2\theta_2} \\
\end{array}
\end{equation}

\begin{equation}
\begin{array}{rcl}

\nabla_\theta \log(P(x_k|\theta)) & = & \left[
\begin{array}{c}
\frac{x_k-\theta_1}{\theta_2} \\
-\frac{1}{2\theta_2}+\frac{(x_k-\theta_1)^2}{2\theta_2} \\
\end{array}\right] \\

\Rightarrow \left\{
\begin{array}{rcl}
\sum_{k=1}^{n}{\frac{1}{\theta_2}(x_k-\theta_1)} & = & 0 \\
-\sum_{k=1}^{n}{\frac{1}{\theta_2}}+\sum_{k=1}^{n}{\frac{(x_k-\theta_1)^2}{\theta_2^2}} & = & 0 \\
\end{array}\right. & \Rightarrow & \left\{
\begin{array}{rcl}
\hat{\mu} & = & \frac{1}{n}\sum_{k=1}^{n}{x_k} \\
\hat{\sigma}^2 & = & \frac{1}{n}\sum_{k=1}^{n}{(x_k-\hat{\mu})^2} \\
\end{array}\right. \\

\end{array}
\end{equation}

\subsubsection{E.g. Multivariate Gaussian}

\begin{equation}
P(x|\mu,\Sigma)=\frac{1}{(2\pi)^{d/2}|\Sigma|^{1/2}}\exp(-\frac{1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu))
\end{equation}

Case: only the mean $\mu$ is unknown

\begin{equation}
\log(P(x_k|\mu)) = -\frac{1}{2}\log((2\pi)^{d}|\Sigma|)-\frac{1}{2}(x_k-\mu)^T\Sigma^{-1}(x_k-\mu)
\end{equation}

\begin{equation}
\begin{array}{rcl}
\nabla_\theta \log(P(x_k|\mu)) & = & \Sigma^{-1}(x_k-\mu) \\
\Rightarrow \sum_{k=1}^{n}{\Sigma^{-1}(x_k-\mu)} = 0 & \Rightarrow & \hat{\mu} = \frac{1}{n}\sum_{k=1}^{n}{x_k}
\end{array}
\end{equation}

Case: Neither the mean $\mu$ nor the covariance matrix $\Sigma$ are known

\begin{equation}
\theta=\left[\begin{array}{rcl}
\theta_1 & = & \mu \\
\theta_2 & = & \Sigma \\
\end{array}\right]
\end{equation}

Convert the covariance matrix $\Sigma$

\begin{equation}
\begin{array}{rcl}
P(D|\Sigma) & = & \prod_{k=1}^{n}{P(x_k|\Sigma)} \\
			& = & \prod_{k=1}^{n}{\frac{1}{\sqrt{(2\pi)^d|\Sigma|}}\exp(-\frac{1}{2}(x_k-\mu)^T\Sigma^{-1}(x_k-\mu))} \\
			& = & \frac{1}{((2\pi)^d|\Sigma|)^{n/2}}\exp(-\frac{1}{2}\sum_{n}^{k=1}{(x_k-\mu)^T\Sigma^{-1}(x_k-\mu)}) \\
\end{array}
\end{equation}

The property of matrix trace
\begin{itemize}
	\item scalar $b^TBb=trace(b^TBb)=trace(Bbb^T)$
	\item $trace(A+B)=trace(A)+trace(B)$
	\item $trace(C(A+B))=trace(CA)+trace(CB)$
	\item $trace(A)$ is the sum of $A$'s eigenvalues
\end{itemize}

\begin{equation}
\begin{array}{rcl}
\sum_{n}^{k=1}{(x_k-\mu)^T\Sigma^{-1}(x_k-\mu)} & = & \sum_{n}^{k=1}{trace((x_k-\mu)^T\Sigma^{-1}(x_k-\mu))} \\
												& = & \sum_{n}^{k=1}{trace(\Sigma^{-1}(x_k-\mu)(x_k-\mu)^T)} \\
												& = & trace(\Sigma^{-1}\sum_{n}^{k=1}{(x_k-\mu)(x_k-\mu)^T})\\
\end{array}
\end{equation}

Assume
\begin{equation}
A=\frac{1}{n}\sum_{k=1}^{n}(x_k-\mu)(x_k-\mu)^T
\end{equation}
and $A$ is fixed.

Then
\begin{equation}
\sum_{n}^{k=1}{(x_k-\mu)^T\Sigma^{-1}(x_k-\mu)} = n~trace(\Sigma^{-1}A)
\end{equation}

Therefore
\begin{equation}
P(D|\Sigma) = \frac{1}{((2\pi)^d|\Sigma|)^{n/2}}\exp(-\frac{n}{2}trace(\Sigma^{-1}A))
\end{equation}

Assume
\begin{equation}
B=\Sigma^{-1}A
\end{equation}
and B's eigenvalues are $\lambda_1,\lambda_2,\dots,\lambda_d$

The property of matrix determinant
\begin{itemize}
	\item $det(AB)=det(A)det(B)$
	\item $det(A^{-1})=det(A)^{-1}$
	\item $det(A)$ is the product of $A$'s eigenvalues
\end{itemize}

Therefore
\begin{equation}
\begin{array}{rcl}
P(D|\Sigma) & = & \frac{1}{((2\pi)^d|\Sigma|)^{n/2}}\exp(-\frac{n}{2}trace(\Sigma^{-1}A)) \\
			& = & \frac{1}{(2\pi)^{dn/2}}|\Sigma|^{-n/2}\exp(-\frac{n}{2}trace(B)) \\
			& = & \frac{1}{(2\pi)^{dn/2}}(\frac{|B|}{|A|})^{n/2}\exp(-\frac{n}{2}trace(B)) \\
			& = & \frac{1}{(2\pi)^{dn/2}}|A|^{-n/2}(\prod_{i=1}^{d}{\lambda_i})^{n/2}\exp(-\frac{n}{2}\sum_{i=1}^{d}{\lambda_i}) \\
\end{array}
\end{equation}

\begin{equation}
\log(P(D|\Sigma))=-\frac{dn}{2}\log(2\pi)-\frac{n}{2}\log(|A|)+\frac{n}{2}\sum_{i=1}^{d}{\log(\lambda_i)}-\frac{n}{2}\sum_{i=1}^{d}{\lambda_i}
\end{equation}

\begin{equation}
\frac{\partial(\log(P(x|\Sigma)))}{\partial\lambda_i}=\frac{n}{2\lambda_i}-\frac{n}{2}=0 \Rightarrow \lambda_i=1
\end{equation}

Therefore $B \sim I$, and $\Sigma^{-1}A=I \Rightarrow \hat{\Sigma}=A=\frac{1}{n}\sum_{k=1}^{n}(x_k-\mu)(x_k-\mu)^T$.

\subsection{Bayesian Estimation (BE)}
Steps:
\begin{itemize}
	\item The parameters are assumed to be random variables with some known priori distribution.
	\item Bayesian approach aims at estimating the posterior density $P(\theta|D)$
	\item The MAPE(Maximum A Posteriori Estimate) of $\theta$ is the value of $\theta$ that maximizes the posterior density $P(\theta|D)$.
\end{itemize}
Method:
\begin{itemize}
	\item Estimate $P(x|D)$ from sample data, and assume it has a known parametric form. So $P(x|\theta)$ is completely known, but $\theta$ is random variable and has its own PDF.
	\item $P(x|D)=\int{P(x,\theta|D)d\theta}=\int{P(x|\theta)P(\theta|D)d\theta}$
	\item Posterior $P(\theta|D)=\frac{P(D|\theta)P(\theta)}{P(D)}$, where $P(D|\theta)=\prod_{k=1}^{n}{P(x_k|\theta)}$
	\item Since $P(D)$ is constant, therefore $\hat{\theta}=\argmax_\theta\{P(D|\theta)P(\theta)\}$
\end{itemize}

\subsubsection{E.g. Univariate Gaussian}
...

\subsubsection{E.g. Multivariate Gaussian}
...

\pandochline
\section{Principle Component Analysis (PCA)}
Purpose: get projection vectors $\omega$ that maximize the variance of the projected data.
\begin{equation}
\max_\omega(Var(\omega^Tx)),\text{~where~}||\omega||=1
\end{equation}

Derivation: Assume the objective function
\begin{equation}
\begin{array}{rcl}
J(\omega) & = & Var(\omega^Tx) \\
		  & = & E(\omega^T(x-\mu)(x-\mu)^T\omega) \\
		  & = & \omega^TE((x-\mu)(x-\mu)^T)\omega \\
		  & = & \omega^T\Sigma\omega~~~~(\text{where~} ||\omega||=1)
\end{array}
\end{equation}
Then use Lagrangian optimization method to get the $\omega^*$
\begin{equation}
\begin{array}{rcl}
L(\omega,\lambda) & = & \omega^T\Sigma\omega - \lambda(\omega^T\omega-1) \\
\frac{\partial L(\omega,\lambda)}{\partial\omega} & = & 2\Sigma\omega-2\lambda\omega=0 \\
& \Rightarrow & \Sigma\omega=\lambda\omega
\end{array}
\end{equation}

To maximize $J(\omega)$, we need to maximize $\omega^T\Sigma\omega=\lambda \rightarrow \lambda_{max}$

\end{document}


\section{Training Procedure}%
\label{sec:training_procedure}

%
By augmenting traditional HMC methods with these trainable functions, we hope
to obtain a sampler that has the following key properties:
%
\begin{enumerate}
    \item Fast mixing (i.e.\ able to quickly produce uncorrelated samples).
    \item Fast burn-in (i.e.\ rapid convergence to the target distribution).
    \item Ability to mix across energy levels.
    \item Ability to mix between modes.
\end{enumerate}
%
Following the results in~\cite{10.2307/24308995}, we design a loss function
with the goal of maximizing the expected squared jumped distance (or
analogously, minimizing the lag-one autocorrelation).
%
To do this, we first introduce
\begin{equation}
  \delta(\xi, \xip) = \delta((x^{\prime}, v^{\prime}, d^{\prime}), (x, v, d))
  \equiv \| x - x^{\prime}\|^2_2.  \label{eq:metric_orig}
\end{equation}
%
Then, the expected squared jumped distance is given by $\mathbb{E}_{\xi\sim
p(\xi)} \left[\delta(\mathbf{FL}_{\theta}\xi, \xi) A(\mathbf{FL}_{\theta}\xi |
\xi)\right]$.
%
By maximizing this objective function, we are encouraging transitions that
efficiently explore a local region of state-space, but may fail to explore
regions where very little mixing occurs.
%
To help combat this effect, we define a loss function
%
\begin{equation}
    \ell_{\lambda}(\xi, \xi^{\prime}, A(\xi^{\prime}|\xi)) =
        \frac{\lambda^2}{\delta(\xi,\xi^{\prime}) A(\xi^{\prime}|\xi)} -
        \frac{\delta(\xi,\xi^{\prime}) A(\xi^{\prime}|\xi)}{\lambda^2}
    \label{eq:loss_ell}
\end{equation}
%
where $\lambda$ is a scale parameter describing the characteristic length scale
of the problem.
%
Note that the first term helps to prevent the sampler from becoming stuck in a
state where it cannot move effectively, and the second term helps to maximize
the distance between subsequent moves in the Markov chain.

The sampler is then trained by minimizing $\ell_{\lambda}$ over both the target
and initialization distributions.
%
Explicitly, for an initial distribution $\pi_0$ over $\mathcal{X}$, we define
the initialization distribution as $q(\xi) = \pi_0(x) \mathcal{N}(v; 0, I)
p(d)$, and minimize
%
\begin{equation}
    \mathcal{L}(\theta)\equiv \mathbb{E}_{p(\xi)}\left[\ell_{\lambda}(\xi,
    \mathbf{FL}_{\theta}\xi, A(\mathbf{FL}_{\theta}\xi|\xi))\right] + \lambda_b
    \mathbb{E}_{q(\xi)}\left[\ell_{\lambda}(\xi, \mathbf{FL}_{\theta}\xi,
    A(\mathbf{FL}_{\theta} \xi| \xi))\right].
    \label{eq:loss_L}
\end{equation}
%
For completeness, we include the full algorithm~\cite{2017arXiv171109268L} used
to train L2HMC in Alg.~\ref{alg:l2hmc}.
%
\begin{algorithm}[htbp]%
  % \centering
    \SetKwProg{Fn}{def}{\string:}{}%
    \SetKwFunction{Range}{range}%
    \SetKwFor{For}{for}{\string:}{}%
    \SetKwIF{If}{ElseIf}{Else}{if}{:}{elif}{else:}{}%
    \SetKwFor{While}{while}{:}{fintq}%
    \AlgoDontDisplayBlockMarkers\SetAlgoNoEnd%
    % \SetAlgoNoLine%
    \DontPrintSemicolon%
    \SetKwInOut{Input}{input}\SetKwInOut{Output}{output}%
    \caption{Training procedure for the L2HMC algorithm.}%
    \Input{%
      \vspace{-5pt}
      \begin{enumerate}
        \item A (potential) energy function, $U: \mathcal{X} \rightarrow
          \mathbb{R}$ and its gradient $\nabla_x U: \mathcal{X} \rightarrow
          \mathcal{X}$\
          \vspace{-10pt}
        \item Initial distribution over the augmented state space, $q$
          \vspace{-10pt}
        \item Number of iterations, $N_{\mathrm{train}}$
          \vspace{-10pt}
        \item Number of leapfrog steps, $N_{\mathrm{LF}}$
          \vspace{-10pt}
        \item Learning rate schedule, ${(\alpha_{t})}_{t\leq N_{\text{train}}}$
          \vspace{-10pt}
        \item Batch size, $N_{\mathrm{samples}}$
          \vspace{-10pt}
        \item Scale parameter, $\lambda$
          \vspace{-10pt}
        \item Regularization strength, $\lambda_b$
      \end{enumerate}
    }\;
    \vspace{-15pt}
    Initialize the parameters of the sampler, $\theta$\;
    Initialize ${\{\xi_{p^{(i)}}\}}_{i\leq N_{\mathrm{samples}}}$ from
    $q{(\xi)}$\; \For{$t = 0$ \KwTo\ $N_{\mathrm{train}}$}{%
      Sample a minibatch, ${\left\{\xi_{q}^{(i)}\right\}}_{i\leq
      N_{\mathrm{samples}}}$ from $q{(\xi)}$.\; $\mathcal{L}\leftarrow 0$\;
      \For{$i = 1$ \KwTo$N_{\mathrm{LF}}$} {%
          $\xi_{p}^{(i)} \leftarrow\ \mathbf{R}\,\xi_p^{(i)}$\;
          $\mathcal{L} \,\,\,\,\leftarrow\mathcal{L} +
          \ell_{\lambda}\left(\xi_p^{(i)}, \FLq\xi_p^{(i)}, A
            (\FLq\xi^{(i)}_p|\xi^{(i)}_p)\right) + \lambda_b
            \ell_{\lambda}\left(\xi^{(i)}_q, \FLq\xi^{(i)}_q,
            A (\FLq\xi^{(i)}_q|\xi^{(i)}_q)\right)$\;
          $\xi_p^{(i)} \leftarrow \FLq\xi^{(i)}_p$ with probability
        $A(\FLq\xi^{(i)}_p|\xi^{(i)}_p)$\; }%
      \vspace{2pt}
      $\theta\ \leftarrow\ \theta-\alpha_t \nabla_{\theta} \mathcal{L}$\;
    }%
\label{alg:l2hmc}
\end{algorithm}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%