\documentclass[12pt]{article}

\usepackage{amsmath}
\usepackage{amssymb}

\title{GAVisTool\\User Manual\\Version 0.1}
\author{Spencer T. Parkin}

\newcommand{\G}{\mathbb{G}}
\newcommand{\V}{\mathbb{V}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\B}{\mathbb{B}}
\newcommand{\nvao}{o}
\newcommand{\nvai}{\infty}

\begin{document}
\maketitle

This is the manual for the GAVisTool program.  It is the quickest way to learn
how to use this software, provided the reader is already familiar with geometric
algebra and the conformal model.  For more information on these topics, see
"Geometric Algebra for Computer Science" by Dr. Leo Dorst, et. al.  Another good
source of information on these topics can be found in Dr. Christian Perwass's "Geometric
Algebra with Applications in Engineering."

\section{The Console Window}

When the GAVisTool program comes up, you are presented with two windows: the console
window and the canvas window.  Ignoring the canvas window for the moment, the console
window is where you will interact with the GA environment by inputting expressions in the
text box at the bottom, then seeing the evaluation of these expressions as output in the
main part of the window.

\subsection{The GAVisTool Language}

A design goal of the GAVisTool software was that of not requiring the user to learn a language
unique to the software.  On the contrary, the tool better serves its user as a means to learning
the language of geometric algebra and mathematics in general.  With that in mind, the language
recognized by GAVisTool deviates from the language of mathematics only where unique features
of the GAVisTool software are concerned.  In fact, if you're familiar with basic mathematics, then
you already know the entire syntax of the GAVisTool language.  Specificly,
its syntax consists of left and right unary operators, infix binary operator notation,
and function call syntax.  That's it.  It's not any more or less complicated than that.  All that
remains then for the user to learn is what unary and binary operators are supported,
what functions are available, and what types of numbers are the subject of all these things.

\subsubsection{Numbers}

The concept of number is abstracted in the software to mean whatever is applicable
to current environment.  As of this writing, the only environment presented to the
user is the GA environment, and so the numbers in this context are simply the elements
of any geometric algebra defined over the field of real numbers.  The specific GA being
used depends on what basis vectors the user chooses to mix into his or her
expressions.  The following table lists the currently recognized constants in the
GA environment.
\begin{equation*}
\begin{array}{ccl}
\mbox{Console} & \mbox{Math Symbol} & \mbox{Description} \\
\hline
\mbox{no} & \nvao & \mbox{Null vector at origin} \\
\mbox{ni} & \nvai & \mbox{Null vector at infinity} \\
\mbox{e0} & e_0 & \mbox{Unit X-axis Euclidean vector} \\
\mbox{e1} & e_1 & \mbox{Unit Y-axis Euclidean vector} \\
\mbox{e2} & e_2 & \mbox{Unit Z-axis Euclidean vector} \\
\mbox{I} & e_0\wedge e_1\wedge e_2\wedge \nvao\wedge \nvai & \mbox{Unit psuedo-scalar of 3D CGA} \\
\mbox{i} & e_0\wedge e_1\wedge e_2 & \mbox{Unit psuedo-scalar of 3D EGA} \\
\mbox{E0} & e_1\wedge e_2 & \mbox{Unit YZ-plane Euclidean 2-blade} \\
\mbox{E1} & e_2\wedge e_0 & \mbox{Unit XZ-plane Euclidean 2-blade} \\
\mbox{E2} & e_0\wedge e_1 & \mbox{Unit XY-plane Euclidean 2-blade}
\end{array}
\end{equation*}
The vectors $e_0$, $e_1$ and $e_2$ form a right-handed orthonormal basis for 3-dimensional
Euclidean space.

You may enter any one of the above variable names into the console to see it recognized
by the GA environment.  The first five symbols given in the table form the basis for the
geometric algebra used by the conformal model of 3-dimensional space.  The remaining
five variables are provided for convenience.

Also recognized by the GA environment are scalar literal values.  These are bits of
text such as "3.14159" and "0.12345" and so on.

\subsubsection{Unary and Binary Operators}

Many unary and binary operators are recognized by the GA enviornment, but the first
of note are simply those that will allow us to generate linear combinations of the basis
vectors given in the previous section.  This is done just as you would expect.
For example, try inputting the following expressions.
...

\subsubsection{Functions}

\subsubsection{Variables}

When the GA environment encounters a named number that it doesn't recognize
as a built-in constant, it treats it as a variable, and remembers variable values.
To store a value (a number) in a variable, use the assignment operator.

\section{The Canvas Window}

The console window by itself is useful for making calculations in geometric algebra, but the main
purpose of GAVisTool is to provide a way to visualize the results of your mathematical expressions.
This is where the canvas window comes in.

\subsection{Bind Targets}

Bind targets are things that appear in the canvas window and that are bound to a variable
in the GA environment.  The type of bind target and how its bound to the variable determine
how that bind target will interpret the value of the variable to which it is bound.  For example,
if we have a variable named $\mbox{sph}$ that we would like to visualize as a conformal
sphere, then we can input the expression $\mbox{bind\_sphere(sph)}$ to create and bind to it
a bind target in the canvas window that is designed to interpret $\mbox{sph}$ as a conformal sphere.
Once bound, any changes we make to the value of the variable $\mbox{sph}$ will be reflected
as changes to the way the sphere is manifested in the canvas window by the bind target.
Conversly, any changes we make to the bind target (a sphere in this case) will be reflected
as changes to the value of the variable $\mbox{sph}$ in the console window.  (We'll go over
canvas interaction in the next section.)

Another type of bind target is a user interface.  As a single element of GA may be characterized
by a number of properties, these properties may be manifested through a simple user interface.
These interfaces appear as floatling control panels that can be docked about the canvas window.
As of this writing, the only currently available bind target interface is a scalar interface which can
be bound to a scalar (real number) variable in the console window.

The following table lists the currently available bind target functions.
\begin{equation*}
\begin{array}{ll}
\mbox{Function} & \mbox{Description} \\
\hline
\mbox{bind\_point} & \mbox{Bind a variable to a conformal point} \\
\mbox{bind\_sphere} & \mbox{Bind a variable to a conformal sphere} \\
\mbox{bind\_circle} & \mbox{Bind a variable to a conformal circle} \\
\mbox{bind\_pointpair} & \mbox{Bind a variable to a conformal point-pair} \\
\mbox{bind\_line} & \mbox{Bind a variable to a conformal line} \\
\mbox{bind\_plane} & \mbox{Bind a variable to a conformal plane} \\
\mbox{bind\_flatpoint} & \mbox{Bind a variable to a conformal flat-point} \\
\mbox{bind\_scalar\_iface} & \mbox{Bind a variable to a scalar interface} \\
\end{array}
\end{equation*}
Left out of the table for ease of reading were functions such as $\mbox{bind\_dual\_point}$
and $\mbox{bind\_dual\_sphere}$.  These interpret elements of GA as dual points and dual
spheres, respectively.  A dual version of all bind target functions for conformal geometries
is provided.

Also left out of the table was a function named $\mbox{bind\_inferred\_geo}$.
A bind target for a specific conformal geometry assumes a certain form for
elements of GA it is given to interpret.  The inferred geometry bind target, however,
analyzes a given element first to determine what type of conformal geometry it
may represent, then interprets it as that geometry.  As of this writing, this
feature has not yet been implemented.

\subsection{Canvas Interaction}

Just as with the console window, the canvas window too provides both output and input.
Specifically, through the canvas window the user can interact with the set of
currently created bind targets.  Continuing with the conformal sphere example in the
previous section, right-clicking on this sphere in the canvas window will select it.  You can
then move the sphere in a plane orthogonal to the viewing direction by holding the
CTRL key and left mouse button down while dragging the mouse in the canvas window.
Holding the CTRL and SHIFT keys down while moving the mouse wheel will rotate
the selected bind target on an axis horizontal and perpendicular to the viewing direction,
and going through the center of the bind target's geometry.
(Notice that conformal spheres don't rotate, because they are invariant under rotations
about their center.)  Without the SHIFT key down, that same action will scale the
bind target's geometry up and down.

Right-clicking on no bind target's geometry unselects the currently selected bind target, if any.

\subsection{Canvas Camera Control}

Since geometry movement control is based on the camera orientation, we must have a way
of moving the camera around.  We would also like to be able to look at the scene from
different perpsectives.

Holding the left mouse button down while dragging the mouse in the canvas window allows
you to rotate the viewing location about the focal point.  Doing so while the ALT key is down allows you to pan
the camera in a plane orthogonal to the viewing direction, which also changes the focal point.  The focal
point is an invisible point in space in front of the camera.
If a bind target is selected, then hitting the SPACE bar will set the focal point of the camera to the
center of the bind target's geometry.  Subsequent rotations of the camera about the
focal point allow you to then view the bind target's geometry from different angles.

\section{Constraints}

The GAVisTool software maintains a set of constraints between the current
set of bind targets.  Like bind targets, until you actually create a constraint,
there are none maintained by the system.  A constraint names one or more
bind targets as its set of inputs and one or more bind targets as its set of outputs.
When a constraint executes, it pulls values from the variables bound to its
input bind targets, performs some sort of calculation and/or algorithm on this data,
and then pushes result values to the variables bound to its output bind targets.
A constraint is schedualed to execute when it is detected that one or more of
its inputs has changed.

One can imagine that with multiple constraints maintained by the system, these
form a dependency graph as one constraint's set of inputs may overlap with
another contraint's set of outputs.  Indeed, when a contraint executes, it may
cause other constraints to execute as well.  There is no need to worry about
creating circular dependencies.  When a single bind target is changed, an entire
chain of constraints is schedualed for execution before actually being executed.
If a constraint can't be schedualed for execution, (due to a circular dependency), it is simply not
executed.  This schedualling failure is not necessarily a design flaw on the part
of the author of the constraint system.  When rigging together a bunch of bind
targets with constraints, it is not uncommon to constrain two bind targets to
one another using two mutually dependent constraints.

As of this writing, there is currently only one type of constraint that can be
created, but it is a powerful one.  It is called a formulated constraint.

\section{Scripts}

\end{document}


\section{Orthogonality}

\begin{outcome}
  \begin{enumerate}
  \item Determine whether two vectors in an inner product space are
    orthogonal.
  \item Find the orthogonal complement of a set of vectors.
  \item Determine whether a set of vectors is orthogonal and/or orthonormal.
  \item Check whether a basis is orthogonal and/or orthonormal.
  \item Calculate the Fourier coefficients of a vector with respect to
    an orthogonal basis.
  \end{enumerate}
\end{outcome}

\begin{definition}{Orthogonality}{inner-product-orthogonality}
  Let $\vect{u}$ and $\vect{v}$ be vectors in an inner product space.
  We say that $\vect{u}$ and $\vect{v}$ are \textbf{orthogonal}%
  \index{vector!orthogonal}%
  \index{orthogonal vectors} if
  \begin{equation*}
    \iprod{\vect{u}, \vect{v}} = 0.
  \end{equation*}
  We also write $\vect{u}\orth\vect{v}$ to indicate that $\vect{u}$
  and $\vect{v}$ are orthogonal.
\end{definition}

We note that the zero vector $\vect{0}$ is orthogonal to all vectors,
because $\iprod{\vect{u},\vect{0}} = 0$ follows from the linearity of
the inner product. We also note that $\vect{u}\orth\vect{v}$ if and
only if $\vect{v}\orth\vect{u}$; this follows from the symmetry of the
inner product.

\begin{definition}{Orthogonal complement}{inner-product-orthogonal-complement}
  Let $S$ be a subset of an inner product space $V$. The
  \textbf{orthogonal complement}%
  \index{orthogonal complement} of $S$ is the set
  \begin{equation*}
    S^{\orth} = \set{\vect{v}\in V \mid
      \mbox{
        $\iprod{\vect{v},\vect{w}} = 0$ for all $\vect{w}\in S$
      }}.
  \end{equation*}
\end{definition}

\begin{proposition}{Orthogonal complement}{inner-product-orthogonal-complement}
  If $S$ is any subset of an inner product space $V$, then $S^{\orth}$
  is a subspace.
\end{proposition}

\begin{proof}
  We clearly have $\vect{0}\in S^{\orth}$, because $\vect{0}$ is
  orthogonal to all vectors. To show that $S^{\orth}$ is closed under
  addition, assume $\vect{v}, \vect{v}'\in S^{\orth}$. We have to show
  $\vect{v}+\vect{v}'\in S^{\orth}$. So take an arbitrary $\vect{w}\in
  S$. Then we have
  \begin{equation*}
    \iprod{\vect{v}+\vect{v}',\vect{w}}
    = \iprod{\vect{v},\vect{w}} + \iprod{\vect{v}',\vect{w}}
    = 0 + 0 = 0.
  \end{equation*}
  Therefore, $\vect{v}+\vect{v}'\in S^{\orth}$. Finally, to show that
  $S^{\orth}$ is closed under scalar multiplication, assume
  $\vect{v}\in S^{\orth}$ and $k\in\R$. We have to show
  $k\vect{v}\in S^{\orth}$. So take an arbitrary
  $\vect{w}\in S$. Then we have
  \begin{equation*}
    \iprod{k\vect{v},\vect{w}}
    = k\iprod{\vect{v},\vect{w}}
    = k0 = 0.
  \end{equation*}
  Therefore, $k\vect{v}\in S^{\orth}$. It follows that $S^{\orth}$ is
  a subspace of $V$.
\end{proof}

Here is an illustration of a subspace $S$ of $\R^3$ and its orthogonal
complement $S^{\orth}$:

\begin{center}
  \begin{tikzpicture}[x={(1cm,0cm)},y={(1cm,0.5cm)},z={(0cm,1cm)},rotate=0]
    \draw[thick,blue!80!black](-2,0,0) -- (2,0,0);
    \filldraw[draw=red!80,fill=red!10](0,-1,-1.5) -- (0,-1,1.5) --
    (0,1,1.5) -- (0,1,-1.5) -- cycle;
    \path[red!80] (0,0,0.5) node[above] {$S$};
    \draw[thick,blue!80!black](0,0,0) -- node[above, pos=0.7] {$S^{\orth}$} (2,0,0);
  \end{tikzpicture}
\end{center}

\begin{example}{Orthogonal complement}{inner-product-orthogonal-complement}
  Consider the inner product space $\Poly_3$ of polynomials of degree
  at most $3$, with the inner product defined by
  \begin{equation*}
    \iprod{f,g} = \int_{-1}^{1} f(x)g(x)\,dx.
  \end{equation*}
  Find the orthogonal complement of $\set{x^2}$.
\end{example}

\begin{solution}
  We have to compute the set of all polynomials of the form
  $p(x)=ax^3+bx^2+cx+d$ that are orthogonal to $x^2$. So let us
  compute the inner product:
  \begin{eqnarray*}
    \iprod{p(x), x^2}
    &=& \int_{-1}^{1} (ax^3+bx^2+cx+d)x^2\,dx \\
    &=& \int_{-1}^{1} ax^5+bx^4+cx^3+dx^2\,dx \\
    &=& 0a  + \frac{2}{5}b + 0c + \frac{2}{3}d.
  \end{eqnarray*}
  Setting this equal to $0$, we see that $\iprod{p(x), x^2}=0$ if and
  only if $\frac{2}{5}b + \frac{2}{3}d = 0$, or equivalently,
  $3b+5d=0$. The basic solutions are
  \begin{equation*}
    \begin{mymatrix}{c} a \\ b \\ c \\ d \end{mymatrix}
    = \begin{mymatrix}{c} 1 \\ 0 \\ 0 \\ 0 \end{mymatrix},\quad
    \begin{mymatrix}{c} a \\ b \\ c \\ d \end{mymatrix}
    = \begin{mymatrix}{c} 0 \\ 5 \\ 0 \\ -3 \end{mymatrix},\quad
    \begin{mymatrix}{c} a \\ b \\ c \\ d \end{mymatrix}
    = \begin{mymatrix}{c} 0 \\ 0 \\ 1 \\ 0 \end{mymatrix},
  \end{equation*}
  giving the following basis for the space of polynomials orthogonal
  to $x^2$:
  \begin{equation*}
    \set{x^3,\quad 5x^2-3, \quad x}.
  \end{equation*}
\end{solution}

We now consider orthogonal sets and bases.

\begin{definition}{Orthogonal and orthonormal sets of vectors}{orthogonal-set}
  A set of vectors $\set{\vect{u}_1,\ldots,\vect{u}_k}$ in an inner
  product space is called an \textbf{orthogonal set}%
  \index{orthogonal set} if the vectors are non-zero and pairwise
  orthogonal, i.e., for all $i$, $\vect{u}_i\neq\vect{0}$ and for all
  $i\neq j$, $\vect{u}_i\orth\vect{u}_j$.
  \smallskip\smallskip

  Moreover, the set of vectors $\set{\vect{u}_1,\ldots,\vect{u}_k}$ is
  called \textbf{orthonormal}%
  \index{orthonormal set} if it is orthogonal and each vector is
  normalized, i.e., $\norm{\vect{u}_i}=1$.
\end{definition}

The interest of orthogonal and orthonormal sets of vectors lies, among
other things, in the fact that they are automatically linearly
independent.

\begin{proposition}{Orthogonal set is linearly independent}{orthogonal-linear-independence}
  If $\set{\vect{u}_1,\ldots,\vect{u}_k}$ is an orthogonal set of
  vectors, then $\vect{u}_1,\ldots,\vect{u}_k$ are linearly
  independent.
\end{proposition}

\begin{proof}
  Assume $\set{\vect{u}_1,\ldots,\vect{u}_k}$ is an orthogonal set. To
  show that $\vect{u}_1,\ldots,\vect{u}_k$ are linearly
  independent, assume
  \begin{equation*}
    a_1\vect{u}_1 + \ldots + a_k\vect{u}_k = \vect{0}.
  \end{equation*}
  We must show that $a_1,\ldots,a_k = 0$. So pick some
  $i\in\set{1,\ldots,k}$. We compute
  \begin{eqnarray*}
    \iprod{a_1\vect{u}_1 + \ldots + a_k\vect{u}_k, \vect{u}_i}
    &=& a_1\iprod{\vect{u}_1,\vect{u}_i}
        + \ldots
        + a_i\iprod{\vect{u}_i,\vect{u}_i}
        + \ldots
        + a_k\iprod{\vect{u}_k,\vect{u}_i} \\
    &=& 0
        + \ldots
        + a_i\iprod{\vect{u}_i,\vect{u}_i}
        + \ldots
        + 0 \\
    &=& a_i \iprod{\vect{u}_i,\vect{u}_i}.
  \end{eqnarray*}
  On the other hand,
  \begin{equation*}
    \iprod{a_1\vect{u}_1 + \ldots + a_k\vect{u}_k, \vect{u}_i}
    = \iprod{\vect{0}, \vect{u}_i} = 0.
  \end{equation*}
  It follows that $a_i \iprod{\vect{u}_i,\vect{u}_i} = 0$. Since
  $\iprod{\vect{u}_i,\vect{u}_i}\neq 0$, it follows that
  $a_i=0$. Since the choice of $i$ was arbitrary, it follows that
  $a_1,\ldots,a_k = 0$, and $\vect{u}_1,\ldots,\vect{u}_k$ are
  linearly independent.
\end{proof}

\begin{example}{An orthogonal set of functions}{orthogonal-set-sin-cos}
  Consider the inner product space $C[0,2\pi]$. The following
  functions form an (infinite) orthogonal set:
  \begin{eqnarray*}
    g_0(x) &=& 1 \\
    f_1(x) &=& \sin x \\
    g_1(x) &=& \cos x \\
    f_2(x) &=& \sin 2x \\
    g_2(x) &=& \cos 2x \\
    &\vdots& \\
    f_k(x) &=& \sin kx \\
    g_k(x) &=& \cos kx \\
    &\vdots&
  \end{eqnarray*}
\end{example}

\begin{proof}
  We have to check that any two functions are orthogonal to each
  other. This is true because of trigonometric identities. We have the
  following trigonometric formulas:
  \begin{eqnarray*}
    \sin\alpha\sin\beta &=& \frac{1}{2}(\cos(\alpha-\beta) - \cos(\alpha+\beta)), \\
    \cos\alpha\cos\beta &=& \frac{1}{2}(\cos(\alpha-\beta) + \cos(\alpha+\beta)), \\
    \sin\alpha\cos\beta &=& \frac{1}{2}(\sin(\alpha-\beta) + \sin(\alpha+\beta)).
  \end{eqnarray*}
  Using these, we can compute the relevant inner products. Assume $i\neq j$. Then
  \begin{eqnarray*}
    \iprod{g_i,g_j}
    &=& \int_{0}^{2\pi} \cos(ix)\cos(jx)\,dx \\
    &=& \frac{1}{2}\int_{0}^{2\pi} \cos((i-j)x) + \cos((i+j)x)\,dx \\
    &=& \frac{1}{2}\bigbracket{\frac{1}{(i-j)} \sin((i-j)x) + \frac{1}{(i+j)}
      \sin((i+j)x)}_0^{2\pi} \\
    &=& 0,
  \end{eqnarray*}
  and therefore $g_i\orth g_j$. The proofs of $f_i\orth f_j$ and
  $f_i\orth g_j$ are similar.
\end{proof}

\begin{definition}{Orthogonal and orthonormal bases}{orthogonal-basis}
  Let $V$ be an inner product space, and let $W$ be a subspace of
  $V$. We say that $\set{\vect{u}_1,\ldots,\vect{u}_k}$ is an
  \textbf{orthogonal basis}%
  \index{orthogonal basis}%
  \index{basis!orthogonal} for $W$ if it is an orthogonal set and
  spans $W$.
  \smallskip\smallskip

  If, moreover, each $\vect{u}_i$ is normalized, we say that
  $\set{\vect{u}_1,\ldots,\vect{u}_k}$ is an \textbf{orthonormal
    basis}%
  \index{orthonormal basis}%
  \index{basis!orthonormal} for $W$.
\end{definition}

We note that, by
Proposition~\ref{prop:orthogonal-linear-independence}, every
orthogonal (or orthonormal) basis is automatically linearly
independent, and therefore an actual basis of $W$.

\begin{example}{Orthogonal, orthonormal, and non-orthogonal bases}{orthogonal-basis}
  Consider $\R^2$ as an inner product space with the usual dot product.
  \begin{itemize}
  \item $\set{\begin{mymatrix}{r} 1 \\ 0 \end{mymatrix},
      \begin{mymatrix}{r} 0 \\ 1 \end{mymatrix}}$ is an orthonormal
    basis of $\R^2$.
  \item
    $\set{\frac{1}{\sqrt{2}}\begin{mymatrix}{r} 1 \\ 1 \end{mymatrix},
      \frac{1}{\sqrt{2}}\begin{mymatrix}{r} 1 \\ -1 \end{mymatrix}}$
    is an orthonormal basis of $\R^2$.
  \item $\set{\begin{mymatrix}{r} 2 \\ 0 \end{mymatrix},
      \begin{mymatrix}{r} 0 \\ 3 \end{mymatrix}}$ is an orthogonal
    basis of $\R^2$, but not orthonormal.
  \item $\set{\begin{mymatrix}{r} 1 \\ 1 \end{mymatrix},
      \begin{mymatrix}{r} 1 \\ -1 \end{mymatrix}}$ is an orthogonal
    basis of $\R^2$, but not orthonormal.
  \item $\set{\begin{mymatrix}{r} 1 \\ 0 \end{mymatrix},
      \begin{mymatrix}{r} 1 \\ 1 \end{mymatrix}}$ is a basis of
    $\R^2$, but neither orthogonal nor orthonormal.
  \end{itemize}
\end{example}

So why are we interested in orthogonal and orthonormal bases? The main
reason is that finding coordinates is {\em much easier} when the bases
are orthogonal (and even better when they are orthonormal). We have
the following property:

\begin{proposition}{Fourier coefficients}{fourier-coefficients}
  Let $B=\set{\vect{u}_1,\ldots,\vect{u}_n}$ be an orthogonal basis of
  some space $W$, and suppose $\vect{v}\in W$. Then
  \begin{equation*}
    \vect{v} = a_1\vect{u}_1 + \ldots + a_n\vect{u}_n,
  \end{equation*}
  where
  \begin{equation*}
    a_i = \frac{\iprod{\vect{u}_i,\vect{v}}}{\iprod{\vect{u}_i,\vect{u}_i}}.
  \end{equation*}
  In this situation, the coordinates $a_1,\ldots,a_n$ are also called
  the \textbf{Fourier coefficients}%
  \index{Fourier coefficients} of $\vect{v}$ (with respect to the
  orthogonal basis $B$).
  \smallskip\smallskip

  In case $B$ is an orthonormal basis, the formula is even simpler. In
  that case:
    \begin{equation*}
    a_i = \iprod{\vect{u}_i,\vect{v}}.
  \end{equation*}
\end{proposition}

\begin{proof}
  Since $B$ is a basis of $W$, we know that there exist coefficients
  $a_1,\ldots,a_n$ such that
  $\vect{v} = a_1\vect{u}_1 + \ldots + a_n\vect{u}_n$. It remains to
  verify that the coefficients satisfy the required formula. This is a
  simple calculation. We have
  \begin{eqnarray*}
    \iprod{\vect{u}_i,\vect{v}}
    &=& \iprod{\vect{u}_i, a_1\vect{u}_1 + \ldots + a_n\vect{u}_n} \\
    &=& a_1\iprod{\vect{u}_i,\vect{u}_1}
        + \ldots
        + a_i\iprod{\vect{u}_i,\vect{u}_i}
        + \ldots
        + a_n\iprod{\vect{u}_i,\vect{u}_n} \\
    &=& 0
        + \ldots
        + a_i\iprod{\vect{u}_i,\vect{u}_i}
        + \ldots
        + 0 \\
    &=& a_i \iprod{\vect{u}_i,\vect{u}_i}.
  \end{eqnarray*}
  Since $\vect{u}_i\neq \vect{0}$, we have
  $\iprod{\vect{u}_i,\vect{u}_i}\neq 0$ by the positive definite
  property. We can therefore divide both sides of the equation by
  $\iprod{\vect{u}_i,\vect{u}_i}$ to obtain
  \begin{equation*}
    a_i = \frac{\iprod{\vect{u}_i,\vect{v}}}{\iprod{\vect{u}_i,\vect{u}_i}},
  \end{equation*}
  as desired. Finally, if the basis is orthonormal, then
  $\iprod{\vect{u}_i,\vect{u}_i}=1$, and the denominator disappears,
  so that $a_i = \iprod{\vect{u}_i,\vect{v}}$.
\end{proof}

\begin{example}{Fourier coefficients}{fourier-coefficients}
  Suppose $B=\set{\vect{u}_1,\vect{u}_2,\vect{u}_3}$ is an orthogonal
  basis for an inner product space $V$, such that
  $\norm{\vect{u}_1}=1$, $\norm{\vect{u}_2}=\sqrt{2}$, and
  $\norm{\vect{u}_3}=2$.  Moreover, suppose that $\vect{v}\in V$ is a
  vector such that $\iprod{\vect{v},\vect{u}_1} = 3$,
  $\iprod{\vect{v},\vect{u}_2} = -1$, and
  $\iprod{\vect{v},\vect{u}_3} = 2$.  Find the coordinates of
  $\vect{v}$ with respect to $B$.
\end{example}

\begin{solution}
  We have to find $a_1,a_2,a_3$ such that
  \begin{equation*}
    \vect{v} = a_1\vect{u}_1 + a_2\vect{u}_2 + a_3\vect{u}_3.
  \end{equation*}
  We have sufficient information to compute $a_1,a_2,a_3$. Namely,
  \begin{eqnarray*}
    a_1
    &=& \frac{\iprod{\vect{u}_1,\vect{v}}}{\iprod{\vect{u}_1,\vect{u}_1}}
        ~=~ \frac{\iprod{\vect{u}_1,\vect{v}}}{\norm{\vect{u}_1}^2}
        ~=~ \frac{3}{1} ~=~ 3, \\
    a_2
    &=& \frac{\iprod{\vect{u}_2,\vect{v}}}{\iprod{\vect{u}_2,\vect{u}_2}}
        ~=~ \frac{\iprod{\vect{u}_2,\vect{v}}}{\norm{\vect{u}_2}^2}
        ~=~ \frac{-1}{2} ~=~ -0.5, \\
    a_3
    &=& \frac{\iprod{\vect{u}_3,\vect{v}}}{\iprod{\vect{u}_3,\vect{u}_3}}
        ~=~ \frac{\iprod{\vect{u}_3,\vect{v}}}{\norm{\vect{u}_3}^2}
        ~=~ \frac{2}{4} ~=~ 0.5.
  \end{eqnarray*}
\end{solution}

Proposition~\ref{prop:fourier-coefficients} shows that if
$\set{\vect{u}_1,\ldots,\vect{u}_n}$ is an orthogonal basis, then we
can solve $\vect{v} = a_1\vect{u}_1 + \ldots + a_n\vect{u}_n$ without
having to solve a system of equations. While this is a useful thing to
be able to do, perhaps it is merely a convenience (solving a system of
equations would also be fine). However, there is another very useful
property of Fourier coefficients. The coefficient $a_i$ only depends
on the basis vector $\vect{u}_i$, and not on any of the other basis
vectors. This is useful in situations where only {\em part} of an
orthogonal basis is known. We can calculate the corresponding
coefficients without having to know the rest of the basis.

\begin{example}{Finding partial coordinates}{partial-coordinates}
  Suppose $B=\set{\vect{u}_1,\vect{u}_2,\vect{u}_3,\vect{u}_4}$ is an
  orthogonal basis of $\R^4$. We have been told that
  \begin{equation*}
    \vect{u}_1 = \begin{mymatrix}{c} 1 \\ 2 \\ -1 \\ 0 \end{mymatrix}
    \quad\mbox{and}\quad
    \vect{u}_2 = \begin{mymatrix}{c} 0 \\ 1 \\ 2 \\ 3 \end{mymatrix},
  \end{equation*}
  but it is not known what $\vect{u}_3$ and $\vect{u}_4$ are. Find the
  first two coordinates of the vector
  \begin{equation*}
    \vect{v} = \begin{mymatrix}{c} 1 \\ 0 \\ 2 \\ 0 \end{mymatrix}
  \end{equation*}
  with respect to the basis $B$.
\end{example}

\begin{solution}
  We have $\vect{v} = a_1\vect{u}_1 + a_2\vect{u}_2 + a_3\vect{u}_3 +
  a_4\vect{u}_4$, where
  \begin{equation*}
    a_1
    = \frac{\iprod{\vect{u}_1,\vect{v}}}{\iprod{\vect{u}_1,\vect{u}_1}}
    = \frac{-1}{6}
    \quad\mbox{and}\quad
    a_2
    = \frac{\iprod{\vect{u}_2,\vect{v}}}{\iprod{\vect{u}_2,\vect{u}_2}}
    = \frac{4}{14}.
  \end{equation*}
  So the first two coordinates are $-\frac{1}{6}$ and $\frac{2}{7}$.
\end{solution}

A word of warning is in order: Fourier coefficients do not work when
the basis is not orthogonal. Consider the following picture, where
$\vect{u}_1$ and $\vect{u}_2$ are not orthogonal.
\begin{equation*}
  \begin{tikzpicture}[scale=2]
    \draw (-2,0) -- (3,0);
    \draw (-0.2,-0.4) -- (1.1,2.2);
    \draw[thick,red,fill=yellow!20] (0,0) -- (2,0) -- (2.5,1) -- (0.5,1) -- cycle;
    \draw[dashed,green!70!black] (2.5,1) -- (2.5,0) node[below]{$a_1$};
    \draw[dashed,green!70!black] (2.5,1) -- (0.9,1.8) node[left,yshift=3]{$a_2$};
    \draw[thick,blue,->] (0,0) -- node[below]{$\vect{u}_1$} (1,0);
    \draw[thick,blue,->] (0,0) -- node[left, pos=0.55]{$\vect{u}_2$} (0.5,1);
    \draw[thick,blue,->] (0,0) -- node[above]{$\vect{v}$} (2.5,1);
  \end{tikzpicture}
\end{equation*}
The {\em coordinates} of $\vect{v}$ with respect to
$\vect{u}_1,\vect{u}_2$ are $(2,1)$, because
$\vect{v}=2\vect{u}_1+1\vect{u}_2$, as indicated by the shaded
parallelogram. On the other hand, the formula for the {\em Fourier
  coefficients} is concerned with the orthogonal projections of
$\vect{v}$ onto $\vect{u}_1$ and $\vect{u}_2$, as indicated by the
dashed lines. It yields the coefficients
\begin{eqnarray*}
  a_1 &=& \frac{\iprod{\vect{u}_1,\vect{v}}}{\iprod{\vect{u}_1,\vect{u}_1}} ~\approx~ 2.5, \\
  a_2 &=& \frac{\iprod{\vect{u}_2,\vect{v}}}{\iprod{\vect{u}_2,\vect{u}_2}} ~\approx~ 1.8.
\end{eqnarray*}
These are not the same as the coordinates of $\vect{v}$, because the
dashed lines are orthogonal to $\vect{u}_1$ and $\vect{u}_2$, instead
of parallel to them. In summary, the Fourier coefficients of a vector
are equal to its coordinates {\em only if the basis is orthogonal}.

When $\vect{u}_1,\ldots,\vect{u}_k$ are orthogonal, we have a
convenient formula for the norm of
$a_1\vect{u}_1+\ldots+a_k\vect{u}_k$:

\begin{proposition}{Norm of orthogonal linear combination}{norm-orthogonal}
  Let $V$ be an inner product space, and suppose that
  $\vect{u}_1,\ldots,\vect{u}_k$ are orthogonal. Then
  \begin{equation*}
    \norm{a_1\vect{u}_1+\ldots+a_k\vect{u}_k}^2
    = a_1^2\norm{\vect{u}_1}^2 + \ldots + a_k^2\norm{\vect{u}_k}^2.
  \end{equation*}
\end{proposition}

\begin{proof}
  We have
  \begin{eqnarray*}
    \norm{a_1\vect{u}_1+\ldots+a_k\vect{u}_k}^2
    &=&
    \iprod{a_1\vect{u}_1+\ldots+a_k\vect{u}_k,a_1\vect{u}_1+\ldots+a_k\vect{u}_k} \\
    &=& \begin{array}[t]{@{}c@{~}c@{~}c@{~}c@{~}c@{~}c@{~}c@{~}c}
          &   a_1^2\iprod{\vect{u}_1,\vect{u}_1}
          &+& a_1a_2\iprod{\vect{u}_1,\vect{u}_2}
          &+& \ldots
          &+& a_1a_n\iprod{\vect{u}_1,\vect{u}_n} \\
          +&  a_2a_1\iprod{\vect{u}_2,\vect{u}_1}
          &+& a_2^2\iprod{\vect{u}_2,\vect{u}_2}
          &+& \ldots
          &+& a_2a_n\iprod{\vect{u}_2,\vect{u}_n} \\
          +&  \vdots
          && \vdots
          &&
          && \vdots \\
          +&  a_na_1\iprod{\vect{u}_n,\vect{u}_1}
          &+& a_na_2\iprod{\vect{u}_n,\vect{u}_2}
          &+& \ldots
          &+& a_n^2\iprod{\vect{u}_n,\vect{u}_n}
        \end{array} \\
    &=& a_1^2\norm{\vect{u}_1}^2 + \ldots + a_k^2\norm{\vect{u}_k}^2.
  \end{eqnarray*}
  Here we have used the fact that $\iprod{\vect{u}_i,\vect{u}_j}=0$
  when $i\neq j$.
\end{proof}

We conclude this section by making a connection between arbitrary
inner products and dot products.  Once an orthonormal basis has been
chosen on an inner product space, computing inner products is
essentially the same as computing dot products. The following
proposition makes this more precise.

\begin{proposition}{Inner product and dot product}{inner-product-dot-product}
  Let $V$ be an inner product space, and suppose that
  $B=\set{\vect{u}_1,\ldots\vect{u}_n}$ is an orthonormal basis. Then
  for any pair of vectors $\vect{v},\vect{w}\in V$, their inner
  product is equal to the dot product of their coordinate vectors with
  respect to the basis $B$, i.e.
  \begin{equation*}
    \iprod{\vect{v},\vect{w}}
    = \coord{\vect{v}}_B \dotprod \coord{\vect{w}}_B.
  \end{equation*}
\end{proposition}

\begin{proof}
  By definition of coordinate vectors, we have
  \begin{equation*}
    \coord{\vect{v}}_B
    = \begin{mymatrix}{c} a_1 \\ \vdots \\ a_n \end{mymatrix}
    \quad\mbox{and}\quad
    \coord{\vect{w}}_B
    = \begin{mymatrix}{c} b_1 \\ \vdots \\ b_n \end{mymatrix},
  \end{equation*}
  where $\vect{v} = a_1\vect{u}_1 + \ldots + a_n\vect{u}_n$
  and $\vect{w} = b_1\vect{u}_1 + \ldots + b_n\vect{u}_n$.
  We calculate
  \begin{eqnarray*}
    \iprod{\vect{v},\vect{w}}
    &=& \iprod{a_1\vect{u}_1 + \ldots + a_n\vect{u}_n, b_1\vect{u}_1 +
        \ldots + b_n\vect{u}_n} \\
    &=& \begin{array}[t]{@{}c@{~}c@{~}c@{~}c@{~}c@{~}c@{~}c@{~}c}
          &   a_1b_1\iprod{\vect{u}_1,\vect{u}_1}
          &+& a_1b_2\iprod{\vect{u}_1,\vect{u}_2}
          &+& \ldots
          &+& a_1b_n\iprod{\vect{u}_1,\vect{u}_n} \\
          +&  a_2b_1\iprod{\vect{u}_2,\vect{u}_1}
          &+& a_2b_2\iprod{\vect{u}_2,\vect{u}_2}
          &+& \ldots
          &+& a_2b_n\iprod{\vect{u}_2,\vect{u}_n} \\
          +&  \vdots
          && \vdots
          &&
          && \vdots \\
          +&  a_nb_1\iprod{\vect{u}_n,\vect{u}_1}
          &+& a_nb_2\iprod{\vect{u}_n,\vect{u}_2}
          &+& \ldots
          &+& a_nb_n\iprod{\vect{u}_n,\vect{u}_n}
        \end{array} \\
    &=& a_1b_1 + a_2b_2 + \ldots + a_nb_n \\
    &=& \coord{\vect{v}}_B \dotprod \coord{\vect{w}}_B.
  \end{eqnarray*}
  Here we have used the fact that $\iprod{\vect{u}_i,\vect{u}_j}=1$
  when $i=j$ and $\iprod{\vect{u}_i,\vect{u}_j}=0$ otherwise, which
  holds because $B$ is orthonormal.
\end{proof}


\input{../header_function}

%---------- start document ---------- %
 \section{multiplicative -- multiplicative number theoretic functions}\linkedzero{multiplicative}
%
All functions of this module accept only positive integers,
unless otherwise noted.
%
  \subsection{euler -- the Euler totient function}\linkedone{multiplicative}{euler}
   \func{euler}
   {%
     \hiki{n}{integer}
   }{%
     \out{integer}
   }\\
   \spacing
   % document of basic document
   \quad Return the number of numbers relatively prime to \param{n}
   and smaller than \param{n}.  In the literature, the function is
   referred often as \(\varphi\).

  \subsection{moebius -- the M\"obius function}\linkedone{multiplicative}{moebius}
   \func{moebius}
   {%
     \hiki{n}{integer}
   }{%
     \out{integer}
   }\\
   \spacing
   % document of basic document
   \quad Return:
   \begin{description}
   \item[-1] if n has odd distinct prime factors,
   \item[1] if n has even distinct prime factors, or
   \item[0] if n has a squared prime factor.
   \end{description}
   In the literature, the function is referred often as \(\mu\).

  \subsection{sigma -- sum of divisor powers)}\linkedone{multiplicative}{sigma}
   \func{sigma}
   {%
     \hiki{m}{integer},\ %
     \hiki{n}{integer}
   }{%
     \out{integer}
   }\\
   \spacing
   % document of basic document
   Return the sum of \param{m}-th powers of the factors of \param{n}.
   The argument \param{m} can be zero, then return the number of factors.
   In the literature, the function is referred often as \(\sigma\).
%
\begin{ex}
>>> multiplicative.euler(1)
1
>>> multiplicative.euler(2)
1
>>> multiplicative.euler(4)
2
>>> multiplicative.euler(5)
4
>>> multiplicative.moebius(1)
1
>>> multiplicative.moebius(2)
-1
>>> multiplicative.moebius(4)
0
>>> multiplicative.moebius(6)
1
>>> multiplicative.sigma(0, 1)
1
>>> multiplicative.sigma(1, 1)
1
>>> multiplicative.sigma(0, 2)
2
>>> multiplicative.sigma(1, 3)
4
>>> multiplicative.sigma(1, 4)
7
>>> multiplicative.sigma(1, 6)
12L
>>> multiplicative.sigma(2, 7)
50
\end{ex}%Don't indent!(indent causes an error.)
\C

%---------- end document ---------- %

\input{../footer}


% Author: Jonathan Siegel
% This chapter aims to provide an introduction to the online learning framework.
% We consider only the convex problems and first-order (i.e. gradient-based) methods.

\section{Online Optimization}
In this chapter we introduce the concept of online learning.

In the online setting, we observe a sequence of objective functions $f_i$. We generate iterates $x_i$ based on the
observed functions $f_i$ with the goal of minimizing the regret $R(x,z^*)$ where
\begin{equation}
 R(x_1,...,x_n,z) = \displaystyle\sum_{i = 1}^n (f_i(x_i) - f_i(z))
\end{equation}
and $z^*\in\arg\min_{z} \displaystyle\sum_{i = 1}^n f_i(z)$. Notice the point $z$ is fixed in hindsight while the $x_i$ vary.

We begin by deriving regret bounds for the dual averaging method in the convex, Lipschitz case. The dual averaging method
is given by
\begin{equation} \label{online_DA}
 x_{n + 1} = \arg\min_x \left(\displaystyle\sum_{i = 1}^n \langle sg_i, x\rangle + \frac{\alpha_{n+1}}{2}\|x - x_1\|_2^2\right)
\end{equation}
where $g_i\in \partial f_i(x_i)$. Note that if $\alpha_i = 1$, then we simply have subgradient descent with
stepsize $s$. We have the following theorem.
\begin{theorem}
 Assume that each $f_i$ is convex and Lipschitz with constant $M$. Let $x_i$ be given by iteration (\ref{online_DA})
 with $\alpha_i \leq \alpha_{i+1}$.
 Then for any $z$, we have
 \begin{equation}
  R(x_1,...,x_n,z) \leq \frac{\alpha_n}{2s} \|z - x_1\|_2^2 + \frac{sM^2}{2}\displaystyle\sum_{i = 1}^n \frac{1}{\alpha_i}
 \end{equation}
 \end{theorem}
 \begin{proof}
  We note that by convexity, we have
  \begin{equation}\label{convexity_bound}
   R(x_1,...,x_n,z) = \displaystyle\sum_{i = 1}^n (f_i(x_i) - f_i(z)) \leq \displaystyle\sum_{i = 1}^n\langle g_i, x_i - z\rangle
   = \frac{1}{s}\displaystyle\sum_{i = 1}^n\langle sg_i, x_i - z\rangle
  \end{equation}
  We proceed by rewriting the sum as follows
  \begin{equation}\label{sum_rewriting}
   \displaystyle\sum_{i = 1}^n\langle sg_i, x_i - z\rangle = \displaystyle\sum_{i = 1}^n\langle sg_i, x_n - z\rangle +
   \displaystyle\sum_{i = 1}^{n-1} \displaystyle\sum_{j = 1}^i \langle sg_j, x_i - x_{i+1}\rangle
  \end{equation}
  We now bound the quantity
  \begin{equation}
   \displaystyle\sum_{j = 1}^i \langle sg_j, x_i - y\rangle
  \end{equation}
  for arbitrary $y$ as follows. We note that by (\ref{online_DA}), we have
  \begin{equation}
   x_i = \arg\min_x \displaystyle\sum_{j = 1}^{i-1} \langle sg_j, x\rangle + \frac{\alpha_i}{2}\|x - x_1\|_2^2 = \arg\min_x f_i(x)
  \end{equation}
  This means that $0\in \partial f_i(x_i)$ and so $sg_i\in \partial f_i^\prime(x_i)$ where
  $f_i^\prime(x) = f_i(x) + \langle sg_i, x\rangle$. Note additionally that $f_i$ and thus $f_i^\prime$ are strongly
  convex with convexity parameter $\alpha_i$. This implies that for any $y$,
  \begin{equation}
   f_i^\prime(x_i) - f_i^\prime(y) \leq \frac{1}{2\alpha_i}\|sg_i\|_2^2 \leq \frac{s^2M^2}{2\alpha_i}
  \end{equation}
  since $\|g_i\|_2\leq M$ by the Lipschitz assumption on $f_i$. Plugging in $y = x_{i+1}$ and $y = z$, and
  expanding $f_i^\prime$, we see that
  \begin{equation}
   (\ref{sum_rewriting})\leq \frac{\alpha_n}{2} \|z - x_1\|_2^2
   + \frac{1}{2}\displaystyle\sum_{i = 2}^n (\alpha_{i-1} - \alpha_i)\|x_i - x_1\|_2^2 + \frac{s^2M^2}{2}\displaystyle\sum_{i = 1}^n \frac{1}{\alpha_i}
  \end{equation}
  Now the assumption that $\alpha_{i-1} - \alpha_i\leq 0$ implies that
  \begin{equation}
   (\ref{sum_rewriting})\leq \frac{\alpha_n}{2} \|z - x_1\|_2^2 + \frac{s^2M^2}{2}\displaystyle\sum_{i = 1}^n \frac{1}{\alpha_i}
  \end{equation}
  Plugging this into equation (\ref{convexity_bound}), we get
  \begin{equation}
   R(x_1,...,x_n,z) \leq \frac{\alpha_n}{2s} \|z - x_1\|_2^2 + \frac{sM^2}{2}\displaystyle\sum_{i = 1}^n \frac{1}{\alpha_i}
  \end{equation}

 \end{proof}

 Note that setting $\alpha_i = 1$ and $s_i = \frac{1}{\sqrt{n}}$ produces a regret which is $O(\sqrt{n})$. It can be
 shown that this is optimal for convex Lipschitz functions in the black-box setting.
\section{Cycle SGD}
We now use the results on online optimization to derive convergence results for the cycle SGD algorithm.
To this end, we let $f_1,...,f_n$, be a collection of convex, Lipschitz functions and suppose we want
to minimize
\begin{equation}
 f(x) = \frac{1}{n}\displaystyle\sum_{i = 1}^n f_i(x)
\end{equation}
The cycle SGD for $k$ epochs is the same as running the online algorithm (\ref{online_DA}) with $\alpha_i = 1$ on the sequence of
functions $f_1,...,f_{nk}$ where $f_i = f_{i\mod n}$, i.e. we repeat the same sequence over and over again.
Given any such order, we obtain the regret bound from the previous section
\begin{equation}
 \displaystyle\sum_{i = 1}^{nk} (f_i(x_i) - f_i(x^*)) \leq \frac{1}{2s} \|x^* - x_1\|_2^2 + \frac{knsM^2}{2}
\end{equation}

Now let $\bar{x} = \frac{1}{nk}\displaystyle\sum_{i = 1}^{nk} x_i$ be the average of the iterates $x_i$ and note that
\begin{equation}
 f(\bar{x}) - f(x^*) = \frac{1}{n}\displaystyle\sum_{i = 1}^{n} (f_i(\bar{x}) - f_i(x^*))
 \leq \frac{1}{kn^2}\displaystyle\sum_{i = 1}^{n} \displaystyle\sum_{j = 1}^{nk} (f_i(x_j) - f_i(x^*))
\end{equation}
Rewriting this, we get
\begin{equation}
 f(\bar{x}) - f(x^*) \leq \frac{1}{kn^2}\displaystyle\sum_{i = 1}^{n} \displaystyle\sum_{j = 1}^{nk} (f_i(x_{k(i,j)}) - f_i(x^*)) + (f_i(x_j) - f_i(x_{k(i,j}))
\end{equation}
where $k(i,j)$ is the index in the same block as $j$ which is congruent to $i$ mod $n$.

The regret bound implies that
\begin{align}
 \frac{1}{kn^2}\displaystyle\sum_{i = 1}^{n} \displaystyle\sum_{j = 1}^{nk} (f_i(x_{k(i,j)}) - f_i(x^*))
 &= \frac{1}{nk}\displaystyle\sum_{i = 1}^{nk} (f_i(x_i) - f_i(x^*)) \\
 &\leq \frac{1}{2nks} \|x^* - x_1\|_2^2 + \frac{sM^2}{2}
\end{align}

It remains to bound
$$\frac{1}{kn^2}\displaystyle\sum_{i = 1}^{n} \displaystyle\sum_{j = 1}^{nk} (f_i(x_j) - f_i(x_{k(i,j}))
$$

Note that for each $i$, $f_i$ is $M$-Lipschitz and since $x_{i+1} = x_{i} + sg_i$, we have $|x_j - x_{k(i,j)}|\leq sM|j - k(i,j)|$.
This means that
\begin{equation}
 \frac{1}{kn^2}\displaystyle\sum_{i = 1}^{n} \displaystyle\sum_{j = 1}^{nk} (f_i(x_j) - f_i(x_{k(i,j})) \leq \frac{sM^2}{kn^2}\displaystyle\sum_{i = 1}^{n} \displaystyle\sum_{j = 1}^{nk} |j - k(i,j)|
\end{equation}
It is a simple matter to check that for each $j$, we have
\begin{equation}
 \displaystyle\sum_{i = 1}^{n}|j - k(i,j)| \leq \frac{n(n-1)}{2}
\end{equation}
which means that
\begin{equation}
 \frac{1}{kn^2}\displaystyle\sum_{i = 1}^{n} \displaystyle\sum_{j = 1}^{nk} (f_i(x_j) - f_i(x_{k(i,j})) \leq \frac{sM^2(n-1)}{2}
\end{equation}
Putting all of this together, this means that
\begin{equation}
 f(\bar{x}) - f(x^*) \leq \frac{1}{2}\left(\frac{1}{nks} \|x^* - x_1\|_2^2 + sM^2n\right)
\end{equation}
If we let $R = \|x^* - x_1\|_2$, then setting $s = \frac{R}{Mn\sqrt{k}}$ gives a bound
\begin{equation}
 f(\bar{x}) - f(x^*) \leq \frac{1}{2\sqrt{k}}\left(R^2 + M^2\right)
\end{equation}

\section{Shuffle SGD}
Finally, we conside the shuffle SGD algorithm. Let $f_1,...,f_n$, be a collection of convex, Lipschitz functions and suppose we want
to minimize
\begin{equation}
 f(x) = \frac{1}{n}\displaystyle\sum_{i = 1}^n f_i(x)
\end{equation}
The shuffle SGD algorithm chooses the order $f_1,...,f_n$ randomly in each epoch. The convergence rate of cycle SGD
applies for the shuffle SGD as well. However, it should be possible to obtain an improved convergence
rate if the order is randomly shuffled each time. However, I believe the convergence rate of the shuffle SGD
algorithm is still an open problem, all of the literature I'm aware of make additional assumptions beyond
the Lipschitz and convex assumptions. See \cite{shamir2016without} for instance.

\iffalse
\section{Introduction}
\subsection{Basic Concepts}
\subsection{Example: Learning from Expert Advice}
\section{Gradient Descent}
\subsection{Convex Case}
\subsection{Strongly Convex Case}
\section{Mirror Descent}
\subsection{Example: Learning from Expert Advice as Mirror Descent}
\fi


\section{Inner Products and Norms}

\subsection{Inner Products}

	Dot products are a form of inner product.

	Let's apply the dot product to the vectors $<v_1,v_2,\ldots,v_n>\cdot<w_1,w_2,\ldots,w_3>$ to show that the dot product is an inner product.

	\begin{equation}
		=v_1w_1+v_2w_2+\ldots+v_nw_n
	\end{equation}

	Therefore, $v\times v$ goes to $\R$. So we know that

	\begin{equation}
		\vec{v}\cdot\vec{v}=v_1^2+v_2^2+\ldots+v_n^2
	\end{equation}

	In general, we can assume that $ | | \vec{v} | | = \sqrt{\vec{v}\vec{v}}$. We should also keep in mind that $\vec{v}\cdot\vec{w}= | | \vec{v} | | | | \vec{w} | | \cos\theta$

	\begin{definition}
		AAn inner product of $V$ is a function $<,>:v\times v\to\R$ such that

		\begin{itemize}
			\item\begin{align}
					<c\vec{u}+d\vec{v},w> &= c<\vec{u},\vec{v}>+d<\vec{v},\vec{u}>\\
				<\vec{u},c\vec{v}+d\vec{w}&=c<\vec{u},\vec{v}>+d<\vec{u},\vec{w}>
			\end{align}
			\item $<\vec{v},\vec{w}> = <\vec{w}, \vec{v}>$
			\item $<\vec{v},\vec{v}>\ge_0$ while $<0,0> = \vec{0}$.
		\end{itemize}
	\end{definition}

	A vector space with an inner product is an inner product space.

	\begin{definition}
		IIf $V$ is an inner product space, then it's magnitude is

		\begin{equation}
			| | \vec{v} | | = \sqrt{<\vec{v},\vec{v}>}
		\end{equation}
	\end{definition}

	Let's take a look at a weighted inner product on $\R^3$. We are going to let $r_1,r_2,r_3>0$. We can define $<\vec{v},\vec{w}>$ as $r_1v_1w_1+r_2v_2w_2+r_3v_3w_3$

	\begin{problem}
		Let's define $[a,b]\le\R$. Consider $\mathbb{C}^0[a,b]$. This is a vector space. Define
		\begin{equation}
			<f(x),g(x)> = \int_a^bf(x)g(x)dx
		\end{equation}

		This is an inner product, so we also know that

		\begin{equation}
			| | f | | = \sqrt{\int_a^b\left(f(x)\right)^2dx}
		\end{equation}

		This equation is the $L^2$ norm.
	\end{problem}

\subsection{Inequalities}

	Recall that $\vec{v}\cdot\vec{w} = | | \vec{v} | | | | \vec{w} | |\cos\theta$, where $\theta$ is the angle between $\vec{v}$ and $\vec{w}$. Now $-1\le\cos\theta\le_1$, so we know that

	\begin{equation}
		| | \vec{v}\cdot\vec{w} | | \le | | \vec{v} | | | | \vec{w} | |
	\end{equation}

	This is the Cauchy-Shuartz inequality.

	\begin{theorem}
		FFor any inner product space

		\begin{equation}
			||<\vec{v},\vec{w}>||\le | | \vec{v}  | | | | \vec{w} | |
		\end{equation}
	\end{theorem}

	\begin{definition}
		IIf $\vec{v},\vec{w}\in V$, we say $\vec{v}$ and $\vec{w}$ are orthogonal if $<\vec{v},\vec{w}> = 0$
	\end{definition}

	\begin{problem}
		Let's look at an example of checking orthogonality of two equations $x,x^2-y\in\mathbb{C}^0[0,1]$. In order to do this we need to find the $L^2$ norm of the equations.
		\begin{align*}
			\left<x,x^2-\frac{1}{2}\right> &= \int_0^1x\left(x^2-\frac{1}{2}\right)dx\\
					    &=\int_0^1\left(x^3-\frac{1}{2}x\right)dx\\
					    &=\left.\frac{1}{4}x^4-\frac{1}{4}x^2\right|_0^1=0
		.\end{align*}
		Because the result of the inner product was zero, we know that $x,x^2-\frac{1}{2}$ are orthogonal in the $L^2$ norm.
	\end{problem}

	\begin{theorem}
		TThe triangle inequality states that if $V$ is an inner product space,

		\begin{equation}
			| | <\vec{v},\vec{w}> | | = | | \vec{v} | | + | | \vec{w} | |
		\end{equation}
	\end{theorem}

	Because we know that if we take the dot product of the same vector itself, $<a,b,c>\cdot<a,b,c>$, we get all of the items squared $<a^2,b^2,c^2>$, and because we know that $| | \vec{v} | | = \sqrt{<\vec{v},\vec{v}>}$, we get an idea of size. So we can define the unit ball to be
	\begin{equation}
		\{\vec{v}\in V\Big| | | \vec{v} | | = 1\}
	\end{equation}

\subsection{Norms}

	Equation (3.5) gives us the "size" of $\vec{V}$.

	\begin{definition}
		AA norm on $V$ is a function $ | | \cdot | |:V\to\R$ such that
		\begin{itemize}
			\item $| | \vec{v} | | = 0$ if and only if $\vec{v}=0$
			\item $| | c\vec{v} | | = |c|\cdot| | \vec{v} | |$
			\item $| | \vec{v}+\vec{w} | | \le | | \vec{v} | | + | | \vec{w} | |$
		\end{itemize}

		If $||<\vec{v},\vec{w}>||\le | | \vec{v}  | | | | \vec{w} | |$, then that is a norm. There are other norms to learn about.

	\end{definition}

	\begin{problem}
		Consider $V=\R^n$. We know that the magnitude of $\vec{V}_p$ is
		\begin{equation}
			\sqrt[p]{|\vec{v_1}|^p+|\vec{v_2}|^p+|\vec{v_3}|^p}.
		\end{equation}
		So if $v=\R^2$, $p=2$ we have
		\begin{equation}
			| |<x,y>| | _2=\sqrt{x^2+y^2}.
		\end{equation}
		But if we were to have $p=3$, we would have
		\begin{equation}
			| | <x,y> | |_3=\sqrt[3]{x^3+y^3}
		\end{equation}

		In $| | \cdot | |_3$, the size is $\sqrt[3]{3^3+4^3} \approx_4.5$
	\end{problem}

	In the 4 term, $| | \cdot | |_4$, the unit circle is the $(x,y)$'s such that $\sqrt[4]{x^4\cdot Y_4} =1$

	\begin{equation}
		\boxed{x^4+y^4=1}
	\end{equation}

	Another norm on $R^n$ is the super-norm. This is where

	\begin{equation}
		| |<x_1,x_2,\ldots,x_n>| |_\infty = max\{|x_1|,|x_2|,\ldots,|x_n|\}
	\end{equation}

	Here's a quick example: The super-norm for $<3,4>$ is

	\begin{equation}
		| | <3,4> | |_\infty=4
	\end{equation}

	because the maximum value in the set is $4$.

	Something to keep in mind is $| | <x,y> | |=|x|+|y|$.

	\begin{theorem}
		LLet $| | \cdot | |_A$ and $| | \cdot | |_B$ be two norms on $\R^n$. Then there exists positive numbers $0<c<k$ such that
		\begin{equation}
			c\cdot| | \vec{v} | |_A<| | \vec{v} | |_B<k \cdot | | \vec{v} | |_A
		\end{equation}
	\end{theorem}
		Let's consider $V\in\R^2$. Let's take a look at $| | \cdot | |_2$ and $| | \cdot | | _\infty$. Where $\vec{V}= <v_1,v_2>$.

	\begin{equation}
		\frac{1}{\sqrt{2} }\cdot | | \vec{v} | |_2\le | | \vec{v} | | _\infty < 1\cdot | | \vec{v} | |_2
	\end{equation}

	We can also define norms on matrices.

	\begin{theorem}
		IIf $| | \cdot | |$ is a norm on $\R^2$ and $A$ is an $m\times n$ matrix, then

		\begin{equation}
			| | A | | = max\{| | A\cdot\vec{u}| | \Big| | |\vec{u} | | = 1
		\end{equation}
	\end{theorem}

	These matrix norms satisfy the following:

	\begin{enumerate}
		\item $| | A\cdot\vec{v} | | \le | | A | | \cdot | | \vec{v} | |$
		\item $ | | A\cdot B | | \le | | A | | \cdot | | B | |$
		\item $| | A^k| | \le | | A | | ^k$
	\end{enumerate}

	Let's take a quick look at $| | A | |_\infty$

	\begin{definition}
		TThe $i^{th}$ absolute row sum of $A$ is the sum of the absolute values of the entries in the $i^{th}$ row.
	\end{definition}

	\begin{theorem}
		y$| | A | |_\infty$ the maximum absolute row sum.
	\end{theorem}

	Here's an example of using the $| | A | |_\infty$ value. Let $A=\SmallMatrix{-3&2\\5&4}$. We can determine that the maximum absolute row sum of $A$ is 8. This is because we can do

	\begin{align}
		|-3|+|2|&=5\\
		|5|+|3|&=\boxed{8}
	\end{align}
\subsection{Positive Definite Matrices}

	Consider the following two equations:

	\begin{align}
		\vec{x}=x_1\vec{e_1}+x_2\vec{e_2}+\ldots+x_n\vec{e_n}\\
		\vec{y}=y_1\vec{e_1}+y_2\vec{e_2}+\ldots+v_n\vec{e_n}
	\end{align}

	We can analyze this as

	\begin{equation}
		<\vec{x},\vec{y}> = [x_1,x_2,\ldots,x_n]\cdot \begin{bmatrix} k_{11}&\ldots&k_{1n}\\ \vdots&\ddots&\vdots\\ k_{n_1}&\ldots&k_{nn} \end{bmatrix} \begin{bmatrix} y_1\\y_2\\ \vdots y_n \end{bmatrix}
	\end{equation}

	\begin{align}
		k_{ij}&= <e_i,e_j>\\
		&=\vec{x}^Tk\vec{y}\\
		k&=k^T
	\end{align}

	This means that $k$ is symmetrical across the diagonal.

	\begin{equation}
		\begin{bmatrix} k_{11}&k_{12}&k_{13}\\ k_{12}&k_{22}&k_{23}\\ k_{13}&k_{23}&k_{33} \end{bmatrix}
	\end{equation}

	\begin{definition}
		AA $n\times n$ matrix $A$ is a symmetrical positive definite matrix if $A=A^{T}$ and $x^{t}k<x>0$.
	\end{definition}

	\begin{theorem}
		EEvery inner product on $\R^{n}$ is given by $<x,y> = \vec{x}^{T}k\vec{y}$ where $k$ is a symmetrical positive definite matrix. So $<\vec{x},\vec{y}>$ is a dot product or a weighted product.
		\begin{align}
			<\vec{x},\vec{y}> &= \vec{x}^{T}k\vec{y}\\
			k^{T}&=k\\
			\vec{v}^{T}k\cdot\vec{v} &>0\\
			\vec{v}\neq_0
		\end{align}
	\end{theorem}

	Let's take a look at an example for this:

	\begin{problem}
		Let $k=\left[\begin{smallmatrix}2&0\\0&3\end{smallmatrix}\right]$. First we need to check to see if $k^T=k$. By just looking at $k$, we can see that $k^{T}=k$. Next we need to do the following calculation to see if $\left[\begin{smallmatrix}x\\y\end{smallmatrix}\right]$ is the weighted inner product of the matrix $k$.

		\begin{align}
			\begin{bmatrix} x&y \end{bmatrix} \begin{bmatrix} 2&0\\0&3 \end{bmatrix} \begin{bmatrix} x\\y \end{bmatrix} &= \begin{bmatrix} x&y \end{bmatrix} \begin{bmatrix} 2x\\3x \end{bmatrix} \\
					 =2x^2+2y^2&>0\\
			\begin{bmatrix} x\\y \end{bmatrix} &\neq\begin{bmatrix} 0\\0 \end{bmatrix}
		\end{align}
		Therefore we know that
	\end{problem}

	\begin{problem}
		Let's consider the following problem

		\begin{equation}
			\begin{bmatrix} x&y \end{bmatrix} \begin{bmatrix} 4&-2\\-2&3 \end{bmatrix} \begin{bmatrix} x\\y \end{bmatrix}
		\end{equation}

		If we let $A$ be the numerical matrix, we can see that $A^{T}=A$. Let's simplify the equation from before

		\begin{align}
			\begin{bmatrix} x&y \end{bmatrix} \begin{bmatrix} 4x-2y\\-2x+3y \end{bmatrix}&=4x^2-2xy-2xy+3x^2\\
					 &=4x^2-4xy+3y^2\\
			(2x-y)^2+2y^2&>0\begin{bmatrix} x\\y \end{bmatrix} \neq\begin{bmatrix} 0\\0 \end{bmatrix}
		\end{align}
		Now we can see that this is a positive definite matrix.
	\end{problem}
	If we are given a symmetric matrix, $k$, the polynomial $x^{T}kx$ is a quadratic form of $k$.

	\begin{problem}
		Let's consider $k=\begin{bmatrix} 1&-3\\-3&2 \end{bmatrix}$. Let's find the quadratic form of $k$. First we need to write $k$ like so:

		\begin{align}
			\begin{bmatrix} x&y \end{bmatrix} \begin{bmatrix} 1&-3\\-3&2 \end{bmatrix} \begin{bmatrix} x\\y \end{bmatrix} &=\begin{bmatrix} x&y \end{bmatrix} \begin{bmatrix} x-3y\\-3x+2y \end{bmatrix} \\
					 &=x^3-3xy-3xy+2y^2\\
					 &=x^3-6xy+2y^2\\
		\end{align}
		 Therefore we know that the quadratic form of $k$ is $x^3-6xy-2y^2$.
	\end{problem}

  For a positive definite matrix, $k=k^{T}$ and $x^{T}kx>0$ for all $\vec{x}\neq 0$

  \begin{theorem}
    EEvery inner product in $\R^n$ is given by
    \begin{equation}
      <x,y> = x^{T}ky \text{ for }xy\in\R^n
    \end{equation}

    Let $v$ be an inner product space and $\vec{v_1},\ldots,\vec{v_2}$. The gram matrix of $v$ is
    \begin{equation}
      K=
      \begin{bmatrix}
        <v_1,v_2>&<v_1,v_2>&\ldots&<v_1,v_n>\\
        <v_2,v_1>&<v_2,v_2>&\ldots&<v_2,v_n>\\
        \vdots&\vdots&\ddots&\vdots\\
        <v_n,v_1>&<v_n,v_2>&\ldots&<v_n,v_n>
      \end{bmatrix}
    \end{equation}
  \end{theorem}

  \begin{definition}
    AA is a matrix that is $n\times n$. A is a positive semidefinite matrix if $A^{T}=A$ and $\vec{x}^{T}A\vec{x}\ge 0$
  \end{definition}

  \begin{theorem}
    AAll gram matrices are positive semi-definite. They are positive definite if and only if $\vec{v_1},\ldots,\vec{v_n}$ are linearly independent.
  \end{theorem}

  Suppose we are in $R^{m}$ and the inner produt is the dot product. Let $\vec{v_1},\ldots \vec{v_n}\in\R^{m}$\newline
  Let $A=[v_1,v_2,v_3,\ldots,v_n]$. Then $K=A^{T}$. Let A be a gram matrix generated by $v_1,\ldots,v_n$ with the dot product.

  \begin{align}
    K=A^{T}A&=\begin{bmatrix} \vec{v_1}\\\vec{v_2}\\ \vdots\\\vec{v_n} \end{bmatrix}
    \begin{bmatrix} \vec{v_1}&\vec{v_2}&\ldots&\vec{v_n} \end{bmatrix} \\
                             &=
    \begin{bmatrix}
      <v_1,v_1>&<v_1,v_2>&\ldots&<v_1,v_n>\\
      <v_2,v_1>&<v_2,v_2>&\ldots&<v_2,v_n>\\
      \vdots&\vdots&\ddots&\vdots\\
      <v_n,v_1>&<v_n,v_2>&\ldots&<v_n,v_n>
    \end{bmatrix}
  \end{align}

  \begin{prop}
    GGiven an $m\times n$ matrix $A$. The following are true
    \begin{enumerate}
      \item The $m\times n$ matrix $k=A^{T}A$ is positive definite.
      \item $A$ has linearly independent columns.
      \item $rank (A)=n$
      \item $Ker(A)=\{0\}$
    \end{enumerate}
  \end{prop}

  \begin{theorem}
    EEvery inner product on $\R^{n}$ is given by
    \begin{equation}
      <\vec{x},\vec{y}> = \vec{x}^{T}\cdot c\vec{y}
    \end{equation}
    where $C$ is a symetric, positive definite $n\times n$ matrix.
  \end{theorem}

  Let $\vec{v_1},\ldots,\vec{v_n}\in\R^{n}$. Let $A=[\vec{v_1}\vdots \vec{v_2} \vdots \ldots \vdots \vec{v_n}]$. Then $K=A^{T}CA$ is the gram matrix with respect to the inner product $\vec{v}^{T}C\vec{w}$.

  \begin{theorem}
    SSuppose $A$ is an $m\times n$ matrix with linearly independent collumns. Suppose $C$ is any positive definite $m\times m$ matrix. Then $\vec{v}^{T}C\vec{w}$
  \end{theorem}

  \begin{definition}
    TThe hilbert matrix $H=(h_{ij})$ where $h_{ij}=\frac{1}{i+j-1}$
  \end{definition}

  The $3\times 3$ hilbert matrix:

  \begin{equation}
    \begin{bmatrix}
      \frac{1}{1}&\frac{1}{2}&\frac{1}{3}\\
      \frac{1}{2}&\frac{1}{3}&\frac{1}{4}\\
      \frac{1}{3}&\frac{1}{4}&\frac{1}{5}
    \end{bmatrix}
  \end{equation}

  \begin{problem}
    Make a gram matrix, not in $\R^{m}$. Let $V=\mathbb{C}^{0}[0,1]$. Use the $L^{2}$ inner product.

    \begin{equation}
      <f,g> = \int_0^{1}f(x)g(x)dx
    .\end{equation}

    $1,x,x^2$ are linearly independent.

    \begin{align}
      <v_i,v_k> &= \int_0^{1}x^{i-1}x^{j-1}dx \\
                &=\int_0^{1}x^{i+j-2}dx\\
                &=\frac{1}{i+j-1}x^{i+j-1}\\
                &=\frac{1}{i+j-1}\\
                \to \begin{bmatrix}
                  \frac{1}{1}&\frac{1}{2}&\frac{1}{3}\\
                  \frac{1}{2}&\frac{1}{3}&\frac{1}{4}\\
                  \frac{1}{3}&\frac{1}{4}&\frac{1}{5}
                \end{bmatrix}
    .\end{align}
  \end{problem}
	\newpage


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS624: Analysis of Algorithms
% Copyright 2015 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 5}

In Figure 3 of Lecture Note 4, where do the costs in the right-hand column come from? Show that they make sense.
In particular, show why some of the costs are exact, and are upper bounds (with $\leq$ signs).

\subsection*{Solution}

The costs indicated at each level of the recursion tree are actually run time costs of calling the \textsc{Partition} method on subarrays.
As the \textsc{Partition} algorithm has a runtime of $\mathcal{O}(n)$ for an array of length $n$, it is not surprising that no matter how many times with divide the array to subarrays of length larger than 1, the runtime cost at each level would still remain the same.

In case at each call of \textsc{Partition} the array is divided into two subarrays of equal length, we can expect that all levels would have a similar constant value.
However, if \textsc{Partition} doesn't give the pivot value at middle of the array, at some level length of the subarrays that are the smaller parts given by \textsc{Partition} would reach 1 and \textsc{Partition} would have no effect, thus no cost.
This makes the runtime cost of higher levels in the recursion tree in Figure 3 less than the constant value given in earlier levels.


%!TEX root = /Users/stevenmartell/Documents/iSCAM-project/docs/iSCAM-guide/userGuide/usrGuide.tex
\section{Statistical functions \& probability distributions}
\begin{multicols}{2}
Many of the statistical functions commonly used in R have been written as negative log likelihoods and are in the \texttt{stats.cxx} library.  In this appendix is the documentation for the available functions in the stats.cxx library.  For the most part I have implemented the function based on the description from the R language, so it is possible to use \texttt{?function} name in R to learn more about the funciton.  Here I provide the formula, the actual code used to implement the function and a description of the variables. Note that some of the functions have been overloaded several times to deal with variables, vectors or a matrix.
\end{multicols}

\paragraph{dbeta} The beta distribution.
\[
	p(x|a,b) = - \ln(\Gamma(a+b))+(\ln(\Gamma(a))+\ln(\Gamma(b)))-(a-1)\ln(x)-(b-1)*\ln(1-x)
\]
the mean is given by $a/(a+b)$ and the variance is $\dfrac{ab}{(a+b)^2(a+b+a)}$
\begin{verbatim}
//beta distribution
dvariable dbeta(const dvariable& x, const double a, const double b)
{
	return - gammln(a+b)+(gammln(a)+gammln(b))-(a-1.)*log(x)-(b-1.)*log(1.-x);
}
\end{verbatim}

\paragraph{dgamma} The gamma distribution.
\[
 p(x|a,b) = -a \ln(b)+\ln(\Gamma(a))-(a-1)\ln(x)+bx
\]
where the mean and variance are given by $E(x) = a/b$ and $V(x) = a/b^2$.   Note that you can also compute $a$ and $b$ from $E(x)$ and $V(x)$ using the following relationships:
\[
 a=\frac{E(x)^2}{V(x)}, \quad b=\frac{E(x)}{V(x)}
\]
\noindent The following code is implemented in \texttt{stats.cxx} library:
\begin{verbatim}
//gamma
dvariable dgamma(const dvariable &x, const double a, const double b)
{
	return -a*log(b)+gammln(a)-(a-1.)*log(x)+b*x;
}
\end{verbatim}

\paragraph{dnorm} The normal distribution
\[
	p(x|\mu,\sigma) = 0.5\ln(2\pi)+\ln(\sigma)+0.5\frac{(x-\mu)^2}{\sigma^2}
\]
where the mean is $\mu$ and the variance is $\sigma^2$.
\begin{verbatim}
//normal distribution
dvariable dnorm(const dvariable& x, const double& mu, const double& std)
{
	double pi=3.141593;
	return 0.5*log(2.*pi)+log(std)+0.5*square(x-mu)/(std*std);
}
\end{verbatim}

\paragraph{dlnorm} The log normal distribution
\[
	p(x|\mu,\sigma) = 0.5\ln(2\pi)+\ln(\sigma)+\ln(x)+0.5\frac{(\ln(x)-\mu)^2}{\sigma^2}
\]
where the log mean is $\mu$ and the log variance is $\sigma^2$.
\begin{verbatim}
//log normal distribution
dvariable dlnorm(const dvariable& x, const double& mu, const double& std)
{
	double pi=3.141593;
	return 0.5*log(2.*pi)+log(std)+log(x)+square(log(x)-mu)/(2.*std*std);
}
\end{verbatim}

\section{Scripts}
\subsection{Bash script for creating the iscamArchive.zip \& .tar.gz files}
The following is a bash script that is used for archiving the source code files for \iscam\ and the scripts for the R code that is hosted on the Goole site.

%\begin{verbatim}
\begin{lstlisting}
#
#  bash script for creating the iscam archive
#
clear
echo "Archiving iscam files"

# script for creating the Tape Archive file
tar -czvf iscamArchive.$(date +%Y.%m.%d).tgz.gz Examples baranov.cxx\
 iscam.tpl iscam.dat iscam.r\
 stats.cxx iSCAMwin.txt Riscam_1.0.tar.gz

# script for creating a windoz zip file
zip -r iscamArchive.$(date +%Y.%m.%d).zip Examples baranov.cxx\
 iscam.tpl iscam.dat iscam.r\
 stats.cxx iSCAMwin.txt Riscam_1.0.tar.gz

exit 0
\end{lstlisting}
%\end{verbatim}

\section{R-code for figures and Tables}
%\begin{multicols}{2}

%		\tiny
%	\begin{alltt}
%	  \input{../iscam.R}\label{HakeDataFile}
%	\end{alltt}
%	\normalsize
%\lstset{language=R}

Here is the R-script that is used for reading in report files from ADMB.  Currently this is also compiled as an R-package.
\begin{scriptsize}
\lstinputlisting[language=R]{../../../src/r-code/read.admb.R}
\end{scriptsize}
%\end{multicols}


\section{Test Description and Success Criteria}
A series of unit tests are performed to check the validity of this module's operation.  The expected outcome is calculated independently in python and the results are seen to be correct to machine error.

\section{Test Parameters}
Three base voltages $V_{0}$  are tested where $V_{0} \in (5.0, -7.5, 0.0)$.    The input voltages are then setup as
\begin{equation}
\bm V = V_{0} \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}
+ \begin{bmatrix} 0.0 \\ 1.0 \\ 1.5 \end{bmatrix}
\end{equation}
Other inputs to the module are:
\begin{verbatim}
testModule.voltage2TorqueGain =[ 1.32, 0.99, 1.31]        # [Nm/V] conversion gain
testModule.scaleFactor =[ 1.01, 1.00, 1.02] #[unitless] scale factor
testModule.bias =[0.01, 0.02, 0.04]	# [Nm] bias
\end{verbatim}

\section{Test Results}
The unit test results are in the following Tables.

\input{AutoTeX/baseVoltage5.0.tex}
\input{AutoTeX/baseVoltage-7.5.tex}
\input{AutoTeX/baseVoltage0.0.tex}

All tests pass:
\begin{table}[H]
	\caption{Test results}
	\label{tab:results}
	\centering \fontsize{10}{10}\selectfont
	\begin{tabular}{c | c  } % Column formatting,
		\hline
		\textbf{Voltage Test} 						  		&\textbf{Pass/Fail} \\ \hline
		5.0	   			& \input{AutoTex/passFail5.0} \\ \hline
		-7.5	   			& \input{AutoTex/passFail-7.5} \\ \hline
		0.0	   			& \input{AutoTex/passFail0.0} \\ \hline
	\end{tabular}
\end{table}


\documentclass[twoside]{MATH77}
\usepackage{multicol}
\usepackage[fleqn,reqno,centertags]{amsmath}
\begin{document}
\begmath 3.2 Gaussian (Normal) Random Numbers and Vectors

\silentfootnote{$^\copyright$1997 Calif. Inst. of Technology, \thisyear \ Math \`a la Carte, Inc.}

\subsection{Purpose}

Generate pseudorandom numbers or vectors from the Gaussian (normal)
distribution.

\subsection{Usage}

\subsubsection{Generating Gaussian (normal) pseudorandom numbers}

\paragraph{Program Prototype, Single Precision}

\begin{description}
\item[REAL]  \ {\bf SRANG, X}
\end{description}
$$
\fbox{{\bf X = SRANG()}}
$$
\paragraph{Argument Definitions}

\begin{description}
\item[SRANG]  \ [out] The function returns a pseudorandom number from the
Gaussian (normal) distribution with mean zero and unit standard deviation.
\end{description}

\subsubsection{Generating Gaussian (normal) pseudorandom vectors}

Given an N-vector, ${\bf \mu }$, and a symmetric positive-definite N$\times $%
N matrix, $A$, the objective is to compute pseudorandom N-vectors, ${\bf x}$%
, from the N-dimensional Gaussian (normal) distribution having mean vector, $%
{\bf \mu }$, and covariance matrix, $A$.

On the first call to SRANGV with a new covariance matrix, $A$, the user must
set HAVEC = .false. to indicate that the Cholesky factor of $A$ has not yet
been computed. SRANGV will replace $A$ in storage by its Cholesky factor, $C$%
, and set HAVEC = .true. to indicate that the array A() now contains $C$
rather than $A$.

On each call to SRANGV an N-vector, ${\bf x}$, is returned as a pseudorandom
sample from the specified distribution.

\paragraph{Program Prototype, Single Precision}

\begin{description}
\item[INTEGER]  \ {\bf NDIM, N, IERR}

\item[REAL]  \ {\bf A}(NDIM$\geq $N, $\geq $N){\bf , U}($\geq $N){\bf , X}($%
\geq $N)

\item[LOGICAL]  \ {\bf HAVEC}
\end{description}

Assign values to A(,), NDIM, N, U(), and HAVEC.

\begin{center}
\fbox{\begin{tabular}{@{\bf }c}
CALL SRANGV(A, NDIM, N, U,\\
X, HAVEC, IERR)\\
\end{tabular}}
\end{center}

Computed values will be returned in X() and IERR, and the contents of A(,)
and HAVEC may be changed.

\paragraph{Argument Definitions}

\begin{description}
\item[A(,)]  \ [inout] When HAVEC is .false., A(,) contains a user-supplied N%
$\times $N symmetric positive-definite covariance matrix, $A$. Only the
diagonal and subdiagonal elements of $A$ need be given. This subroutine will
not access the super-diagonal locations in the array, A(,). This subroutine
will replace $A$ by its lower-triangular Cholesky factor, $C$, and set HAVEC
= .true.

When HAVEC is .true., A(,) is assumed to contain the Cholesky factor, $C$.

\item[NDIM]  \ [in] First dimension of the array A(,). Require NDIM $\geq $
N.

\item[N]  \ [in] Order of the covariance matrix $A$ and dimension of the
vectors U and X.  Require N $\geq $ 1.

\item[U()]  \ [in] Contains the N-dimensional mean vector, ${\bf \mu }.$

\item[X()]  \ [out] On return will contain the N-dimensional generated
random vector.

\item[HAVEC]  \ [inout] See description above for A(,).

\item[IERR]  \ [out] IERR will only be referenced when the subroutine is
entered with HAVEC = .false. If the Cholesky factorization is successful,
the subroutine will set HAVEC = .true. and IERR = 0. Otherwise, it will
leave HAVEC = .false. and set IERR to the index of the row of $A$ in which
failure was noted. In this latter case the results returned in A(,) and X()
will not be useful.
\end{description}

\subsubsection{Modifications for Double Precision}

For double-precision usage, change the REAL statements to DOUBLE PRECISION
and change the initial ``S" of the function and subroutine names to ``D."
Note particularly that if the function name, DRANG, is used it must be typed
DOUBLE PRECISION either explicitly or via an IMPLICIT statement.

\subsection{Examples and Remarks}

The program DRSRANG demonstrates the use of SRANG to compute Gaussian random
numbers and uses SSTAT1 and SSTAT2 to compute and print statistics and a
histogram based on a sample of~10000 numbers delivered by SRANG.

To compute Gaussian random numbers with mean XMEAN and standard deviation
STDDEV, one can use the statement
$$
\text{X = XMEAN + STDDEV }*\text{ SRANG()}
$$
The program DRSRANGV demonstrates the use of SRANGV to compute pseudorandom
vectors from a 3-dimensional Gaussian distribution with mean vector and
covariance matrix specified as%
$$
{\bf \mu } =\left[
\begin{array}{c}
1 \\
2 \\
3
\end{array}
\right] \quad \text{and}\quad A=\left[
\begin{array}{rrr}
0.05 & 0.02 & 0.01 \\
0.02 & 0.07 & -0.03 \\
0.01 & -0.03 & 0.06
\end{array}
\right] .
$$
To fetch or set the seed used in the underlying pseudorandom integer
sequence use the subroutines described in Chapter~3.1.

\subsection{Functional Description}

\subparagraph{Method}

The algorithm for generation of Gaussian random numbers is based on
\cite{Bell:1968:A334}--\nocite{Box:1958:NOG}\cite{vonNeumann:1959:VTU}.
This method draws pairs of uniform random numbers $x$ from
[0,~1] and $y$ from [$-$1,~1] until a pair is obtained that
satisfies $x^2 + y^2 \leq 1$.
The probability of satisfying this constraint is $\pi /4 \approx 0.785$. It
then draws another uniform random number $u$ from [0,~1] and computes
\begin{eqnarray*}
s & = & x^2 + y^2\\
t & = & \sqrt{-2\ \log u} \,/s\\
g_1 & = & t(x^2 - y^2)\\
g_2 & = & 2xyt
\end{eqnarray*}
The numbers $g_1$ and $g_2$ are independent samples from the Gaussian
distribution with mean zero and unit standard deviation. The number $g_1$ is
returned when it is computed and $g_2$ is saved and returned the next time a
Gaussian random number is requested. The saved value will be discarded if
the underlying uniform sequence is reinitialized by a call to RAN1 or RANPUT
of Chapter~3.1.

This method is a mathematically exact transformation from the uniform
distribution to the Gaussian distribution so the statistical quality of the
delivered numbers depends entirely on the quality of the uniform
pseudorandom numbers used. The uniform numbers are obtained by calling
SRANUA or DRANUA, using the array in common block /RANCMS/ or /RANCMD/ as a
buffer as described in Chapter~3.1.

For the generation of Gaussian random vectors we are given a mean vector $%
{\bf \mu }$ and a symmetric positive-definite covariance matrix $A$. The
Cholesky method is used to factor the given covariance matrix $A$ as $A =
CC^t$ where $C$ is a lower triangular matrix. Then for each vector to be
delivered the method first constructs a vector ${\bf g}$ whose components
are independent samples from the Gaussian distribution with mean zero and
unit standard deviation. It then computes
$$
{\bf x} = {\bf \mu }+C{\bf g}
$$
which is a sample from the N-dimensional Gaussian distribution with mean $%
{\bf \mu }$ and covariance matrix $A$.

Values returned as double-precision random numbers will have random bits
throughout the word, however the quality of randomness should not be
expected to be as good in a low-order segment of the word as in a high-order
part.

\bibliography{math77}
\bibliographystyle{math77}

\subsection{Error Procedures and Restrictions}

While computing the Cholesky factorization of $A$, subroutine SRANGV or
DRANGV may determine that the matrix is not positive-definite. In that case
it will return with IERR set to the index of the row of the matrix at which
the problem was detected. In this non-positive-definite case the contents of
the array A(,) will have been altered, HAVEC will still have the value
.false., and X() will not contain useful results.

When the Cholesky factorization is successful, IERR will be set to zero.

The conditions that N $\geq $ 1, and NDIM $\geq $ N are required but
not checked.

\subsection{Supporting Information}

The source language is ANSI Fortran~77.

\begin{tabular}{@{\bf}l@{\hspace{5pt}}l}
\bf Entry & \hspace{.35in} {\bf Required Files}\vspace{2pt} \\
DRANG & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
DRANG, ERFIN, ERMSG, RANPK1, RANPK2\rule[-5pt]{0pt}{8pt}}\\
DRANGV & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
DRANG, DRANGV, ERFIN, ERMSG, RANPK1, RANPK2\rule[-5pt]{0pt}{8pt}}\\
SRANG & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
ERFIN, ERMSG, RANPK1, RANPK2, SRANG\rule[-5pt]{0pt}{8pt}}\\
SRANGV & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
ERFIN, ERMSG, RANPK1, RANPK2, SRANG, SRANGV}\\\end{tabular}

Based on subprograms written for JPL by Carl Pitts, Heliodyne Corp., April,
1969. Adapted to Fortran~77 for the JPL MATH77 library by C. L. Lawson and
S. Y. Chiu, JPL, April~1987. November~1991: Lawson reorganized and renamed
common blocks. Improved coordination between Gaussian and uniform sequences
on reinitializations.

\begcodenp

\lstset{language=[77]Fortran,showstringspaces=false}
\lstset{xleftmargin=.8in}

\centerline{\bf \large DRSRANG}\vspace{10pt}
\lstinputlisting{\codeloc{srang}}
\newpage

\vspace{30pt}\centerline{\bf \large ODSRANG}\vspace{10pt}
\lstset{language={}}
\lstinputlisting{\outputloc{srang}}
\lstset{language=[77]Fortran,showstringspaces=false}
\lstset{xleftmargin=.8in}
\newpage

\centerline{\bf \large DRSRANGV}\vspace{10pt}
\lstinputlisting{\codeloc{srangv}}
\newpage

\vspace{30pt}\centerline{\bf \large ODSRANGV}\vspace{10pt}
\lstset{language={}}
\lstinputlisting{\outputloc{srangv}}
\end{document}


\documentclass{article}

\usepackage{amsmath}
\usepackage{numprint}

\author{Daniel Fernandes Martins (danielfmt)}
\title{Question \#4 Solution}

\begin{document}

\maketitle

\section{Partial Derivative of $E(u,v)$}

In this exercise, we are asked to compute the partial derivative
$\frac{\partial E}{\partial u}$ of an error function that defines a
hypothetical error surface.

\begin{equation*}
\frac{\partial}{\partial u} E(u,v) = (ue^v -2ve^{-u})^2
\end{equation*}

Let's define the partial derivative of each term, step-by-step. First, we apply
the \textit{power rule}, which says $\frac{\partial}{\partial x} x^n = n \cdot x^{n-1}$:

\begin{equation*}
\frac{\partial}{\partial u} E(u,v) = 2(ue^v -2ve^{-u})^1 \\
\frac{\partial}{\partial u} (ue^v -2ve^{-u})
\end{equation*}

Notice that we also applied the \textit{chain rule} to the squared term since
it contain terms of $u$.

Now, the term $ue^v$, which is $u$ multiplied by a constant, is really simple
to differentiate:

\begin{equation*}
\frac{\partial}{\partial u} (ue^v) = 1 \cdot e^v = e^v
\end{equation*}

Finally, to differentiate a term that looks like $e^u$ we just need to multiply
it by the derivative of $u$, which is 1 or -1 depending on the sign:

\begin{equation*}
\frac{\partial}{\partial u} (-2ve^{-u}) = -2ve^{-u} \cdot -1 = 2ve^{-u}
\end{equation*}

Putting everything together, we have the solution:

\begin{equation*}
\frac{\partial}{\partial u} E(u, v) = \\
2(ue^v -2ve^{-u})(e^v + 2ve^{-u})
\end{equation*}

\end{document}


\section{Adaptive AR Model Based Time-Frequency Estimation}
\subsection{AR modelling of FM signal}
\begin{figure}[htb]
	\centering
    \includegraphics[width=0.36\textwidth]{fig/32/32a1.eps}
    \caption{Time-variant frequency of FM signal}
    \label{fig:3_2_a1}
\end{figure}
\noindent
A FM signal with time-variant frequency as shown in Fig.\ref{fig:3_2_a1} is modulated with adding white noise in distribution of $\mathcal N\in$ (0, 0.05). The frequency is composed of three segments with constant, linear and quadratic parts, resulting in the non-stationary FM signal.
\begin{figure}[htb]
     \centering
     \hspace{-0.4cm}
     \begin{subfigure}[b]{0.33\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32a2.eps}
     \end{subfigure}
    \hspace{-0.4cm}
     \begin{subfigure}[b]{0.33\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32a3.eps}
     \end{subfigure}
    \hspace{-0.4cm}
     \begin{subfigure}[b]{0.33\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32a4.eps}
     \end{subfigure}
    \caption{Whole FM AR estimation with order=1, 10 and 15}
    \label{fig:3_2_a2}
\end{figure}\\
When using the MATLAB function \texttt{aryule} to estimate the coefficient for entire signal, Fig.\ref{fig:3_2_a2} depicts the performance of estimations with different orders. For AR(1) modelling, the estimated peak of frequency is inaccurate since the function is incapable to estimate non-stationary signal. With increasing the order of estimated model, only the constant frequency at $100Hz$ is successfully estimated. However, other segments frequencies are still not captured accurately. Thus, the previous method in Part 2.2 is not applicable for FM signal.
\begin{figure}[htb]
     \centering
     \hspace{-0.4cm}
     \begin{subfigure}[b]{0.32\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32a5.eps}
     \end{subfigure}
    \hspace{-0.4cm}
     \begin{subfigure}[b]{0.32\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32a6.eps}
     \end{subfigure}
    \hspace{-0.4cm}
     \begin{subfigure}[b]{0.32\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32a7.eps}
     \end{subfigure}
     \\
     \hspace{-0.4cm}
     \begin{subfigure}[b]{0.32\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32a8.eps}
     \end{subfigure}
    \hspace{-0.4cm}
     \begin{subfigure}[b]{0.32\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32a9.eps}
     \end{subfigure}
    \hspace{-0.4cm}
     \begin{subfigure}[b]{0.32\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32a10.eps}
     \end{subfigure}
        \caption{Block-based AR estimation with order=1 and 15}
        \label{fig:3_2_a3}
\end{figure}\\
In addition, the frequency is time-varying function with constant, linear and quadratic segments. Thus, the block-based estimation is implemented with segments length $N=500$ samples. Fig.\ref{fig:3_2_a3} illustrates the performance of estimated AR model with order 1 and 15. The constant frequency part can be successfully estimated with $p=1$ presented in a peak value at $100Hz$. However, the linear and quadratic segments perform inferior with inaccurate frequencies. By increasing the capacity of estimated model, the frequency range of segment can be approximately estimated with acceptable uncertainty, as shown in second row of Fig.\ref{fig:3_2_a3}. Nevertheless, the linear and quadratic relationship can not be observed based on the estimation since they are not satisfied with stationary.
\subsection{CLMS based estimated AR coefficient}
In this section, the CLMS algorithm is applied to estimate non-stationary signal. Fig.\ref{fig:3_2_b} shows the performance of estimating AR(1) coefficients with varying step-size $\mu$. It is obviously that the CLMS can adaptively capture the time-variant frequencies. However, the different step-size $\mu$ also affects the performance of the CLMS estimation. With small step-size, the learning curve can not converge, leading to inadequate estimation. The performance at $\mu=0.01$ is improved in spite of lacking beginning parts of the constant frequency. Setting $\mu=0.05$ introduces an optimal estimation, while larger step causes oscillation in convergence. As a consequence, there are large variance and distortions of spectrum.
\begin{figure}[htb]
     \centering
     \hspace{0.4cm}
     \begin{subfigure}[b]{0.35\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32b1.eps}
     \end{subfigure}
    \hspace{0.4cm}
     \begin{subfigure}[b]{0.35\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32b2.eps}
     \end{subfigure}
    \\
    \hspace{0.4cm}
     \begin{subfigure}[b]{0.35\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32b3.eps}
     \end{subfigure}
     \hspace{0.4cm}
     \begin{subfigure}[b]{0.35\textwidth}
         \centering
         \includegraphics[width=\textwidth]{fig/32/32b4.eps}
     \end{subfigure}
        \caption{CLMS-based AR estimation with different step $\mu$}
        \label{fig:3_2_b}
\end{figure}


\section{Neighbourhoods}

\subsection{Neighbourhood topology}

We have a set \(X\).

For each element \(x\in X\), there is a non-empty set of neighbourhoods \(N\in \mathbf N(x)\) where \(x\in N\subseteq X\) such that:

\begin{itemize}
\item If \(N\) is a subset of \(M\), \(M\) is a neighbourhood.
\item The intersection of two neighbourhoods of \(x\) is a neighbourhood of \(x\).
\item \(N\) is a neighbourhood for each point in some \(M\subseteq N\)
\end{itemize}


\chapter{AutoDiff}
\section{Introduction}
Suppose $\mathcal F:\mathbb U \mapsto \mathbb V$, (usually $\mathbb U = \mathbb R^n$ and $\mathbb V\in\mathbb R^m$) and the tangent space of a point $u\in\mathbb U$ with $v=\mathcal F(u) \in \mathbb V$ is $T_u\mathbb U$ and $T_v\mathbb V$. The dual of  $T_u\mathbb U$ and $T_v\mathbb V$ is  $(T_u\mathbb U)'$ and $(T_v\mathbb V)'$. Then for any $u\in\mathbb U$, $\mathcal{F}'(u)=\nabla \mathcal{F}(u)$ is the linear operator from $T_u\mathbb U$ to $T_v\mathbb V$ and $(\nabla \mathcal{F}(u))^T$ is the linear from $(T_v\mathbb V)'$ to $(T_u\mathbb U)'$.

 For any $u_0\in \mathbb U$, $v_0=\mathcal{F}(u_0)\in\mathbb V$, and assume the coat of time to calculate each entry of $\mathcal F$ or $\mathcal{F}'$ can be bounded by a constant, and let $F_{time}$ is the cost of calculating $\mathcal F$.
\begin{itemize}
	\item  Suppose $u$ is depend on a variable $s$. If we have already known $x^1=\dfrac{\partial u}{\partial s}(u_0)\in T_{u_0}\mathbb U$, the forward autodiff algorithm is to calculate
	\begin{equation}
	\dfrac{\partial v}{\partial s}(u_0)=	\dfrac{\partial \mathcal F}{\partial s}(u_0)=\nabla \mathcal F(u_0) \dfrac{\partial u}{\partial s}(u_0)=\nabla \mathcal F(u_0) x^1
	\end{equation}
	and the cost of time is $O(F_{time})$
 	\item  Suppose $t$ is a variable depend on $v$. If we have already known $y_1=(\dfrac{\partial t}{\partial v}(v_0))^T\in (T_{v_0}\mathbf V)'$ the backward autodiff algorithm is to calculate
 	\begin{equation}
 	(\dfrac{\partial t}{\partial u}(u_0))^T=(\dfrac{\partial v}{\partial u}(u_0))^T(\dfrac{\partial t}{\partial v}(v_0))^T=(\nabla \mathcal{F}(u_0))^T y_1
 	\end{equation}
 	and the cost of time is $O(F_{time})$
\end{itemize}

If we want to calculate $\nabla F$ by autodiff, we can set $x^1=e^1,...,e^n$ or set $y_1=e_1,...,e_m$, where $e^i$ is the $i$th column of $I_n$ and $e_i$ is the $i$th column of $I_m$. This means the cost of forward algorithm is $O(nF_{time})$ and the cost of backward algorithm is $O(mF_{time})$.

Now we have a series of $\mathcal F_j:\mathbb R^{n_j-1}\mapsto\mathbb R^{n_j}$, and $t=\mathcal{F}_J\circ\cdots\circ \mathcal{F}_1(s)$, if we use forward algorithm for all functional composition, the cost is $O(n_0(n_1+\cdots+n_J)F_{time})$, if we use backward  algorithm for all functional composition, the cost is $O(n_J(n_{J-1}+\cdots+n_0)F_{time})$. For deep learning problem, the $n_0$ is very large but $n_0=1$, so backward algorithm is much better.


\section{Markov decision processes}

\subsection{Introduction}

An MDP is defined by:
\begin{itemize}
    \item A set of states $s \in S$
    \item A set of actions $a \in A$
    \item A transition function $T(s, a, a')$ - the model - that represents the probability that $a$ leads to $s'$ fro $s$
    \item A reward function $R(s, a, s')$
    \item (Optional) A terminal state
\end{itemize}

The action outcome depends only on the current state, but not on the history of actions

\subsection{Policies}

A policy is a function that $P: S \mapsto A$, that gives an action for each state. An optimal policy $p^*$ maximizes the expected utility if followed.

\subsection{Discounting rewards}

Solution to avoid infinite rewards:
\begin{itemize}
    \item Finite horizon: (similar to depth-limited search)
    \begin{itemize}
        \item Terminate episodes after a fixed T steps (e.g. life)
        \item Gives non-stationary policies ($p$ depends on time left)
    \end{itemize}
    \item Discounting: smaller $\gamma$ means smaller “horizon” – shorter term focus
    \begin{math}
    U(\overline{r}) = \sum_{t=0}^{\infty}{\gamma^t r_t} \leq \frac{R_{max}}{1-\gamma}
    \end{math}
    \item Absorbing state: guarantee that for every policy, a terminal state will eventually be reached
\end{itemize}

\subsection{Solving MDPs}

Given:
\begin{itemize}
    \item $V^*(s)$  the expected utility starting in $s$ and acting optimally
    \item $Q^*(s, a)$ - q-state - the expected utility starting out having taken action $a$ from state $s$ and thereafter acting optimally
    \item $\pi^*(s)$ the optimal action from state $s$
    \item $V_k(s)$ the expected utility starting in $s$ and acting optimally if the game ends in $k$ more time steps
\end{itemize}

\subsubsection{Value iteration}

Given the Bellman equation:
\begin{equation}
    V^*(s) = \underset{a}{max}(R(s,a, s') + \gamma V_k(s'))
\end{equation}

\paragraph{Algorithm} Repeat until convergence:
\begin{itemize}
    \item Start with $V_0(s) = 0$: no time steps left means an expected reward sum of zero
    \item Given vector of $V_k(s)$ values, do one ply of expectimax from each state:
    \begin{math}
    V_{k+1}(s) = \underset{a}{max}{R(s,a, s') + \gamma V_K(s')}
    \end{math}
\end{itemize}

\paragraph{Properties}
\begin{itemize}
    \item Complexity of each iteration: $O(S^2A)$
    \item Theorem: will converge to unique optimal values
    \item Basic idea: approximations get refined towards optimal values
    \item Policy may converge long before values do
\end{itemize}

\subsection{Policy extraction}

\begin{equation}
    \pi^*(s) = \underset{a}{arg max}\sum_{s'}{T(s,a,a')[R(s,a, s') + \gamma V_k(s')]}
\end{equation}

\subsection{Policy iteration}

\paragraph{Algorithm} Repeat steps until policy converges:
\begin{itemize}
    \item Policy evaluation: calculate utilities for some fixed policy (not optimal utilities!) until convergence
    \begin{math}
    V^{\pi_i}_{k+1}(s) \leftarrow \sum_{s'}{T(s,a,a')[R(s,\pi_i(s)=a, s') + \gamma V^{\pi_i}_K(s')]}
    \end{math}
    If the state space is finite, the utilities can also be computed by solving a set of linear equations.
    \item Policy improvement: update policy using one-step look-ahead with resulting converged (but not optimal!) utilities as future values
    \begin{math}
    \pi^*_{i+1}(s) = \underset{a}{argmax}\sum_{s'}{T(s,a,a')[R(s,a, s') + \gamma V_k^{\pi_i}(s')]}
    \end{math}
\end{itemize}


\chapter{Group Theory}

A group is a set of elements that can move between themselves. That movement is denoted by the multiplication of elements. We call the entire group itself $G$, which contains $g_1, g_2, ... g_N$ elements ($N$ is the order of $G$). The group is defined in the following way
\begin{enumerate}
	\item $g_1g_2~\exists ~G$ for all $g_1, g_2~\exists~ G$. Which says when we multiply two elements, we get another element that is still within the group back
	\item $(g_1g_2)g_3 = g_1(g_2g_3)$ for $g_1, g_2, g_3~\exists~ G$. Which says when given a specific order to multiply the elements, the sequencing of multiplication doesn't matter
	\item There is an identity element $e$ which gives $eg = ge = g$ for all $g\exists G$. This says there is one element that does not change any of the other elements.
	\item For every element $g$, there is an inverse $g^{-1}$ such that $gg^{-1} = e$. Which allows us, from any element, to return to the identity
\end{enumerate}
The order of any of these elements $g$ are given by the smallest integer $k$ such that $g^k = e$.
An isomorphism between two groups technically is a bijective map that preserves the group operations. This means that any of the operations from one group can be mapped exactly on to those of another.

\subsection{Quaternion Group}
The Quaternion Group $Q_8$ is defined such that
\begin{align}
	i^2 = j^2 = k^2 = ijk = -1
\end{align}

%
%TODO:
%\begin{itemize}
%	\item SO(3) can have matrices that represent it of arbitrary size?
%	\item Put example of what a matrix tensor product looks like (from comp notes)
%	\item Irreducible representation
%\end{itemize}
%
%
\subsection{Lie Groups}
Of particular importance in physics are \textbf{Lie groups}, groups in which you can change infinitesimally between their elements. This allows you to Taylor expand ... TODO

\subsection{SO(3)}
Consider the addition of angular momentum of two spin $1/2$ particles, $a$ and $b$ the Hilbert space in which they live both have dimension
\begin{align}
d_i = (2j_i + 1)
\end{align}
where $i ~\exists~(a,b)$. So $d_i = 2$ in the case of spin 1/2 particles ($\uparrow$ and $\downarrow$). The dimension of their combination is the product of both dimensions
\begin{align}
d_{ab} = (2j_a+1)(2j_b+1)
\end{align}
giving us 4. Considering each particle individually, we have a set of operators that "represent" the legal group operations that are allowed to make on the ket that would match with another potential bra. Following the same notation we look for the tensor product of the two representations
\begin{align}
D^{(j_a)}\otimes D^{(j_b)}
\end{align}

It can be shown \cite{sakurai} that when you combine any two particles with angular momentum $j, j'$ you can describe any transformation on the system as a whole in terms of a sum of irreducible representations.

\begin{align}
D^{(j)}\otimes D^{(j')} = \bigoplus^{j+j'}_{l=|j-j'|} D^{(l)}
\end{align}

Groups themselves are abstract entities defined by just the rules to get between elements. In physics, we use a \textbf{representation} of a group, which is the group put into matrix form such that it obeys the same rules as its definition.
\begin{gather}
\begin{align}
\textrm{Group} && \textrm{Representation}\\
g_1g_2 = g_3 && D(g_1)D(g_2) = D(g_3)
\end{align}
\end{gather}
Where $D(g_1)$ is a matrix representation of the group element $g_1$ etc.