%********************************************************************
% Appendix
%*******************************************************
% If problems with the headers: get headings in appendix etc. right
%\markboth{\spacedlowsmallcaps{Appendix}}{\spacedlowsmallcaps{Appendix}}
\chapter{Appendix: Python implementation}
\label{ch:appendix:python:implementation}

\section{Engineering Python's modules}

In this appendix we talk about how we implemented a subset of the \emph{Riordan
group} theory using the Python programming language.  Although our code is pure
Python code, we rest on the \emph{Sage} \cite{sage} mathematical framework for
symbolic computations\marginpar{pure Python classes, delegating to \emph{Sage}}.

We start from the abstract world of mathematics and, gradually, introduce the
architecture, principles, design choices and, finally, hierarchies of classes
that allow us to fullfil our implementation.

\subsection{From mathematics to code}
\label{subsection:python:appendix:from:math:to:code}

Mathematically, a function $colouring$ has been implemented. Let $\mathcal{M}$
be a Riordan array and $m_{nk} \in \mathcal{M}$ its generic element, moreover
define a set of $k$ colours $\lbrace c_0, \ldots, c_{k-1} \rbrace$. So,
function $colouring$ has the following type:
\begin{displaymath}
    colouring : \mathbb{N} \times\mathbb{N} \rightarrow \lbrace c_0, \ldots, c_{k-1} \rbrace
\end{displaymath}
Let $p\in\mathbb{N}$, usually a prime number, the following definitions have been implemented:
\begin{itemize}
    \item colour $\mathcal{M}$ associating to each remainder class
        $r \in \lbrace[0],\ldots,[p-1]\rbrace$ a different colour $c_r$:
        \begin{displaymath}
            colouring_{p}(n,k) = c_{r} \leftrightarrow m_{nk} \equiv_{p} r
        \end{displaymath}
    \item \emph{bi}-colour $\mathcal{M}$ with $c_0$ if $m_{nk}$
        is a multiple of $p$, otherwise use a colour $c_1$:
        \begin{displaymath}
            colouring_{p}(n,k) = c_{0} \leftrightarrow p | m_{nk}
        \end{displaymath}
    \item a less used one, \emph{bi}-colour $\mathcal{M}$ with $c_0$
        if $m_{nk}$ is a prime, otherwise use a colour $c_1$
            \marginpar{``prime'' definition of $colouring$
                seems to produce triangles coloured at random\ldots}:
        \begin{displaymath}
            colouring(n,k) = c_{0} \leftrightarrow
                \nexists s\in\lbrace 2,\ldots,m_{nk}-1\rbrace.s|m_{nk}
        \end{displaymath}
\end{itemize}

We choose the \emph{Python} language to implement such a function and we rest
on the \emph{Sage} \cite{sage} framework to do hard computational stuff, like
Taylor expansion of a function or solving equations \emph{symbolically}.

Our implementation aims at a subset of the \emph{Riordan group} theory,
focusing on simplicity and sound design principles. The interest of the author
for the \emph{Smalltalk} programming language \cite{Goldberg:1983:SLI:273} has
    influenced the design of the code base, with a strong focus on the concept
    of \emph{messaging} between objects as the core programming paradigm. We
    believe that keeping such an approach, classes are structured to be really
    extendible, keeping the whole implementation limited to few core concept.
    In addition to this orientation, we have integrated some particular
    features supplied by the Python programming language in order to ease the
    experience of playing with our objects.

We \marginpar{all Python code and \LaTeX files of this work is under \emph{Git}
version control} wont describe all the ``nitty-gritty'' details of the
implementation, on the other hand we focus on the architecture, pointing out
the concepts, and some design principles and patterns used to implement it.
Finally, we review a set of core classes, pointing the interested reader to the
complete implementation, which is freely available on line, in the \emph{Git}
repository \cite{nocentini:git:repository}.

\subsection{Architecture}

Talking about \emph{an architecture} to describe the implementation of a single
\emph{functionality}, namely the $colouring$ function, seems to be wasteful.
However, \emph{Smalltalkers} like to introduce a little language for each
problem they tackle, which is the same to say they introduce \emph{an
architecture}, eventually. What is important is the spirit and the principle
behind their approach: \emph{messaging}. As \citeauthor{kay:on:messaging}
states in \cite{kay:on:messaging}, what is really important is the relation and
the net of messages that a set of objects send to each other. Pairing this
principle with the one shown in
\cite{friedman:felleisen:few:java:few:patterns}, about looking at behavior as
an object, we have a solid track to follow.

Our objects are instances of a few class hierarchies, many of which can be seen
\marginpar{dispatching and action objects} as \emph{dispatching objects}, while
actual behavior is performed by a very restricted set of them, what we call
\emph{action objects}. So, the whole $colouring$ functionality is taken apart
into a set of small hierarchies, such that each one of them catches a
\emph{context} that can happen in a study session; example contexts are the
desired type of partitioning or if the representation should have a plain
layout or a centered one\ldots Each object which is an instance of a little
hierarchy, doesn't know how to do hard symbolic computation or how to generate
a result \TeX\,file. Its only responsibility is to inform that \emph{the
requested functionality should be performed under the context it represents}.
This is pretty similar to the approach advised by \emph{functional
programming}, where \emph{pattern matching} over type constructors is a compact
way to express the same computation as we do with a net of \emph{messages}.

Only a small set of \emph{action objects}\marginpar{an action object is the
recipient of a chain of dispatching messages}, also known as \emph{visitors} if
we look at them from the design patterns \cite{Gamma:1995:DPE:186897} point of
view, code the actual implementation because only them know the \emph{entire}
context.  With this approach, if we would like to implement a new
\emph{functionality}, we have to \emph{create} a new \emph{action object}; on
the other hand, if we would like to augment the context with new informations,
we are required to \emph{refine} methods already implemented in \emph{action
objects}, without updating the remaining classes.

Finally, this approach leads quite naturally to\marginpar{favor
\emph{composition} over \emph{inheritance}} favor \emph{composition} over
\emph{inheritance}. Due to the \emph{dynamic} typing of Python, without the
constraint to assign \emph{explicitly} a type to every expression, using the
proposed methodology we have built a code base that is highly polymorphic,
without constraining objects in rigid inheritance structures.

\subsection{Core classes and hierarchies}

\subsubsection{\emph{RiordanArray} class and \emph{Characterization} hierarchy}

The \emph{RiordanArray} class is the first of a set of \emph{interface} classes
that the client will work with.  Its responsibility is to denote the
\emph{mathematical} concept of Riordan array, but kept alone, it couldn't do
many things. It can be \emph{indexed} to supply a coefficient $m_{nk}$ it
contains but asking for raw matrix expansion or doing group operations, such as
multiplication or inversion, depends on how the Riordan array has been
\emph{characterized}, and this context information is caught by the
\emph{Characterization} hierarchy\marginpar{instances built from a class in the
Characterization hierarchy are dispatching objects}. We provide the following
characterizations:
\begin{itemize}
    \item by \emph{sequences}, that is using $A$ and $Z$ sequences, together with
        coefficient $m_{00}$;
    \item by \emph{matrices}, that is using $A$-matrices, where each $A$-matrix can
        be expressed as a table of coefficients $\lbrace a_{nk}\rbrace_{n,k\in\mathbb{N}}$
        or as a sequence of \ac{gf}, one for each matrix's row;
    \item by \emph{subgroup}, which essentially allows the client to use \emph{analytic
        functions} $d$ and $h$ to define the desired array. Up to now, only one
        subgroup is implemented, the \emph{vanilla} subgroup which resembles the
        formal way to define a Riordan array $\mathcal{M}$ as a pair of functions $(d(t),h(t))$.
        We will extend the set of \emph{subgroups} toward the one studied in the literature:
        this will allow the client to play with our object having a \emph{mathematical}
        experience, for example if an array in the \emph{Renewal} subgroup is desired,
        the client should only define function $d$, since function $h$ is implicitly
        defined as $h(t)=t\,d(t)$.
\end{itemize}

Therefore, with the introduction of \emph{characterization} context, denoted by
an object instance of the previous classes, it is possible for a
\emph{RiordanArray} instance to \emph{dispatch} messages on such
characterization that it cannot fullfil by itself, for example messages about
doing group operations, raw matrix expansion, building a formal
\emph{mathematical} name, formatted as a pair of functions $(d(t),h(t))$.

\subsubsection{\emph{TriangleColouring} class and auxiliary hierarchies}

The \emph{TriangleColouring} class is the second \emph{interface} class, which
denotes the desired colouring, namely a representatation of a Riordan array
under a congruence relation $\equiv_{p}$, for some modulo $p$. An object,
instance of this class, builds context objects about the desired layout of the
triangle (\emph{centered} or \emph{left-aligned}) or if negative coefficient in
an inverse array should be associated with \emph{lighter} colour variant
respect to positive coefficients, when both belong to the same remainder class of
$\equiv_{p}$ relation. Such additional context objects are instances of
\emph{auxiliary hierarchies}, such as \emph{TriangleShape} and
\emph{NegativeChoices}.

Python language allows to \emph{override} some \emph{special} methods that the
virtual machine calls on an object when it is used in a particular
\emph{syntatic context}. \marginpar{\mintinline{python}|__getindex__(index)| is
another message that the vm sends to an object $obj$ when it is indexed, as in
$obj[n,k]$} One of such method is \mintinline{python}|__call__|, which is sent
to an object \mintinline{python}|obj| when it is used like a function, for
example: \mint{python}|obj(array=anArray, partitioning=aPartitioning)| for some
objects \mintinline{python}|anArray| and \mintinline{python}|aPartitioning|. We
have \emph{overridden} such a method in order to provide a more \emph{fluent}
interface: recall that, eventually, our interest is to implement the function
$colouring$, so it is nice to be able to use the object that denotes such
functionality \emph{like} a function, as in \emph{mathematics} we are able to
do.

\subsubsection{\emph{Partitioning} hierarchy}

\emph{Partitioning} hierarchy denotes how the \emph{modular transformation} is
applied to an array $\mathcal{M}$. It is composed of three classes, one for
each implementation of \emph{math} function $colouring$ described in
\autoref{subsection:python:appendix:from:math:to:code}.

\subsubsection{\emph{ActionUsingSubgroupCharacterization} hierarchy}

Previous classes, hierarchies and auxiliaries build instances that belong to a
set of objects that we call \emph{dispatching objects}, since their responsibilities
are to dispatch messages, where each such object augment the message with the
context information it is responsible for.

Classes in the \emph{ActionUsingSubgroupCharacterization} hierarchy, on the other hand,
build instances that belong to a set of objects that we call \emph{action objects},
since they \emph{can} perform a functionality since they \emph{know} the entire context,
having collected all context informations \emph{dispatched} from remaining objects.
There are the following classes in this hierarchy, each one denoting a \emph{major}
functionality that is necessary to fullfil the implementation of the $colouring$
function\marginpar{action objects}:
\begin{itemize}
    \item \emph{ExpansionActionUsingSubgroupCharacterization} class denotes
        the operation of doing raw matrix expansion of an array $\mathcal{M}$;
    \item \emph{FormalDefLatexCodeUsingSubgroupCharacterization} class denotes
        the operation of producing a formal description of the array,
        formatted as $\mathcal{M}=(d(t),h(t))$;
    \item \emph{BuildInverseActionUsingSubgroupCharacterization} class denotes
        the operation of \emph{inverting} an array $\mathcal{M}$ to produce
        the array $\mathcal{M}^{-1}$.
\end{itemize}

All the above classes stricly depend on how the Riordan array has been
characterized, therefore they code a different implementation for each such
characterization. Looking at them from this point of view, it is easy to
recognize that the set of \emph{action objects} is actually a set of
\emph{visitors}. It is interesting how the\marginpar{Visitor pattern}
\emph{Visitor pattern} can be generalized to a net of \emph{dispatching
messages}, without limiting its application to hierarchies denoting a recursive
domain.

\section{A \emph{Sage} study file}

In this section we show a typical session where we study the \emph{Motzkin}
array $\mathcal{M}$.  First we report the complete script that we have used to
generate matrix raw expansion and coloured representations of
$\mathcal{M}_{\equiv_{p}}$, for some prime $p$; after we take it apart
describing the meaning of each chunk of code.

\subsection{The complete \emph{study} script for Motzkin array $\mathcal{M}$...}

\inputminted[
    mathescape,
    linenos,
    %numbersep=5pt,
    %gobble=2,
    frame=lines,
    framesep=2mm,
    breaklines
    ]{python}{../sympy/motzkin/motzkin-for-document-inclusion.sage}

\subsection{...and taking it apart, bite after bite}

Having seen the complete session script for array $\mathcal{M}$, in the
following paragraphs we take it apart, describing each little chunk in greater
detail.

\inputminted[
    mathescape,
    linenos,
    %numbersep=5pt,
    %gobble=2,
    frame=lines,
    framesep=2mm,
    breaklines,
    firstline=1,
    lastline=16
    ]{python}{../sympy/motzkin/motzkin-for-document-inclusion.sage}

In this chunk we do some boring \emph{imports} and prepare a variable
\mintinline{python}|tex_parent_prefix| in order to localize the destination
path for files that will be generated. Finally, variable \mintinline{python}|t|
is defined in order to denote a \emph{math} undeterminate variable
$t$\marginpar{\mintinline{python}|t| denotes a \emph{Python object}, $t$
denotes a \emph{math variable}}.

\inputminted[
    mathescape,
    linenos,
    %numbersep=5pt,
    %gobble=2,
    frame=lines,
    framesep=2mm,
    breaklines,
    firstline=19,
    lastline=25
    ]{python}{../sympy/motzkin/motzkin-for-document-inclusion.sage}

Here we define the desired partitioning, in this case we want
$\mathcal{M}_{\stackrel{\circ}{\equiv_{7}}}$, and the colouring object. It
keeps track of the number of rows, namely $127$; if the representation should
be centered or aligned to the left, as a raw matrix expansion looks like; and,
finally, if negative coefficients in the inverse array $\mathcal{M}^{-1}$
should get lighter colour variants respect to ones that positive coefficient in
$\mathcal{M}$ get, under congruence relation $\equiv_{7}$.

\inputminted[
    mathescape,
    linenos,
    %numbersep=5pt,
    %gobble=2,
    frame=lines,
    framesep=2mm,
    breaklines,
    firstline=32,
    lastline=45
    ]{python}{../sympy/motzkin/motzkin-for-document-inclusion.sage}

Here we define function $d$ and function $h$ of $\mathcal{M}$ and build an
object that denotes the \emph{mathematical} concept of Riordan array, composed
of the following slots\marginpar{we provide strong integration with \LaTeX}:
\begin{itemize}
    \item a \emph{subgroup}, in this case we use the \emph{plain vanilla} definition by
        providing functions $d$ and $h$ directly;
    \item a \emph{name}, which will be used to build unique filenames for
        the generated \TeX\,files;
    \item a \emph{mathematical name}, as the ones used in this document, such as $\mathcal{M}$ and so on,
        used within \emph{captions} environments, for example;
    \item an \emph{additional text} in order to augment array's description, it will be attached
        to \emph{captions} environments again. In this text \emph{any} \LaTeX\, code can be used.
\end{itemize}

\inputminted[
    mathescape,
    linenos,
    %numbersep=5pt,
    %gobble=2,
    frame=lines,
    framesep=2mm,
    breaklines,
    firstline=48,
    lastline=56
    ]{python}{../sympy/motzkin/motzkin-for-document-inclusion.sage}

Here some computation happen, actually: application of
\mintinline{python}|colouring| object expands $\mathcal{M}$ as a matrix
$\lbrace m_{nk}\rbrace_{n,k\in\mathbb{N}}$, up to row $126$ included, starting
from row $0$. Finally, it writes a bunch of files containing results, splitted
according to their content.

\inputminted[
    mathescape,
    linenos,
    %numbersep=5pt,
    %gobble=2,
    frame=lines,
    framesep=2mm,
    breaklines,
    firstline=58,
    lastline=62
    ]{python}{../sympy/motzkin/motzkin-for-document-inclusion.sage}

Here we tackle the settings for computing
$\mathcal{M}^{-1}$\marginpar{computing $\mathcal{M}^{-1}$ providing an helper
lambda expression}. Mathematically, it is necessary to compute the
\emph{computational inverse} $\hat{h}$ of function $h$ and this is the main
difficult point to solve, for \emph{Sage} \cite{sage} too.  \emph{Sage}
framework is pretty powerful, for instance it can compute function
$\hat{h}_{\mathcal{P}}$ for the Pascal array $\mathcal{P}$, but for Motzkin
array things get complicated because a radical appears in function $h$.

For this reason, \emph{message} \mintinline{python}|inverse| sent to the object
denoting $\mathcal{M}$ is a curious one, because it asks for an \emph{help}
function which will allow \emph{Sage} to solve an equation, yielding function
$\hat{h}$ eventually. Formally, to find $\hat{h}$ we could solve a
\emph{functional} equation $h(\hat{h}(t))=t$ because $h$ is known.  Otherwise,
we could use the \emph{change of variable} trick yielding $\hat{h}(y)$ if we
can solve $h(t)=y$:
\begin{displaymath}
    \left.\left[\hat{h}(y)=t\right|y=h(t)\right]
\end{displaymath}
so, at the end, we have to solve another equation respect to
variable $t$, obtaining it as a function of variable $y$.

This is exactly the aim of the helper function. It receive three arguments,
namely variable $y$, variable $t$ and an equation over function $h$ such that:
\begin{displaymath}
    y=h(t)
\end{displaymath}
which in the case of array $\mathcal{M}$ rewrites as:
\begin{displaymath}
    y=\frac{1-t-\sqrt{1-2t-3t^{2}}}{2t}
\end{displaymath}
our help is to manipulate the equation in order to remove the radical. Therefore
we can do the following steps, in the given order:
\begin{displaymath}
    \begin{split}
        y&=\frac{1-t-\sqrt{1-2t-3t^{2}}}{2t} &\quad\text{given}\\
        2ty &=1-t-\sqrt{1-2t-3t^{2}} &\quad\text{multiply by } 2t\\
        -2ty &=-1+t+\sqrt{1-2t-3t^{2}} &\quad\text{multiply by } -1\\
        1-t-2ty &=\sqrt{1-2t-3t^{2}} &\quad\text{add } -(t-1)\\
        \left(1-t-2ty\right)^{2} &=1-2t-3t^{2} &\quad\text{square}\\
    \end{split}
\end{displaymath}
which is exactly the meaning of the supplied lambda: \mint{python}|lambda y, t,
equation: (equation*2*t*(-1) -(t-1))**2|

\emph{Sage} is now able to solve the last equation respect to variable $t$ and it
can find the desired function $\hat{h}$ as:
\begin{displaymath}
    \left.\left[\hat{h}(y)=\frac{y}{1+y+y^{2}}\right|
        y=\frac{1-t-\sqrt{1-2t-3t^{2}}}{2t} \right]
\end{displaymath}

To be truly precise,\marginpar{the supplied helper works as a certificate as
well} the above \emph{helper} lambda expression is both an help to \emph{Sage}
to build the inverse of an array  and it is also a \emph{certificate} for the
computed compositional inverse function $\hat{h}$.  Under the hood it is used
also at the end of the search of $\hat{h}$ to check if it really satisfies
$\hat{h}(h(t))=t$.

\inputminted[
    mathescape,
    linenos,
    %numbersep=5pt,
    %gobble=2,
    frame=lines,
    framesep=2mm,
    breaklines,
    firstline=65,
    lastline=74
    ]{python}{../sympy/motzkin/motzkin-for-document-inclusion.sage}

Finally, as before, we actually do the computation that will expand
$\mathcal{M}^{-1}$ as a matrix of coefficients and will produce desired
\LaTeX\,files composed of code necessary to build the modular representation
$\left(\mathcal{M}^{-1}\right)_{\equiv_{7}}$.
\\\\
This is the most complete example our simple implementation allows us to do and,
although simple, it generates \emph{objects of beauty}.


\chapter{Expression Manipulation} \label{chap:manipulation}

The basic description of the types in Chap.~\ref{chap:basics} allows for defining expressions and evaluating them.
However, for most algorithms expressions need to be manipulated in one way or another, which is the topic of this chapter.
Unless otherwise noted, all manipulations considerered in the following can be used for both compiletime and runtime expressions using the same interface.

\TIP{Manipulation functionality resides in folder \lstinline|viennamath/manipulation/|.
 The respective header files are not included automatically with \lstinline|viennamath/expression.hpp| and need to be included as required.}

 \section{Evaluation}
All {\ViennaMath} expressions can be evaluated to a floating point number using the parenthesis operator.
A vector of values needs to be passed for the evaluation. In the special case that only the first entry of a vector is required for evaluation, it suffices to
pass the value directly.
%Depending on the number of variables in the expression, either a scalar or a vector needs to be passed for evaluation. For conv

Using \lstinline|operator()|, however, is possibly not an option for a generic interface with non-ViennaMath types.
For this reason, \lstinline|viennamath::eval()| provides a generic evaluation interface.
The first argument is the expression to be evaluated, and the second argument is the tuple with the values to be substituted for the variables.
For example, the expression $x^2$ is defined and evaluated at $x=2$ as follows:
\begin{lstlisting}
 ct_constant<2> c2;
 ct_variable<0> x;
 eval( x*x, 2.0 ); //     runtime evaluation
 eval( x*x,  c2 ); // compiletime evaluation
\end{lstlisting}
Note that compiletime evaluation is only performed when both arguments are fully compiletime compatible.
As soon as one part of the expression cannot be handled at compile time, a fallback to runtime evaluation is carried out.
A hybrid evaluation in such cases is postponed to future releases of {\ViennaMath}.

A vector of values needs to be passed as second argument, if a variable formally refers to any other than the first coordinate in a vector.
Let us consider several use-cases of \lstinline|eval()| consisting of various combinations of compiletime and runtime expressions:
\begin{lstlisting}
 ct_constant<3> c3;
 ct_variable<1> y;
 expr f = x*x + y;     // conversion to runtime expression
 eval( f, 2.0 );       // runtime exception: insufficient values provided
 eval( x*x + y, c2 );  // compilation error: insufficient values provided
 eval( x*x + y,
       make_vector(2.0, 3.0) ); //    runtime evaluation
 eval( x*x + y,
       make_vector(c2, c3) );  // compiletime evaluation
 eval( 2.0, 0.0 );             // possible thanks to overloading
\end{lstlisting}
Since the runtime wrapper \lstinline|expr| hides information from the compiler, an exception is thrown at runtime if insufficient values are provided for evaluation.
For a full compiletime evaluation, insufficient parameters are already detected at an earlier stage.
The helper function \lstinline|make_vector() | generates a suitable vector type both for the runtime and the compiletime case.
Instead of using \lstinline|make_vector()|, a STL vector (\lstinline|std::vector<double>|) or any compatible type can also be passed.
Also note that the last line in the code snippet shows the benefit of using \lstinline|eval()| instead of \lstinline|operator()|:
Scalars can also be 'evaluated' and are thus reinterpreted as constant functions.

 \section{Substitution} \label{sec:substitute}
Formally, the evaluation of an expression can be seen as a substitution of the variables with values.
A generalization is to replace arbitrary expressions with another expression in third expression.
This is accomplished by the function \lstinline|substitute()| defined in \lstinline|viennamath/manipulation/substitute.hpp|:
\begin{lstlisting}
 expr f = (x + y) * (x - y);
 substitute(x, y, f);         // returns (y + y) * (y - y) = 0
 substitute(x, 1, f);         // returns (1 + y) * (1 - y)
 substitute(x + y, x - y, f); // returns (x - y) * (x - y)
\end{lstlisting}
As with \lstinline|eval|, substitutions are carried out at compiletime if all parameters are compiletime expressions.

 \section{Expansion}
It is often desired to expand an expression given as a product of other terms. For example, instead of $2(x+y)$ one may want to have $2x - 2y$.
Such a functionality is provided by the function \lstinline|expand()| defined in \lstinline|viennamath/manipulation/expand.hpp|:
\begin{lstlisting}
 expand( c2 * (x+y) );
\end{lstlisting}
where \lstinline|c2| is a compiletime constant and \lstinline|x|, \lstinline|y| are compiletime variables.
Note that {\ViennaMathversion} does not support the expansion of runtime expressions yet.

\NOTE{ {\ViennaMathversion} supports compiletime expansion only. }

 \section{Simplification}
In the course of manipulating expressions, simple operations such as $x+0$ or $x/1$ may appear.
However, such terms constitute unnecessary overhead for later evaluations, thus it is desirable to have these operations dropped.
Such a simplification of the expression can be achieved with the function \lstinline|simplify()| defined in \lstinline|viennamath/manipulation/simplify.hpp|:
\begin{lstlisting}
 simplify( x + 1.0 * y - 0.0 );  // returns x + y
 simplify( x * (2.0 + 3.0) + (y * 0) / (x * 1) ); // returns 5x
\end{lstlisting}
\lstinline|simplify()| is available both for runtime and compiletime manipulation.

 \section{Differentiation}
The differentiation of an expression with respect to one or several variables is central to many algorithms, among which the Newton scheme is presumably the
most widely used. Differentiation routines in {\ViennaMath} reside in \lstinline|viennamath/manipulation/diff.hpp| and are used in a canonical way by passing
the expression to be differentiated as first argument and the differentiation variable as second argument:
\begin{lstlisting}
 diff( x + y, x );              // returns 1
 diff( (2.0 - x)*(3.0 + y), y); // returns 2.0 - x
\end{lstlisting}
As usually, compiletime expressions are differentiated at compiletime, while runtime expressions are differentiated at runtime.
An exemplary Newton-solver demonstrating the use of differentiation routines can be found in \lstinline|examples/newton_solve.cpp|.

 \section{Integration}
The integration of an expression over an interval can be accomplished in two ways: The first option is by analytical integration, provided that an
antiderivative of the integrand can be found easily. The second option is by numerical quadrature using a suitable quadrature rule.

 %% Compiletime: Analytic
Analytical integration is in {\ViennaMathversion} available for compiletime types and polynomials as integrands only.
The file \lstinline|viennamath/manipulation/integrate.hpp| provides the function \lstinline|integrate()|, which takes the integration interval
as first argument, the integrand as second argument and the integration variable as third argument. For example, the integral
\begin{align}
 \int_0^1 x^2 \: \mathrm{d} x
\end{align}
is computed analytically at compile time using the lines
\begin{lstlisting}
 integrate( make_interval(c0, c1),
            x * x,
            x);              //returns the expression 1/3
\end{lstlisting}
where \lstinline|c0| and \lstinline|c1| denote the compiletime constants $0$ and $1$ and are passed to the helper function \lstinline|make_interval()|
to generate a suitable compiletime interval with the provided lower and upper bound. Note that analytic integration can also be nested and use polynomial lower
and upper bounds. For example, integration of $x^2$ over the unit triangle with vertices $(0,0)$, $(1,0)$ and $(0,1)$ is achieved via
\begin{lstlisting}
 integrate( make_interval(c0, c1),
            integrate( make_interval(c0, c1 - x),
                       x * x,
                       y),
            x);              //returns 1/12
\end{lstlisting}

\NOTE{ Analytic integration in {\ViennaMathversion} is available for polynomial integrands at compiletime only. }

 %% Runtime: Quadrature
\begin{table}
\centering
\begin{tabular}{|l|l|l|l|}
\hline
Name & {\ViennaMath} Type   & Shortcut & Accuracy \\
\hline
$1$-point Gauss   & \lstinline|rt_gauss_quad_1| & \lstinline|gauss_quad_1| & $1$ \\
\hline
\end{tabular}
\caption{Overview of numerical quadrature rules in {\ViennaMath}. \label{tab:quadrature}}
\end{table}

In order to compute an integral numerically using a quadrature rule, the respective rule from Tab.~\ref{tab:quadrature} needs to be instantiated first.
For a $1$-point Gauss rule, this is accomplished by
\begin{lstlisting}
 rt_numerical_quadrature<InterfaceType>
    integrator(new rt_gauss_quad_1<InterfaceType>());
\end{lstlisting}
for a suitable runtime interface type. If the default interface type is to be used, the shortcut types can be used for convenience:
\begin{lstlisting}
 numerical_quadrature integrator(new gauss_quad_1());
\end{lstlisting}
To carry out the numerical quadrature, two options exist: The first is to pass the integration interval, the integrand, and
the integration variable as separate arguments to \lstinline|operator()| of the \lstinline|integrator| object:
\begin{lstlisting}
 integrator( make_interval(0.0, 1.0),
             x * x,
             x);             // returns the value 0.25
\end{lstlisting}
The second option for numerical quadrature is to encode the integral directly in the expression:
\begin{lstlisting}
  expr my_integral = integral( make_interval(0, 1), x * x, x );
\end{lstlisting}
which encodes $\int_0^1 x^2 \: \mathrm{d} x$ directly in an expression. For numerical quadrature, only the encoded form needs to be passed to the quadrature
rule then
\begin{lstlisting}
 integrator(my_integral);    // returns the value 0.25 (1-point Gauss)
\end{lstlisting}

 \section{Extract Coefficient}
Given a polynomial $p(x,y) = 1 + x + y + 2xy$ it can be of interest to extract individual parts of the polynomial.
For example, one wishes to extract the coefficient of $x$, which is $(1+2y)$. In such a case,
the function \lstinline|coefficient()| defined in \lstinline|viennamath/manipulation/coefficient.hpp| can be used.
The first parameter is the variable or expression for which the coefficient should be returned, and the second argument is the expression from which the
coefficient is to be extracted:
\begin{lstlisting}
 coefficient(x, c1 + x + y + c2*x*y);  //returns 1+2y
\end{lstlisting}
Note that higher-order terms in the variable are also returned. For example, the coeffient of $x$ in $x+x^2$ is obtained as $(1+x)$.

\NOTE{ \lstinline|coefficient()| is in {\ViennaMathversion} available for compiletime types only. }

 \section{Drop Dependent Terms}
In order to drop all terms in an expression which depend on a certain expression type (not necessarily a variable), the convenience function
\lstinline|drop_dependent_terms()| from \lstinline|viennamath/manipulation/drop_dependent_terms.hpp| can be used. As the name suggests, all terms with a
dependency on the expression passed as first parameter are dropped in the expression passed as second variable.
For example, all terms depending on $x$ in $1+x+y+2xy$ are dropped using the line
\begin{lstlisting}
 drop_dependent_terms(x, c1 + x + y + c2*x*y);  //returns 1+y
\end{lstlisting}
in order to obtain $1+y$.

\NOTE{ \lstinline|drop_dependent_terms()| is in {\ViennaMathversion} available for compiletime types only. }


\newpage\section{Research Stuffs for later}

	\prob{https://artofproblemsolving.com/community/c6h1499703p8903397}{AoPS}{}{Let $ABC$ be a triangle with incenter $I.$ $L_a,$ $L_b,$ $L_c$ are symmedian points of triangles $IBC,$ $ICA,$ $IAB.$ Let $X,Y,Z$ be the reflections of $I$ through $L_a,$ $L_b,$ $L_c.$

		\begin{itemize}

			\item Prove that $AX,$ $BY,$ $CZ$ and $OI$ are concurrent.

			\item Let $I_a,$ $I_b,$ $I_c$ be the excenters of $ABC.$ Prove that $I_aX,$ $I_bY,$ $I_cZ$ are concurrent at a point $P$ and isogonal conjugate of $P$ with respect to triangle $I_aI_bI_c$ lies on Euler line of $ABC.$

	\end{itemize}}

	\prob{https://artofproblemsolving.com/community/c6h1180223}{buratinogigle Tough P1}{}{Let $ABC$ be a triangle inscribed in circle $(O)$ with $A$-excircle $(J)$. Circle passing through $A,B$ touches $(J)$ at $M$. Circle passing through $A,C$ touches $(J)$ at $N$. $BM$ cuts $CN$ at $P$. Prove that $AP$ passes through tangent point of $A$-mixtilinear incircle with $(O)$.}


\documentclass[./\jobname.tex]{subfiles}
\begin{document}

\section{State of the Art}
\label{chap:state_of_the_art}

\subsection{Finite Element Method}
Currently, the Finite Element Method is the go-to approach to solve partial differential equations. The domain $\Omega$ on which the \gls{pde} is posed, is discretised into multiple smaller elements - as the name suggests. Thus, \gls{fem} counts to the category of meshed methods. The underlying solution function $u(\mathbf{x})$ to the PDE is then approximated by so called ``basis-functions'' $\Phi(\mathbf{x})$ limited to these finite elements. This thesis uses the open-source Netgen/NGSolve \gls{fem} package \cite{schoberl_ngsolvengsolve_2020}.\\
The general steps taken to solve a PDE with an FEM solver are:
\begin{enumerate}
	\item \underline{Step: Strong Form} \\
		  This is the standard formulation of the linear \gls{pde}. $\mathbf{L}$ and $\mathbf{B}$ are linear differential operators that include the derivatives. \\
		  \begin{equation}
		  \label{eq: strong form}
			  \begin{split}
			  	\mathbf{x} \in \mathbb{R}^2 \\
			  	u(\mathbf{x}), f(\mathbf{x}), g(\mathbf{x}): \Omega \rightarrow \mathbb{R} \\
				\mathbf{L} u(\mathbf{x}) = f(\mathbf{x}) \text{ on $\Omega$} \\
				\mathbf{B} u(\mathbf{x}) = g(\mathbf{x}) \text{ on $\partial \Omega$}
			  \end{split}
		  \end{equation}
		  Further, only Dirichlet boundary conditions are considered, thus the boundary operator is always the identity matrix $\mathbf{B} = \mathbb{I}$. Therefore, the linear operator on the boundary $\mathbf{B}$ can be disregarded, resulting in  \\
		  \begin{equation}
		  	u(\mathbf{x})|_{\partial \Omega} = g(\mathbf{x}) .
		  \end{equation}

	\item \underline{Step: Weak Form} \\
		  The next step is to reformulate the strong form into a usable weak form. This is equivalent to the strong form but written in an integral notation. In this equation, the $A$, $b$ and $c$ correspond to the constant factors of the derivatives in strong form. For the sake of completeness, this is kept abstract. In the \gls{pde}s considered in this work, $A = \mathbb{I}$ and $b=\mathbf{0}$, $c = 0$. Currently, the so-called test-function $v(\mathbf{x})$ is an arbitrary function, but it has to be 0 on the boundary $v(\mathbf{x})|_{\Omega} = 0$. The choice of the test-function correspond to different \gls{fem} types (\cite[p. 6f]{shen_spectral_2011}).\\
		  \begin{equation}
		  \label{eq: weak form}
		  \begin{split}
			  a(u,v) &
			  \begin{cases}
				  & ~ \int_{\Omega} - (\nabla^T A \nabla) u(\mathbf{x}) v(\mathbf{x}) dV \\
				  & - \int_{\Omega} b^T \nabla u(\mathbf{x}) v(\mathbf{x}) dV \\
				  & + \int_{\Omega} c u(\mathbf{x}) v(\mathbf{x}) dV
			  \end{cases} \\
			  F(v) &
			  \begin{cases}
			  	& = \int_{\Omega} f(\mathbf{x}) v(\mathbf{x}) dV
			  \end{cases}
		  \end{split}
		  \end{equation}
	\item \underline{Step: Discretisation of $\Omega$} \\
		  Create a mesh of finite elements that span the whole domain. Usually these are triangles. Thus, this step is sometimes called ``triangulation''.
	\item \underline{Step: Basis functions} \\
		  Choose a basis function $\Phi(\mathbf{x})$ that can be used to approximate the solution $u(\mathbf{x}) \approx u_{h}(\mathbf{x}) = \sum_{i = 1}^{N} u_i \Phi_i(\mathbf{x})$. A common choice are Lagrange or Chebyshev polynomials. In the Galerkin type \gls{fem}, the test-function $v(\mathbf{x})$ is the same as the trail-function, thus $v(\mathbf{x}) = \sum_{j = 1}^{N} v_j \Phi_j(\mathbf{x})$. The choice of the basis function $\Phi(\mathbf{x})$ largely influences the computational effort.  $\Phi(\mathbf{x})$ should have a small support, to produce a thinly populated matrix $A$ in the linear system of equations \eqref{eq:linear_system_of_equations} below.
	\item \underline{Step: Solution} \\
		  In the weak form, as seen in equation \eqref{eq: weak form}, $a(u,v)$ is a continuous bilinear form and $F(v)$ is a continuous linear functional. Substituting $u$ and $v$ with their corresponding approximation from \mbox{step 4} results in
		  \begin{equation}
		  \sum_{j=1}^{N} v_j \sum_{i=1}^{N} u_i a(\Phi_i, \Phi_j) = \sum_{j=1}^{N} v_j F(\Phi_j).
		  \end{equation}
		  Dividing by the $v_j$ values on both sides results in a linear system of equations, where the constant factors $u_i$ need to be determined.
		  \begin{equation}
		  \label{eq:linear_system_of_equations}
		  \underbrace{\sum_{i=1}^{N} u_i a(\Phi_i, \Phi_j)}_{\mathbf{A u}} = \underbrace{F(\Phi_j)}_{\mathbf{b}} \text{ for $j=1,...N$}
		  \end{equation}
\end{enumerate}

Modern solvers include more complex and advanced techniques to further improve the solution error and the computation time. Some of the most important concepts that are also available in NGSolve are listed here.

\begin{itemize}
	\item \underline{Static Condensation}: \\
		  Depending on the number of discrete elements, the $\mathbf{A}$ matrix can be very large. Inverting large matrices is very time consuming. Static condensation, also called Guyan reduction (\cite{guyan_reduction_1965}), reduces this dimensionality by exploiting the structure of $\mathbf{A}$.
	\item \underline{Preconditioner}: \\
		  Instead of solving the $\mathbf{A}^{-1}$ exactly, this can also be approximated by a matrix that is similar to $\mathbf{A}^{-1}$. The actual inverse can be iteratively approximated. NGSolve implements multiple different preconditioners and it even allows to create your own method.
	\item \underline{Adaptive Mesh Refinement}: \\
		The accuracy of a FEM-approximated solution mainly depends on the density of the mesh. Typically, finer meshes tend to produce more accurate solutions, but the computation time is longer. This trade-off can be overcome by a self-adaptive mesh. NGSolve implements that in an adaptive loop that executes:
		\begin{itemize}
			\item Solve PDE (with coarse mesh)
			\item Estimate Error (for every element)
			\item Mark Elements (that have the greatest error)
			\item Refine Elements (that were previously marked)
			\item Repeat until \gls{dof} exceed a specified $N$
		\end{itemize}
\end{itemize}

\subsection{Computational Intelligence Methods}
\label{chap:literature_overview}
The following table \ref{tab:literature_research} gives a brief overview of these papers and sorts them historically. In general, all of these papers from the table use the \gls{wrm}, or some variant of that concept, to transform their differential equation into an optimisation problem. This serves as the fitness function and is necessary to evaluate a possible candidate solution and perform the evolutionary selection. The fitness function is also called objective function and these terms are used interchangeably in this paper. In short, the residual $R$ is defined through the differential equation itself and can be calculated by $R(u(\mathbf{x})) = \mathbf{L}u(\mathbf{x}) - f(\mathbf{x})$. The residual can be thought of as a functional that substitutes $u(\mathbf{x})$ with an approximate solution $u_{apx}(\mathbf{x})$ and returns a numerical score. \\
This paper mainly builds on the work presented in \cite{chaquet_using_2019}. In their paper they describe an algorithm that approximates a solution with a linear combination of Gaussian \gls{rbf} as kernels:
\begin{equation}
u(\mathbf{x})_{apx} = \sum_{i=1}^{N} \omega_i e^{\gamma_i (\left||\mathbf{x} - \mathbf{c}_i\right||^2)}
\end{equation}
The approximated function $u(\mathbf{x})_{apx}$ can be fully determined by a finite number of parameters: $\omega_i, \gamma_i, \mathbf{c}_i$. These are stacked together into a vector $\mathbf{p_{apx}}$ and called the decision variables which are optimised by the algorithm. The objective function can be seen in equation \eqref{eq:fit_func_chaquet}.
\begin{equation}
\label{eq:fit_func_chaquet}
\begin{split}
F(u_{apx}(\mathbf{x})) = \frac{1}{m (n_C + n_B)} \\ \left[ \sum_{i=1}^{n_C} \right. \xi (\mathbf{x}_i) || \mathbf{L}u_{apx}(\mathbf{x}_i) - f(\mathbf{x}_i)||^2 \\ + \left. \phi \sum_{j=1}^{n_B} || \mathbf{B}u_{apx}(\mathbf{x}_j) - g(\mathbf{x}_j)||^2 \right]
\end{split}
\end{equation}
The multipliers $\xi(\mathbf{x}_i)$ and $\phi$ are weighting factors for either the inner or the boundary term. The whole term is normalised with the number of collocation points. The parameters of the kernels are determined via a \gls{cma_es} \cite{hansen_reducing_2003}. To further improve the solution, the evolutionary algorithm is coupled with a \gls{ds} method to carry out the local search. The authors show empirically that the local search significantly improves the performance by testing the algorithm on a set of 32 differential equations.

\begin{table}[h]
	\centering
	\noindent\adjustbox{max width=\linewidth}{
		\begin{tabular}{|c|c|c|c|}

			\hline
			\rowcolor[HTML]{\farbeTabA}

			Paper & Algorithm & Representation & Problems \\ \hline

			\multilinecell{\cite{howard_genetic_2001}} & \multilinecell{GP} & \multilinecell{polynomial of \\ arbitrary length} & \multilinecell{one-dimensional \\ steady-state \\ model of \\ convection-diffusion \\ equation} \\ \hline

			\multilinecell{\cite{kirstukas_hybrid_2005}} & \multilinecell{GP} & \multilinecell{algebraic \\ expression} & \multilinecell{heating of thin rod \\ heating by current} \\ \hline

			\multilinecell{\cite{tsoulos_solving_2006}} & \multilinecell{GE} & \multilinecell{algebraic term} & \multilinecell{set of ODEs \\ system of ODEs \\ and PDEs} \\ \hline

			\multilinecell{\cite{mastorakis_unstable_2006}} & \multilinecell{GA\\(global); \\ DS\\(local)} & \multilinecell{5th order \\ polynomial}& \multilinecell{unstable \\ ODEs} \\ \hline

			\multilinecell{\cite{sobester_genetic_2008}} & \multilinecell{GP \\ and \\ RBF-NN} & \multilinecell{algebraic term \\ for inner; \\ RBF for boundary} & elliptic PDEs \\ \hline

			\multilinecell{\cite{howard_genetic_2011}} & \multilinecell{GP} & function value grid & \multilinecell{convection–diffusion \\ equation \\ at different \\ Peclet numbers } \\ \hline

			\multilinecell{\cite{chaquet_solving_2012}} & \multilinecell{ES} & \multilinecell{partial sum \\ of Fourier series} & \multilinecell{testbench of \\ ODEs \\ system of ODEs \\ and PDEs} \\ \hline

			\multilinecell{\cite{babaei_general_2013}} & \multilinecell{PSO} & \multilinecell{partial sum\\of Fourier series} & \multilinecell{integro-differential equation\\system of linear ODEs \\ Brachistochrone \\ nonlinear Bernoulli} \\ \hline

			\multilinecell{\cite{panagant_solving_2014}} & \multilinecell{DE} & \multilinecell{polynomial of \\ unspecified order} & \multilinecell{set of 6 \\ different PDEs}  \\ \hline

			\multilinecell{\cite{sadollah_metaheuristic_2017}} & \multilinecell{PSO\\HS\\WCA} & \multilinecell{partial sum\\of Fourier series} & \multilinecell{singular BVP} \\ \hline

			\multilinecell{\cite{chaquet_using_2019}} & \multilinecell{CMA-ES\\(global); \\ DS\\(local)} & \multilinecell{linear combination \\ of Gaussian kernels} & \multilinecell{testbench of \\ ODEs \\ system of ODEs \\ and PDEs}\\ \hline

			\multilinecell{\cite{fateh_differential_2019}} & \multilinecell{DE} & \multilinecell{function value\\grid} & elliptic PDEs \\ \hline

		\end{tabular}
	}
	\unterschrift{Literature research on the general topic of stochastic solver and their application. The papers are sorted by date of release. }{}{}
	\label{tab:literature_research}
\end{table}

\subsection{Differential Evolution}

The differential evolution framework was first introduced in \cite{storn_differential_1997}. Due to its simple and flexible structure, it quickly became one of the most successful evolutionary algorithm. Over the years, several adaptations to the original framework have been proposed. Some of them currently count to the best performing algorithms, as the 100-Digit Challenge at GECCO 2019 \cite{suganthan_suganthancec2019_2020} shows. The main \gls{de} framework consists of three necessary steps that continuously update a population of possible solutions. The population can be interpreted as a matrix, where each row-vector $\mathbf{x}_i$, also called individual, represents a point within the search domain and has a fitness according to the fitness function $f(\mathbf{x}_i): \mathbb{R}^n \rightarrow \mathbb{R}$. The goal is to minimise the fitness function. These steps are performed in a loop until a predefined termination condition is reached. Each step is controlled by a user-defined parameter:
\begin{itemize}
	\item \underline{Mutation}: \\
		  Mutation strength parameter F;\\
		  The mutation uses the information from within the population to create a trial vector $v_i$. This is done by scaling the difference between some vectors in the population. The \textit{/current-to-pbest/1} mutation operator can be seen in equation \eqref{eq:mut_rand_1} where $x_i$ is the current individual, $x_{best}^p$ is one random vector of the p\% top vectors, $x_{r1}$ is a random vector from the population while $\tilde{x}_{r2}$ is randomly chosen from the population and the archive. $x_{r1}$ and $\tilde{x}_{r2}$ must not describe the same individual.
		  \begin{equation}
		  \label{eq:mut_rand_1}
		  v_i = x_{i} + F_i(x_{best}^p - x_{i}) + F_i(x_{r1} - \tilde{x}_{r2})
		  \end{equation}
	\item \underline{Crossover}: \\
		  Crossover probability parameter CR;\\
		  The crossover procedure randomly mixes the information between the trial vector $v_i$ and a random candidate from the population $x_{i}$ to create a new trial vector $u_i$. The binomial crossover from equation \eqref{eq:crs_bin} randomly takes elements from both vectors, where $K$ is a random index to ensure that at least one element from the trial vector $v_i$ is taken.
		  \begin{equation}
		  \label{eq:crs_bin}
		  u_{ij}=\begin{cases}
		  v_{ij}, &\text{if $j = K \lor rand[0,1] \leq CR$}\\
		  x_{ij}, &\text{otherwise}
		  \end{cases}
		  \end{equation}
	\item \underline{Selection}: \\
		  Population size N;\\
		  The selection replaces the old candidate $x_i$ if the trial candidate $u_i$ is better as measured by the fitness function. This is performed for every individual in the population, then the next generation is started.
\end{itemize}  ~\\
In modern \gls{de} variants, these parameters are self-adapted during the evolutionary process. This means that the algorithms can balance out between exploration of the search-space and exploitation of promising locations. A prominent example of a modern \gls{de} with self-adaption is JADE, which is described in \cite{zhang_jade_2009}. The adaption is performed by taking successful F and CR values of the last generation into account. If a certain setting is successful in generating better candidates, newly selected F and CR gravitate towards that setting.

\end{document}


\documentclass[a4paper]{article}

%% Language and font encodings
\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage[T1]{fontenc}

%% Sets page size and margins
\usepackage[a4paper,top=3cm,bottom=2cm,left=3cm,right=3cm,marginparwidth=1.75cm]{geometry}

%% Useful packages
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{graphicx}
\usepackage[colorinlistoftodos]{todonotes}
\usepackage[colorlinks=true, allcolors=blue]{hyperref}

\title{Judson's Abstract Algebra: Chapter 9}
\date{}

\begin{document}
\maketitle

\section*{1}

Prove that $\mathbb{Z} \cong n \mathbb{Z}$ for $n \neq 0$.

\vspace{\baselineskip}

Let $n \in \mathbb{Z}, n \neq 0$. Consider the function $f : \mathbb{Z} \rightarrow n \mathbb{Z}$

$$f(x) = nx.$$

Let $x,y \in \mathbb{Z}$. Note that

$$f(x+y) = n(x+y) = nx + ny = f(x) + f(y).$$

Assume $f(x) = f(y)$. Then

$$f(x) = f(y)$$
$$nx = ny$$
$$x = y$$

so $f$ is injective. Note that $f$ is surjective by definition. Since $f$ is a bijection and preserves the group operation $f$ is an isomorphism.

\section*{2}

Prove that $\mathbb{C}^*$ is isomorphic to the subgroup of $GL_2(\mathbb{R}$ consisting of matrices of form

$$
  \begin{pmatrix}
    a & b \\
    -b & a
  \end{pmatrix}.$$

We will call the subgroup described above $M$. Consider a function $f : \mathbb{C}^* \rightarrow M$ defined by

$$f(a+bi) = \begin{pmatrix}
    a & b \\
    -b & a
  \end{pmatrix}.$$

Let $a,b,c,d \in \mathbb{R}$ where $ab \neq 0$ and $cd \neq 0$. First note that the function preserves the group operation since

\begin{align*}
f((a+bi)(c+di)) &= f(ac-db + (db + da)i) \\
&= \begin{pmatrix}
    ac-db & cb + da \\
    cb + da & ac - db
  \end{pmatrix} \\
&= = \begin{pmatrix}
    a & b \\
    -b & a
  \end{pmatrix}
  = \begin{pmatrix}
    c & d \\
    -d & c
  \end{pmatrix} \\
&= f(a+bi)f(c+di)
\end{align*}

Note that $f$ is surjective because $a,b$ are arbitrary real numbers. Notice that $f$ is injective because of element-wise equality of matrices.

\section*{7}

Show that any cyclic group of order $n$ is isomorphic to $\mathbb{Z}_n$.

\vspace{\baselineskip}

Let $G$ be a cyclic group of order $n$ generated by $a$ and define a function $\phi : \mathbb{Z}_n \rightarrow G$ by $\phi(k) = a^k$ where $0 \leq k < n$. Let $x,y \in \mathbb{Z}_n$.
Notice that since

$$\phi(x + y) = a^{x+y} = a^x a^y = \phi(x) \phi(y)$$

that $\phi$ preserves the group operation. Assume $\phi(x) = \phi(y)$. Then

$$\phi(x) = \phi(y)$$
$$a^x = a^y$$
$$a^x a^{-y} = e$$
$$a^{x-y} = e.$$

Hence $x - y \equiv 0 \mod n$ and therefore $x \equiv y \mod n$ and $\phi$ is injective. Let $g \in G$. Since $G$ is cyclic, there exists $k \in \mathbb{Z}, 0 \leq k < n$ such that

$$g = a^k = \phi(k)$$

therefore $\phi$ is surjective. This proves that $\phi$ is an isomorphism, hence any cyclic group of order $n$ is isomorphic to $\mathbb{Z}_n$.

\section*{9}

Let $G = \mathbb{R} \setminus \{ -1 \}$ and define a binary operation on $G$ by

$$a * b = a + b + ab.$$

Prove that $G$ is a group under this operation. Show that $(G, *)$ is isomorphic to the multiplicative group of nonzero real numbers.

\vspace{\baselineskip}

Let $a,b,c \in G$. Note that $G$ contains an identity since

$$a * 0 = a + 0 + 0a = a.$$

Note $G$ contains inverses since

\begin{align*}
a * \frac{-a}{1+a} &= a + \frac{-a}{1+a} + \frac{-a^2}{1+a} \\
&= \frac{a + a^2}{1+a} + \frac{-a}{1+a} + \frac{-a^2}{1+a} \\
&= 0.
\end{align*}

Consider

\begin{align*}
(a*b) * c &= (a + b + ab) * c \\
&= a + b + ab + c + c (a + b + ab) \\
&= a + b + c + ab + ac + bc + abc \\
&= b + c + bc + a + a (b + c + bc) \\
&= a * (b + c + bc) \\
&= a * (b*c)
\end{align*}

shows that $*$ is associative. We must now find a bijection $f : G \rightarrow \mathbb{R} \setminus \{ 0 \}$. Let $f(x) = 1 + x$. Consider

\begin{align*}
f(a*b) &= f(a + b + ab) \\
&= (1 + a + b + ab) \\
&= (1 + a) (1 + b) \\
&= f(a) f(b).
\end{align*}

Hence $f$ preserves the group operation. It is clear that $f$ is injective since

\begin{align*}
f(a) &= f(b) \\
1 + a &= 1 + b \\
a &= b.
\end{align*}

It is clear that $f$ is surjective since for $x \in \mathbb{R} \setminus \{ 0 \}$

$$f(x - 1) = x$$

and $(x - 1) \in G$.

\section*{27}

Let $G \equiv H$. Show that if $G$ is cyclic, then so is $H$.

\vspace{\baselineskip}

Let $a$ be a generator of $G$. Then for all $g \in G$ there exists $k \in \mathbb{Z}$ such that $a^k = g$. Let $f: G \rightarrow H$ be an isomorphism. Let $h \in H$. There exists $g \in G$ such that

\begin{align*}
h &= f(g) \\
&= f(a^k) \\
&= f(a)^k.
\end{align*}

Notice that $f(a)$ is a generator of $H$ since $h$ is arbitrary. This proves that $H$ is a cyclic group.

\section*{28}

Prove that any group $G$ of order $p$, $p$ a prime, must be isomorphic to $\mathbb{Z}_p$.

\vspace{\baselineskip}

Recall Theorem 9.3: If $G$ is a cyclic group of order $n$, then $G$ is isomorphic to $\mathbb{Z_n}$. Also recall Corollary 6.7: Let $|G| = p$ with $p$ a prime number. Then $G$ is cyclic and any $g \in G$ such that $ \neq e$ is a generator. The desired result follows immediately from combining these two results.

\section*{34}

An \textit{automorphism} of a group $G$ is an isomorphism with itself. Prove that complex conjugation is an automorphism of the additive group of complex numbers; that is, show that the map $\phi(a+bi) = a - bi$ is an isomorphism from $\mathbb{C}$ to $\mathbb{C}$.

\vspace{\baselineskip}

Note that since

\begin{align*}
\phi((a+bi) + (c+di)) &= \phi((a+c) + (b+d)i) \\
&= (a+c) - (b+d)i \\
&= (a-bi) + (c-di) \\
&= \phi(a+bi) + \phi(c+di)
\end{align*}

we know that $\phi$ preserves the group operation. The proof that $\phi$ is surjective is trivial (to get any element apply $\phi$ to the element's inverse). To show that $\phi$ is injective consider

$$\phi(a+bi) = \phi(c+di)$$
$$\overline{\phi(a+bi)} = \overline{\phi(c+di)}$$
$$a + bi = c + di.$$

\section*{35}

Prove that $a + ib \rightarrow a- ib$ is an automorphism of $\mathbb{C}^*$.

\vspace{\baselineskip}

Call the map $f$. Note that by problem 34 we know that $f$ is a bijection. To show that $f$ preserves the group operation consider

\begin{align*}
f((a+bi) (c+di)) &= f((ac - bd) + (ad + bc)i) \\
&= (ac - bd) - (ad + bc)i \\
&= (a-bi)(c-di) \\
&= f(a+bi) f(c+di).
\end{align*}

\section*{37}

We will denote the set of all automorphisms of $G$ by Aut$(G)$. Prove that Aut$(G)$ is a subgroup of $S_G$, the group of permutations of $G$.

\vspace{\baselineskip}

Note that the identity automorphism is the function $x \rightarrow x$. Clearly any automorphism composed with another automorphism results in an automorphism. Since automorphisms are bijections clearly inverses are contained within Aut$(G)$. This proves Aut$(G)$ is a subgroup of $S_G$.

\section*{50}

Prove that $A \times B$ is abelian if and only if $A$ and $B$ are abelian.

\vspace{\baselineskip}

Assume $A$ and $B$ are abelian. Let $(a_1, b_1), (a_2, b_2) \in A \times B$. Note that since

\begin{align*}
(a_1, b_1) (a_2, b_2) &= (a_1 a_2, b_1 b_2) \\
&= (a_2 a_1, b_2 b_1) \\
&= (a_2, b_2) (a_1, b_1)
\end{align*}

it is true that $A \times B$ is abelian. Assume that $A \times B$ is abelian. Let $(a_1, b_1), (a_2, b_2) \in A \times B$. Consider

\begin{align*}
(a_1, b_1) (a_2, b_2) &= (a_2, b_2) (a_1, b_1) \\
(a_1 a_2, b_1 b_2) &= (a_2 a_1, b_2 b_1).
\end{align*}

This shows that $a_1 a_2 = a_2 a_1$ and that $b_1 b_2 = b_2 b_1$, therefore $A$ and $B$ are abelian.

\end{document}


\section{Accuracy of the merge algorithm}

The overall numerical accuracy of any algorithm, is of crucial
relevance in the area of Numerical Analysis; in particular in the
subarea we care about in this project (Numerical Linear
Algebra). \Rehurek offers detailed and promising accuracy comparisons
between his proposal and several other available implementations,
though he does that mainly for the serial executions (single node) of
small/medium corpus sizes (not the Wikipedia experiment described in
previous section). Another pending task to verify the accuracy against
a golden standard, could be to perform experiments on a supercomputer
with enough RAM to hold the large corpus matrix; using an standard
SVD software like \cite{svdlibc}. The authors of these lines have added
such pending task, to the TODO list for further stages of
this project. \\

Despite of the above, an interesting analysis about the effect of
nested truncation that \cref{alg:svd-dist} introduces, is exposed in
\Rehurek Phd thesis \cite{rehurek11a}. Citing the work of Zha et al
(\cite{zha00}), he remarks that his distributed SVD algorithm meets the conditions to
be an stable algorithm (on the numerical sense), though no longer
exact. This should not surprise us, as the almost perfect parallelism
achieved can not come without a price: every merge of two SVD
factorizations, as produced by \cref{alg:merge-svd}, introduces some
error, in the sense that the following equality does not hold: \\

\begin{equation}
\label{eq:merge-svd-eq}
\func{SVD_k}(\begin{bmatrix}A_1 \mid A_2\end{bmatrix}) =
\func{SVD_k}(\begin{bmatrix}\func{SVD_k}(A_1) \mid \func{SVD_k}(A_2)\end{bmatrix})
\end{equation}
\hfill

In other words, calculating truncated SVD against the original input
matrices $A_1$ and $A_2$ (concatenated by columns), is not the same as
calculating the same over their truncated $SVD_k$ approximations. Let
us recall that the matrix produced by $SVD_k(A)$ is just an approximation
of original matrix $A$. \\

The precision lost by accepting as inputs rank-$k$ approximations,
instead of the original matrices, is not that bad though; it is shown in
\cite{zha00} and reused by Rehurek in \cite{rehurek11a}, that the
typical matrix $A$ that emerges from Natural Language
Applications like LSI, ``do indeed possess the necessary structure and
that in this case, a rank-$k$ approximation of A can be expressed as a
combination of rank-k approximations of its submatrices without a
serious loss of precision''. \\

The above quote means, that the equality \cref{eq:merge-svd-eq} can
be considered to hold in practice. In strict theory we shall replace the
equality sign by an approximation sign though, as the equality sign
can be stated only on the idealistic case of exact arithmetic. Hence,
we can claim that: \\

\begin{equation}
\label{eq:merge-svd-app}
\func{SVD_k}(\begin{bmatrix}A_1 \mid A_2\end{bmatrix}) \approx
\func{SVD_k}(\begin{bmatrix}\func{SVD_k}(A_1) \mid \func{SVD_k}(A_2)\end{bmatrix})
\end{equation}
\hfill

This is in part, because the $SVD_k$ factorization of a matrix $A$ is
not actually just an approximation (as we claimed paragraphs above),
it is  ``the best'' approximation by a matrix of rank $k$, per the
Eckart-Young Theorem \cite{eckart36}. The \cref{eq:merge-svd-app},
derived from the work of \cite{zha00}, can be considered the angular
stone for the divide-and-conquer strategy of
the distributed \cref{alg:svd-dist}. Without it, we would not know if
it is valid to use the $\func{SVD}_k$ calculation itself, as a way of
combining two already calculated $\func{SVD}_k$ factorizations. A quite
interesting research path for this project, could be to seek for other
alternatives for doing the merge; or, to confirm that the scheme
proposed by \Rehurek is the optimal way of merging two truncated SVD
factorizations.


% Appendix A

\chapter{CFR pseudo-code} % Main appendix title

\label{AppendixCFR} % For referencing this appendix elsewhere, use \ref{AppendixA}

\lhead{Appendix B. \emph{CFR pseudo-code}} % This is for the header on each page - perhaps a shortened title

\begin{lstlisting}[caption={CFR pseudo-code}, label={lst:cfr}]
Class PokerEnv {...}
% A class representing the poker env, it's implementation
% will not be shown in this pseudo-code

Struct StategyInfo {
    float value;
    string action;
}

Struct NodeInfo {
    % you should store an hash key of this env intead of
    % the full PokerEnv, we are doing this for readability
    PokerEnv env
    float regret
    Map<PokerEnv, strategys> strategys
}

nodes = {}

void runOnce(env, proba):
    currentNode = nodes[env]

    if (env.isOver()):
        currentNode.regret = env.reward * proba
        return

    actions = env.getAvailableActions()
    leadingNodes = {}
    total = 0

    % explore and compute values of the leading nodes
    FOR action in actions:
        copy = env.copy()
        copy.play(action)
        if (!currentNode.strategys[copy]):
            currentNode.strategys[copy] = {
                value: 1 / actions.size
                action: action
            }
        runOnce(copy, currentNode.strategys[copy].value * proba)
        leadingNodes.append(nodes[copy])

    sumNegativeRegret = 0
    sumPositiveRegret = 0
    for FOR node in leadingNodes:
        regret = -node.regret
        if (regret > 0):
            sumPositiveRegret += regret;
        else:
            sumNegativeRegret += regret;

    % update strategies
    FOR node IN leadingNodes:
        regret = -node.regret
        if (sumPositiveRegret > 0):
            currentNode.strategys[node.env].value = regret > 0 ? regret / sumPositiveRegret : 0;
        else if (sumNegativeRegret != 0):
            currNode.strategys[node.env].value = regret == 0 ? 1.0f : 1 - abs(regret / sumNegativeRegret);
        else:
            currNode.strategys[node.env].value = 1;

    % weight the stratgies so it sum up to 1
    float totalStrategy = 0
    FOR node in leadingNodes:
        totalStrategy += currentNode.strategys[node.env].value
    FOR node in leadingNodes:
        currNode.strategys[node.env].value /= totalStrategy;

    % compute the regret of this node
    FOR node in leadingNodes:
        currentNode.regret += proba * -node.regret * currentNode.strategies[node.env].value

env = PokerEnv(public_cards,private_cards,...)
% Initial environemnt of which you want the CFR Value
% It takes all the necessary information to create a moment in the game.
for _ in NB_ITERATION:
    runOnce(env, 1)

% CFR values are in nodes[env].strategies
FOR strategy in nodes[env].strategies:
    env.generateRandomCardsFor2ndPlayer()
    print("Action {strategy.action} -> strategy.value")
\end{lstlisting}


\lab{Multi-Armed Bandit Problems}{Multi-Armed Bandit Problems}
\objective{This lesson explains Markov decision processes --
specifically multi-armed bandit problems -- and how to solve them by
Thompson Sampling. An application to web page experiments is addressed.}

\section*{Markov Decision Processes}
Previously we considered dynamic programming problems.
These problems involve making sequential decisions in some optimal way,
possibly under uncertainty.  Dynamic programming problems are closely
related to a class of problems called Markov Decision Processes.

A Markov Decision Process (MDP) involves the following elements:

\begin{itemize}
\item   A set of decision times,
\item   A set of states,
\item   A set of actions,
\item   A set of rewards dependent on the state and action, and
\item   Transition probabilities dependent on states and actions.
\end{itemize}
In this lab we will consider discrete time problems, so that our
set of decision times is $t = 1,2,3,\ldots$.
\begin{comment}
In the dynamic programming problems considered in the previous labs,
the set of states was the set of all possible levels of wealth and
the actions were how much wealth to save for the next period.
The rewards were given by the utility function $u(\cdot)$,
and we considered both deterministic and stochastic transitions.
\end{comment}
\section*{Bandit Problems}
In particular, we will consider what is called the multi-armed bandit
problem.  The name comes from the following example.
Suppose there is a row of $N$ slot machines (``one-armed bandits")
that each pays out with probability $p_i$, $i= 1,2,\ldots,N$,
where the probabilities are unknown to the gambler.
The gambler seeks to determine the sequence of levers to pull in
order to maximize her winnings.

Bandit problems have a wide range of applications.
One can consider the ``arms" to be the different treatments in a
clinical trial, the different forms of advertising a product,
or the different research projects a company might invest in.

We now formulate the multi-armed bandit problem precisely.
For simplicity, suppose there are $2$ arms, though the
discussion is easily extended to $N$ arms.
In each time period $t= 1,2,\ldots$, one arm can be pulled.
With unknown probability $p_i$, the $i$-th arm gives
reward $1$ and with probability $1-p_i$ it gives reward $0$.
Note we have now defined a set of decision times, actions, and rewards.
We define the state to be a 4-tuple giving the number of successful and unsuccessful
pulls on each arm up to that point, written as
\begin{equation}\label{state}
(a_1,b_1,a_2,b_2),
\end{equation}
where $a_i$ is the number of successful pulls on arm $i$ and $b_i$
is the number of unsuccessful pulls on arm $i$.
As we pull the arms, we must balance between pulling the arm that has
the highest expected payoff and pulling all arms in order to gain
information about the probabilities $p_i$.
This trade-off is often referred to as exploring versus exploiting.
In essence, while gaining rewards, we will also come up with
estimates of the $p_i$ that improve our decision making.
We will do so by Bayesian updating.

\section*{Bayesian Updating}
While a full exposition of Bayesian inference is well beyond the
scope of this lab, the essential concepts are fairly straightforward.
We recognize that we do not know the values of the $p_i$,
but given our past history of successful and unsuccessful
pulls, we can say something about what range we think they might be in.
This is different than guessing a specific value for the $p_i$.
Instead of specifying what we think is the most likely value of $p_i$,
we can encode our knowledge of and uncertainty about $p_i$ in a
probability distribution,
like those seen in Figure \ref{fig:priors}.
\begin{figure}

\centering
\includegraphics[width=\textwidth]{priors.pdf}
\caption{Bayesian priors}
\label{fig:priors}
\end{figure}

If you have had $1$ success and $2$ failures, you might
think that the success probability
$p_i$ is around $\frac{1}{3}$.
However, you have very little information at this point,
so rather than assuming the exact value for $p_i$,
you might represent your belief about $p_i$
by the blue probability distribution in Figure \ref{fig:priors}.
We call the distribution that describes our belief about $p_i$
the \emph{prior distribution} of $p_i$.  If we incorporate new information,
we get a new distribution, called the \emph{posterior distribution}.
As we collect even more information, the posterior distribution becomes
the new prior distribution. This framework allows us to fluidly incorporate
new information as we seek to determine the value of $p_i$.

As we get more information, the probability curve gets narrower.
In the figure, we see that the distributions have expected
value of $1/3$ and become tighter around that value.
This is fitting since the parameter values have success $\alpha$ one
third of the time and the tighter distributions correspond to having
more prior information.  If we need an estimate for $p_i$ we can use
the expected value of our distribution corresponding to $p_i$.
We will denote this estimate by $\overline{p_i}$.

We will use this Bayesian framework in our approach to the multi-armed bandit problem.
We do not know the $p_i$, but each time we pull an arm, we update our distribution of
where we think it might be.
In particular, we use a beta distribution for each $p_i$.
A beta distribution is a continuous probability distribution on the interval $[0,1]$,
which corresponds nicely to the possible values of the $p_i$.  Beta distributions have
two parameters (given by $\alpha$ and $\beta$ in the figure).

In our problem, the state (given by \eqref{state}) can be thought of as representing two
beta distributions -- one for each $p_i$ with parameters $a_i$ and $b_i$.
The details of the updating process are unimportant for now, except
for this important property: the two parameters for the beta distribution
correspond exactly with successes and failures.
So if we start with a prior distribution Beta$(a,b)$ and the next pull is success,
our posterior distribution is Beta$(a+1,b)$. If the next pull is a failure, however, then
the posterior distribution is Beta$(a,b+1)$.
Observe how this corresponds with the way our state evolves.

\section*{Simulation and Sampling in a Bayesian Framework}
Along with the estimate $\overline{p_i}$, we can also compute many other useful
quantities in our Bayesian problem via random sampling.  Essentially, we can take
random draws from our distribution and estimate the mean, median, or other
quantities based on the value of those quantities in the random sample.
This is a very powerful concept that will be explored in more detail in future labs.

\begin{problem}
Write a function \li{sim\_data} that accepts an $n\times 2$ array, where each
row represents the parameters of a beta distribution, respectively, and a positive integer $k$
that represents the number of random draws to return.
The function should return a $k\times n$ matrix where each row has a random sample
from each of the $n$ arms.  This can be accomplished with ease using the NumPy function
\li{numpy.random.beta}.
\label{prob:simdata}
\end{problem}

\begin{comment}
\begin{problem}
Suppose one of the arms in a bandit problem has the state (or prior distribution of $p_i$)
Beta$(100,200)$ corresponding to $100$ successes and $200$ failures.
Simulate 10,000 data points for the distribution Beta$(100,200)$.
Compute $\overline{p_i}$ by finding the mean of the simulated points.
Compute the median.  Also compute the $95$th percentile using the command
\li{scipy.stats.mstats.mquantiles(data, .95)} where \li{data} is the array
containing the 10,000 simulated data points.
\end{problem}
\end{comment}
\section*{Direct Dynamic Programming Solution}
This framework lends itself well to a dynamic programming type solution.
We introduce the value function $R$ that depends on the state, and gives
the optimal expected value that can be achieved
starting from the state.  Then we have
\begin{equation}
\label{recurs}
\begin{aligned}
R(a_1,b_1,&a_2,b_2) =\\
 \max&\left\{\overline{p}_1\cdot[1 + \beta R(a_1+1,b_1,a_2,b_2)] +
 (1-\overline{p}_1)\beta R(a_1,b_1+1,a_2,b_2)\right. ,\\
&  \left.\overline{p}_2\cdot[1 + \beta R(a_1,b_1,a_2+1,b_2)] +
(1-\overline{p}_2)\beta R(a_1,b_1,a_2,b_2+1)\right\}
\end{aligned}
\end{equation}
The two terms in the maximization represent the expected value of pulling lever
one or lever two respectively.  For example, by pulling lever one, we expect
to get a reward of 1 with probability $\overline{p_1}$.
We must also account for the expected value of future rewards
(discounted by $\beta$) moving to a state with one more success on arm 1.
With probability $1-\overline{p_1}$, lever one does not give a reward and so
rewards are simply the discounted expected reward starting in the next state.

Notice that the expressions inside of $R(\cdot)$ on the right side have parameter
values that add to one greater than the $R(\cdot)$ on the left side.
So if we want to compute $R(1,1,1,1)$ we need to know
$R(2,1,1,1)$, $R(1,2,1,1)$, $R(1,1,2,1)$ and $R(1,1,1,2)$
(all possible combinations of parameters that add up to 5).
To compute these, we need to know the reward corresponding to all possible
combinations of parameters that add up to 6, and so on.  Consequently, we could
make a guess for all $R$ for all parameter combinations that add up to some large $N$,
then work backward until we get to $R(1,1,1,1)$.  This is backward induction,
just as we saw in dynamic programming.  However, the number of computations in this
problem grow much too quickly because of the branching nature of having multiple arms.
In fact, if there are more than two arms this method of computation is infeasible!

\begin{comment}
\section*{Gittins Index Solution}
One way we might hope to solve a bandit problem is by computing some sort of ``index''
for each arm.  That is, we want a number associated with each arm that in some sense
captures the value of pulling that arm.   Ideally such an index would depend only on that arm,
and not on the others.  We could then compare the indices for all of the arms
and pull the arm with highest index.  It turns out that bandit problems can be
solved optimally by such methods and are computationally more feasible than the
dynamic programming approach we saw above.

To compute an index for this problem, we consider comparing an arm with unknown
payoff probability $p_i$ to an arm with known payoff probability $p$.  Then equation \eqref{recurs}
becomes
\begin{equation}\label{index}
\begin{aligned}
R(p,a_i,b_i) = \max&\left\{\frac{p}{1-\beta} \right. ,\\
&  \left.\hat{p}_i\cdot[1 + \beta R(p,a_i+1,b_i)] + (1-\hat{p}_i)\beta R(p, a_i,b_i+1)\right\}.\\
\end{aligned}
\end{equation}

We determine the expected value of pulling the first arm to be $\frac{p}{1-\beta}$
noting that if we pull the deterministic arm once, we will continue pulling it
forever as we gain no new information.  In this case the expected reward from pulling
the known arm is $p + \beta p + \beta^2 p + \cdots = \frac{p}{1-\beta}$.

If we can find the $p$ such that we are indifferent between the deterministic
arm and the unknown arm, this will give us an index that quantifies the value of arm $p_i$
(this can be proved).

Putting all of this together, our algorithm for solving the multi-armed bandit
problem with two arms is as follows.  For each arm $i$, compute \eqref{recurs}
over a range of $p$ values to find the $p$ such that you would be indifferent
between the arm with probability $p$ and arm $i$. Store this as the index $\lambda_i$.
Compare the $\lambda_i$ and pull the arm with largest index.

In order to compute \eqref{index} we use dynamic programming, starting with a guess
for $R$ for parameters that add up to some large $N$ and use backward induction.
In the process, we find $R$ and $\lambda_i$ for each combination of parameters
that adds up to any $n\leq N$. Thus we do not have to compute new $\lambda_i$ after
each pull, we can just look them up.

\section*{Algorithm Outline}
This section will guide you through creating a function that will compute the indices
for a given arm.  It will involve writing a number of functions that you can save in the same
.py file.

First, we need a function that will compute all the pairs of $a,b$ that add up
to some $N$.  We also want the user to be able to input the minimum values of
$a$ and $b$ of interest to avoid unnecessary computation.  For example if in practice
all of the arms have $a_i \geq 5$ and $b_i \geq 10$, then we are uninterested in smaller $a$, $b$.
The following code accepts the value of $N$ and a minimum value of a and b and
returns an $N$ by 2 array with the $a$'s and $b$'s such that $a + b = N$.
\begin{lstlisting}
# computes pairs of numbers starting with mina and minb that
# add up to N

def compute_indices(N,mina,minb):
    import scipy as sp
    avec = sp.arange(mina,N-minb+1)
    avec = sp.reshape(avec,(avec.shape[0],1))
    bvec  = sp.arange(N-mina, minb-1,-1)
    bvec = sp.reshape(bvec,(bvec.shape[0],1))
    values = sp.hstack((avec,bvec))

    return values
\end{lstlisting}

In order to perform the backward induction, we need to be able to estimate the value
$R(p,a,b)$ for $a+b = N$.  To do so we have to estimate the second quantity in \eqref{index}.
We will estimate it as $(\frac{a}{a+b})/(1-\beta)$.  This is the value one would get by
pulling an arm with $p_i$ equal to the expected value of $Beta(a,b)$ forever.
\begin{problem}
Write a function ``end\_reward" that accepts $p,a,b,$ and $\beta$ and returns the
estimated value of $R(p,a,b)$ for $a,b$ such that $a+b=N$.
\end{problem}

For convenience it will be nice to have a function that will compute $R(p,a,b)$
given values of $p,\overline{p},a,b,\beta,N$ as well as values of $R(p,a+1,b)$ and $R(p,a,b+1)$.
This should follow directly from \eqref{index}.
\end{comment}

\section*{Thompson Sampling}
\begin{comment}
There is a method for computing the optimal solution to the multi-armed bandit problem
based on what is commonly known as the Gittins Index Theorem.
Computationally it is similar to the dynamic programming approach, but it is significantly less
costly.
Unfortunately, for large scale problems, it can still be too costly.
\end{comment}
There are, however, many heuristic methods of solving the multi-armed bandit problem.
In particular we will use a method known as Thompson Sampling, or Randomized Probability Matching.
The idea is that we should choose arm $i$ with probability equal to the probability
that arm $i$ is the best arm.  So if we believe there is an $80\%$ probability that
arm 2 is the best arm we will pull it $80\%$ of the time.  The other $20\%$ of the
time we would pull arm one which will help give more information about the true value of $p_1$.
In this way, we will pull most often the arms from which we expect the most rewards;
however, we will also pull other arms with some probability so that we accomplish some
exploration and gain information on all arms until we are confident we have found the best arm.

\begin{problem}
Write a function \li{arm_probs} that accepts a $k \times n$ array of data computed by the
function from Problem \ref{prob:simdata}.

This function should return a vector of the probabilities that each of the $n$ arms is optimal.
This can be computed for each arm by determining in how many of the $k$ simulations
that arm had the highest value.  Dividing this number by $k$ will give the proportion
of the simulations for which this arm had the greatest probability of success.
This proportion is interpreted as the probability that the arm is the optimal arm.
\end{problem}

In some applications, we might want to run computations once, then pull many arms instead
of computing before each pull.  In these cases, rather than compute these probabilities,
choose an arm, then recompute, it can be more convenient to view the probabilities as weights.
For example we can view the probabilities as the weights of how to distribute the next 100 pulls.
Suppose the probabilities resulting from the previous function (with two arms) are $0.4$ and $0.6$.
Then we would allot 40 of the next 100 pulls to arm 1 and 60 of the next 100 pulls to arm 2.
Then we could compute new weights for the next 100 pulls.

\begin{problem}
Using the results from the previous problems, write a function \li{get_pulls} that determines how many
times each arm should be pulled in the next $M$ pulls, where $M$ is an input to the function.
The function should accept a vector of probabilities of the form returned by the
function in the previous problem, and a number of pulls $M$.
Note that you will have to round since the number of pulls for each arm must be in integer.
Return a vector of length $n$ (where $n$ is the number of arms) that gives the number of
pulls out of the next $M$ for each arm.  Make sure the entries sum to $M$.
\end{problem}

You now have code that solves the version of the multi-armed bandit problem described here.
We first start all arms with the state (or prior) Beta$(1,1)$,
which is the uniform distribution, meaning we have no information on the $p_i$.
Then we compute the weights for the next $M$ pulls.
After those $M$ pulls we compute new weights and continue on in this pattern.
In some applications we may not continue this forever, but might instead have some stopping
criterion for when we think we have identified the best arm.
A common stopping criterion is to stop when one of the arms has a $95\%$ probability
(or some other specified probability) of being the optimal arm.
In the framework we have set up, this is already computed at each step and, thus, is easy to check for.

We now investigate how this process can be applied
in web page testing.

\section*{Web Page Experiments}
Bandit problems provide a way to compare the success of different variations of a web page.
Suppose a business wants to test new versions of a web page.
The goal of the page might be to get the user to click a certain link, make a purchase, etc.
When the user does this, we call it a conversion.  The proportion of web page visits
that results in a conversion is called the conversion rate, or CvR.
The website designer wants to determine which variation of the web page has the best CvR.

We can model this situation as a bandit problem by considering each page as a different arm.
Each page has some unknown probability (the CvR) that a user will perform the desired action.
The company then wants to experiment with giving different users different versions of
the page in order to determine which variation is most successful.

This method is the same that is used by Google Analytics.  Take a moment to skim their description
located here:
\url{http://analytics.blogspot.com/2013/01/multi-armed-bandit-experiments.html}.

We will apply the Thompson Sampling method described above (the same method
used by Google)
to solve this problem.  We will simulate the results and attempt to replicate Google's results
found at the website above.

\section*{The Experiment}
Here we describe how we will design our web page experiment using bandits and how we will
simulate it.
We will have some number of variations of a web page, $n$.  Each day, the web-page will receive
100 visitors
(for the purposes of our simulation).  Twice each day, at the beginning and after 50 visits,
we will compute the number of each variation to deliver to the next 50 visitors using the
method described above.

\begin{problem}
Write a function \li{sim_day} that simulates the experiment for one day.
The function should accept a vector of length $n$ of the true probabilities (CvR)
for different web page variations.  It should also accept the state of the variations
at the beginning of the day; this is an $n \times 2$ array with the number of previous
successes plus one in the first column and the number of previous failures plus one in
the second column (so if it is the first day, the state for any arm would be $(1,1)$,
corresponding to a Beta$(1,1)$ distribution).

The function can be outlined like this: first compute how many times each variation
should be used in the next 50 visits (refer to these numbers as the ``pulls").
When determining the pulls, you will need to use
the \li{sim_data} function.  Use 100 as the number of draws here and throughout this lab.

Next, ``visit'' each page the number of times given by the pulls.
Use the NumPy function \li{numpy.random.binomial} to compute the outcome of each visit.
To illustrate, suppose \li{cvr} is the length $n$ vector giving the true CvR values
and \li{pulls} is the length $n$ vector giving the number of times to visit each page
in the next 50 visits. We calculate the outcome of visiting page \li{i} the
specified number of times as follows:
\begin{lstlisting}
>>> outcome = np.random.binomial(pulls[i], cvr[i])
\end{lstlisting}

As you make these 50 visits, update the state of each arm by adding the outcome of the visits
to each page to the corresponding entry of the state vector.
To illustrate, suppose \li{state} is the $n \times 2$ vector giving the beta parameters
for each web page. After we compute \li{outcome} as shown above, we update the
state of the $i$-th web page as follows:
\begin{lstlisting}
>>> state[i,0] += outcome
>>> state[i,1] += pulls[i] - outcome
\end{lstlisting}

Once you have finished these 50 visits and updated the state of each arm,
repeat the process for the next 50 visits (i.e. recompute the pulls,
visit the web pages, and update the state of each arm).

After these 100 total visits,
the function should return the resulting $n\times 2$ array giving the states of the web pages,
as well as the length $n$ array giving the number of pulls for each arm used for the final
50 visits.
\end{problem}

In this manner we continue from day to day, always updating the state of each arm
(the beta distribution for the CvR of each arm).  We have three criteria to determine
when to stop the experiment.
Firstly, the experiment must run at least two weeks to make sure the
results are not overly influenced by a small number of random draws.

The second stopping criterion is that there be a $95\%$ probability that one
of the variations is the best variation.
This is the same as saying that, of the weights for each variation, the largest
is greater than $.95$.

It may seem that these two criteria should be enough; however, in some cases, the
experiment could last a very long time using just these criteria.
For example, consider the case that two of the web page variations have nearly the same CvR.
In this case it will be very difficult to determine which is best.
It will also not be very important since the results are so similar.
Thus we use a measure called the \emph{potential value remaining}
in the experiment as the third criterion.  The potential value remaining is computed by
simulating many draws for each arm.  Using this data, the potential value remaining
for arm $i$ is obtained by computing the following for each simulated data point:
\begin{equation}\label{valrem}
\frac{\theta_{max} - \theta^*}{\theta^*},
\end{equation}
where $\theta_{max}$ is the largest value for the random draw and $\theta^*$ is the
value of the arm that is currently believed to be the best
(the arm with the highest weighting, or probability of being optimal).

The result is some distribution of numbers between $0$ and $1$ that we can think
of as the distribution of value remaining.  For example, if $50\%$ of the numbers are 0,
then about $50\%$ of the time the arm that is currently believed optimal will perform the best.
The potential value remaining is the $95$th percentile of this distribution.
If the potential value remaining were $.2$, we could interpret it as meaning
that there is about a $5\%$ chance that another arm beats the current best arm by $.2$ or more.
We stop the experiment if this value is less than $1\%$ of the current best arm's CvR.
This way we stop the experiment if there seems to be little chance of improvement over
the current best arm, regardless of whether we've met the $95\%$ tolerance for the weights.

The value remaining can be computed using the following code:
\begin{lstlisting}
import scipy as sp
def val_remaining(data,prob):
    champ_ind = np.argmax(prob)
    thetaM = np.amax(data,1)
    valrem = (thetaM - data[:,champ_ind])/data[:,champ_ind]
    pvr = sp.stats.mstats.mquantiles(valrem,.95)
    return pvr
\end{lstlisting}
where \li{data} is simulated using the \li{sim_data} function and
and \li{prob} is a vector containing the probabilities that each arm is optimal
(obtained from the \li{arm_probs} function).

\begin{problem}
Write a function \li{sim_experiment} that simulates the problem described above,
using the stopping criteria described above.
The function should accept a vector of the true probabilities of the arms.

Your code should be formatted as follows.
First, initialize the state vector to reflect our Beta$(1,1)$ prior:
\begin{lstlisting}
>>> state = np.ones((n,2))
\end{lstlisting}
(here \li{n} is the number of arms, as usual).
It will also be helpful to have a list \li{pull_list} that stores
the pull vectors from each day of simulation.
\begin{lstlisting}
>>> pull_list = []
\end{lstlisting}

Now we are ready to launch into the iteration.
Write a while-loop that checks for the stopping criteria.
It might look something like this:
\begin{lstlisting}
while (days<14) or (max_p <= .95 and max_pbar/100 <= pvr):
\end{lstlisting}
Here, \li{days} is a count variable keeping track of how many days (iterations) the experiment
has been running. The variable \li{max_p} gives the maximal entry of the vector of
probabilities returned by \li{arm_probs}. The variable \li{max_pbar} gives the value
$a_i/(a_i+b_i)$, where $i$ is the index corresponding to the arm with the highest
probability of being the best arm. Finally, \li{pvr} is the value returned by
\li{val_remaining}. You will need to initialize these four variables to appropriate
values before the while loop.

In each iteration, run your \li{sim_day} function to simulate one day of the experiment:
\begin{lstlisting}
>>> state, pulls = sim_day(cvr, state)
\end{lstlisting}
We next need to update our stopping criteria variables.
As described above, simulate data from the current state, and calculate the
probability vector of each arm being the best:
\begin{lstlisting}
>>> data = sim_data(state, n_sim)
>>> probs = arm_probs(data)
\end{lstlisting}
Compute the maximum probability, as well as the corresponding maximal $\bar{p_i}$:
\begin{lstlisting}
>>> max_ind = np.argmax(probs)
>>> max_p = probs[max_ind]
>>> max_pbar = state[max_ind, 0]/(state[max_ind,:].sum())
\end{lstlisting}
Finally, compute the potential value remaining:
\begin{lstlisting}
>>> pvr = val_remaining(data, probs)
\end{lstlisting}
Don't forget to increment your \li{days} variable, as well as store the pulls from
that day:
\begin{lstlisting}
>>> days += 1
>>> pull_list.append(pulls)
\end{lstlisting}

Once the iterations terminate, return the state vector (\li{state}),
return the pulls from each day in the form of an array (\li{np.array(pull_list)}),
return the optimal arm (given by \li{max_ind}), and finally the total amount
of days that the simulation ran (\li{days}).
\end{problem}

Now let's see how our bandit performs with specific examples.

\begin{problem}
Suppose a web page has two variations and the true CvR of the original is
$.04$ and the true CvR of the new variation is $.05$.
Create a plot similar to \ref{fig:weights1} that shows how the proportion of visits
assigned to the pages changes from day to day until the optimal page is chosen and the experiment
stops:
\begin{lstlisting}
>>> cvr = np.array([.04,.05])
>>> n = cvr.size
>>> state, pulls, max_ind, days = sim_experiment(cvr)
>>> d = np.arange(days)
>>> for i in xrange(n):
>>>     plt.plot(d, pulls[:,i])
>>> plt.show()
\end{lstlisting}

Next, run the same simulation 200 times and keep track of how many days
the experiment took in each case in an array called \li{dayvec}.
Create a histogram that shows the
number of days it takes to complete the experiment.
The following code will create such a histogram:
\begin{lstlisting}
>>> hist, bins = sp.histogram(dayvec, bins = 12)
>>> width = (bins[1]-bins[0])
>>> center = (bins[:-1]+bins[1:]) / 2
>>> plt.bar(center, hist, align = 'center', width = width, color = 'g')
>>> plt.show()
\end{lstlisting}
Also track which arm is determined to be optimal in each simulation.
What percent of the time did the bandit find the optimal arm?

Create the same two types of plots, this time with six variations having
weights $.04,.02,.03,.035,.045,.05$.  This time only run the simulation 100 times.
What percent of the time did the bandit find the optimal arm in this case?
\end{problem}

\begin{figure}
\centering
\begin{subfigure}[t]{.49\textwidth}
\centering
\includegraphics[width=\textwidth]{weights1.pdf}
\caption{Optimal arm probabilities for two arms.}
\label{fig:weights1}
\end{subfigure}
\begin{subfigure}[t]{.49\textwidth}
\centering
\includegraphics[width=\textwidth]{weights2.pdf}
\caption{Optimal arm probabilities for six arms.}
\label{fig:weights2}
\end{subfigure}
\end{figure}

% \begin{figure}[h]
% \centering
% \includegraphics[width=\textwidth]{weights2.pdf}
% \caption{Optimal arm probabilities in the six arm case}
% \label{fig:weights2}
% \end{figure}

\section*{Comparison with Classical Tests}
A more classical approach to this problem would be to split traffic between each
variation for a predetermined amount of time, which should give enough data to
determine the best arm with some level of confidence.
Using the bandit approach described here has significant advantages over a classical test.
There are two main reasons why the bandit approach is more efficient.

The first reason is that the bandit method generally converges more quickly.
A standard test would require splitting the web page views between the different
variations over a long period of time.  According to Google's explanation in the
website mentioned at the beginning of this lab, the two arm case would take 223 days
and the 6 arm case would take 919 days.  The results from the simulations you performed
should show that on average the bandit method finishes much faster.
There are other ways we could choose our stopping criteria that may result in even shorter experiment times.
In general, we can always adjust the tolerance of our stopping criteria to shorten experiment time or increase accuracy.

The second reason the bandit approach is more efficient is that, as we gain more information,
we allocate more visits to the variation that we believe has a better CvR.
In the classical method we would split the visits evenly until the end of the experiment.
This way we gain many more conversions during testing than we would using classical tests.