\section{Front Trees}
\label{section:front-trees}
\par
To illustrate the different types of front trees, and their
transformations we do for the sake of efficiency,
we will use an an example the matrix R2D100, a matrix generated by
first randomly triangulating the unit square with 100 grid points.
The resulting matrix has 100 rows and columns.
We ordered the matrix using a generalized nested dissection
algorithm from the {\bf SPOOLES} library.
On the left in Figure~\ref{fig:R2D100} is the triangulation.
On the right we have labeled the grid points with their place in
the nested dissection ordering.
Note that vertices 90 through 99 form a separator of the graph.
Vertices 0 through 47 are found on the right of the separator,
vertices 48 through 89 are found on the left
\par
\begin{figure}[htbp]
\caption{R2D100: randomly triangulated, 100 grid points}
\label{fig:R2D100}
\begin{center}
\mbox{
\psfig{file=R2D100.eps,width=3.0in,height=3.00in}
}
\mbox{
\psfig{file=R2D100perm.eps,width=3.0in,height=3.00in}
}
\end{center}
\end{figure}
\par
\subsection{Vertex elimination trees}
\label{subsection:vtx-elim}
\par
Recall that the four ordering methods from
Section~\ref{section:ordering} return an {\tt ETree} object.
% In addition to the tree structure, an {\tt  ETree} object contains
% other information: the number of internal and external rows
% and columns to a
% front, and a map from each row and column to the front it belongs in.
There is another way to construct a tree using the {\tt Graph} object
and the permutation vectors.
The following code fragment shows how to do this.
\begin{verbatim}
ETree   *vetree ;
int     *newToOld, *oldToNew ;
Graph   *graph ;

vetree = ETree_new() ;
ETree_initFromGraphWithPerms(vetree, graph, newToOld, oldToNew) ;
\end{verbatim}
The {\tt vetree} object in the code fragment above is a
{\it vertex elimination tree}
\cite{liu90-etree}, \cite{sch82-etree},
where each front contains one vertex.
\par
Figure~\ref{fig:R2D100-tree-vtx} contains the vertex elimination tree
for this ordering.
The vertex elimination tree is
a representation of the partial order by which
the vertices in the graph may be eliminated.\footnote{
Vertex $j$ is the parent of $i$ if $j$ is the first vertex greater
than $i$ such that $L_{j,i} \ne 0$.
}
The dependencies of the rows and columns form a tree structure.
The leaves of the tree (our trees hang
upside down with the leaves at the bottom and the root at the top)
represent vertices which can be eliminated first.
The parents of those leaf nodes can be eliminated next,
and so on, until finally the vertices represented
by the root of the tree will be eliminated last.
\par
\begin{figure}[htbp]
\caption{Vertex elimination tree for R2D100, 100 rows and columns}
\label{fig:R2D100-tree-vtx}
\begin{center}
\mbox{
\psfig{file=vtree.eps,width=5.0in,height=5.00in}
}
\end{center}
\end{figure}
\par
The elimination tree illustrates the dependence of the vertices.
The basic  rule is that a vertex {\it depends} only on its descendents
and will {\it affect} only its ancestors.
It should be clear that the tree allows us to identify independent,
parallel computation.
For example, the computation of the factor entries in the
subtree rooted at vertex 47 is completely independent of the
subtree rooted at vertex 89, so we could identify one process to
compute the left subtree and another to compute the right subtree.
\par
\subsection{Fundamental supernode trees}
\label{subsection:fs-tree}
\par
While the vertex elimination tree is useful to communicate the data
dependencies, it is not a good granularity on which to
base a factorization or solve, in serial or in parallel.
It is important to group vertices together in some meaningful way
to create larger data structures that will be more efficient with
respect to storage and computation.
% The first step in this direction is to group together vertices
% that form a chain with no branches in the tree.
Any grouping of vertices imposes a block structure on the matrix.
The {\it fundamental supernode tree}
\cite{ash89-relaxed}
has these property:
any node in the tree is
\begin{itemize}
\item either a leaf,
\item or has two or more children,
\item or its nonzero structure
      is not contained in that of its one child.
\end{itemize}
The top tree in Figure~\ref{fig:fs-trees}
shows the vertex elimination tree with the ``front'' number of each
vertex superimposed on the vertex.
The bottom tree is the fundamental supernode tree.
Figure~\ref{fig:R2D100-fs-mtx} shows the block partition superimposed on
the structure of the factor $L$.
Note this one important property:
within any block column and below the diagonal block,
a row is either zero or dense.
\par
\begin{figure}[htbp]
\caption{Top: vertex elimination tree with the vertices mapped to
the fundamental supernode that contains them.
Bottom: fundamental supernode tree.}
\label{fig:fs-trees}
\begin{center}
\mbox{
\psfig{file=fsvtree.eps,width=5.0in,height=5.00in}
}
\par
\mbox{
\psfig{file=fstree.eps,width=4.71in,height=2.63in}
}
\end{center}
\end{figure}
\par
\begin{figure}[htbp]
\caption{Block structure of $L$ with the fundamental supernode
partition.}
\label{fig:R2D100-fs-mtx}
\begin{center}
\mbox{
\psfig{file=fsmtx.eps,width=5.0in,height=5.00in}
}
\end{center}
\end{figure}
\par
The code fragment to convert a tree into a fundamental supernode tree is
given below.
\begin{verbatim}
ETree   *fsetree, *vetree ;
int     maxzeros ;
IV      *nzerosIV ;

nzerosIV = IV_new() ;
IV_init(nzerosIV, vetree->nfront, NULL) ;
IV_fill(nzerosIV, 0) ;
maxzeros = 0 ;
fsetree = ETree_mergeFrontsOne(vetree, maxzeros, nzerosIV) ;
\end{verbatim}
The {\tt ETree\_mergeFrontsOne()} method constructs a new {\tt ETree}
object from the {\tt vetree} object.
When a node $J$ has a single child $I$, it looks to see whether merging
$I$ and $J$ together will add more than a given number of zeroes into
the block columns of $I$ and $J$.
(The nonzero rows of the block of $I$ and $J$ together is the union of
the nonzero rows of blocks $I$ and $J$ separately, and all nonzero rows
are stored as dense rows.)
To create a fundamental supernode tree, the number of
zeros allowed into a block column is zero, i.e., the nonzero structure
of the fundamental supernode tree contains no zeros.
The {\tt nzerosIV} object contains a running count of the number of zero
entries present in the factor storage.
It will be used in later calls to other transformation methods.
\par
\subsection{Amalgamated or relaxed supernode trees}
\label{subsection:am-tree}
\par
A factorization based on the fundamental supernode tree requires
no more operations than one based on the vertex elimination tree.
There are many small supernodes at the lower levels of the tree.
By {\it amalgamating} small but connected sets of supernodes together
into larger supernodes we can reduce the overhead of the processing
all of the small supernodes at the expense of adding
entries to the factors and operations to compute the factorization.
This amalgamation of supernodes generally leads to an overall
increase in efficiency
\cite{ash89-relaxed},
\cite{duf83-multifrontal}.
We call the result the {\it amalgamated}
or {\it relaxed} supernode tree.
\par
The top tree in Figure~\ref{fig:am-trees}
shows the vertex elimination tree with the ``front'' number of each
vertex superimposed on the vertex.
The bottom tree is the amalgamated supernode tree.
Figure~\ref{fig:R2D100-am-mtx} shows the block partition superimposed on
the structure of the factor $L$.
\par
\begin{figure}[htbp]
\caption{Top: fundamental supernode tree with the supernodes mapped to
the amalgamated supernode that contains them.
Bottom: amalgamated supernode tree.}
\label{fig:am-trees}
\begin{center}
\mbox{
\psfig{file=amvtree.eps,width=5.0in,height=5.00in}
}
\par
\mbox{
\psfig{file=amtree.eps,width=3.25in,height=1.38in}
}
\end{center}
\end{figure}
\par
\begin{figure}[htbp]
\caption{Block structure of $L$ with the amalgamated supernode
partition.}
\label{fig:R2D100-am-mtx}
\begin{center}
\mbox{
\psfig{file=ammtx.eps,width=5.0in,height=5.00in}
}
\end{center}
\end{figure}
\par
The code fragment to create this amalgamated tree is found below.
\begin{verbatim}
ETree   *ametree ;

maxzeros = 20 ;
ametree = ETree_mergeFrontsAll(fsetree, maxzeros, nzerosIV) ;
\end{verbatim}
This method will merge a node with {\it all} of its children if it will
not result in more than {\tt maxzeros} zeros inside the new block.
On input, {\tt nzerosIV} object keeps count of the number of zeroes
already in the blocks of {\tt fsetree}, and on return it will
contain the number of zeros in the blocks of {\tt ametree}.
\par
\subsection{Splitting large fronts}
\label{subsection:sp-tree}
\par
There is one final step to constructing the tree that governs the
factorization and solve.
Large matrices will generate large supernodes at the topmost levels
of the tree.
For example, a $k \times k \times k$ grid with a 27 point finite
difference operator, when ordered by nested dissection, has a root
supernode with $k^2$ rows and columns.
The data structure for a top level supernode can be very large,
too large to fit into memory.
In a parallel environment, we follow the convention that each node
in the tree is handled by one process.
Having a very large node at the top levels of the tree will
severely decrease the parallelism available to the computations.
\par
The solution to both problems, large data structures and limited
parallelism, is to split large supernodes into pieces.
We can specify a maximum size for the nodes in the tree, and split
the large supernode into pieces no larger than this maximum size.
This will keep the data structures to a manageable size and increase
the available parallelism.  We call the resulting tree the {\it front}
tree because it represents the final computational unit for the
factorization, the frontal matrix.
\par
The amalgamated supernode tree has been transformed so that except for
the leaf nodes, which are not changed, no node in the tree has more
than four vertices.
The top tree in Figure~\ref{fig:sp-trees}
shows the vertex elimination tree with the ``front'' number of each
vertex superimposed on the vertex.
The bottom tree is the amalgamated and split supernode tree.
Figure~\ref{fig:sp-mtx} shows the block partition superimposed on
the structure of the factor $L$.
Splitting large nodes into smaller nodes will not increase the
factor storage or operation counts, in fact, as we shall soon see,
it is possible to decrease them slightly when compared to the
amalgamated tree before splitting.
\par
\begin{figure}[htbp]
\caption{Left: tree after the large supernodes have been split.
Right: tree with nodes mapped back to their amalgamated supernode.}
\label{fig:sp-trees}
\begin{center}
\mbox{
\psfig{file=spvtree.eps,width=5.00in,height=5.000in}
}
\quad
\mbox{
\psfig{file=sptree.eps,width=3.25in,height=2.21in}
}
\end{center}
\end{figure}
\par
\begin{figure}[htbp]
\caption{Block structure of $L$ with the amalgamated and split
         supernode partition.}
\label{fig:sp-mtx}
\begin{center}
\mbox{
\psfig{file=spmtx.eps,width=5.0in,height=5.00in}
}
\end{center}
\end{figure}
\par
The code fragment to split the large fronts is found below.
\begin{verbatim}
ETree   *spetree ;
int     maxsize, seed ;

maxsize = 4 ;
spetree = ETree_splitFronts(ametree, NULL, maxsize, seed) ;
\end{verbatim}
This method creates and returns an {\tt ETree} object where each front
has {\tt maxsize} or fewer internal rows and columns, except for the
fronts that are leaves in the tree.
Here we imposed the condition that no non-leaf front has more than four
vertices.
The second parameter in the calling sequence is non-{\tt NULL}
if the graph has nonunit vertex weights.
The last parameter is a seed for a random number generator.
When we identify a front with more than {\tt maxsize} internal rows and
columns, there are many ways to split the front into smaller fronts.
We try to keep the sizes of the fronts roughly equal, but which vertices
to play into which fronts is not specified.
We shuffle the vertices using a random number generator and assign
vertices to smaller fronts in a block manner.
\par
\subsection{Results}
\label{subsection:tree-results}
\par
This front tree is now the defining
structure for the numerical factorization and solve steps.
The structure of the front tree defines the order of the computations
that will be carried out in the factorization and the solve.
The composition of the front tree can have a profound effect on storage
and performance of the factorization and solves.
\par
Our {\tt R2D100} matrix was small enough to illustrate the steps in the
transformation of the front tree, but is not large enough to
realistically display how the front tree influences the differences
in storage and speed of the computations.
Now we look at the {\tt R3D13824} matrix.
Table~\ref{table:R3D13824-tree-stats} contains some statistics for a
sequence of front trees.
The original front tree came from our nested dissection ordering.
\par
\begin{table}[htbp]
\label{table:R3D13824-tree-stats}
\caption{R3D13824: front tree transformations}
\begin{center}
\begin{tabular}{l|rrrrr}
& CPU & \# fronts & \# indices & \# entries & \# operations \\ \hline
 original  &        &  6001 & 326858 & 3459359 & 1981403337 \\
 fs tree   & 0.040  &  6000 & 326103 & 3459359 & 1981403337 \\
 merge one & 0.032  &  3477 & 158834 & 3497139 & 2000297117 \\
 merge all & 0.020  &   748 &  95306 & 3690546 & 2021347776 \\
 merge any & 0.012  &   597 &  85366 & 3753241 & 2035158539 \\
 split     & 0.043  &   643 & 115139 & 3753241 & 2035158539 \\
 final     & 0.423  &   643 & 115128 & 3752694 & 2034396840
\end{tabular}
\end{center}
\end{table}
\par
There are 13824 rows and columns in the matrix, and 6001 fronts in the
nested dissection tree.
While there is an average of two rows and columns per front,
most of the fronts are singleton fronts at the lower levels of the tree.
The top level front has 750 internal rows and columns.
\par
\begin{itemize}
\item
In the first step we create an fundamental supernode tree with a call to
{\tt ETree\_mergeFrontsOne()} with {\tt maxzeros = 0}.
We see that the number of fronts decreases by one and the number of
entries does not change.
\item
The second step is also a call to
{\tt ETree\_mergeFrontsOne()}, this time with {\tt maxzeros = 1000}.
Here we merge fronts with only one child with that child,
in other words, only chains of nodes can merge together.
Note how the number of fronts is decreased by almost one half,
and the number of factor entries and operations increase by 1\%.
\item
The third step is a call to {\tt ETree\_mergeFrontsAll()}
with {\tt maxzeros = 1000}, where we try
to merge a node with all of its children if possible.
The number of fronts decreases again by a factor of five, while the
number of factor entries and operations increases by 7\% and 2\%,
respectively, when compared with the original factor matrices.
\item
The fourth step
is a call to {\tt ETree\_mergeFrontsAny()}
with {\tt maxzeros = 1000}, where we try
to merge a front with any subset of its children.
The number of fronts decreases further, and the factor entries and
operations increase by 8\% and 3\%, respectively.
\item
In the fifth step
is a call to {\tt ETree\_splitFronts()}
with {\tt maxsize = 64}, where we try
split the large fronts into smaller fronts.
Note that the number of factor entries and operations do not seem to
increase, while the number of fronts increases by about 8\%.
In reality, a large front that is split into smaller fronts may have a
non-dense block column structure, a one of its smaller fronts may have
rows in its block column of $L$ that are zero, whereas that same row in
the larger front is nonzero.
\end{itemize}
Merging fronts and splitting fronts can have a large effect on the
computational performance of a factor and solve.
Table~\ref{table:R3D13824-comp-stats} contains some results for
solving linear systems of equations for the {\tt R3D13824} matrix
using the five different front trees.
\par
\begin{table}[htbp]
\label{table:R3D13824-comp-stats}
\caption{R3D13824:
         factor and solve timings for five different front trees.}
\begin{center}
\begin{tabular}{l|ccccccc}
& & \multicolumn{2}{c}{factor} & & \multicolumn{2}{c}{solve} & total \\
& init & CPU & mflops & postprocess & CPU & mflops & CPU \\ \hline
 original  & 4.0 & 131.7 & 15.0 & 5.0 & 7.3 & ~7.6 & 148.0 \\
 fs tree   & 3.3 & 130.4 & 15.2 & 5.4 & 7.8 & ~7.1 & 146.9 \\
 merge one & 3.1 & 119.9 & 16.7 & 2.7 & 4.6 & 12.1 & 130.3 \\
 merge all & 3.0 & 120.7 & 16.7 & 1.4 & 3.6 & 16.2 & 128.7 \\
 merge any & 3.0 & 121.6 & 16.7 & 1.4 & 3.5 & 16.9 & 129.5 \\
 split     & 3.0 & ~84.9 & 24.0 & 1.9 & 3.5 & 17.1 & ~93.3
\end{tabular}
\end{center}
\end{table}
\par
The first thing to notice is that factorization performance improves
slightly as small fronts are merged together.
The large improvement comes when the fronts are split.
The explanation of this behavior is that all {\it inter-front}
computation is done using BLAS3 kernels for the operation
$Y := Y - L * D * U$, where $L$ and $U$ are dense matrices,
$D$ is diagonal or block diagonal with $1 \times 1$ and $2 \times 2$
pivots, and $Y$ is dense.
The {\it intra-front} computations, done entirely within the block
columns of $L$ and block rows of $U$, are done using BLAS1 kernels.
This is necessary when pivoting for stability.
Had we chosen to write BLAS3 kernels for the intra-front
computations when pivoting is not enabled, the factorization timings for
the first five front trees would have been higher.
But splitting fronts into smaller fronts is necessary for parallel
computations, so it made sense to make it the recommended route for
serial computations as well.
There would be very little difference in speed had the intra-front
computations been done with BLAS3 kernels compared with using the
final front tree, for the intra-front computations are a small fraction
of the total number of operations.
\par
The solve time improves dramatically when small fronts are merged
together into larger fronts.
Our solves are submatrix algorithms, where the fundamental kernel is an
operation
$Y_J := B_J - L_{J,I} X_I$
and
$X_J := Y_J - U_{I,J} Y_J$,
and is designed to be a BLAS2 kernel (when $X$ and $Y$ have a single
column) or BLAS3 kernel (when $X$ and $Y$ are matrices).
When fronts are small, particularly with one internal row and column,
the submatrices that take part are very small.
The overhead for the computations takes far more time than the
computations themselves.
\par
This multistep process of merging, merging again, etc, and finally
splitting the front trees is tedious.
There are simple methods that do the process in one step.
\begin{verbatim}
ETree   *etree, *etree2, *etree3 ;
int     maxfrontsize, maxzeros, seed ;

etree2 = ETree_transform(etree, NULL, maxzeros, maxfrontsize, seed) ;
etree3 = ETree_transform2(etree, NULL, maxzeros, maxfrontsize, seed) ;
\end{verbatim}
Inside The {\tt ETree\_transform()} method
is a sequence of four transformations:
\begin{itemize}
\item
Merge small fronts into larger fronts
using the {\tt ETree\_mergeFrontsOne()} method.
\item
Then merge small fronts into larger fronts
using the {\tt ETree\_mergeFrontsAll()} method.
\item
Then merge small fronts into larger fronts
using the {\tt ETree\_mergeFrontsAny()} method.
\item
Then merge a large front into a chain of smaller fronts
using the {\tt ETree\_splitFronts()} method.
\end{itemize}
The {\tt ETree\_transform2()} method differs from the
{\tt ETree\_transform()} method in that it omits the
setp with {\tt ETree\_mergeFrontsAny()}.
Either method will be suitable in most cases.
\par
However, there are some times one method is to be preferred over
the other.
If we look again at the vertex elimination tree
in Figure~\ref{fig:R2D100-tree-vtx}, we see the top level separator
with nodes $\{90,\cdots,99\}$, and the two second level separators
with nodes $\{45,\cdots,47\}$ and $\{87,\cdots,89\}$.
If one looks at their block columns in Figure~\ref{fig:R2D100-fs-mtx}
we see that either of the two second level separators could be merged
with the top level separator without introducing any zero entries
into the factor.
Using the {\tt ETree\_mergeFrontsAny()} method could merge the top
level separator with one of its two children, and produce an
imbalanced tree, not as well suited for parallel computation had
the two separators not been merged.
\par
In a parallel environment, it is much more efficient to not merge
the top level separator with either of its second level separators.
The transformation methods in {\bf SPOOLES 1.0} created front trees that
were not as efficient for parallel processing, precisely because of
the use of the ``merge-with-any'' step.
This led us to write three separate merging methods to replace the
single method from the 1.0 release, and thus give us the ability to
avoid the trees unsuitable for parallel computation.
\par
The values of {\tt maxzeros} and {\tt maxsize} will have a fair
amount of influence on the efficiency of the factor and solves.
This is illustrated in Table~\ref{table:R3D13824-maxzero-maxsize}
for the {\tt R3D13824} matrix and a number of different combinations
of {\tt maxzeros} and {\tt maxsize}.
\par
\begin{table}[htbp]
\label{table:R3D13824-maxzero-maxsize}
\caption{R3D13824: the influence of {\tt maxzeros} and {\tt maxsize}.}
\begin{center}
\begin{tabular}{cc|ccccccc}
& & & \multicolumn{2}{c}{factor} & & \multicolumn{2}{c}{solve} & total\\
{\tt maxzeros} & {\tt maxsize}
   & init & CPU & mflops & postprocess & CPU & mflops & CPU \\ \hline
$~~~~~0$ & $\infty$ & 3.3 & 129.8 & 15.3 & 5.3 & 7.8 & ~7.1 & 146.2 \\
$~~~~10$ & $\infty$ & 3.5 & 129.2 & 15.3 & 3.3 & 5.3 & 10.5 & 141.3 \\
$~~~100$ & $\infty$ & 3.0 & 119.3 & 16.7 & 2.0 & 3.9 & 14.4 & 128.2 \\
$~~1000$ & $\infty$ & 3.0 & 121.8 & 16.7 & 1.4 & 3.5 & 17.0 & 129.7 \\
$~10000$ & $\infty$ & 3.5 & 138.1 & 16.8 & 1.5 & 4.0 & 17.8 & 147.1 \\
$~~1000$ &    32    & 3.3 & ~89.8 & 22.7 & 2.6 & 4.1 & 14.7 & ~99.8 \\
$~~1000$ &    48    & 3.1 & ~85.8 & 23.7 & 2.1 & 3.6 & 16.5 & ~94.6 \\
$~~1000$ &    64    & 3.1 & ~85.2 & 23.9 & 1.9 & 3.5 & 17.1 & ~93.7 \\
$~~1000$ &    80    & 3.0 & ~85.9 & 23.7 & 1.8 & 3.4 & 17.4 & ~94.1 \\
$~~1000$ &    96    & 3.0 & ~86.1 & 23.6 & 1.8 & 3.4 & 17.6 & ~94.3 \\
$~~~~~0$ &    32    & 3.3 & 100.3 & 19.8 & 6.6 & 7.9 & ~7.0 & 118.1 \\
$~~~~~0$ &    48    & 3.2 & ~97.0 & 20.4 & 6.1 & 7.6 & ~7.3 & 113.9 \\
$~~~~~0$ &    64    & 3.2 & ~95.8 & 20.7 & 5.4 & 6.9 & ~8.0 & 111.3 \\
$~~~~~0$ &    80    & 3.2 & ~96.7 & 20.5 & 5.5 & 7.1 & ~7.8 & 112.5 \\
$~~~~~0$ &    96    & 3.2 & ~97.2 & 20.4 & 5.2 & 6.7 & ~8.3 & 112.3
\end{tabular}
\end{center}
\end{table}
\par
\par
As the matrix size grows, the number of zero entries we can allow
into a front can also grow.
We recommend using {\tt maxzeros} somewhere between
{\tt 0.01*neqns} and {\tt 0.1*neqns}.
The exact value isn't crucial, what is important is to have the
smaller subtrees at the lower levels of the tree merged together.
The {\tt maxsize} parameter specifies the ``panel'' size of the
large fronts, and so influences the granularity of the BLAS3
computations in the factorization and solve.
If {\tt maxsize} is too large, then too much of the computations
in the factorization is done inside a front, which uses a slow
kernel.
If {\tt maxsize} is too small, then the fronts are too small to get
much computational efficiency.
We recommend using a value between {\tt 32} and {\tt 96}.
Luckily, the factor and solve times are fairly flat within this
range.
A value of {\tt 64} is what we customarily use.


\documentclass[]{article}
\usepackage{amssymb,amsmath}
\usepackage{hyperref}
\usepackage{graphicx}
\usepackage{natbib}
\bibliographystyle{plainnat}

\date{}
\title{MgSi Paper Supplement}

\begin{document}

	\maketitle
	\section{Methods}\label{methods}

	\subsection{Equilibrium Reactions}\label{equilibrium-reactions}
	We cast the equilibrium reaction equations in terms of total moles for ease of computation. We assume that all activities are equal to one in our formation, which gives
	\begin{align}
	K_{1} &= \frac{a_{Mg}a_{O}}{ a_{MgO}} = \frac{X_{Mg} X_O }{X_{MgO}} = \frac{M_{Mg}M_{O}M_m}{M_{MgO}M_c^2}\\
	K_{2} &= \frac{a_{Si}a_{O}^2}{ a_{SiO_2}} = \frac{X_{Si} X_O^2 }{X_{SiO_2}} = \frac{M_{Si}M_{O}^2M_m}{M_{SiO_2}M_c^3}\\
	K_{3} &= \frac{a_{Fe}a_{O}}{ a_{FeO}} = \frac{X_{Fe} X_O }{X_{FeO}} = \frac{M_{Fe}M_{O}M_m}{M_{FeO}M_c^2}\\
	K_{4} &= \frac{a_{MgO}a_{SiO_2}}{ a_{MgSiO_3}} = \frac{X_{MgO} X_{SiO_2} }{X_{MgSiO_3}} = \frac{M_{MgO}M_{SiO_2}}{M_{MgSiO_3}M_m}\\
	K_{5} &= \frac{a_{FeO}a_{SiO_2}}{ a_{FeSiO_3}} = \frac{X_{FeO} X_{SiO_2} }{X_{FeSiO_3}} = \frac{M_{FeO}M_{SiO_2}}{M_{FeSiO_3}M_m}
	\end{align}
	where $M_c$ and $M_m$ are the total moles in the core and mantle interaction layer.

	Then, to compute the change in equilibrium, we take the derivative of each equation with respect to temperature, which gives
	\begin{align}
	\frac{(\partial_T K_1)}{K_1} &= \frac{(\partial_T M_{Mg})}{M_{Mg}} + \frac{(\partial_T M_{O})}{M_{O}} + \frac{(\partial_T M_{m})}{M_{m}} - \frac{2(\partial_T M_{c})}{M_{c}} - \frac{(\partial_T M_{MgO})}{M_{MgO}}
	\\
	\frac{1}{K_2}\partial_T K_2 &= \frac{1}{M_{Si}}\partial_T M_{Si} +
	\frac{2}{M_{O}}\partial_T M_{O} + \frac{1}{M_{m}}\partial_T M_{m} -
	\frac{3}{M_{c}}\partial_T M_{c} -
	\frac{1}{M_{SiO_2}}\partial_T M_{SiO_2} \\
	\frac{1}{K_3}\partial_T K_3 &= \frac{1}{M_{Fe}}\partial_T M_{Fe} +\frac{1}{M_{O}}\partial_T M_{O} +\frac{1}{M_{m}}\partial_T M_{m}-\frac{2}{M_{c}}\partial_T M_{c} - \frac{1}{M_{FeO}}\partial_T M_{FeO}\\
	\frac{1}{K_4}\partial_T K_4 &= \frac{1}{M_{MgO}}\partial_T M_{MgO} +\frac{1}{M_{SiO_2}}\partial_T M_{SiO_2} -\frac{1}{M_{MgSiO_3}}\partial_T M_{MgSiO_3} -\frac{1}{M_{m}}\partial_T M_{m} \\
	\frac{1}{K_5}\partial_T K_5 &= \frac{1}{M_{FeO}}\partial_T M_{FeO} +
	\frac{1}{M_{SiO_2}}\partial_T M_{SiO_2} -
	\frac{1}{M_{FeSiO_3}}\partial_T M_{FeSiO_3} -
	\frac{1}{M_{m}}\partial_T M_{m} \, .
	\end{align}
	We have analytical expressions for how the equilibrium constants $K_i$ vary with temperature, so the independent variables in this system consist of the nine $M_i$ species and two total values $M_c$ and $M_m$. We wish to solve this system to obtain a set of first-order coupled ODEs. Therefore, with eleven variables and five equations, we need six more constraints. Four constraints are provided by maintaining the total number of moles of each atomic species between the core, interaction layer, and exchange with the background mantle:
	\begin{align}
	0 &= \partial_T M_{Mg} + \partial_T M_{MgO} + \partial_T M_{MgSiO_3} + \left(\partial_T M_{MgO}\right)_{erosion} + \left(\partial_T M_{MgSiO_3}\right)_{erosion} \\
	0 &= \partial_T M_{Si} + \partial_T M_{SiO_2} + \partial_T M_{MgSiO_3} + \partial_T M_{FeSiO_3} + \left(\partial_T M_{SiO_2}\right)_{erosion} +\left(\partial_T M_{MgSiO_3}\right)_{erosion} +\left(\partial_T M_{FeSiO_3}\right)_{erosion}\\
	0 &= \partial_T M_{Fe} + \partial_T M_{FeO} + \partial_T M_{FeSiO_3} + \left(\partial_T M_{FeO}\right)_{erosion} + \left(\partial_T M_{FeSiO_3}\right)_{erosion}\\
	0 &= \partial_T M_{O}+\partial_T M_{MgO}+ \partial_T M_{FeO} +2\partial_T M_{SiO_2} + 3\partial_T M_{MgSiO_3} + 3\partial_T M_{FeSiO_3} +\left(\partial_T M_{MgO}\right)_{erosion} +\left(\partial_T M_{FeO}\right)_{erosion} +2\left(\partial_T M_{SiO2}\right)_{erosion} +3\left(\partial_T M_{MgSiO_3}\right)_{erosion} +3\left(\partial_T M_{FeSiO_3}\right)_{erosion} \,.
	\end{align}
	Then, the final two constraints are provided by ensuring mass continuity in the core and mantle
	\begin{align}
	0 &=\partial_T M_{c}+ \partial_T M_{Mg}+ \partial_T M_{Fe} +\partial_T M_{Si} + \partial_T M_{O}\\
	0 &= \partial_T M_{m}+ \partial_T M_{MgO}+ \partial_T M_{FeO} +\partial_T M_{SiO_2} + \partial_T M_{MgSiO_3} + \partial_T M_{FeSiO_3} +\left(\partial_T M_{MgO}\right)_{erosion} +\left(\partial_T M_{FeO}\right)_{erosion} +\left(\partial_T M_{SiO2}\right)_{erosion} +\left(\partial_T M_{MgSiO_3}\right)_{erosion} +\left(\partial_T M_{FeSiO_3}\right)_{erosion} \,.
	\end{align}

	\subsection{Interaction Layer Erosion}
	As detailed in the main text, this layer is removed and replaced with fresh background mantle on a timescale $\tau$ governed by mantle convection. To model this, we create an empirical function that pushes the mantle interaction layer composition towards the background mantle composition on a timescale governed by $\tau$:
	\begin{equation}
	\left(\partial_t M_i\right)_{erosion}=\frac{sgn(M_{i,b}-M_i)}{\tau}[(|M_i / M_{i,b}-1|+1)^2-1] \,.
	\end{equation}
	In the absence of continued exsolution, this expression returns the layer composition to the background mantle composition on the timescale $\tau$ of mantle convection. For incorporation into the above equations, the derivative with respect to time must be converted to a derivative with respect to temperature
	\begin{equation}
	\left(\partial_T M_i\right)_{erosion} = \frac{\left(\partial_t M_i\right)_{erosion}}{ \tfrac{dT_c}{dt}}
	\end{equation}
	where $\tfrac{dT_c}{dt}$ is simply the change in temperature at the CMB.

	\subsection{Equilibrium Constants}
	We use data from \citet{Badro2016}, and \citet{Hirose2017} to compute equilibrium constants in our model. For eq (1-3) we compute equilibrium constants using the form
	\begin{equation}\label{eq:K}
	K_i(T) = 10^{a+b/T} ,\qquad \partial_T K_i(T) = \tfrac{b}{T} 10^{a+b/T}
	\end{equation}
	TUSHAR TODO
	We find that computing the full expression given in \citet{Hirose2017} introduces numerical instabilities in our model due to the changing silicon and oxygen content in our core. Therefore, we choose to use the functional form \eqref{eq:K} while modifying the reported fit values to more accurately fit the reported experimental outcomes.

	\subsection{First-Order ODE System} \label{solution}

	With total moles in the mantle \(M_m\) and core \(M_c\), There are
	eleven components \(M_i\) in the system and eleven equations. This
	system is solved to obtain a system of first-order nonlinear ODEs of the
	form
	$$\partial_T M_i = f(M_i, K_j, \partial_T K_j)$$
	where \(M_i\) represents each molar species in the mantle and core and \(K_j\) is each equilibrium constant. Due to the highly non-linear nature of this set of equations, each equation has several thousand terms. Further simplification of the equations may be possible to reduce the number of terms, but there is no significant performance penalty when computing the system so it was not considered necessary.

	\section{Coupled Thermo-Chemical Evolution} \label{coupled-thermo-chemical-evolution}

	The chemical reactions are incorporated into the core-mantle thermal
	evolution model. With two temperatures \(T_{CMB}\) and \(T_{UM}\), and
	nine molar species, this creates and eleven-component state vector
	\(\mathbf{x}\). At each timestep, the mantle uses \(T_{CMB}\) and
	\(T_{UM}\) to compute a heat flux at the surface and core-mantle
	boundary and uses this to compute \(\partial_t T_{UM}\). Then, the
	system of chemical reaction equations uses the current \(T_{CMB}\) and
	molar concentrations \(M_i\) to compute \(\partial_T M_i\). The core
	uses \(Q_{CMB}\) from the mantle and \(\partial_T M_i\) from exsolution
	of light elements to compute \(\partial_t T_{CMB}\). Finally, this
	change in temperature is used to compute the change in molar
	concentrations with time by
	\(\partial_t M_i = \partial_T M_i \partial_t T_{CMB}\). This allows
	for the computation of \(\partial_t \mathbf{x}\) at any state.

	We compute the heat release with change in CMB temperature using
	$$\tilde{Q}_{g,i} \partial_t T_c = Q_{g,i}$$
	where \(\tilde{Q}_{g,i}\) depends upon the change in wt\% of the light element with temperature (see Nimmo 2015 eq. 73). This is easily computed given \(\partial_T M_i\). Then, we include these in addition to the inner-core and secular cooling terms to obtain
	$$\tilde{Q}_T = \tilde{Q}_s+\tilde{Q}_g+\tilde{Q}_L +\tilde{Q}_{g,MgO}+\tilde{Q}_{g,FeO}+\tilde{Q}_{g,SiO2}+\tilde{Q}_{L,MgO}+\tilde{Q}_{L,FeO}+\tilde{Q}_{L,SiO2}$$
	which is used the the calculation of the core energy budget (see Nimmo
	2015 eq. 73). We modify the expression for entropy in the core in a similar manner, including terms for exsolution of light elements
	$$\tilde{E}_T = \tilde{E}_s+\tilde{E}_g+\tilde{E}_L+\tilde{E}_H +\tilde{E}_{g,MgO}+\tilde{E}_{g,FeO}+\tilde{E}_{g,SiO2}$$
	(see Nimmo 2015, eq. 72). Note that this expression does not include contributions from latent heat release from exsolution of light elements because heat produced at the CMB does not contribute to thermal convection.

	As noted in the main text, we do not allow light elements to diffuse into the core because they would form a thin stratified layer and fail to mix into the bulk core. We implement this by requiring that \(\partial_t M_i \le 0\) for i=Mg,Si,O. We do allow Fe to diffuse into the core, but this happens only rarely as a result of changing mantle composition due to mantle overturn in our models and must be matched by exsolution of light elements. Oxygen cannot diffuse into the core and therefore the O concentration cannot increase, which requires that any FeO entering the core be matched by MgO or SiO2 pulling Oxygen out at the same or faster rate.

		\bibliography{citations}
\end{document}


\Lecture{Jayalal Sarma}{Oct 5, 2020}{14}{Mobius Inversion and applications}{K Sampreeth Prem}{$\alpha$}{JS}

\section{Introduction}
The focus on this lecture will be if we have two funtions $f$ and $g$ such that g can be expressed as a function of $f$ by a peculiar relation then how can we express $f$ in terms of $g$. We have already seen one such inversion during the discussion of stronger version of PIE where we have two fuctions mapping from sets to numbers. In this lecture we will look at a different setup where we have two functions $f:\{\mathbb{N} \} \to \{\mathbb{N}\}$ and $g:\{\mathbb{N} \} \to \{\mathbb{N} \}$ and if $g$ can be expressed in a peculiar way w.r.t $f$ then we want to invert to express $f$ in terms of $g$.This is called \textbf{Möbius Inversion}.
Before we formally state the theorem and prove it, we will look at two functions namely Euler's function and Möbius function and derive some of their properties.We will then establish a relation between these two functions to prove the Möbius Inversion and then look at an application of the theorem.

\section{Euler's Function} \label{sec:Euler's Function}
Euler's function, also known as phi-function $\Phi(n)$, counts the number of integers between 1 and n inclusive, which are coprime to n.
$$
\Phi(n) = |\{k \in \mathbb{N} | 1\leq k \leq n, gcd(n,k)=1\}| \label{fun:euler's}
$$
We have already derived the formula for $\Phi(n)$ in \ref{subsec:euler's function application} which is
\begin{align}
\Phi(n) = n\prod_{i=1}^{k}(1-\frac{1}{p_i}) \textrm{ ~~~where~~  $n = \prod_{i=1}^{k}(p_i)^{a_i} $}\label{for:Euler's formula}
\end{align}
\subsection{Properties of $\Phi(n)$} \label{subsec:Properties of Euler function}

\begin{property}
If $gcd(n1,n2)=1 \implies \Phi(n_1)\Phi(n_2) = \Phi(n_1n_2) $
\end{property}

\begin{proof}
We have derived this in \ref{subsec:euler's function application}
\end{proof}

\begin{property}
$ \sum_{d|n}^{} \Phi(d) = n $
\end{property}

\begin{proof}
Fix $d$ as a divisor of $n$. Consider the following sets $ A = \{x\in\{1,2,..n\} | gcd(x,n)=d\}$ and $B = \{y\in [\frac{n}{d}] | gcd(y,\frac{n}{d})=1\}$. Now we will establish a bijection between these two sets. The cardinality of the second set is $\Phi(\frac{n}{d})$ (by definition of $\Phi$) but instead of counting explicitly we will establish a bijection between A and B. Consider the mapping $x \mapsto \frac{x}{d}$ ,this mapping is well defined because $gcd(x,n) = d$ so $d|x$ and $\frac{x}{d}$ is an integer whose value is at most $\frac{n}{d}$ so it is in the range of B.The mapping is injective because division is injective. Now take any element from $1$ to $\frac{n}{d}$, multiply it with $d$ then we will get a number $p$ such that $gcd(p,n) = d$ because $d$ is the divisor of both $n$ and $p$. So the mapping is also surjective and hence a bijection.
Observe that the property can be rewritten as,
$$
\sum_{k = \frac{n}{d}}\Phi(\frac{n}{k})
$$
From the bijection established,$$
\sum_{d}|\{x\in\{1,2,..n\}|gcd(x,n)=d\}| = \sum_{k}\Phi(\frac{n}{k})
$$
Notice that the summation on L.H.S will be equal to $n$ that's because every number $l \in [1,n]$ will be counted exactly once in the summation when the variable of summation $d=gcd(x,n)$.\\
So,
\begin{align*}
n =  \sum_{k}\Phi(\frac{n}{k})\\
\sum_{k}\Phi(\frac{n}{k}) = n
\end{align*}
Hence, the proof.
\end{proof}

\section{Möbius Function} \label{sec:Möbius Function}
For any positive integer $n$, Möbius Function $\mu(n)$ is defined as,
53

	\[
	\mu(n) = \begin{cases}
	+1 , & ~if~ ~n~ ~is~ ~a~ ~\href{https://en.wikipedia.org/wiki/Square-free_integer}{square-free} ~positive~ ~integer~ ~with~ ~an~ ~even~ ~number~ ~of~ ~prime~ ~factors~~\\
 	-1 , & ~if~ ~n~ ~is~ ~a~ ~square-free~ ~positive~ ~integer~ ~with~ ~an~ ~odd~ ~number~ ~of~ ~prime~ ~factors~~\\
	0 , & otherwise
	\end{cases}
	\]
\noindent
For example,$15=3*5$ has even number of prime factors which are square-free, hence $\mu(15)=1$. Similarly, $30=2*3*5$ and $20={2^2}*5$ has $\mu$ values equal to $-1$ and $0$. \\
Now, consider the factors of $30$ and their $\mu$ values
\begin{center}
\begin{tabular}{c c c c c c c c c }
 Factors & 30 & 15 & 10 & 6 & 5 & 3 & 2 & 1 \\
 $\mu$ & -1 & 1 & 1 & 1 & -1 & -1 & -1 & 1 \\
\end{tabular}
\end{center}
\noindent
We Observe the sum of $\mu$ values of the factors of $30$ are equal to $0$. This isn't a coincidence and we will formally define this property and prove it.\\

\begin{property} \label{prp:2}
$\sum_{d|n}\mu(d) = \begin{cases} \label{subsec: Properties of Möbius Function }
	1 , & ~if~ n=1\\
	0 , & otherwise
	\end{cases}
	$
\end{property}

\begin{proof}

$$\mu(1) = 1 ~~(~by~ ~definition~)$$
Take $n= \prod_{i=1}^{k}(p_i)^{a_i}$, we will consider only those factors of $n$ whose prime factors have multiplicity $1$ other factors of $n$ which have prime factors multiplicity greater than 1 have $\mu$ value $0$ and does not contribute to the sum.
\begin{align}
\sum_{d|n}\mu(d) &= \sum_{d|p_1p_2...p_k}\mu(d) \label{pf:pt1} \\
&= \sum_{I\subseteq[k]}{(-1)}^I \label{pf:pt2} \\
&= \sum_{i=0}^{k} {k \choose i}{(-1)}^i\nonumber\\
&= 0 \nonumber
\end{align}
Notice that in $\ref{pf:pt1}$ the $\mu$ value depends only on whether we are picking odd number of primes or an even number and not on actual primes themselves so we could rewrite the summation to
$\ref{pf:pt2}$.
\end{proof}
\noindent
Now, we will see a corollary of the above propery that connects the two functions $\mu$ and $\Phi$. \\
\begin{corollary} \label{subsec:relation between euler function and mobius function}
$\frac{\Phi(n)}{n} = \sum_{d|n}\frac{\mu(d)}{d} $
\begin{proof}
From \ref{subsec:euler's function application}
\begin{align}
    \Phi(n) &= n\prod_{i=1}^{k}(1-\frac{1}{p_i}) \label{cor:pt1}\\
    &= n\sum_{I \subseteq [k]} (-1)^{|I|}\frac{1}{(\prod_{i \in I}p_i)}\label{cor:pt2}\\
    &= n\sum_{d|p_1...p_k}\frac{\mu(d)}{d} \label{cor:pt3} \\
    &= n\sum_{d|n}\frac{\mu(d)}{d} \label{cor:pt4}
\end{align}
In step \ref{cor:pt2}, $I\subseteq[k]$ can be thought of as those prime indices that are picked which contribute to $d$ so the ${(\prod_{i \in I}p_i)}$ becomes equal to $d$  and the numerator $(-1)^{|I|}$ is exactly $\mu(d)$. The generalisation of step \ref{cor:pt3} to \ref{cor:pt4} is done following the observation that if $d|n$ but $d\nmid\prod p_1..p_k$  then $d$ must have a prime factor whose multiplicity is greater than 1. Therefore, $\mu(d)$ becomes $0$ for such $d$'s.
\end{proof}
\end{corollary}

\section{Möbius Inversion} \label{sec:Möbius Inversion}
$f,g:\mathbb{N} \to \mathbb{R} ~satisfying,$ $\forall n ~g(n) = \sum_{d|n} f(d)  ~~then,$
$$\forall n ~f(n) = \sum_{d|n} \mu(d)d(\frac{n}{d})$$

\begin{proof}
Consider,
\begin{align}
    \sum_{d|n}\mu(d)g(\frac{n}{d}) &= \sum_{k=\frac{n}{d}}\mu(\frac{n}{d})g(d)  \label{pf:pt1}\\
    &= \sum_{d|n}\mu(\frac{n}{d})g(d) \label{pf:pt2} \\
    &=\sum_{d|n}\mu(\frac{n}{d})(\sum_{d'|d}f(d')\label{pf:pt5} \\
    &=\sum_{d'|n}C_{d'}f(d')  \label{pf:pt3}\\
    &=f(n) \label{pf:pt4}
\end{align}
At \ref{pf:pt3} $C_{d'}$ represents the constant that tells us how many times $f(d')$ is going to appear in the summation. Note that, if $d|n$ then $(\frac{n}{d})|n$ so using a change of variable we were able to reach \ref{pf:pt2} from \ref{pf:pt2}. \\
Now, we have two tasks, first calculate the value of $C_{d'}$ and next to show the summation value of \ref{pf:pt3} is $f(n)$.\\
\item {\textbf{Plan:} Evaluate $(C_{d'})$ for differnet $d'$}
\item \underline{\textbf{Case1:} }\text{~~If $d'=n$ then, $f(n)$ occurs only once in the summation \ref{pf:pt5}}. So, $C_n=1.$ \\

\item \underline{\textbf{Case2:}} $~~d'<n ~~and~~ d'|n.$
\begin{align}
    C_{d'} &= \sum_{d'|d|n} \mu(\frac{n}{d}) &\label{cs2:p1}\\
    &= \sum_{d|n}\mu(\frac{n}{d}) &\label{cs2:p2}\\
    &= \sum_{d|n}\mu(d) &\label{cs2:p3}\\
    &= 0 \nonumber
\end{align}
In step  $d \neq 1$ so by Property 14.3.1 (\ref{prp:2}) we get the value 0.
Observe that in step \ref{cs2:p1} the summation value doesn't depend on $d'$ so by change of variable we reach step \ref{cs2:p2}.Similarly, we reached step \ref{cs2:p3}  from \ref{cs2:p2} using change of variable that we have seen earlier.\\
Since we have calculated the value of $C_{d'}$ for different $d'$s  we will now show the value of summation in \ref{pf:pt3} is $f(n)$.
\begin{align}
\sum_{d'|n}C_{d'}f(d') &= C_nf(n)~~+~~\sum_{(d'~<~n)|n}C_{d'}f(d')\nonumber\\
&= 1*f(n) + 0 \nonumber\\
&= f(n) \nonumber
\end{align}
Hence, proved.
\end{proof}
\subsection{Application of Möbius Inversion} \label{Application of Möbius Inversion}
\textbf{Problem Statement:} Count the number of circular sequences/strings of $0$'s and $1$'s of length $n$.\\
\textbf{Solution:} We know that there are $2^n$ possible strings of $0$'s and $1$'s but some of them are clockwise rotations of each other and we consider them equal.So the circular sequences which are clockwise rotations of one another are equivalent. \\
Consider the following example of circular sequences of $0$'s and $1$'s of length 9. In order to represent a circular string in a normal form we fix a starting point and iterate it until it's length.    \\
A = 001011010\\
B = 110100010\\
C = 100100100\\
By our definition of equivalence $A \equiv B \not\equiv C$. \\
Observe that sequence C is periodic, it has a repeating subsequence 100 whereas sequences A and B do not contain any such repeating sub sequence.We call them aperiodic. In general a sequence is said to be aperiodic if it cannot be written as several times of a shorter sequence. \\
For every circular sequence there is a unique aperiodic circular sequence that we can associate with.\\
Instead of proving this claim we will write it as an example \\
For $n=6$,\\
$
000000  \Rightarrow  0 \\
001001  \Rightarrow 001 \\
111111  \Rightarrow  1 \\
010010  \Rightarrow 010 \\
001001  \Rightarrow 001 \\
$
Observe that the last two examples are equivalent and their circular subsequences are also equivalent. So their is an unique association (bijection) of circular sequences and their circular subsequences. \\
Another obvious observation is that the length of the circular subsequence must divide the original sequence. \\
Let,
\begin{align}
M(d) ~=~ ~no.~ ~of~ ~periodic~ ~circular~ ~sequences~ ~of~ ~length~ ~d~ \nonumber
\end{align}
Denote, $N_n$ as the number of circular sequences of length n, then \\
\begin{equation} \label{eq:circular seq}
    N_n = \sum_{d|n}M(d)
\end{equation}
Since we don't know the values of $d$ we are summing up over all values of $d$ that divides $n$. \\
Now we need to compute $M(d)$ as a function of $d$.
\textbf{Aim:} Compute $M(d)$ as a function of $d$.
\textbf{Solution:} Consider,the query how many aperiodic sequences of length d. Observe, that we have removed circular so equivalent ones are counted different. So, for every aperiodic circular sequence of length $d$, every shift of it is counted different hence, there are $dM(d)$ aperiodic sequences of length d. \\
We know that there are $2^n$ sequences(not circular) of length $n$ each one uniquely corresponds to an aperiodic sequence.\\
Hence,
\begin{align}
    2^n &= \sum_{d|n}dM(d)
\end{align}

Now, this is of the form $g(x) = \sum_{d|n}f(d)$ where, $g(x) = 2^n$ and $f(d)=d.M(d)$. \\
Using Möbius Inversion,

\begin{align}
    f(n) &= nM(n) \nonumber \\
    &= \sum_{d|n}\mu(d)g(\frac{n}{d}) \nonumber\\
    &= \sum_{d|n}\mu(d)2^{\frac{n}{d}}  \nonumber\\
From~ \ref{eq:circular seq},  \nonumber\\
    N_n &= \sum_{d|n}M(d)  \nonumber\\
    &= \sum_{d|n} \frac{1}{d}(\sum_{l|d}\mu(l)~2^{\frac{d}{l}}) \nonumber\\
    &= \sum_{d|n} \frac{1}{d}~\sum_{l|d}\mu(\frac{d}{l})~2^l \nonumber\\
    &= \sum_{l|n}(\sum_{l|d|n}\frac{1}{d}\mu(\frac{d}{l})2^l \nonumber\\
     &= \sum_{l|n}\sum_{1|{\frac{d}{l}}|{\frac{n}{l}}}\frac{1}{d}~\mu(\frac{d}{l})~2^l\nonumber \nonumber\\
    Substituting ~d~=~k~l, \nonumber \\
     &= \sum_{l|n}\sum_{k|{\frac{n}{l}}}\frac{2^l}{kl}~\mu(k)\nonumber\\
    &= \sum_{l|n}\frac{2^l}{l}\sum_{k|{\frac{n}{l}}}\frac{\mu(k)}{k} \nonumber\\
    &= \sum_{l|n}\frac{2^l}{l}~\frac{\Phi(\frac{n}{l})}{\frac{n}{l}}\nonumber\\
    &= \sum_{l|n}\frac{2^l~\Phi(\frac{n}{l})}{n} \nonumber\\
    N_n &= \frac{1}{n}\sum_{l|n}2^l~\Phi(\frac{n}{l}) \nonumber
\end{align}

\Lecture{Jayalal Sarma}{Oct 5, 2020}{15}{Generating Functions}{Simran}{$\alpha$}{JS}
\section{Introduction}
In this lecture, we will see a new tool for solving counting problems, namely generating functions method. To get an intuition of this method, let's think about a counting problem with parameter $n$ and say, we are interested in counting the number of objects of size $n$, denoted  by $a_n$. We can view $a_n$ as an infinite sequence of integers $(a_0,a_1,a_2,\dots)$, where we write down  the first few terms of sequence explicitly . Next we try to match it up with some known integer sequence, say Maclaurin sequence or fabonacci sequence . \\
If we find some initial matching, say first 4 or 5 terms, then we try for a bijection between matching sequence and the corresponding problem to our setting.\\
The overall idea of this method is to view the counting problem as infinite sequences and then encode these sequences using the algebraic expression (namely power series) and then do some power series manipulation to recover some meaningful things corresponding to the counting problem we are interested in .
\section{Generating Functions}
\begin{definition}\textbf{Generating Functions}
The generating function for a sequence of non-negative integers $(a_n)_{n\in \mathrm{N}}=(a_0,a_1,a_2,\dots)$ is given by $$F(x)=\sum_{n=0}^{\infty}a_nx^n$$
\end{definition}
\textbf{Remarks:}
\begin{itemize}
    \item[1] We are not going to evaluate the power series for any random value of $x$, say $F(100)$, as the series may not even be convergent .
\item[2] We think of $x$ in the expression $F(x)$ as a formal variable
\end{itemize}
We will indirectly use these expression in order to achieve our combinatorial goals without actually worrying about the convergence of the expression.

\noindent Let's look at an example to get more flavour of the idea introduced.

Consider a sequence with the first few terms as $a_0=1,a_1=42,a_2=23,\dots$.
The series corresponding to it is given by $A(x)=1+42x+23x^2+\dots$.
\\The advantage with this translation is that we can manipulate sequences in useful ways with simple algebra.

\noindent For the given $A(x)$, let's look at what happens when we multiply it by $x$.

We have $xA(x)=x+42x^2+23x^3+\dots$.
So we get, $a_0=0, a_2=42, a_3=23, \dots$ as the coefficient of the series $xA(x)$.
Hence we see that multiplying by $x$ corresponds to shifting the sequence to the right by 1.

\subsection{Operations on Generating Functions}
\noindent Let us look at other algebraic operations that can be easily and more naturally performed on the given generating functions, $F(x)=\sum_{n=0}^{\infty}a_nx^n$ and $G(x)=\sum_{n=0}^{\infty}b_nx^n$
\begin{itemize}
    \item[1] \textbf{Addition:} $$F(x)+G(x)=\sum_{n=0}^{\infty}(a_n+b_n)x^n$$
    \item[2] \textbf{Multiplication by a scalar $\lambda \in \mathbb{R}$} $$\lambda F(x)=\sum_{n=0}^{\infty}\lambda a_nx^n$$
    \item[3] \textbf{Differentiation } $$\frac{d}{d(x)}(F(x))= F'(x)=\sum_{n=1}^{\infty}(nx^{n-1})a_n$$
    Substituting $n$ with $n+1$, we get $$F'(x)=\sum_{n=0}^{\infty}(n+1)a_{n+1}x^n$$
   Suppose the sequence corresponding to  $F(x)$ is $(a_0,a_1,a_2,\dots)$ , then the sequence corresponding to  $F'(x)$ is clearly $(a_1,2a_2,3a_3,\dots)$
   \item[4] \textbf{Multiplication}
   $$F(x)G(x)=\sum_{n=0}^{\infty}(\sum_{k=0}^n a_nb_{n-k})x^n$$
\end{itemize}
\paragraph{Remark:} Note that the coefficients resulting after applying differentiation or multiplication to the generating function may sometimes not always be meaningful, but as we proceed we will look at few examples where the coefficients from multiplication will help us in countings.
\subsection{A Quick Example: Maclaurin Series}
The Maclaurin series is given by  $1+x+x^2+\dots$ which corresponds to the sequence $(1,1,1,\dots)$. The shorthand algebraic expression corresponding to this series is given by  $$F(x)=\frac{1}{1-x}$$  To see this note that
 $$(1-x)F(x)=F(x)-xF(x)=(1+x+x^2+\dots)-(x+x^2+x^3+\dots)=1$$
  Hence we have a neat shorthand algebraic expression for the Maclaurin Series . We will always think of this expression  as a formal power series and never going to substitute the value for $x$.
Now, let us demonstrate few of the discussed algebraic operations on the generating function of Maclaurin Series and use it or manipulate it to reach some new generating functions and the corresponding sequence
\begin{itemize}
    \item Differentiating $F(x)=\frac{1}{1-x}$ with respect to $x$,
    $$F'(x)=\sum_{n=0}^{\infty}(n+1)a_{n+1}x^n=\frac{1}{(1-x)^2}$$
    So, we can say that $F'(x)$ is the generating function for the sequence $b_n=(n+1)=(1,2,3,\dots)$
    \item Substituting $x=-y$ in the generating function $F(x)$ gives us a new generating function $G(y)$ corresponding to the alternating sequence $(1,-1,1,-1,\dots)$.\\ Formally, we have $G(y)=F(-y)=\frac{1}{1+y}$.
\end{itemize}
\section{Applying Generating Functions To Counting Problems}
In this section we will look at  the ways to use generating functions to solve the counting problems.

\paragraph{Example 1:} Consider the situation where we need to distribute $n$ votes to $k$ candidates such that every candidate gets atleast one vote.

\noindent \textbf{Note:}  In this example, we will see the correspondence of the multiplication of two generating functions to our counting problem.

\noindent Let $a_n^{(k)}$ denote the number of ways of distributing $n$ votes to $k$ candidates with each candidate getting atleast one vote and Let $B^{(k)}=\sum_{n=0}^{\infty}a_n^{(k)}x^n$, be the generating function corresponding to $a_n^{(k)}$.\\
Let us look at the case where we have only candidate, i.e $k=1$, we have:
\[
  a_n^{(1)} =
  \begin{cases}
   0, & \text{for } n=0 \\
    1, & \text{for }n\ge 1
  \end{cases}
\]
So, $a_n^{(1)}=(0,1,1,\dots)$, which is shifted version of Maclaurin Series. So the corresponding generating function is given by $B^{(1)}(x)=x\frac{1}{1-x}$

\noindent \textbf{Observation:} Let there be $s$ male candidates and $t$ female candidates. So $a_n^{(s+t)}$ can be thought as distributing $n$ votes to $s+t$ candidates. Suppose male candidates got $l$ votes and female candidates got $n-l$ votes. Also, each candidate gets 1 vote. So,
$$a_n^{(s+t)}=\sum_{l=0}^n a_l^{(s)}a_{n-l}^{(t)}$$

Using the observation, we get $$B^{(s+t)}(x)= B^{(s)}B^{(t)}$$
Note that $$a_n^{(s+t)}x^{s+t}=\left(\sum_{l=0}^n a_l^{(s)}a_{n-l}^{(t)}\right)x^{s+t}$$
So, the product of two generating function has a meaning in this context, hence we can use it. We are interested in $B^{(k)}(x)$ and we have $B^{(1)}(x)$. Hence, $$B^{(k)}(x)=\left(B^{(1)}(x)\right)^k=\left(\frac{x}{1-x}\right)^k$$
Now, we have a generating function for the count we wanted. Our next step is to recover the count from this generating function.\\

\noindent \textbf{Aim:} To write down the Generating Function in the form of individual coefficient and then read off $a_n^{(k)}$\\
We have $B^{(k)}(x)=\frac{x^k}{(1-x)^k}$
Observe that, $$\frac{d^{k-1}}{dx^{k-1}}\left(\frac{1}{1-x}\right)=(k-1)!\left(\frac{1}{(1-x)^k}\right)$$
Using the observation, we have

\begin{align*}
    B^{(k)}(x) & = \frac{x^k}{(k-1)!}\left(\frac{d^{k-1}}{dx^{k-1}}\left(\frac{1}{1-x}\right)\right)\\
    & = \frac{x^k}{(k-1)!}\left(\frac{d^{k-1}}{dx^{k-1}}(1+x+x^2+\dots)\right)\\
    &= \frac{x^k}{(k-1)!}\left(\sum_{n=k-1}^\infty n(n-1)\dots (n=k+2)x^{n-k+1}\right) \\
    & =\sum_{n=k-1}^\infty \frac{n(n-1)\dots (n=k+2)}{(k-1)!} x^{n+1}
\end{align*}
Using the expansion of ${n \choose k-1}$, we get
\begin{align*}
     B^{(k)}(x) & =\sum_{n=k-1}^\infty {n \choose k-1} x^{n+1}\\
     & =\sum_{n=k}^\infty {n-1 \choose k-1} x^{n}\\
     &= \sum_{n=0}^\infty {n-1 \choose k-1} x^{n}
\end{align*}
Hence by reading off the coefficient of $x^n$, we have $a_n^{(k)}={n-1 \choose k-1}$.

\paragraph{} To summarise the above example,
 we had a Counting problem, from the counting world we went to generating functions world and then manipulated it to get the power series of the required count and then we recovered $a_n^{(k)}$.

\paragraph{Example 2:} Suppose  Count the number of non negative solutions to $a+b+c=n$, such that, \begin{itemize}
    \item $a$ is an even integer
\item $b$ is a non negative integer
\item $c\in \{0,1,2\}$
\end{itemize}
Intuitively, we can think of this as having  $n$ votes and 3 candidates where the first candidate, $a$, should receive an even number of votes and $b,c$ receives at
most 2 votes.\\
The idea here is to use generating functions to split the problem nicely.\\
Let us consider three simpler settings ,
\begin{itemize}
    \item[1] Suppose we have only one candidate $a$.\\ So, our equation reduces to $a=n$ and $a$ should get even votes. So we have, \begin{itemize}
        \item if $n$ is odd, there is no solution
        \item if $n$ is even, there is exactly one solution
    \end{itemize}
    Hence the corresponding sequence for this case looks like $(1,0,1,\dots)$ and the generating function is given by
    $$A(x)=\sum_{n=0}^\infty x^{2n}=\sum_{n=0}^\infty (x^2)^n=\frac{1}{1-x^2}$$
    \item[2] Suppose we have $b$ as the only candidate.\\ So, our equation reduces to $b=n$ and $b$ should get non-negative number of votes. So, whatever the value of $n$ is there exists a unique solution for the equation $b=n$.\\  Hence the corresponding sequence for this case looks like $(1,1,1,\dots)$ and the generating function is given by
    $$B(x)=\sum_{n=0}^\infty x^n=\frac{1}{1-x}$$
    \item[3] Suppose we have $c$ as the only candidate.\\ So, our equation reduces to $c=n$ and $c\in \{0,1,2\}$. Clearly,
    \begin{itemize}
        \item the number of solution is unique for $n \in \{0,1,2\}$
        \item the number of solutions is 0 for $n>2$.
    \end{itemize} Hence, the generating function for this function is given by $$C(x)=1+x+x^2$$
\end{itemize}
\textbf{Observation :} We are looking for non negative solutions of $a+b+c=n$, satisfying the three conditions discussed above.The coefficient of $x^n=x^{a+b+c}$  gives us the required number of solutions in the cases, where the coefficient of $x^a,x^b$ and $x^c$ will give us the number of solutions where $a$ is even, $b$ is non-negative integer and $c\in \{0,1,2\}$ respectively.

\noindent Using these the generating function of the count we are interested in  is given by multiplying the generating functions of the cases discussed. Hence we have, $$F(x)=A(x)B(x)C(x)=\frac{1+x+x^2}{(1-x^2)(1-x)}$$
Now we want to write $F(x)$ as sum of terms where we will have the well known Maclaurin Series equation, to simplify our work \\
Using the method of partial fraction we have
\begin{align}
    F(x) &= \frac{R}{1+x}+\frac{S}{1-x}+\frac{T}{(1-x)^2} \nonumber\\
    & \implies \frac{1+x+x^2}{(1-x^2)(1-x)}=\frac{R}{1+x}+\frac{S}{1-x}+\frac{T}{(1-x)^2}
\end{align}
Multiplying both sides by $(1-x^2)(1-x)$, we get
\begin{align*}
    1+x+x^2 &=R(1-x)^2+S(1-x)(1+x)+T(1+x)\\
    &= (R+S+T)+(T-2R)x+(R-S)x^2
\end{align*}
Equating the coefficients and solving we get,
$$R=1/4,S=-3/4,T=3/2$$
Finally substituting the values of $R,S$ and $T$ we have

$$F(x)=\frac{1}{4}\left(\sum_{n=0}^\infty (-1)^nx^n\right)-\frac{3}{4}\left(\sum_{n=0}^\infty x^n\right)+\frac{3}{2}\left(\sum_{n=0}^\infty (n+1)x^n\right)$$
Reading off the coefficients of $x^n$, the number of solutions of $a+b+c=n$ satisfying the properties specified is given by $$ a_n=\frac{(-1)^n}{4}-\frac{3}{4}+\frac{3(n+1)}{2}$$
\section{Catalan numbers using generating functions}
 We already have come across Catalan numbers in Week3 lectures and seen that $$C_n=\frac{1}{n+1}{2n \choose n }$$
In this section we will derive this count using generating functions. Here we interpret Catalan Numbers $C_n$ as number of ways of forming well formed paranthesised expression with n pairs of brackets. In this example we will also see the notion of recurrence relation.
\paragraph{Recurrence Relation for Catalan numbers}
Basically, the recurrence relation for a sequence $(a_0,a_1,a_2,\dots,a_n,\dots)$, tries to figure out the relation between $a_n$ and the previous $a_i$'s.\\
Let $(a_0,a_1,a_2,\dots)$ denote the sequence for the given counting problem, where we use $a_n$ to denote the number of ways of forming well formed paranthesised expression with $n$ pairs of brackets.\\
To count $a_n$, we approach in the following steps:
\begin{itemize}
    \item[1] Place one pair of bracket $($ and $)$
    \item[2] Place $k$ pairs of brackets inside the already placed pair of brackets
    \item[3] Place the remaining $n-k-1$ pairs of brackets adjacent to this placed bracketing
\end{itemize}
With this conditioning on $k$, we can write
$$a_n=\sum_{k=0}^{n-1}a_k a_{n-k-1}$$
So, generating function for $a_n$
$$F(x)=\sum_{n=0}^\infty a_n x^n=a_0+\sum_{n=1}^\infty a_n x^n$$
Here $a_0$ is the  number of ways of forming well formed paranthesised expression with 0 pairs of brackets, which is unique.
\\ So, $$F(x)=1+\sum_{n=1}^\infty a_n x^n$$
Substituting for $a_n$ using the recurrence relation obtained, we get
\begin{align*}
    F(x) & = 1+\sum_{n=1}^\infty\left(\sum_{k=0}^{n-1} (a_k a_{n-k-1})x^n\right)\\
    &= 1+x\left(\sum_{n=0}^\infty\left(\sum_{k=0}^n (a_k a_{n-k-1})x^n\right)\right)\\
    & =1+x(F(x)F(x))
\end{align*}
So, we get
\begin{align}
    F(x)&=1+x(F(x)^2) \nonumber \\
    & \implies x(F(x))^2-F(x)+1=0
\end{align}

Solving the quadratic equation, we get
$$F(x)=\frac{1-\sqrt{1-4x}}{2x}$$
Now, to simplify  $F(x)$ into some known power series form, we need to simply the $\sqrt{1-4x}$ term in the function. For this we need a separate tool, informally the generalisation of the binomial theorem.

\paragraph{Binomial Theorem :} For every $n,k \in \mathbb{N}$, we have
$$(1+x)^n=\sum_{k=0}^n {n \choose k}x^k= \sum_{k=0}^\infty {n \choose k}x^k $$

Now, we will see the extended version of the Binomial Theorem, known as Newton's Binomial Theorem, where we consider any $n \in \mathbb{R}$
\begin{theorem}\text{Newton's Binomial Theorem} \\ For all non-zero $n \in \mathbb{R}$, we have $$(1+x)^n= \sum_{k=0}^\infty {n \choose k}x^k$$
Proof Omitted
\end{theorem}

\noindent So, for any $n \in \mathbb{R}, n \ne 0$, we can compute the value of ${n \choose k}$ by using the expression $${n \choose k}=\frac{n(n-1)\dots (n-k+1)}{k!}$$.

\paragraph{Example:} Take $n=-7/2$ and $k=5$, so we have
$${-7/2 \choose 5}=\frac{(-7/2)(-9/2)(-11/2)(-13/2)(-15/2)}{5!}=-\frac{9009}{256}$$

\noindent Now, getting back to the generating function for the catalan numbers
\begin{equation} \label{gen fn}
    F(x)=\frac{1-\sqrt{1-4x}}{2x}=\frac{1-(1-4x)^{1/2}}{2x}
\end{equation}
Let us look at a lemma which will help us simplify the square root term.
\begin{lemma} \label{Lemma1}
For $ n \in \mathbb{N}$, we have $${1/2 \choose n}=(-1)^{n+1}{2n-2 \choose n-1}\frac{1}{2^{2n-1}}\cdot \frac{1}{n}$$
\end{lemma}
\begin{proof}
Left as an exercise to the readers. \textit{Hint}: Use induction on $n$

\end{proof}
\noindent Applying \ref{Lemma1} to $\sqrt{1+x}=(1+x)^{1/2}$, we have
\begin{align} \label{Simplifying Sqrt}
    \sqrt{1+x} &=\sum_{n=0}^\infty {1/2 \choose k}x^n \nonumber \\
    &= 1+ \sum_{n=1}^\infty (-2){2n-2 \choose n-1}\frac{1}{2^{2n}}\cdot \frac{1}{n}(-1)^{n+1}
\end{align}

Substituting \ref{Simplifying Sqrt} in \ref{gen fn} we get,
\begin{align*}
    F(x) &= \frac{1}{2x}\sum_{n=1}^\infty 2{2n-2 \choose n-1}\frac{(-1)^n(-4x)^n}{2^{2n}n} \\
    & = \frac{1}{x}\sum_{n=1}^\infty {2n-2 \choose n-1}\frac{1}{n}x^n\\
    & = \sum_{n=0}^\infty {2n \choose n}\frac{1}{n+1}x^n
\end{align*}
Hence the generating function for the catalan numbers is,
$$F(x)=\sum_{n=0}^\infty {2n \choose n}\frac{1}{n+1}x^n$$
and the coefficient $a_n={2n \choose n}\frac{1}{n+1}$ denotes the catalan numbers.

\section{Live Discussion Session, Oct 8}

Consider a standard dice with $(1,2,3,4,5,6)$ as the number on it's faces. Let us look at the sum values that can be obtained when two standard dice are rolled together and number of different ways in which the pair of standard  dice results to the sum value . Note that the minimum sum that can be obtained is $2$ (when  $1$ appears on each dice) and the maximum sum that a pair of standard dice can yeild is $12$ (when  $6$ appears on each dice).

\begin{center}
\begin{tabular}{c c c c c c c c c }
 Sum Value & 2 & 3 & 4 & \dots & 12   \\
 \# of pairs & 1 & 2 & 3 & \dots & 1  \\
\end{tabular}
\end{center}

Now, let's compute the value $\text{Pr[Sum value} = 8]$. We know that there are $36$ different pairs as an outcome when we two standard dice are rolled together. The pairs that result in $8$ are $(2,6),(3,5),(4,4),(5,3),(6,2)$. So we get
$$\text{Pr[Sum value} = 8]=\frac{5}{36}$$
\subsection{Crazy Dice Problem and Generating Functions}
The question of interest here is can we find a pair of dice with
$(a_1,a_2,a_3,a_4,a_5,a_6)$ and $(b_1,b_2,b_3,b_4,b_5,b_6)$ as their faces, such that throwing the two dice in sequence gives us the same sum distribution as a standard dice.
Let's try to figure it out using the generating functions method.
First, we try to relate a standard dice with $(1,2,3,4,5,6)$ as the number on it's faces to a sequence and then using the sequence we can figure out it's generating function.
\paragraph{Generating Function for a Standard Dice} Let $a_n$ denote the number of ways in which a dice when rolled can produce the number $n$. So for the standard dice ($1\le n \le 6$) we know there is a unique way to get each face $\{1,2,\dots,6\}$. So, the infinite sequence representing it is $(1,1,1,1,1,1,0,0,\dots)$ and the corresponding generating function is $$F(x)=x+x^2+x^3+x^4+x^5+x^6$$
The generating function for the sum value when two dice are thrown in a sequence can be given by multiplying the respective generating function of each die.

So, for two standard dice thrown in a sequence, the generating function for the sum of the values appeared on each die is given by
\begin{align} \label{Gen fnc: sum standard dice}
    f(x)=(x+x^2+x^3+x^4+x^5+x^6)^2
\end{align}
\paragraph{Observation:} For two dice with  $(a_1,a_2,a_3,a_4,a_5,a_6)$ and $(b_1,b_2,b_3,b_4,b_5,b_6)$ as their faces such that they fit in the requirement of our question, the product of their generating functions must be equal to \ref{Gen fnc: sum standard dice}

Let $p(x)=x^{a_1}+x^{a_2}+\dots+x^{a_6}$ and $p(x)=x^{b_1}+x^{b_2}+\dots+x^{b_6}$ denote the generating function of the two dice considered respectively.

So, from the observation we have that
\begin{align} \label{Eq1}
    p(x)q(x)=(x+x^2+x^3+x^4+x^5+x^6)^2
\end{align}
Using few algebraic manipulation we get $$x+x^2+x^3+x^4+x^5+x^6 = x(1+x)(1+x+x^2)(1-x+x^2)$$. Substituting this in \ref{Eq1}, we have
\begin{align} \label{Eq2}
    p(x)q(x)&=(x(1+x)(1+x+x^2)(1-x+x^2))^2\nonumber\\
    & = x^2(1+x)^2(1+x+x^2)^2(1-x+x^2)^2
\end{align}
An additional constraint on $p(x)$ and $q(x)$ is that $p(1)=q(1)=6$, since the sequence corresponding to both these must have 6 terms.

Now our task is to distribute the factors of $x^2(1+x)^2(1+x+x^2)^2(1-x+x^2)^2$ to $p(x)$ and $q(x)$ such that the \ref{Eq2} and the constraint $p(1)=q(1)=6$ is satisfied.
One such valid distribution of the factors is $p(x)=x(1+x)(1+x+x^2)$
and $q(x)=x(1+x)(1+x+x^2)(1-x+x^2)^2$. Simplifying these gives us
\begin{align}\label{eqn 3}
    p(x)=x + 2x^2 + 2x^3 + x^4
\end{align}
\begin{align}\label{eqn 4}
    q(x)=x + x^3 + x^4 + x^5 + x^6 + x^8
\end{align}
Using \ref{eqn 3}, we get $(a_1,a_2,a_3,a_4,a_5,a_6)=(1,2,2,3,3,4)$ and from \ref{eqn 4}, we get $(b_1,b_2,b_3,b_4,b_5,b_6)=(1,3,4,5,6,8)$.

So, we were able to get two dice with  faces $(1,2,2,3,3,4)$ and $(1,3,4,5,6,8)$ such that throwing these two dice in sequence gives us the same sum distribution as throwing a pair of standard dice.

\Lecture{Jayalal Sarma}{Oct 5, 2020}{16}{Recurrence relation for Derangements}{K Sampreeth Prem}{$\alpha$}{JS}

\section{Introduction}
We were looking at the tool named generating functions and towards the end of last lecture we have seen recurrence relation for catalan numbers.The idea was to express the $n^th$ term of the sequence as a function of the previous terms and along with the generating functions obtain a closed form expression for the generating function, then use the series expansion to get the $n^th$ term of the infinite sequence.
In this lecture we will focus on getting a recurrence relation for Derangements.

\section{Derangements}\label{Derangements}
A Derangement is a permutation in which none of the objects appear in their "natural" (i.e., ordered) place.
$$
D_n = | \{\sigma ~\in~ S_n | \forall i ~\sigma(i)\neq i\}|
$$
We have already seen the expression for $D_n$ using P.I.E in \ref{subsec:derangements application}
which is,
$$
(\sum_{k=0}^{n} \frac{(-1)^k}{k!})n!
$$
Now we will try to obtain this expression using recurrence relations.
\subsection{Recurrence relations for Derangements}
Before we state the recurrence relations, let's fix the initial conditions. \\
$D_0 = 1~$ because there are no elements out of their natural position so we take it as $1$  and $D_1=0~$ since, we cannot have a derangement on one element.
\begin{recurrence relation} \label{rel:1}
\begin{align}
    D_n ~=~ (n-1)~(D_{n-1}~+~D_{n-2})
\end{align}
\end{recurrence relation}
\begin{proof}
We will use a double counting argument to prove this.\\
On \textbf{L.H.S} it's the number of derangements on n elements. \\
On \textbf{R.H.S}, let $\sigma\in S_n$ denote the derangement. So, $\sigma(n)~\neq~n$.\\
Let $\sigma(n)=k$, now there are two cases, \\
\textbf{Case 1:} $\sigma(k)=n$ \\
So, we can remove  $n$ and $k$ from the set $\{1,...,n\}$ and by appropriate renaming the new $\sigma$($say~ \sigma'$) corresponds to a derangement $S_n-2$.Since there are $n-1$ ways of choosing $k$\\
\textbf{Case 2:} $\sigma(k)\neq n$ \\
Consider the following bijection to a derangement in $S_{n-1}$.\\
Swap $\sigma(k)$ and $\sigma(n)$ in the permutation. In the resulting permutation($say~ \sigma'$) $\sigma'(n)=j (j \neq n)$ and $\sigma'(k)=k$. This isn't a derangement because $\sigma'(k)=k$. So remove $k$ and by appropriate renaming $\sigma'$ gives a derangement in $S_{n-1}$. \\
So from \textbf{Case 1} and \textbf{Case 2} we conclude by fixing $k$ there are $(D_{n-1}~+~D_{n-2}~)$ derangements possible. Since, $k$ is arbitrary and there are $n-1$ ways of choosing $k$ we get number of derangements possible as,   $(n-1)~(D_{n-1}~+~D_{n-2})$.
\end{proof}
\begin{recurrence relation} \label{rel:2}
\begin{align}
    D_n ~=~ nD_{n-1}~+~(-1)^n
\end{align}
\end{recurrence relation}
\begin{proof}
From \ref{rel:1},
\begin{align}
     D_n ~&=~ (n-1)~(D_{n-1}~+~D_{n-2}) \nonumber \\
     D_n~-~nD_{n-1}~&=~-(D_{n-1}~-~(n-1)D_{n-2}) \nonumber
     \end{align}
     Substitute $A_n = D_n - nD_{n-1}$ \label{rpf:1},\\
    \begin{align}
     A_n~&=~-A_{n-1} \nonumber\\
     A_n~&=~(-1)^n \nonumber\\
     D_n~-~nD_{n-1}~&=~(-1)^n \nonumber
\end{align}
\end{proof}
\begin{recurrence relation} \label{rel:3}
\begin{align}
    D_n ~=~ n! - \sum_{k=0}^{n-1} {n \choose k}~D_k
\end{align}
\end{recurrence relation}
\begin{proof}
\begin{align}
    D_n ~&=~ n! - \sum_{k=0}^{n-1} {n \choose k}~D_k \nonumber \\
         n! ~&=~ D_n + \sum_{k=0}^{n-1} {n \choose k}~D_k \nonumber \\
     n! ~&=~ \sum_{k=0}^{n} {n \choose k}~D_k \label{rpf:1}
\end{align}
Now we will argue \ref{rpf:1} using double counting argument.\\
On \textbf{L.H.S} it's the number of permutations on $n$ numbers. \\
On \textbf{R.H.S}, observe that each permutation has some points which are in their natural position i.e., $\sigma(i)~=~i$ and others are derangements on the remaining elements.Now we condition on number of fixed points and there are ${n\choose k }$ ways of fixing $k$ points and remaining form derangements.Notice that we are not over counting across sums because once we fix the number of fixed points one permutation cannot have two different number of fixed points.
\end{proof}
\noindent
We have stated and proved the recurrence relations associated with derangements.
Now, we will derive the formula for Derangements using the recurrence relations.\\
Consider \ref{rel:2},  \\
\begin{align}
    D_n ~&=~ nD_{n-1}~+~(-1)^n \nonumber\\
    D_n ~-~ nD_{n-1} ~&=~ (-1)^n \nonumber
    \end{align}
    Divide throughout by $n!$
    \begin{align}
    \frac{D_n}{n!} ~-~ \frac{D_{n-1}}{{n-1}!}  ~&=~ \frac{(-1)^n}{n!} \nonumber\\
      Substitute~B_n~&=~\frac{D_n}{n!} \nonumber \\
    B_{n}~-~B_{n-1} ~&=~ \frac{(-1)^n}{n!} \nonumber
    \end{align}
    \begin{align}
    B_{n} ~&=~ B_{n-1}~+~\frac{(-1)^n}{n!} \nonumber \\
    B_{n} ~&=~ B_{n-1}~+~\frac{(-1)^n}{n!} \nonumber \\
     B_{n} ~&=~ B_{n-2}~+~\frac{(-1)^{n-1}}{{n-1}!}~+~\frac{(-1)^n}{n!} \nonumber
\end{align}
By repeatedly expanding the terms on \textbf{R.H.S},we get
\begin{align}
    B_{n} ~&=~ \sum_{k=0}^{n}\frac{(-1)^k}{k!} \nonumber \\
    D_{n} ~&=~ n!(\sum_{k=0}^{n} \frac{(-1)^k}{k!}) \nonumber
\end{align}


\subsection*{MILP (complete for ReLU, NP-complete)}
\begin{tabular}{|l@{~~}l}
\textbf{Objective} & $\min_{x_1, \ldots, x_n} c_1 x_1 + \ldots + c_n x_n$ \\
\textbf{Constraints} & $a_{i,1} x_1 + \ldots + a_{i,n} x_n \le b_i$, $1 \le i \le m$ \\
\textbf{Bounds} & $x_j \in [l_j, u_j]$ or $x_j \in \mathbb{Z}$, $1 \le j \le n$
\end{tabular}
\paragraph{Affine layer} $y=Wx+b$
\paragraph{ReLU layer}
\begin{tabular}[t]{l@{}l@{}r}
$y \le x - l(1-a)$ & \rdelim{\}}{5}{*} & $x \in [l, u]$\\
$y \le u a$ & & $a = 0 \Rightarrow (y=0$ \\
$y \ge x$ & & $\land~ x \in [l, 0])$ \\
$y \ge 0$ & & $a=1 \Rightarrow (y = x$\\
$a \in \{0, 1\}$ & & $\land~ x \in [0, u])$
\end{tabular}
\paragraph{Bounds}
$x_i \in[l_i, u_i]$ (precomputed box bounds for neurons) or $x_i' \in [x_i - \epsilon, x_i + \epsilon]$ (inputs)
\paragraph{Objective} $\min o_\text{label} - o_i$ (verification succeeds iff $o_\text{label} > o_i$)


\section*{Exercise 20.3-4}
\subsection*{What happens if you call vEB-TREE-INSERT with an element, that is already in the vEB tree?}

The procedure will terminate without inserting anything:
\\
Line 1-2 is not possible, since there is at least one element in the tree. We know this, since $x$ is a duplicate of an element in the tree.
\\
Line 3-4 is not possible either, since $x$ is a duplicate of an element in the tree, so the lowest possible value for $x$ is $x=V.min$
\\
Line 6-8 is not possible either, so eventually we will hit line 9, where the recursive call brings us to a smaller cluster. This will keep happening until we hit a base-case vEB tree, meaning we no longer fulfill line 5, and moves on to line 10-11.
\\
Since line 10-11 is not possible either (again, $x$ is a duplicate of an element in the tree, so the highest possible value for $x$ is $x=V.max$), so the procedure terminates without doing anything.

\subsection*{What happens if you call vEB-TREE-DELETE with an element, that is not in the vEB tree?}

A wrong value will be deleted:
\\
Line 1-3 simply checks if the tree has only one element, and if so, deletes it, regardless of the value of $x$.
\\
Line 4-8 checks if we are in a base-case vEB tree, and if so, updates $V.min$ and $V.max$, based on the value of $x$, which is assumed to be either 0 or 1 (the only two possible values in the base-case vEB tree). Since $x$ is non of these values, this update will end up deleting $V.max$ and leaving $V.min$ unchanged.
\\
Line 9-12 is not possible, since we know, that $x$ is not in the tree, and so we will hit line 12, where the recursive call brings us to a smaller cluster. This will keep happening until we hit a base-case vEB tree, which brings us to line 4-8, which ends up deleting $V.max.$
\\
Line 14-22 is not possible to reach.

\subsection*{Explain why the procedures exhibit the behavior that they do}

Recursive calls will always bring us down to a smaller tree, until we hit a base case.

\subsection*{Show how to modify vEB trees and their operations so that we can check in constant time whether an element is present}

We introduce an extra array, which has $u$ elements, that are bits. For each element in the tree, the value of the corresponding entry in the array represents, whether the element is present (1) or not (0).


\subsection{part c}
For transfer function we use common architecture.
\begin{figure}[H]
    \caption{Architecture}
    \centering
    \includegraphics[width=12cm]{../Figure/Q1/Q1_c/architecture.png}
\end{figure}
\begin{itemize}
    \item r to y refrence
    $$
    \dfrac{y}{r} = \dfrac{C(s)G(s)}{1+C(s)G(s)}
    $$
    \begin{figure}[H]
        \caption{r to y bode magnitude}
        \centering
        \includegraphics[width=12cm]{../Figure/Q1/Q1_c/bode_r2y.png}
    \end{figure}
    System has a good performance at high frequency but not good performance at low frequency.
    \item du to y distubance
    $$
    \dfrac{y}{du} = \dfrac{G(s)}{1+C(s)G(s)}
    $$
    \begin{figure}[H]
        \caption{du to y bode magnitude}
        \centering
        \includegraphics[width=12cm]{../Figure/Q1/Q1_c/bode_du2y.png}
    \end{figure}
    System has a better performance at high frequency but pretty good performance at low frequency.
    \item dy to y distubance
    $$
    \dfrac{y}{dy} = \dfrac{1)}{C(s)G(s)}
    $$
    \begin{figure}[H]
        \caption{dy to y bode magnitude}
        \centering
        \includegraphics[width=12cm]{../Figure/Q1/Q1_c/bode_dy2y.png}
    \end{figure}
    System has a good performance at high frequency but very bad performance at low frequency.
    \item n to y noise
    $$
    \dfrac{y}{du} = \dfrac{-C(s)G(s)}{1+C(s)G(s)}
    $$
    \begin{figure}[H]
        \caption{n to y bode magnitude}
        \centering
        \includegraphics[width=12cm]{../Figure/Q1/Q1_c/bode_n2y.png}
    \end{figure}
    System has a good performance at high frequency but not good performance at low frequency.
\end{itemize}


\subsection{Time--discretization of the linear case~(\ref{first-DS3}) }
Let us now proceed with the time discretization of~(\ref{eq:toto1}) with FirstOrderLinearR~(\ref{first-DS3})  by a fully implicit scheme :
\begin{equation}
  \begin{array}{l}
    \label{eq:toto1-DS3}
     M x^{\alpha+1}_{k+1} = M x_{k} +h\theta A x^{\alpha+1}_{k+1}+h(1-\theta) A x_k + h \gamma r^{\alpha+1}_{k+1}+ h(1-\gamma)r(t_k)  +hb\\[2mm]
     y^{\alpha+1}_{k+1} =  C x^{\alpha+1}_{k+1} + D \lambda ^{\alpha+1}_{k+1} +Fz +e\\[2mm]
     r^{\alpha+1}_{k+1} = B \lambda ^{\alpha+1}_{k+1} \\[2mm]
  \end{array}
\end{equation}

\[R_{\free} = M(x^{\alpha}_{k+1} - x_{k}) -h\theta A x^{\alpha}_{k+1} - h(1-\theta) A x_k -hb_{k+1} \]
\[R_{\free} = W(x^{\alpha}_{k+1} - x_{k}) -h A x_{k} -hb_{k+1} \]

\paragraph{Resulting Newton step (only one step)}
For the sake of simplicity, let us assume that $\gamma =1$
\begin{equation}
  \begin{array}{l}
     (M -h\theta A)x^{\alpha+1}_{k+1} = M x_{k} +h(1-\theta) A x_k + hr^{\alpha+1}_{k+1} + hb\\[2mm]
     y^{\alpha+1}_{k+1} =  C x^{\alpha+1}_{k+1} + D \lambda ^{\alpha+1}_{k+1} +Fz + e \\[2mm]
     r^{\alpha+1}_{k+1} = B \lambda ^{\alpha+1}_{k+1}\\[2mm]
  \end{array}
\end{equation}
that lead to with: $ (M -h\theta A) = W$
\begin{equation}
  \begin{array}{l}
     x^{\alpha+1}_{k+1} = W^{-1}(M x_{k} +h(1-\theta) A x_k + r^{\alpha+1}_{k+1} +hb) = x\free + W^{-1}(r^{\alpha+1}_{k+1})\\[2mm]
     y^{\alpha+1}_{k+1} =  ( D+hCW^{-1}B) \lambda ^{\alpha+1}_{k+1} +Fz + CW^{-1}(M
     x_k+h(1-\theta)Ax_k + hb) +e \\[2mm]
  \end{array}
\end{equation}
with $x_{\free} = x^{\alpha}_{k+1} + W^{-1}(-R_{\free})= x^{\alpha}_{k+1} - W^{-1}(W(x^{\alpha}_{k+1}
- x_k) -hAx_k-hb_{k+1} )= W^{-1}(Mx_k +h(1-\theta)Ax_k +h b_{k+1})$
\begin{equation}
  \begin{array}{l}
     y^{\alpha+1}_{k+1} =  ( D+hCW^{-1}B) \lambda ^{\alpha+1}_{k+1} +Fz + Cx_{\free}+e\\[2mm]
     r^{\alpha+1}_{k+1} = B \lambda ^{\alpha+1}_{k+1}\\[2mm]
  \end{array}
\end{equation}

\paragraph{Coherence with previous formulation}
\[y_p = y^{\alpha}_{k+1} -\mathcal R^{\alpha}_{yk+1} + C^{\alpha}_{k+1}(x_p -x^{\alpha}_{k+1}) -
D^{\alpha}_{k+1} \lambda^{\alpha}_{k+1} \]
\[y_p = Cx_k + D \lambda _k  + C(\tilde x_{\free}) -D \lambda_k +Fz + e\]
\[y_p = Cx_k   + C(\tilde x_{\free})  +Fz + e\]
\[y_p = Cx_k   + C(\tilde x_{\free})  +Fz + e\]
\[y_p = C(x_{\free})  +Fz + e\]

%In the case of the system~(\ref{eq:deux}) with a affine function $f$ or $\theta =0$, the the MLCP matrix $W$ can be computed before the beginning of the time loop saving a lot of computing effort.  In the case of the system (\ref{eq:trois}) with $\theta=\gamma=0$, the MLCP matrix $W$ can be computed before the beginning of the Newton loop.
\clearpage

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "DevNotes"
%%% End:


\hypertarget{group__numpp__structures__matrices}{}\section{Matrices}
\label{group__numpp__structures__matrices}\index{Matrices@{Matrices}}

Module containing different matrix implementation.

\subsection*{Modules}
\begin{DoxyCompactItemize}
\item
\hyperlink{group__numpp__structures__matrices__dense}{Dense}
\begin{DoxyCompactList}\small\item\em Module containg constexpr dense matrix implementation. \end{DoxyCompactList}\item
\hyperlink{group__numpp__structures__matrices__sparse}{Sparse}
\begin{DoxyCompactList}\small\item\em Module containg sparse matrix implementations. \end{DoxyCompactList}\end{DoxyCompactItemize}

\subsection{Detailed Description}
Module containing different matrix implementation.

\begin{DoxyWarning}{Warning}
{\bfseries Compile-\/time implementation of dense matrix will be inefficient for bigger matrices compile-\/wise, but runtime is much faster.~\newline
 Sparse vector isn\textquotesingle{}t be affected by this way of implementation}
\end{DoxyWarning}
This is a summary header including every matrix structure provided by numpp.

{\bfseries \begin{DoxyWarning}{Warning}
If you are looking for more compile-\/runtime balanced implementations check Eigen, Armadillo or other libraries
\end{DoxyWarning}

\begin{DoxyCode}
\textcolor{preprocessor}{#include"numpp/structures/matrices.hpp"}
\end{DoxyCode}
 }


%% State Space Modelling of Dynamic Systems
%% Lecture 20: State Feedback Control
\def\FileDate{10/02/02}
\def\FileVersion{1.0}
% ----------------------------------------------------------------
% Notes pages *********************************************************
% ----------------------------------------------------------------

\begin{slide}
   \heading{State Feedback Control}
One of the advantages of state space models is that it is possible to apply state feedback to place the closed loop poles into any desired positions.

\textbf{State Space Design Methodology}

\begin{enumerate}
	\item Design control law to place closed loop poles where desired
	\item If full state not available for feedback, then design an \emph{Observer} to compute the states from the system output
	\item Combine \emph{Observer} and \emph{Controller} -- this takes the place of the \emph{Classical Compensator}
	\item Introduce the \emph{Reference Input} -- affects the closed loop zeros but not the poles making it possible to improve the transient response and tracking accuracy
\end{enumerate}
\end{slide}

\begin{slide}
   \heading{State Feedback Compensator}
   \begin{center}
   	\resizebox{280pt}{!}{\includegraphics{pictures/statefb.pdf}}
   \end{center}
\end{slide}

\ifslidesonly
\begin{slide}
   \heading{This Lecture}
   \begin{itemize}
   	\item Finding the control law
   	\item State feedback for controller canonical form
   	\item Transfer function model
   	\item Ackermann's formula
   	\item Effect of state feedback on closed-loop zeros
   	\item Effect of plant zeros on the feedback gains
   	\item Pole-selection for good design
   \end{itemize}
\end{slide}
\fi
\section*{Finding the Control Law} % (fold)
\label{sec:finding_the_control_law}

We shall only consider SISO systems here.

\input{frag1}
\ifslidesonly
\begin{slide}
   \heading{Finding the Control Law (1)}
   \input{frag1}
\end{slide}
\fi

\input{frag2}
\ifslidesonly
\begin{slide}
	\heading{Finding the Control Law (2)}
   \input{frag2}
\end{slide}
\fi

\input{frag3}
\ifslidesonly
\begin{slide}
	\heading{Finding the Control Law (3)}
   \input{frag3}
\end{slide}
\fi

\input{frag4}
\ifslidesonly
\begin{slide}
	\heading{Finding the Control Law (4)}
   \input{frag4}
\end{slide}
\fi

\subsection*{Example 1} % (fold)
\label{sub:example_1}

\textbf{Problem}: Given,
\[
{\bf{\dot x}} = \left[ {\begin{array}{*{20}c}
   { - 4} & 0  \\
   0 & { - 11}  \\
\end{array}} \right]{\bf{x}} + \left[ {\begin{array}{*{20}c}
   1  \\
   { - 1}  \\
\end{array}} \right]u
\]
find the feedback control law which places the closed-loop poles at: $-10\pm j10$.

\textbf{SOLUTION}:
\begin{eqnarray*}
	0 & = & \det \left[ {s{\bf{I}} - {\bf{A}} + {\bf{BK}}} \right] = \det \left\{ {\left. {\left[ {\begin{array}{*{20}c}
	   {s + 4} & 0  \\
	   0 & {s + 11}  \\
	\end{array}} \right] + \left[ {\begin{array}{*{20}c}
	   1  \\
	   { - 1}  \\
	\end{array}} \right]\left[ {\begin{array}{*{20}c}
	   {k_1 } & {k_2 }  \\
	\end{array}} \right]} \right\}} \right. \\
	0 & = & \det \left[ {\begin{array}{*{20}c}
	   {s + 4 + k_1 } & {k_2 }  \\
	   { - k_1 } & {s + 11 - k_2 }  \\
	\end{array}} \right] \\
	0 & = & (s + 4 + k_1 )(s + 11 - k_2 ) - (k_2 )( - k_1 ) \\
	0 & = & (s+4+k_1)(s+11-k_2)+k_1k_2
\end{eqnarray*}
\begin{equation}
	\label{eq:3}
	s^2+(15+k_1-k_2)s+(44+11k_1-4k_2)=0
\end{equation}

Now the desired CE is:
\[
\alpha_c(s)=(s+10-j10)(s+10+j10) = 0
\]
\begin{equation}\label{eq:4}
	s^2+20s+200=0
\end{equation}

Therefore matching coefficients in Eqs. (\ref{eq:3}) and (\ref{eq:4}):
\[
\begin{array}{c}
 s^2 :1 = 1 \to {\rm{OK}} \\
 s^1 :15 + k_1  - k_2  = 20 \to k_1  - k_2  = 5 \\
 s^0 :44 + 11k_1  - 4k_2  = 200 \to 11k_1  - 4k_2  = 156 \\
 \end{array}
\]

Solving for the $k$'s:
\[
\left[ {\begin{array}{*{20}c}
   1 & { - 1}  \\
   {11} & { - 4}  \\
\end{array}} \right]\left[ {\begin{array}{*{20}c}
   {k_1 }  \\
   {k_2 }  \\
\end{array}} \right] = \left[ {\begin{array}{*{20}c}
   5  \\
   {156}  \\
\end{array}} \right]
\]
% MathType!MTEF!2!1!+-
% faaagaart1ev2aaaKnaaaaWenf2ys9wBH5garuavP1wzZbqedmvETj
% 2BSbqefm0B1jxALjharqqtubsr4rNCHbGeaGqiVu0Je9sqqrpepC0x
% bbL8FesqqrFfpeea0xe9Lq-Jc9vqaqpepm0xbba9pwe9Q8fs0-yqaq
% pepae9pg0FirpepeKkFr0xfr-xfr-xb9Gqpi0dc9adbaqaaeGaciGa
% aiaabeqaamaabaabaaGcbaWaamWaaeaafaWabeGabaaabaGaam4Aam
% aaBaaaleaacaaIXaaabeaaaOqaaiaadUgadaWgaaWcbaGaaGOmaaqa
% baaaaaGccaGLBbGaayzxaaGaeyypa0ZaaSaaaeaacaaIXaaabaGaey
% OeI0IaaGinaiabgUcaRiaaigdacaaIXaaaamaadmaabaqbamqabiGa
% aaqaaiabgkHiTiaaisdaaeaacaaIXaaabaGaeyOeI0IaaGymaiaaig
% daaeaacaaIXaaaaaGaay5waiaaw2faamaadmaabaqbamqabiqaaaqa
% aiaaiwdaaeaacaaIXaGaaGynaiaaiAdaaaaacaGLBbGaayzxaaGaey
% ypa0ZaaSaaaeaacaaIXaaabaGaaG4naaaadaWadaqaauaadeqaceaa
% aeaacaaIXaGaaG4maiaaiAdaaeaacaaIXaGaaGimaiaaigdaaaaaca
% GLBbGaayzxaaGaeyypa0ZaamWaaeaafaWabeGabaaabaGaaGymaiaa
% iMdacaGGUaGaaGinaiaaiMdacaaIYaGaaGyoaaqaaiaaigdacaaI0a
% GaaiOlaiaaisdacaaIYaGaaGyoaaaaaiaawUfacaGLDbaaaaa!5BC6!
\[
\left[ {\begin{array}{*{20}c}
   {k_1 }  \\
   {k_2 }  \\
\end{array}} \right] = \frac{1}{{ - 4 + 11}}\left[ {\begin{array}{*{20}c}
   { - 4} & 1  \\
   { - 11} & 1  \\
\end{array}} \right]\left[ {\begin{array}{*{20}c}
   5  \\
   {156}  \\
\end{array}} \right] = \frac{1}{7}\left[ {\begin{array}{*{20}c}
   {136}  \\
   {101}  \\
\end{array}} \right] = \left[ {\begin{array}{*{20}c}
   {19.429}  \\
   {14.429}  \\
\end{array}} \right]
\]

Therefore the required feedback control law is:
% MathType!MTEF!2!1!+-
% faaagaart1ev2aaaKnaaaaWenf2ys9wBH5garuavP1wzZbqedmvETj
% 2BSbqefm0B1jxALjharqqtubsr4rNCHbGeaGqiVu0Je9sqqrpepC0x
% bbL8FesqqrFfpeea0xe9Lq-Jc9vqaqpepm0xbba9pwe9Q8fs0-yqaq
% pepae9pg0FirpepeKkFr0xfr-xfr-xb9Gqpi0dc9adbaqaaeGaciGa
% aiaabeqaamaabaabaaGcbaGaamyDaiabg2da9iaadkhacqGHsislda
% WadaqaauaadeqabiaaaeaacaaIXaGaaGyoaiaac6cacaaI0aGaaGOm
% aiaaiMdaaeaacaaIXaGaaGinaiaac6cacaaI0aGaaGOmaiaaiMdaaa
% aacaGLBbGaayzxaaGaaCiEaaaa!3DCD!
\[
u = r - \left[ {\begin{array}{*{20}c}
   {19.429} & {14.429}  \\
\end{array}} \right]{\bf{x}}
\]

\textbf{COMMENT}
This matching of coefficients can always be done, though it is tedious for $n>3$, \textbf{EXCEPT} in the case of the \emph{Control Canonical Form}.

% subsection example_1 (end)

% section finding_the_control_law (end)

\section*{State Feedback in the Case of the Control Canonical Form} % (fold)
\label{sec:state_feedback_in_the_case_of_the_control_canonical_form}

\input{frag5}
\ifslidesonly
\begin{slide}
	\heading{Control Canonical Form Simplifies Calculation (1)}
   \input{frag5}
\end{slide}
\fi

\input{frag6}
\input{frag7}
\ifslidesonly
\begin{slide}
	\heading{Control Canonical Form (2)}
   \input{frag6}
\end{slide}
\begin{slide}
	\heading{Control Canonical Form (3)}
   \input{frag7}
\end{slide}
\fi

\input{frag8}
\ifslidesonly
\begin{slide}
	\heading{Control Canonical Form (4)}
   \input{frag8}
\end{slide}
\fi

\subsection*{Example 2} % (fold)
\label{sub:example_2}

\textbf{Problem}: Given the system TF:
\[
G(s) = \frac{7}{(s+4)(s+11)}
\]
find the control law for the control canonical form which places the closed loop poles at $s=−10\pm j10$.

\textbf{SOLUTION}:
\[
G(s) = \frac{7}{(s+4)(s+11)} = \frac{7}{(s^2+15s+44)}
\]

The control canonical form has matrices:
\[
{\bf{A}} = \left[ {\begin{array}{*{20}c}
   { - 15} & { - 44}  \\
   1 & 0  \\
\end{array}} \right];\quad {\bf{B}} = \left[ {\begin{array}{*{20}c}
   1  \\
   0  \\
\end{array}} \right];\quad {\bf{C}} = \left[ {\begin{array}{*{20}c}
   0 & 7  \\
\end{array}} \right];\quad {\bf{D}} = 0
\]

\textbf{NB}:  $\mathbf{C}$ is obtained from the TF numerator $(0s+7)$.
so:
\[
{\bf{A}} - {\bf{BK}} = \left[ {\begin{array}{*{20}c}
   { - 15 - k_1 } & { - 44 - k_2 }  \\
   1 & 0  \\
\end{array}} \right]
\]
and the closed loop CE is:
\begin{equation}
	s^2+(15+k_1)s+(44+k_2)=0 \label{eq:7}
\end{equation}
The desired CE is:
\[
\alpha_c(s)=(s+10-j10)(s+10+j10) = 0
\]
\begin{equation}\label{eq:8}
	s^2+20s+200=0
\end{equation}

Comparing Eqs. (\ref{eq:7})  and  (\ref{eq:8}) gives:
% MathType!MTEF!2!1!+-
% faaagaart1ev2aaaKnaaaaWenf2ys9wBH5garuavP1wzZbqedmvETj
% 2BSbqefm0B1jxALjharqqtubsr4rNCHbGeaGqiVu0Je9sqqrpepC0x
% bbL8FesqqrFfpeea0xe9Lq-Jc9vqaqpepm0xbba9pwe9Q8fs0-yqaq
% pepae9pg0FirpepeKkFr0xfr-xfr-xb9Gqpi0dc9adbaqaaeGaciGa
% aiaabeqaamaabaabaaGcbaGaaGymaiaaiwdacqGHRaWkcaWGRbWaaS
% baaSqaaiaaigdaaeqaaOGaeyypa0JaaGOmaiaaicdacqGHsgIRcaWG
% RbWaaSbaaSqaaiaaigdaaeqaaOGaeyypa0JaaGynaaaa!3A5E!
\[
15 + k_1  = 20 \to k_1  = 5
\]
and
% MathType!MTEF!2!1!+-
% faaagaart1ev2aaaKnaaaaWenf2ys9wBH5garuavP1wzZbqedmvETj
% 2BSbqefm0B1jxALjharqqtubsr4rNCHbGeaGqiVu0Je9sqqrpepC0x
% bbL8FesqqrFfpeea0xe9Lq-Jc9vqaqpepm0xbba9pwe9Q8fs0-yqaq
% pepae9pg0FirpepeKkFr0xfr-xfr-xb9Gqpi0dc9adbaqaaeGaciGa
% aiaabeqaamaabaabaaGcbaGaaGymaiaaiwdacqGHRaWkcaWGRbWaaS
% baaSqaaiaaigdaaeqaaOGaeyypa0JaaGOmaiaaicdacqGHsgIRcaWG
% RbWaaSbaaSqaaiaaigdaaeqaaOGaeyypa0JaaGynaaaa!3A5E!
\[
44 + k_2  = 200 \to k_2  = 156
\]
giving the control law as:
\[
u = r - \left[ {\begin{array}{*{20}c}
   {5} & {156}  \\
\end{array}} \right]{\bf{x}}
\]

% subsection example_2 (end)Given the system TF:

% section state_feedback_in_the_case_of_the_control_canonical_form (end)

\section*{A Transfer Function Model of State Feedback} % (fold)
\label{sec:a_transfer_function_model_of_state_feedback}

\input{frag9}
\ifslidesonly
\begin{slide}
	\heading{Transfer Function Model of State Feedback (1)}
   \input{frag9}
\end{slide}
\fi

\input{frag10}
\ifslidesonly
\begin{slide}
	\heading{Transfer Function Model of State Feedback (2)}
   \input{frag10}
\end{slide}
\fi

% section a_transfer_function_model_of_state_feedback (end)

\section*{Ackermann's Formula} % (fold)
\label{sec:ackermann_s_formula}

\subsection*{State Feedback Design for any Form of State Space Model} % (fold)
As stated previously, the derivation of the feedback law is tedious for systems of order  $n>3$  except in the case of the controller canonical form. One approach to the problem is to transform the given model to controller canonical form, derive the control law in terms of these states and then transform back to the original system. Ackermann derived the following formula by this method.
\ifslidesonly
\begin{slide}
   \heading{State Feedback Design for any Form of State Space Model}
\begin{itemize}
	\item As stated previously, the derivation of the feedback law is tedious for systems of order  $n>3$  except in the case of the controller canonical form.
	\item One approach to the problem is to transform the given model to controller canonical form, derive the control law in terms of these states and then transform back to the original system.
	\item Ackermann derived the following formula by this method.
\end{itemize}
\end{slide}
\fi

\subsection*{The formula} % (fold)
\label{sub:the_formula}

\input{frag11}
\ifslidesonly
\begin{slide}
	\heading{Ackermann's Formula}
   \input{frag11}
\end{slide}
\fi

\input{frag12}
\ifslidesonly
\begin{slide}
	\heading{Explanation of the Terms}
   \input{frag12}
\end{slide}
\fi

\input{frag13}
\ifslidesonly
\begin{slide}
	\heading{Caveats}
   \input{frag13}
\end{slide}
\fi

\subsection*{Matlab Function}
\input{frag14}
\ifslidesonly
\begin{slide}
	\heading{Matlab Function}
   \input{frag14}
\end{slide}
\fi

% subsection the_formula (end)

\subsection*{Example 3} % (fold)
\label{ssec:example_2}

\textbf{Problem}:
Given:
\[
{\bf{A}} = \left[ {\begin{array}{*{20}c}
   1 & 2  \\
   { - 1} & 1  \\
\end{array}} \right]\quad {\rm{and}}\quad {\bf{B}} = \left[ {\begin{array}{*{20}c}
   1  \\
   { - 2}  \\
\end{array}} \right]
\]
find the feedback vector $\mathbf{K}$ to place the closed loop poles at $s = -1,\ -1$ using Ackermann's formula.

\textbf{SOLUTION}:
\[
\alpha_c(s) = (s + 1)(s + 1) = s^2 + 2s + 1
\]
therefore
\[
\alpha_c(s) = \mathbf{A}s^2 + 2\mathbf{A}s + \mathbf{I}
\]
\[
\alpha _c ({\bf{A}}) = \left[ {\begin{array}{*{20}c}
   { - 1} & 4  \\
   { - 2} & { - 1}  \\
\end{array}} \right] + 2\left[ {\begin{array}{*{20}c}
   1 & 2  \\
   { - 1} & 1  \\
\end{array}} \right] + \left[ {\begin{array}{*{20}c}
   1 & 0  \\
   0 & 1  \\
\end{array}} \right] = \left[ {\begin{array}{*{20}c}
   2 & 8  \\
   { - 4} & 2  \\
\end{array}} \right]
\]

\[
{\bf{AB}} = \left[ {\begin{array}{*{20}c}
   { - 3}  \\
   { - 3}  \\
\end{array}} \right];\quad \mathcal{C} = \left[ {{\bf{A}} \vdots {\bf{AB}}} \right] = \left[ {\begin{array}{*{20}c}
   1 & { - 3}  \\
   { - 2} & { - 3}  \\
\end{array}} \right]
\]

\begin{eqnarray*}
	{\bf{K}} & = & \left[ {\begin{array}{*{20}c}
	   0 &  \cdots  & 0 & 1  \\
	\end{array}} \right]\mathcal{C}^{ - 1} \alpha _c ({\bf{A}}) \\
	& = & \left[ {\begin{array}{*{20}c}
	   0 & 1  \\
	\end{array}} \right]\left[ {\begin{array}{*{20}c}
	   1 & { - 3}  \\
	   { - 2} & { - 3}  \\
	\end{array}} \right]^{ - 1} \left[ {\begin{array}{*{20}c}
	   2 & 8  \\
	   { - 4} & 2  \\
	\end{array}} \right] \\
	& = & \left[ {\begin{array}{*{20}c}
	   0 & 1  \\
	\end{array}} \right]\frac{\left[ {\begin{array}{*{20}c}
	   { -3 } & { 3 }  \\
	   { 2 } & { 1 }  \\
	\end{array}} \right]}{-3-(+6)} \left[ {\begin{array}{*{20}c}
	   2 & 8  \\
	   { - 4} & 2  \\
	\end{array}} \right] \\
	& = & \left[ {\begin{array}{*{20}c}
	   0 & 1  \\
	\end{array}} \right]\frac{1}{-9}\left[ \begin{array}{*{20}c}
	   { -18 } & { -18 }  \\
	   { 0 } & { 18 }  \\
	\end{array} \right] \\
	& = & \left[ {\begin{array}{*{20}c}
	   0 & 1  \\
	\end{array}} \right]\left[ \begin{array}{*{20}c}
	   { 2 } & { 2 }  \\
	   { 0 } & { -2 }  \\
	\end{array} \right] \\
	& = & \left[ {\begin{array}{*{20}c}
	   0 & -2  \\
	\end{array}} \right]
\end{eqnarray*}
% subsection example_3 (end)

\subsection*{Solution in Matlab} % (fold)
\label{sub:solution_in_matlab}
\begin{verbatim}
% Using the formula
A = [1 2; -1 1]; B = [1; -2];
alpha_c = A * A + 2 * A + eye(2);
K = [0 1] * inv(ctrb(A, B)) * alpha_c

% Using the function Acker
P = [-1, -1] % vector of desired pole locations
Ka = acker(A, B, P)
\end{verbatim}
\ifslidesonly
\begin{slide}
   \heading{Solution in Matlab(1)}
\begin{verbatim}
A = [1 2; -1 1]; B = [1; -2];
alpha_c = A * A + 2 * A + eye(2);
K = [0 1] * inv(ctrb(A, B)) * alpha_c
\end{verbatim}
\end{slide}
\begin{slide}
   \heading{Solution in Matlab(2)}
\begin{verbatim}
% Using the function Acker
P = [-1, -1] % vector of desired pole locations
Ka = acker(A, B, P)
\end{verbatim}
\end{slide}
\fi
% subsection solution_in_matlab (end)

% section ackermann_s_formula (end)

\section*{Effect of State Feedback on the Closed Loop Zeros} % (fold)
\label{sec:effect_of_state_feedback_on_the_closed_loop_zeros}

\input{frag15}
\ifslidesonly
\begin{slide}
	\heading{Closed-Loop System}
   \input{frag15}
\end{slide}
\fi

\input{frag16}
\ifslidesonly
\begin{slide}
	\heading{Closed-loop Transfer Function}
   \input{frag16}
\end{slide}
\fi

\input{frag17}
\ifslidesonly
\begin{slide}
	\heading{Effect of State-Feedback on Closed-Loop Zeros}
   \input{frag17}
\end{slide}
\fi

% section effect_of_state_feedback_on_the_closed_loop_zeros (end)

\section*{Effect of Zero Locations on the Feedback Gains} % (fold)
\label{sec:effect_of_zero_locations_on_the_feedback_gains}

\input{frag18}
\ifslidesonly
\begin{slide}
	\heading{Effect of Zero Locations on the Feedback Gains -- Example}
   \input{frag18}
\end{slide}
\fi

\input{frag19}
\ifslidesonly
\begin{slide}
	\heading{Solution}
   \input{frag19}
\end{slide}
\fi

\ifslidesonly
\begin{slide}
   \heading{Comments on Solution}
\[
	k_1 =  \frac{p-p_c}{p-z}
\]
\input{frag20}
\end{slide}
\fi
\input{frag20}

\input{frag21}
\ifslidesonly
\begin{slide}
	\heading{Effect of Zero Locations on the Feedback Gains}
   \input{frag21}
\end{slide}
\fi

% section effect_of_zero_locations_on_the_feedback_gains (end)

\section*{Pole Selection for Good Design} % (fold)
\label{sec:pole_selection_for_good_design}

\input{frag22}
\ifslidesonly
\begin{slide}
	\heading{Pole Selection for Good Design (1)}
   \input{frag22}
\end{slide}
\fi

\input{frag23}
\ifslidesonly
\begin{slide}
	\heading{Pole Selection for Good Design (2)}
   \input{frag23}
\end{slide}
\fi

\subsubsection*{Comments} % (fold)
\label{ssub:comments}

\input{frag24}
\ifslidesonly
\begin{slide}
	\heading{Comments}
   \input{frag24}
\end{slide}
\fi

\input{frag25}
\ifslidesonly
\begin{slide}
	\heading{Optimal State Feedback}
   \input{frag25}
\end{slide}
\fi

% subsubsection comments (end)

% section pole_selection_for_good_design (end)

\ifslidesonly
\begin{slide}
   \heading{Summary of this Lecture}
   \begin{itemize}
   	\item Finding the control law
   	\item State feedback for controller canonical form
   	\item Transfer function model
   	\item Ackermann's formula
   	\item Effect of state feedback on closed-loop zeros
   	\item Effect of plant zeros on the feedback gains
   	\item Pole-selection for good design
   \end{itemize}
\end{slide}
\fi

%----------------------------------------------------------------
% The end of notes
% ----------------------------------------------------------------
\endinput

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:


\section{Introduction}%
\label{sec:l2hmc_intro}
We describe a new technique for performing Hamiltonian Monte-Carlo
(HMC) simulations called: `Learning to Hamiltonian Monte Carlo'
(L2HMC)~\cite{2017arXiv171109268L} which expands upon the traditional HMC by
using a generalized version of the leapfrog integrator that is parameterized by
weights in a neural network.
%
Hamiltonian Monte-Carlo improves upon the random-walk guess and check strategy
of generic MCMC by integrating Hamilton's equations along approximate
iso-probability contours of phase space.
%
In doing so, we are able to explore
the phase space more efficiently by taking larger steps between proposed
configurations while maintaining high acceptance rates.
%
In order to demonstrate the usefulness of this new approach, we use various
metrics for measuring the performance of the trained (L2HMC) sampler vs.\ the
generic HMC sampler.

First, we will look at applying this algorithm to a two-dimensional Gaussian
Mixture Model (GMM).
%
The GMM is a notoriously difficult distribution for HMC due to the vanishingly
small likelihood of the leapfrog integrator traversing the space between the
two modes.
%
Conversely, we see that through the use of a carefully chosen training
procedure, the trained L2HMC sampler is able to successfully discover the
existence of both modes, and mixes (`tunnels') between the two with ease.
%
Additionally, we will observe that the trained L2HMC sampler mixes much faster
than the generic HMC sampler, as evidenced through their respective
autocorrelation spectra.

This ability to reduce autocorrelations is an important metric for measuring
the efficiency of a general MCMC algorithm, and is of great importance for
simulations in lattice gauge theory and lattice QCD.\@
%
Following this, we introduce the two-dimensional $U(1)$ lattice gauge theory
and describe important modifications to the algorithm that are of particular
relevance for lattice models.
%
Ongoing issues and potential areas for improvement are also discussed,
particularly within the context of high-performance computing and long-term
goals of the lattice QCD community.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%