\section{Pattern Matching by Example} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\label{sec:bg}

Pattern matching is an abstraction mechanism that provides syntax very close to
mathematical notations. It allows the user tersely describe a (possibly
infinite) set of values accepted by the pattern. A \emph{pattern} represents a
predicate on values, and is usually more concise and readable than equivalent
imperative code. An interesting peculiarity about patterns is that a given
pattern is rarely repeated in several places in code. This context sensitivity
makes them unattractive for code reuse through a dedicated predicate, since the
predicate will typically only be used once.

Pattern matching is closely related to \emph{algebraic data types} and
\emph{equational reasoning}. In ML and Haskell an \emph{Algebraic Data Type} is
a data type each of whose values are picked from a disjoint sum of (possibly
recursive) data types, called \emph{variants}. Each variant is marked with a
unique symbolic constant called a \emph{constructor}. Constructors provide a
convenient way of creating a value of its variant type as well as discriminating
among variants through pattern matching. Consider a simple expression language:
\begin{center}
$exp$ \is{} $val$ \Alt{} $exp+exp$ \Alt{} $exp-exp$ \Alt{} $exp*exp$ \Alt{} $exp/exp$
\end{center}

\noindent
This can be represented in OCaml as an algebraic data type:

\begin{lstlisting}[language=Caml,keepspaces,columns=flexible]
type expr = Value  of int
          | Plus   of expr * expr
          | Minus  of expr * expr
          | Times  of expr * expr
          | Divide of expr * expr ;;
\end{lstlisting}

\noindent
The set of values described by such an algebraic data type is defined
inductively as the least set closed under constructor functions of its variants.
A simple evaluator of such expressions can be defined:

\begin{lstlisting}[language=Caml,keepspaces,columns=flexible]
let rec eval e = match e with
            Value  v     -> v
          | Plus  (a, b) -> (eval a) + (eval b)
          | Minus (a, b) -> (eval a) - (eval b)
          | Times (a, b) -> (eval a) * (eval b)
          | Divide(a, b) -> (eval a) / (eval b) ;;
\end{lstlisting}

\noindent
There are two critical differences between algebraic data types and classes in
object-oriented languages: definition of an algebraic data type is \emph{closed},
while its variants are \emph{disjoint}. Closedness means that once we have listed
all the variants a given algebraic data type may have we cannot extend it with
new variants without modifying its definition. Disjointedness means that a value
of an algebraic data type belongs to exactly one of its variants. Neither is
the case in object-oriented languages. Classes are \emph{extensible},
since new variants can be added through subclassing, as well as
\emph{hierarchical}, since variants are not necessarily disjoint and can form
subtyping relation between themselves. The above algebraic data type can be
encoded in C++ as following:

\begin{lstlisting}[columns=flexible]
struct Expr { virtual @$\sim$@Expr(){}};
struct Value  : Expr { int value; };
struct Plus   : Expr { Expr* e1; Expr* e2; }; ...
struct Divide : Expr { Expr* e1; Expr* e2; };
\end{lstlisting}

\noindent
while a similar evaluator in our SELL is almost as terse as OCaml code:

\begin{lstlisting}[columns=flexible]
int eval(const Expr* e)
{
  Match(e) {
    Case(Value,  n)    return n;
    Case(Plus,   a, b) return eval(a) + eval(b);
    Case(Minus,  a, b) return eval(a) - eval(b);
    Case(Times,  a, b) return eval(a) * eval(b);
    Case(Divide, a, b) return eval(a) / eval(b);
  } EndMatch
}
\end{lstlisting}

\noindent
An expression \code{e} used as an argument of a \emph{matching expression} or a
\emph{match statement}, is usually referred to as \emph{subject}, and its type
as a \emph{subject type}. We will also refer to the type of the value expected
by a given pattern as a \emph{target type}.

\emph{Polymorphic variants} in OCaml\cite{garrigue-98} and \emph{open data
types} in Haskell\cite{LohHinze2006} allow addition of new variants later.
These extensions are simpler however as they maintain the
disjointedness property: open data types do not introduce any subtyping relation,
while the subtyping relation on polymorphic variants relates anonymous
combinations of variants and not the variants themselves. In contrast,
the \emph{nominative subtyping} of object-oriented languages does not maintain
the disjointness property, making objects effectively belong to multiple classes.

Closedness of algebraic data types is particularly useful for reasoning about
programs by case analysis and allows the compiler to perform an automatic
\emph{incompleteness} check -- test of whether a given match statement
covers all possible cases. Similar reasoning about programs involving extensible
data types is more involved as we are dealing with potentially open set of
variants. \emph{Completeness} check in such scenario reduces to checking presence
of a case that handles the static type of the subject. Absence of such a case,
however, does not necessarily imply incompleteness, only potential incompleteness,
as the answer will depend on the actual set of variants available at run-time.

A related notion of \emph{redundancy} checking arises from the
tradition of using \emph{first-fit} strategy in pattern matching. It warns the
user of any \emph{case clause} inside a match statement that will
never be entered because of a preceding one being more general. Object-oriented
languages, typically prefer \emph{best-fit} strategy because it is not prone
to errors where semantics of a statement might change depending on the ordering
of preceding definitions.

The patterns used in functions \codeocaml{eval} and \codehaskell{eitherLift} to
identify and decompose a concrete variant of an algebraic data types are
generally called \emph{tree patterns} or \emph{constructor patterns}. Their
analog in object-oriented languages is often referred to as \emph{type pattern}
since it may involve type testing and type casting. Special cases of tree patterns
are \emph{list patterns} and \emph{tuple patterns} that make use of special list
and tuple constructors \codehaskell{:} and \codehaskell{(,,...,)}.

Pattern matching can also be applied to built-in types.
In Haskell, a factorial can be defined by equations:

\begin{lstlisting}[language=Haskell]
factorial 0 = 1
factorial n = n * factorial (n-1)
\end{lstlisting}

\noindent
Here 0 in the left hand side of the first \emph{equation} is an example of a
\emph{value pattern} that will only match when the actual argument passed is 0.
The \emph{variable pattern} \codehaskell{n} in the left hand side of the second
equation will match any value, \emph{binding} variable \codehaskell{n} to it in
the right hand side of equation. The \emph{wildcard pattern} \codehaskell{_}
will match any value, neither binding it to a variable nor even obtaining it.
Value patterns, variable patterns and wildcard patterns are
generally called \emph{primitive patterns}. Patterns like variable and wildcard
patterns that never fail to match are called \emph{irrefutable}, in contrast to
\emph{refutable} patterns like value patterns, which may fail to match.

In Haskell 98\cite{Haskell98Book} factorial could alternatively be defined as:

\begin{lstlisting}[language=Haskell]
factorial 0 = 1
factorial (n+1) = (n+1) * factorial n
\end{lstlisting}

\noindent
The \codehaskell{(n+1)} pattern in the left hand side of equation is an example of
\emph{n+k pattern}. According to its informal semantics ``Matching an $n+k$
pattern (where $n$ is a variable and $k$ is a positive integer literal) against
a value $v$ succeeds if $v \ge k$, resulting in the binding of $n$ to $v-k$, and
fails otherwise''\cite{haskell98}. n+k patterns were introduced into Haskell to
let users express inductive functions on natural numbers in much the same way as
functions defined through case analysis on algebraic data types. Besides
succinct notation, this could facilitate automatic proof of
termination of such functions by the compiler.
However, numerous debates over semantics and usefulness of n+k patterns
resulted in their removal from the Haskell
2010 standard\cite{haskell2010}. Generalization of n+k patterns, called
\emph{application patterns} has been studied by Oosterhof\cite{OosterhofThesis}.
Application patterns essentially treat n+k patterns as equations, while matching
against them attempts to solve or validate the equation.

Our library language supports generalized n+k patterns in a different form
(\textsection\ref{sec:slv}). We do not restrict ourselves with equational view
of the n+k patterns, but allow the user to specify suitable semantics.
In our library, we can define fast computation of Fibonacci numbers like this:

\begin{lstlisting}[keepspaces]
int fib(int n)
{
    variable<int> m;
    Match(n) {
      When(1)         return 1;
      When(2)         return 1;
      When(2*m)     return sqr(fib(m+1)) - sqr(fib(m-1));
      When(2*m+1) return sqr(fib(m+1)) + sqr(fib(m));
    } EndMatch
}
\end{lstlisting}

\noindent
A \emph{guard}
is a predicate attached to a pattern that may make use of the variables bound in
it. The result of its evaluation will determine whether the case clause and the
body associated with it will be \emph{accepted} or \emph{rejected}.

In OCaml, we can define rules for rewriting terms in
our $exp$ language with the help of guards:

\begin{lstlisting}[language=Caml,keepspaces,columns=flexible]
let collect e = match e with
      Plus(Times(e1,e2), Times(e3,e4)) when e1 = e3 -> Times(e1, Plus(e2,e4))
    | Plus(Times(e1,e2), Times(e3,e4)) when e2 = e4 -> Times(Plus(e1,e3), e4)
    | e -> e ;;
\end{lstlisting}

\noindent
Attempting to write the first case clause as \codeocaml{Plus(Times(e,e2),
Times(e,e4))} would have been relying on \emph{equivalence patterns}.
Neither OCaml nor Haskell support such patterns, but
Miranda\cite{Miranda85} and Tom\cite{Moreau:2003} do.

The example above illustrates yet another common pattern-matching facility --
\emph{nesting of patterns}. Constructor patterns composed
solely of (distinct) variables are called \emph{simple pattern}s
and others are called \emph{nested pattern}s.
Nested checks are hard to handle using the visitor design pattern, as they are often
too context-dependent to extract them into a dedicated visitor.
In such cases, users typically prefer \emph{type tests} and \emph{type
casts}. Our library handles such cases using nested patterns:
\begin{lstlisting}
const Expr* collect(const Expr* e)
{
  const Expr *e1, *e2, *e3, *e4;
  Match(e) {
    When(match<Plus>(match<Times>(e1,e2),match<Times>(e3 |= e1==e3,e4)))
        return new Times(e1, new Plus(e2,e4));
    When(match<Plus>(match<Times>(e1,e2),match<Times>(e3,e4 |= e2==e4)))
        return new Times(new Plus(e1,e3), e4);
    When()
        return e;
  } EndMatch
}
\end{lstlisting}

\noindent
Decomposing algebraic data types through pattern matching has an important
drawback that was originally spotted by Wadler\cite{Wadler87}: they expose
concrete representation of an abstract data type, which conflicts with the
principle of \emph{data abstraction}. To overcome the problem he proposed the
notion of \emph{views} that represent conversions between different
representations that are implicitly applied during pattern matching. As an
example, imagine polar and cartesian representations of complex numbers. A user
might choose polar representation as a concrete representation for the abstract
data type \codeocaml{complex}, treating cartesian representation as view or vice
versa:\footnote{We use the syntax from Wadler's original paper for this example}

\begin{lstlisting}[language=Haskell,columns=flexible]
complex ::= Pole real real
view complex ::= Cart real real
  in  (Pole r t) = Cart (r * cos t) (r * sin t)
  out (Cart x y) = Pole (sqrt(x^2 + y^2)) (atan2 x y)
\end{lstlisting}

\noindent
The operations then might be implemented in whatever representation is the most
suitable, while the compiler will implicitly convert representation if needed:

\begin{lstlisting}[language=Haskell,columns=flexible]
  add  (Cart x1 y1) (Cart x2 y2) = Cart (x1 + x2) (y1 + y2)
  mult (Pole r1 t1) (Pole r2 t2) = Pole (r1 * r2) (t1 + t2)
\end{lstlisting}

\noindent
The idea of views were later adopted in various forms in several languages:
Haskell\cite{views96}, Standard ML\cite{views98}, Scala (in the form of
\emph{extractors}\cite{EmirThesis}) and F$\sharp$ (under the name of
\emph{active patterns}\cite{Syme07}). We demonstrate our support of views in
\textsection\ref{sec:view}.


\section{Parameter Estimation}
Parameter estimation involves a process of obtaining a parameter set $\boldsymbol{\chi}$ of a CDM model that minimizes the difference between $\boldsymbol{\sigma}^M$-$\boldsymbol{\epsilon}^M$ and $\left<\boldsymbol{\sigma}\right>$-$\left<\boldsymbol{\epsilon}\right>$ for all load paths. Herein, the parameter estimation was conducted using calibration algorithms, a subset of optimization which attempts to minimize a least-squares objective function \cite{matott_ostrich:_2008}. Optimization algorithms are often described as either deterministic, which find the local optimum precisely, or heuristic, which find the global optimum approximately.

%Optimization algorithms are often described as either deterministic (local search) or heuristic (global search). Deterministic optimization algorithms primarily focus on searching for the optima within the local parameter space by iteratively converging towards a solution. Heuristic optimization algorithms explore the entire parameter space approximately and provide an estimate of the global optima. Heuristic techniques are useful for highly non-linear problems, where there are numerous local optima within the prescribed parameter space. When searching the global parameter-space deterministically becomes too computationally demanding, heuristic methods are used, at the cost of completeness and accuracy. A compromise between speed and accuracy can be obtain by strategically using both types of algorithms.

A combination of two optimization algorithms is used to assess the optimal parameter set. An initial heuristic algorithm is applied to search for the approximate global optima, followed by a deterministic algorithm as a local refinement of the optimal parameter set. Particle Swarm Optimization (PSO) is used for the global heuristic search, whereas the Levenberg-Marquardt Algorithm (LMA) is used for the local deterministic search.

%The PSO algorithm was developed by  as a byproduct of modeling the cooperative-competitive nature of social behaviour in birds as they flocked searching for food. The PSO algorithm, in a conceptual sense, consists of a series of 'particles' (birds) which 'swarm' through the entire parameter space (sky) searching for the global optima (food) using a combination of individual 'particle' knowledge and global 'swarm' (flock) knowledge.

%The Levenburg-Marquardt Algorithm (LMA) was proposed by \citet{marquardt_algorithm_1963} which builds off of the work of \citet{levenberg_method_1944}. This calibration algorithm combines a quasi-Newton approach with a conjugate gradient technique in order to efficiently minimize non-linear least-squares problems.

The parameter estimation works by iteratively running a single element CDM model, subject to boundary conditions provided by the homogenized DEM simulations, with successive parameter sets that intelligently adapt in order to converge to the DEM data. The approach used here is similar is aim to the Least Squares method was briefly described in \citet{wellmann_homogenization_2008}.


% Note that the text in the [] brackets is the one that will
% appear in the table of contents, whilst the text in the {}
% brackets will appear in the main thesis.

%% APPENDIX HEADER ////////////////////////////////////////////////////////////////////////////////////
\chapter{Elementary matrices of the equation of motion.}
\label{app:matrices}
%% APPENDIX CONTENT ///////////////////////////////////////////////////////////////////////////////////

\section{Mass and stiffness matrices}
The formulae of matrices for 3D elements are:
\begin{eqnarray}
	\textbf{M}_{dd}^e & = & \int_{V_e}\textbf{N}^T\rho \textbf{N} dV_e,\\
	\textbf{K}_{dd}^e & = & \int_{V_e}{\textbf{B}_d^e}^T\textbf{c}\textbf{B}_d^edV_e,
\end{eqnarray}
where \textbf{c} is the stiffness tensor, \(\rho\) is mass density, and \(V_e\) is the element volume.

In the case of the 2D elements, matrices are defined as:
\begin{eqnarray}
	\textbf{M}_{dd}^e & = &
	\left [
	\begin{array}{cc}
		\textbf{M}^e & 0\\
		0 & \textbf{J}^e
	\end{array}
	\right] =
	\int_{\Omega_e}\textbf{N}^T\rho
	\left [
	\begin{array}{ccccc}
		h & 0 & 0 & 0 & 0 \\
		& h & 0 & 0 & 0 \\
		&  & h & 0 & 0\\
		&  &  & \frac{h^3}{12} & 0\\
		Sym. &  &  &  & \frac{h^3}{12}
	\end{array} \right]
	\textbf{N} d\Omega_e,\\
	\textbf{K}_{dd}^e & = & \int_{\Omega_e}{\textbf{B}_b^e}^T
	\left[
	\begin{array}{cc}
		\textbf{A} & \textbf{B}\\
		\textbf{B} & \textbf{D}
	\end{array} \right]
	\textbf{B}_b^ed \Omega_e+\int_{\Omega_e}{\textbf{B}_s^e}^T\hat{\textbf{A}}\textbf{B}_s^ed \Omega_e,
\end{eqnarray}
where \(h=h_t+h_b\) is the element thickness, while \(h_{t(b)}\) is the distance between mid-plane and top(bottom) surface of the element, and \(\Omega_e\) is the element area:
\begin{eqnarray}
	\textbf{A} & = & \textbf{c}_{ij}\,(h_t-h_b),\qquad i,j=1,2,6\nonumber\\
	\textbf{B} & = & 1/2\, \textbf{c}_{ij}\,(h_t^2-h_b^2),\qquad i,j=1,2,6\nonumber\\
	\textbf{D} & = & 1/3\, \textbf{c}_{ij}\,(h_t^3-h_b^3),\qquad i,j=1,2,6\nonumber\\
	\hat{\textbf{A}} & = & 5/4\, \textbf{c}_{ij}\,\left[h_t-h_b-4/3\left(h_t^3-h_b^3\right)/h^2\right],\qquad i,j=4,5.
\end{eqnarray}
\section{\ac{pzt} matrices}

The dielectric conductivity matrix \(\textbf{K}_{\phi \phi}^e\) and piezoelectric coupling matrix \(\textbf{K}_{u \phi}^e\) are defined:
\begin{eqnarray}
	\textbf{K}_{d\phi}^e & = & \int_{V_e}{\textbf{B}_d^e}^T\textbf{e}^T \textbf{B}_{\phi}^ed V_e,\\
	\textbf{K}_{\phi \phi}^e & = & -\int_{V_e}{\textbf{B}_{\phi}^e}^T
	{\textbf{\(\epsilon\)}^S}^T \textbf{B}_{\phi}^edV_e.
\end{eqnarray}


\chapter{Introduction}
\label{chap:Introduction}
%
\section{System and the experimental setup}
\label{sec:intro1}
%
For this experiment, we are going to identify the parameters of the Rectilinear
Control System (Model 210).
The experimental control system is comprised of the three subsystems shown in
Figure \ref{fig:dynamicalsystem}. The first of these is the electromechanical
plant which consists of the spring/mass mechanism, its actuator and sensors.
The design features a brushless DC servomotor, high resolution encoders,
adjustable masses, and reconfigurable plant type.
%
\begin{figure}[ht]
\centering
\includegraphics[width=0.8\linewidth]{linlrge}
\caption{Dynamical system}
\label{fig:dynamicalsystem}
\end{figure}
\subsection{Parameters}
The system is configured with three bodies above the mass carriage suspension
is an anti-friction ball bearing type with approximately $\pm 3$
[\si{\centi\meter}] of available travel.
The linear drive is comprised of a gear rack suspended on an anti-friction
carriage and pinion (pitch diameter 7.62 [\si{\centi\meter}]) coupled to the
brushless servo-motor shaft.
Optical encoders measure the mass carriage positions - also via a rack and
pinion with a pinion pitch about 3.18 [\si{\centi\meter}].
The bodies are connected by known stiffness springs, and a spring connects the
third mass to the frame. Instead, the first body is rigidly connected to a
pinion gear with a live-powered motor with a PC interface.
The position of each body is provided by an encoder. The position zeros are at
the equilibrium positions of the springs.
For the springs we use the nominal values provided:
%
\begin{itemize}
	\item $k_1 = k_2 = 800$ [\si{\newton\per\meter}] between $m_1$ and $m_2$, $m_2$
and $m_3$;
	\item $k_3 = 400$ [\si{\newton\per\meter}] between $m_3$ and the ground.
\end{itemize}
%
The shifts $x_1$, $x_2$, $x_3$ are provided in encoder counts, where the
relationship (\ref{eq:encodercounts}) between the measured counts and the
displacement was used.
Where $r_{e}$ is the radius of the encoder and $2\pi r_{e} = 0.0706$
[\si{\meter}]; 16000 is the number of counts per encoder revolution.
%
\begin{equation}
	\Delta x = 2\pi \cdot r_{e} \cdot \frac{\Delta count}{16000}
	\label{eq:encodercounts}
\end{equation}
%
The input data are given by the voltage \si{\volt}.
The following relation between the applied voltage and the applied force holds:
$f = (k_a \cdot k_t \cdot k_{mp}) \cdot v$.\\
Where:\begin{description}
	\item $k_a$ is the Servo Amp gain:
	\begin{equation*}
		k_a \approx 2 \quad [\text{\si{\ampere\per\volt}}]
	\end{equation*}
	\item $k_t$ is the Servo Motor Torque constant:
	\begin{equation*}
		k_t	\approx	0.1 \quad 	[\text{\si{\newton\meter\per\ampere}}]
	\end{equation*}
	\item $k_{mp}$ is the Motor Pinion pitch radius inverse:
	\begin{equation*}
		k_{mp} 	= 26.25 \quad	[\text{\si{\per\meter}}]
	\end{equation*}
\end{description}
%
\section{The dynamical model}
\label{sec:dynamicalmodel}
%
\subsection{Assumption}
\label{subsec:assumption}
The system described in the previous chapter is modelled as a linear system and
for this reason some simplifications are made.
It is considered that all the bodies move on the same axis, assuming therefore
that the rack meshed by the pinion plots the force on this axis, so that a
straight motion is assumed.
In the model there are only viscous frictions.
The block containing the engine with the attachment unit and rack is considered
rigidly connected to the mass m, according to the equation:
\begin{equation}
	\label{eq:reducedinertia}
	\begin{cases}
		m_{1} &=  m_{11} + \frac{J_{\text{motor}}}{r^2}\\
		c_{1} &=  c_{11} + \frac{c_{\text{motor}}}{r^2}
	\end{cases}
\end{equation}
In equation \eqref{eq:reducedinertia}: $r$ is the radius of the pinion-rack
coupling, $J_{\text{motor}}$ the inertia of the motor, $c_{\text{motor}}$ the
rotational damping.
While $m_{11}$ and $c_{11}$ are respectively the mass and damping of the first body.
%
\subsection{Equation of motion}
\label{subsec:equationofomotion}
We describe the equations of motion for each body according to the embodiments
reported in \eqref{eq:equationmotion}, the schematic representation is
observable in the figure \ref{fig:modelscheme}
%
\begin{equation}
	\label{eq:equationmotion}
	\begin{array}{l}
		m_1 \ddot{x}_{1} = k_1 (x_2 - x_1) - c_1 \dot{x}_{1} + g_{\text{v}} \cdot v	\\
		m_2 \ddot{x}_{2} = k_1 (x_1 - x_2) + k_2 (x_3 - x_2) - c_2 \dot{x}_{2} \\
		m_3 \ddot{x}_{3} = k_2 (x_2 - x_3) - c_3 \dot{x}_{3} - k_3 x_3	\\
	\end{array}
\end{equation}
%
The equation in matrix form is shown below (\ref{eq:matrixform}):
%
\begin{equation}
	\label{eq:matrixform}
	\begin{bmatrix}
		m_1	&  	0	&	0	\\
 		0	& 	m_2	&	0	\\
 		0	&  	0	&	m_3	\\
	\end{bmatrix}
	\ddot{x}+
	\begin{bmatrix}
		c_1	&	0	&	0	\\
  		0	&	c_2	&  	0	\\
  		0	&	0	& 	c_3	\\
	\end{bmatrix}
	\dot{x}+
	\begin{bmatrix*}[c]
		k_1		&	-k_1			&       0	\\
 		-k_1		& 	k_1 + k_2	&     -k_2	\\
  		0		&	-k_2			& k_2 + k_3	\\
	 \end{bmatrix*}
 	x=\begin{bmatrix}
 	g_{\text{v}}	\\
 	0 	\\
 	0
 	\end{bmatrix} \cdot v
\end{equation}
%
\begin{figure}[hb]
	\centering
    \resizebox{\linewidth}{!}{\input{sketchplant.tex}}
	\caption{model system}
	\label{fig:modelscheme}
\end{figure}


\subsection{Syntax}
The following table summarizes the various operators and expression syntax.
The arithmetic operators have the expected precedence, that is,
multiplication and division are evaluated before add and subtract.
Subexpressions surrounded by parentheses have highest precedence.

\begin{center}
\begin{tabular}{clll}
{\it Math} & & {\it Eigenmath} & {\it Comment} \\
\\
$a=b$ & & \verb$a == b$ & {\it test for equality} \\
\\
$-a$ & & {\tt -a} & {\it negation} \\
\\
$a+b$ & & {\tt a+b} & {\it addition} \\
\\
$a-b$ & & {\tt a-b} & {\it subtraction} \\
\\
$ab$ & & {\tt a b} & {\it multiplication, alternatively,} \verb$a*b$ \\
\\
$\displaystyle\frac{a}{b}$ & & {\tt a/b} & {\it division}\\
\\
$\displaystyle\frac{a}{bc}$ & & {\tt a/b/c} & {\it division operator is left-associative} \\
\\
$a^2$ & & {\tt a{\char94}2} & {\it power}\\
\\
$\sqrt{a}$ & & \verb$sqrt(a)$ & {\it square root, alternatively,} \verb$a^(1/2)$ \\
\\
$a(b+c)$ & & {\tt a (b+c)} & {\it note the space in between, alternatively,} \verb$a*(b+c)$ \\
\\
$f(a)$ & & {\tt f(a)} & {\it function} \\
\\
$\begin{pmatrix}a\\ b\\ c\end{pmatrix}$ & & {\tt (a,b,c)} & {\it vector} \\
\\
$\begin{pmatrix}a&b\\ c&d\end{pmatrix}$ & & {\tt ((a,b),(c,d))} & {\it matrix} \\
\\
$F^1{}_2$ & & {\tt F[1,2]} & {\it tensor component access} \\
\\
-- & & \verb$"hello, world"$ & {\it string literal} \\
\\
$\pi$ & & {\tt pi} & -- \\
\\
$e$ && {\tt exp(1)} & {\it natural number}
\end{tabular}
\end{center}


\documentclass{article}
\author{xuyou}
\title{Proof For Prop 1 and 2 in GraphWave}
\usepackage{xeCJK}
\usepackage{bm}
\begin{document}
\maketitle
\section{note for prop 1}
\subsection{equation 1}
\begin{center}
\[
\sum _ { m=1  }^N\left( \Psi _ { m a } ^ { ( s ) } \right) ^ { 2 }
=\Psi_{aa}^{(2s)}
\]
\end{center}
because left side equal \bm{$1_a^T$}$Ug(s\Lambda)g(s\Lambda)U^T$\bm{$1_a$}=
$\Psi_{aa}^{(2s)}$
\subsection{equation 2}
\begin{center}

\[
\sum_{m=1}^N \Psi_{ma}^{(s)}=1 \quad s\ for\ any\ value
\]
\end{center}
%\paragraph{}
because 1 is the eigenvalue of $\Psi^{(s)}$ for eigenvector \textbf{1}
\section{note for prop 2}
$\Psi^s$ eigenvector for eigenvalue 1
\begin{center}
$U_{.1}=\frac{1}{\sqrt{N}}$
\end{center}
so\begin{center}
$\left|\Psi_{aa}^{(s+1)}-\frac{1}{N}\right|=\left|\sum_{j=1}^Ne^{-\lambda_j(s+1)}U_{ja}^2-\frac{1}{N}\right|=\left| \sum _ { j = 2 } ^ { N } e ^ { - \lambda _ { j } ( s + 1 ) } U _ { j a } ^ { 2 } \right|\leq
\left| \sum _ { j = 2 } ^ { N }e^{\lambda_2s} e ^ { - \lambda _ { j } s  } U _ { j a } ^ { 2 } \right|=e ^ { - \lambda _ { 2 } } \left| \Psi _ { a a } ^ { ( s ) } - \frac { 1 } { N } \right|$

\end{center}
\end{document}


\documentclass[twoside]{MATH77}
\usepackage{multicol}
\usepackage[fleqn,reqno,centertags]{amsmath}
\usepackage{url}
\begin{document}

\begmath 10.0 Overview of Fourier Transforms and Spectral Analysis

\silentfootnote{$^\copyright$1997 Calif. Inst. of Technology, \thisyear \ Math \`a la Carte, Inc.}

\subsection{Introduction}

The subroutines in this chapter compute discrete Fourier transforms,
using the fast Fourier transform (FFT). Discrete Fourier transforms
can be used in turn to approximate Fourier coefficients or evaluate
truncated Fourier series, to approximate power spectra, to compute
convolutions and lagged products (as might be required in computing
correlations and in filtering), and in several other applications
where the speed of the FFT has been found of value, see, $e.g.$,
\cite{Cooley:1966:AFF}.

The purpose of this introductory section is to outline what is
available, to help the reader in the selection of the appropriate
subroutine, to outline how the discrete transform can be used for the
computation of convolutions and lagged products, to examine the
errors introduced in using the discrete transform as an approximation
to other transforms, and to suggest a computational procedure to
those with doubts on how to proceed.

Lower case English letters are used for functions of $t$, the
independent variable, and Greek letters for Fourier transform
functions, that are functions of $\omega $, with units of
radians/(units of $t)$.

\subsection{Subroutines Available}

The one dimensional discrete transform pairs available are indicated
below.  In ``SRFT1/DRFT1'' (for example) SRFT1 is the name for the
single precision version and DRFT1 is the name for the double
precision version. In all cases N $=2^M$ where M is a nonnegative
integer and W $=e^{2\pi i/N}=\cos 2\pi /{N} +i\ \sin 2\pi /N.$

SRFT1/{DRFT1} One dimensional real transform\vspace{-6pt}
\begin{equation}\label{O1}
x_j=\sum_{k=0}^{N-1}\xi _kW^{jk},\quad j=0,1,...,N-1\vspace{-6pt}
\end{equation}
\vspace{-15pt}
\begin{equation}\label{O2}
\xi _k=\frac 1N\sum_{j=0}^{N-1}x_jW^{-jk},\quad k=0,1...,N-1
\end{equation}
STCST/DTCST Trigonometric transform\vspace{-6pt}%
\begin{multline}\label{O3}
y_j=\frac 12\alpha _0+\,\sum_{k=1}^{(N/2)-1}\left[
\alpha _k\cos \frac{2\pi jk}N+\beta _k\sin \frac{2\pi jk}N\right]\\
+\frac 12\alpha _{N/2}(-1)^j,\quad j=0,1,...,N-1
\end{multline}
\vspace{-20pt}
\begin{equation}
\begin{split}\label{O4}
\alpha _k&=\frac 2N\sum_{j=0}^{N-1}y_j\cos \frac{2\pi jk}N,\quad
k=0,1,...,\frac N2\\
\beta _k&=\frac 2N\sum_{j=1}^{N-1}y_j\sin \frac{2\pi jk}N,\quad
k=1,2,...,\frac N2-1
\end{split}
\end{equation}

STCST/DTCST Cosine transform%
\begin{multline}\label{O5}
y_j=\frac 12\alpha _0+\sum_{k=1}^{N-1}\alpha _k\cos
\frac{\pi jk}N+\frac 12\alpha _N(-1)^j,\\
j=0,1,...,N
\end{multline}
\vspace{-15pt}
\begin{multline}\label{O6}
\alpha _k=\frac 2N\left[ \frac 12y_0+\sum_{j=1}^{N-1}y_j\cos
\frac{\pi jk} N+\frac 12y_N(-1)^k\right]\\
k=0,1,...,N
\end{multline}
\vspace{-15pt}

STCST/DTCST Sine transform
\begin{equation}\label{O7}
y_j=\sum_{k=1}^{N-1}\beta _k\sin \frac{\pi jk}N,\quad
j=1,2,...,N-1
\end{equation}
\vspace{-10pt}
\begin{equation}\label{O8}
\beta _k=\frac 2N\sum_{j=1}^{N-1}y_j\sin \frac{\pi jk}N,\quad
k=1,2,...,N-1
\end{equation}

SCFT/DCFT Complex transform
\begin{equation}\label{O9}
z_j=\sum_{k=0}^{N-1}\zeta _kW^{jk},\quad j=0,1,...,N-1
\end{equation}
\vspace{-10pt}
\begin{equation}\label{O10}
\zeta _k=\frac 1N \sum_{j=0}^{N-1}z_jW^{-jk},\quad k=0,1,...,N-1
\end{equation}

All variables in the above equations are real, except $W$, $\xi $,
$z$, and $ \zeta $ which are complex. For each of the transform pairs
given, either equation can be derived from the other -- there are no
approximations involved.

Taking real and imaginary parts of Eq.\,(2) and making comparisons with Eq.
(4), it is clear that if $x_j=y_j$, then 2 $\Re \xi _k=\alpha _k$ and
2 $\Im \xi _k=-\beta _k$. Thus the trigonometric transform and the real
transform are closely related. Since SRFT1 is slightly more efficient and
shorter than STCST, it is recommended unless one has a distinct preference
for the trigonometric form. If one has data that are even (or odd) then
one can save a factor of two in both storage and computation by using the
cosine (or sine) transform in STCST. One could use SCFT for real data by
setting $\Im z_j=0$, but since this requires twice the storage and twice
the work as SRFT1, SCFT is recommended only for complex data.

Subroutines STCST, SCFT, and SRFT (similar to SRFT1) can be used for
data in more than one dimension. As above, there is a connection
between the real transform and the trigonometric transform, but the
relations connecting the two are not as simple. Indexing of the
coefficients in SRFT is more complicated, and thus we recommend STCST
for multi-dimensional real data unless one prefers the form of the
solution provided by SRFT.

\subsection{Discrete Convolutions and Correlations}

Here we consider computing sums of the form
\begin{equation}\label{O11}
c_n=\sum_{j=0}^{N-1}a_jb_{n\pm j},\quad n=0,1,...,N-1
\end{equation}
Initially it is assumed that $a_j$ and $b_j$ are periodic with period N.
(That is, if $j$ is not in the range $0\leq j\leq {N-1}$, it is replaced by
its value mod N.) Thus $\alpha _k$ and $\beta _k$ can be defined by%
\begin{equation*}
a_j=\sum_{k=0}^{N-1}\alpha _kW^{jk},\quad \text{and}\quad
b_j=\sum_{k=0}^{N-1}\beta _kW^{jk}.
\end{equation*}
Substitution in Eq.\,(11) yields
\begin{equation}\label{O12}
\begin{split}
 c_n&=\sum_{j=0}^{N-1}\left[ \sum_{k^{\prime }=0}^{N-1}\alpha
_{k^{\prime }}W^{jk^{\prime }}\right] \left[ \sum_{k=0}^{N-1}\beta
_kW^{(n\pm j)k}\right] \vspace{4pt} \\ \displaystyle\phantom{c_n}%
&=\sum_{k^{\prime }=0}^{N-1}\sum_{k=0}^{N-1}\alpha _{k^{\prime }}\beta
_kW^{nk}\sum_{j=0}^{N-1}W^{j(k^{\prime }\pm k)}.
\end{split}
\end{equation}
From the easily verified fact that
\begin{equation}\label{O13}
\sum_{j=0}^{N-1}W^{j(k^{\prime }\pm k)}=
\begin{cases}
N & \text{if }k^{\prime }\equiv \mp k\text{ mod }N \\
0 & \text{if }k^{\prime }\not \equiv \mp k\text{ mod }N
\end{cases}
\end{equation}
and since $\alpha _k$ is periodic with period N there follows from Eq.\,(12)
\begin{equation}\label{O14}
\begin{array}{ll}
\displaystyle c_n=N\sum_{k=0}^{N-1}\alpha _{\mp k}\beta _kW^{nk},&%
n=0,1,...,N-1\rule[-20pt]{0pt}{20pt}\\
\displaystyle \phantom{c_n}=N\sum_{k=0}^{N-1}\gamma _kW^{nk},&%
\text{where }\gamma _k=\alpha _{\mp k}\beta _k.
\end{array}
\end{equation}

The $c_n$ can be computed most efficiently by the indirect route of
using the FFT to compute the $\alpha ^{\prime }$s and $\beta ^{\prime
}$s from the $a^{\prime }$s and $b^{\prime }s$, then computing
$\gamma _k$ as defined in Eq.\,(14), and finally using the FFT to
compute the $c^{\prime }$s from the $\gamma ^{\prime }s$. For real
data, SRFT1 is recommended for computing the transforms. Note that
with SRFT1 $\alpha _k$, $\beta _k$, and $\gamma _k$ are only computed
for $k=0,1,...,N/2$; for $k=(N/2)+1,...,N-1$, one must use the fact
that for real data $\alpha _{N-k}=\overline{\alpha }_k\
(\overline{z}=$ conjugate of $z).$

Ordinarily one has $a_j$ and $b_j$ defined for $j=0,1,...,J-1$,
assumed to be~0 for other values of $j$, and desires $c_n$ for
$n=0,1,...,L-1$, where $L\leq J$. The above procedure can be used to
get the first $L$ values of $c_n $ if one sets $N=$ smallest power of
$2>J+L$ and sets $a_j$ and $b_j=0$ for $j=J,J+1,...,N-1.$ (If $J+L$
is equal to or just slightly greater than a power of~2, it pays to
reduce $L$ so that $J+L$ is one less than a power of 2, and to
compute $c_n$ directly from Eq.\,(11) for values of $n=new\ L,...,$
desired $L-1$.) Direct computation of the $c_n$ for $n=0,1,...,L-1$
from Eq.  (11) requires $L(2J-L+1)/2$ multiplies and adds. Using
SRFT1 and the procedure above requires approximately $(9/4)N\ \log
_2N$ multiplies, $(33/8)N(\log _2N+1)$ adds, and a little additional
overhead. If $a_j=b_j$ then the counts using SRFT1 can be multiplied
by $2/3$. The fastest procedure clearly depends on the values of
$J,L$, and $N$. For $J<64$, or for values of $L$ small relative to
$J$, the direct method is fastest, see \cite{Cooley:1966:AFF}.

Note that no assumption need be made about $a_j$ and $b_j$; the only
errors introduced in the calculation of $c_n$ using SRFT1 are
round-off errors.

\subsection{Estimating Power Spectra}

To within a constant factor (different normalizations are used), an
estimate of the power at the $k^{th}$ frequency (defined in E below)
is given by $|\xi _k|^2=(\Re \xi _k)^2+(\Im \xi _k)^2$, where the
$\xi _k$ are obtained from SRFT1, Eq.\,(2) above. This estimate
suffers from the same type of errors discussed below for the case of
computing Fourier integrals.

\subsection{Replacement of Continuous Transforms with Discrete Ones}

To simplify the material that follows we consider only the case of
complex functions. Results for real data follow immediately by
considering real and imaginary parts of the variables and equations
below. Proofs for the transform pairs are given in many books on
Fourier transforms, although different scalings for $\varphi $ and
$\omega $ are used.  We have used definitions that maximize symmetry
while matching up with the form of the discrete transform provided by
the FFT.

For continuous data we have the following transform pairs.

Fourier Integral $\left( \int_{-\infty }^\infty |f(t)|\,dt\text{\quad
exists}\right)$%
\begin{equation}\label{O15}
f(t)=\int_{-\infty }^\infty \varphi (\omega )e^{2\pi i\omega
t}d\omega
\end{equation}
\vspace{-10pt}
\begin{equation}\label{O16}
\varphi (\omega )=\int_{-\infty }^\infty f(t)e^{-2\pi i\omega
t}dt
\end{equation}
Fourier Series $\left( s(t)=s(t+T)\right) $%
\begin{equation}\label{O17}
s(t)=\sum_{k=-\infty }^\infty \sigma _ke^{2\pi ikt/T}
\end{equation}
\vspace{-10pt}
\begin{equation}\label{O18}
\sigma _k=\frac 1T\int_{-T/2}^{T/2}s(t)e^{-2\pi ikt/T}dt
\end{equation}
The discrete transform Eq.\,(10) can be used to approximate Eq.\,(16)
if the infinite limits are replaced by finite limits and $f$ is
sampled at equally spaced points between the two limits. As an
example let the infinite limits be replaced by $-T/2$ and T$/2$, and
let
\begin{equation}\label{O19}
z_j=
\begin{cases}
\displaystyle f(-\frac T2+j\Delta t),&j=1,2,...,N-1\\
\displaystyle \frac 12\left[ f(-\frac T2)+f(\frac T2)\right] ,& j=0,
\end{cases}
\end{equation}
\vspace{-10pt}
\begin{equation}\label{O20}
\Delta t=\frac TN.
\end{equation}
This value of $z_0$ gives significantly better results than simply
setting $z_0=f(-T/2)$. With the assumption that the contribution to
the integral in Eq.\,(16) is negligible for $|t|>T/2$, the trapezoidal
rule gives
\begin{equation}\label{O21}
\begin{split}
\varphi (\omega )&\approx \Delta t\sum_{j=0}^{N-1}z_je^{-\pi
i\omega (-T+2j\Delta t)} \\
&\approx \frac TNe^{\pi i\omega T}\sum_{j=0}^{N-1}z_je^{-2\pi ij\omega T/N}.
\end{split}
\end{equation}
In order that Eq.\,(21) have the form of Eq.\,(10), the solution is
obtained for $\omega =\omega _k$, where
\begin{equation}\label{O22}
\omega _k=\frac kT,\quad k=0,1,...N-1.
\end{equation}
Then
\begin{equation}\label{O23}
\varphi (\omega _k)\approx \frac TNe^{\pi
ik}\sum_{j=0}^{N-1}z_jW^{-jk}\quad (W=e^{2\pi i/N}).\hspace{-10pt}
\end{equation}
Thus $\varphi (\omega _k)\approx Te^{\pi ik}\zeta _k$, $\zeta _k$
defined as in (10). The factor $e^{\pi ik}(=(-1)^k)$ is due to
shifting the lower limit of $-T/2$ on the integral to a lower limit
of~0 on the summation.

Note that the $k^{th}$ frequency, $2 \pi \omega _k$, depends only on
T, the length of the interval over which $f$ is sampled. Thus, from
Eq.\,(22), $2 \pi \omega _k \text{ radians/(units of }t) =k/T$
cycles/(units of $t)$. From Eqs.\,(20) and Eq.\,(21) it follows that
the largest frequency for which a result is obtained is $\omega
_{N-1}=(N-1)/(N\Delta T)$ cycles.  For real data, our approximation
is such that $\varphi (\omega _{N-k})$ is the conjugate of $\varphi
(\omega _{-k})$, and thus in this case the largest effective
frequency is \begin{equation}\label{O24} \omega _{N/2}=\frac
1{2\Delta t}\ \text{cycles}/\text{(units of } t\text{).}
\end{equation} This frequency is commonly called the {\em Nyquist
frequency.} Note that this frequency depends only on the sampling
interval $\Delta t.$

All that has been said above for Eq.\,(16) applies almost word for
word to Eq.\,(18), except that Eq.\,(18) does not require replacing
infinite limits by finite ones. Because of the factor $1/T$ appearing
in Eq.\,(18), in place of $\varphi (\omega _k)\approx Te^{\pi
ik}\zeta _k$, we have $\sigma _k\approx e^{\pi ik}\zeta _k.$

\subsection{Errors Introduced by Using the Discrete Transform}

The discrete Fourier transform when used as above is a crude
approximation to a continuous transform. Its primary virtue is the
speed with which it can be computed using the FFT ({\bf F}ast {\bf
F}ourier {\bf T}ransform). Many procedures have been suggested for
computing continuous transforms and power spectra, some of which
permit the use of the FFT and some which do not.  References in
Section H below give a sampling of what has been suggested.

Any computational procedure involves making assumptions about either
$f(t)$ or $\varphi (\omega )$ (or both), and any assumption about one
implies something about the other. Answers to questions such as the
following help one in understanding the implications of a
computational procedure.

(Q1) How are the true and computed transform related?

(Q2) What is the error in the computed transform?

(Q3) What assumptions (if any) are made or implied concerning the transform?

(Q4) What assumptions are made about the function at points where its value
is not used?

(Q5) At the points where the value of the function is used, how is the
function related to a function that would give the true transform at
selected values of $\omega ?$

These questions are considered below for the case of the discrete
transform. We begin by giving some results required in the analysis,
then consider the effect of limiting the sampling of $f$ to a finite
range, the effect of discrete sampling, a combination of the two, and
finally the effect on the discrete transform of filling in with zeros.

\subsubsection{Convolution Theorems}

For Fourier integrals we have
\begin{equation}\label{O25}
\varphi (\omega )=\varphi _1(\omega )\varphi _2(\omega )\qquad
\text{if and only if}
\end{equation}
\vspace{-10pt}
\begin{equation}\label{O26}
f(t)=\int_{-\infty }^\infty f_1(\tau )f_2(t-\tau )\,d\tau .
\end{equation}
This result can be derived from Eqs.\,(15) and (16), and is also true
if $\varphi $ and $f$ are interchanged throughout Eqs.\,(25) and (26).

In the case of Fourier series there are two convolution theorems. We
work with the three transform pairs $(a,\alpha )$, $(b,\beta )$, and
$(c,\gamma )$. If $c(t)=a(t)b(t)$, then
\begin{equation}\label{O27}
\gamma _n=\frac 1T\int_{-T/2}^{T/2}\sum_{k=-\infty }^\infty
\sum_{k^{\prime }=-\infty }^\infty \alpha _k\beta _{k^{\prime }}e^{2\pi
it(k+k^{\prime }-n)/T}dt.
\end{equation}
Interchanging integration and summation, the integral is~0 if $k^{\prime
}\neq n-k$, and is T if $k^{\prime }=n-k$. Thus
\begin{equation}\label{O28}
\gamma _n=\sum_{k=-\infty }^\infty \alpha _k\beta _{n-k}.
\end{equation}
The second convolution theorem starts with
\begin{multline}\label{O29}
c(t)=\frac 1T\int_{-T/2}^{T/2}a(\frac{1+\tau }2)b(\frac{1-\tau
}2)\,d\tau \vspace{4pt} \\
\hspace{-15pt}=\frac 1T\int_{-T/2}^{T/2}
\sum_{k=-\infty }^\infty \sum_{k^{\prime }=-\infty
}^\infty \!\!\!\alpha _k\beta _{k^{\prime }}e^{2\pi i[t(k+k^{\prime })+\tau
(k-k^{\prime })]/2T}d\tau .
\end{multline}
Moving the integral inside the summations, we get a nonzero result only
for $k^{\prime }=k$ in which case the integral is $\exp (2\pi itk/T)$.
Thus%
\begin{equation*}
c(t)=\sum_{k=-\infty }^\infty \alpha _k\beta _ke^{2\pi itk/T},\quad
\text{and so}
\end{equation*}
\begin{equation}\label{O30}
\gamma _k=\alpha _k\beta _k.
\end{equation}
The convolution theorem for the discrete Fourier transform has already been
given in Eqs.\,(11) and (14).

\subsubsection{Fourier Transform of a Step Function}

For Fourier integrals, let
\begin{equation}\label{O31}
r_T(t)=
\begin{cases}
1 & -\frac T2<t<\frac T2\\
0 & |t|>\frac T2
\end{cases}
\end{equation}
\begin{equation}\label{O32}
\begin{split}
\rho _T(\omega )&=\int_{-\infty }^\infty r_T(t)e^{-2\pi i\omega
t}dt\vspace{4pt} \\
\phantom{\rho _T(\omega )}&=\int_{-T/2}^{T/2}e^{-2\pi i\omega t}dt=
\frac{\sin \pi \omega T}{\pi \omega }.
\end{split}
\end{equation}
Similarly, one has the transform pair $(\frac 1{\pi t}\sin \pi t\Omega $, $r_\Omega
(\omega )).$

For Fourier series, we are interested in the case
\begin{equation}\label{O33}
\hat \rho _k^{(K)}=
\begin{cases}
1 & -K\leq k<K \\
0 & k\geq K\text{ or }k<-K
\end{cases}
\end{equation}
\vspace{-10pt}
\begin{equation}\label{O34}
\begin{split}
\displaystyle\hat r^{(K)}&=\sum_{k=-\infty }^\infty \hat \rho _k^{(K)}e^{2\pi
itk/T}=\sum_{k=-K}^{K-1}e^{2\pi itk/T}\vspace{4pt} \\
\displaystyle &=\dfrac{2i(\sin 2\pi Kt/T)}{e^{2\pi it/T}-1}\vspace{4pt}\\
\displaystyle &=\dfrac{(1+e^{-2\pi it/T})(\sin 2\pi Kt/T)}{\sin 2\pi t/T}.
\end{split}
\end{equation}
For the discrete Fourier transform, consider
\begin{equation}\label{O35}
\tilde r_{j+kN}^{(J)}=
\begin{cases}
1 & 0\leq j\leq J-1 \\
0 & J\leq j\leq N-1
\end{cases}
k=0,\pm 1,\pm 2,...
\end{equation}
\begin{equation*}
\tilde \rho _k^{(J)}=\frac 1N\sum_{j=0}^{N-1}\tilde r_j^{(J)}W^{-jk}=\frac
1N\sum_{j=0}^{J-1}W^{-jk}
\end{equation*}
\begin{equation}\label{O36}
\tilde \rho _k^{(J)}=
\begin{cases}
J/N & k=0,\pm N,\pm 2N,...\\
\dfrac{1-W^{-Jk}}{N(1-W^{-k})} & \text{otherwise.}
\end{cases}
\end{equation}

\subsubsection{Errors Due to a Finite Range}

Consider approximating $\varphi (\omega )$ in Eq.\,(16) with
\begin{equation}\label{O37}
\hat \varphi (\omega )=\int_{-T/2}^{T/2}f(t)e^{-2\pi i\omega t}dt.
\end{equation}
This is equivalent to answering (Q4) with $f(t)=0$ for $|t|>T/2.$

To answer (Q1), rewrite the above equation as follows
\begin{equation}\label{O38}
\hat \varphi (\omega )=\int_{-\infty }^\infty r_T(t)f(t)e^{-2\pi
i\omega t}dt,
\end{equation}
and using Eqs.\,(25) and (26) (with $f$ and $\varphi $ interchanged)
\begin{equation}\label{O39}
\hat \varphi (\omega )=\int_{-\infty }^\infty \varphi (\omega )
\frac{\sin \pi T(\omega -w)}{\pi (\omega -w)}\,dw.
\end{equation}
Thus the effect of a finite range is a loss of resolution due to the
smearing as indicated in Eq.\,(39).

One answer to (Q2) is to subtract both sides of Eq.\,(39) from $\varphi
(\omega )$. A more useful result is obtained by subtracting Eq.\,(37)
from Eq.\,(16).
\begin{equation}\label{O40}
\varphi (\omega )-\hat \varphi (\omega )=\int_{T/2}^\infty \left[
f(t)e^{-2\pi i\omega t}+f(-t)e^{2\pi i\omega t}\right] \,dt.
\end{equation}
Thus a bound on the error is given by
\begin{equation}\label{O41}
|\varphi (\omega )-\hat \varphi (\omega )|\leq \int_{T/2}^\infty
\left[ |f(t)|+|f(-t)|\right] \,dt.
\end{equation}
If $\int_{-\infty }^\infty |f^{(j)}(t)|\,dt$ exists for $j=0,1,...,J+1$,
integration by parts of the two terms in the integral of Eq.\,(40) yields $(k$ an
integer)
\begin{multline}\label{O42}
\varphi (k/T)-\hat \varphi (k/T)=\hfill\vspace{4pt} \\
\hspace{-10pt}(-1)^k\sum_{j=0}^J\left[ \frac{-iT}{2\pi k}\right]
^{(j+1)}\left[ f^{(j)}(T/2)-f^{(j)}(-T/2)\right] +R,
\end{multline}
where R is a remainder term that goes to~0 as $k\rightarrow \infty $.
Except for the form of the remainder, the same result is obtained in the
same way from the negative of the right side of Eq.\,(37), where $\hat
\varphi $ is defined. Thus for large $\omega $, $\hat \varphi (\omega )$ is
mainly error unless derivatives of $f$ at $-T/2$ are very nearly equal to
those at T$/2$. If the $j^{th}$ derivatives of $f$ at the endpoints are not
equal, then $\hat \varphi (\omega )$ decreases no faster than $\omega
^{-(j+1)}$ for large $\omega $. We find below that nearly equal derivatives
at the ends of the interval are also important when estimating $\hat
\varphi $ using the FFT.

To answer (Q5) consider%
\begin{equation*}
\begin{split}
\varphi (k/T)&=\int_{-\infty }^\infty f(t)e^{-2\pi ikt/T}dt\\
&=\sum_{j=-\infty}^\infty \int_{-T/2+jT}^{T/2+jT}f(t)e^{-2\pi ikt/T}dt\\
&=\int_{-T/2}^{T/2}\sum_{j=-\infty }^\infty f(t+jT)e^{-2\pi ik(t+jT)/T}dt.
\end{split}
\end{equation*}
And since $e^{-2\pi ijk}=1$, we have
\begin{equation}\label{O43}
\begin{split}
\varphi (k/T)&=\int_{-T/2}^{T/2}\hat f(t)e^{-2\pi ikt/T}dt, \quad
\text{where}\vspace{4pt} \\
\hat f(t)&=\sum_{j=-\infty }^\infty f(t+jT)
\end{split}
\end{equation}
If $f(t)$ is replaced by $\hat f(t)$ as defined above, then $\hat \varphi
(k/T)=\varphi (k/T).$

\subsubsection{Errors Due to Discrete Sampling}

Given $f(j\Delta t)$, $j=0$, $\pm 1$, $\pm 2$, ..., we have (proceeding much
as was done in obtaining Eq.\,(42))%
\begin{equation*}
\begin{split}
f(j\Delta t)&=\int_{-\infty }^\infty \varphi (\omega )e^{2\pi
i\omega j\Delta t}d\omega\\
&=\sum_{k=-\infty }^\infty \int_{(k-1/2)/\Delta
t}^{(k+1/2)/\Delta t}\varphi (\omega )e^{2\pi i\omega j\Delta t}d\omega
\end{split}
\end{equation*}
\begin{equation}\label{O44}
\begin{split}
f(j\Delta t)&=\int_{-1/2\Delta t}^{1/2\Delta t}\tilde \varphi
(\omega )e^{2\pi i\omega j\Delta t}d\omega ,\quad \text{where}\\
\tilde \varphi (\omega )&=\sum_{k=-\infty }^\infty \varphi (\omega
+\frac k{\Delta t}).
\end{split}
\end{equation}
Clearly $\tilde \varphi (\omega )$ is periodic with period $1/\Delta t$,
thus from Eqs.\,(18) and (17)
\begin{equation}\label{O45}
\tilde \varphi (\omega )=\Delta t\sum_{j=-\infty }^\infty f(j\Delta
t)e^{-2\pi i\omega j\Delta t}.
\end{equation}
If no assumptions are made about $f$ or $\varphi $, then given just
$f(j\Delta t)$, $j=0$, $\pm 1$, ... values of $\varphi $ for
frequencies that differ by a multiple of $1/\Delta t$ cycles are
irrevocably mixed. This phenomenon is commonly called {\em aliasing.}

When sampling at discrete points the usual assumption made is that $f$ is
band-limited. That is, $\varphi (\omega )=0$ for $|\omega |>1/(2\Delta t)$.
Thus the computed transform is equal to $\hat \varphi $ as given in Eq.
(45), and questions (Q1) and (Q2) are answered by the expression for $\hat
\varphi$ as given in Eq.\,(44).

To answer (Q4) and (Q5), note that if $\tilde f$ is a function whose true
transform is 0 for $|\omega |>1/(2\Delta t)$ and is $\varphi (\omega )$
otherwise, then
\begin{equation}\label{O46}
\tilde f(t)=\int_{-\infty }^\infty r_{1/\Delta t}(\omega )\varphi
(\omega )e^{2\pi i\omega t}d\omega ,
\end{equation}
and using Eqs.\,(25), (26), (31), and (32)
\begin{equation}\label{O47}
\tilde f(t)=\int_{-\infty }^\infty f(\tau )\frac{\sin (\pi
(t-\tau )/\Delta t)}{\pi (t-\tau )}\,d\tau .
\end{equation}
Note the symmetries between $f$ and $\varphi $ in Eqs.\,(38) and (46); (39) and
(47); and (43) and (44). Relations symmetric to Eqs.\,(40) -- (42) are easily
obtained, but we do not bother to do so here.

\subsubsection{Errors Due to Discrete Sampling on a Finite Interval}

Here we consider errors that result from evaluating the integral in Eq.\,(18)
using the discrete Fourier transform. The results also apply to the integral
in Eq.\,(37), and thus errors examined here combined with those due to a
finite range give the total error due to replacing a Fourier integral with a
discrete Fourier transform.

The aliasing problem is revealed much as it was obtained in Eq.\,(44). Let
$\Delta t=T/N$, and from Eq.\,(17)%
\begin{equation*}
\begin{split}
s(j\Delta t)&=\!\sum_{k=-\infty }^\infty \!\sigma _ke^{2\pi
ijk/N}=\!\sum_{m=-\infty }^\infty \!\!\sum_{k=mN}^{(m+1)N-1}\!\!\sigma
_kW^{jk}\\
&=\sum_{k=0}^{N-1}\sum_{m=-\infty }^\infty \sigma _{k+mN}W^{jk},
\end{split}
\end{equation*}
\begin{equation}\label{O48}
s(j\Delta t)=\sum_{k=0}^{N-1}\tilde \sigma _kW^{jk},\text{ where }%
\tilde \sigma _k=\sum_{m=-\infty }^\infty \sigma _{k+mN}.
\end{equation}
And, as for Eq.\,(45), there results from Eqs.\,(9) and (10)
\begin{equation}\label{O49}
\tilde \sigma _k=\frac 1N\sum_{j=0}^{N-1}s(j\Delta t)W^{-jk}.
\end{equation}
Since $\tilde \sigma _{k-N}=\tilde \sigma _k$, and $W^{j(k-N)}=W^{jk},$ Eq.
(48) can be rewritten to make more apparent the assumption that $s$ is
band-limited. Subtracting N from all indices $k\geq N/2$, there results
\begin{equation}\label{O50}
s(j\Delta t)=\sum_{k=-N/2}^{(N/2)-1}\tilde \sigma _kW^{jk}.
\end{equation}
As before, if we assume that $s(t)$ is band-limited, questions (Q1) and (Q2)
are answered by the expression for $\tilde \sigma $ given in Eq.\,(48).

To answer (Q4) and (Q5) as in Eq.\,(47) we use Eqs.\,(29), (30), (33), and
(34) to obtain from
\begin{equation}\label{O51}
\tilde s(t)=\sum_{k=-\infty }^\infty \sigma _k\hat \rho
_k^{(N/2)}e^{2\pi ikt/T},
\end{equation}
\vspace{-10pt}
\begin{multline}\label{O52}
\tilde s(t)=\frac 1T\int_{-T/2}^{T/2}\Biggl [s(\frac{t+\tau }%
2)\left( 1+e^{-\pi i(t-\tau )/T}\right) \vspace{4pt} \\
\times \frac{\sin (\pi N(t-\tau )/2T)}{\sin (\pi (t-\tau )/T)}\Biggr ]%
\,d\tau .
\end{multline}
Another answer to (Q2) can be obtained using the Euler-Maclaurin
formula.  Let $S_k(t)=s(t)e^{-2\pi ikt/T}$, define $S_{k,j}^{(\mu )}$
to be the $\mu ^{th}$ derivative of $S_k$ evaluated at $t=-\frac
T2+j\,\Delta t$, and $S_{k,j}=S_{k,j}^{(0)}$. Then the
Euler-Maclaurin formula applied to Eq.\,(18) gives
\begin{equation}\label{O53}
\hspace{-10pt}
\sigma _k=\frac{\Delta t}T\left \{\frac 12\left[
S_{k,N}+S_{k,0})\right] +\sum_{j=1}^{N-1}S_{k,j} + C_k^{(r)} \right \} +
R_k^{(r)},
\end{equation}\vspace{-5pt}
where
\begin{equation*}
C_k^{(r)}=\sum_{\nu =1}^r\frac{B_{2\nu }(\Delta
t)^{2\nu -1}}{2\nu !}\left[ S_{k,N}^{(2\nu -1)}-
S_{k,0}^{(2\nu -1)}\right] ,
\end{equation*}
$B_{2\nu }$ is the $2\nu ^{th}$ Bernoulli number, and $R_k^{(r)}$ is
a remainder term. The terms in Eq.\,(53) that contain S are what one
would use following the procedure given in Section E above. The terms
involving a $(2\nu -1)^{st}$ derivative of S are correction terms,
which if not used, give an indication (which can be misleading) of
the error. To examine the errors as they depend on the derivatives of
$s$, we write
\begin{equation}\label{O54}
\hspace{-5pt}S_k^{(\mu )}(t)=\sum_{j=1}^\mu \binom
\mu js^{(j)}(t)\left[ \frac{-2\pi ik}T\right] ^{\mu -j}e^{-2\pi ikt/T},
\end{equation}
\vspace{-10pt}
\begin{equation}\label{O55}
\hspace{-5pt}S_k^{(\mu )}(\pm T/2)=\sum_{j=1}^\mu \binom
 \mu j\left[ \frac{-2\pi ik}T\right] ^{\mu -j}(-1)^ks^{(j)}(\pm T/2).
\end{equation}
Substitution into $C_k^{(r)}$ and a little algebraic manipulation gives the
following result.
\begin{equation}\label{O56}
\hspace{-10pt}\begin{array}{l}\displaystyle
\frac{\Delta t}TC_k^{(r)}\!=\!\frac{(-1)^k}%
N\sum_{j=0}^{2r-1}b_j^{(r)}\!\left[ \frac{iT}{2\pi k}\right] ^j\!\!\left[
s^{(j)}(\frac T2)-s^{(j)}(-\frac T2)\right],\\
~
\end{array}% Weird stuff with array needed to avoid overlap on label.
\end{equation}
where
\begin{equation}\label{O57}
b_j^{(r)}=\sum_{\nu =\lfloor j/2\rfloor +1}^r\binom{2\nu
-1}j\frac{B_{2\nu }(-2\pi ik/N)^{2\nu -1}}{(2\nu )!},
\end{equation}
and $\lfloor j/2\rfloor $ is the integer part of $j/2$. Note the similarities
between Eqs.\,(42) and (56). The degree of continuity in the periodic
extension of the function sampled on $[-T/2$, T$/2]$ is extremely important
in determining how well the discrete Fourier transform will approximate
either of the other Fourier transforms.

\subsubsection{Filling in with Zeros}

The requirement that N be a power of~2 imposed by the FFT routines in
this chapter may be inconvenient. It is sometimes suggested that if
one has $N^{\prime }$ function values, that the remaining
$N-N^{\prime }$ values be set to~0, where N is the first power of
$2\geq N^{\prime }$. Let $z_j$ denote the true values of the function
for $j=0,1,...,N-1$, and let $\zeta _k $ and $\hat \zeta _k$ denote
the true and computed coefficients for the discrete Fourier transform
of $z$. Then clearly
\begin{equation}\label{O58}
\zeta _k-\hat \zeta _k=\frac 1N\sum_{j=N^{\prime
}}^{N-1}z_jW^{-jk}.
\end{equation}
And, using Eqs.\,(11), (14), (35), and (36)
\begin{equation}\label{O59}
\hat \zeta _k=\sum_{\nu =0}^{N-1}\zeta _\nu \frac{%
1-W^{-N^{\prime }(k-\nu )}}{N(1-W^{-(k-\nu )})}
\end{equation}
where for $\nu =k$ the multiplier of $\zeta _\nu $ is $N^{\prime }/N.$

It is perhaps more instructive to consider the connection between $\hat
\zeta _k$ and $\tilde \zeta _k$, where
\begin{equation}\label{O60}
\tilde \zeta _k=\frac 1{N^{\prime }}\sum_{j=0}^{N^{\prime
}-1}z_je^{-2\pi ijk/N^{\prime }}.
\end{equation}
By extending the definitions of $\hat \zeta _k$, and $\tilde \zeta _k$ to
noninteger $k$ in the obvious way, one obtains
\begin{equation}\label{O61}
\begin{split}
\hat \zeta _{kN/N^{\prime }}&=\frac 1N\sum_{j=0}^{N^{\prime
}-1}z_je^{-2\pi ijk/N^{\prime }}=\frac{N^{\prime }}N\tilde \zeta _k
\text{,\quad and}\\
\tilde \zeta _{kN^{\prime }/N}&=\frac 1{N^{\prime }}
\sum_{j=0}^{N^{\prime }-1}z_je^{-2\pi ijk/N}=\frac N{N^{\prime }}
\hat \zeta _k
\end{split}
\end{equation}
Thus $\frac N{N^{\prime }}\hat \zeta _k$ can be thought of as a way
of computing $\tilde \zeta $ with a smaller than usual $\Delta \omega
$. In particular, if $N=2N^{\prime }$, $\hat \zeta _{2k}=\tilde \zeta
_k/2.$ It is sometimes suggested that $N^{\prime }$ extra zeros be
added even when $N^{\prime }$ is a power of~2. By doing this one
can use the FFT to get the autocorrelation from Eqs.\,(11) and (14),
and the power spectrum, which can be defined as the Fourier transform
of the autocorrelation function, is obtained automatically. An
alternative method for those who like this approach is to compute
$\tilde \zeta _k/2$ $(N^{\prime }$ a power of~2) to get $\hat \zeta
_{2k}$, and obtain $\hat \zeta _{2k+1}$ by setting it equal to the
$k^{th}$discrete Fourier coefficient of $z^{\prime }$ where
$z_j^{\prime }=z_je^{\pi ij/N^{\prime }},\ j=0,1,...,N^{\prime }-1.$

If one zeros $\zeta _k$ for $k=N^{\prime }$, $N^{\prime }+1$, ..., N, then
instead of defining a trigonometric polynomial of degree $N-1$ that passes
through $z_j$, $j=0,1,...,N-1,$ the $\zeta ^{\prime }s$ define a
trigonometric polynomial of degree $N^{\prime }-1$, that fit the $z_j$ in a
least-squares sense. Multiplication by the Lanczos sigma factors, see below,
should usually give better smoothing characteristics for nonperiodic data.

\subsection{Recommendations}

The FFT gives good results if one is sampling a periodic function over one
period. Results are less satisfactory for other cases. What we suggest here
should usually give an improvement over the FFT, however, in many cases one
can undoubtedly do ever better.

It is assumed that trends in the data $(e.g.$, a linear trend) have been
removed, and that appropriate action has been taken for wild points or gaps
in the data. By a trend, we mean a smooth function S, so that the
(estimated) average value of $|S(t)-z(t)|$ is as small as possible for $t$
outside the sampling interval. In some cases one may want to add the Fourier
transform of S to the computed transform of $z$.

For functions that are not periodic it makes little sense to attempt a
representation in terms of a discrete set of frequencies. In Eq.\,(61) the
value of the discrete transform for noninteger values is defined. What we
propose here amounts to computing $\zeta _k$, which is an average of the
values of the discrete transform for nearby noninteger values of $k$.
Starting from Eq.\,(21), consider
\begin{align}
\varphi _a(\omega )=&\frac 1{2a}\int_{\omega -a}^{\omega
+a}\varphi (\alpha )\,d\alpha \vspace{4pt}\notag \\
=&\frac TN\sum_{j=0}^{N-1}z_j\frac
1{2a}\int_{\omega -a}^{\omega +a}e^{\pi i\alpha T(N-2j)/N}d\alpha
\vspace{4pt}\notag \\
=\frac TNe^{\pi i\omega T}&\sum_{j=0}^{N-1}\left[
\frac{\sin [\pi aT(N-2j)/N]}{\pi aT(N-2j)/N}\right]
z_je^{-2\pi ij\omega T/N}.\label{O62}
\end{align}
With $\omega =\omega _k$, Eq.\,(62) is the same as Eq.\,(23) except
for the multiplier of $z_j$. The choice $a=1/T$ is attractive for
several reasons:  it is the smallest value of $a$ that gives a
multiplier of~0 for $z_0$(and for $z_N$ if it were used), and thus
helps to minimize the effect of a discontinuity in the periodic
extension of $z_j$; the value of $\varphi _{1/T}(\omega _k)$ is the
average value of $\varphi $ from $\omega _{k-1}$ to $\omega
_{k+1}$, so not much resolution is lost; finally, if Eq.\,(39) is
integrated from $\omega -a$ to $\omega +a$ as was done in Eq.\,(21),
one finds that the choice $a=1/T$ minimizes the effect of $\varphi
(\hat \omega )$ on $\hat \varphi _k(\omega )$ for values of $\hat
\omega $ remote from $\omega $. Thus, we define
\begin{equation}\label{O63}
\sigma _j=\begin{cases}
\dfrac{\sin \pi j/N}{\pi j/N}\vspace{4pt} & j\neq 0\\
1 & j=0
\end{cases}
\end{equation}
and the multiplier of $z_j$ in Eq.\,(62) is $\sigma _{N-2j}$. And since
$\sigma _{-N+2j}=\sigma _{N-2j}$, $z_{N-j}$ has the same multiplier.

The $\sigma _j$ defined in Eq.\,(63) are the Lanczos sigma factors, see
\cite{Lanczos:1956:AA} for a different motivation in their derivation, and
for instructive examples.  If one wants a greater smoothing, rather than
increasing $a$, we recommend averaging the average.  Thus if the same
procedure is applied to Eq.\,(62) as was applied to Eq.\,(21) one finds
that $\sigma _{N-2j}$ is replaced by $\sigma _{N-2j}^2$.  This process can
of course be repeated, depending on how much resolution one is willing to
give up.  The $\sin \pi j/N$ are readily available from the sine table
required by the FFT subroutines as illustrated in the example for SRFT1.

The choice of $a = 1/T$ is not appropriate if one has filled out with
zeros because $N^{\prime}$ values are given and $N^{\prime}$ is not a
power of~2.  We assume the given values of $z$ are stored in $z_j$,
$j = N^{\prime}/2$, ..., $[(N+N^{\prime})/2]$, and that both ends
have been zero filled. Then one should set $a =N/(N^{\prime}T).$

The $\sigma $ factors are also useful for smoothing. Given real data,
$x$, one can replace $x_j$ by the average value of the trigonometric
polynomial interpolating the data on the interval $x_{j-1}\leq x\leq
x_{j+1}$ using the $\sigma $ factors. Simply compute the $\zeta _k$
using the FFT, multiply $\zeta _k$ by $\sigma _{2k}$, $k=1$, 2,
..., N$/2$, and then compute the inverse transform. This application
is illustrated in the example for SRFT.

When the number of data points is not a power of two, one may have an
interest in using a mixed radix algorithm such as that in
\cite{Singleton:1969:AAC}, or one could use the codes given here together
with the technique described in \cite{Bailey:1991:TFF}.  Before making the
decision to go with something more complicated than padding the data
with~0's, (or when padding the data with~0's), one should understand the
connections described from Eq.\,(60) to just below Eq.\,(61).
\nocite{Hamming:1962:NMS}
\nocite{Jenkins:1968:SAA}
\nocite{Havie:1973:REI}
\nocite{Abramovici:1973:TAC}
\nocite{Lyness:1971:TCF}
\nocite{VandeVooren:1968:OCI}
\nocite{Stetter:1966:NAF}
\nocite{Clendenin:1966:AMN}
\nocite{VanLoan:1992:CFF}

\bibliography{math77}
\bibliographystyle{math77}

For information on the web, see:\newline
{\bf http://theory.lcs.mit.edu/$^\sim$fftw/fft-links.html}

Fred T. Krogh, JPL, November~1974. Minor additions and corrections,
October~1991, October~1993, and April~1998.
\end{multicols}
\end{document}


\subsection{Loss functions for hard classifiers}

Don't want answers outside \(0\) and \(1\).

\subsubsection{F score}

\subsubsection{F1 score}

\(F_1\) score: \(\dfrac{2PR}{(P+R)}\)

may not just care about accuracy, eg breast cancer screening

high accurancy can result from v basic model (ie all died on titanic)

\subsubsection{Receiver Operating Characteristic (ROC) Area Under Curve (AUC)}


\section{Example Equations Explained}

    \subsection{Permutations}

        \subsubsection{Factorial Notation}
            \begin{example}
                Solve the following equation for n. State restrictions.
                \begin{equation*}
                    \frac{n!}{(n-3)!}=2n
                \end{equation*}
            \end{example}
            To solve for n, we must first divide out the denominator, which in this case is $(n-3)!$. Before all of this, we should state our restriction. Since a factorial may not be negative, and $(n-3)!$ would equal $-3!$ our restriction is that \emph{$n\geqslant3$}. You may notice there is no $(n-3)!$ in the top, but the solution for this is simple. If you recall the equation for factorial notation, we can expand the top of this question.  Following this, the equation expands to:
            \begin{equation*}
                \frac{n\cdot(n-1)\cdot(n-2)\cdot(n-3)!}{(n-3)!}=2n
            \end{equation*}
            Next, the $(n-3)!$s cancel out, to bring you this:
            \begin{equation*}
                n\cdot(n-1)\cdot(n-2) = 2n
            \end{equation*}
            Now we can divide out the n from both sides. To get this:
            \begin{equation*}
                (n-1)\cdot(n-2) = 2
            \end{equation*}
            We can't divide any further, so we'll need to expand.
            \begin{equation*}
                n^2 -3n+2 = 2
            \end{equation*}
            Move the 2 over so we can turn this into something we can factor.
            \begin{equation*}
                n^2 -3n=0
            \end{equation*}
            Use factor by grouping to factor this equation.
            \begin{equation*}
                n(n-3)=0
            \end{equation*}
            So either $n=0$ or $n-3=0$. Following our restriction, n be greater than 3, so $n=0$ is inadmissible, therefore the answer must be $n=3$.
    \clearpage
        \subsubsection{Rule of Product and Rule of Sum}
            \begin{example}
                Using only the numbers from $0\cdots4$, how many three digit numbers can you find if the number 4 \textbf{MUST} be included. No repetition.
            \end{example}
            For this question we must follow the Rule of Sum.

    \clearpage
        \subsubsection{Problem Solving with Permutations}
            \begin{example}
                Try to find the total number of arrangements of the numbers 0-7(8 digits) that are even. No repetition is allowed, and the number cannot start with a 0.
            \end{example}
            For this question, you can use the \textbf{Case Method} and break the question down into numbers that end in a zero, and numbers that don't end in a zero, yet still end in an even number.
            \begin{parcolumns}{2}
                \colchunk[1]{\\\textbf{Numbers that end in a zero:}
                    $$7! \cdot 1 = 5040$$ \\
                    \textbf{Explanation:} One has 7 possible numbers for the first place, and only zero as the final possible number.
                }
                \colchunk[2]{\\\textbf{Numbers that end in 2,4,6:}
                    $$6 \cdot 6 \cdot 3 = 12960$$ \\
                    \textbf{Explanation:} The last number is 3, and represents the possible even numbered endings (2,4,6). The reason why it is $6 \cdot 6$, it is because zero cannot be first, yet it can be second.\\
                }
                \colplacechunks
            \end{parcolumns}
            This question isn't over yet though. Finally, you will have to add the solutions together, as you may have found all of the answers for the possible even numbers (0,2,4,6), but they haven't been added together yet.
            \begin{equation*}
                5040 + 12960 = 18000
            \end{equation*}
            \emph{In conclusion}, there will be 18000 possible combinations of even numbers by using the numbers 0-7 (8 digits in total).

    \subsection{Combinations}

        \subsubsection{Set Theory}

        \subsubsection{Intro to Combinations}
            \begin{example}
                A committee of 4 people is chosen from 10 students, 6 parents, and 10 teachers. Order does not matter. How many ways can we make the committee if there are a) no restrictions, b) At least one teacher on the committee or c) Exactly three students on the committee.
            \end{example}
            \textbf{a) No restrictions:}\\
            We can assume that this is a combination, as there are no restrictions other than order does not matter. Therefore we only need to fill in the combination equation, which turns out to be C(26,4).
            \begin{equation*}
                \binom{26}{4} = 14950
            \end{equation*}
            \textbf{b) At least one teacher.}
            There are two ways to handle this question, we can solve it either directly, or indirectly. In this example, I will choose indirectly. In this case, we want every solution with a teacher in it, and do not want the solution with zero teachers. Therefore we must subtract the unwanted solutions from all the solutions. This would work out to be:
            \begin{equation*}
                \binom{26}{4} - \binom{10}{0}\cdot\binom{16}{4} = 13130
            \end{equation*}
            \textbf{c) Exactly three students.}
            For this question, we are forced to do it one way and one way only, as the question refers to \emph{exactly three} students. To start we must have 3 students on the committee, and then since we have one space left on the committee, we must fill that spot with someone who \emph{IS NOT} a student, as that would ruin our "exactly three" need. This works out to be:
            \begin{equation*}
                \binom{10}{3}\cdot\binom{24-10}{1} = 1680
            \end{equation*}


% This LaTeX was auto-generated from an M-file by MATLAB.
% To make changes, update the M-file and republish this document.

\subsection*{covNoise.m}

\begin{par}
Independent covariance function, ie "white noise", with specified variance. The covariance function is specified as:
\end{par} \vspace{1em}
\begin{par}
k(x\^{}p,x\^{}q) = s2 * \ensuremath{\backslash}delta(p,q)
\end{par} \vspace{1em}
\begin{par}
where s2 is the noise variance and \ensuremath{\backslash}delta(p,q) is a Kronecker delta function which is 1 iff p=q and zero otherwise. The hyperparameter is
\end{par} \vspace{1em}
\begin{par}
logtheta = [ log(sqrt(s2)) ]
\end{par} \vspace{1em}
\begin{par}
For more help on design of covariance functions, try "help covFunctions".
\end{par} \vspace{1em}
\begin{par}
(C) Copyright 2006 by Carl Edward Rasmussen, 2006-03-24.
\end{par} \vspace{1em}

\begin{lstlisting}
function [A, B] = covNoise(logtheta, x, z)
\end{lstlisting}

\subsection*{Code}

\begin{lstlisting}
if nargin == 0, A = '1'; return; end              % report number of parameters

s2 = exp(2*logtheta);                                          % noise variance

if nargin == 2                                      % compute covariance matrix
  A = s2*eye(size(x,1));
elseif nargout == 2                              % compute test set covariances
  A = s2;
  B = 0;                               % zeros cross covariance by independence
else                                                % compute derivative matrix
  A = 2*s2*eye(size(x,1));
end
\end{lstlisting}


    \filetitle{cumsumk}{Cumulative sum with a k-period leap}{tseries/cumsumk}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
Y = cumsumk(X,K,Rho,Range)
Y = cumsumk(X,K,Rho)
Y = cumsumk(X,K)
Y = cumsumk(X)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\item
  \texttt{X} {[} tseries {]} - Input data.
\item
  \texttt{K} {[} numeric {]} - Number of periods that will be leapt the
  cumulative sum will be taken; if not specified, \texttt{K} is chosen
  to match the frequency of the input data (e.g. \texttt{K = -4} for
  quarterly data), or \texttt{K = -1} for indeterminate frequency.
\item
  \texttt{Rho} {[} numeric {]} - Autoregressive coefficient; if not
  specified, \texttt{Rho = 1}.
\item
  \texttt{Range} {[} numeric {]} - Range on which the cumulative sum
  will be computed and the output series returned.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{Y} {[} tseries {]} - Output data constructed as described
  below.
\end{itemize}

\paragraph{Options}\label{options}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{'log='} {[} \texttt{true} \textbar{} \emph{\texttt{false}} {]}
  - Logarithmise the input data before, and de-logarithmise the output
  data back after, running \texttt{x12}.
\end{itemize}

\paragraph{Description}\label{description}

If \texttt{K \textless{} 0}, the first \texttt{K} observations in the
output series \texttt{Y} are copied from \texttt{X}, and the new
observations are given recursively by

\begin{verbatim}
Y{t} = Rho*Y{t-K} + X{t}.
\end{verbatim}

If \texttt{K \textgreater{} 0}, the last \texttt{K} observations in the
output series \texttt{Y} are copied from \texttt{X}, and the new
observations are given recursively by

\begin{verbatim}
Y{t} = Rho*Y{t+K} + X{t},
\end{verbatim}

going backwards in time.

If \texttt{K == 0}, the input data are returned.

\paragraph{Example}\label{example}

Construct random data with seasonal pattern, and run X12 to seasonally
adjust these series.

\begin{verbatim}
x = tseries(qq(1990,1):qq(2020,4),@randn);
x1 = cumsumk(x,-4,1);
x2 = cumsumk(x,-4,0.7);
x1sa = x12(x1);
x2sa = x12(x2);
\end{verbatim}

The new series \texttt{x1} will be a unit-root process while \texttt{x2}
will be stationary. Note that the command on the second line could be
replaced with \texttt{x1 = cumsumk(x)}.


\chapter{Cubic functions}
\section{Defined on $\mdr$}
Let $f:\mdr \rightarrow \mdr, f(x) = a \cdot x^3 + b \cdot x^2 + c \cdot x + d$
be a cubic function with $a \in \mdr \setminus \Set{0}$ and
$b, c, d \in \mdr$.

\begin{figure}[htp]
    \centering
\begin{tikzpicture}
    \begin{axis}[
        legend pos=south east,
        axis x line=middle,
        axis y line=middle,
        grid = major,
        width=0.8\linewidth,
        height=8cm,
        grid style={dashed, gray!30},
        xmin=-3, % start the diagram at this x-coordinate
        xmax= 3, % end   the diagram at this x-coordinate
        ymin=-3, % start the diagram at this y-coordinate
        ymax= 3, % end   the diagram at this y-coordinate
        axis background/.style={fill=white},
        xlabel=$x$,
        ylabel=$y$,
        tick align=outside,
        minor tick num=-3,
        enlargelimits=true,
        tension=0.08]
      \addplot[domain=-3:3, thick,samples=50, red] {x*x*x};
      \addplot[domain=-3:3, thick,samples=50, green] {x*x*x+x*x};
      \addplot[domain=-3:3, thick,samples=50, blue] {x*x*x+2*x*x};
      \addplot[domain=-3:3, thick,samples=50, orange] {x*x*x+x};
      \addlegendentry{$f_1(x)=x^3$}
      \addlegendentry{$f_2(x)=x^3 + x^2$}
      \addlegendentry{$f_2(x)=x^3 + 2 \cdot x^2$}
      \addlegendentry{$f_1(x)=x^3 + x$}
    \end{axis}
\end{tikzpicture}
    \caption{Cubic functions}
\end{figure}

%
%\section{Special points}
%\todo[inline]{Write this}
%
%\section{Voronoi}
%
%For $b^2 \geq 3ac$
%
%\todo[inline]{Write this}

\subsection{Calculate points with minimal distance}
\begin{theorem}\label{thm:no-finite-solution}
    There cannot be a finite, closed form solution to the problem of finding
    a closest point $(x, f(x))$ to a given point $P$ when $f$ is
    a polynomial function of degree $3$ or higher.
\end{theorem}

\begin{proof}
    Suppose you could solve the closest point problem for arbitrary
    cubic functions $f = ax^3 + bx^2 + cx + d$ and arbitrary points $P = (x_P, y_P)$.

    Then you could solve the following problem for $x$:
    \begin{align}
        0  &\stackrel{!}{=} \left ((d_{P,f}(x))^2 \right )'\\
           &=-2 x_p + 2x -2y_p(f(x))' + (f(x)^2)'\\
           &= 2 f(x) \cdot f'(x) - 2 y_p f'(x) + 2x - 2 x_p\\
           &= f(x) \cdot f'(x) - y_p f'(x) + x - x_p\\
           &= \underbrace{f'(x) \cdot \left (f(x) - y_p \right )}_{\text{Polynomial of degree 5}} + x - x_p
    \end{align}

    General algebraic equations of degree 5 don't have a solution formula.\footnote{TODO: Quelle}
    Although here seems to be more structure, the resulting algebraic
    equation can be almost any polynomial of degree 5:\footnote{Thanks to Peter Košinár on \href{http://math.stackexchange.com/a/584814/6876}{math.stackexchange.com} for the idea.}

    \begin{align}
        0  &\stackrel{!}{=} f'(x) \cdot \left (f(x) - y_p \right ) + (x - x_p)\\
        &= \underbrace{3 a^2}_{= \tilde{a}} x^5 + \underbrace{5ab}_{= \tilde{b}}x^4 + \underbrace{2(2ac + b^2 )}_{= \tilde{c}}x^3 &+& \underbrace{3(ad+bc-ay_p)}_{= \tilde{d}} x^2 \\
        & &+& \underbrace{(2 b d+c^2+1-2 b y_p)}_{= \tilde{e}}x+\underbrace{c d-c y_p-x_p}_{= \tilde{f}}\\
        0 &\stackrel{!}{=} \tilde{a}x^5 + \tilde{b}x^4 + \tilde{c}x^3 + \tilde{d}x^2 + \tilde{e}x + \tilde{f}
    \end{align}

    \begin{enumerate}
        \item For any coefficient $\tilde{a} \in \mdr_{> 0}$ of $x^5$ we can choose $a := \frac{1}{3} \sqrt{\tilde{a}}$ such that we get $\tilde{a}$.
        \item For any coefficient $\tilde{b} \in \mdr \setminus \Set{0}$ of $x^4$ we can choose $b := \frac{1}{5a} \cdot \tilde{b}$ such that we get $\tilde{b}$.
        \item With $c := -2b^2 + \frac{1}{4a} \tilde{c}$, we can get any value of $\tilde{c} \in \mdr$.
        \item With $d := -bc + a y_p + \frac{1}{a} \tilde{d}$, we can get any value of $\tilde{d} \in \mdr$.
        \item With $y_p := \frac{1}{2b}(2bd + c^2)\cdot \tilde{e}$, we can get any value of $\tilde{e} \in \mdr$.
        \item With $x_p := cd - c y_P+\tilde{f}$, we can get any value of $\tilde{f} \in \mdr$.
    \end{enumerate}

    The first restriction guaratees that we have a polynomial of
    degree 5. The second one is necessary, to get a high range of
    $\tilde{e}$.

    This means that there is no finite solution formula for the problem of
    finding the closest points on a cubic function to a given point,
    because if there was one, you could use this formula for finding
    roots of polynomials of degree 5. $\qed$
\end{proof}

\subsection{Another approach}
Just like we moved the function $f$ and the point to get in a
nicer situation, we can apply this approach for cubic functions.

\begin{figure}[htp]
    \centering
\begin{tikzpicture}
    \begin{axis}[
        legend pos=south east,
        axis x line=middle,
        axis y line=middle,
        grid = major,
        width=0.8\linewidth,
        height=8cm,
        grid style={dashed, gray!30},
        xmin=-3, % start the diagram at this x-coordinate
        xmax= 3, % end   the diagram at this x-coordinate
        ymin=-3, % start the diagram at this y-coordinate
        ymax= 3, % end   the diagram at this y-coordinate
        axis background/.style={fill=white},
        xlabel=$x$,
        ylabel=$y$,
        tick align=outside,
        minor tick num=-3,
        enlargelimits=true,
        tension=0.08]
      \addplot[domain=-3:3, thick,samples=50, red] {x*x*x};
      \addplot[domain=-3:3, thick,samples=50, green] {x*x*x+x};
      \addplot[domain=-3:3, thick,samples=50, orange] {x*x*x-x};
      \addplot[domain=-3:3, thick,samples=50, blue, dotted] {x*x*x+2*x};
      \addplot[domain=-3:3, thick,samples=50, lime, dashed] {x*x*x+3*x};
      \addlegendentry{$f_1(x)=x^3$}
      \addlegendentry{$f_2(x)=x^3 + x$}
      \addlegendentry{$f_1(x)=x^3 - x$}
      \addlegendentry{$f_2(x)=x^3 + 2 \cdot x$}
      \addlegendentry{$f_2(x)=x^3 + 3 \cdot x$}
    \end{axis}
\end{tikzpicture}
    \caption{Cubic functions with $b = d = 0$}
\end{figure}

First, we move $f_0$ by $\frac{b}{3a}$ in $x$ direction, so

\[f_1(x) = ax^3 + \frac{b^2 (c-1)}{3a} x + \frac{2b^3}{27 a^2} - \frac{bc}{3a} + d \;\;\;\text{ and }\;\;\;P_1 = (x_P + \frac{b}{3a}, y_P)\]

because

\begin{align}
    f_1(x) &= a \left (x - \frac{b}{3a} \right )^3 + b \left (x-\frac{b}{3a} \right )^2 + c \left (x-\frac{b}{3a} \right ) + d\\
           &= a \left (x^3 - 3 \frac{b}{3a}x^2 + 3 (\frac{b}{3a})^2 x - \frac{b^3}{27a^3} \right )
             +b \left (x^2 - \frac{2b}{3a} x + \frac{b^2}{9a^2} \right )
             +c x - \frac{bc}{3a} + d\\
            &= ax^3 - bx^2 + \frac{b^2}{3a}x - \frac{b^3}{27 a^2}\\
            & \;\;\;\;\;\;+ bx^2 - \frac{2b^2}{3a}x + \frac{b^3}{9a^2}\\
            & \;\;\;\;\;\;\;\;\;\;\;\; + c x - \frac{bc}{3a} + d\\
            &= ax^3 + \frac{b^2}{3a}\left (1-2+c \right )x + \frac{b^3}{9a^2} \left (1-\frac{1}{3} \right )- \frac{bc}{3a} + d
\end{align}

The we move it in $y$ direction by $- (\frac{2b^3}{27 a^2} - \frac{bc}{3a} + d)$:

\[f_2(x) = ax^3 + \frac{b^2 (c-1)}{3a} x \;\;\;\text{ and }\;\;\;P_2 = (x_P + \frac{b}{3a}, y_P - (\frac{2b^3}{27 a^2} - \frac{bc}{3a} + d))\]

Multiply everything by $\sgn(a)$:

\[f_3(x) = \underbrace{|a|}_{=: \alpha}x^3 + \underbrace{\frac{b^2 (c-1)}{3|a|}}_{=: \beta} x \;\;\;\text{ and }\;\;\;P_2 = (x_P + \frac{b}{3a}, \sgn(a) (y_P - \frac{2b^3}{27 a^2} + \frac{bc}{3a} - d))\]

Now the problem seems to be much simpler. The function $\alpha x^3 + \beta x$
with $\alpha > 0$ is centrally symmetric to $(0, 0)$.

\todo[inline]{Und weiter?}

\subsection{Number of points with minimal distance}
As this leads to a polynomial of degree 5 of which we have to find
roots, there cannot be more than 5 solutions.
\todo[inline]{Can there be 3, 4 or even 5 solutions? Examples!

After looking at function graphs of cubic functions, I'm pretty
sure that there cannot be 4 or 5 solutions, no matter how you
chose the cubic function $f$ and $P$.

I'm also pretty sure that there is no polynomial (no matter what degree)
that has more than 3 solutions.}

\subsection{Interpolation and approximation}
\subsubsection{Quadratic spline interpolation}
You could interpolate the cubic function by a quadratic spline.

\subsubsection{Bisection method}
\todo[inline]{TODO}

\subsubsection{Newtons method}
One way to find roots of functions is Newtons method. It gives an
iterative computation procedure that can converge quadratically
if some conditions are met:

\begin{theorem}[local quadratic convergence of Newton's method\footnotemark]
    Let $D \subseteq \mdr^n$ be open and $f: D \rightarrow \mdr^n \in C^2(\mdr)$.
    Let $x^* \in D$ with $f(x^*) = 0$ and the Jaccobi-Matrix $f'(x^*)$
    should not be invertable when evaluated at the root.

    Then there is a sphere
    \[K := K_\rho(x^*) = \Set{x \in \mdr^n | \|x- x^*\|_\infty \leq \rho} \subseteq D\]
    such that $x^*$ is the only root of $f$ in $K$. Furthermore,
    the elements of the sequence
    \[ x_{n+1} = x_n - \frac{f'(x_n)}{f(x_n)}\]
    are for every starting value $x_0 \in K$ again in $K$ and
    \[\lim_{n \rightarrow \infty} x_k = x^*\]
    Also, there is a constant $C > 0$ such that
    \[\|x^* - x_{n+1} \| = C \|x^* - x_n\|^2 \text{ for } n \in \mathbb{N}_0\|\]
\end{theorem}
\footnotetext{Translated from German to English from lecture notes of "Numerische Mathematik für die Fachrichtung Informatik
und Ingenieurwesen" by Dr. Weiß, KIT}

The approach is extraordinary simple. You choose a starting value
$x_0$ and compute

\[x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}\]

As soon as the values don't change much, you are close to a root.
The problem of this approach is choosing a starting value that is
close enough to the root. So we have to have a \enquote{good}
initial guess.
\clearpage

\subsubsection{Muller's method}
Muller's method was first presented by David E. Muller in 1956.

\todo[inline]{Paper? Might this be worth a try?}

\subsubsection{Bisection method}
The idea of the bisection method is the following:

Suppose you know a finite intervall $[a,b]$ in which you have
exactly one root $r \in (a,b)$ with $f(r) = 0$.

Then you can half that interval:
    \[[a, b] = \left [a, \frac{a+b}{2} \right ] \cup \left [\frac{a+b}{2}, b \right ]\]

Now three cases can occur:
\begin{enumerate}
    \item[Case 1] $f(\frac{a+b}{2})=0$: You have found the exact root.
    \item[Case 2] $\sgn(a) = \sgn(\frac{a+b}{2})$: Continue searching in $[\frac{a+b}{2}, b]$
    \item[Case 3] $\sgn(b) = \sgn(\frac{a+b}{2})$: Continue searching in $[a, \frac{a+b}{2}]$
\end{enumerate}

\todo[inline]{Which intervall can I choose? How would I know that there is exactly one root?}

\subsubsection{Bairstow's method}
Cite from Wikipedia:
The algorithm first appeared in the appendix of the 1920 book "Applied Aerodynamics" by Leonard Bairstow. The algorithm finds the roots in complex conjugate pairs using only real arithmetic.

[...]
\todo[inline]{Find a source for the following!}
A particular kind of instability is observed when the polynomial has odd degree and only one real root.

\section{Defined on a closed interval $[a,b] \subseteq \mdr$}
The point with minimum distance can be found by:
\[\underset{x\in[a,b]}{\arg \min d_{P,f}(x)} = \begin{cases}
 S_3(f, P) &\text{if } S_3(f, P) \cap [a,b] \neq \emptyset\\
  TODO     &\text{if } S_3(f, P) \cap [a,b] = \emptyset
    \end{cases}\]


\section{Experiments}

We have implemented a tool chain that takes as input a description
of the polytopes in terms of vertices, and the parallelepiped boundaries,
and it returns the placement vectors for the arrangement, if any
exists.

In order to compute the facets of the polytopes we rely on the CGAL
library~\cite{CGAL}. We then solve the SMT problem with some
efficient and open-source SMT-solver, such as CVC4~\cite{CVC4}, OpenSMT~\cite{OpenSMT},
or Z3~\cite{Z3}. We use CGAL again
to export the solution returned by the SMT-Solver into the VRML graphical models
that can be seen in this paper. The source code of our tool-chain can
be downloaded from~\url{https://github.com/bobosoft/polytopepacking}.
At the same address it is possible to obtain the problem descriptions, the
encoded smt2 files, and the 3D models of the results.

As an experiment to test our proof-of-concept tool-chain we have
encoded and solved the first problem described in~\cite{sto03}, where
7 polytopes are placed in a parallelepiped of length 12 and width 10.
\cite{sto03} shows a placement of height 27, which is a local optima for
the approach of that paper. By running our tool-chain we were able to
prove, in about 10 minutes, that a height of 23 is actually sufficient.
We were not able to obtain a model for height 22 within an hour, and
the polytope placement is shown in Figure~\ref{fig:p7}. In these
trials we have increased and decreased the value for the height manually,
but this process can be automated in a way that is standard among
optimizing solvers.

A more automated implementation, and more exhaustive and detailed
experimentation are left as future work.

%% \begin{figure}
%% \begin{center}
%% \begin{tabular}{cccc}
%% \begin{minipage}{.2\textwidth}
%% \includegraphics[scale=.2]{poly_1.png}
%% \end{minipage}
%% &
%% \begin{minipage}{.2\textwidth}
%% \includegraphics[scale=.2]{poly_2.png}
%% \end{minipage}
%% &
%% \begin{minipage}{.2\textwidth}
%% \includegraphics[scale=.2]{poly_3.png}
%% \end{minipage}
%% &
%% \begin{minipage}{.2\textwidth}
%% \includegraphics[scale=.2]{poly_4.png}
%% \end{minipage}
%% \\
%% \begin{minipage}{.2\textwidth}
%% \includegraphics[scale=.2]{poly_5.png}
%% \end{minipage}
%% &
%% \begin{minipage}{.2\textwidth}
%% \includegraphics[scale=.2]{poly_6.png}
%% \end{minipage}
%% &
%% \begin{minipage}{.2\textwidth}
%% \includegraphics[scale=.2]{poly_7.png}
%% \end{minipage}
%% \end{tabular}
%% \end{center}
%% \caption{The polytopes that are to be packed.}
%% \label{fig:p}
%% \end{figure}

\begin{figure}
\begin{center}
\begin{tabular}{cc}
\begin{minipage}{.4\textwidth}
\includegraphics[scale=.5]{problem_7_1.png}
\end{minipage}
&
\begin{minipage}{.4\textwidth}
\includegraphics[scale=.5]{problem_7_2.png}
\end{minipage}
\end{tabular}
\end{center}
\caption{Two views of the packing placement for 7 polytopes as per~\cite{sto03}
for a height of 23.}
\label{fig:p7}
\end{figure}


\bigskip
\textit{This paper is the report of the \textbf{Group 43} for the first assignment
of the course.}

\section{Search Algorithms and their relations}

\begin{enumerate}
 \item Give a consistent heuristic for this problem. Prove that it is admissible.
    \begin{framed}
        Our heuristic is a function of the distance with a manhattan
        distance heuristic (ignoring walls) between
        the current character position and the \euro{} position. It is
        admissible because it will never overestimate the cost of reaching
        the goal, as such we will always find an optimal solution with A*
        search.
    \end{framed}
 \item Show on the left maze the states (board positions) that are visited during an
execution of a uniform-cost graph search. We assume that when different states
in the fringe have the smallest value, the algorithm chooses the state with
the smallest coordinate (\textit{i, j}) ((0, 0) being the bottom left position, \textit{i} being the
horizontal index and \textit{j} the vertical one) using a lexicographical order.
    \begin{framed}
        \begin{center}
        \includegraphics[width=0.5\linewidth]{figure1_left.png}
        \end{center}

        The squares are visited following the numbering order and red nodes
        are not included in the final path.
    \end{framed}
  \item Show on the right maze the board positions visited by A graph search with
a manhattan distance heuristic (ignoring walls). A state is visited when it is
selected in the fringe and expanded. When several states have the smallest
path cost, this uniform-cost search visits them in the same lexicographical order
as the one used for uniform-cost graph search.
    \begin{framed}
        \begin{center}
        \includegraphics[width=0.5\linewidth]{figure1_right.png}
        \end{center}

        The squares are visited following the numbering order and red nodes
        are not included in the final path.
    \end{framed}

\end{enumerate}

\section{Sokoban planning problem}

\begin{enumerate}
 \item As illustrated on Figure 3 some situations cannot lead to a solution. Are there
other similar situations? If yes, describe them.
  \begin{framed}
        Other similar situations that cannot lead to a solution happen :
        \begin{itemize}
            \item When a box is in a corner formed by walls and
                this corner is not a target position for boxes ;
            \item When two boxes are next to each other and both of the boxes cannot be moved so they act as
                walls.
        \end{itemize}
  \end{framed}
  \item Why is it important to identify dead states in your successor function? How
are you going to implement it?
    \begin{framed}
        Identifying as many dead states as possible will allow us to reduce
        the search space by stopping the exploration of branches that fell
        in a dead state. Allowing us to reduce the time that our algorithm
        need to find a solution. We are going to implement it by
        identifying the situations mentioned in the answer to the previous
        question.
    \end{framed}
  \item Describe possible (non trivial) heuristic(s) to reach a goal state (with reference
if any). Is(are) your heuristic(s) admissible and/or consistent?
    \begin{framed}
    A first interesting heuristic would be to give a fixed penalty for each
    boxes that isn't on a target, allowing our algorithm to investigate
    potential solution first. But this heuristic that is neither consistent
    nor always admissible (depending on the penalty) has a quite a few
    shortcoming as it might take a long time before the algorithm finds a
    state worth investigating and that, if a box has to reach a target by
    passing over another target, it will explore all the children of this
    state before considering other solution. \newline

    A second interesting heuristic, that solve the shortcomings of the
    previous one, would be the sum of the manhattan distance from each
    boxes to the closest target. Although this heuristic is not always
    consistent as two boxes could calculate the manhattan distance with the
    same target, it is always admissible as it won't ever overestimate the
    cost to reach a solution. As this heuristic is also fast to compute, it
    is extremely interesting to implement it or a close derivative in our
    solution.
    \end{framed}
  \item Implement this problem. Extend the \textit{Problem} class and implement the necessary
methods and other class(es) if necessary. Your file must be named \textit{sokoban.py}.
You program must print to the standard output a solution to the sokoban instance
given in argument satisfying the described format. You will receive the name
of the instance and you have to read from the two files .init and .goal, for
instance, given instance instance1 you will read from files instance1.init
and instance1.goal .
    \begin{framed}
        \includegraphics[width=0.8\linewidth]{test_passed.png}
    \end{framed}
  \item Experiment, compare and analyze informed (\textit{astar\_graph\_search}) and unin-
formed (\textit{breadth\_first\_graph\_search}) graph search of aima-python3 on the 15
instances of sokoban provided. Report in a table the time, the number of
explored nodes and the number of steps to reach the solution. Are the
number of
explored nodes always smaller with \textit{astar\_graph\_search}, why?
When no solution can be found by a strategy in a reasonable time (say 5 min),
explain the reason (time-out and/or swap of the memory).
    \begin{framed}
    	\begin{tabular}{l|l|l}
    					& A* 		& Breadth-first\\
            \hline
    		sokoInst01 	& 9ms		& 180ms \\
    					& 71		& 2790 \\
    					& 15		& 15 \\
    		sokoInst02 	& 1s66		& 1s94 \\
    					& 20947 	& 30368 \\
    					& 65		& 65 \\
    		sokoInst07 	& 450ms		& 710ms \\
    					& 5311		& 9394 \\
    					& 98		& 98 \\
    		sokoInst08 	& 600ms		& 940ms \\
    					& 6679		& 13628 \\
    					& 108		& 90 \\
    		sokoInst15 	& 15s		& 68s27 \\
    					& 146013 	& 802991 \\
    					& 68		& 56 \\
    	\end{tabular}

        The number of explored with an A* graph search is always smaller or
        equals to the BFS, smaller if the heuristic is good and put on the
        worst cases a big weight ; equals if the heuristic is very bad.
    \end{framed}
  \item What are the performances of your program when you don’t perform dead state
detection?
    \begin{framed}
      \todo[inline]{Réponse à faire}
    \end{framed}
\end{enumerate}


\section*{IGO Preparation 2020}

\prob{}
{}{}{
    Let $ABC$ be an arbitrary triangle,$P$ is the intersection point of the altitude from
    $C$ and the tangent line from $A$ to the circumcircle. The bisector of angle $A$
    intersects $BC$ at $D$ . $PD$ intersects $AB$ at $K$, if $H$ is the orthocenter then
    prove : $HK\perp AD$
}

\prob{}
{}{}{
    We have $4$ circles in plane such that any two of them are tangent to each other. we
    connect the tangency point of two circles to the tangency point of two other circles.
    Prove that these three lines are concurrent.
}

\prob{}
{}{}{
    In an acute triangle $ABC$, points $D,E,F$ are the feet of the altitudes from $A,B,C$,
    respectively. A line through $D$ parallel to $EF$ meets $AC$ at $Q$ and $AB$ at $R$.
    Lines $BC$ and $EF$ intersect at $P$. Prove that the circumcircle of triangle $PQR$
    passes through the midpoint of $BC$.
}

\prob{}
{}{}{
    Given $\triangle ABC$ inscribed in $(O)$ an let $I$ and $I_a$ be it's incenter and
    $A$-excenter, respectively.

    Tangent lines to $(O)$ at $C,B$ intersect the angle bisector of $A$ at $M,N$,
    respectively. Second tangent lines through $M,N$ intersect $(O)$ at $X,Y$.

    Prove that $XYII_a$ is cyclic.
}

\prob{}
{}{}{
    Given triangle $ABC$, $D$ is the foot of the external angle bisector of $A$, $I$ its
    incenter and $I_a$ its $A$-excenter. Perpendicular from $I$ to $DI_a$ intersects the
    circumcircle of triangle in $A'$. Define $B'$ and $C'$ similarly. Prove that $AA',BB'$
    and $CC'$ are concurrent.
}


\subsection{Selection}

How to restrict variables? best subset. iterate through alol and test.

not feasible for large numbers of variables. forward selection, backward selection, L1 alternatives.

removing variable does: increase bias. may reduce variance of prediction


\chapter{PCA and Autoencoder}
\input{6DL/PCA}

\chapter{Proper orthogonal decomposition}
\input{6DL/POD}