\newpage
\subsection{Goal}

The useage of direct methods (one-dimensional methods of exhaustive search, dichotomy, golden section search; multidimensional methods of exhaustive search, Gauss, Nelder-Mead) in the tasks of unconstrained nonlinear optimization.

\subsection{Formulation of the problem}

\paragraph{I.}

Use the one-dimensional methods of \textit{exhaustive search}, \textit{dichotomy} and \textit{golden section} search to find an approximate (with precision $\varepsilon = 0.001$) solution $x: f(x) \rightarrow \min$  for the following functions and domains:

\begin{enumerate}
    \item $f(x) = x^3, x \in [0, 1]$;
    \item $f(x) = \lvert x - 0.2 \rvert, x \in [0, 1]$;
    \item $f(x) = x\sin{\frac{1}{x}}, x \in [0.01, 1]$.
\end{enumerate}

Calculate the number of $f$-calculations and the number of iterations performed in each method and analyze the results.
Explain differences (if any) in the results obtained.

\paragraph{II.}

Generate random numbers $\alpha \in (0, 1)$ and $\beta \in (0, 1)$.
Furthermore, generate the noisy data $\{x_k, y_k\}$, where $k = 0, ..., 100$, according to the following rule:

\begin{equation}
    y_k = \alpha x_k + \beta + \delta_k, x_k = \frac{k}{100},
\end{equation}

where $\delta_k \sim N(0, 1)$ are values of a random variable with standard normal distribution.
Approximate the data by the following linear and rational functions:

\begin{enumerate}
    \item $F(x, a, b) = ax + b$ (linear approximant),
    \item $F(x, a, b) = \frac{a}{1 + bx}$ (rational approximant),
\end{enumerate}

by means of least squares through the numerical minimization (with precision $\varepsilon = 0.001$ of the following function:

\begin{equation}
    D(a, b) = \sum^{100}_{k=0}(F(x_k, a, b) - y_k)^2.
\end{equation}

To solve the minimization problem, use the methods of exhaustive search, Gauss and Nelder-Mead.
If necessary, set the initial approximations and other parameters of
the methods.
Visualize the data and the approximants obtained in a plot \textbf{separately for each type of approximant}.
Analyze the results obtained (in terms of number of iterations, precision, number of function evaluations, etc.).

\subsection{Brief theoretical part}

Optimization methods are numerical methods for finding optimal (in some sense) values of objective functions, for example, in the framework of mathematical models of certain processes.

\textit{Direct optimization methods} (\textit{zero-order optimization methods}) use only the values of the function $f$ itself, but not its derivatives, when searching for $x*$.
These methods are particularly applicable for continuous (and not necessarily differentiable) functions $f = f(x)$ of a single variable $x$ on the segment $Q = [0, 1]$, but also have a low convergence rate.

\paragraph{Exhaustive search}

In the \textbf{exhaustive search} algorithm, tests are performed at points that are determined by evenly dividing the interval $[a, b]$ by $N$ identical subintervals.
The smallest value of the function $f(x)$ is selected from the calculated values.
Let this value be reached at point $x_k$.
Then, due to the unimodality of the function $f(x)$, the subintervals $[a, x_{k-1}]$, $[x_{k+1}, b]$ can be excluded from consideration, i.e., the interval $[x_{k-1}, x_{k+1}]$ can be made the next uncertainty interval.

\paragraph{Dichotomy method}

In the \textbf{dichotomy algorithm}, tests are performed in pairs.
The coordinates of each subsequent pair of tests are separated by the value $\delta_x < \varepsilon_x$, where $\varepsilon_x$ is the required accuracy of the solution.
Tests are performed in the middle of the interval.
For the values of $f(x)$ obtained at these points, one half of the interval is excluded from further consideration due to the unimodality of the function $f(x)$.
The value of $\delta_x$ is determined by the required accuracy of the solution.

\paragraph{Golden section.}

Consider the interval $[a, b]$.
A point $c$ is said to fulfill the Golden section of the interval $[a, b]$ if $\frac{c - a}{b - a} = \tau$, where $\tau = \frac{\sqrt{5} - 1}{2} \approx 0.618$ is the solution of the quadratic equation $\tau^2 + \tau - 1 = 0$.
Requires two times less computations of the function $f(x)$ than in the iteration method.

\paragraph{Gauss method.}

The essence of the method is to minimize the function along each of the coordinates in turn at each iteration.

\paragraph{Nelder-Mead.}

If the function $f(x)$ is a ravine function, the efficiency of the simplex method in solving problem:
\begin{equation}
    \min_{x \in R^n} f(x) = f(x*) = f*
\end{equation}
is significantly reduced due to the fact that the regular simplex cannot be "extended" along the ravine.
The Nelder-Mead method (the deformable polyhedron method) is a development of the simplex method and uses the deformation of the current simplex (not necessarily regular) in the search process.
The method uses the following operations on simplexes:

\begin{itemize}
    \item reflection;
    \item reduction;
    \item compression;
    \item stretching.
\end{itemize}

\subsection{Results}

For the first and second functions, the local minimum coincides with the global one, and the third function has many local minima.
However, the $x_{min}$ values for different methods and functions are the same.
The main difference between these methods is the number of iterations performed.
If for the dichotomy method and the Golden section method, the number of iterations is almost the same.
Then the exhaustive method goes through all the values of the function (with a step equal to $\varepsilon$), so the number of iterations is much larger.

\begin{enumerate}
    \item{
        $f(x) = x^3, x \in [0, 1]$:
        \begin{itemize}
            \item{\textit{exhaustive\_search}: count of function evaluations and iterations = (1000, 1000);}
            \item{\textit{dichotomy\_method}: count of function evaluations and iterations = (22, 11);}
            \item{\textit{golden\_section}: count of function evaluations and iterations = (17, 15).}
        \end{itemize}
    }
    \item{
        $f(x) = \lvert x - 0.2 \rvert, x \in [0, 1]$:
        \begin{itemize}
            \item{\textit{exhaustive\_search}: count of function evaluations and iterations = (1000, 1000);}
            \item{\textit{dichotomy\_method}: count of function evaluations and iterations = (22, 11);}
            \item{\textit{golden\_section}: count of function evaluations and iterations = (17, 15).}
        \end{itemize}
    }
    \item{
        $f(x) = x\sin{\frac{1}{x}}, x \in [0.01, 1]$:
        \begin{itemize}
            \item{\textit{exhaustive\_search}: count of function evaluations and iterations = (900, 900);}
            \item{\textit{dichotomy\_method}: count of function evaluations and iterations = (20, 10);}
            \item{\textit{golden\_section}: count of function evaluations and iterations = (17, 15).}
        \end{itemize}
    }
\end{enumerate}

It is known that optimization by \textit{linear approximation} has a unique solution, so these methods give similar optimal values for $a$ and $b$, regardless of the choice of initial approximations.
In the case of a \textit{rational approximation}, significant nonlinearities occur, so the result depends on the initial approximations. It is worth adding that inside the Gauss method, a brute-force algorithm was used, so the graphs for them are similar (Figure \ref{ris:direct}).

\begin{figure}[H]
    \center
    \includegraphics[width=\textwidth]{img/plot.png}
    \caption{Direct methods of optimization.}
    \label{ris:direct}
\end{figure}

\subsection{Conclusion}

In the course of the laboratory work, direct methods were implemented and analyzed within the problem of the unconstrained optimization problem.

\subsection{Appendix}

The source code is located \href{https://github.com/vanSultan/anal_dev_algo/tree/lab_02}{here}: \url{https://github.com/vanSultan/anal_dev_algo/tree/lab_02}.


\subsection{Strengths}
\begin{itemize}
  \item Using Gaussian surface and Gauss theorem in physical electromagnetism, we deduce the process of fungi wood decomposition in accordance with objective laws and mathematical logic.
  \item Based on \textit{Figure~1} and \textit{Figure~2} given in problem sheet, after performing the function fitting derivation, we carry out the fungi selection operation, which simplifies the overall model building process, and obtains more practical data.
  \item Starting from Single-group Logistic Model, we use Multi-groups Logistic Model to accurately and comprehensively describe the interactions between different species of fungi.
  \item In the case of insufficient time series data, we use the Gray Prediction Model to predict and analyze the survival of fungal species combination and the change trend of decomposition in different environmental conditions.
  \item From the perspective of the ecosystem, we carry out an ecologically significant analysis of the impact of fungal population biodiversity on the carbon cycle in authentic nature.
\end{itemize}


\section{Implementation of non-pharmaceutical interventions} \label{npi}
A major part of the rationale for the development of this model was to capture the past impact of non-pharmaceutical interventions (NPIs) and produce future scenarios projections with the implementation or release of such interventions.

\subsection{Isolation and quarantine}
For persons who are identified with symptomatic disease and enter clinical stratum 3, self-isolation is assumed to occur and their infectiousness is modified as described above. The proportion of ambulatory symptomatic persons passively identified through the public health response is determined by the case detection rate as described above in Section \ref{cdr}.

\subsection{Community quarantine or ``lockdown" measures}
For all NPIs relating to reduction of human mobility or “lockdown” (i.e. all NPIs other than isolation and quarantine), these interventions are implemented through dynamic adjustments to the age-assortative mixing matrix. The baseline mixing matrices have the major advantage of allowing for disaggregation of total contact rates by location, i.e. home, work, school and other locations. This disaggregation allows for the simulation of various NPIs in the local context by dynamically varying the contribution of each location to reflect the historical implementation of the interventions.

For each location $L$ (home, school, work, other locations) the age-specific contact matrix $\mathbf{C^L} = (c_{i,j}^L) \in \mathbb{R}_{+}^{16 \times 16}$ is defined such that $c_{i,j}^L$ is the average number of contacts that a typical individual aged $i$ has with individuals aged $j$. The original matrices from the United Kingdom are denoted $\mathbf{Q^L} = (q_{i,j}^L) \in \mathbb{R}_{+}^{16 \times 16}$, where $q_{i,j}^L$ is defined using the same convention as for $c_{i,j}^L$. The matrices $\mathbf{Q^L}$ were extracted using the R package ``socialmixr'' (v 0.1.8).

Let $\pi_j$ denote the proportion of people aged $j$ in the UK, and $\rho_j$ the proportion of people aged $j$ in Victoria. The contact matrices $\mathbf{C^L}$ were obtained from:
$$
c_{i,j}^L = q_{i,j}^L \times \frac{\pi_j}{\rho_j} .
$$

The overall contact matrix results from the summation of the four location-specific contact matrices: \(C_{0}=C_{H}+C_{S}+C_{W}+C_{L}\), where \(C_{H}\), \(C_{S}\), \(C_{W}\) and \(C_{L}\) are the age-specific contact matrices associated with households, schools, workplaces and other locations, respectively.

In our model, the contributions of the matrices \(C_{S}\), \(C_{W}\) and \(C_{L}\) vary with time such that the input contact matrix can be written:
\[C(t)= C_{H}+ s(t)^{2}C_{S}+ w(t)^{2}C_{W}+l(t)^{2}C_{L}\]

The modifying functions are each squared to capture the effect of the mobility changes on both the infector and the infectee in any given interaction that could potentially result in transmission. The modifying functions incorporate both macro-distancing and microdistancing effects, depending on the location.

\subsection{School closures/re-openings}
Reduced attendance at schools is represented through the function \(s(t)\), which represents the proportion of all students currently attending on-site teaching. If schools are fully closed, \(s(t)=0\) and \(C_{S}\) does not contribute to the overall mixing matrix \(C(t)\). \(s(t)\) is calculated through a series of estimates of the proportion of students attending schools, to which a smoothed step function is fitted. Note that the dramatic changes in this contribution to the mixing matrix with school closures/re-openings is a more marked change than is seen with the simulation of policy changes in workplaces and other locations (which are determined by empiric data and so do not vary so abruptly or reach a value of zero).

\subsection{Workplace closures}
Workplace closures are represented by quadratically reducing the contribution of workplace contacts to the total mixing matrix over time. This is achieved through the scaling term \(w(t)^{2}\) which modifies the contribution of \(C_{W}\) to the overall mixing matrix \(C(t)\). The profile of the function \(w(t)\) is set by fitting a polynomial spline function to Google mobility data for workplace attendance (Table \ref{tab:mobility_map}).

\subsection{Community-wide movement restriction}
Community-wide movement restriction (or ``lockdown") measures are represented by proportionally reducing the contribution of the other locations contacts to the total mixing matrix over time. This is achieved through the scaling term \(l(t)^{2}\) which modifies the contribution of \(C_{L}\) to the overall mixing matrix \(C(t)\). The profile of the function \(l(t)\) is set by fitting a polynomial spline function to an average of Google mobility data for various locations, as indicated in Table \ref{tab:mobility_map}.

\subsection{Household contacts}
The contribution of household contacts to the overall mixing matrix \(C(t)\) is fixed over time. Although Google provides mobility estimates for residential contacts, the nature of these data are different from those for each of the other Google mobility types. They represent the time spent in that location, as opposed to the other categories, which measure a change in total visitors rather than the duration. The daily frequency with which people attend their residence is likely to be close to one and we considered that household members likely have a daily opportunity for infection with each other household member. Therefore, we did not implement a function to scale the contribution of household contacts to the mixing matrix with time.

\begin{table}[ht]
\renewcommand{\baselinestretch}{1}
    \begin{tabular}{| p{4.4cm} | p{4.4cm} | p{5cm} |}
        \hline
        \textbf{location} & \textbf{Approach} & \textbf{Google mobility types} \\
        \hline
        School & Policy response & Not applicable \\
      \hline
      Household & Constant & Not applicable \\
      \hline
      Workplace & Google mobility & Workplace \\
      \hline
      Other locations & Google mobility &
      Unweighted average of: \begin{itemize}
			\item Retail and recreation
          \item Grocery and pharmacy
          \item Parks
          \item Transit stations
      \end{itemize}\\
      \hline
    \end{tabular}
    \title{Mapping of Google mobility data to contact locations.}
    \caption{\textbf{Mapping of Google mobility data to contact locations.}}
    \label{tab:mobility_map}
\end{table}

\subsection{Microdistancing}
\label{microdist}
Interventions other than those that prevent people coming into contact with one another are thought to be important to COVID-19 transmission and epidemiology, such as maintaining interpersonal physical distance and the wearing of face coverings. We therefore implemented a ``microdistancing” function to represent reductions in the rate of effective contact that is not attributable to persons visiting specific locations and so is not captured through Google mobility data. This microdistancing function reduces the values of all non-household contributions to the mixing matrices by a certain proportion. These time-varying functions multiplicatively scale the location-specific contact rate modifiers \(s(t)\), \(w(t)\) and \(l(t)\).


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath,amssymb}
\usepackage{csquotes}
\usepackage[pdfborder={0 0 0}]{hyperref}
\newcommand\set[1]{\left\{#1\right\}}
\newcommand\card[1]{\left|#1\right|}

\begin{document}
\section{Exercise 10}

\begin{verbatim}
#!/usr/bin/env python

import itertools

def experiment(n):
    distribution = [0] * (n + 1)
    for outcome in itertools.product({'L', 'R'}, repeat=2*n-1):
        bags = {'L': n, 'R': n}
        for out in outcome:
            bags[out] -= 1
            if bags[out] == 0:
                break

        print(outcome, bags)

        outcome_k = max(bags.values())
        distribution[outcome_k] += 1

    return distribution

print(experiment(5))
\end{verbatim}

For $n=5$, it gives $k=1$ in 140 cases, $k=2$ in 140 cases, $k=3$ in 120 cases, $k=4$ in 80 cases and $k=5$ in 32 cases.
See OEIS A172068 \enquote{Triangular array T(n,k) = the number of n-step one dimensional walks that return to the origin exactly k times.}.

\[ \Omega = \set{0,1}^{2n-k} \]
\[ \mathcal A = \mathcal P(\Omega) \]
\[ \mathbb P(A_k) = \frac{\card{A_k}}{\card{\Omega}} \]

$0$ represents a left drawing, $1$ represents a right drawing.

\[ \implies A_k = R_k + L_k \]

Regards $L_k$, there must be a $1$ at its tail.
Before that $(n+1)$ ones and $n-k$ zeros will be distributed on $2n - k - 1$ places.
\[ \implies {2n-k-1 \choose n-1} \]

The probability that right-sided box is chosen: ${1 \over 2}^{n-1}$
The probability that left-sided box is chosen: ${1 \over 2}^{n-k}$

In the last trial, we chose a right-sided box, so $\cdot \frac12$.
The probability that right-sided box is empty and left-sided box has
$k$ elements,
\[
  {2n - k - 1 \over n-1} \cdot {1 \over 2}^{n-1} {1 \over 2} {1 \ over 2}^{n-k} = {2n-k-1 \over n-1} \cdot {1 \over 2}^{2n-k}
\]
$L_k$ has same cardinality like $R_k$, hence times $2$.

\[
  \mathbb P(A_k) = 2 \cdot {2n-k-1 \choose n-1} {1 \over 2}^{2n-k} = {2n-k-1 \over n-1} \cdot {1 \over 2}^{2n-k-1}
\]

\end{document}


\documentclass{article}

\usepackage{fancyhdr}
\usepackage{extramarks}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amsfonts}
\usepackage[plain]{algorithm}
\usepackage{algpseudocode}
\usepackage{matlab-prettifier}
\usepackage{graphicx}
\usepackage[export]{adjustbox}
%
% Basic Document Settings
%
\lstMakeShortInline[style=Matlab-editor]"

\topmargin=-1in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.5in
\textheight=9.0in
\headsep=0.25in

\linespread{1.1}

\rhead{\firstxmark}
\lfoot{\lastxmark}
\cfoot{\thepage}

\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}

%
% Homework Details
%   - Title
%   - Due date
%   - Class
%   - Section/Time
%   - Instructor
%   - Author
%

\newcommand{\hmwkTitle}{AMATH 482 Homework 3: PCA}
\newcommand{\hmwkDueDate}{February 26, 2019}
\newcommand{\hmwkClassInstructor}{Professor Nathan Kutz}
\newcommand{\hmwkAuthorName}{\textbf{Skyler Hallinan}}

%
% Title Page
%

\title{
    \textmd{\textbf{\text{ } \hmwkTitle}}\\
}

\author{\hmwkAuthorName}
\date{}

\begin{document}
\maketitle

\section*{\fontsize{19}{15}\selectfont Abstract}
		We were given a four sets of three videos, where each set corresponded to a different oscillation scenario. Each of the three videos represented samples taken from a different location; the cameras were in different locations and observed the same movement from different angles. We performed principal component analysis on the four systems of oscillating data to convert the six variables (three sets of X and Y coordinates at $n$ points in time) into six principal components. We then looked at the energy content of each of these components, and projected the data onto significant principal components basis sets and compared them to the original oscillation behavior.
\section*{\fontsize{19}{15}\selectfont Introduction and Overview}
	We have taken videos of four different oscillatory situations, with different noise and rotation behavior. We are assuming that we have no underlying knowledge of the mechanics or equations behind this motion, and we are trying to experimentally determine how they function. For each of these four situations, we have somewhat oversampled; we have used three cameras at different locations to track the oscillatory motion of the bucket on a spring. There is a light on the bucket to track the movement, so we can convert the video of the bucket oscillating to $(x,y)$ coordinates by tracking the light in each frame, and add them to a data matrix, for all three cameras.
	Once we have this matrix of size $6 \times$ FrameNumber, we will need to perform SVD on it. \\ \\
The SVD will show us which principle components are the most important, and how much energy each principal component captures. We can also see how many principal component dimensions are significant to see if we can accurately reduce the dimension of our system to it's accurate number. We will ultimately remove noise and redundancy from this data by reducing its dimension.
	Although some videos may only need one principal component, such as simple harmonic motion going up and down, some will be better represented by having more principal components, such as test 4, which has a rotation factor that cannot be expressed in only one dimension. We will see how accurate our PCA will be in determining the number of significant principal components needed to express a system.
\section*{\fontsize{19}{15}\selectfont Theoretical Background}
	The Singular Value Decomposition is used to reformat a matrix into the following:
	\begin{align*}
\mathbf { A } = \mathbf { U } \boldsymbol { \Sigma } \mathbf { V } ^ { * } \text{ where} \\
\begin{array} { l } { \mathbf { U } \in \mathbb { C } ^ { m \times m } \text { is unitary } } \\ { \mathbf { V } \in \mathbb { C } ^ { n \times n } \text { is unitary } } \\ { \boldsymbol { \Sigma } \in \mathbb { R } ^ { m \times n } \text { is diagonal } } \end{array}
	\end{align*}
	The diagonal values of $\sigma$ are nonnegative and ordered from largest to smallest. We can also compute the SVD with:
\begin{align*}
\mathbf { A } ^ { T } \mathbf { A } & = \left( \mathbf { U } \boldsymbol { \Sigma } \mathbf { V } ^ { * } \right) ^ { T } \left( \mathbf { U } \boldsymbol { \Sigma } \mathbf { V } ^ { * } \right) \\ & = \mathbf { V } \boldsymbol { \Sigma } \mathbf { U } ^ { * } \mathbf { U } \boldsymbol { \Sigma } \mathbf { V } ^ { * } \\ & = \mathbf { V } \boldsymbol { \Sigma } ^ { 2 } \mathbf { V } ^ { * }  \\ \\
\mathbf { A } \mathbf { A } ^ { T } & = \left( \mathbf { U } \boldsymbol { \Sigma } \mathbf { V } ^ { * } \right) \left( \mathbf { U } \boldsymbol { \Sigma } \mathbf { V } ^ { * } \right) ^ { T } \\ & = \mathbf { U \Sigma V } ^ { * } \mathbf { V } \boldsymbol { \Sigma } \mathbf { U } ^ { * } \\ & = \mathbf { U } \boldsymbol { \Sigma } ^ { 2 } \mathbf { U } ^ { * } \\ \\
\mathbf { A } ^ { T } \mathbf { A V } & = \mathbf { V } \Sigma ^ { 2 } \\ \mathbf { A } \mathbf { A } ^ { T } \mathbf { U } & = \mathbf { U } \boldsymbol { \Sigma } ^ { 2 }
\end{align*}
Computing the normalized eigenvectors for the final two equations gives orthonormal basis vectors for $U$ and $V$. In addition, the square root of the eigenvalues of these equations produces the singular values $\sigma_j$. The SVD enables for every matrix to be diagonal if the proper bases for the domain and the range are used.
	One of the primary applications of the SVD is for Principal Component Analysis (PCA). The PCA allows for large, complex, and even somewhat random sets of data to be reduced to lower dimensions of dynamics without any knowledge of underlying behavior. This allows one to quantify low dimensional dynamics in systems like a mass spring system. We can place in the data in a matrix with $X \in \mathbb { R } ^ { m \times n }$, where $m$ is number of measurements and $n$ is number of data points taken over time. From this matrix, we must address two issues: noise in the data, which comes from a low signal to noise ratio SNR, as well as redundancy, which comes from oversampling from multiple cameras. \\ \\
	The covariance matrix will show correlations between all variables in the system; strong statistically dependent variables can be classified as redundant. Since we are looking for the covariance of a specific matrix, we have $\mathbf { C } _ { \mathbf { X } } = \frac { 1 } { n - 1 } \mathbf { X X } ^ { T }$, where $C_X$ is a square $m \times m$. This covariance matrix is key to understanding redundancies in data; large values correspond to redundancy. However, large diagonal terms also correspond to large variances, which suggest strong fluctuations in specific variables, and help identify important components. Large variances are thus dynamics of interest, while small variances are non-interesting. Thus, our goal is to view this $C_X$ matrix ordered from largest to smallest, with off-diagonal values of 0 (diagonilization).
	The SVD does this. Each singular direction in the SVD captures as much energy as possible, which is measured by the singular values $\theta_j$.  We know that SVD can diagnonalize any matrix via using the appropriate bases $U$ and $V$ as shown above. We define the transformed variable $Y = U*X$ where $U$ is the unitary transformation associated with the SVD ($\mathbf { X } = \mathbf { U } \boldsymbol { \Sigma } \mathbf { V } ^ { * }$). We then calculate the variance in Y:
 \begin{align*}
C_Y = \frac { 1 } { n - 1 } \mathbf { Y } \mathbf { Y } ^ { T }\\
= \frac { 1 } { n - 1 } \left( \mathbf { U } ^ { * } \mathbf { X } \right) \left( \mathbf { U } ^ { * } \mathbf { X } \right) ^ { T } \\
= \frac { 1 } { n - 1 } \mathbf { U } ^ { * } \left( \mathbf { X } \mathbf { X } ^ { T } \right) \mathbf { U } \\
= \frac { 1 } { n - 1 } \mathbf { U } ^ { * } \mathbf { U } \boldsymbol { \Sigma } ^ { 2 } \mathbf { U } \mathbf { U } ^ { * } \\
C_Y= \frac { 1 } { n - 1 } \mathbf { \Sigma } ^ { 2 }
\end{align*}

In general, PCA suggests that we are expanding our solution in another orthonormal basis, where we can diaganolize the system. We are normally given a function $f(x,t)$, which we must expand in a different basis representation such that $f ( x , t ) \approx \sum _ { j = 1 } ^ { N } a _ { j } ( t ) \phi _ { j } ( x )$. From this expansion, we can consider the energy in each principal component direction, which are the singular values $\sigma_j$. One note when we do SVA and PCA is that we must subtract the mean for each row $x_j$. PCA also assumes linearity, and that larger variances are more important. Finally, PCA may not always produce optimal results.

\section*{\fontsize{19}{15}\selectfont Algorithm Development}
	Our procedure was similar for all four cases. We first loaded in the three videos corresponding to different cameras. We started with one video. We found the number of frames for each one using the "size" command. We went through each of the frames of the video in a loop. For each frame, we converted it to grayscale using "rgb2gray" and "double" so we could analyze it. Since the bucket had a light attached to the top of it, from this grayscale image, in order to track where the bucket was at a specific point in time, we looked at the max values in the grayscale matrix corresponding to the image at a point of time. We created a threshold matrix where we found all locations where the value was greater than 250 (we had to lower this in some of the video case due to sparse data). Then, we used the $find$ function to locate the indeces of these points, and the $ind2sub$ function to output the exact coordinates of all these bright points. We then averaged these coordinates, and added this to our data array. This gave us a matrix of size $2 \times$FrameNumber. We repeated this with the other two videos, to get three matrices. \\ \\
	We had two issues: The videos had different frame numbers, and were not in sync. In order to make them in sync, we changed the first frame in each of the videos to be the frame corresponding with the lowest Y coordinate. Once we did this, we identified the shortest video (in terms of frame length), and trimmed the other two matrices so they had the same frame number. We then added this to one large matrix, such that we had 6 rows corresponding to $X$ and $Y$ coordinates taken from 3 videos, and a number of columns equal to the lowest frame number.\\ \\
	With this data matrix, we then subtracted the mean of each row from the data to center the data. We then used took the svd of the transpose of our data matrix, and divided by $\sqrt{n-1}$, using "svd(data'/sqrt(n-1)" to get three matrices, "[u,s,v]". We extracted the values of the diagonal matrix "s" and squared them to get our variances for each principal component. We then plotted normalized variances (divided by the sum) to see which principal components were significant. Finally, we plotted projections of our data on the significant principal component orthonormal bases to reconstruct our observed data. We repeated this procedure for all four sets of data.

\begin{figure}[H]
\begin{center}
\includegraphics[width = 8cm]{oscmotion1}
\includegraphics[width = 8cm]{energy1}
\caption{\label{fig:scaled_diss} (left)  Plots of original z displacement vs new basis (test1)}
\caption{\label{fig:scaled_diss} (right) Plot of variances of principal components (test 1)}
\end{center}
\end{figure}

\begin{figure}[H]
\begin{center}
\includegraphics[width = 8cm]{oscmotion2}
\includegraphics[width = 8cm]{energy2}
\caption{\label{fig:scaled_diss} (left) Plots of original z displacement vs new basis (test2) }
\caption{\label{fig:scaled_diss} (right) Plot of variances of principal components (test 2)}
\end{center}
\end{figure}

\section*{\fontsize{19}{15}\selectfont Computational Results}
	\textbf{(test 1) Ideal Case} \\
As seen in Figure 2, we saw that we had only that we only had one principal component which corresponded to a 91.78 \% energy capturing. The rest of the components had very low energy, which fits this video; we had a single bucket oscillating in one direction, so it should only need one principle component. In addition, our transformation of the first principal component resulted in accurate data. We also see that our projection onto the first principal component basis results in very similar data to what we experimentally measured. This shows that this system can be accurately reproduced and represented by one principal component. \\ \\
\textbf{(test 2) noisy case:}
Looking at Figure 4 in this case, we see that maybe two principal components could be considered significant; the second diagonal variance has a somewhat high value of 0.1946, while our first principal component has an energy captured of 0.6345. Thus, it seems that we have shown that two principal components are needed for this system, although it is the same as system 1, only with noise. Thus noise in our data has thrown off some of our PCA calculations. \\
We see that because there was so much noise in the system, our observed data consistently shows noise as it oscillates. This is also true in our projection to the principal component basis. However, although there is lots of noise in both systems, there is clearly oscillatory behavior. \\ \\
\textbf{(test 3) horizontal displacement}
We see that for our horizontal displacement in Figure 6, we see up to 4 principal components that seem to capture significant amounts of energy relative to eachother: We have values of the first four energy captured of 0.4671, 0.2489, 0.1475, 0.1213. Although there is a large drop off from the first principal component to the last, they are all relatively close together. \\ \\
\textbf{(test 4) horizontal displacement and rotation}
We see that for our energy captured in Figure 8, there seem to be 3 significant principal components that have high energy captured, with values of  0.6385, 0.2286, and 0.0910. This corresponds to the data we observed, as we had both oscillatory motion and spinning motion. It seems that PCA has captured the multi-dimensional nature of the horizontal displacement and rotation, as there is both a pendulum nature, as well as simple harmonic motion

\begin{figure}[H]
\begin{center}
\includegraphics[width = 8cm]{oscmotion4}
\includegraphics[width = 8cm]{energy4}
\caption{\label{fig:scaled_diss} (left) Plots of original z displacement vs new basis (test4)}
\caption{\label{fig:scaled_diss} (right) Plot of variances of principal components (test 4)}
\end{center}
\end{figure}

\begin{figure}[H]
\begin{center}
\includegraphics[width = 8cm]{oscmotion3}
\includegraphics[width = 8cm]{energy3}
\caption{\label{fig:scaled_diss} (left) Plots of original z displacement vs new basis (test 3) }
\caption{\label{fig:scaled_diss} (right) Plot of variances of principal components (test 3)}
\end{center}
\end{figure}

\section*{\fontsize{19}{15}\selectfont Summary and Conclusions}
We were able to track an oscillating bucket for different scenarios and perform PCA on our datasets to see how many significant, orthonormal, principal components existed. We were able to plot both the energy captured by specific nodes, as well as our original data projected onto the various basis sets of the principal components. We observe that our projections are accurate to some extent. We have seen that even without underlying system dynamics, we can experimentally determine the minimum number of principal components needed to represent a system, and reclassify redundant systems into lower order ones.

\pagebreak

\section*{\fontsize{19}{15}\selectfont Appendix A}
\subsection*{MATLAB functions used and implementation}
"diag(X)" : Returns a column vector of the diagonal values of a matrix. We used this to generate our variance numbers from SVD. \\ \\
"find(X)" : Returns a vector of non-zero indeces in a matrix. We used this in our conditional matrix to find locations in the image of bright spots, which corresponded to moving object. We then used in "ind2sub" to find direct coordinates.  \\ \\
"[i,j,k] = ind2sub(siz,IND)" : Given a index "IND", it returns the "i", "j", and "k" indeces, which can then be translated into spacial or frequency domain coordinates. We used this command to find the matrix coordinates of our bright spots of our black-white frames in the videos, which corresponded to the light on the bucket. \\ \\
"max(A)" : If "A" is a vector, then it returns the maximum value of this vector. We used this for normalization of data. \\ \\
"mean(A)" : Returns the mean of the vector "A". We used this to average the coordinates above the threshold value to locate the object.
"[M,I] = min(A)" : Returns the minimum value "M" and the index "I" of the array. We used this to find the minimum value in the first 20 frames of the video, to start at a minimum value. \\ \\
"plot3(X1,Y1,Z1)" : Displays a 3-dimensional plot of a set of datapoints through inputs "X1", "Y1", "Z1". We used this to visualize the path of the marble in the intestine after noise was removed and 20 data points were obtained in 3-dimensional space. \\ \\
"rgb2gray(X)" : Given a 3D image matrix "X", returns a 2D image matrix that corresponds to the grayscale version. We used this for image processing, to calculate the brightest spot on each frame for object tracking. \\ \\
"size(X)" : Returns the dimensions of the matrix "X". We used this to calculate frame numbers of our video files. \\ \\
"zeros(A,B,C)" : Creates a matrix filled with zeros of size "A x B x C". We used this to initialize an average matrix, where we stored and added 20 64x64x64 fourier transformed matrices together. \\ \\

\section*{\fontsize{19}{15}\selectfont Appendix B}
\subsection*{MATLAB code}
\begin{lstlisting}[style=Matlab-editor]
clear all; close all; clc;
load('cam1_1.mat')
load('cam2_1.mat')
load('cam3_1.mat')
%%
%Play Videos
numFrames1a =size(vidFrames1_1,4);
numFrames2a =size(vidFrames2_1,4);
numFrames3a =size(vidFrames3_1,4);

%%
for k = 1:numFrames1a
    mov1a(k).cdata = vidFrames1_1(:,:,:,k);
    mov1a(k).colormap = [];
end

%Play video
width = 50;
filter = zeros(480,640);
filter(300-2.6*width:1:300+2.6*width, 350-width:1:350+width) = 1;

data1 = [];
for j=1:numFrames1a
    X=frame2im(mov1a(j));

    Xabw = rgb2gray(X);
    X2 = double(X);

    Xabw2 = double(Xabw);
    Xf = Xabw2.*filter;
    thresh = Xf > 250;
    indeces = find(thresh);
    [Y, X] = ind2sub(size(thresh),indeces);

    data1 = [data1; mean(X), mean(Y)];

%     subplot(1,2,1)
%     imshow(uint8((thresh * 255))); drawnow
%     subplot(1,2,2)
%     imshow(uint8(Xf)); drawnow
end

%pause(5)
close all;
%%
for k = 1:numFrames2a
    mov2a(k).cdata = vidFrames2_1(:,:,:,k);
    mov2a(k).colormap = [];
end
width = 50;
filter = zeros(480,640);
filter(250-3*width:1:250+3*width, 290-1.3*width:1:290+1.3*width) = 1;

data2 = [];
%Play video
for j=1:numFrames2a
    X=frame2im(mov2a(j));

    Xabw = rgb2gray(X);
    X2 = double(X);

    Xabw2 = double(Xabw);
    Xf = Xabw2.*filter;
    thresh = Xf > 250;
    indeces = find(thresh);
    [Y, X] = ind2sub(size(thresh),indeces);

    data2 = [data2; mean(X), mean(Y)];

%     subplot(1,2,1)
%     imshow(uint8((thresh * 255))); drawnow
%     subplot(1,2,2)
%     imshow(uint8(Xf)); drawnow
end

%%
for k = 1:numFrames3a
    mov3a(k).cdata = vidFrames3_1(:,:,:,k);
    mov3a(k).colormap = [];
end

width = 50;
filter = zeros(480,640);
filter(250-1*width:1:250+2*width, 360-2.5*width:1:360+2.5*width) = 1;

data3 = [];
%Play video
for j=1:numFrames3a
    X=frame2im(mov3a(j));

    Xabw = rgb2gray(X);
    X2 = double(X);

    Xabw2 = double(Xabw);
    Xf = Xabw2.*filter;
    thresh = Xf > 247;
    indeces = find(thresh);
    [Y, X] = ind2sub(size(thresh),indeces);

    data3 = [data3; mean(X), mean(Y)];

%     subplot(1,2,1)
%     imshow(uint8((thresh * 255))); drawnow
%     subplot(1,2,2)
%     imshow(uint8(Xf)); drawnow
end

%%
[M,I] = min(data1(1:20,2));
data1  = data1(I:end,:);

[M,I] = min(data2(1:20,2));
data2  = data2(I:end,:);

[M,I] = min(data3(1:20,2));
data3  = data3(I:end,:);

%%
data2 = data2(1:length(data1), :);
data3 = data3(1:length(data1), :);

%% Method 2
alldata = [data1';data2';data3'];
%alldata = alldata';

[m,n]=size(alldata); % compute data size
mn=mean(alldata,2); % compute mean for each row
alldata=alldata-repmat(mn,1,n); % subtract mean

[u,s,v]=svd(alldata'/sqrt(n-1)); % perform the SVD
lambda=diag(s).^2; % produce diagonal variances

Y= alldata' * v; % produce the principal components projection

sig=diag(s);
%%
figure()
plot(1:6, lambda/sum(lambda), 'mo', 'Linewidth', 2);
title("Case 1: Energy of each Diagonal Variance");
xlabel("Diagonal Variances"); ylabel("Energy Captured");

figure()
subplot(2,1,1)
plot(1:218, alldata(2,:),1:218, alldata(4,:), 1:218, alldata(6,:), 'Linewidth', 2)
ylabel("Displacement (pixels)"); xlabel("Time (frames)");
title("Case 1: Original displacement across Z axis and XY-plane");
legend("Z", "XY")
subplot(2,1,2)
plot(1:218, Y(:,1),'r','Linewidth', 2)
ylabel("Displacement (pixels)"); xlabel("Time (frames)");
title("Case 1: Displacement across principal component directions");
legend("PC1")

clear all; close all; clc;
load('cam1_2.mat')
load('cam2_2.mat')
load('cam3_2.mat')
%%
%Play Videos
numFrames1a =size(vidFrames1_2,4);
numFrames2a =size(vidFrames2_2,4);
numFrames3a =size(vidFrames3_2,4);

%%
for k = 1:numFrames1a
    mov1a(k).cdata = vidFrames1_2(:,:,:,k);
    mov1a(k).colormap = [];
end

%Play video
width = 50;
filter = zeros(480,640);
filter(300-2.6*width:1:300+2.6*width, 350-width:1:350+2*width) = 1;

data1 = [];
for j=1:numFrames1a
    X=frame2im(mov1a(j));

    Xabw = rgb2gray(X);
    X2 = double(X);

    Xabw2 = double(Xabw);
    Xf = Xabw2.*filter;
    thresh = Xf > 250;
    indeces = find(thresh);
    [Y, X] = ind2sub(size(thresh),indeces);

    data1 = [data1; mean(X), mean(Y)];

% subplot(1,2,1)
% imshow(uint8((thresh * 255))); drawnow
% subplot(1,2,2)
% imshow(uint8(Xf)); drawnow
end

%pause(5)
close all;
%%
for k = 1:numFrames2a
    mov2a(k).cdata = vidFrames2_2(:,:,:,k);
    mov2a(k).colormap = [];
end
width = 50;
filter = zeros(480,640);
filter(250-4*width:1:250+4.5*width, 290-2.5*width:1:290+2.7*width) = 1;

data2 = [];
%Play video
for j=1:numFrames2a
    X=frame2im(mov2a(j));

    Xabw = rgb2gray(X);
    X2 = double(X);

    Xabw2 = double(Xabw);
    Xf = Xabw2.*filter;
    thresh = Xf > 249;
    indeces = find(thresh);
    [Y, X] = ind2sub(size(thresh),indeces);

    data2 = [data2; mean(X), mean(Y)];

%     subplot(1,2,1)
%     imshow(uint8((thresh * 255))); drawnow
%     subplot(1,2,2)
%     imshow(uint8(Xf)); drawnow
end

%%
for k = 1:numFrames3a
    mov3a(k).cdata = vidFrames3_2(:,:,:,k);
    mov3a(k).colormap = [];
end

width = 50;
filter = zeros(480,640);
filter(250-1*width:1:250+2.6*width, 360-2.5*width:1:360+2.7*width) = 1;

data3 = [];
%Play video
for j=1:numFrames3a
    X=frame2im(mov3a(j));

    Xabw = rgb2gray(X);
    X2 = double(X);

    Xabw2 = double(Xabw);
    Xf = Xabw2.*filter;
    thresh = Xf > 246;
    indeces = find(thresh);
    [Y, X] = ind2sub(size(thresh),indeces);

    data3 = [data3; mean(X), mean(Y)];

%     subplot(1,2,1)
%     imshow(uint8((thresh * 255))); drawnow
%     subplot(1,2,2)
%     imshow(uint8(Xf)); drawnow
end

%%
[M,I] = min(data1(1:20,2));
data1  = data1(I:end,:);

[M,I] = min(data2(1:20,2));
data2  = data2(I:end,:);

[M,I] = min(data3(1:20,2));
data3  = data3(I:end,:);

%%
data2 = data2(1:length(data1), :);
data3 = data3(1:length(data1), :);

%% Method 2
alldata = [data1';data2';data3'];
%alldata = alldata';

[m,n]=size(alldata); % compute data sizea
mn=mean(alldata,2); % compute mean for each row
alldata=alldata-repmat(mn,1,n); % subtract mean

[u,s,v]=svd(alldata'/sqrt(n-1)); % perform the SVD
lambda=diag(s).^2; % produce diagonal variances

Y= alldata' * v; % produce the principal components projection

sig=diag(s);
%%
figure()
plot(1:6, lambda/sum(lambda), 'mo', 'Linewidth', 2);
title("Case 2: Energy of each Diagonal Variance");
xlabel("Diagonal Variances"); ylabel("Energy Captured");

figure()
subplot(2,1,1)
plot(1:295, alldata(2,:), 1:295, alldata(1,:),'Linewidth', 2)
ylabel("Displacement (pixels)"); xlabel("Time (frames)");
legend("Z", "XY")
title("Case 2: Original displacement across Z axis and XY-plane (cam 1)");
subplot(2,1,2)
plot(1:295, Y(:,1),1:295, Y(:,2),'r','Linewidth', 2)
ylabel("Displacement (pixels)"); xlabel("Time (frames)");
title("Case 2: Displacement across principal component directions");
legend("PC1", "PC2")

clear all; close all; clc;
load('cam1_3.mat')
load('cam2_3.mat')
load('cam3_3.mat')
%%
%Play Videos
numFrames1a =size(vidFrames1_3,4);
numFrames2a =size(vidFrames2_3,4);
numFrames3a =size(vidFrames3_3,4);

%%
for k = 1:numFrames1a
    mov1a(k).cdata = vidFrames1_3(:,:,:,k);
    mov1a(k).colormap = [];
end

%Play video
width = 50;
filter = zeros(480,640);
filter(300-1.5*width:1:300+3*width, 350-1.5*width:1:350+2*width) = 1;

data1 = [];
for j=1:numFrames1a
    X=frame2im(mov1a(j));

    Xabw = rgb2gray(X);
    X2 = double(X);

    Xabw2 = double(Xabw);
    Xf = Xabw2.*filter;
    thresh = Xf > 250;
    indeces = find(thresh);
    [Y, X] = ind2sub(size(thresh),indeces);

    data1 = [data1; mean(X), mean(Y)];

% subplot(1,2,1)
% imshow(uint8((thresh * 255))); drawnow
% subplot(1,2,2)
% imshow(uint8(Xf)); drawnow
end

%pause(5)
close all;
%%
for k = 1:numFrames2a
    mov2a(k).cdata = vidFrames2_3(:,:,:,k);
    mov2a(k).colormap = [];
end
width = 50;
filter = zeros(480,640);
filter(250-3*width:1:250+3.5*width, 290-2.5*width:1:290+2.7*width) = 1;

data2 = [];
%Play video
for j=1:numFrames2a
    X=frame2im(mov2a(j));

    Xabw = rgb2gray(X);
    X2 = double(X);

    Xabw2 = double(Xabw);
    Xf = Xabw2.*filter;
    thresh = Xf > 249;
    indeces = find(thresh);
    [Y, X] = ind2sub(size(thresh),indeces);

    data2 = [data2; mean(X), mean(Y)];

%     subplot(1,2,1)
%     imshow(uint8((thresh * 255))); drawnow
%     subplot(1,2,2)
%     imshow(uint8(Xf)); drawnow
end

%%
for k = 1:numFrames3a
    mov3a(k).cdata = vidFrames3_3(:,:,:,k);
    mov3a(k).colormap = [];
end

width = 50;
filter = zeros(480,640);
filter(250-1.8*width:1:250+2.3*width, 360-2.5*width:1:360+2.7*width) = 1;

data3 = [];
%Play video
for j=1:numFrames3a
    X=frame2im(mov3a(j));

    Xabw = rgb2gray(X);
    X2 = double(X);

    Xabw2 = double(Xabw);
    Xf = Xabw2.*filter;
    thresh = Xf > 246;
    indeces = find(thresh);
    [Y, X] = ind2sub(size(thresh),indeces);

    data3 = [data3; mean(X), mean(Y)];

%     subplot(1,2,1)
%     imshow(uint8((thresh * 255))); drawnow
%     subplot(1,2,2)
%     imshow(uint8(Xf)); drawnow
end

%%
[M,I] = min(data1(1:20,2));
data1  = data1(I:end,:);

[M,I] = min(data2(1:20,2));
data2  = data2(I:end,:);

[M,I] = min(data3(1:20,2));
data3  = data3(I:end,:);

%%
data1 = data1(1:length(data3), :);
data2 = data2(1:length(data3), :);

%% Method 2
alldata = [data1';data2';data3'];
%alldata = alldata';

[m,n]=size(alldata); % compute data sizea
mn=mean(alldata,2); % compute mean for each row
alldata=alldata-repmat(mn,1,n); % subtract mean

[u,s,v]=svd(alldata'/sqrt(n-1)); % perform the SVD
lambda=diag(s).^2; % produce diagonal variances

Y= alldata' * v; % produce the principal components projection

sig=diag(s);
%%
figure()
plot(1:6, lambda/sum(lambda), 'mo', 'Linewidth', 2);
title("Case 3: Energy of each Diagonal Variance");
xlabel("Diagonal Variances"); ylabel("Energy Captured");

figure()
subplot(2,1,1)
plot(1:237, alldata(2,:),'Linewidth', 2)
ylabel("Displacement (pixels)"); xlabel("Time (frames)");
title("Case 3: Original displacement across Z axis and XY-plane");
subplot(2,1,2)
plot(1:237, Y(:,1), 1:237, Y(:,2), 1:237, Y(:,3),1:237, Y(:,4),'r','Linewidth', 2)
ylabel("Displacement (pixels)"); xlabel("Time (frames)");
title("Case 3: Displacement across principal component directions");
legend("PC1", "PC2", "PC3", "PC4")

clear all; close all; clc;
load('cam1_4.mat')
load('cam2_4.mat')
load('cam3_4.mat')
%%
%Play Videos
numFrames1a =size(vidFrames1_4,4);
numFrames2a =size(vidFrames2_4,4);
numFrames3a =size(vidFrames3_4,4);

%%
for k = 1:numFrames1a
    mov1a(k).cdata = vidFrames1_4(:,:,:,k);
    mov1a(k).colormap = [];
end

%Play video
width = 50;
filter = zeros(480,640);
filter(300-1.5*width:1:300+3*width, 350-1.5*width:1:350+2.4*width) = 1;

data1 = [];
for j=1:numFrames1a
    X=frame2im(mov1a(j));

    Xabw = rgb2gray(X);
    X2 = double(X);

    Xabw2 = double(Xabw);
    Xf = Xabw2.*filter;
    thresh = Xf > 247;
    indeces = find(thresh);
    [Y, X] = ind2sub(size(thresh),indeces);

    data1 = [data1; mean(X), mean(Y)];

% subplot(1,2,1)
% imshow(uint8((thresh * 255))); drawnow
% subplot(1,2,2)
% imshow(uint8(Xf)); drawnow
end

%pause(5)
close all;
%%
for k = 1:numFrames2a
    mov2a(k).cdata = vidFrames2_4(:,:,:,k);
    mov2a(k).colormap = [];
end
width = 50;
filter = zeros(480,640);
filter(250-4*width:1:250+3*width, 290-2.5*width:1:290+2.7*width) = 1;

data2 = [];
%Play video
for j=1:numFrames2a
    X=frame2im(mov2a(j));

    Xabw = rgb2gray(X);
    X2 = double(X);

    Xabw2 = double(Xabw);
    Xf = Xabw2.*filter;
    thresh = Xf > 249;
    indeces = find(thresh);
    [Y, X] = ind2sub(size(thresh),indeces);

    data2 = [data2; mean(X), mean(Y)];

%     subplot(1,2,1)
%     imshow(uint8((thresh * 255))); drawnow
%     subplot(1,2,2)
%     imshow(uint8(Xf)); drawnow
end

%%
for k = 1:numFrames3a
    mov3a(k).cdata = vidFrames3_4(:,:,:,k);
    mov3a(k).colormap = [];
end

width = 50;
filter = zeros(480,640);
filter(250-3*width:1:250+1*width, 360-1.8*width:1:360+2.9*width) = 1;

data3 = [];
%Play video
for j=1:numFrames3a
    X=frame2im(mov3a(j));

    Xabw = rgb2gray(X);
    X2 = double(X);

    Xabw2 = double(Xabw);
    Xf = Xabw2.*filter;
    thresh = Xf > 234;
    indeces = find(thresh);
    [Y, X] = ind2sub(size(thresh),indeces);

    data3 = [data3; mean(X), mean(Y)];

%     subplot(1,2,1)
%     imshow(uint8((thresh * 255))); drawnow
%     subplot(1,2,2)
%     imshow(uint8(Xf)); drawnow
end

%%
[M,I] = min(data1(1:20,2));
data1  = data1(I:end,:);

[M,I] = min(data2(1:20,2));
data2  = data2(I:end,:);

[M,I] = min(data3(1:20,2));
data3  = data3(I:end,:);

%%
data1 = data1(1:length(data3), :);
data2 = data2(1:length(data3), :);

%% Method 2
alldata = [data1';data2';data3'];
%alldata = alldata';

[m,n]=size(alldata); % compute data sizea
mn=mean(alldata,2); % compute mean for each row
alldata=alldata-repmat(mn,1,n); % subtract mean

[u,s,v]=svd(alldata'/sqrt(n-1)); % perform the SVD
lambda=diag(s).^2; % produce diagonal variances

Y= alldata' * v; % produce the principal components projection

sig=diag(s);
%%
figure()
plot(1:6, lambda/sum(lambda), 'mo', 'Linewidth', 2);
title("Case 4: Energy of each Diagonal Variance");
xlabel("Diagonal Variances"); ylabel("Energy Captured");

figure()
subplot(2,1,1)
plot(1:375, alldata(2,:),'Linewidth', 2)
ylabel("Displacement (pixels)"); xlabel("Time (frames)");
title("Case 4: Original displacement across Z axis and XY-plane");
subplot(2,1,2)
plot(1:375, Y(:,1), 1:375, Y(:,2), 1:375, Y(:,3), 'Linewidth', 2)
ylabel("Displacement (pixels)"); xlabel("Time (frames)");
title("Case 4: Displacement across principal component directions");
legend("PC1", "PC2", "PC3")

\end{lstlisting}

\end{document}


[30 min late, first zoom lecture]

\section{The Point on Infinity}
The transformation $w=1/z$ establishes a one-to-one correspondence between
points in the $z$-plane and there is the $w$-plane,
except for the singular point at $z=0$.
The resolve this,
we attach an ``improper point'' to the $w$-plane.
This point is called ``the point of infinity''
and is denoted by $\infty$.
The entire complex plane together with the point of infinity is called the
``extended complex plane''.

You need to know what it is and move on.

Certain functions are analytic in the entire complex plane,
they are \emph{entire functions}.
However, they need not be analytic at the point of infinity.
To check whether something is analytic at infinity,
you have to do this mapping $w=1/z$.
Once you do that mapping,
look at how the function behaves at $w=0$.

To check whether a function $g(z)$ is analytic at $\iinfty$,
substitute $w=1/z$ and check analyticity at $w=0$.

\begin{question}
    There is no number line,
    it's a complex plane.
    What does infinity mean in a complex plane?
\end{question}
It's not a simple thing.
The claim is that in the complex plane,
infinity can be represented by a single point,
which is not true in the real line where $+\infty$
is different to $-\infty$.
But in the complex plane,
there is only one $\infty$.
The only reason I am discussing is that you will hear this occasionally so you
just need to have some idea what people are talking about.
It's not like we're actually going to use it for anything.
If you want to know more,
you're going to have to do the reading yourself.
Infinity is a treacherous number,
it's not a simple thing.

Any other questions?
OK let's move on.

\section{Conformal mapping}
Mappings by analytic functions possess and important geometric property known as
\emph{conformal}.
A mapping in the plane is said to be ``conformal''
if it preserves the angle between any two curves,
both in magnitude and orientation.

Let me illustrate that with a picture.
Let's say this is the $z=x+iy$ plane,
with real part $x$ and imaginary part $y$.
Let's say that is the curve $C_1$
and that is the curve $C_2$.
They intersect at a point and
draw their tangents at this point.
Let's say there's an angle $\beta$ between the tangents of the curves at that
point.
Let's look at the map of these curves $z\to w = u + iv$.
Every point of the curve $C_1$ gets mapped to a curve in the $w$ plane $C_1'$.
Every point of the curve $C_2$ gets mapped to a curve in the $w$ plane $C_2'$.
Again, $C_1'$ and $C_2'$ still intersect at a point.
The claim is that the angle between the tangents of the curves in the map is the
same as before, $\beta$.
So yeah.

In other words,
if the mapping is conformal,
the angle between the curves is preserved.
And the claim is that if you do a mapping via an analytic function,
it's automatically conformal.
Mappings with analytic functions are automatically conformal.
It's guaranteed with one caveat I will explain in a moment.

We want to show that the mapping $w=f(z)$
is conformal at every point where $f(z)$ is analytic
except where the derivative $f'(z)=0$.
A point where $f'(z)=0$ is called a \emph{critical point}.

Consider a small change $\Delta z$ at a point $z$.
Then the derivative is
\begin{align}
    \frac{df}{dz} = \frac{dw}{dz}
    = \lim_{\Delta z\to 0} \frac{\Delta w}{\Delta z}
\end{align}
taking the argument which is the angle
\begin{align}
    \arg \lim_{\Delta z\to 0} \frac{\Delta w}{\Delta z}
    = \lim_{\Delta z\to 0} \arg \Delta w
    - \lim_{\Delta z\to 0} \arg \Delta z
    = \arg \frac{df}{dz} =: \alpha
\end{align}
which is a constant for a given point.

Take the ratio of two complex numbers and take the argument to get the angle
between them.

Any line that passes through $z_0$
has its image rotated by an angle $\alpha$.
That means if you had two lines passing through $z_0$,
they would both be rotated by the same angle $\alpha$.
Hence for any pair of lines through $z_0$,
their images in the $w$-plane are rotated through the same angle.
Hence the angle between the lines is the same as in the $z$-plane.

Remember,
you have two curves $C_1$ and $C_2$,
both of which get rotated by the same angle.

You might remember there was a caveat.
At points $z$ where $f'(z)=0$,
the angle $\alpha$ is not well-defined,
and the argument fails.

That's the caveat.
Any analytic function where there is a region where the derivative doesn't
vanish,
if you have this property of conformality,
the image of any curves is rotated by a fixed angle at that point,
so the angle between pairs of curves is preserved.

\begin{question}
    What do you mean by orientation?
\end{question}
What I mean is that you are rotated by this angle $\alpha$ in the same
direction.

This always ends up taking a bit longer than I expect.

So we agree that the angle is preserved.
But what about relative sizes?

From the definition of a derivative,
we have
\begin{align}
    \lim_{z\to z_0}
    \underbrace{\left|
        \frac{f(z) - f(z_0)}{z - z_0}
    \right|}_{
        = \frac{|f(z) - f(z_0)|}{|z - z_0|}
    }
    =
    |f'(z_0)|
\end{align}
Let's think about what this means.
$|z - z_0|$ is the length of $\Delta z$
and $|f(z) - f(z_0)|$ is the length between the image points.
So in the limit $\Delta z\to 0$,
the denominator is the distance in the $z$ plane,
but the numerator is the distance between the same two points in the $w$ plane.
That means there's a rescaling between the points and the images.
So if you have a distance between two points,
that distance is rescaled by a factor
which only depends on $z_0$.

That means if you have a curve that passes through a point,
in the neighbourhood of hat point,
the image is rescaled by that factor.
If you have multiple curves passing through that point,
all the curves are rescaled by that factor.
You also know they're rotated.
So every cure is rotated by some amount
and rescaled by some amount
in that neighbourhood.

I understand it's not easy to observe these things.
If I write everything down,
I can go on and on,
so let me explain again from scratch.

We want $z-z_0$ to go infinitesimal.
The denominator is the distance between two points in the $z$-plane.

Let me write this.
The mapping $w=f(z)$ rescales the lengths of infinitesimal lines passing through
$z_0$ by the factor $|f'(z_0)|$.
Therefore the image of a small figure always has the same shape as the original
figure,
even though it has been rescaled and rotated.
However, a large figure may have an image with a completely different shape.

\section{Conformal Mapping and Harmonic Functions}
OK so here is the claim,
which I proved in my notes,
but the proof is tedious and I wonder if I want to do it.
I will state the claim and give and outline of the proof,
and you can have the pleasure of filling in the proof.

\begin{theorem}
    A harmonic function of two variables $(x, y)$
    remains harmonic under a change of variables from $(x, y)$
    to $(u, v)$ if $z = x + iy$ and $w= u + iv$
    are related by a conformal transformation.
\end{theorem}
I'll explain in a moment what that means.

OK so here's the idea.
Let $h$ be a harmonic function.
What that means is
\begin{align}
    \frac{\partial^2 h}{\partial x^2}
    + \frac{\partial^2 h}{\partial y^2} = 0.
\end{align}
The claim is that if $u=u(x, y)$ and $v=v(x, y)$
such that $w = f(z)$,
$w=u+iv$ and $z=x+iy$.
Then
\begin{align}
    \frac{\partial^2 h}{\partial u^2}
    + \frac{\partial^2 h}{\partial v^2} =0
\end{align}
provided $f'(z)\ne 0$.

This is a very useful result,
used to prove lots of results in fluid dynamics and electrostatics.
You're going to apply this to electrostatics.
Unfortunately,
it's a trick that only works in 2D.
Unfortunately it doesn't help you solve Laplace's equation in arbitrary
dimensions,
but it is a powerful method in 2D.

I look at my notes and I actually did prove this.
Let me just outline how it goes.
The proof is kind of like brute force.
There are more clever proofs in the books I suggested,
but all of them rely on some additional assumptions or rely on some theorem I
didn't prove.

\begin{proof}
    Write using the chain rule
    \begin{align}
        \frac{\partial h}{\partial x} &=
        \frac{\partial h}{\partial u} \frac{\partial u}{\partial x}
        + \frac{\partial h}{\partial v} \frac{\partial v}{\partial x}
    \end{align}
    then differentiate again
    \begin{align}
        \frac{\partial^2 h}{\partial x^2} &=
        \left\{
            \frac{\partial^2 h}{\partial u^2}
            \left(\frac{\partial u}{\partial x} \right)^2
            +
            \frac{\partial^2 h}{\partial u \partial v}
            \left( \frac{\partial u}{\partial x} \right)
            \left( \frac{\partial v}{\partial x} \right)
            + \frac{\partial h}{\partial u}
            \frac{\partial^2 u}{\partial x^2}
        \right\}\\\nonumber
        &\qquad +
        \left\{
            \frac{\partial^2 h}{\partial v^2}
            \left(\frac{\partial v}{\partial x} \right)^2
            +
            \frac{\partial^2 h}{\partial u \partial v}
            \left( \frac{\partial u}{\partial x} \right)
            \left( \frac{\partial v}{\partial x} \right)
            + \frac{\partial h}{\partial v}
            \frac{\partial^2 v}{\partial x^2}
        \right\}
    \end{align}
    Similarly,
    \begin{align}
        \frac{\partial^2 h}{\partial y} = \cdots
    \end{align}
    and it gets messy\ldots
    Then you have to use the Cauchy-Riemann equations to cancel out some terms.
    Then you use the fact that $u$ and $v$ are harmonic functions themselves.
    After a lot of algebra,
    you get
    \begin{align}
        \frac{\partial^2 h}{\partial x^2} +
        \frac{\partial^2 h}{\partial y^2} &=
        \left(
            \frac{\partial^2 h}{\partial u^2}
            + \frac{\partial^2 h}{\partial v^2}
        \right)
        |f'(z)|^2
        = 0
    \end{align}
    where we need the assumption $f'(z)\ne 0$.
    Then
    \begin{align}
        \frac{\partial^2 h}{\partial u^2}
        + \frac{\partial^2 h}{\partial v^2}
        = 0
    \end{align}
\end{proof}


\documentclass{article}

\usepackage{stmaryrd}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{relsize}
\usepackage{bm}
\usepackage{IEEEtrantools}
\usepackage{graphicx}
\usepackage[font={small,it}, width=\textwidth]{caption}
\usepackage{subcaption}
\usepackage{hyperref}
\usepackage{cases}
\usepackage{xfrac}
\usepackage{comment}
\usepackage{framed}
\usepackage[ ddmmyyyy ]{datetime}
\usepackage{fancyhdr}
\usepackage{enumitem}
\usepackage{cite}
\usepackage{float}
\usepackage{multirow}

\newcommand{\source}[1]{\caption*{\hfill Source: {#1}} }

\usepackage{mathtools}
\DeclareMathOperator{\sign}{sign}
\DeclareMathOperator{\sat}{sat}

\oddsidemargin = 20pt
\textwidth = 420pt

\hypersetup{
     colorlinks   = true,
     linkcolor    = blue
}

\title{Dual Motor Control}
\author{Marcus Greiff}

\begin{document}
\maketitle

\section{Introduction}

\subsection{Cart model}
The most basic cart model consists of simply setting up the equations of motion for the cart with the motor torques as input signals,
\begin{equation}\label{eq:motion}
M\ddot{x} = \frac{u_1+u_2}{d} - F_f(\dot{x}).
\end{equation}
Here, the control signals $u_1$ and $u_2$ are the torques generated by the two motors, $d$ is the cart wheel radius and $M$ is the cart mass. In this simple linear model, the friction force, $F_f(\dot{x}) = b\dot{x}$, is simply proportional to the cart speed with some constant $b$. By introducing the state variables $\mathbf{x} = [x, \dot{x}]^T$ and letting the control signal $\mathbf{u} = [u_1, u_2]^T$, the system can be written as
\begin{equation}\label{eq:linModelCart}
\dot{\mathbf{x}} = \mathbf{A}_c\mathbf{x} + \mathbf{B}_c\mathbf{u} =
\begin{bmatrix}
0 & 1\\
0 & b/M
\end{bmatrix}\mathbf{x}  +
\frac{1}{Md}
\begin{bmatrix}
0 & 0\\
1 & 1
\end{bmatrix}\mathbf{u},
\end{equation}
This state space representation was used to derive certain controllers, but does not take the many non-linearities of the system into account, most notably the backlash, motor torque saturation and non-linear friction. In the original model implementation (see Figure~\ref{fig:SimModel}), the linear dynamics were implemented separate to the non-linearities in order to investigate their effect separately. However, in the open source project, a simpler linear cart was implemented to show the effect of the non-linear torque splitting and how the cascade control loop is constructed (see \texttt{DMC.slx}).

\begin{figure}[htbp]
\centering
\includegraphics[width=0.9\textwidth]{figures/CartSimulink.png}
\rule{35em}{0.5pt}
\caption{The Simulink cart model with motor torque saturation (blue), the backlash model (red), the non-linear friction model (green) and the linear cart dynamics (purple).}
\label{fig:SimModel}
\end{figure}

\subsection{Backlash compensation}
As this work aspires to be reused in parallel kinematics projects, the complete dynamics of the final robot will be very different from that of a simple cart. In order for the backlash compensation to be reusable, it should therefore be largely independent of system dynamics. Consequently, approaches such as those using state-dependent algebraic Ricatti equation (SDARE) method cannot be used~\cite{friedland1997feedback}. Instead, a simple non-linear filter is proposed to (i) operate the motors synchronously at large reference offsets to improve the system response, and (ii) to operate the two motors in the opposing directions when the reference offset is small to completely get rid of backlash and improving accuracy in stationarity. This method of compensation is more economical in terms of power consumption than the method described in a Cairen's thesis on DMC control ~\cite{Patrik:2013}, where the motors were working in opposing directions at all times.

The non-linear filter exploits the fact that the effective force applied to the cart, $u$, is proportional to the sum of the motor torques, $T_a$ and $T_b$. As long as there exists good torque control of the motors and the controllers are tuned equally, we may perturb the motor torques slightly by some constant, $\epsilon_{bw}$, and still achieve the same effective force
\begin{equation}\label{eq:splitEq}
\frac{u_1 + u_2}{d}  = \frac{\sat(T_a) + \sat(T_b)}{d} = \frac{\sat(T_a +  \epsilon_{bw}) + \sat(T_b - \epsilon_{bw})}{d}.
\end{equation}
This statement holds only when absolute value of the perturbed torque reference is within the bounds of saturation, but proves no issue for our application, as $\epsilon_{bw} << \max_t(|T_a|,|T_b|)$ and the absolute saturation limit can be lowered by $\epsilon_{bw}$ to guarantee that~\eqref{eq:splitEq} holds. As we only include feed forward terms of acceleration, velocity and position, we denote the $i^{th}$ derivative of the control error by $e^{(i)}(t) = x^{(i)}(t)  - x^{(i)}_{ref}(t)$ and let $\tilde{e}(t) = \max_i(|e^{(i)}(t)|)$ with $r = 2$. Using equation~\eqref{eq:splitEq}, we then demand that $T_a = T_b$ when the control error is above some threshold $\tilde{e}(t) > \epsilon_{th}$, and that reference torques are only split continuously for smaller control errors, such that $\tilde{e}(t) \equiv 0\Rightarrow T_a = -T_b \Rightarrow u_1 + u_2 = \epsilon_{bw}-\epsilon_{bw} = 0$. With this approach, both conditions (i) and (ii) will be satisfied, eliminating backlash completely when all control error derivatives are small. In summary the filter function, $f(u,e)$, has to be chosen so that $f(u,0) = \epsilon_{th}$ and $f(u,\epsilon_{bw}) = 0$, and general filter can then be written
\begin{equation}
h(u,e) =
\begin{cases}
\tilde{e}(t) = \max\limits_{i = 0,...,r}(|e^{(i)}(t)|) \quad r\in\mathbb{N}_0\\
\frac{1}{2}\{u, u\} \qquad\qquad\qquad\quad\;\; \text{if}\; |\tilde{e}(t)| > \epsilon_{th}\\
\frac{1}{2}\{u \pm f(u,\tilde{e})\} \qquad\quad\quad\;\;\text{if}\; |\tilde{e}(t)| \leq \epsilon_{th}\\
\end{cases}
\;\;\text{where}\;\;\epsilon_{th},\epsilon_{bw}\in \mathbb{R}_+,\;\; \epsilon_{th}\leq\epsilon_{bw}
\end{equation}
An initial linear filter function,
\begin{equation}
f_1(u,\tilde{e}) = \epsilon_{bw} \cdot (\epsilon_{th}  - u \cdot \sign(\tilde{e})) \in C^1,
\end{equation}
was implemented in Simulink and simulated with good results, but behaved strangely with certain control schemes (notably second order sliding mode control) due to it not having a second derivative. To remedy this, the conditions $f^{\prime}(0) = f^{\prime}(\epsilon_{bw}) = 0$ were enforced, and a trigonometric function was used instead of a linear one (see Figure~\ref{fig:filter}), making the filter smooth and continuous in the $n^{th}$ derivative,
\begin{equation}
f_2(u,\tilde{e}) = \frac{\epsilon_{bw}}{2}\Big(1+\cos\Big(\frac{\pi \tilde{e}}{\epsilon_{th}}\Big)\Big) \in C^N.
\end{equation}
Note that this ``$C^N$-filter" can be implemented such that the control error is with reference to position, velocity, acceleration or a combination of the three.

\begin{figure}[htbp]
\centering
\includegraphics[width=0.7\textwidth]{figures/CN_filter.png}
\includegraphics[width=0.9\textwidth]{figures/CNFILT.png}
\rule{35em}{0.5pt}
\caption{\textbf{Top:} The simulink implementation of the  $C^N$-filter with $r=2$. \textbf{Bottom:} The $C^N$-filter with $r=0$ applied to the input $u(t) = 8e^{-t}cos(2\pi t)$ (blue), with the outputs $T_a$ (red) and $T_b$ (green), and the sum of the outputs (black). The splitting is done when the positional error is small, and approaches a stationary split $T_a=-T_b$ when $e\approx 0$.}
\label{fig:filter}
\end{figure}

\subsection{Trajectory planning}
The trajectory planning has two separate functions. The first is to simply to take reference function of position, velocity or acceleration, and then generate the signals required in the feed-forward loop. For example, if a sinusoidal velocity reference is to be followed, the trajectory planner should evaluate and output velocities in time, as well as it's derivative (acceleration) and primitive (position). The second mode of operation is to detect sudden reference changes in position ( steps), and then approximate a fourth degree reference polynomial which moves the cart to the required position in optimal time given constraints on velocity, $\hat{v}$, acceleration, $\hat{a}$, and jerk, $\hat{j}$, and jerk derivative, $\hat{d}$. The first problem is trivial, but the second is more difficult can be formulated mathematically as
\begin{equation}\label{eq:optProb}
\text{Minimize}(t_f)\;\;\text{ subject to}\;\;\\
\begin{cases}
x(t_0) = r_0, x(t_f) = r_f\\
\dot{x}(t_0) = \ddot{x}(t_0) = x^{(3)}(t_0) = x^{(4)}(t_0)=0\quad (*)\\
\dot{x}(t_f) = \ddot{x}(t_f) = x^{(3)}(t_f) = x^{(4)}(t_f) = 0 \\
|x^{(4)}(t)|<\hat{d},|x^{(3)}(t)| < \hat{j}, |\ddot{x}(t)|<\hat{a}, |\dot{x}(t)| <\hat{v} \quad(**)\\
\end{cases}
\end{equation}
by defining the time of a change in reference as $t_0 = 0$, and the time at which the new reference position is reached as $t_f > t_0$. Here, $r_0$ is the positional reference before $t_0$ and $r_f$ is the desired position. We also require the jerk and all lower derivatives to be continuous at all times. In future iterations, it is worth considering setting the derivatives at $t_0$ as free variables $(*)$, to properly allow the computation of new reference steps while the cart is moving. Here we make the assumption of $(*)$ and use the symmetry of the problem, to which the general solution is a set of polynomial splines of order $p\leq4$,
\begin{equation}\label{eq:gensol}
\begin{cases}
x^{(4)}(t) = d_i\\
x^{(3)}(t) =  d_it+j_i\\
\ddot{x}(t) = \dfrac{{d_i}}{2}t^2 + {j_i}t + a_i\\
\dot{x}(t) = \dfrac{{d_i}}{6}t^3 + \dfrac{{j_i}}{2}t^2 + {a_i}t + v_i\\
x(t) = \dfrac{{d_i}}{24}t^4 + \dfrac{{j_i}}{6}t^3 + \dfrac{{a_i}}{2}t^2 + {v_i}t + x_i.
\end{cases}
\end{equation}
These splines exist on a time interval $[t_i,t_{i+1}]$, and are defined as sets $S_i = \{d_i, j_i, a_i, v_i, x_i,t_i, t_{i+1}\}$, and the idea is to start simple and then introduce the constraints $(**)$ one by one. Due to the boundary conditions on velocity, the acceleration has to be zero at $t=t_f/2$, which implies that the jerk has to be zero at  $t = t_f/4,2t_f/4,3t_f/4$. It is then clear that the jerk derivative must be alternating between $x^{(4)}(t)=\pm\hat{d}$ on eight equidistant time intervals, $t_d$.
The jerk derivative in each spline $S_i$ is then given by the corresponding element in the vector
\begin{equation}\label{eq:djerksol}
\mathbf{d} = \sign(r_f - r_0)\cdot\hat{d}\cdot[1,-1,-1,1,1,-1,1]\\
\end{equation}
where
\begin{equation}\label{eq:djerkref}
t_d=\sqrt[4]{\dfrac{|r_f - r_0|}{8\hat{d}}},
\end{equation}
is determined from~\eqref{eq:gensol} and the remaining constants are chosen to preserve continuity. When adding the remaining constraints $(**)$, we need to introduce periods of constant jerk of length $t_j$, constant acceleration of length $t_a$ and constant velocity of length $t_v$, such that the constraints are preserved. The general solution is then
\begin{equation}
\begin{cases}\label{eq:fullsol}
\mathbf{d} = \sign(r_f - r_0)\cdot\hat{d}\cdot[1,0,-1,0,-1,0,1,0,1,0,-1,0,1]\\
\mathbf{t} =[t_d,t_j,t_a,t_j,t_d,t_v,t_d,t_j,t_a,t_j,t_d,]
\end{cases}
\end{equation}
where the jerk derivative and time interval length in each spline $S_i$ is given by the corresponding element in $\mathbf{d}$ and $\mathbf{t}$ respectively. The time interval lengths and splines are then determined in five steps.

\subsubsection*{Step 1}
We start by finding $t_d$, as the maximum time that the jerk derivative can be kept zero separate without violating $(**)$, which is given by
\begin{equation}\label{eq:td}
t_{d} = \min\begin{Bmatrix}\sqrt[4]{\dfrac{|r_f - r_0|}{8\hat{d}}}, \sqrt[3]{\dfrac{\hat{v}}{2\hat{d}}}, \sqrt{\dfrac{\hat{a}}{\hat{d}}},\dfrac{\hat{j}}{\hat{d}}\end{Bmatrix}
\end{equation}
as the initial conditions on the derivative terms are $\mathbf{x}(t_0) = \mathbf{0}$. Now, if $t_d$ depends on the first term (i.e. the index of the minimum value in ~\eqref{eq:td} is 1), we violate a constraint on jerk derivative and nothing else, and the solution takes the form of ~\eqref{eq:djerksol} described above with $t_j=t_a=t_v=0$. If the jerk is first violated (index 2), we need to introduce periods of constant jerk, shown in \textbf{Step 2}. If the acceleration is first violated (index 3), $t_j = 0$, and we proceed with \textbf{Step 3}. Finally, if the constraint on velocity is first violated (index 4), $t_j=t_a=0$, and we compute a period of constant velocity in \textbf{Step 4}.

\subsubsection*{Step 2}
According to~\eqref{eq:gensol} and~\eqref{eq:fullsol}, the maximum $t_j$ at which the constraint $|x^{(3)}(t)| < \hat{j}$ is still satisfied, is given by the real and positive solution to
\begin{equation}\label{eq:tjj}
t_j^3 + 5t_dt_j^2 + 8t_d^2t_j + 4t_d^3-\dfrac{|r_f - r_0|}{2\hat{d}t_d} = 0.
\end{equation}
This choice of $t_j$ ensures that we arrive at the correct terminal point, $x(t_f) = r_f$. However, we must also check if the constraint on acceleration and velocity is violated. By examining~\eqref{eq:gensol} and~\eqref{eq:fullsol}, these constraints are violated exactly when
\begin{equation}\label{eq:tja}
\sup\limits_{t_0\leq t\leq t_f}|\ddot{x}(t)| = \hat{a} \Rightarrow t_j = \dfrac{\hat{a}- \hat{d}t_d^2}{\hat{j}}.
\end{equation}
and for the velocity
\begin{equation}\label{eq:tjv}
\sup\limits_{t_0\leq t\leq t_f}|\dot{x}(t)| = \hat{v} \Rightarrow
\hat{j}(t_d + t_j)t_a + \hat{d}(t_d^3 + t_d^2t_j)+ \hat{j}(2t_jt_d + t_j^2+t_d^2)-\hat{v} =0
\end{equation}
The smallest real solution $tj>0$ to ~\eqref{eq:tjj}~\eqref{eq:tja}~\eqref{eq:tjv} is chosen so that the solution complies with all constraints.
If this $t_j$ is given by the constraint on jerk ~\eqref{eq:tjj}, we arrive at our terminal point keeping all constraints in $(**)$ and we are done.
If $t_j$ is given by the constraint on acceleration ~\eqref{eq:tja}, we proceed with \textbf{step 3}, and if the constraint on velocity~\eqref{eq:tjv} is violated first, $t_a = 0$ and we move straight to \textbf{Step 4}.

\subsubsection*{Step 3}
A period of constant acceleration, $t_a$, is computed such that the terminal point $x(t_f) = r_f$. This time is given by the real an positive solution to
\begin{equation}
(t_d^2+t_dt_j)t_a^2+
(6t_d^3+9t_d^2t_j+3t_dt_j^2)t_a+
8t_d^4+16t_d^3t_j+10t_d^2t_j^2+2t_dt_j^3-\dfrac{|r_f - r_0|}{\hat{d}} = 0
\end{equation}
as derived from equation~\eqref{eq:gensol} and~\eqref{eq:fullsol}. However, we might then violate the velocity constraint, and must therefore compute the $t_a$ at which the maximum velocity is reached. This is done by solving~\eqref{eq:tjv} for $t_a$. Just as in \textbf{Step 2}, if the constraint on velocity is not breached, the algorithm arrives at the terminal point and we are done. If the velocity constraint is violated (i.e. $t_f$ is given by~\eqref{eq:tjv}) we proceed with \textbf{Step 4}.

\subsubsection*{Step 4}
Here, a period of constant velocity, $t_v$, is introduced. No $t_v>0$ can violate any of the constraints in $(**)$ since the speed is constant during $t_f$, and the period time is simply computed so that  $x(t_f) = r_f$. This can either be done similarly to $t_j$ and $t_a$, by finding $t_v$ analytically from~\eqref{eq:gensol} and~\eqref{eq:fullsol}. In the implementation, this is done automatically in \textbf{Step 5} by checking the end point position, $x_p$, and velocity, $v_p$, of the 7th spline and computing $t_v = x_p/v_p$.

\subsubsection*{Step 5}
In this final step, the splines are created based on $~\eqref{eq:fullsol}$ to preserve continuity in all derivatives between the splines. In doing so, a complete trajectory can be generated in the cycle where the positional reference step is taken (see the function block \texttt{computeTrajectory}), and then evaluated at a given point in time (see the function block \texttt{evaluateTrajectory}). This approach was taken to decrease the computational effort and allow a higher cycle frequency.

\begin{figure}
\centering
\includegraphics[height = 6cm, width = 0.7\linewidth]{figures/Splines.png}
\rule{35em}{0.5pt}
\caption{The fourth order trajectory generated for a positional reference step from $r_0 = 5$ to $r_f=10$ [cm] at the time $t_0 = 0$, given the bounds $[\hat{d},\hat{j},\hat{a},\hat{v}]= [10,3,2,4]$ (dashed). The optimal time given the harsh constraints is $t_f\approx 4.3$.}
\label{fig:posref}
\end{figure}

\clearpage\subsection{Cart control}
Since previous work done in backlash compensation using DMC and positional PID control resulted in a very low repeatable positional accuracy, this method was used as a starting point in the controller design~\cite{Patrik:2013}. However, this gave bad results when implemented in TwinCAT, and was not investigated further due to a lack of time. Instead, two  approaches commonly used in the industry for rigid robotics, the feed-forward PID-control and the cascade control, were implemented in Simulink to make use of the information generated by the trajectory planner (see Figure ~\ref{fig:controlSimulink})~\cite{PID2015feedforward}.

\begin{figure}
\centering
\includegraphics[width = 0.8\linewidth]{figures/PIDfeed-forward.png}
\includegraphics[width = 0.8\linewidth]{figures/CascadeControl.png}
\caption{The feed-forward PID control  (top) and the cascade control (bottom) as implemented in Simulink.}
\label{fig:controlSimulink}
\end{figure}

In the physical process, the two motors of the cart are controlled by a pair of EPOS3 70/10 drivers developed by Maxon. These drivers can be configured to have the motor follow various reference signals, such as position, velocity and torque. Given the structure of the $C^N$-filter, the PID-controlled current loop over the motors was auto-tuned to follow a reference torque.

A model of the asynchronous squirrel cage motors was created in Simulink using the ``asynchronous macine'' based on the work of Li et al. (see \texttt{motor\_model.slx}, Gitlab)~\cite{li2013modeling}. Due to the poor data sheet of the motors, there were too many unknown parameters to make good use the motor model when simulating the system. Consequently, for the purpose of crude simulations of control in Matlab, the driver control loop was omitted and a 1:1 relation of reference torque and actual motor torque was assumed.

In the feed-forward scheme, the control errors in position, velocity and acceleration are amplified by feed forward gains and summed in to a single error signal. This error is fed to the PID control, which in turn generates a control signal corresponding to an effective total motor torque. After the friction compensation, the signal control is split by the $C^N$-filter and fed to the motor drivers as reference torques.

In the cascade control, a P-PI loop is used to to compensate errors in position and velocity successively. Simulations showed that good control could be achieved both when using the P-PI loop (see FIgure~\ref{fig:controlSimulink}), that both speed and accuracy could be increased by introducing a second PI-term on acceleration. However, this P-PI-PI control could not be realized in TwinCAT due to noisy signals and highly inaccurate acceleration measurements.

The tuning of the P-PI loop was relatively simple in Matlab but proved very difficult in the real process. In order to both have good transient behavior and make the cart follow it's optimal trajectory, gain scheduling was used to increase the P-parts on position and velocity while moving the cart, and scaling down both P-gains by a factor of 100 when approaching the reference position (see Figure ~\ref{fig:CascadeControl}). In addition, error in velocity was completely disregarded when below a certain tolerance if also the positional error of an order of a few mm. In doing so, integral action work exclusively on positional control.

\begin{figure}
\centering
\includegraphics[width = 0.8\linewidth]{figures/CascadeTwincat.png}
\caption{The cascade control as implemented in Simulink (top) and TwinCat (bottom).}
\label{fig:CascadeControl}
\end{figure}

\subsection{Anti-Windup}
When running the physical process with the P-PI cascade control and the trajectory planning set to only constrain jerk derivative, small positional oscillations were present as the cart approached the terminal trajectory position, $r_f$ (see the torque control signals in \textbf{Section 5}, Figure~\ref{fig:torquesplit}). In order for the cart to reach a stable position in a short amount of time, the influence of the oscillations had to be minimized. When investigating the problem, it seemed largely dependent on the I-gain. This was confirmed in the Simulink model, where the behavior was replicated when using cascade control with high I-gains along with third order trajectory planning (see Figure~\ref{fig:AntiWindupResults}). Here the oscillations are most clearly visible in the plot of acceleration and velocity, and significant enough to cause large position errors when compared to the desired repeatable accuracy on the micrometer scale.
\begin{figure}
\centering
\includegraphics[width = 0.3\linewidth,height=6cm]{figures/PositionAW.png}%
\includegraphics[width = 0.3\linewidth,height=6cm]{figures/VelocityAW.png}%
\includegraphics[width = 0.3\linewidth,height=6cm]{figures/AccelerationAW.png}
\caption{Cart position (left), velocity (center) and acceleration (right) when running the cart model along a third-order trajectory (black, dashed) with conditioned AW P-PI cascade control, and regular P-PI cascade control.}
\label{fig:AntiWindupResults}
\end{figure}

A good way of mitigating the oscillatory behavior induced by $I$-gain is to use one of many anti-windup schemes (AW). Four different approaches were investigated, based on the work on J. Espina et al.~\cite{espina2009speed}. In this survey of fast AW schemes, the authors concluded that for PI-control of permanent magnet synchronous machines, the conditioned AW had the lowest settling time and overshoot of the methods considered. This conditioned AW scheme simply detects when the control signal gets saturated, which in our case is when difference in reference torque before and after saturation is non-zero. In this state, the integrator holds it's most recent value. The conditioned AW scheme was implemented in Simulink using the memory block and a relational operator for checking when the control is saturated (see Figure ~\ref{fig:AntiWindupSimulink}). Note that the $D$-term was included with a filter-coefficient $N$ for the purpose of testing AW PID control, but that the $D$-gain was set to zero in the cascade control implementation. As expected, the oscillations are greatly suppressed and we reach stable desired position much faster than with conventional $PI$-regulators.

\begin{figure}
\centering
\includegraphics[width = 0.5\linewidth]{figures/AntiwindupSimulink.png}
\caption{The Simulink model of the continuous time conditioned AW-PID regulator.}
\label{fig:AntiWindupSimulink}
\end{figure}

The simulation was run with rough estimations of parameters such as the cart mass and motor rotor inertia, and was merely used to determine the effect of conditioned AW control, which yielded a significant improvement in control. Due to lack of time, this was never implemented in TwinCAT, but it is definitely worth investigating in future iterations of the TwinCAT software.

\newpage\bibliography{Bibliography}{}
\bibliographystyle{IEEEtran}
\end{document}


\color{black}
\subsection*{fitting a self organizing map to in-game data}
The file \texttt{q3dm1-path2.csv} contains a sequence
\begin{equation*}
\vec{x}[1]:\vec{x}[2]:\vec{x}[3]:\vec{x}[3]:\cdots:\vec{x}[n]
\end{equation*}
of 3D locations the avatar of a human player was seen at while moving around the Quake III map \textit{q3dm1}. When plotted, these data look like this
\begin{center}
\includegraphics[width=0.75\textwidth]{q3dm1-data-path2.pdf}
\end{center}

Fit a self organizing map of $k=24$ neurons into the given data points $\vec{x}[t]$. This SOM should have the following topological- or \emph{map space} structure
\begin{center}
\includegraphics[width=0.6\textwidth]{som-infty-topology.pdf}
\end{center}
\newpage

Plot the data in \texttt{q3dm1-path2.csv} together with the SOM you fitted to it. Plot data points in black and SOM weights and their connections in \textcolor{blue}{blue}.

\textbf{Note:} if your fitted SOM looks ``twisted'', then run the training algorithm again, until you obtain a better fit.
%%%%%
%%%%% enter your answer here, i.e. replace "placeholder.pdf" by the name of the graphics file you created
%%%%%
\begin{center}
\includegraphics[width=0.7\textwidth]{som24.png}
\end{center}
%%%%%
%%%%%
%%%%%

\vspace{2cm}
Once you have fitted your SOM, determine how well its weights $\vec{w}_1, \ldots, \vec{w}_k$ represent the data. To do this, compute the mean squared error
\begin{equation*}
E = \frac{1}{n} \sum_{t=1}^n \, \min_i \, \dsq{\vec{x}[t]}{\vec{w}_i}
\end{equation*}
Round your result to \emph{two} decimals and enter it here \color{blue}
%%%%%
%%%%% enter your answer after the '=' sign
%%%%%
\begin{equation*}
E = 1671.85
\end{equation*}
%%%%%
%%%%%
%%%%%
%%%%%
\color{black}
\newpage