\chapter{Functional Methods in Quantum Field Theory}\label{chap:QFT}
This chapter introduces a treatment of quantum field theory using functional methods. The main goal is to become familiar with the physical concepts and the notation used throughout this work and to derive the flow equation for the average effective action, introduced by Christof Wetterich in 1993 \cite{Wetterich1992}.
For the derivation of the flow equation we are following \cite{Gies2006, PawlowskiNPgaugeLecture}.

\section{Generating Functionals and Correlation Functions}
Consider a theory setting of $N$ real scalar fields $\varphi_a(x), a \in \{1,\dots,N\}$ in $d$-dimensional Euclidean space. The corresponding partition sum in presence of sources $J_a(x)$ reads
\begin{align}
	Z[J] = \frac{1}{\mathcal{N}} \int \D\varphi \operatorname{e}^{-\S + J\cdot\varphi}.
	\label{eqn:partition}
\end{align}
The action $\mathcal{S}$ is specified together with an ultraviolet cutoff scale $\Lambda$, later being the momentum scale where we initialize the flow equations and some normalization factor $\mathcal{N}$.\\
In this notation, the scalar product sums over field components and integrates over all space,
\begin{align}
	J\cdot\varphi = \int_x J_a(x) \ \varphi_a(x) = \int_p \tilde{J}_a(p) \ \tilde{\varphi}_a(p),
\end{align}
with
\begin{align}
\int_x = \int_{\mathbb{R}^d} \dd^d x \qquad \text{and} \qquad \int_p = \int_{\mathbb{R}^d} \frac{\dd^d p}{(2\pi)^d}.
\end{align}

The partition sum $Z[J]$ is called a \textit{generating functional}. It directly allows us to compute field expectation values
\begin{align}
	\phi := \cf{\varphi} = \eval{\frac{1}{Z}\frac{\delta Z}{\delta J}}_{J=0} = \int \D\varphi \ \varphi \ \operatorname{e}^{-\S + J\cdot\varphi}
\end{align}
and higher order correlation functions
\begin{align}
\cf{\varphi(x_1) \cdots \varphi(x_n)} := \cf{\varphi^n} = \frac{1}{Z}\eval{\frac{\delta^n Z}{\delta^n J}}_{J=0} = \int \D\varphi \ \overbrace{\varphi_1 \cdots \varphi_n}^{:= \ \varphi^n} \ \operatorname{e}^{-\S + J\cdot\varphi}
\end{align}
via functional differentiation. This means, we are basically able to compute all contributing Feynman diagrams for our theory setting, if we have knowledge of its corresponding (grand) canonical partition sum. \\
 For a more efficient description of the theory in terms of only the \textit{connected} correlation functions, we define the \textit{Schwinger functional} $W[J]$ as the logarithm of $Z[J]$,
\begin{align}
W[J] = \ln Z[J].
\label{eqn:Schwinger}
\end{align}
It is the generating functional for the connected correlation functions. The normalization factor $\mathcal{N}$, introduced in (\ref{eqn:partition}) enters here as an additive constant, which drops out for all higher order correlation functions, except for the zero-point function. This term is connected to the thermodynamic quantities of the system and becomes important, when external parameters such as temperature, volume or the chemical potential are varied. For the case of quantum gravity, it is linked to the cosmological constant $\Lambda$. Nevertheless, in general we are only interested in correlation functions with $n\geq 1$ and therefore we drop this term.\\
Consider for example the connected two-point function $G_{ab}(x,y) = G_{\alpha\beta}$\footnote{To save on notation, we introduce collective indices $\alpha = (x,a)$ or $(q,a)$ in momentum space.}, known as the propagator, correlating the field $\varphi_a$ at spacetime point $x$ with the field $\varphi_b$ at $y$,
\begin{equation}
\begin{aligned}
	G_{\alpha\beta} &= \frac{\delta^2W[J]}{\delta J_{\alpha}\delta J_{\beta}} = \frac{\delta}{\delta J_{\alpha}}\left(\frac{1}{Z}\frac{\delta Z}{\delta J_{\beta}}\right)  \\[10pt]
				&= \frac{1}{Z}\left(\frac{\delta^2Z}{\delta J_{\alpha}\delta J_{\beta}}\right) - \frac{1}{Z^2}\left(\frac{\delta Z}{\delta J_{\alpha}}\right)\left(\frac{\delta Z}{\delta J_{\beta}}\right)\\[10pt]
				&= \cf{\varphi_{\alpha}\varphi_{\beta}} - \phi_{\alpha}\phi_{\beta} = \cf{\varphi_{\alpha}\varphi_{\beta}}_{\text{c}}.
\end{aligned}
\label{eqn:G_connected}
\end{equation}
The propagator is the key object in functional approaches to quantum field theory. It depends on the chosen background via $J$. \\
It is still possible to make our computations even more efficient, because $W[J]$ still contains some redundant information. Connected correlation functions can be separated into so-called one-particle irreducible (1PI) and one-particle reducible ones. The 1PI correlation functions are those, whose corresponding Feynman diagrams can \textit{not} be separated into two disconnected ones by cutting a single internal line. As an example, contributing 1PI and reducible diagrams to the connected four-point function for Yukawa theory are depicted in figure (\ref{fig:1PI_Yukawa}). \\
\begin{figure}[t]
\centering
\includegraphics[width=0.8\textwidth]{figs/TikZ/1PI_Yukawa}
\caption[Contributing one-particle reducible and 1PI diagrams to the four-point-function in Yukawa theory]{Contributing one-particle reducible and 1PI diagrams to the four-point-function in Yukawa theory, inspired by \cite{FloerchingerWetterichQFT}.}
\label{fig:1PI_Yukawa}
\hrulefill
\end{figure}
The generating functional for the  1PI correlation functions, the \textit{effective action} $\Gamma$, is obtained from the Schwinger functional via a Legendre transformation,
\begin{equation}
	\Gamma[\phi]=\sup _{J}\left\{\int_{x} J(x) \phi(x)-W[J]\right\}=\int_{x} J_{\mathrm{sup}}(x) \phi(x)-W\left[J_{\mathrm{sup}}\right],
\label{eqn:Def_Gamma}
\end{equation}
where $J_{\mathrm{sup}}$ has to be understood as a field-dependent current $J_{\mathrm{sup}}[\phi]$. In the following, we will drop the subscript, its meaning is implicitly understood.
The quantum equation of motion derived from $\Gamma$ reads
\begin{align}
	J(x) = \frac{\delta\Gamma[\phi]}{\delta\phi(x)}.
	\label{eqn:quantum_eom}
\end{align}
It allows us to understand the dynamics of field expectation values, taking the effects of all quantum fluctuations into account.
From a physical point of view, the effective action $\Gamma$ is the quantum analogue of the classical action $\mathcal{S}$. The performed Legendre transformation leads us to a mean field description of our theory with $\phi = \cf{\varphi}$ on a given background, as introduced before. The symmetries of the classical action are in general still present in the effective action.\\
In terms of the effective action, higher order correlation functions are again obtained by performing functional derivatives, but now w.\,r.\,t. the mean field $\phi$,
\begin{align}
	\Gamma^{(n)}\left(x_{1}, \ldots, x_{n}\right)=\frac{\delta^{n} \Gamma}{\delta \phi\left(x_{1}\right) \cdots \delta \phi\left(x_{n}\right)}.
\end{align}
With the definition of the effective action (\ref{eqn:Def_Gamma}), we find
\begin{equation}
	\operatorname{e}^{-\Gamma[\phi]}=\int_{\Lambda} \mathcal{D} \varphi \exp \left(-\mathcal{S}[\phi+\varphi]+\int_x \frac{\delta \Gamma[\phi]}{\delta \phi(x)} \varphi(x)\right).
\end{equation}
The solution of such functional integro-differential equations is highly non-trivial. To solve this problem, we want to make use of the Functional Renormalization Group. The general idea of this approach is to introduce a scale-dependent action $\Gamma_k$, interpolating between the bare, microscopic action $\mathcal{S}$ and the full quantum effective action $\Gamma$. A more formal motivation and a derivation of the equation governing this interpolation process is presented in the next section.
 \section{Functional Renormalization Group}
The Functional Renormalization Group (FRG) is a mathematical tool, allowing us to investigate the dynamics of physical systems on different energy (momentum) scales. This idea is based on a continuous version of Leo P. Kadanoff's block spin model on the lattice \cite{Kadanoff1966} and was developed by Kenneth G. Wilson in 1971 \cite{Wilson1971}. It aims at solving the theory by integrating successively momentum shell by momentum shell, being the reason why the path integral approach to quantum field theory provides a suitable framework. The main advantage of the FRG approach is, that no regularization or renormalization procedure has to be applied. The latter one is already implemented systematically, which secures the self-consistency of the approach. As this section is only supposed to introduce the basics of the FRG, we refer the interested reader to more complete reviews, e.\,g. \cite{Pawlowski2005, Gies2006}, particularly for applications in different areas of physics.

As a first step towards a FRG equation we need to introduce an infrared cutoff scale $k$ in our theory, below which the modes are not integrated out. A common way to introduce such a scale is by  adding a scale-dependent cutoff term $\Delta\mathcal{S}_k$ in the definition of the partition sum (\ref{eqn:partition}) and therefore automatically also in the definition of the Schwinger functional (\ref{eqn:Schwinger}):
\begin{align}
W_{k}[J]=\ln Z_{k}[J]=\ln \int \mathcal{D} \varphi  \operatorname{e}^{-\mathcal{S}[\varphi]+J \cdot \varphi-\Delta \mathcal{S}_{k}[\varphi]}.
\label{eqn:Wk}
\end{align}
The physical scale $k$ we introduced here is known as \textit{renormalization scale} and has units of inverse length, meaning large $k$ correspond to small distances and vice versa. The cutoff term $\Delta\mathcal{S}_k$ is a quadratic functional depending on the field $\varphi$:
\begin{align}
	\Delta \mathcal{S}_{k}[\varphi]=\frac{1}{2} \varphi \cdot R_{k} \cdot \varphi=\frac{1}{2} \int_{x, y} \varphi_{\alpha} \ R_{k, \alpha\beta} \ \varphi_{\beta}.
\end{align}
The function $R_k$ is called \textit{regulator}. It plays an important role for this formulation of quantum field theory. The regulator is chosen such that only the propagation for momentum modes with $p^2 \lesssim k^2$ is suppressed. The most important physical limits are summarized in the following:
\begin{align}
	R_{k}(p^2) \rightarrow\left\{\begin{array}{ll}{k^{2}} & {\text { for } p \rightarrow 0} \\ {0} & {\text { for } p \rightarrow \infty} \\ {0} & {\text { for } k \rightarrow 0} \\ {\infty} & {\text { for } k \rightarrow \Lambda}\end{array}\right.
\label{eqn:regulator_limits}
\end{align}
A convenient choice of the regulator is given by
\begin{align}
	R_k(p^2) = p^2 \cdot r_k(y),
\end{align}
with $ y := \frac{p^2}{k^2}$, and a dimensionless regulator shape function $r_k$, only depending on the dimensionless momentum ratio $y$. There is a plethora of different types of shape functions, but for the computations performed in this work we restrict ourselves to a class of rather simple, so-called Litim-type regulators with shape functions
\begin{align}
r_k(y) = \left(\frac{1}{y} - 1\right)\theta(1-y),
\label{eqn:Litim}
\end{align}
where $\theta$ is the Heaviside step function. This class of \textit{sharp} regulators is a good choice for finding analytic FRG equations in simple approximations. For numerical approaches, exponential regulators such as depicted in figure (\ref{fig:exp_regulator}) are well suited.\\
\begin{figure}[t]
\centering
\includegraphics[width=0.4\textwidth]{figs/Plots/regulator_plot}
\caption[Shape of a typical exponential regulator function $R(p^2)$ and its derivative w.\,r.\,t. the RG time $t$.]{Shape of a typical exponential regulator function $R(p^2)$ and its derivative w.\,r.\,t. the RG time $t$. The regulator has a finite value for momenta smaller than $k^2$ and therefore acts as a suppressing mass term. The peak of $\partial_tR_k$ around $k^2=p^2$ clearly shows the implementation of Wilsons idea of shell-wise momentum integration.}
\label{fig:exp_regulator}
\hrulefill
\end{figure}
At this point it is quite convenient to introduce the \textit{RG time} $t$ as
\begin{align}
	t = \ln\left(\frac{k}{\Lambda}\right) \qquad \longrightarrow\qquad \partial_t = \frac{\partial}{\partial\ln(k/\Lambda)} = \frac{k}{\Lambda}\frac{\partial}{\partial(k/\Lambda)} = k \partial_k,
\end{align}
where $\Lambda$ is a fixed reference scale. Usually one chooses the ultraviolet cutoff scale, where the flow is initialized.\\
In this setting, (\ref{eqn:Wk}) provides a good starting point for solving the theory by successively lowering the cutoff scale $k$ infinitesimally and integrating out all momentum modes $\varphi_{p\approx k}$. This procedure can be formalized by taking a scale derivative of our scale-dependent functional (\ref{eqn:Wk}):
\begin{equation}
\begin{aligned} \partial_{t} W_{k}[J] &=-\frac{1}{2} \int \mathcal{D} \varphi \ \varphi(-p) \partial_{t} R_{k}(p) \varphi(p) \operatorname{e}^{-\mathcal{S}[\varphi]+ J \cdot\varphi - \Delta \mathcal{S}_{k}[\varphi]} \\ &=-\frac{1}{2} \int_p \partial_{t} R_{k}(p) G_{k}(p)+\partial_{t} \Delta S_{k}[\phi],
\label{eqn:dtW}
\end{aligned}
\end{equation}
where we used the definition of the connected propagator:
\begin{equation}
G_k = \frac{\delta^2 W_k[\phi]}{\delta\phi(x)\delta\phi(y)}.
\label{eqn:Gk}
\end{equation}
The \textit{flowing} or \textit{effective average action} $\Gamma_k$ is then again defined via a modified Legendre transformation, including the insertion of $\Delta S_k$:
\begin{equation}
	\Gamma_{k}[\phi]=\sup _{J}\left(\int_x J(x) \phi(x)-W_{k}[J]\right)-\Delta S_{k}[\phi].
\end{equation}
This yields the modified, scale-dependent quantum equation of motion:
\begin{align}
	J(x) = \frac{\delta\Gamma_k[\phi]}{\delta\phi(x)} + \left(R_k\phi\right)(x).
\end{align}
Compared to the scale-independent version (\ref{eqn:quantum_eom}), we find an additional, regulator dependent term, but with the properties of the regulator presented in (\ref{eqn:regulator_limits}) in mind, we see that in the limit $k\rightarrow 0 $ the initial equation of motion is restored.
We find
\begin{equation}
	\frac{\delta J(x)}{\delta \phi(y)}=\frac{\delta^{2} \Gamma_{k}[\phi]}{\delta \phi(x) \delta \phi(y)}+R_{k}(x, y).\label{eqn:dJdphi}
\end{equation}
With the help of these relations we are able to show that
\begin{equation}
\begin{aligned} \delta\left(x-x^{\prime}\right) =\frac{\delta J(x)}{\delta J\left(x^{\prime}\right)}&=\int_y \frac{\delta J(x)}{\delta \phi(y)} \frac{\delta \phi(y)}{\delta J\left(x^{\prime}\right)} \\[10pt] &=\int_y\left(\Gamma_{k}^{(2)}[\phi]+R_{k}\right)(x, y) \ G_{k}\left(y-x^{\prime}\right).
\end{aligned}
\end{equation}
Here, we used (\ref{eqn:dJdphi}) and the definition of $G_k$ (\ref{eqn:Gk}).
This yields the following important identity:
\begin{equation}
	G_k = \left(\Gamma_k^{(2)} + R_k\right)^{-1}.
	\label{eqn:inverse_prop_identity}
\end{equation}
Altogether, we arrive at the \textit{flow equation}, a.\,k.\,a. the \textit{Wetterich equation} for the average effective action:
\begin{equation}
\begin{aligned}
\partial_t \Gamma_k[\phi] &\overset{\phantom{(\ref{eqn:dtW})}}{=} -\partial_t W_k +\int\left(\partial_t J\right) \phi - \partial_t \Delta S_k[\phi] = - \partial_t W_k[J] - \partial_t \Delta S_k[\phi] \\[5pt]
&\overset{(\ref{eqn:dtW})}{=} \frac{1}{2} \int_p G_{k}(p) \ \partial_{t} R_{k}(p)\\[5pt]
&\overset{(\ref{eqn:inverse_prop_identity})}{=}\frac{1}{2} \operatorname{STr}\left[\left(\Gamma_{k}^{(2)}[\phi]+R_{k}\right)^{-1}\partial_{t} R_{k}\right].
\end{aligned}
\label{eqn:Wetterich}
\end{equation}
The supertrace $\operatorname{STr}$ sums over all internal indices and integrates over momentum space. For Grassmann fields, it also involves the inclusion of a minus sign. We will drop the $\operatorname{S}$ for the rest of this work, its meaning should be understood implicitly.
The flow equation can be represented diagrammatically as a $1$-loop equation:
\begin{figure}[H]
\centering
\begin{gather}
\begin{aligned}
\includegraphics[scale=1.1]{figs/TikZ/wetterich_equation}
\end{aligned}
\end{gather}
\end{figure}
\vspace{-0.7cm}
The full propagator $\left[\Gamma_k^{(2)} + R_k\right]^{-1}$ is represented as usual as a single, double, dashed etc. line, dependent on the field content.
The crossed circle $\otimes$ denotes the insertion of the respective regulator or more precisely its derivative w.\,r.\,t. the RG time $t$. Here $\partial_{t} R_{k, ij}(p, q)=\partial_{t} R_{k}(p^2)(2 \pi)^{d} \  \delta_{i j} \ \delta(p-q)$ and therefore the trace on the r.\,h.\,s. effectively sums over just one index $i$ and integrates over one loop momentum $p$. \\
\begin{figure}[t]
\centering
\includegraphics{figs/TikZ/regulator_dependence}
\caption[Flow of $\Gamma_k$ through infinite-dimensional theory space for different regulators.]{Flow of $\Gamma_k$ through infinite-dimensional theory space for different regulators, inspired by \cite{Gies2006}. Although the trajectories in theory space, governed by the flow equation (\ref{eqn:Wetterich}) may be different, they flow towards the same quantum effective action $\Gamma_{k\rightarrow 0} \equiv \Gamma$.}
\label{fig:theory_space}
\hrulefill
\end{figure}
It is important to mention, that the Wetterich equation is an \textit{exact} equation, no approximations have been made. The only modification, the implementation of $\Delta S_k$ vanishes in the limit $k\rightarrow 0$. Solutions of the flow equation correspond to trajectories in \textit{theory space}, the space spanned by all (= infinitely many) dimensionless couplings $g_{\alpha}$. The choice of the regulator has direct impact on the exact form of the trajectory. This is often referred to as \textit{scheme dependence}. Nevertheless, for all regulators satisfying the properties (\ref{eqn:regulator_limits}) it is guaranteed, that the flow will lead to the same quantum effective action $\Gamma$. For a visualization of this idea, have a look at figure (\ref{fig:theory_space}).  In principle this means, that $\lim_{k\rightarrow 0}\Gamma_k\equiv\Gamma$, but in most practical cases it is unavoidable to employ truncation schemes to be able to solve the flow equation. A plethora of different truncation schemes has been developed recently, details concerning the most important schemes can be found e.\,g. in the reviews about the FRG we referred to at the beginning of this section. We want to conclude this chapter with a more formal discussion of the concept of theory space, before proceeding to an introduction of the basic concepts of (classical) gravity.
\section{Renormalization Group Flow and Theory Space}
We want to use this section to formalize the concept of theory space we introduced in the last section and to discuss important characteristics of the renormalization group flow such as the $\beta$-functions and their zeros, the fixed points of the flow. For this part, we mainly follow \cite{ReuterSaueressig2012}. \\
The theory space is defined as the space spanned by all dimensionless couplings of the theory. To be more precise, it consists of all (action) functionals $A:\Phi \mapsto A[\Phi]$, that are compatible with the imposed symmetries of the theory such as e.\,g. diffeomorphism invariance in the case of (quantum) gravity. \\
The flow equation (\ref{eqn:Wetterich}) defines a vector field $\vec{\beta}$ in theory space whose integral curves are the trajectories $\Gammak$ parametrized by the scale $k$. Assuming the existence of a complete set of basis functionals $\left\{P_{\alpha}[\ \cdot \ ]\right\}$, we can expand $\Gamma_k$ as follows:
\begin{equation}
	\Gamma_{k}[\Phi, \bar{\Phi}]=\sum_{\alpha=1}^{\infty} \bar{g}_{\alpha}(k) P_{\alpha}[\Phi, \bar{\Phi}].
\end{equation}
Here, the expansion coefficients $ \bar{g}_{\alpha}(k)$ are given by the generalized couplings. Inserting this ansatz into the flow equation (\ref{eqn:Wetterich}), yields a set of infinitely many coupled differential equations for the couplings:
\begin{equation}
	k \partial_{k} \bar{g}_{\alpha}(k)=\bar{\beta}_{\alpha}\left(\bar{g}_{1}, \bar{g}_{2}, \cdots ; k\right), \qquad \alpha=1,2, \cdots
\end{equation}
The \textit{beta functions} $\bar{\beta}_{\alpha}\left(\bar{g}_{1}, \bar{g}_{2}, \cdots ; k\right)$  are the components of the vector field $\vec{\beta}$ and arise from an expansion of the trace on the r.\,h.\,s. of the flow equation in terms of the functional basis\footnote{The expansion reads: $\frac{1}{2}\tr{\cdots} = \sum_{\alpha=1}^{\infty} \bar{\beta}_{\alpha}\left(\bar{g}_{1}, \bar{g}_{2}, \cdots ; k\right) P_{\alpha}[\Phi, \bar{\Phi}]$.}. Up to this point, we are still dealing with dimensionful couplings $\bar{g}$, but as mentioned earlier usually the flow equation is expressed in terms of \textit{dimensionless couplings}
\begin{equation}
	g_{\alpha} \equiv k^{-d_{\alpha}} \bar{g}_{\alpha},
\end{equation}
where $d_{\alpha}$ is the canonical mass dimension of the respective coupling. The \textit{essential} couplings\footnote{Essential in this sense means, that they can not be absorbed into the fields via a rescaling.} provide a set of coordinates for the theory space. This allows us to interpret the idea of renormalization theory in a new, geometrical way: We need to construct \enquote{infinitely long} trajectories $\Gammak$, that lie \textit{entirely} in theory space. In this case, the couplings are prevented from diverging and we are able to define a consistent quantum field theory. \newpage
A \textit{fixed point} $g^{*}$ of the flow is a zero point of the vector field $\vec{\beta}$, i.\,e.  $\beta_{\alpha}(g^{*})\equiv 0 \ \forall \alpha$. The existence of such fixed points is crucial for our discussion of Asymptotic Safety as an approach to quantum gravity, based on the concepts we introduced here. \\
\begin{figure}[t]
	\centering
	\includegraphics[width=0.5\textwidth]{figs/TikZ/hypersurface}
	\caption[Visualization of a fixed point with its corresponding UV hypersurface $\Sigma_{\mathrm{UV}}$ in theory space.]{Visualization of a fixed point $g^{*}$ (orange dot) with its corresponding UV hypersurface $\Sigma_{\mathrm{UV}}$ and trajectories starting at $g^{*}$ (green) in theory space. The flow points towards the IR. Trajectories starting off the surface (red) are pulled towards the FP along the irrelevant direction (here: $g_1$) until  the IR repulsive directions $g_2$ and $g_3$ dominate and drive the flow away from $g^{*}$. This figure is inspired by \cite{Eichhorn2018}.}\label{fig:hypersurface}
	\hrulefill
\end{figure}
In general, one distinguishes different classes of fixed points. The \textit{Gaussian} or \textit{non-inter-acting} fixed points  (GFP) are classified by $g^{*}_{\alpha}=0 \ \forall\alpha$. This class of fixed points is relevant for perturbation theory, where the limit $k\rightarrow\infty$ is taken at such a GFP\footnote{E.\,g. in Yang-Mills theory, the concept of \textit{Asymptotic freedom}, where the couplings tend to zero in the limit $k\rightarrow\infty$, is based on the existence of an UV attractive Gaussian fixed point, rendering the theory perturbatively renormalizable \cite{GrossWilczek1973}.}. If at least one of the couplings $g^{*}_{\alpha}\neq 0$, the fixed point is classified as \textit{Non-Gaussian} or \textit{interacting} (NGFP). The idea of Asymptotic Safety relies on the existence of such a NGFP, rendering the theory \enquote{safe} from divergences in the ultraviolet (UV) regime. An important characteristic of a fixed point is its stability or more precisely if it is \textit{attractive} or \textit{repulsive} for near RG trajectories. Additionally one distinguishes between infrared ($k\rightarrow0$) and ultraviolet ($k\rightarrow\infty$) attractive (repulsive) fixed points. To analyze this behavior, the flow near a fixed point is linearized, i.\,e.
\begin{equation}
	\partial_{t} g_{\alpha}(k)=\sum_{j=1}^{\infty} B_{\alpha j}\left(g_{j}-g_{j}^{*}\right),
	\label{eqn:linearized_flow}
\end{equation}
where we defined the \textit{stability matrix} \ $\mathbf{B} = B_{\alpha j} = \partial_j\beta_{\alpha}(g^{*}_{\alpha}) $. The solution of the differential equation (\ref{eqn:linearized_flow}) reads:
\begin{equation}
	g_{\alpha}(k)=g_{\alpha}^{*}+\sum_{j=1}^{\infty} C_{j} V_{\alpha}^{j}\left(\frac{k}{k_{0}}\right)^{\theta_{j}}.
\end{equation}
Here, the $V^{j}$ are the eigenvectors of the stability matrix with eigenvalues $\theta_j$ a.\,k.\,a. \textit{critical exponents}. In general, the $\theta_{j}$, are complex numbers. We use the real part of the critical exponents to classify the coupling as \textit{relevant} (= attractive) or \textit{irrelevant} (= repulsive):
\begin{align}
	g^{*}_{\alpha} \ \text{ is } \left\{\begin{array}{ll}{\text{relevant }} & {\text { for } \ \mathfrak{Re}\left(\theta_j\right) > 0} \\[10pt] {\text{irrelevant }} & {\text { for } \  \mathfrak{Re}\left(\theta_j\right) < 0} \\ \end{array}\right..
\end{align}
Fixed points with critical exponents $\theta_{j}=0$ are called \textit{marginal}. Based on this classification, it follows quite naturally to define an UV (or IR)  \textit{critical hypersurface} $\Sigma_{\mathrm{UV}}$ in theory space for a NGFP, consisting of all points that are pulled into the NGFP for increasing $k$. The dimension of $\Sigma_{\mathrm{UV}}$ is equal to the number of UV relevant couplings. This means, that trajectories lying on such a hypersurface tend to flow towards the fixed point in the UV limit. To visualize this idea, a schematic sketch of such a hypersurface in a $3$-dim. theory space is depicted in figure (\ref{fig:hypersurface}).


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% UMB-CS114-2015F: Introduction to Programming in Java
% Copyright 2015 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/UMB-CS114-2015F
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\def \topDirectory {.}
\def \texDirectory {\topDirectory/src/main/tex}

\documentclass[12pt,letterpaper,twoside]{article}
\usepackage{\texDirectory/template/style/directives}
\usepackage{\texDirectory/template/style/assignment}
\input{\texDirectory/template/config}

\begin{document}

\doc{title}{Solution to Assignment 4}
\doc{date-pub}{Nov 04, 2015 at 5:30 PM}
\doc{date-due}{Nov 18, 2015 at 5:30 PM}
\doc{points}{8}

\prepare{header}

\section*{Question 1}

An $n \times n$ matrix is called a \textit{positive Markov matrix} if each element is positive and the sum of the elements in each column is 1.
Write a program \texttt{MarkovMatrix.java} that prompts the user to enter a $3 \times 3$ matrix of double values.
Use a method with the following signature to test if the given matrix is a Markov matrix.

\begin{terminal}
public static boolean isMarkovMatrix(double[][] matrix)
\end{terminal}

Your program is expected to function as shown in following examples:

\begin{terminal}
$ javac MarkovMatrix.java
$ java MarkovMatrix
Enter Row 1: 0.15 0.875 0.375
Enter Row 2: 0.55 0.005 0.225
Enter Row 3: 0.30 0.12 0.4
Markov matrix given.
\end{terminal}

\newpage

\subsection*{Solution}

\lstset{language=Java,tabsize=4}
\begin{lstlisting}
import java.util.Scanner;
public class MarkovMatrix {
	public static void main(String[] args) {
		double[][] matrix = initMatrix(3, 3);
		if (isMarkovMatrix(matrix))
			System.out.println("Markov matrix given.");
		else
			System.out.println("Matrix not Markov.");
	}
	public static double[][] initMatrix(int row, int col) {
		Scanner input = new Scanner(System.in);
		double[][] matrix = new double[row][col];
		for (int i = 0; i < row; i++) {
			System.out.printf("Enter Row %d: ", i + 1);
			for (int j = 0; j < col; j++)
				matrix[i][j] = input.nextDouble();
		}
		input.close();
		return matrix;
	}
	public static boolean isMarkovMatrix(double[][] matrix) {
		for (int i = 0; i < matrix[0].length; i++) {
			double sum = 0;
			for (int j = 0; j < matrix.length; j++) {
				if (matrix[j][i] <= 0)
					return false;
				sum += matrix[j][i];
			}
			if (sum != 1)
				return false;
		}
		return true;
	}
}
\end{lstlisting}

\newpage

\section*{Question 2}

Write a program \texttt{PointAndSphere2.java} that prompts user to enter respectively, coordinates of a point, coordinates of the center of a sphere and radius of the sphere.
Your program would then determine the location of the point with respect to the sphere.
The point should be an instance of class \texttt{Point} and the sphere should be an instance of class \texttt{Sphere}.
Following is an example of an accepted output format.

\begin{terminal}
$ javac Point.java Sphere.java PointAndSphere2.java
$ java PointAndSphere2
Coordinates of Point: 1 1 1
Coordinates of Sphere: 0 0 0
Radius of Sphere: 1.7
The point is outside the sphere.
\end{terminal}

\subsection*{Solution}

\lstset{language=Java,tabsize=4}
\begin{lstlisting}
public class Point {
	public double[] coordinates;
	public Point(double[] coordinates) {
		this.coordinates = coordinates;
	}
	public double getDistance(Point point) {
		double sum = 0;
		for (int i = 0; i < 3; i++)
			sum += Math.pow(this.coordinates[i] - point.coordinates[i], 2);
		return Math.sqrt(sum);
	}
}
\end{lstlisting}

\lstset{language=Java,tabsize=4}
\begin{lstlisting}
public class Sphere {
	public Point center;
	public double radius;
	public Sphere(Point center, double radius) {
		this.center = center;
		this.radius = radius;
	}
}
\end{lstlisting}

\newpage

\lstset{language=Java,tabsize=4}
\begin{lstlisting}
import java.util.Scanner;
public class PointAndSphere2 {
	public static void main(String[] args) {
		double[] pointCoordinates = promptCoordinates("Point");
		double[] sphereCoordinates = promptCoordinates("Sphere");
		double radius = promptRadius();
		Point point = new Point(pointCoordinates);
		Point center = new Point(sphereCoordinates);
		Sphere sphere = new Sphere(center, radius);
		double dist = sphere.center.getDistance(point);
		if (dist > sphere.radius)
			System.out.println("The point is outside the sphere.");
		else if (dist == sphere.radius)
			System.out.println("The point is on the sphere.");
		else
			System.out.println("The point is inside the sphere.");
	}
	public static double[] promptCoordinates(String name) {
		Scanner input = new Scanner(System.in);
		System.out.printf("Coordinates of %s: ", name);
		double[] coordinates = new double[3];
		for (int i = 0; i < coordinates.length; i++)
			coordinates[i] = input.nextDouble();
		return coordinates;
	}
	public static double promptRadius() {
		Scanner input = new Scanner(System.in);
		System.out.print("Radius of Sphere: ");
		double radius = input.nextDouble();
		return radius;
	}
}
\end{lstlisting}

\newpage

\section*{Question 3}

Write a program \texttt{MatrixFiller2.java} that prompts user for a number $x$ between 1 to 9 and instantiates an $x \times x$ matrix from class \texttt{Matrix.java} whose elements are randomly generated from range 1 to $x^2$.
Following is an expected sample run of your program.

\begin{terminal}
$ javac Matrix.java MatrixFiller2.java
$ java MatrixFiller2
Size of Matrix: 4
 05 13 07 16
 12 02 10 01
 09 14 14 08
 02 05 01 14
\end{terminal}

\subsection*{Solution}

\lstset{language=Java,tabsize=4}
\begin{lstlisting}
public class Matrix {
	int[][] elements;
	public Matrix(int row, int col) {
		this.elements = new int[row][col];
		for (int i = 0; i < row; i++)
			for (int j = 0; j < col; j++) {
				int rand = (int) (Math.random() * row * col) + 1;
				this.elements[i][j] = rand;
			}
	}
	public void display() {
		int row = this.elements.length;
		int col = this.elements[0].length;
		for (int i = 0; i < row; i++) {
			for (int j = 0; j < col; j++)
				System.out.printf(" %02d", elements[i][j]);
			System.out.println();
		}
	}
}
\end{lstlisting}

\newpage

\lstset{language=Java,tabsize=4}
\begin{lstlisting}
import java.util.Scanner;
public class MatrixFiller2 {
	public static void main(String[] args) {
		Scanner input = new Scanner(System.in);
		System.out.print("Size of Matrix: ");
		int size = input.nextInt();
		input.close();
		Matrix matrix = new Matrix(size, size);
		matrix.display();
	}
}
\end{lstlisting}

\section*{Question 4}

Write a class \texttt{Circle.java} from which we can instantiate a circle by giving its radius and use it as is shown in the following program.

\lstset{caption=}
\lstset{language=Java,tabsize=4}
\begin{lstlisting}
import java.util.Scanner;
public class Circles {
	public static void main(String[] args) {
		Scanner input = new Scanner(System.in);
		System.out.print("Enter radius: ");
		double radius = input.nextDouble();
		input.close();
		Circle myCircle = new Circle(radius);
		double area = myCircle.getArea();
		double perimeter = myCircle.getCircumference();
		System.out.printf("Area: %.2f, Perimeter: %.2f\n", area, perimeter);
	}
}
\end{lstlisting}

\subsection*{Solution}

\lstset{language=Java,tabsize=4}
\begin{lstlisting}
public class Circle {
	double radius;
	public Circle(double radius) {
		this.radius = radius;
	}
	public double getArea() {
		double area = Math.PI * Math.pow(this.radius, 2);
		return area;
	}
	public double getCircumference() {
		double circumference = 2 * Math.PI * this.radius;
		return circumference;
	}
}
\end{lstlisting}

\section*{Question 5}

The code snippet given below is content of a file \texttt{Kitten.java} found in a public repository.
Unfortunately, the program cannot be executed because the file \texttt{Cat.java} which defines the class \texttt{Cat} is missing.
You are expected to develop the class \texttt{Cat} in a file \texttt{Cat.java} such that \texttt{Kitten.java} is successfully executed.

\lstset{caption=}
\lstset{language=Java,tabsize=4}
\begin{lstlisting}
import java.util.Scanner;
public class Kitten {
	public static void main(String[] args) {
		Cat myCat = new Cat("Kitty");
		double[] movement = promptMove(myCat);
		myCat.move(movement[0], movement[1]);
		myCat.showPosition();
		myCat.showDistance();
	}
	public static double[] promptMove(Cat myCat) {
		Scanner input = new Scanner(System.in);
		char[] directions = {'X', 'Y'};
		double[] movement = new double[directions.length];
		for (int i = 0; i < directions.length; i++) {
			System.out.printf("Distance to move in %c direction: ", directions[i]);
			movement[i] = input.nextDouble();
		}
		input.close();
		return movement;
	}
}
\end{lstlisting}

\newpage

Following is a sample expected run of the program.

\begin{terminal}
$ javac Cat.java Kitten.java
$ java Kitten
Distance to move in X direction: 3
Distance to move in Y direction: 4
Kitty is in (3.0, 4.0).
Kitty is 5.00 units away from (0, 0).
\end{terminal}

\subsection*{Solution}

\lstset{language=Java,tabsize=4}
\begin{lstlisting}
public class Cat {
	public String name;
	public double dirX = 0;
	public double dirY = 0;
	public Cat(String name) {
		this.name = name;
	}
	public void move(double dirX, double dirY) {
		this.dirX += dirX;
		this.dirY += dirY;
	}
	public void showPosition() {
		System.out.printf("%s is in (%.1f, %.1f).\n", this.name, this.dirX, this.dirY);
	}
	public void showDistance() {
		double distance = Math.sqrt(Math.pow(this.dirX, 2) + Math.pow(this.dirY, 2));
		System.out.printf("%s is %.2f units away from (0, 0).\n", this.name, distance);
	}
}
\end{lstlisting}

\end{document}


\section{Disorder Terms}

\begin{slide}{Structural Disorder and the Pair Distribution Function}

  \begin{cenpage}{135mm}

  An EXAFS measurement averages billions of {\BlueEmph{snapshots}} of the
  local structure:

\begin{cenpage}{88mm}
\begin{itemize}
\item Each absorbed x-ray generates 1 photo-electron.
\item the photo-electron / core-hole pair lives for about
  $10^{-15}$ s --  much faster than the thermal vibrations ($10^{-12}$ s).
\item An EXAFS measurement samples $10^4$ (dilute fluorescence) to $10^{10}$
  absorbed x-rays for each energy point.
\end{itemize}
\end{cenpage}

\vmm
 \vmm

So far, we've put this in the EXAFS Equation as \hspace{2mm}
$\chi \sim N \exp({-2k^2\sigma^2}) $

\vmm \hrule \vmm \onslide+<2->

\begin{columns}
  \begin{column}{70mm}
    More generally, EXAFS samples the

    \vmm
    {\RedEmph{Partial Pair Distribution Function}}

    \vmm

    {\RedEmph{$g(R)$}} =   probability that an
    atom is a distance $R$ away from the absorber.

    \vspace{8mm}

    \end{column}
  \begin{column}{65mm}

    \includegraphics[width=60mm]{figs/errors/gnxas}
    \end{column}
  \end{columns}
\end{cenpage}
\end{slide}

\begin{slide}{EXAFS and The Pair Distribution Function}

  \begin{cenpage}{135mm}
  To fully account for a highly disordered local structure, we should use

\[
    \chi(k)  = \Biggl\langle \sum_j {\frac{f_j(k)e^{i2kR_j + \delta_j(k)}}{kR_j^2}} \Biggr\rangle
\]

where $ \langle x \rangle = \int dR\, x \, g(R) / \int dR\, g(R) $ --
averaging over the billions+ of snapshots.

\vmm\pause
$R$ won't change too much, so we'll neglect the changes to $1/R^2$:

\[
  \chi \approx
  \sum_j {f_j(k){\frac{e^{ i\delta_j(k)} }{kR_j^2}}}   \biggl\langle e^{i2kR_j}  \biggr\rangle
\]

each path in the sum now has a $g(R)$ with respect to the absorbing atom.

\vmm \hrule \vmm \onslide+<3->

The {\RedEmph{the cumulant expansion}} relates $\langle e^x\rangle$ to $\langle x \rangle$, the moments of $g(x)$:

\[ \biggl\langle e^{i2kR} \biggr\rangle
= \exp \bigg[ \sum_{n=1}^{\infty} { \frac{(2ik)^n}{n!}}C_n  \bigg].
\]

\vfill
\end{cenpage}
\end{slide}

%% Slide
\begin{slide}{The Cumulants and Moments of a Distribution Function}

  \begin{cenpage}{135mm}
  The cumulants $C_n$ of $g(R)$ are related to the moments of $g(R)$:
  $\langle r^n \rangle$,

  with $r= R - R_0$ and $R_0$ is the centroid of the distribution:

  \vmm

    \begin{tabular}{lll}
      $C_1 = \Delta R$  & {\BlueEmph{\tt{deltar}}}    & $ = \langle r \rangle    $    \\
      $C_2  = \sigma^2$ &  {\BlueEmph{\tt{sigma2}}} & $ = \langle r^2 \rangle - \langle r \rangle^2    $   \\
      $ C_3 $ & {\BlueEmph{\tt{third}}} & $ = \langle r^3 \rangle - 3 \langle r^2 \rangle
      \langle r \rangle  + 2 \langle r \rangle^3   $    \\
      $ C_4 $ & {\BlueEmph{\tt{fourth}}} & $ =  \langle r^4 \rangle - 3 \langle r^2 \rangle^2
      - 4\langle r^3 \rangle \langle r \rangle
      +12  \langle r^2 \rangle  \langle r \rangle^2
      - 6\langle r \rangle^4  $   \\
    \end{tabular}

    \vmm

    \begin{postitbox}{80mm}
    $C_3$ (the {\RedEmph{third cumulant}}) can be important in many cases.
  \end{postitbox}

    \vmm\hrule\vmm
    \onslide+<2->

\begin{columns}
  \begin{column}{70mm}

    But: Sometimes, the cumulant expansion isn't good enough.  One can also build models by using
    paths spaced in $R$ (say, at 0.2 {\AA} steps), and model the amplitude of each Path with a
    distribution like (following GNXAS):

    \[    g(R, N, R_0, \sigma, \beta) = \frac{2N [  e^{-\alpha} \alpha^{q-1}]}{\sigma\beta\Gamma(q) }\]

    where  $\alpha = q + 2(R-R_0)/(\beta\sigma)$,  and $q = 4/\beta^2$

\vspace{5mm}

    \end{column}
  \begin{column}{65mm}
    \onslide+<2->
      \includegraphics[width=60mm]{figs/errors/gnxas_histogram}
    \end{column}
  \end{columns}

  \end{cenpage}
\end{slide}


\documentclass[11pt]{article}

\usepackage{alltt,fullpage,graphics,color,epsfig,amsmath, amssymb}
\usepackage{hyperref}
\usepackage{boxedminipage}
\usepackage[ruled,vlined]{algorithm2e}

\newcommand{\floor}[1]{\lfloor #1 \rfloor}
\newcommand{\ceil}[1]{\lceil #1 \rceil}

\title{CS 510 Assignment 1:A Poisson distribution }
\author{Daniel Campos}
\date{February 7th,2021}
\begin{document}
\maketitle
\section{Log Likelihood of Poisson distribution}
\subsection{Solution}
First we find the likelihood function
\begin{equation}
    L(u) = \prod_{i = 1}^{n} \frac{{u^x}{e^{-u}}}{x!}
\end{equation}
Next we find the log likelihood function
\begin{equation}
L(u) = \sum_{i=1}^{n} (x_i*log(u) - u - log(x_i)!)
\end{equation}
\begin{equation}
L(u) = log(u) * \sum_{i=1}^{n} x_i - n*u - \sum_{i=1}^{n} log(x_i)!
\end{equation}
\section{Derivative of the log likelihood}
\subsection{Solution}
\begin{equation}
    \frac{d}{du} = \frac{\sum_{i=1}^{n} x_i}{u} - n
\end{equation}
\section{Solve for MLE of U}
\subsection{Solution}
We take our prior derivative and set to 0
\begin{equation}
    \frac{d}{du} = \frac{\sum_{i=1}^{n} x_i}{u} - n = 0
\end{equation}
\begin{equation}
    n= \frac{\sum_{i=1}^{n} x_i}{u}
\end{equation}
\begin{equation}
    nu= \frac{\sum_{i=1}^{n} x_i}{1}
\end{equation}
\begin{equation}
    u= \frac{\sum_{i=1}^{n} x_i}{n}
\end{equation}
\begin{equation}
    u= \bar{x}
\end{equation}
\end{document}


\subsection{Simulation}
In this section, the synchronisation errors caused by the hardware, the carrier phase and frequency shift are simulated, then algorithms allowing to reduce them are introduced.

\subsubsection{Synchronisation errors}

On \autoref{fig:CFOphi} is shown how the CFO affects the constellation. The linearly increasing phase shift caused by the CFO causes a rotation of the symbols. This effect is first manually corrected to study the impact of ISI only.

\begin{figure}[H]
    \centering
    \input{fig/CFOphi}
    \caption{Illustration of the effect of the CFO phase shift on a 4-QAM constellation for $\frac{E_b}{N_0} = 15 \si{\deci\bel}$}
    \label{fig:CFOphi}
\end{figure}

The effect of the CFO ISI on the BER curves is shown in \autoref{fig:CFOisi}. Low values of a frequency mismatch between the emitter and receiver quartz (expressed in parts per million, \textit{ppm}) do not disturb the BER too much but increasing value prevent correct communication.

\begin{figure}[H]
    \centering
    \input{fig/cfoISI}
    \caption{Illustration of the effect of the CFO ISI on the BER curves for a 4-QAM modulation}
    \label{fig:CFOisi}
\end{figure}

Next, a sampling time shift is simulated. The impact of this shift is depicted in \autoref{fig:samplingshift}.

\begin{figure}[H]
    \centering
    \input{fig/samplingshift}
    \caption{Effect of the sampling shift on the BER for 4-QAM}
    \label{fig:samplingshift}
\end{figure}

\subsubsection{Correction algorithms}

As explained later, the sampling time shift is the first unwanted effect that is tackled, thanks to the Gardner algorithm. \autoref{fig:gardner} shows how the Gardner algorithm is able to correct the sampling time shift.

\begin{figure}[H]
    \centering
    \input{fig/gardner}
    \caption{Gardner compensation of the sampling shift without CFO for a 4-QAM modulation}
    \label{fig:gardner}
\end{figure}

The main parameter of the Gardner algorithm is the factor $\kappa$. \autoref{fig:gardnerK} shows how increasing $\kappa$ allows to have faster convergence, but at the cost of a less stable estimate. All the subsequent Figures have been generated for a 4-QAM modulation.

\begin{figure}[H]
    \centering
    \input{fig/gardnerK}
    \caption{Influence of $\kappa$ on the convergence of the Gardner algorithm}
    \label{fig:gardnerK}
\end{figure}

The Gardner algorithm is robust to CFO, as shown in \autoref{fig:gardnerCFO}. Its convergence is nearly unaffected by the presence of realistic CFO.

\begin{figure}[H]
    \centering
    \input{fig/gardnerCFO}
    \caption{Illustration of the robustness of the Gardner to CFO, $\kappa = 0.1$}
    \label{fig:gardnerCFO}
\end{figure}

After the correction of the sampling time shift, the CFO and the Time of Arrival (ToA) are estimated by sending a pilot (a known sequence) in between frames of data. The main parameters are the length of the pilot $N$ and the size of the averaging windows $K$. The following Figures depict the depency of the estimation of the ToA and the CFO on these parameters. It appears that when $N=40$ and $K=16$, both estimations can be trusted when the SNR is high enough ($> \si{5}{dB}$).

\begin{figure}[H]
    \centering
    \input{fig/dftoaN}
    \caption{Impact of the pilot length on the CFO estimation for a 4-QAM}
    \label{fig:dftoaN}
\end{figure}

\begin{figure}[H]
    \centering
    \input{fig/stdevt_N}
    \caption{Impact of the pilot length on the ToA estimation for a 4-QAM}
    \label{fig:shiftoaN}
\end{figure}

\begin{figure}[H]
    \centering
    \input{fig/stdevf_K}
    \caption{Impact of the size of the averaging window $K$ on the CFO estimation for a 4-QAM}
\end{figure}

\begin{figure}[H]
    \centering
    \input{fig/stdevt_K}
    \caption{Impact of the size of the averaging window $K$ on the ToA estimation for a 4-QAM}
\end{figure}

On the next two figures, the robustness of the frame acquisition to CFO is clearly visible as it has nearly no impact on the estimations.

\begin{figure}[H]
    \centering
    \input{fig/stdevt_CFO}
    \caption{Impact of the CFO and the sample time shift on the CFO estimation}
\end{figure}

\begin{figure}[H]
    \centering
    \input{fig/stdevf_CFO}
    \caption{Impact of the CFO and the sample time shift on the CFO estimation}
\end{figure}

\subsection{Questions}

\subsubsection{Questions regarding the simulation}

\paragraph{Derive analytically the baseband model of the channel including the synchronisation errors.} \mbox{}

Assuming that the oscillators from the emitter and the receiver differ from each other by a pulsation $\Delta \omega$ and a phase shift $\phi$, the signal at the receiver can be written as a function of the complex envelope of the emitted signal $e_s(t)$ (neglecting the noise):

\begin{equation*}
    r(t) = \Re \left\{ e_s(t) \right\} \cos ( \omega_c t ) + \Im \left\{ e_s(t) \right\} \sin ( \omega_c t )
\end{equation*}

The complex envelope must be computed to go to the baseband model.

\begin{equation*}
    \begin{split}
    e_r(t) & = r(t) \cos \left[ (\omega_c + \Delta \omega) t \right] + r(t) \sin \left[ (\omega_c + \Delta \omega) t \right] \\
    & =  \cos \left[ (\omega_c + \Delta \omega) t \right] \times \left[\Re \left\{ e_s(t) \right\} \cos ( \omega_c t ) + \Im \left\{ e_s(t) \right\} \sin ( \omega_c t ) \right]\\
    & +  \sin \left[ (\omega_c + \Delta \omega) t \right] \times \left[\Im \left\{ e_s(t) \right\} \cos ( \omega_c t ) + \Im \left\{ e_s(t) \right\} \sin ( \omega_c t ) \right]
    \end{split}
\end{equation*}

Using the Simpsons rules and lowpass filtering out the high frequency components, one can find

\begin{equation*}
\begin{split}
    &e_r(t) =  \Re \left\{ e_s(t) \right\} \cos ( \Delta \omega_c t + \phi) - j   \Im \left\{ e_s(t) \right\} \sin ( \Delta \omega_c t + \phi)\\
    &\hspace{3cm} \Rightarrow e_r(t) = e_s(t) \cdot e^{j\Delta \omega t + \phi}
\end{split}
\end{equation*}

Hence, the synchronisation errors can be modelled by a multiplication of the baseband model by $ e^{j\Delta \omega t + \phi}$.

\paragraph{How do you separate the impact of the carrier phase drift and ISI due to the CFO in your simulation?} \mbox{}

The linearly increasing phase shift is manually perfectly compensated after the convolution with the second halfroot filter, taking into account the discarded samples of the convolution.

\paragraph{How do you simulate the sampling time shift in practice?} \mbox{}

First, the sampling frequency is significantly increased to increase the accuracy. The sampling time shift is then implemented as a shift in the indexes when downsampling the received signal.

\paragraph{How do you select the simulated $E_b/N_0$ ratio?} \mbox{}

The SNR should be sufficiently high so that the various algorithms in the communication chain converge. However, a noiseless simulation is not realistic. The SNR is thus fixed to typical values ranging from 5 to 10 \si{\deci\bel}.

\paragraph{How do you select the lengths of the pilot and data sequences?} \mbox{}

The length of a data sequence is selected to ensure a correct phase interpolation between two pilot sequences. If it is too long, the phase interpolation is may be incorrect since $e^{j (x+ k 2 \pi)} = e^{j x}$ and the residual CFO may be incorrectly determined.
The length of the pilot should be sufficiently long to get a correct estimate of the ToA and the CFO. As shown in the results of the simulations, the remaining CFO at SNR greater than \si{5}{dB} is sufficiently low when $N>20$. However, the length and rate of repetitions of the pilot should not be too high because they reduce the channel throughput.

\subsubsection{Questions regarding the communication system}

\paragraph{In which order are the synchronisation effects estimated and compensated? Why?} \mbox{}

The Gardner's algorithm being robust to CFO, it is first used to correct the sampling time shift. Then the CFO and the ToA are handled by the frame acquisition. Between the the pilot sequences, the residual CFO is approximated by linear interpolation.

\paragraph{Explain intuitively how the error is computed in the Gardner algorithm. Why is the Gardner algorithm robust to CFO?} \mbox{}

At a given time, the estimation of the time shift is obtained from the previous estimation weighted by a certain factor. This factor depends on the middle sample between the two time steps. The direction of the correction is given by the sign of the middle value, and the magnitude of the correction is given by its magnitude. The correction will be very low if a zero crossing happens. For example, if the sampling happens too late for a downwards crossing, the value of the correction factor will be negative.

By looking at the mathematical form of the feedback implemented in the Gardner algorithm, it appears that for reasonable values of the CFO, the phases will nearly cancel each other thanks to the conjugate operator.

\paragraph{Explain intuitively why the differential cross-correlator is better suited than the usual cross-correlator? Isn’t it interesting to start the summation at $k$ = 0 (no time shift)?} \mbox{}

The usual cross-correlator is nearly optimal according to the ML estimator but it necessitates a computationally heavy 2D exhaustive search in order to be robust to CFO. Instead, the differential cross-correlator allows for a solution of lower complexity which first estimates the ToA then the CFO.

It is not interesting to start the summation at $k=0$ since the term $D_0 \left[ n \right]$ does not carry any information other than the power of the window.

\newpage

\paragraph{Are the frame and frequency acquisition algorithms optimal? If yes, give the optimisation criterion.} \mbox{}

The algorithm is not optimal because the optimisation is not done for both the ToA and the CFO at the same time. However, it is sufficient when both CFO and the noise have reasonable values, as it is the case in practical applications.

The optimisation criterion is the maximum likelihood of observing the received symbol $y$ knowing the pilot and the CFO.


\chapter{A Proof of Theorem \ref{SubG8}}\label{AppB}
This chapter is devoted to a concrete proof of Theorem \ref{IrreducibleSubs}.
We use simillar methods to the ones we already applied in chapter 3. After
reducing the lattices we compare the sublattices with the rational ones introduced
in \cite{Kunyavski} and \cite{Nicole1}.
 \section{ (5,6,3 ) }
The group is generated by
$$
\left[ \begin {array}{ccccc} -1&0&0&0&0\\0&0&0&0&-
1\\0&0&0&-1&0\\0&0&-1&0&0
\\0&-1&0&0&0\end {array} \right].
$$
The corresponding lattice is sign permutation. This implies rationality of
the corresponding torus.
 \section{(5,18,28)}
The group is generated by
$$
 \left[ \begin {array}{ccccc} 1&0&0&0&0\\0&0&1&0&0
\\0&1&0&0&0\\0&0&0&0&1
\\0&0&0&1&0\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} -1&0&0&0&0\\0&0&0&0&1
\\0&0&0&1&0\\0&0&1&0&0
\\0&1&0&0&0\end {array} \right]
$$

The corresponding lattice is a sign permutation lattice. Thus it is hereditarily rational.

 \section{(5,19,14)}
Algorithm 2 produces the change of basis matrix
$$
\left[ \begin {array}{ccccc} 0&1&0&0&0\\ 0&0&1&0&0
\\ 0&0&0&1&0\\ 0&0&0&0&1
\\ 1&0&1&0&0\end {array} \right]
$$
With the above transformation we can see the new representative is generated by
$$
 \left[ \begin {array}{cccc|c} 0&0&1&-1&0\\0&-1&0&-2
&0\\1&0&0&1&0\\0&0&0&1&0
\\ \hline 0&1&0&1&1\end {array} \right]
 \left[ \begin {array}{cccc|c} 0&0&1&0&0\\0&1&0&2&0
\\1&0&0&0&0\\0&0&0&-1&0
\\\hline 0&0&0&-1&1\end {array} \right]
$$
we can produce
$$
\exactseq{}
.$$
The corresponding group to $M$ has GAP ID [4,5,1,10] and also can be generated by
$$
 \left[ \begin {array}{ccc|c} 1&0&-1&0\\ 0&1&-1&0
\\ 0&0&-1&0\\ \hline 0&0&0&-1\end {array}
 \right]
 \tand
 \left[ \begin {array}{ccc|c} 0&1&0&0\\ 1&0&0&0
\\ 0&0&1&0\\  \hline 0&0&0&-1\end {array}
 \right]
$$
So $M$ decomposes into a direct sum of a rank one sign permutation lattice
(which is hereditarily rational) and a rank 3 lattice given by a group, $H$,
generated by
$$
\left[ \begin {array}{cc|c} 1&0&-1\\ 0&1&-1
\\ \hline 0&0&-1\end {array}
 \right]
 \tand
 \left[ \begin {array}{cc|c} 0&1&0\\ 1&0&0
\\ \hline 0&0&1\end {array}
 \right]
$$
Looking at the generators of $H$ tells us we can form
$$0 \longrightarrow \Z^- \longrightarrow L_H \longrightarrow P \longrightarrow 0$$
where $P$ is given by the group generated by
$$
\begin{bmatrix}
1&0\\
0& 1
\end{bmatrix}
\tand
\begin{bmatrix}
0&1\\
1&0
\end{bmatrix}
.$$
Since $P$ is a permutation lattice, by Corollary \ref{permcoker} we conclude
that [4,5,1,10] is hereditarily rational. This implies our desired result
which is hereditarily rationality of (5,19,14).

 \section{(5,22,14)}
The group is generated by
$$
  \left[ \begin {array}{cccc|c} 1&0&0&0&-1\\0&1&0&0&1
\\0&0&1&0&1\\0&0&0&1&-1
\\ \hline 0&0&0&0&-1\end {array} \right] ,
 \left[ \begin {array}{cccc|c} 1&0&0&0&0\\0&0&0&1&-1
\\0&0&1&0&0\\0&1&0&0&1
\\ \hline 0&0&0&0&1\end {array} \right]
\tand
 \left[ \begin {array}{cccc|c} 0&0&1&0&1\\0&1&0&0&0
\\1&0&0&0&-1\\0&0&0&1&0
\\ \hline 0&0&0&0&1\end {array} \right]
$$
Now we define $P$ to be the lattice corresponding to, $H$, generated by
$$
  \left[ \begin {array}{cccc} 1&0&0&0\\0&1&0&0
\\0&0&1&0\\0&0&0&1
\end {array} \right] ,
 \left[ \begin {array}{cccc} 1&0&0&0\\0&0&0&1
\\0&0&1&0\\0&1&0&0
\end {array} \right]
\tand
 \left[ \begin {array}{cccc} 0&0&1&0\\0&1&0&0
\\1&0&0&0\\0&0&0&1
\end {array} \right],
$$
We can see the corresponding lattice to (5,22,14), $L$, fits into the following exact sequence
$$0\longrightarrow \Z^- \longrightarrow L \longrightarrow P \longrightarrow 0.$$
and since $P$ is permutation, by Corollary \ref{permcoker} we can conclude that $L$ is hereditarily rational.
 \section{(5,57,8)}
The group is generated by
$$
\left[ \begin {array}{ccccc} -1&0&0&0&0\\0&0&0&1&0
\\0&-1&0&0&0\\0&0&0&0&1
\\0&0&-1&0&0\end {array} \right]
$$
and the corresponding lattice is a sign permutation lattice which is hereditarily rational.
  \section{(5,81,54)}
The group is generated by
$$
 \left[ \begin {array}{ccccc} 1&0&1&0&1\\0&1&0&0&0
\\0&0&0&0&-1\\0&0&1&1&1
\\0&0&-1&0&0\end {array} \right],
 \left[ \begin {array}{ccccc} 0&0&1&0&0\\-1&0&0&1&0
\\1&0&0&0&0\\0&1&1&0&0
\\-1&0&-1&0&-1\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 0&0&-1&-1&-1\\0&1&0&0
&0\\0&0&1&0&0\\-1&0&-1&0&-1
\\0&0&0&0&1\end {array} \right]
$$
The dual group is generated by
$$
  \left[ \begin {array}{ccccc} 1&0&0&0&0\\0&1&0&0&0
\\1&0&0&1&-1\\0&0&0&1&0
\\1&0&-1&1&0\end {array} \right] ,
 \left[ \begin {array}{ccccc} 0&-1&1&0&-1\\0&0&0&1&0
\\1&0&0&1&-1\\0&1&0&0&0
\\0&0&0&0&-1\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 0&0&0&-1&0\\0&1&0&0&0
\\-1&0&1&-1&0\\-1&0&0&0&0
\\-1&0&0&-1&1\end {array} \right]
$$
Algorithm (2) produces the change of basis matrix
$$
 \left[ \begin {array}{ccccc} 0&1&0&0&0\\ 0&0&1&0&0
\\ 0&0&0&1&0\\ 0&0&0&0&1
\\ 1&-2&1&-1&-1\end {array} \right]
$$
With the above transformation we can see the new representative is generated by
$$
 \left[ \begin {array}{cccc|c} 1&2&0&2&0\\0&-1&0&-2&0
\\0&2&1&2&0\\0&0&0&1&0
\\ \hline 0&1&0&1&1\end {array} \right],
 \left[ \begin {array}{cccc|c} 0&2&1&0&0\\0&-1&0&0&0
\\1&2&0&0&0\\0&0&0&-1&0
\\ \hline 0&1&0&0&1\end {array} \right]
\tand
 \left[ \begin {array}{cccc|c} 1&-2&-2&-2&0\\0&2&1&1
&0\\0&-2&-1&-2&0\\0&-1&-1&0&0
\\ \hline 0&-1&-1&-1&1\end {array} \right]
$$
Now by considering $M$ to be the corresponding lattice to
$$
 \left[ \begin {array}{cccc} 1&2&0&2\\0&-1&0&-2
\\0&2&1&2\\0&0&0&1\end {array}
 \right],
 \left[ \begin {array}{cccc} 0&2&1&0\\0&-1&0&0
\\1&2&0&0\\0&0&0&-1\end {array}
 \right]
 \tand
 \left[ \begin {array}{cccc} 1&-2&-2&-2\\0&2&1&1
\\0&-2&-1&-2\\0&-1&-1&0
\end {array} \right]
$$
we can produce
$$
\exactseq{}
.$$
The generators of [4,13,7,5] (another representative of the corresponding conjugacy class to $M$) are
$$
\left[ \begin {array}{ccc|c} 0&-1&1&0\\ 0&-1&0&0
\\ 1&-1&0&0\\ \hline 0&0&0&-1\end {array}
 \right],
 \left[ \begin {array}{ccc|c} 0&1&-1&0\\ 1&0&-1&0
\\ 0&0&-1&0\\ \hline 0&0&0&1\end {array}
 \right]
 \tand
 \left[ \begin {array}{ccc|c} 0&0&-1&0\\ 1&0&-1&0
\\ 0&1&-1&0\\ \hline 0&0&0&1\end {array}
 \right]
$$
The generators of rank 3 lattice are
$$
 \left[ \begin {array}{ccc} 0&-1&1\\ 0&-1&0
\\ 1&-1&0\end {array} \right],
 \left[ \begin {array}{ccc} 0&1&-1\\ 1&0&-1
\\ 0&0&-1\end {array} \right]
\tand
 \left[ \begin {array}{ccc} 0&0&-1\\ 1&0&-1
\\ 0&1&-1\end {array} \right]
$$
the CrystCatZClass of the former group is [3,4,6,4] which is rational by \cite{Kunyavski} and its subgroups are  [ 3, 1, 1, 1 ], [ 3, 2, 1, 2 ], [ 3, 2, 2, 2 ], [ 3, 3, 1, 4 ], [ 3, 3, 2, 4 ], [ 3, 4, 2, 2 ] and  [ 3, 4, 6, 4 ] where all of them are rational. This implies that (5, 81, 54) is hereditarily rational.
  \section{(5,98,28)}
The group is generated by
$$
 \left[ \begin {array}{ccccc} 0&-1&0&0&0\\-1&0&0&0&0
\\-1&0&1&0&-1\\0&-1&0&1&1
\\-1&1&0&0&-1\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 1&0&0&0&0\\0&1&0&0&0
\\0&1&-1&0&0\\0&0&1&0&-1
\\0&1&-1&-1&0\end {array} \right]
$$
The dual group is generated by
$$
 \left[ \begin {array}{ccccc} 0&-1&-1&0&-1\\-1&0&0&
-1&1\\0&0&1&0&0\\0&0&0&1&0
\\0&0&-1&1&-1\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 1&0&0&0&0\\0&1&1&0&1
\\0&0&-1&1&-1\\0&0&0&0&-1
\\0&0&0&-1&0\end {array} \right]
$$
Algorithm (1) produces the change of basis matrix
$$
 \left[ \begin {array}{ccccc} 1&0&-1&0&0\\ 0&1&-1&0&0
\\ 0&0&0&1&0\\ 0&0&0&0&1
\\ 2&-2&-1&1&-2\end {array} \right]
$$
With the above transformation we can see the new representative is generated by
$$
  \left[ \begin {array}{cccc|c} -6&-5&0&-2&0\\5&4&0&2
&0\\-3&-3&1&0&0\\5&5&0&1&0
\\ \hline 3&2&0&1&1\end {array} \right]
\tand
 \left[ \begin {array}{cccc|c} 5&6&0&0&0\\-4&-5&0&0&0
\\1&2&0&-1&0\\-3&-4&-1&0&0
\\ \hline -2&-3&0&0&1\end {array} \right]
$$
Now by considering $M$ to be the corresponding lattice to
$$
\left[ \begin {array}{cccc} -6&-5&0&-2\\5&4&0&2
\\-3&-3&1&0\\5&5&0&1\end {array}
 \right]
 \tand
 \left[ \begin {array}{cccc} 5&6&0&0\\-4&-5&0&0
\\1&2&0&-1\\-3&-4&-1&0
\end {array} \right]
$$
we can produce
$$
\exactseq{}
.$$
The generators of [4,13,3,3] are
$$
 \left[ \begin {array}{ccc|c} 0&0&1&0\\ 0&1&0&0
\\ 1&0&0&0\\ \hline 0&0&0&-1\end {array}
 \right]
 \left[ \begin {array}{ccc|c} 0&0&-1&0\\ 1&0&-1&0
\\ 0&1&-1&0\\ \hline 0&0&0&-1\end {array}
 \right]
$$
The generators of rank 3 lattice are
$$
  \left[ \begin {array}{ccc} 0&0&1\\ 0&1&0
\\ 1&0&0\end {array} \right]
\tand
 \left[ \begin {array}{ccc} 0&0&-1\\ 1&0&-1
\\ 0&1&-1\end {array} \right]
$$
the GAP ID of the former group is [3,4,6,4] which is hereditarily rational by the argument given in the previous case. So (5,98,28) is hereditarily rational.
  \section{(5,99,57)}
The group is generated by
$$
 \left[ \begin {array}{ccccc} 1&0&1&0&1\\0&1&0&0&0
\\0&-1&-1&0&0\\0&0&1&1&1
\\0&1&0&0&-1\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 0&1&1&0&0\\1&0&0&-1&0
\\0&0&0&1&0\\0&0&1&0&0
\\0&0&-1&-1&-1\end {array} \right]
$$
The dual group is generated by
$$
\left[ \begin {array}{ccccc} 1&0&0&0&0\\0&1&-1&0&1
\\1&0&-1&1&0\\0&0&0&1&0
\\1&0&0&1&-1\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 0&1&0&0&0\\1&0&0&0&0
\\1&0&0&1&-1\\0&-1&1&0&-1
\\0&0&0&0&-1\end {array} \right]
$$
Algorithm (1) produces the change of basis matrix
$$
\left[ \begin {array}{ccccc} 0&1&0&0&0\\ 0&0&1&0&0
\\ 0&0&0&1&0\\ 0&0&0&0&1
\\ 1&2&-1&-1&1\end {array} \right]
$$
With the above transformation we can see the new representative is generated by
$$
\left[ \begin {array}{cccc|c} 1&-2&0&-2&0\\-1&0&0&1
&0\\0&2&1&2&0\\1&-1&0&-2&0
\\ \hline 0&1&0&1&1\end {array} \right]
\tand
 \left[ \begin {array}{cccc|c} -2&-2&-1&0&0\\1&1&1&0
&0\\1&2&0&0&0\\-1&-2&-1&-1&0
\\ \hline 1&1&0&0&1\end {array} \right]
$$
Now by considering $M$ to be the corresponding lattice to
$$
\left[ \begin {array}{cccc} 1&-2&0&-2\\-1&0&0&1
\\0&2&1&2\\1&-1&0&-2\end {array}
 \right]
 \tand
 \left[ \begin {array}{cccc} -2&-2&-1&0\\1&1&1&0
\\1&2&0&0\\-1&-2&-1&-1
\end {array} \right]
$$
we can produce
$$
\exactseq{}
.$$
The generators of [4,12,4,7] are
$$
 \left[ \begin {array}{ccc|c} 0&0&1&0\\ 0&1&0&0
\\ 1&0&0&0\\ \hline 0&0&0&-1\end {array}
 \right]
 \left[ \begin {array}{ccc|c} 0&0&-1&0\\ 1&0&-1&0
\\ 0&1&-1&0\\ \hline 0&0&0&1\end {array}
 \right]
$$
The generators of rank 3 lattice are
$$
 \left[ \begin {array}{ccc} 0&0&1\\ 0&1&0
\\ 1&0&0\end {array} \right]
 \left[ \begin {array}{ccc} 0&0&-1\\ 1&0&-1
\\ 0&1&-1\end {array} \right]
$$
the GAP ID of the former group is [3,4,6,4] which is hereditarily rational by argument given in the previous case. So (5,99,57) is hereditarily rational.
  \section{(5,164,2)}
The group is generated by
$$
 \left[ \begin {array}{cc|ccc} -1&1&0&0&0\\-1&0&0&0&0
\\ \hline 0&0&0&-1&0\\0&0&0&0&1
\\0&0&-1&0&0\end {array} \right]
$$
The corresponding lattice decomposes into a rank 2 lattice which is hereditarily rational and a rank 3 sign permutation lattice which is also hereditarily rational. Hence (5,164,2) is hereditarily rational.
  \section{(5,174,2)}
The group is generated by
$$
  \left[ \begin {array}{ccccc} 0&0&-1&0&-1\\0&0&0&1&
1\\-1&0&0&0&-1\\0&1&0&0&-1
\\0&0&0&0&1\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 0&0&-1&0&-1\\-1&0&1&0
&0\\0&0&-1&1&0\\0&0&-1&0&0
\\0&1&1&0&0\end {array} \right]
$$
The dual group is generated by
$$
 \left[ \begin {array}{ccccc} 0&0&-1&0&0\\0&0&0&1&0
\\-1&0&0&0&0\\0&1&0&0&0
\\-1&1&-1&-1&1\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 0&-1&0&0&0\\0&0&0&0&1
\\-1&1&-1&-1&1\\0&0&1&0&0
\\-1&0&0&0&0\end {array} \right]
$$
Algorithm (1) produces the change of basis matrix
$$
 \left[ \begin {array}{ccccc} 1&0&0&0&0\\ 0&0&1&0&0
\\ 0&0&0&1&0\\ 0&0&0&0&1
\\ 1&-1&0&0&-1\end {array} \right]
$$
With the above transformation we can see the new representative is generated by
$$
\left[ \begin {array}{cccc|c} 0&-1&1&0&0\\-1&0&0&-1
&0\\0&0&0&-1&0\\0&0&-1&0&0
\\ \hline 0&0&-1&-1&1\end {array} \right]
\tand
 \left[ \begin {array}{cccc|c} -1&0&0&-1&0\\0&-1&1&0
&0\\0&-1&0&0&0\\1&0&0&0&0
\\ \hline 1&-1&0&0&1\end {array} \right]
$$
Now by considering $M$ to be the corresponding lattice to
$$
 \left[ \begin {array}{cccc} 0&-1&1&0\\-1&0&0&-1
\\0&0&0&-1\\0&0&-1&0\end {array}
 \right]
 \tand
 \left[ \begin {array}{cccc} -1&0&0&-1\\0&-1&1&0
\\0&-1&0&0\\1&0&0&0\end {array}
 \right]
$$
we can produce
$$
\exactseq{}
.$$
The generators of [4,17,13] are
$$
 \left[ \begin {array}{cc|cc} -1&-1&0&0\\ 0&1&0&0
\\ \hline 0&0&1&1\\  0&0&0&-1\end {array}
 \right]
 \tand
 \left[ \begin {array}{cc|cc} -1&-1&0&0\\ 1&0&0&0
\\ \hline 0&0&-1&-1\\ 0&0&1&0\end {array}
 \right]
$$
and the lattice decomposes into rank 2 lattices which we know they are hereditarily rational. This implies that (5,174,2) is hereditarily rational.
  \section{(5,174,5)}
The group is generated by
$$
 \left[ \begin {array}{cc|ccc} -1&0&0&0&0\\1&1&0&0&0
\\ \hline 0&0&0&0&1\\0&0&0&1&0
\\0&0&1&0&0\end {array} \right]
\tand
 \left[ \begin {array}{cc|ccc} -1&-1&0&0&0\\1&0&0&0&0
\\ \hline 0&0&0&0&1\\0&0&-1&0&0
\\0&0&0&-1&0\end {array} \right]
$$
and the lattice decomposes into a rank 2 lattice and a rank 3 sign permutation lattice, both of which are hereditarily rational. This implies that (5,174,5) is hereditarily rational.
  \section{(5,389,4)}
The group is generated by
$$
 \left[ \begin {array}{ccccc} 1&0&0&-1&0\\0&1&0&-1&0
\\0&0&0&-1&-1\\0&0&0&-1&0
\\0&0&-1&1&0\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 1&0&0&-1&0\\1&0&0&0&1
\\1&-1&0&0&0\\1&0&-1&0&0
\\-1&0&0&0&0\end {array} \right]
$$
The dual group is generated by
$$
 \left[ \begin {array}{ccccc} 1&0&0&0&0\\0&1&0&0&0
\\0&0&0&0&-1\\-1&-1&-1&-1&1
\\0&0&-1&0&0\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 1&1&1&1&-1\\0&0&-1&0&0
\\0&0&0&-1&0\\-1&0&0&0&0
\\0&1&0&0&0\end {array} \right]
$$
Algorithm (1) produces the change of basis matrix
$$
 \left[ \begin {array}{ccccc} 0&1&0&0&0\\ 0&0&1&0&0
\\ 0&0&0&1&0\\ 0&0&0&0&1
\\ 1&0&1&0&-1\end {array} \right]
$$
With the above transformation we can see the new representative is generated by
$$
 \left[ \begin {array}{cccc|c} 1&0&-1&0&0\\0&0&0&-1&0
\\0&0&-1&0&0\\0&-1&0&0&0
\\ \hline 0&0&-1&0&1\end {array} \right]
\tand
 \left[ \begin {array}{cccc|c} 0&0&0&1&0\\-1&0&1&0&0
\\0&-1&0&0&0\\0&0&-1&0&0
\\ \hline 0&0&-1&0&1\end {array} \right]
$$
Now by considering $M$ to be the corresponding lattice to
$$
 \left[ \begin {array}{cccc} 1&0&-1&0\\0&0&0&-1
\\0&0&-1&0\\0&-1&0&0\end {array}
 \right]
 \tand
 \left[ \begin {array}{cccc} 0&0&0&1\\-1&0&1&0
\\0&-1&0&0\\0&0&-1&0\end {array}
 \right]
$$
we can produce
$$
\exactseq{}
.$$
the above group is [4,21,3,2] which has the following subgroups [ 4, 1, 1, 1 ], [ 4, 3, 1, 3 ], [ 4, 5, 1, 1 ], [ 4, 11, 1, 1 ], [ 4, 17, 1, 2 ], [ 4, 17, 1, 3 ], [ 4, 21, 1, 1 ] and [ 4, 21, 3, 2 ]
where all of them are rational.

  \section{(5,901,3)}
The group is generated by
$$
 \left[ \begin {array}{ccccc} 0&1&0&0&0\\1&0&0&0&0
\\0&0&0&0&-1\\0&0&0&1&0
\\0&0&-1&0&0\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 0&0&0&0&-1\\1&0&0&0&0
\\0&1&0&0&0\\0&0&1&0&0
\\0&0&0&-1&0\end {array} \right]
$$
The dual group is generated by
$$
 \left[ \begin {array}{ccccc} 0&1&0&0&0\\1&0&0&0&0
\\0&0&0&0&-1\\0&0&0&1&0
\\0&0&-1&0&0\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 0&1&0&0&0\\0&0&1&0&0
\\0&0&0&1&0\\0&0&0&0&-1
\\-1&0&0&0&0\end {array} \right]
$$
Algorithm (1) produces the change of basis matrix
$$
 \left[ \begin {array}{ccccc} 1&0&0&0&0\\ 0&0&1&0&0
\\ 0&0&0&1&0\\ 0&0&0&0&1
\\ 1&1&1&1&-1\end {array} \right]
$$
With the above transformation we can see the new representative is generated by
$$
 \left[ \begin {array}{cccc|c} -1&0&0&0&0\\-1&0&0&-1
&0\\-1&0&1&0&0\\1&-1&0&0&0
\\ \hline 1&0&0&0&1\end {array} \right]
\tand
 \left[ \begin {array}{cccc|c} -1&0&0&-1&0\\-1&0&0&0
&0\\-1&1&0&0&0\\1&0&-1&0&0
\\ \hline 1&0&0&0&1\end {array} \right]
$$
Now by considering $M$ to be the corresponding lattice to
$$
 \left[ \begin {array}{cccc} -1&0&0&0\\-1&0&0&-1
\\-1&0&1&0\\1&-1&0&0\end {array}
 \right]
 \tand
 \left[ \begin {array}{cccc} -1&0&0&-1\\-1&0&0&0
\\-1&1&0&0\\1&0&-1&0\end {array}
 \right]
$$
we can produce
$$
\exactseq{}
.$$
The lattice $M$ corresponds to [4,27,3,1] with subgroups [ 4, 1, 1, 1 ], [ 4, 3, 1, 3 ], [ 4, 27, 1, 1 ], [ 4, 27, 3, 1 ] where all of them are rational. So (5,901,3) is hereditarily rational.

 \section{(5,918,4)}
The group is generated by
$$
  \left[ \begin {array}{ccccc} 0&-1&0&-1&0\\ \noalign{\medskip}-1&1&0&1
&1\\ -1&0&0&0&0\\ 0&-1&1&0&0
\\ 1&0&0&-1&0\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 0&1&0&1&1\\ 0&-1&-1&-1
&-1\\ 0&0&-1&0&-1\\ -1&0&1&1&1
\\ 0&1&0&0&1\end {array} \right]
$$
The dual group is generated by
$$
\left[ \begin {array}{ccccc} 0&-1&-1&0&1\\ -1&1&0&-
1&0\\ 0&0&0&1&0\\ -1&1&0&0&-1
\\ 0&1&0&0&0\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} 0&0&0&-1&0\\ 1&-1&0&0&
1\\ 0&-1&-1&1&0\\ 1&-1&0&1&0
\\ 1&-1&-1&1&1\end {array} \right]
$$
Algorithm (1) produces the change of basis matrix
$$
  \left[ \begin {array}{ccccc} 0&1&1&-2&1\\ 1&0&0&0&0
\\ 0&1&0&0&0\\ 0&0&1&0&0
\\ 0&0&0&1&0\end {array} \right]
$$
With the above transformation we can see the new representative is generated by
$$
  \left[ \begin {array}{ccccc} 1&0&1&1&0\\ -1&0&-1&0&0
\\ -2&1&-1&0&0\\ 2&0&1&0&0
\\ -1&0&-1&0&1\end {array} \right]
\tand
 \left[ \begin {array}{ccccc} -1&-1&-1&-1&0\\ 1&-1&1
&0&0\\ 1&1&2&2&0\\ -1&0&-2&-1&0
\\ 1&0&1&1&1\end {array} \right]
$$
Now by considering $M$ to be the corresponding lattice to
$$
\left[ \begin {array}{cccc} 1&0&1&1\\ -1&0&-1&0
\\ -2&1&-1&0\\ 2&0&1&0\end {array}
 \right]
 \tand
 \left[ \begin {array}{cccc} -1&-1&-1&-1\\ 1&-1&1&0
\\ 1&1&2&2\\ -1&0&-2&-1
\end {array} \right]
$$
we can produce
$$
\exactseq{}
.$$
The lattice $M$ corresponds to [4,31,1,2] which is a subgroup of $[4,31,7,1]$. In \cite{Nicole1} it is shown that $[4,31,7,1]$ is hereditarily rational and so is (5,918,4).


\documentclass[14pt]{extarticle}

\usepackage{listings}
\usepackage{xcolor}
\usepackage{amsmath,amsfonts}
\usepackage{hyperref}

\title{GEometric Algebra}

\author{LAK132}

\begin{document}

\color{white}
\pagecolor{black!90}

\setlength{\parindent}{0pt}
\setlength{\parskip}{6pt}

\maketitle

\pagebreak
\tableofcontents

%
% Dot product
%

\pagebreak
\section{Dot product}
\label{sec:dot-product}

The dot product (\(\cdot\)) of 2 vectors (\(a\) and \(b\))
creates a scalar:

\newcommand{\dotpart}[4]{ #1_#2 #3_#4 }

\newcommand{\expdot}[3][]{
  #1 \dotpart{#2}{1}{#3}{1} + \cdots + #1 \dotpart{#2}{n}{#3}{n}}

\( a \cdot b = \expdot{a}{b} \)

Note:

\( a \cdot b = b \cdot a \)

%
% Outer product
%

\pagebreak
\section{Outer product}
\label{sec:outer-product}

The outer product (\(\wedge\)) of 2 vectors (\(a\) and \(b\))
creates a bivector (\(\mathbf{A}\)):

\newcommand{\wedgepart}[4]{(#1_#2 #3_#4 - #1_#4 #3_#2) (e_#2 \wedge e_#4)}

\newcommand{\expwedge}[2]{
  \wedgepart{#1}{1}{#2}{2} + \cdots + \wedgepart{#1}{n}{#2}{1}}

\newcommand{\expwedgepre}[3]{
  #1 \wedgepart{#2}{1}{#3}{2} + \cdots + #1 \wedgepart{#2}{n}{#3}{1}}

\newcommand{\expwedgepost}[3]{
  \wedgepart{#1}{1}{#2}{2} #3 + \cdots + \wedgepart{#1}{n}{#2}{1} #3}

\newcommand{\expwedgeprepost}[4]{
  #1 \wedgepart{#2}{1}{#3}{2} #4 + \cdots + #1 \wedgepart{#2}{n}{#3}{1} #4}

\(
  \mathbf{A}
  = a \wedge b \\
  = \expwedge{a}{b}
  % = (a_1 b_2 - a_2 b_1) (e_1 \wedge e_2)
  % + (a_2 b_3 - a_3 b_2) (e_2 \wedge e_3)
  % + \cdots
  % + (a_n b_1 - a_1 b_n) (e_n \wedge e_1)
\)

Where \(n\) is the dimension of \(a\) and \(b\),
and \(e_{1..n}\) are the basis vectors.

If we let
\( A_{e_i e_j} = (a_{e_i} b_{e_j} - a_{e_j} b_{e_i})(e_i \wedge e_j) \),
then this simplifies to:

\(
  \mathbf{A}
  = a \wedge b \\
  = (a_1 e_1 + \cdots + a_n e_n) \wedge (b_1 e_1 + \cdots + b_n e_n) \\
  = a_1 b \\
  = A_{e_1 e_2} + A_{e_2 e_3} + \cdots + A_{e_n e_1}
\)

Chaining:

\(
  a_1 \wedge a_2 \wedge \cdots \wedge a_r
  = \frac{1}{r!} \sum\limits_{\sigma \in \mathfrak{G}_r}
  \textrm{sgn} (\sigma) a_{\sigma (1)} a_{\sigma (2)} \cdots a_{\sigma (r)} ,
\)

Note:

\( a \wedge b = - (b \wedge a) \)

In 2D:

\(
  a \wedge b
  = \mathbf{A}
  = A_{xy}
  = (a_x b_y - a_y b _x) (x \wedge y)
\)

In 3D:

\(
  a \wedge b
  = \mathbf{A}
  = A_{xy} + A_{yz} + A_{zx} \\
  = (a_x b_y - a_y b_x) (x \wedge y)
  + (a_y b_z - a_z b_y) (y \wedge z)
  + (a_z b_x - a_x b_z) (z \wedge x)
\)

\pagebreak
In 3D we also find that the outer product looks very similar to
the cross product:

\(
  a \wedge b = \\
  (a_x b_y - a_y b_x) (x \wedge y) + \\
  (a_z b_x - a_x b_z) (z \wedge x) + \\
  (a_y b_z - a_z b_y) (y \wedge z)
\)

\(
  a \times b = \\
  (a_x b_y - a_y b_x) z + \\
  (a_z b_x - a_x b_z) y + \\
  (a_y b_z - a_z b_y) x
\)

%
% Geometric product
%

\pagebreak
\section{Geometric product}
\label{sec:geometric-product}

The geometric product of 2 vectors (\(a\) and \(b\))
creates a rotor (\(\mathbf{R}\)):

\(
  ab
  = \mathbf{R}
  = \frac{1}{2} (ab + ba) + \frac{1}{2} (ab - ba)
  = (a \cdot b) + (a \wedge b)
\)

For perpendicular vectors (\(c\) and \(d\)) we find that \( c \cdot d = 0 \),
hence:

\( cd = (c \cdot d) + (c \wedge d) = 0 + (c \wedge d) = c \wedge d \)

Because basis vectors are perpendicular,
we can use this to simplify the outer product equation:

\(
  a \wedge b \\
  = (a_1 b_2 - a_2 b_1) e_1 e_2
  + (a_2 b_3 - a_3 b_2) e_2 e_3
  + \cdots
  + (a_n b_1 - a_1 b_n) e_n e_1 \\
  = (a_1 b_2 - a_2 b_1) e_{12}
  + (a_2 b_3 - a_3 b_2) e_{23}
  + \cdots
  + (a_n b_1 - a_1 b_n) e_{n1}
\)

In 2D:

\(
  ab
  = (a_x b_x + a_y b_y)
  + (a_x b_y - a_y b_x) xy
\)

In 3D:

\(
  ab \\
  = (a_x b_x + a_y b_y + a_z b_z)
  + (a_x b_y - a_y b_x) xy
  + (a_y b_z - a_z b_y) yz
  + (a_z b_x - a_x b_z) zx
\)

%
% Reflection
%

\pagebreak
\section{Reflection}
\label{sec:reflection}

Given a vector \(v\) and unit vector \(a\), \(v_\parallel\) is \(v\)
projected onto \(a\):

\( v_\parallel = a (a \cdot v) \)

\(v_\perp\) is the the vector perpendicular to \(a\) that sums with
\(v_\parallel\) to equal \(v\):

\( v_\perp = v - v_\parallel \)

To reflect \(v\) by the plane perpendicular to \(a\),
we define the function \(R_a(v)\):

\( R_a(v) = -ava \)

\(
  va \\
  = (v \cdot a) + (v \wedge a) \\
  = ((v_\parallel + v_\perp) \cdot a)
  + ((v_\parallel + v_\perp) \wedge a) \\
  = (v_\parallel \cdot a) + (v_\perp \cdot a)
  + (v_\parallel \wedge a) + (v_\perp \wedge a) \\
  = (v_\parallel \cdot a) + (0) + (0) + (v_\perp \wedge a) \\
  = (v_\parallel \cdot a) + (v_\perp \wedge a) \\
  = (v_\parallel \cdot a) + (v_\perp a)
\)

\(
  ava \\
  = a (v a) \\
  = a ((v_\parallel \cdot a) + (v_\perp a)) \\
  = (a (v_\parallel \cdot a)) + (a (v_\perp a)) \\
  = (a (v_\parallel \cdot a)) + (a v_\perp a) \\
  = v_\parallel + (a v_\perp a) \\
  = v_\parallel - (v_\perp a a) \\
  = v_\parallel - v_\perp
\)

\(
  -ava \\
  = -(v_\parallel - v_\perp)\\
  = v_\perp - v_\parallel \\
  = v - 2 a (a \cdot v)
\)

%
% Rotation
%

\pagebreak
\section{Rotataion}
\label{sec:rotation}

% Basis vectors table:
%
% 2D:
%   2   1
%
% 3D:
%   2   1
% 123   3
%
% 4D:
%   2   1 123
% 123   3   2
% 134 234   4
%
% 5D:
%   2   1 123 124
% 123   3   2 234
% 134 234   4   3
% 145 245 345   5
%
% 6D:
%   2   1 123 124 125
% 123   3   2 234 235
% 134 234   4   3 345
% 145 245 345   5   4
% 156 256 356 456   6
%
% 7D:
%   2   1 123 124 125 126
% 123   3   2 234 235 236
% 134 234   4   3 345 346
% 145 245 345   5   4 456
% 156 256 356 456   6   5
%   6 126 136 146 156   1

\subsection{2D}
\label{subsec:rotation-2D}

Rotor \( R = R_a + R_{xy} \mathbf{xy} \)

Vector \( v = v_x \mathbf{x} + v_y \mathbf{y} \)

Trivector \( T = R v = R_a v + R_{xy} \mathbf{xy} v \)

\(
  = R_a (v_x \mathbf{x} + v_y \mathbf{y})
  + R_{xy} \mathbf{xy} (v_x \mathbf{x} + v_y \mathbf{y})
\)

\(
  = R_a v_x \mathbf{x} + R_a v_y \mathbf{y}
  + R_{xy} v_x \mathbf{xyx} + R_{xy} v_y \mathbf{xyy}
\)

\(
  = R_a v_x \mathbf{x} + R_a v_y \mathbf{y}
  - R_{xy} v_x \mathbf{y} + R_{xy} v_y \mathbf{x}
\)

\(
  = (R_a v_x + R_{xy} v_y) \mathbf{x}
  + (R_a v_y - R_{xy} v_x) \mathbf{y}
\)

Rotor \( R' = R_a - R_{xy} \mathbf{xy} \)

Vector \( v' = R v R' = T R' \)

\(
  = ((R_a v_x + R_{xy} v_y) \mathbf{x} + (R_a v_y - R_{xy} v_x) \mathbf{y})
  (R_a - R_{xy} \mathbf{xy})
\)

\(
  = ((R_a v_x + R_{xy} v_y) \mathbf{x}
     + (R_a v_y - R_{xy} v_x) \mathbf{y}) R_a \\
  - ((R_a v_x + R_{xy} v_y) \mathbf{x}
     + (R_a v_y - R_{xy} v_x) \mathbf{y}) R_{xy} \mathbf{xy}
\)

\(
  = ((R_a v_x + R_{xy} v_y) \mathbf{x} R_a
     + (R_a v_y - R_{xy} v_x) \mathbf{y} R_a) \\
  - ((R_a v_x + R_{xy} v_y) \mathbf{x} R_{xy} \mathbf{xy}
     + (R_a v_y - R_{xy} v_x) \mathbf{y} R_{xy} \mathbf{xy})
\)

\(
  = ((R_a R_a v_x + R_a R_{xy} v_y) \mathbf{x}
     + (R_a R_a v_y - R_a R_{xy} v_x) \mathbf{y}) \\
  - ((R_a R_{xy} v_x + R_{xy} R_{xy} v_y) \mathbf{xxy}
     + (R_a R_{xy} v_y - R_{xy} R_{xy} v_x) \mathbf{yxy})
\)

\(
  = ((R_a^2 v_x + R_a R_{xy} v_y) \mathbf{x}
     + (R_a^2 v_y - R_a R_{xy} v_x) \mathbf{y}) \\
  - ((R_a R_{xy} v_x + R_{xy}^2 v_y) \mathbf{y}
     - (R_a R_{xy} v_y - R_{xy}^2 v_x) \mathbf{x})
\)

\(
  = ((R_a^2 v_x + R_a R_{xy} v_y) \mathbf{x}
     + (R_a^2 v_y - R_a R_{xy} v_x) \mathbf{y}) \\
  + ((R_a R_{xy} v_y - R_{xy}^2 v_x) \mathbf{x}
     - (R_a R_{xy} v_x + R_{xy}^2 v_y) \mathbf{y})
\)

\(
  = ((R_a^2 v_x + R_a R_{xy} v_y)
     + (R_a R_{xy} v_y - R_{xy}^2 v_x)) \mathbf{x} \\
  + ((R_a^2 v_y - R_a R_{xy} v_x)
     - (R_a R_{xy} v_x + R_{xy}^2 v_y)) \mathbf{y}
\)

\(
  = (R_a^2 v_x + R_a R_{xy} v_y + R_a R_{xy} v_y - R_{xy}^2 v_x) \mathbf{x} \\
  + (R_a^2 v_y - R_a R_{xy} v_x - R_a R_{xy} v_x - R_{xy}^2 v_y) \mathbf{y}
\)

\(
  = (R_a^2 v_x + 2 R_a R_{xy} v_y - R_{xy}^2 v_x) \mathbf{x}
  + (R_a^2 v_y - 2 R_a R_{xy} v_x - R_{xy}^2 v_y) \mathbf{y}
\)

\(
  = ((R_a^2 - R_{xy}^2) v_x + 2 R_a R_{xy} v_y) \mathbf{x}
  + ((R_a^2 - R_{xy}^2) v_y - 2 R_a R_{xy} v_x) \mathbf{y}
\)

\(
  R = \mathbf{xy}
  = (\mathbf{x} \cdot \mathbf{y}) + (\mathbf{x} \wedge \mathbf{y})
  = 0 + 1 \mathbf{xy}
\)

\(
  = ((0^2 - 1^2) v_x + 0 v_y) \mathbf{x}
  + ((0^2 - 1^2) v_y - 0 v_x) \mathbf{y}
\)

\(
  = - v_x \mathbf{x} + - v_y \mathbf{y}
\)

\end{document}


\section{Reinforcement Learning for Automated Trading}
\label{sec:application_to_systematic_trading}

Many financial applications can be seen as sequential decision problems which naturally fall in the stochastic optimal control framework introduced above. In this section we discuss how reinforcement learning algorithms can be applied to the asset allocation problem, where an agent invests his capital on various assets available in the market.

\subsection{Asset Allocation With Transaction Costs}
The asset allocation problem consists of determining how to dynamically invest the available capital in a portfolio of different assets in order to maximize the expected total return or another relevant performance measure. Let us consider a financial market consisting of $I+1$ different stocks that are traded only at discrete times $t \in \{0, 1, 2, \ldots\}$ and denote by ${Z}_t = {(Z_t^0, Z_t^1, \ldots, Z_t^I)}^T$ their prices at time $t$. Typically, $Z_t^0$ refers to a riskless asset whose dynamic is given by $Z_t^0 = {(1 + X)}^t$ where $X$ is the deterministic risk-free interest rate. The investment process works as follows: at time $t$, the investor observes the
state of the market $S_t$, consisting for example of the past asset prices and other relevant economic variables, and subsequently chooses how to rebalance his portfolio, by specifying the units of each stock ${n}_t = {(n_t^0 , n_t^1 , \ldots , n_t^I)}^T$ to be held between $t$ and $t+1$. In doing so, he needs to take into account the transaction costs that he has to pay to the broker to change his position.  At time $t+1$, the investor realizes a profit or a loss from his investment due to the stochastic variation of the stock values. The investor’s goal is to maximize a given performance measure. Let $W_t$ denote the wealth of the investor at time $t$. The profit realized between $t$ and $t+1$ is simply given by the difference between the trading results and the transaction costs payed to the broker. More formally
\begin{equation*}
	\Delta W_{t+1} = W_{t+1} - W_t = \text{PNL}_{t+1} - \text{TC}_{t}
\end{equation*}
where $\text{PNL}_{t+1}$ denotes the profit due to the variation of the
portfolio asset prices between $t$ and $t+1$
\begin{equation*}
	\text{PNL}_{t+1} = {n}_t \cdot \Delta{Z}_{t+1} = \sum^{I}_{i=0}
	n_t^i (Z_{t+1}^i - Z_t^i)
\end{equation*}
and $\text{TC}_t$ denotes the fees payed to the broker to change the portfolio
allocation and on the short positions
\begin{equation*}
	\text{TC}_t = \sum^{I}_{i=0} \delta_p^i \left| n_t^i - n_{t-1}^i\right| Z_t^i
				- \delta_f W_t \ind{{n}_t \neq {n}_{t-1}}
				- \sum^{I}_{i=0} \delta_s^i {(n_t^i)}^{-} Z_t^i
\end{equation*}
The transaction costs consist of three different components. The first term
represent a transaction cost that is proportional to the change in value of the
position in each asset. The second term is a fixed fraction of the total value
of the portfolio which is payed only if the allocation is changed. The last
term represents the fees payed to the broker for the shares borrowed to build a
short position. The portfolio return between $t$ and $t+1$ is thus given by
\begin{equation}\label{eq:portfolio_return}
	X_{t+1} = \frac{\Delta W_{t+1}}{W_t} = \sum^{I}_{i=0} \left[ a_t^i
	X_{t+1}^i - \delta_i \left| a_t^i - \tilde{a}_t^i \right| - \delta_s
	{(a_t^i)}^- \right] - \delta_f \ind{{a}_t \neq \tilde{{a}}_{t-1}}
\end{equation}
where
\begin{equation*}
	X_{t+1}^i = \frac{\Delta Z_{t+1}^i}{Z_t^i}
\end{equation*}
is the return of the $i$-th stock between $t$ and $t+1$,
\begin{equation*}
	a_t^i = \frac{n_t^i Z_t^i}{W_t}
\end{equation*}
is the fraction of wealth invested in the $i$-th stock between time $t$ and
$t+1$ and finally
\begin{equation*}
	\tilde{a}_t^i = \frac{n_{t-1}^i Z_t^i}{W_t} = \frac{a_{t-1}^i (1+X_t^i)}
	{1 + X_t}
\end{equation*}
is the fraction of wealth invested in the $i$-th stock just before the
reallocation. We assume that the agent invests all his wealth at each step, so
that $W_t$ can be also interpreted as the value of his portfolio. This
assumption leads to the following constraint on the portfolio weights
\begin{equation}
	\sum^{I}_{i=0} a_t^i = 1 \;\;\;\;\; \forall t \in \{0, 1, 2, \ldots\}
\end{equation}
We notice that we are neglecting the typical margin requirements on the short
positions, which would reduce the available capital at time $t$. Considering
margin requirements would lead to a more complex constraint on the portfolio
weights which would be difficult to treat in the reinforcement learning
framework. Plugging this constraint into Eq. (\ref{eq:portfolio_return}), we
obtain
\begin{equation}\label{eq:portfolio_return_benchmark}
	X_{t+1} = X + \sum^{I}_{i=1} a_t^i (X_{t+1}^i - X) - \sum^{I}_{i=0}
	\left[\delta_i \left| a_t^i - \tilde{a}_t^i \right| - \delta_s^i
	{(a_t^i)}^-\right] - \delta_f \ind{{a}_t \neq \tilde{{a}}_{t-1}}
\end{equation}
which highlights the role of the risk-free asset as a benchmark for the
portfolio returns. The total profit realized by the investor between $t=0$ and
$T$ is
\begin{equation*}
	\Pi_T = W_T - W_0 = \sum^{T}_{t=1} \Delta W_t = \sum^{T}_{t=1} W_t X_t
\end{equation*}
The portfolio return between $t=0$ and $T$ is given by
\begin{equation*}
	X_{0,T} = \frac{W_T}{W_0} - 1 = \prod_{t=1}^T (1+X_t) - 1
\end{equation*}
In order to cast the asset allocation problem in the reinforcement learning
framework, we consider the log-return of the portfolio between $t=0$ and $T$
\begin{equation}
	R_{0,T} = \log \frac{W_T}{W_0} = \sum^{T}_{t=1} \log(1+X_t) = \sum_{t=1}^T
	R_t
\end{equation}
where $R_{t+1}$ is the log-return of the portfolio between $t$ and $t+1$
\begin{equation}
	R_{t+1} = \log \left\{ 1 + \sum^{I}_{i=0} \left[ a_t^i X_{t+1}^i - \delta_i
	\left| a_t^i - \tilde{a}_t^i \right| - \delta_s {(a_t^i)}^- \right] -
	\delta_f \ind{{a}_t \neq \tilde{{a}}_{t-1}}\right\}
\end{equation}
The portfolio log-return can be used as the reward function of a
RL algorithm, either in a offline or in an online approach.

\subsection{Reinforcement Learning Application}
In the previous section we derived the reward function for the asset allocation problem with transaction costs. In order to apply the policy gradient algorithms discussed in the previous sections we still need to define the state space, the action space and the agent's policy. For simplicity, we limit ourselves to the case of a single risky asset, i.e. $I = 1$, but the discussion could be generalized to the multi-asset case.\\
We assume that at each time step the agent considers the $P+1$ past returns of the risky asset, i.e. $\{X_t, X_{t-1}, \ldots, X_{t-P}\}$. In order to properly incorporate the effects of transaction costs into his decision process, the agent must keep track of its current position $\tilde{a}_t$. The state of the system is thus given by $S_t = \{X_t, X_{t-1}, \ldots, X_{t-P}, \tilde{a}_t\}$ We might also include some external variables $Y_t$ that may be relevant to the trader, such as the common technical indicator used in practice. Furthermore, these input variables may be used to construct more complex features for example using some deep learning techniques, such as a deep auto-encoder.\\
The agent, or trading system, specifies the portfolio weights $a_t = (a_t^0, a_t^1)^T$ according to a long-short strategy, i.e. the agent may be long ($a_t^1 = +1$) or short ($a_t^1 = -1$) on the risky-asset while $a_t^0 = 1 - a_t^1$ since the agents invests all the available capital at each time step. In the GPOMDP framework we assume that the agent selects $a_t^1$ according to a Boltzmann policy, i.e.
\begin{equation}
	\pi_\theta(s, +1) = \frac{e^{\theta^T s}}{1 + e^{\theta^T s}} \;\;\;\;\; \pi_\theta(s, -1) = \frac{1}{1 + e^{\theta^T s}}
\end{equation}
where we included a bias term in the parameters and in the state. In the parameter-based formulation, we assume that agent selects actions according to the binary controller
\begin{equation}
	F_\theta(s) = \sign(\theta^T s)
\end{equation}
where the controller parameters are normally distributed $\theta \sim \calN(\mu, \diag(\sigma))$. Since the formulation of the asset allocation problem given above is non-episodic, we actually applied the online version of the algorithms discussed above. The main considerations made above still hold and we refer to the full thesis for the details.


\section{The Ratio and Root Tests}\label{sec:ratioroottest}

Does the series $\ds\sum_{n=0}^\infty {n^5\over 5^n}$ converge? It is
possible, but a bit unpleasant, to approach this with the integral
test or the comparison test, but there is an easier way. Consider what
happens as we move from one term to the next in this series:

\[\cdots+{n^5\over5^n}+{(n+1)^5\over 5^{n+1}}+\cdots\]

The denominator goes up by a factor of 5, $\ds 5^{n+1}=5\cdot5^n$, but the
numerator goes up by much less: $\ds (n+1)^5=n^5+5n^4+10n^3+10n^2+5n+1$,
which is much less than $\ds 5n^5$ when $n$ is large, because $\ds 5n^4$ is
much less than $\ds n^5$. So we might guess that in the long run it begins
to look as if each term is $1/5$ of the previous term. We have seen
series that behave like this: The geometric series.

\[\sum_{n=0}^\infty {1\over 5^n} = {5\over4},\]
So we might try comparing the given series to some
variation of this geometric series. This is possible, but a bit
messy. We can in effect do the same thing, but bypass most of the
unpleasant work.

The key is to notice that
\[
  \lim_{n\to\infty} {a_{n+1}\over a_n}=
  \lim_{n\to\infty} {(n+1)^5\over 5^{n+1}}{5^n\over n^5}=
  \lim_{n\to\infty} {(n+1)^5\over n^5}{1\over 5}=1\cdot {1\over5}
    ={1\over 5}.
\]

This is really just what we noticed above, done a bit more formally:
in the long run, each term is one fifth of the previous term. Now pick
some number between $1/5$ and $1$, say $1/2$. Because
$$\lim_{n\to\infty} {a_{n+1}\over a_n}={1\over5},$$
then when $n$ is big enough, say $n\ge N$ for some $N$,

\[
  {a_{n+1}\over a_n}<{1\over2} \quad \hbox{so}\quad a_{n+1}<{a_n\over2}.
\]

So $\ds a_{N+1}< a_N/2$, $\ds a_{N+2}<a_{N+1}/2<a_N/4$,
$\ds a_{N+3}<a_{N+2}/2<a_N/8$, and so on. The general form is
$\ds a_{N+k}< a_N/2^k$. So if we look at the series

\[
  \sum_{k=0}^\infty a_{N+k}=
  a_N+a_{N+1}+a_{N+2}+a_{N+3}+\cdots+a_{N+k}+\cdots,
\]
its terms are less than or equal to the terms of the sequence

\[
  a_N+{a_N\over2}+{a_N\over4}+{a_N\over8}+\cdots+{a_N\over2^k}+\cdots=
  \sum_{k=0}^\infty {a_N\over 2^k} = 2a_N.
\]

So by the comparison test, $\ds\sum_{k=0}^\infty a_{N+k}$ converges,
and this means that $\ds\sum_{n=0}^\infty a_{n}$ converges, since
we've just added the fixed number $\ds a_0+a_1+\cdots+a_{N-1}$.

Under what circumstances could we do this? What was crucial was that
the limit of $\ds a_{n+1}/a_n$, say $L$, was less than 1 so that we could pick a
value $r$ so that $L<r<1$. The fact that $L<r$ ($1/5<1/2$ in our
example) means that we can compare the series $\sum a_n$ to $\sum
r^n$, and the fact that $r<1$ guarantees that $\sum r^n$
converges. That's really all that is required to make the argument
work. We also made use of the fact that the terms of the series were
positive; in general we simply consider the absolute values of the
terms and we end up testing for absolute convergence.

\begin{theorem}{The Ratio Test}{RatioTest}
Suppose that $\ds\lim_{n\to \infty} |a_{n+1}/a_n|=L$. If $L<1$
the series $\sum a_n$ converges absolutely,
if $L>1$ the series diverges, and if
$L=1$ this test gives no information.
\end{theorem}
\begin{proof}
The example above essentially proves the first part of this, if we
simply replace $1/5$ by $L$ and $1/2$ by $r$.
Suppose that $L>1$, and pick $r$ so that $1<r<L$.
Then for $n\ge N$, for some $N$,
\[{|a_{n+1}|\over |a_n|} > r \quad \hbox{and}\quad |a_{n+1}| > r|a_n|.\]

This implies that $\ds |a_{N+k}|>r^k|a_N|$, but since $r>1$ this means
that $\ds\lim_{k\to\infty}|a_{N+k}|\not=0$, which means also that
$\ds\lim_{n\to\infty}a_n\not=0$. By the divergence test, the series
diverges.

To see that we get no information when $L=1$, we need to exhibit two
series with $L=1$, one that converges and one that diverges. The series
$\sum 1/n^2$ and $\sum 1/n$ provide a simple example.
\end{proof}

The ratio test is particularly useful for series involving
the factorial function.

\begin{example}{}{}
Analyze $\ds\sum_{n=0}^\infty  \frac{5^n}{n!}$.
\end{example}
\begin{solution}
$$
  \lim_{n\to\infty} {5^{n+1}\over (n+1)!}{n!\over 5^n}=
  \lim_{n\to\infty} {5^{n+1}\over 5^n}{n!\over (n+1)!}=
  \lim_{n\to\infty} {5}{1\over (n+1)}=0.
$$
Since $0<1$, the series converges.
\end{solution}

A similar argument justifies a similar test
that is occasionally easier to apply.

\begin{theorem}{The Root Test}{RootTest}
Suppose that $\ds\lim_{n\to \infty} |a_n|^{1/n}=L$. If $L<1$
the series $\sum a_n$ converges absolutely,
if $L>1$ the series diverges, and if
$L=1$ this test gives no information.
\end{theorem}

The proof of the root test is actually easier than that of the ratio
test, and is left as an exercise.

\begin{example}{}{}
Analyze $\ds\sum_{n=0}^\infty {5^n\over n^n}$.
\end{example}
\begin{solution}
The ratio test turns out to be a bit difficult on this series (try
it). Using the root test:
$$
  \lim_{n\to\infty} \left({5^n\over n^n}\right)^{1/n}=
  \lim_{n\to\infty} {(5^n)^{1/n}\over (n^n)^{1/n}}=
  \lim_{n\to\infty} {5\over n}=0.
$$
Since $0<1$, the series converges.
\end{solution}

The root test is frequently useful when $n$ appears as an exponent in
the general term of the series.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Opensolutionfile{solutions}[ex]
\section*{Exercises for \ref{sec:ratioroottest}}

\begin{enumialphparenastyle}

\begin{ex}
Compute $\ds\lim_{n\to\infty} |a_{n+1}/a_n|$ for the series
$\sum 1/n^2$.
\end{ex}

\begin{ex}
Compute $\ds\lim_{n\to\infty} |a_{n+1}/a_n|$ for the series
$\sum 1/n$.
\end{ex}

\begin{ex}
Compute $\ds\lim_{n\to\infty} |a_n|^{1/n}$ for the series
$\sum 1/n^2$.
\end{ex}

\begin{ex}
Compute $\ds\lim_{n\to\infty} |a_n|^{1/n}$ for the series
$\sum 1/n$.
\end{ex}

\begin{ex}
Determine whether the series converge.

\begin{multicols}{2}
\begin{enumerate}
	\item $\ds\sum_{n=0}^\infty (-1)^{n}{3^n\over 5^n}$
	\item $\ds\sum_{n=1}^\infty {n!\over n^n}$
	\item $\ds\sum_{n=1}^\infty {n^5\over n^n}$
	\item $\ds\sum_{n=1}^\infty {(n!)^2\over n^n}$
\end{enumerate}
\end{multicols}
\begin{sol}
\begin{enumerate}
	\item converges
	\item converges
	\item converges
	\item diverges
\end{enumerate}
\end{sol}
\end{ex}

\begin{ex}
Prove Theorem \ref{thm:RootTest}, the root test.
\end{ex}

\end{enumialphparenastyle}


\chapter{Relationships are pointfree funcoids}

\begin{thm}
  $(\tofcd, \torldin )$ are components
  of a complete pointfree funcoid.
\end{thm}

\begin{proof}
  For every ultrafilters $x$ and $y$ we have $x
  \mathrel{[\tofcd (f \sqcap \torldin
  g)]} y \Leftrightarrow x \times^{\mathsf{RLD}} y \nasymp f \sqcap
  \torldin  g \Leftrightarrow x
  \times^{\mathsf{RLD}} y \sqsubseteq
  \torldin  g \land x \times^{\mathsf{RLD}} y
  \nasymp f \sqcap \torldin g \Leftrightarrow x \times^{\mathsf{FCD}} y \in
  \atoms g : x \times^{\mathsf{RLD}} y \nasymp f \sqcap \torldin g
  \Leftrightarrow
  x \times^{\mathsf{FCD}} y \in
  \atoms g : x \times^{\mathsf{RLD}} y \nasymp f
  \Leftrightarrow
  x \times^{\mathsf{FCD}} y \in \atoms g \land x
  \times^{\mathsf{FCD}} y \sqsubseteq \tofcd f
  \Leftrightarrow x \mathrel{[g \sqcap \tofcd f]} y$.

  Thus $\tofcd (f \sqcap \torldin  g) =
  g \sqcap \tofcd f$. Consequently $f \sqcap
  \torldin  g = \bot \Leftrightarrow g \sqcap
  \tofcd f = \bot$ that is $g \nasymp \tofcd f
  \Leftrightarrow f \nasymp \torldin  g$.

  It is complete by theorem~\bookref{rels-dist}.
\end{proof}

We will also prove in another way that $\tofcd$,
$\torldin $ are components of pointfree funcoids:

\begin{thm}
  $\torldin $ is a component of a pointfree funcoid
  (between filters on boolean lattices).
\end{thm}

\begin{proof}
  Consider the pointfree funcoid $\mathscr{R}$ defined by the formula
  $\left\langle \mathscr{R} \right\rangle^{\ast} F =
  \torldin  F$ for binary relations $F$ (obviously it
  does exists). Then $\left\langle \mathscr{R} \right\rangle f = \left\langle
  \mathscr{R} \right\rangle \bigsqcap^{\mathsf{FCD}} \up^{\Gamma}
  f = \bigsqcap^{\mathsf{RLD}}_{F \in \up^{\Gamma} f}
  \left\langle \mathscr{R} \right\rangle^{\ast} F =
  \bigsqcap^{\mathsf{RLD}}_{F \in \up^{\Gamma} f}
  \torldin  F = \torldin
  \bigsqcap^{\mathsf{FCD}}_{F \in \up^{\Gamma} f} F =
  \torldin  f$.
\end{proof}

\begin{thm}
  $\tofcd$ is a component of a complete pointfree funcoid
  (between filters on boolean lattices).
\end{thm}

\begin{proof}
  Consider the pointfree funcoid $\mathscr{Q}$ defined by the formula
  $\left\langle \mathscr{Q} \right\rangle^{\ast} F = \tofcd F$
  for binary relations $F$ (obviously it does exists). Then $\left\langle
  \mathscr{Q} \right\rangle f = \left\langle \mathscr{Q} \right\rangle
  \bigsqcap^{\mathsf{RLD}} \up f = \text{(because $\up f$
  is a filter base)} = \bigsqcap^{\mathsf{FCD}}_{F \in \up f}
  \left\langle \mathscr{Q} \right\rangle^{\ast} F =
  \bigsqcap^{\mathsf{FCD}}_{F \in \up f} \tofcd F
  = \bigsqcap^{\mathsf{FCD}}_{F \in \up f} F =
  \bigsqcap^{\mathsf{FCD}} \up f = \tofcd f$.
\end{proof}

\begin{prop}
$\tofcd \bigsqcap S = \bigsqcap_{f\in S}\tofcd f$ if~$S$ is
a filter base of reloids (with the same sources and destinations).
\end{prop}

\begin{proof}
Theorem~\bookref{supfun-genbase}.
\end{proof}

\begin{conjecture}
$\torldin \bigsqcap S = \bigsqcap_{f\in S}\torldin f$ if~$S$ is
a filter base of funcoids (with the same sources and destinations).
\end{conjecture}


\section{Methodology}\label{sec:methodology:clustering}

In order to understand the make up of \bslongs{} in \cplop{}, we chose to investigate how a density-based clustering algorithm might group the \isols{} we have collected so far and how that might affect a rudimentary \mst{} technique based off of clustering: cluster an unknown \isol{} along with the \isols{} in \cplop{} and classify it as the most plural species of the cluster.
Density-based clustering ties in well with our notion of closely-related strains of \ecoli{} (the relation being the separate \pearson{} comparison of each \itsshort{} region we use to compare \isols{}).
\index{\ecoli{} strain}
From the computer science point of view, a bacterial strain is essentially a cluster of \ecoli{} \isol{} representations stored in \cplop{}.
Our \mst{} method, thus, works as follows:
\begin{enumerate}
    \item \textbf{Strain Identification.} Identify bacterial strains in \cplop{} by clustering
    all \cplop{} \isols{}.
    \item \textbf{MST.} Given an isolate of unknown origin, find the cluster it belongs to.
    Return the \spec{} of the plurality of isolates in the cluster.
\end{enumerate}

Our clustering algorithm is the density-based clustering algorithm developed by Johnson \cite{johnson2015density}.
It extends \dbscan{} for the case of two \compfuncs{} between data points (our isolates are compared based on the two \itsshort{} regions) and implements an efficient spatial data structure to manage the retrieval of the data points.

\dbscan{} can easily use an efficient range query technique to find nearby points and speed up clustering time considerably by taking advantage of the \trieq{} that proper metric spaces have.
Unfortunately, \pearson{} does not encode a metric space, because it fails the \trieq{}\footnote{$d(x,z)\leq d(x,y) + d(y,z)$}.
This complicates range queries, discussed in \autoref{sec:background:dbscan}, because spatial indexes tend to rely on the \trieq{}, usually with \euclid{}, to argue that certain points can be ignored during a spatial index tree traversal.

To allow us the use of a spatial data structure with the \pearson{} to store data points during the clustering procedure we use instead the \clustplop{}, which we derive by recognizing in \autoref{eq:pearson} that \pearson{} is made up of \zscores{} of \pcveca{} and \pcvecb{}.
The \zscore{} of \pcveca{} is:
\[
\zscoreeq{\pca}
\]
where \vecavg{\pca} and \vecstddev{\pca} are the mean and standard deviation of the values in a single pyroprint respectively.
Thus, for clustering, we compare \pyros{} using the Euclidean distance \euczfunclabel{} of $z$-scores.
\begin{equation}\label{eq:euclidean_zscores}
\eucz{\pca}{\pcb}
\end{equation}
where \numdims{} is the number of dimensions.
This allows us to use spatial indexes and \bigo{\log{n}} lookup in \dbscan{}.

Each \isol{} is represented in \cplop{} by a pair of \pyros{}: one each from each \Ssixt{} and \Sfive{} region, complicating the use of \dbscan{} and the meaning of $\alpha$ threshold.
We handle this in \dbscan{} by performing two range queries, one each for \Ssixt{} and \Sfive{}, taking the intersection of the two results.
We must, however, pick a suitable \eps{} for each \itsshort{} region.

\cplop{} uses a threshold value of $\alpha = 0.995$  to compare two \pyros{}.
\Pyros{} with \pearson{} above $\alpha$ are considered to represent the same \dna{} material, while \pyros{} with a \pearson{} below $\alpha$ are considered to represent different \dna{} material \cite{Shealy:SeniorProject, soliman2013cplop, SolimanDVMBNWKG12}.

The number of dispensations \numdims{} used to build a \pyro{} differ for the \Ssixt{} and \Sfive{} regions.
Because  $\numdims{}_{\Ssixt{}} \neq \numdims{}_{\Sfive{}}$, the original $\alpha$ under the space defined by \eqref{eq:euclidean_zscores} no longer applies in the same way to both regions.
An alternative formulation of \eqref{eq:euclidean_zscores}, with respect to the \pearson{} \pcfunclabel{}, is:
\begin{equation}\label{eq:euclidean_zscores_alternate}
\euczalternate{\pca}{\pcb}
\end{equation}
where \numdims{} is the number of dimensions and $\numdims{}_{\pcveca{}} = \numdims{}_{\pcvecb{}} = \numdims{}$.
Using \autoref{eq:euclidean_zscores_alternate}, we can convert $\alpha$ to the values in \autoref{tab:converted_thresholds}.
We use these converted $\alpha$ values as the \eps{} for each \itsshort{} region's \codefn{RangeQuery}.
\begin{table}
\centering
\caption{Converted $\alpha$ threshold to fit the new metric space defined by \eqref{eq:euclidean_zscores}.}
\label{tab:converted_thresholds}
\begin{tabular}{|c|c|c|}
\hline
\textbf{\itsshort{} Region} & \Ssixt{}     & \Sfive{}     \\
                            & $\alpha$     & $\alpha$     \\ \hline
\pcfunc{\pca}{\pcb}         & 0.995        & 0.995        \\ \hline
\numdims{}                  & \Ssixtdims{} & \Sfivedims{} \\ \hline
\euczfunc{\pca{}}{\pcb{}}   & 0.9747       & 0.9644       \\ \hline
\end{tabular}
\end{table}

When clustering \cplop{} isolates using our density-based clustering algorithm, we need to set up the two parameters at our disposal: \minneigh{} and \eps{}.
For \eps{} we choose the two values shown in Table \ref{tab:converted_thresholds} converted from the 0.995 \pearson{} threshold of pyroprint similarity.
Essentially, we only want to consider the \eps{}-neighborhood of a \pyro{} that contains the other \pyros{} that we consider to represent the same DNA material.

For the \minneigh{} parameter, we use \textit{grid search} running our clustering with \minneigh{} set to $1, 2, 3, 4, 5, 6$, and $7$.
The \minneigh{} value adjusts how strict our definition of a cluster is.
That is, the higher the value of \minneigh{}, the more neighbors a core point must have with \eps{} of it and its neighbors to become a cluster.
Balancing this value with the coverage of our algorithm is crucial to its success, because for too low of a value, we may not have a clear plurality in a cluster, while too high of a value may miss some smaller clusters that might classify our unknown \isol{} into something other than noise.