\documentclass[a4paper]{article}
\usepackage[a4paper,margin=25mm]{geometry}
\usepackage{graphicx,subcaption}
\usepackage{amsmath,amsfonts}
\title{Notes on Louvain Modularity}
\author{G.A. Jarrad}
\begin{document}
\maketitle
\numberwithin{equation}{section}
\numberwithin{figure}{section}
\numberwithin{table}{section}
\section{Undirected Modularity}\label{sec:Q}
This section is motivated by the application of the Louvain modularity clustering algorithm to an undirected graph,
as (incompletely) described by \cite{blondel08}.

Let $D_{ij}\ge 0$ represent the weight of a directed edge
$i\rightarrow j$ (if one exists) from vertex $i\in{\cal V}$ to vertex $j\in{\cal V}$.
Then the equivalent undirected edge has weight
$A_{ij}=D_{ij}+D_{ji}-\delta_{ij}D_{ii}$, such that $A_{ij}=A_{ji}$. The total
weight of all edges for vertex i (its so-called {\em vertex weight}) is then given by
\begin{eqnarray}
  A_{i\cdot} & = & \sum_{j\in{\cal V}} A_{ij}~=~\sum_{j\in{\cal V}} A_{ji}~=~A_{\cdot i}\,.
\label{eq:in:out}
\end{eqnarray}
The sum of all vertex weights is then given by
\begin{eqnarray}
  A_{\cdot\cdot} & = & \sum_{i\in{\cal V}} A_{i\cdot}
  ~=~\sum_{i\in{\cal V}}\sum_{j\in{\cal V}} A_{ij}\,.
\end{eqnarray}
Note that this counts self edges (i.e.\ $i$ -- $i$) once and all other edges (i.e.\ $i$ -- $j$,
$i\ne j$) twice. The Louvain modularity algorithm \cite{blondel08} in fact assumes that there are no self edges, but we
shall include them here for completeness.

The modularity score of a clustered, undirected graph is then given by
\begin{eqnarray}
  Q & = & \sum_{i\in{\cal V}}\sum_{j\in{\cal V}} \left[
  \frac{A_{ij}}{A_{\cdot\cdot}}-
  \frac{A_{i\cdot}A_{\cdot j}}{A_{\cdot\cdot}^2}
  \right]\,\delta(c_i,c_j)\,,
\end{eqnarray}
where $c_i$ is the index of the cluster containing vertex $i$. Note that
$\delta(c_i,c_j)=1$ if and only if $c_i=c_j=g$ for some cluster index $g$. Then $Q$ can be
partitioned by cluster as
\begin{eqnarray}
  Q & = & \sum_{g=1}^G\sum_{i\in{\cal V}_g}\sum_{j\in{\cal V}_g} \left[
  \frac{A_{ij}}{A_{\cdot\cdot}}-
  \frac{A_{i\cdot}A_{\cdot j}}{A_{\cdot\cdot}^2}
  \right]~\doteq~\sum_{g=1}^G Q_g\,,
\label{eq:Qg}
\end{eqnarray}
where the $g$-th cluster contains vertices ${\cal V}_g=\{i\!\in\!{\cal V}\;|\;c_i\!=\!g\}$,
and therefore ${\cal V}=\bigcup_{g=1}^G {\cal V}_g$.

We now observe that the sum of edge weights of vertex $i$ for all edges to and from cluster $g$
is given by
\begin{eqnarray}
  A_{i,g} & = & \sum_{j\in{\cal V}_g} A_{ij}\,,
\end{eqnarray}
and hence the {\em internal} cluster weight, namely the total weight of all edges internal to
cluster $g$, is given by
\begin{eqnarray}
   \Sigma_g^{\tt int} & = & \sum_{i\in{\cal V}_g}\sum_{j\in{\cal V}_g} A_{ij}
   ~=~\sum_{i\in{\cal V}_g} A_{i,g}\,.
\label{eq:sig:int}
\end{eqnarray}
Note that this value, like $A_{\cdot\cdot}$, also counts self edges ($i$ -- $i$) once and all other edges
($i$ -- $j$) twice.
Conversely, the {\em external} cluster weight, namely the total weight of all edges from vertices
in cluster $g$ to and from vertices in other clusters, is given by
\begin{eqnarray}
   \Sigma_g^{\tt ext} & = & \sum_{i\in{\cal V}_g}\sum_{j\in\bar{\cal V}_g}
   A_{ij} ~=~\sum_{j\in\bar{\cal V}_g} A_{j,g}\,,
\label{eq:sig:ext}
\end{eqnarray}
where $\bar{\cal V}_g={\cal V}\backslash{\cal V}_g$ (or ${\cal V}-{\cal V}_g$).
Note that these external edge weights are only counted once per cluster, but the
edge $i$ -- $j$ is counted separately for both the cluster containing vertex $i$
and the other cluster containing vertex $j$.

The total weight of cluster $g$ is then given by
\begin{eqnarray}
   \Sigma_g^{\tt tot} & = & \Sigma_g^{\tt int} + \Sigma_g^{\tt ext}
\nonumber\\&=&
   \sum_{i\in{\cal V}_g}\sum_{j\in{\cal V}_g} A_{ij}
   + \sum_{i\in{\cal V}_g}\sum_{j\in\bar{\cal V}_g} A_{ij}
\nonumber\\&=&
   \sum_{i\in{\cal V}_g}\sum_{j\in{\cal V}} A_{ij}~=~\sum_{i\in{\cal V}_g}A_{i\cdot}\,.
\label{eq:sig:tot}
\end{eqnarray}
We now observe that
\begin{eqnarray}
    \sum_{i\in{\cal V}_g}\sum_{j\in{\cal V}_g} A_{i\cdot}A_{\cdot j}
    & = & \sum_{i\in{\cal V}_g}A_{i\cdot}\sum_{j\in{\cal V}_g} A_{\cdot j}
\nonumber\\&=&
    \left(\sum_{i\in{\cal V}_g}A_{i\cdot}\right)\left(\sum_{j\in{\cal V}_g} A_{j\cdot}\right)
\nonumber\\&=&
    \left(\sum_{i\in{\cal V}_g}A_{i\cdot}\right)^2 ~=~ \left(\Sigma_g^{\tt tot}\right)^2\,.
\label{eq:sum:sq}
\end{eqnarray}
Hence, from equation~\eqref{eq:Qg}, we see that the modularity score of the $g$-th cluster
simplifies to become
\begin{eqnarray}
   Q_g & = & \sum_{i\in{\cal V}_g}\sum_{j\in{\cal V}_g} \left[
  \frac{A_{ij}}{A_{\cdot\cdot}}-
  \frac{A_{i\cdot}A_{\cdot j}}{A_{\cdot\cdot}^2}
  \right]
~=~    \frac{\Sigma_g^{\tt int}}{A_{\cdot\cdot}}
    -\left(\frac{\Sigma_g^{\tt tot}}{A_{\cdot\cdot}}\right)^2\,,
\label{eq:Qg:only}
\end{eqnarray}
from equations~\eqref{eq:sig:int} and~\eqref{eq:sum:sq}.

This cluster modularity score $Q_g$ now gives us a handle on how to compute changes in score
due to changes in the graph clustering, with the aim of choosing a clustering that maximises
the total modularity score $Q$.
Suppose we merge a singleton cluster containing only vertex $k$ with another cluster $g$ to form a new
cluster $g\oplus k$ (technically, the new cluster is ${\cal V}_{g\oplus k}={\cal V}_g\bigcup\{k\}$).
Then, from equation~\eqref{eq:sig:int}, the new internal cluster weight is
given by
\begin{eqnarray}
   \Sigma_{g\oplus k}^{\tt int} & = & \sum_{i\in{\cal V}_g\bigcup\{k\}}
   \sum_{j\in{\cal V}_g\bigcup\{k\}} A_{ij}
\nonumber\\&=&
    \sum_{i\in{\cal V}_g}\sum_{j\in{\cal V}_g} A_{ij}
    +\sum_{i\in{\cal V}_g}A_{ik}+\sum_{j\in{\cal V}_g}A_{kj}+A_{kk}
\nonumber\\&=&
    \Sigma_g^{\tt int}+2A_{k,g}+A_{kk}\,.
\label{eq:sig:int:add}
\end{eqnarray}
Similarly, the new total cluster weight is given by
\begin{eqnarray}
    \Sigma_{g\oplus k}^{\tt tot} & = & \sum_{i\in{\cal V}_g\bigcup\{k\}} A_{i\cdot}
~=~
    \sum_{i\in{\cal V}_g} A_{i\cdot}+A_{k\cdot}
    ~=~\Sigma_{g}^{\tt tot}+A_{k\cdot}\,,
\label{eq:sig:tot:add}
\end{eqnarray}
from equation~\eqref{eq:sig:tot}.
Consequently, the modularity score of the new cluster is given by
\begin{eqnarray}
Q_{g\oplus k} & = &
\frac{\Sigma_{g\oplus k}^{\tt int}}{A_{\cdot\cdot}}
    -\left(\frac{\Sigma_{g\oplus k}^{\tt tot}}{A_{\cdot\cdot}}\right)^2
\nonumber\\& = &
\frac{\Sigma_{g}^{\tt int}+2A_{k,g}+A_{kk}}{A_{\cdot\cdot}}
    -\left(\frac{\Sigma_{g}^{\tt tot}+A_{k\cdot}}{A_{\cdot\cdot}}\right)^2\,,
\label{eq:Qgpk}
\end{eqnarray}
from equations~\eqref{eq:Qg:only}--\eqref{eq:sig:tot:add}.
By extension, a singleton cluster containing only vertex $k$ is notionally
formed by merging the vertex with an empty cluster having zero cluster weights, and so the
modularity score of the singelton cluster is just
\begin{eqnarray}
    Q_k & = & \frac{A_{kk}}{A_{\cdot\cdot}}
    -\left(\frac{A_{k\cdot}}{A_{\cdot\cdot}}\right)^2\,.
\label{eq:Qk}
\end{eqnarray}
Note that the Louvain modularity algorithm \cite{blondel08} starts by placing every vertex $k\in{\cal V}$ into
its own singleton cluster, and so initially there are $G=|{\cal V}|$ such clusters.
Also note that, as mentioned above, the Louvain algorithm \cite{blondel08} implicitly assumes that $A_{kk}=0$.

We can now compute the total change in modularity caused by adding singleton vertex $k$
to cluster $g$, namely
\begin{eqnarray}
    \Delta Q_{(g,k)\rightarrow g\oplus k} & = & Q_{g\oplus k}-Q_g-Q_k
\nonumber\\&=&
    \left[\frac{\Sigma_{g}^{\tt int}+2A_{k,g}+A_{kk}}{A_{\cdot\cdot}}
    -\left(\frac{\Sigma_{g}^{\tt tot}+A_{k\cdot}}{A_{\cdot\cdot}}\right)^2\right]
-\left[\frac{\Sigma_{g}^{\tt int}}{A_{\cdot\cdot}}
    -\left(\frac{\Sigma_{g}^{\tt tot}}{A_{\cdot\cdot}}\right)^2\right]
    -\left[\frac{A_{kk}}{A_{\cdot\cdot}}
    -\left(\frac{A_{k\cdot}}{A_{\cdot\cdot}}\right)^2\right]
\nonumber\\&=&
\frac{\Sigma_{g}^{\tt int}+2A_{k,g}+A_{kk}}{A_{\cdot\cdot}}
-\frac{(\Sigma_{g}^{\tt tot})^2+2\Sigma_{g}^{\tt tot}
A_{k\cdot}+A_{k\cdot}^2}{A_{\cdot\cdot}^2}
-\frac{\Sigma_{g}^{\tt int}+A_{kk}}{A_{\cdot\cdot}}
+\frac{(\Sigma_{g}^{\tt tot})^2+A_{k\cdot}^2}{A_{\cdot\cdot}^2}
\nonumber\\&=&
    \frac{2A_{k,g}}{A_{\cdot\cdot}}-\frac{2\Sigma_g^{\tt tot}A_{k\cdot}}{A_{\cdot\cdot}^2}\,,
\label{eq:delta:Q:plus}
\end{eqnarray}
from equations~\eqref{eq:Qg:only} and \eqref{eq:Qgpk}--\eqref{eq:Qk}.
Conceptually, this is just the score change upon destroying clusters $k$ and $g$ and then creating a new cluster $g\oplus k$.

In the converse situation, we instead want to remove vertex $k$ from the cluster $g\oplus k$
 and restore $k$ to its singleton cluster. Since the
action of adding vertex $k$ to group $g$ (above) is reversible, then removing the vertex must result in a change of
modularity score opposite to the change in score caused by adding the vertex.
This is just the score change involved in destroying cluster $g\oplus k$ and creating clusters $g$ and $k$.
 Hence, we obtain
\begin{eqnarray}
    \Delta Q_{g\oplus k\rightarrow(g,k)} & = & Q_g+Q_k-Q_{g\oplus k}
~=~    -\frac{2A_{k,g}}{A_{\cdot\cdot}}+\frac{2\Sigma_g^{\tt tot}A_{k\cdot}}{A_{\cdot\cdot}^2}\,,
\end{eqnarray}
where now $A_{k,g}$ is computed from $A_{k,g\oplus k}$ via
\begin{eqnarray}
    A_{k,g\oplus k} & = & \sum_{j\in{\cal V}_g\bigcup\{k\}}A_{kj}
    ~=~\sum_{j\in{\cal V}_g}A_{kj}+A_{kk}~=~A_{k,g}+A_{kk}
\nonumber\\
\Rightarrow A_{k,g} & = & A_{k,g\oplus k} - A_{kk}\,,
\end{eqnarray}
and $\Sigma_g^{\tt tot}$ is computed from $\Sigma_{g\oplus k}^{\tt tot}$ as
\begin{eqnarray}
    \Sigma_{g}^{\tt tot} & = & \Sigma_{g\oplus k}^{\tt tot}-A_{k\cdot}\,,
\end{eqnarray}
from equation~\eqref{eq:sig:tot:add}.
In terms of the existing cluster $g\oplus k$, the score change is thus
\begin{eqnarray}
    \Delta Q_{g\oplus k\rightarrow(g,k)} & = &
    -\frac{2(A_{k,g\oplus k}-A_{kk})}{A_{\cdot\cdot}}
    +\frac{2(\Sigma_{g\oplus k}^{\tt tot}-A_{k\cdot})A_{k\cdot}}{A_{\cdot\cdot}^2}\,.
\label{eq:delta:Q:minus}
\end{eqnarray}
Hence, the combined action of removing vertex $k$ from cluster $g\oplus k$ (to form cluster $g$)
and adding it to cluster $g'$ (to form cluster $g'\oplus k$) gives rise to the total change of score
\begin{eqnarray}
    \Delta Q_{(g\oplus k,g')\rightarrow(g,g'\oplus k)} & = & \Delta Q_{g\oplus k\rightarrow(g,k)} +
    \Delta Q_{(g',k)\rightarrow g'\oplus k}\,,
\end{eqnarray}
utilising equations~\eqref{eq:delta:Q:plus} and~\eqref{eq:delta:Q:minus}.

\section{Directed Modularity}
This section is motivated by the application of the Louvain modularity clustering algorithm to a directed graph,
as described by \cite{browet14}.
In distinction from the case of Section~\ref{sec:Q}, we now take $A_{ij}=D_{j}\ge 0$ to be the directed edge weight from vertex $i$ to vertex $j$ (if such an
edge exists), such that $A_{ij}\ne A_{ji}$ in general; this breaks a number of assumptions used in Section\ref{sec:Q}.
For example, equation~\eqref{eq:in:out} now becomes
\begin{eqnarray}
  s^{\tt out}_i~=~A_{i\cdot}~=~\sum_{j\in{\cal V}} A_{ij}\,,&&
  s^{\tt in}_j~=~\sum_{i\in{\cal V}} A_{ij}~=~A_{\cdot j}\,,
\end{eqnarray}
mixing in some of the notation from \cite{browet14}.
Similarly, if we consider the $g$-th cluster, then the external cluster weight from equation~\eqref{eq:sig:ext} is now replaced by
\begin{eqnarray}
   \Sigma_g^{\tt out}~=~\sum_{i\in{\cal V}_g}\sum_{j\in\bar{\cal V}_g} A_{ij}\,, &&
   \Sigma_g^{\tt in}~=~\sum_{i\in\bar{\cal V}_g}\sum_{j\in{\cal V}_g} A_{ij}\,,
\end{eqnarray}
such that
\begin{eqnarray}
   S^{\tt out}_g & = & \sum_{i\in{\cal V}_g} s^{\tt out}_i~=~\sum_{i\in{\cal V}_g}\sum_{j\in{\cal V}}A_{ij}~=~\Sigma_g^{\tt int}+\Sigma_g^{\tt out}\,,
\end{eqnarray}
and
\begin{eqnarray}
   S^{\tt in}_g & = & \sum_{j\in{\cal V}_g} s^{\tt in}_j~=~\sum_{i\in{\cal V}}\sum_{j\in{\cal V}_g}A_{ij}~=~\Sigma_g^{\tt int}+\Sigma_g^{\tt in}\,,
\end{eqnarray}
using equation~\eqref{eq:sig:int}.
It follows that the modularity score $Q_g$ of the $g$-th cluster becomes
\begin{eqnarray}
   Q_g & = & \sum_{i\in{\cal V}_g}\sum_{j\in{\cal V}_g} \left[
  \frac{A_{ij}}{A_{\cdot\cdot}}-
  \frac{A_{i\cdot}A_{\cdot j}}{A_{\cdot\cdot}^2}
  \right]
~=~    \frac{\Sigma_g^{\tt int}}{A_{\cdot\cdot}}
    -\frac{S_g^{\tt out}S_g^{\tt in}}{A_{\cdot\cdot}^2}\,,
\end{eqnarray}
instead of equation~\eqref{eq:Qg:only}.

We now consider the effect of merging the $g$-th cluster with the singleton cluster containing only vertex $k$, resulting
in a new cluster $g\oplus k$.
The internal weight $\Sigma^{\tt int}_{g\oplus k}$ of this new cluster is now
no longer given by equation~\eqref{eq:sig:int:add}, but instead by
\begin{eqnarray}
   \Sigma_{g\oplus k}^{\tt int} & = & \sum_{i\in{\cal V}_g\bigcup\{k\}}
   \sum_{j\in{\cal V}_g\bigcup\{k\}} A_{ij}
\nonumber\\&=&
    \sum_{i\in{\cal V}_g}\sum_{j\in{\cal V}_g} A_{ij}
    +\sum_{i\in{\cal V}_g}A_{ik}+\sum_{j\in{\cal V}_g}A_{kj}+A_{kk}
\nonumber\\&=&
    \Sigma_g^{\tt int}+W(g,k)+W(k,g)+A_{kk}\,.
\end{eqnarray}
More generally \cite{browet14}, $W(g,k)$ is the edge weight from all vertices in the $g$-th cluster to vertex $k$, excluding any self edge
$k\rightarrow k$ if $k$ happens to be a member of the $g$-th cluster.  Thus
\begin{eqnarray}
  W(g,k) & = & \sum_{i\in{\cal V}_g\backslash\{k\}}A_{ik}\,,
\label{eq:W:g:k}
\end{eqnarray}
and conversely
\begin{eqnarray}
  W(k,g) & = & \sum_{j\in{\cal V}_g\backslash\{k\}}A_{kj}\,,
\label{eq:W:k:g}
\end{eqnarray}
which is the edge weight from vertex $k$ to all vertices (except $k$) in the $g$-th cluster.

Similarly, the cluster merge $g\oplus k$ results in new out and in edge weights given by
\begin{eqnarray}
    S_{g\oplus k}^{\tt out} & = & \sum_{i\in{\cal V}_g\bigcup\{k\}} A_{i\cdot}
~=~
    \sum_{i\in{\cal V}_g} A_{i\cdot}+A_{k\cdot}
    ~=~S_{g}^{\tt out}+s^{\tt out}_k\,,
\label{eq:S:g:k:out}
\\
    S_{g\oplus k}^{\tt in} & = & \sum_{j\in{\cal V}_g\bigcup\{k\}} A_{\cdot j}
~=~
    \sum_{j\in{\cal V}_g} A_{\cdot j}+A_{\cdot k}
    ~=~S_{g}^{\tt out}+s^{\tt in}_k\,,
\label{eq:S:g:k:in}
\end{eqnarray}
respectively; these equations supercede equation~\eqref{eq:sig:tot:add}.
Consequently, the modularity score of the new cluster is now given by
\begin{eqnarray}
Q_{g\oplus k} & = &
\frac{\Sigma_{g\oplus k}^{\tt int}}{A_{\cdot\cdot}}
    -\frac{S_{g\oplus k}^{\tt out}S_{g\oplus k}^{\tt in}}{A_{\cdot\cdot}^2}
\nonumber\\& = &
\frac{\Sigma_{g}^{\tt int}+W(g,k)+W(k,g)+A_{kk}}{A_{\cdot\cdot}}
    -\frac{\left(S_{g}^{\tt out}+A_{k\cdot}\right)\left(S_{g}^{\tt in}+A_{\cdot k}\right)}{A_{\cdot\cdot}^2}\,,
\end{eqnarray}
instead of by equation~\eqref{eq:Qgpk}.
Similarly, the modularity score of the singleton cluster $k$ is now just
\begin{eqnarray}
    Q_k & = & \frac{A_{kk}}{A_{\cdot\cdot}}
    -\frac{A_{k\cdot}A_{\cdot k}}{A_{\cdot\cdot}^2}\,,
\end{eqnarray}
replacing equation~\eqref{eq:Qk}.
Finally, the change in score due to merging clusters $g$ and $k$ is
\begin{eqnarray}
    \Delta Q_{(g,k)\rightarrow g\oplus k} & = & Q_{g\oplus k}-Q_g-Q_k
\nonumber\\&=&
    \left[\frac{\Sigma_{g}^{\tt int}+W(g,k)+W(k,g)+A_{kk}}{A_{\cdot\cdot}}
    -\frac{\left(S_{g}^{\tt out}+A_{k\cdot}\right)\left(S_{g}^{\tt in}+A_{\cdot k}\right)}{A_{\cdot\cdot}^2}\right]
\nonumber\\&&
{}-\left[\frac{\Sigma_{g}^{\tt int}}{A_{\cdot\cdot}}
    -\frac{S_{g}^{\tt out}S_{g}^{\tt in}}{A_{\cdot\cdot}^2}\right]
    -\left[\frac{A_{kk}}{A_{\cdot\cdot}}
    -\frac{A_{k\cdot}A_{\cdot k}}{A_{\cdot\cdot}^2}\right]
\nonumber\\&=&
    \frac{W(g,k)+W(k,g)}{A_{\cdot\cdot}}-\frac{S_g^{\tt out}A_{\cdot k}+S_g^{\tt in}A_{k\cdot}}{A_{\cdot\cdot}^2}\,,
\end{eqnarray}
which replaces equation~\eqref{eq:delta:Q:plus}.

We can now compute the change in modularity score involved with removing vertex $k$ from some cluster $g\oplus k$, resulting in cluster $g$.
As in Section~\ref{sec:Q}, this is just the negative of the score change from merging vertex $k$ with cluster $g$, namely
\begin{eqnarray}
    \Delta Q_{g\oplus k\rightarrow (g,k)} & = &
    -\frac{W(g,k)+W(k,g)}{A_{\cdot\cdot}}+\frac{S_{g}^{\tt out}A_{\cdot k}+S_{g}^{\tt in}A_{k\cdot}}{A_{\cdot\cdot}^2}
\nonumber\\&=&
    -\frac{W(g\oplus k,k)+W(k,g\oplus k)}{A_{\cdot\cdot}}
    +\frac{(S_{g\oplus k}^{\tt out}-A_{k\cdot})A_{\cdot k}+(S_{g\oplus k}^{\tt in}-A_{\cdot k})A_{k\cdot}}{A_{\cdot\cdot}^2}
\,,
\end{eqnarray}
from equations~\eqref{eq:W:g:k}--\eqref{eq:W:k:g} and~\eqref{eq:S:g:k:out}--\eqref{eq:S:g:k:in}. This replaces equation~\eqref{eq:delta:Q:minus}.
The total change in modularity score involved in moving vertex $k$ from cluster $g\oplus k$ to some other cluster $g'$ is therefore
\begin{eqnarray}
    \Delta Q_{(g\oplus k,g')\rightarrow(g,g'\oplus k)} & = & \Delta Q_{g\oplus k\rightarrow(g,k)} +
    \Delta Q_{(g',k)\rightarrow g'\oplus k}
\nonumber\\
& = &
\frac{W(g',k)+W(k,g')-W(g\oplus k,k)-W(k,g\oplus k)}{A_{\cdot\cdot}}
\nonumber\\&&
{}-\frac{(S_{g'}^{\tt out}-S_{g\oplus k}^{\tt out}+A_{k\cdot})A_{\cdot k}+(S_{g'}^{\tt in}-S_{g\oplus k}^{\tt in}+A_{\cdot k})A_{k\cdot}}
{A_{\cdot\cdot}^2}
\,.
\end{eqnarray}
Multiplying $\Delta Q$ by $A_{\cdot\cdot}$ gives equation~(3.26) of \cite{browet14}
with $c_1=g\oplus k$, $c_2=g'$, $s_k^{\tt out}=A_{k\cdot}$ and $s_k^{\tt in}=A_{\cdot k}$,
except that their use of $m$
(the total number of edges) should be
replaced with $m_w=A_{\cdot\cdot}$ (the total weight of all edges).

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{thebibliography}{9}

\bibitem{blondel08}
  Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte, Etienne Lefebvre,
  \textit{Fast unfolding of communities in large networks},
  Journal of Statistical Mechanics: Theory and Experiment 2008 (10), P10008; ArXiv: https://arxiv.org/abs/0803.0476.

\bibitem{browet14}
  Arnaud Browet,
  \textit{Algorithms for community and role detection in networks},
  Doctoral thesis, Louvain-la-Neuve, September 30, 2014;
  https://perso.uclouvain.be/arnaud.browet/files/thesis/thesis.pdf.

\end{thebibliography}

\end{document}


\section{Introduction}

The objective of the exercise is to study several methods of measuring the
speed of sound in air: the resonance method, the phase comparison method, and
the time difference method, including successive difference method in
measurement data processing.

Sound is a mechanical wave that propagates through a compressible medium.
It's a longitudinal wave since the direction of vibrations of the medium is
the same as the direction of propagation. Sound with the frequency higher
than $20,000\ Hz$ is called \emph{ultrasound}, which is chosen as the signal
source in this experiment because its wavelength is short enough to measure
the speed precisely.

The phase speed $v$, the frequency $f$ and the length $\lambda$ of a wave are
related by the formula
\begin{equation}\label{vlf}
    v=\lambda f.
\end{equation}

For motion with constant speed $v$ along a straight line, we have
\begin{equation}\label{vlt}
    v=\frac{L}{t},
\end{equation}
where $L$ is the distance travelled over time $t$.


% pathAPEXCalculus_Source/text/06_Hyperbolic_Functions.tex
%c13ffb9  on Jun 15, 2015
%@APEXCalculus APEXCalculus Updated files from Version 2.0 to Version 3.0.
%1 contributor
%RawBlameHistory     421 lines (353 sloc)  20.7 KB
\section{Hyperbolic Functions}\label{sec:hyperbolic}

This section defines the derivatives of the hyperbolic functions and describes some of their properties. Recall their definition.

\begin{definition}{Hyperbolic Functions}{def:hyperbolic_functions}
{\noindent%
\begin{minipage}{.5\textwidth}
\begin{enumerate}
\item		$\ds \cosh x = \frac{e^x+e^{-x}}2$\index{hyperbolic function!definition}
\item		$\ds \sinh x = \frac{e^x-e^{-x}}2$
\item		$\ds \tanh x = \frac{\sinh x}{\cosh x}$
\end{enumerate}
\end{minipage}
\begin{minipage}{.5\textwidth}
\begin{enumerate}\addtocounter{enumi}{3}
\item		$\ds \sech x = \frac{1}{\cosh x}$
\item		$\ds \csch x = \frac{1}{\sinh x}$
\item		$\ds \coth x = \frac{\cosh x}{\sinh x}$
\end{enumerate}
\end{minipage}
}\\
\end{definition}

\begin{example}{Exploring properties of hyperbolic functions}{ex_hf1}
\noindent Use Definition \ref{def:hyperbolic_functions} to rewrite the following expressions.
\begin{enumerate}
\item		$\frac{d}{dx}\big(\cosh x\big)$
\item		$\frac{d}{dx}\big(\sinh x\big)$
\item		$\frac{d}{dx}\big(\tanh x\big)$
\end{enumerate}
\end{example}

\begin{solution}
{
\begin{enumerate}
\item  \hfill$\begin{aligned}[t]
	\frac{d}{dx}\big(\cosh x\big) &= \frac{d}{dx}\left(\frac{e^x+e^{-x}}2\right) \\
					&= \frac{e^x-e^{-x}}2\\
					&= \sinh x.
	\end{aligned}\hfill$

So $\frac{d}{dx}\big(\cosh x\big) = \sinh x.$

\item  \hfill$\begin{aligned}[t]
	\frac{d}{dx}\big(\sinh x\big) &= \frac{d}{dx}\left(\frac{e^x-e^{-x}}2\right) \\
					&= \frac{e^x+e^{-x}}2\\
					&= \cosh x.
	\end{aligned}\hfill$

So $\frac{d}{dx}\big(\sinh x\big) = \cosh x.$

\item  \hfill$\begin{aligned}[t]
	\frac{d}{dx}\big(\tanh x\big) &= \frac{d}{dx}\left(\frac{\sinh x}{\cosh x}\right) \\
					&= \frac{\cosh x \cosh x - \sinh x \sinh x}{\cosh^2 x}\\
					&= \frac{1}{\cosh^2 x}\\
					&= \sech^2 x.
	\end{aligned}\hfill$

So $\frac{d}{dx}\big(\tanh x\big) = \sech^2 x.$
\end{enumerate}
\vskip-\baselineskip
}
\end{solution}

The following Key Idea summarizes the derivatives relating to hyperbolic functions. Each can be verified by referring back to Definition \ref{def:hyperbolic_functions}.
\textbf{Derivatives}
\begin{enumerate}
\item $\frac{d}{dx}\big(\cosh x\big) = \sinh x$
\item $\frac{d}{dx}\big(\sinh x\big) = \cosh x$
\item $\frac{d}{dx}\big(\tanh x\big) = \sech^2 x$
\item $\frac{d}{dx}\big(\sech x\big) = -\sech x\tanh x$
\item $\frac{d}{dx}\big(\csch x\big) = -\csch x\coth x$
\item $\frac{d}{dx}\big(\coth x\big) = -\csch^2x$
\end{enumerate}

%We practice using Key Idea \ref{idea:hyperbolic_identities}.\\

%\example{ex_hf2}{Derivatives and integrals of hyperbolic functions}{
%Evaluate the following derivatives and integrals.

%\begin{minipage}[t]{.5\linewidth}
%\begin{enumerate}
%\item		$\ds\frac{d}{dx}\big(\cosh 2x\big)$
%\item		$\ds\int \sech^2(7t-3)\ dt$
%\end{enumerate}
%\end{minipage}
%\begin{minipage}[t]{.5\linewidth}
%\begin{enumerate}\addtocounter{enumi}{2}
%\item		$\ds \int_0^{\ln 2} \cosh x\ dx$
%\end{enumerate}
%\end{minipage}
%}
%{\begin{enumerate}
%\item		Using the Chain Rule directly, we have $\frac{d}{dx} \big(\cosh 2x\big) = 2\sinh 2x$.
%
%Just to demonstrate that it works, let's also use the Basic Identity found in Key Idea \ref{idea:hyperbolic_identities}: $\cosh 2x = \cosh^2x+\sinh^2x$.
%\begin{align*}\frac{d}{dx}\big(\cosh 2x\big) = \frac{d}{dx}\big(\cosh^2x+\sinh^2x\big) &= 2\cosh x\sinh x+ 2\sinh x\cosh x\\ &= 4\cosh x\sinh x.
%\end{align*}
%Using another Basic Identity, we can see that $4\cosh x\sinh x = 2\sinh 2x$. We get the same answer either way.

%\item	  We employ substitution, with $u = 7t-3$ and $du = 7dt$. Applying Key Ideas \ref{idea:linearsub}  and \ref{idea:hyperbolic_identities} we have:
%$$ \int \sech^2 (7t-3)\ dt = \frac17\tanh (7t-3) + C.$$
%
%\item		$$\int_0^{\ln 2} \cosh x\ dx = \sinh x\Big|_0^{\ln 2} = \sinh (\ln 2) - \sinh 0 = \sinh(\ln 2).$$
%
%We can simplify this last expression as $\sinh x$ is based on exponentials:
%$$\sinh(\ln 2) = \frac{e^{\ln 2}-e^{-\ln 2}}2 = \frac{2-1/2}{2} = \frac34.$$
%\end{enumerate}
%\vskip-1.5\baselineskip


\documentclass[12pt, titlepage, oneside]{article}

\input{settings}

\begin{document}

	\textbf{ELECENG 3TQ3}\\
	\textbf{Elston A.}

	\section{Lecture 1}

\subsection{Probability vs. Statistics}

\items
\item Statistics is probability formulate differently
\item Statistics applies to probability to extract information about data and recommend what decisions should be made
\eitems

\b{Probabilistic: } You have a fair coin, you flip it 100 times, what is the probability to getting more than a 70\% heads (We expect a small number).

\b{Statistical: } You have a coin of unknown fairness, you flip it 100 times and get 70\% heads. Your job would be to draw conclusions about that coin.

The main difference is that you know the conditions when you are deriving probability of a particular outcome. In statistics, you know the outcome and you are trying to obtain conclusions about the underlying relations, models, and hidden patterns in the data.

\subsection{Frequency vs. Bayes}

Consider a simple coin toss and assume you toss it $n$ times. Let $nh$ be the number of heads and $nt = n - nh$ be the number of tails.

The \b{Frequentist} estimates the probability of heads as $Ph = nh/n$. That is the frequency of the "head" event.

The \b{Beyesian} attempts to find prior knowledge such as where was the coin minted and built. This "prior" information is then used for analyzing the data.

\subsection{Counting and Sets}

We need to determine all the possible all the outcomes in order to determine probabilities. To accomplish this we would need to understand sets, unions, intersections, and complements. Also aspects of of combinatorics calculus is needed in order to analyze problems eg. selecting $n$ out of $m$ objects.

\ex Toss a fair coin 4 times. What is the probability to get exactly two tails?

We can approach this using the counting approach by finding all the possible outcomes.

S = \{ HHHH, HHHT, HHTH, HHTT, HTHH, HTHT, HTTH, HTTT, THHH, THHT, THTH, THTT, TTHH, TTHT, TTTH, TTTT\}

Then we find the set of favorable outcomes

A = \{HHTT, HTHT, HTTH, THHT, THTH, TTHH\}

So the probability of the event to exactly two tails in 4 tosses is the number of favourable outcomes divided by the total outcomes = 6/16.

But this type of approach becomes impossible once the space grows. \ex if we had 100 tosses.

\subsection{Formalizing Sets/Counting}

A \b{set} is a collection of elements: \ex patients with blood pressure over 140, people with brown hair, etc.

We use $x \in S$ to denote \b{element} x belonging to the set $S$: \ex John has brown hair, hence John is an element in the set of people with brown hair.

We say a set $A$ is a \b{subset} of $S$ if all the elements in $A$ belongs to $S$: \ex we can pick 6 people from the people with brown hair set and make a new set. This new set would be a subset of the original set of people with brown hair.

An \b{empty set} is the set with no elements denoted as $\emptyset$

\subsection{Set Operations}

\b{Complement}: The complement of $A$ in $S$ is all the elements in $S$ that are not in $A$. We denote this as $S-A$ or $A^c$.

\ex Let $B$ be the set of all the people with brown hair and shorter than 6 feet. The complement of this would be all the people who do not have brown hair or are taller than 6 feet.

\b{Union}: The union of $A$ and $B$ is the set of all the elements that are in $A$ or $B$ (or in both). We denote this as: $A \u B$.

\b{Intersection}: The intersection of $A$ and $B$ is the set of all elements that are in both $A$ and $B$. We denote this as: $A\n B$.

\b{Disjoint}: Two sets are called disjoint if they have no mutual elements

\b{Difference}: The difference of $A$ and $B$ is the set of all elements in A, but are not in B and is denoted as $A \backslash B$.

\ex Consider a set of 10 cities \{Berlin, Munich, Madrid, Barcelona, Beijing, Shanghai,  Portland, Washington DC, Ottawa, Toronto\}

Consider two subsets: 1) M is a set of capitols and 2) N is a set of cities in Europe

M=\{Berlin, Madrid, Beijing, Washington DC, Ottawa\}

N=\{Berlin, Munich, Madrid, Barcelona\}

\items
\item Intersection: \{Berlin, Madrid\}
\item Union: \{Berlin, Munich, Madrid, Barcelona, Beijing, Washington DC, Ottawa\}
\item Complement of M in S : \{Munich, Barcelona, Shanghai, Portland, Toronto\}
\item Difference  = \{Beijing, Washington DC, Ottawa\}
\eitems

\subsection{De Morgans Laws}

\begin{align}
(A \u B)^c = A^c \n B^c
\end{align}
\begin{align}
(A \n B)^c = A^c \u B^c
\end{align}

\subsection{Proof of De Morgans Laws}

\begin{align*}
&x \in (A \n B)^c\\
&\iff x \notin (A \n B) \\
&\iff x \in A^c \u x \in B^c
\end{align*}
\begin{align*}
&x \in (A \u B)^c\\
&\iff x \notin (A \u B)\\
&\iff x \in A^c \n x \in B^c
\end{align*}

\subsection{Venn Diagrams}

 \begin{figure}[h!]
 	\centering
 	\includegraphics[width=0.5\linewidth]{images/S}
 	\caption{Set $S$ of all possible outcomes}
 	\label{fig:s}
 \end{figure}
 \begin{figure}[h!]
 	\centering
 	\includegraphics[width=0.2\linewidth]{images/A}
 	\caption[]{The set of all the outcomes in $A$}
 	\label{fig:a}
 \end{figure}

 \begin{figure}[h!]
 	\centering
 	\includegraphics[width=0.2\linewidth]{images/B}
 	\caption[]{The set of all the outcomes in $B$}
 	\label{fig:b}
 \end{figure}

 \begin{figure}[h!]
 	\centering
 	\includegraphics[width=0.3\linewidth]{images/Union}
 	\caption[]{Union of set $A$ and set $B$}
 	\label{fig:union}
 \end{figure}

\begin{figure}[h!]
	\centering
	\includegraphics[width=0.2\linewidth]{images/intersection}
	\caption{Intersection of $A$ and $B$}
	\label{fig:intersection}
\end{figure}

\begin{figure}[h!]
	\centering
	\includegraphics[width=0.3\linewidth]{images/Ac}
	\caption{Complement of $A$ or $A^c$}
	\label{fig:ac}
\end{figure}
\newpage
\begin{figure}[h!]
	\centering
	\includegraphics[width=0.3\linewidth]{images/B-A}
	\caption{$B-A$ or $B \backslash A$}
	\label{fig:b-a}
\end{figure}

\subsection{Set Product and Additional Set Terminology}

$S \times T = \{ (s,t) \in S \text{ and } t \in T\}$

\ex $\{1,2\} \times \{3,4,5\} = \{(1,3),(1,4),(1,5), (2,3), (2,4), (2,5)\}$

Cardinal number is the number of elements in the set. Cardinal number is denoted as $\#s$ or $card\{S\}$ or $|S|$.

Inclusion - Exclusion principle: $|A \u B| = |A| + |B| - |A \n B| $

We subtract $|A \n B|$ to ensure we do not double count.

\end{document}


%% -*- coding:utf-8 -*-
\chapter{Monads}

Monads are very important for pure functional programming languages
such as Haskell. We will start with \mynameref{def:monoid}
consideration, continue with the formal mathematical definition for
monad and
will finish with programming languages examples later.

\section{Monoid in \textbf{Set} category}
We are going to consider \mynameref{def:monoid} in the terms of Set theory and
will try to give the definition that is based rather on morphisms then
on internal set structure i.e. we will use
\mynameref{def:categorical_approach}. Let $M$ is a set and by the
monoid
definition (\cref{def:monoid})
$\forall m_1, m_2 \in M$ we can define a new element of the set
$\mu(m_1, m_2) \in M$. Later we will use the following notation for
the $\mu$:
\[
\mu(m_1, m_2) \equiv m_1 \cdot m_2.
\]
If the $(M, \cdot)$ is monoid then the following 2 conditions have to
be satisfied. The first one (associativity) declares that $\forall
m_1, m_2, m_3 \in M$
\[
m_1 \cdot ( m_2 \cdot m_3) = ( m_1 \cdot
m_2 ) \cdot m_3.
\]
The second one (identity presence) says that
\(
\exists e \in M
\) such that $\forall m \in M$:
\begin{equation}
m \cdot e = e \cdot m = m.
\label{eq:monoid2}
\end{equation}

With the first one we can define $\mu$ as a
\mynameref{def:morphism} in the following way
\[
\mu: M\times M \to M,
\]
where $M \times M$ is the \mynameref{ex:set_product} in the
\mynameref{def:setcategory}. I.e. $M \times M, M \in \catob{Set}$ and
$\mu \in \cathom{Set}$.  Consider other objects of $\cat{Set}$: $A =
M \times \left( M \times M \right)$ and $A' = \left( M \times M \right)
\times M$. They are not the same but there is a trivial
\mynameref{def:isomorphism} between them $A \cong_\alpha A'$, where
the isomorphism $\alpha$ defined as
\[
\alpha(x,(y,z)) = ((x,y),z).
\]
Consider the action of \mynameref{def:product_of_morphisms}
$\idm{M} \times \mu$ on $A$:
\[
\idm{M} \times \mu \left(x,\left(y,z\right)\right) =
\left(\idm{M}(x),\mu\left(y,z\right)\right) =
\left(x, y \cdot z\right) \in M \times M
\]
i.e. $\idm{M} \times \mu: M \times \left( M \times M \right) \to M
\times M$. If we act $\mu$ on the result then we can obtain:
\begin{eqnarray}
\mu \left(\idm{M} \times \mu \left(x,\left(y,z\right)\right)\right) =
\left(\idm{M}(x),\mu\left(y,z\right)\right) =
\nonumber \\
=
\mu\left(x, y \cdot z\right) = x \cdot (y\cdot z) \in M,
\nonumber
\end{eqnarray}
i.e.
$\mu \circ \left(\idm{M} \times \mu\right): M \times \left( M \times M
\right) \to M$.

For $A'$ we have the following one:
\begin{eqnarray}
\mu\circ\left(\mu \times \idm{M}\right)\left(\left(x,y\right),z\right)
= \mu\left(x \cdot y, z\right) = (x \cdot y) \cdot z.
\nonumber
\end{eqnarray}
Monoid associativity requires
\[
x \cdot (y\cdot z) =
(x \cdot y) \cdot z
\]
i.e. the morphisms is shown in \cref{fig:monoid_mu_alpha} commute:
\begin{equation}
\label{eq:monad_monoid_mu}
\mu\circ\left(\mu \times \idm{M}\right) =
\mu \circ \left(\idm{M} \times \mu\right) \circ \alpha.
\end{equation}

\begin{figure}
  \centering
  \begin{tikzpicture}[ele/.style={fill=black,circle,minimum
        width=.8pt,inner sep=1pt},every fit/.style={ellipse,draw,inner
        sep=-2pt}]

    % the texts

    \node[ele,label=above:$M\times\left(M \times M\right)$] (M31) at (0,3) {};
    \node[ele,label=above:$\left(M \times M\right)\times M$] (M32) at (6,3) {};
    \node[ele,label=below:$M \times M$] (M21) at (0,0) {};
    \node[ele,label=below:$M \times M$] (M22) at (6,0) {};
    \node[ele,label=below:$M$] (M) at (3,0) {};

    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M31) to
    node[sloped,above]{$\cong_\alpha$} (M32);
    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M31) to
    node[sloped,below]{$\idm{M} \times \mu$} (M21);
    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M32) to
    node[sloped,below]{$\mu \times \idm{M}$} (M22);
    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M22) to
    node[sloped,above]{$\mu$} (M);
    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M21) to
    node[sloped,above]{$\mu$} (M);
  \end{tikzpicture}
  \caption{Commutative diagram for $\mu\circ\left(\mu \times
    \idm{M}\right) = \mu \circ \left(\idm{M} \times \mu\right) \circ
    \alpha$.}
  \label{fig:monoid_mu_alpha}
\end{figure}
Very often the isomorphism $\alpha$ is omitted i.e.
\[
M\times\left(M \times M\right)
= \left(M \times M\right)\times M = M^3
\]
and the morphism
equality \eqref{eq:monad_monoid_mu} is written as follow
\[
\mu\circ\left(\mu \times \idm{M}\right) =
\mu \circ \left(\idm{M} \times \mu\right).
\]
The corresponding commutative diagram is shown in
\cref{fig:monoid_mu}.
\begin{figure}
  \centering
  \begin{tikzpicture}[ele/.style={fill=black,circle,minimum
        width=.8pt,inner sep=1pt},every fit/.style={ellipse,draw,inner
        sep=-2pt}]

    % the texts

    \node[ele,label=above:$M^3$] (M3) at (0,3) {};
    \node[ele,label=above:$M \times M$] (M21) at (3,3) {};
    \node[ele,label=below:$M \times M$] (M22) at (0,0) {};
    \node[ele,label=below:$M$] (M) at (3,0) {};

    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M3) to
    node[sloped,below]{$\idm{M} \times \mu$} (M21);
    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M3) to
    node[sloped,above]{$\mu \times \idm{M}$} (M22);
    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M22) to
    node[sloped,above]{$\mu$} (M);
    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M21) to
    node[sloped,above]{$\mu$} (M);
  \end{tikzpicture}
  \caption{Commutative diagram for $\mu\circ\left(\mu \times
    \idm{M}\right) = \mu \circ \left(\idm{M} \times \mu\right)$.}
  \label{fig:monoid_mu}
\end{figure}

For \eqref{eq:monoid2} consider a morphism $\eta$ from
\mynameref{def:singleton_set}
\footnote{
 It also is called \cite{bib:maclane98} as a one point set
}
$I = \{0\}$ to the special element $e \in M$ such that
$\forall m \in M: e \cdot m = m \cdot e = m$. I.e. $\eta: I \to M$ and
$e = \eta(0)$. Consider 2 sets $B = I \times M$ and $B' = M \times I$.
We have 2 \mynameref{def:isomorphism}s: $B \cong_\lambda M$ and $B'
\cong_\rho M$ such that
\[
\lambda(m) = 0 \times m
\]
and
\[
\rho(m) = m \times 0.
\]

If we apply the products (see \mynameref{def:product_of_morphisms}) $\eta \times \mu$ and
$\mu \times \eta$ on $B$ and $B'$ respectively then we get
\begin{eqnarray}
\eta \times \idm{M} \left(0 \times m\right) = e \times m,
\nonumber \\
\idm{M} \times \eta \left(m \times 0\right) = m \times e.
\nonumber
\end{eqnarray}
After the application of $\mu$ on the result we obtain
\begin{eqnarray}
\mu \left(\eta \times \idm{M} \left(0 \times m\right) \right)
= \mu \left( e \times m \right) = e \cdot m,
\nonumber \\
\mu \left( \idm{M} \times \eta \left(m \times 0\right) \right) =
\mu \left(m \times e \right) = m \cdot e.
\nonumber
\end{eqnarray}
\begin{figure}
  \centering
  \begin{tikzpicture}[ele/.style={fill=black,circle,minimum
        width=.8pt,inner sep=1pt},every fit/.style={ellipse,draw,inner
        sep=-2pt}]

    % the texts

    \node[ele,label=above:$M$] (M') at (0,3) {};
    \node[ele,label=above:$M \times M$] (M21) at (3,3) {};
    \node[ele,label=below:$M \times M$] (M22) at (0,0) {};
    \node[ele,label=below:$M$] (M) at (3,0) {};

    \draw [double equal sign distance] (M') to (M);
    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M') to
    node[sloped,below]{$\lambda$} (M21);
    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M') to
    node[sloped,above]{$\rho$} (M22);
    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M22) to
    node[sloped,above]{$\mu$} (M);
    \draw[->,thick,shorten <=2pt,shorten >=2pt] (M21) to
    node[sloped,above]{$\mu$} (M);
  \end{tikzpicture}
  \caption{Commutative diagram for $\mu \circ (\eta \times \idm{M})
    \circ \lambda = \mu \circ (\idm{M} \times \mu) \circ \rho =
    \idm{M}$ .}
  \label{fig:monoid_eta_lambda_rho}
\end{figure}
The \eqref{eq:monoid2} leads to the following equation for morphisms
\[
\mu \circ (\eta \times \idm{M}) \circ \rho =
\mu \circ (\idm{M} \times \mu) \circ \lambda =
\idm{M}
\]
or the commutative diagram show on \cref{fig:monoid_eta_lambda_rho}.

Before given a formal definition lets look at the operations were used
for the construction. The first one is the product of 2 objects:
\[
M \times M.
\]
We also have 2 pairs of morphisms:
\begin{eqnarray}
\mu: M \times M \to M,
\nonumber \\
\idm{M}: M \to M.
\nonumber
\end{eqnarray}
and
\begin{eqnarray}
\eta: I \to M,
\nonumber \\
\idm{M}: M \to M.
\nonumber
\end{eqnarray}
The pairs can be combined into one using
\mynameref{def:product_of_morphisms} as follows:
\begin{eqnarray}
\mu \times \idm{M}: \left(M \times M\right) \times M \to M \times M,
\nonumber \\
\idm{M} \times \mu: M \times \left(M \times M\right) \to M \times M
\nonumber
\end{eqnarray}
and
\begin{eqnarray}
\eta \times \idm{M}: I \times M \to M \times M,
\nonumber \\
\idm{M} \times \eta: M \times I \to M \times M.
\nonumber
\end{eqnarray}
The same structure
\footnote{not only objects mapping but also morphisms mapping}
is used by
\mynameref{def:functor}
and
especially by \mynameref{ex:product_bifunctor}.

%% If we take into consideration that one-point set is
%% \mynameref{ex:set_terminal_object} then we can conclude that the
%% monoid can be defined for instance in a \mynameref{def:cartesian_closed_category}
%% as follow
Now we are ready to provide the monoid definition in the terms of morphisms.
\begin{definition}[Monoid]
\label{def:monoid_category}
Consider \mynameref{def:setcategory} $\cat{C}$ with a
\mynameref{def:singleton_set} $t \in \catob{C}$. The \mynameref{def:cartesian_product}
with \mynameref{def:product_of_morphisms} forms a
\mynameref{def:bifunctor} $\times$ (see \cref{ex:product_bifunctor}).
The object $m \in
\catob{C}$ is called \textit{monoid} if the following
conditions satisfied:
\begin{enumerate}
\item there is a \mynameref{def:morphism} $\mu: m \times m \to m$ in
  the category
\item there is another morphism $\eta: t \to m$ in the category
\item the morphisms satisfy the following conditions:
\begin{equation}
\mu\circ\left(\mu \times
    \idm{M}\right) = \mu \circ \left(\idm{M} \times \mu\right) \circ
    \alpha,
\label{eq:monoid_def_mu}
\end{equation}
\begin{equation}
\mu \circ (\eta \times \idm{M})
\circ \lambda = \mu \circ (\idm{M} \times \mu) \circ \rho =
\idm{M}
\label{eq:monoid_def_eta}
\end{equation}
where $\alpha$ (associator) is an \mynameref{def:isomorphism} between
$m \times (m \times m)$ and $(m \times m) \times m$. $\lambda, \rho$
are other isomorphisms:
\[
m \cong_\lambda t \times m
\]
and
\[
m \cong_\rho m \times t
\]
\end{enumerate}
\end{definition}

\section{Monoidal category}
As we saw in the categorical definition for monoid (see
\cref{def:monoid_category}) the category $\cat{C}$ should satisfy
several conditions to have an object as monoid. Lets formalise the conditions.
\begin{definition}[Monoidal category]
\label{def:monoidal_category}
A category $\cat{C}$ is called \textit{monoidal category} if it is
equipped with a \mynameref{def:monoid} structure i.e. there are
\begin{itemize}
\item \mynameref{def:bifunctor} $\otimes: \cat{C} \times \cat{C} \tof
  \cat{C}$ called \textit{monoidal product} \index{Monoidal
    product!definition}
\item an \mynameref{def:object} $e$ called unit object or identity object
\end{itemize}

The elements should satisfy (up to \mynameref{def:isomorphism})
several conditions. The first one:
associativity:
\begin{equation}
a \otimes \left( b \otimes c \right) \cong_\alpha
  \left( a \otimes b \right) \otimes c,
\nonumber
\end{equation}
where $\alpha$ is called associator. \index{Associator!definition}
The second condition says that
$e$ can be treated as left and right identity:
\begin{eqnarray}
a \cong_\lambda e \otimes a,
\nonumber \\
a \cong_\rho a \otimes e,
\nonumber
\end{eqnarray}
where $\lambda, \rho$ are called as left and right unitors respectively. \index{Left unitor!definition} \index{Right unitor!definition}
\end{definition}

In the \mynameref{def:setcategory} we have $\times$ as the monoidal
product (see \cref{ex:product_bifunctor}). There is also a
morphism $\eta$ from terminal object $t$ to $e$
\cite{bib:stackexchange:terminalinmonoid} (see \cref{def:monoid_category}).

\begin{definition}[Strict monoidal category]
\label{def:strict_monoidal_category}
\index{Associator}
A \mynameref{def:monoidal_category} $\cat{C}$ is said to be \textit{strict} if the
associator, left and right unitors are all identity morphisms i.e.
\[
\alpha = \lambda = \rho = \idm{C}.
\]
\index{Left unitor} \index{Right unitor}
\end{definition}

\begin{remark}[Monoidal product]
\label{rem:monoidal_product}
The monoidal product is a binary operation that specifies the exact
monoidal structure. Often it is called as \textit{tensor product} but
we will avoid the naming because it is not always the same as the
\mynameref{def:tensor_product} introduced for
\mynameref{def:hilbert_space}s. We also note that the monoidal product is
a \mynameref{def:bifunctor}.
\end{remark}

\section{Category of endofunctors}

The \mynameref{ex:fun_category} is an example of a category. We can
apply additional limitation and consider only
\mynameref{def:endofunctor}s i.e. we will look at the category
$[\cat{C}, \cat{C}]$ - the category of functors from category $\cat{C}$ to
the same category. One of the most popular math definition of a monad
is the following:
``All told, a monad in X is just a monoid in the category of
endofunctors of X''\cite{bib:maclane98}.
Later we will give an explanation for that one.

We start with the formal definition of category of endofunctors and a
tensor product in the category
\begin{definition}[Category of endofunctors]
\label{def:category_of_endofunctors}
Let $\cat{C}$ is a category, then  the category $[\cat{C}, \cat{C}]$ of
functors from category $\cat{C}$ to the same category is called the
category of endofunctors. The monoidal product in the category is the
functor composition.
\end{definition}

\begin{definition}[Monad]
  \label{def:monad}
  The monad $M$ is an \mynameref{def:endofunctor} with 2
  \mynameref{def:nt}s:
  \begin{equation}
    \label{eq:monad_mu}
    \mu: M \circ M \tont M
  \end{equation}
  and
  \begin{equation}
    \label{eq:monad_eta}
    \eta: \idf{C} \tont M,
  \end{equation}
  where $\idf{C}$ is \mynameref{def:idfunctor}.

  The $\eta, \mu$ should satisfy the following conditions:
  \begin{eqnarray}
    \mu \circ M \mu = \mu \circ \mu M,
    \nonumber \\
    \mu \circ M \eta = \mu \circ \eta M = \idnt{M},
    \label{eq:monad}
  \end{eqnarray}
  where $M \mu, M \eta$ - \mynameref{def:rw}s, $\mu M, \eta M$ -
  \mynameref{def:lw}s, $\idnt{M}$ - \mynameref{def:idnt} for $M$.
  \mynameref{def:vertical_composition} is used in the equations.

  The monad will be denoted later as $\left<M, \mu, \eta\right>$.
\end{definition}

Lets look at the requirements \eqref{eq:monad} more closely. Notice
that the functor composition is associative:
\[
M \circ ( M \circ M ) = (M \circ M) \circ M = M^3.
\]
Secondly
all rewrite it with \eqref{eq:lw} and \eqref{eq:rw} as follows
\begin{eqnarray}
  \mu \circ \left( \idnt{M} \star \mu \right) =
  \mu \circ \left( \mu \star \idnt{M} \right),
  \nonumber \\
  \mu \circ \left( \idnt{M} \star \eta \right) =
  \mu \circ \left( \eta \star \idnt{M} \right) = \idnt{M}.
  \label{eq:monad_p1}
\end{eqnarray}
Thus we can notice that the pair of operations (composition $\circ$
and \mynameref{def:horizontal_composition} $\star$) forms the bifunctor (see
\mynameref{rem:bifunctor_fun_cat}).

The morphism $\idnt{M} \star \mu$ acts on $M \circ ( M \circ M )$ as
\[
\idnt{M} \star \mu : M \circ ( M \circ M ) \tont M \circ (M \otimes M)
\]
thus
\[
\mu \circ (\idnt{M} \star \mu) : M \circ ( M \circ M ) \tont M \otimes (M \otimes M).
\]
Similarly
\[
\mu \circ (\mu \star \idnt{M}) : (M \circ  M) \circ M  \tont ( M \otimes M) \otimes M.
\]
I.e. the both morphisms start at the same object $M^3$ and finish also
at the same point. The equality
\begin{equation}
\mu \circ (\idnt{M} \star \mu) =
\mu \circ (\mu \star \idnt{M})
\label{eq:monoidalobject1}
\end{equation}
is similar to the conditions on the \cref{fig:monoid_mu} and can be
written as \cref{fig:monad_monoid1}. Thus if we compare
\eqref{eq:monoidalobject1} and \eqref{eq:monoid_def_mu} then we can say
that they are same if we replace $\star$ sign with $\times$ one. I.e.
in the case we can say that the monad looks like a
\mynameref{def:monoid_category}.

\begin{figure}
  \centering
  \begin{tikzpicture}[ele/.style={fill=black,circle,minimum
        width=.8pt,inner sep=1pt},every fit/.style={ellipse,draw,inner
        sep=-2pt}]

    % the texts

    \node[ele,label=left:$M^3$] (m3) at (0,4) {};
    \node[ele,label=left:$M^2$] (m21) at (0,0) {};
    \node[ele,label=right:$M^2$] (m22) at (4,4) {};
    \node[ele,label=right:$M$] (m) at (4,0) {};

    \draw[->,thick,shorten <=2pt,shorten >=2pt] (m3) to
    node[sloped,above]{$\idnt{M} \star \mu$} (m21);
    \draw[->,thick,shorten <=2pt,shorten >=2] (m3) to
    node[sloped,above]{$\mu \star \idnt{M}$} (m22);
    \draw[->,thick,shorten <=2pt,shorten >=2] (m21) to
    node[sloped,above]{$\mu$} (m);
    \draw[->,thick,shorten <=2pt,shorten >=2] (m22) to
    node[sloped,above]{$\mu$} (m);
  \end{tikzpicture}
  \caption{Monad as monoid in the category of endofunctors.}
  \label{fig:monad_monoid1}
\end{figure}

For the identity element consider the same trick: replace in
\eqref{eq:monoid_def_eta} tensor
product $\times$ with \mynameref{def:horizontal_composition} $\star$ and
morphisms $\idm{M}, \rho, \lambda$ with identity natural transformation
$\idnt{M}$. Thus the equation
\[
\mu \circ (\eta \times \idm{M})
\circ \lambda = \mu \circ (\idm{M} \times \mu) \circ \rho =
\idm{M}
\]
will be replaced with
\[
\mu \circ (\eta \star \idnt{M}) = \mu \circ (\idnt{M} \star \mu) =
\idnt{M}
\]
that is the exact we want to get (see second equation of
\eqref{eq:monad_p1}).

%% There should be an identity element for each \mynameref{def:monoid}.
%% Lets show that the second equation of  \eqref{eq:monad_p1} provides us
%% the required element. Lets $U =
%% \eta\left(\idf{C}\right)$.
%% We want to show that
%% \begin{equation}
%% U \otimes M = M \otimes U = M,
%% \nonumber
%% \end{equation}
%% that is exactly required as identity presence in the
%% \mynameref{def:monoid} definition.
%% Consider the action of the left part of second equation
%% \eqref{eq:monad_p1} on $M = M \circ \idf{C}$:
%% \begin{eqnarray}
%% \mu \circ \left( \idnt{M} \star \eta \right) \left[M \circ \idf{C}\right] =
%% \nonumber \\
%% =
%% \mu \left[M \circ U\right] = M \otimes U
%% \nonumber
%% \end{eqnarray}
%% For the middle part of  of second equation
%% \eqref{eq:monad_p1}
%% \begin{eqnarray}
%% \mu \circ \left( \eta \star \idnt{M} \right) \left[M\right] =
%% \mu \circ \left( \eta \star \idnt{M} \right) \left[\idf{C} \circ M\right] =
%% \nonumber \\
%% =
%% \mu \left[U \circ M\right] = U \otimes M.
%% \nonumber
%% \end{eqnarray}
%% Thus
%% finally
%% \[
%% M \otimes U = U \otimes M = M
%% \]
%% that finals the proof of monoidal structure.

\section{Monads in programming languages}
There are several examples of \mynameref{def:monad} implementation in
different programming languages:

\subsection{Haskell}

\begin{example}[Monad][\textbf{Hask}]
\label{ex:monad_haskell}
In Haskell monad can be defined from \mynameref{ex:functor_haskell} as follows
\footnote{real definition is quite different from the presented one}
\begin{minted}{haskell}
    class Functor m => Monad m where
        return :: a -> m a
        (>>=)  :: m a -> (a -> m b) -> m b
\end{minted}

To show how this one can be get we can start from a definition that is
similar to the math definition:
\begin{minted}{haskell}
    class Functor m => Monad m where
        return :: a -> m a
        join  :: m (m a) -> m a
\end{minted}
where \mintinline{haskell}{return} can be treated as $\eta$
\eqref{eq:monad_eta} and
\mintinline{haskell}{join} as $\mu$ \eqref{eq:monad_mu}. In the case
the bind operator \mintinline{haskell}{>>=} can be implemented as follows
\begin{minted}{haskell}
(>>=)  :: m a -> (a -> m b) -> m b
ma >>= f = join ( fmap f ma )
\end{minted}

\end{example}

\subsection{C++}
The monad in C++ use the functor definition from \mynameref{ex:functor_cpp}
\begin{minted}{c++}
// from functor.h
template < template< class ...> class M, class A, class B>
M<B> fmap(std::function<B(A)>, M<A>);

// file: monad.h
template < template< class ...> class M, class A>
M<A> pure(A);

template < template< class ...> class M, class A>
M<A> join(M< M<A> >);
\end{minted}
where \mintinline{c++}{pure} can be treated as $\eta$
\eqref{eq:monad_eta} and
\mintinline{haskell}{join} as $\mu$ \eqref{eq:monad_mu}. In the case
the bind operator can be implemented as follows
\begin{minted}{c++}
template < template< class ...> class M, class A, class B>
M<B> bind(std::function< M<B> (A) > f, M<A> a) {
  return join( fmap<>(f, a) );
};
\end{minted}

\subsection{Scala}

\begin{example}[Monad][\textbf{Scala}]
The monad concept is Scala is more close to formal math definition for
\mynameref{def:monad}. It can be defined as follows
\footnote{real definition is quite different from the presented one}
\label{ex:monad_scala}
\begin{minted}{scala}
trait M[A] {
  def flatMap[B](f: A => M[B]): M[B]
}

def unit[A](x: A): M[A]
\end{minted}
I.e. \mintinline{scala}{flatMap} can be considered as $\mu$ and
\mintinline{scala}{unit} as $\eta$.
\end{example}

TBD

\section{Kleisli category}

\begin{definition}[Kleisli category]
\label{def:kleisli_category}
Let $\cat{C}$ is a category, $M$ is an \mynameref{def:endofunctor} and
$\left<M, \mu, \eta\right>$ is a \mynameref{def:monad}. Then we can
construct a new category $\cat{C_M}$ as follows:
\begin{eqnarray}
\catob{C_M} = \catob{C},
\nonumber \\
\hom_{\cat{C_M}}\left(a, b\right) =
\hom_{\cat{C}}\left(a, M(b)\right)
\nonumber
\end{eqnarray}
i.e. objects of categories $\cat{C}$ and $\cat{C_M}$ are the same but
morphisms from $\cat{C_M}$ form a subset of morphisms $\cat{C}$:
$\cathom{C_M} \subset \cathom{C}$. The category is called as
\textit{Kleisli category}.

The identity morphism in the Kleisli category is the
\mynameref{def:nt} $\eta$ \eqref{eq:monad_eta} defined by the monad
$\left<M, \mu, \eta\right>$:
\[
\idm{C_M} = \eta
\]
\end{definition}

\begin{remark}[Kleisli category composition]
\mynameref{def:kleisli_category} has a non trivial composition rules.
If we have 2 \mynameref{def:morphism}s from $\cathom{C_M}$:
\[
f_M: a \to b
\]
and
\[
g_M: b \to c.
\]
The morphisms have correspondent ones in $\cat{C}$:
\[
f: a \to M(b)
\]
and
\[
g: b \to M(c).
\]
The composition $g_M \circ f_M$ gives a new morphism
\[
h_M = g_M \circ f_M: a \to c.
\]
The corresponding one from $\cat{C}$ is
\[
h: a \to M(c).
\]
It has to be pointed that the compositions in $\cat{C}$ and
$\cat{C_M}$ are not the same:
\[
g_M \circ f_M \ne g \circ f.
\]
\end{remark}

\mynameref{def:kleisli_category} widely spread in programming
especially it provides good description for different types of
computations, for instance \cite{bib:Moggi91, bib:milewski2018category}
\begin{itemize}
\item \textbf{Partiality} i.e. then a function not defined for each input, for
  instance the following expression is undefined (or partially
  defined) for $x = 0$: $f(x) = \frac{1}{x}$
\item \textbf{Non-Determinism} i.e. then multiply output are possible
\item \textbf{Side-effects} i.e. then a function communicates with
  an environment
\item \textbf{Exception} i.e. when some input is incorrect and can
  produce an abnormal result. Therefore it is the same as
  \textbf{Partiality} and will be considered below as the same type of
  computation.
\item \textbf{Continuation} i.e. when we need to save the current
  state of the computation and be able to restore it on demand later
\item \textbf{Interactive input} i.e. a function that reads data from
  an input device (keyboard, mouse, etc.)
\item \textbf{Interactive output} i.e. a function that writes data to
an output device (monitor etc.)
\end{itemize}

\subsection{Partiality and Exception}

Partial functions and exceptions can be processed via monad be called
as Maybe. There will be implementations in different languages below.
And the usage example for the following function implementation
\[
h(x) = \frac{1}{2 \sqrt{x}}.
\]
The function is a composition of 3 functions:
\begin{eqnarray}
f_1(x) = \sqrt{x},
\nonumber \\
f_2(x) = 2 \cdot x,
\nonumber \\
f_3(x) = \frac{1}{x}
\label{eq:monadmaybe_ex_f}
\end{eqnarray}
and as result the goal can be implemented as the following
composition:
\begin{equation}
h = f_3 \circ f_2 \circ f_1.
\label{eq:monadmaybe_ex_h}
\end{equation}
$f_2$ is a \mynameref{def:pure_function} and defined $\forall x \in \mathbb{R}$. The
functions $f_1, f_3$ are partially defined.

\begin{example}[Maybe monad][\textbf{Hask}]
\label{ex:maybe_monad_haskell}
The Maybe monad can be implemented as follows
\begin{minted}{haskell}
  instance Monad Maybe where
    return = Just
    join Just( Just x) = Just x
    join _ = Nothing
\end{minted}

Our functions \eqref{eq:monadmaybe_ex_f} can be implemented as follows
\begin{minted}{haskell}
f1 :: (Ord a, Floating a) => a -> Maybe a
f1 x = if x >= 0 then Just(sqrt x) else Nothing

f2 :: Num a => a -> Maybe a
f2 x = Just (2*x)

f3 :: (Eq a, Fractional a) => a -> Maybe a
f3 x = if x /= 0 then Just(1/x) else Nothing
\end{minted}

The $h$ \eqref{eq:monadmaybe_ex_h} is the composition via bind
operator:
\begin{minted}{haskell}
h :: (Ord a, Floating a) => a -> Maybe a
h x = (return x) >>= f1 >>= f2 >>= f3
\end{minted}

The usage example is the following:
\begin{minted}{bash}
*Main> h 4
Just 0.25
*Main> h 1
Just 0.5
*Main> h 0
Nothing
*Main> h (-1)
Nothing
\end{minted}

\end{example}

\begin{example}[Maybe monad][\textbf{C++}]
\label{ex:maybe_monad_cpp}
The Maybe monad can be implemented as follows
\begin{minted}{c++}
template <class A> using Maybe = std::optional<A>;

template < class A, class B>
Maybe<B> fmap(std::function<B(A)> f, Maybe<A> a) {
  if (a) {
    return f(a.value());
  }
  return {};
}

template < class A>
Maybe<A> pure(A a) {
  return a;
}

template < class A>
Maybe<A> join(Maybe< Maybe<A> > a){
  if (a) {
    return a.value();
  }
  return {};
}
\end{minted}

Our functions \eqref{eq:monadmaybe_ex_f} can be implemented as follows
\begin{minted}{c++}
std::function<Maybe<float>(float)> f1 =
    [](float x) {
      if (x >= 0) {
        return Maybe<float>(sqrt(x));
      }
      return Maybe<float>();
    };

std::function<Maybe<float>(float)> f2 = [](float x) { return 2 * x; };

std::function<Maybe<float>(float)> f3 =
    [](float x) {
      if (x != 0) {
        return Maybe<float>(1 / x);
      }
      return Maybe<float>();
    };
}
\end{minted}

The $h$ \eqref{eq:monadmaybe_ex_h} is the composition via bind
operator:
\begin{minted}{c++}
auto h(float x) {
  Maybe<float> a = pure(x);
  return bind(f3,bind(f2,bind(f1, a)));
};
\end{minted}

\end{example}

\subsection{Non-Determinism}

TBD

\subsection{Side effects and interactive input/output}

TBD

\subsection{Continuation}

TBD

\section{Examples}

\subsection{Quantum mechanics}

\begin{definition}[Tensor product]
  \label{def:tensor_product}
  TBD
\end{definition}

The tensor product in quantum mechanics is used for
representing a system that consists of multiple systems. For instance
if we have an interaction between an 2 level atom ($a$ is excited
state $b$ as a ground state) and one mode light then the
atom has its own Hilber space $\mathcal{H}_{at}$ with $\ket{a}$ and
$\ket{b}$ as basis
vectors.  Light also has its own Hilber space $\mathcal{H}_f$ with Fock state
$\{\ket{n}\}$ as the basis.
\footnote{
  Really the $\mathcal{H}_f$ is infinite dimensional Hilber space and
  seems to be out of our assumption about \textbf{FdHilb} category as
  a collection of finite dimensional Hilber spaces only.
}
The result system that describes both atom
and light is represented as the tensor product $\mathcal{H}_{at}
\otimes \mathcal{H}_f$.

The morphisms of \textbf{FdHilb} category have a connection with
\mynameref{def:tensor_product}. Consider the so called Hilbert-Schmidt
correspondence for finite dimensional Hilbert spaces i.e. for given
$\mathcal{A}$ and $\mathcal{B}$ there is a natural isomorphism between
the tensor product and linear maps (aka morphisms) between
$\mathcal{A}$ and $\mathcal{B}$:
\[
\mathcal{A}^\ast \otimes \mathcal{B} \cong \hom(\mathcal{A}, \mathcal{B})
\]
where $\mathcal{A}^\ast$ - \mynameref{def:dual_space}.

TBD


% !TEX root = main.tex

\chapter{Non-parametric methods}\label{chap:nonparametric}

%% intro
%A typical problem in statistics is to estimate the distribution of a random variable from a set of observations.
%
%\smallskip
So far we have assumed that the parametric form of the distribution is known, for example $X\sim\text{Bernoulli}(\theta)$ where $\theta\in[0,1]$ is unknown or $X\sim\text{Poisson}(\theta)$ where $\theta>0$ is unknown. We now look at methods for estimating the distribution of a random variable where the parametric form of the distribution is not known. Such methods are called \emph{non-parametric} or \emph{distribution-free} methods.

\smallskip
We consider only continuous distributions, so we can assume that the inverse CDF $F^{-1}(u)$ exists for all $u\in[0,1]$ and that the probability of different observations taking the same value is zero.

\input{10A_quantiles}
\input{10B_location_models}
\input{10C_one_sample_tests}


%!TEX root = TDT4265-Summary.tex
\section{Epipolar geometry}
With one camera, you know the direction to each pixel/object in an image, but not its distance. With two cameras and epipolar geometry, this can be determined.

\begin{figure}[htbp]
    \centering
    \includegraphics[width=0.95\linewidth]{images/Epipolar_geometry}
    \caption{Epipolar geometry}
    \label{fig:epipolar-geometry}
\end{figure}

In Figure \ref{fig:epipolar-geometry}, both cameras see the object $\V{X}$. The line $\V{O}_L$--$\V{X}$ from the left camera to $\V{X}$, is seen as the epipolar line $\e_R$--$\x_R$ by the right camera, marked in red. (Correspondingly, $\x_L$--$\e_L$ is an epipolar line for the left camera.) This means that $\V{X}$ must lie on the red epipolar line, but we don't know where (this is the fundamental ambiguity). This provides a constraint that can be used to create a model describing the geometric transformation between the two image planes.

\subsection{Essential matrix}
The epipolar constraint from the previous section can be formulated as
\begin{equation}
    \y_1\T \M{E} \y_0 = 0
\end{equation}
where $\y_1$ and $\y_0$ are normalized image coordinates and $\M{E}$ is the essential matrix. This relation holds if the coordinates correspond to the same point in 3D space. Thus, the essential matrix relates points in global coordinates.

\subsection{Fundamental matrix}
The fundamental matrix $\M{F}$ does the same as the essential matrix, but relates points given in image pixel coordinates, such as $\x_L$ and $\x_R$ in Figure \ref{fig:epipolar-geometry}. A similar relation holds for image coordinates that describe the same point in space:
\begin{equation}
    \x_1\T \M{F} \x_0 = 0
\end{equation}

\subsection{Structure from motion (SfM)}
SfM is a method for using data from many 2D images to create a 3D map. It's based on finding features in all the images taken (a huge amount is necessary). Match features between images to estimate camera location, and then pixel distances.


\documentclass[main.tex]{subfiles}
\begin{document}

\subsection{Spectrum of synchrotron radiation}

\marginpar{Saturday\\ 2020-8-29, \\ compiled \\ \today}

We will use a rough, heuristic approach.
Recall that the spectrum depends on
%
\begin{align}
\frac{ \dd{w}}{ \dd{\omega } \dd{\Omega }} \propto \abs{\hat{E}(\omega )}^2
\,,
\end{align}
%
where \(\hat{E}(\omega )\) is the Fourier transform of the electric field \(E(t)\).

When we compute the Fourier transform we do so referring to the time as seen by the observer; we will see that the observer in any specific direction will only see the radiation for a very short amount of time each revolution because of relativistic beaming.
In order to see this, consider a single charge moving along a helical path.

Let us choose two points 1 and 2 along the trajectory, such that the path length between them is \(\Delta s\) and the angle difference is \(\Delta \theta \). Let us also denote the radius of curvature of the trajectory as \(a\): then we will have \(\Delta s = a \Delta \theta \).

We know that radiation is emitted in a cone with aperture \(\sim 1/\gamma \): let us choose the two points so that they are at the edges of where the radiation starts and then stops being received by the observer.
Then, radiation emitted between the two points will always hit the observer.

We know that in this case \(\Delta \theta  = 2 / \gamma \) by the geometry of the setup, since \(2 / \gamma \) is the angle between two diametrically opposed points in the cone.
The equation of motion is
%
\begin{align}
m \gamma \dv{\vec{v}}{t} = \frac{q}{c} \vec{v} \times \vec{B}
\,,
\end{align}
%
so the modulus of the velocity change, for small enough time differences, is
%
\begin{align}
\frac{ \abs{ \Delta \vec{v}}}{ \Delta t} = \frac{q}{mc \gamma } v B \sin \alpha
\,,
\end{align}
%
where \(\alpha \) is the angle between the velocity and \(\vec{B}\), which is called the \emph{pitch angle}.
Since the modulus of \(\vec{v}\) is constant, we can write \(\abs{ \Delta \vec{v}} = v \Delta \theta\), so that
%
\begin{align}
v \frac{ \Delta \theta }{\Delta t} = \frac{q}{mc \gamma } vB \sin \alpha
\,,
\end{align}
%
but since the path travelled is given by \(\Delta s = v \Delta t\) we have
%
\begin{align}
v^2 \frac{ \Delta \theta }{ \Delta s} &= \frac{q}{mc \gamma } v B \sin \alpha  \\
 \frac{ \Delta \theta }{ \Delta s} &= \frac{q}{mc v \gamma } B \sin \alpha \\
 \frac{ 2/ \gamma  }{ \Delta s} &= \frac{q}{mc v \gamma } B \sin \alpha \\
 \Delta s = \frac{2 mc v}{q B \sin \alpha }
\,,
\end{align}
%
which is related to the frequency \(\omega _B = q B / mc \gamma \) by
%
\begin{align}
\Delta s = \frac{2 b}{\gamma \omega _B \sin \alpha  }
\,.
\end{align}

Then, we can derive the time needed to go from point 1 to point 2:
%
\begin{align}
\Delta t = \frac{\Delta s}{v} = \frac{2}{\gamma \omega _B \sin \alpha }
\,,
\end{align}
%
however we must be careful: although this is the time as measured in the lab frame, the motion of the particle is highly relativistic and the radiation is emitted in two different places.

Let us call \(t_1^A\) and \(t_2^A\) the times at which the radiation corresponding to  the start and the end of the pulse arrive at the observer.
They will be given by
%
\begin{align}
t_1^{A} = t_1 + \frac{L}{c} + \frac{\Delta s}{c}
\qquad \text{and} \qquad
t_2^{A} = t_2 + \frac{L}{c}
\,.
\end{align}

This means that
%
\begin{align}
t_1^A- t_2^{A} = t_2 - t_1 - \frac{\Delta s}{c}
= \frac{2}{\gamma \omega_B \sin \alpha } - 2 \frac{v/c}{\gamma \omega _B \sin \alpha }
= \frac{2}{\gamma \omega_B \sin \alpha } \qty(1 - \frac{v}{c})
\,,
\end{align}
%
but if the motion is highly relativistic we can approximate
%
\begin{align}
1 - \frac{v}{c} \approx 1 - \qty(1 - \frac{1}{2 \gamma^2}) = \frac{1}{2 \gamma^2}
\,,
\end{align}
%
therefore
%
\begin{align}
\Delta t^{A} = \frac{2}{\gamma \omega _B \sin \alpha } \frac{1}{2 \gamma^2} = \frac{1}{\gamma^3 \omega _B \sin \alpha }
\,,
\end{align}
%
so the reduction of the pulse duration is shorter by a factor \(\sim 1/ \gamma^3\) than \(1 / \omega _B\): it becomes \textbf{very short!}

Therefore, the spectrum will be rather broad in frequency, of the order \(\sim \gamma^3 \omega _B\).

Let us define the \textbf{critical pulsation} and the \textbf{critical frequency}:
%
\begin{align}
\omega _c = \frac{3}{2} \gamma^3 \omega _B \sin \alpha
\qquad \text{and} \qquad
\nu _c = \frac{3}{4 \pi } \gamma^3 \omega _B \sin \alpha
\,,
\end{align}
%
which will roughly be around the cutoff of the spectrum.

We can see by looking at the old formulas we derived that the dependence of the emitted power per unit solid angle on the angle is always in the form
%
\begin{align}
\frac{ \dd{w}}{ \dd{t} \dd{\Omega }} = F(\gamma \theta )
\,,
\end{align}
%
which is a consequence of relativistic beaming. Therefore, this will also hold for the dependence of the electric field.

Now, we want to find out how \(\theta \) is related to \(t\): we know that \(\theta = s/ a\) while \(t = (s/v) (1 - v/c)\); so
%
\begin{align}
t &= \frac{a \theta }{v} \qty(1 - \frac{v}{c}) \\
\gamma t &= \frac{\gamma  a \theta }{v} \qty(1 - \frac{v}{c}) \\
\gamma \theta &= \gamma t \frac{v}{a} \qty(1 - \frac{v}{c})^{-1}
\approx \gamma t \omega _B \sin \alpha 2 \gamma^2 \propto \omega _c t
\,.
\end{align}

Therefore, the electric field depends on time like \(E(t) \propto g(\omega _c t)\) for some function \(g\).
Its Fourier transform will then look like
%
\begin{align}
\hat{E}(\omega ) &\propto \int_{- \infty }^{\infty } g(\omega _c t) e^{-i \omega t} \dd{t}  \\
&\propto \int_{-\infty }^{\infty } g(z) e^{-i \frac{\omega}{\omega _c} z} \dd{z}
\,,
\end{align}
%
meaning that \emph{the dependence of \(\hat{E}(\omega )\) on \(\omega \) is only through the ratio \(\omega / \omega _c\).}

Recall that the power per unit solid angle is given by
%
\begin{align}
\frac{ \dd{w}}{ \dd{\omega } \dd{\Omega }} \propto \abs{\hat{E}(\omega )}^2
\,,
\end{align}
%
so if we integrate over the solid angle and divide by the period we find that the power per unit frequency is
%
\begin{align}
\frac{ \dd{w}}{ \dd{t} \dd{\omega }} = c_1 F \qty(\frac{\omega }{\omega _c} )
\,,
\end{align}
%
where \(c_1 \) is some constant and \(F\) is some function.
We can evaluate \(c_1 \) since we know what is the total radiated power:
%
\begin{align}
\dv{w}{t} = P
= \frac{2}{3} r_0^2 c \gamma^2 \beta_{\perp}^2B^2
= \frac{2}{3} \frac{q^{4} \beta^2 \sin^2 \alpha B^2 \gamma^2}{m^2 c^3}
\,,
\end{align}
%
where we used the fact that \(\beta _\perp^2 = \beta^2 \sin^2 \alpha \) and \(r_0^2= q^{4} / m^2 c^{4}\).
In terms of the constant \(c_1 \) and the function \(F\) this will be the integral over the frequency space:
%
\begin{align}
\dv{w}{t}
= c_1 \int_0^{\infty} F \qty(\frac{\omega }{\omega _c}) \dd{\omega }  =
c_1 \omega _c \int_0^{\infty } F(z) \dd{z} \overset{!}{=}
\frac{2}{3} \frac{q^{4} \beta^2 \sin^2 \alpha B^2 \gamma^2}{m^2 c^3}
\,,
\end{align}
%
so we can write the value of the constant \(c_1 \) as
%
\begin{align}
c_1 &= \frac{2}{3} \frac{q^{4} \beta^2 \sin^2 \alpha B^2 \gamma^2}{m^2 c^3} \frac{2}{3 \gamma^3 \omega _B \sin \alpha } \frac{1}{
\int_0^{\infty } F(z) \dd{z}
}   \\
&= \frac{q^3 \beta^2  \sin \alpha B}{mc^2} \frac{1}{\int_0^{ \infty } F(z) \dd{z}}
\,.
\end{align}

As we shall show later the integral will evaluate to \(\int_0^{ \infty } F(z) \dd{z} = 2 \pi / \sqrt{3}\), so, as long as the particle is ultrarelativistic (\(\beta \approx 1\)) we will have
%
\begin{align}
c_1 = \frac{q^3 \sin \alpha B}{mc^2} \frac{\sqrt{3}}{ 2 \pi }
\,,
\end{align}
%
therefore
%
\begin{align}
\frac{ \dd{w}}{ \dd{t} \dd{\omega }} = \frac{\sqrt{3}}{ 2 \pi }
\frac{q^3 B \sin \alpha }{mc^2} F \qty( \frac{\omega }{\omega _c})
\,.
\end{align}

The explicit expression of \(F\) can be derived \cite[par.\ 6.4]{rybickiRadiativeProcessesAstrophysics1979}, but we will not do it here. It is given by the modified Bessel function
%
\begin{align}
F(z) =  z \int_{z}^{\infty } k_{5/3} (x) \dd{x}
\,.
\end{align}

It is a smooth function of \(z\), going to zero at 0 and \(+ \infty \), and it has its peak around \num{.29}, where it reaches a value close to 1 (this means that the peak of synchrotron radiation is around \(\omega \sim \num{.29} \omega _c\)). Its decay is then exponential.
Asymptotically, we have that for \(z \ll 1\)
%
\begin{align}
F(z) \sim \frac{4}{\sqrt{3} \Gamma (1/3)} \qty(\frac{z}{2})^{1/3}
\,,
\end{align}
%
while for \(z \gg 1\):
%
\begin{align}
F(z) \sim \qty(\frac{\pi }{2})^{1/2} e^{-z} z^{1/2}
\,.
\end{align}

\subsubsection{Synchrotron emission from a nonthermal electron distribution}

In many astrphyiscal applications we need to consider the emission from electrons whose energies do not follow a thermal distribution but instead something like a powerlaw; the energy losses and gains caused by synchrotron emission actually cause the spectrum to get closer to a powerlaw, which gives further justification for considering this situation.

So, we will consider a population such that the number density of electrons per unit energy is given by
%
\begin{align}
\dv{N}{E} = C E^{-P}
\,.
\end{align}

Since the energy is given by \(E = mc^2 \gamma \), we can also write
%
\begin{align}
\dd{N} = C \gamma^{-P} \dd{\gamma }
\,.
\end{align}

Then, the emission per unit volume, frequency and time will be
%
\begin{align}
\frac{ \dd{w}}{ \dd{t} \dd{V} \dd{\omega }}
&= \int_{\gamma_1 }^{\gamma_2 } \frac{ \dd{N}}{ \dd{\gamma }} \frac{ \dd{w}}{ \dd{t} \dd{\omega }}  \dd{\gamma }  \\
&= \int_{\gamma_1 }^{\gamma_2 } c \gamma^{-P } c_1 F \qty(\frac{\omega }{\omega _c}) \dd{\gamma }
\,.
\end{align}

Remember that \(\omega_c\) depends on \(\gamma \), as \(\omega _c \propto \gamma^{2}\), so we cannot bring it out of the integral.
We can write this integral, keeping only the variable parts, as
%
\begin{align}
\frac{ \dd{w}}{ \dd{t} \dd{V} \dd{\omega }}
&\propto \int_{\gamma_1 }^{\gamma_2 } \gamma^{-P} F \qty(\frac{\omega }{\omega _c}) \dd{\gamma }  \\
&\propto \int_{x_1 }^{x_2 } \omega^{-P / 2} x^{P/2} \omega^{1/2} \qty(x)^{3/2} F \qty(x) \dd{ x}  \\
&= \int_{x_1 }^{x_2 } \omega^{-(P-1)/2} x^{(P-3) / 2} F \qty(x) \dd{x}   \\
&= \omega^{- (P-1 ) / 2} \int_{x_1 }^{x_2 } x^{(P-3) / 2} F(x) \dd{x}
\,,
\end{align}
%
\todo[inline]{in the second equality, should the integration bounds not change?}
where \(x = \omega / \omega _c\).
The integral bounds \(x_1 \) and \(x_2 \) will depend on \(\omega \) in general --- depending on what the regime of validity of the powerlaw --  but we will brutally approximate \(x_1 \sim 0 \) and \(x_2 \sim + \infty \).
In this case, the integral will not depend on \(\omega \), therefore
%
\begin{align}
\frac{ \dd{w}}{ \dd{t} \dd{V} \dd{\omega }}
\propto \omega^{- (P-1) / 2}
\,.
\end{align}

So, the spectrum of the synchrotron radiation emitted by an electron population whose energies are distributed according to a powerlaw will \textbf{also be a powerlaw}.
The new spectral index is given by \(S = (P-1) / 2\).

\end{document}


\input{../header_function}

%---------- start document ---------- %
 \section{bigrange -- range-like generator functions}\linkedzero{bigrange}
%
  \subsection{count -- count up}\linkedone{bigrange}{count}
   \func{count}
   {%
     \hikiopt{n}{integer}{0}
   }{%
     {\em iterator}
   }\\
   \spacing
   % document of basic document
   \quad Count up infinitely from \param{n} (default to \(0\)).
   See \linklibrary{itertools}.\linklibraryone{itertools\#count}{count}.\\
   \spacing
   % added document
   %\spacing
   % input, output document
   \quad \param{n} must be int, long or rational.Integer.\\

  \subsection{range -- range-like iterator}\linkedone{bigrange}{range}
   \func{range}
   {%
     \hiki{start}{integer},\ %
     \hikiopt{stop}{integer}{None},\ %
     \hikiopt{step}{integer}{1}
   }{%
     {\em iterator}
   }\\
   \spacing
   % document of basic document
   \quad Return a range-like iterator which generates a finite integer sequence.\\
   \spacing
   % added document
   \quad  It can generate more than
   \linklibrary{sys}.\linklibraryone{sys\#maxint}{maxint} elements,
   which is the limitation of the \linklibraryone{functions\#range}{range}
   built-in function.\\
   \spacing
   % input, output document
   \quad The argument names do not correspond to their roles, but
   users are familiar with the
   \linklibraryone{functions\#range}{range} built-in function of
   \python and understand the semantics.
   Note that the output is not a list.\\
%
\begin{ex}
>>> range(1, 100, 3) # built-in
[1, 4, 7, 10, 13, 16, 19, 22, 25, 28, 31, 34, 37, 40, 43, 46,
 49, 52, 55, 58, 61, 64, 67, 70, 73, 76, 79, 82, 85, 88, 91,
 94, 97]
>>> bigrange.range(1, 100, 3)
<generator object at 0x18f8c8>
\end{ex}%Don't indent!(indent causes an error.)

  \subsection{arithmetic\_progression -- arithmetic progression iterator}\linkedone{bigrange}{arithmetic\_progression}
   \func{arithmetic\_progression}
   {%
     \hiki{init}{integer},\ %
     \hiki{difference}{integer}
   }{%
     {\em iterator}
   }\\
   \spacing
   % document of basic document
   \quad Return an iterator which generates an arithmetic progression
   starting from \param{init} and \param{difference} step.\\

  \subsection{geometric\_progression -- geometric progression iterator}\linkedone{bigrange}{geometric\_progression}
   \func{geometric\_progression}
   {%
     \hiki{init}{integer},\ %
     \hiki{ratio}{integer}
   }{%
     {\em iterator}
   }\\
   \spacing
   % document of basic document
   \quad Return an iterator which generates a geometric progression
   starting from \param{init} and multiplying \param{ratio}.\\

  \subsection{multirange -- multiple range iterator}\linkedone{bigrange}{multirange}
   \func{multirange}
   {%
     \hiki{triples}{list of range triples}
   }{%
     {\em iterator}
   }\\
   \spacing
   % document of basic document
   \quad Return an iterator over Cartesian product of elements of ranges.\\
   \spacing
   % added document
   \quad Be cautious that using multirange usually means you are
   trying to do brute force looping.\\
   \spacing
   % input / output
   \quad The range triples may be doubles {\tt (start, stop)} or single {\tt (stop,)},
   but they have to be always tuples.\\

%
\begin{ex}
>>> bigrange.multirange([(1, 10, 3), (1, 10, 4)])
<generator object at 0x18f968>
>>> list(_)
[(1, 1), (1, 5), (1, 9), (4, 1), (4, 5), (4, 9), (7, 1),
 (7, 5), (7, 9)]
\end{ex}%Don't indent!(indent causes an error.)

  \subsection{multirange\_restrictions -- multiple range iterator with restrictions}\linkedone{bigrange}{multirange\_restrictions}
   \func{multirange\_restrictions}
   {%
     \hiki{triples}{list of range triples},\ %
     \hiki{**kwds}{keyword arguments}
   }{%
     {\em iterator}
   }\\
   \spacing
   % document of basic document
   \quad {\tt multirange\_restrictions} is an iterator similar to the
   {\tt multirange} but putting restrictions on each ranges.\\
   \spacing
   % added document
   \quad Restrictions are specified by keyword arguments: \param{ascending},
   \param{descending}, \param{strictly\_ascending} and
   \param{strictly\_descending}.

   A restriction \param{ascending}, for example, is a sequence that
   specifies the indices where the number emitted by the range should
   be greater than or equal to the number at the previous index.
   Other restrictions \param{descending}, \param{strictly\_ascending}
   and \param{strictly\_descending} are similar.  Compare the examples
   below and of \linkingone{bigrange}{multirange}.

%
\begin{ex}
>>> bigrange.multirange_restrictions([(1, 10, 3), (1, 10, 4)], ascending=(1,))
<generator object at 0x18f978>
>>> list(_)
[(1, 1), (1, 5), (1, 9), (4, 5), (4, 9), (7, 9)]
\end{ex}%Don't indent!(indent causes an error.)
\C
%---------- end document ---------- %

\input{../footer}


\subsection{The Open Mapping Theorem}
We want to study complex analytic functions $f:R\to\mathbb C$ where $R$ is a Riemann surface.
The case where $R$ is compact is especially interesting as we can greatly constrain them using the open mapping theorem.
\begin{theorem}\label{open_mapping}
    Any non-constant, analytic map of Riemann surfaces $f:R\to S$ is an open map.
\end{theorem}
\begin{proof}
    By the identity principle for Riemann surfaces shows that $f$ is not constant in any open subset.
    Let $W\subset R$ be open and $p\in W$.
    Pick charts $(\phi,U)$ containing $p$ and $(\psi,V)$ containing $f(p)$, then by the open mapping theorem on complex plane, $\psi\circ f(U\cap W\cap f^{-1}(V))$ is an open neighbourhood of $\psi\circ f(p)$ in $\psi(f(W)\cap V)$, so $f(U\cap W\cap f^{-1}(V))$ is an open neighbourhood of $f(p)$ in $f(W)$.
\end{proof}
\begin{corollary}
    Let $f:R\to S$ be a non-constant, analytic map of Riemann surfaces.
    If $R$ is compact, then $f$ is surjective and $S$ is also compact.
\end{corollary}
\begin{proof}
    By Theorem \ref{open_mapping}, $f(R)$ is open.
    But $R$ is compact, so $f(R)$ is compact, hence $f(R)$ is also closed as $S$ is Hausdorff.
    But then $S$ is path-connected hence connected, therefore $S=f(R)$ and hence is compact.
\end{proof}
\begin{corollary}
    Every analytic function on a compact Riemann surface is constant.
\end{corollary}
\begin{proof}
    $\mathbb C$ is not compact.
\end{proof}


%!TEX root = ../../template.tex
\section{Literature review}%
\label{sec:literature_review}

\subsection{Tomography}%
\label{sub:tomography}

Tomography is the cross-sectional imaging of an object through the use
of transmitted or reflected waves, captured by the object exposure to
the waves from a set of known angles. It has many different applications
in science, industry, and most prominently,
medicine~\cite{DeChiffre2014}. Since the invention of the \gls{CT}
machine in 1972, by Hounsfield~\cite{Gunderman2006}, tomographic imaging
techniques have had a revolutionary impact, allowing doctors to see
inside their patients, without having to subject them to more invasive
procedures~\cite{Kak2001}.

Mathematical basis for tomography were set by Johannes Radon in 1917. At
the time, he postulated that  it is possible to represent a function
written in $\mathbb{R}$ in the space of straight lines, $\mathbb{L}$
through the function's line integrals. A line integral is an integral in
which the function that is being integrated is evaluated along a curved
path, a line. In the tomographic case, these line integrals represent a
measurement on a ray that traverses the \gls{ROI}.  Each set of line
integrals, characterized by an incidence angle, is called a projection
(see Figure~\ref{fig:projection}). To perform a tomographic
reconstruction, the machine must take many projections around the
object. To the set of projections arranged in matrix form by detector
and projection angle, we call sinogram. All reconstruction methods,
analytical and iterative, revolve around going from reality to sinogram
to image~\cite{Bruyant2002, Kak2001, Herman1973, Herman1995, Herman2009,
Defrise2003}.

\begin{figure}[htpb]
    \centering
    \includegraphics[width=0.5\textwidth]{img/png/projections.png}
    \caption{A schematic representation of a projection acquisition. In
    this image, taken from ~\cite{Herman2009}, the clear line that comes
    down at a diagonal angle is a projection.}
    \label{fig:projection}
\end{figure}

There are two broad algorithm families when it comes to tomographic
reconstruction, regarding the physics of the problem. It can involve
either non-diffracting sources (light travels in straight lines), such
as the X-Rays in a conventional \gls{CT} exam; or diffracting sources,
such as micro-waves or ultrasound in more research-oriented
applications~\cite{Kak2001}. In this document, I will not address the
latter family, since I will not be applying them in my work.

In any tomographic procedure, the first step is to gather information
from the target object. The first concept one requires for this is to
determine the problem's geometry. There are many different possible
geometry, however, there are two that are more important for this
thesis: parallel and fan-beam geometries. In the parallel case, there
are as many light sources as there are detectors. Light travels between
the source and the detector in straight lines, and the whole set rotates
around the object's location. Fan-beam geometries are characterized by
having only one light source which rotates around the target object. In
this geometry, a set of detectors are placed on the other side of the
object, and the lines (rays) between the source and the detectors
describe a fan, thus the name of the technique~\cite{Herman2009,
Kak2001}.

As far as algorithms are concerned, there are two main types: analytical
and iterative. The first family includes the most famous algorithm for
these applications, the \gls{FBP}. Iterative algorithms work by
iteratively searching for a solution to the reconstruction equation,
which is basically an underdetermined system of equations (far less
equations than unknowns).  There are numerous algorithms that work in
this way, but in my work, I have identified three that are extensively
used, both in the field of atmospheric tomography and in medicine:
\gls{ART}, \gls{SART}, and \gls{MLEM}. The simulator that was developed
as part of this project uses the last two and \gls{FBP}. All these
techniques are further discussed and presented in
Chapter~\ref{cha:literature_review}.

\subsection{\gls{DOAS}}%
\label{sub:doas}

Since the beginning of the 20\textsuperscript{th} century, scientists
have been using spectroscopy to measure reactive trace gases in the
atmosphere, especially ozone. The basis for these applications were set
by Bouguer, Lambert and Beer, which have separately presented the law
(Lambert-Beer's) that determines the relationship between light
extinction and the concentration of an absorber, when it must traverse a
medium in which this absorber is present. \gls{DOAS} is one of the
methods that is applied for this purpose. It was developed in 1976, by
Perner and his colleagues~\cite{Perner1976}, to detect and quantify the
hydroxyl radical in the atmosphere. The book by Jochen Stütz and Ulrich
Platt~\cite{Platt2007} is considered by most researchers one of the most
important references in the field and is present in most bibliographies
of the literature in this subject (it is also one of the main references
in this thesis). Platt, in particular, has been working with the
technique since its beginning, as one of the elements of Perner's team
that published the article about the hydroxyl radical mentioned some
lines above~\cite{Perner1976}.

Besides \gls{DOAS}, Lambert-Beer's law is the basis of many quantitative
spectroscopy applications. However, most of these techniques are used in
a laboratory context, in which conditions are controlled and very well
known. Atmospheric studies do not have this luxury. In the open
atmosphere, there are a number of factors, like Mie and Rayleigh
scattering, atmospheric turbulence or thermal fluctuations in the
optical path that make outdoor spectral measurements more complicated.
\gls{DOAS} is able to circumvent these difficulties by measuring
differential absorptions, which is to say the difference in absorption
between two different wavelengths~\cite{Platt2007, Merlaud2013}.

There are two modalities for \gls{DOAS} experiments, Active and Passive,
which differ mainly on the use of artificial or natural light sources,
respectively. Both methods have their advantages and disadvantages.
Active systems are more similar to a bench spectroscopy experiment.
Conditions are more controlled (starting with the light source) and
therefore, results are usually more reliable and precise, not to mention
simpler to reach, since there is no need to account for complex physical
phenomena, like radiative transfer~\cite{Platt2007}. However, these
systems do require additional material, and many times entire
infrastructures have to be built around them~\cite{Pundt2005}. Passive
systems, on the other hand, can be comprised of just a computer, a
spectrometer and a telescope, making them instrumentally much simpler
than their active counterparts. This flexibility also comes with the
possibility to develop new interesting sub-techniques, like
\gls{maxdoas}, which allows (for instance) the determination of the
stratospheric contribution of a certain trace gas (in opposition to its
tropospheric contribution). The mathematical processing of the acquired
spectra, nonetheless, is much more complex.

\subsection{DOAS Tomography}%
\label{sub:doas_tomography}

\gls{DOAS} tomography is a relatively new subject within the realm of
\gls{DOAS}. It consists in the application of tomographic methods to
reconstruct a two-dimensional or three-dimensional \emph{map} of the
concentrations of trace gases in study. The seminal paper that
originated this and other remote sensing tomographic techniques was
published by Byer and Shepp in 1979~\cite{Byer1979}. It is not by any
means a very populated literary space. In a systematic review that was
performed as part of a course I took during the development of this PhD
thesis (see Section~\ref{sec:doas_tomography}), I managed to identify a
total of 13 papers that were clearly about \gls{DOAS} tomography. In
doing this review, I have also found that the largest \gls{DOAS}
tomography study was performed in Germany, during the first years of the
21\textsuperscript{st} century. This research campaign, called
BAB-II~\cite{Laepple2004}, aimed to measure and map traffic-related
concentrations for \gls{no2} in the motorway that goes between
Heidelberg and Mannheim. A more recent effort that is mention worthy is
the paper by the Stutz~\cite{Stutz2016}, in which the group created a
tomographic system that was able to perform as a fence  line monitor for
a refinery in Houston, Texas. In between the two studies, and right
after BAB-II, Erna Frins published her paper~\cite{Frins2006}, in which
she described the use of a \gls{maxdoas} system alternately pointed
towards sun-illuminated and dark targets in a tomographic manner.

Besides the birds eye view of the literary panorama of the \gls{DOAS}
tomography field, this systematic review allowed me to understand two
important gaps in the technology being employed for this research. The
first is that all of the studies that were featured in my review used a
very low number of tomographic projections (some dozens of line
integrals). This is a problem because it is the single most important
factor for the resolution of any tomographic procedure, and although
resolution is not an absolutely critical factor in atmospheric analysis
(because the sizes of the target objects - gas plumes - are very large
and diffuse), it is an important system feature that should be improved.
Moreover, the second important pattern that was clear from my research
was that all but one of the described systems were fixed, and the one
that was mobile was composed of a minimum of two spectral acquisition
devices and had one of the lowest numbers of projections in all papers.
This is a very important gap. \gls{DOAS} tomography has the ability not
only to measure but also to map pollutant concentrations and can be an
invaluable technique in the fight to understand and track the movement
of pollutant plumes, but the fact that the available systems require
dedicated infrastructure to operate and have no mobility at all may be
the most important factor leading to the almost non-existing investment
\gls{DOAS} tomography systems.

More than half of the world's population is expected to be living in
cities by the year 2050, and in the next 10 years we will see an
increase in the number of so-called mega cities of more than
30\%~\cite{CABI2019}. This puts a lot of pressure on governmental
agencies and municipalities (specially in the West) to "smarten up"
their urban infrastructures, so that cities can harbour their
inhabitants with reasonable quality of life for everyone. In fact, a
recent report by \gls{oecd} concluded that in 2010, estimated costs due
to \gls{AP} were around 1.7 trillion American dollars, just in
\gls{oecd} countries.  The flexibility and mobility that the system I am
developing brings to the table are two very heavy points in its favour
as a pollution mapping tool which is unobtainable with the traditional
methods, whether \emph{in-situ} or remote.