\iffalse
Noether's theorem says that continuous symmetries of physical systems gives rise to conservation laws. In this class we'll see some examples of low dimensional Lie groups and how they give rise to various phenomenon in physics like time dilation and length contraction in special relativity, spin states of electrons.

Keywords: bilinear forms, signature, SO(2), SO(3), Spin, SO(1,3), Minkowski space and relativity, Noether's theorem, Lie groups.

Prereqs: Linear algebra, Group theory
Homework: Recommended
\fi

\input{../preamble}

\begin{document}
\title{Rotations}
\author{Apurva Nakade}
\thispagestyle{fancy}
\maketitle

\emph{I did not have time to proof-read these notes, these are likely to have more errors than usual :-/}$\\\\$

Let's start by analyzing the orthogonal group in 2 dimensions $O(2)$.
\begin{align}
	O(2)
	&=
	\left\{ \begin{bmatrix} a & b \\ c & d \end{bmatrix} : \begin{bmatrix} a & b \\ c & d \end{bmatrix} \begin{bmatrix} a & c \\ b & d \end{bmatrix} = I_2\right\} \\
	SO(2)
	&=
	\left\{ \begin{bmatrix} a & b \\ c & d \end{bmatrix} : \begin{bmatrix} a & b \\ c & d \end{bmatrix} \begin{bmatrix} a & c \\ b & d \end{bmatrix} = I_2, ad-bc = 1 \right\}
\end{align}
By a direct computation we can show that every element of $O(2)$ is one of the two forms (Exercise \ref{thm:exO2})
\begin{align}
	\label{eq:O2}
	\begin{bmatrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{bmatrix}, & \begin{bmatrix} \cos \theta & \sin \theta \\ \sin \theta & -\cos \theta \end{bmatrix}
\end{align}
These matrices have determinants $1$ and $-1$ respectively and represent rotations and reflections in $\R^2$. The eigenvalues are of the form $e^{i\theta}, e^{-i\theta}$ for the rotation matrices and $\pm 1$ for the reflection ones and so we get

\begin{proposition}
	Every matrix in $O(2)$ is either a rotation, in which case it is similar to a matrix of the form $\begin{bmatrix} e^{i\theta} & \\ & e^{-i\theta} \end{bmatrix}$ or a reflection about a line, in which case it is similar to a matrix of the form $\begin{bmatrix} -1 & \\ & 1 \end{bmatrix}$.
\end{proposition}

\section{Orthogonal matrices}
This method does not generalize to higher dimensions (or does it?) instead we use eigenvalues to analyze the matrices.

\begin{thm}[Spectral theorem]
	Every matrix in $O(n)$ and $U(n)$ is diagonalizable over the complex numbers.
\end{thm}
Recall that diagonalizable means that the matrix is similar to a diagonal matrix i.e. it becomes diagonal after doing some base change. Even though $O(n)$ has real entries it's eigenvalues and eigenvectors might be complex i.e. the eigenvectors can be vectors in $\C^n$ instead of $\R^n$.

Because $O(n) \subseteq U(n)$ it suffices to analyze the eigenvectors of unitary matrices. Let $M \in U(n)$ be a unitary matrix. By the Spectral theorem there exist $n$ eigenvectors $v_1, \ldots, v_n \in \C^n$ with corresponding eigenvalues $\lambda_1, \ldots, \lambda_n$ i.e. $A v_i = \lambda_i v_i$. Using the definition of unitary matrices we must have
\begin{alignat}{4}
	               &   & \innerp{Av_i}{Av_i}                         & = \innerp{v_i}{v_i} \\
	\implies \quad &   & \innerp{\lambda_i v_i}{\lambda_i v_i}       & = \innerp{v_i}{v_i} \\
	\implies \quad &   & \conj \lambda_i \lambda_i \innerp{v_i}{v_i} & = \innerp{v_i}{v_i} \\
	\implies \quad &   & \conj \lambda_i \lambda_i                   & = 1
\end{alignat}
As $O(n)	\subseteq U(n)$ the same holds for $O(n)$ so we get the following proposition.

\begin{proposition}
	Every eigenvalue of an unitary or an orthogonal matrix is a complex number of norm 1 and hence is of the form $e^{i \theta}$ for some $\theta$.
\end{proposition}

\subsection{Orthogonal matrices in 3 dimensions}
Consider a matrix $A \in O(3)$, by the previous section $A$ has 3 eigenvalues of the form $\lambda_1 = e^{i \theta_1}$, $\lambda_2 = e^{i \theta_2}$, $\lambda_3 = e^{i \theta_3}$ for some $\theta_1$, $\theta_2$, $\theta_3$. But $O(3)$ has real entries and hence the complex eigenvalues of $A$ should come in conjugate pairs. The only way this can happen is if $\theta_1 = 0$ or $\pi$ and $\theta_2 = - \theta_3$.

\begin{proposition}
	For any $A \in O(3)$ the eigenvalues of $A$ are of the form $\lambda_1 = \pm 1 , \lambda_2 = e^{i \theta}, \lambda_3 = e^{-i \theta}$ for some $\theta$. Further $\lambda_1 = 1$ iff $A \in SO(3)$.
\end{proposition}

If $A \in SO(3)$ then $A$ is similar to
\begin{align}
	\label{eq:O3Type1}
	A \sim \begin{bmatrix} 1 &   &   \\ & e^{i \theta} & \\ & & e^{-i\theta} \end{bmatrix} \sim  \begin{bmatrix} 1 &   &   \\ &\cos \theta & -\sin \theta \\ &\sin \theta & \cos \theta \end{bmatrix}
\end{align}
This is saying that any matrix in $SO(3)$ represents rotation around an axis.

If $A \in O(3) \setminus SO(3)$ then $A$ is similar to
\begin{align}
	\label{eq:O3Type2}
	A \sim \begin{bmatrix} -1 &   &   \\ & e^{i \theta} & \\ & & e^{-i\theta} \end{bmatrix} \sim  \begin{bmatrix} -1 &   &   \\ &\cos \theta & -\sin \theta \\ &\sin \theta & \cos \theta \end{bmatrix}
\end{align}
This is saying that any matrix in $O(3) \setminus SO(3)$ represents rotation around an axis followed by a reflection along the perpendicular plane.

\begin{proposition}
	Every linear transformation of $\R^3$ that preserves distances is either a rotation about an axis or a rotation about an axis followed by a rotation about the perpendicular plane.
\end{proposition}

\iffalse
\section{$SU(2)$}
The arguments above prove that every unitary matrix is similar to a diagonal matrix with entries of the form $e^{i\theta}$. Since unitary matrices can have entries in complex numbers there are no conditions on the $\theta$'s.

Let us restrict to $n=2$ and consider the matrices in $SU(2)$

\begin{align}
	SU(2) = \{ A \in M_{2 \times 2}(\R) : A ^* A = I_2, \det A = 1\}
\end{align}

As we did for $O(2)$ we can explicitly write down the matrices in $SU(2)$. Let $\begin{bmatrix} a & b \\ c & d \end{bmatrix}$ be a matrix in $SU(2)$ then the conditions on $SU(2)$ imply that $d = \conj a$ and $c = -\conj b$ and $\norm{a}^2 + \norm{b}^2 = 1$ i.e.
\begin{align}
	SU(2) = \left\{ \begin{bmatrix} a & b \\ -\conj{b} & \conj{a} \end{bmatrix} : a, b \in \C \mbox{ and } \norm{a}^2 + \norm{b}^2 = 1 \right\}
\end{align}
Let $a = x_1 + i y_1$ and $b = x_2 + i y_2$. The the condition $\norm{a}^2 + \norm{b}^2 = 1$ is equivalent to $x_1^2 + y_1^2 + x_2^2 + y_2^2 = 1$, but this is exactly the equation of the sphere in $\R^3$.

\begin{thm}
	As a (topological) space $SU(2)$ is isomorphic to $S^3$.
\end{thm}
Note that this implies that every point in $S^3$ gives rise to a unitary transformation of $\C^2$.
\fi

\section{Quaternions}
There is another way to talk about rotations, using quaternions! Recall that \textbf{quaternions} form a non-abelian group, denoted $\mathbb{H}$, that is isomorphic as a set to $\R^4$. Elements of $\mathbb{H}$ are of the form $a + bi + cj + dk$ and satisfy the relations
\begin{align}
	i^2 = j^2 = k^2 = -1, ij = k, jk = i, ki = j
\end{align}

\iffalse

Similar to complex numbers we have conjugation and norm on quaternions given by
\begin{align}
	\norm{a + bi + cj + dk}^2 & = a^2 + b^2 + c^2 + d^2 \\
	\conj{a + bi + cj + dk}   & = a - bi - cj - dk
\end{align}
and we also have the identity
\begin{align}
	(a + bi + cj + dk).\conj{(a + bi + cj + dk)} = \norm{a + bi + cj + dk}^2
\end{align}
In particular note that if $p$ has norm 1 then $p^-1 = \conj p$.
\fi

A quaternion $p \in \mathbb{H}$ defines a linear transformation $\Phi(p):\mathbb{H} \rightarrow \mathbb{H}$ that sends $v \mapsto p v p^{-1}$. These transformation turn out to be rotations when restricted to the unit quaternion group!

Let $S\mathbb{H}$ denote the group of unit quaternions i.e. $\{ p \in \mathbb{H} : \norm{p} = 1\}$. We think of $\R^3$ as the set of \emph{purely imaginary} quaternions i.e. the vector $(x,y,z)$ represents the quaternion $xi + yj + zk$. It turns out to be the case that when $p \in S \mathbb{H}$ the transformation $v \mapsto p v p^{-1}$ preserves the set of purely imaginary quaternions. In fact a much stronger result holds.

\begin{thm}
	\label{thm:quaternions}
	The map sending $p\in S\mathbb{H}$ to $\Phi(p)$ defines a homomorphism
	\begin{align}
		\Phi : S \mathbb{H} \rightarrow SO(3)
	\end{align}
	This homomorphism is surjective with kernel $\Z/2$.
\end{thm}
The proof of this has several steps and is in Exercises in  \ref{sec:exQuaternions}.

The group $S\mathbb{H}$ shows up in several avatars in various branches of mathematics. It is the spin group in 3 dimensions, denoted $Spin(3)$. Because $SO(3)$ is the group of rotation of $\R^3$ the above theorem is asserting that there are two quaternions over each rotation of $\R^3$. In physics this fact becomes relevant because in quantum mechanics certain systems have $S\mathbb{H}$ as their symmetry groups and for such systems there are is a physical quantity, called \textbf{spin} which has two possible values for each value of the angular moment.

\iffalse
\begin{proof}
	Suppose $\norm{p} = 1$ for some $p \in \mathbb{H}$. We need to show that for $\alpha, \beta \in \R^3$ we have $\innerp{p\alpha}{p\beta} = \innerp{\alpha}{\beta}$. We need a good way to manipulate inner products. Let $\Re(a+bi+cj+dk) = a$ denote the real part of quaternions. Then it is easy to see that $\innerp{\alpha}{\beta} = \Re(\alpha \beta)$. We're reduced to showing thatwhen $\norm{p} = 1 $ we have
	\begin{align}
		\label{eq:random}
		\Re(p \alpha p ^{-1}.p \beta p ^{-1}) = \Re{(\alpha\beta)}
	\end{align}

	Now we invoke the notion of conjugate quaternions. Similar to complex numbers we have conjugation and norm on quaternions given by
	\begin{align}
		\conj{a + bi + cj + dk}   & = a - bi - cj - dk      \\
		\norm{a + bi + cj + dk}^2 & = a^2 + b^2 + c^2 + d^2
	\end{align}
	and we also have the identity
	\begin{align}
		(a + bi + cj + dk).\conj{(a + bi + cj + dk)} = \norm{a + bi + cj + dk}^2
	\end{align}
	In particular note that if $p$ has norm 1 then $p^{-1} = \conj p$. Plugging this back in \eqref{eq:random} gives us the desired result.

	The kernel of this homomorphism is exactly the quaternions $p$ such that for all $v$ we have
	\begin{align}
		p v p ^{-1} = v
	\end{align}
	The only such $p$'s are the purely \emph{real} quaternions i.e. the ones with no $i,j,k$ components. The only such quaternions of norm 1 are $p = \pm 1$ and so the kernel of $\Phi$ is $\Z/2$.
\end{proof}
\fi

\newpage
\section{Exercise}
\begin{exercise}
	\label{thm:exO2}
	Consider a matrix $A = \begin{bmatrix} a & b \\ c & d \end{bmatrix} \in O(2)$.
	\begin{enumerate}
		\item Show that for some $\theta$, $\phi$ we must have $a = \cos \theta$, $b = \sin \theta$, and $c = \cos \phi$, $d = \sin \phi$.
		\item Find the relations between $\theta$ and $\phi$ and prove that every matrix in $O(2)$ is of the form \eqref{eq:O2}.
		\item Describe the matrices in \eqref{eq:O2} geometrically and compute their eigenvalues.
	\end{enumerate}
\end{exercise}

\iffalse
\begin{exercise}
	\label{thm:exO(n)}
	What is wrong with the following proof?
	Suppose $A \in O(n)$ and let $v$ be an eigenvector of $A$ with eigenvalue $\lambda$ then
	\begin{alignat}{4}
		         &   & \innerp{Av}{Av}               & = \innerp{v}{v} \\
		\implies &   & \innerp{\lambda v}{\lambda v} & = \innerp{v}{v} \\
		\implies &   & \lambda^2 \innerp{v}{v}       & = \innerp{v}{v} \\
		\implies &   & \lambda^2                     & =  1
	\end{alignat}
	Hence every eigenvalue of $A$ is $\pm 1$.
\end{exercise}
\fi

\begin{exercise}
	Show that the matrices
	\begin{align}
		\begin{bmatrix} 1  &   &   \\ &\cos \theta & \sin \theta \\ &\sin \theta & -\cos \theta \end{bmatrix}
		&&
		\begin{bmatrix} -1 &   &   \\ &\cos \theta & \sin \theta \\ &\sin \theta & -\cos \theta \end{bmatrix}
	\end{align}
	are in $O(3)$. What do these geometrically represent? Find the matrices of type \eqref{eq:O3Type1} or \eqref{eq:O3Type2} to which these are similar.
\end{exercise}

\begin{exercise}
	Describe the matrices in $SO(n)$ geometrically for arbitrary positive integer $n$. Do these matrices still represent rotations? What is the difference between matrices in $SO(2n)$ and matrices in $SO(2n+1)$.
\end{exercise}

\subsection{Quaternions}
\label{sec:exQuaternions}
The following exercises prove theorem \ref{thm:quaternions}.

\begin{exercise}
	The first step is to figure out how to deal with inner products using quaternions. Let $\Re(a + bi + cj + dk) = a$ denote the real part of quaternions.
	\begin{enumerate}
		\item Show that for two vectors $x, y \in \R^3$ the dot product $\innerp{x}{y}$ is equal to $-\Re(xy)$.
		\item Show that for any quaternion $p \conj{p} = \norm{p}^2$ and hence if $p \in S\mathbb{H}$ then $p^{-1} = \conj{p}$.
		\item Show that for $p \in S \mathbb{H}$ and $v \in \mathbb{H}$ we have $ \Re{(x)} = \Re{(p x p^{-1}) }$. This implies in particular that $\Phi(p)$ takes the purely imaginary quaternions to purely imaginary quaternions.
		\item Show that for $p \in S \mathbb{H}$ and $x, y \in \R^3$ we have $ \innerp{x}{y} = \innerp{px\conj{p}}{px\conj{p}}$.
	\end{enumerate}
\end{exercise}

\begin{exercise}
	Let $p \in S \mathbb{H}$ be a unit quaternion. The above exercise proves that the transformation $\Phi(p)$ preserves the dot product.
	\begin{enumerate}
		\item Show that for $q \in \mathbb{H}$ we have $\Phi(pq) = \Phi(p)\Phi(q)$ and hence we have a group homomorphism $\Phi: S\mathbb{H}\rightarrow SO(3)$.
		      (It is $SO(3)$ and not $SO(4)$ because we're looking at the transformations of the space of purely imaginary quaternions.)
		\item Argue that because $S\mathbb{H}$ is connected the image of $\Phi$ should be a subgroup of $SO(3)$ and hence $\Phi$ is a homomorphism $S\mathbb{H} \rightarrow SO(3)$.
		\item Show that for the unit quaternion $p = \cos (\theta/2) + \sin (\theta/2)(x i+yj+zk)$ the transformation $\Phi(p)$ fixes the vector $(x,y,z)$. Use this to argue that $\Phi$ is surjective.
		\item Show that the center of $S\mathbb{H}$ is the set of purely real quaternions. Argue that the kernel of $\Phi$ is $\Z/2$.
	\end{enumerate}
\end{exercise}

\begin{exercise}
	\begin{align}
		O_8 = \{ \pm 1, \pm i, \pm j, \pm k \}
	\end{align}
	Let $O_8 \subseteq \mathbb{H}$ be the finite quaternion group. Describe the image of $O_8$ under the homomorphism $\Phi$ (defined in Section \ref{sec:quaternions}).
\end{exercise}

\iffalse
\begin{exercise}
	Show that $O(n)$, $SO(n)$, $U(n)$ and $SU(n)$ are compact subsets of $M_{n \times n}(\R \mbox{ or } \C)$, but $GL_n(\R)$, $SL_n(\R)$, $GL_n(\C)$ and $SL_n(\C)$ are not.
\end{exercise}

\begin{exercise}
	Find the center of the groups $GL_n(\R)$ and $SL_n(\C)$.
\end{exercise}

\begin{exercise}
	A \textbf{maximal torus} of a matrix group $G$ is a maximal abelian subgroup of $G$ i.e. a subgroup $H \subseteq G$ is called a maximal torus if $H$ is abelian and if an abelian subgroup $H' \subseteq G$ contains $H$ then $H=H'$.

	Find a maximal torus of each of the following groups: $U(n)$, $SU(n)$, $SO(2n)$ and $SO(2n+1)$.

\end{exercise}
\fi

\end{document}

03 Symmetries of Spaces/02 Rotations.tex


\chapter{Abstract}

{
% for aesthetic reasons, this page will not have indented paragraphs
\setlength{\parskip}{1ex plus 0.5ex minus 0.2ex}
\setlength{\parindent}{0pt}

In 1968, Milnor asked if a finitely-generated group could have volume growth that is neither exponential nor polynomial (so-called `intermediate'), and if there is an algebraic classification of groups with polynomial volume growth.
We consider the analogous questions for geodesic growth.

We show that no virtually abelian group can have intermediate geodesic growth.
In particular, we completely characterise the geodesic growth for every virtually abelian group.
We show that the geodesic growth is either polynomial of an integer degree with rational geodesic growth series, or exponential with holonomic geodesic growth series.
In addition, we show that the language of geodesics is blind multicounter.
These results hold for each finite weighted monoid generating set of any virtually abelian group.

A direct consequence of Gromov's classification of polynomial volume growth is that if a group has polynomial geodesic growth with respect to some finite generating set, then it is virtually nilpotent.
Until now, the only known examples with polynomial geodesic growth were all virtually abelian.
We furnish the first example of a virtually 2-step nilpotent group having polynomial geodesic growth with respect to a certain finite generating set.

Holt and R\"over proved that finitely-generated bounded automata groups have indexed co-word problems.
We sharpen this result to show that their co-word problem is ET0L.
We do so using an equivalent machine model known as a cspd automaton.
This extends a result of \citeauthor{ciobanu2018} who showed this for the first Grigorchuk group by explicitly constructing an ET0L grammar.

}


%auto-ignore
\providecommand{\MainFolder}{..}
\documentclass[\MainFolder/Text.tex]{subfiles}

\begin{document}

\section{Construction of a Hodge propagator for spheres}
\allowdisplaybreaks
\label{Sec:GreenSphere}

The standard Riemannian volume form on the round sphere $\Sph{n}\subset \R^{n+1}$ is the restriction of the following closed form on $\R^{n+1}\backslash \{0\}$:
\[ \Vol(x) \coloneqq \frac{1}{|x|^{n+1}}\sum_{i=1}^{n+1} (-1)^{i+1} x^i \Diff x_1 \dotsm \widehat{\Diff x_i} \dotsm \Diff x_{n+1}. \]
Here $\widehat{\Diff x_i}$ means that $\Diff{x_i}$ is omitted.
We denote the Riemannian volume of $\Sph{n}$ by
\[ V\coloneqq \int_{\Sph{n}} \Vol. \]
The $n$-form $\HKer$ from Proposition~\ref{Lemma:HKer} reads
\begin{equation*}
\HKer =  \frac{1}{V}\bigl(\Pr_1^*\Vol +  (-1)^{n} \Pr_2^*\Vol\bigr).
\end{equation*}
According to Proposition~\ref{Prop:GKer}, the equation which we want to solve reads
\begin{equation} \label{Eq:GreenKernel}
\Dd \Prpg = \frac{1}{V}\bigl((-1)^{n}\Pr_1^*\Vol +  \Pr_2^*\Vol\bigr).
\end{equation}
We denote
\[\tilde{\Prpg}\coloneqq V\Prpg\quad\text{and}\quad \tilde{\HKer}\coloneqq V\HKer.\]

The following lemma will be used to construct a solution to~\eqref{Eq:GreenKernel}.
\begin{Lem}[Relative Poincar\'e Lemma]\label{Lem:ChainHtpy}
Let $M$ be a smooth oriented manifold and $\psi: [0,1]\times M \rightarrow M$ a smooth map. Consider the operator $T: \DR^*(M) \rightarrow \DR^{*-1}(M)$ defined by
\[ T(\eta) \coloneqq \FInt{[0,1]} \psi^*\eta\quad\text{for all }\eta\in \DR(M), \]
where we integrate along the fiber of the oriented fiber bundle $\Pr_2: [0,1]\times M \rightarrow M$. Then we have
\[ \Dd\circ T + T\circ \Dd = \psi_1^* - \psi_0^*. \]
\end{Lem}
%
\begin{proof}
Stokes' formula from Proposition~\ref{Prop:StokesForm} gives
\[ \Dd \FInt{[0,1]}\psi^*\eta = -\Bigl( \FInt{[0,1]} \Dd \psi^* \eta - \FInt{\Bdd [0,1]} \psi^* \eta \Bigr) = - \FInt{[0,1]} \psi^* \Dd \eta + \psi^*_1 \eta - \psi^*_0 \eta\]
for all $\eta \in \DR(M)$.
\end{proof}

\begin{Proposition}[Solution to \eqref{Eq:GreenKernel}] \label{Prop:GKerSph}
For all $(x,y)\in (\Sph{n}\times \Sph{n}) \backslash \Diag$, let
\begin{equation} \label{Eq:GreenKernelMC1}
\Prpg (x,y) \coloneqq (-1)^{n} \sum_{k=0}^{n-1} g_k(x,y)\omega_k(x,y),
\end{equation}
where
\begin{equation}\label{Eq:FunctionsGk1}
 g_k(x,y) \coloneqq \int_{0}^1 \frac{t^k(t-1)^{n-1-k}}{(2t(t-1)(1+x\cdot y) + 1)^{\frac{n+1}{2}}} \Diff{t}
\end{equation}
and
\begin{equation} \label{Eq:FormOmega1}
\omega_k(x,y) \coloneqq \begin{multlined}[t]\frac{1}{k!} \frac{1}{(n-1-k)!} \sum_{\sigma \in \Perm_{n+1}} (-1)^\sigma x^{\sigma_1} y^{\sigma_2} \Diff{x^{\sigma_3}} \dotsm \Diff{x^{\sigma_{2+k}}} \\ \Diff{y^{\sigma_{3+k}}} \dotsm \Diff{y^{\sigma_{n+1}}}.\end{multlined}
\end{equation}
The form~\eqref{Eq:GreenKernelMC1} is a smoooth solution to~\eqref{Eq:GreenKernel} on $(\Sph{n}\times \Sph{n}) \backslash \Diag$.
\end{Proposition}
%\Red{Can $\omega$ be written as a product in the sense that in $n=1$ is $\omega_{00}=x\cdot Ry$.?}
%
\begin{proof} %Checked on 19.9.17
Define the set
\[ N \coloneqq (\R^{n+1}_{\neq 0}\times\R^{n+1}_{\neq 0})\backslash \{(x,a x) \mid x\in \R^{n+1}, a>0\}. \]
It is an open thickening of $(\Sph{n}\times \Sph{n})\backslash \Diag$ in $\R^{n+1} \times \R^{n+1}\backslash \Diag $. Consider the smooth deformation retraction
\begin{align*}
 \psi : [0,1] \times N & \longrightarrow N \\
 (t,x,y) &\longmapsto \psi_t(x,y) \coloneqq (x,(1-t)y-tx)
\end{align*}
with
\[ \psi_0(x,y)=(x,y) \quad \text{and}\quad \psi_1(x,y)= (x,-x)\quad\text{for all }(x,y)\in N. \]
The retraction is depicted in Figure~\ref{Fig:Retraction}.
\begin{figure}
\centering
%\includegraphics[width=.3\textwidth,trim=5.1cm 20.5cm 7.6cm 1.3cm,clip]
\input{\GraphicsFolder/contraction.tex}
\caption[Retraction of the configuration space $C_2(\Sph{n})$ to $\Sph{n}$.]{Retraction $\psi_t = (\psi_t^1, \psi_t^2)$. A point of $\Sph{n}\times \Sph{n}$ is visualized as a pair of points on $\Sph{n}$.}\label{Fig:Retraction}
\end{figure}
Denote by $A: \R^{n+1} \rightarrow \R^{n+1}$, $x\mapsto -x$ the antipodal map. It is easy to see that
\[ A^* \Vol = (-1)^{n+1} \Vol, \]
and hence
\begin{equation*}
\psi_1^* \tilde{\HKer} = \psi_1^* \Pr_1^*\Vol + (-1)^n \psi_1^*\Pr_2^* \Vol  = \Pr_1^*\Vol + (-1)^n \Pr_1^*A^* \Vol = 0.
\end{equation*}
Define
\begin{equation}  \label{Eq:GExpr}
\Prpg \coloneqq (-1)^{n+1} \FInt{[0,1]} \psi^*\HKer.
\end{equation}
Let $T: \DR^*(N)\rightarrow \DR^{*-1}(N)$ be the cochain homotopy from Lemma~\ref{Lem:ChainHtpy} associated to $\psi$. Because $\Dd \HKer=0$, we get
\[ \Dd \Prpg = (-1)^{n+1} \Dd T(\HKer) = (-1)^{n+1}(\Dd T + T \Dd)\HKer = (-1)^{n+1}(\psi_1^*-\psi_0^*)\HKer = (-1)^{n}\HKer. \]
For every $i=1$,~$\dotsc$, $n+1$, we have
\[ \psi^*(\Diff{x^i}) = \Diff{x^i}\quad\text{and}\quad\psi^*(\Diff{y^i}) = (1-t)\Diff{y^i} - t\Diff{x^i} - (y^i+x^i)\Diff{t}. \]
We compute
%
\begin{align*}
 &  \FInt{[0,1]} \psi^* \tilde{\HKer}\\
 & \quad = (-1)^{n}\FInt{[0,1]} \psi^* \Pr_2^* \Vol \\
 & \quad = (-1)^{n+1}\FInt{[0,1]} \sum_{i=1}^{n+1} (-1)^{i}\frac{((1-t)y^i - t x^i)}{{\Abs{(1-t)y-tx}^{n+1}}} \psi^*(\Diff{y^1} \dotsm \widehat{\Diff{y^i}} \dotsm \Diff{y^{n+1}}) \\
 & \quad = (-1)^{n+1} \sum_{1\le i<j \le n+1} (-1)^{i+j}(x^i y^j - y^i x^j) \FInt{[0,1]}  \frac{\Diff{t} \psi^*(\Diff{y^1} \dotsm \widehat{\Diff{y^i}} \dotsm \widehat{\Diff{y^j}} \dotsm \Diff{y^{n+1}})}{\Abs{(1-t)y-tx}^{n+1}} \\
 & \quad = \begin{multlined}[t] - \sum_{k=0}^{n-1} \Bigl(\int_0^1 \frac{t^k(t-1)^{n-1-k}}{\Abs{(1-t)y-tx}^{n+1}} \Diff{t}\Bigr) \sum_{1\le i<j\le n+1} (-1)^{i+j+1} (x^i y^j - y^i x^j) \\
 \sum_{\mathclap{\substack{\sigma: \{1,\dotsc,n-1\}\rightarrow \{1,\dotsc,\hat{i},\dotsc,\hat{j},\dotsc,n+1\} \\ \sigma_1<\dots < \sigma_k \\ \sigma_{k+1}<\dots < \sigma_{n-1}}}} (-1)^\sigma \Diff{x^{\sigma_1}} \dotsm \Diff{x^{\sigma_k}}\Diff{y^{\sigma_{k+1}}} \dotsm \Diff{y^{\sigma_{n-1}}}.\end{multlined}
\end{align*}
The formulas~\eqref{Eq:FunctionsGk1} and~\eqref{Eq:FormOmega1} are obtained from this by writing
\[ \Abs{(1-t)y - tx}^2 = 2t(t-1)(1+x \cdot y) + 1 \]
in the denominator of the integrand and by simple combinatorics in the form part, respectively. Smoothness of~$\Prpg$ on $(\Sph{n}\times \Sph{n})\backslash \Diag$ follows from the expression~\eqref{Eq:GExpr}.
\end{proof}
Note that $g_k$ are smooth functions on $(\Sph{n}\times \Sph{n})\backslash \Diag$.
%
\begin{Example}[$\Prpg$ for $\Sph{1}$ and $\Sph{2}$]\phantomsection\label{Example:Circle}% On 15.9.17
\begin{ExampleList}
\item Let
\[\alpha: (\Sph{1}\times\Sph{1})\backslash \Diag \rightarrow (0,2\pi)\]
be the smooth function assigning to a pair $(x,y)\in  (\Sph{1}\times\Sph{1})\backslash \Diag$ the counterclockwise angle from $x$ to $y$. Let $\alpha_1$, $\alpha_2 \in [0,2\pi)$ be such that $x=\cos(\alpha_1)\StdBasis_1 + \sin(\alpha_1)\StdBasis_2$ and $y=\cos(\alpha_2)\StdBasis_1+\sin(\alpha_2)\StdBasis_2$ for the standard Euclidean basis $\StdBasis_1$, $\StdBasis_2$ of $\R^2$. It is easy to see that
\[ \alpha(x,y) = \begin{cases} \alpha_2 - \alpha_1 & \text{if }\alpha_1<\alpha_2, \\
\alpha_2-\alpha_1+2\pi & \text{if }\alpha_1>\alpha_2. \end{cases} \]
Therefore, we get
\[ \Diff{\alpha} = \Diff{\alpha_2} -\Diff{\alpha_1} = -2\pi H \quad\text{on } (\Sph{1}\times\Sph{1})\backslash \Diag. \]
On the other hand, we can compute $\Prpg$ from~\eqref{Eq:GreenKernelMC1} as follows. Using the substitution $u=2t-1$, we get for all $x$, $y\in \Sph{1}$ with $x\neq \pm y$ the following:
%
\begin{align*}
 g_{0}(x,y) &= \int_0^1 \frac{\Diff{t}}{2t(t-1)(1+x\cdot y) +1} \\
 & =  \frac{1}{1-x\cdot y}\int_{-1}^1 \frac{\Diff{u}}{\frac{1+x\cdot y}{1- x\cdot y}u^2 + 1} \\
 & = \frac{2}{\sqrt{1-(x\cdot y)^2}}\arctan\Bigl(\sqrt{\frac{1+x\cdot y}{1-x\cdot y}}\Bigr) \\
 & = \frac{\pi - \arccos(x\cdot y)}{\sqrt{1-(x\cdot y)^2}} \\
 & = \frac{\pi - \arccos(x\cdot y)}{ \Abs{x^1 y^2 - x^2 y^1}} \\
 & = \frac{\pi - \alpha(x,y)}{x^1 y^2 - x^2 y^1}.
\end{align*}
The third from last equality can be obtained by trigonometric considerations and the second from last equality by an algebraic manipulation with the denominator. We will explain the last equality. Consider the matrix
\[ R=\begin{pmatrix}
0 & -1 \\ 1 & 0
\end{pmatrix} \]
representing the counterclockwise rotation by $\frac{\pi}{2}$. The function $\arccos: (-1,1) \rightarrow (0,\pi)$ satisfies
\[ \arccos(x\cdot y) = \begin{cases}
                        \alpha(x,y) & \text{if }y\cdot Rx > 0, \\
                        2\pi - \alpha(x,y) & \text{if }y\cdot Rx<0.
                       \end{cases}\]
The last equality becomes clear when we notice that $x^1 y^2 - x^2 y^1 = y\cdot Rx$.

Finally, we have $\omega_{0}(x,y) = x^1 y^2 - x^2 y^1$, and hence
\begin{align*}
2\pi \Prpg(x,y) &=  - g_{0}(x,y) \omega_{0}(x,y) \\
& = \alpha(x,y) - \pi \\
& = \pi - \alpha(y,x).
\end{align*}
\item For $n=2$, we get the formulas
\allowdisplaybreaks
\begin{align*}
g_0(x,y) & = - g_1(x,y) = \frac{1}{x\cdot y - 1}\quad\text{and} \\
\omega_0(x,y) &=(x^2 y^3 - x^3 y^2) \Diff{y^1} +(x^3 y^1 - x^1 y^3) \Diff{y^2} + (x^1 y^2 - x^2 y^1) \Diff{y^3} \\ &=\sum_{i=1}^3 (x\times y)^i \Diff{y^i}.
\end{align*}
The formula for $\omega_1(x,y)$ is obtained from the formula for $\omega_0(x,y)$ by replacing~$\Diff{y}$ with~$\Diff{x}$.\qedhere
\end{ExampleList}
\end{Example}

Consider the diagonal action of the orthogonal group $O(n+1)$ on $\R^{n+1}\times\R^{n+1}$ by matrix multiplication.
%\[ R(x,y)\coloneqq (Rx,Ry)\quad \text{for all }(x,y)\in \R^{n+1}\times\R^{n+1}, R\in O(n+1), \]
%where $Rx$ denotes the standard action on $\R^{n+1}$.
\begin{Proposition}[Symmetries of $\Prpg$]\label{Prop:SymmetryOfG}
Consider $\Prpg$ from Proposition~\ref{Prop:GKerSph}. For all $R\in O(n+1)$, we have
\[ R^* \Prpg = (-1)^R \Prpg, \]
where $(-1)^R = \det(R)$. Moreover, if $\tau$ denotes the twist map, then
\[\tau^*\Prpg = (-1)^{n}\Prpg. \]
\end{Proposition}
%
\begin{proof} % Check on 13.9.17
We will use the thickening $N$, the antipodal map $A$ and the expression~\eqref{Eq:GExpr} for~$\Prpg$ from the proof of Proposition~\ref{Prop:GKerSph}.

It is easy to check that both $\tau$ and $R$ preserve $N$. Let~$\tilde{\tau}$ and $\tilde{R}$ be the isomorphisms of the fiber bundle $\Pr_2: [0,1]\times N \rightarrow N$ given by
\[ \tilde{\tau}(t,x,y) \coloneqq (1-t,y,x)\quad \text{and}\quad \tilde{R}(t,x,y) \coloneqq (t,Rx,Ry) \]
for all $(t,x,y)\in [0,1]\times N$. Then $\tilde{\tau}$ covers $\tau$ and $\tilde{R}$ covers $R$. A simple computation directly from Definition~\ref{Def:FibInt} shows that the fiberwise integration commutes with the pullback along a bundle morphism if the bundle map and the base map are both either orientation preserving or reversing. In our case, we have
\[ (-1)^{\tau + \tilde{\tau}} = -1\quad \text{and}\quad (-1)^{R+\tilde{R}} = 1. \]
Using this and the equation
\[ \Pr_2 \circ \psi \circ \tilde{\tau} = A\circ \Pr_2 \circ \psi, \]
we get firstly
\allowdisplaybreaks
\begin{align*}
\tau^* \FInt{[0,1]} \psi^*\tilde{\HKer} &=  - \FInt{[0,1]} \tilde{\tau}^* \psi^*\Pr_2^*\Vol \\ &= - \FInt{[0,1]} \psi^*\Pr_2^*A^* \Vol \\ &= (-1)^n \FInt{[0,1]} \psi^*\Pr_2^*\Vol \\ & = (-1)^n \FInt{[0,1]} \psi^*\tilde{\HKer}
\end{align*}
and secondly
\[ R^* \FInt{[0,1]} \psi^* \HKer = \FInt{[0,1]} \tilde{R}^* \psi^* \HKer = \FInt{[0,1]} \psi^* R^*\HKer = (-1)^{n+1} \FInt{[0,1]} \psi^* \HKer. \]
This proves the proposition.
\end{proof}
Both diffeomorphisms $R$ and $\tau$ preserve $\Delta$, and hence they extend to diffeomorphisms of $\Bl_\Diag(\Sph{n}\times\Sph{n})$. If also $\Prpg$ extends, then the statement of Proposition~\ref{Prop:SymmetryOfG} holds for $\Prpg$ on $\Bl_\Diag(\Sph{n}\times\Sph{n})$.

In the rest of the section, we will be proving that $\Prpg$ extends smoothly to $\Bl_{\Diag}(\Sph{n}\times \Sph{n})$. This is a local problem at the boundary, where we introduce the following radial coordinates. Define the set
\[ X \coloneqq \{(r,\eta,x)\in [0,\infty)\times \Sph{n}\times\Sph{n} \mid \eta\cdot x = 0\}, \]
and let $\kappa: X \longrightarrow \Bl_\Diag(\Sph{n}\times \Sph{n})$ be the map defined by
\begin{align*}
\kappa(r,\eta,x) &\coloneqq \begin{cases}
\Bigl(x,\dfrac{x+r\eta}{\Abs{x+r\eta}}\Bigr)\in (\Sph{n}\times \Sph{n})\backslash \Diag & \text{for }r>0, \\[2ex]
[(-\eta,\eta)]\in P^+ N_{(x,x)}\Diag & \text{for }r=0.
\end{cases}
\end{align*}
Recall that the oriented projectivization $P^+$ was defined in Definition~\ref{Def:SphBlow}.
For the upcoming computations, it is convention to define the map $\gamma: \R \rightarrow (-1,1)$ by
\[ \gamma(r) \coloneqq \frac{r}{\sqrt{1+r^2}+1} \quad\text{for all }r\in \R. \]
It is a diffeomorphism with inverse $r = \frac{2 \gamma}{1-\gamma^2}$.

\begin{Lem}[Parametrization of collar neighborhood] \label{Lem:NewBlowupParam}
The subset $X\subset \R\times \R^{n+1}\times\R^{n+1}$ is a submanifold with boundary, and the map $\kappa: X \longrightarrow \Bl_\Diag(\Sph{n}\times \Sph{n})$ is an embedding onto a neighborhood of $\Bdd \Bl_\Diag(\Sph{n}\times \Sph{n})$.
\end{Lem}
%
\begin{proof}
The set $X$ is a Cartesian product of $[0,\infty)$ and a regular level set; therefore, it is a submanifold with boundary. The inclusion $\Sph{n}\times \Sph{n} \subset \R^{n+1}\times \R^{n+1}$ induces an embedding of manifolds with boundary $\Bl_\Diag(\Sph{n}\times\Sph{n})\subset \Bl_\Diag(\R^{n+1}\times \R^{n+1})$.  Consider the global chart $\tilde{\Id}: \Bl_\Diag(\R^{n+1}\times \R^{n+1}) \rightarrow [0,\infty) \times \Sph{n} \times \R^{n+1}$ from~\eqref{Eq:BlowUpChart} induced by the identity. We have
\[ \begin{aligned}Y &\coloneqq \tilde{\Id}(\Bl_\Diag(\Sph{n}\times \Sph{n})) \\ &= \{(\tilde{r},w,u)\in [0,\infty) \times \Sph{n} \times \R^{n+1} \mid \Abs{u}^2+\tilde{r}^2=1,\ w\cdot u = 0\}, \end{aligned}\]
where we denote $r$ on $Y$ by $\tilde{r}$ in order to distinguish it from $r$ on $X$. It suffices to prove the claim for the map $\mu\coloneqq \tilde{\Id}\circ\kappa: X \rightarrow Y$. For $(r,\eta,x)\in X$, we compute
\[ \mu(r,\eta,x) = \biggl( \frac{\gamma}{\sqrt{1+\gamma^2}}, \frac{1}{\sqrt{1+\gamma^2}}(\gamma x - \eta), \frac{1}{1+\gamma^2}(x+\gamma \eta) \biggr). \]
This formula defines a smooth map of $\R\times \R^{n+1}\times\R^{n+1}$.
%Let $\eta, x \in \Sph{n}$ be such that $\eta\cdot x = 0$ and let $r\in \R$. Equations
%\[ x = u + R \omega \quad\text{and}\quad \frac{x+r\eta}{\Abs{x+r\eta}} = u-R\omega \]
%are solved for
%\begin{equation} \label{Eq:Extension}
%R = \frac{\gamma}{\sqrt{1+\gamma^2}}, \quad
% \omega = \frac{1}{\sqrt{1+\gamma^2}}(\gamma x - \eta), \quad u  = \frac{1}{1+\gamma^2}(x+\gamma \eta).
% \end{equation}
%We used $\Abs{x+r\eta} = \sqrt{1+r^2} = \frac{1+\gamma^2}{1-\gamma^2}$.
%The solution satisfies $\Abs{\omega}=1$, $\Abs{u}^2 + R^2 = 1$ and $\omega \cdot u = 0$ which implies $(R,\omega,u)\in Y$. Therefore, it defines a smooth extension of $\mu$
%\[ \begin{aligned} \tilde{\mu}: \R\times \R^{n+1} \times \R^{n+1} &\longrightarrow \R\times \R^{n+1}\times \R^{n+1} \\
%(r,\eta,x) &\longmapsto (R,\omega,u). \end{aligned} \]
It is a local diffeomorphism because its Jacobian is non-vanishing:
\[ |\Jac{\mu}| = \frac{\partial \tilde{r}}{\partial r} \Bigl(\frac{\partial w}{\partial \eta}\frac{\partial u}{\partial x}  - \frac{\partial w}{\partial x}\frac{\partial u}{\partial \eta}\Bigr)^{n+1} = (-1)^{n+1}(1+\gamma^2)^{-\frac{n+4}{2}} \frac{\partial \gamma}{\partial r}. \]
Moreover, the map $\mu$ is injective, maps $X$ into $Y$ and $\Bdd X$ onto $\Bdd Y$. The claim follows.
\end{proof}
%
Consider the action of $O(n+1)$ on $X$ defined by
\[ R\cdot(r,\eta,x) \coloneqq (r,R\eta,Rx)\quad\text{for all }(r,\eta,x)\in X\text{ and }R\in O(n+1). \]
Via $\kappa$, this agrees with the diagonal action of $O(n+1)$ on $\Bl_\Diag(\Sph{n}\times\Sph{n})$. Denote
\[ \Prpg'\coloneqq \kappa^* \Prpg \in \Omega^{n-1}(\Int(X)). \]
From Proposition~\ref{Prop:SymmetryOfG} we get
\begin{equation} \label{Eq:SymmetryOfGPrime}
R^* \Prpg' = (-1)^R \Prpg'\quad \text{for all }R\in O(n+1).
\end{equation}
%
Consider the smooth curve (see Figure~\ref{Fig:CurveOnSphere})
\begin{equation*}\label{Eq:CurveZetaDef}
 \begin{aligned} \zeta': [0,\infty) &\longrightarrow  X \\
                    r &\longmapsto (r,e_n,e_{n+1}). \end{aligned}
\end{equation*}
We have the following lemma.
%
\begin{figure}[t]\centering
\input{\GraphicsFolder/zeta.tex}
\caption[A curve approaching the diagonal in the configuration space $C_2(\Sph{n})$.]{The curve $\zeta\coloneqq \kappa \circ \zeta'$ is given by $\zeta(r)=\bigl(e_{n+1},\frac{e_{n+1}+r e_n}{\Abs{e_{n+1}+r e_n}}\bigr)$ for $r>0$.}\label{Fig:CurveOnSphere}
\end{figure}
%
\begin{Lem}[Smooth extension along curve] \label{Lem:ExtAlongCurve}
The form $\Prpg'$ extends smoothly to $X$ if and only if the map $\Prpg'\circ \zeta' : (0,\infty)\rightarrow \Lambda^{n-1}T^* X$ extends smoothly to the interval $[0,\infty)$.
\end{Lem}
%
\begin{proof}
As for the non-trivial implication, let $(0,\eta_0,x_0) \in X$ be a boundary point. Pick vectors $v_1$,~$\dotsc$, $v_{n-1}\in \R^{n+1}$ so that the vectors $v_1$,~$\dotsc$, $v_{n-1}$, $\eta_0$, $x_0$ are linearly independent, and define the set
\[ U\coloneqq\{(r,\eta,x)\in X \mid v_1,\,\dotsc,\,v_{n-1},\,\eta,\,x \text{ are linearly independent}\}. \]
It is an open neighborhood of $(0,\eta_0,x_0)$ in $X$. Applying the Gram-Schmidt orthogonalization to $v_1$,~$\dotsc$, $v_{n-1}$, $\eta$, $x$, we find a smooth map $R: U \rightarrow O(n+1)$ such that
\[ R(r,\eta,x)\cdot (r,\eta,x) = (r,e_n,e_{n+1}) \quad \text{for all }(r,\eta,x)\in U. \]
The equation~\eqref{Eq:SymmetryOfGPrime} implies
\[ \Prpg'(r,\eta,x) =(-1)^R R(r,\eta,x)^*\bigl(\Prpg'(r,e_n,e_{n+1})\bigr)\quad\text{for all }(r,\eta,x)\in \Int(U), \]
where $R(r,\eta,x)^*: \Lambda^* T^* X \rightarrow \Lambda^* T^* X$ is the smooth cotangential map which is induced by the diffeomorphism $R(r,\eta,x): X\rightarrow X$, and which maps the fiber over $z\in X$ to the fiber over $R(r,\eta,x)^{-1} z$. By the assumption, all maps in the composition are smooth in their arguments. The lemma follows.
\end{proof}

\begin{Lem}[Local expression at boundary] \label{Lem:FormulaAlongCurve}
On the interval $(0,\infty)$, we have
\begin{equation*}\label{Eq:FormulaAlongCurve}
\tilde{\Prpg}'\circ\zeta' = (-1)^{n+1}(1+\gamma^2)^{-\frac{n-1}{2}} \sum_{k=0}^{n-1} \gamma^{n-k} (h_{k}\circ\gamma)(\nu_{k}\circ\zeta'),
\end{equation*}
where the functions $h_{k}: (0,1)\rightarrow \R$ are defined by
\begin{equation*}
  h_{k}(\gamma)\coloneqq \int_{-1}^1 \frac{(u+\gamma^2)^{k}(u-1)^{n-1-k}}{(u^2+\gamma^2)^{\frac{n+1}{2}}} \Diff{u} \quad \text{for all }\gamma\in (0,1)
\end{equation*}
and the forms $\nu_{k}\in \Omega(X)$ are defined by
\begin{align*}
 \nu_{k}(r,x,\eta) &\coloneqq \frac{1}{k!(n-1-k)!}\sum_{\sigma\in \Perm_{n-1}} (-1)^\sigma \Diff{x^{\sigma_1}}\dotsm
\Diff{x^{\sigma_k}} \Diff{\eta^{\sigma_{k+1}}} \dotsm \Diff{\eta^{\sigma_{n-1}}}.
\end{align*}
\end{Lem}

\begin{proof}
We start with the following formula from the proof of Proposition~\ref{Prop:GKerSph}:
\[ \tilde{\Prpg} =  \sum_{1\le i<j \le n+1} (-1)^{i+j}(x^i y^j - y^i x^j) \FInt{[0,1]}  \frac{\Diff{t} \psi^*(\Diff{y^1} \dotsm \widehat{\Diff{y^i}} \dotsm \widehat{\Diff{y^j}} \dotsm \Diff{y^{n+1}})}{\Abs{(1-t)y-tx}^{n+1}}. \]
We restrict to the points $(x,y)=\kappa(r,e_n,e_{n+1})$ with $r>0$. There, we have
\begin{align*}
&x^1= \dotsb =x^n=0,\ x^{n+1}=1, \\
&y^1 = \dotsb = y^{n-1} =0,\ y^n= \frac{2\gamma}{1+\gamma^2},\ y^{n+1} = \frac{1-\gamma^2}{1+\gamma^2}.
% &\eta^1 = \dotsb = \eta^{n-1}=\eta^{n+1}=0,\ \eta^{n}=1.
% &\Diff{x^{n+1}} = 0,\ \Diff{\eta^n}=0,\\Diff{\eta^{n+1}}=-\Diff{x^n}.
\end{align*}
Under the substitution $u = 2t-1$, we get
\[\Abs{(1-t)y-t x}^2 = \frac{4 t (t-1)}{1+\gamma^2}+1=\frac{u^2+\gamma^2}{1+\gamma^2}. \]
We make the following preliminary computations:
\[ \begin{aligned}
x^i y^j - y^i x^j & = 0\quad\text{for }1\le i \le n-1\text{ and }i<j\le n+1, \\
x^n y^{n+1} - y^n x^{n+1} & = -\frac{2\gamma}{1+\gamma^2}, \\
\kappa^*(\Diff{y^i}) & = \frac{1}{1+\gamma^2}\bigl((1-\gamma^2)\Diff{x^i}+ 2\gamma \Diff{\eta^i}\bigr)\quad\text{for } 1\le i \le n-1. \end{aligned} \]
We plug these in the formula for $\tilde{\Prpg}$ and get
% \Red{//There might by $\pm$ issue when exchanging $\varphi^*$ and $\int^{[0,1]}$ as $\varphi$ is not orientation preserving in every dimension.// There is no issue since we extend $\varphi$ from the base space $X$ to $X\times [0,1]$ and the extension preserves the total space orientation if and only if the base map preserves total space orientation}
%
\begin{align*}
 \tilde{\Prpg}'(\zeta'(r)) &= 2\gamma(1+\gamma^2)^{\frac{n-1}{2}} \FInt{[0,1]}\Diff{t}\frac{\prod_{i=1}^{n-1} \bigl((1-t)\kappa^*(\Diff{y^i}) - t\Diff{x^i}\bigr)}{(u^2+\gamma^2)^{\frac{n+1}{2}}}  \\
&=(-1)^{n+1}\gamma(1+\gamma^2)^{-\frac{n-1}{2}}\FInt{[-1,1]} \Diff{u} \frac{\prod_{i=1}^{n-1}\bigl((u+\gamma^2)\Diff{x^i} + \gamma(u-1)\Diff{\eta^i}\bigr)}{(u^2+\gamma^2)^{\frac{n+1}{2}}}  \\
& =(-1)^{n+1}(1+\gamma^2)^{-\frac{n-1}{2}}\sum_{k=0}^{n-1} \gamma^{n-k}\Bigl( \int_{-1}^1 \frac{(u+\gamma^2)^{k}(u-1)^{n-1-k}}{(u^2+\gamma^2)^{\frac{n+1}{2}}} \Diff{u}\Bigr) \nu_{k}.
\end{align*}
The lemma follows.
% Checked on 28.9.
\end{proof}
%
%Because the overall coefficient $(1+\gamma^2)^{-\frac{n-1}{2}}$ is smooth and non-zero on $[0,\infty)$, and the forms $\nu_{k,n-1-k}$, $k=0,\ldots,n-1$ are linearly independent smooth forms on $X$, the map $G\circ \gamma$ extends smoothly to $[0,\infty)$ if and only if the functions $\gamma^{n-k}h_{k,n-1-k}$ for all $k=0,\ldots,n-1$ do.
%
\begin{Lem}[Integrals depending on parameter] \label{Lem:GeneralIntegralExtension}
Let $n\in \N$, and let $l\in\{ 0, 1,~\dotsc, n-1\}$. The function $F_{n,l}: (0,\infty)\rightarrow \R$ defined by
\begin{equation}\label{Eq:GeneralIntegral}
F_{n,l}(t) \coloneqq \int_{-1}^1 \frac{t^{n-l} u^l}{(u^2+t^2)^{\frac{n+1}{2}}} \Diff{u}\quad\text{for all }t\in(0,\infty)
\end{equation}
extends smoothly to $[0,\infty)$.
\end{Lem}
\begin{proof}
We have
\[ F_{1,0}(t) = 2 \arctan\Bigl(\frac{1}{t}\Bigr)=\pi - 2 \arctan(t) \quad\text{for all }t\in (0,\infty). \]
The right-hand side is a smooth function on $\R$.

For $n\ge 2$, we deduce the recursive formula
\[ F_{n,0}(t) = \frac{1}{n-1}\Bigl((n-2)F_{n-2,0}(t)+\frac{2 t^{n-2}}{(1+t^2)^{\frac{n-1}{2}}}\Bigr).\]
%We see that $F_{n,0}$ extends smoothly to $\R$ for all $n$ by induction.
%by defining
%\[ F_{n,0}(0)\coloneqq \frac{(n-2)!!}{(n-1)!!}\begin{cases} \pi & \text{for }n \text{ odd},\\ 2 & \text{for }n \text{ even}.
%\end{cases}\]
If $l$ is odd, then $F_{n,l}\equiv 0$ for all $n$ because the integrand of \eqref{Eq:GeneralIntegral} is odd as a function of $u$.

For $n\ge 3$ and even $2\le l \le n-1$, we deduce yet another recursive formula
\[  F_{n,l}(t) = \frac{1}{n-l}\Bigl((l-1) F_{n,l-2}(t) -  \frac{2 t^{n-l}}{(1+t^2)^{\frac{n-1}{2}}} \Bigr).
\]
%by defining
%\[ F_{n,l}(0)\coloneqq \begin{cases} \frac{(l-1)!!(n-l-2)!!}{(n-2)!!} F_{n,0} & \text{for }l\text{ even} \\ 0 & \text{for }l\text{ odd}\end{cases} \]
The claim for all $F_{n,l}$ follows by induction.
\end{proof}

\begin{Proposition}[Smooth extension to boundary]\label{Prop:GKerBdd}
The form $\Prpg$ from~\eqref{Eq:GreenKernelMC1} extends smoothly to $\Bl_\Diag(\Sph{n}\times\Sph{n})$.
\end{Proposition}
%
\begin{proof}
According to Lemmas~\ref{Lem:NewBlowupParam} and~\ref{Lem:ExtAlongCurve}, it suffices to show that the curve $\Prpg'\circ \zeta': (0,\infty)\rightarrow \Lambda^{n-1}T^* X$ extends smoothly to $[0,\infty)$. Lemma~\ref{Lem:FormulaAlongCurve} gives an expression for $\Prpg'\circ \zeta'$ as a linear combination of smooth forms $\nu_{k}\in \Omega^{n-1}(X)$ with coefficients $\gamma^{n-k}(h_{k}\circ \gamma)$ for $k=0$,~$\dotsc$, $n-1$ multiplied by the overall coefficient $(-1)^n (1+\gamma^2)^{-\frac{n-1}{2}}$. We expand
\begin{equation*}
\gamma^{n-k}(h_{k}\circ\gamma) = \sum_{a=0}^k\sum_{b=0}^{n-1-k}(-1)^{n-1-k-b} \binom{k}{a} \binom{n-1-k}{b} \int_{-1}^1 \frac{\gamma^{n+k-2a}u^{a+b}}{(u^2+\gamma^2)^{\frac{n+1}{2}}} \Diff{u}
\end{equation*}
and notice that we can write
\begin{equation*}
 \int_{-1}^1 \frac{\gamma^{n+k-2a}u^{a+b}}{(u^2+\gamma^2)^{\frac{n+1}{2}}} \Diff{u} = \gamma^{k-a+b}(F_{n,a+b}\circ \gamma)
\end{equation*}
for the function $F_{n,l}$ from~\eqref{Eq:GeneralIntegral} with $l\coloneqq a+b$. Because $0\le l \le n-1$, Lemma~\ref{Lem:GeneralIntegralExtension} asserts that $F_{n,l}$ extends smoothly to $[0,\infty)$. Because $k-a+b\ge 0$, the entire coefficient at $\nu_{k}$ extends smoothly to $[0,\infty)$ for every $k=0$,~$\dotsc$, $n-1$. The lemma follows.
\end{proof}

%\begin{Example}
%\[
%\Prpg'(\zeta'(r)) = \begin{cases}
% \dfrac{1}{2\pi}\bigl(\pi - \arctan(r)\bigr) & \text{for }n=1, \\[2ex]
%\dfrac{1}{4\pi}\Bigl(\Bigl(1+\dfrac{1}{\sqrt{1+r^2}}\Bigr) \Diff{\eta^1} - \dfrac{r}{\sqrt{1+r^2}} \Diff{x^1}\Bigr)& \text{for }n=2, \\[2ex]
% \begin{gathered}[b]\frac{1}{2\pi^2}\Bigl(\frac{1}{2}\Bigl(\pi - \arctan(r)+\frac{r}{1+r^2}\Bigr)\Diff{\eta^1}\Diff{\eta^2} - \frac{r^2}{2(1+r^2)}\SplitEq (\Diff{x^1}\Diff{\eta^2} -\Diff{x^2}\Diff{\eta^1}) + \Bigl(\pi - \arctan(r) - \frac{r}{2(1+r^2)}\Bigr)\Diff{x^1}\Diff{x^2}\Bigr) \end{gathered} & \text{for }n=3.
%\end{cases}
%\]
%%We see that the right hand side indeed smoothly extends $G'\circ \zeta'$ to $[0,\infty)$.
%\end{Example}

We summarize our results in the following proposition:

\begin{Proposition}[Hodge propagator for $\Sph{n}$]\label{Proposition:GreenKernel}
The form $\Prpg$ from~\eqref{Eq:GreenKernelMC1} defines a Hodge propagator for $\Sph{n}$ satisfying Definition~\ref{Def:GreenKernel}. Moreover, we have the symmetries
\begin{align*}
R^* \Prpg &= (-1)^R \Prpg\quad \text{for all }R\in O(n+1)\text{ and} \\
\tau^* \Prpg & = (-1)^n \Prpg.
\end{align*}
\end{Proposition}
\begin{proof}
The proposition is a summary of Propositions~\ref{Prop:GKerSph},~\ref{Prop:SymmetryOfG} and~\ref{Prop:GKerBdd}.
\end{proof}
%\begin{proof}
%The proposition is proved in Section~\ref{Section:Proof1}: Lemma~\ref{Lemma:GKer} transforms~\eqref{Eq:CochainHomotopy} into the differential equation~\eqref{Eq:GreenKernel} for $\Prpg$, Lemma~\ref{Prop:GKerSph} provides a solution, in Lemma~\ref{Prop:GKerBdd} we show that the solution extends smoothly to the blow-up, and finally in Lemma~\ref{Prop:SymmetryOfG} we show that $\Prpg$ satisfies the symmetry property~\eqref{Eq:SymProp}.
%\end{proof}

\begin{Remark}[Better notation due to R. Bryant, see  \cite{MO291535}] \label{Remark:Bryant}
Pick an oriented basis $e_1$,~$ \dotsc$, $e_{n+1}$ of $\R^{n+1}$ as generators of the exterior algebra $\Lambda^*(\R^{n+1})$, and view $x$, $y$, $\Diff{x}$, $\Diff{y}$ as $\Lambda^*(\R^{n+1})$-valued forms on~$\R^{n+1}$. For example, we view $x$ as the map $x\in \R^{n+1} \mapsto \sum_{i=1}^{n+1} x^i e_i \in \Lambda^1(\R^{n+1})$ and $\Diff{x}$ as the map $x\in \R^{n+1} \mapsto \sum_{i=1}^{n+1} (\Diff{x}_i)_x e_i \in \Lambda^1(\R^{n+1})$. There is a natural wedge product on the space of $\Lambda^*(\R^{n+1})$-valued forms. If $\omega$ is a top-form, we denote by $[\omega]$ the coefficient of $\omega$ at $e_1 \wedge \dotsm \wedge e_{n+1}$. Then it holds
\[ \omega_k(x,y) = \frac{1}{k!}\frac{1}{(n-1-k)!}[x\wedge y \wedge (\Diff{x})^{k} \wedge (\Diff{y})^{n-1-k}]. \]

Note that if we view $e_i$ as odd variables, then $[\cdot]$ corresponds to the odd integration~$\int \mathrm{D}e(\cdot)$. It would be interesting to know whether this notation simplifies some proofs, especially if Lemma~\ref{Lemma:ABVanishing} can be deduced from abstract algebraic facts or rules valid for odd integration.
%and
%\[ \Vol(x) = \frac{1}{n!} [x \wedge (\Diff{x})^n].\]
\end{Remark}
\end{document}


\chapter{Poisson denoising}
\label{ch_denoising}

\markright{Poisson denoising}

\section{MS-VST + IUWT}

Under the hypothesis of homogeneous Poisson intensity, the stabilized wavelet coefficients $d_j$ behave like centered Gaussian variables of standard deviation $\sigma_{(j)}$. We can detect significant coefficients with binary hypothesis testing as in Gaussian denoising.

Under the null hypothesis $\mathcal{H}_0$ of homogeneous Poisson intensity, the distribution of the stabilized wavelet coefficient $d_j[k]$ at scale $j$ and location index $k$ can be written as:
\begin{equation}
\label{ }
p(d_j[k]) = \frac{1}{\sqrt{2\pi}\sigma_j}\exp(-d_j[k]^2 / 2 \sigma_j^2) .
\end{equation}

The rejection of the hypothesis $\mathcal{H}_0$ depends on the double-sided p-value:
\begin{equation}
\label{ }
p_j[k] = 2 \frac{1}{\sqrt{2\pi}\sigma_j}\int_{|d_j[k]|}^{+\infty} \exp(-x^2 / 2 \sigma_j^2) dx .
\end{equation}

Consequently, to accept or reject $\mathcal{H}_0$, we compare each $|d_j[k]|$ with a critical threshold $\kappa \sigma_j$, $\kappa= 3,4 \text{ or } 5$ corresponding respectively to significance levels. This amounts to deciding that:
\begin{itemize}
  \item if $|d_j[k]| \geqslant  \kappa \sigma_j$, $d_j[k]$ is significant.
  \item if $|d_j[k]| < \kappa \sigma_j$, $d_j[k]$ is not significant.
\end{itemize}

Then we have to invert the MS-VSTS scheme to reconstruct the estimate. However, although the direct inversion is possible (Eq. (\ref{eq30})), it can not guarantee a positive intensity estimate, while the Poisson intensity is always nonnegative. A positivity projection can be applied, but important structures could be lost in the estimate. To tackle this problem, we reformulate the reconstruction as a convex optimisation problem and solve it iteratively with an algorithm based on Hybrid Steepest Descent (HSD)~\citep{wave:yamada01}.

We define the multiresolution support $\mathcal{M}$, which is determined by the set of detected significant coefficients after hypothesis testing:
\begin{equation}
\label{eq33}
\mathcal{M} := \{ (j,k) | \text{if } d_j[k] \text{ is declared significant} \} .
\end{equation}

We formulate the reconstruction problem as a convex constrained minimization problem:

\begin{equation}
\label{eq34}
\begin{split}
\text{Arg} \min_{\mathbf{X}} \| \mathbf{ \Phi}^{T}\mathbf{X}\|_1,
\text{s.t.} \\ \: \left\{\begin{array}{c}\mathbf{X} \geqslant 0 , \\\forall (j,k)\in \mathcal{M},      (\mathbf{ \Phi}^{T}\mathbf{X})_j[k]=(\mathbf{ \Phi}^{T}\mathbf{Y})_j[k] , \end{array}\right.
\end{split}
\end{equation}
where $\mathbf{\Phi}$ denotes the IUWT synthesis operator.

This problem is solved with the following iterative scheme: the image is initialised by $\mathbf{X}^{(0)} = 0$, and the iteration scheme is, for $n=0$ to $N_{\max}-1$:

\begin{eqnarray}
\tilde{\mathbf{X}} &=& P_{+}[\mathbf{ X}^{(n)} + \mathbf{ \Phi} P_{\mathcal{M}} \mathbf{ \Phi}^{T} (\mathbf{ Y} - \mathbf{ X}^{(n)})] \\
\mathbf{X}^{(n+1)} &=& \mathbf{ \Phi}\text{ST}_{\lambda_n}[\mathbf{ \Phi}^{T}\tilde{\mathbf{X}}]
\end{eqnarray}
where $P_{+}$ denotes the projection on the positive orthant, $P_{\mathcal{M}}$ denotes the projection on the multiresolution support $\mathcal{M}$:
\begin{equation}
P_{\mathcal{M}}d_j[k] = \left\{\begin{array}{cc} d_j[k] & \text{if} \  (j,k) \in \mathcal{M} , \\0 & \text{otherwise} \end{array} . \right.
\end{equation}
and $\text{ST}_{\lambda_n}$ the soft-thresholding with threshold $\lambda_n$:
\begin{equation}
\text{ST}_{\lambda_n} [d] = \left\{\begin{array}{cc} \mathrm{sign}(d)(|d| - \lambda_n) & \text{if} \ |d| \geqslant \lambda_n , \\0 & \text{otherwise} \end{array} . \right.
\end{equation}
We chose a decreasing threshold $\lambda_n = \frac{N_{\max} - n}{N_{\max} - 1},n=1,2,\cdots,N_{\max}$.

The final estimate of the Poisson intensity is: $\hat{\mathbf{\Lambda}} = \mathbf{X}^{(N_{\max})}$. Algorithm~\ref{alg1} summarizes the main steps of the MS-VSTS + IUWT denoising algorithm.

\begin{algorithm}[!h]
\caption{MS-VSTS + IUWT Denoising}
\label{alg1}
\begin{algorithmic}[1]
\REQUIRE $\quad$ data $a_0:=\mathbf{Y}$, number of iterations $N_{\max}$, threshold $\kappa$ \\
\underline{\emph{\textbf{Detection}}} \\
\FOR{$j=1$ to $J$}
\STATE Compute $a_j$ and $d_j$ using (\ref{eq27}).
\STATE Hard threshold $|d_j[k]|$ with threshold $\kappa \sigma_j$ and update $\mathcal{M}$.
\ENDFOR \\
\underline{\emph{\textbf{Estimation}}} \\
\STATE Initialize $\mathbf{X}^{(0)}=0$, $\lambda_0 = 1$.
\FOR{$n=0$ to $N_{\max}-1$}
\STATE $\tilde{\mathbf{X}}= P_{+}[\mathbf{ X}^{(n)} + \mathbf{ \Phi} P_{\mathcal{M}} \mathbf{ \Phi}^{T} (\mathbf{ Y} - \mathbf{ X}^{(n)})]$.
\STATE $\mathbf{X}^{(n+1)} = \mathbf{ \Phi}\text{ST}_{\lambda_n}[\mathbf{ \Phi}^{T}\tilde{\mathbf{X}}]$.
\STATE $\lambda_{n+1} = \frac{N_{\max} - (n+1)}{N_{\max} - 1}$.
\ENDFOR
\STATE Get the estimate $\hat{\mathbf{\Lambda}} = \mathbf{X}^{(N_{\max})}$.

\end{algorithmic}
\end{algorithm}

\section{Multi-resolution support adaptation}

When two sources are too close, the less intense source may not be detected because of the negative wavelet coefficients of the brightest source. To avoid such a drawback, we may update the multi-resolution support at each iteration. The idea is to withdraw the detected sources and to make a detection on the remaining residual, so as to detect the sources which may have been missed at the first detection.

At each iteration $n$, we compute the MS-VSTS of $\mathbf{X}^{(n)}$. We denote $d^{(n)}_j[k]$ the stabilised coefficients of $\mathbf{X}^{(n)}$. We make a hard thresholding on $(d_j[k]-d^{(n)}_j[k])$ with the same thresholds as in the detection step. Significant coefficients are added to the multiresolution support $\mathcal{M}$.

\begin{algorithm}
\caption{MS-VSTS + IUWT Denoising + Multiresolution Support Adaptation}
\label{alg4}
\begin{algorithmic}[1]
\REQUIRE $\quad$ data $a_0:=\mathbf{Y}$, number of iterations $N_{\max}$, threshold $\kappa$ \\
\underline{\emph{\textbf{Detection}}} \\
\FOR{$j=1$ to $J$}
\STATE Compute $a_j$ and $d_j$ using (\ref{eq27}).
\STATE Hard threshold $|d_j[k]|$ with threshold $\kappa \sigma_j$ and update $\mathcal{M}$.
\ENDFOR \\
\underline{\emph{\textbf{Estimation}}} \\
\STATE Initialize $\mathbf{X}^{(0)}=0$, $\lambda_0 = 1$.
\FOR{$n=0$ to $N_{\max}-1$}
\STATE $\tilde{\mathbf{X}}= P_{+}[\mathbf{ X}^{(n)} + \mathbf{ \Phi} P_{\mathcal{M}} \mathbf{ \Phi}^{T} (\mathbf{ Y} - \mathbf{ X}^{(n)})]$.
\STATE $\mathbf{X}^{(n+1)} = \mathbf{ \Phi}\text{ST}_{\lambda_n}[\mathbf{ \Phi}^{T}\tilde{\mathbf{X}}]$.
\STATE Compute the MS-VSTS on  $\mathbf{X}^{(n)}$ to get the stabilised coeffcients $d^{(n)}_j$.
\STATE Hard threshold $|d_j[k]-d^{(n)}_j[k]|$ and update $\mathcal{M}$.
\STATE $\lambda_{n+1} = \frac{N_{\max} - (n+1)}{N_{\max} - 1}$.
\ENDFOR
\STATE Get the estimate $\hat{\mathbf{\Lambda}} = \mathbf{X}^{(N_{\max})}$.

\end{algorithmic}
\end{algorithm}

The main steps of the algorithm are summarized in Algorithm~\ref{alg4}. In practice, we use Algorithm~\ref{alg4} instead of Algorithm~\ref{alg1} in our experiments.

\section{MS-VST + Curvelets}

Insignificant coefficients are zeroed by using the same hypothesis testing framework as in the wavelet scale. At each wavelet scale $j$ and ridgelet band $k$, we make a hard thresholding on curvelet coefficients with threshold $\kappa \sigma_{j,k}$, $\kappa= 3,4 \text{ or } 5$. Finally, a direct reconstruction can be performed by first inverting the local ridgelet transforms and then inverting the MS-VST + IUWT~(Equation~(\ref{eq30})). An iterative reconstruction may also be performed.

Algorithm~\ref{algcurv} summarizes the  main steps of the MS-VSTS + Curvelets denoising algorithm.

\begin{algorithm}
\caption{MS-VSTS + Curvelets Denoising}
\label{algcurv}
\begin{algorithmic}[1]
\STATE Apply the MS-VST + IUWT with $J$ scales to get the stabilized wavelet subbands $d_j$.
\STATE Set $B_1 = B_{\min}$.
\FOR{$j=1$ to $J$}
\STATE Partition the subband $d_j$ with blocks of side-length $B_j$ and apply the digital ridgelet transform to each block to obtain the stabilized curvelets coefficients.
\IF {$j$ modulo $2=1$}
\STATE $B_{j+1} = 2 B_j$
\ELSE
\STATE $B_{j+1} =  B_j$
\ENDIF \\
\STATE HTs on the stabilized curvelet coefficients.
\ENDFOR \\
\STATE Invert the ridgelet transform in each block before inverting the MS-VST + IUWT.

\end{algorithmic}
\end{algorithm}

\section{Experiments}

The method was tested on simulated Fermi data. The simulated data are the sum of a Milky Way diffuse background model and 1000 gamma ray point sources. We based our Galactic diffuse emission model intensity on the model $gll\_iem\_v02$ obtained at the Fermi Science Support Center~\citep{Models}
. This model results from a fit of the LAT photons with various gas templates as well as inverse Compton in several energy bands. We used a realistic point-spread function for the sources, based on Monte Carlo simulations of the LAT and accelerator tests, that scale approximately as $0.8(E/1GeV)^{-0.8}$ degrees. The position of the 205 brightest sources were taken from the Fermi 3-month source list~\citep{Abdo}. The position of the 795 remaining sources follow the LAT 1-year Point Source Catalog~\citep{Catalog}
  sources distribution: each simulated source was randomly sorted in a box of $\Delta$l=5$^o$ and $\Delta$b=1$^o$ around a LAT 1-year catalog source. We simulated each source assuming a power-law dependence with its spectral index given by the 3-month source list and the first year catalog. We used an exposure of $3.10^{10} s.cm^2$ corresponding approximatively to one year of Fermi all-sky survey around 1 GeV. The simulated counts map shown here correspond to photons energy from 150 MeV to 20 GeV.

Fig.~\ref{rechsd} compares the result of denoising with MS-VST + IUWT (Algorithm~\ref{alg1}), MS-VST + curvelets (Algorithm~\ref{algcurv}) and Anscombe VST + wavelet shrinkage on a simulated Fermi map. Fig.~\ref{recface} shows one HEALPix face of the results.
As expected from theory, the Anscombe method produces poor results to  denoise Fermi data, because the underlyning intensity is too weak.
Both wavelet and curvelet denoising on the sphere  perform much better.
For this application, wavelets are slightly better than curvelets ($SNR_{wavelets} = 65.8 dB$, $SNR_{curvelets} = 37.3 dB$, $SNR (dB) = 20 \log (\sigma_{signal} / \sigma_{noise})$). As this image contains many point sources, thisresult is expected. Indeed wavelet are better than curvelets to represent isotropic objects.

\begin{figure}[htb]
\centering{
\hbox{
\includegraphics[width=3in,height=2.4in]{13822fg10.pdf}
\includegraphics[width=3in,height=2.4in]{13822fg11.pdf}}
\hbox{
\includegraphics[width=3in,height=2.4in]{13822fg12.pdf}
\includegraphics[width=3in,height=2.4in]{13822fg13.pdf}}
\hbox{
\includegraphics[width=3in,height=2.4in]{13822fg14.pdf}
\includegraphics[width=3in,height=2.4in]{13822fg15.pdf}}

\caption{\emph{Top Left}: Fermi simulated map without noise.
\emph{Top Right}: Fermi simulated map with Poisson noise.
\emph{Middle Left}: Fermi simulated map denoised with Anscombe VST + wavelet shrinkage.
\emph{Middle Right}: Fermi simulated map denoised with MS-VSTS + curvelets (Algorithm~\ref{algcurv}).
\emph{Bottom Left}: Fermi simulated map denoised with MS-VSTS + IUWT (Algorithm~\ref{alg1}) with threshold $5\sigma_j$.
\emph{Bottom Right}: Fermi simulated map denoised with MS-VSTS + IUWT (Algorithm~\ref{alg1}) with threshold $3\sigma_j$.
Pictures are in logarithmic scale.}
\label{rechsd}
}
\end{figure}

\begin{figure*}
\begin{center}
\includegraphics[width=2.5in]{13822fg16.pdf} \hfill
\includegraphics[width=2.5in]{13822fg17.pdf} \hfill
\includegraphics[width=2.5in]{13822fg18.pdf} \hfill
\includegraphics[width=2.5in]{13822fg19.pdf} \hfill
\includegraphics[width=2.5in]{13822fg20.pdf} \hfill
\includegraphics[width=2.5in]{13822fg21.pdf} \hfill
\caption{View of a single HEALPix face from the results of Figure~\ref{rechsd}.
\emph{Top Left}: Fermi simulated map without noise.
\emph{Top Right}: Fermi simulated map with Poisson noise.
\emph{Middle Left}: Fermi simulated map denoised with Anscombe VST + wavelet shrinkage.
\emph{Middle Right}: Fermi simulated map denoised with MS-VSTS + curvelets (Algorithm~\ref{algcurv}).
\emph{Bottom Left}: Fermi simulated map denoised with MS-VSTS + IUWT (Algorithm~\ref{alg1}) with threshold $5\sigma_j$.
\emph{Bottom Right}: Fermi simulated map denoised with MS-VSTS + IUWT (Algorithm~\ref{alg1}) with threshold $3\sigma_j$.
Pictures are in logarithmic scale.
}
\label{recface}
\end{center}
\end{figure*}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Written By Michael Brodskiy
% Class: Analytic Geometry & Calculus III (Math-292)
% Professor: V. Cherkassky
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\documentclass[12pt]{article}
\usepackage{alphalph}
\usepackage[utf8]{inputenc}
\usepackage[russian,english]{babel}
\usepackage{titling}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{enumitem}
\usepackage{amssymb}
\usepackage[super]{nth}
\usepackage{everysel}
\usepackage{ragged2e}
\usepackage{geometry}
\usepackage{fancyhdr}
\geometry{top=1.0in,bottom=1.0in,left=1.0in,right=1.0in}
\newcommand{\subtitle}[1]{%
  \posttitle{%
    \par\end{center}
    \begin{center}\large#1\end{center}
    \vskip0.5em}%

}
\usepackage{hyperref}
\hypersetup{
colorlinks=true,
linkcolor=blue,
filecolor=magenta,
urlcolor=blue,
citecolor=blue,
}

\urlstyle{same}

\title{Lecture IV Notes}
\date{\today}
\author{Michael Brodskiy\\ \small Professor: V. Cherkassky}

\begin{document}

\maketitle

\subsection{The Distance From a Point to a Plane}

The distance from any point to a plane may be found using the formula $\frac{|ax+by+cz+d|}{\sqrt{a^2+b^2+c^2}}$, where $a$-$d$ are the coefficients of the equation of the plane, and $x$-$z$ are the coordinates of the point.

\subsection{The Distance From a Plane to a Plane}

Find one point on either plane, and repeat the process to find a distance from a point to a plane.

\section{Cylinders and Quadric Surfaces}

The equation of a quadric surface is given by the equation: $Ax^2+By^2+Cz^2+Dxy+Eyz+Fxz+Gx+Hy+Iz+J$, where $A$-$J$ are constants.\\

There are six different figures that should be known:

\begin{tabular}{|p{.45\textwidth}||p{.45\textwidth}|}

\hline
  Figure & Equation \\
\hline
Ellipsoid: A Figure in Which All Traces are Ellipses & $\frac{x^2}{a^2}+\frac{y^2}{b^2}+\frac{z^2}{c^2}=1$\\
\hline
  Cone: A Figure in Which Horizontal Traces are Ellipses and Vertical Traces in $x$ and $y$ are Hyperbolas & $\frac{x^2}{a^2}+\frac{y^2}{b^2}=\frac{z^2}{c^2}$\\
\hline
Elliptic Paraboloid: Horizontal Traces are Ellipses and Vertical Traces are Parabolas & $\frac{x^2}{a^2}+\frac{y^2}{b^2}=\frac{z}{c}$\\
\hline
Hyperboloid of One Sheet: Horizontal Traces are Ellipses and Vertical Traces are Hyperbolas & $\frac{x^2}{a^2}+\frac{y^2}{b^2}-\frac{z^2}{c^2}=1$\\
\hline
Hyperbolic Paraboloid: Horizontal Traces are Hyperbolas and Vertical Traces are Parabolas & $\frac{x^2}{a^2}-\frac{y^2}{b^2}=\frac{z}{c}$\\
\hline
Hyperboloid of Two Sheets: Horizontal Traces are Ellipses in $z$ and Vertical Traces are Hyperbolas & $-\frac{x^2}{a^2}-\frac{y^2}{b^2}+\frac{z^2}{c^2}=1$\\
\hline

\end{tabular}

\end{document}


\chapter{Introduction} \label{sec:Introduction}
Optimizing predictive models on datasets obtained from citizen-science projects can be computationally expensive as these datasets grow in size. Consequently, running models based on Multi-layered Neural Networks, Integer Programming, and other optimization routines can become more computationally difficult as the number of parameters increase, despite using the faster Central Processing Units (CPUs) in the market. Incidentally, it becomes difficult for citizen-science projects, which often deal with large datasets, to scale if the organizers do not employ special processing units to run neural networks as optimization models, which require extensive tensor operations. One such special processing unit is the Graphical Processing Unit (GPU), which offers numerous cores to parallelize computation. GPUs can often outperform CPUs in computing such predictive models if these models \textit{heavily} rely on large-scale tensor operations \cite{ParallelNVIDIA, cuDNNPaper}. By using GPUs over CPUs to accelerate computation on a citizen-science project, the model could achieve better optimization in less time, enabling the project to scale.

\section{Avicaching} \label{sec:Avicaching}
Part of the eBird project, which aims to ``maximize the utility and accessibility of the vast numbers of bird observations made each year by recreational and professional bird watchers'' \cite{EBird}, Avicaching is an incentive-driven game trying to homogenize the spatial distribution of citizens' (agents') observations \cite{Xue2016Avi1, Xue2016Avi2}. Since the dataset of agents' observations in eBird is geographically heterogeneous (concentrated in some places like cities and sparse in others), Avicaching homogenizes the observation set by placing rewards at and attracting agents to under-sampled locations \cite{Xue2016Avi1}. For the agents, collecting rewards increases their `utility' (excitement, fun etc.), while for the organizers, a more homogeneous observation dataset means better sampling and higher confidence in using it for other models.
\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{avicaching_change}
    \caption[Avicaching 2014-15 Results in Tompkins and Cortland Counties]{Avicaching 2014-15 Results in Tompkins and Cortland Counties, NY: With the previous model \cite{Xue2016Avi1, Xue2016Avi2, EBird}, Avicaching was able to attract `eBirders' to under-sampled locations by distributing rewards over the location set.}
    \label{fig:Avicaching 2014-15 Results in Tompkins and Cortland Counties}
\end{figure}

To accomplish this task of specifying rewards at different locations based on the historical records of observations, Avicaching would learn how agents change their behavior when a certain sample of rewards was applied to the set of locations, and then redistribute rewards across the locations based on those learned parameters \cite{Xue2016Avi2}. This requirement naturally translates into a  optimization problem, which we implement using multi-layered neural networks and linear programming.

\section{Important Questions} \label{sec:Important Questions}
Although the previously devised solutions to Avicaching were conceptually effective \cite{Xue2016Avi1, Xue2016Avi2}, using CPUs to solve Mixed Integer Programming and (even) shallow neural networks made the models impractical to scale. Solving the problems faster would have also allowed organizers to find better results (more optimized). These concerns form the pivots of our research study.

\subsection{Solving Faster} \label{sec:Important Questions - Solving Faster}
We were interested in using GPUs to run our optimization models because of their capability to accelerate problems based on large tensor operations \cite{ParallelNVIDIA, cuDNNPaper}. Newer generation NVIDIA GPUs, equipped with thousands of CUDA (NVIDIA's parallel computing API) cores \cite{NVIDIA}, could have empowered Avicaching's organizers to scale the game, if the underlying models were computed using simple arithmetic operations on tensors rather than using conditional logic (see~\Cref{sec:Avoiding Specific Operations in GPUs}). Since even the faster CPUs --- in the range of Intel Core i7 chipsets --- are sequential in processing and do not provide as comparable parallel processing\footnote{CPUs often have multiple cores nowadays, but very few compared to what many GPUs have.} as GPUs do, we sought to solve the problem much faster using GPUs. But \textbf{how fast could we do it?}

\subsection{Better Results} \label{sec:Important Questions - Better Results}
For learning parameters in agents' change of behavior on a fixed set of rewards, the previously devised sub-model delivered predictions that differed $26\%$ from Ground Truth \cite[Table~1]{Xue2016Avi2}. This model was then used to redistribute rewards in a budget. If we could get closer to the Ground Truth, i.e., better learn the parameters for the change, we could redistribute rewards with superior prediction/accuracy. Since the organizers require the \textit{best} distribution of rewards, we will need a set of learned parameters that is closer to the Ground Truth (in terms of Normalized Mean Squared Error \cite[Section~4.2]{Xue2016Avi2}). In a gist, we aimed to \textbf{learn the parameters more suitably}, and \textbf{find the  best allocation of rewards.}

\subsection{Adjusting the Model's Features} \label{sec:Important Questions - Adjusting the Model's Features}
Once the model starts delivering better results than the previously devised models, one thinks if some characteristics\footnote{Tinkering with hyper-parameters like learning rate or adding a weight regularizer.} of the model can be changed to get more preferable results (one could also build a better model). While a goal of ``getting better results'' is an unending struggle, there is a trade-off with practicality as these adjustments take time and computation power to test --- and we didn't have unlimited resources. Therefore, we asked if one could \textbf{reasonably adjust the model's features to improve performance and optimization.}

\section{Computation Using GPUs} \label{sec:Computation Using GPUs}
The use of GPUs has changed drastically in the last decade --- from rendering superior graphics to parallelizing floating point operations. Companies like NVIDIA are now providing General Purpose GPUs (GPGPUs) that are capable of executing parallel algorithms using thousands of cores and threads \cite{NVIDIA}. Furthermore, by working with newly-developed parallel programming APIs like CUDA, one can handle a GPU's threads more efficiently and optimize a task's datapath \cite{CUDADocs}. In the next sections, we briefly describe NVIDIA GPU's architecture, instruction set, and best practices for optimization. Although we do not implement the CUDA back-end manually and use PyTorch's implementation \cite{PTDocs}, understanding the basics of the processor was helpful for designing our models.

\subsection{GPU Architecture} \label{sec:GPU Architecture}
We describe the structure and abstractions of an NVIDIA GPU with Pascal architecture\footnote{We use NVIDIA Quadro P4000 (Pascal architecture) for our tests.}. These devices are multi-threaded multi-processors for executing ``fine-grained threads efficiently'' \cite[Appendix~B.4]{PattersonARM}. Unlike Personal Computer CPUs, which currently comprise 1-8 cores with fast and low-latency datapaths, GPUs provide scalable parallel processing power with high throughput and latency\footnote{High latency is undesirable.} \cite{PattersonARM, DemystifyingGPU}. Informally, GPUs are often referred to as a collection of many `dumb' cores, unlike the few `smart' cores in a CPU. Nonetheless, GPUs are efficient in executing simple instructions in parallel, using hierarchies of cores, threads, and memory.

\subsubsection{Core Organization}
In most architectures by NVIDIA, many Scalar Processors (SP or CUDA cores) are clustered in a Streaming Multiprocessor (SM), which are then organized in Grid Processing Clusters (GPCs). A GPU can have several GPCs, as shown in \Cref{fig:Organization of Cores in NVIDIA Pascal GP100 GPU} \cite{CUDADocs, ParallelNVIDIA, DemystifyingGPU}.
\begin{figure}
    \centering
    \begin{subfigure}{\textwidth}
        \centering
        \includegraphics[width=\textwidth]{gpu_pascal}
        \caption[Organization of Cores in NVIDIA Pascal GP100 GPU]{Organization of Cores in NVIDIA Pascal GP100 GPU: In 6 GPCs and 60 SMs, 3840 SP or CUDA Cores (depicted by green blocks) are arranged \cite{PascalWhitepaper, ParallelNVIDIA}.}
        \label{fig:Organization of Cores in NVIDIA Pascal GP100 GPU}
    \end{subfigure}\vspace*{1em}
    \begin{subfigure}{\textwidth}
        \centering
        \includegraphics[width=.7\textwidth]{gpu_pascal_sm}
        \caption[Constituents of a Streaming Multiprocessor]{Constituents of a Streaming Multiprocessor \cite{PascalWhitepaper, ParallelNVIDIA}.}
        \label{fig:Constituents of a Streaming Multiprocessor}
    \end{subfigure}
    \caption[The NVIDIA Pascal Architecture]{The NVIDIA Pascal Architecture: A GPU, like a typical CPU, contains multiple hierarchies of memory. In addition, NVIDIA also builds a hierarchy of core organization.}
    \label{fig:The NVIDIA Pascal Architecture}
\end{figure}

The Streaming Multiprocessor (\Cref{fig:Constituents of a Streaming Multiprocessor}), which is the primary multi-threaded multi-processor, houses a shared memory unit for all SPs along with registers and Special Function Units (SFUs), which can calculate specific mathematical functions in fewer clock cycles than threads. The SM is responsible for relaying instructions to threads (in SPs) and maintaining synchronized parallel computation at barriers \cite{PascalWhitepaper, DemystifyingGPU}. GPCs and other hierarchies provide abstractions to memory and program access.

\subsubsection{Memory Organization}
Even the memory units are organized into hierarchies like multi-level caches in CPUs (\Cref{fig:The NVIDIA Pascal Architecture}). In a SM, there are: dedicated local memory units for threads, register files shared by several SPs in a SM, and shared memory units for all SP Cores in a SM \cite{PascalWhitepaper, ParallelNVIDIA}.

The global internal memory of the GPU (like the RAM of a computer), which is housed separately around the clusters of SMs, stores the datasets for a program. Data is then distributed in the shared and local memory units once threads start executing an instruction \cite[Appendix~B]{PattersonARM}. Moreover, similar to the multi-level cache structure in CPUs, GPUs also have on-chip cache hierarchy (\Cref{fig:The NVIDIA Pascal Architecture}), which enables threads to quickly access working data, thus increasing the performance. On a side note, GPUs' caches are often not big enough to hold full datasets as programs' often require large datasets, leading to higher miss rates than those of caches in CPUs \cite[Appendix~B]{PattersonARM}.

\subsubsection{Thread Organization}
According to NVIDIA, the parallel structure is obtained by organizing threads into a hierarchy (``grids'' of ``blocks'' of ``warps'' of ``threads''), where threads of a warp (an abstraction) execute in lockstep per clock cycle \cite{CUDADocs, DemystifyingGPU} \cite[Appendix~B]{PattersonARM}. In other words, warps are the basic units of execution in a GPU, which receive singular instructions from the control unit, and forward them to their constituent threads for execution \cite{CUDADocs, DemystifyingGPU}. This abstraction allows the GPU to execute independent sub-tasks in parallel on a large scale (\Cref{fig:GPU - Thread Organization in a GPU}), reducing the total runtime of the program.
\begin{figure}[!htbp]
    \centering
    \includegraphics[width=.55\textwidth]{gpu_threads_blocks}
    \caption[Thread Organization in a GPU]{Thread Organization in a GPU: Warps are only an abstraction for identifying threads executing a single instruction; however, threads are physically organized into blocks and grids \cite{CUDADocs,ParallelNVIDIA}}
    \label{fig:GPU - Thread Organization in a GPU}
\end{figure}

NVIDIA calls this setup ``Single-Instruction Multiple-Thread (SIMT)'' \cite{CUDADocs} \cite[Appendix~B.4]{PattersonARM}, where a single instruction is executed in lockstep by multiple threads in a warp (though allowed to branch and diverge). The differences between SIMT and Single-Instruction Multiple-Data (SIMD), a common feature in CPUs, are very subtle. While SIMD distributes data sequences into vectors to be executed by a single core/thread of the CPU (data-level parallelism), SIMT requires a particular thread to only operate on a scalar element of the dataset  \cite{PattersonARM}. In this way, having thousands of threads running in locksteps provides better throughput than doing vector operations per clock cycle. SIMT also enables programmers to write a program meant for a single, independent thread instead of managing vectors in their code, which promotes ease of use and programming \cite[Chapter~4]{CUDADocs}.

\subsubsection{Instruction Set}
The Instruction Set of a GPU is limited, which causes complex instructions to take several clock cycles. Even though a Special Functional Unit (SFU) computes complex functions (sine, square-root, reciprocal function etc.) in fewer clock cycles, their count is small compared to the commonplace threads \cite[Appendix~B]{PattersonARM}. The instruction set for Pascal architecture contains basic operations to load and store memory, perform basic floating point, integer (add, subtract, multiply, divide, min, max, left-shift etc.), binary logical (or, and, xor etc.), and branch-jump operations \cite{CUDABinUtils, DemystifyingGPU}. This comprises a very basic set of instructions unlike those in Intel CPUs, for example, which have coalesced and dedicated datapaths for complex instructions\footnote{Intel x86 CPUs have Complex Instruction Set Computer (CISC) Design.} \cite{PattersonARM}.

\subsection{Avoiding Specific Types of Operations in GPUs} \label{sec:Avoiding Specific Operations in GPUs}
GPUs are not better than CPUs at many operations, unlike popular perception. With parallel processing comes concerns of synchronization delays, blocking instructions, program correctness etc. Moreover, since GPUs are separate units of processors, usually connected to CPUs by PCIe lanes, back and forth memory transfer can cause slowdowns in overall performance. Even the GPUs low-level caches are not big enough to deliver hit rates as compared to CPUs. Often there exists a tradeoff when programming with GPUs, and one should take care to avoid such delays.

\subsubsection{Branch and Diverge}
Since NVIDIA GPUs deal with program correctness through intra-warp synchronization (keeping warp threads in lockstep), threads must remain in sync even when conditional blocks force different datapaths for different threads. This ultimately reduces performance as some threads become inactive, which can aggregate and slowdown the performance of the whole program.

To avoid such delays and maximize throughput, one should elude from using extensive branching in the program. Using conditional checks can change a thread's datapath, which can possibly stall other threads in the warp from proceeding to the next instruction in the program. This behavior is referred to as `branch and diverge' (\Cref{fig:Decrease in GPU Throughput due to Branch and Diverge}), which occurs when some threads in a warp follow a different datapath due to conditional branching, `diverging' from the group. Now when the warp's threads are forced to converge, those diverging threads would take longer to complete the instruction, causing the non-diverging threads to wait. This radically decreases the throughput of the warp, which again diverges from the block, adding up the latency \cite[Appendix~B]{PattersonARM}\cite{DemystifyingGPU}.
\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{gpu_branch_diverge}
    \caption[Decrease in GPU Throughput due to Branch and Diverge]{Decrease in GPU Throughput due to Branch and Diverge: In this case, while some threads execute \texttt{A, B}, others execute \texttt{X, Y}. If those sets of operations are all executed by threads of the same warp, the threads could diverge from the group and decrease the throughput \cite{ParallelNVIDIA}.}
    \label{fig:Decrease in GPU Throughput due to Branch and Diverge}
\end{figure}

To ensure program correctness, the diverging threads in a warp are stalled while the other non-diverging threads are executed and vice-versa, according to NVIDIA \cite{CUDADocs,ParallelNVIDIA}. For example, in \Cref{fig:Decrease in GPU Throughput due to Branch and Diverge}, the 4 threads with thread-indexes $=\{0,1,2,3\}$ will have to wait while the other threads in the warp execute instructions \texttt{X} and \texttt{Y}, and, the other threads will be stalled while the 4 threads execute instructions \texttt{A} and \texttt{B}. This leads to a $50\%$ decrease in GPU performance as instructions get `serialized' \cite{ParallelNVIDIA}, though ensuring program correctness. The overhead can slowdown the run drastically if the program is extensively branched\footnote{On a side note, functions like \texttt{min, max, abs} would not add overhead due to branch and diverge since they have devoted datapaths, and operation codes in the GPUs' instruction set.}. This overhead can either be decreased by reducing the number of conditional statements or by intentionally directing conditional statements to threads to different warps in the block, which execute independently \cite{ParallelNVIDIA}. The latter, at the maximum, only affects full program synchronization if the operations in conditional blocks have unequal runtimes. Furthermore, the performance is usually much better than the otherwise decrease due to diverging intra-warp threads.

\subsubsection{Memory Limitations}
PCIe lanes between GPU's internal memory and system's main memory are not as fast as on-chip cache access\cite{CUDADocs, ParallelNVIDIA}. This means that transferring datasets back and forth between GPUs and CPUs can add up considerably. Often the limits of GPU's internal memory or required operations on a dedicated device can force the user to transfer datasets multiple times. Even so, one should look for optimizations that can reduce the number of transfers whenever possible, especially when large datasets are involved.

This suggestion is also applicable when transferring datasets in memory intra-GPU (between caches). Miss rates in a GPU's local caches are often high due to limited size, and data requests from higher-level caches can often be expensive.

\subsection{GPUs' Strengths} \label{sec:Gpu's Strengths}
Inferring from GPUs' weaknesses and architecture, we can say that GPUs are good at handling independent sub-tasks with less diverging sections. These independent sub-tasks can usually be clubbed into matrices and tensors, which the threads operate on in parallel. The fewer conditional branching we have in our algorithms, the better.

Therefore, GPUs are particularly good at doing linear algebra \cite{PattersonARM, ParallelNVIDIA} since most arithmetic operations on matrix elements can be independently executed without branching. Avoiding branching often requires extensive dataset preprocessing; consequently, one should meticulously design the structure of the models' datasets. Tensor-based datasets are often required in machine learning problems, graphics rendering routines, graph-based algorithms etc., which GPUs can potentially accelerate \cite{PattersonARM}.


\section{The \counti algorithm}
\Label{sec:counti}

The \counti algorithm in the \cxx Standard Library \cite[\S
28.5.9]{cxx-17-draft} counts
the frequency of occurrences for a particular element in
a sequence.
For our purposes we have modified
the generic implementation
to that of arrays of type \valuetype.
The signature now reads:

\begin{lstlisting}[style = acsl-block]

  size_type
  count(const value_type* a, size_type n, value_type v);
\end{lstlisting}

Informally, the function returns the number of occurrences of
\inl{v} in the array \inl{a}.

\subsection{The logic function \Count}

When trying to specify \counti we are faced with the situation that
\acsl does not provide a definition of counting a value in an array.\footnote{
  This statement is not quite true because the \acsl documentation
  lists \inl{numof} as one of several \emph{higher order logic constructions} \cite[\S 2.6.7]{ACSLSpec}.
  However, these \emph{extended quantifiers} are mentioned only as experimental features.
}
We therefore start with an axiomatic definition of \emph{logic function} \Count
that captures the basic intuitive features of counting on an array section.
The expression \inl{Count(a,m,n,v)} returns the number of
occurrences of \inl{v} in \inl{a[m],...,a[n-1]}.

The specification of \counti will then be fairly short because it employs
our \emph{logic function}
\Count whose (considerably) longer definition is given in the
Listings~\ref{logic:Count-1} and~\ref{logic:Count-2}.\footnote{
 This definition of \Count is a generalization of
 the \emph{logic function} \inl{nb_occ} of the \acsl specification \cite{ACSLSpec}.
}

\begin{itemize}
\item
The \acsl keyword \inl{axiomatic}
is used to structure the specification and gather the logic function \Count and related lemmas.
Note that the interval bounds \inl{m} and \inl{n} and the return value for \Count are of type \inl{integer}.

\item
The logic functions \Count is recursively defined.
It consist of two checks: whether the range is empty and for the value of
the "current" element in the array. The recursion goes down on the range length.
We also provide an overloaded version of \Count that accepts only
the length of an array, thus relieving the use the supply the argument $m=0$ for the
case of a complete array.

\item
Lemma \logicref{CountEmpty} covers the cases of empty ranges.

\item
Lemmas \logicref{CountHit} and
\logicref{CountMiss} reduce
counting of a range of length~$n-m$ to a range of length~$n-m-1$.

\item
Lemmas \logicref{CountOne} and \logicref{CountSingle} built on on top of \CountHit
and \CountMiss.
Using them simplifies several \coq proofs.
They also slightly change the induction scheme from $n-1 \rightarrow n$
to $n \rightarrow n+1$.
\end{itemize}

\begin{logic}[hbt]
\begin{minipage}{\textwidth}
\lstinputlisting[linerange={1-45}, style=acsl-block, frame=single]{Source/Count.acsl}
\end{minipage}
\caption{\label{logic:Count-1}The logic function \Count (1)}
\input{Listings/Count.acsl.labels.tex}
\input{Listings/Count.acsl.index.tex}
\end{logic}

\FloatBarrier

\begin{itemize}

\item
The logic function \Count depends only on the set \inl{a[m..n-1]} of memory locations.
Lemma \logicref{CountUnchanged} makes this claim explicit by ensuring that
\Count produces the same result
if the values \inl{a[0..n-1]} do not change between two program states indicated
by the labels~\inl{K} and~\inl{L}.
We use here predicate \logicref{Unchanged} to express the premise.

\item
Lemma \logicref{CountEqual} is a generalization of lemma \CountUnchanged for
the case of comparing \Count on two arrays.

\item
Lemmas \logicref{CountUnion} and \logicref{CountCut}
allow to deal with partitions of arrays.
\end{itemize}

\begin{logic}[hbt]
\begin{minipage}{\textwidth}
\lstinputlisting[linerange={46-90}, style=acsl-block, frame=single]{Source/Count.acsl}
\end{minipage}
\caption{\label{logic:Count-2}The logic function \Count (2)}
\end{logic}

\FloatBarrier

\begin{itemize}
\item
Lemmas \logicref{CountSingleBounds} and \logicref{CountBounds}
express lower and upper bounds of \Count.
Lemma \logicref{CountIncreasing} states that \Count is a monotonically increasing.

\item
Finally, lemmas \logicref{CountSingleShift} and \logicref{CountShift}
state that \Count is invariant under array shifts.
\end{itemize}

We mention here also lemma \logicref{CountSomeEqual}
which brings together properties of \logicref{Count} and \logicref{Find}.

\input{Listings/CountFind.acsl.tex}

\clearpage

\subsection{Formal specification of \counti}

In the contract of \specref{counti} we use the logic function
\logicref{Count}
Note that our specification also states that the result of \counti is non-negative
and less than or equal the size of the array.

\input{Listings/count.h.tex}

\subsection{Implementation of \counti}

The following listing shows a possible implementation of \implref{counti}.
Note that we refer to the logic function \Count in one of the loop invariants.

\input{Listings/count.c.tex}

\clearpage


\documentclass[letterpaper]{article}

\usepackage[utf8x]{luainputenc}
\usepackage{aeguill}
%\usepackage{nopageno}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{mathrsfs}
\usepackage{fullpage}
\usepackage{fancyhdr}
\setlength{\headheight}{12pt}
\pagestyle{fancy}
\chead{Linear Algebra}
\lhead{October 12, 2015}
\rhead{Jon Allen}
\allowdisplaybreaks

\newcommand{\abs}[1]{\left\lvert #1 \right\rvert}

\begin{document}
%\renewcommand{\labelenumii}{\alph{enumii}.}
%\renewcommand{\labelenumiii}{\alph{enumiii}.}
%\renewcommand{\labelenumi}{(\arabic{enumi})}
\section*{3.1}
%6, 8, 9(b,c), 11, 15, 16.
\begin{enumerate}
\setcounter{enumi}{5}
\item
  \begin{enumerate}
  \item
  Let $U$ and $V$ be subspaces of $\mathbb{R}^n$. Define the \emph{intersection} of $U$ and $V$ to be
  \[U\cap V=\{\mathbf{x}\in \mathbb{R}^n:\mathbf{x}\in U\text{ and }\mathbf{x}\in V\}\]
  Show that $U\cap V$ is a subspace of $\mathbb{R}^n$. Give two examples.

  We know that $\mathbf{0}\in U$ and $\mathbf{0}\in V$. And so $\mathbf{0}\in U\cap V$. Now if we take any $\mathbf{u},\mathbf{v}\in U\cap V$ then $\mathbf{u},\mathbf{v}\in U$ and $\mathbf{u},\mathbf{v}\in V$ and so $\mathbf{u}+\mathbf{v}\in U$ and $\mathbf{u}+\mathbf{v}\in V$. Thus $\mathbf{u}+\mathbf{v}\in  U\cap V$. Similarly $\alpha\mathbf{u}\in U$ and $\alpha\mathbf{u}\in V$ for any $\alpha\in \mathbb{R}$. Thus $\alpha\mathbf{u}\in U\cap V$.
  \item
  Is $U\cup V=\{\mathbf{x}\in \mathbb{R}^n:\mathbf{x}\in U\text{ or }\mathbf{x}\in V\}$ always a subspace of $\mathbb{R}^n$? Give a proof or counterexample.

  Let $V=\{3n:\forall n\in \mathbb{Z}\}$ and $U=\{2n:\forall n\in \mathbb{Z}\}$. Then $3\in V$ and $2\in U$. Therefore $3,2\in U\cup V$. But $3+2=5\not\in V$ and $5\not\in U$. Therefore $5\not\in U\cup V$ and so we have a counterexample.
  \end{enumerate}
\setcounter{enumi}{7}
\item
Let $\mathbf{v}_1,\dots,\mathbf{v}_k\in \mathbb{R}^n$ and let $\mathbf{v}\in \mathbb{R}^n$. Prove that $\text{Span}(\mathbf{v}_1,\dots,\mathbf{v}_k)=\text{Span}(\mathbf{v}_1,\dots,\mathbf{v}_k,\mathbf{v})$ if and only if $\mathbf{v}\in \text{Span}(\mathbf{v}_1,\dots,\mathbf{v}_k)$.

Define $A=\text{Span}(\mathbf{v}_1,\dots,\mathbf{v}_k)$ and $B=\text{Span}(\mathbf{v}_1,\dots,\mathbf{v}_k,\mathbf{v})$.

We know that $\mathbf{v}\in B$ and so if we assume that $A=B$ then $\mathbf{v}\in A$ follows immediately. Now lets assume that $\mathbf{v}\in A$. Then for any $\mathbf{x}\in A$ we have $\mathbf{x}=\alpha_1\mathbf{v}_1+\dots+\alpha_k\mathbf{v}_k+0\mathbf{v}\in B$.
Now, because $\mathbf{v}=\alpha_1\mathbf{v}_1+\dots+\alpha_k\mathbf{v}_k$ Then any element $\mathbf{x}\in B$ we have $\mathbf{x}=\beta_1\mathbf{v}_1+\dots+\beta_k\mathbf{v}_k+\beta\mathbf{v}=\beta_1\mathbf{v}_1+\dots+\beta_k\mathbf{v}_k+\beta(\alpha_1\mathbf{v}_1+\dots+\alpha_k\mathbf{v}_k)=(\beta_1+\beta\alpha_1)\mathbf{v}_1+\dots+(\beta_k+\beta\alpha_k)\mathbf{v}_k\in A$.
$\Box$
\item
Determine the intersection of the subspaces $\mathcal{P}_1$ and $\mathcal{P}_2$ in each case:
  \begin{enumerate}
  \setcounter{enumii}{1}
  \item
    $\mathcal{P}_1=\text{Span}\left((1,2,2),(0,1,1)\right),\mathcal{P}_2=\text{Span}\left((2,1,1),(1,0,0)\right)$
    \begin{align*}
      a\left(\begin{array}{r}1\\2\\2\end{array}\right)
      +b\left(\begin{array}{r}0\\1\\1\end{array}\right)
      -c\left(\begin{array}{r}2\\1\\1\end{array}\right)
      -d\left(\begin{array}{r}1\\0\\0\end{array}\right)
      =0\Rightarrow
      \left[\begin{array}{rrrr}
      1&0&2&1\\
      2&1&1&0\\
      2&1&1&0
      \end{array}\right]\Rightarrow
      \left[\begin{array}{rrrr}
      1&0&2&1\\
      0&1&-3&-2\\
      0&0&0&0
      \end{array}\right]
    \end{align*}
    And so we have two free variables. Which means given any element in $\mathcal{P}_1$ we can find that element in $\mathcal{P}_2$ and the other way around. That is $\mathcal{P}_1=\mathcal{P}_2$
  \item
    $\mathcal{P}_1=\text{Span}\left((1,0,-1),(1,2,3)\right),\mathcal{P}_2=\{\mathbf{x}:x_1-x_2+x_3=0\}$
    Converting $\mathcal{P}_2$ to standard form we have $\mathbf{x}=\left(\begin{array}{c}x_1\\x_1+x_3\\x_3\end{array}\right)$ or $\mathcal{P}_2=\text{Span}\{(1,1,0),(0,1,1)\}$. And so we put everything in an array as above
    \[
    \left[\begin{array}{rrrr}
    1&1&-1&0\\
    0&2&-1&-1\\
    -1&3&0&-1
    \end{array}\right]
    \left[\begin{array}{rrrr}
    1&1&-1&0\\
    0&2&-1&-1\\
    0&4&-1&-1
    \end{array}\right]
    \left[\begin{array}{rrrr}
    1&1&-1&0\\
    0&2&0&0\\
    0&2&-1&-1
    \end{array}\right]
    \left[\begin{array}{rrrr}
    1&1&-1&0\\
    0&1&0&0\\
    0&0&1&1
    \end{array}\right]
    \left[\begin{array}{rrrr}
    1&0&0&1\\
    0&1&0&0\\
    0&0&1&1
    \end{array}\right]
    \]
    Which gives us one free variable, and so the two planes intersect on a line. We notice that $(1,0,-1)=(1,1,0)-(0,1,1)$ and so since $(1,0,-1)$ is in $P_1$ and in $\mathcal{P}_2$ so any element in $\text{Span}(1,0,-1)$ is in the intersection, thus we have found our line.
  \end{enumerate}
\setcounter{enumi}{10}
\item
Suppose $V$ and $W$ are orthogonal subspaces of $\mathbb{R}^n$, i.e., $\mathbf{v}\cdot\mathbf{w}=0$ for every $\mathbf{v}\in V$ and every $\mathbf{w}\in W$.
Prove that $V\cap W=\{\mathbf{0}\}$.

Start by assuming there exists some element  $\mathbf{u}\in V\cap W$ such that $\mathbf{u}\ne \mathbf{0}$.
Then because $\mathbf{u}\in V$ and $\mathbf{u}\in W$ we know that $\mathbf{u}\cdot\mathbf{u}=\mathbf{0}$.
And of course $\mathbf{u}\cdot\mathbf{u}=||\mathbf{u}||^2$.
But $\mathbf{u}\ne 0$ and so $||\mathbf{u}||^2>0$ and $\mathbf{u}\cdot\mathbf{u}\ne 0$ which leaves us with a contradiction.
We must assume then that the intersection of these sets contains only $\mathbf{0}$.
\setcounter{enumi}{14}
\item
Let $A$ be an $m\times n$ matrix. Let $V\subset \mathbb{R}^n$ and $W\subset \mathbb{R}^m$ be subspaces.
  \begin{enumerate}
  \item
  Show that $E=\{\mathbf{x}\in \mathbb{R}^n:A\mathbf{x}\in W\}$ is a subspace of $\mathbb{R}^n$.

  Of course $A\mathbf{0}=\mathbf{0}\in W$. Let us choose $\mathbf{x},\mathbf{y}\in E$. Then we have $A\mathbf{x}\in W$ and $A\mathbf{y}\in W$ and so $A\mathbf{x}+A\mathbf{y}\in W$. But $A\mathbf{x}+A\mathbf{y}=A(\mathbf{x}+\mathbf{y})$ and so $\mathbf{x}+\mathbf{y}\in E$. And if $A\mathbf{x}\in W$ then $\alpha A\mathbf{x}\in W$ for any $\alpha\in \mathbb{R}$. So because $\alpha A\mathbf{x}=A(\alpha\mathbf{x})$ then we have $\alpha\mathbf{x}\in E$. And so $E$ is a subspace.
  \item
  Show that $F=\{\mathbf{y}\in \mathbb{R}^m:\mathbf{y}=A\mathbf{x}\text{ for some }\mathbf{x}\in V\}$ is a subspace of $\mathbb{R}^m$

  Observe that $\mathbf{0}=A\mathbf{0}$ and $\mathbf{0}\in V$. And so $\mathbf{0}\in F$. Now, let us take $\mathbf{u},\mathbf{v}\in F$. Then $\mathbf{u}=A\mathbf{x}_1$ and $\mathbf{v}=A\mathbf{x}_2$ where $\mathbf{x}_1,\mathbf{x}_2\in V$. Now $\mathbf{u}+\mathbf{v}=A\mathbf{x}_1+A\mathbf{x}_2=A(\mathbf{x}_1+\mathbf{x}_2)$. But $\mathbf{x}_1+\mathbf{x}_2\in V$ and so $\mathbf{u}+\mathbf{v}\in F$. And if we choose $\alpha\in \mathbb{R}$ then $\alpha\mathbf{u}=\alpha A\mathbf{x}=A(\alpha\mathbf{x})$ for some $\mathbf{x}\in V$. And since $\mathbf{x}\in V$ then $\alpha\mathbf{x}\in V$ and so $\alpha\mathbf{u}\in F$. Thus $F$ is a subspace.
  \end{enumerate}
\item
Suppose $A$ is a symmetric $n\times n$ matrix. Let $V\subset \mathbb{R}^n$ be a  subspace with the property that $A\mathbf{x}\in V$ for every $\mathbf{x}\in V$. Show that $A\mathbf{y}\in V^\perp$ for all $\mathbf{y}\in V^\perp$

We choose some $\mathbf{y}\in V^\perp$ and $\mathbf{x}\in V$. Then $\mathbf{x}\cdot\mathbf{y}=\mathbf{0}$. Now if $\mathbf{x}A\mathbf{y}=\mathbf{0}$ then $A\mathbf{y}\in V^\perp$. Of course $\mathbf{x}A\mathbf{y}=A^T\mathbf{x}^T\mathbf{y}=(A\mathbf{x})\mathbf{y}$. But $A\mathbf{x}\in V$ and so $(A\mathbf{x})\mathbf{y}=0$ therefore $A\mathbf{y}\in V^\perp$
\end{enumerate}
\end{document}