\section{Analytical Model}
\label{sec:analytical-model}
In this Section we define the analytical model of the system.
%
In particular, we will show the queue model, Markov Chain and performance metrics formulas for the system both running the Off-Loading Policy 1 and the Off-Loading Policy 2.
%
At the end, we will explain how we solved the analytical model to obtain theoretical results for the target case.

\paragraph{Queue Model}
As the system is characterized by Poisson arrivals and Exponential services, we could consider treating the queue model as a Jackson Network.
Nevertheless, this would not be correct without a very strong assumption on the routing probabilities, In fact, a Jackson Network requires the routing probabilities  to be static, i.e. independent of the system state, and this is not the case.

So, \textit{we assume the routing probabilities to be static in order to unlock the potential of the Jackson Network in our analytical dissertation}.

In Figures~\ref{fig:analytical-model-queue-1} and \ref{fig:analytical-model-queue-2}, we show the queue models for the system with Off-Loading Policy 1 and Off-Loading Policy 2, respectively.

\begin{figure}
	\includegraphics[width=\columnwidth]{fig/analytical-model-queue-1}
	\caption{Queue model for the system with OP1.}
	\label{fig:analytical-model-queue-1}
\end{figure}

\begin{figure}
	\includegraphics[width=\columnwidth]{fig/analytical-model-queue-2}
	\caption{Queue model for the system with OP2.}
	\label{fig:analytical-model-queue-2}
\end{figure}

The definition of the routing probabilities relies on the following subsets of Cloudlet states $S_{clt}$, whose definition strictly depends on the adopted off-loading policy:

\begin{itemize}
	\item $A_{clt,1}$:  subset of states where a task belonging to the $1^{st}$ class is accepted in the Cloudlet.

	In OP1, a $1^{st}$ class task is accepted in the Cloudlet as long as not all the $N$ resources are occupied.
	In OP2, a $1^{st}$ class task is accepted in the Cloudlet as long as not all the $N$ resources are occupied or there is at least one $2^{nd}$ class task to be interrupted.

	\begin{equation}
		A_{clt,1} :=
		\left\{
		\begin{array}{ll}
			\begin{aligned}
				& \{(n_{clt,1},n_{clt,2})\in S_{clt} : \\
				& n_{clt,1}+n_{clt,2}<N\}
			\end{aligned} & \mbox{if } OP1 \\
			\\
			\begin{aligned}
				& \{(n_{clt,1},n_{clt,2})\in S_{clt} : \\
				& n_{clt,1}+n_{clt,2}<N \vee n_{clt,2}>0\}
			\end{aligned} & \mbox{if } OP2
		\end{array}
		\right.
	\end{equation}

	\item $A_{clt,2}$: subset of states where  a task belonging to the $2^{nd}$ class is accepted in the Cloudlet.

	In OP1, a $2^{nd}$ class task is accepted in the Cloudlet as long as not all the $N$ resources are occupied.
	In OP2, a $2^{nd}$ class task is accepted in the Cloudlet as long as not all the $N$ resources are occupied and the number of resources occupied by $2^{nd}$ class tasks satisfies the condition $(n_{clt,1}+n_{clt,2}<S)$.

	\begin{equation}
		A_{clt,2} :=
		\left\{
		\begin{array}{ll}
			\begin{aligned}
				& \{(n_{clt,1},n_{clt,2})\in S_{clt} : \\
				& n_{clt,1}+n_{clt,2}<N\}
			\end{aligned} & \mbox{if } OP1 \\
			\\
			\begin{aligned}
				& \{(n_{clt,1},n_{clt,2})\in S_{clt} : \\
				& n_{clt,1}+n_{clt,2}<N \wedge n_{clt,1} + n_{clt,2}<S\}
			\end{aligned} & \mbox{if } OP2
		\end{array}
		\right.
	\end{equation}

	\item $R_{clt,2}$: subset of states where  a task belonging to the $2^{nd}$ class is interrupted in the Cloudlet and it is restarted in the Cloud.

	In OP1, the set is empty because task interruption is not provided by the policy.
	In OP2, a $2^{nd}$ class task is interrupted in the Cloudlet and restarted in the Cloud as long as the former does not have free resources and there is at least one $2^{nd}$ class task to interrupt.

	\begin{equation}
		R_{clt,2} :=
		\left\{
		\begin{array}{ll}
			\begin{aligned}
				& \emptyset
			\end{aligned} & \mbox{if } OP1 \\
			\\
			\begin{aligned}
				& \{(n_{clt,1},n_{clt,2})\in S_{clt} : \\
				& n_{clt,1}+n_{clt,2}\geq S \wedge n_{clt,2}>0\}
			\end{aligned} & \mbox{if } OP2
		\end{array}
		\right.
	\end{equation}
\end{itemize}

Given such sets, the routing probabilities  are accordingly defined:

\begin{itemize}

	\item $a_{clt,1}$: the probability for a demanding $1^{st}$ class task to be accepted in the Cloudlet.

	\begin{equation}
		a_{clt,1} := \sum_{s\in A_{clt,1}} \pi_{s}
	\end{equation}

	\item $a_{clt,2}$: the probability for a demanding $2^{nd}$ class task to be accepted in the Cloudlet.

	\begin{equation}
		a_{clt,2} := \sum_{s\in A_{clt,2}} \pi_{s}
	\end{equation}

	\item $r_{clt,2}$: the probability for a running $2^{nd}$ class task to be interrupted in the Cloudlet and restarted in the Cloud.

	\begin{equation}
		\begin{split}
			r_{clt,2} & = \sum_{s\in R_{clt,2}} \pi_{s} \Big(\frac{\lambda_{1}}{\lambda_{1}+\lambda_{2}}\Big) \\
		\end{split}
	\end{equation}
\end{itemize}

\paragraph{Markov Chain}
As the system is characterized by Poisson arrivals\footnote{same as Exponential inter-arrivals.} and Exponential services, the Markovian condition holds true and we can then determine the Markov Chain\footnote{If the Markovian condition is not satisfied, Markov Chain solution must be considered only as an approximation.} whose resolution allows us to compute the routing probabilities.

For sake of simplicity, we consider here the simple case with $N=2$ in order to (i) give an idea of the system of equations to be solved and (ii) graphically recognize the critical states.
In fact, the representation fo the Markov Chain and the associated equations would be impractical for the target case $N=20$, due to the combinatorial explosion of the state space.

In Figure~\ref{fig:analytical-model-markov-1} we show the Markov Chain for the system with Off-Loading Policy 1 and $N=2$.
In this chain, red states represent those where both the $1^{st}$ and $2^{nd}$ class traffic is blocked by the Cloudlet and forwarded to the Cloud.
The associated flow balance equations are listed in Equation~\ref{eqn:analytical-model-markov-1}.

\begin{figure}
	\includegraphics[width=\columnwidth]{fig/analytical-model-markov-1}
	\caption{Markov Chain for the system with OP1 (N=2).}
	\label{fig:analytical-model-markov-1}
\end{figure}

\begin{equation}
	\begin{split}
		\pi_{0,0}(\lambda_{1}+\lambda_{2})& = \pi_{1,0}\mu_{clt,1}+\pi_{0,1}\mu_{clt,2} \\
		\pi_{0,1}(\lambda_{1}+\lambda_{2}+\mu_{clt,2}) & = \pi_{0,0}\lambda_{2}+\pi_{1,1}\mu_{clt,1}+\pi_{0,2}2\mu_{clt,2} \\
		\pi_{1,0}(\lambda_{1}+\lambda_{2}+\mu_{clt,1}) & = \pi_{0,0}\lambda_{1}+\pi_{1,1}\mu_{clt,2}+\pi_{2,0}2\mu_{clt,1} \\
		\pi_{1,1}(\mu_{clt,1}+\mu_{clt,2}) & = \pi_{0,1}\lambda_{1}+\pi_{1,0}\lambda_{2} \\
		\pi_{0,2}(2\mu_{clt,2}) & = \pi_{0,1}\lambda_{2} \\
		\pi_{2,0}(2\mu_{clt,1}) & = \pi_{1,0}\lambda_{1} \\
		1 & = \pi_{0,0}+\pi_{0,1}+\pi_{1,0}+\pi_{1,1}+\pi_{0,2}+\pi_{2,0}\\
	\end{split}
	\label{eqn:analytical-model-markov-1}
\end{equation}

In Figure~\ref{fig:analytical-model-markov-2} we show the Markov Chain for the system with Off-Loading Policy 2 and $N=S=2$.
In this chain, the red state represents the one where both the $1^{st}$ and $2^{nd}$ class traffic is blocked; whereas the orange states represent those where (i) a $1^{st}$ class arrival is accepted in Cloudlet, causing the restart in Cloud of a random running $2^{nd}$ class task and (ii) a $2^{nd}$ class arrival is blocked by the Cloudlet and forwarded to the Cloud.

\begin{figure}
	\includegraphics[width=\columnwidth]{fig/analytical-model-markov-2}
	\caption{Markov Chain for the system with OP2 (N=S=2).}
	\label{fig:analytical-model-markov-2}
\end{figure}

\begin{equation}
	\begin{split}
		\pi_{0,0}(\lambda_{1}+\lambda_{2})& = \pi_{1,0}\mu_{clt,1}+\pi_{0,1}\mu_{clt,2} \\
		\pi_{0,1}(\lambda_{1}+\lambda_{2}+\mu_{clt,2}) & = \pi_{0,0}\lambda_{2}+\pi_{1,1}\mu_{clt,1}+\pi_{0,2}2\mu_{clt,2} \\
		\pi_{1,0}(\lambda_{1}+\lambda_{2}+\mu_{clt,1}) & = \pi_{0,0}\lambda_{1}+\pi_{1,1}\mu_{clt,2}+\pi_{2,0}2\mu_{clt,1} \\
		\pi_{1,1}(\lambda_{1}+\mu_{clt,1}+\mu_{clt,2}) & = \pi_{0,1}\lambda_{1}+\pi_{1,0}\lambda_{2}+\pi_{0,2}\lambda_{1} \\
		\pi_{0,2}(\lambda_{1}+2\mu_{clt,2}) & = \pi_{0,1}\lambda_{2} \\
		\pi_{2,0}2\mu_{clt,1} & = \pi_{1,0}\lambda_{1}+\pi_{1,1}\lambda_{1} \\
		1 & = \pi_{0,0}+\pi_{0,1}+\pi_{1,0}+\pi_{1,1}+\pi_{0,2}+\pi_{2,0}\\
	\end{split}
	\label{eqn:analytical-model-markov-2}
\end{equation}

In Figure~\ref{fig:analytical-model-markov-2-1} we show the Markov Chain for the system with Off-Loading Policy 2 and $N=2,S=1$.
In this chain, the red state represents the one where both the $1^{st}$ and $2^{nd}$ class traffic is blocked; the orange state represents the one where (i) a $1^{st}$ class arrival is accepted in Cloudlet, causing the restart in Cloud of a random running $2^{nd}$ class task and (ii) a $2^{nd}$ class arrival is blocked by the Cloudlet and forwarded to the Cloud; whereas the grey state represents the one where a $1^{st}$ class arrival is accepted in Cloudlet and a $2^{nd}$ class arrival is blocked by the Cloudlet and forwarded to the Cloud.

\begin{figure}
	\includegraphics[width=\columnwidth]{fig/analytical-model-markov-2-1}
	\caption{Markov Chain for the system with OP2 (N=2, S=1).}
	\label{fig:analytical-model-markov-2-1}
\end{figure}

\begin{equation}
	\begin{split}
		\pi_{0,0}(\lambda_{1}+\lambda_{2})& = \pi_{1,0}\mu_{clt,1}+\pi_{0,1}\mu_{clt,2} \\
		\pi_{0,1}(\lambda_{1}+\mu_{clt,2}) & = \pi_{0,0}\lambda_{2} \\
		\pi_{1,0}(\lambda_{1}+\mu_{clt,1}) & = \pi_{0,0}\lambda_{1}+\pi_{2,0}2\mu_{clt,1} \\
		\pi_{2,0}2\mu_{clt,1} & = \pi_{1,0}\lambda_{1} \\
		1 & = \pi_{0,0}+\pi_{0,1}+\pi_{1,0}+\pi_{2,0}\\
	\end{split}
	\label{eqn:analytical-model-markov-2-1}
\end{equation}

\paragraph{Accepted Workload}
Given the routing probabilities we can determine the following accepted workloads:

\begin{itemize}
	\item \textit{Cloudlet}: arrival rate of tasks belonging to $j$-th class accepted in Cloudlet, valid both for OP1 and OP2:
	\begin{equation}
	\lambda_{clt,j} = a_{clt,j}\lambda_{j}
	\end{equation}

	\item \textit{Cloud}: arrival rate of tasks belonging to $j$-th class forwarded to Cloud, valid both for OP1 and OP2:
	\begin{equation}
	\lambda_{cld,j} = (1-a_{clt,j})\lambda_{j}
	\end{equation}

	\item \textit{Restarts}: rate of tasks belonging to $2^{nd}$ class interrupted in Cloudlet and restarted in Cloud, valid only for OP2:
	\begin{equation}
	\lambda_{r} = r(\lambda_{1}+\lambda_{2})
	\end{equation}
\end{itemize}

\paragraph{Performance metrics}
Given the accepted workloads we can determine the following performance metrics for classed tasks in each subsystem\footnote{In every formula, we omitted the symbol $E[\cdot]$ of the expected value in order to lighten the notation.}.

\begin{itemize}

	\item $1^{st}$ class in Cloudlet:
	\begin{equation}
		\begin{split}
			N_{clt,1} &= \sum_{s: (n_{clt,1},n_{clt,2}) \in S_{clt}} \pi_{s} n_{clt,1}  \\
			X_{clt,1} &= \lambda_{clt,1} \\
			T_{clt,1} &= \frac{N_{clt,1}}{X_{clt,1}} \\
		\end{split}
	\end{equation}

	\item $2^{nd}$ class in Cloudlet:
%\footnote{We assume here to know the expected time lost in Cloudlet by $2^{nd}$ class tasks before their interruption, i.e. $E[T_{clt,2,lost}]$.In particular, as we are not able to determine it from the Markov Chain in a simple way, we will assume the experimental value computed by the simulator.}:
	\begin{equation}
		\begin{split}
			N_{clt,2} &= \sum_{s: (n_{clt,1},n_{clt,2}) \in S_{clt}} \pi_{s} n_{clt,2}  \\
			%E[N_{clt,2}] &= \lambda_{clt,2}E[T_{clt,2}]-\lambda_{r} E[T_{clt,2,lost}] \\
			%E[N_{clt,2}] &= \lambda_{clt,2}E[T_{clt,2}]-E[N_{cld,2}]^{[R]} \\
			X_{clt,2} &= \lambda_{clt,2} - \lambda_{r} \\
			T_{clt,2} &= \frac{N_{clt,2}}{X_{clt,2}} \\
		\end{split}
	\end{equation}

It is important to notice here that we assumed that the Cloudlet throughput for $2^{nd}$ class tasks is only given by the portion of traffic that is served to completion by the Cloudlet, thus excluding the portion of interrupted traffic.

	\item $1^{st}$ class in Cloud:
	\begin{equation}
		\begin{split}
			T_{cld,1} &= \frac{1}{\mu_{cld,1}} \\
			N_{cld,1} &= \lambda_{cld,1}T_{cld,1} \\
			X_{cld,1} &= \lambda_{cld,1} \\
		\end{split}
	\end{equation}

	\item $2^{nd}$ class in Cloud (NR: not restarted\footnote{$2^{nd}$ class tasks that are served by the Cloud because they have not been accepted in the Cloudlet.}):
	\begin{equation}
	\begin{split}
		T_{cld,2}^{[NR]} &= \frac{1}{\mu_{cld,2}} \\
		N_{cld,2}^{[NR]} &= \lambda_{cld,2}T_{cld,2}^{[NR]} \\
		X_{cld,2}^{[NR]} &= \lambda_{cld,2} \\
	\end{split}
	\end{equation}

	\item $2^{nd}$ class in Cloud (R: restarted\footnote{$2^{nd}$ class tasks that are served by the Cloud because they have been interrupted in the Cloudlet.}):
	\begin{equation}
	\begin{split}
		T_{cld,2}^{[R]} &= T_{setup}+ T_{cld,2}^{[NR]} \\
		N_{cld,2}^{[R]} &= \lambda_{r} T_{cld,2}^{[R]} \\
		X_{cld,2}^{[R]} &= \lambda_{r} \\
	\end{split}
	\end{equation}

	It is important to notice here that we decided to assign the entire setup time to the Cloud.

	\item $2^{nd}$ class in Cloud (both restarted and not restarted):
	\begin{equation}
	\begin{split}
		T_{cld,2} &= \sum_{m=NR,R} \frac{N_{cld,2}^{[m]}}{N_{cld,2}}T_{cld,2}^{[m]} \\
		N_{cld,2} &= \sum_{m=NR,R} N_{cld,2}^{[m]} \\
		X_{cld,2} &= \sum_{m=NR,R} X_{cld,2}^{[m]} \\
	\end{split}
	\end{equation}
\end{itemize}

Then we can determine the following performance metrics for each subsystem:

\begin{itemize}
	\item Cloudlet:
	\begin{equation}
	\begin{split}
		T_{clt} &= \sum_{j=1,2} \frac{N_{clt,j}} {N_{clt}} T_{clt,j} \\
		N_{clt} &= \sum_{j=1,2} N_{clt,j} \\
		X_{clt} &= \sum_{j=1,2} X_{clt,j} \\
	\end{split}
	\end{equation}

	\item Cloud:
	\begin{equation}
	\begin{split}
		T_{cld} &= \sum_{j=1,2} \frac{N_{cld,j}} {N_{cld}} T_{cld,j} \\
		N_{cld} &= \sum_{j=1,2} N_{cld,j} \\
		X_{cld} &= \sum_{j=1,2} X_{cld,j} \\
	\end{split}
	\end{equation}

	It is important to notice here that we assumed that the Cloud throughput is given by all the traffic that is served to completion by the Cloudlet, i.e. both the portion of traffic directly assigned to the Cloud and the one that is restarted in the Cloud.
	\end{itemize}

Finally, we can determine the following performance metrics for the whole system:

\begin{equation}
\begin{split}
	T &= \sum_{i=cld,clt} \frac{N_{i}} {N} T_{i} \\
	N &= \sum_{i=cld,clt} N_{i} \\
	X &= \sum_{i=cld,clt} X_{i} \\
\end{split}
\end{equation}

\paragraph{Resolution}
We solved the analytical models with a custom Python script executing the following steps:

\begin{itemize}
\item receives as input the system configuration
\item  generates the Markov Chain representing the Cloudlet
\item  generates the system of equations from the Markov Chain
\item  computes limiting probabilities by solving the system
\item  computes routing probabilities
\item  computes performance metrics and
\item  displays a report of results.
\end{itemize}

Analytical results are presented in Tables~\ref{tbl:evaluation-performance-metrics-1}, \ref{tbl:evaluation-performance-metrics-2-5} and \ref{tbl:evaluation-performance-metrics-2-20}, alongside with their experimental counterparts.
%
We preferred to present analytical and experimental results in a unique common view, in order to provide the reader with an idea on how the simulator approximates analytical results.


\section{Affine approximations}

{\it Remark} 6.1.
To show that a sub-box of the parameter box ${\cal W}$ is killed by one of the interesting conditions (plus associated killerword) we need to show that at each point in the sub-box, the killerword evaluated at that point satisfies the given condition (see
Section 5).  That is, we are simply analyzing a certain function from the sub-box to ${\bf C}$.

As described in Remark FIXME(6.5), this analysis can be pulled back from the sub-box in question to the closed polydisc
$A = \{(z_0,z_1,z_2) \in {\bf C}^3 : |z_k| \le 1$  for $k \in \{0,1,2\}\}.$
Loosely,  we will analyze such a function on $A$ by using  Taylor approximations consisting of an affine approximating
function together with a bound on the ``error" in the approximation (this could also be described as a ``remainder bound").  This ``error"
is separate from round-off error,  which will be analyzed in Sections 7 and 8.

\demo{Problems {\rm FIXME(6.2)}}
There are two immediate problems likely to arise from this Taylor approximation approach.
The first problem is the appearance of unpleasant functions such as
Arccosh.  We have already taken care of this problem by ``exponentiating'' our preliminary parameter space ${\cal P}$.  This resulted in all functions under consideration being built up from the co-ordinate functions $ L^{\prime}, D^{\prime},$ and $ R^{\prime}$ on ${\cal W}$ by means of the elementary operations $+,\ -,\ \times,\ /,\ \sqrt.$

Second,  for a given ``built-up function'' the computer needs to be able to compute the Taylor approximation, and the error term.  This will be handled
by developing combination formulas for elementary operations (see the propositions below).  Specifically, given two Taylor
approximations with error terms representing functions $g$ and $h$ and an elementary operation on $g$ and $h$, we will show how to
get the Taylor approximation with error term for the resultant function from the two original Taylor approximations.

A similar approach was developed independently by Stolfi and Figuereido (see [FS]).
\enddemo

{\it Remark} FIXME(6.3).
We set up the Taylor approximation approach rigorously
as follows in Definition FIXME(6.4).
The notation will be a bit unusual, but we are motivated by a desire to stay close to the notation used in the checker computer programs, {\it verify}.  However, it should be pointed out that the formulas in this
section will be superseded by the ones in Section 8, which incorporate a round-off error analysis.  It is the Section 8 formulas that are
used in {\it verify}.

\vglue8pt {\it Definition {\rm FIXME(6.4)}}.  An {\it AffApprox}  $x$ is a five-tuple
$(x.f;\ x.f_0,\ x.f_1,\ x.f_2;\ x.e),$
consisting of four complex numbers $x.f,\ x.f_0,\ x.f_1,\ x.f_2$ and one real number $x.e,$  which represents all functions
$g: A \rightarrow {\bf C}$ such that
$$|g(z_0,z_1,z_2) - (x.f + x.f_0 z_0 + x.f_1 z_1 + x.f _2 z_2)| \le x.e$$
for all $(z_0,z_1,z_2) \in A.$  That is, $x$ represents all functions from $A$ to ${\bf C}$ that are $x.e$-well-approximated by the affine function $x.f + x.f_0 z_0 + x.f_1 z_1 + x.f _2 z_2$.  We will denote this set of functions associated with $x$ by $S(x)$.
\vglue6pt

{\it Remark} 6.5.
As mentioned in Remark FIXME(6.1), given a sub-box to analyze, instead of working with functions defined on the sub-box, we will work with corresponding functions defined on $A.$  Specifically, rather than build up a function by elementary operations performed on the co-ordinate functions
$L^{\prime}, D^{\prime}, R^{\prime}$
restricted to the given sub-box, we will perform the elementary operations on the following functions defined on $A$,
$$(p_0 + i p_3; s_0 + i s_3,0,0; 0)\  \ (p_1 + i p_4; 0, s_1 + i s_4,0;0) \   \ (p_2 + i p_5; 0,0, s_2 + i s_5; 0)$$
where $(p_0 + i p_3, p_1 + i p_4, p_2 + i p_5)$ is the center of the sub-box in question, and the $s_i$ describe the six dimensions of the box. In the computer programs, these three functions are called {\it along}, {\it ortho}, and {\it whirle,} respectively, and $p_i$ and $s_i$
 are denoted {\it pos}[$i$] and {\it size}[$i$], respectively.
\vglue6pt
After the following remarks, we state and prove the combination formulas.

\vglue6pt {\it Remarks} 6.6.
i) In order to co-ordinate numbering with Section 8, we will break with the convention used previously in this paper and start the
numbering of the propositions with FIXME(6.1). However, we will end this section with Example FIXME(6.7).
\vglue2pt
ii)  The negation of a set of functions is the set consisting of the negatives of the original functions, and similarly for other operations.
\vglue2pt
iii)  The propositions that follow include in their statements the definitions of the various operations on AffApproxes.  What needs to
be proved is that the $S$ functions behave as expected.  For example, we need to show that under the definition given for addition, the
set of functions $S(x+y)$ contains all functions obtained by adding a function from $S(x)$ to a function from $S(y).$

\nonumproclaim{Proposition FIXME(6.1) {\rm (unary minus)}} If $x$ is an {\rm AffApprox,} then
$S(-x) = -(S(x))$ where
$$-x \equiv (-x.f;\ -x.f_0,\ -x.f_1,\ -x.f_2;\ x.e).$$
\endproclaim

{\it Proof}.
$$|g(z_0,z_1,z_2) - (x.f + x.f_0 z_0 + x.f_1 z_1 + x.f_2 z_2)| \le e$$
if and only if
\vglue6pt
\hfill $|-g(z_0,z_1,z_2) - (-x.f - x.f_0 z_0 - x.f_1 z_1 - x.f_2 z_2)| \le e.$ \hfill\qed

\nonumproclaim{Proposition FIXME(6.2) {\rm (addition)}} \hskip-8pt If $x$ and $y$ are {\rm AffApproxes,} then $S(x + y) \supseteq S(x)
+ S(y)${\rm ,} where
$$x + y \equiv (x.f + y.f; x.f_0 + y.f_0, x.f_1 + y.f_1, x.f_2 + y.f_2; x.e + y.e).$$
\endproclaim

\demo{Proof}  If $g\in S(x)$ and $h \in S(y)$ then we must show that $g + h \in S(x + y).$
\begin{eqnarray*}
&&|(g + h)(z_0,z_1,z_2) \\
&&\quad- ((x.f + y.f) + (x.f_0 + y.f_0) z_0 + (x.f_1 + y.f_1) z_1 + (x.f_2 + y.f_2) z_2)| \\
&&\qquad
\le |g(z_0,z_1,z_2) -
(x.f + (x.f_0) z_0 + (x.f_1) z_1 + (x.f_2) z_2)| \\
&&\quad\qquad +
|h(z_0,z_1,z_2) - (y.f + (y.f_0) z_0 + (y.f_1) z_1 + (y.f_2) z_2)|
                                        \\
&&\qquad \le x.e + y.e.\\
\noalign{\vskip-36pt}
\end{eqnarray*}
\enddemo

\nonumproclaim{Proposition FIXME(6.3) {\rm (subtraction)}} \hskip-8pt If $x$ and $y$ are {\rm AffApproxes,} then $S(x - y) \supseteq
S(x) - S(y)${\rm ,}
 where
$$x - y \equiv (x.f - y.f; x.f_0 - y.f_0, x.f_1 - y.f_1, x.f_2 - y.f_2; x.e + y.e).$$
\endproclaim

To co-ordinate numbering with Section 8 (where we incorporate round-off error into these formulas) we include the following special
cases of Propositions FIXME(6.2) and FIXME(6.3).  Similarly for Propositions FIXME(6.7, 6.9, and 6.10).

In what follows,  ``double" refers to a real number, and has an associated AffApprox, with the last four entries zero.  When we do
machine arithmetic in Sections FIXME(7 and 8), doubles will be  machine numbers.

\nonumproclaim{Proposition FIXME(6.4) {\rm (addition of an AffApprox and a double)}} If $x$ is an {\rm AffApprox}  and $y$ is a double{\rm ,}
 then $S(x +
y)
\supseteq S(x) + S(y)${\rm ,} where
$$x + y \equiv (x.f + y; x.f_0, x.f_1 , x.f_2 ; x.e).$$
\endproclaim

\nonumproclaim{Proposition FIXME(6.5) {\rm (subtraction of a double from an AffApprox)}}  If $x$ is an {\rm AffApprox} and $y$ is a double{\rm ,}
 then
$S(x - y)
\supseteq S(x) - S(y)${\rm ,} where
$$x - y \equiv (x.f - y; x.f_0, x.f_1 , x.f_2 ; x.e).$$
\endproclaim

\nonumproclaim{Proposition FIXME(6.6) {\rm (multiplication)}}  If $x$ and $y$ are {\rm AffApproxes,}
 then $S(x \times y) \supseteq S(x) \times
S(y)${\rm ,} where
\begin{eqnarray*}
x \times y &\equiv& (x.f \times y.f; x.f \times y.f_0 + x.f_0 \times y.f, \\
&& \quad
x.f \times y.f_1 + x.f_1 \times y.f, x.f \times y.f_2 + x.f_2 \times y.f;
\\
&& \quad
({\rm size}(x) + x.e) \times ({\rm size}(y) + y.e) + (|x.f| \times y.e + x.e \times |y.f|))
                                            \end{eqnarray*}
 with ${\rm size}(x) = |x.f_0| + |x.f_1| + |x.f_2|$ and ${\rm size}(y) = |y.f_0| + |y.f_1| + |y.f_2|.$
\endproclaim

\demo{Proof}  If $g \in S(x)$ and $h \in S(y)$ then we must show that $g\times h \in S(x \times y).$  That is, we need to show
\begin{eqnarray*}
&&\hskip-.25in |(g \times h)(z_0,z_1,z_2)   -
((x.f \times y.f) +
(x.f \times y.f_0 + x.f_0 \times y.f) z_0 \\
&\hskip-8pt+\hskip-8pt& (x.f \times y.f_1 + x.f_1 \times y.f) z_1 + (x.f \times y.f_2 + x.f_2 \times y.f) z_2)|
\\
\le&\hskip-8pt\hskip-8pt&\hskip-16pt ({\rm size}(x) + x.e) \times ({\rm size}(y) + y.e) + (|x.f| \times y.e + x.e \times |y.f|).
\end{eqnarray*}
Note that for any point $(z_0, z_1, z_2) \in A$ and any functions $g \in S(x)$ and $h \in S(y)$ we can find complex numbers $u, v$ with
$|u| \le 1$ and $|v| \le 1$, such that
$$ g(z_0, z_1, z_2) = x.f + (x.f_0 z_0 + x.f_1 z_1 + x.f_2 z_2) + (x.e) u$$ and
$$ h(z_0, z_1, z_2) = y.f + (y.f_0 z_0 + y.f_1 z_1 + y.f_2 z_2) + (y.e) v.$$
Multiplying out, we see that
\begin{eqnarray*}
&&\hskip-36pt(g \times h) (z_0, z_1, z_2)\\
& = &
(x.f \times y.f) +
(x.f \times y.f_0 + x.f_0 \times y.f) z_0\\
&& +\ (x.f \times y.f_1 + x.f_1 \times y.f) z_1 +
(x.f \times y.f_2 + x.f_2 \times y.f) z_2\\&& + \
(x.f \times y.e) v + (x.e \times y.f) u +
((x.f_0 z_0 + x.f_1 z_1 + x.f_2 z_2) + (x.e) u)\\
&& \times\
((y.f_0 z_0 + y.f_1 z_1 + y.f_2 z_2) + (y.e) v).
                                              \end{eqnarray*}
Hence,
\begin{eqnarray*}
&&\hskip-48pt |(g \times h) (z_0, z_1, z_2) - ((x.f \times y.f) \\
& & + \
((x.f \times y.f_0 + x.f_0 \times y.f) z_0\\
&& +\ (x.f \times y.f_1 + x.f_1 \times y.f) z_1 +
(x.f \times y.f_2 + x.f_2 \times y.f) z_2))|
 \\
&\le& (|x.f| y.e + x.e |y.f|) +
({\rm size}(x) + x.e) \times ({\rm size}(y) + y.e).\\
\noalign{\vskip-36pt}
\end{eqnarray*}
\enddemo

\vglue9pt
\nonumproclaim{Proposition FIXME(6.7) {\rm (an AffApprox multiplied by a double)}}
If $x$ is an {\rm AffApprox}  and $y$ is a double{\rm ,} then $S(x \times y) \supseteq S(x) \times S(y)${\rm ,} where
$$ x \times y \equiv (x.f \times y;  x.f_0 \times y,
x.f_1 \times y,  x.f_2 \times y;
x.e \times |y|). $$
\endproclaim

\nonumproclaim{Proposition FIXME(6.8) {\rm (division)}} If $x$ and $y$ are {\rm AffApproxes}  with $|y.f| > {\rm size}(y) + y.e${\rm ,}
 then $S(x / y) \supseteq
S(x) / S(y)${\rm ,} where
\begin{eqnarray*}
x / y &\equiv& (x.f / y.f; (-x.f \times y.f_0 + x.f_0 \times y.f)/((y.f)^2),\\&&\qquad
(-x.f \times y.f_1 + x.f_1 \times y.f)/((y.f)^2),
\\
&&\qquad
(-x.f \times y.f_2 + x.f_2 \times y.f)/((y.f)^2);
\\
&&\qquad (|x.f| + {\rm size}(x) + x.e) /(|y.f| - ({\rm size}(y) + y.e))\\
&&  - \
((|x.f|/|y.f| + {\rm size}(x)/|y.f|) + |x.f| {\rm size}(y)/(|y.f||y.f|))).
                                           \end{eqnarray*}
\endproclaim

\demo{Proof}
For notational convenience, denote $(x.f_0 z_0 + x.f_1 z_1 + x.f_2 z_2)$ by $x.f_k z_k$ and similarly for $y.f_k z_k$ and so on.
As above, note that for any point $(z_0, z_1, z_2) \in A$ and any functions $g \in S(x)$ and $h \in S(y)$ we can find complex numbers $u, v$ with $|u| \le 1$ and $|v| \le 1$, such that
$$ g(z_0, z_1, z_2) = x.f + (x.f_k z_k) + (x.e) u$$ and
$$ h(z_0, z_1, z_2) = y.f + (y.f_k z_k) + (y.e) v.$$

We compare $(g/h)(z_0, z_1, z_2)$ with its putative affine approximation.  That is, we analyze
\begin{eqnarray*}
&&\big|(x.f + (x.f_k z_k) + (x.e) u)/(y.f + (y.f_k z_k) + (y.e) v)\\
&&\qquad\quad - \
((x.f / y.f) + {(x.f_k) y.f - x.f (y.f_k) \over (y.f)^2} z_k)\big|
          . \end{eqnarray*}
Putting this over a common denominator  of
$|((y.f)^2)(y.f + (y.f_k z_k) + (y.e) v)|$
and cancelling equal terms (in the numerator) we are left with a quotient whose numerator is
\begin{eqnarray*} &&
|x.e ((y.f)^2) u - (x.f_k) y.f (y.f_k) z_k - x.f ((y.f_k)^2) z_k\\
&&\qquad\quad  + \
(x.f) y.f (y.e) v + x.f_k (y.f) y.e (v) z_k - x.f (y.f_k) y.e (v) z_k|.
\end{eqnarray*}
We must show this (first) quotient is bounded by
\begin{eqnarray*}
&&
(|x.f| + {\rm size}(x) + x.e) /(|y.f| - ({\rm size}(y) + y.e)) \\
&&\qquad\quad - \
((|x.f|/|y.f| + {\rm size}(x)/|y.f|) + |x.f| {\rm size}(y)/(|y.f||y.f|)).
\end{eqnarray*}
Putting this over a common denominator of
$|(y.f)|^2 (|y.f| - ({\rm size}(y) + y.e))$
and cancelling equal terms (in the numerator) we are left with a second quotient, whose numerator is
$$x.e |y.f|^2 - (-|x.f| |y.f| y.e - {\rm size}(x) |y.f| ({\rm size}(y) + y.e) -
|x.f| {\rm size}(y) ({\rm size}(y) + y.e))$$
and we see that all terms in this numerator are positive.
Further, the terms in the numerators of the first and second quotients correspond in a natural way, and each term in the numerator of the second quotient  is greater than or equal to the absolute value of its corresponding term in the numerator of the first quotient.

Finally,
 because the denominator in the second quotient is less than or equal to the absolute value of the denominator in the first quotient, we
see that the absolute value of the first quotient is less than or equal to the second quotient, as desired. \enddemo

\nonumproclaim{Proposition FIXME(6.9) {\rm (division of a double by an AffApprox)}} If $x$ is a double and $y$ is an {\rm AffApprox }
 with $|y.f| > {\rm size}(y) + y.e${\rm ,} then $S(x / y) \supseteq S(x) / S(y)${\rm ,} where
\begin{eqnarray*}
x / y &\hskip-8pt\equiv\hskip-8pt& (x / y.f; -x \times y.f_0 /((y.f)^2),
-x.f \times y.f_1 /((y.f)^2), -x.f \times y.f_2 /((y.f)^2);
\\
&\hskip-8pt\hskip-8pt&\qquad (|x|  /(|y.f| - ({\rm size}(y) + y.e)) -
(|x|/|y.f|  + |x| {\rm size}(y)/(|y.f||y.f|))).
                                              \end{eqnarray*}
\endproclaim

\nonumproclaim{Proposition FIXME(6.10) {\rm (division of an AffApprox by a double)}} If $x$ is an {\rm AffApprox}  and $y$ is a double
with $|y| > 0${\rm ,} then $S(x / y) \supseteq S(x) / S(y)${\rm ,} where
$$
x / y \equiv (x.f / y; x.f_0 / y,
x.f_1 / y, x.f_2 / y;
x.e/ |y| ).
$$
\endproclaim

Finally, we do the square root.

\nonumproclaim{Proposition FIXME(6.11) {\rm (square root)}} If $x$ is an {\rm AffApprox}  with $|x.f| > {\rm size}(x) + x.e${\rm ,}
 then $S(\sqrt x)
\supseteq
\sqrt {S(x)}${\rm ,} where
\begin{eqnarray*}
\sqrt x& = &\left(\sqrt {x.f};
 {x.f_0 \over 2 \sqrt {x.f}},
 {x.f_1 \over 2 \sqrt {x.f}},
 {x.f_2 \over 2 \sqrt {x.f}};
\right.\\
&& \left. \sqrt {|x.f|} - \left({{\rm size}(x) \over 2 \sqrt {|x.f|}} + \sqrt {|x.f| - ({\rm size}(x) + x.e)}\right)
                                              \right).
\end{eqnarray*}
\endproclaim
If $|x.f| \le {\rm size}(x) + x.e$ then we use the crude estimate $$\left(0;0,0,0;\sqrt{|x.f| + {\rm size}(x) + x.e}\right).$$

The branch of the square root of a complex number is determined by the construction of the square root of a complex in Proposition FIXME(7.14).  In fact, the square root is in the first or fourth quadrant.

\demo{Proof}
As above, note that for any point $(z_0, z_1, z_2) \in A$ and any function $g \in S(x)$ we can find a complex number $u$ with
$|u|   \le 1$, such that
$$ g(z_0, z_1, z_2) = x.f + (x.f_k z_k) + (x.e) u.$$
Also, because $|x.f| > {\rm size}(x) + x.e.$ we see that the argument of
$x.f + (x.f_k z_k) + (x.e) u$ is within $\pi/2$ of the argument of
$x.f$, and therefore, we can require that $\sqrt{g(z_0,z_1,z_2)}$ have argument within $\pi/4$ of the argument of $\sqrt{x.f}.$

We need to show that
\begin{eqnarray*}
&&\hskip-35pt \left|\sqrt {x.f + x.f_k z_k + (x.e) u} - (\sqrt{x.f} + {x.f_k z_k\over 2 \sqrt {x.f}})\right|\\[4pt]
&&\qquad \le \sqrt{|x.f|} - \left({{\rm size}(x) \over 2 \sqrt {|x.f|}} + \sqrt {|x.f| - ({\rm size}(x) + x.e)}\ \right).
\end{eqnarray*}
Or, after we multiply both sides by $\sqrt {|x.f|}$,
\begin{eqnarray*}
&&
\hskip-35pt\left| \sqrt {x.f (x.f + x.f_k z_k + (x.e) u)} - (x.f + (x.f_k) z_k /2) \right|\\
&&\quad \le \left( |x.f| - {\rm size}(x) /2\right) - \sqrt {|x.f|(|x.f| - ({\rm size}(x) +
x.e))}.
\end{eqnarray*}
 The two  sides of the inequality are of the form $ A - B$ and $C - D$, and we
``simplify" by multiplying by
${ A+ B \over A + B}$ and ${ C+ D \over C + D}$.   We now show that the (absolute value of the) left-hand numerator is less than or equal to the right-hand numerator.  Later, we will show that the (absolute value of the) left-hand denominator is larger than or equal to the right-hand denominator.
The   left-hand numerator is
\begin{eqnarray*}
&&\hskip-35pt |x.f (x.f + x.f_k z_k + (x.e) u) - (x.f + (x.f_k) z_k /2)^2|\\[4pt]
&&\qquad
 = |(x.f)^ 2 + x.f (x.f_k) z_k + x.f (x.e) u - (x.f)^2\\[4pt]
&&\qquad\quad - x.f (x.f_k) z_k - ((x.f_k)^2) (z_k)^2/4|\\[4pt]
&&\qquad = |x.f (x.e) u - ((x.f_k)^2) (z_k)^2/4|.
\end{eqnarray*}
The right-hand numerator is
\begin{eqnarray*}
&&(|x.f| - {\rm size}(x)/2)^2 - |x.f| (|x.f| - ({\rm size}(x) + x.e))\\[4pt]
&&\qquad = |x.f|^2 - |x.f| {\rm size}(x) + {\rm size}(x)^2/4 - |x.f|^2 + |x.f| {\rm
size}(x) + |x.f| x.e\\[4pt]
&&\qquad = |x.f| x.e + {\rm size}(x)^2/4.
\end{eqnarray*}
So the left-hand numerator is indeed less than or equal to the right-hand numerator.
\vglue4pt
We now compare the denominators, but only after dividing each by
$\sqrt {|x.f|}$.
The left-hand denominator is
$$\left|\sqrt {x.f + x.f_k z_k + (x.e) u} +
\left(\sqrt {x.f} + {x.f_k z_k\over 2 \sqrt{x.f}}\right)\right|$$
while the right-hand denominator is
$$\sqrt{|x.f|} - {{\rm size}(x) \over 2 \sqrt {|x.f|}} + \sqrt {|x.f| - ({\rm size}(x) + x.e)}.$$
The claim that the left-hand denominator is greater than or equal to the right-hand denominator is a bit complicated.  First, compare the $\sqrt{x.f}$ term and the $\sqrt{|x.f|}$ terms.  They are the same distance from the origin.  Next, note that as $z_k$ and $u$ take on all
relevant values, $x.f + x.f_k z_k + (x.e) u$ describes a disk centered at $x.f$
with radius   less than $ |\sqrt {x.f}|$.  Hence, $\sqrt {x.f + x.f_k z_k + (x.e) u}$ describes a convex set containing $\sqrt{x.f}$.  This set
is symmetric about the line joining the origin and $\sqrt{x.f}$. Further, $\sqrt{x.f} + \sqrt {x.f + x.f_k z_k + (x.e) u}$ describes a convex
set containing $2 \sqrt{x.f}$.   This set is also symmetric about the line joining the origin and  $\sqrt{x.f}$.
It is easy enough to see that no points on this convex symmetric set get closer to the origin than $\sqrt{|x.f|}  + \sqrt {|x.f| - ({\rm size}(x) + x.e)}$.

Finally, because $|{x.f_k z_k \over 2 \sqrt{x.f}}| \le {{\rm size}(x) \over 2 \sqrt {|x.f|}},$ no points of $$\sqrt {x.f} + \sqrt {x.f + x.f_k z_k + (x.e) u} +
{x.f_k z_k \over 2 \sqrt{x.f}}$$
can get closer to the origin than
\vglue12pt
\hfill ${\displaystyle \sqrt{|x.f|} + \sqrt {|x.f| - ({\rm size}(x) + x.e)} -
{{\rm size}(x) \over 2 \sqrt {|x.f|}}.} $ \enddemo

\vglue12pt
{\it Example} FIXME(6.7) (Continuation of Example FIXME(5.4)).
We can now complete the analysis begun in Example FIXME(5.4),
because we can describe $f$ and $w$ as 2-by-2 matrices of AffApproxes.
We note the minor quibble that the full definition of AffApprox is
given in Section 8, where round-off error is incorporated into
the remainder/error-bound term.

For convenience,
we repeat the description of the sub-box under investigation.
The sub-box $Z(s01011)$ with
\begin{small}
$$s = 001000110001110111001111000101111111101111100111001111000001111011110111$$
\end{small}%

\noindent is the region where
$$\left(\begin{array}{rll}
-1.381589027741\ldots &\hskip-6pt \le  {\rm Re}(L')&  \hskip-6pt\le  -1.379848991182\ldots\\[7pt]
-1.378124546093\ldots &\hskip-6pt \le  {\rm Re}(D')&\hskip-6pt  \le  -1.376574349753\ldots\\[7pt]
0.999893182771\ldots  &\hskip-6pt\le {\rm Re}(R')& \hskip-6pt  \le  1.001274250703\ldots\\[7pt]
-2.535837191243\ldots &\hskip-6pt \le  {\rm Im}(L')&\hskip-6pt  \le  -2.534606799593\ldots\\[7pt]
2.535404997792\ldots & \hskip-6pt\le  {\rm Im}(D') &\hskip-6pt \le  -2.534308843448\ldots\\[7pt]
-0.001953125000\ldots &\hskip-6pt \le {\rm  Im}(R')&\hskip-6pt  \le  0.000000000000\ldots
\end{array}\right).$$

For this sub-box, we get (printing only 10 decimal places,
for visual convenience):
\begin{small}
$$  \hskip-12pt
f = \left[\begin{array}{cc}
  \left(\begin{array}{c}
    -0.8677851121   + i 1.4607429651;\cr
   \phantom{-} 0.0000248810   - i 0.0003125810,\\[4pt]
   \phantom{-} 0.0000000000   + i 0.0000000000,\\[4pt]
   \phantom{-} 0.0000000000   + i 0.0000000000;\\[4pt]
   \phantom{-} 0.0000000289
 \end{array}\right)
 &
  \left(\begin{array}{c}
    0.0000000000 + i 0.0000000000;\\[4pt]
    0.0000000000 + i 0.0000000000,\\[4pt]
    0.0000000000 + i 0.0000000000,\\[4pt]
    0.0000000000 + i 0.0000000000;\\[4pt]
    0.0000000000
  \end{array}\right)
 \\[12pt]
  \left(\begin{array}{c}
    0.0000000000 + i 0.0000000000;\\[4pt]
    0.0000000000 + i 0.0000000000,\\[4pt]
    0.0000000000 + i 0.0000000000,\\[4pt]
    0.0000000000 + i 0.0000000000;\\[4pt]
    0.0000000000
  \end{array} \right)
 &
  \left(\begin{array}{c}
    -0.3006023265 - i 0.5060039953;\\[4pt]
    -0.0000909686 - i 0.0000593570,\\[4pt]
   \phantom{-} 0.0000000000 + i 0.0000000000,\\[4pt]
   \phantom{-} 0.0000000000 + i 0.0000000000;\\[4pt]
   \phantom{-} 0.0000000301
  \end{array}\right)
\end{array}\right]
$$\end{small}
and
\begin{small}
$$
\hskip-16pt
w = \left[\begin{array}{cc}
  \left(\begin{array}{c}
    -0.5845111829 + i 0.4773282853;\\[4pt]
   \phantom{-} 0.0000000000 + i 0.0000000000,\\[4pt]
   -0.0000296707 - i 0.0001657332,\\[4pt]
   -0.0004345111 - i 0.0001209539;\\[4pt]
    0.0000002590
  \end{array}\right)
 &
  \left(\begin{array}{c}
    -0.2840228472 + i 0.9825063583;\\[4pt]
  \phantom{-}  0.0000000000 + i 0.0000000000,\\[4pt]
  \phantom{-}  0.0000516606 - i 0.0001128245,\\[4pt]
   \phantom{-} 0.0005776611 - i 0.0001998632;\\[4pt]
    0.0000006462
  \end{array}\right)
 \\[12pt]
  \left(\begin{array}{c}
    -0.2832291572 + i 0.9833572297;\\[4pt]
   \phantom{-} 0.0000000000 + i 0.0000000000,\\[4pt]
   \phantom{-} 0.0000515806 - i 0.0001129408,\\[4pt]
    -0.0005778031 + i 0.0002005440;\\[4pt]
    0.0000002806
  \end{array}\right)
 &
  \left(\begin{array}{c}
    -0.5846352333 + i 0.4764792236;\\[4pt]
   \phantom{-} 0.0000000000 + i 0.0000000000,\\[4pt]
    -0.0000294917 - i 0.0001656653,\\[4pt]
   \phantom{-} 0.0004341392 + i 0.0001213070;\\[4pt]
    0.0000005286
  \end{array}\right)
\end{array}\right].$$
\end{small}
Calculation of
$g = f^{-1}wf^{-1}w^{-1}f^{-1}w^{-1}fw^{-1}f^{-1}w^{-1}f^{-1}wf^{-1}wfww$ gives
\begin{small}
$$ \hskip-18pt
g = \left[\begin{array}{cc}
  \left(\begin{array}{c}
    -0.5764337542 + i 0.4752708071;\\[4pt]
    -0.0031657223 - i 0.0001436786,\\[4pt]
    -0.0017723577 + i 0.0000352928,\\[4pt]
    -0.0011623491 + i 0.0017516088;\\[4pt]
    0.0008229225
  \end{array}\right)
 &
  \left(\begin{array}{c}
    -0.2704033973 + i 0.9822741250;\\[4pt]
    -0.0045902952 - i 0.0019135041,\\[4pt]
    -0.0026219461 - i 0.0007506230,\\[4pt]
    -0.0002823450 + i 0.0033805602;\\[4pt]
    0.0008037640
  \end{array}\right)
 \\[9pt]
  \left(\begin{array}{c}
    -0.2861207992 + i 0.9766064999;\\[4pt]
    -0.0002777968 + i 0.0020330488,\\[4pt]
   \phantom{-} 0.0000837571 + i 0.0010241875,\\[4pt]
   \phantom{-} 0.0028322367 - i 0.0005972336;\\[4pt]
   \phantom{-} 0.0018172437
  \end{array}\right)
 &
  \left(\begin{array}{c}
    -0.5861133046 + i 0.4624368851;\\[4pt]
    -0.0021932627 + i 0.0040523411,\\[4pt]
    -0.0008612361 + i 0.0022394639,\\[4pt]
   \phantom{-} 0.0061581377 - i 0.0005862070;\\[4pt]
    0.0017738513
  \end{array}\right)
\end{array}\right].
$$
\end{small}
We then get
$$
{\rm length}(g) = \left(\begin{array}{c}
    -1.3588762105 - i 2.4897230182;\\[4pt]
  \phantom{-}  0.0030210500 - i 0.0182284729,\\[4pt]
   \phantom{-} 0.0007938572 - i 0.0096614614,\\[4pt]
    -0.0122034521 + i 0.0074353043;\\[4pt]
    0.0080071969
  \end{array}\right)
$$
and
$$
{{\rm length}(g) \over L'} =
  \left(\begin{array}{c}
   \phantom{-} 0.9825397896 - i 0.0008933519;\\[4pt]
   \phantom{-} 0.0053701602 + i 0.0037789019,\\[4pt]
   \phantom{-} 0.0028076072 + i 0.0018421952,\\[4pt]
    -0.0002400615 - i 0.0049443045;\\[4pt]
   \phantom{-} 0.0027802966
  \end{array}\right).
$$

This is not quite good enough to kill the sub-box, since
$|{\rm length}(g)/L'|$ can be as high as $1.0001951323$.

When we subdivide $Z(s01011)$, we have to analyze two sub-boxes,
$Z(s010110)$ and $Z(s010111)$.
For $Z(s010110)$, the same calculation on the region
$$\begin{array}{rll}
-1.381589027741073400 &\hskip-6pt \leq {\rm Re}(L') &\hskip-6pt\leq -1.379848991182205200\\[4pt]
-1.378124546093485700 &\hskip-6pt\leq {\rm Re}(D')
&\hskip-6pt\leq -1.376574349753672900\\[4pt]
0.999893182771602220 &\hskip-6pt\leq {\rm Re}(R')&\hskip-6pt \leq 1.001274250703607400\\[4pt]
-2.535837191243490300&\hskip-6pt \leq {\rm Im}(L')&\hskip-6pt \leq
-2.534606799593201600\\[4pt]
-2.535404997792558600 &\hskip-6pt\leq {\rm Im}(D') &\hskip-6pt\leq -2.534308843448505900\\[4pt]
-0.001953125000000000&\hskip-6pt \leq {\rm Im}(R')
&\hskip-6pt\leq -0.000976562500000000
\end{array}
$$
gives
$$
{{\rm length}(g) \over L'} =
  \left(\begin{array}{c}
 \phantom{-}   0.9814518667 + i 0.0008103446;\\[4pt]
\phantom{-}      0.0053616729 + i 0.0037834001,\\[4pt]
  \phantom{-}    0.0028027236 + i 0.0018435245,\\[4pt]
    -0.0013175066 - i 0.0032448794;\\[4pt]
    0.0019033926
  \end{array}\right),
$$
and we can then bound $\left|{{\rm length}(g) \over L'}\right| \leq 0.9967745579$,
which kills $Z(s010110)$.

On $Z(s010111)$, the calculation gives
$$
{{\rm length}(g) \over L'} =
  \left(\begin{array}{c}
\phantom{-}      0.9836225919 - i 0.0025990177;\\[4pt]
  \phantom{-}    0.0053786346 + i 0.0037743930,\\[4pt]
 \phantom{-}     0.0028124892 + i 0.0018408583,\\[4pt]
    -0.0013333182 - i 0.0032343347;\\[4pt]
    0.0019044429
  \end{array}\right)
$$
and $\left|{{\rm length}(g) \over L'}\right| \leq 0.9989610507$,
which kills $Z(s010111)$.


Train networks to be provably robust (instead of experimentally result as in PGD training).

\paragraph{Opt.:}
$\displaystyle \argmin_\theta \E_{(x, y) \sim D} \left[ \max_{z \in \gamma(\Sharp{\operatorname{NN}}(S(x)))} \loss(\theta; z, y)\right]$

\paragraph{Loss} $\displaystyle \loss(z, y) := \max_{q \neq y} (z_q - z_y) = \max_{q \neq y} (\operatorname{box}(z_q - z_y))$

\paragraph{CE loss} $\loss(z, y) = \operatorname{CE}(z', y)$, with $z'_y := l_y$, $z'_q := u_q$ for $q \neq y$

\paragraph{Universal Approximation} For any neural network, there exists a network with the same properties that can be analyzed exactly with Box.

\paragraph{Complexity} Using complex relaxations generally leads to worse results in provability than with Box (more complex optimization problem)

\paragraph{COLT} For each layer, find $x_l \in S_l$ that maximizes loss in final layer, and use it.

\paragraph{COLT projection} Write zonotope as $Z = A \cdot [-1, 1]^d$, compute $e = A^{-1}\cdot x$, clip $e$ to $[-1, 1]$, projection is $A\cdot e_\text{clip}$.
This projection is \emph{sound} (result inside zonotope), but \emph{not optimal}.


    \filetitle{lognormal}{Create function proportional to log of log-normal distribution}{logdist/lognormal}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
F = logdist.lognormal(Mean,Std)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\item
  \texttt{Mean} {[} numeric {]} - Mean of the log-normal distribution.
\item
  \texttt{Std} {[} numeric {]} - Std dev of the log-normal distribution.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{F} {[} function\_handle {]} - Function handle returning a
  value proportional to the log of the log-normal density.
\end{itemize}

\paragraph{Description}\label{description}

See \href{logdist/Contents}{help on the logdisk package} for details on
using the function handle \texttt{F}.

\paragraph{Example}\label{example}


\section{Sequential colimits}

\emph{Note: This chapter currently contains only the statements of the definitions and theorems, but no proofs. I hope to make a complete version available soon.}

\subsection{The universal property of sequential colimits}

Type sequences are diagrams of the following form.
\begin{equation*}
\begin{tikzcd}
A_0 \arrow[r,"f_0"] & A_1 \arrow[r,"f_1"] & A_2 \arrow[r,"f_2"] & \cdots.
\end{tikzcd}
\end{equation*}
Their formal specification is as follows.

\begin{defn}
An \define{(increasing) type sequence} $\mathcal{A}$ consists of
\begin{align*}
A & : \N\to\UU \\
f & : \prd{n:\N} A_n\to A_{n+1}.
\end{align*}
\end{defn}

In this section we will introduce the sequential colimit of a type sequence.
The sequential colimit includes each of the types $A_n$, but we also identify each $x:A_n$ with its value $f_n(x):A_{n+1}$.
Imagine that the type sequence $A_0\to A_1\to A_2\to\cdots$ defines a big telescope, with $A_0$ sliding into $A_1$, which slides into $A_2$, and so forth.

As usual, the sequential colimit is characterized by its universal property.

\begin{defn}
\begin{enumerate}
\item A \define{(sequential) cocone} on a type sequence $\mathcal{A}$ with vertex $B$ consists of
\begin{align*}
h & : \prd{n:\N} A_n\to B \\
H & : \prd{n:\N} f_n\htpy f_{n+1}\circ H_n.
\end{align*}
We write $\mathsf{cocone}(B)$ for the type of cones with vertex $X$.
\item Given a cone $(h,H)$ with vertex $B$ on a type sequence $\mathcal{A}$ we define the map
\begin{equation*}
\mathsf{cocone\usc{}map}(h,H) : (B\to C)\to \mathsf{cocone}(B)
\end{equation*}
given by $f\mapsto (f\circ h,\lam{n}{x}\mathsf{ap}_f(H_n(x)))$.
\item We say that a cone $(h,H)$ with vertex $B$ is \define{colimiting} if $\mathsf{cocone\usc{}map}(h,H)$ is an equivalence for any type $C$.
\end{enumerate}
\end{defn}

\begin{thm}\label{thm:sequential_up}
Consider a cocone $(h,H)$ with vertex $B$ for a type sequence $\mathcal{A}$. The following are equivalent:
\begin{enumerate}
\item The cocone $(h,H)$ is colimiting.
\item The cocone $(h,H)$ is inductive in the sense that for every type family $P:B\to \UU$, the map
\begin{align*}
\Big(\prd{b:B}P(b)\Big)\to {}& \sm{h:\prd{n:\N}{x:A_n}P(h_n(x))}\\
& \qquad \prd{n:\N}{x:A_n} \mathsf{tr}_P(H_n(x),h_n(x))={h_{n+1}(f_n(x))}
\end{align*}
given by
\begin{equation*}
s\mapsto (\lam{n}s\circ h_n,\lam{n}{x} \mathsf{apd}_{s}(H_n(x)))
\end{equation*}
has a section.
\item The map in (ii) is an equivalence.
\end{enumerate}
\end{thm}

\subsection{The construction of sequential colimits}

We construct sequential colimits using pushouts.

\begin{defn}
Let $\mathcal{A}\jdeq (A,f)$ be a type sequence. We define the type $A_\infty$ as a pushout
\begin{equation*}
\begin{tikzcd}[column sep=large]
\tilde{A}+\tilde{A} \arrow[r,"{[\idfunc,\sigma_{\mathcal{A}}]}"] \arrow[d,swap,"{[\idfunc,\idfunc]}"] & \tilde{A} \arrow[d,"\inr"] \\
\tilde{A} \arrow[r,swap,"\inl"] & A_\infty.
\end{tikzcd}
\end{equation*}
\end{defn}

\begin{defn}
The type $A_\infty$ comes equipped with a cocone structure consisting of
\begin{align*}
\mathsf{seq\usc{}in} & : \prd{n:\N} A_n\to A_\infty \\
\mathsf{seq\usc{}glue} & : \prd{n:\N}{x:A_n} \mathsf{in}_n(x)=\mathsf{in}_{n+1}(f_n(x)).
\end{align*}
\end{defn}

\begin{constr}
We define
\begin{align*}
\mathsf{seq\usc{}in}(n,x)\defeq \inr(n,x) \\
\mathsf{seq\usc{}glue}(n,x)\defeq \ct{\glue(\inl(n,x))^{-1}}{\glue(\inr(n,x))}.
\end{align*}
\end{constr}

\begin{thm}
Consider a type sequence $\mathcal{A}$, and write $\tilde{A}\defeq\sm{n:\N}A_n$. Moreover, consider the map
\begin{equation*}
\sigma_{\mathcal{A}}:\tilde{A}\to\tilde{A}
\end{equation*}
defined by $\sigma_{\mathcal{A}}(n,a)\defeq (n+1,f_n(a))$. Furthermore, consider a cocone $(h,H)$ with vertex $B$.
The following are equivalent:
\begin{enumerate}
\item The cocone $(h,H)$ with vertex $B$ is colimiting.
\item The defining square
\begin{equation*}
\begin{tikzcd}[column sep=large]
\tilde{A}+\tilde{A} \arrow[r,"{[\idfunc,\sigma_{\mathcal{A}}]}"] \arrow[d,swap,"{[\idfunc,\idfunc]}"] & \tilde{A} \arrow[d,"{\lam{(n,x)}h_n(x)}"] \\
\tilde{A} \arrow[r,swap,"{\lam{(n,x)}h_n(x)}"] & A_\infty,
\end{tikzcd}
\end{equation*}
of $A_\infty$ is a pushout square.
\end{enumerate}
\end{thm}

\subsection{Descent for sequential colimits}

\begin{defn}
The type of \define{descent data} on a type sequence $\mathcal{A}\jdeq (A,f)$ is defined to be
\begin{equation*}
\mathsf{Desc}(\mathcal{A}) \defeq \sm{B:\prd{n:\N}A_n\to\UU}\prd{n:\N}{x:A_n}\eqv{B_n(x)}{B_{n+1}(f_n(x))}.
\end{equation*}
\end{defn}

\begin{defn}
We define a map
\begin{equation*}
\mathsf{desc\usc{}fam} : (A_\infty\to\UU)\to\mathsf{Desc}(\mathcal{A})
\end{equation*}
by $B\mapsto (\lam{n}{x}B(\mathsf{seq\usc{}in}(n,x)),\lam{n}{x}\mathsf{tr}_B(\mathsf{seq\usc{}glue}(n,x)))$.
\end{defn}

\begin{thm}
The map
\begin{equation*}
\mathsf{desc\usc{}fam} : (A_\infty\to\UU)\to\mathsf{Desc}(\mathcal{A})
\end{equation*}
is an equivalence.
\end{thm}

\begin{defn}
A \define{cartesian transformation} of type sequences from $\mathcal{A}$ to $\mathcal{B}$ is a pair $(h,H)$ consisting of
\begin{align*}
h & : \prd{n:\N} A_n\to B_n \\
H & : \prd{n:\N} g_n\circ h_n \htpy h_{n+1}\circ f_n,
\end{align*}
such that each of the squares in the diagram
\begin{equation*}
\begin{tikzcd}
A_0 \arrow[d,swap,"h_0"] \arrow[r,"f_0"] & A_1 \arrow[d,swap,"h_1"] \arrow[r,"f_1"] & A_2 \arrow[d,swap,"h_2"] \arrow[r,"f_2"] & \cdots \\
B_0 \arrow[r,swap,"g_0"] & B_1 \arrow[r,swap,"g_1"] & B_2 \arrow[r,swap,"g_2"] & \cdots
\end{tikzcd}
\end{equation*}
is a pullback square. We define
\begin{align*}
\mathsf{cart}(\mathcal{A},\mathcal{B}) & \defeq\sm{h:\prd{n:\N}A_n\to B_n} \\
& \qquad\qquad \sm{H:\prd{n:\N}g_n\circ h_n\htpy h_{n+1}\circ f_n}\prd{n:\N}\mathsf{is\usc{}pullback}(h_n,f_n,H_n),
\end{align*}
and we write
\begin{equation*}
\mathsf{Cart}(\mathcal{B}) \defeq \sm{\mathcal{A}:\mathsf{Seq}}\mathsf{cart}(\mathcal{A},\mathcal{B}).
\end{equation*}
\end{defn}

\begin{defn}
We define a map
\begin{equation*}
\mathsf{cart\usc{}map}(\mathcal{B}) : \Big(\sm{X':\UU}X'\to X\Big)\to\mathsf{Cart}(\mathcal{B}).
\end{equation*}
which associates to any morphism $h:X'\to X$ a cartesian transformation of type sequences into $\mathcal{B}$.
\end{defn}

\begin{thm}
The operation $\mathsf{cart\usc{}map}(\mathcal{B})$ is an equivalence.
\end{thm}

\subsection{The flattening lemma for sequential colimits}

The flattening lemma for sequential colimits essentially states that sequential colimits commute with $\Sigma$.

\begin{lem}
Consider
\begin{align*}
B & : \prd{n:\N}A_n\to\UU \\
g & : \prd{n:\N}{x:A_n}\eqv{B_n(x)}{B_{n+1}(f_n(x))}.
\end{align*}
and suppose $P:A_\infty\to\UU$ is the unique family equipped with
\begin{align*}
e & : \prd{n:\N}\eqv{B_n(x)}{P(\mathsf{seq\usc{}in}(n,x))}
\end{align*}
and homotopies $H_n(x)$ witnessing that the square
\begin{equation*}
\begin{tikzcd}[column sep=7em]
B_n(x) \arrow[r,"g_n(x)"] \arrow[d,swap,"e_n(x)"] & B_{n+1}(f_n(x)) \arrow[d,"e_{n+1}(f_n(x))"] \\
P(\mathsf{seq\usc{}in}(n,x)) \arrow[r,swap,"{\mathsf{tr}_P(\mathsf{seq\usc{}glue}(n,x))}"] & P(\mathsf{seq\usc{}in}(n+1,f_n(x)))
\end{tikzcd}
\end{equation*}
commutes. Then $\sm{t:A_\infty}P(t)$ satisfies the universal property of the sequential colimit of the type sequence
\begin{equation*}
\begin{tikzcd}
\sm{x:A_0}B_0(x) \arrow[r,"{\tot[f_0]{g_0}}"] & \sm{x:A_1}B_1(x) \arrow[r,"{\tot[f_1]{g_1}}"] & \sm{x:A_2}B_2(x) \arrow[r,"{\tot[f_2]{g_2}}"] & \cdots.
\end{tikzcd}
\end{equation*}
\end{lem}

In the following theorem we rephrase the flattening lemma in using cartesian transformations of type sequences.

\begin{thm}
Consider a commuting diagram of the form
\begin{equation*}
\begin{tikzcd}[column sep=small,row sep=small]
A_0 \arrow[rr] \arrow[dd] & & A_1 \arrow[rr] \arrow[dr] \arrow[dd] &[-.9em] &[-.9em] A_2 \arrow[dl] \arrow[dd] & & \cdots \\
& & & X \arrow[from=ulll,crossing over] \arrow[from=urrr,crossing over] \arrow[from=ur,to=urrr] \\
B_0 \arrow[rr] \arrow[drrr] & & B_1 \arrow[rr] \arrow[dr] & & B_2 \arrow[rr] \arrow[dl] & & \cdots \arrow[dlll] \\
& & & Y \arrow[from=uu,crossing over]
\end{tikzcd}
\end{equation*}
If each of the vertical squares is a pullback square, and $Y$ is the sequential colimit of the type sequence $B_n$, then $X$ is the sequential colimit of the type sequence $A_n$.
\end{thm}

\begin{cor}
Consider a commuting diagram of the form
\begin{equation*}
\begin{tikzcd}[column sep=small,row sep=small]
A_0 \arrow[rr] \arrow[dd] & & A_1 \arrow[rr] \arrow[dr] \arrow[dd] &[-.9em] &[-.9em] A_2 \arrow[dl] \arrow[dd] & & \cdots \\
& & & X \arrow[from=ulll,crossing over] \arrow[from=urrr,crossing over] \arrow[from=ur,to=urrr] \\
B_0 \arrow[rr] \arrow[drrr] & & B_1 \arrow[rr] \arrow[dr] & & B_2 \arrow[rr] \arrow[dl] & & \cdots \arrow[dlll] \\
& & & Y \arrow[from=uu,crossing over]
\end{tikzcd}
\end{equation*}
If each of the vertical squares is a pullback square, then the square
\begin{equation*}
\begin{tikzcd}
A_\infty \arrow[r] \arrow[d] & X \arrow[d] \\
B_\infty \arrow[r] & Y
\end{tikzcd}
\end{equation*}
is a pullback square.
\end{cor}

\subsection{Constructing the propositional truncation}\label{sec:propositional-truncation-constr}
The propositional truncation can be used to construct the image of a map, so we construct that first. We construct the propositional truncation of $A$ via a construction called the \define{join construction}, as the colimit of the sequence of join-powers of $A$
\begin{equation*}
  \begin{tikzcd}
    A \arrow[r] & \join{A}{A} \arrow[r] & \join{A}{(\join{A}{A})} \arrow[r] & \cdots
  \end{tikzcd}
\end{equation*}
The join-powers of $A$ are defined recursively on $n$, by taking\footnote{In this definition, the case $A^{\ast1}\defeq A$ is slightly redundant because we have an equivalence
\begin{equation*}
  \join{A}{\emptyt}\simeq A.
\end{equation*}
Nevertheless, it is nice to have that $A^{\ast 1}\jdeq A$.}
\begin{align*}
  A^{\ast0} & \defeq \emptyt \\
  A^{\ast 1} & \defeq A \\
  A^{\ast(n+2)} & \defeq \join{A}{A^{\ast (n+1)}}.
\end{align*}
We will write $A^{\ast\infty}$ for the colimit of the sequence
\begin{equation*}
  \begin{tikzcd}
    A \arrow[r,"\inr"] & \join{A}{A} \arrow[r,"\inr"] & \join{A}{(\join{A}{A})} \arrow[r,"\inr"] & \cdots.
  \end{tikzcd}
\end{equation*}
The sequential colimit $A^{\ast\infty}$ comes equipped with maps $\inseq_n:A^{\ast (n+1)}\to A^{\ast\infty}$, and we will write
\begin{equation*}
  \eta\defeq\inseq_0:A\to A^{\ast\infty}.
\end{equation*}
Our goal is to show $A^{\ast\infty}$ is a proposition, and that $\eta:A\to A^{\ast\infty}$ satisfies the universal property of the propositional truncation of $A$. Before showing that $A^{\ast\infty}$ is indeed a proposition, let us show in two steps that for any proposition $P$, the map
\begin{equation*}
  (A^{\ast\infty}\to P)\to (A\to P)
\end{equation*}
is indeed an equivalence.

\begin{lem}\label{lem:extend_join_prop}
Suppose $f:A\to P$, where $A$ is any type and $P$ is a proposition.
Then the precomposition function
\begin{equation*}
\blank\circ\inr:(\join{A}{B}\to P)\to (B\to P)
\end{equation*}
is an equivalence, for any type $B$.
\end{lem}

\begin{proof}
  Since the precomposition function
  \begin{equation*}
    \blank\circ\inr:(\join{A}{B}\to P)\to (B\to P)
  \end{equation*}
  is a map between propositions, it suffices to construct a map
  \begin{equation*}
    (B\to P)\to (\join{A}{B}\to P).
  \end{equation*}
  Let $g:B\to P$. Then the square
  \begin{equation*}
    \begin{tikzcd}
      A\times B \arrow[r,"\proj 2"] \arrow[d,swap,"\proj 1"] & B \arrow[d,"g"] \\
      A \arrow[r,swap,"f"] & P
    \end{tikzcd}
  \end{equation*}
  commutes since $P$ is a proposition. Therefore we obtain a map $\join{A}{B}\to P$ by the universal property of the join.
\end{proof}

\begin{prp}\label{prp:universal-property-brck}
Let $A$ be a type, and let $P$ be a proposition. Then the function
\begin{equation*}
\blank\circ \eta : (A^{\ast\infty}\to P)\to (A\to P)
\end{equation*}
is an equivalence.
\end{prp}

\begin{proof}
  Since the map
  \begin{equation*}
    \blank\circ \eta : (A^{\ast\infty}\to P)\to (A\to P)
  \end{equation*}
  is a map between propositions, it suffices to construct a map in the converse direction.

  Let $f:A\to P$. First, we show by recursion that there are maps
  \begin{equation*}
    f_n:A^{\ast(n+1)}\to P.
  \end{equation*}
  The map $f_0$ is of course just defined to be $f$. Given $f_n:A^{\ast(n+1)}$ we obtain $f_{n+1}:\join{A}{A^{\ast(n+1)}}\to P$ by \cref{lem:extend_join_prop}. Because $P$ is assumed to be a proposition it is immediate that the maps $f_n$ form a cocone with vertex $P$ on the sequence
  \begin{equation*}
    \begin{tikzcd}
      A \arrow[r,"\inr"] & \join{A}{A} \arrow[r,"\inr"] & \join{A}{(\join{A}{A})} \arrow[r,"\inr"] & \cdots.
    \end{tikzcd}
  \end{equation*}
  From this cocone we obtain the desired map $(A^{\ast\infty}\to P)$.
\end{proof}

\begin{prp}\label{prp:isprop-infjp}
The type $A^{\ast\infty}$ is a proposition for any type $A$.
\end{prp}

\begin{proof}
  By \cref{lem:isprop_eq} it suffices to show that
  \begin{equation*}
    A^{\ast\infty}\to \iscontr(A^{\ast\infty}).
  \end{equation*}
  Since the type $\iscontr(A^{\ast\infty})$ is already known to be a proposition by \cref{ex:isprop_istrunc}, it follows from \cref{prp:universal-property-brck} that it suffices to show that
\begin{equation*}
A\to \iscontr(A^{\ast\infty}).
\end{equation*}

Let $x:A$. To see that $A^{\ast\infty}$ is contractible it suffices by \cref{ex:seqcolim_contr} to show that $\inr:A^{\ast n}\to A^{\ast(n+1)}$ is homotopic to the constant function $\const_{\inl(x)}$. However, we get a homotopy $\const_{\inl(x)}\htpy \inr$ immediately from the path constructor $\glue$.
\end{proof}

All the definitions are now in place to define the propositional truncation of a type.

\begin{defn}
  For any type $A$ we define the type
  \begin{equation*}
    \trunc{-1}{A}\defeq A^{\ast\infty},
  \end{equation*}
  and we define $\eta:A\to\trunc{-1}{A}$ to be the constructor $\seqin_0$ of the sequential colimit $A^{\ast\infty}$. Often we simply write $\brck{A}$ for $\trunc{-1}{A}$.
\end{defn}

The type $\trunc{-1}{A}$ is a proposition by \cref{prp:isprop-infjp}, and
\begin{equation*}
  \eta:A\to\trunc{-1}{A}
\end{equation*}
satisfies the universal property of propositional truncation by \cref{prp:universal-property-brck}.

\begin{prp}
  The propositional truncation operation is functorial in the sense that for any map $f:A\to B$ there is a unique map $\brck{f}:\brck{A}\to\brck{B}$ such that the square
  \begin{equation*}
    \begin{tikzcd}
      A \arrow[r,"f"] \arrow[d,swap,"\eta"] & B \arrow[d,"\eta"] \\
      \brck{A} \arrow[r,swap,"\brck{f}"] & \brck{B}
    \end{tikzcd}
  \end{equation*}
  commutes. Moreover, there are homotopies
  \begin{align*}
    \brck{\idfunc[A]} & \htpy \idfunc[\brck{A}] \\
    \brck{g\circ f} & \htpy \brck{g}\circ\brck{f}.
  \end{align*}
\end{prp}

\begin{proof}
  The functorial action of propositional truncation is immediate by the universal property of propositional truncation. To see that the functorial action preserves the identity, note that the type of maps $\brck{A}\to\brck{A}$ for which the square
  \begin{equation*}
    \begin{tikzcd}
      A \arrow[r,"\idfunc"] \arrow[d,swap,"\eta"] & A \arrow[d,"\eta"] \\
      \brck{A} \arrow[r,densely dotted] & \brck{A}
    \end{tikzcd}
  \end{equation*}
  commutes is contractible. Since this square commutes for both $\brck{\idfunc}$ and for $\idfunc$, it must be that they are homotopic. The proof that the functorial action of propositional truncation preserves composition is similar.
\end{proof}

\subsection{Proving type theoretical replacement}

Our goal is now to show that the image of a map $f:A\to B$ from an essentially small type $A$ into a locally small type $B$ is again essentially small. This property is called the type theoretic replacement property. In order to prove this property, we have to find another construction of the image of a map. In order to make this construction, we define a join operation on maps.

\begin{defn}
  Consider two maps $f:A\to X$ and $g:B\to X$ with a common codomain $X$.
  \begin{enumerate}
  \item We define the type $\join[X]{A}{B}$ as the pushout
    \begin{equation*}
      \begin{tikzcd}
        A\times_X B \arrow[r,"\pi_2"] \arrow[d,swap,"\pi_1"] & B \arrow[d,"\inr"] \\
        A \arrow[r,swap,"\inl"] & \join[X]{A}{B}.
      \end{tikzcd}
    \end{equation*}
  \item We define the \define{join} $\join{f}{g}:\join[X]{A}{B}\to X$ to be the unique map for which the diagram
        \begin{equation*}
      \begin{tikzcd}
        A\times_X B \arrow[r,"\pi_2"] \arrow[d,swap,"\pi_1"] & B \arrow[d,"\inr"] \arrow[ddr,bend left=15,"g"] \\
        A \arrow[r,swap,"\inl"] \arrow[drr,bend right=15,swap,"f"]  & \join[X]{A}{B} \arrow[dr,densely dotted,swap,"\join{f}{g}"] \\
        & & X
      \end{tikzcd}
    \end{equation*}
  \end{enumerate}
\end{defn}

The reason to call the map $\join{f}{g}$ the join of $f$ and $g$ is that the fiber of $\join{f}{g}$ at any $x:X$ is equivalent to the join of the fibers of $f$ and $g$ at $x$.

\begin{lem}
  Consider two maps $f:A\to X$ and $g:B\to X$. Then there is an equivalence
  \begin{equation*}
    \fib{\join{f}{g}}{x}\simeq\join{\fib{f}{x}}{\fib{g}{x}}
  \end{equation*}
  for any $x:X$.
\end{lem}

\begin{proof}
  Consider the commuting cube
  \begin{equation*}
    \begin{tikzcd}
      & \fib{f}{x}\times\fib{g}{x} \arrow[dl] \arrow[dr] \arrow[d] \\
      \fib{f}{x} \arrow[d] & A\times_X B \arrow[dl] \arrow[dr] & \fib{g}{x} \arrow[d] \arrow[dl,crossing over] \\
      A \arrow[dr] & \unit \arrow[from=ul,crossing over] \arrow[d] & B \arrow[dl] \\
      & X
    \end{tikzcd}
  \end{equation*}
  In this cube, the bottom square is a canonical pullback square. The two squares in the front are pullbacks by \cref{lem:fib_pb}, and the top square is a pullback square by \cref{lem:prod_pb}. Therefore it follows by \cref{rmk:strongly-cartesian} that all the faces of this cube are pullback squares, and hence by \cref{thm:effectiveness-pullback} we obtain that the square
  \begin{equation*}
    \begin{tikzcd}
      \join{\fib{f}{x}}{\fib{g}{x}} \arrow[d,densely dotted] \arrow[r] & \unit \arrow[d] \\
      \join[X]{A}{B} \arrow[r,swap,"\join{f}{g}"] & X
    \end{tikzcd}
  \end{equation*}
  is a pullback square. Now the claim follows by the uniqueness of pullbacks, which was shown in \cref{cor:uniquely-unique-pullback}.
\end{proof}

\begin{lem}
Consider a map $f:A\to X$, an embedding $m:U\to X$, and $h:\mathrm{hom}_X(f,m)$. Then the map
\begin{equation*}
\mathrm{hom}_X(\join{f}{g},m)\to \mathrm{hom}_X(g,m)
\end{equation*}
is an equivalence for any $g:B\to X$.
\end{lem}

\begin{proof}
Note that both types are propositions, so any equivalence can be used to prove the claim. Thus, we simply calculate
\begin{align*}
\mathrm{hom}_X(\join{f}{g},m) & \eqvsym \prd{x:X}\fib{\join{f}{g}}{x}\to \fib{m}{x} \\
& \eqvsym \prd{x:X}\join{\fib{f}{x}}{\fib{g}{x}}\to\fib{m}{x} \\
& \eqvsym \prd{x:X}\fib{g}{x}\to\fib{m}{x} \\
& \eqvsym \mathrm{hom}_X(g,m).
\end{align*}
The first equivalence holds by \cref{ex:triangle_fib}; the second equivalence holds by \cref{ex:fib_join}, also using \cref{ex:equiv_precomp,lem:postcomp_equiv} where we established that that pre- and postcomposing by an equivalence is an equivalence; the third equivalence holds by \cref{lem:extend_join_prop,lem:postcomp_equiv}; the last equivalence again holds by \cref{ex:triangle_fib}.
\end{proof}

For the construction of the image of $f:A\to X$ we observe that if we are given an embedding $m:U\to X$ and a map $(i,I):\mathrm{hom}_X(f,m)$, then $(i,I)$ extends uniquely along $\inr:A\to \join[X]{A}{A}$ to a map $\mathrm{hom}_X(\join{f}{f},m)$. This extension again extends uniquely along $\inr:\join[X]{A}{A}\to \join[X]{A}{(\join[X]{A}{A})}$ to a map $\mathrm{hom}_X(\join{f}{(\join{f}{f})},m)$ and so on, resulting in a diagram of the form
\begin{equation*}
\begin{tikzcd}
A \arrow[dr] \arrow[r,"\inr"] & \join[X]{A}{A} \arrow[d,densely dotted] \arrow[r,"\inr"] & \join[X]{A}{(\join[X]{A}{A})} \arrow[dl,densely dotted] \arrow[r,"\inr"] & \cdots \arrow[dll,densely dotted,bend left=10] \\
& U
\end{tikzcd}
\end{equation*}

\begin{defn}
Suppose $f:A\to X$ is a map. Then we define the \define{fiberwise join powers}
\begin{equation*}
f^{\ast n}:A_X^{\ast n} X.
\end{equation*}
\end{defn}

\begin{constr}
Note that the operation $(B,g)\mapsto (\join[X]{A}{B},\join{f}{g})$ defines an endomorphism on the type
\begin{equation*}
\sm{B:\UU}B\to X.
\end{equation*}
We also have $(\emptyt,\ind{\emptyt})$ and $(A,f)$ of this type. For $n\geq 1$ we define
\begin{align*}
A_X^{\ast (n+1)} & \defeq \join[X]{A}{A_X^{\ast n}} \\
f^{\ast (n+1)} & \defeq \join{f}{f^{\ast n}}.\qedhere
\end{align*}
\end{constr}

\begin{defn}
We define $A_X^{\ast\infty}$ to be the sequential colimit of the type sequence
\begin{equation*}
\begin{tikzcd}
A_X^{\ast 0} \arrow[r] & A_X^{\ast 1} \arrow[r,"\inr"] & A_X^{\ast 2} \arrow[r,"\inr"] & \cdots.
\end{tikzcd}
\end{equation*}
Since we have a cocone
\begin{equation*}
\begin{tikzcd}
A_X^{\ast 0} \arrow[r] \arrow[dr,swap,"f^{\ast 0}" near start] & A_X^{\ast 1} \arrow[r,"\inr"] \arrow[d,swap,"f^{\ast 1}" near start] & A_X^{\ast 2} \arrow[r,"\inr"] \arrow[dl,swap,"f^{\ast 2}" xshift=1ex] & \cdots \arrow[dll,bend left=10] \\
& X
\end{tikzcd}
\end{equation*}
we also obtain a map $f^{\ast\infty}:A_X^{\ast\infty}\to X$ by the universal property of $A_X^{\ast\infty}$.
\end{defn}

\begin{lem}\label{lem:finfjp_up}
Let $f:A\to X$ be a map, and let $m:U\to X$ be an embedding. Then the function
\begin{equation*}
\blank\circ \seqin_0: \mathrm{hom}_X(f^{\ast\infty},m)\to \mathrm{hom}_X(f,m)
\end{equation*}
is an equivalence.
\end{lem}

\begin{thm}\label{lem:isprop_infjp}
For any map $f:A\to X$, the map $f^{\ast\infty}:A_X^{\ast\infty}\to X$ is an embedding that satisfies the universal property of the image inclusion of $f$.
\end{thm}

\begin{lem}
Consider a commuting square
\begin{equation*}
\begin{tikzcd}
A \arrow[r] \arrow[d] & B \arrow[d] \\
C \arrow[r] & D.
\end{tikzcd}
\end{equation*}
\begin{enumerate}
\item If the square is cartesian, $B$ and $C$ are essentially small, and $D$ is locally small, then $A$ is essentially small.
\item If the square is cocartesian, and $A$, $B$, and $C$ are essentially small, then $D$ is essentially small.
\end{enumerate}
\end{lem}

\begin{cor}
Suppose $f:A\to X$ and $g:B\to X$ are maps from essentially small types $A$ and $B$, respectively, to a locally small type $X$. Then $A\times_X B$ is again essentially small.
\end{cor}

\begin{lem}
Consider a type sequence
\begin{equation*}
\begin{tikzcd}
A_0 \arrow[r,"f_0"] & A_1 \arrow[r,"f_1"] & A_2 \arrow[r,"f_2"] & \cdots
\end{tikzcd}
\end{equation*}
where each $A_n$ is essentially small. Then its sequential colimit is again essentially small.
\end{lem}

\begin{thm}\label{thm:replacement}
  For any map $f:A\to B$ from an essentially small type $A$ into a locally small type $B$, the image of $f$ is again essentially small.
\end{thm}

\begin{cor}
  Consider a $\UU$-small type $A$, and an equivalence relation $R$ over $A$ valued in the $\UU$-small propositions. Then the set quotient $A/R$ is essentially small.
\end{cor}

\begin{exercises}
\exercise \label{ex:seqcolim_shift}
Show that the sequential colimit of a type sequence
\begin{equation*}
\begin{tikzcd}
A_0 \arrow[r,"f_0"] & A_1 \arrow[r,"f_1"] & A_2 \arrow[r,"f_2"] & \cdots
\end{tikzcd}
\end{equation*}
is equivalent to the sequential colimit of its shifted type sequence
\begin{equation*}
\begin{tikzcd}
A_1 \arrow[r,"f_1"] & A_2 \arrow[r,"f_2"] & A_3 \arrow[r,"f_3"] & \cdots.
\end{tikzcd}
\end{equation*}
  \exercise Let
  \begin{tikzcd}
    P_0 \arrow[r] & P_1 \arrow[r] & P_2 \arrow[r] & \cdots
  \end{tikzcd}
  be a sequence of propositions. Show that
  \begin{equation*}
    \eqv{\colim_n(P_n)}{\exists_{(n:\N)} P_n}.
  \end{equation*}
\exercise \label{ex:seqcolim_contr}Consider a type sequence
\begin{equation*}
\begin{tikzcd}
A_0 \arrow[r,"f_0"] & A_1 \arrow[r,"f_1"] & A_2 \arrow[r,"f_2"] & \cdots
\end{tikzcd}
\end{equation*}
and suppose that $f_n\htpy \mathsf{const}_{a_{n+1}}$ for some $a_n:\prd{n:\N}A_n$. Show that the sequential colimit is contractible.
\exercise Define the $\infty$-sphere $\sphere{\infty}$ as the sequential colimit of
\begin{equation*}
\begin{tikzcd}
\sphere{0} \arrow[r,"f_0"] & \sphere{1} \arrow[r,"f_1"] & \sphere{2} \arrow[r,"f_2"] & \cdots
\end{tikzcd}
\end{equation*}
where $f_0:\sphere{0}\to\sphere{1}$ is defined by $f_0(\bfalse)\jdeq \inl(\ttt)$ and $f_0(\btrue)\jdeq \inr(\ttt)$, and $f_{n+1}:\sphere{n+1}\to\sphere{n+2}$ is defined as $\susp(f_n)$. Use \cref{ex:seqcolim_contr} to show that $\sphere{\infty}$ is contractible.
\exercise Consider a type sequence
\begin{equation*}
\begin{tikzcd}
A_0 \arrow[r,"f_0"] & A_1 \arrow[r,"f_1"] & A_2 \arrow[r,"f_2"] & \cdots
\end{tikzcd}
\end{equation*}
in which $f_n:A_n\to A_{n+1}$ is weakly constant in the sense that
\begin{equation*}
\prd{x,y:A_n} f_n(x)=f_n(y)
\end{equation*}
Show that $A_\infty$ is a mere proposition.
\exercise Show that $\N$ is the sequential colimit of
\begin{equation*}
  \begin{tikzcd}
    \Fin(0) \arrow[r,"\inl"] & \Fin(1) \arrow[r,"\inl"] & \Fin(2) \arrow[r,"\inl"] & \cdots.
  \end{tikzcd}
\end{equation*}
\end{exercises}


\chapter{Hypothesis testing} % Casella berger, S373
\section{Introduction}
This chapter explains hypothesis testing and P-values. A hypothesis is defined as below.
\begin{defn}
A hypothesis is a statement about a population parameter.
\end{defn}
Furthermore one tests two hypotheses against each other. A definiton of this is as follows.
\begin{defn}
The two complementary hypotheses in a hypothesis testing problem are called the null hypothesis and the alternative hypothesis. They are denoted by $H_0$ and $H_1$, respectivly.
\end{defn}
To do a hypothesis test one need to calculate a test statistic from data. For a one sided test one can assume that the data is outside the null model for large values of the test statistic. In the case of a two sided test the data can be outside the null hypothesis for small values of the test statistic too. This leads to calculation of p-values
\section{P-values} % s397
A p-value can give the result of a hypothesis test. The p-value is defined in definiton ??.
\begin{theorem}
Let $W(\boldsymbol{X})$ be a test statistic such that large values of $W$ give evidence that $H_1$ is true. For each sample point $\boldsymbol{x}$, define
\begin{equation}
p(\boldsymbol{x}) = \sup_{\theta \in \Theta_0} P_\theta (W(\boldsymbol{X}) \geq W(\boldsymbol{x})).
\end{equation}
Then, $p(\boldsymbol{X})$ is a valid p-value.
\end{theorem}
The $\Theta_0$ is the subset of the parameter space for the null model.
\\
\\
P-values can also be defined by using sufficient statistics. A p-value is then defined as
\begin{equation}
p(\boldsymbol{x}) = P(W(\boldsymbol{X}) \geq W(\boldsymbol{x}) | S = S(\boldsymbol{x})),
\label{eq:pvalue}
\end{equation}
where $S(\boldsymbol{x})$ is a sufficient statistic. A valid p-value is defined by
\begin{defn}
A p-value $p(\boldsymbol{X})$ is a test statistic satisfying $0 \leq p(\boldsymbol{x}) \leq 1$ for every sample point $\boldsymbol{x}$. Small values of $p(\boldsymbol{X})$ give evidence that $H_1$ is true. A p-value is valid if, for every $\theta \in \Theta_0$ and every $0 \leq \alpha \leq 1$,
\begin{equation}
P_\theta (p(\boldsymbol{X}\leq \alpha)) \leq \alpha.
\end{equation}
\label{defn:validpvalue}
\end{defn}
By this definition the p-value given a sufficient statistic is valid as shown below
\begin{equation}
P_\theta(p(\boldsymbol{X}) \leq \alpha ) = \sum_x P(p(\boldsymbol{X}) \leq \alpha | s =s) P_\theta (S=s) \leq \sum_s \alpha P_\theta (S=) \leq \alpha.
\end{equation}
\\
\\ % http://www.math.ntnu.no/~bo/Fordypning/2015/Marius/NHPP-Iran-paper.pdf
\section{Test statistics}
\label{sec:teststat}
As mentioned earlier to calculate the p-value one need a test statistic. A test statistic is often a function which takes a sample as input and gives out a meassure of an certain attribute.For our data we will use several types of test statistics. In the test statistics we are using tranformed times of our events. This is has also been done in \cite{lindqvist2011monte}. The transformed times have an uniform distribution between 0 and 1. This is because if the times $t_1,...,t_n$ is from a NHPP then $\Lambda(t_1),...,\Lambda(t_1)$ is a homogenous Poisson process with intensity 1. The tranformed times are defined as below.
\begin{equation}
V_j = \frac{\Lambda(t_j)}{\Lambda(\tau)}.
\label{eq:transtimes}
\end{equation}
The different statistics will be introduced below.
\subsection{Greenwood statistic}
The Greenwood statistic is a two sided statistic where values are rejected for small and large values.
\begin{equation}
G = \sum_{j=1}^{n+1} (\hat{V_j} - \hat{V}_{j-1})^2
\end{equation}
\subsection{Laplace statistic}
This one is also a two sided statistic and are rejected for small and large values.
\begin{equation}
L = \sqrt{\frac{12}{n}} \sum_{j=1}^{n+1} \left( \hat{V_j} - \frac{1}{2} \right)
\end{equation}
\subsection{Modified Cramer von Mises statistic}
\begin{equation}
W^2 = \sum_{j=1}^{n} \left[ \hat{V}_j - \frac{(2j-1)}{2n} \right]^2 + \frac{1}{12n}
\end{equation}
\subsection{Modified Kolmogorov-Smirnov statistic}
\begin{equation}
D = max[D^+ , D^-]
\end{equation}
\begin{equation}
D^+ = \max_{1 \leq j \leq n} \left( \frac{j}{n} - \hat{V}_j \right)
\end{equation}
\begin{equation}
D^- = \max_{1 \leq j \leq n} \left( \hat{V}_j - \frac{(j-1)}{n} \right)
\end{equation}
\\
\\
Both of the Modified Cramer von Mises statistic and Modified Kolmogorov:Smirnov statistic rejects the null hypothesis for large values of the statistic.

\section{Test statistics and p-values in NHPP}
To be able to use the test statistics in section \ref{sec:teststat} we need to find the tranformed times as defined in equation \ref{eq:transtimes}. To do this the parameters for the rate function can be estimated. These paramters can be found by using maximum likelihood estimators as defined in chapter \ref{chap:like}. From equation \ref{eq:loglike} we see that finding the maximum likelihood might not be trivial because of the large lambda term. However the maximum likelihood can be found numerically in R programming language by using the built in function Optim. Furthermore the integral in equation \ref{eq:largelambda} can also be solved numerically. Hence the tranformed times and test statistics can be calculated.
\\
\\
To calculate the p-value in equation \ref{eq:pvalue} we refer to \cite{iranNHPP}. From this paper we have that the p-value can be estimated by
\begin{equation}
\hat{p} = \#{W^* \geq W_{obs}}/M,
\end{equation}
where $W*$ is a test statistic for a sample, $W_{obs}$ and $M$ is the number of samples.


\section{Generic Programming} \label{sec:generic_programming}

Ever since the humble beginnings of \Cpp\ there's been extensive support for different kinds of \emph{polymorphism}, which can be split into two ``main'' categories: \emph{run-time} polymorphism and \emph{compile-time} polymorphism. The former is a key component in class-based OOP, and takes the form of inheritance, while the latter one enables \emph{generic programming}, and is synonymous with \emph{templates}.

Templates are a bit special in \Cpp\ since they are \emph{unconstrained}. Meaning, when we \emph{instantiate} e.g. function templates, the compiler will first generate the function (that's the instantiation part) by replacing \texttt{T} with whatever type you've passed to the function, and only \underline{after}, check if the syntax is correct! This leads to some very unfortunate side-effects, as you'll see soon enough :)

But before digging into the gritty details on that, let's start with a simple example of ``regular programming'' (i.e. with no templates), which we'll then use to iteratively build a scenario where generic programming will be needed. This will help us understand the though process that goes into designing a generic function, and how \emph{requirements} are gathered by the function's author.

Below is a function that calculates the arithmetic mean by taking in two pointers to a C-style array of \texttt{doubles}, one for the first, and last elements. Notice, that even if we would mangle the names, we would still recognize this as the \texttt{mean}, since the operations (summing elements and dividing by size), would still be familiar to us, since we know what \texttt{\{\}}, \texttt{+=}, \texttt{/} do to \texttt{doubles}.

\lstinputlisting[linerange={3-10}]{examples/mean.cc}

\noindent By using \emph{operator overloading}, we can make our user-defined types behave just like built-in types. This is very powerful, because it allows us to transfer our knowledge about e.g. \texttt{doubles}, and apply it for our own types as well, by using the \underline{\emph{same syntax}} as before. This last part is particularly important!

\lstinputlisting[linerange={6-16}]{examples/point2.hh}

\lstinputlisting[linerange={5-12}]{examples/centroid.cc}

\noindent As can be seen above, providing all those overloads to \texttt{point2} proved fruitful, since we can now represent the algorithm to find the centroid of a cluster of points in a very natural way, quite similarly to how we did the \texttt{mean} function. In fact, this function is \emph{exactly} the same as \texttt{mean}, with the only difference being that the type \texttt{double} $\rightarrow$ \texttt{point2} and that now \texttt{mean} $\rightarrow$ \texttt{centroid}. Surely there must be a way to generalize this type of ``coincidence'' in \Cpp? If you've done any GP at all before, you probably know where this is going!

\vspace{1em}\noindent\textbf{Note:} these are \underline{not} good examples on how you should write these functions, for instance, what happens when \texttt{begin >= end}? And why use pointers? Many examples in this primer are like this too, so don't use it in production!

\subsection{Unconstrained Templates} \label{sec:unconstrained_templates}

    Obviously, both \texttt{mean} and \texttt{centroid} are trying to find an \texttt{average}, which we can express as a higher-level idea with \emph{function templates}, as shown below. We've essentially just swapped \texttt{double}/\texttt{point2} by \texttt{T}, a \emph{template parameter}. These are defined in \emph{template parameter lists}, which have either \emph{type} or \emph{non-type template parameters}. By default, template parameters are \emph{unconstrained}, which means they may take \emph{any} type or value (using non-type parameters).

    Now, assume you're a user of this function, and wish to use it with your own user-defined type. What are the \emph{requirements} our type needs to \emph{satisfy}? As authors of this function, we know \texttt{T} needs to be \texttt{DefaultConstructible} in line 4, \texttt{Summable} with \texttt{T} in line 7, and \texttt{Scalable} by \texttt{double} in line 8. However, the user doesn't know that, and there's no way to tell by looking at the function signature... Do we have to go and look at the implementation?

    While that might work for this short toy example, it certainly won't scale. A possible solution (adopted by the STL) is to document everything on paper.

    \lstinputlisting[linerange={7-7,11-18}]{examples/average.cc}

    While writing documentation for the requirements on \texttt{T} certainly works, there is no guarantee the user will read it. And even if the user \emph{does} read it, there is still the possibility he/she will implement the requirements incorrectly. \textbf{Note:} for a ``good'' example on this, see: \emph{Type Requirements} in \href{https://en.cppreference.com/w/cpp/algorithm/sort}{\texttt{std::sort}}.

    Another reasonable alternative, and probably the most common approach, is to implement the requirements iteratively by \emph{trial-and-error}. We just let the compiler try to instantiate the template, and the hope is that the compiler will tell us what requirements are missing, so we can implement them. Even this approach is problematic for the user. For instance, consider this snippet:

    \begin{lstlisting}
std::list l { 5, 1, 2, 4, 3 };
std::sort(l.begin(), l.end()); \end{lstlisting}

\noindent Spits around 50 lines of template instantiation errors in GCC 8.1. While this is actually pretty tame, and not that hard to figure out what's wrong, it will still scare away a lot of people. The compiler is somewhat helpful, and colors e.g. the \texttt{std::\_\_lg(\_\_last - \_\_first) * 2}, telling us that it can't compile because \texttt{std::\_List\_iterator<int>} doesn't have \texttt{operator-}. However, the most helpful hint is hidden away between those lines, and the true cause is that \texttt{std::list} uses \texttt{ForwardIterators}, and \texttt{std::sort} expects a pair of \texttt{RandomAccessIterators}. Both of these are concepts, and their requirements are defined in the standard library specification. The programmer needs to the read documentation to locate the problem anyway!

To put a little bit more salt to the wound, consider this simple example:

    \begin{lstlisting}
std::set<Widget> w; // defined as struct Widget {  }; elsewhere
w.insert(Widget{}); \end{lstlisting}

\noindent Gives around 412 lines of template instantiation errors. This is still pretty tame in comparison to what some templated libraries output when you make a small mistake, and some of the longer errors even crash terminal emulators! Why can't we just get the same quality of errors as with non-templated code?

There is a reason why compilers can't be more helpful in these situations. As I've mentioned before, unconstrained templates only validate syntax \emph{after} the template has been instantiated, which means it's very hard to track down the source of the problem. Especially if the instantiation error happens far from the ``call site'', resulting in a long \emph{template instantiation stack}. It's so hard in fact, that compilers don't even needed to provide any diagnostics for these problems (according to the standard). Luckily, compilers still try, but the chances for these error messages getting vastly better in future compiler versions are slim, since these problems are equivalent to the \emph{halting problem}.

It seems we've hit a dead-end with unconstrained templates. Maybe we'll just have to endure the pain of using templated code, or consider it a rite of passage for every \Cpp\ programmer that wishes to venture forth. We'll just have to live with fragile template interfaces and unintuitive error messages...

    \subsubsection*{Concepts to the Rescue!}

    Don't fear, concepts is here! Below we have the \texttt{average} from before, with the exception of lines 2-4, which have been squeezed in-between the template parameter list and the function signature. These apply \emph{type constraints} on \texttt{T}, which are just a set of \emph{predicates} that must be \emph{fulfilled} \underline{before} instantiation.

    Without getting boggled down on syntax (we'll have plenty of that later), lines 2-4 say that: we require \texttt{T} to be \texttt{DefaultConstructible}, \texttt{T} must also be \texttt{SummableWith} another \texttt{T}, and \texttt{T} has to be \texttt{ScalableWith} a \texttt{double}. All of these concepts together constrain \texttt{T} to a set of types we want to allow.

    \lstinputlisting[linerange={7-18}]{examples/average.cc}

    \noindent Even though that seems pretty neat, what have we gained from doing this? Apart from the extra verboseness, function authors now need to spend  time thinking about the constraints on \texttt{T}, which might seem like ``a waste of time''. After all, it's not like there aren't implicit ``constraints'' already defined in the body of the function: giving a ``bad'' \texttt{T} won't compile, even without concepts. There needs to be a good reason \emph{why} this extra verboseness is needed at all!

    Luckily, this extra verboseness solves a lot of the problems we've described, and easily outweigh the cost of having to write a couple of extra lines of code. Because the constraints on \texttt{T} are checked \emph{before} instantiation occurs, a more reasonable error message can be displayed instead of the ``junk'' we get today. Also, because constraints are part of the function signature, the interface is no longer fragile, and the user can see what concepts they need to implement.

    This completely eliminates the need to document template requirements, because the constraints are explicitly stated in the function signature. Also, the ``trial-and-error'' approach to implementing requirements is no longer unreasonable, because the compiler can now give (more) useful information! Calling \texttt{std::sort} with a \texttt{std::list} will now give something similar to: \emph{``constraints not satisfied: \texttt{std::list::iterator} is not \texttt{RandomAccess}''}, followed by a concise summary of all the requirements you need to implement.

    Also, concepts allow us to overload function templates using constraints. However, we've been able to do that for a while now, by using \emph{type traits} and SFINAE, or even with \emph{tag dispatching}. In fact, both of these are ``competitors'' to concepts, and fill similar niches. Let's have a look at both of these as well!

\subsection{Type Traits and SFINAE} \label{sec:type_traits_and_sfinae}

    Even though this primer is mainly focused on concepts, it would be foolish not to compare it with existing methods that already limit template parameters. One of these are \emph{type traits + SFINAE (Substitution Failure Is Not An Error)}. Despite the scary looking acronym, the idea behind SFINAE is quite simple: ``if replacing \texttt{T} gives invalid syntax, don't emit an error, find alternatives first'', which means we can ``swap'' between overloads depending on these ``failures''.

    And this is where type traits come in handy, since they allow us to ``trigger'' these failures by using them together with \texttt{enable\_if}, a compile-time switch, which evaluates to a non-failing type if the type trait is satisfied. The type traits themselves are just clever constructs that evaluate to \texttt{true} when \texttt{T} has certain properties, e.g. \texttt{is\_integral\_v<int>} gives us \texttt{true}. One of the problems with type traits is that they are not obvious to specify.

    \noindent \textbf{Note:} for a deeper dive see \emph{Bendersky's} \href{https://eli.thegreenplace.net/2014/sfinae-and-enable_if/}{SFINAE and \texttt{enable\_if}} blog post.

    For instance, \texttt{EqualityComparable} from the \Cpp11 standard library:

    \begin{table}[h]
    \begin{tabular}{ccc}
        \toprule
        \bf{Expression} & \bf{Return Type} & \bf{Requirement Specification} \\
        \midrule
        \texttt{x == y} & \textbf{\texttt{bool}} convertible & \makecell[l]{\texttt{==}\, is an equivalence relation, that is,\\
                                                                it has the following properties:\\
                                                                $\rightarrow$ for all \texttt{x}, \texttt{x == x}\\
                                                                $\rightarrow$ if \texttt{x == y}, then \texttt{y == x}\\
                                                                $\rightarrow$ if \texttt{x == y}, \texttt{y == z}, then \texttt{x == z}} \\
        \bottomrule
    \end{tabular}
    \end{table}

    \noindent could be defined as the type trait below, which gives \texttt{true} iff \texttt{T} and \texttt{U} have \texttt{operator==(T,U)}. While this wasn't all that complicated in this case, it's still annoying that 87.5\% of this code is just pure boilerplate. It's line 7 that is actually doing the heavy lifting, everything else is just ``unnecessary'' code. Also, if you're very observant, you'll notice that this type trait doesn't even satisfy all of the requirements above, since it doesn't support commutativity.

    Even line 7 isn't obvious, since we need to use \texttt{std::declval} to acquire references to \texttt{T} and \texttt{U} that don't need to go through constructors, so that they can be evaluated with \texttt{decltype}, and finally do what we came to do: check if \texttt{operator==(T,U)} is available. I hope that you will agree that this seems like an awfully roundabout way of doing this. And this is one of the easier type traits to specify, expect extra machinery for anything less trivial.

    \lstinputlisting[linerange={6-13}]{examples/sfinae.hh}

    \subsubsection*{Concepts to the Rescue!}

    Don't fear, concepts is here! Why do we have to go through all that trouble? In the end, the problem we're trying to solve is simple: for any \texttt{T} and \texttt{U}, check if \texttt{x == y} and \texttt{y == x} are expressions that compile (where \texttt{x} $\in$ \texttt{T}, \texttt{y} $\in$ \texttt{U}). Below we define \texttt{EqualityComparable} by using concepts. Prettier, right?

    Again, without delving too much into the syntax, line 2 defines a concept called \texttt{EqualityComparableWith}, that requires for any \texttt{x} $\in$ \texttt{T} and \texttt{y} $\in$ \texttt{U}: \texttt{x == y}, \texttt{x != y}, \texttt{y != x}, \texttt{y == x}, be valid expressions evaluable to \texttt{bool}. e.g. \texttt{EqualityComparableWith<const char*, std::string>} gives \texttt{true}, because \texttt{std::string} has all operators for comparing C-style strings. Note that this is way more powerful than \texttt{is\_equality\_comparable} since it also checks commutativity properties. Something that might be surprising is that it's also non-equality comparable, and is a trend you'll see with ``good'' concept definitions, since they \emph{prefer complete} rather than \emph{minimal} concepts.

    Back in the \texttt{average} example, where we apply constraints via concepts, you might have been wondering how to define e.g. \texttt{DefaultConstructible}. And you'll be pleased to know all of them are really simple to define as well, just like the one below! If you want to take a peek: \texttt{examples/concepts.h}.

    \lstinputlisting[linerange={57-63}]{examples/concepts.h}

    \noindent And now for something completely different, but still related to type traits, and more specifically, the many issues with SFINAE. Consider this example: a factory which creates different numbers depending on the constructor that was called. If \texttt{NumberFactory} is called with an integer type, \texttt{create\_number} returns \texttt{42}, but if it's a floating point type, we get e.g. \texttt{0xDEADBEEF} instead.

    \lstinputlisting[linerange={8-22}]{examples/factory.hh}

    Sounds simple right? And one would expect the \texttt{NumberFactory} in the previous page to already work, since we're eliminating the ``bad'' constructors with SFINAE, via the \texttt{is\_integral} \& \texttt{is\_floating\_point} type traits. However, that doesn't quite work, because the call will be ambiguous. SFINAE isn't part of overload resolution. As far as the compiler knows, both constructors are exactly the same. How to fix this? We make them different!:

    \lstinputlisting[linerange={28-44}]{examples/factory.hh}

    \noindent Essentially, by introducing a \texttt{dummy} argument, we have changed the signature of the function, which means the compiler no longer considers this ambiguous, and SFINAE can finally take over as usual. While this works, I'd like to argue that this isn't nice, and causes unnecessary headaches for something so simple. Wouldn't it be awesome to not have to think about these edge-cases anymore?

    \subsubsection*{Concepts to the Rescue!}

    Don't fear, concepts is here! Since constraints are part of overload resolution, the compiler won't consider these constructors ambiguous anymore. In fact, constraints are part of the function's signature! No more edge-cases with this!

    \lstinputlisting[linerange={50-62}]{examples/factory.hh}

\subsection{Tag Dispatching} \label{sec:tag_dispatching}

    Another generic programming technique, heavily used in the \Cpp\ standard library, is \emph{tag dispatching} together with \emph{trait classes}. It's a way to choose the function to call/dispatch depending on the properties of a type by using \emph{tags}. A tag is just an empty class (déjà vu?) used to control the function dispatch.

    This is better explained with an example, which I've shamelessly borrowed from \emph{Boost's} excellent \href{https://www.boost.org/community/generic_programming.html#tag_dispatching}{Generic Programming Techniques} survey. If you've played around with \texttt{std::advance} before, you'll know that it's an algorithm which increments an iterator $n$ times, and gives you that ``advanced'' iterator. What you might \emph{not} know, is that there are two versions of it: one which runs in $\mathcal{O}(n)$ time, and another in $\mathcal{O}(1)$ time, depending on the properties of the type that was passed. Which makes sense when you think about it, since a \texttt{ForwardIterator} can only call \texttt{++i} while a \texttt{RandomAccessIterator} can just directly call \texttt{i += n}. It does this by using tag dispatching, like this:

    \lstinputlisting[linerange={13-31}]{examples/advance.cc}

    \noindent Notice that \texttt{tagged\_advance} needs an extra argument in the end: the tag, which is used to dispatch the right \texttt{tagged\_advance} from within \texttt{advance}. It does this selection by using \texttt{iterator\_traits<T>::category}, which is a pre-defined type alias for finding out what type of iterator \texttt{T} is. This is quite an elegant approach, but requires us to specify new ``empty'' classes for new iterator types. Worst of all though, we need to provide a specialization inside the \texttt{iterator\_traits} class! Isn't there a better way to solve this?

    \lstinputlisting[linerange={36-40}]{examples/advance.cc}

    \subsubsection*{Concepts to the Rescue!}

    Don't fear, concepts is here! Instead of having to control dispatch with a tag, how about choosing the correct function overload right from the start? With concepts, this is possible, since we can specify the properties on \texttt{T} explicitly!

    \lstinputlisting[linerange={47-50,52-59,61-62}]{examples/advance.cc}

    \noindent At this point, you should be somewhat convinced that concepts are neat, and that existing \Cpp\ language features aren't ``good'' enough for some scenarios. It's finally time to actually start talking about Concepts! I hope you're hyped!