\input{../../../utils/header_article.tex}

\title{Regression Discontinuity Design (RDD)\thanks{We are indebted to Liudmila Kiseleva for the design of the problem set.}}
\subtitle{-- Problem Set 3 --}
\date{}

\begin{document}\maketitle\vspace{-2cm}

In the problem set 3 we are going to practice RDD in \cite{Lee.2008} framework presented in the lecture 11. We employ the original simplified data set on the individual candidates for the US House of Representatives from 1946 to 1998. If a candidate obtains more votes than his or her competitors, he or she takes the office.  Each elected candidate represents one of 435 congressional districts. The elections are held every two years. We seek the answer to the question whether winning the election has a causal influence on the probability that the candidate will win the next election.\\
\\
The observations of the data set \href{https://github.com/HumanCapitalAnalysis/microeconometrics/tree/master/problem-sets/03-regression-discontinuity-design/data}{\texttt{individ\_final.dta}} are clustered by district and election year. It consists of the following variables:
\begin{enumerate}

\item \emph{outcome} is a treatment variable; it is coded as 1 if a candidate won the election in the corresponding year and 0 -- otherwise.
\item \emph{outcomenext} is an outcome variable. It is coded as 1 if a candidate won the next election; as 0 if he or she did not win the next election; and as -1 if he or she did not participate in the next election.
\item \emph{difshare} is an assignment variable; it is the winning candidate's vote share minus the vote share of the highest performing competitor. Therefore, 0 is the cutoff point: a candidate whose vote share is more than 0 is automatically assigned to treatment.

\end {enumerate}

\section*{Task A. Theoretical foundation}

\begin{boenumerate}

  \item \emph{What is the main assumption that makes RDD possible?} Define the local randomization condition in the simplified setup presented in the lecture.

\end{boenumerate}

\section*{Task B. Graphical presentation using local averages}

A major advantage of the RD design over competing methods is its transparency, which can be illustrated using graphical methods. A standard way of graphing the data is to divide the assignment variable  into a number of bins, making sure there are two separate bins on each side of the cutoff point. Then, the average value of the outcome variable can be computed for each bin and graphed against the mid-points of the bins.

\begin{boenumerate}

\item Create a new variable that groups the assignment variable values into 400 bins  with a size of 0.005.

\item Since we are interested in a causal influence on the probability that the candidate will win the next election based on winning the current election, drop the rows that do not have a comparable next election.

\item Find the mean of the outcome variable for each bin or, in other words, local average. Draw this relationship on the scatterplot.

\item For better visuality we also add to the graph the fitted values of logistic regression around the cutoff. For this apply logistic regression separately on either side of the threshold (we take the bins with the share values from -0.25 to 0.25 and use the package \texttt{LogisticRegression} from \texttt{sklearn.linear\_model}). Extract probability estimates. Add them to the scatterplot in the proximity of cutoff. \emph{Do you observe a discontinuity at the cutoff point?}
\end{boenumerate}

\section*{Task C. Local linear regression (LLR)}

 LLR as a method restricts the estimation to observations close to the cutoff. It is based on the assumption that  regression lines within the bins around the cutoff point are close to linear. That helps to avoid some of the drawbacks of other parametric/non-parametrics approaches (\cite{Lee.2010a})

\begin{boenumerate}

\item Run the LLR with a specification $Y = \alpha_r + \tau D + \beta X + \gamma X D + \epsilon$, where X is rectricted by a bandwidth: $h \geq X \geq -h$. Interpret the result. Experiment with few bandwidths on your choice.

\end{boenumerate}

\section*{Task D. Cross-validation}

As you might find, the treatment effect result is sensitive to the bandwidth choice. In general, choosing a bandwidth in estimation involves finding an optimal balance between precision and bias. One the one hand, using a larger bandwidth yields more precise estimates as more observations are available to estimate the regression. On the other hand, the linear specification is less likely to be accurate (\cite{Lee.2010a})\\
\\
We are going to review one of the approaches for choosing a bandwidth --  cross-validation “leave one out” procedure. The main idea is to take an observation $i$ in the data, leave it out, run LLR, and use the estimates to predict the value of $Y$ at $X = X_i$.  Proceeding with each observation separately on each side of the cutoff, we obtain the predicted values of $Y$ that can be compared to the actual values. The optimal bandwidth is then a value of $h$ that minimizes the mean square of the difference between the predicted and actual values of $Y$. And overall mean square error is simply the average of the squares of the prediction errors  on each side of the cutoff.

\begin{boenumerate}

\item If you want to practice your Python skills, we recommend to work with the packages \texttt{LeaveOneOut()} and \texttt{cross\_val\_score} from \texttt{sklearn.model\_selection} and to write the code that finds the optimal bandwidth. Otherwise, we created our draft in \texttt{auxiliary.py}; you can use it to produce your solution. Draw the graph showing the relationship between the bandwidth and the mean square error. \emph{What is the optimal bandwidth for LLR in our framework?}

\end{boenumerate}

\bibliographystyle{apacite}
\bibliography{../../../submodules/bibliography/literature}

\end{document}


\documentclass{article}
    % General document formatting
    \usepackage[margin=0.7in]{geometry}
    \usepackage[parfill]{parskip}
    \usepackage[utf8]{inputenc}
    \usepackage{mathrsfs}
    \usepackage{amsmath}
    \usepackage{amssymb}
    \usepackage{tikz}
    \usepackage{fancyhdr}
    \usepackage{multicol}

    \usetikzlibrary{positioning}

\pagestyle{fancy}
\fancyhf{}
\rhead{Edgar Jacob Rivera Rios - A01184125}

\renewcommand{\labelenumi}{\alph{enumi})}

\begin{document}
\section*{6.1.1}
\begin{equation*}
  Z = -3x_{1} + 2x_{2} + 10x_{3} - x_{4} \rightarrow max
\end{equation*}
s.t
\begin{align*}
-x_{1} + 2x_{2} - 3x_{3} + 4x_{4} &\leq 7\\
       - 3x_{2} + 2x_{3} - 5x_{4} &\leq 12\\
2x_{1} + 2x_{2}          + 7x_{4} &\leq 15\\
x_{j} \geq 0, j &= 1, 2, 3, 4
\end{align*}
\begin{enumerate}
  \item Rewrite the problem as an augmented LP problem ready to be solved by the simplex method.

  See file 6.1.1.txt

  \item Work through the simplex method step by step to solve the augmented LP problem.

  See file 6.1.1.txt
\end{enumerate}

\section*{6.1.2}
Work through the simplex method step by step to solve the following problem

Maximize
\begin{equation*}
  Z = 4x_{1} + 3x_{2} + 6x_{3}
\end{equation*}
\begin{align*}
  3x_{1} + x_{2} + 3x_{3} &\leq 30\\
  2x_{1} + 2x_{2} + 3x_{3} &\geq 10\\
  x_{1} \geq 0, x_{2} \geq 0, x_{3} &\geq 0,
\end{align*}

See file 6.1.2.txt

\end{document}


\subsection{Clustering}\label{sec:clustering}
%Another way of optimising the computation of the link model is to introduce clustering of nodes, used to reduce the amount of links required for the link model computation.
The second option for optimising the computation of the link model, is to introduce clustering of nodes. As mentioned in \autoref{sec:pathloss}, links existing in the same physical environment should experience similar shadow fading \gls{pathloss}. This means that we are be able to cluster nodes with a minimum loss of precision, provided that we minimise difference of the correlation (the angle) between links, before and after clustering.

%minimise the difference of the angles between links before and after clustering (correlation)

%This means that we should be able to compute clusters of nodes, and use the clusters to faster compute the link model of multiple nodes at the same time, as we would, depending on the number of clusters, have a far smaller link matrix. \medbreak

%For our purposes, we have chosen to use a density-based clustering algorithm, more specifically the \gls{optics} algorithm (\autoref{algo:optics}). The idea behind density-based clustering is that for every node in a cluster, the neighbourhood, within a given radius, has to contain at least a minimum number of other nodes~\cite[p.~50]{Ankerst:1999:OOP:304182.304187}. A set of nodes in a network, $\textbf{N} \subseteq \mathcal{X}$, along with a distance parameter $\varepsilon$, denoting the maximum radius of a nodes neighbourhood, and $MinNodes \geq 2$ denoting the minimum number of nodes required to form a cluster, such that a single node will not be considered a cluster. The goal is to find a set of clusters, $C$, such that we are able to form a new network with the centroids of our set of clusters, as well any outlying node not contained in a cluster, $\text{NOISE} \subseteq \textbf{N}$, while minimising the radius of the clusters.

%\subsubsection{Metric Space}
% Minimise the diameter using parameters
% Maximum diameter as input, give me the smallest number of k-nodes, such that every other node is within the diameter of at least one node.
% Output: Some number of nodes, which is the smallest possible diameter

%metric space
\subsubsection{Metric Space}
A metric space is a pair $( \mathcal{X}, d )$ where $ \mathcal{X} $ is a set and $\textbf{d}:\mathcal{X} \times \mathcal{X} \rightarrow [0, \infty )$ is a metric, satisfying the following axioms:

\begin{itemize}
    \item Positivity: $d(x, y) \geq 0$
    \item Reflexivity: $d(x, y) = 0 \Longleftrightarrow x = y$
    \item Symmetry: $d(x, y) = d(y, x)$
    \item Triangle inequality: $d(x, z) \leq d(x, y) + d(y, z)$
\end{itemize}

Our chosen metric space is $\mathbb{R}^2$ using Euclidean distance.

%$\mathbb{R}^2$

\subsubsection{Problem Statement}

%A set $\textbf{N} \subseteq \mathcal{X}$, is provided together with parameters $\varepsilon$. The goal is to find a subset of clusters, such that the maximum angle between any node to nodes in the cluster is minimised.
A set $\textbf{N} \subseteq \mathcal{X}$, is provided together with a parameter $k$, $k \leq |\textbf{N}|$. Initially, the set $\textbf{N}$ is a set of clusters, each containing a single node. The goal is to find a set of clusters, such that the computation time required for computing the link model can be reduced, while minimising the loss of precision in the stochastic shadow fading part. We minimise the loss of precision by minimising the difference of the correlation ($\Delta\theta$) between links to any node in a cluster, to the centroid of the cluster. The centroid of a cluster $x$ is defined as $cent(x) = \sum\limits_{n \in x} n \div |x|$. \smallbreak
%
The problem is defined as follows:\smallbreak

%This correlation difference is computed using the autocorrelation function from \autoref{eq:pathlossautocorrelation}.

% \autoref{figure:clusteringgoal} contains an example of this
%For example, we want to minimise the difference $\Delta\theta$ between the links $l_{n,u}$, $l_{n, c}$, in relation to the link $l_{n,m}$ in \autoref{figure:clusteringgoal}. The difference is computed using the autocorrelation function from \autoref{eq:pathlossautocorrelation}: $\Delta\theta = r(l_{n,m}, l_{n,u}) - r(l_{n,m},l_{n,c})$, where $u \in c$, $n \not\in c$, $m \not\in c$.

%The goal is to find a
%subset of clusters, such that the difference of the correlation between ($\Delta\theta$) links to any node in a cluster, to the centroid of the cluster, is minimised, while reducing the number of clusters, thus reducing the computational time required to compute the link model, while minimising the loss of precision.

%s an example, in \autoref{figure:clusteringgoal}, we want to minimise the difference $\Delta\theta$, between the links $l_{n,u}$, $l_{n, C}$, in relation to the link $l_{n,m}$, or $\Delta\theta = r(l_{n,m}, l_{n,u}) - r(l_{n,m},l_{n,C})$, where $u \in C$, $n \not\in C$, $m \not\in C$, and $r$ is the autocorrelation function from \autoref{eq:pathlossautocorrelation}.

%such that the difference between an angle any node from outside the cluster to the centroid of the cluster, and the original node in the cluster is minimised.
%\autoref{figure:clusteringgoal} demonstrate an example of this, where we see three clusters of nodes.
%$\delta\theta$

%such that the difference in correlation between links is minimised, while maximising the number of clusters.

%Minimise the difference between the original correlation matrix, and the new correlation matrix created using the centroid of the clusters.

For a metric space $(\mathcal{X}, d)$,
\begin{itemize}
    \item Input: A set $\textbf{N} \subseteq \mathcal{X}$ and a parameter $k$, $k \leq |\textbf{N}|$.
    \item Output: A set of clusters $C \subseteq 2^{\textbf{N}}$, such that
          \begin{enumerate}
              \item $\bigcup\limits_{x \in C}x = \textbf{N}$,
              \item $\forall x, y \in C, \ \text{if} \ x \neq y \ \text{then} \ x \cap y = \emptyset$, and
              \item $|C| \leq k$
          \end{enumerate}
    \item Goal: Minimise the $\Delta\theta(C)$ function:\smallbreak $\Delta\theta(C) = \mathlarger{\sum}\limits_{n_1, n_2, n_3 \ \in \ \textbf{N}} \left( r(l_{n_1,n_2}, l_{n_1,n_3}) - r(l_{cent(C(n_1),cent(C(n_2)}, l_{cent(C(n_1),cent(C(n_3)}) \right)^2$

          % \item Input: a set $\textbf{N} \subseteq \mathcal{X}$, and a parameter $\varepsilon$.
          %       %    \item Output: a set of clusters $C$ where, for each nodes in the cluster, the neighbourhood of a given radius $\varepsilon$ of the nodes has to contain at least a minimum number of nodes $MinNodes$.
          % \item Output: a set of clusters $C$. % færre elementer end det set vi startede med
          % \item Goal: minimise the difference in correlation, $\Delta\theta$, between links $l_{n,c}$, $l_{n,u}$, with relation to $l_{n, m}$, for any $c \in C$, $u \in c$, $n \not\in c$, and $m \not\in c$, where $n \neq m$.

          %$\forall c \in C, \forall u \in c, \forall m \not\in c, \forall n \not\in c, $

          %$\Delta\theta$, $\forall c \in C,  $  $\Delta\theta = r(l_{n,m}, l_{n,u}) - r(l_{n,m},l_{n,C})$
          %    \item Goal: minimise the cost $r^C_\infty(\textbf{N}) = \max\limits_{n \in \textbf{N}} d(n, C)$.
          %\item Goal: minimise the size of the network by creating a new network consisting of the set of nodes $centroids(C) \cup \text{NOISE}$
          %\item Goal: Minimise the cost $r^C_\infty(\textbf{P}) = \max{p \in \textbf{P}} d(p, C)$
\end{itemize}

%\todo[inline]{How to describe the goal? Is it the goal of the particular clustering algorithm, or our goal for using the clustering algorithm? In the second case, the goal is to reduce the number of overall nodes in the network, by creating clusters of nodes very close to each other, and using the centroid of the clusters for the link model.}

\autoref{figure:clusteringgoal} illustrates the goal, where we want to minimise the correlation difference $\Delta\theta$ between the nodes $n$, $m$, and $u$ (as well of the rest of the nodes), to the centroids of their respective clusters $C(n)$, $C(m)$, and $C(u)$, where $C(n)$ is the cluster in $C$ that contains $n$, such that $n \in C(n)$

\begin{figure}[ht]
    \centering
    \includegraphics[width=.5\textwidth]{figures/clustering/clustering.png}
    \caption{Minimising the difference of the correlation ($\Delta\theta$) between links to nodes in a cluster to the centroid of the cluster.}
    \label{figure:clusteringgoal}
\end{figure}

%\subsubsection{Choice of Clustering Algorithm}\label{sec:clustering:choice}
%In the paper~\cite{proceeding:clustering-Survey}, the authors conduct a survey on density based clustering algorithms, giving an overview on existing algorithms. As mentioned above, we want the clustering to minimise the difference of the correlation ($\Delta\theta$) between links. We can approximate this by clustering nodes that are close to each other. The algorithm needs to handle clusters of varying densities and it should allow noise (outliers). \smallbreak

%The \acrshort{dbscan} algorithm is a widely used algorithm, but does work very well when clustering with varying densities.

%The \acrshort{vdbscan} is a variation of the \acrshort{dbscan} that fixes the problem with varying densities, and even automatically finds the parameters normally needed when running the algorithm.

%This however does not make it possible, to more aggressively specify the density threshold. The \gls{vdbscan}'s found optimal parameters, might not create the desired small clusters that is needed according to our clustering problem statement.\smallbreak

%\gls{optics} however can handle variable densities although slower than \gls{dbscan}. The complexity of \gls{optics} is O$(n^2)$, but if implemented with a spatial index has the complexity of O$(n\log n)$. As such we choose to use the \gls{optics} algorithm.

\subsubsection{Approximation using OPTICS}
We have chosen a greedy approach using the \gls{optics} algorithm~\cite{Ankerst:1999:OOP:304182.304187} to approximate solutions for our clustering problem. \gls{optics} is an algorithm for finding density-based clusters, and the run-time of the algorithm is O($n \ \cdot $ run-time of a neighbourhood query)~\cite[p.~53]{Ankerst:1999:OOP:304182.304187}. We chose the \gls{optics} algorithm based on a survey on density-based clustering algorithms in \cite{proceeding:clustering-Survey}.\smallbreak

The algorithm requires three parameters: our input set $\textbf{N}$, $\varepsilon$, describing the maximum distance between two nodes to consider for clustering, and $MinPts$, describing the minimum number of nodes required to form a new cluster. For our case, the parameter $MinPts = 2$. With this, the \gls{optics} algorithm will return the set of clusters $C \subseteq 2^{\textbf{N}}$, where any nodes not satisfying the $MinPts$ parameter will be a singleton cluster. \autoref{algo:clustering} presents the pseudo code description of our greedy approach, that repeatedly computes the a set of clusters $C$, until $|C| \leq k$, each time incrementing the $\varepsilon$ variable by 10 meters.

\begin{algorithm}[H]
    \DontPrintSemicolon
    \KwResult{A set of clusters $C \subseteq 2^{\textbf{N}}$, where $|C| \leq k$}
    \SetKwFunction{FCluster}{Cluster}
    \SetKwProg{Fn}{Function}{:}{}
    \Fn{\FCluster{\textbf{N}, $k$}}{
        $\varepsilon \leftarrow 0$ m\;
        $C \leftarrow \text{OPTICS}(\textbf{N}, \varepsilon, 2)$\;

        \Repeat{$|C| \leq k$}{
            $\varepsilon \leftarrow \varepsilon + 10$ m\;
            $C \leftarrow \text{OPTICS}(\textbf{N}, \varepsilon, 2)$\;
        }

        \KwRet $C$\;
    }

    \caption{Greedy approach using the \gls{optics} algorithm.}
    \label{algo:clustering}
\end{algorithm}

% The way we want to use the \gls{optics} algorithm is to create many clusters with a higher density (a lower value for $\varepsilon$), as bigger, lower density clusters, would not make sense for the purposes of the link model, given the loss of precision this would incur. The \gls{optics} algorithm in itself does not assign cluster memberships to nodes in the network~\cite[p.~52]{Ankerst:1999:OOP:304182.304187}. Instead, the algorithm produces a particular ordering of the nodes, from which we can extract actual cluster memberships from. This ordering is produced by computing two values for every node in the network: the \textit{core-distance} (\autoref{eq:coredist}) and the \textit{reachability-distance} (\autoref{eq:reachdist}) Additionally, a node is a \textit{core node} if at least $MinNodes$ are found within its $\varepsilon$-neighbourhood~\cite[p.~52]{Ankerst:1999:OOP:304182.304187}.
% \begin{equation}\label{eq:coredist}
%     coredist_{\varepsilon,MinNodes}(n) =
%     \begin{cases}
%         \text{UNDEFINED}                                          & \text{if} |N_\varepsilon(n)| < MinNodes \\
%         MinNodes\text{-th smallest distance to}\ N_\varepsilon(n) & \text{otherwise}
%     \end{cases}
% \end{equation}
% \begin{equation}\label{eq:reachdist}
%     reachdist_{\varepsilon,MinNodes}(o, n) =
%     \begin{cases}
%         \text{UNDEFINED}                                    & \text{if} |N_\varepsilon(n)| < MinNodes \\
%         max(coredist_{\varepsilon,MinNodes}(n), dist(o, n)) & \text{otherwise}
%     \end{cases}
% \end{equation}

% The \gls{optics} (\autoref{algo:optics}) algorithm processes each node only once, and performs one $\varepsilon$-neighbourhood query during the processing of a node, at \autoref{algo:optics:getneighbours1} and \autoref{algo:optics:getneighbours2}. This means that the run-time of the algorithm is heavily dependent on the $\varepsilon$-neighbourhood query, i.e., the run-time for the \gls{optics} algorithm is O($n \ \cdot $ run-time of an $\varepsilon$-neighbourhood query)~\cite[p.~53]{Ankerst:1999:OOP:304182.304187}. The algorithm starts by setting the reachability-distance of each node to \texttt{UNDEFINED} on \autoref{algo:optics:setreachdistundefined}. Note that the very first node processed will be added to the ordered list, at \autoref{algo:optics:addfirstnode}, with a reachability distance of \texttt{UNDEFINED}, and that we assume \texttt{UNDEFINED} to be greater than any defined distance~\cite[p.~54]{Ankerst:1999:OOP:304182.304187}. Next, we check if the node $n$ is a core node on \autoref{algo:optics:checkifcorepoint1}, and if yes, we initialise an empty priority queue \texttt{Seeds} on \autoref{algo:optics:initseeds}. With this, we call the \texttt{Update} function on \autoref{algo:optics:updateseeds}, which updates the priority queue with the reachability-distance of any unprocessed node in the $\varepsilon$-neighbourhood of $n$. Finally, we repeat the process for any node $m$ in the priority queue, and move on to the next unprocessed node in $N$.\medbreak

% \begin{algorithm}[ht]
%     \DontPrintSemicolon
%     \KwResult{Ordered list of nodes N$'$}
%     \SetKwFunction{FOptics}{Optics}
%     \SetKwFunction{FUpdate}{Update}
%     \SetKwProg{Fn}{Function}{:}{}
%     \Fn{\FOptics{N, $\varepsilon$, MinNodes}}{
%         N$'$ = ordered list\;

%         \ForEach{$n \in$ N}{
%             n.reachdist = UNDEFINED\;\label{algo:optics:setreachdistundefined}
%             n.processed = false\;
%         }

%         \ForEach{unprocessed $n \in$ N}{
%             N$_\varepsilon$ = getNeighbours(n, $\varepsilon$)\;\label{algo:optics:getneighbours1}
%             n.processed = true\;
%             N$'$.insert(n)\;\label{algo:optics:addfirstnode}

%             \If{coredist$_{\varepsilon,MinNodes}$(n) $\neq$ UNDEFINED}{\label{algo:optics:checkifcorepoint1}
%                 Seeds = empty priority queue\;\label{algo:optics:initseeds}
%                 Update(N$_\varepsilon$, n, Seeds, $\varepsilon$, MinNodes)\;\label{algo:optics:updateseeds}

%                 \ForEach{next $m \in \text{Seeds}$}{
%                     M$_\varepsilon$ = getNeighbours(m, $\varepsilon$)\;\label{algo:optics:getneighbours2}
%                     m.processed = true\;
%                     N$'$.insert(m)\;

%                     \If{coredist$_{\varepsilon,MinNodes}$(m) $\neq$ UNDEFINED}{\label{algo:optics:checkifcorepoint2}
%                         Update(M$_\varepsilon$, m, Seeds, $\varepsilon$, MinNodes)\;
%                     }
%                 }
%             }
%         }

%         \KwRet N$'$\;
%     }\;

%     \Fn{\FUpdate{N, n, Seeds, $\varepsilon$, MinNodes}}{
%         coredist = coredist$_{\varepsilon,MinNodes}$(n)\;

%         \ForEach{$o \in$ N}{
%             \If{o.processed = false}{
%                 reachdist = reachdist$_{\varepsilon,MinNodes}$(o, n)\;

%                 \If{o.reachdist = UNDEFINED}{
%                     \tcp{o is not in Seeds}
%                     o.reachdist = reachdist\;
%                     Seeds.insert(o, reachdist)\;
%                 }
%                 \Else{
%                     \tcp{o is in Seeds, check for improvement}
%                     \If{reachdist $<$ o.reachdist}{
%                         o.reachdist = reachdist\;
%                         Seeds.move-up(o, reachdist)\;
%                     }
%                 }

%             }
%         }
%     }

%     \caption{The OPTICS algorithm~\cite{Ankerst:1999:OOP:304182.304187}.}
%     \label{algo:optics}
% \end{algorithm}

% \todo[inline]{Write pseudocode for the cluster extraction algorithm.}
% %\todo[inline]{Add reachability plot for a sample run}
% \todo[inline]{Add table demonstrating parameterised runs, amount of clusters vs resulting nodes in the network, as well as the radius of the clusters}
% \todo[inline]{Maybe use a smaller area and with fewer/smaller actual clusters for the demonstration.  }

% \begin{table}[H]
%     \centering
%     \begin{tabular}{|l|l|l|l|}
%         \hline
%         $\varepsilon$ & Clusters & Correlation difference & Duration \\\hline
%         0.05 km       & 99       & 2.330                  & 2 ms     \\\hline
%         0.06 km       & 99       & 2.330                  & 2 ms     \\\hline
%         0.07 km       & 99       & 2.330                  & 2 ms     \\\hline
%         0.08 km       & 99       & 2.330                  & 2 ms     \\\hline
%         0.09 km       & 97       & 4.909                  & 2 ms     \\\hline
%     \end{tabular}
%     \caption{Computation time measurement for NearPD and Cholesky decomposition.}
%     \label{table:clusteringtime}
% \end{table}

\subsubsection{Experiment}\label{sec:clustering:experiments}
To measure the performance of our greedy approach, we conducted an experiment where we generated 1000 nodes with random locations in a $10 \times 10$ kilometre area, and ran the \texttt{Cluster} algorithm from \autoref{algo:clustering}, with different values for the $k$ parameter. The purpose of this experiment is to measure the computational time used to form a number of clusters $|C| \leq k$. The results of this experiment can be seen in \autoref{table:clustering-benchmark}.

%To measure the performance of our clustering approach, we ran an experiment. The experiment consisted of running the clustering algorithm on 1000 randomly generated nodes, create a fully connected network of the clusters and measure the computational time used on clustering. \autoref{table:clustering-benchmark} contain the results of this experiment.

\begin{table}[H]
    \begin{tabular}{|c|c|c|c|}\hline
        Max clusters ($k$) & Resulting clusters & Links  & Clustering       \\\hline%& Cholesky \\\hline
        1000               & 1000               & 499500 & 244 milliseconds \\\hline%& \dots    \\\hline
        900                & 884                & 390286 & 1 second         \\\hline%& \dots    \\\hline
        800                & 771                & 296835 & 2 seconds        \\\hline%& \dots    \\\hline
        700                & 692                & 239086 & 2 seconds        \\\hline%& \dots    \\\hline
        600                & 592                & 174936 & 3 seconds        \\\hline%& \dots    \\\hline
        500                & 463                & 106953 & 4 seconds        \\\hline%& \dots    \\\hline
        450                & 424                & 89676  & 4 seconds        \\\hline%& \dots    \\\hline
        400                & 389                & 75466  & 4 seconds        \\\hline%& \dots    \\\hline
        350                & 322                & 51681  & 5 seconds        \\\hline%& \dots    \\\hline
        300                & 299                & 44551  & 5 seconds        \\\hline%& \dots    \\\hline
        %  250          & 229                & 26106  & 249 milliseconds & \dots           \\\hline
        %  200          & 183                & 16653  & 250 milliseconds & 32 minutes      \\\hline
        %  150          & 136                & 9180   & 250 milliseconds & 325 seconds     \\\hline
        %  100          & 98                 & 4753   & 250 milliseconds & 45 seconds      \\\hline
        %  50           & 37                 & 666    & 253 milliseconds & 67 milliseconds \\\hline
    \end{tabular}
    \caption{Experiment results from benchmarking time required to cluster.}
    \label{table:clustering-benchmark}
\end{table}

%To verify that the clustering actually improves the cholesky computations, experiments have been run. The results can be soon in \autoref{table:clustering-benchmark}.\medbreak

%The experiment consisted of iteratively running our clustering on 1000 randomly generated nodes until the amount of clusters desired is achieved, create a fully connected network of the clusters and measure the used on clustering and computing the path loss with the cholesky. The columns in \autoref{table:clustering-benchmark} are as followed:

%\begin{description}
%    \item[Min clusters:] The minimum allowed amount of clusters. In the \gls{optics} algorithm, one does not specify an amount of desired clusters, hence a minimum needs to be specified to stop clustering once this minimum has been reached.
%    \item[Resulting clusters:] The resulting amount of clusters.
%    \item[Links:] The total amount of links.
%    \item[Clustering time:] How long the clustering took.
%    \item[Cholesky:] How long the cholesky computation took. Some fields in the cholesky column are marked as \dots. This is because the computation took too long to let it finish as the correlation matrix simply became too big.
%\end{description}

% write about the results

%\autoref{figure:clustering:nodes} demonstrates a sample run of the algorithm on a network of 2000 nodes, with 1000 nodes uniformly distributed across the entire 15.5 km by 7.8 km area, two clusters containing 200 nodes each, and two clusters containing 300 nodes each. The low $\varepsilon = 0.08$ km value means that only very densely packed nodes will be added to the clusters. Ideally, a lot of the outliers in the figure should be able to be clustered with $MinNodes=2$, but increasing the $\varepsilon$ could also mean that the radius of the clusters could become too large.

%based on features called \textit{core-distance} and\textit{reachability-distance}~\cite{Ankerst:1999:OOP:304182.304187},
%1179
%\begin{figure}[ht]
%    \centering
%    \begin{subfigure}[t]{.31\textwidth}
%        \includegraphics[width=\linewidth]{figures/clustering/2k.png}
%        %\caption{2000 nodes.}
%        \label{figure:clustering:2knodes}
%    \end{subfigure}
%    \begin{subfigure}[t]{.3223\textwidth}
%        \includegraphics[width=\linewidth]{figures/clustering/2k80m2pts.png}
%        %\caption{2000 nodes clustered.}
%        \label{figure:clustering:2k80m2ptsnodes}
%    \end{subfigure}
%    \caption{Result of the OPTICS algorithm with parameters $\varepsilon = 0.08$ km, and %$MinNodes = 2$.}
%    \label{figure:clustering:nodes}
%\end{figure}

%7.81 km * 15.57 km

%\begin{figure}[ht]
%    \centering
%    \includegraphics[width=.7\textwidth]{figures/clustering/2k-80m-2pts.png}
%    \caption{2000 nodes}
%    \label{figure:clustering:2k}
%\end{figure}


%
% STAT 100: Chance and Data Analysis - A Course Overview
% Section: Analysis of Population Proportions
%
% Author: Jeffrey Leung
%

\section{Analysis of Population Proportions}
	\label{sec:analysis-of-population-proportions}
\subsection{Margin of Error and Confidence Interval}
	\label{subsec:analysis-of-population-proportions:margin-of-error-and-confidence-interval}
\begin{easylist}

	& \emph{Parameter:} Value which summarizes population data
		&& Calculation requires collection of data from the entire population (i.e. see \emph{census}, subsection~\ref{subsec:data-collection:methods})
		&& Estimation is often calculated from a sample statistic
			&&& E.g. If 33\% of a random sample of Canadian adults support the Conservative Party, then the proportion of all Canadian adults who support the Conservative Party is estimated to be 33\%.
		&& Denoted by $p$
			&&& Sample proportion/statistic is denoted by $\hat{p}$

	\medskip
	& \emph{Margin of error:} Percentage value of the uncertainty of an estimated population proportion
		&& Calculation (for a 95\% confidence level):
		\begin{math}
			Margin\ of\ error = \frac{1}{\sqrt{sample\ size}}
		\end{math}
			&&& Unit: Percentage
		&& Valid only for a random sample
		&& Dependent on a confidence level
			&&& \emph{Confidence level:} Degree of certainty of the accuracy of a population proportion estimate
				&&&& Unit: Percentage
				&&&& Often 95\% (expressed as a fraction; 19 times out of 20 = $\frac{19}{20}$)
		&& Interpretation (with the confidence level): \smallskip \\
		If many random samples of <sample size, subject> of the population are taken and the sample proportion of <statistic> is calculated for each sample, <confidence level percentage> of the sample proportions will be within $\pm$ <margin of error percentage> of the population proportion.

		&& E.g. ``A probability sample of this design and sample size would carry a margin of error in the range of $\pm 1.2\%$, 19 times out of 20.'' \smallskip \\
		Margin of error: $\pm 1.2\%$ \\
		Confidence level: $\frac{19}{20} = 95\%$
		&& E.g. Given a study of Canadian adults who support the Conservatives with a random sample of 6005 Canadian adults and a margin of error of $\pm$ 1.2\%, 19 times out of 20: \smallskip \\
		If many random samples of 6005 Canadian adults of the population are taken and the sample proportion of Canadian adults who support the Conservatives is calculated for each sample, 95\% of the sample proportions will be within $\pm$ 1.2\% of the population proportion.

	\medskip
	& \emph{Confidence interval:} Set of values which the population proportion is within
		&& Calculation:
		\begin{math}
			Confidence\ interval = sample\ proportion \pm margin\ of\ error
		\end{math}
			&&& Unit: Percentage range
		&& Interpretation: \smallskip \\
		Using the sample data, we are <confidence level percentage> confident that the population proportion of <statistic> is between <confidence interval lower bound percentage> and <confidence interval upper bound percentage>.
			&&& Always specify the population proportion
		&& E.g. Given the sample proportion of Canadian adults who will vote for the Conservatives as 33\% for 6005 subjects, the confidence interval is: \\
		\begin{math}
			Sample\ proportion \pm \frac{1}{\sqrt{sample\ size}}
			= 33\% \pm \frac{1}{\sqrt{6005}}
			= 33\% \pm 1.2904...\%
			\approx (31.7\%, 34.3\%)
		\end{math} \\
		Using the sample data, we are 95\% confident that the population proportion of Canadian adults who support the Conservative Party is between 31.7\% and 34.3\%.

		&& Analyzing the confidence interval: Given a value, check whether or not all values of the confidence interval satisfy the condition
			&&& E.g. Given the confidence interval for the population proportion of Canadian adults who support the Conservative Party to be (31.7\%, 34.3\%), can you conclude that more than 30\% of all Canadian adults support the Conservative Party? \smallskip \\
			Yes; all values in the confidence interval are greater than 30\%.
			&&& E.g. Given the confidence interval for the population proportion of Canadian adults who support the Conservative Party to be (31.7\%, 34.3\%), can you conclude that more than 34\% of all Canadian adults support the Conservative Party? \smallskip \\
			No; there exist values in the confidence interval which are less than 34\%.

\end{easylist}
\subsection{Bias and Variability}
	\label{subsec:analysis-of-population-proportions:bias-and-variability}
\begin{easylist}

	& \emph{Bias:} Consistent under-estimation or over-estimation of results
		&& Can be reduced by:
			&&& Avoiding \hyperref[subsec:data-collection:sampling-errors]{non-sampling errors}
			&&& Ensuring fair representation of the population
			&&& Using random sampling
		&& Increasing sample size does not reduce bias

	& \emph{Sampling variability:} Degree of variability between random samples
		&& Quantified by the \hyperref[subsec:analysis-of-population-proportions:margin-of-error-and-confidence-interval]{margin of error}
		&& Can be reduced by:
			&&& Increasing sample size

\end{easylist}
\subsection{Hypothesis Testing for Population Proportions}  %TODO move this to a different file
	\label{subsec:analysis-of-population-proportions:hypothesis-testing-for-population-proportions}
\subsubsection{Introduction}
	\label{subsubsec:analysis-of-population-proportions:hypothesis-testing-for-population-proportions:introduction}
\begin{easylist}

	& \emph{Hypothesis test:} Calculation which determines whether a claim/research hypothesis is supported by evidence

	& \emph{Null hypothesis (H\textsubscript{o}):} Statement that a population proportion is equal to a given value (which may be another population proportion)
		&& Assumed to be possible until contradicting evidence is found
	& \emph{Alternative hypothesis (H\textsubscript{a}):} Statement that a population proportion is less than, not equal to, or greater than a given value (which may be another population proportion)
	& E.g. The sample proportion of Canadian adults who want to legalize marijuana was found to be 59\%. Test whether or not the population proportion of Canadian adults who want to legalize marijuana is greater than 50\%. \\
	H\textsubscript{o}: The population proportion of Canadian adults who want to legalize marijuana is equal to 50\%. \\
	H\textsubscript{a}: The population proportion of Canadian adults who want to legalize marijuana is greater than 50\%.

	\medskip
	& \emph{Test statistic:} Standardized value representing a numerical summary of sample data
		&& Calculated from sample data; analyzed to determine the p-value
		&& E.g. Z-statistic, t-statistic, chi-square statistic
		&& \emph{Z-statistic:} Test statistic which can be calculated to determine whether an inequal relationship between population proportions exists
			&&& Formula for one population proportion compared against a given percentage: \smallskip \\
			\begin{displaymath}
				z-statistic =
				\frac
				{
					\hat{p} - p_{0}
				}
				{
					\sqrt
					{
						\frac
						{
							p_{0} \cdot (1 - p_{0})
						}
						{
							n
						}
					}
				}
			\end{displaymath}
			\Deactivate
			\begin{center}
				\begin{tabular}{ l r @{ = } l }
					where & $\hat{p}$ & sample proportion \\
					& $p_{0}$ & population proportion if H\textsubscript{0} is true \\
					& $n$ & sample size
				\end{tabular}
			\end{center}
			\Activate

			\medskip
			&&& Formula for two population proportions compared against each other: \medskip \\
			\begin{displaymath}
				z-statistic =
				\frac
				{
					\hat{p_{1}} - \hat{p_{2}}
				}
				{
					\sqrt
					{
						\hat{p} \cdot (1-\hat{p}) \cdot
						(
							\frac{1}{n_{1}} + \frac{1}{n_{2}}
						)
					}
				}
			\end{displaymath}
			\Deactivate
			\begin{center}
				\begin{tabular}{ l r @{ = } l }
					where & $\hat{p_{x}}$ & sample proportion of the $x$\textsuperscript{th} set of data \\
					& $\hat{p}$ & combined sample proportion \\
					& $n_{x}$ & sample size of the $x$\textsuperscript{th} set of data
				\end{tabular}
			\end{center}
			\Activate

				&&&& Combined sample proportion is the sum of the number of subjects satisfying the condition of sample dataset 1 and the number of subjects satisfying the condition of sample dataset 2, divided by the sum of the number of subjects of each dataset

		&& \emph{Chi-square statistic:} %TODO

	\medskip
	& \emph{Probability value (p-value):} Probability that a given result is obtained through chance, calculated from sample data
		&& Unit: Percentage
		&& Interpretation: If many random samples of the given sample size and population are chosen and the sample proportion in question is calculated, then the p-value represents the percentage of the sample populations which would support the alternative hypothesis.
			&&& E.g. Given that 50\% of all Canadian adults support the legalization of marijuana, the probability of calculating a sample proportion of 59\% or higher through random sampling is equal to the p-value (0.62\%).
		&& The lesser the p-value, the greater the evidence for the alternative hypothesis
		&& Calculated from a test statistic; compared against the significance level to determine the amount of evidence for the alternative hypothesis
		&& For the interpretation of the p-value compared against the significance level, see \hyperref[subsec:analysis-of-population-proportions:statistical-significance]{statistical significance}
		&& For the interpretation of the magnitude, see table~\ref{tab:p-value-magnitude-chart}

		\Deactivate
		\begin{table}[!htb]
			\centering
			\caption{P-Value Magnitude Chart}
			\label{tab:p-value-magnitude-chart}
			\begin{tabular}{ r c l l }
				\multicolumn{3}{ c }{P-value} & Strength of Evidence to Support H\textsubscript{a} \\
				 10\% < & p-value & & No evidence \\
				  5\% < & p-value & $\leq$  10\% & Weak evidence \\
				  1\% < & p-value & $\leq$   5\% & Some evidence \\
				0.1\% < & p-value & $\leq$   1\% & Strong evidence \\
				        & p-value & $\leq$ 0.1\% & Very strong evidence
			\end{tabular}
		\end{table}
		\Activate

		&& P-value of a z-statistic: Area under the standard normal distribution where z-statistic equals the z-score (see %\ref standard normal table
			&&& If the alternative hypothesis is `less than', the p-value is the area to the left of the z-statistic
			&&& If the alternative hypothesis is `not equal to', the p-value is the area to the left of the negative absolute value of the z-statistic an the area to the right of the positive absolute value of the z-statistic
			&&& If the alternative hypothesis is `greater than', the p-value is the area to the right of the z-statistic
		&& P-value of a chi-square statistic: %TODO

	& \emph{Significance level:} %TODO

	& General process:
		&& Find the test statistic using the sample data
		&& Find the p-value using the test statistic
		&& Compare the p-value to the significance level
			&&& Conclusion: ``Since the p-value (<p-value>) is <less than/greater than> the significance level (<significance level>), we <do not> reject the null hypothesis. There is <sufficient/insufficient> evidence to conclude that <alternative hypothesis>.''

\end{easylist}
\subsection{One Population Proportion (Z-Statistic)}
	\label{subsec:analysis-of-population-proportions:one-population-proportion-z-statistic}
\begin{easylist}

	&
	& E.g. %TODO

\end{easylist}
\subsection{Two Population Proportions (Z-statistic)}
	\label{subsec:analysis-of-population-proportions:two-population-proportions-z-statistic}
\begin{easylist}

	& Process:
		&& Compute the proportion of subjects in each test group who satisfy the condition
		&& Compare the proportions using a bar graph
		&& Conclude which group has a greater/lesser proportion
		&& To compare population proportions, conduct a hypothesis test (see %TODO ref
			&&& Null hypothesis states that the population proportions are equal; alternative hypothesis states that the population proportions are unequal (less than, not equal to, or greater than)
			&&& Formula for the z-statistic for two population proportions:

	& E.g. Are the proportions...

\end{easylist}
\subsubsection{Confidence Interval}
	\label{subsubsec:analysis-of-population-proportions:two-population-proportions-z-statistic:confidence-interval}
\begin{easylist}

	& Formula (95\% confidence interval):
	\begin{math}
		(\hat{p_{1}} - \hat{p_{2}}) \pm 2 \cdot
		\sqrt
		{
			\frac
			{
				\hat{p_{1}} \cdot (1 - \hat{p_{1}})
			}{
				n_{1}
			}
			+ \frac
			{
				\hat{p_{2}} \cdot (1-\hat{p_{2}})
			}{
				n_{2}
			}
		}
	\end{math}

	& Example: [..] How does the proportion of people who have lung cancer...

	& There may be no difference between the proportion of students who use iPhones in UBC compared to the proportion of students who use iPhones in SFU.

\end{easylist}
\subsection{Multiple Population Proportions (Chi-Square Statistic)}
	\label{subsec:analysis-of-population-proportions:multiple-population-proportions-chi-square-statistic}
\begin{easylist}

	& The greater the difference, the greater the chi-square statistic
	& P-value always uses the right side of the normal distribution
	& Only concludes whether a relationship exists between two variables
	& Can compare many population proportions

\end{easylist}
\subsection{Errors}
	\label{subsec:analysis-of-population-proportions:errors}
\begin{easylist}

	& \emph{Type I error:} Rejection of H\textsubscript{o} from analysis of the sample data when H\textsubscript{o} is true
		&& I.e. False positive/confirmation of the alternative hypothesis; finding evidence where there is none
		&& May occur when H\textsubscript{o} is rejected
		&& E.g. Judging a person for a crime: \\
		H\textsubscript{o}: The person is not guilty. \\
		H\textsubscript{a}: The person is guilty. \\
		Truth: The person is not guilty (H\textsubscript{o} is true). \\
		Decision: The person is guilty (H\textsubscript{o} is rejected).
		&& Probability of its occurrence is directly proportional to the \hyperref[subsubsec:analysis-of-population-proportions:hypothesis-testing-for-population-proportions:introduction]{significance level}
			&&& Reducing significance level (and therefore, the probability of the type I error) increases the probability of the type II error

	& \emph{Type II error:} Failure to reject H\textsubscript{o} from analysis of the sample data when H\textsubscript{a} is true
		&& I.e. Failing to find evidence which exists
		&& May occur when H\textsubscript{o} is not rejected
		&& E.g. Judging a person for a crime: \\
		H\textsubscript{o}: The person is not guilty. \\
		H\textsubscript{a}: The person is guilty. \\
		Truth: The person is guilty (H\textsubscript{a} is true). \\
		Decision: The person is not guilty (H\textsubscript{o} is not rejected).
		&& Probability of its occurrence is inversely proportional to the \hyperref[subsubsec:analysis-of-population-proportions:hypothesis-testing-for-population-proportions:introduction]{significance level}
			&&& Increasing significance level (and therefore, the probability of the type II error) increases the probability of the type I error

	& E.g. A company will renew a contract with a radio station only if the station can find sufficient evidence to support that more than 20\% of the listeners have heard their ad. The station conducts a random survey of 400 people, 100 of which have heard the ad. \smallskip \\
	H\textsubscript{o}: 20\% of the listeners have heard the ad. \\
	H\textsubscript{a}: More than 20\% of the listeners have heard the ad. \smallskip \\
	A type I error will occur if 20\% of the listeners have heard the ad, but the sample data provides sufficient evidence to conclude that more than 20\% of the listeners have heard the ad. H\textsubscript{o} is true but rejected; H\textsubscript{a} is false but accepted. \\
	The possibility of this error can be reduced by decreasing the significance level. \smallskip \\
	A type II error will occur if more than 20\$ of the listeners have heard the ad, but the sample data does not provide sufficient evidence to reject the hypothesis that 20\% of the listeners have heard the ad. H\textsubscript{o} false but not rejected; H\textsubscript{a} is true but not affirmed. \\
	The possibility of this error can be reduced by increasing the significance level.

\end{easylist}
\subsection{Statistical Significance}
	\label{subsec:analysis-of-population-proportions:statistical-significance}
\begin{easylist}

	& P-value represents the probability that a given sample proportion is found, if the null hypothesis is correct, and therefore the probability of obtaining a difference of | sample proportion - test proportion | from the test proportion

	& \emph{Statistically significant:} Result which is unlikely to occur by chance
		&& Statistically significant result: P-value is less than the significance level
		&& Not statistically significant result: P-value is greater than the significance level

	& If a p-value is less than the significance level, then the difference between the sample proportion and the test proportion is statistically significant at the given significance level
	& If a p-value is greater than the significance level, then the difference between the sample proportion and the test proportion is not statistically significant at the given significance level

\end{easylist}
\clearpage


\section{Numerical Experiments}

	This section intends to be the most important in the chapter since it will take advantage of everything that has been previously studied and expand in the best possible way the implementation of the spectral methods studied in the previous section. For this, we will focus on the nonlinear problem defined in (\ref{IVP_Burgers}) directly, because after understanding these tools you want to have the ability to attack more complex problems such as (\ref{navierstokes}) which it is nonlinear, and in general, in most problems, it is not always possible to obtain analytical solutions, and then an option is to approximate them with numerical methods. \\

	According to what was studied in the previous section, we will start solving our problem with each method considering $\alpha> 0$, and later we will study the case when $\alpha = 0$. To find solutions we are going to use Euler's method, either explicit or implicit, to solve in the variable $t$, and due to the importance of carrying out numerical studies, we will try to give a detailed methodology about each method implemented to construct an algorithm that can be useful in the realization of computational codes. Finally, with the aim of giving a more detailed analysis, we will present some simulations of the results obtained.

	\subsection{Numerical Solutions for Burgers' Equation with Viscosity}
	\input{burgers_equation/deterministic/numerical_experiments/viscid/Viscid_Galerkin}
	\input{burgers_equation/deterministic/numerical_experiments/viscid/Viscid_Collocation}
	\input{burgers_equation/deterministic/numerical_experiments/inviscid/Inviscid}


\section{Simple model} \slabel{model}

We base our model on the buoyancy-drag models of \cite{Oron2001}:
\begin{equation}
(\rho_1 + \rho_2) \mathcal{V} \ddot{h} = (\rho_2 - \rho_1) g \mathcal{V} - C \dot{h}^2 \rho \mathcal{A}
\end{equation}
where $\rho_1$ and $\rho_2$ are the densities of the light and heavy fluid,
$\mathcal{V}$ is the characteristic volume of the bubble
$g$ is the acceleration,
$C$ is a drag-like coefficient, and
$\mathcal{A}$ is the characteristic cross sectional area of the bubble.
Making the Boussinesq approximation, $\rho_1 \approx \rho_2$ yields:
\begin{equation}
\ddot{h} = A g - \frac{C}{2} \dot{h}^2 \frac{\mathcal{A}}{\mathcal{V}}
\end{equation}
In the self-similar regime there is only one length-scale, so $\mathcal{A}/\mathcal{V} \sim 1 / \lambda$.
However, in the single-mode regime that is the focus of this study, the bubbles are elongated, producing two length scales: a span-wise scale $\lambda$ and a stream-wise scale $h$.
Therefore, for the smRTI $\mathcal{A}/\mathcal{V} \sim \frac{1}{h}$ and the model of Oron \etal yields un-bounded velocities:
\begin{equation}
\ddot{h} = A g - \frac{C}{2} \frac{\dot{h}^2}{h}
\end{equation}
Because the strength of the form drag relative to buoyancy decreases at high aspect ratio, we must consider other drag terms, such as skin drag, that grow at least linearly with $h$.

\subsection{Dynamics}

We begin by listing the external forces the bubble experiences.  The first is the buoyant force:
\begin{equation}
F_b = C_0 A g \lambda^2 h,
\end{equation}
where $C_0$ is an unknown coefficient.
The next is the form drag:
\begin{equation}
F_f = C_1 \lambda^2 \dot{h}^2,
\end{equation}
where $C_1$ is similar to a drag coefficient.
The next is the viscous, or skin, drag:
\begin{equation}
F_s = C_2 \nu h \dot{h},
\end{equation}
where $C_2$ is another unknown coefficient and
$\nu$ is the kinematic vicosity.

To complete the dynamic equation, we must characterize the inertia of the bubble.
The bubble is roughly cylindrical with a height $h$, so we expect an inertial term of the form $\lambda^2 h$.
However, consider the limit of $h \rightarrow 0$.
Here, streamlines must extend from bubble to spike, which has a characteristic separation $\lambda$ for an inertial term of the form $\lambda^3$.
Therefore, we expect the inertia to be a mix of a term that goes as $\lambda^2 h$ and one that goes as $\lambda^3$:
\begin{equation} \elabel{inertia}
I = C_3 \lambda^2 h + C_4 \lambda^3,
\end{equation}
where $C_3$ and $C_4$ are two more unknown coefficients.

The complete dynamic equation is:
\begin{equation}
\ddot{h} = \frac{C_0 A g \lambda^2 h - C_1 \lambda^2 \dot{h}^2 - C_2 \nu h \dot{h}}{C_3 \lambda^2 h + C_4 \lambda^3}
\end{equation}
Without loss of generality, we can let $C_0 = 1$ and simplify:
\begin{equation} \elabel{dynamics}
\ddot{h} = \frac{A g h - C_1 \dot{h}^2 - C_2 \nu (h/\lambda^2) \dot{h}}{ C_3 h + C_4 \lambda }
\end{equation}
We can non-dimensionalize by defining a dimensionless length and time:
\begin{equation}
z = \frac{h}{\lambda} \qquad \tau = \sqrt{\frac{A g}{\lambda}} t,
\end{equation}
which simplifes:
\begin{equation}
\ddot{z} = \frac{z - C_1 \dot{z}^2 - C_2 \text{Gr}^{-1/2} z \dot{z}}{C_3 z + C_4},
\end{equation}
where
the derivative is with repsect to $\tau$ and
$\text{Gr} = A_0 g \lambda^3 \nu^{-2}$ is the Grashof number.

\subsection{Mixing}

As the bubble height grows, the velocity approaches a terminal value specified by the balance between buoyancy and skin drag.
At terminal velocity, the flux of pure fluid into the bubble is bounded.
However, the interfacial mixing continues to grow with the interfacial area, which grows with $h$.
Therefore, for any finite diffusivity, the bubble will ultimately diffuse away.
For this reason, we must include the effects of interficial mixing, at least to the first order.

The quantity of mixed fluid, $m$ is computed directly from the time, bubble height, diffusivity, and initial interface thicnkess.
The quantity of mixed fluid is defined as the integral of the absolute value of the scalar:
\begin{equation}
	m(t) = \int \left( 1-\text{abs}\left[\phi(x,y,z,t)\right] \right) dV,
\end{equation}
where we assume the mean scalar is zero, $\int \phi dV = 0$.

We approximate the volume integral by a 1D integral across the interface multiplied by the surface area:
\begin{equation}
	m(t) \approx S \int \left( 1- \text{abs}\left[\phi_1(r)\right] \right) dr
\end{equation}
where $S$ is the surface area and
$\phi_1$ is a model 1D scalar profile:
\begin{equation}
\phi_1(r) = \frac{1}{2} \left( \erf\left[\frac{r}{\delta}\right] - \erf\left[\frac{r - d}{\delta}\right] \right),
\end{equation}
where $\delta$ is the interface width and
$d$ is the diameter of the bubble.

The surface area has contributions from the bubble tip and side walls:
\begin{equation} \elabel{surface_area}
S = \left(C_6 \lambda^2 + C_5 \lambda h\right)
\end{equation}
where $C_5$ and $C_6$ are unknown coefficients.
$C_5$ scales the perimeter of span-wise slices of the bubble while $C_6$ rescales bubble tip.

To the first order, the diameter is half the wavelength: $d \approx \lambda / 2$.
However, the cylindrical bubbles do not always fill the span-wise domain.
This can be seen by values of $C_5$ that are below $4$, the value corresponding to space-filling rectangular bubbles.
Therefore, we adjust the diameter using $C_5$:
\begin{equation}
d = \frac{\lambda}{2} \frac{C_5}{4}
\end{equation}

The interface width is modeled by simple 1D diffusion:
\begin{equation}
\delta(t) = 2 \sqrt{D (t + t_0)},
\end{equation}
where $t_0$ is chosen to match $\delta(0)$ to the initial condition.

We perform the integral through the bubble:
\begin{equation} \elabel{profile1d}
\begin{split}
	\int_{-d/2}^{d/2} \left(1- \left|\phi_1(r)\right| \right) dr &= \frac{\delta}{\sqrt{\pi}} \left( 1 - \exp\left[-\frac{d^2}{\delta^2}\right]\right) \\
&+ d \left(1 - \erf\left[\frac{d}{\delta}\right]\right).
\end{split}
\end{equation}

The mixed mass must still be connected to the dynamics equation via the Atwood number:
\begin{equation} \elabel{effective-atwood}
A = A_0 \left( 1 - \frac{m}{V}\right),
\end{equation}
where $A_0$ is `pure` Atwood number and
$V$ is the volume of the bubble.
As in the dynamics equation, we define the volume as a mixture of $\lambda^3$ and $\lambda^2 h$:
\begin{equation}
V = \left(C_8 \lambda^3 + C_7 \lambda^2 h\right),
\end{equation}
where $C_7$ and $C_8$ scale the volume analogously to $C_5$ and $C_6$.

The volume of mixed fluid, $m(t)$, can be measured directly in the simulations.
This gives meaning to the value of $m(t)$ independent of the ratio $m(t)/V$.
Therefore, unlike in the dynmics, where a coefficient could be discarded without loss of generality, all four of $C_5, C_6, C_7$ and $C_8$ are nessesary.
The overall scale factor can not be removed if we want to compare to mixed volume measurements.

\subsection{Coefficient constraints}
First, consider the limit where $D = 0$, $\nu = 0$, and $h \rightarrow 0$.
The dynamical equation becomes
\begin{equation} \elabel{first_constraint}
\ddot{h} = \frac{A g }{C_4 \lambda} h,
\end{equation}
which matches Rayleigh's original linear stability analysis if
\begin{equation}
C_4 = 1/(2 \pi).
\end{equation}

When $\nu > 0$, the growth rate $\ddot{h}$ is given by Duff's linear theory:
\begin{equation}
\ddot{h} = \left(\sqrt{A g k + \nu^2 k^4} - \nu k^2\right)^2 h,
\end{equation}
where $k = 2\pi / \lambda$ is the wavenumber.
Setting this equal to \eref{first_constraint} yields:
\begin{equation} \elabel{c4}
C_4 = \frac{1 + 2x\left(\sqrt{1 + x^2} + x\right)}{2\pi},
\end{equation}
where:
\begin{equation}
x = \sqrt{\frac{8 \pi^3 \nu^2}{A g \lambda^3}} = \sqrt{\frac{(2 \pi)^3}{\text{Gr}}}
\end{equation}
and Gr is the Grashof number.

Next, consider the initial quantity of mixed mass for small sharp interfaces, $\delta(0), a_0 \rightarrow 0$.
We assume the initial condition is an error function profile:
\begin{equation}
M(t=0) = \lambda^2 \int_{-\infty}^{\infty} \text{erf}\left[\frac{z}{\delta}\right] = \frac{2\lambda^2 \delta}{\sqrt{\pi}}.
\end{equation}
Equating this to the product of \eref{surface_area} and \eref{profile1d}
\begin{equation}
\frac{2 \lambda^2 \delta}{\sqrt{\pi}}= \frac{C_6 \lambda^2 \delta}{\sqrt{\pi}},
\end{equation}
which implies that $C_6 = 2$.

Next, consider the limit when $\delta \rightarrow 0 $ and $h \rightarrow 0$.
In the linear theory, the Atwood number is rescaled:
\begin{equation}
A = \frac{A_0}{1 + \pi^{-1/2} k \delta} = A_0 \left(1 - \frac{k \delta}{\sqrt{\pi} + k \delta}\right)
\end{equation}
We equate this to \eref{effective-atwood}:
\begin{equation}
\frac{2 \pi \delta}{\lambda (\sqrt{\pi} + 2 \pi \delta / \lambda)} = \frac{C_6 }{C_8}\frac{\delta}{\lambda \sqrt{\pi} },
\end{equation}
or
\begin{equation}
\frac{C_6}{C_8} = \frac{2 \pi \sqrt{\pi}}{\sqrt{\pi} + 2 \pi \delta / \lambda}.
\end{equation}
The variable $\delta$ is associated with the mixing model, not the dynamics, so it would be convenient to have $C_8$ independent of $\delta$.
We've defined $C_6 = 2$ in the limit of $\delta(0), a_0 \rightarrow 0$, so we can add a term that goes to zero at $\delta = 0$:
\begin{equation}
C_6 = \frac{2}{1 + 2 \sqrt{\pi} \delta / \lambda},
\end{equation}
which constrains:
\begin{equation}
C_8 = \frac{1}{2\pi},
\end{equation}
which is the same as $C_4$ in the inviscid case.

\subsection{Coefficient estimation}

\begin{figure*}
\begin{subfigure}[b]{\columnwidth}
\includegraphics[width=\columnwidth]{figs/slice}
\caption{Scalar $\phi$}
\end{subfigure}
\begin{subfigure}[b]{\columnwidth}
\includegraphics[width=\columnwidth]{figs/slice_w}
\caption{Vertical component of the velocity, $w$}
\end{subfigure}
\caption{ \flabel{bubble_geom}
Slices of the scalar and vertical component of the velocity at early times and high Grashof number.
The arrows indiciate the dependence of the model terms on different span-wise length scales, and are identical in both figures.
$C_1$ is related to the maximum cross sectional diameter of the bubble in the velocity field.
$C_2$ is related to the nominal side-wall diameter of the bubble in the velocity field.
$C_5$ is related to the nominal side-wall diameter of the bubble in the scalar field.
$\delta$ is related to the interface thickness in the scalar field.
The slice is of the plane $x=y$, which passes through only bubble centers.
}
\end{figure*}

The parameter $C_1$ scales the form drag and serves as a drag coefficient.
Because we have let $C_0 = 1$, the force balance is really aggreated over two rising bubbles and two falling spikes, each with diameter $\lambda / 2$.
Therefore, we multiply the force on a single bubble of diameter $\lambda/2$ by 4.
Now, we relate $C_1$ to the drag coefficient $C_d$ in the drag equation:
\begin{equation}
	C_1 \lambda^2 \dot{h}^2 = 2 C_d \mathcal{A} \dot{h}^2,
\end{equation}
where $\mathcal{A}$ is the cross sectional area,
so $C_1$ can be estimated using drag coefficients of similar objects:
\begin{equation} \elabel{prior_c1}
C_1 = 2 C_d \frac{
	\mathcal{A}}{\lambda^2},
\end{equation}
where $\mathcal{A} \approx (\lambda/2)^2$.
Initially, the bubble tip is a flat plate, which has $C_d = 1.28$.
At late times, the bubble is closer to an elongated cylinder, which has $C_d = 0.82$, but with a somewhat streamlined tip, which further reduces drag.
We expect $C_1 \approx 0.64$, but possibly much smaller if the bubble takes a streamlined shape.
However, if the bubble spreads to have a diameter greater than $\lambda / 2$, $C_1$ could be greater than $0.64$.

Next, onsider the limit when $h \rightarrow \infty$ and $D = 0$:
The dynamical equation becomes
\begin{equation}
\ddot{h} = \frac{A_0 g - C_2 \nu (1/\lambda) \dot{h}}{C_3}
\end{equation}
which leads to a terminal velocity of:
\begin{equation} \elabel{visc_vel}
\dot{h} = \frac{A_0 g \lambda^2 }{C_2 \nu},
\end{equation}
or a non-dimensional velocity, i.e. Froude number:
\begin{equation}
\text{Fr} = \frac{d z}{d \tau} = \frac{\sqrt{\text{Gr}}}{C_2}.
\end{equation}
The case of extended bubbles and spikes affected only by viscous drag is highly analogous to flow through a square duct.
The pressure drop, $\Delta p$, along a duct is given by the Darcey-Weisbach formula:
\begin{equation}
\Delta p = \frac{f_D}{2} \frac{v^2 L}{d},
\end{equation}
where $L$ is the length of the duct,
$v$ is the mean velocity,
$d$ is the hydraulic diameter,
and $f_D$ is the Darcy friction factor.
In our case, $L = h$, $v = \dot{h}$, and $\Delta p = A_0 g h$, so:
\begin{equation}
A g = \frac{f_D}{2} \frac{\dot{h}^2}{d},
\end{equation}
For laminar flows in circular pipes, $f_D = \bar{f}_D = 64 / \text{Re}$, so:
\begin{equation}
A g = \frac{f_D}{\bar{f}_D} 32 \nu \frac{\dot{h}}{d^2}.
\end{equation}
The hydraulic diameter $d = \lambda / 2$, so:
\begin{equation}
\dot{h} = \frac{\bar{f}_D}{f_D} \frac{A_0 g \lambda^2}{128 \nu} .
\end{equation}
This gives an estimate for $C_2$:
\begin{equation}
C_2 \approx 128 \frac{f_D}{\bar{f}_D},
\end{equation}
where the ratio $f_D / \bar{f}_D$ is affected by the geometry and departure from laminar flow.
For example, for square ducts $f_D/\bar{f}_D \approx 0.889$, so $C_2 \approx 114$~\cite{ghiaasiaan2011convective}.

The product of the coefficient $C_5$ and $\lambda h$ gives the interfacial area of the side of the bubble.
Therefore, $C_5$ captures information both about the bubble shape and the bubble diameter.
If the bubbles had diameter $\lambda / 2$ and were smooth and rectangular, then $C_5 \approx 4$
If the bubble has a lower surface area shape, e.g. cylindrical, or is thinner then $C_5 < 4$.

These diameters, along with the relevant span-wise length scales in the preceeding coefficients, are sketched in \fref{bubble_geom}.
The diameter associated with $C_1$ is defined as the entrainment width at the bubble tip, in contrast to the width at the bubble center used for $C_2$.
The diameter associated with $C_5$ is defined similarly to $C_2$, but with respect to the scalar interface.
The interface width $\delta$ also depends on the scalar representation of the bubble.


% As a sample LaTeX document, this is an actual assignment
% written in LaTeX with my template for MATH 417,
% Honors Real Variables (Measure Theory) at University of Alberta.
% This source has been released with permission with the instructor,
% Professor John C. Bowman as the solutions are available at
% https://www.math.ualberta.ca/~bowman/m417/assign1.pdf

% Of course, don't plagiarize! Just because you have the solutions
% to whatever problems there is does not mean you should copy it.
% The only way to learn math is to do the problems yourself!

% Moreover, I do not warrant any correctness of these solutions.
% These are simply what I have written which may contain mistakes.

% UPDATE (2/1/19): Unfortunately, by the request of my professor
% John C. Bowman with potential plagiarism in future years,
% I removed all the solutions and replaced them with [Redacted].
% However, all the problem texts are kept.

% ------- END DISCLAIMER -------------

\documentclass{article}
\usepackage{import}

% Change this if you want a different margin.
% Current margin is smaller than the LaTeX default but larger than
% Standard word documents.
\newcommand{\SMALLMARGINS}{1.5in}

% Change this directory if you put your template files in a different directory.
\newcommand{\basedir}{../../}

% Comment out this if you want an alternative header (that I no longer support)
\newcommand{\USESCSTYLE}{}

% If you are using Crowdmark or require new page for each problem, uncomment this.
% \newcommand{\CROWDMARK}{}

\import{\basedir/base/}{mathPreamble}

% Assignment number and Course Name
\newcommand{\assignmentnum}{1}
\newcommand{\coursename}{MATH 417}

% Who are you, obviously.
\newcommand{\studentname}{Jamie van Lindenberg}
\newcommand{\studentid}{1234567}
\newcommand{\studentemail}{rassamee@ualberta.ca}
\newcommand{\coursesec}{Q1}

\begin{document}

% Comment this out if you do not want the title.
% Alternatively, use
% \import{\basedir/base/}{hwTitle}

% for a cover page.
\integratedtitle

\problemsec

Let $(S_n)_1^\infty$ sequences of sets.
Define
\[
    \liminf_{n\to\infty} S_n\defeq \bigcup_{k=1}^\infty \bigcap_{n=k}^\infty S_n, \quad \limsup_{n\to\infty} S_n\defeq \bigcap_{k=1}^\infty \bigcup_{n=k}^\infty S_n.
\]
Show that
\[
    \liminf_{n\to\infty} S_n\subset \limsup_{n\to\infty} S_n.
\]

\begin{solution}
    [Redacted].
\end{solution}

\problemsec
Let $(S_j)_1^\infty$ sequence of sets.
Define $T_1=S_1$ and
\[
    T_j\defeq S_j \setminus \bigcup_{i=1}^{j-1}S_i, \quad j\in 2,3, \ldots.
\]
Prove that $(T_j)_1^\infty$ is a sequence of pairwise disjoint sets satisfying
\[
    \bigcup_{j=1}^n T_j=\bigcup_{j=1}^n S_j, \quad \text{all }n\in\naturals
\]

\begin{solution}
    [Redacted].
\end{solution}

\problemsec
Suppose that $A, B\subset \real^d$ with $m^*(A),m^*(B)<\infty$.
Show that
\[
    \abs{m^*(A)-m^*(B)}\le m^*(A\symdiff B)
\]

\begin{solution}
    [Redacted].
\end{solution}

\problemsec
Give examples of
\begin{enumerate}
    \item a bounded countable union of Jordan measurable sets that is not Jordan measurable;
    \item a bounded countable intersection of Jordan measurable sets that is not Jordan measurable.
\end{enumerate}

\begin{solution}
    [Redacted].
\end{solution}

\problemsec
Determine whether each of the following sets are countable.
Justify your answers.

\subsection{Part (a)}
The set of all mappings from $1\ldots,N$ to $\naturals$ where $N\in\naturals$ is fixed.

\begin{solution}
    [Redacted].
\end{solution}

\subsection{Part (b) and (c)}
The set of all mappings $S$ from $\naturals$ to $[0,1]$ and the restriction $R$ of $S$ such that each $f\in R$ is eventually $0$ (i.e. $\lim_{n\to\infty} f(n)=0$ for $f\in R$)

\begin{solution}
    [Redacted].
\end{solution}

\subsection{Part (d)}
The set of all finite subset of $\naturals$.

\begin{solution}
    [Redacted].
\end{solution}

\problemsec

Let $(x_\alpha)_{\alpha\in A}$ a collection of $x_\alpha \in \real_{\ge 0}$ such that
\[
    \sum_{\alpha\in A} x_\alpha <\infty.
\]
Show that $x_\alpha=0$ for all but at most countably many $\alpha \in A$, even if $A$ is itself uncountable.

\begin{solution}
    [Redacted].
\end{solution}

\problemsec
Let $S$ a set.
Prove that there is no surjective map from $S$ to $\powerset(S)$.

\begin{solution}
    [Redacted].
\end{solution}

\problemsec

Let $S$ a countable set.
Show that
\[
    \mathfrak{F}(S)\defeq \set{F\subset S: F \text{ finite}}
\]

\begin{solution}
    [Redacted].
\end{solution}
\end{document}


\chapter{Integer domains}\label{ch4:integers}
In this chapter we focus on the domain of integers, and try to determine sufficient conditions for the non existence of under-approximation abstract domains.

\section{Sum completeness}
The concept of complete abstraction (Definition \ref{ch2:def:complete-abstr}) for a function is very important in over-approximation abstract interpretation because it correspond to analyses without false alarms. In general it's really rare that all the basic transfer functions of a program are complete, with boolean guards almost never being such. However, it's not uncommon that \textit{some} of them are complete: for instance, in the interval domain sum and multiplication by a constant are complete operations.

In under-approximation, completeness is much harder to achieve: the reason is the fact that complete abstractions allow to ``distribute" $\alpha$ over the operation, so having even a single set that is abstracted to $\bot$ forces, by strictness of concrete operations, all concrete sets that can be obtained applying the operation to that specific set to be $\bot$ as well. This idea is more easily seen in the proof of the following proposition.
\begin{prop}\label{ch3:th:sum-complete-trivial}
	Let $\ugi{\pow(\setZ)}{\alpha}{A}$ be an under-approximating Galois insertion. If $A$ is complete for the concrete sum\footnote{In this proposition we're considering as usual the additive extension of $+$.} then it's trivial.
\end{prop}
\begin{proof}
	Let $+^{\flat}$ be the complete abstraction of $+$.
	Since all sound under-approximations for a singleton set are $\emptyset$ and the singleton itself, we can distinguish two cases.

	If all singletons $\{ n \}$ are representable in $A$ then, by union closure (Proposition \ref{ch2:th:under-gc-union-closure}) we get that any $S \in \pow(\setZ)$ is also in $A$ since it's union of singletons. This in turn entails $A = \pow(\setZ)$, that is trivial.

	Otherwise, there exists an integer $\bar{n}$ such that $\alpha(\{ \bar{n} \}) = \emptyset$. Consider then an arbitrary set of integers $S \subseteq \setZ$.
	\begin{align*}
		\alpha(S) &= \alpha(S - \{ \bar{n} \} + \{ \bar{n} \} )      &&[\text{concrete operation}] \\
		&= \alpha(S - \{ \bar{n} \}) +^{\flat} \alpha(\{ \bar{n} \}) &&[\text{completeness of }+^{\flat}] \\
		&= \alpha(S - \{ \bar{n} \}) +^{\flat} \emptyset             &&[\alpha(\{ \bar{n} \}) = \emptyset] \\
		&\subseteq \alpha(S - \{ \bar{n} \}) + \emptyset = \emptyset &&[\text{correctness of }+^{\flat}]
	\end{align*}
	This means that any set $S$ is abstracted to $\emptyset$, so $A = \{ \emptyset \}$.
\end{proof}

In our opinion, the reason this doesn't happen in over-approximation is an instance of the asymmetry born of basic transfer functions. As we introduced in the previous chapter, many operations, defined as the additive extension of basic constructs, increase the output cardinality. For instance, the sum of two sets of integers have a cardinality that is at least the largest of the two inputs. This asymmetry allows to get many subsets of the concrete domain as the result of applying the function to a small set (for instance a singleton) but not to a big one.
Then in under-approximation small sets can be composed by union to make the abstract domain grow until it's too big to be feasible, while in over-approximation sets are composed by intersection. To emulate this behaviour, we would need big sets (eg. the whole $C$ without a single value) so that their intersections give rise to lots of smaller sets, but this conflicts with the ``direction" of basic transfer functions.

Of course in general completeness is a very strong property to require, so in the next section we'll introduce a weaker notion and show that it is enough to prevent the existence of under-approximation abstract domains.

\section{Non emptying-ness}
Completeness is a concept tailored to over-approximation: we use it to avoid false alarms while over-approximating the set of possible values. In bug catching, an alternative, weaker definition may be enough
\begin{definition}[Non emptying]\label{ch3:def:non-emptying}
	Let $\ugc{C}{\alpha}{\gamma}{A}$ be an under-approximation Galois connection, $f : C \rightarrow C$ a monotone function on $C$ and $f^{\flat} = \alpha \circ f \circ \gamma$ its best correct approximation in $A$.
	We say that $f$ is \textit{non emptying} (in $A$) if, for any concrete value $c$, if both $\alpha(c) \neq \bot$ and $\alpha(f(c)) \neq \bot$ then also $f^{\flat}(\alpha(c)) \neq \bot$.
\end{definition}

Unlike completeness, this definition doesn't mean that the analysis will find the best possible result the abstraction can, but just that if it starts from something ($\alpha(c) \neq \bot$) and it can find something ($\alpha(f(c)) \neq \bot$) then it will find at least one of the possible results ($f^{\flat}(\alpha(c)) \neq \bot$).
The rationale behind it is that the analysis shouldn't get to $\bot$ because, as stated in the previous chapter, recovery from it is hard. The definition exactly prevents this when it's caused only by imprecision of the abstracted function.

Clearly completeness implies non emptying-ness since $f^{\flat}(\alpha(c)) = \alpha(f(c)) \neq \bot$, but the converse is not true, as shown in the following example.
\begin{example}\label{ch3:ex:ne-not-complete}
	Consider the under-approximation domain $\Int_0$ of intervals containing $0$ (Example \ref{ch2:ex:intervals-0}). Note that in this example we write intervals to denote subsets of $\setZ$, following the identification given by the Galois insertion, as well as use both $\bot$ and $\emptyset$ to refer to the same object (the empty subset of $\setZ$).

	Consider the additive extension of the function $f(x) = x + 1$. This function is not complete in the abstract domain since, for instance on the concrete element $S = \{ -1 \}$, we have
	\begin{align*}
		&\alpha(f(S)) = \alpha(\{ 0 \}) = [0, 0] \\
		&f^{\flat}(\alpha(S)) = f^{\flat}(\bot) = \bot
	\end{align*}

	However this function is non emptying.
	First we show that any concrete element $S \in \pow(\setZ)$ is such that $\alpha(S) \neq \emptyset$ if and only if $0 \in S$.
	Suppose that $0 \in S$. Then by monotonicity of $\alpha$
	\[
	\alpha(S) \supseteq \alpha(\{ 0 \}) = [0, 0] \supset \emptyset
	\]
	Conversely, assume that $\alpha(S) \neq \emptyset$. Since all elements of $\Int_0$ but the empty set contains $0$, and $\alpha \preceq \id$, we get
	\[
	0 \in \alpha(S) \subseteq S
	\]
	that is exactly $0 \in S$.

	Consider now an arbitrary concrete element $S$ such that $\alpha(S) = \bot$ and $\alpha(f(S)) = \bot$. As shown above, the former condition is equivalent to $0 \in S$, and the latter to $0 \in f(S)$, that is in turn equivalent to $-1 \in S$. Using these two hypothesis, we can show
	\begin{align*}
		&S \supseteq \{ -1, 0 \} \\
		\implies& \alpha(S) \supseteq \alpha(\{ -1, 0 \}) = [-1, 0] \\
		\implies& f(\alpha(S)) \supseteq f([-1, 0]) = [0, 1] \\
		\implies& f^{\flat}(\alpha(S)) = \alpha(f(\alpha(S))) \supseteq \alpha([0, 1]) = [0, 1] \supset \emptyset
	\end{align*}
	that is exactly $f^{\flat}(\alpha(S)) \neq \bot$, the non emptying condition.
\end{example}
However, even this weaker notion allows to prove some results on $A$ under some assumptions, as we show in the remainder of this chapter.

We assume there is an under-approximation Galois insertion $\ugi{\pow(C)}{\alpha}{A}$. Moreover, we say an element $S \in \pow(C)$ is \textit{representable} if it belongs to $A$, or equivalently if $\alpha(S) = S$.

\begin{definition}\label{ch3:def:repr-with-set}
	Let $S \subseteq C$ be a subset of $C$. We say that $d \in C$ is \textit{representable with $S$} if $S \cup \{ d \}$ is representable. We call $R(S)$ the set of elements of $C$ representable with $S$, ie.
	\[
	R(S) = \{ d \in C \svert \alpha(\{ d \} \cup S) = \{ d \} \cup S \}
	\]
\end{definition}
For the sake of brevity, we shall write $R$ for $R(\emptyset)$, the set of representable values of $C$, and $R(c)$ for $R(\{ c \})$ where $c \in C$ is any concrete value.

We now present a lemma about non emptying functions that makes reasoning easier. This lemma is weaker than the definition, but is nevertheless one of the main tools we use when considering non emptying functions.

\begin{lemma}\label{ch3:th:f-non-repr-pair}
	Let $f: C \rightarrow C$ be non emptying, $c \in R$ and the pair $\{ c, \bar{c} \}$ be not representable, ie. $\bar{c} \notin R(c)$. If $f(\bar{c}) \in R$ then also $f(c) \in R$.
\end{lemma}
\begin{proof}
	Since $\{ c, \bar{c} \} \supseteq \{ c \}$ we have
	\[
	\alpha(\{ c, \bar{c} \}) \supseteq \alpha(\{ c \}) = \{ c \}
	\]
	where the equality follows because $c \in R$ is representable. Since by correctness
	\[
	\{ c, \bar{c} \} \supseteq \alpha(\{ c, \bar{c} \})
	\]
	and $\alpha(\{ c, \bar{c} \})$ can't be the pair $\{ c, \bar{c} \}$ because this is not representable and hence not in the image of $\alpha$, it should be the case that $\alpha(\{ c, \bar{c} \}) = \{ c \}$.

	Now
	\[
	\alpha(f(\{ c, \bar{c} \})) = \alpha(\{ f(c), f(\bar{c}) \}) \supseteq \alpha(\{ f(\bar{c}) \}) = \{ f(\bar{c}) \}
	\]
	where the last equality follows by the hypothesis that $f(\bar{c}) \in R$.
	This in particular means that $\alpha(f(\{ c, \bar{c} \})) \neq \emptyset$, and together with the fact that $\alpha(\{ c, \bar{c} \}) = \{ c \} \neq \emptyset$, since $f$ is non emptying we get that
	\[
	f^{\flat}(\alpha(\{ c, \bar{c} \})) \neq \emptyset
	\]

	From this we find
	\begin{align*}
		\emptyset &\subset f^{\flat}(\alpha(\{ c, \bar{c} \})) \\
		&= f^{\flat}(\{ c \}) \\
		&= \alpha \circ f(\{ c \}) \\
		&= \alpha(\{ f(c) \})
	\end{align*}
	Again by correctness we have $\alpha(\{ f(c) \}) \subseteq \{ f(c)\}$, and since this can't be empty it should be exactly $\alpha(\{ f(c) \}) = \{ f(c) \}$, that is $f(c) \in R$.
\end{proof}

The goal of this lemma is to use non emptying functions to get new representable elements, as this will be our main tool to prove non existence of under-approximation abstract domains.

We now apply this definition on integer domains in two different situations, on the infinite domain $\pow(\setZ)$ and on the finite $\pow([-N; N])$ of machine integers.

\subsection{Infinite domain}
In this subsection we focus on the infinite concrete domain $\pow(\setZ)$.
\begin{assumption}
	We assume that an abstract domain $A$, to be feasible for analyses, must be at most countable.
\end{assumption}
We do this assumption because we want to represent abstract elements with an amount of bits comparable with that of concrete \textit{values}, to have a complexity comparable with a single concrete execution of the program and not exponentially greater. Thus, we require the size of the abstract domain to be that of $\setZ$, the set of values handled by the program, and not the concrete domain $\pow(\setZ)$.

We first present a simple cardinality estimate that will be useful in the proof of the following result. The goal of this lemma is to show that some sets must be ``small" (in some sense, in this case finite), so we can find a contradiction showing that one of these sets is actually ``big". We follow this line of reasoning in almost all of our proofs of non existence of abstract domain with some properties.

\begin{lemma}\label{ch3:th:R-S-bound-integer-inf}
	For any fixed subset $S \subseteq \setZ$, $R(S)$ is finite.
\end{lemma}
\begin{proof}
	By union closure of the abstract domain (Proposition \ref{ch2:th:under-gc-union-closure}), any set $S \cup T$ for $T \subseteq R(S)$ is representable too, since it can be expressed as the union of representable sets:
	\[
	S \cup T = \bigcup\limits_{x \in T} (S \cup \{ x \})
	\]
	and $S \cup \{ x \}$ is representable since $x \in R(s)$.

	The number of those sets is the cardinality of $\pow(R(S))$, and since $A$ is at most countable the set $R(S)$ can't be infinite.
\end{proof}

We now present the first result that shows that requiring some functions to be non emptying makes impossible to create an abstract domain. The main idea is to define an infinite sequence of representable elements, that is in contradiction with the previous lemma that says that $R = R(\emptyset)$ is finite.
In order to define such a sequence, we want to use Lemma \ref{ch3:th:f-non-repr-pair}: we start from an initial representable $n_0$ and from a value $\bar{n}$ not representable with it, then find a non-emptying $f$ that maps $\bar{n}$ into $n_0$, so that $f(\bar{n})$ is representable and we can then apply the lemma to get the new representable element $f(n_0)$. We then iterate this procedure, changing $f$, to build the infinite sequence.

We believe the assumption that there exists an initial representable value is not very restrictive since initializations like \code{x = 0} must be abstracted to $\bot$ if $0$ is not representable.

\begin{prop}\label{ch3:th:ne-sum-nonexsistence-inf}
	Let $\ugi{\pow(\setZ)}{\alpha}{A}$ be an under-approximation Galois insertion, and assume that there is an integer $n_0$ that is representable. Then it can't be the case that all the functions of the form $f_n(x) = x + n$ are non emptying in $A$.
\end{prop}

\begin{proof}
	Assume by contradiction that all $f_n$ are non emptying in $A$.
	By hypothesis, $n_0 \in R$, and $R(n_0)$ is at most finite by Lemma \ref{ch3:th:R-S-bound-integer-inf}. Since $\setZ$ is infinite, this means there exists an $\bar{n} \in \setZ \setminus R(n_0)$, that is an element such that the pair $\{ n_0, \bar{n} \}$ is not representable.

	Let $d = n_0 - \bar{n}$ and consider $f_d$. We assumed it to be non emptying, so we can apply Lemma \ref{ch3:th:f-non-repr-pair}: $n_0$ is representable while the pair $\{ n_0, \bar{n} \}$ is not, and
	\[
	f_d(\bar{n}) = \bar{n} + d = \bar{n} + n_0 - \bar{n} = n_0
	\]
	so it's representable. Hence also $f_d(n_0) = n_0 + d$ is representable.

	Following this idea, we can prove by induction on $t$ that $n_0 + t d$ is representable for all $t$. The base step $t = 0$ is the hypothesis that $n_0$ is representable.
	For the inductive step, assume $n_0 + (t - 1) d$ is representable, and consider $f_{t d}$. We assumed this non emptying, so we can apply again Lemma \ref{ch3:th:f-non-repr-pair} to the pair $\{ n_0, \bar{n} \}$:
	\[
	f_{t d}(\bar{n}) = \bar{n} + t d = \bar{n} + n_0 - \bar{n} + (t - 1) d = n_0 + (t - 1) d
	\]
	that is representable by inductive hypothesis. So we get that $f_{t d}(n_0) = n_0 + t d$ is representable too, that is exactly the inductive step.

	Since $\bar{n}$ is not representable while $n_0$ is, we have $n_0 \neq \bar{n}$, or equivalently $d \neq 0$, hence $\{ n_0 + t d \svert t \in \setN \}$ is infinite.
	Moreover $\{ n_0 + t d \svert t \in \setN \} \subseteq R$ by the induction above, but this is impossible since $R$ must be finite by Lemma \ref{ch3:th:R-S-bound-integer-inf}.
\end{proof}

\subsection{Finite domain}
Now we move to a slightly different setting: we assume as concrete domain $\pow([-N, N])$, the power set of a finite, symmetric interval for some large integer value $N$, and we assume all operations are performed ``in machine arithmetic", that is whenever the result is greater than $N$ it wraps back to $-N$ because of overflows. This correspond to work modulo $2N + 1$ taking the unique representative of each congruence class in the interval $[-N, N]$ of interest.
\begin{assumption}
	We assume that an abstract domain $A$, to be feasible, must have cardinality polynomial in $N$.
\end{assumption}
This assumption guarantees that the number of bits required to represent an abstract element is linear in that for concrete elements.

In this section we'll use asymptotic notation for some quantities. For this to be completely formal we should define a sequence of abstract domain $A_N$, each one for the concrete domain $\pow([-N, N])$, then define a sequence of values for each quantity we want to estimate, and take the limit of this sequence for $N$ going to infinity. However we do believe all these formal details would clutter both statements and proofs, making hard to get insight. For this reason, we avoid all this, just (ab)using the intuitive meaning associated with the notation.

\begin{lemma}\label{ch3:th:R-S-bound-integer-fin}
	For any fixed subset $S \subseteq \setZ$, $\lvert R(S) \rvert = O(\log(N))$.
\end{lemma}
\begin{proof}
	As in the proof of Lemma \ref{ch3:th:R-S-bound-integer-inf}, by union closure any set $S \cup T$ for $T \subseteq R(S)$ is representable. We then have
	\[
	\poly(N) = \abs{A} \ge \abs{\pow(R(S))} = 2^{\abs{R(S)}}
	\]
	so, taking log at both sides, $\abs{R(S))} = O(\log(N))$.
\end{proof}

The following proposition uses the same proof line as Proposition \ref{ch3:th:ne-sum-nonexsistence-inf} above: we define a sequence of representable elements, and prove that they are too much since, by the previous lemma, $R$ is quite small.

\begin{prop}\label{ch3:th:ne-sum-nonexsistence-fin}
	Let $\ugi{\pow([-N, N])}{\alpha}{A}$ be an under-approximation Galois insertion, and assume that there is an integer $n_0$ that is representable. Then it can't be the case that all the functions of the form $f_n(x) = x + n$ (modulo $2N + 1$) are non emptying in $A$.
\end{prop}
\begin{proof}
	Let $r = \abs{R(n_0)}$. By the previous lemma \ref{ch3:th:R-S-bound-integer-fin} we know that $r = O(\log(N))$. Fix an element $\bar{n} \notin R(n_0)$ not representable with $n_0$ such that
	\[
	d = n_0 - \bar{n} \le r + 1
	\]
	This element should exists because otherwise all elements in the interval $[n_0 - r - 1; n_0 - 1]$ (modulo $2N + 1$) would be representable with $n_0$, that is impossible since they are $r + 1 = \abs{R(n_0)} + 1$.

	Following the proof of proposition \ref{ch3:th:ne-sum-nonexsistence-inf}, we can show by induction that for all $t \ge 1$ the value $f_{td}(\bar{n}) = n_0 + (t - 1) d$ is representable. However, all these values are different from one another for
	\[
	1 \le t < \frac{2N + 1}{d}
	\]
	and we also know that
	\[
	\frac{2N + 1}{d} > \frac{2N}{r + 1} = \frac{N}{O(\log(N))}
	\]
	But this is a contradiction since all these values are representable while $\abs{R} = O(\log(N))$ by lemma \ref{ch3:th:R-S-bound-integer-fin}, and
	\[
	\frac{N}{\log(N)} = \omega( \log(N) )
	\]
\end{proof}

\subsection{Consequences}
The previous two Propositions \ref{ch3:th:ne-sum-nonexsistence-inf} and \ref{ch3:th:ne-sum-nonexsistence-fin} show how requiring some functions to be non emptying prevents the existence of under-approximation abstract domains. We considered sums, some of the most common basic transfer functions over integers, and proved that no abstract domain small enough makes them all non-emptying, both for the infinite domain of all integers $\setZ$ and for a finite, machine representable subset $[-N; N]$.

We presented the notion of non-emptying function motivated by observations of Chapter \ref{ch3:comparison}. We can define correct under-approximation domains, for instance by taking complements of known over-approximation ones (see Example \ref{ch2-5:ex:complement-intervals}), but the point is that these are not useful for real analyses. Requiring a domain to be non-emptying for some basic transfer functions is an attempt to force that usefulness, and we showed no domain satisfies this condition for integers.

Of course this is not definitive evidence, since there may exist practical domains that doesn't make all sums non-emptying, or big domains (that is, not satisfying the assumption of having cardinality comparable with the set of values and not its power set) that can be represented efficiently, but we believe this to be nevertheless an interesting clue about the difficulty of designing under-approximation abstract domains.