\section{Background}
\label{sec:background}

\subsection{Learning to Rank}
\label{sec:learningtorank}

\textit{Learning to rank} is the application of machine learning (ML) to the creation of models that can rank. Ranking, in its most general form, is the ordering of a collection of documents (also referred to as samples). The book \textit{Learning to Rank for Information Retrieval} by \citeauthor{liu2009learningtorank} \cite{liu2009learningtorank} enumerates three distinguishable approaches to phrasing and solving ranking problems. They are referred to as pointwise, pairwise, and listwise, and shall be explained briefly in the following. The second part of this section deals with ranking metrics which are used to evaluate the performance of ranking model, i.e. the quality of its ranking outputs.

The \textbf{pointwise approach} deals with models that determine the rank of a single sample. It can be subdivided into the three subcategories (1) regression based, (2) classification based, and (3) ordinal regression based. For (1) the problem is considered a regression problem, where the rank of a sample is interpreted as a point on the real number line which the model seeks to predict correctly. With (2) the ranks are discrete and considered classes, for a given input, the model predicts a class which can be mapped to a rank. Ranks can also be grouped into bins, such that an entire range of adjacent ranks corresponds to a single class. While the classes in (2) do not have an ordering that is necessarily apparent to the model, the third subcategory (3) takes the ordinal relationship of ranks into account. The goal is to find a scoring functions such that thresholds can be found to distinguish the outputs into the different, ordered categories.

A possible loss function for pointwise training is to interpret the scaled and rounded model output $y\in\mathbb{R}$ as the predicted rank of an input. A loss function could simply penalize divergence from the target output $\hat{y}\in\mathbb{R}$, e.g. in a quadratic fashion, $(y-\hat{y})^2$.

Opposed to the pointwise approach, the \textbf{pairwise approach} does not seek to predict the rank of a single sample. Instead the focus is on predicting the relative order between two given samples, i.e. whether the first sample $\bm{x}^{(i)}$ has a greater or a lower rank than the second one $\bm{x}^{(j)}$. This problem can be considered a binary classification task on a tuple input $(\bm{x}^{(i)},\bm{x}^{(j)})$, where the first class means $\bm{x}^{(i)}\triangleright \bm{x}^{(j)}$ ($\bm{x}^{(i)}$ has a higher rank than $\bm{x}^{(j)}$) and the second class means $\bm{x}^{(i)}\triangleleft \bm{x}^{(j)}$.

\citeauthor{Burges:learningtorankwithsgd} propose a probabilistic cost function for pairwise training \cite{Burges:learningtorankwithsgd}.
They represent the ground truth as a matrix $\bm{\overline{P}}\in[0,1]^{m\times m}$, where $\overline{P}_{i,j}$ is the probability of sample $\bm{x}^{(i)}$ to have a higher rank than $\bm{x}^{(j)}$, i.e. $P(\bm{x}^{(i)}\triangleright \bm{x}^{(j)})$.
The model itself is a function $f:\mathbb{R}^d\rightarrow\mathbb{R}$ mapping a sample $\bm{x}^{(i)}$ to a scalar. The loss for two given samples $\bm{x}^{(i)}$ and $\bm{x}^{(j)}$ is defined as
\begin{align}
-\overline{P}_{i,j}\times\left(f\left(\bm{x}^{(i)}\right)-f\left(\bm{x}^{(j)}\right)\right)+\log\left(1+e^{f\left(\bm{x}^{(i)}\right)-f\left(\bm{x}^{(j)}\right)}\right)
\end{align}
This formulation has some favorable properties: It allows for uncertainty in the relative ordering of items in that the $\bm{\overline{P}}$ is a probability and could be e.g. $\overline{P}_{i,j}=\frac{1}{2}$, if the relative ranking of two items was unknown. Furthermore, the loss asymptotes to a linear function. According to the authors, this is likely to be more robust with noisy labels than a quadratic cost.

The \textbf{listwise approach} deals with models taking an entire list of samples as their input, with the goal of ordering them. This can be either done by mapping them to real values and penalizing wrong ordering, or mapping the samples discretely to a permutation.

Once a model has produced a ranking, it must be evaluated. Since the magnitude of a loss function is often rather less meaningful, other measures, so called \textbf{ranking metrics}, have been defined. \citeauthor{tfranking} name four, namely \textit{Mean Reciprocal Rank (MRR)}, average of positions of examples weighted by their relevance values (\textit{ARP}), \textit{Discounted Cumulative Gain (DCG)}, and \textit{Normalized DCG (NDCG)} in \cite{tfranking}. They also make the important point that ``in ranking, it is preferable to have fewer errors at higher ranked positions than at the lower ranked positions, which is reflected in many metrics''.

In real world applications, the training of ranking models can be much more complicated. For example, updating a ranking model based on what users click leads to wrong updates because the feedback from users is biased. \citeauthor{unbiasedlearningtorank} deal with that problem and propose \textit{unbiased learning to rank} in \cite{unbiasedlearningtorank}. This is, however, outside the scope of this work, because we do not deal with direct user feedback.

\subsection{Graph Networks}
\label{sec:graphnetworks}

The most basic type of neural network (NN), consisting solely of fully connected layers, converts a vector of fixed size into another vector of fixed size.
In contrast, CNNs can convert variably-sized inputs into variably-sized outputs. In image classification or segmentation tasks, the inputs are often scaled to a fixed size and the convolutional layers serve as feature extractors for a fully-connected classification head.
Recurrent neural networks can convert a sequence of vectors into another sequence.

While some of those NN types \textit{can} handle variably-sized inputs, they do not deal with sets very well: Consider for instance an (unordered) set of images and the task to assign a single label to the entire set. All architectures from above could be applied to that task by simply concatenating all the images into one large vector. However, they would implicitly try to assign a semantic meaning to the ordering in which the samples are being presented to them, so called relational inductive bias. One could augment the dataset by shuffling the samples of a set before concatenating them but that does not solve the underlying problem of the architecture. Alternatively, one might process the images in the set individually and combine the outputs. This setup has the drawback that information exchange between internal, intermediate representations of the model cannot take place. The feature representation of one image cannot influence the processing of other images, which could, however, be a requirement or at least helpful when dealing with a set of images.

Graph networks (GNs) are designed to handle graphs and sets naturally. They can deal with any size of graph and do not regard the ordering of nodes. While processing the nodes, information can freely flow between them. \citeauthor{deepmind:graphnets} have brought several different types of graph networks under a common hood \cite{deepmind:graphnets}. We follow their notation and will recapitulate the relevant parts of the paper in the following paragraphs. That is the formal definition of a directed, attributed multi-graph with a global attribute, as well as the definition of GNs.

Let a graph $G$ be a 3-tuple $G=\left(\bm{u},\mathbb{V},\mathbb{E}\right)$, where $\bm{u}$ is a vector of global attributes.
$\mathbb{V}=\left\{\bm{v}_i\right\}_{i=1}^{N^{(v)}}$ is a set of $N^{(v)}$ nodes (also referred to as vertices), each consisting solely of an attribute vector.
$\mathbb{E}=\left\{\left(\bm{e}_k,r_k,s_k\right)\right\}_{k=1}^{N^{(e)}}$ is a set of $N^{(e)}$ edges, each of which connects a sender node $\bm{v}_{s_k}$ to a receiver node $\bm{v}_{r_k}$, and contains an attribute vector $\bm{e}_k$.

The attribute vectors $\bm{u}$, $\bm{v}_i$, and $\bm{e}_k$ can contain any kind of information. In fact they could even be graphs themselves.

GNs are NNs containing at least one GN block. \citeauthor{deepmind:graphnets} describe it as follows: A GN block contains three update functions, $\phi$, and three aggregation functions, $\rho$,
\begin{align}
    \bm{e}'_k=&\phi^e\left(\bm{e}_k,\bm{v}_{r_k},\bm{v}_{s_k},\bm{u}\right)\\
    \bm{\overline{e}}'_i=&\rho^{e\rightarrow v}\left(\mathbb{E}'_i\right)\\
    \bm{v}'_i=&\phi^v\left(\bm{\overline{e}}'_i,\bm{v}_i,\bm{u}\right)\\
    \bm{\overline{e}}'=&\rho^{e\rightarrow u}\left(\mathbb{E}'\right)\\
    \bm{\overline{v}}'=&\rho^{v\rightarrow u}\left(\mathbb{V}'\right)\\
    \bm{u}'=&\phi^u\left(\bm{\overline{e}}',\bm{\overline{v}},\bm{u}\right)\,,
\end{align}
where $\mathbb{E}'_i=\left\{\left(\bm{e}'_k,r_k,s_k\right):r_k=i\right\}_{k=1}^{N^{(e)}}$ is the set of all edges pointing to the $i$th node; $\mathbb{V}'=\left\{\bm{v}'_i\right\}_{i=1}^{N^{(v)}}$; $\mathbb{E}'=\bigcup_i\mathbb{E}'_i$.

\begin{enumerate}
    \item $\phi^e$ is the \textbf{edge update function} and applied to every edge in the graph. For the $k$th edge it computes a new attribute vector $\bm{e}'_k$ based on the previous state $\bm{e}_k$, the attributes of the two nodes that this edge connects, namely $\bm{v}_{r_k}$ and $\bm{v}_{s_k}$, and the global state $\bm{u}$.
    \item $\rho^{e\rightarrow v}$ is the \textbf{edge aggregation function} and applied once per node in the graph. It computes an aggregation of all edges pointing to the $i$th node.
    \item $\phi^v$ is the \textbf{node update function}. It is applied to every node in the graph and computes the new attribute vector $\bm{v}'_i$ based on the aggregation of all edges pointing to the node, the previous state of the node, and the global state.
    \item $\rho^{e\rightarrow u}$ is the \textbf{global edge aggregation function} which computes an aggregation $\bm{\overline{e}}'$ of all edges in the graph.
    \item $\rho^{v\rightarrow u}$ is the \textbf{node aggregation function} which compuates an aggregation $\bm{\overline{v}}'$ of all nodes in the graph.
    \item $\phi^u$ is the \textbf{global attribute update function}. It takes both node and edge aggregation, as well as the previous global attribute $\bm{u}$, and computes a new global attribute $\bm{u}'$.
\end{enumerate}

The functions are being evaluated in a specific sequence, visualized in Figure~\ref{fig:fullgraphblock}. GN blocks do not alter the structure of the graph, i.e. they convert a graph into another graph isomorphic to the input. In order to extract a fixed size representation from a GN block, the global attribute can be used.

\begin{figure}
    \centering

    \begin{tikzpicture}[]

        \draw[dashed] (0,-5) rectangle (6,1);

        % inputs and outputs
        \node (in_u) at (-1,0) [draw=none,fill=none]{$\bm{u}$};
        \node (out_u) at (7,0) [draw=none,fill=none] {$\bm{u}'$};
        \node (in_V) at (-1,-2) [draw=none,fill=none]{$\mathbb{V}$};
        \node (out_V) at (7,-2) [draw=none,fill=none]{$\mathbb{V}'$};
        \node (in_E) at (-1,-4) [draw=none,fill=none]{$\mathbb{E}$};
        \node (out_E) at (7,-4) [draw=none,fill=none]{$\mathbb{E}'$};

        % processing
        \node (phi_u) at (5,0) [draw, circle]{$\phi^u$};
        \node (phi_v) at (3,-2) [draw, circle]{$\phi^v$};
        \node (phi_e) at (1,-4) [draw, circle]{$\phi^e$};

        \node (rho_ev) at (2,-3) [draw, process]{$\rho^{e\rightarrow v}$};
        \node (rho_eu) at (4,-3) [draw, process]{$\rho^{e\rightarrow u}$};
        \node (rho_vu) at (4,-1) [draw, process]{$\rho^{v\rightarrow u}$};

        % edges
        \draw[->] (in_u) -- (phi_u);
        \draw[->] (in_u) -- (phi_v);
        \draw[->] (in_u) -- (phi_e);
        \draw[->] (phi_u) -- (out_u);
        \draw[->] (in_V) -- (phi_v);
        \draw[->] (in_V) -- (phi_e);
        \draw[->] (phi_v) -- (out_V);
        \draw[->] (in_E) -- (phi_e);
        \draw[->] (phi_e) -- (out_E);
        \draw[->] (phi_e) -- (rho_ev);
        \draw[->] (rho_ev) -- (phi_v);
        \draw[->] (phi_e) -- (rho_eu);
        \draw[->] (phi_v) -- (rho_vu);
        \draw[->] (rho_vu) -- (phi_u);
        \draw[->] (rho_eu) -- (phi_u) ;

    \end{tikzpicture}
    \caption[Full GN block]{Visualization of the evaluation order in a \textbf{full GN block}. Arrows indicate inputs as well as temporal dependency. The three inputs to a GN block are on the left, the three outputs (transformations of the inputs) are on the right-hand side. Several variation of the depicted version exist. The full GN block is the most generic GN block. The variations independent recurrent block, message-passing neural network, non-local neural network, relation network, and deep set exist \cite{deepmind:graphnets}.}
    \label{fig:fullgraphblock}
\end{figure}

GNs in related forms have been applied to a variety of different problems from different domains:
\begin{itemize}
    \item \textbf{Social sciences}: \cite{graphnetsreddit} apply GNs to post data from the social network Reddit. \cite{graphnetscitationgraph} use convolutional GNs. They run experiments on citation networks and on a knowledge graph dataset.
    \item \textbf{Physics}: \cite{graphnetsphysicsengine} and \cite{graphnetsphysics2} train GNs that focus on the modelling of physical relationships between objects.
    \item \textbf{Medicine}: \cite{graphnetsproteininterface} consider the prediction of interfaces between proteins, a challenging problem with important applications in drug discovery and design.
    \item \textbf{Algorithms}: See e.g. \cite{selsam:satsolver} and \cite{dai:graphnetscombinatorialalgo}.
\end{itemize}

GNs can be trained with stochastic gradient descent (SGD), since they are fully differentiable, if aggregation and attribute update functions are differentiable.

\cite{selsam:satsolver} train a GN on a boolean satisfiability task (SAT-solver) with a single bit of supervision, namely whether or not the statement is satisfiable. The authors thereby show that GNs can be successfully trained on NP-complete tasks with very little supervision, namely just a single bit. The method presented in the paper extracts a fixed-size information (true or false) from the GN by computing the mean of all nodes. This fits into \cite{deepmind:graphnets} Deep Mind's GN framework, since the output bit can be interpreted as a global state. Depending on the application, a model's output might be a graph itself, in such cases, the last layer can be a GN block.

\subsection{Screenshot Processing}
\label{sec:screenshotprocessing}

CNNs were originally inspired by the visual cortex of humans, see \cite{lecun:lenet}, and had their most notable breakthrough with the success of AlexNet \cite{krizhevsky:imagenet} in the domain of image classification. The convolutional layers are sliding multiple, learnable kernels over two dimensions of the input while looking at all channels simultaneously. This concept allows for two-dimensional translation invariant feature extraction and has proven to be both effective and efficient.

Most commonly, CNNs are applied to natural images, for instance pictures of animals taken in nature or car traffic scenes from a driver's point of view.
Screenshots, i.e. images showing the content of a monitor, differ from natural images: Often they do not contain smooth transitions, do not feature a large color variety, and contain many equally colored areas such as white background.
CNNs have been shown to have a relational inductive bias towards natural images, which is being utilized in denoising, super-resolution, and inpainting problems \cite{deepimageprior}.
The question arises whether CNNs are also applicable to screenshots, which have different characteristics. There is little work that uses CNNs for screenshot processing, which can either be attributed to them handling screenshots just as fine as natural images or few applications.

The closest to us is the work of \citeauthor{beltramelli:pix2code}, who feeds screenshots into a CNN in order to extract structual information from them \cite{beltramelli:pix2code}. He tries to convert screenshots of a user interface into code that describes that exact layout. In his setup, a CNN is combined with a long short-term memory (LSTM) \cite{hochreiter1997lstm}: The former is responsible for the feature extraction, whereas the latter converts the extracted features into a sequence of layout descriptions.

More specifically, \citeauthor{beltramelli:pix2code} does not perform any pre-processing and feeds images of size $256\times 256$ into the model. The model consists of three convolutional blocks, each of which contains two convolutional layers with kernels of size $3\times 3$ and stride 1, followed by $2\times 2$ max-pooling and dropout \cite{srivastava2014:dropout} with a probability of $p=0.25$. The third convolutional block is followed by two fully connected layers with $1024$ units each. Both regularized with $p=0.3$ dropout. The filter count for the convolutions is $32$, $64$, and $128$ for the three blocks, respectively.

\subsection{Open PageRank}
\label{OpenPageRank}
\textit{Open PageRank} \cite{OpenPageRank} is an initiative, which has the goal to enable webmasters to determine the page rank for their domains and easily compare with other domains. The initiative was constituted after Google had shut down the \textit{PageRank}-toolbar, leaving a void in the industry. The Open PageRank initiative provides freely available data that was built on top of \textit{Common Crawl} \cite{CommonCrawl}, which provides high quality crawl data of webp ages since 2013.

Open PageRank uses the number of backlinks of a domain found in \textit{Common Crawl} to calculate the page rank. The greater the number of links, different domains use to refer to the given domain, the better the page rank of the given domain according to Open PageRank.

The list offered by Open PageRank contains about 10 million domains ordered by their page rank. Each entry in the list consists of the following comma-separated attributes: rank of the domain, domain name, and calculated score by Open PageRank.

The list is avilable at \url{https://www.domcop.com/openpagerank/}.

\begin{table}
	\center
	\begin{tabular}{r l}
		\textbf{Rank} & \textbf{Website}  \\ \hline
		\#1 & \url{fonts.googleapis.com} \\
		\#2 & \url{facebook.com} \\
		\#3 & \url{youtube.com} \\
		\#4 & \url{twitter.com} \\
		\#5 & \url{google.com} \\
		\#6 & \url{plus.google.com} \\
		\#7 & \url{instagram.com} \\
		\#8 & \url{linkedin.com}\\
		\#9 & \url{en.wikipedia.org}\\
		\#10 & \url{maxcdn.bootstrapcdn.com}\\
	\end{tabular}
	\caption[Top ten websites from \textit{Open PageRank}]{The top ten websites in the world according to \textit{Open PageRank}.}
	\label{tab:toptensites}
\end{table}

\subsection{Puppeeter}
\label{puppeeter}
\texttt{Puppeteer}\footnote{Puppeeter is available at \url{https://github.com/GoogleChrome/puppeteer}.} is a open-source library for the JavaScript runtime \texttt{Node.js}, which provides a high-level API to control instances of the browsers \texttt{Chrome} or \texttt{Chromium}. One of the main goals of \texttt{Puppeteer} is to grow adoption in automated browser testing \cite{PuppeteerFAQ}.  For this purpose, it wraps the \texttt{Chrome DevTools Protocol} in JavaScript, which allows web developers to instrument, inspect and debug instances of \texttt{Chrome}. Technically, the protocol is HTTP-based and exposed as a RESTful API at the port for debugging by \texttt{Chrome}.

The protocol offers various functionalities, which are grouped into domains \cite{DevToolsProtocol}. Some of the interesting domains for our use-case are:
\begin{description}
	\item[Page]: API to load and take a screenshot of the given website.
	\item[DOM]: API to read and manipulate the DOM of the given website.
	\item[Network]: API to intercept network requests, track downloaded data and network issues.
	\item[Emulation]: API to emulate different geolocation, network bandwith and mobile devices.
\end{description}

During start up \texttt{Puppeteer} starts an instance of \texttt{Chrome}, attaches to the instances using the port for debugging and afterwards custom code of the user will be executed against the instance. The instance will be started in \texttt{headless}-mode meaning that no UI will be visible.

\subsection{Chromium Embedded Framework}
\label{cef}
\texttt{Chromium Embedded Framework}\footnote{\texttt{CEF} is available at \url{https://bitbucket.org/chromiumembedded}.} (short: \texttt{CEF}) was designed ground-up with performance and ease of use in mind. The framework exposes \texttt{C++} interfaces to control instances of the browsers \texttt{Chrome} or \texttt{Chromium} with default implementation for all features requiring little or no integration work. Furthermore, the community added wrappers for the base implementation to support a wide-range of operating systems and programming languages.
\texttt{CEF} has a total of three versions, whereas \texttt{CEF 2} was abandoned due to the appearance of the \texttt{Chromium Content API} which will be discussed later. \texttt{CEF 1} is a single process implementation and based on the old \texttt{Chromium WebKit API}, due to the deprecation of the \texttt{Chromium WebKit API} it is no longer supported and developed. \texttt{CEF 3} is a multi process implementation and based on the current \texttt{Chromium Content API}.

\subsection{Kubernetes}
\label{k8s}
Kubernetes\footnote{K8s is available at \url{https://kubernetes.io}.} (K8s) is an open source container orchestration solution that was originally designed by \textit{Google} in 2014. It aims to provide a platform where containers and containerized applications can be deployed, scaled, and managed \cite{Wikik8s}. K8s orchestrates computing, networking, and storage infrastructure on behalf of user workloads. This means applications are automatically scaled depending on incoming workload.

K8s is much of the simplicity of Platform as a Service with the flexibility of Infrastructure as a Service \cite{k8s}. K8s offers platform abstraction at container level. Therefore, developers are responsible for building their own application containers rather than only the application itself. This is rewarded with more freedom in application development, but comes at the cost of lower productivity.

\subsubsection*{Kubernetes Components and Concepts}
The most popular container runtime in the industry is \textit{Docker}. Container runtimes are responsible for serving the communication to the operating system (OS) and the container environment, where applications can be configured and ran.
\begin{figure}
	\centering
	\includegraphics[width=0.7\textwidth]{resources/k8s_architecture}
	\caption[Architecture diagram of K8s]{This diagram illustrates the main parts: The \textit{master node} representing the central unit for the orchestration of the Kubernetes cluster. The \textit{worker nodes} running the user-specified workloads represented as \textit{pods} on the Kubernetes cluster. Arrows indicate control flow. The cluster can be controller and orchestrated by the user using the \textit{kubectl}-CLI. \cite{K8sArch}}
	\label{k8s_architecture}
\end{figure}
K8s consist of three main components \textit{kubectl}, \textit{master node}, and the \textit{worker nodes}. Kubectl represents the client interface to communicate with the master node, by sending requests against the API server. The master node represents the administrative entrypoint and is responsible for managing the whole K8s cluster \cite{K8sArch}. It consists of the \textit{API server}, \textit{etcd storage}, \textit{scheduler}, and \textit{controller manager}:

\begin{itemize}
	\item \textit{API server} represents the entry point for all incoming REST commands. It processes and executes them according to their business logic.
	\item \textit{etcd storage} is a simple key-value store, where information about the \textit{pods} and the cluster is saved.
	\item \textit{scheduler} is responsible for container distribution among the worker nodes by using information about the worker nodes.
	\item \textit{controller-manager} consists of several controllers such as the replication controller, which is responsible for monitoring pods and recreating crashed pods.
\end{itemize}
Beside the master node, a K8s cluster consists of one or more worker nodes, which are controlled by the controllers of the master node. Each worker node consists of \textit{kubelet}, \textit{kubeproxy}, and \textit{pods}:

\begin{itemize}
	\item \textit{kubelet} communicates with the API server, ensuring the requested containers are up and running.
	\item \textit{kube-proxy} acts as a proxy and load balancer on the worker node. It is responsible for the communication with the outside world.
	\item \textit{pods} are the smallest unit in K8s. They encapsulate one or several containers in the same shared context, which means that containers within the same pods share the same IP, storage, and memory.
\end{itemize}


\chapter{PCPs \& Hardness of Approximation}

In this chapter, we will review some of the fundamental results in hardness of
approximation. One of the main results is an alternate characterization for \NP.
Recall the definition of \NP\ (\lref[Definition]{def:np}).
For the verifier of an \NP\ problem, the proof length is at most a polynomial in
$n$. The alternate characterization deals with probabilistic verifiers, that
query only a few bits in the proof.

\begin{definition}[{$\PCP_{c,s}[t,q,R]$}]
A $\PCP_{c,s}[t(n),q(n), R]$-Verifier (\PCP\ stands for Probabilistically
Checkable Proofs)
for a decision problem $L$, is an algorithm $V$
which takes an input $x$, a random string
$y \in \{0,1\}^{t(n)}$, has oracle access to a proof $\pi$ over alphabet $[R]$ of length $2^{t(n)}$
 and satisfies the following properties:
 \begin{itemize}
 \item $V$ runs in polynomial time in the length of $x$ for all $y,\pi$.
\item $V$ queries $\pi$ in at most $q(n)$ bits on all inputs.
\item Completeness: If $x \in L$ then there exists $\pi$ such that $\Pr_y[V(x,\pi)=1]\geq c$.
\item Soundness : If $x \notin L$ then for any $\pi$, $\Pr_y[V(x, \pi)=1] \leq
s$.
 \end{itemize}
 $\PCP_{c,s}[t,q,R]$ is the class of decision problems that have
a $ \PCP_{c,s}[t,q,R]$-Verifier.
 \end{definition}
 It is not difficult to see that,
for any $q$, constant $R$ and $s<c$, $$\PCP_{c,s}[O(\log n),q,R] \subseteq
\NP.$$ A major breakthrough in PCP characterizations, which lead to many
hardness of approximation results was the following theorem due to Arora \&
Safra~\cite{AroraS1998} and Arora \etal~\cite{AroraLMSS1998}.
\begin{theorem}[PCP Theorem]
There exists constant $s <1, q,R$ such that $$\NP \subseteq
\PCP_{1,s}[O(\log n),q,R].$$
\end{theorem}
In the above form, it is not clear
how the theorem might be useful in hardness of approximation results. The
theorem can be stated equivalently as a hardness of approximation result.

\begin{theorem}[Hardness of Approximating \MAXTSAT] There exists a constant $s
<1$, such that it is \NPHard\ to distinguish satisfiable \MAXTSAT\ instances,
from ones for which any assignment can satisfy at most $s$ fraction of the
clauses. \end{theorem}

It is easy to see that the above theorem is equivalent to having a
$\PCP_{1,s}[O(\log n),3,2]$-Verifier for \MAXTSAT, whose checks are \MAXTSAT\
clauses. The equivalence follows from viewing such verifiers as
\MAXTSAT\ instances and vice versa.
 Any such verifier could be converted to a \MAXTSAT\ instance.
The instance is obtained by adding a variable for every bit of the proof,
and adding a \MAXTSAT\ clause for every check the verifier makes.
Any \MAXTSAT\ instance, has a trivial \PCP\ verification procedure,
whose proof is the assignment and the verifier chooses a random clause,
and checks if it is satisfied by the assignment.

\section{Label Cover}

The PCP theorem stated in terms hardness of approximation of \MAXTSAT, does not
give optimal inapproximability results (see H\aa stad \cite{Hastad2001}).
To get strong results, it is used in
conjunction with the parallel repetition theorem of Raz~\cite{Raz1998}.
This strong version of PCP
theorem is usually stated in terms of the \LabelCover\ problem.

\begin{definition}[\LabelCover]

	\label{def:label-cover} An instance $G=(U,V,E,L,
	R,\{\pi_e\}_{e\in E})$ of the {\LC} constraint satisfaction
	problem consists of a bi-regular bipartite graph $(U,V,E)$,
        two sets of alphabets $R$ and $L$ and a projection map $\pi_e : R \rightarrow
        L$ for every edge $e\in E$.
        Given a labeling $\ell : U \rightarrow R, \ell:V \rightarrow
        L$, an edge $e = (u,v)$ is said to be satisfied by $\ell$ if
        $\pi_e(\ell(v)) = \ell(u)$.

        $G$ is said to be \emph{at most $\delta$-satisfiable} if every
        labeling satisfies at most a $\delta$ fraction of the
        edges.

	An instance of \UG\ is a label cover instance where $L=R$ and the constraints
	$\pi$ are permutations.
\end{definition}

We consider label cover instances obtained from $\TSAT$ instances in the
following natural manner.
\begin{definition}[$r$-repeated label cover]
\label{def:label-cover}
Let $\phi$ be a $\TSAT$ instance with $X$ as the set of
variables and $C$ the set of clauses. The $r$-repeated bipartite label cover
instance $I(\phi)$ is specified by:
\begin{itemize}
\item A graph $G:=(U,V,E)$,
where $U:=C^r, V:=X^r$. \item $\Sigma_U := \{0,1\}^{3r},\Sigma_V := \{0,1\}^r$.
\item There is an edge $(u,v) \in E$ if the tuple of variables $v$ can be
obtained from the tuple of clauses $u$ by replacing each clause by a variable in
it.
\item The constraint $\pi_{uv}:\{0,1\}^{3r}\rightarrow \{0,1\}^{r}$ is
simply the projection of the assignments on $3r$ variables in all the clauses in
$u$ to the assignments on the $r$ variables in $v$. \item For each $u$ there is
a set of $r$ functions $\{f^u_i:\{0,1\}^{3r} \rightarrow \{0,1\} \}_{i=1}^r$
such that $f^u_i(a)=0$ iff the assignment $a$ satisfies the $i$th clause in $u$.
Note that $f^u_i$ depends only on the $3$ variables in the $i$th clause.
\end{itemize}
A labeling $L_U:U\rightarrow \Sigma_U,L_V:V\rightarrow \Sigma_V$
satisfies an edge $(u,v)$ iff $\pi_{uv}(L_U(u))=L_V(v)$ and $L_U(u)$ satisfies
all the clauses in $u$. Let $\OPT(I(\phi))$ be the maximal fraction of
constraints that can be satisfied by any labeling.
\end{definition}
The
following theorem is obtained by applying  parallel repetition
theorem of Raz~\cite{Raz1998} with $r$ repetitions on hard instances of
\MAXTSAT\ where each variable occurs the same number of
times (see Feige's result~\cite{Feige1998}) and a structural property proved by
H\aa stad \cite[Lemma 6.9]{Hastad2001}.

\begin{theorem} \label{thm:label-cover}
There is an
algorithm which on input a \TSAT\ instance $\phi$ and $r\in \N$ outputs an
$r$-repeated label cover instance $I(\phi)$ in time $n^{O(r)}$ with the
following properties.
\begin{itemize}
\item Completeness: If $\phi \in \TSAT$, then $\OPT(I(\phi))=1$.
\item Soundnes: If $\phi \notin \TSAT$, then $\OPT(I(\phi)) \leq
2^{-\epsilon_0 r}$ for some universal constant $\epsilon_0\in (0,1)$.
\item Smooth Projections:
$$\forall v \in V, \alpha \subset R, \qquad Pr_u \left[ |\pi_{uv}(\alpha)| <|\alpha|^{c_0}\right] \leq \frac{1}{|\alpha|^{c_0}}.$$
\end{itemize}
Moreover, the underlying graph $G$ is both left and right regular.
\end{theorem}

 For our hardness results for
$3$-uniform $3$-colorable hypergraphs, we need a multipartite version of label
cover, satisfying a smoothness condition.
\begin{definition}[\cite{Khot2002b}]
Let $I$ be a bipartite label cover instance specified by
$\left((U,V,E),\Sigma_U,\Sigma_V,\Pi\right)$. Then $I$ is $\eta$-\emph{smooth}
iff for every $u \in U$ and two distinct labels $a,b \in \Sigma_U$
$$\Pr_v[\pi_{uv}(a) = \pi_{uv}(b) ] \leq \eta,$$
where $v$ is a random neighbour of $u$.
\end{definition}
\begin{definition}[$r$-repeated $\ell$-layered $\eta$-smooth label cover]
\label{def:multilayer}
Let $T:=\lceil\ell/\eta\rceil$ and $\phi$ be
a \TSAT\ instance with $X$ as the set of variables and $C$ the set of clauses.
The $r$-repeated $\ell$-layered $\eta$-smooth label cover instance $I(\phi)$ is
specified by:
\begin{itemize}
\item An $\ell$-partite graph with vertex sets
$V_0, \cdots V_{\ell-1}$. Elements of $V_i$ are tuples of the form $(C',X')$
where $C'$ is a set of $(T+\ell - i)r$ clauses and $X'$ is a a set of $ir$
variables.
\item $\Sigma_{V_i} := \{0,1\}^{m_i}$ where $m_i:={3(T+\ell - i)r
+ir}$ which corresponds to all Boolean assignments to the clauses and variables
corresponding to a vertex in layer $V_i$.
\item For $0 \leq i < j < \ell$,
$E_{ij} \subseteq V_i \times V_j$ denotes the set of edges between layers $V_i$
and $V_j$. For $v_i \in V_i, v_j \in V_j$, there is an edge $(v_i,v_j) \in
E_{ij}$ iff $v_j$ can be obtained from $v_i$ by replacing some $(j-i)r$ clauses
in $v_i$ with variables occurring in the clauses respectively.
\item The
constraint $\pi_{v_i v_j}$ is the projection of assignments for clauses and
variables in $v_i$ to that of $v_j$.
\item For each $i <\ell$, $v_i \in V_i$,
there are $(T+\ell - i)r$ functions $f_j^{v_i}:\{0,1\}^{3(T+\ell - i)r
+ir}\rightarrow \{0,1\}$, one for each clause $j$ in $v_i$ such that
$f_j^{v_i}(a)=0$ iff $a$ satisfies the clause $j$. This function only depends on
the $3$ coordinates in $j$. \end{itemize} Given a labeling $L_i:V_i\rightarrow
\Sigma_{V_i}$ for all the vertices, an edge $(v_i,v_j) \in E_{ij}$ is satisfied
iff $L_i(v_i)$ satisfies all the clauses in $v_i$, $L_j(v_j)$ satisfies all the
clauses in $v_j$ and $\pi_{v_i v_j}(L_i(v_i)) = L_j(v_j)$. Let
$\OPT_{ij}(I(\phi))$ be the maximum fraction of edges in $E_{ij}$ that can be
satisfied by any labeling.
\end{definition}
The following theorem was proved by
Dinur~\etal~\cite{DinurGKR2005} in the context of hypergraph vertex cover
inapproximability (also see results of Dinur, Regev \& Smyth~\cite{DinurRS2005}).
\begin{theorem}
\label{thm:layered-label-cover}
There is an algorithm which on input a \TSAT\
instance $\phi$ and $\ell,r\in \N, \eta \in [0,1)$ outputs a $r$-repeated
$\ell$-layered $\eta$-smooth label cover instance $I(\phi)$ in time
$n^{O((1+1/\eta)\ell r)}$ with the following properties.
\begin{enumerate}
\item $\forall~ 0\leq i < j < \ell$, the bipartite label cover instance on
$I_{ij}=\left((V_i,V_j,E_{ij}),\Sigma_{V_i},\Sigma_{V_j},\Pi_{ij}\right)$ is
$\eta$-smooth.
\item For $1<m<\ell$, any $m$ layers $0\leq i_1< \cdots <i_m\leq
\ell-1$, any $S_{i_j} \subseteq V_{i_j}$ such that $|S_{i_j}| \geq
\frac{2}{m}|V_{i_j}|$, there exists distinct ${i_j}$ and ${i_{j'}}$ such that
the fraction of edges between $S_{i_j}$ and $S_{i_{j'}}$ relative to
$E_{i_ji_{j'}}$ is at least $1/m^2$.
\item If $\phi \in \TSAT$, then there is a
labeling for $I(\phi)$ that satisfies all the constraints.
\item If $\phi \notin \TSAT$, then
$$\OPT_{i,j}(I(\phi)) \leq 2^{-\Omega(r)}, \quad \forall 0\leq i <j \leq \ell.$$
\end{enumerate}
\end{theorem}

\section{Unique Games Conjecture}

Khot observed~\cite{Khot2002} that if the label sets in the \LabelCover\
instance are the same and the projections are permutations, then
the hardness reductions could be simplified.
He made the
conjecture that \LabelCover\ is hard to approximate to any constant factor,
restricted to such kind of instances.
\begin{definition}[Unique Games Conjecture]
For every $\delta$ there exists a large enough $R$ such that it
is hard to distinguish between \UniqueGame\ instances $G$ have label
size  $R$ from the following cases.
\begin{itemize}
\item \YES\ case : $\OPT(G)\geq 1-\delta$
\item \NO\ case :  $\OPT \leq \delta$
\end{itemize}
\end{definition}

Starting with the work of Khot ~\cite{Khot2002}, it was shown that
UGC, explains the lack of efficient approximation algorithms for a
variety of problems (eg. Vertex Cover, MAX-CUT).


\section{The Dot Product}\label{sec:3Ddotproduct}

The goal of this section is to answer the following question. Given two
vectors, what is the angle between them?

Since vectors have no position, we are free to place vectors wherever we
like. If the two vectors are placed tail-to-tail, there is now a
reasonable interpretation of the question: we seek the measure of the
smallest angle between the two vectors, in the plane in which they lie.
Figure~\ref{fig:angle between vectors} illustrates the situation.\index{vector!angle between}

\begin{figure}[H]
\centerline{
\vbox{\beginpicture
\normalgraphs
\setcoordinatesystem units <6truemm,6truemm>
\setplotarea x from 0 to 7, y from 0 to 4
\arrow <4pt> [0.35, 1] from 0 0 to 3 4
\arrow <4pt> [0.35, 1] from 0 0 to 7 3
\setdashes
\plot 3 4 7 3 /
\put {$\vect{v}$} [b] <0pt,3pt> at 3 4
\put {$\vect{w}$} [l] <3pt,0pt> at 7 3
\put {$\theta$} [bl] <11pt,8pt> at 0 0
\endpicture}}
\caption{The angle between vectors $\vect{v}$ and $\vect{w}$. \label{fig:angle between vectors}}
\end{figure}

Since the angle $\theta$ lies in a triangle, we can compute it using a
bit of trigonometry, namely, the law of cosines. Remember that the law of cosines states $c^2 = a^2 + b^2 - 2ab \cos C$.

The lengths of the sides of the
triangle in Figure~\ref{fig:angle between vectors} are $|\vect{v}|$,
$|\vect{w}|$, and $|\vect{v}-\vect{w}|$. Let $\vect{v}=\langle v_1,v_2,v_3\rangle$ and $\vect{w}=\langle w_1,w_2,w_3\rangle$; then
\begin{align*}
  |\vect{v}-\vect{w}|^2&=|\vect{v}|^2+|\vect{w}|^2-2|\vect{v}||\vect{w}|\cos\theta	\\
  2|\vect{v}||\vect{w}|\cos\theta&=|\vect{v}|^2+|\vect{w}|^2-|\vect{v}-\vect{w}|^2	\\
  &=v_1^2+v_2^2+v_3^2+w_1^2+w_2^2+w_3^2-(v_1-w_1)^2-(v_2-w_2)^2-(v_3-w_3)^2	\\
  &=v_1^2+v_2^2+v_3^2+w_1^2+w_2^2+w_3^2	\\
  &\qquad-(v_1^2-2v_1w_1+w_1^2)
  -(v_2^2-2v_2w_2+w_2^2)-(v_3^2-2v_3w_3+w_3^2)	\\
  &=2v_1w_1+2v_2w_2+2v_3w_3	\\
  |\vect{v}||\vect{w}|\cos\theta&=v_1w_1+v_2w_2+v_3w_3	\\
  \cos\theta&=(v_1w_1+v_2w_2+v_3w_3)/(|\vect{v}||\vect{w}|)
\end{align*}
A bit of simple arithmetic with the coordinates of $\vect{v}$ and $\vect{w}$ allows us to compute the cosine of the angle between them. If
necessary we can use the arccosine to get $\theta$, but in many
problems $\cos\theta$ turns out to be all we really need.

The numerator of the fraction that gives us $\cos\theta$ turns up a
lot, so we give it a name and more compact notation: we call it
the \dfont{dot  product}\index{dot product}, and write it as
\[
\vect{v}\cdot\vect{w} = v_1w_1+v_2w_2+v_3w_3
\]
This is the same symbol we use for ordinary multiplication, but there
should never be any confusion; you can tell from context whether we
are ``multiplying'' vectors or numbers. (We might also use the dot for
scalar multiplication: $a\cdot\vect{v}=a\vect{v}$; again, it is clear
what is meant from context.)

\begin{example}{}{}
Find the angle between the vectors $\vect{v}=\langle 1,2,1\rangle$ and
$\vect{w}=\langle 3,1,-5\rangle$.
\end{example}

\begin{solution}
We know that
$\cos\theta=\vect{v}\cdot\vect{w}/(|\vect{v}||\vect{w}|)=
(1\cdot3 + 2\cdot1 + 1\cdot(-5))/(|\vect{v}||\vect{w}|)=0$, so
$\theta=\pi/2$, that is, the vectors are perpendicular.
\end{solution}

\begin{example}{}{}
Find the angle between the vectors $\vect{v}=\langle 3,3,0\rangle$ and
$\vect{w}=\langle 1,0,0\rangle$.
\end{example}

\begin{solution}
We compute
\begin{align*}
\cos\theta &= (3\cdot1 + 3\cdot0 + 0\cdot0)/(\sqrt{9+9+0}\sqrt{1+0+0})	\\
&= 3/\sqrt{18} = 1/\sqrt2
\end{align*}
so $\theta=\pi/4$.
\end{solution}

The following are some special cases worth looking at.

\begin{example}{}{}
Find the angles between:
\begin{enumerate}
\item $\vect{v}$ and $\vect{v}$
\item $\vect{v}$ and $-\vect{v}$
\item $\vect{v}$ and $\vect{0}=\langle 0,0,0\rangle$
\end{enumerate}
\end{example}

\begin{solution}
\begin{enumerate}
\item
$\ds \cos\theta= \vect{v}\cdot\vect{v}/(|\vect{v}||\vect{v}|)=(v_1^2+v_2^2+v_3^2)/
(\sqrt{v_1^2+v_2^2+v_3^2}\sqrt{v_1^2+v_2^2+v_3^2})=1$, so the angle
between $\vect{v}$ and itself is zero, which of course is correct.

\item
$\ds\cos\theta=\vect{v}\cdot-\vect{v}/(|\vect{v}||-\vect{v}|)=(-v_1^2-v_2^2-v_3^2)/
(\sqrt{v_1^2+v_2^2+v_3^2}\sqrt{v_1^2+v_2^2+v_3^2})=-1$, so the angle
is $\pi$, that is, the vectors point in opposite directions, as of
course we already knew.

\item
$\ds \cos\theta= \vect{v}\cdot\vect{0}/(|\vect{v}||\vect{0}|)=(0+0+0)/
(\sqrt{v_1^2+v_2^2+v_3^2}\sqrt{0^2+0^2+0^2})$, which is undefined.
On the other hand, note that since $\vect{v}\cdot\vect{0}=0$ it looks
at first as if $\cos\theta$ will be zero, which as we have seen means
that vectors are perpendicular; only when we notice that the
denominator is also zero do we run into trouble. One way to ``fix''
this is to adopt the convention that the zero vector $\vect{0}$ is
perpendicular to all vectors; then we can say in general that if
$\vect{v}\cdot\vect{w}=0$, $\vect{v}$ and $\vect{w}$ are perpendicular.
\end{enumerate}
\end{solution}

Generalizing the examples, note the following useful facts:

\begin{itemize}
	\item	If $\vect{v}$ is parallel or anti-parallel to $\vect{w}$ then
	$\vect{v}\cdot\vect{w}/(|\vect{v}||\vect{w}|)=\pm1$, and conversely, if
	$\vect{v}\cdot\vect{w}/(|\vect{v}||\vect{w}|)=1$, $\vect{v}$ and $\vect{w}$	are parallel, while if $\vect{v}\cdot\vect{w}/(|\vect{v}||\vect{w}|)=-1$, $\vect{v}$ and $\vect{w}$ are anti-parallel. (Vectors are
	parallel if they point in the same direction,
	anti-parallel if they point in opposite directions.) \index{vector!parallel}\index{vector!anti-parallel}
	\item	If $\vect{v}$ is perpendicular to $\vect{w}$ then
	$\vect{v}\cdot\vect{w}/(|\vect{v}||\vect{w}|)=0$, and conversely if
	$\vect{v}\cdot\vect{w}/(|\vect{v}||\vect{w}|)=0$ then
	$\vect{v}$ and $\vect{w}$ are perpendicular. \index{vector!perpendicular}
\end{itemize}

Given two vectors, it is often useful to find the \dfont{projection}\index{vector!projection}
of one vector onto the other, because this turns out to have important
meaning in many circumstances. More precisely, given $\vect{v}$ and
$\vect{w}$, we seek a vector parallel to $\vect{w}$ but with length
determined by $\vect{v}$ in a natural way, as shown in
Figure~\ref{fig:vector projection}. $\vect{p}$ is chosen so that the
triangle formed by $\vect{v}$, $\vect{p}$, and $\vect{v}-\vect{p}$
is a right triangle.

\begin{figure}[H]
\centerline{
\vbox{\beginpicture
\normalgraphs
\setcoordinatesystem units <6truemm,6truemm>
\setplotarea x from 0 to 7, y from 0 to 4
\arrow <4pt> [0.35, 1] from 0 0 to 3 4
\arrow <4pt> [0.35, 1] from 0 0 to 3.98 1.71
\setdashes
\arrow <4pt> [0.35, 1] from 0 0 to 7 3
\plot 3 4 3.98 1.71 /
\put {$\vect{v}$} [b] <0pt,3pt> at 3 4
\put {$\vect{w}$} [l] <3pt,0pt> at 7 3
\put {$\vect{p}$} [tl] <3pt,-3pt> at 3.98 1.71
\put {$\theta$} [bl] <11pt,8pt> at 0 0
\endpicture}}
\caption{$\vect{p}$ is the projection of $\vect{v}$ onto $\vect{w}$. \label{fig:vector projection}}
\end{figure}

Using a little trigonometry, we see that
$$
  |\vect{p}|=|\vect{v}|\cos\theta=
  |\vect{v}|{\vect{v}\cdot\vect{w}\over|\vect{v}||\vect{w}|}=
  {\vect{v}\cdot\vect{w}\over|\vect{w}|};
$$
this is sometimes called the \dfont{scalar projection of $\vect{v}$ onto $\vect{w}$}\index{vector!scalar projection}. To get $\vect{p}$
itself, we multiply this length by a vector of length one parallel to
$\vect{w}$:
$$
  \vect{p}= {\vect{v}\cdot\vect{w}\over|\vect{w}|}{\vect{w}\over|\vect{w}|}=
  {\vect{v}\cdot\vect{w}\over|\vect{w}|^2}\vect{w}.
$$
Be sure that you understand why $\vect{w}/|\vect{w}|$ is a vector of
length one (also called a
\dfont{unit vector}\index{unit vector}) parallel to $\vect{w}$.

The discussion so far implicitly assumed that $0\le\theta\le\pi/2$.
If $\pi/2<\theta\le\pi$, the picture is like
Figure~\ref{fig:obtuse vector projection}.
In this case $\vect{v}\cdot \vect{w}$ is negative, so the vector
$${\vect{v}\cdot\vect{w}\over|\vect{w}|^2}\vect{w}$$
is anti-parallel to $\vect{w}$, and its length is
$$\left|{\vect{v}\cdot\vect{w}\over|\vect{w}|}\right|.$$
In general, the scalar projection of $\vect{v}$ onto $\vect{w}$
may be positive or negative. If
it is negative, it means that the projection vector is anti-parallel
to $\vect{w}$ and that the length of the projection vector is the
absolute value of the scalar projection. Of course, you can also
compute the length of the projection vector as usual, by applying the
distance formula to the vector.

\begin{figure}[H]
\centerline{
\vbox{\beginpicture
\normalgraphs
\setcoordinatesystem units <6truemm,6truemm>
\setplotarea x from -4 to 7, y from -2 to 4
\arrow <4pt> [0.35, 1] from 0 0 to -4.966 0.586
\arrow <4pt> [0.35, 1] from 0 0 to -3.98 -1.71
\setdashes
\arrow <4pt> [0.35, 1] from 0 0 to 7 3
\plot -4.966 0.586 -3.98 -1.71 /
\put {$\vect{v}$} [b] <0pt,3pt> at -4.966 0.586
\put {$\vect{w}$} [l] <3pt,0pt> at 7 3
\put {$\vect{p}$} [tl] <3pt,-3pt> at -3.98 -1.71
\put {$\theta$} [b] <0pt,3pt> at 0 0
\endpicture}}
\caption{$\vect{p}$ is the projection of $\vect{v}$ onto $\vect{w}$. \label{fig:obtuse vector projection}}
\end{figure}

Note that the phrase ``projection onto $\vect{w}$'' is a bit misleading
if taken literally; all that $\vect{w}$ provides is a direction; the
length of $\vect{w}$ has no impact on the final vector. In
Figure~\ref{fig:short projection}, for example, $\vect{w}$ is shorter than
the projection vector, but this is perfectly acceptable.

\begin{figure}[H]
\centerline{
\vbox{\beginpicture
\normalgraphs
\setcoordinatesystem units <6truemm,6truemm>
\setplotarea x from 0 to 7, y from 0 to 4
\arrow <4pt> [0.35, 1] from 0 0 to 3 4
\setdashes
\arrow <4pt> [0.35, 1] from 0 0 to 2.3333 1
\put {$\vect{v}$} [b] <0pt,3pt> at 3 4
\put {$\vect{w}$} [t] <0pt,-5pt> at 2.3333 1
\put {$\theta$} [bl] <11pt,8pt> at 0 0
\setcoordinatesystem units <6truemm,6truemm> point at -9 0
\setplotarea x from 0 to 7, y from 0 to 4
\setsolid
\arrow <4pt> [0.35, 1] from 0 0 to 3 4
\arrow <4pt> [0.35, 1] from 0 0 to 3.98 1.71
\setdashes
\arrow <4pt> [0.35, 1] from 0 0 to 2.3333 1
\plot 3 4 3.98 1.71 /
\put {$\vect{v}$} [b] <0pt,3pt> at 3 4
\put {$\vect{w}$} [t] <0pt,-5pt> at 2.3333 1
\put {$\vect{p}$} [tl] <3pt,-3pt> at 3.98 1.71
\put {$\theta$} [bl] <11pt,8pt> at 0 0
\endpicture}}
\caption{$\vect{p}$ is the projection of $\vect{v}$ onto $\vect{w}$. \label{fig:short projection}}
\end{figure}

Physical force is a vector quantity. It is often necessary to compute
the ``component'' of a force acting in a different direction than the
force is being applied.

\begin{example}{Components of Force Vector}{components of force vector}
Suppose a ten pound weight is resting on an inclined plane---a pitched roof, for example. Gravity
exerts a force of ten pounds on the object, directed straight down. It
is useful to think of the component of this force directed down and
parallel to the roof, and the component down and directly into the
roof. These forces are the projections of the force vector onto
vectors parallel and perpendicular to the roof. Suppose the roof is
tilted at a $30^\circ$ angle, as in Figure~\ref{fig:components of force}. Compute the component of the force directed down the roof and the component of the force directed into the roof.
\end{example}

\begin{solution}
A vector parallel to the roof is $\langle-\sqrt3,-1\rangle$,
and a vector perpendicular to the roof is
$\langle 1,-\sqrt3\rangle$.  The force vector is $\vect{F}=\langle
0,-10\rangle$. The component of the force directed down the roof is
then
\begin{align*}
  \vect{F}_1&={\vect{F}\cdot
  \langle-\sqrt3,-1\rangle\over|\langle-\sqrt3,-1\rangle|^2}
  \langle-\sqrt3,-1\rangle
  ={10\over 2}{\langle-\sqrt3,-1\rangle\over2}=
  \langle -5\sqrt3/2,-5/2\rangle
\end{align*}
with length 5.  The component of the force directed into the roof is
\begin{align*}
  \vect{F}_2&={\vect{F}\cdot
  \langle1,-\sqrt3\rangle\over|\langle1,-\sqrt3\rangle|^2}
  \langle1,-\sqrt3\rangle
={10\sqrt3\over 2}{\langle1,-\sqrt3\rangle\over2}=
\langle 5\sqrt3/2,-15/2\rangle\cr
\end{align*}
with length $5\sqrt3$. Thus, a force of 5 pounds is pulling the object
down the roof, while a force of $5\sqrt3$ pounds is pulling the object
into the roof.
\end{solution}

\begin{figure}[H]
\centerline{
\vbox{\beginpicture
\normalgraphs
\setcoordinatesystem units <2truecm,2truecm>
\setplotarea x from 0 to 3.5, y from 0 to 2
\plot 0 0 3.464 2 /
%\setplotsymbol ({\twelvepoint.})
\arrow <4pt> [0.35, 1] from 2.5 1.443 to 2.067 1.193
\arrow <4pt> [0.35, 1] from 2.5 1.443 to 2.5 0.443
\arrow <4pt> [0.35, 1] from 2.5 1.443 to 2.933 0.683
\put {$\vect{F}$} [t] <0pt,-5pt> at 2.5 0.443
\put {$\vect{F}_2$} [tl] <3pt,-3pt> at 2.933 0.683
\put {$\vect{F}_1$} [br] <-3pt,3pt> at 2.067 1.193
\endpicture}}
\caption{Components of a force. \label{fig:components of force}}
\end{figure}

The dot product has some familiar-looking properties that will be
useful later, so we list them here. These may be proved by writing the
vectors in coordinate form and then performing the indicated
calculations; subsequently it can be easier to use the properties
instead of calculating with coordinates.

\begin{theorem}{Dot Product Properties}{dot product properties}
If $\vect{u}$, $\vect{v}$, and $\vect{w}$ are vectors and $a$ is a real
number, then
\begin{enumerate}
	\item	$\ds \vect{u}\cdot\vect{u} = |\vect{u}|^2$
	\item	$\vect{u}\cdot\vect{v} = \vect{v}\cdot\vect{u}$
	\item	$\vect{u}\cdot(\vect{v}+\vect{w}) =
	\vect{u}\cdot\vect{v}+\vect{u}\cdot\vect{w}$
	\item	$(a\vect{u})\cdot\vect{v}=a(\vect{u}\cdot\vect{v})
	=\vect{u}\cdot(a\vect{v})$
\end{enumerate}\index{dot product!properties}
\end{theorem}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Opensolutionfile{solutions}[ex]
\section*{Exercises for \ref{sec:3Ddotproduct}}

\begin{enumialphparenastyle}

\begin{ex}
Find $\langle 1,1,1\rangle\cdot\langle 2,-3,4\rangle$.
\begin{sol}
	$3$
\end{sol}
\end{ex}

\begin{ex}
Find $\langle 1,2,0\rangle\cdot\langle 0,0,57\rangle$.
\begin{sol}
	$0$
\end{sol}
\end{ex}

\begin{ex}
Find $\langle 3,2,1\rangle\cdot\langle 0,1,0\rangle$.
\begin{sol}
	$2$
\end{sol}
\end{ex}

\begin{ex}
Find $\langle -1,-2,5\rangle\cdot\langle 1,0,-1 \rangle$.
\begin{sol}
	$-6$
\end{sol}
\end{ex}

\begin{ex}
Find $\langle 3,4,6\rangle\cdot\langle 2,3,4\rangle$.
\begin{sol}
	$42$
\end{sol}
\end{ex}

\begin{ex}
Find the cosine of the angle between $\langle 1,2,3\rangle$
and $\langle 1,1,1\rangle$; use a calculator if necessary to find the angle.
\begin{sol}
	$\sqrt6/\sqrt7$, $\approx 0.39$
\end{sol}
\end{ex}

\begin{ex}
Find the cosine of the angle between $\langle -1, -2,-3\rangle$
and $\langle 5,0,2\rangle$; use a calculator if necessary to find the angle.
\begin{sol}
	$-11\sqrt{14}\sqrt{29}/406$, $\approx 2.15$
\end{sol}
\end{ex}

\begin{ex}
Find the cosine of the angle between $\langle 47,100,0\rangle$
and $\langle 0,0,5\rangle$; use a calculator if necessary to find the angle.
\begin{sol}
	$0$, $\pi/2$
\end{sol}
\end{ex}

\begin{ex}
Find the cosine of the angle between $\langle 1,0,1 \rangle$
and $\langle 0,1,1\rangle$; use a calculator if necessary to find the angle.
\begin{sol}
	$1/2$, $\pi/3$
\end{sol}
\end{ex}

\begin{ex}
Find the cosine of the angle between $\langle 2,0,0\rangle$
and $\langle -1,1,-1\rangle$; use a calculator if necessary to find the angle.
\begin{sol}
	$-1/\sqrt3$, $\approx 2.19$
\end{sol}
\end{ex}

\begin{ex}
Find the angle between the diagonal of a cube and one of the
edges adjacent to the diagonal.
\begin{sol}
	$\arccos(1/\sqrt3)\approx 0.96$
\end{sol}
\end{ex}

\begin{ex}
Find the scalar and vector projections of $\langle 1,2,3\rangle$
onto $\langle 1,2,0\rangle$.
\begin{sol}
	$\sqrt{5}$, $\langle 1,2,0\rangle$.
\end{sol}
\end{ex}

\begin{ex}
Find the scalar and vector projections of $\langle 1,1,1\rangle$
onto $\langle 3,2,1\rangle$.
\begin{sol}
	$3\sqrt{14}/7$, $\langle 9/7,6/7,3/7\rangle$.
\end{sol}
\end{ex}

\begin{ex}
A force of 10 pounds is applied to a wagon, directed at an
angle of $30^\circ$. Find the component of this force pulling the
wagon straight up, and the component pulling it horizontally along
the ground.
\begin{sol}
	$\langle 0,5\rangle$, $\langle 5\sqrt3,0\rangle$
\end{sol}

\begin{figure}[H]
\centerline{
\vbox{\beginpicture
\normalgraphs
\setcoordinatesystem units <5truemm,5truemm>
\setplotarea x from 0 to 11, y from 0 to 4
\putrule from 0 0 to 11 0
\putrule from 6 0.5 to 6 2
\putrule from 6 2 to 2 2
\putrule from 2 2 to 2 0.5
\putrule from 2 0.5 to 6 0.5
\circulararc 360 degrees from 3.5 0.5 center at 3 0.5
\circulararc 360 degrees from 5.5 0.5 center at 5 0.5
\plot 6 1.5 8 2.655 /
%\setplotsymbol ({\twelvepoint.})
\arrow <4pt> [0.35, 1] from 8.5 2.943 to 10 3.81
\put {$\vect{F}$} [bl] <3pt,3pt> at 10 3.81
\endpicture}}
\caption{Pulling a wagon. \label{fig:pulling a wagon}}
\end{figure}
\end{ex}

\begin{ex}
A force of 15 pounds is applied to a wagon, directed at an
angle of $45^\circ$. Find the component of this force pulling the
wagon straight up, and the component pulling it horizontally along
the ground.
\begin{sol}
	$\langle 0,15\sqrt2/2\rangle$,$\langle 15\sqrt2/2,0\rangle$
\end{sol}
\end{ex}

\begin{ex}
Use the dot product to find a non-zero vector $\vect{w}$
perpendicular to both $\vect{u}=\langle 1,2,-3\rangle$ and
$\vect{v}=\langle 2,0,1\rangle$.
\begin{sol}
	Any vector of the form $\langle a, -7a/2, -2a\rangle$
\end{sol}
\end{ex}

\begin{ex}
Let $\vect{x}=\langle 1,1,0 \rangle$ and $\vect{y}=\langle
2,4,2 \rangle$.  Find a unit vector that is perpendicular to both $\vect{x}$ and $\vect{y}$.
\begin{sol}
	$\langle 1/\sqrt3,-1/\sqrt3,1/\sqrt3\rangle$
\end{sol}
\end{ex}

\begin{ex}
Do the three points $(1,2,0)$, $(-2,1,1)$, and $(0,3,-1)$
form a right triangle?
\begin{sol}
	No.
\end{sol}
\end{ex}

\begin{ex}
Do the three points $(1,1,1)$, $(2,3,2)$, and $(5,0,-1)$
form a right triangle?
\begin{sol}
	Yes.
\end{sol}
\end{ex}

\begin{ex}
Show that $|\vect{v}\cdot\vect{w}|\le|\vect{v}||\vect{w}|$
\end{ex}

\begin{ex}
Let $\vect{x}$ and $\vect{y}$ be perpendicular vectors.  Use
Theorem~\ref{thm:dot product properties} to prove that $|\vect{x}|^2+|\vect{y}|^2=|\vect{x}+\vect{y}|^2$.  What is this result better known as?
\end{ex}

\begin{ex}
Prove that the diagonals of a rhombus intersect at right angles.
\end{ex}

\begin{ex}
Suppose that $\vect{z}=|\vect{x}| \vect{y} + |\vect{y}| \vect{x}$
where $\vect{x}$, $\vect{y}$, and $\vect{z}$ are all nonzero vectors.  Prove
that $\vect{z}$ bisects the angle between $\vect{x}$ and $\vect{y}$.
\end{ex}

\begin{ex}
Prove Theorem~\ref{thm:dot product properties}.
\end{ex}

\end{enumialphparenastyle}


\subsection{Parallel transport}

Move and therefore change basis, but components are the same.


%% Decision Procedure %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{The BLT Decision Procedure}
\label{sec:dp}

To describe our decision procedure, we assume that we are attempting
to check whether the constraints below are satisfiable:
%
\begin{equation}
\label{eq:prob-matrix}
    \v{l} \le \mat{A} \v{x} \le \v{u}.
    \tag{P1}
\end{equation}
%
We let $n$ denote the number of free integer variables in $\v{x} \in \ZZ^n$,
and let $m$ denote the number of constrained linear forms.  The coefficients
are rational, so $\mat{A} \in \QQ^{m \times n}$ and $\v{l},\v{u} \in \QQ^m$.

Without loss of generality we assume that $n \le m$ and that $\mat{A}$ has
rank $n$ (full rank). In case the problem at hand is such that $n > m$ and/or
that $\mat{A}$ is less than full rank, one can compute a basis of the column
space, say $\mat{A}'$, that meets the requirement (cf. \cite{Cohen}, \S~2.7.1).
The new system $\v{l} \le \mat{A}' \v{y} \le \v{u}$ is equisatisfiable with
the original and solutions of the new system
determine one or more solutions of the original.

Geometrically, we can think of $\v{l}$ and $\v{u}$ as opposite corners of an
$m$-dimensional hyperrectangle defined by
%
\[ \Co := \{ \v{z} \in \RR^m  \mid  \v{l}_i \le \v{z}_i \le \v{u}_i \}. \]
%
We refer to this as the \emph{constraint set} of the problem.
Without loss of generality we may scale the rows of \eqref{eq:prob-matrix} so
that the width of $\Co$ is the same along every axis, i.e.
%
\begin{equation}
    \label{eq:cube}
    \v{u}_i - \v{l}_i = \v{u}_j - \v{l}_j \quad \forall \, i,j \in \{1,\ldots,m\}.
\end{equation}
%
Note that this transformation makes $\Co$ a \emph{hypercube}.
We let $d_\Co$ denote the common width, and let $r_\Co = d_\Co/2$ denote the
corresponding radius of the hypercube.

The problem given by \eqref{eq:prob-matrix} can also be characterized as
trying to find a common point in both a hypercube and a
lattice.
The columns of $\mat{A}$, regarded as vectors, generate a lattice.
%
Let $\{\v{b}_1, \v{b}_2, \ldots, \v{b}_n\}$ denote the column vectors of $\mat{A}$ and define:
%
\begin{equation}
    \label{eq:lattice-def}
    \La := \left\{ a_1 \v{b}_1 + \cdots + a_n \v{b}_n \in \RR^m \mid
            a_i \in \ZZ \right\}
\end{equation}
%
By our assumption that $\mat{A}$ has full rank,
$\{\v{b}_1, \ldots, \v{b}_n\}$ are linearly independent,
and there is a one-to-one
correspondance between elements in $\La \cap \Co$ and
satifying assignments to~\eqref{eq:prob-matrix}.
%
It follows that checking the satisfiability of~\eqref{eq:prob-matrix}
is equivalent to deciding whether $\La \cap \Co$ is non-empty.

The set $\La \cap \Co$ is guaranteed to be finite as a
lattice must have a finite number of elements in any space with
a bounded volume.
%
Due to the one-to-one correspondance, the number of
solutions to~\eqref{eq:prob-matrix} must be finite as well, and
hence a procedure capable of enumerating the elements in $\La \cap \Co$ can
be used as a decision procedure for checking the satisfiability
of~\eqref{eq:prob-matrix}.

Before describing such a procedure, we note that a nice feature
of the lattice and hypercube formulation is that it provides
a simple way to estimate the number of satisfying assignments. The
\emph{volume} of a lattice $\La$ can be defined in several ways (see
\cite{Lenstra}, \S5), but the simplest computationally is $\vol(\La) =
\abs{\det{\m{B}}}$ where the columns of $\m{B}$ generate $\La$. Then, the
number of elements of $\La \cap \Co$ is approximately
$\vol{\Co}\:{/}\:\vol{\La}$.
We will
use this to compute the number of expected solutions for the JPEG preimage
problems described in the Section~\ref{sec:jpeg}.

% We will lateFigure
%\ref{fig:solution_count} shows a plot of this estimate for a specific family
%of problems.

%% Decision Procedure %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Enumerating Lattice Elements}
\label{ssec:dp}

% What I want to say...

% Our algorithm is a search over the tree of partial assignments $x_1 = a_1, x_2
% = a_2, \ldots, x_j = a_j$. We use a \emph{best-first} search strategy, where
% "best" at each step means making the assignment to the next variable
% such that the resulting layer is the \emph{closest} one to center among the
% remaining possibilities. The best-first search is also pruned by keeping track
% of a best known upper bound on the distance of a closest lattice vector to
% center. Layers whose distance to center exceeds the known upper bound are not
% examined.
%
% In section \ref{ssec:cvp-inf}, we discuss a slightly more general family of
% search strategies and show that they have good properties. After that we give
% some details about our implementation.
%

We now turn our attention to the problem of enumerating the elements
$\La \cap \Co$.  Recall that, without loss of generality, we have
taken $\Co$ to be a hypercube. Let $\v{p}$ denote the geometric center
of $\Co$:
%
\begin{equation}
    \label{eq:center}
    \v{p} := (\frac{\v{l}_1 + \v{u}_1}{2}, \ldots, \frac{\v{l}_m + \v{u}_m}{2}).
\end{equation}
With respect to the $\linf$ metric, $\Co$ is a closed ball of radius $r_\Co$,
centered at $\v{p}$, and hence the elements in $\La \cap \Co$ are
precisely those that are at most a distance $r_\Co$ from $\v{p}$.

Algorithms for finding lattice elements that are close to a given point have been
extensively studied and many algorithmic approaches to it exist; see
\cite{AgrellEtAl} for a good survey.
%
We have developed a complete search procedure by adapting the Schnorr-Euchner
algorithm for computing the closest vector point to a
lattice~\cite{Schnorr-Euchner}.

%In adapting the algorithm, we have made three changes: (1) We use the $\linf$
%metric instead of the Euclidean $\ltwo$
%metric; (2) rather than compute the closest element, we immediately return
%when we find an element within the hypercube; and (3) we prune search paths
%when the $R_\Co$
%Our adaption %$\linf$, both because of its performance characteristics and the ease with
%which we could change metrics.

% In particular, $\v{z} \in \Co$ if and only if $\norminf{z - p} \le r_{\Co}$.
%In the algorithm below we use \proc{Center} to denote the calculation
%\eqref{eq:center}.

%Now, suppose we have a procedure $\proc{Closest}_\infty$ which takes as input
%a lattice $\La \subset \RR^m$, a point $\v{q} \in \RR^m$, and returns a
%closest lattice vector to $\v{q}$, not in the usual $L^2$ metric, but in the
%$\linf$ metric. Assuming this for the moment, we arrive at our decision
%procedure for \eqref{eq:prob-matrix}.

%\begin{algorithm}[H]
    %\SetLine
%    \KwIn{a lattice $\La \subset \RR^m$ and a hypercube $\Co$}
%    \KwOut{``SAT'' and a $\v{z} \in \La \cap \Co$, or ``UNSAT''}
%    $\v{p} \leftarrow \proc{Center}(\Co)$\;
%    $\v{r} \leftarrow \proc{Closest}_\infty(\La, \v{p})$\;
%    \eIf{$\v{r} \in \Co$}{
%        \KwRet{$(\text{SAT}, \v{r})$}\;
%    }{
%        \KwRet{UNSAT}\;
%    }
%    \caption{Decision procedure for lattice points in a hypercube}
%\end{algorithm}

%The first branch is obviously correct. For the alternative, simply observe
%that if the closest lattice vector to $\v{p}$ in the $\linf$ metric is not
%contained in $\Co$ then there can be no lattice vector in $\Co$ (any such
%vector would be strictly closer to $\v{p}$).

%The utility of this simple procedure obviously hinges on
%$\proc{Closest}_\infty$.

%\subsection{Closest Vector Search with Infinity Norm}
%\label{ssec:cvp-inf}

%The closest vector problem has been extensively studied and many algorithmic
%approaches to it exist; see \cite{AgrellEtAl} for a good survey. In our
%implementation BLT we have chosen to adapt the Schnorr-Euchner strategy for
%$\linf$, both because of its performance characteristics and the ease with
%which we could change metrics.
\newcommand{\ruleref}{\textbf{Split}}

We model our search procedure as a non-deterministic transition rule on partial
assignments to the vectors $\v{x}$.  The procedure begins with the empty
assignment $\emptyset$, and incrementally assigns values to variables in $\v{x}$.
If the transition rule terminates with a complete assignment
$\v{u} \in \ZZ^n$, then $\v{u}$ is a solution to the constraint
problem.
%
\begin{equation}
%\label{eq:rules-split}
%\tag{\textbf{Split}}
\mathbf{(Split)}
\hspace{20pt} \theta \ \Rightarrow \ \theta \cup \{\,j \mapsto s\,\}
   \ \textbf{where}\,\begin{cases}
      j \in \{\,1, \ldots, n\,\} \setminus \fn{dom}(\theta)\\
      s \in \ZZ\ \textbf{s.t.}
      \ \La^\RR_{\theta \cup \{\,j \mapsto s\,\}} \cap \Co \neq \emptyset
      \end{cases}
\end{equation}

This rule takes a partial assignment $\theta$, and extends it with an additional
binding $j \mapsto s$ such that the real-affine linear space
$\La^\RR_{\theta \cup \{\,j \mapsto s\,\}}$ of the resulting assignment
intersects with $\Co$.  This rule models backtracking implicitly; at each
step, we may find that there is no legal value $s$ to assign $j$.  If this
occurs, our procedure must backtrack to a previous step, and explore an
alternative assignment.

We can show that the set of lattice points in $\Co$ can be enumerated by
applying~\ruleref{} transitively starting
from $\emptyset$.
%
Before stating the theorem, we first observe that each point $\v{x} \in \La$
can be expressed as the weighted sum of the columns in $\mat{A}$,
(i.e.,~$\v{x} = \mat{A}\v{u}$ for some unique $\v{u} \in \ZZ^n$).
%
\begin{thm}
  For each vector $\v{u} \in \ZZ^n$, $\mat{A} \v{u} \in \Co$ iff. there is
  a derivation $\emptyset \Rightarrow^{+} \v{u}$.
\end{thm}
\begin{proof}
%
To see that $\emptyset \Rightarrow^{+} \v{u}$ implies $\mat{A} \v{u} \in \Co$,
observe that the proceeding step must have shown that
$\La^\RR_{\v{u}} \cap \Co \neq \emptyset$.  Since $\v{u}$
is a complete assignment, $\La^\RR_{\v{u}} = \{\,\mat{A} \v{u}\,\}$, and
hence $\mat{A} \v{u} \in \Co$.

To see that $\mat{A} \v{u} \in \Co$ implies $\emptyset \Rightarrow^{+} \v{u}$,
observe that for all
partial assignments $\theta$, $\La^\RR_\theta \cap \Co \neq \emptyset$
implies that $\emptyset \Rightarrow^{+} \theta$
by induction on the number of bindings in $\theta$.
%
For a complete assignment $\v{u}$, $\La^\RR_{\v{u}} = \{\,\mat{A} \v{u}\}$.
Hence, if $\mat{A} \v{u}$ is in $\Co$, then
$\La^\RR_{\v{u}}$ is a non-empty subset of $\Co$ and $\varnothing \Rightarrow^{+} \v{u}$.
\end{proof}
%
We note that the above theorem holds regardless of the order in which we choose
the values of $j$ in~\ruleref{}.  We only need to consider all valid
assignments to $s$ once we have chosen $j$.  An implementation then has
a choice in which it can use different heuristics to search for an
assignment.
%
We will briefly describe
the heuristics used by BLT in Section~\ref{ssec:blt-optimizations}.

The previous theorem shows that the transition rule is sound and complete from
a logical point of view, to show that it is computable
we prove the following:

\begin{thm}
  The set of partial assignments $\theta$ such that
  $\emptyset \Rightarrow^{+} \theta$ is finite and
  computable.
  \label{thm:computable}
\end{thm}

\begin{proof}
As each application of~\ruleref{} adds an additional
binding to the substitution, the number of applications along any
path is bounded by $n$.  To show that the set of $\theta$ is finite,
we must show that the number of potential values of $s$ used to
instantiate~\ruleref{} is both finite and computable.
More precisely, we must prove that
there are at most a finite number of integers $s \in \ZZ$ such that
\begin{equation}
\La^\RR_{\theta \cup \{\,j \mapsto s\,\}} \cap \Co \neq \emptyset.
\label{eq:extend_theta}
\end{equation}
%
Observe that for any $u,k \in \ZZ$ with $k \neq 0$, the affine set
$\La^\RR_{\theta \cup \{\,j \mapsto u+k\,\}}$ can be
obtained by shifting the set $\La^\RR_{\theta \cup \{\,j \mapsto u\,\}}$
by a multiple $k$ of the basis vector $\v{b}_j$.  As $\v{b}_j$ is linearly
independent from the other basis vectors, it follows that
$\La^\RR_{\theta \cup \{\,j \mapsto u\,\}}$ and
$\La^\RR_{\theta \cup \{\,j \mapsto u+k\,\}}$
are disjoint and separated by some
positive distance $k \times d_{\theta,j}$ where $d_{\theta,j}$ is the distance
between the adjacent hyperplanes $\La^\RR_{\theta \cup \{\,j \mapsto 0\,\}}$ and
$\La^\RR_{\theta \cup \{\,j \mapsto 1\,\}}$
%
As the distance between any two points $\v{x}, \v{y} \in \Co$ is at most
$d_\Co$, it follows that the number of distinct $s$
satisfying~\eqref{eq:extend_theta} is at most
$d_\Co / d_{\theta,j}$.  Moreover, as both $\Co$ and
$\La^\RR_{\theta \cup \{\,j \mapsto s\,\}}$ are convex, there
must be a bounded interval $s \in \{\,l,l+1,\dots, u-1, u\,\}$
of values satisfying~\eqref{eq:extend_theta}.

Rather than compute the bounds explicitly, we
compute the $\linf$-distance between $\La^\RR_\theta$ and the point $\v{p}$ at
the center of $\Co$ using the reduction to linear programming described at the
end of the Section~\ref{sec:preliminaries} in equation~\eqref{eq:set_distance}.
%
This reduction to linear programming allows one to find an assignment
$\v{y} \in \RR^{n}$ so that $\mat{A} \v{y}$ is one of the points
in $\La^\RR_\theta$ with minimal distance to $\v{p}$.  We can then start by
considering for $s$ the points $\{\,\floor{\v{y}_j},\,\floor{\v{y}_j-1},\dots\,\}$
and $\{\,\ceil{\v{y}_j},\,\ceil{\v{y}_j+1},\dots\,\}$ until we have
explored all the assignments in the set $\{\,l,l+1,\dots, u\,\}$.
%
\end{proof}

%\proc{Split} says we can take a sublayer in $S$ and decompose it into its
%component sub-sublayers by choosing an unassigned index $j$. The assignments
%$s_i$ are taken to be precisely those for which $d_\infty(\La^\RR_{I \cup (j,
%    s_i)}, p) < r$. This is always a finite set of consecutive integers.
% See figure \ref{fig:layers} for illustration.
%
%\proc{Prune} removes sublayers that we know have no points closer to $\v{p}$
%than $r$.
%
%Finally, \proc{Satisfiable} applies to 0-dimensional sublayers, i.e. lattice
%points. If $\La_I = \{ \v{y} \}$ and the distance from $\v{y}$ to $\v{p}$ is
%less than $r_\Co$, we have found a satisfying assignment $I$.

% \begin{figure}
%     \centering
%     \vspace{0.1cm}
%     \includegraphics[width=0.48\textwidth]{lattice-layers}
%     \caption{\proc{Split} $\La_I$ into sublayers}
%     \label{fig:layers}
% \end{figure}

%Starting at some initial state $S_0$, a \emph{derivation} in this system is a
%sequence of transitions $S_0 \Rightarrow S_1 \Rightarrow \cdots$ using any of
%the three rules as long as their precondition applies. We call any state of
%the form $(\emptyset, r, \v{z})$ a final state.
%
%\begin{lem}
%    \label{lem:search}
%    The transition system described above always terminates in a final state, i.e.
%    \begin{enumerate}
%        \item every derivation is finite,
%        \item the only states in which no rule applies are final states.
%    \end{enumerate}
%\end{lem}
%
%\begin{proof}
%Consider the function $\nu$ which maps states to $n+1$-tuples of natural
%numbers: $\nu(S, r, \v{z})_i = \#\{\La_I \in S \mid \abs{I}=i \}$ for $i=0,\ldots,n$.
%We claim that all three transition rules cause $\nu$ to strictly decrease in
%the lexicographic order. For \proc{Prune} and \proc{Record} this is obvious.
%Applying $\proc{Split}_{j,I}$ causes $\nu_{\abs{I}}$ to decrease by 1 and
%$\nu_{\abs{I}+1}$ to increase (by an amount bounded above by a constant
%multiple of $r$). Hence $\nu$ strictly decreases in lexicographic order and so
%every derivation is finite.
%
%For (2), suppose to the contrary $(S, r, \v{z})$ is a state such that $S \ne
%\emptyset$ but none of the transition rules apply. Choose a $\La_I \in S$. If
%$\abs{I} < n$ then $\proc{Split}_{j,I}$ applies for some $j$. Otherwise, if
%$\abs{I} = n$ then either $d_\infty(\La^\RR_I, \v{p}) \ge r$, in which case
%$\proc{Prune}_I$ applies, or the opposite holds, in which case
%$\proc{Record}_I$ applies. Thus we have a contradiction.
%\end{proof}
%
%\begin{thm}
%    \label{thm:closest}
%    Let $\v{z}_0$ be an arbitrary lattice vector and $r_0 = d_\infty(\v{z}_0,
%    \v{p})$. Then any derivation starting with initial state
%    $(\{\La_\emptyset\}, r_0, \v{z}_0)$ terminates at a final state $(\{\},
%    r_f, \v{z}_f)$ in which $\v{z}$ is a closest vector in $\La$ to $\v{p}$.
%\end{thm}
%
%\begin{proof}
%Assume to the contrary that there is a \emph{closer} lattice vector
%$\tz$. Clearly $\tz \in \La_\emptyset = \La$. Further, if
%we are at a state $(S, r, \v{z})$ in which $\proc{Split}_{j,I}$ applies and $\tz \in
%\La_I$, then there is a new sublayer produced by $\proc{Split}$ that contains
%$\tz$. This follows from the definition of \proc{Split}, since $\tz$ is in
%\emph{some} sublayer of $\La_I$, say $\La_{I \cup (j,s)}$ and
%$d_\infty(\La^\RR_{I \cup (j,s)}, \v{p}) \le d_\infty(\tz, \v{p}) \le r$, the
%latter inequality following from our assumption that $\tz$ is closer than the
%final vector $\v{z}_f$ and hence than $\v{z}$. Similarly it should be clear
%that if at any state, $\tz \in \La_I$ and $\La_I \in S$, then $\proc{Prune}_I$
%does not apply.

%The above argument implies that at some point in our derivation, the
%$0$-dimensional sublayer $\La_J = \{ \tz \}$ must appear. The only rule that
%removes it is $\proc{Record}_J$. Hence, $d_\infty(\tz, \v{p}) \le r_f =
%d_\infty(\v{z}_f, \v{p})$, a contradiction.
%\end{proof}

% The Schnorr-Euchner strategy is best understood as a recursive
% search operation. Recall the problem is to find a $\v{z} \in \La$ such that
% $\norminf{\v{z}-\v{p}}$ is minimal.
%
%
% The main idea in the Schnorr-Euchner search strategy is to recursively search
% for a closet vector in a finite number of the layers, ordered by increasing
% distance from the target point. First we discuss which layers are searched and
% then how to compute the $\linf$ distance from the target point to a layer.
%
% Suppose for a moment that an upper bound $\rho$ is known on the distance from a
% closest vector to $\v{p}$. In this case it suffices to search a finite number
% of consecutive layers.
%
% \begin{lemma}
%     \label{lem:layers-to-search}
%     If $d(\La, p) \le \rho$ then a closest vector is contained in some layer
%     $\mathcal{Y}_a$ such that $\mathcal{Y}^\RR_a \cap H \ne \emptyset$ and
%     moreover $\{ a \in \ZZ \mid \mathcal{Y}^\RR_a \cap H \ne \emptyset \}$ is
%     bounded set of consecutive integers.
% \end{lemma}
% \begin{proof}
%     TODO
% \end{proof}
% The order in which layers are searched makes a significant difference in
% practice. We discuss the choice our implementation makes at the end of the
% section.

\subsection{Implementation Decisions}
\label{ssec:blt-optimizations}

Turning the previous section into a working and efficient
procedure involves many more choices and details than
we have room to describe.  We would like to indicate, however a couple choices
we have made in implementing BLT.

\textbf{Search Strategy.}
%
In implementing the transition system,
we have chosen to adopt a strategy similar to Schnorr and Euchner
in~\cite{Schnorr-Euchner}.
%
We use the LLL algorithm~\cite{Lenstra} to generate a reduced basis, and
fix the basis vectors by sorting in order of decreasing $L^2$ magnitude.
%
We then proceed by applying \ruleref{} in a depth first order
with the sequence of $j$'s chosen according to our basis order.
%
The variables with the largest magnitude are typically the
most-constrained variables in our problems, as they have the largest
distance between adjacent sublayers.
%
Choosing the most constrained variable is a common strategy in
constraint satisfication, and we have found the strategy effective
in this case as well.

The other choice we have with split is to consider which values of $s$ to
explore.  To maximize the likelihood of finding a satisfying assignment,
we would like to choose a value for $s$ that imposes the least
constraints on subsequent assignments.
This could be done by choosing
an assignment to $s$ that maximizes the volume of the intersection
between the hypercube $\Co$ and real-affine set
$\La^\RR_{\theta \cup \{\,j \mapsto s\,\}}$.

Unfortunately, we do not know of an efficient way to compute the
$s$ with the maximal volume\footnote{In~\cite{dyer_freize88}, the
authors show that the related problem of computing the volume of the
intersection of the unit cube and a rational halfspace is \#P-hard.},
but we have developed a proxy that works well in practice.
%
As alluded to in the proof of Theorem~\ref{thm:computable}, we
use linear programming to find an initial assignment to $s$ that minimizes
the $\linf$-distance between the center of the hypercube $\v{p}$ and
$\La^\RR_{\theta \cup \{\,j \mapsto s\,\}}$.
%
Since the distance between the sublayer and center point is minimal, we
can expect that the volume of the sublayer within the hypercube should
be maximal or near maximal.  If this assignment is found
infeasible, and we backtrack, then we explore adjacent assignments
$s + \delta, s - \delta, s + 2\delta, \ldots{}$, where $\delta = \pm 1$
depending on orientation, in order of increasing distance.

%In implementing the transition system,
%we have chosen to adopt a strategy similar to Schnorr and Euchner in
%\cite{Schnorr-Euchner}. After choosing a lattice basis, we fix an ordering of
%the basis vectors and proceed by applying $\proc{Split}_{j,-}$ for the
%sequence of $j$'s corresponding to our basis order. After splitting say
%$\La_I$ into $\La_{I \cup (i,a_1)}, \ldots, \La_{I \cup (i,a_k)}$ we choose
%the sublayer among these that is closest to $\v{p}$ and apply \proc{Split}
%there.  Thus we have a best-first search that always follows the closest
%sublayer first. In this way we quickly reach a $0$-dimensional sublayer,
%namely a lattice point, and apply \proc{Record}. This point is known as the
%\emph{Babai point} in the literature and gives us a good starting $r$ and
%$\v{z}$ value for our state.

%After reaching the Babai point, we backtrack to the $1$-dimensional layer it
%came from and decide to either \proc{Prune} or \proc{Record} at the other
%sublayers there, doing so in order of increasing distance from $\v{p}$
%\footnote{If the closest sublayer was $\La_{I \cup (j,s)}$ and $\v{p}$ is
%``above'' it with respect to the direction of $\v{v}_j$, then one can show
%that the assignments $(s_i) = (s, s+1, s-1, s+2, \ldots)$ enumerate the
%sublayers in order of increasing distance from $\v{p}$.}.  The advantage
%of making this choice is that when we hit a sublayer that \proc{Prune} applies
%to, then we infer that all the other sublayers at this level can be pruned and
%we jump up another level.

%Our implementation deviates slightly from the transition system
%described here in order to make some crucial optimizations. When solving a
%bounded integer constraint problem, the goal is to find a lattice vector
%satisfying the constraints. Accordingly, in our $\proc{Closest}_\infty$
%implementation we check at each application of \proc{Record} whether the new
%point satisfies the constraints and if it does we return early as there is no
%need to find the closest solution. In the problems we've studied, early exit
%like this saves an enormous amount of time, in particular when the Babai point
%mentioned above already satisfies the constraints.

%Finally, note that we've described $\proc{Closest}_\infty$ as starting with
%the state $(\La_\emptyset, r_0, \v{z}_0)$ for some $\v{z}_0 \in \La$ and $r_0 =
%d_\infty(\v{z}_0, \v{p})$. However, if our constraint set $\Co$ is a hypercube
%with radius $r_\Co$, it's obviously better to start with $r_0 = r_\Co$ as this
%is likely to be a much tighter upper bound. Some of the arguments above need
%to be modified to account for this, but the performance gain is significant.

% \paragraph{Lattice Basis.} In section \ref{sec:bcp} we hinted
% choosing a good lattice basis to work with is important. For solving the
% problems presented in section \ref{sec:jpeg}, it is essential. As a
% pre-processing step in BLT we compute a reduced lattice basis once and for all
% using Algorithm 2.6.3 of \cite{Cohen} as implemented in \cite{NTL}. This is
% done \emph{after} the hypercube scaling since, depending on the nature of the constraints
% present, this step can make the lattice matrix quite non-orthogonal.

%Joe
\textbf{Layer-point distance.}
%
Due to efficiency concerns as well as implementation issues with linking
GMP with other Haskell code that BLT is linked against,
we compute the distance using a conventional linear programming solver,
GLPK \cite{GLPK}, which uses IEEE double precision floating point for
its calculations.  If the distance calculation is inaccurate, there is the
potential to prune a sublayer that is mistaken for being slightly too far
away,
and consequently BLT may incorrectly return UNSAT.  In cases where it
returns SAT, the model is checked against the problem for certainty. In
principle the distance calculations can be done using exact
arithmetic\footnote{GLPK supports this directly.} or arbitrary-precision
floating point arithmetic, but we have not attempted to do so yet.


\documentclass[letterpaper, 11pt]{article}

\usepackage[pdfusetitle]{hyperref}
\usepackage[margin=1in, top = 0.5in, bottom = 0.5in]{geometry}
\usepackage{framed}

\pagestyle{empty}

%% \usepackage{listings}
%% \lstloadlanguages{C++}
%% \lstset{
%%   basicstyle=\ttfamily,
%%   language=C++,
%%   breaklines=true,
%%   columns=fullflexible,
%%   showstringspaces=false,
%%   commentstyle=\color[rgb]{0.5,0,0}\itshape,
%% }

\begin{document}

\title{Split block Bloom filters}
\author{Jim Apple \\
  jbapple@apache.org}
\maketitle
\thispagestyle{empty}

Bloom filters and other approximate membership query structures (including quotient filters and cuckoo filters) typically have worst-case operation costs that are linear (or worse) in $\lg (1/\varepsilon)$. \cite{cuckoo-filter,quotient-filter}
In Bloom filters, this is driven by the $\lg (1/\varepsilon)$ hash functions to be evaluated.\footnote{And the same number of bits to access after hashing.}
In dictionary-based filters like cuckoo filters and quotient filters, this is from searching through a near-full table for an open slot to write a fingerprint in.

This brief note describes a Bloom filter in which all operations scale independently of $1/\varepsilon$, with worst-case $O(1)$ operations, first created for Apache Impala in early 2016.\footnote{\url{https://github.com/apache/impala/commit/b35f6d070c8e6b51079f962b448ecc2b0eb74c1a}}\footnote{Rediscovered in 2018 by \cite{ultra-fast}.}
The price paid for this performance is that these filters use a pre-determined number of hash functions, limiting their utility for false positive probabilities outside of $[0.4\%, 19\%]$.
The central ideas are:

\begin{enumerate}
\item Use block Bloom filters to reduce the number of cache lines to access down to one.~\cite{block}
\item Within each block, use a ``split'' Bloom filter that sets one bit in each of several sections, rather than several bits in one section.~\cite{split-bloom}
\item Use eight hash functions in order to fit cleanly into SIMD lanes.\footnote{Four or sixteen would work, too.}
\end{enumerate}

A value is inserted in a split block Bloom filter by first selecting one 256-bit block from the filter by hashing the key once.
Then the key is hashed eight more times to a range of $[0,32)$ using SIMD instructions and multiply-shift universal hashing: $h_{s_i}(x) = \lfloor(s_i \cdot x) / 2^{27}\rfloor$, where $s_i$ are odd seeds.~\cite{multiply-shift}
One bit is set in each of eight contiguous 32-bit lanes within the 256-bit block using the results of the eight hash functions.
Lookup is symmetric; deletions are not supported; code is available at the end of this document.

Each of the central ideas of split block Bloom filters can negatively affect $\varepsilon$ compared to standard Bloom filters.
For instance, in the same space it takes for a split block Bloom filter to support $\varepsilon = 1.0\%$, a standard Bloom filter achieves a false positive rate of $0.63\%$
The false positive rate of split block Bloom filters can be approximated from \cite[Equation 3]{block} and \cite[Section 2.1]{split-bloom}

\[
\sum_{i=0}^\infty P_{256/(m/n)}(i) (1 - (1-8/256)^i)^8
= \sum_{i=0}^\infty P_a(i) (1 - (1-1/32)^i)^8
\]

where $P$ is the Poisson distibution, $n$ is the number of distinct hash values, $m$ is the size of the filter in bits, and $a$ is the average number of distinct hash values per block.
As long as $a \in [20,52]$ ($\varepsilon \in [0.40\%, 19\%]$), the false positive probability of split block Bloom filters is no more than twice that of a standard Bloom filter with the same $n$ and $m$.

In tradeoff for this increased $\varepsilon$, we get very high speed.
For instance, comparing against the 8-bit version of cuckoo filters\footnote{This test is replicable from the original cuckoo filter repo, \url{https://github.com/efficient/cuckoofilter}}\footnote{More benchmarks are also shown in \cite{lemire-xor-filter,ribbon-filter,bloom-overtakes}.}:

\begin{tabular}{|r|r|r|r|}
  \hline  & {\bf 100k elements} & {\bf 1M elements} & {\bf 100M elements} \\
  \hline {\bf Size} & 131KB & 1MB & 134M \\
  \hline {\bf Cuckoo insert (M/s)} & 71 & 33 & 14 \\
  \hline {\bf SBBF insert (M/s)} & 416 & 182 & 32 \\
  \hline {\bf Cuckoo lookup (M/s)} & 281 & 139 & 23 \\
  \hline {\bf SBBF lookup (M/s)} & 400 & 186 & 43 \\
  \hline {\bf Cuckoo $\varepsilon$} & 2.37\% & 2.97\% & 2.33\% \\
  \hline {\bf SBBF $\varepsilon$} & 1.03\% & 2.74\% & 0.91\% \\
  \hline
\end{tabular}

Because of their per-slot metadata, cuckoo filters and quotient filters mainly shine at false positive probabilities less than 0.5\%.
For higher probabilities like those in this table, which are in the target range for Impala's use cases, split block Bloom filters are appropriate, even if not the theoretically optimal.
Split block Bloom filters are now also used in Apache Arrow, Apache Kudu, and Apache Parquet.
The filter is available in a standalone package at \url{https://github.com/jbapple/libfilter}.

See the figure below for a cut-down but working C version of split block Bloom filters.

\section*{Acknowledgements}
Thank you to Daniel Lemire for helpful discussions and inspiring questions.

\bibliographystyle{alpha}
\bibliography{doc}

\appendix{}
\begin{figure}
  \begin{framed}
\begin{verbatim}
#include <immintrin.h>
#include <stdint.h>

// Take a hash value and get the block to access within a filter with
// num_buckets buckets.
uint64_t block_index(const uint64_t hash, const uint32_t num_buckets) {
  return ((hash >> 32) * num_buckets) >> 32;
}

// Takes a hash value and creates a mask with one bit set in each 32-bit lane.
// These are the bits to set or check when accessing the block.
__m256i make_mask(uint32_t hash) {
  const __m256i ones = _mm256_set1_epi32(1);
  // Set eight odd constants for multiply-shift hashing
  const __m256i rehash = {INT64_C(0x47b6137b) << 32 | 0x44974d91,
                          INT64_C(0x8824ad5b) << 32 | 0xa2b7289d,
                          INT64_C(0x705495c7) << 32 | 0x2df1424b,
                          INT64_C(0x9efc4947) << 32 | 0x5c6bfb31};
  __m256i hash_data = _mm256_set1_epi32(hash);
  hash_data = _mm256_mullo_epi32(rehash, hash_data);
  // Shift all data right, reducing the hash values from 32 bits to five bits.
  // Those five bits represent an index in [0, 31)
  hash_data = _mm256_srli_epi32(hash_data, 32 - 5);
  // Set a bit in each lane based on using the [0, 32) data as shift values.
  return _mm256_sllv_epi32(ones, hash_data);
}

void add_hash(uint64_t hash, uint32_t num_buckets, __m256i filter[]) {
  const uint64_t bucket_idx = block_index(hash, num_buckets);
  const __m256i mask = make_mask(hash);
  __m256i *bucket = &filter[bucket_idx];
  // or the mask into the existing bucket
  _mm256_store_si256(bucket, _mm256_or_si256(*bucket, mask));
}

_Bool find_hash(uint64_t hash, uint32_t num_buckets, const __m256i filter[]) {
  const uint64_t bucket_idx = block_index(hash, num_buckets);
  const __m256i mask = make_mask(hash);
  const __m256i *bucket = &filter[bucket_idx];
  // checks if all the bits in mask are also set in *bucket. Scalar
  // equivalent: (~bucket & mask) == 0
  return _mm256_testc_si256(*bucket, mask);
}
\end{verbatim}
  \end{framed}
\end{figure}

\end{document}

%%  LocalWords:  Kudu


\documentclass[twocolumn]{article}

\title{Topic 13: Distributed Systems \\ \Large{Blockchains}}
\author{Hilmar G\'ustafsson}
\begin{document}
\maketitle

\section{Cryptographical Tools}
A summary of the tools necessary for Blockchains.

\subsection{Hash}
A hash function $H$ takes binary input of arbitrary length, and creates fixed-length output of it.
\begin{center}
 $H: X = \{0,1\}^* \rightarrow \{0,1\}^L$ \\ typically where $L \in \{128, 160, 256, 512\}$
\end{center}

For security purposes, it is important that a small change in the input results in a large change in the output.
Collisions exist, but they are hard to find with this property.

\subsection{Signatures using Asymmetric Cryptography}
Digital signatures are typically done using three algorithms, and a known hash $H$.

\paragraph{KeyGen} An algorithm which returns two keys: a private signing key, and a public verification key.

\paragraph{Sign} An algorithm which computes the signature of some input, given the private key.
This input is typically a hash of some data to be sent.

\paragraph{Verify} Decrypts the signature using the public key and compare the result with the hash of the received data.

\subsection{Merkle Trees}
A complete binary tree of hashes.
It is built starting from an initial set of symbols.
Also known as a hash tree.
Explots a hash function H (SHA1, MD5).
Leaves are H applied to the initial symbols.
Internal nodes are H applied to the sons of a node.
\section{Blockchain Basics}
A blockchain is a digitized, decentralized, public ledger of all cryptocurrency transactions. Constantly growing as 'mined' blocks (the most recent transactions) are recorded and added to it in chronological order, it allows market participants to keep track of digital currency transactions without central recordkeeping \footnote{Merkle tree}.
The Blockchain in essence is a collection of Blocks.
Each Block is composed of the following elements: data, a hash pointer, and a timestamp.
\subsection{Hash Pointers}
Pointer to where some info is stored.
Cryptographic hash of the info.
If we have a hash pointer, we can: (1) ask to get the info back, and (2) verify that it hasn't changed.
Similar to how programs to download are often accompanied by a hash, to verify that you downloaded the correct file.
\subsection{Consensus}
The Blockchain network is in principle completely asynchronous and decentralized.
In terms of currency, the problem to solve is double-spending, i.e. being able to spend the same money more than once.

\paragraph{Nakamoto Consensus} Assuming that 1. most nodes are honest, and 2. adding a block is computationally expensive, we can ignore received blockchains which are shorter than ours, for two reasons: 1. it was created later than mine, and 2. it can be out of sync.
This assumes that the honest nodes have more CPU power than attacker nodes.

\subsection{Proof of Work}
Provide a computational puzzle that is hard to solve when you want to add a valid block, but easy to solve when you want to verify that a block is valid.
Specifically, the new block must be a combination of a nonce and a header.
The nonce is accepted when $H(\mathtt{header})$ starts with $n$ zeros.
$n$ defines how hard it is to add a new block.
\paragraph{Example} Say the hashing algorithm is \texttt{SHA256}, and $n = 36$. Each nonce has the probability $2^{-36}$ of success. The nonce must therefore be brute-forced, but only requires one hash to verify.

\section{Blockchain Details}
\subsection{Transactions}
A transaction contains the following data:
\begin{itemize}
 \item A list of input transactions
 \item A list of tuples of the recipient public key, and the amount to send.
 \item Personal signature, signed with private key
\end{itemize}

\noindent
In order to verify a transaction, one must:

\begin{enumerate}
 \item Verify the signature using the public key of the sender
 \item Verify the signatures of each of the input transactions
 \item Ensure that the money has not been spent between the input transactions and the new transaction.
\end{enumerate}

Since the transaction is signed using the private key, only the owner of the identity can transfer the money from that point.
A single user can have an arbitrary amount of identities.

\subsection{User Miners}
A miner does the following:
\begin{enumerate}
 \item Verify all the transactions by looking that input transactions are covered and properly signed
 \item Compute the Merkle root hash for the transactions
 \item Solve the puzzle on the previous block, for immutability
 \item Broadcast the new header
 \item Go on collecting new transactions for next block
\end{enumerate}

Miners receive compensation for computing the next block. However, it is estimated that we waste \$15 million / day on energy to power the miners.

\subsection{Smart Contracts}
\paragraph{Definition} Computer protocols that facilitate, verify or enforce the negotiation or performance of a contract, or that make a contractual clause unnecessary. Define the rules and penalties around an agreement in the same way that a traditional contract does, but also automatically enforce those obligations, i.e. code is law.

\paragraph{Ethereum} Contracts are the main building blocks of Ethereum, the second most popular blockchain.
The contract is a program which lives inside the distributed Ethereum network and has its own ether balance of Ether\footnote{Ether is the cryptocurrency / cryptofuel of Ethereum}, memory and code.
Every time you send a transaction to a contract, it executes its code, which can store data, send transactions and interact with other contracts.
In order to run the contract, you create a transaction of Ether to the contract, optionally with some input information.
The contract runs until it completes or runs out of Ether.
The Ether paid is awarded to the winning miner.
Each miner runs the smart contract, and produces the same output.
The winning miner will publish the block to the rest of the network.
Other miners validate that they get the same result.
The contracts may be compiled online using the Solidity Compiler\footnote{https://remix.ethereum.org/}.

\section{Summary}
When to use blockchains:
\begin{enumerate}
 \item Are there multiple parties in the ecosystem?
 \item Is establishing trust between all the parties an issue?
 \item Is it critical to have a tamper-proof and permanent record of transactions?
 \item Are we securing the ownership or management of a finite resource?
 \item Does this ecosystem benefit from improved transparency?
\end{enumerate}

\end{document}