\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{indentfirst}
\usepackage{amssymb}
\usepackage{fancyhdr}
\usepackage[margin=1in]{geometry}
\title{LX331: Assignment 4}
\author{Duy Nguyen}
\date{3 March 2017}

\pagestyle{fancy}
\fancyhf{}
\rhead{Nguyen \thepage}

\begin{document}
\maketitle

\section{Exploring semantic ambiguity with propositional logic}

So in (3a) we have the ambiguous sentence: \textit{I didn’t go to Phonetics \textbf{or} Syntax 1 today}.

In this sentence, let  $p =$ I went to Phonetics today and $q =$ I went to Syntax 1 today, then we can write our interpretations of the sentence in propositional logic as follow:

(3b) $\sim (p \lor q)$

(3c) $\sim (p \oplus q)$
\begin{center}
\begin{tabular}{cc|cc}
    $p$ & $q$ &  $\sim (p \lor q)$ & $\sim (p \oplus q)$ \\ \hline
    T & T & F & T \\
    T & F & F & F \\
    F & T & F & F \\
    \textbf{F} & \textbf{F} & \textbf{T} & \textbf{T} \\
\end{tabular}
\end{center}

Note here that we can see (3b) entails (3c) as there is no row that $\sim (p \lor q)$ is true and $\sim (p \oplus q)$ is false. The entailment relationship between (3b) and (3c) show that there is not really two different meaning of \textit{or}.

\section{The conversational implicatures of \textit{or}-sentences}

With $p$ = You have small children, $q$ = You need special assistance, and $r$ = You may board the flight early; we have:

(1) If you have small children, or you need special assistance, then you may board the flight early.

$ p \lor q \rightarrow r$

(7) If you have small children, and you need special assistance, then you may board the flight early.

$ p \& q \rightarrow r$
\begin{center}
\begin{tabular}{ccc|cc|cc}
    $p$     & $q$     & $r$   &  $p \lor q$ &  $(p \lor q) \rightarrow r$          & $p \& q$  & $(p \& q) \rightarrow r$               \\ \hline
    T       & T       & T     & T           & \textbf{T}                                     & T         & \textbf{T}                    \\
    T       & T       & F     & T           & F                                     & T         & F                    \\
    T       & F       & T     & T           & \textbf{T}                                     & F         & \textbf{T}                    \\
    T       & F       & F     & T           & F                                     & F         & T                    \\
    F       & T       & T     & T           & \textbf{T}                                     & F         & \textbf{T}                    \\
    F       & T       & F     & T           & F                                     & F         & T                    \\
    F       & F       & T     & F           & \textbf{T}                                     & F         & \textbf{T}                    \\
    F       & F       & F     & F           & \textbf{T}                                     & F         & \textbf{T}                    \\
\end{tabular}
\end{center}

From the truth table, we can see that (1) asymmetrically entails (7). This make sense, as the category that gets to board the flight early are: a. have small children; b. need special assistance; and c. both have small children and need special assistance. Thus the meaning of and both is include in (1).

However, the construction like (1) and (7) will not work for the exclusive or meaning. That is an utterance of (4) will not entails (6). From these two relationships, the listener can infer the if the \textit{or} carries an exclusive implication by the fact if that version implies an \textit{and} utterance.

\section{Presuppositions vs. Entailments}

\subsection*{(8)} From an utterance of (8a), we can easily see that it contains more information than both (8b) and (8c). We can see that through redundancy test where:

\# The woman who murdered Arturo was arrested. In fact, she was arrested.

\# The woman who murdered Arturo was arrested. In fact, she murdered Arturo.

Therefore, sentence \textbf{(8a) entails both (8b) and (8c)}.

To test for presupposition, we construct the s-family of (8a).
\begin{center}
\begin{tabular}{r|l}
    $S$ & The woman who murdered Arturo was arrested. \\
    $\sim S$ & The woman who murdered Arturo was not arrested \\
    $S?$ & Was the woman, who murdered Arturo, arrested? \\
    $if-S$ & If the woman, who murdered Arturo, arrested, \\
    & the victim's lawyer will press charges. \\
    $maybe-S$ & Perhaps the woman, who murdered Arturo, was arrested.\\
\end{tabular}
\end{center}
At this point, we can see that through looking at the s-family, we can't not be sure if the woman was arrested, but we can say with stronger belief that Arturo was murdered, possibly by a woman. Therefore \textbf{(8a) presupposes (8c) and does not presupposes (8b)}

\subsection*{(9)} From an utterance of (9a), the listener will know two pieces of information. The woman was arrested and Arturo was murdered. We again uses the redundancy test here:

\#The woman who was arrested murdered Arturo. In fact, she was arrested.

\#The woman who was arrested murdered Arturo. In fact, she murdered Arturo,

Both show redundancy thus, we can say that \textbf{(9a) entails (9b) and (9c)}.

Let's construct the s-family for (9a).
\begin{center}
\begin{tabular}{r|l}
    $S$ & The woman who was arrested murdered Arturo. \\
    $\sim S$ & The woman who was arrested didn't murdered Arturo \\
    $S?$ & Did the woman, who was arrested, murdered Arturo? \\
    $if-S$ & If the woman, who was arrested, murder Arturo, \\
    & then how did she escaped prison in the first place? \\
    $maybe-S$ & Perhaps the woman, who was arrested, murdered Arturo.\\
\end{tabular}
\end{center}
In contrast with (8), here we are not sure of the fact that Arturo being murder. However, one thing stay the same is that the woman was arrested. Thus we can see that \textbf{(9a) presupposes (9b) but not (9c)}.

From here we could also draw a general observation that the information contained in the relative clause remains and could serve as a trigger for presupposition.

\subsection*{(10)} The utterance of (10a) tells us two pieces of information: John's baldness and his children's baldness. Let's try non-denialability test on the (10b) and (10c).

\# John is bald, and John's children are bald too. But he doesn't have any children.

\# John is bald, and John's children are bald too. But it's not true that a member of his family his bald.

Both sentences show contradictions, thus we can say that \textbf{(10a) entails (10b) and (10c)}.

Let's construct the s-family for (10a).
\begin{center}
\begin{tabular}{r|l}
    $S$         & John is bald, and John's children are bald too.       \\
    $\sim S$    & It's not true that John is bald, and his children are also bald.    \\
    $S?$        & Was it true that John is bald and his children are also bald?     \\
    $if-S$      & If it's true that John is bald and his children are also bald,      \\
                & then it's probably genetic.     \\
    $maybe-S$   & Maybe John is bald and his children are also bald.\\
\end{tabular}
\end{center}
Here, we are not to sure on the fact that John or his children are bald but we can tell that he has children. Thus we can say that \textbf{(10a) presuppose (10b) but not (10c)}.

\subsection*{(11)} Similar to (9) and (8), we now switches some elements of the utterance around. We can tell the sentences still entail each other since we can reuse our non-deniability test.

\# John has children, and John’s children are bald. But he doesn't have any children.

\# John has children, and John’s children are bald. But it's not true that a member of his family his bald.

Therefore \textbf{(11a) entails (11b) and (11c)}.

Before we move on to constructing the s-family, we should note that in (11), we don't know whether John is bald.
\begin{center}
\begin{tabular}{r|l}
    $S$         & John has children, and John's children are bald.       \\
    $\sim S$    & It's not true that John has children, and his children are bald.    \\
    $S?$        & Was it true that John has children and his children are bald?     \\
    $if-S$      & If it's true that John has children and his children are bald,      \\
                & then it's still probably genetic.     \\
    $maybe-S$   & Maybe John has children and his children are bald.\\
\end{tabular}
\end{center}
It's interesting here. Like other sentences, we see that one of the fact of the sentence become ambiguous: Whether John has children. However, in this case, $\sim S$ in the s-family is semantically unsound: If John doesn't have children then who are bald?

I think in this case, if the utterance doesn't presuppose (11b) \textit{John has children} then it could not presuppose (11c) \textit{John's children are bald}. I think this is because (11c) depends on (11b) as (11c) entails (11b). Upon hearing \textit{John's children ...} the speaker will automatically understand (11b) \textit{John has children}.

Therefore, I think \textbf{(11a) does not presuppose (11b) or (11c)}.

\end{document}


% !TeX root = ../main.tex
\section{The Integration of Functions into Logic Programming}
In this section we give a simple implementation of the amalgamation of functional and logic programming languages, for a more detailed and extended discussion the reader can see \cite{HANUS1994583}. The traditional approach to study a logic programming  is to define an \textit{operational semantics} of an abstract interpreter for the language. This interpreter is an idealised machine that performs the most simple and (sometimes) atomic\footnote{Example of atomic computation step of an idealised machine is the motion of the \textit{reading head} of the Turing Machine, defined by Alan Turing.} operations called \textit{computation}. The definition of computation is the core of the language because it says how one can reason about the behaviour, power and limitation of the machine itself. It is by a mathematical definition that we can predict properties of machines and study the \textit{soundness} and \textit{completeness} of this idealised interpreter.

The interest in the fusion of logic programming and functional programming language have the advantages from both the functional and logic point of view. In comparison with pure functional languages, functional logic languages have more expressive power due to the availability of features like function inversion, partial data structures, and logical variables. In comparison with pure logical languages, functional logic languages have a more efficient operational behaviour since functions allow a more deterministic evaluation principles than pure logic predicates. These two principal consideration are the main initial motivation for integrating these two types of languages.

The integration of functions into logic programming is very simple from a syntactic point of view. For this purpose, we have to extend a logic programming by the addition:
\begin{enumerate}
	\item A method to define new functions.
	\item A possibility to use these functions inside program clauses.
\end{enumerate}

In pure Prolog systems the only \textit{equality predicate} was the built-in syntactic equality. It is by adding the support for terms to be equal \textit{modulo} some equality predicate without being syntactically equal that one can define functional logic programs. For example, the following equational program (equivalent to the TRS defined in the Example \ref{example:app-concat-standard-narrowing})

\begin{align*}
    \appFunc{\nilList}{x} &= x. \\
    \appFunc{x \cdot y}{z} &= x \cdot \appFunc{y}{z}.
\end{align*}
Using this clauses for equality, we can prove that the term $\appFunc{[1,2]}{[3]}$ is equal to $[1,2,3]$.

In order to give the precise semantics of functional logic programs and to fix some basic notions of logic programming we give some definitions below, for an introduction to logic programming one is refereed to the classic book \cite{Clocksin:1984:PP:2343}.

\begin{definition}
    Let $\mathcal{P}$ be a set of \textit{predicate symbols} including the binary equality predicate $=$. A literal $p(t_1, \dots, t_n)$ consists of an n-ary predicate symbol applied to $n$ argument terms. An logical equation is a literal with $=$ as predicate symbol.
\end{definition}

\begin{definition}
    A clause has the form
    $$L_0 \leftarrow L_1, \dots, L_n$$
    $(n \geq 0)$, where $L_0, L_1, \dots, L_n$ are literals. It is called (conditional) equation if $L_0$ is an equation, and unconditional equation if $L_0$ is an equation and $n = 0$.
\end{definition}

\begin{definition}
    An \textit{program objective} $G$ is a logic clause of the form:
    $$\leftarrow B_1, \dots, B_m$$
\end{definition}

A clause $C$ is a \textit{variant} of another cause $D$ if it is an instance of $D$ by some variable renaming substitution $\theta$, i.e. $C = D \theta$.

\begin{definition}
    A \textit{functional logic program} $P$ is a finite set of clauses.
\end{definition}

Note that if we have to evaluate a function applied to ground terms during a unification in a functional logic program, we can simply evaluate this function call as in functional languages by applying appropriate rules to this call. This is called a rewrite step. But, if in the objective one have free variables, one needs to instantiate these variables to some terms and then operates on a rewrite step. This is called \textit{narrowing}, as we have seen in this report early.

That introduce the needs of narrowing-based strategy on evaluation of functions defined inside a logic programming language. We already have established the conditions for this strategy be indeed an terminating procedure for the equational theory defined by this equational classes (in Proposition \ref{proposition:narrowing-as-E-unification}).

The computation in pure Prolog languages is by the means of $SLD$-resolution. This is a complete and sound operational semantic for pure Prolog systems. This evaluation rule use standard unification for solving the resolution of objectives with free variables. In order to extend logic programs we need to be able to compute reasoning with equations. This is done by incorporating narrowing-based unification procedure in the resolution step.

We now see one example of this type of integration. (From the survey of Michael Hanus, see \cite{HANUS1994583}).

\subsection{Constructor-Based Programs}
In order to implement a functional logic language based on basic narrowing we have to manage the set of basic positions in program clauses and we try to apply all rules at all basic positions in each step. That yields a highly non-deterministic execution principle. On the other hand, pure functional languages deterministically select the position where the rules are applied next (innermost positions for \textit{eager evaluation} languages and outermost positions for \textit{lazy evaluation} languages). An approach to achieve similar strategy for functional logic languages is the partition of the signature of the program into a disjoint set $\constructors$ of constructors and a set $\defFunctions$ of defined functions.

Constructors are used to build data types, whereas defined functions operate on these data types. Constructors terms are always in normal form, whereas defined functions are defined by equational rules. This enable us to use narrowing techniques on evaluation strategy. We call a term \textit{innermost} if it has the form $f(t_1, \dots, t_n)$, with $f \in \defFunctions$ and $t_1, \dots, t_n$ are terms built with $\var$ and $\constructors$. A functional logic program is \textit{constructor-based} if the left-hand side of each rule is an innermost term.

In constructor-based functional logic programs, we can solve equations by innermost narrowing strategy, as we already have established early in the report. To define a evaluation principle for this kind of logic functional programs, one needs to take care of some issues. For instance, innermost evaluation only give raise to an incomplete semantics as the example below shall demonstrate.
\begin{example}
    Consider the following rules where $a$ is a constructor:
    \begin{align*}
        f(x) &= a. \\
        g(a) &= a.
    \end{align*}
    Since $f$ is a constant function mapping all inputs to $a$, the identity substitution $[]$ is a solution of the equation $f(g(x)) = a$. However, the only innermost narrowing derivation is
    $$f(g(x)) = a \narrow_{[x/a]} f(a) = a \narrow_{[]} a = a \narrow \top$$
    i.e. innermost narrowing computes only the more specific solution $[x / a]$.
\end{example}

To the operational semantics of constructor-based programs be complete, one need to impose several conditions in the forms of functions defined by clauses. The most important one is: innermost narrowing evaluation is complete if all functions are totally defined, i.e. the only irreducible ground terms are constructors terms. The next example (\cite{HANUS1994583}), shows the incompleteness of innermost narrowing in the presence of partial functions.

\begin{example}
    Consider the following rules, where $a$ and $b$ are constructors:
    \begin{align*}
        f(a, Z) &= a. \\
        g(b) &= b.
    \end{align*}
    If we want to solve the equation $f(X, g(X)) = a$, then there is the successful narrowing derivation
    $$f(X, g(X)) = a \narrow_{[X / a]} a = a$$
    by applying the first rule to the term $f(X, g(X))$. However, this derivation is not innermost, and the only innermost narrowing derivation is not successful:
    $$f(X,g(X)) = a \narrow_{[X/b]} f(b,b) = a$$
    Therefore, innermost narrowing cannot compute the solution.
\end{example}

\begin{theorem}[\cite{HANUS1994583}]
    If $E$ is a finite set of constructor-based unconditional equations such that the rewrite relation $\contr_E$ is convergent and all function are totally defined, then innermost narrowing is complete w.r.t ground substitutions.
\end{theorem}

For a precise description of this strategy (under the conditions of the above theorem) we represent a equational literal in a goal by a skeleton and an environment part. The skeleton is an equation composed of terms occurring in the original program, and the environment is a substitution which as to be applied to the equation in order to obtain  the actual literal.

\begin{definition}
     The initial equation $E$ is represented by the pair $\pair{E}{[]}$. If $\pair{E}{\sigma}$ is a literal, then a derivation step in the innermost narrowing calculus is one the following two forms. Let $p$ be an innermost position:
     \begin{enumerate}
         \item \textit{Narrowing:} Let $l = r$ be a variant of a rule such that $\restr{E}{p}\sigma$ and $l$ are unifiable with $mgu$ $\tau$. Then $\pair{E[r]_p}{\sigma \tau}$ is the next literal derived by an innermost basic narrowing step.
         \item \textit{Innermost Reflection:} Let $\gamma$ be the substitution $[x / \restr{E}{p}\sigma]$, with the condition that $x$ is a fresh variable. Then $\pair{E[x]_p}{\sigma \gamma}$ is the next literal derived by an innermost reflection step.
     \end{enumerate}
\end{definition}


\graphicspath{{sliding\_puzzle/fig}}
\chapter{Solvability of a NxN sliding puzzle}
\label{chap:Solvability of a NxN sliding puzzle}

{\color{red} \Huge{info in this chapter to still be integrated somewhere else into report if needed. Else will leave it out}}

\section{General puzzle description}
Let us assume that we have an NxN puzzle, then we have NxN number of blocks. We can represent the puzzle as an NxN array, then we stack the array into a one dimensional array of 1 x (N*N). For example see the 4x4 puzzle in Figure \ref{fig:sliding_puzzle} we have a 1 x 16 array as:
Array = (12,7,8,13,4,9,2,11,3,6,15,14,5,1,10).
Before we describe the conditions for a sliding puzzle to be solvable, we first define the term “inversion”. Assuming the the first index of the 1xN 2 array starts at the left top corner (valued 12) in
Figure \ref{fig:sliding_puzzle}, and that it runs from [0,(N*N)-1]. Then an inversion occurs when Array[index] >
Array[index+1] where index is an arbitrary integer between 0 and N*N-1. Hence in Figure \ref{fig:sliding_puzzle} we have a
total: sum of inversions(Array) = 11 + 6 + 6 + 8 + 3 + 5 + 1 + 5 + 1 + 2 + 4 + 3 + 1 + 0 = 56.

\begin{figure}[!htb]
	\centering
	\includegraphics[width=0.25\linewidth]{sliding_puzzle/fig/puzzle.png}
	\caption{Example of a sliding puzzle}
	\label{fig:sliding_puzzle}
\end{figure}

\section{Conditions for solvability}
Even and odd sized boards are analysed separately (where size = N).

For odd sized boards where N is odd we have the puzzle only being solvable if and only if the boards
has an even number of inversions. The proof for this can be deduced by looking at Figure 2 and noting that for every switch of the blank block we have an even change in the sum of inversions of the board. \cite{princeton_8puzzle_assignment}

\begin{figure}[!htb]
	\centering
	\includegraphics[width=1\linewidth]{sliding_puzzle/fig/princeton_odd_boards.png}
	\caption{Odd boards with change in blank piece only having even inversion change \cite{princeton_8puzzle_assignment}}
	\label{fig:sol_odd_board}
\end{figure}

For even sized boards where N is even we have the board solvable if and only if the number of
inversions plus the row of the blank square is odd. This is illustrated in Figure 3.

\begin{figure}[!htb]
	\centering
	\includegraphics[width=1\linewidth]{sliding_puzzle/fig/princeton_even_board_solvability.png}
	\caption{Even board solvability \cite{princeton_8puzzle_assignment}}
	\label{fig:sol_even_board}
\end{figure}

Half of all puzzle configurations are unsolvable. \cite{Notes_15_puzzle} This means that we only have N! / 2 configurations
that are solvable for an NxN board. This was proven using parity in the paper in \cite{Notes_15_puzzle}. Sliding puzzles
can be solved relatively quickly with today’s processing of computers for puzzles for example an 5x5
puzzle was solved in 205 tile moves in 2016. \cite{Domain_cube_forum}

The issue more so lies in finding the shortest path to solving a puzzle. This specific problem of solving
with the least amount of tile moves of a sliding puzzle has been defined as NP (non-deterministic polynomial-time) hard. NP hardness is are problems that are as least as hard as NP.
Where in computational complexity theory NP (non-deterministic polynomial-time) is a has a solution
with a proof variable to be in polynomial time by a deterministic Turing Machine. A Turing machine is
a mathematical model defining an abstract machine which manipulates symbols according to a set of
rules. \cite{Computation_finite_and_infinite_machines}

In simpler terms a problem is NP if it can be solved within a time that is a polynomial function of the
input. For instance if we define the time to solve a problem as ‘T’ and the input data as ‘D’. Then as
long as T = polynomial function (D) then a problem is NP.


\subsection{Introduction}

Take physical disk drives, create logical disk drives.

\subsection{Striping}

A single file is spread over multiple disks.

Can be done at bit/byte/block level.

\subsection{Mirroring}

Same data on multiple drives

\subsection{Parity}

Used for error detection.

In protocol for sending/saving we say that all bits must be even (eg 1100) (or odd)

For given bit of info, we add parity bit to guarantee that bit is indeed even/odd. 1100 becomes 11000; 1000 becomes 10000

Cannot correct errors, just detect them. only detects if odd number of errors.

Parity can be stored on dedicate disk, distributed.

alternative to parity bit: hamming code

\subsection{RAID levels}

RAID 0: Uses striping across disks, but no redunency. allows for improved read/write times. if any drive fails, all fail
RAID 1: Data written identically to two drives. read times increased, as with raid 0, due to mirroring. writing is slower. no parity or striping
RAID 2: bit level striping and hamming code. rarely used
RAID 3: rarely used. byte level stripping. Dedicated parity disk
RAID 4: dedicated parity disk
RAID 5: block level striping, distributed parity
RIAD 6: double distributed parity

\subsection{ZFS}

RAID-Z: Under ZFC, similar to RAID 5

\subsection{Off-site backups}


\def\Relength{{\rm Relength}}
\def\length{{\rm length}}
\def\Arccosh{{\rm Arccosh}}
\def\trace{{\rm trace}}
\def\distance{{\rm distance}}

\vglue-8pt
\section{Killerwords and the parameter space}
\vglue-4pt

{\it Notation and conventions} FIXME(1.1).
A hyperbolic $3$-manifold is a Riemannian $3$-manifold of constant sectional curvature $-1$.  All hyperbolic $3$-manifolds under
consideration will be closed and orientable. We will work in the upper-half-space model for
hyperbolic 3-space:  ${\bf H}^3 = \{(x,y,z): z > 0\}$ with
metric ${\rm ds_H} = {\rm ds_E}/z.$ The distance between two points $w$ and $v$ in ${\bf H}^3$ will be denoted $\rho(w,v).$

It is well known that
${\rm Isom}_+({\bf H}^3) = {\rm PSL(2,}{\bf C}),$
where an element of
${\rm PSL(2,}{\bf C})$ acts as a M\"obius transformation on the bounding (extended) complex plane and the extension to upper-half-space is the natural extension
 (see [Bea]).
If $M$ is a hyperbolic $3$-manifold, then $M={\bf H}^3/\Gamma$ where $\Gamma$ is a
discrete, torsion-free subgroup of ${\rm PSL(2,}{\bf C})$.

For computational convenience, we will often normalize so that the (positive) $z$-axis is the axis of an isometry.  As such, we set up some special notation.
Let $B_{(0;\infty)}$ denote the oriented geodesic $\{(0,0,z): 0< z < \infty \}$,
with negative endpoint $(0,0,0).$  (An {\it endpoint} of an axis refers to a limit point of the axis on $S^2_{\infty}$.) Let $B_{(-1;1)}$ denote the
oriented geodesic with negative endpoint $(-1,0,0)$ and positive endpoint $(1,0,0)$.

When working in a group $G$ generated by $f$ and $w$ and looking
at words in $f,w, f^{-1}, w^{-1}$ we will often let $F$ and $W$ denote
$f^{-1}$ and $w^{-1},$ respectively.

\advance\theoremcount by 1
 \numbereddemo{Definition}
 If $f$ is an isometry, then we define
$${\it Relength}(f)=\inf\{\rho(w,f(w))\mid w\in {\bf H}^3\}.$$
Thus $\Relength(f)=0$ if and only if $f$ is either a parabolic or elliptic
isometry.  If $\Relength(f) > 0,$ then $f$
is hyperbolic and maps a unique geodesic $\sigma$ in ${\bf H}^3$ to itself. In that case
$\sigma$ is oriented (the negative end
being the repelling fixed point on $S^2_\infty$)  and the isometry $f$ is the
composition of a  rotation of $t \ ({\rm mod}\ 2\pi)$
radians along $\sigma$ (the sign of the angle of rotation is determined by the right-hand rule) followed by a pure translation of ${\bf H}^3$ along $\sigma$ of
$l = \Relength(f)$.  We define
${\it length}(f)=l+it,$ and call $A_f = \sigma$ the {\it axis} of $f.$  Now,  $A_f$ is an oriented interval with endpoints in $S^2_{\infty},$ the
orientation being induced from $\sigma.$

If the geodesic $\sigma$ is  given a fixed orientation, we define an $l+it$
translation $f$ along $\sigma$  to be a distance $l$ translation in the positive
direction, followed by a rotation of $\sigma$ by $t$ radians.  Of course if
$l < 0,$ then each point of $\sigma$ gets moved $-l$ in the negative direction.
Also, via the right-hand rule, the orientation determines what is meant
by a $t$-radian rotation.  Thus if $l > 0,$ the orientation induced on
$\sigma$ by $f$ (as in the previous paragraph) equals the given orientation.
If $l < 0,$ then the induced orientation is opposite to the given orientation
and $f$ is a $-(l+it)$ translation of $-\sigma$ in the sense of the previous
paragraph.

If $f$ is elliptic, then $f$ is a rotation of $t$ radians where
$0 \le t \le \pi$ about some oriented geodesic, and we define $\length(f) = t i.$
If $f$ is parabolic or the identity, we define $\length(f) = 0 + i0.$
So, for all isometries we have that $\Relength = {\rm Re}(\length).$
\enddemo

\numbereddemo{Definition}   If
$G$ is a subgroup of ${\rm Isom}_+({\bf H}^3),$ then  we say that $f$ is a {\it shortest} element  in $G$ if $f\neq {\rm id}$ and
$\Relength(f)\le \Relength(g)$
for all $g\in G, \ g\neq {\rm id}.$
\enddemo

\numbereddemo{Definition}
  If $\sigma,\ \tau$ are disjoint oriented geodesics  in
${\bf H}^3$ which do not meet at infinity,
then define
${\it distance}(\sigma,\tau)=\length(w)$ where
$w\in {\rm Isom}_+({\bf H}^3)$ is the hyperbolic
element which translates ${\bf H}^3$
along the unique common perpendicular between $\sigma$ and $\tau$ and which takes the
oriented geodesic $\sigma$ to the
oriented geodesic $\tau$.  The oriented common perpendicular from $\sigma$ to $\tau$ is
called the {\it orthocurve} between $\sigma$
and $\tau$.  The {\it ortholine} between $\sigma$ and $\tau$ is the complete oriented
geodesic in ${\bf H}^3$ which contains the
orthocurve between $\sigma$ and~$\tau$.

If $\sigma$ and $\tau$ intersect at one point in ${\bf H}^3$ then
there is an elliptic isometry $w$ taking $\sigma$ to $\tau$ fixing $\sigma \cap \tau.$  Again,  define $\distance(\sigma, \tau) = \length(w).$  In this case, the orthocurve is the point $\sigma \cap \tau,$ and the ortholine $O$ from $\sigma$ to $\tau$ is oriented
so that $\sigma,\ \tau,\ O$ form a right-handed frame.

If $\sigma$ and $\tau$ intersect at infinity, then there is no unique common perpendicular, hence no ortholine, and we define
$\distance(\sigma,\tau) = 0 + i0,$ or $0 + i\pi$ depending on whether or not $\sigma$ and $\tau$ point in the same direction at their intersection point(s) at infinity.

Define ${\it Redistance} = {\rm Re} (\distance).$

As defined, Redistance is nonnegative.
In Definition FIXME(1.8) and in Sections FIXME(2 and 3),
it will be useful to have a broader definition. Given an oriented
geodesic $\alpha$ in ${\bf H}^3$ orthogonal to oriented geodesics $\beta$ and $\gamma$, define $d_\alpha(\beta,\gamma) \in {\bf C}$ where a
$d_\alpha(\beta,\gamma)$ translation of ${\bf H}^3$  along $\alpha$ takes $\beta$ to $\gamma.$
\enddemo

\numbereddemo{Definition}
A tube of radius $r$ about a geodesic $\delta_0$ in ${\bf H}^3$ is
$\{w \in {\bf H}^3 \mid \rho(w, v) \le r$ for some $v \in \delta_0 \}.$
If $\delta$
is a simple closed geodesic in the hyperbolic $3$-manifold $N$ and if
$\{\delta_i \}$ is the set of pre-images of $\delta$ in ${\bf H}^3$, then define
{\it tuberadius}($\delta)={1 \over 2} \min
\{{\rm Redistance}(\delta_i,\delta_j) \mid i\neq j \}.$
If $r = {\rm tuberadius}(\delta),$
then define a {\it maximal tube} about $\delta$ to be the
image of a tube of radius $r$ about $\delta_0.$  Note that
tuberadius$(\delta)=
\sup\{r \mid $ there exists an embedded tubular neighborhood of
radius $r$ about $\delta \}.$
\enddemo

\numbereddemo{Definition}
Our desire to understand tuberadii about closed geodesics, and especially about a simple closed geodesic $\delta$, leads us to investigate certain 2-generator subgroups $G =
\langle f,w\rangle$  of ${\rm Isom}_+({\bf H}^3)$ with the generator $f$ corresponding to a primitive isometry fixing $\delta_0$ and the generator
$w$ corresponding to an element taking $\delta_0$ to its nearest covering translate.  We investigate these 2-generator groups by using certain
subsets of ${\bf C}^3$ as parameter spaces.

A {\it marked {\rm (2-}generator\/{\rm )} group} is a triple $\{G,f,w\}$ consisting of a 2-generator subgroup $G$ of ${\rm Isom}_+({\bf H}^3)$
and an ordered pair of isometries $f,w$ of ${\bf H}^3$ which generate $G$  such that $\Relength(f) > 0$ and if $A_f$ is the axis of $f,$ then $w(A_f)
\cap A_f  = \emptyset$  (here, intersection is taken in ${\bf H}^3 \cup S^2_\infty$). Two marked groups $\{G_1,f_1,w_1\}$ and $\{G_2,f_2,w_2\}$
are {\it conjugate} if $G_1$ and $G_2$ are conjugate via an element of ${\rm Isom}_+({\bf H}^3)$ and this conjugating element takes $f_1$ to $f_2$
and
$w_1$ to $w_2.$  Within any conjugacy class of marked groups is a unique normalized element
$\{G,f,w\}$ where $f$ is a
positive translation  along the (oriented) geodesic $B_{(0;\infty)},$  and the
orthocurve from $w^{-1}(B_{(0;\infty)})$ to $B_{(0;\infty)}$ lies on
$ B_{(-1;1)}$
on the negative side of $ B_{(-1;1)}\cap B_{(0;\infty)}.$
To minimize notation, we will frequently equate a conjugacy class with its normal representative.
\enddemo

Given $(L,D,R)=(l+it, d+ib, r+ia)\in{\bf C}^3$ with $l > 0,\ d > 0,$ one  can associate a group $G$ generated by  elements $f$ and $w$ as follows.  Define $f$ to be an $l+it$ translation along $B_{(0;\infty)}$ and $w$ to be a $d+ib$ translation along $ B_{(-1;1)}$ followed by an $r+ia$ translation along $B_{(0;\infty)}$ (here, $r$ can be negative, in which case this is equivalent to
a
$-r-ia$ translation along $-B_{(0;\infty)}$).  Conversely if $\{G,f,w\}$ is a normalized marked group then $f$ is an $L$ translation of $B_{(0;\infty)}$
and $w$ is a $D$ translation of
$ B_{(-1;1)}$ followed by an $R$ translation of $B_{(0;\infty)}.$ Thus
${\cal P}^\prime = \{(l+it,d+ib,r+ia)\in {\bf C}^3 | \ l>0, d>0\}$
parametrizes the
set of conjugacy classes of marked groups.  In particular, the parametrization is surjective and locally one-to-one.

We are primarily interested in the set
${\cal T}^{\prime}\subset {\cal P}^{\prime}$
which parametrizes all conjugacy classes of
marked groups
$\{G,f,w\}$ for which $f$ is a shortest element  of $G$ which (positively) translates $B_{(0;\infty)}$ and $w\in G$ takes $ B_{(0;\infty)}$ to a
{\it nearest} translate $w(B_{(0;\infty)})$ such that
$-\Relength(f)/2< {\rm Re}(d_ {B_{(0;\infty)}})$
(ortholine from $w^{-1}( B_{(0;\infty)})$ to $ B_{(0;\infty)}$),
(ortholine from $ B_{(0;\infty)}$ to $w(B_{(0;\infty)}))))
\le \Relength(f)/2.$
See Figure FIXME(1.1).
Note that because $f$ is shortest and
$\Relength(f) > 0,$ it follows that $G$ must be
discrete, torsion-free, and parabolic-free.

\numbereddemo{{R}emark}  ${\cal T}^{\prime}$ consists of those parameters corresponding to marked groups $\{G,f,w\}$ such that $l$ is the real
length of a shortest element of $G,\  d$ is the real distance between $ B_{(0;\infty)}$ and a nearest
translate, and $-l/2<r\le l/2.$
In what follows, it is
essential to remember that an element $\alpha$ of ${\cal P^{\prime}}$ corresponds not only
to a group $G,$ but to a marked group.
To further establish the point, we note that,
for elements of ${\cal T}^{\prime},$
the parameter $l$ is an invariant of $G$ alone (that is, $l$ is the shortest real length of an element of $G$), while  the parameter $d$ is determined by $G$ and $f$ (that is, the notion of ``nearest" used to define $w$ in the definition of ${\cal T}^{\prime}$ requires a choice of $f$).

As mentioned in the introduction to this paper, we are only interested in the subset of ${\cal T}^{\prime}$ corresponding to
parameters $\alpha$ with $d \le \ln(3).$  The
following two propositions imply this subset of ${\cal T}^{\prime}$ lives in a compact subset of ${\cal P}^{\prime}.$
\enddemo

 \figin{fig1.1}{500}
 \centerline{Figure FIXME(1.1)}

\numbereddemo{Definition} \hglue-8pt Let ${\cal P}\subset{\cal P}^{\prime}$ be the set of those parameters $\alpha
= (l+it$, $d+ib, r+ia)$ such
that
 \vglue3pt
 a)       $0.0978\le l \le 1.289785$,
\vglue2pt b)\enspace      $l/2 \le d\le \ln(3)$,
\vglue2pt c) \enspace   $0\le r \le l/2$,
\vglue2pt d) \enspace      $-\pi \le t \le 0$,
\vglue2pt e) \enspace	$-\pi \le b \le \pi$,
\vglue2pt f) \enspace	$-\pi \le a \le \pi$.
\vglue2pt\noindent
Define ${\cal T}={\cal T}^{\prime}\cap {\cal P}.$
\enddemo

The point of the definition of ${\cal P}$ and ${\cal T}$ is as follows.  We want to analyze by computer the relationship between lengths of shortest geodesics  and their tuberadii in hyperbolic $3$-manifolds.  We were naturally led to the parameter space
${\cal P}^{\prime}$ and its subset ${\cal T}^{\prime}.$  But
${\cal P}^{\prime}$ is problematic from the computational viewpoint because it is noncompact.  We wish to replace
${\cal P}^{\prime}$ by ${\cal P}$ which is compact, and
${\cal T}^{\prime}$ by ${\cal T}$ in our computer analysis.  This is carried out in Lemma FIXME(1.13).  Note that we worked to make ${\cal P}$ as small as reasonable to save computation time;  for example, the $t$ and $r$ restrictions above cut down the parameter space by a factor of 4 over the obvious $t$ and $r$ restrictions.

By [G; Lemma 5.9] (or see Example A.3 in the appendix) a closed orientable hyperbolic $3$-manifold $N$
satisfies the insulator condition provided that
tuberadius$(\delta) > \ln(3)/2$ for some closed geodesic $\delta\subset N.$  Thus we
are led to consider:

\numbereddemo{Definition}  A word $h$ in $w,f,w^{-1},f^{-1}$ for which statement a) (resp.\ b)) in Remark FIXME(1.17) holds for each $\beta\in {\cal P}_i$
and for which $h_\beta$ is not a power of $f_\beta$ for each $\beta\in {\cal P}_i$ is
called a {\it killerword} for ${\cal P}_i$ with respect to contradiction a) (resp.\ b)).

\numbereddemo{Summary}  With seven exceptions,  to each of the approximately one
billion regions partitioning ${\cal P},$ we will
associate a killerword and a contradiction.  \enddemo

\numbereddemo{{R}emark}  Computers are well suited for partitioning a set such as ${\cal P}$
into many regions $\{{\cal P}_i \},$ and finding a
killerword $h_i$ which eliminates all $\alpha_i \in {\cal P}_i $ due to contradiction $C_i.$
Depending on the contradiction, we find
computable expressions  for approximations of the values of
$\Relength(h_\beta)$ or Redistance$(h_\beta(B_{(0;\infty)}), B_{(0;\infty)})$ and
thus use the computer to eliminate all of~${\cal P}_i .$
\enddemo

\vglue8pt
{\it Definition} FIXME(1.22). Let
\vglue4pt
\centerline{$ {\cal W} = \{ (x_0,x_1,x_2,x_3,x_4,x_5) : |x_i| \le 4 \times 2^{(5 - i) /6} {\rm \ for\ } i = 0,1,2,3,4,5 \}$}
\begin{eqnarray*}
\noalign{\vskip-12pt}
 \supset \exp({\cal P}) =
\{(x_0,x_1,x_2,x_3,x_4,x_5)\mid \hskip-16pt&\hskip-16pt &
x_0 + i x_3=\exp(e),  \enspace x_1 + i x_4
  = \exp(f),  \\
\hskip-16pt&\hskip-16pt& x_2 + i x_5 = \exp(g) \
{\rm where} (e,f,g)\in {\cal P}\}\\
\noalign{\vskip-28pt}
\end{eqnarray*}
 and let
$${\cal S}=\exp({\cal T}).$$
As we are taking $\exp$ of the various complex co-ordinates, it is notationally convenient to replace our complex parameters
$L = l+it,\ D = d+ib,\ R = r+ia$ by exponentiated versions.  That is, let
\begin{eqnarray*}
L^\prime &\hskip-8pt=\hskip-8pt& \exp(L) = \exp(l+it),\enspace D^\prime = \exp(D) = \exp(d + ib),
\\ R^\prime &\hskip-8pt=\hskip-8pt& \exp(R) = \exp(r+ia).
\end{eqnarray*}

{\it Remarks} FIXME(1.23).
i) We work with ${\cal W}$ instead of $\exp({\cal P})$ because we want our initial parameter space to be a (6-dimensional) box that is easily
subdivided.  This has the side effect that certain regions (sub-boxes)
 ${\cal W}_i$ of ${\cal W}$ will be eliminated because they are outside of $\exp({\cal P})$ not because of the analogues of conditions a) and b) in
Remark FIXME(1.17).
The entire collection of conditions is given in Section FIXME(5).

ii)  The presence of the factor $2^{(5-i)/6}$ in the definition of ${\cal W}$ is explained in Construction FIXME(5.3).
Briefly, the main reason for including it
is to make the shape of regions stay as uniform and ``round" as possible under subdivision.
This makes the Taylor approximations efficient, hence fast.
\vglue4pt
iii)  We chose the co-ordinates of ${\cal W}$  so that $L^\prime = x_0 + i x_3,\ D^\prime = x_1 + i x_4,\ R^\prime = x_2 + i x_5\ $ to gain a mild
computer advantage.
\advance\theoremcount by 3

\proclaim{Lemma}  If $(L^\prime, D^\prime, R^\prime)\in {\cal W}$ and  $\{G,f,w\}$ is the associated normalized
marked group{\rm ,} then $f$ and $w$ have matrix representatives
 $$ f = \left(\matrix{\sqrt{L^\prime}&0\cr
0 & 1/\sqrt{L^\prime}\cr}\right), \leqno{{\rm a)}}$$

  $$ w = \left(\matrix{\sqrt{R^\prime} * ch & \sqrt{R^\prime} * sh \cr sh / \sqrt{R^\prime} & ch/\sqrt{R^\prime}\cr}\right) \leqno{\rm b)}$$
where
$ch = (\sqrt{D^\prime} + 1/\sqrt{D^\prime})/2\ \ {\rm and}\ \
sh = (\sqrt{D^\prime} - 1/\sqrt{D^\prime})/2.$
\endproclaim

\demo{Proof}  a)  In our set-up  the (oriented) axis of $f$ is $B_{(0;\infty)}$.
As such, $f$ corresponds to a diagonal matrix, with diagonal entries $p$ and $p^{-1},$  with $|p| >1.$
The action of $f$ on the bounding complex plane is simply multiplication by $p^2.$  Extending this action to upper-half-space in the natural way rotates the $z$-axis by angle $\arg(p^2)$ and sends $(0,0,1)$ to $(0,0,|p|^2).$
 Thus, $${\rm Im}(\length(f)) = \arg(p^2) = {\rm Im}(\ln(p^2))$$ and,
using the hyperbolic metric,
$${\rm Re}(\length(f)) = \ln(|p|^2) = {\rm Re}(\ln(p^2)).$$
That is, $\length(f) = \ln(p^2)$ and
$$p = \pm \exp(\length(f)/2) = \pm \sqrt{\exp(\length(f))} = \pm \sqrt{\exp(L)} = \pm \sqrt{L^\prime}.$$ Now, we take the positive square
root (taking the negative square root produces the other lift from
${\rm PSL(2,}{\bf C})$ to ${\rm SL(2,}{\bf C})$).

b)  $w = \beta \circ \alpha$ where $\beta$ is translation of distance $R$ along $ B_{(0;\infty)}$ and $\alpha$ is translation of distance $D$ along $ B_{(-1;1)}$.  Thus,
a matrix representative of $\beta$ is $$ \left(\matrix{\sqrt{R^\prime} & 0 \cr 0 & 1/\sqrt{R^\prime}\cr}\right)$$ and a matrix representative of $\alpha$ can be computed to be $$\left(\matrix{\cosh(D/2) & \sinh(D/2) \cr \sinh(D/2) & \cosh(D/2)\cr}\right).$$ But
$\cosh(D/2) = (\exp(D/2) + \exp(-D/2))/2 =
(\sqrt{D^\prime} + 1/\sqrt{D^\prime})/2 = ch$ and similarly for $sh.$
Thus, $$\alpha = \left(\matrix{ch & sh \cr sh & ch\cr}\right)$$ and b) follows by matrix multiplication.
\enddemo

\proclaim{Lemma} If $h \in {\rm Isom}_+({\bf H}^3)$ is represented by the matrix $$A = \left(\matrix{a &  b \cr c & d\cr}\right)\in {\rm SL}(2,{\bf
C}),$$ then
\begin{itemize}
\ritem{a)} $\exp(\Relength (h))= |\trace(A)/2 \pm \sqrt{(\trace(A)/2)^2 - 1}|^2,$

\ritem{b)} $\exp({\rm Redistance} (h(B_{(0;\infty)}), B_{(0;\infty)}))=|{\rm orthotrace}(A) \pm$\hfill \noindent $\sqrt{({\rm
orthotrace}(A))^2 - 1}|$  where
{\rm orthotrace(}$A) = ad + bc.$ $\phantom{\sum^\int}$
\end{itemize}
In both cases{\rm ,} the $+, -$ produce reciprocal values for the right\/{\rm -}\/hand side{\rm ,}
  and we take the one producing the larger value{\rm ,} unless the value is $1${\rm ,} in
which case there is no need to choose.
\endproclaim

\demo{Proof}  a)  If $A$ is elliptic or parabolic, the proof is straightforward (the trace of a parabolic is $\pm 2$ while the trace of an elliptic
is a real number between 2 and -2).

We assume $A$ is hyperbolic.  Because trace is a conjugacy invariant, we can assume the oriented axis of $A$ is $ B_{(0;\infty)}.$  Thus $A$ is a diagonal matrix with $p$ and $p^{-1}$ along the diagonal with
$|p| > 1,$ and, as in the proof of Lemma FIXME(1.24), we see that
$\exp({\rm length}(h)) = p^2.$  Of course,  $\trace(A)= p + p^{-1}$, and it is easy enough to solve for $p.$   Specifically,
$p = \trace(A)/2 \pm \sqrt{(\trace(A)/2)^2 - 1}.$
Thus,
\begin{eqnarray*}
\exp(\Relength (h))& =& |\exp(\length(h))|  =
|p|^2\\[5pt]
&=& |(\trace(A)/2) \pm \sqrt{(\trace(A)/2)^2 - 1}|^2.
\end{eqnarray*}
\vglue4pt
b)  If $ B_{(0;\infty)}$ and $h(B_{(0;\infty)})$ intersect at infinity, then the proof is straightforward.  For example,
if $h$ fixes the point $(0,0,0)$ at infinity, then $c = 0,\ ad = 1$ and the formula holds.  Similarly for the other cases in which $ B_{(0;\infty)}$ and $h(B_{(0;\infty)})$ intersect at infinity.

We assume $ B_{(0;\infty)}$ and $h(B_{(0;\infty)})$ do not intersect at infinity. We will compute the length of $k$, the square of the transformation taking $ B_{(0;\infty)}$ to $h(B_{(0;\infty)})$ along their ortholine.
Let $\tau$ be 180-degree rotation about $ B_{(0;\infty)},$ then $(h \circ \tau \circ h^{-1})$ is 180-degree rotation about $h(B_{(0;\infty)}),$ and
we have that  $k = (h \circ \tau \circ h^{-1}) \circ \tau.$
Now, $\tau$ and $h$ are represented by the matrices
$$ \left(\matrix{ i & 0 \cr
                                0  & -i \cr} \right) {\rm \ \ and\ \ }
                         \left(\matrix{ a & b \cr
                                c  & d \cr} \right)  \in {\rm SL}(2,{\bf C}).$$
Hence,   $k = (h \circ \tau \circ h^{-1}) \circ \tau$ can be computed to have matrix
representation $$\left(\matrix{ ad + bc & 2ab \cr
                                2cd & ad + bc \cr} \right).$$
Thus,
\begin{eqnarray*}
&&\hskip-66pt\exp({\rm Redistance} (h(B_{(0;\infty)}), B_{(0;\infty)}))\\
& = &
\exp(\Relength(k)/2)= \sqrt{|\exp(\length(k)|}\\
& =&
|(\trace(k)/2) \pm \sqrt{(\trace(k)/2)^2 - 1}|\\
& =&
|(ad + bc) \pm \sqrt{(ad + bc)^2 - 1}|. \\
\noalign{\vskip-36pt}
\end{eqnarray*}
\enddemo

\numbereddemo{{R}emarks} i)  It follows from Lemma FIXME(1.25) that if $h$ is a word in $f,w,f^{-1},w^{-1},$ then for any
parameter value $\alpha\in {\cal W},$
$$\exp(\Relength(h_\alpha)), \hbox{ and }
  \exp({\rm Redistance}(h_\alpha(B_{(0;\infty)}), B_{(0;\infty)}))$$ can be computed using only the
operations $+, -, \times, /, \sqrt{}.$
\vglue4pt
 ii) During the course of the computer work needed to prove the main theorems, the parameter space
${\cal W}$ was decomposed into sub-boxes by computer via a recursive subdivision process:
Given a sub-box  being analyzed, either it can be {\it killed directly} (that is, eliminated by a killerword and associated condition as described in
Remark FIXME(1.17),
or for the trivial reason described in Remark FIXME(1.23i)), or it cannot.
If it cannot be killed directly, it is subdivided in half by a hyperplane
$\{x_i = c \}$ (where $i$ runs through the various co-ordinate dimensions
cyclically) and the two pieces are analyzed separately, and so on.

As such, a sub-box of ${\cal W}$ can be described by a sequence of 0's and 1's where 0 means ``take the lesser $x_i$ values" and 1 means ``take the greater $x_i$ values."
For the decomposition of ${\cal W}$ into sub-boxes, all the
sub-box descriptions could be neatly encoded into one tree (although in practice we found it preferable to use several trees to describe the entire
decomposition.  See \S 5).
\vglue4pt
iii) In the following proposition, seven {\it exceptional boxes} are described as sequences of 0's and 1's.
Four of the exceptional boxes---$X_0, X_4, X_5, X_6$---are each the union of two abutting sub-boxes, $X_0 = X_{0a} \cup X_{0b}$ and so on.  It is a pleasant exercise to work through the fact that they abut.
It should be noted that had the set-up for ${\cal W}$ been different, more sub-boxes (or perhaps fewer) might have been needed to construct the seven
exceptional regions.

It is also a pleasant exercise to calculate by hand the co-ordinate ranges of the various sub-boxes.  For example, the range of the last co-ordinate (i.e., $x_5$) of the sub-box
\eject

\noindent
$X_{6a} =
111000000001000111\
111111110101001111\
011111010111111111$\hfill
\vglue4pt
  \hfill $
110001001011000111\ 0$
\vglue4pt\noindent
is found by taking the 6th entry, the 12th entry, the 18th entry, and so on.  These entries are 011111111111.  The first entry (0) means take the
lesser $x_5$ values, and produces the interval $[-4,0].$  The second entry (1) means take the greater $x_5$ values, and produces the interval
$[-2,0].$  The third entry (1) produces $[-1,0].$  Continuing, we see that $X_{6a}^{\phantom{|}}$ has $-2^{-9} \le x_5 = {\rm Im}(R') \le 0.$  The other
co-ordinates can be computed in the same fashion, although they must at the end be multiplied by the factor $2^{(5 - i)/6}$ (see the definition of the
initial box
${\cal W}$).  The range of co-ordinate values for each  exceptional  box $X_0, X_1, \ldots, X_6$ is given in Table FIXME:1.1 (a limited number of significant
digits is given), and  then a range of co-ordinates for exceptional regions (in ${\cal P}$)
${\cal R}_i \supset \exp^{-1}(X_i)$ is given (see Remarks FIXME:1.30i) and 1.30ii)) in Table FIXME:1.2. (Note that this use of the symbol ${\cal R}_i$ differs
slightly from the use in \S 0.)
Finally, two {\it quasi-relators} are given in Proposition FIXME:1.28 for each exceptional box $X_0, X_1, \ldots, X_6$ (see the next definition).

\numbereddemo{Definition} A {\it quasi-relator} in a sub-box $X$ of ${\cal W}$ is a word in $f,w,F=f^{-1},W=w^{-1}$ that is close to the identity
throughout
$X$ and experimentally appears to be converging to the identity at some point in $X.$  In particular, a quasi-relator rigorously has Relength less than
that of $f$ at all points in $X.$
\enddemo

\proclaim{Proposition}
Within the parameter space ${\cal W}$ but outside the seven exceptional boxes there are no parameter points corresponding to
marked groups $\{G,f,w\}$ where $G$ is
discrete{\rm ,} torsion\/{\rm -}\/free and  parabolic\/{\rm -}\/free\/{\rm ; }
$f$ corresponds to a shortest geodesic $\delta$ of tuberadius $\le
\ln(3)/2${\rm ;} and $w$ takes a particular lift of $\delta$ to
a nearest translate.
 Specifically{\rm ,}
${\cal S} \cap ({\cal W} - \bigcup_{n = 0,\dots, 6} X_n)=\emptyset$ where the $X_n$ are the exceptional boxes

\vglue4pt

 \noindent $X_0 = X_{0a} \cup X_{0b},$
\vglue8pt
\noindent $X_{0a} =
001000110111110001\
101001010101011001\
011011010111101101$\hfill

 \hfill
$100001101101000111\
010001110101100101\
1101110111110100,$
\vglue8pt
 \noindent $X_{0b} =
001001110110110000\
101000010100011000\
011010010110101100$

\hfill
$100000101100000110\
010000110100100100\
1101100111100100,$
 \vglue4pt
\noindent  $X_0\ \ $ quasi\/{\rm -}\/relators\/{\rm :}

$r_1 = fwFwwFwfww,$

$r_2 = FwfwfWfwfw,$

\vglue8pt
\noindent $X_1 =
001000110001110110\
011101000110111110\
100010110000100011$\hfill

\hfill $
101101001101001000\
110101011000000100\
000.$

\noindent $X_1\ \
$ quasi\/{\rm -}\/relators\/{\rm :}

$r_1 = FFwFWFWfWFWFwFFww,$

$r_2 = FFwwFwfwfWfwfwFww,$
\vglue4pt
\noindent $X_2 =
001000110101010010\
101010110001100101\
110111100001101010$\hfill

\hfill $111100100000010001\
111100,$
\vglue4pt
\noindent $X_2\ \ $ quasi\/{\rm -}\/relators\/{\rm :}

$r_1 = FwfwfWffWfwfwFww,$

$r_2 = FFwFFwwFwfwfwFww,$
\vglue4pt
\noindent
$X_3 =
111000000001000110\
011011101101011000\
111101011110001100$\hfill

\hfill
$111111100110110000\
0000100010100010,$
\vglue4pt
\noindent $X_3\ \ $ quasi\/{\rm -}\/relators\/{\rm :}\/

$r_1 = FFwfwFFwwFWFwFWfWFWffWFWfWFwFWFww,$

$r_2 = FFwfwFwfWfwfWWfwfWfwFwfwFFwwFWFww,$
\vglue4pt
\noindent $X_4 = X_{4a} \cup X_{4b},$
\vglue4pt
\noindent $X_{4a} =
111000000001000110\
011001001111101010\
011110110110111101$\hfill

\hfill
$100011111110110110\
10000111101,$
\vglue4pt
\noindent $X_{4b} =
111000000001000110\
011001001111101010\
111110010110011101$\hfill

\hfill
$000011011110010110\
00000101101,$
\vglue4pt
\noindent $X_4\ \ $ quasi\/{\rm -}\/relators\/{\rm :}

$r_1 = FFwfwFwfWfwfWfwFwfwFFwwFWFwFWFww,$

$r_2 = FFwfwFwfwFFwwFWFwFWfWFWfWFwFWFww,$
\vglue4pt
\noindent $X_5 = X_{5a} \cup X_{5b},$
\vglue4pt
\noindent $X_{5a} =
001000110001110111\
001111000101111111\
101111100111001111$\hfill

\hfill
$000001111011110111\ 1,$
\vglue4pt
\noindent  $X_{5b} =
001001110000110110\
001110000100111110\
101110100110001110$\hfill

\hfill
$000000111010110110\ 1,$
\vglue4pt
\noindent $X_5\ \ $ quasi\/{\rm -}\/relators\/{\rm :}

$r_1 = FwFWFwFwfwfWfwfw,$

$r_2 = FwfwfWfWFWfWfwfw,$
\vfil
\noindent $X_6 = X_{6a} \cup X_{6b},$
\vfil
\noindent $X_{6a} =
111000000001000111\
111111110101001111\
011111010111111111$\hfill

\hfill
$110001001011000111\ 0,$
\vglue4pt

\noindent $X_{6b} =
111001000000000110\
111110110100001110\
011110010110111110$\hfill

\hfill
$110000001010000110\ 0,$
\vglue4pt
\noindent $X_6\ \ $ quasi\/{\rm -}\/relators\/{\rm :}\/

$r_1 = FWFwFWfWFwFWFwfw,$

$r_2 = FWFwfwFwfWfwFwfw.$
\endproclaim

\demo{Proof}
The proof follows along the lines presented in Remark FIXME(1.17).
Two computer files contain the data needed for the proof.
The first computer file describes the partition of ${\cal W}$ into
sub-boxes and attaches an integer to each such sub-box,
and the second file, called ``conditionlist" is an ordered list of conditions and killerwords.
The integer associated to a sub-box in the first file describes the numbered condition/killerword from   conditionlist that will eliminate the sub-box
in question (other than those corresponding to the $X_i).$ A computer program named {\it verify} shows
that the conditions and killerwords in question actually do kill off their associated sub-boxes (see Section 5 for more details).  This computer
program addresses the issues of Remark FIXME(1.20).  The code for {\it
verify} is available at the {\it Annals} web site.

In addition, a mild modification of {\it verify} showed that the listed words were quasi-relators for the given sub-boxes.
\enddemo


\chapter{Resampling Methods}
\label{chp:crossvalidation}
In this chapter different types of resampling methods are explained. The validation set approach and cross validation is presented as two ways of estimating the test error for a model. Lastly Bootstrap is presented as a method for estimating properties of a model.

\section{The Validation Set Approach}
\label{sec:VSA}
In the validation set approach the available set of samples is randomly divided into two parts of approximately equal size. One is the training set and the other is the validation set. The idea is then to fit the model on the training set, and then apply the model to the validation set. The resulting prediction error of the validation set is then used as an estimate of the test error. This error rate is typically calculated as the mean squared error (MSE).

\myFigure{validationSetApproach.png}{Splitting data randomly}{fig:valid}{0.6}

The concept of randomly splitting the  data set is illustrated in figure \ref{fig:valid}. The  data set is a series of observations from 1 to n, and the result is two almost equally sized parts, with the observations from the original  data set randomly distributed.

The blue half of the  data set is the training set, this is used for fitting the model. The orange half of the data set is the validation set, used for estimating the prediction error rate.

\subsection{Lab 5.3.1 - The Validation Set Approach}
In this exercise the goal is to use the validation set approach. So as described in the section above the first thing to do is to split the data randomly in two halves. The data used here is the \emph{Auto}  data set, containing a list of car information.

\newpage

\begin{lstlisting}[language=Python, label=lst:fit_valid, caption=Fitting linear regression]
train, test = train_test_split(df, test_size = 0.5)

X_train = train[['horsepower']]
X_test = test[['horsepower']]

Y_train = train['mpg']
Y_test = test['mpg']

lm = linear_model.LinearRegression()
lm.fit(X_train, Y_train)
\end{lstlisting}

Listing \ref{lst:fit_valid} shows the \emph{Auto} data as \emph{df} being split randomly in half by the \emph{train\_test\_split()} function.
\emph{horsepower} is used as a predictor and denoted X while \emph{mpg} is the response value denoted Y. The model is then fitted on the training data \emph{X\_train} and \emph{Y\_train}.

\begin{lstlisting}[language=Python, label=lst:print_valid, caption=Printing MSE]
print("Mean squared error: %.2f"
		% np.mean((lm.predict(X_test) - Y_test) ** 2))
\end{lstlisting}

In Listing \ref{lst:print_valid} the MSE is computed by applying the fitted model to the test sets. The error is 24,42.

Because the relationship between \emph{horsepower} and \emph{mpg} is non-linear the model can be improved upon by using higher polynomial features for fitting the model. Using second degree polynomials is shown in Listing \ref{lst:poly2_valid}. Third degree polynomials was also applied in the exercise.

\begin{lstlisting}[language=Python, label=lst:poly2_valid, caption=Polynomial features with degree = 2]
poly = PolynomialFeatures(degree=2)
X_poly_train_deg2 = poly.fit_transform(X_train)
X_poly_test_deg2 = poly.fit_transform(X_test)

lm.fit(X_poly_train_deg2, Y_train)
\end{lstlisting}

Using polynomials shows a clear improvement of the MSE, but not continuously. Using the second degree polynomial shows a drop in the MSE from 24,42 to 19,46, but using the third degree polynomial shows a slight increase from 19,46 to 19,55.

\section{Leave-one-out cross-Validation}
In validation-set approach only half of the data is used for training the model. This might give an overestimate of the mean squared error when training on the complete data set. The MSE might also vary a lot when doing random splits of the  data set  \citep[pp. 54]{AIbook}. Leave-one-out cross-validation tries to address this drawback and is illustrated in figure \ref{fig:loocv}. It splits the  data set in number of observations n and then uses n-1 observations for training and the last observation as validation set. It then iterates through using all observations as validation. This might address the drawbacks of validation set approach but it is also very time consuming as a model has to be fitted n times.

\myFigure{LOOCV.png}{Leave-one-out cross-validation}{fig:loocv}{0.4}

\subsection{Lab 5.3.2 - Leave-One-Out Cross-Validation}
For the exercise for LOOCV, the \emph{Auto} data is read just like in Lab 5.3.1, but not split into two halves of random values. The \emph{horsepower} and \emph{mpg} data parts are though still distributed into respectively \emph{X\_train} and \emph{Y\_train}.

\begin{lstlisting}[language=Python, label=lst:LOOCV, caption=Leave-one-out cross-validation loop]
for i in range(1, 6):
	lm = linear_model.LinearRegression()
	poly = PolynomialFeatures(degree=i)
	X_poly_train = poly.fit_transform(X_train)

	kf = KFold(n_splits=X_train.shape[0])
	test = cross_val_score(lm, X_poly_train, Y_train, cv = kf,
			scoring = "neg_mean_squared_error")

	print(np.mean(-test))
\end{lstlisting}

In Listing \ref{lst:LOOCV} the LOOCV is done. Lines 2-4 are the same functionality as in Listing \ref{lst:poly2_valid}, where polynomials are used for the validation set approach. However, here the degree is iteratively increasing up to a fifth degree polynomial.

The \emph{KFold} function is used for LOOCV to split the data set in as many folds as there are observations, i.e. making 392 folds. Then everything is used as parameters for the \emph{cross\_val\_score} function. It uses the fitted model, the \emph{horsepower} training set fitted to an i'th degree polynomial, the \emph{mpg} training set and the folds to evaluate a score by cross-validation.

The mean squared error is lastly computed from the result of the cross-validation. Once again it is clear that there is improvement when using second degree polynomial, but from there no clear improvement can be seen. Here the MSE is a bit lower than with the validation set approach. This can be caused by the discussed overestimation of the MSE by only training on half the  data set. For first degree polynomial the MSE is 24.23, for second degree polynomial 19.25 and for third degree polynomial the MSE was calculated to 19.33. All of these error rates showed lower than with validation set approach.

\section{K-Fold Cross-Validation}
K-fold cross-validation is similar to LOOCV, however it surpasses LOOCV in some areas. Instead of using only one observation as the validation set, the  data set is split into k roughly equal-sized parts, called \emph{folds}, and one of these folds is the validation set. All other folds are used as training set. Similar to LOOCV the algorithm then iterates through which fold is the validation set. Since there are less iterations to go through the K-fold cross-validation has a computational advantage over the LOOCV, but that is not all. You can also look at what is called the bias-variance trade-off.

The LOOCV gives an approximately unbiased estimate of test error compared to the validation set approach, due to training on all but one observation. However, in an estimating procedure, another important aspect is the procedure's variance, and this is an aspect where the k-fold cross-validation is superior to LOOCV \citep[pp. 304-305]{datamining}. When performing LOOCV, the result is an average of \emph{n} fitted models, and since the training set for each model only vary by one observation, the outputs are highly correlated. In k-fold cross-validation the training sets in each model has a smaller overlap, and since the mean of highly correlated outputs has high variance, the LOOCV's results have higher variance than the K-fold cross-validation's results. So the bias-variance trade-off is associated with k in K-fold cross-validation. The higher the k, the more folds, and thereby higher correlation giving higher variance. In practice, $k=5$ or $k=10$ gives a good result for estimating the test error \citep[pp. 183]{ISLR} \citep[pp. 54]{AIbook}.

\myFigure{K_fold.png}{Representation of K-fold cross-validation}{fig:K_fold}{0.4}

The idea behind k-fold cross-validation is illustrated in Figure \ref{fig:K_fold}, where the blue folds are the training sets and the orange fold is the validation set. Note that the case where k is equal to the number of observations is what was just described as LOOCV. Having a lower value of k therefore provides a less computationally expensive way of doing cross-validation.

\FloatBarrier
\subsection{Lab 5.3.3 - k-Fold Cross-Validation}
The only difference from Lab 5.3.2 to this exercise is that now the number of folds is set by k, and not the amount of observations. In Listing \ref{lst:K_fold} it can be seen that the number of splits is set to 10, as the exercise specifies $k=10$, and the loop now runs 10 times, up until a 10th degree polynomial. The rest is similar to Listing \ref{lst:LOOCV}.

\begin{lstlisting}[language=Python, label=lst:K_fold, caption=K-fold cross-validation loop]
for i in range(1, 11):
	...
	k_fold = KFold(n_splits=10, shuffle=True)
	...
\end{lstlisting}
\FloatBarrier

In table \ref{table:kfold_polynomials_mse} the result of the k-fold cross-validation is shown from a 1 to 10 degree polynomial. The mean squared error is lower than with validation set approach. Compared to LOOCV it has a higher MSE for some polynomials while for others the MSE is lower. As explained this method is computationally less expensive than LOOCV and this was also obvious when executing the implementation.

\begin{table}[htbp]
	\centering
	\begin{subtable}[t]{2in}
		\centering
		\begin{tabular}{ p{2.5cm} p{1.5cm}  }
			\textbf{Degree} & \textbf{MSE} \\
			\hline
			\\
			1 & 24.22 \\\hline
			\\
			2 & 19.19 \\\hline
			\\
			3 & 19.28 \\\hline
			\\
			4 & 19.54  \\\hline
			\\
			5 & 19.00  \\\hline
		\end{tabular}
		\label{table:mse_validation}
	\end{subtable}
	\quad
	\begin{subtable}[t]{2in}
		\centering
		\begin{tabular}{ p{2.5cm} p{1.5cm}  }
			\textbf{Degree} & \textbf{MSE} \\
			\hline
			\\
			6 & 18.85 \\\hline
			\\
			7 & 19.48  \\\hline
			\\
			8 & 19.41  \\\hline
			\\
			9 & 19.02  \\\hline
			\\
			10 & 19.04  \\\hline
		\end{tabular}
		\label{table:mse_cross}
	\end{subtable}
	\caption{Polynomials from 1 to 10 and calculated mean squared error.}\label{table:kfold_polynomials_mse}
\end{table}

\section{Bootstrap}
This chapter is about the bootstrap method which is a statistical tool that can be used to calculate uncertainty associated with estimators or statistical learning methods. It can for example be used to estimate the standard errors of the coefficients from a linear regression fit. Bootstrap is a simple approach which can be adapted and applied to many different statistical learning methods. Bootstrap relies on random sampling with replacement. Replacement means that the same observations can occur more than once or maybe not at all in the bootstrap data set.

%Link til billede: https://www.draw.io/?lightbox=1&highlight=0000ff&edit=_blank&layers=1&nav=1#G0Bw37nIXex8aXSl91SjQ4MjlvNXc
\myFigure{bootstrap_example.PNG}{A graphical example of the bootstrap approach}{fig:bootstrap}{0.6}

Figure \ref{fig:bootstrap} is a graphical example of the bootstrap approach on a sample containing 3 observations. These observations are randomly sampled with replacement from the original data set Z. For each bootstrap data set $\hat{\alpha}$ can be calculated.

With all $\alpha$ values estimated the mean and standard deviation can be estimated to reason about the accuracy of the bootstrap data sets.

\subsection{Lab 5.3.4 - The Bootstrap}

The goal with Lab 5.3.4 is to show the use of bootstrap on the example used in Section 5.2 \citep[pp. 187-190]{ISLR} which uses the \emph{Portfolio} data set. The second task of Lab 5.3.4 is to estimate the accuracy of the linear regression model on the \emph{Auto} data set.

\subsubsection{Estimating the Accuracy of a Statistic of Interest}

Initially, a function for calculating $\alpha$ is required. The function is shown in listing \ref{lst:alpha}. The function receives \emph{data} which is a vector with the properties X and Y. The second input is \emph{index} which is used to access the wanted X and Y values in the vector \emph{data}.
With the X and Y values the $\alpha$ can now be calculated and returned.
\newpage
\begin{lstlisting}[caption={Function for calculating $\alpha$ in python}, label=lst:alpha, mathescape=true]
def alpha(data,index):
	X = data['X'][index]
	Y = data['Y'][index]
	return ((np.var(Y) - np.cov(X,Y)) / (np.var(X) + np.var(Y) -
		2*np.cov(X,Y)))[0,1]
\end{lstlisting}

The next step is to bootstrap the \emph{Portfolio} data set.
To perform a bootstrap a second function is needed which is seen in listing \ref{lst:bootstrap}.

\begin{lstlisting}[caption={Bootstrap function in python}, label=lst:bootstrap, mathescape=true]
def boot_python(data, function, num_of_iteration):
	n = data.shape[0]
	idx = np.random.randint(0, n, (num_of_iteration, n))
	stat = np.zeros(num_of_iteration)
	for i in xrange(len(idx)):
		stat[i] = function(data, idx[i])
	return {'Mean': np.mean(stat), 'std. error': np.std(stat)}
\end{lstlisting}

The bootstrap method receives three arguments, the first being \emph{data} which is the data that the bootstrap approach should be performed on. The next argument is the function for calculating $\alpha$. The last argument is the number of simulated data that should be created.

Now everything is set to perform a bootstrap on the \emph{Portfolio} data set with an \emph{num\_of\_iteration} set to 1,000. The result is as follows.

\begin{center}
	$\hat{\alpha} = 0.5834$
\end{center}

\begin{center}
	$SE(\hat{\alpha}) = 0.09102$
\end{center}

\subsubsection{Estimating the Accuracy of a Linear Regression Model}

In the next part of Lab 5.3.4 the goal is to use the bootstrap approach to check the variability for $\beta_0$ and $\beta_1$, which is the intercept and slope terms for the linear regression model. The linear regression model in this exercise uses \emph{horsepower} to make a prediction of \emph{mpg}. \emph{horsepower} and \emph{mpg} are part of the \emph{Auto} data set.

First step is to create a function to calculate the intercept and slope, this is seen in listing \ref{lst:boot}. The function \emph{boot} performs a linear regression fit on X and Y.
\newpage
\begin{lstlisting}[caption={Boot function in python}, label=lst:boot, mathescape=true]
def boot(data, index):
	X = data['horsepower'][index]
	Y = data['mpg'][index]
	slope, intercept, r_value, p_value, std_err = stats.linregress(X,Y)
	return [intercept, slope]
\end{lstlisting}

The next step is to adjust the bootstrap function from the last part. The modified version is the same function as in Listing \ref{lst:bootstrap}, except the modified function now returns the mean intercept, standard error, mean slope and standard error slope.

Now with the standard errors of 1,000, the bootstrap estimate for the intercept and slope can be computed.
The bootstrap estimates for the standard error intercept and slope are shown below.

\begin{center}
	$SE(\hat{\beta_0}) = 0.8601$
\end{center}
\begin{center}
	$SE(\hat{\beta_1}) = 0.007335$
\end{center}

The standard errors estimates $SE(\hat{\beta_0})$ and $SE(\hat{\beta_1})$ obtained by the library when performing a linear regression on the \emph{Auto} data set are 0.717 for the intercept and 0.0064 for the slope. These are different from the results which the bootstrap estimates yielded. This is caused by the libraries relying on the assumption of the data being linear, while it is in fact non-linear. In this case bootstrap is therefore more reliable.

The reason is that the \emph{Auto} data set has a non-linear relationship, so using a quadratic model to fit the data should yield a result that is closer to the bootstrap estimates

So a new function for finding the standard error is needed, and this function can be seen in Listing \ref{lst:bootstrap_q}.

\begin{lstlisting}[caption={Function to calculate standard error with a qudratic model}, label=lst:bootstrap_q, mathescape=true]
def boot_fn2(data, index):
	formula = 'mpg ~ horsepower+I(horsepower**2)'
	model = smf.glm(formula=formula, data=data, subset=index)
	result = model.fit()
	return result.params
\end{lstlisting}

The results it yields with a 1,000 simulated observations:

\begin{center}
	$SE(\hat{\beta_0}) = 2.06$
\end{center}
\begin{center}
	$SE(\hat{\beta_1}) = 0.0328$
\end{center}
\begin{center}
	$SE(\hat{\beta_2}) = 0.00012$
\end{center}

The new values from the quadratic formula shows a closer correspondence between the estimates from the bootstrap and the estimates from the library.