\chapter{Combining Turing Machines}
\label{chap:combining}

Recall that Turing machines are unstructured -- the execution of a machine could continue from one state to any other state of the machine.  There is
also no trivial way how to sequentially compose two machines.  We fix these problems and introduce \textit{control-flow} operators, like ``if then
else'', ``sequential composition'', and ``while''.  Thus, we do not need to define transition functions $\delta$ or even states $Q$ for complex
machines, because we only combine machines using these operators.  The transition function of the sequential composition of $M_1$ and $M_2$ is derived
from the transition functions of $M_1$ and $M_2$.  But maybe more importantly, this also makes correctness of machines compositional: From the
correctness of $M_1$ and $M_2$, we can conclude the correctness of the sequential composition of $M_1$ and $M_2$.  Also note that the number of
machine states can be \textit{huge} for complex machines, so it is important to not refer to any concrete machine state, either in the definition or
in the verification of a machine.\footnote{Using this framework, we define and verify a machine with 11537 states.}

We define control-flow operators as \textit{first-class citizen} in Coq's type theory.  As a result, we get a \textit{shallow-embedded} language for
programming multi-tape Turing machines in an imperative way.  The primitive machines defined in Chapter~\ref{chap:basic} serve as the ``primitive
instructions'' of our language.  Another possible approach, called \textit{deep embedding}, is to define a fixed syntax of a language as an inductive
data type.  We would need a compiler from syntax trees of Turing machine programs to actual Turing machines.  This compiler would have to be extended,
whenever we add a new feature to our language.  In our approach, we can just ``extend'' our language by defining a function, and we use the same
verification techniques (i.e.\ realisation and termination) on each stage of the development.

\section{$\Switch$}
\label{sec:switch}

\setCoqFilename{ProgrammingTuringMachines.TM.Combinators.Switch}

Asperti and Ricciotti~\cite{asperti2015} define the control-flow operators sequential composition, conditional, and while.  Their conditional operator
takes a concrete machine state as a parameter.  We noticed that this is not acceptable.  We now introduce a generalised operator on labelled machines,
called $\MS{Switch}$, and derive sequential composition and conditional from this operator.

\begin{definition}[$\MS{Switch}~M~f$][Switch]
  \label{def:Switch}
  Let $M : \TM_\Sigma^n(L)$ and $f : L \to \TM_\Sigma^n(L')$.  We define the machine $\MS{Switch}~M~f : \TM_\Sigma^n(L')$ with the following
  components:
  \begin{alignat*}{3}
    & Q                  &~:=~& Q_M +  \sum_{l:L} Q_{f(l)} \\
    & start              &~:=~& \inl start_M \\
    \delta ~&(\inl q, s) &~:=~&
    \begin{cases}
      \bigl(\inr (lab_M~q, start_{f (lab_M~q)}), (\None, N)^n \bigr) & halt_M(q) \\
      \Let{(q', act) := \delta_M(q, s)}{\left(\inl q', act \right)} & \lnot halt_M(q)
    \end{cases} \\
    \delta ~&(\inr q, s) &~:=~& \Let{(q', act) := \delta_{f(\pi_1 q)} (\pi_2 q, s)}{\bigl( \inr (\pi_1 q, q'), act \bigr)} \\
    halt   ~&(\inl  q)   &~:=~& \false \\
    halt   ~&(\inr  q)   &~:=~& halt_{f(\pi_1~q)} (\pi_2~q) \\
    lab   ~&(\inl  q)   &~:=~& \_ \\
    lab   ~&(\inr  q)   &~:=~& lab_{f(\pi_1~q)} (\pi_2~q)
  \end{alignat*}
\end{definition}

In Definition~\ref{def:Switch}, the $lab$ value for $\inl$ is unimportant, because the lifted states of $M$ are not terminating states for
$\MS{Switch}~M~f$.  We just use a canonical value.

\begin{figure}
  \center
  \input{fig-SwitchExample}
  \caption{Example of a $\MS{Switch}$.  The left box stands for the first machine $M_1:\TM_\Sigma^n(\Bool)$.  The final states $q_1$ and $q_2$ are
    mapped to $\true$ and $q_3$ to $\false$.  After $\MS{Switch}$ reaches one of the injections of the terminal states $q_1, q_2, q_3$ of $M_1$, it
    continues its execution either in the top case-machine $M_2$ or in the bottom case-machine $M_3$.  The halting states of $\MS{Switch}$ are exactly
    the injections of the halting states of the case-machines.}
  \label{fig:match}
\end{figure}

$\MS{Switch}~M~f$ first executes a copy of $M$.  When it reaches a final state $q$ of $M$, it does a ``nop'' action (i.e.\ $(\None,N)^n$) and changes
to the injection of the start state of the machine $f(lab_M~q)$.  When $\MS{Switch}~M~f$ reaches a state that is the injection of a final state of a
machine $f~l$, it terminates.  The correctness part of the semantics can be expressed using the following lemma:

\begin{lemma}[Correctness of $\MS{Switch}~M~f$][Switch_Realise]
  \label{lem:Switch_Realise}
  Let $R \subseteq \Tape_\Sigma^n \times L \times \Tape_\Sigma^n$ and $R'~l \subseteq \Tape_\Sigma^n \times L' \times \Tape_\Sigma^n$ for all $l:L$.
  If $M \Realise R$ and $f~l \Realise R'~l$ for all $l:L$, then
  \[
    \MS{Switch}~M~f \Realise SwitchRel~R~R'
  \]
  with
  \[
    SwitchRel~R~R' := \bigcup_{l:L} \bigl( R\at l \circ R'~l \bigr)
  \]
\end{lemma}

Note that in the correctness relation, we compose the unlabelled relation $R \at l \subseteq \Tape_\Sigma^n \times \Tape_\Sigma^n$ with the labelled
relation $R'~l \subseteq \Tape_\Sigma^n \times L' \times \Tape_\Sigma^n$.  This means that $\Switch~M~f$ terminates in a state with a label of $f~l$,
which has type $L'$.

To specify the running time of $\MS{Switch}~M~f$, we need to know the running time relation in that $M$ terminates, and for each $l:L$ the running
time relation of $f~l$.  We also need to know the correctness relation of $M$, because the running time of $f~l$ depends on the output of $M$, which
is the input of $f~l$.  Also, the choice of the case-machine depends on the label of the terminating state of $M$.

\begin{lemma}[Running Time of $\MS{Switch}~M~f$][Switch_TerminatesIn]
  \label{lem:Switch_Terminates}
  Let $R \subseteq \Tape_\Sigma^n \times L \times \Tape_\Sigma^n$, $T \subseteq \Tape_\Sigma^n \times \Nat$, and
  $T'~l \subseteq \Tape_\Sigma^n \times \Nat$ for all $l:L$.  If $M \Realise R$, $M \TerminatesIn T$, and $f~l \TerminatesIn T'~l$ for all $l:L$, then
  $\MS{Switch}~M~f \TerminatesIn SwitchT~R~T~T'$, where
  \[
    SwitchT~R~T~T' :=
    \lambda~t~k.~ \exists~k_1~k_2.~T~t~k_1 \land 1+k_1+k_2 \le k \land
      \forall~l~t'.~ R~t~(l,t') \rightarrow T'(l)~t'~k_2
  \]
\end{lemma}

We can combine the correctness and running time lemma, in case $M$ and every $f~l$ terminates in constant time.
\begin{lemma}[Correctness of $\MS{Switch}~M~f$ in constant time][Switch_RealiseIn]
  \label{lem:Switch_RealiseIn}
  Let $k_1, k_2:\Nat$.
  If $M \RealiseIn{k_1} R$ and $f~l \RealiseIn{k_2} R'~l$ for every $l:L$, then
  \[ \MS{Switch}~M~f \RealiseIn{1+k_1+k_2} SwitchRel~R~R' \]
\end{lemma}
\begin{proof}
  Follows with the Lemmas~\ref{lem:Realise_total},~\ref{lem:Switch_Realise}, and~\ref{lem:Switch_Terminates}.
\end{proof}

\subsection{Derived Operators}
\label{sec:match-derived-operators}
\setCoqFilename{ProgrammingTuringMachines.TM.Combinators.SequentialComposition}%

As mentioned above, conditional and sequential composition can be defined as instances of the Switch operator.  For the sequential composition
$M_1 \Seq M_2$ with $M_1 : \TM_\Sigma^n(L)$ and $M_2 : \TM_\Sigma^n(L')$, the function $f : L \to \TM_\Sigma^n(L')$, maps all labels of $M_1$ to the
machine $M_2$.
\begin{definition}[Sequential composition][Seq]
  \label{def:Seq}
  Let $M_1 : \TM_\Sigma^n(L)$ and $M_2 : \TM_\Sigma^n(L')$ .
  \[
    M_1 \Seq M_2 := \Switch~M_1~
    \bigl(
    \lambda~\_.~M_2
    \bigr)
  \]
\end{definition}

This means that regardless in which state the machine $M_1$ terminates, the sequential composition $M_1 \Seq M_2$ continues its execution in the start
state of $M_2$.  The following lemma is the correctness lemma of sequential composition for constant-time termination.  The version without
constant-time termination, i.e.\ the lemma we get notationally by removing the step numbers, holds as well.

\begin{lemma}[Correctness of sequential composition][Seq_RealiseIn]
  \label{lem:Seq_RealiseIn}
  If $M_1 \RealiseIn{k_1} R_1$ and $M_2 \RealiseIn{k_2} R_2$, then
  $M_1 \Seq M_2 \RealiseIn{1+k_1+k_2} \bigcup_{l:L} \left(R_1\at l \circ R_2 \right)$.
\end{lemma}

We often have the case that the first machine $M_1$ is labelled over the unit-type $L = \Unit$.  In this case, the relation is
$\left(\bigcup_{l:L} R_1\at l\right) \circ R_2 \equiv R_1 \at \unit \circ R_2 = R_1 \circ R_2$ by the convention that we identify unit-labelled
relations with unlabelled relations.  This means that sequential composition of two machines amounts to composing their correctness relations.

In case either $M_1$ or $M_2$ do not have constant running time, we need the running time lemma, which can be derived from
Lemma~\ref{lem:Switch_Terminates}.
\begin{lemma}[Running Time of sequential composition][Seq_TerminatesIn]
  \label{lem:Seq_TerminatesIn}
  If $M_1 \Realise R_1$, $M_1 \TerminatesIn T_1$, and $M_2 \TerminatesIn T_2$, then
  \[
    M_1 \Seq M_2 \TerminatesIn
    \left(%
      \lambda t~k.~\exists k_1~k_2.~T_1~t~k_1 ~\land~ 1 + k_1 + k_2 \leq k ~\land~ \forall t'~l.~R_1~t~(l,t') \rightarrow T_2~t'~k_2%
    \right)
  \]
\end{lemma}

For the conditional $\If{M_1}{M_2}{M_3}$ with $M_1 : \TM_\Sigma^n(\Bool)$, $M_2, M_3 : \TM_\Sigma^n(L)$, the function $f : \Bool \to \TM_\Sigma^n(L)$
simply maps $\true$ to $M_2$ and $\false$ to $M_3$.  The conditional, as defined below, first executes $M_1$.  If $M_1$ terminates in a state with
label $\true$, it continues the execution in $M_2$, else in $M_3$.

\setCoqFilename{ProgrammingTuringMachines.TM.Combinators.If}%
\begin{definition}[Conditional][If]
  \label{def:If}
  Let $M_1 : \TM_\Sigma^n(\Bool)$, $M_2, M_3 : \TM_\Sigma^n(L)$.
  \[
    \If{M_1}{M_2}{M_3} := \Switch~M_1
    \left(\lambda b.~\cond{b}{M_2}{M_3} \right)
  \]
\end{definition}

\begin{lemma}[Correctness of conditional][If_RealiseIn]
  \label{lem:If_RealiseIn}
  If $M_1 \RealiseIn{k_1} R_1$, $M_2 \RealiseIn{k_2} R_2$, and $M_3 \RealiseIn{k_3} R_3$, then
  $\If{M_1}{M_2}{M_3} \RealiseIn{1+k_1+max(k_2+k_3)} (R_1\at\true \circ R_2) \cup (R_1\at\false \circ R_3)$.
\end{lemma}

The correctness relation $(R_1\at\true \circ R_2) \cup (R_1\at\false \circ R_3)$ captures the idea of the following case-distinction: If the
conditional machine terminates, than the copy of $M_1$ either terminates in a state with label $\true$ or $\false$.  In the first case, the
conditional proceeds in $M_2$, else in $M_3$.

The running time lemma of the conditional can also be derived from Lemma~\ref{lem:Switch_Terminates}.
\begin{lemma}[Running Time of the conditional][If_TerminatesIn]
  \label{lem:If_TerminatesIn}
  If $M_1 \Realise R_1$, $M_1 \TerminatesIn T_1$, $M_2 \TerminatesIn T_2$, and $M_3 \TerminatesIn T_3$, then
  \begin{align*}
    & \If{M_1}{M_2}{M_3} \TerminatesIn \\
    &\quad \bigl( \lambda t~k.~\exists k_1~k_2.~T_1~t~k_1 ~\land~ 1+k_1+k_2 \leq k ~\land~ \\
    &\quad \phantom{\bigl( \lambda t~k.~\exists k_1~k_2.}~\forall b~t'.~R_1~t~(b, t') \rightarrow \cond{b}{T_2~t'~k_2}{T_3~t'~k_2} \bigr)
  \end{align*}
\end{lemma}

Asperti and Ricciotti~\cite{asperti2015} also define sequential composition and a conditional operator.  However, they have two fundamental
differences.  First, they define and verify each operator separately.  Secondly, they do not consider state labelling.  Their conditional operator
takes one concrete ``negative'' state, i.e.\ the machine $M_3$ is only executed if $M_1$ terminates in this particular state.  This makes programming
and reasoning about concrete machines tedious, because complex machines also have many states.  They also have to introduce a separate notion for
correctness, because they do not have states in their original one.  By introducing state-labelled machines and by implementing the more general
$\Switch$ operator, we solve both problems.  We no longer have to specify concrete machine states in the definition and verification of machines.
Moreover, we often exploit the convenient generality of the $\Switch$ operator.  For example, it can be used to implement a machine that copies a
symbol from one tape to another tape, as demonstrated in Chapter~\ref{chap:combining}.  Also note, that Asperti and Ricciotti~\cite{asperti2015} have
no notion of time complexity; their strong notion of realisation only implies termination in an uncertain number of steps.  The idea of
state-labelling and $\Switch$ is due to Y.~Forster and F.~Kunze.

\subsection{Proof of $\Switch$}
\label{sec:match-proofs}

\setCoqFilename{ProgrammingTuringMachines.TM.Prelim}

The idea of the proofs of Lemma~\ref{lem:Switch_Realise} and Lemma~\ref{lem:Switch_Terminates} is to abstract two features of the machine: lifting of
configurations from one abstract machine to another abstract machines, and sequencing of two executions.  We formalise these two concepts for abstract
machines, i.e.\ we argue on the abstract $\Loop$ function.

For the first feature, \textit{lifting}, we assume two types $A$, $B$ for abstract configurations, a function $lift : A \to B$, two step functions
$f : A \to A$, $f' : B \to B$, and two halting functions $h : A \to \Bool$, $h' : B \to \Bool$.  We assume that the step functions $f$ and $f'$ are
compatible with $lift$ in non-halting states of $A$.  Formally, this means:
\begin{equation}
  \label{eq:loop_lift_assumption1}
  \forall a:A.~ h(x)=\false \rightarrow f' (lift~x) = lift (f~x)
\end{equation}

The second assumption is that $h'$ and $h$ are compatible w.r.t.\ $lift$; formally:
\begin{equation}
  \label{eq:loop_lift_assumption2}
  \forall a:A.~h'(lift~x)=h(x)
\end{equation}

Under these two assumptions we can show two lemmas that essentially say that the second abstract machine $B$ simulates the first machine $A$.
\begin{lemma}[Loop lifting][loop_lift]
  \label{lem:loop_lift}
  Under the assumptions~(\ref{eq:loop_lift_assumption1}) and~(\ref{eq:loop_lift_assumption2}):
  \begin{alignat*}{1}
    & \forall (k:\Nat)~(a~a' : A). \\
    & \quad \Loop~f ~h ~a        ~k = \Some{a'} \rightarrow \\
    & \quad \Loop~f'~h'~(lift~a) ~k = \Some {lift~a'}
  \end{alignat*}
\end{lemma}
\begin{proof}
  By induction on $k:\Nat$.
\end{proof}
\begin{lemma}[Loop unlifting][loop_unlift]
  \label{lem:loop_unlift}
  Under the assumptions~(\ref{eq:loop_lift_assumption1}) and~(\ref{eq:loop_lift_assumption2}):
  \begin{alignat*}{1}
    & \forall (k:\Nat)~(a:A)~(b':B). \\
    & \quad \Loop~f'~h'~(lift~a)~k = \Some{b'} \rightarrow \\
    & \quad \exists (a':A).~\Loop~f~h~a~k = \Some{a'} \land b'=lift~a'
  \end{alignat*}
\end{lemma}
\begin{proof}
  By induction on $k:\Nat$.
\end{proof}

\begin{figure}
  \center
  \begin{tikzcd}
    a'                    \arrow[r, "lift"] & \cdot \\
    \cdot \arrow[u, "f"]  \arrow[r, "lift"] & \cdot \arrow[u, "g", swap] \\
    a     \arrow[u, "f"]  \arrow[r, "lift"] & \cdot \arrow[u, "g", swap] \\
  \end{tikzcd}
  \hspace{1cm}
  \begin{tikzcd}
    a'                            \arrow[r, "lift", dotted] & b' \\
    \cdot \arrow[u, "f", dotted]  \arrow[r, "lift", dotted] & \cdot \arrow[u, "g", swap] \\
    a     \arrow[u, "f", dotted]  \arrow[r, "lift"]         & \cdot \arrow[u, "g", swap] \\
  \end{tikzcd}
  \caption{Instances of Lemma~\ref{lem:loop_lift} (left) and Lemma~\ref{lem:loop_unlift} (right) for $k=2$ as commuting diagrams.  Dotted lines denote
    existentials.  Note that the rectangles correspond to condition (\ref{eq:loop_lift_assumption1}).  Only the top ``states'' are terminating
    states.}
  \label{fig:loop_lift_lemmas}
\end{figure}

The Lemmas~\ref{lem:loop_lift} and~\ref{lem:loop_unlift} are visualised in Figure~\ref{fig:loop_lift_lemmas}.

For the second feature, \textit{sequential execution}, we assume another type $A$ with a step function $f \from A \to A$ and two halting functions
$h, h' \from A \to \Bool$.  We assume, that if $a$ is a non-halting state w.r.t.\ $h$, then $a$ also is a non-halting state w.r.t.\ $h'$:
\begin{equation}
  \label{eq:loop_merge_assumption}
  \forall(a:A).~h~a = \false \rightarrow h'~a = \false
\end{equation}

\begin{lemma}[Loop merging][loop_merge]
  \label{lem:loop_merge}
  Under assumption~(\ref{eq:loop_merge_assumption}):
  \begin{align*}
    & \forall (k_1~k_2:\Nat)~(a_1~a_2~a_3 : A).\\
    & \quad \Loop~f~h ~a_1~k_1       = \Some{a_2} \rightarrow\\
    & \quad \Loop~f~h'~a_2~k_2       = \Some{a_3} \rightarrow\\
    & \quad \Loop~f~h'~a_1~(k_1+k_2) = \Some{a_3}
  \end{align*}
\end{lemma}
\begin{proof}
  By induction on $k_1:\Nat$, using Lemma~\ref{lem:loop}~(\ref{lem:loop_monotone}).
\end{proof}
\begin{lemma}[Loop splitting][loop_split]
  \label{lem:loop_split}
  Under assumption~(\ref{eq:loop_merge_assumption}):
  \begin{align*}
    & \forall (k:\Nat)~(a_1~a_3 : A).\\
    & \quad \Loop~f~h'~a_1~k = \Some{a_3} \rightarrow \\
    & \quad \exists (k_1~k_2:\Nat)~(a_2:A).\\
    & \quad \quad \Loop~f~h ~a_1~k_1 = \Some{a_2} \land\\
    & \quad \quad \Loop~f~h'~a_2~k_2 = \Some{a_3} \land\\
    & \quad \quad k_1 + k_2 \leq k
  \end{align*}
\end{lemma}
\begin{proof}
  By complete induction on $k:\Nat$.
\end{proof}

Back to the verification of $\Switch~M~f$.  In the following, we simply write $\Switch$.  For a configuration $c_k$, we write $q_k$ and $t_k$ for the
state and tapes component of $c_k$.

The execution steps of $\Switch$ are essentially a sequence of first, the lifted steps of an execution of $M$, second, a ``nop'' transition, and
third, the lifted steps of the execution of $f~l$.  We give the concrete lifting functions from $M$ to $\Switch$ and from $f~l$ to $\Switch$:
\setCoqFilename{ProgrammingTuringMachines.TM.Combinators.Switch}%
\begin{definition}[Liftings of $\Switch$][lift_confL]
  We define the functions \\$liftL : \Conf_M \to \Conf_\Switch$ and $liftR_l : \Conf_{f(l)} \to \Conf_\Switch$ for all $l:L$:
  \begin{alignat*}{2}
    & liftL   & (q, t) &:= (\inl q,      t) \\
    & liftR_l & (q, t) &:= (\inr (l, q), t)
  \end{alignat*}
\end{definition}

For the sequential lab, we also have to define the lifted halting function of \\$haltConfL : \Conf_\Switch \to \Bool$:
\begin{definition}[Lifted halting function][halt_liftL]
  \begin{alignat*}{2}
    & haltConfL & (\inl~q, t) &:= halt_M(q) \\
    & haltConfL & (\inr~q, t) &:= \true
  \end{alignat*}
\end{definition}

Using Lemma~\ref{lem:loop_merge} and~\ref{lem:loop_lift}, we can show the lemma we need for running time:
\begin{lemma}[Merging parts of executions of $\Switch$][Switch_merge]
  \label{lem:Switch_merge}
  Let $t : \Tape_\Sigma^n$, $k_1, k_2:\Nat$, $c_1 : \Conf_M$ and $c_2 : \Conf_{f(lab_M~q_1)}$.
  \begin{alignat*}{1}
    & M(t) \terminates^{k_1} c_1 \rightarrow\\
    & \bigl( f~(lab_M~q_1) \bigr) (t_1) \terminates^{k_2} c_2 \rightarrow\\
    & \Switch(t) \terminates^{1+k_1+k_2} liftR(c_2)
  \end{alignat*}
\end{lemma}
\begin{proof}
  We apply Lemma~\ref{lem:loop_merge} and have to show:
  \begin{enumerate}
  \item $\forall a:\Conf_\Switch.~ haltConfL~a = \false \rightarrow \MS{haltConf}~a = \false$.  This holds trivially by case-analysis over $a$.
  \item $\Loop~step_\Switch~haltConfL~(\MS{initConf}_\Switch~t)~k_1 =
    \Some{liftL~c_1}$.  By definition, we have $\MS{initConf}_\Switch~t=liftL(\MS{initConf}_M~t)$.  The claim follows with Lemma~\ref{lem:loop_lift}.
  \item $\Loop~step_\Switch~haltConf_\Switch~(liftL~c1)~(1+k_2) =
    \Some{liftR(c_2)}$.  By definition, we know that the first step must be a ``nop'' transition from
    $liftL~c_1$ to\\
    $liftR~\bigl(\MS{initConf}_{f(lab_M~q_1)}~t_1 \bigr)$.  It remains to show that:
    $$\Loop~step_\Switch~haltConf_\Switch~liftR \bigl(\MS{initConf}_{f(lab_M~q_1)}~t_1 \bigr)~k_2 = \Some{liftR(c_2)}$$
    This follows with Lemma~\ref{lem:loop_merge}.
  \end{enumerate}
\end{proof}
The running time Lemma~\ref{lem:Switch_Terminates} follows directly from Lemma~\ref{lem:Switch_merge}.

\begin{lemma}[Splitting execution of $\Switch$][Switch_split]
  \label{lem:Switch_split}
  Let $t : \Tape_\Sigma^n$, $k : \Nat$, $c : \Conf_\Switch$.
  \begin{align*}
    & \Switch(t) \terminates^k c \rightarrow \\
    & \exists (k_1~k_2 : \Nat)~(c_1 : \Conf_M)~(c_2 : \Conf_{f(lab_M~q_1)}). \\
    & \quad M(t) \terminates^{k_1} c_1 ~\land~ \\
    & \quad \bigl( f (lab_M~q_1) \bigr) (t_1) \terminates^{k_2} c_2 ~\land~ \\
    & \quad c = liftR(c_2)
  \end{align*}
\end{lemma}
\begin{proof}
  Analogous to the proof of Lemma~\ref{lem:Switch_merge}, using Lemmas~\ref{lem:loop_split} and~\ref{lem:loop_unlift}.
\end{proof}
The correctness Lemma~\ref{lem:Switch_Realise} follows directly from Lemma~\ref{lem:Switch_split}.

Asperti and Ricciotti~\cite{asperti2015} have a version of Lemma~\ref{lem:loop_lift} and Lemma~\ref{lem:loop_merge}.

\section{$\MS{While}$}
\label{sec:While}
\setCoqFilename{ProgrammingTuringMachines.TM.Combinators.While}%

The machine $\While~M$ essentially behaves like a ``do-while'' loop in imperative languages like C.  At the end of the execution of the loop body $M$,
$M$ decides either to continue or break out of the loop.  If $M$ terminates in a state with label $\None$, then the loop is continued, and if $M$
terminates in $\Some{l}$, the loop breaks and $\While~M$ terminates in a state with label $l$.

\begin{definition}[$\MS{While}~M$][While]
  \label{def:While}
  Let $M : \TM_\Sigma^n(\Option(L))$ and $def:L$.  We define $\MS{While}~M : \TM_\Sigma^n(L)$ with the following components.
  \begin{alignat*}{3}
    & Q              &~:=~& Q_M \\
    & start          &~:=~& start_M \\
    \delta ~& (q, s) &~:=~&
    \begin{cases}
      (start_M, (\None, N)^n) & halt_M(q) \\
      \delta_M (q,s)    & \lnot halt_M(q)
    \end{cases} \\
    halt ~& q      &~:=~& halt_M(q) \land lab_M(q) \neq \None \\
    lab ~& q      &~:=~&
    \begin{cases}
      l   & lab_M(q) = \Some l \\
      def & lab_M(q) = \None
    \end{cases}
  \end{alignat*}
\end{definition}

\begin{figure}
  \center
  \input{fig-WhileExample}
  \caption{Example for $\While~M$ with $M:\TM_\Sigma^n(\Option(\Bool))$.  When $M$ reaches the state $q_1$, the loop is continued, because $q_1$ is in
    assumed to have label $\None$.  The other halting states $q_2$ and $q_3$ have the labels $\Some{\true}$ and $\Some{\false}$.  Therefore $\While~M$
    terminates in a state with label $\true$ or $\false$, when $M$ terminates in $q_2$ or $q_3$, respectively.}
  \label{fig:while-example}
\end{figure}

In Definition~\ref{def:While}, we have to assume that $L$ is inhabited.  However, the choice of $def:L$ is semantically irrelevant, because $\While~M$
only halts in states where $lab_M(q) \neq \None$.

The correctness of $\MS{While}$ can be expressed using the following lemma:

\begin{lemma}[Correctness of $\MS{While}$][While_Realise]
  \label{lem:While_Realise}
  Let $R \subseteq \Tape_\Sigma^n \times \Option(L) \times \Tape_\Sigma^n$.
  If $M \Realise R$, then $\While~M \Realise WhileRel~R$, where
  $WhileRel~R \subseteq \Tape_\Sigma^n \times L \times \Tape_\Sigma^n$
  is inductively defined by the following two rules:
  \[
    \inferrule{R~t~(\Some{l}, t')}{WhileRel~R~t~(l, t')}
    \quad
    \inferrule{R~t~(\None, t') \and WhileRel~R~t'~(l, t'')}{WhileRel~R~t~(l, t'')}
  \]
\end{lemma}

We can also express the correctness relation of $\While$ using the Kleene star:
\begin{fact}[Alternative correctness relation for $\While~M$][OtherWhileRel]
  ~
  \[
    WhileRel~R \equiv (R \at \None)^* \circ \Bigl( \bigcup_{l:L} \bigl( R \at {\Some{l}} \bigr) \att l \Bigr)
  \]
\end{fact}

Both definitions of $WhileRel$ should make clear what $\While~M$ does: It repeats the execution of $M$ as long as it terminates in $\None$, and after
$M$ terminates in $\Some{l}$, $\While~M$ terminates in a state with label $l$.  This is visualised in Figure~\ref{fig:while-example}.

When we want to prove $\While~M \Realise R$ for some machine $M:\TM_\Sigma^n(\Option(L))$ and relation
$R \subseteq \Tape_\Sigma^n \times L \times \Tape_\Sigma^n$, we must of course have already proven that $M \Realise R'$ for some relation
$R' \subseteq \Tape_\Sigma^n \times \Option(L) \times \Tape_\Sigma^n$.  Then we apply the monotonicity Lemma~\ref{lem:Realise_monotone} and the above
correctness Lemma~\ref{lem:While_Realise}, and have to show $WhileRel~R' \subseteq R$.  We can prove this by induction on the inductive predicate,
which is equivalent to applying the following lemma:
\begin{lemma}[Induction for $WhileRel$][WhileInduction]
  \label{lem:WhileInduction}
  ~
  \begin{alignat*}{1}
    & \left(\forall t~t'~l.~R'~t~(\Some{l}, t') \rightarrow R~t~(l,t')\right) \rightarrow \\
    & \left(\forall t~t'~t''~l.~R'~t~(\None, t') \rightarrow R~t'~(l, t'') \rightarrow R~t~(l,t'')\right) \rightarrow \\
    & WhileRel~R' \subseteq R
  \end{alignat*}
\end{lemma}

The running time lemma of $\While$ is dual.  We define a co-inductive termination relation $WhileT~R~T$, where $R$ is the relation that $M$ realises
and $T$ is the running time relation in that $M$ terminates.
\begin{lemma}[Running Time of $\While~M$][While_TerminatesIn]
  \label{lem:While_TerminatesIn}
  If $M \Realise R$ and $M \TerminatesIn T$.  Then $\While~M \TerminatesIn WhileT~R~T$, where $WhileT~R~T \subseteq \Tape_\Sigma^n \times \Nat$ is
  defined as the following co-inductive running time relation:
  \[
    \inferrule{T~t~k_1 \\
      \forall t'~l.~R~t~(\Some{l}, t') \rightarrow k_1 \leq k \\
      \forall t'.~R~t~(\None,t') \rightarrow \exists k_2.~WhileT~R~T~t'~k_2 \land 1+k_1+k_2 \leq k}%
    {WhileT~R~T~t~k}
  \]
\end{lemma}

When we want to show $\While~M \TerminatesIn T$, this is dual to showing $\While~M \Realise R$.  We apply the anti-monotonicity
Lemma~\ref{lem:TerminatesIn_monotone} and the above running time lemma (where $T'$ is the running time relation in that $M$ terminates), and have to
show $T \subseteq WhileT~R~T'$.  For that, we use the co-induction lemma of $WhileT$:
\begin{lemma}[$WhileT$ co-induction][WhileCoInduction]
  \label{lem:WhileCoInduction}
  To show $T \subseteq WhileT~R~T'$, it suffices to show:
  \begin{alignat*}{1}
    &\forall t~k.~T~t~k \rightarrow \\
    &\quad \exists k_1.~T'~t~k_1 ~\land~ \\
    &\quad\quad \left( \forall l~t'.~R~t~(\Some{l}, t') \rightarrow k_1 \leq k \right) ~\land~ \\
    &\quad\quad \left( \forall t'.~R~t~(\None, t') \rightarrow \exists k_2.~T~t'~k_2 ~\land~ 1+k_1+k_2 \leq k \right).
  \end{alignat*}
\end{lemma}
The number $k_1$ is the number of steps needed for the first iteration of the loop.  We have to consider all possible outputs of the first loop.  If
$M$ terminates in label $\Some{l}$ in $k_1$ steps, then $\While~M$ also only needs $k_1$ steps.  However, if $M$ terminates in $\None$, and $\While~M$
needs $k_2$ steps for all other loops, then $\While~M$ needs $1+k_1+k_2$ steps in total.  The one additional step comes from the ``nop''-transition
back to the starting state.

\subsection{Proof of $\While$}
\label{sec:while-proofs}

The running time and correctness proofs are similar to the proofs of $\Switch$, as explained in Section~\ref{sec:match-proofs}.  The configurations of
$\While$ are exactly the configurations of $M$, so the lifting function is the identity function.  However, the fundamental difference between
$\Switch$ and $\While$ is that $\While$ can execute $M$ arbitrarily often; it could also diverge.  As a consequence, we also need complete induction
on step-numbers, in addition to the loop-splitting and loop-merging lemmas.  We present the key lemmas here and the most important parts of the
proofs.

We simply write $\While$ instead of $\While~M$ in this section.  Since we have $\Conf_M = \Conf_\While$, we also only write $\Conf$.

The first lemma says that an execution of $\While$ consists of an execution of $M$ and a (possibly empty) continuation of $\While$:
\begin{lemma}[Splitting the execution of $\While$][While_split]
  \label{lem:While_split}
  Let $c_1, c_3 : \Conf$ and $k:\Nat$.  Then
  \begin{alignat*}{1}
    & \While(c_1) \terminates^k c_3 \rightarrow \\
    & \exists (k_1~k_2 : \Nat)~c_2. \\
    & \quad M(c_1) \terminates^{k_1} c_2 ~\land~ \\
    & \quad \While(c_2) \terminates^{k_2} c_3 ~\land~\\
    & \quad k_1 + k_2 \leq k
  \end{alignat*}
\end{lemma}
\begin{proof}
  Follows with Lemma~\ref{lem:loop_split} and Lemma~\ref{lem:loop_lift}.
\end{proof}

We have two more splitting lemmas, one for the case that $\While$ terminates immediately, and one for the case that $\While$ continues the loop.
\begin{lemma}[Splitting, break case][While_split_term]
  \label{lem:While_split_term}
  Let $c_1$, $c_2 : \Conf$, $k:\Nat$, and $l:L$.
  \begin{alignat*}{1}
    & \While(c_1) \terminates^k c_2 \rightarrow \\
    & haltConf_M~c_1 \rightarrow \\
    & lab_M(q_1) = \Some l \rightarrow \\
    & c_1 = c_2
  \end{alignat*}
\end{lemma}
\begin{proof}
  By Lemma \ref{lem:loop} (\ref{lem:loop_eq_0}), because $c_1$ is a halting state of $\While$.
\end{proof}
\begin{lemma}[Splitting, continue case][While_split_repeat]
  \label{lem:While_split_repeat}
  Let $c_1$, $c_2 : \Conf$, $k:\Nat$.
  \begin{alignat*}{1}
    & \While(c_1) \terminates^k c_2 \rightarrow \\
    & haltConf_M~c_1 \rightarrow \\
    & lab_M(q_1) = \None \rightarrow \\
    & \exists k'.~k = 1+k' ~\land~ \While(t_1) \terminates^{k'} c_2
  \end{alignat*}
\end{lemma}
\begin{proof}
  $\While$ must have taken the ``nop''-transition from $c_1$ to $initConf~t_1$, because $c_1$ is a halting configuration of $M$ but not of $\While$.
\end{proof}

We now can prove the correctness Lemma~\ref{lem:While_Realise} of $\While$.
\begin{proof}
  We assume $\While(t_1) \terminates^k c_3$ and have to show $WhileRel~t_1~(lab_\While~q_3, t_3)$.  We use complete induction on $k:\Nat$.  By
  Lemma~\ref{lem:While_split}, we have $M(t_1) \terminates^{k_1} c_2$ and\\ $\While(c_2) \terminates^{k_2} c_3$, for $k_1+k_2 \leq k$.  Case analysis.
  \begin{enumerate}
  \item $lab_M(q_2) = \Some{l}$.  Then we know by Lemma~\ref{lem:While_split_term}, that $c_2=c_3$.  It remains to show $WhileRel~t_1~(l, t_2)$.  By
    applying the first constructor, it is enough to show $R~t_1~(\Some{l}, t_2)$.  This follows from the realisation of $M$.
  \item $lab_M(q_2) = \None$.  By Lemma~\ref{lem:While_split_repeat}, we know that $\While(t_2) \terminates^{k'_2} c_3$ for $k_2 = 1 + k'_2$.  The
    inductive hypothesis gives $WhileRel~t_2~(lab_\While~q_3, t_3)$.  The goal follows by applying the second constructor and the realisation of $M$.
  \end{enumerate}
\end{proof}

For the running time proofs, we again have lemmas that ``merge'' executions together.

\begin{lemma}[Merging, break case][While_merge_term]
  Let $k : \Nat$, $c_1, c_2 : \Conf$, and $l:L$.
  \begin{alignat*}{1}
    & M(c_1) \terminates^k c_2 \rightarrow \\
    & lab_M(q_2) = \Some{l} \rightarrow \\
    & \While(c_1) \terminates^k c_2
  \end{alignat*}
\end{lemma}
\begin{lemma}[Merging, continue case][While_merge_repeat]
  Let $k_1, k_2 : \Nat$, $c_1, c_2, c_3 : \Conf$.
  \begin{alignat*}{1}
    & M(c_1) \terminates^{k_1} c_2 \rightarrow \\
    & lab_M(q_2) = \None \rightarrow \\
    & \While(t_2) \terminates^{k_2} c_3 \rightarrow \\
    & \While(c_1) \terminates^{1+k_1+k_2} c_3
  \end{alignat*}
\end{lemma}

The running time Lemma~\ref{lem:While_TerminatesIn} follows similarly, by complete induction on $k:\Nat$.

\section{$\MS{Mirror}$}
\label{sec:mirror}

We can define a machine operator that ``mirrors'' a machine $M$.  Whenever $M$ makes a transition with a move to $L$, $\MS{Mirror}~M$ moves the head
to the right instead.  For example, we define a machine $\MS{MoveToSymbol}$ below, that moves the head of the tape right, until it reads a certain
symbol.  Using this operator, we get a machine ``for free'' that moves the head to the left instead.  However, we still have to copy or parametrise
the correctness and running time relations.

\setCoqFilename{ProgrammingTuringMachines.TM.TM}%
Using the proof techniques developed in the previous sections, verifying the $\MS{Mirror}$ operator is very easy.  The function
$\coqlink[mirror_tape]{\MS{mirror}} : \Tape_\Sigma \to \Tape_\Sigma$ swaps the left and right part of a tape.  %
% \begin{definition}[Mirror tape][mirror_tape]
%   \label{def:mirror_tape}
%   Let $t : \Tape_\Sigma$.
%   \begin{alignat*}{2}
%     & \MS{mirror}~(\MS{niltape})         &&~:=~ \MS{niltape} \\
%     & \MS{mirror}~(\MS{leftof}~l~ls)     &&~:=~ \MS{rightof}~l~ls \\
%     & \MS{mirror}~(\MS{rightof}~l~ls)    &&~:=~ \MS{leftof}~l~ls \\
%     & \MS{mirror}~(\MS{midtape}~ls~m~rs) &&~:=~ \MS{midtape}~rs~m~ls
%   \end{alignat*}
% \end{definition}
% \begin{lemma}[Correctness of $\MS{mirror}$]
%   \label{lem:mirror}
%   Let $t:\Tape_\Sigma$.
%   \begin{enumerate}
%   \coqitem[mirror_tape_injective] \label{lem:mirror_tape_injective}
%     $\MS{mirror}$ is injective,
%   \coqitem[mirror_tape_involution] \label{lem:mirror_tape_involution}
%     $\MS{mirror}$ is an involution, i.e.\ $\MS{mirror}(\MS{mirror}(t))=t$,
%   \coqitem[mirror_tape_left] \label{lem:mirror_left}
%     $\MS{left}(\MS{mirror}(t)) = \MS{right}(\MS{mirror}(t))$,
%   \coqitem[mirror_tape_right] \label{lem:mirror_right}
%     $\MS{right}(\MS{mirror}(t)) = \MS{left}(\MS{mirror}(t))$,
%   \coqitem[mirror_tape_current] \label{lem:mirror_current}
%     $\MS{current}(\MS{mirror}(t)) = \MS{current}(t)$.
%   \end{enumerate}
% \end{lemma}
% % \begin{proof}
% %   By case analysis over the tape(s).
% % \end{proof}
Furthermore, we define an injective and involutive function $\coqlink[mirror_move]{\MS{swap}}:\MS{Move} \to \MS{Move}$ that simply swaps the movements
$L$ and $R$.  \setCoqFilename{ProgrammingTuringMachines.TM.Combinators.Mirror}%
\begin{definition}[$\MS{Mirror}~M$][Mirror]
  \label{def:Mirror}
  Let $M:\TM_\Sigma^n(L)$.  The machine $\MS{Mirror}~M : \TM_\Sigma^n(L)$ has the same components as $M$, except:
  \[
    \delta(q, s) :=~ \Let{(q', a) := \delta_M(q,s)}{\bigl(q', \map{(\lambda(w, m).~(w, \MS{swap}~m))}{a} \bigr)}
  \]
\end{definition}

The correctness and termination proofs are similar to the proofs of $\MS{Switch}$ and $\MS{While}$.  The ``lifting'' between configurations of $M$ and
$\MS{Mirror}$ is the injective and involutive function $mirrorConf : \Conf \to \Conf$ that simply mirrors the tapes.

\begin{definition}[Mirror configuration][mirrorConf]
  \label{def:mirrorConf}
  $mirrorConf(q,t) := (q, \map{\MS{mirror}}{t}).$
\end{definition}

\begin{lemma}[Mirroring steps][mirror_step]
  \label{lem:mirror_step}
  Let $c_1 : \Conf$.  Then
  \[ step_M~(mirrorConf~c_1) = mirrorConf~(step_{\MS{Mirror}}~c_1) \]
\end{lemma}

\begin{lemma}[Mirroring executions]
  \label{lem:mirror_loop}
  Let $c_1, c_2 : \Conf$.
  \begin{enumerate}
  \item \label{lem:mirror_lift}
    \coqlink[mirror_lift]{$\MS{Mirror} (c_1) \terminates^k c_2 \rightarrow M (mirrorConf~c_1) \terminates^k (mirrorConf~c_2)$}
  \item \label{lem:mirror_unlift}
    \coqlink[mirror_unlift]{$M (mirrorConf~c_1) \terminates^k (mirrorConf~c_2) \rightarrow \MS{Mirror} (c_1) \terminates^k c_2$}
  \end{enumerate}
\end{lemma}
\begin{proof}
  Claim~\ref{lem:mirror_lift} follows with Lemma~\ref{lem:loop_lift} and Lemma~\ref{lem:mirror_step}.  Claim~\ref{lem:mirror_unlift} follows with
  Lemma~\ref{lem:loop_unlift} and Lemma~\ref{lem:mirror_step}.
\end{proof}

\begin{lemma}[Correctness of $\MS{Mirror}$]
  Let $M \Realise R$.  Then $\MS{Mirror}~M \Realise MirrorRel~R$ with
  \[
    MirrorRel~R := \lambda t~(l, t').~ R~(\map{\MS{mirror}}{t})~(l, \map{\MS{mirror}}{t'})
  \]
\end{lemma}
\begin{proof}
  Follows from Lemma~\ref{lem:mirror_loop}~(\ref{lem:mirror_lift}).
\end{proof}
\begin{lemma}[Running Time of $\MS{Mirror}$]
  Let $M \TerminatesIn T$.  Then $\MS{Mirror}~M \TerminatesIn MirrorT~T$ with
  \[
    MirrorT~T := \lambda t~k.~ T~(\map{\MS{mirror}}{t})~k
  \]
\end{lemma}
\begin{proof}
  Follows from Lemma~\ref{lem:mirror_loop}~(\ref{lem:mirror_unlift}).
\end{proof}

\section{Relabelling Operators}
\label{sec:labelling-op}
\setCoqFilename{ProgrammingTuringMachines.TM.Combinators.Combinators}%

The operators of the above sections of this Chapter modify the behaviour of the machine.  We can also define simple operators that modify the
labelling function $lab : Q_M \to L$.  This is for example useful if we want a machine to terminate in one particular label.

\begin{definition}[$\MS{Relabel}$][Relabel]
  Let $M:\TM_\Sigma^n(L)$ and $g : L \to L'$.
  \[
    \MS{Relabel}~M~g := \left(M, g \circ lab_M\right)
  \]
\end{definition}

\begin{definition}[$\MS{Return}$][Return]
  Let $M:\TM_\Sigma^n(L)$ and $l':L'$.
  \[
    \MS{Return~M~l'} := \MS{Relabel}~M~(\lambda \_.~l')
  \]
\end{definition}

The correctness for these simple operators is obvious.  Note that we do not need lemmas for running time, because Definition~\ref{def:TerminatesIn} of
running time is defined over the bare machine without labelling function.  So the running time lemmas for $M$ also apply for $\MS{Relabel}$ and
$\MS{Return}$.
\begin{lemma}[Correctness of $\MS{Relabel}$ and $\MS{Return}$][Relabel_Realise]
  If $M \Realise R$, then
  \begin{alignat*}{1}
    \MS{Relabel}~M~g & \Realise \bigcup_{l:L} \left( \left(R \at l\right) \att {g(l)} \right) \\
    \MS{Return}~M~l'         & \Realise \Bigl( \bigcup_{l:L} R \at {l} \Bigr) \att {l'}
  \end{alignat*}
\end{lemma}

%%% Local Variables:
%%% TeX-master: "thesis"
%%% End:


\subsection{Representing finite groups}

Finite groups can all be represented with square matrices.


\chapter{The Seifert-Van Kampen theorem}

\begin{thm}
Suppose
\begin{equation*}
\begin{tikzcd}
S \arrow[r,"g"] \arrow[d,swap,"f"] & B \arrow[d,"j"] \\
A \arrow[r,swap,"i"] & X
\end{tikzcd}
\end{equation*}
is a cocartesian square, in which $S$, $A$, and $B$ are assumed to be $0$-connected. Then
\begin{equation*}
\begin{tikzcd}
\pi_1(S) \arrow[r,"g"] \arrow[d,swap,"f"] & \pi_1(B) \arrow[d,"j"] \\
\pi_1(A) \arrow[r,swap,"i"] & \pi_1(X)
\end{tikzcd}
\end{equation*}
is a pushout square of groups.
\end{thm}


\documentclass[a4paper,11pt,article]{memoir}

\usepackage[utf8]{inputenc}
\usepackage{fourier}
\usepackage{amsmath,amssymb}
\usepackage{xstring,ifthen,xcolor}
\usepackage{xspace}

\usepackage{tikz}
\usepackage{pgfmath}
\usetikzlibrary{calc,shapes,positioning}

\title{Mini-Mapper 8:\,Photoencoder signals}
\author{Ian~Ross}

\newdimen\holecircleradius{}
\pgfmathsetlength{\holecircleradius}{10mm}

\newdimen\offsetangle{}

\newcommand{\fulldisk}[1]{%
\pgfmathsetlength{\offsetangle}{#1}
\begin{tikzpicture}[scale=1.75]
  \fill[fill=red]
  (-0.25mm,\the\holecircleradius-1mm) --
  (-0.25mm,\the\holecircleradius+1mm) --
  (0.25mm,\the\holecircleradius+1mm) --
  (0.25mm,\the\holecircleradius-1mm) --
  (-0.25mm,\the\holecircleradius-1mm);

  \draw (0,0) -- (0,\the\holecircleradius+1.5mm);
  \draw (0,0) -- (90+\the\offsetangle:\the\holecircleradius+1.5mm);

  \draw[dashed] (0,0) circle [radius=\the\holecircleradius];

  \foreach \angle in {0,1,...,31}
    \draw (360/32*\angle+90+\the\offsetangle:\the\holecircleradius) circle [radius=0.5mm];
\end{tikzpicture}}

\graphicspath{{figs/}}

\begin{document}

\maketitle

\section*{Introduction}

The idea here is to calculate the form of signal that the
phototransistor in the photointerrupter on the motor encoder disk will
generate as the encoder disk rotates. This isn't strictly necessary,
but it's mildly interesting, and a fun geometrical exercise.
Figure~\ref{fig:setup} shows the basic setup, with the ring of holes
in the encoder disk rotating past the beam of the photointerrupter.

The important factor for the purposes of determining the current
signal generated by the phototransistor in the photointerrupter is the
area of overlap between the hole in the encoder disk and the beam of
infrared radiation emitted by the photodiode in the photointerrupter.

The following parameters are used in the analysis below:
\begin{itemize}
  \item{$d$: the diameter of a single hole in the encoder disk;}
  \item{$D$: the diameter of the circle of centres of the holes in the
    encoder disk;}
  \item{$w$ and $h$: the width and height of the beam from the
    photodiode in the photointerrupter --- for the photointerrupters
    I've been looking at, the beam aperture is rectangular, with the
    narrower dimension horizontal.}
  \item{$\phi$: the angle (anticlockwise positive) between the line
    between the centre of rotation of the encoder disk and the center
    of the photointerrupter beam, and the line betwen the centre of
    rotation of the encoder disk and the centre of the hole under
    consideration.}
\end{itemize}

We make a few assumptions:
\begin{enumerate}
  \item{$d \leq h$ and $d \geq w$: these assumptions give the
    geometrical relations between the optical beam and the hole as
    shown in Figure~\ref{fig:setup}, i.e., the outline of the hole has
    either zero, two or four intersections with the outline of the
    optical beam, with all intersections occurring on the vertical
    sides of the beam.}
  \item{The gaps between the holes in the encoder disk are large
    enough that only one hole can intersect with the optical beam at a
    time.}
\end{enumerate}

\begin{figure}
  \begin{center}
    \fulldisk{0}\hspace{0.5cm}\fulldisk{1}\hspace{0.5cm}\fulldisk{2}
  \end{center}

  \begin{center}
    \fulldisk{3}\hspace{0.5cm}\fulldisk{4}\hspace{0.5cm}\fulldisk{5}
  \end{center}

  \caption{Photointerrupter beam (red, $w = 0.5\,\mathrm{mm} \times h
    = 2\,\mathrm{mm}$) interfacing with holes in motor encoder disk.
    Dashed circle shows line of centres of encoder disk holes
    (diameter $D = 20\,\mathrm{mm}$), with each hole being of diameter
    $d = 1\,\mathrm{mm}$. The images here show the positions of the
    encoder disk $1^\circ$ apart, starting with a hole centrally
    located in front of the photointerrupter beam ($\phi = 0$) and
    showing positions of $\phi = 1^\circ, 2^\circ, 3^\circ, 4^\circ,
    5^\circ$.}\label{fig:setup}
\end{figure}

\section*{Intersections of holes with optical aperture}

Let's think about the overlap of a single hole with the aperture of
the photointerrupter. We'll consider only $\phi \geq 0$ (the situation
for negative $\phi$ is symmetrical). Writing $R = D / 2$ for the
radius of the circle of centres, and $r = d / 2$ for the radius of the
holes, the centre of the hole we're looking at is at $(x_c, y_c)$:
\begin{equation*}
  x_c = -R \sin \phi, \; y_c = R \cos \phi
\end{equation*}
and its equation is
\begin{equation*}
  {(x - x_c)}^2 + {(y - y_c)}^2 = r^2
\end{equation*}
or
\begin{equation*}
  {(x + R \sin \phi)}^2 + {(y - R \cos \phi)}^2 = r^2
\end{equation*}
Points of intersection of this circle with the outline of the optical
beam are given by solving this equation for $y$ when $x = \pm w/2$:
\begin{equation*}
  R^2 \sin^2 \phi \pm w R \sin \phi + w^2/4 + y^2 - 2 R y \cos \phi +
  R^2 \cos^2 \phi - r^2 = 0
\end{equation*}
or, simplifying
\begin{equation}
  \label{eq:intersection}
  y^2 - 2 R y \cos \phi + R^2 - r^2 + w^2/4 \pm w R \sin \phi = 0.
\end{equation}
This discriminant of this equation is
\begin{equation}
  \label{eq:discriminant}
  \Delta_{\pm} = 4 R^2 \cos^2 \phi \mp 4 w R \sin \phi - 4 (R^2 - r^2)
  - w^2.
\end{equation}
Figure~\ref{fig:discriminant} shows values of $\Delta_{\pm}$
from~\eqref{eq:discriminant} as a function of $\phi$. When the hole
extends right across the width of the optical beam aperture, there are
four intersections between the circle defining the outline of the hole
and the outline of the aperture. In this case, $\Delta_{+} \geq 0,
\Delta_{-} \geq 0$. When the outline of the hole intersects with only
one vertical side of the optical beam aperture, then there are two
intersections between the circle and the outline of the aperture and
$\Delta_{+} < 0, \Delta_{-} \geq 0$. When there are no intersections
between the circle and the outline of the aperture then both
discriminants are negative.

The limits in terms of the angular displacement $\phi$ of the ranges
with solutions are given by $\Delta_{\pm} = 0$.
From~\eqref{eq:discriminant}, this gives
\begin{equation*}
  4 R^2 \sin^2 \phi \pm 4 w R \sin \phi - 4 r^2 + w^2 = 0
\end{equation*}
or
\begin{align*}
  \sin \phi &= \frac{\mp 4 w R \pm \sqrt{16 w^2 R^2 - 16 R^2 (w^2 - 4 r^2)}}{8 R^2} \\
  \sin \phi &= \frac{\mp w \pm \sqrt{w^2 - (w^2 - 4 r^2)}}{2 R} \\
  \sin \phi &= \frac{\mp w \pm 2 r}{2 R}
\end{align*}

\begin{equation*}
  \sin \phi = \frac{\mp w/2 \pm r}{R}.
\end{equation*}
The four limiting values for $\phi$ derived from this equation provide
the limits of the two intersection and four intersection cases in the
positive and negative $\phi$ directions. Putting $w =
0.5\,\mathrm{mm}$, $r = 0.5\,\mathrm{mm}$ and $R = 10\,\mathrm{mm}$,
we get limits of $|\phi| \leq 4.30^\circ$ for the two intersections
case, and $|\phi| \leq 1.43^\circ$ for the four intersections case,
which matches with the limits seen in Figure~\ref{fig:discriminant}.

\begin{figure}
  \begin{center}
    \includegraphics[width=\textwidth]{intersection-discriminant}
  \end{center}

  \caption{Discriminant for hole/beam intersection equation as a
    function of angular displacement $\phi$. The boundary of the ``two
    intersections'' region is $\Delta_{-}(\phi)$, while that of the
    ``four intersections'' region is $\Delta_{+}(\phi)$.}\label{fig:discriminant}
\end{figure}

Solutions to the positive and negative versions
of~\eqref{eq:intersection} exist when $\Delta_{\pm} \geq 0$ and are
given by:
\begin{equation}
  \label{eq:solutions}
  y_{\color{black}\pm}^{\color{red}\pm} = R \cos \phi {\color{red}\pm} \frac12
  \sqrt{\Delta_{\pm}} = y_c {\color{red}\pm} \frac12
  \sqrt{\Delta_{\pm}}
\end{equation}
(The sign of the discriminant $\Delta_{\pm}$ matches the sign on the
left hand side. The other plus-or-minus sign is for the usual pair of
solutions to a quadratic equation, and corresponds to the two members
of the pairs of intersection points.) Figure~\ref{fig:solutions} shows
these solutions as a function of $\phi$.

\begin{figure}
  \begin{center}
    \includegraphics[width=\textwidth]{intersection-solutions}
  \end{center}

  \caption{Solutions for hole/beam intersection equation as a function
    of angular displacement $\phi$. The $y_{-}$ values show
    intersections with the left-hand edge of the optical aperture
    (remember positive $\phi$ means movement of the hole to the left
    from top dead centre), and $y_{+}$ values are for intersection
    with the right-hand edge of the aperture. The ``(pos)'' and
    ``(neg)'' labels indicate the two signs in~\eqref{eq:solutions}
    for the two solutions to each quadratic
    equation.}\label{fig:solutions}
\end{figure}

\section*{Hole/aperture overlap area}

Given the intersection information from~\eqref{eq:solutions}, we now
need to calculate the area of the overlap between the encoder disk
hole and the photointerrupter optical aperture. There are three cases
to consider, two requiring calculation and one trivial:
\begin{enumerate}
  \item{There are two intersections between the outline of the hole
    and the optical aperture. In this case, the overlap between the
    hole and aperture is formed by a line segment of the vertical side
    of the aperture and a segment of the circle forming the outline of
    the hole --- this segment lies inside the optical aperture and has
    endpoints given by the intersection points between the hole and
    the vertical side of the aperture (Figure~\ref{fig:overlaps}a).}
  \item{There are four intersections between the outline of the hole
    and the optical aperture. In this case, the overlap is the encoder
    disk hole minus two segments of the circle forming the outline of
    the hole --- these segments lie outside the optical aperture and
    have endpoints given by the intersection points between the hole
    and the vertical sides of the aperture
    (Figure~\ref{fig:overlaps}b).}
  \item{There are no intersections between the outline of the encoder
    disk hole and the optical aperture. In this trivial case, the
    overlap area is identically zero.}
\end{enumerate}
In each of the two cases requiring calculation, the calculation can be
performed on the basis of an expression for the area of a segment of a
circle. For a circle with radius $R$, the area $A$ of a segment
subtending an angle $\theta$ at the centre of the circle is
\begin{equation}
  \label{eq:segment}
  A = \frac{R^2}{2} (\theta - \sin \theta).
\end{equation}

\begin{figure}
  \begin{center}
    \begin{tabular}{cc}
    \includegraphics[height=0.6\textheight]{overlap-2} &
    \includegraphics[height=0.6\textheight]{overlap-4} \\
    (a) Two intersections & (b) Four intersections
    \end{tabular}
  \end{center}

  \caption{Overlap cases: (a) two intersections, so overlap is a
    segment of a circle; (b) four intersections, so overlap is a
    circle minus two segments.}\label{fig:overlaps}
\end{figure}

\subsection*{Case 1: two intersections}

Figure~\ref{fig:overlaps}a shows this case. The intersection points of
the outline of the encoder disk hole and the optical aperture are
$(-w/2, y_{-}^{+})$ and $(-w/2, y_{-}^{-})$. We know the radius of the
encoder disk hole, so in order to apply~\eqref{eq:segment} to
calculate the area of overlap, we need to find the angle $\theta$. To
do this, we construct vectors $\mathbf{v}_{+}$ and $\mathbf{v}_{-}$
between the centre of the encoder disk hole and the intersection
points:
\begin{equation}
  \mathbf{v}_{+} = \begin{pmatrix}
    -w/2 - x_c \\
    y_{-}^{+} - y_c
  \end{pmatrix}, \;\;\;\;\;\;
  \mathbf{v}_{-} = \begin{pmatrix}
    -w/2 - x_c \\
    y_{-}^{-} - y_c
  \end{pmatrix},
\end{equation}
and then
\begin{equation*}
  \cos \theta = \hat{\mathbf{v}}_{+} \cdot \hat{\mathbf{v}}_{-}.
\end{equation*}
(Here $\hat{\mathbf{v}}$ is the unit vector in the direction of vector
$\mathbf{v}$.) First we calculate $\mathbf{v}_{+} \cdot
\mathbf{v}_{-}$, making use of the fact that $y_{-}^{\pm} - y_c = \pm
\frac12 \sqrt{\Delta_{-}}$ (from~\eqref{eq:solutions}):
\begin{equation}
  \mathbf{v}_{+} \cdot \mathbf{v}_{-} = {\left(\frac{w}{2} +
    x_c\right)}^2 + (y_{-}^{+} - y_c) (y_{-}^{-} - y_c) =
         {\left(\frac{w}{2} + x_c\right)}^2 - \frac{\Delta_{-}}{4}.
\end{equation}
Now, we also find that
\begin{align*}
  |\mathbf{v}_{\pm}|^2 = {\left(\frac{w}{2} + x_c\right)}^2 + {(y_{-}^{\pm} -
  y_c)}^2 = {\left(\frac{w}{2} + x_c\right)}^2 + \frac{\Delta_{-}}{4}.
\end{align*}
so that
\begin{equation*}
  |\mathbf{v}_{+}| |\mathbf{v}_{+}| = {\left(\frac{w}{2} +
    x_c\right)}^2 + \frac{\Delta_{-}}{4}
\end{equation*}
and we finally arrive at
\begin{equation*}
  \cos \theta = \frac{\mathbf{v}_{+} \cdot
    \mathbf{v}_{-}}{|\mathbf{v}_{+}| |\mathbf{v}_{+}|} =
  \frac{{\left(\frac{w}{2} + x_c\right)}^2 -
    \frac{\Delta_{-}}{4}}{{\left(\frac{w}{2} + x_c\right)}^2 +
    \frac{\Delta_{-}}{4}}
\end{equation*}
which gives an area for the two intersection case of
\begin{equation*}
  A_2 = \frac{r^2}{2} (\theta - \sin \theta)
\end{equation*}

\subsection*{Case 2: four intersections}

The four intersections case (Figure~\ref{fig:overlaps}b) is similar,
except that we have two angles, $\theta_{-}$ and $\theta_{-}$, one of
each side of the optical aperture, and the area of interest is the
total area of the encoder disk hole minus the area of the two segments
subtended by $\theta_{-}$ and $\theta_{-}$.

The left-hand side of Figure~\ref{fig:overlaps}b can be treated the
same way as for the two intersections case:
\begin{equation}
  \cos \theta_{-} = \frac{\mathbf{w}_{+} \cdot
    \mathbf{w}_{-}}{|\mathbf{w}_{+}| |\mathbf{w}_{+}|} =
  \frac{{\left(\frac{w}{2} + x_c\right)}^2 -
    \frac{\Delta_{-}}{4}}{{\left(\frac{w}{2} + x_c\right)}^2 +
    \frac{\Delta_{-}}{4}}.
\end{equation}
The right-hand side goes the same kind of way, with vectors
$\mathbf{v}_{+}$ and $\mathbf{v}_{-}$ defined as:
\begin{equation}
  \mathbf{v}_{+} = \begin{pmatrix}
    w/2 - x_c \\
    y_{+}^{+} - y_c
  \end{pmatrix}, \;\;\;\;\;\;
  \mathbf{v}_{-} = \begin{pmatrix}
    w/2 - x_c \\
    y_{+}^{-} - y_c
  \end{pmatrix},
\end{equation}
so that
\begin{equation}
  \mathbf{v}_{+} \cdot \mathbf{v}_{-} = {\left(\frac{w}{2} -
    x_c\right)}^2 + (y_{+}^{+} - y_c) (y_{+}^{-} - y_c) =
         {\left(\frac{w}{2} - x_c\right)}^2 - \frac{\Delta_{+}}{4}.
\end{equation}
and
\begin{align*}
  |\mathbf{v}_{\pm}|^2 = {\left(\frac{w}{2} - x_c\right)}^2 + {(y_{+}^{\pm} -
  y_c)}^2 = {\left(\frac{w}{2} - x_c\right)}^2 + \frac{\Delta_{+}}{4}.
\end{align*}
so that
\begin{equation*}
  |\mathbf{v}_{+}| |\mathbf{v}_{+}| = {\left(\frac{w}{2} -
    x_c\right)}^2 + \frac{\Delta_{+}}{4}
\end{equation*}
and we arrive at
\begin{equation*}
  \cos \theta_{+} = \frac{\mathbf{v}_{+} \cdot
    \mathbf{v}_{-}}{|\mathbf{v}_{+}| |\mathbf{v}_{+}|} =
  \frac{{\left(\frac{w}{2} - x_c\right)}^2 -
    \frac{\Delta_{+}}{4}}{{\left(\frac{w}{2} - x_c\right)}^2 +
    \frac{\Delta_{+}}{4}}.
\end{equation*}
The total area in the four intersections case is then:
\begin{equation*}
  A_4 = \pi r^2 - \frac{r^2}{2} (\theta_{+} - \sin \theta_{+}) -
  \frac{r^2}{2} (\theta_{-} - \sin \theta_{-}).
\end{equation*}

\section*{Final results}

The functional dependence of the overlap area between an encoder disk
hole and the optical beam of the interrupter is shown in
Figure~\ref{fig:overlap-areas}, representing the overlap as the
fraction of the photointerrupter optical beam covered by the encoder
hole disk. The assumption is that the current generated by the
phototransistor in the photointerrupter will be proportional to this
fractional overlap.

We can now use these results to generate simulated photocurrent
signals for use in simulations.

\begin{figure}
  \begin{center}
    \includegraphics[width=\textwidth]{overlap-areas}
  \end{center}

  \caption{Dependence of fraction of area of photointerrupter optical
    beam covered by encoder disk hole as a function of angular
    displacement $\phi$, based on $w = 0.5\,\mathrm{mm}$, $r =
    0.5\,\mathrm{mm}$ and $R = 10\,\mathrm{mm}$.}\label{fig:overlap-areas}
\end{figure}

\section*{Generation of simulated signals}

Based on the results so far, we want to be able to generate PWL
(piecewise linear) data files for use in LTSpice simulations. We'll do
this with a Python script. Basically, the script just duplicates the
calculations performed above, and produces a time series of simulated
phototransistor current values, based on a time profile of wheel
velocity. We assume that the phototransistor current is proportional
to the fraction of the optical aperture that is overlapped by the
motor encoder disk hole.

The calculations go something like this, stepping the simulation time
by a constant timestep (I've been using $\Delta t = 0.1\,\mathrm{ms}$):
\begin{enumerate}
  \item{Determine the current linear velocity from the current
    simulation time. The example I've been using starts from rest,
    accelerates to a constant speed, has a constant velocity segment,
    then decelerates back to rest. Acceleration and deceleration are
    both eased using a parabolic easing function.}
  \item{Step the current distance, wheel angle and encoder disk angle
    using simple Euler integration.}
  \item{Normalise the encoder disk angle into the range $|\phi| \leq
    \Delta \phi / 2$, where $\Delta \phi$ is the angular separation
    between adjacent holes in the encoder disk.}
  \item{We can then apply the calculations in the first part of this
    document to the angle $\phi$ to get the area overlap fraction
    between the optical aperture and the hole in the encoder disk.}
  \item{Multiply the area overlap fraction by the maximum light
    current generated by the phototransistor (I'm using
    $I_L(\mathrm{max}) = 2\,\mathrm{mA}$, which is based on the
    ``typical'' value for the Vishay TCST1230 photointerrupter I'm
    planning to use.)}
\end{enumerate}
The results of this procedure are shown in
Figure~\ref{fig:simulation}.

\begin{figure}
  \begin{center}
    \includegraphics[width=\textwidth]{simulation-1}
  \end{center}
  \begin{center}
    \includegraphics[width=\textwidth]{simulation-2}
  \end{center}
  \begin{center}
    \includegraphics[width=\textwidth]{simulation-3}
  \end{center}
  \begin{center}
    \includegraphics[width=\textwidth]{simulation-4}
  \end{center}

  \caption{Simulated phototransistor current based on using a Vishay
    TCST1230 photointerrupter, an encoder disk with 32 holes of 1\,mm
    diameter on a 20\,mm diameter circle, and a linear velocity
    profile starting from rest, increasing to 50\,mm/s in 3 seconds,
    travelling at constant speed for 2 seconds, then decelerating back
    to rest in 3 seconds.}\label{fig:simulation}
\end{figure}

\end{document}


\section{Formulation\label{sec:formulation}}
\subsection{Problem Setup}

We restrict our focus to interior
%\note[DZ]{the pictures usually show an exterior problem?}
%\note[MJM]{added a comment that formulation applies to both problems}
Dirichlet boundary value problems of the form
\begin{align}
  L u(\vx) =  0,& \quad \vx \in \Omega,\\
  u(\vx) =  f(\vx),& \quad \vx \in \partial\Omega = \Gamma,
  \label{eq:pde}
\end{align}
with multiply- or singly-connected domain $\Omega$ of arbitrary genus.
Our approach applies directly to standard integral equation formulations of exterior Dirichlet and Neumann problems; we include results for an exterior Dirichlet problem in \cref{sec:results-torii}.
Here $L$ is a linear elliptic operator and $f$ is at least $C^k$.
While our method can be applied to any non-oscillatory elliptic \pde, we use the following equations in our examples:
\begin{equation}
  Lu =\begin{cases}
    \Delta u & \text{Laplace}\\
    \Delta u  - \nabla p, \quad \nabla \cdot u = 0 & \text{Stokes}\\
    \Delta u  + \frac{1}{1-2\nu}\nabla \nabla\cdot u & \text{Navier (linear elasticity)}\\
  \end{cases}
  \label{eq:pdes}
\end{equation}

We follow the approach of \cite{YBZ}.
We can express  the solution at a point $\vx \in \Omega$ in terms of the double-layer potential
\begin{equation}
  u(\vx) = D[\phi](\vx) = \int_\Gamma \frac{\partial G(\vx,\vy)}{\partial \vn(\vy)} \phi(\vy) d\vy_\Gamma,
  \label{eq:double_layer}
\end{equation}
where $G(\vx,\vy)$ is the \textit{fundamental solution} or \textit{kernel} of \cref{eq:pde}, $\vn(\vy)$ is the normal at $\vy$ on $\Gamma$ pointing into the exterior of $\Omega$, and $\phi$ is an unknown function, or \textit{density}, defined on $\Gamma$.
We list the kernels associated with the \abbrev{PDE}s\xspace in \cref{eq:pdes} in \cite[Section 1]{morse2020bsupplementary}.
Using the jump relations for the interior and exterior limits of $u(\vx)$ as $\vx$ tends towards $\Gamma$ \cite{K,mikhlin2014integral,pozrikidis1992boundary,parton1982integral}, we know that \cref{eq:double_layer} is a solution to \cref{eq:pde} if $\phi$ satisfies
\begin{equation}
  \left(\frac{1}{2}I + D + M\right)[\phi](\vx) = f(\vx), \vx \in \Gamma
  \label{eq:int-eq}
\end{equation}
with identity operator $I$.
We will refer to $\phi$ as the \textit{density} and $u(\vx)$ as the \textit{potential} at $\vx$.
The double-layer integrals in this equation are \textit{singular}, due to the singularity in the integrand of \cref{eq:double_layer}.
Additionally, as $\vx$ approaches $\Gamma$, \cref{eq:double_layer} becomes a \textit{nearly singular} integral.

The operator $M$ completes the rank of $\frac{1}{2}I + D$ to ensure invertibility of \cref{eq:int-eq}.
If $\frac{1}{2}I + D$ is full-rank, $M = 0$.
When $\frac{1}{2}I + D$ has a non-trivial null space, $M$ accounts for the additional constraints to complete the rank of the left-hand side of \cref{eq:int-eq}.
For example, for the exterior Laplace problem on $\ell$ multiply-connected domains, the null space of $\frac{1}{2}I + D$ has dimension $\ell$ \cite{ST}.
The full set of cases for each kernel is considered in this work and their corresponding values of $M$ have been detailed in \cite{YBZ}.

%We assume that $f(\vx)$ is $C^k$ on $\Gamma$, since the convergence order of our integration scheme depends on smoothness of $f$.
%This implies that the convergence order is at most $k$ (see \cref{sec:quad_error}).

\subsection{Geometry representation \label{sec:geom-def}}

\begin{figure}[!htb]
  \centering
  %\setlength\figureheight{1.9in}
  %\setlength\figurewidth{2.1in}
  \begin{minipage}{\textwidth}
      \includegraphics[width=\linewidth]{figs/quadrisection.pdf}
  \end{minipage}\hfill
  \mcaption{fig:geometry-representation}{Patch Quadrisection}{
      Right: the standard domain $\mathcal{I}^2$ of a single surface or quadrature patch.
    Middle: a collection of subdomains $\mathcal{D}_i$ of $E_r$, produced by quadrisection.
    Each $\mathcal{D}_i$ corresponds to a map $\eta_i$ such that $\mathcal{D}_i = \eta_i(\mathcal{I}^2)$; a single $\mathcal{D}_i$ is highlighted in bold.
    Left: the image of $E_r$ under the patch $\gamma_r$.
    The final image of each subdomain is outlined, with the image of $\mathcal{D}_i$ in bold.
  }
\end{figure}
We assume that the smooth domain boundary $\Gamma$ is given by a \textit{quadrilateral mesh} consisting of quadrilateral faces $Q_r$, referred to as \textit{quads}.
Each quad is associated with a parametric domain $\I^2 =  [-1,1]^2 = E_r$, along with embeddings $\gamma_r : E_r \to \mathbb{R}^3$ for each quad such that $Q_r = \gamma_r(E_r)$.
We assume that the quad mesh is \textit{conforming}, i.e., two non-disjoint faces either share a whole edge or a single vertex; examples of this are shown in \Cref{fig:greens-id-test-cases,fig:solver-conv-test-cases}.
We assume that no two images $\gamma_r(E_r)$ intersect, except along the shared edge or vertex.
The surface $\Gamma$ is the union of patches $\cup_r \gamma_r(E_r) = \cup_r Q_r$.
We also assume that $\Gamma$ is sufficiently smooth to recover the solution of \cref{eq:pde} up to the boundary \cite{K} and is at least $C^k$.

To represent the surface geometry, we approximate $\Gamma$ with a collection of \emph{B\'ezier patches}, given by a linear combination of tensor-product Bernstein polynomials
\begin{equation}
    \vP_i(s,t) = \sum_{\ell =0}^n\sum_{m =0}^n \vector{a}^{(i)}_{\ell m }B_\ell^n(s)B_m^n(t),
  \label{eq:tensor-product}
\end{equation}
where  $B_\ell^n(t) = \binom{n}{\ell} t^{n-\ell}(1-t)^{\ell}$ for each $\ell$ are the $n$-th degree Bernstein polynomials, $i$ denotes the index of a patch in the collection and $\vector{a}_{\ell m}^{(i)} \in \mathbb{R}^3$.
Each patch $\vP$ is a vector function from $\I^2$ to $\mathbb{R}^3$, so $s,t\in [-1,1]$.
We will refer to this approximation of $\Gamma$ as $\hat{\Gamma}$. % and detail its construction in \cref{sec:admissible}.
\edit{}{This representation is advantageous because the B\'ezier coefficients provide a direct connection to surface geometry, as shown in \cref{fig:patch-coeffs-bbox}.}
%More specifically, we use a \emph{forest of quad trees of B\'ezier patches}.

The domain $E_r$ of each embedding function $\gamma_r$ is adaptively refined using \emph{quadrisection}, i.e., splitting a square domain into four square subdomains of equal size. %This yields a quad tree of subdomains for each face of the quad mesh.
Quadrisection induces a \textit{quadtree} structure on each $E_r$.
The root of the quadtree is the original domain $\I^2$ and each node of the tree is related by a single quadrisection of a subdomain of $E_r$.
The leaves of the quadtree form a collection of subdomains $\mathcal{D}_i$ whose union equals $E_r$, as shown in \cref{fig:geometry-representation}-middle.
Given an indexing scheme of all $\mathcal{D}_i$'s over all $E_r$'s, we define the function $r(i)$ that maps the leaf node index $i$ to its root node index $r$ in the quadtree forest, indicating that $\mathcal{D}_i \subset E_r$.
For each $r$, $E_r$ can have a distinct sequence of associated quadrisections and therefore a distinct quadtree structure.
We refer to the process of \textit{refinement} or \textit{refining a patch $\vP$} as the construction of such quadtrees for each $E_r$ subject to some set of criteria.

On each $\mathcal{D}_i$ at the quadtree leaves, we define a B\'ezier patch and reparametrize each patch over $\I^2$ by defining the affine map $\eta_i: \I^2 \to E_{r(i)}$ such that $\eta_i(\I^2) = \mathcal{D}_i \subseteq E_{r(i)}$.
%On each $\mathcal{D}_i$ at the quadtree leaves, we define a \emph{B\'ezier patch} given by a linear combination of  tensor-product Bernstein polynomials on $\mathcal{D}_i$:
%\begin{equation}
%    P(s,t) = \sum_{\ell =0}^n\sum_{m =0}^n \vector{a}_{\ell m }B_\ell^n(s)B_m^n(t),
%  \label{eq:tensor-product}
%\end{equation}
%where  $B_k^n(t) = \binom{n}{k} t^{n-k}(1-t)^{k}$ is $n$-th degree Bernstein polynomials.
It follows that the set of subdomains $\{ \eta_i(\I^2) \,| \,r(i) = \kappa\}$ form a cover of $E_\kappa$ and $\{ \gamma_\kappa(\eta_i(\I^2))\, | \,r(i) = \kappa\}$ likewise covers $\gamma_\kappa(E_\kappa)$.
We summarize this setup in \Cref{fig:geometry-representation}; examples of surfaces of this form can be seen in \Cref{fig:greens-id-test-cases,fig:solver-conv-test-cases,fig:torii,fig:vessel}.
%Since we have made no assumptions on the explicit form of $\gamma_r$, this approximation stage constructs a consistent surface representation that we can leverge in our algorithms in \cref{sec:admissible} without sacrificing geometric fidelity.
%At each step of quadrisection, there is a parent-child relationship formed between the subdomain and its four resulting subdomains.
%In \Cref{fig:geometry-representation}-right, we have a single leaf of this quadtree, denoted $D_i$.

%The coefficients or \emph{control points} $\vector{a}_{\ell m}$ in \cref{eq:tensor-product} are computed to fit the B\'ezier patches to the input embeddings $\gamma_r$.
%Each patch $P_i$ in $\Pcoarse$ is reparametrized on $\I^2$ and each domain is associated with a leaf of the forest of quad trees.
%We refer to the domain of $P_i$ by $D_{i}$, noting that $D_i = \I^2$.
%Each such domain $D_i$ corresponds to a subdomain of $E_{r(i)}$, where $r(i)$ is the index of the embedding $\gamma_{r(i)}$ from which
%$P_i$ was obtained by refinement.
%Define the  map $\eta_i: D_i \rightarrow E_{r(i)}$, which embeds the domain $D_i$ into the copy of $\I^2$ corresponding to $\gamma_{r(i)}$.
%The set of maps $\{\eta_i \mid r(i) = k\}$ cover $\I^2$ for each embedding map $\gamma_k$.
%We summarize this setup in \cref{fig:geometry-representation}.

%In the simplest case, the input is already in B\'ezier form and no additional processing is necessary; the overall accuracy of our method is limited by therefore by the smoothness of the given polynomial surface.
%In general, $\Gamma$ can be defined by arbitrary functions.
%The domain on which the complete approximate surface $\hat{\Gamma}$ is defined is the union of $E_{r(i)}$ identified along shared edges.
%The complete set of B\'ezier patches defined on these domains is denoted $\Pcoarse$.

\subsection{Problem discretization \label{sec:discretization}}
We use two collections of patches in the form described above: $\Pcoarse$ and $\Pfine$.
The patches in $\Pcoarse$, called \emph{surface patches}, determine $\Gammah$ from $\Gamma$ and the set of patches $\Pfine$, called \emph{quadrature patches}, are obtained by further quadrisection of the surface patches in $\Pcoarse$.
The geometry of $\Gammah$ is not changed by this additional refinement of $\Pcoarse$, but the total number of subdomains $E_{r(i)}$ is increased.
We will detail the geometric criteria that $\Pcoarse$ and $\Pfine$ must satisfy in \cref{sec:geom_criteria}.
Discretizing $\Gammah$ with with a quadrature rule based on $\Pfine$ results in a denser sampling of $\Gammah$ than a similar discretization of $\Pcoarse$.
We will refer to $\Pcoarse$ as the \textit{coarse discretization} of $\hat{\Gamma}$ and $\Pfine$ as the \textit{upsampled} or \textit{fine discretization} of $\hat{\Gamma}$.

We index the patches in $\vP_i \in \Pcoarse$ by $i = 1,\hdots N$; we can then rewrite \cref{eq:double_layer} as a sum of integrals over surface patches:
\begin{equation}
  u(\vx) = \sum_{i=1}^N\int_{\vP_i} \frac{\partial G(\vx,\vy)}{\partial \vn(\vy)} \phi(\vy) d\vy_{\vP_i}.
  \label{eq:double_layer_patches}
\end{equation}

%The coefficients $a_{ij}$ of $P(u,v)$, or \textit{control points}, have geometric relevance: the convex hull of the control points contains $P$, yielding a simple algorithm for computing a bounding box for a patch image.

%The control points of B\'ezier patches also admit an efficient algorithm to compute control points on subdomains of the domain of $P$ via a \emph{subdivision} \cite{F}. This fact is used in \cref{sec:adaptive_upsampling,sec:mark_near}. Subdivision is used for two purposes: obtain an accurate surface approximation with B\'ezier patches, and obtain admissible \emph{quadrature patches} from surface patches, as explained below.

%In the case of surfaces specified by a black-box evaluator on the initial quad mesh,  we fit tensor-product B\'ezier patches to each $\gamma_i$, adaptively subdividing each original patch as necessary, until a desired absolute error in function values is achieved. In the former case, the accuracy of the solution is restricted by the accuracy of the given surface approximation.

%Ideally, each surface patch is defined as in \cref{eq:tensor-product}; however, we do not assume that this is the case in practice.
%If surface patches are defined in another tensor-product polynomial basis or splines, one can perform a change of variables to express them in the form of \cref{eq:tensor-product}.

%An important property of the method presented in this work is that we make minimal assumptions on the input geometry definition and boundary data; the accuracy our method can achieve is ultimately bounded by their smoothness.

We discretize functions defined on $\Gammah$, such as
\cref{eq:double_layer_patches}, at $q$-node composite tensor-product
Clenshaw-Curtis quadrature points on $\I^2$ of patches in $\Pcoarse$.
We refer to these points and weights on a single patch $\vP_i$ as $x_j$ and $w_j^{\lbl{CC}}$ respectively, for $j = 1\ldots q^2$.
%The weights are given by $w_j = \sqrt{g_j}w_j^{\lbl{CC}}$, with $g_j$ being the determinant of the metric tensor of $\vP$ at $x_j$ and $w_j^{\lbl{CC}}$ is the Clenshaw-Curtis weight at $x_j$.
The quadrature point $\vy_{ij}$ from $\vP_i$ is defined as $\vy_{ij} = \vP_{i}(\eta_i(x_j))$.
We assume that the boundary condition $f$ is given by a black-box evaluator on $\mathbb{R}^3$ that can be used to obtain values at $\vy_{ij}$.
For clarity, we reindex the surface points by a global index $I = 1, \hdots, q^2N$.
%For the remainder of this work, we will suppress the explicit dependence of $P_i$ on the parametrization $\eta_i$. \note[MJM]{check that this is needed}
We discretize the double layer integral \cref{eq:double_layer_patches} on $\Pcoarse$ to approximate the solution $u(\vx)$:
%\begin{equation}
%  \left(\frac{1}{2}I + \hat{D} + \hat{M}\right)[\phi](\vy_\ell) = f(\vy_\ell), \quad \ell=1,\hdots N
%  \label{eq:int-eq-disc}
%\end{equation}
%with the discretized double layer operator $\hat{D}[\phi](\vy)$ defined as
\begin{equation}
    u(\vx,\Pcoarse) \approx \hat{u}(\vx,\Pcoarse) = \sum_{i=1}^N\sum_{j=1}^{q^2} \frac{\partial G(\vx,\vy_{ij})}{\partial \vn(\vy_{ij})} \phi_{ij} \sqrt{g_{ij}}w_j^{\lbl{CC}}
    = \sum_{I=1}^{q^2N}\frac{\partial G(\vx,\vy_I)}{\partial \vn(\vy_I)} \phi_I \hat{w}_I
  \label{eq:double_layer_disc}
  %\label{eq:double_layer_disc}
\end{equation}
with $g_{ij}$ being the determinant of the metric tensor of $\vP_i$ at $x_j$ and $\hat{w}_{i\cdot q^2+j} = \sqrt{g_{ij}}w_{j}^{\lbl{CC}}$.
In other words, $\hat{u}(\vx, \Pcoarse) = \hat{D}[\phi](\vx)$, where $\hat{D}[\phi](\vx) \approx D[\phi](\vx)$.

We can also discretize functions with tensor-product Clenshaw-Curtis nodes on the domains of patches in $\Pfine$.
The values of functions on $\Pfine$ are \emph{interpolated} from their values on the quadrature nodes of $\Pcoarse$ rather than being computed directly on $\Pfine$.
We call this interpolation from $\Pcoarse$ to $\Pfine$ \textit{upsampling}.
We denote the quadrature nodes and weights on $\Pfine$ by $\tilde{x}_j$ and $\tilde{w}_j$ with a similar global index $J$ and refer to them as the \textit{upsampled} nodes and weights.
Identical formulas are used for computing quadrature on $\Pfine$ with the nodes and weights $\tilde{x}_j$, $\tilde{w}_j$ on $\Pfine$, denoted $u(\vx,\Pfine)$ and $\hat{u}(\vx,\Pfine)$, repsectively.

In the next section, we describe the algorithm to compute an accurate approximation to the singular/near-singular double-layer integral in \cref{eq:double_layer}, using a quadrature rule for smooth functions (\cref{eq:double_layer_disc}) as a building block.
This algorithm allows us to compute the matrix-vector products $A\phi$, for a vector of values $\phi$ defined at the quadrature points $\vy_I$, where $A$ is the discrete operator obtained from the left-hand side of \cref{eq:int-eq} after approximating $D[\phi](\vy)$ with the singular integration scheme.
As a result, we can solve the linear system using \gmres, which only requires a matrix-vector product
\begin{equation}
  A\phi = f,
  \label{eq:linear_system}
\end{equation}
where $f$ is the boundary condition sampled at the points $\vy_I$. The evaluation of these integrals is accelerated in a standard manner using the fast multipole method (\fmm)\cite{MB,ying2004kernel,greengard1987fast}.
%The index in the sum in \cref{eq:double_layer_disc} ranges over all quadrature nodes on all quadrature patches.
%This results in a linear system for the values of the density at the quadrature nodes on the boundary:
%\begin{equation}
%  A\phi = f, \quad A_{ij} = \frac{\delta_{ij}}{2} + \frac{\partial G(\vy_i, \vy_j)}{\partial \vn(\vy_j)} + M_{ij},
%  \label{eq:linear_system}
%\end{equation}
%where $\delta_{ij}$ is the Kronecker delta. \note[DZ]{need to define $M_{ij}$.}
%$A$ is dense, but we can apply $A$ to a vector in linear time,  due to the low-rank structure of off-diagonal blocks, using fast-multipole method (\fmm). The integral equation is well-conditioned and
%in conjunction with GMRES to solve \cref{eq:linear_system}.
%However, the entries of this system matrix become singular as they approach the diagonal; singular quadrature rules are need to compute these values accurately.


\subsection{Multi-valued logic}

We can have logic with more than two states.


% \tableofcontents
\cleardoublepage
\pagenumbering{roman}
  \maketitle
\tableofcontents
$\: $
\vspace{1cm}
\newpage
\setlength{\epigraphwidth}{\textwidth}
\epigraph{\it Perhaps I can best describe my experience of doing mathematics in terms of a journey through a dark unexplored mansion. You enter the first room of the mansion and it’s completely dark. You stumble around bumping into the furniture, but gradually you learn where each piece of furniture is. Finally after six months or so, you find the light switch, you turn it on, and suddenly it’s all illuminated. You can see exactly where you were. Then you move into the next room and spend another six months in the dark. So each of these breakthroughs, while sometimes they’re momentary, sometimes over a period of a day or two, they are the culmination of—and couldn’t exist without—the many months of stumbling around in the dark that precede them.}{Andrew Wiles}
\vspace{2cm}

\section*{Introduction}
In this class, our goal is to understand how the simple operation of adding multi-digit numbers using the ``carrying process'' is really an example of a much general structure, called a group extension, which in turn is related to group cohomology.

We will start with the very concrete world of arithmetic, and gradually increase the level of abstraction and eventually define some form of group cohomology.
We will mostly work with abelian groups, and time permitting will switch to non-abelian groups toward the end.\\\\

Today, we want to understand how the following are equivalent to each other.
\begin{mdframed}
  \adjustbox{scale=1,center}{%
    \begin{tikzcd}
      &\mbox{Adding 2-digit numbers}
        \ar[ddddl, leftrightarrow]
        \ar[ddddr, leftrightarrow, end anchor={[xshift=0ex]}]
        &
      \\\\\\\\
      \mbox{2-cocycle condition}\ar[rr, dashed, leftrightarrow]& & \text{Group extensions}
    \end{tikzcd}
  }
\end{mdframed}

\cleardoublepage
\pagenumbering{arabic}


\chapterimage{head2.png} % Chapter heading image
\chapter{Bayesian Inference Framework}
\section{Graphical Model}
\begin{definition}[$(X(t), Y(t))$]
We denote a particular configuration as $(X(t), Y(t))$
\begin{equation}
        (X(t), Y(t))=(x_{t_0}, x_{t_1}, \cdots, x_{t_{N_p}}, y_{t_1}, \cdots, y_{t_{N_p}})
\end{equation}
\end{definition}

\begin{definition}[The complete likelihood function]
The complete likelihood function is the joint probability distribution
\begin{equation}
        P(X(t), Y(t); \bm{\theta})= p(x_{t_0})\prod_{t=1}^{N_p} p(x_{t_{\tau}}|x_{t_{\tau-1}}) p(y_{t_{\tau}}|x_{t_{\tau}})
\end{equation}
where $\bm{\theta}=\{F(x), D_1,D_2,...,D_{N_v}\}$
\begin{center}
        \includegraphics[scale=0.8]{ch2/graphical_model.pdf}
\end{center}
\end{definition}

\section{Define Alpha and Beta}
\begin{definition}[$p(x_t|Y(t))$]
\begin{equation}
        p(x_t|Y) = \frac{P(Y|x_t)p(x_t)}{P(Y)}
\end{equation}
where we use Bayes theorem $P(A|B)=\frac{P(B|A)P(A)}{P(B)}$. And we can further decompose $P(Y|x_t)$
\begin{align*}
        P(Y|x_t) &= p(y_1,...,y_t|x_t)p(y_{t+1},...,y_{N_p}|x_t) \\
        p(x_t) &= p_{\text{eq}}(x) = \sqrt{p_{\text{eq}}(x)}\sqrt{p_{\text{eq}}(x)}
\end{align*}
Therefore, we get
\begin{equation}
        p(x_t|Y) = \frac{p(y_1,...,y_t|x_t)p(y_{t+1},...,y_{N_p}|x_t) p_{\text{eq}}(x)}{P(Y)}
\end{equation}
\end{definition}

\begin{definition}[$\alpha(x_t)$]
\begin{align}
        \alpha(x_t) &= p(y_1,...,y_t|x_t)\sqrt{p_{\text{eq}}(x)} = \left< \alpha_t | x_t \right> \\
        \left< \alpha_{t_0} | x_{t_0} \right> & = \rho_{\text{eq}}(x_{t_0}) = \psi_1(x_{t_0})
\end{align}
\end{definition}

\begin{definition}[$\beta(x_t)$]
\begin{align}
        \beta(x_t) &= p(y_{t+1},...,y_{N_p}|x_t)\sqrt{p_{\text{eq}}(x)} = \left< x_t | \beta_t \right> \\
        \left< x_t | \beta_{t_{N_p}} \right> &= \rho_{\text{eq}}(x_{t})= \psi_1(x_{t})
\end{align}
\end{definition}

\section{Time Propagation}
\begin{definition}[Fokker-Planck Equation]
\begin{equation}
        \frac{\partial \rho(x,t)}{\partial t} = -\textbf{H} \rho(x,t)
\end{equation}
where $\rho(x,t)=\frac{p(x,t)}{\sqrt{p_{\rm{eq}}(x)}}$, and
\begin{align}
        \textbf{H} &= -D \frac{\partial^2}{\partial x^2} + \frac{1}{2}D\frac{d F(x)}{dx} + \frac{1}{4} D F^{2}(x) \\
        p_{\rm{eq}}(x) &= \exp{(-V(x))} \\
        F(x) &= - \frac{dV(x)}{dx}
\end{align}
Furthermore, we can get the eigenbasis $\{\psi_i(x)\}$
\begin{equation}
        \textbf{H} \left| \psi_i \right> = \lambda_i \left| \psi_i \right>
\end{equation}
And the formal solution of this Hermitian PDE can be written as:
\begin{equation}
        \rho(x,\Delta t) = e^{-\textbf{H} \Delta t} \rho(x,0) = \bm{\Psi} e^{-\bm{\Lambda} \Delta t}  \bm{\Psi}^{\dagger} \rho(x,0)
\end{equation}
If $\rho(x,0)$ can be expressed by eigenbasis:
\begin{equation}
        \rho(x,0) = a_1 \psi_1(x)+\cdots+a_{N_v} \psi_{N_v}(x)
\end{equation}
$\rho(x,\Delta t)$ becomes
\begin{equation}
        \rho(x,\Delta t) = a_1  e^{(-\lambda_{1}\Delta t)} \psi_1(x)+ \cdots + a_{N_v} e^{(-\lambda_{N_v}\Delta t)}\psi_{N_v}(x)
\end{equation}
\end{definition}

\section{Photon Operator}
\begin{definition}[Build Photon Matrix $\left<\psi_i|\textbf{y}|\psi_j\right>$]
\begin{center}
        \includegraphics[scale=0.4]{ch2/build_photon_mat.pdf}
\end{center}
\end{definition}

\section{Forward algorithm}
\begin{definition}[$\left< \alpha_{t_{\tau}} \right|$]
\begin{align}
        \left< \alpha_{t_{\tau}} \right| &= \left< \alpha_{t_0} \right|e^{-\textbf{H}\Delta t}\textbf{y}_1 ... e^{-\textbf{H}\Delta t} \textbf{y}_{\tau} \\
        \left< \alpha_{t_{\tau}} \right| &= \left< \alpha_{t_{\tau-1}} \right|e^{-\textbf{H}\Delta t} \textbf{y}_{\tau}
\end{align}
\begin{center}
        \includegraphics[scale=0.75]{ch2/alpha_example.pdf}
\end{center}
\end{definition}

\begin{definition}[$P(Y(t))$]
\begin{equation}
        P(Y(t))= \langle \alpha_{t_0} |e^{-\textbf{H}\Delta t}\textbf{y}_1 ... e^{-\textbf{H}\Delta t} \textbf{y}_{t_{N_p}} | \beta_{t_{N_p}} \rangle = \left< \alpha_{t_{N_p}} | \beta_{t_{N_p}} \right>
\end{equation}
\begin{align}
        P(Y(t)) &= \int \left< \alpha_{t_{N_p}} | x_{N_p} \right> \left< x_{N_p}| \beta_{t_{N_p}} \right> dx_{N_p} \\
        &= \int \rho_{\text{eq}}(x_{N_p}) p(y_1,...,y_{N_p}|x_{N_p}) \rho_{\text{eq}}(x_{N_p}) dx_{N_p} \\
        &= \int p(y_1,...,y_{N_p}|x_{N_p}) p_{\text{eq}}(x_{N_p}) dx_{N_p}
\end{align}
\end{definition}

\begin{definition}[Scaling Factor: $p(y_1)$]
\begin{align*}
        p(y_1) &= \left<\alpha_{t_1}|\psi_1\right> = \int \rho_{\text{eq}}(x_{1}) p(y_1|x_1) \rho_{\text{eq}}(x_{1}) dx_1 \\
        &= \int p(y_1|x_1) p_{\text{eq}}(x_{1}) dx_1
\end{align*}
\begin{align}
        \left< \hat{\alpha}_{t_1} \right| &= \frac{\left< \alpha_{t_1} \right|}{\left<\alpha_{t_1}|\psi_1\right>} \\
        \left< \hat{\alpha}_{t_1} | x_1 \right> &= \frac{\rho_{\text{eq}}(x_{1}) p(y_1|x_1)}{p(y_1)}
\end{align}
\end{definition}

\begin{definition}[Scaling Factor: $p(y_t|y_1,...,y_{t-1})$]
\begin{align*}
        &\left< \hat{\alpha}_{t_{\tau-1}} | e^{-\textbf{H}\Delta t}\textbf{y}_\tau | \psi_1 \right> \\
        &=  \int \rho_{\text{eq}}(x_{\tau-1}) p(y_1,...,y_{\tau-1}|x_{\tau-1}) p(x_{\tau}|x_{\tau-1}) p(y_{\tau}|x_{\tau}) \rho_{\text{eq}}(x_{\tau-1}) dx_{\tau-1} dx_{\tau} \\
        &= \int p_{\text{eq}}(x_{\tau-1}) p(y_1,...,y_{\tau-1}|x_{\tau-1}) p(x_{\tau}|x_{\tau-1}) p(y_{\tau}|x_{\tau}) dx_{\tau-1} dx_{\tau} \\
        &= \int p(y_1,...,y_{\tau-1},x_{\tau-1}) p(x_{\tau}|x_{\tau-1}) p(y_{\tau}|x_{\tau}) dx_{\tau-1} dx_{\tau} \\
        &= \int p(y_1,...,y_{\tau-1},y_{\tau}, x_{\tau-1}, x_{\tau}) dx_{\tau-1} dx_{\tau} \\
        &= \frac{p(y_1,...,y_{\tau-1},y_{\tau})}{p(y_1,...,y_{\tau-1})} \\
        &= p(y_{\tau}|y_1,...,y_{\tau-1}) = c_{\tau}
\end{align*}
\end{definition}

\begin{definition}[$\left< \hat{\alpha}_{t_\tau} \right|$]
\begin{equation}
        \left< \hat{\alpha}_{t_\tau} \right| = \frac{\left< \hat{\alpha}_{t_{\tau-1}} \right| e^{-\textbf{H}\Delta t}\textbf{y}_{\tau} }{\left< \hat{\alpha}_{t_{\tau-1}} | e^{-\textbf{H}\Delta t}\textbf{y}_{\tau} | \psi_1 \right>}
\end{equation}
\end{definition}

\section{Backward algorithm}
\begin{definition}[$\left| \beta_{t_\tau} \right> $]
\begin{align}
        \left| \beta_{t_\tau} \right> &= e^{-\textbf{H}\Delta t} \textbf{y}_{\tau+1} ... e^{-\textbf{H}\Delta t} \textbf{y}_{N_p} \left| \beta_{t_{N_p}} \right> \\
        \left| \beta_{t_{N_p-1}} \right> &= e^{-\textbf{H}\Delta t} \textbf{y}_{N_p} \left| \beta_{t_{N_p}} \right> \\
        \left| \beta_{t_{\tau}} \right> &= e^{-\textbf{H}\Delta t} \textbf{y}_{\tau+1} \left| \beta_{t_{\tau+1}} \right>
\end{align}
\begin{center}
        \includegraphics[scale=0.6]{ch2/beta_example.pdf}
\end{center}
\end{definition}

\begin{definition}[$\left| \hat{\beta}_{t_\tau} \right> $]
\begin{equation}
        \left| \hat{\beta}_{t_{\tau}} \right> = \frac{e^{-\textbf{H}\Delta t} \textbf{y}_{\tau+1} \left| \beta_{t_{\tau+1}} \right>}{c_{\tau}}
\end{equation}
\end{definition}

\section{Likelihood Function}
\begin{definition}[Simple Example: Algebra]
The likelihood for the following example is
\begin{equation}\label{eq:likelihood}
    P(Y(t);\bm{\theta}) = L(\bm{\theta})= \langle \alpha_{t_0} |e^{-\textbf{H}\Delta t}\textbf{y}_1 e^{-\textbf{H}\Delta t} \textbf{y}_2 | \beta_{t_2} \rangle
\end{equation}
\begin{center}
    \includegraphics[scale=0.8]{ch2/graphical_model_simple_example.pdf}
\end{center}
We insert $\textbf{1}=\sum_i |\psi_i\left>\right<\psi_i|$ in between each $e^{-\textbf{H}\Delta t}$ in the eq.(\ref{eq:likelihood}),
\begin{align}
    P(Y(t)) &= \sum_{\{i,j,m,l\}}\langle \alpha_{t_0} |\psi_i\left>\right<\psi_i| e^{-\textbf{H}\Delta t_1} |\psi_j\left>\right<\psi_j| \textbf{y}_1 |\psi_m\left>\right<\psi_m| e^{-\textbf{H}\Delta t_2} |\psi_l\left>\right<\psi_l| \textbf{y}_2 | \beta_{t_2} \rangle \\
    &= \sum_{\{i,j,m,l\}} P(X(t),Y(t))
\end{align}
where
\begin{equation}
    P(X(t),Y(t)) = \langle \alpha_{t_0} |\psi_i\left>\right<\psi_i| e^{-\textbf{H}\Delta t_1} |\psi_j\left>\right<\psi_j| \textbf{y}_1 |\psi_m\left>\right<\psi_m| e^{-\textbf{H}\Delta t_2} |\psi_l\left>\right<\psi_l| \textbf{y}_2 | \beta_{t_{2}} \rangle
\end{equation}
and we then take a derivative with respect to $\theta$
\begin{align*}
    &\frac{\delta P(X,Y)}{\delta \theta} = \left<\alpha_{t_0} |\psi_i \right> \frac{\delta \left<\psi_i| e^{-\textbf{H}\Delta t_1} |\psi_j \right>}{\delta \theta} \left<\psi_j| \textbf{y}_1 |\psi_m \right> \left<\psi_m| e^{-\textbf{H}\Delta t_2} |\psi_l \right> \left<\psi_l| \textbf{y}_2 | \beta_{t_{2}} \right> \\
    &+ \left<\alpha_{t_0} |\psi_i \right> \left<\psi_i| e^{-\textbf{H}\Delta t_1} |\psi_j \right> \left<\psi_j| \textbf{y}_1 |\psi_m \right> \frac{\delta \left<\psi_m| e^{-\textbf{H}\Delta t_2} |\psi_l \right>}{\delta \theta} \left<\psi_l| \textbf{y}_2 | \beta_{t_{2}} \right>
\end{align*}
therefore
\begin{equation}
    \frac{\delta P(Y)}{\delta \theta} = \sum_{\{i,j,m,l\}} \frac{\delta P(X,Y)}{\delta \theta} = \sum_{\tau=1}^{2} \left<\alpha^k_{t_{\tau-1}} \right| \frac{\delta e^{-\textbf{H}\Delta t_{\tau}}}{\delta \theta} \textbf{y}_{\tau} \left| \beta^k_{t_{\tau}}\right>
\end{equation}
and
\begin{equation}
    \frac{\delta \ln{P(Y)}}{\delta \theta} = \frac{1}{P(Y;\theta^k)}\frac{\delta P(Y)}{\delta \theta} = \frac{1}{P(Y;\theta^k)} \sum_{\tau=1}^{2} \left<\alpha^k_{t_{\tau-1}} \right| \frac{\delta e^{-\textbf{H}\Delta t_{\tau}}}{\delta \theta} \textbf{y}_{\tau} \left| \beta^k_{t_{\tau}}\right>
\end{equation}
As you can see, $Q$ appear as
\begin{equation}
    Q(\theta) = \frac{1}{P(Y;\theta^k)} \sum_{\tau=1}^{2} \left<\alpha^k_{t_{\tau-1}} | e^{-\textbf{H}\Delta t_{\tau}} \textbf{y}_{\tau} | \beta^k_{t_{\tau}}\right>
\end{equation}
\end{definition}

\begin{definition}[General Case: $Q$]
Because
\begin{align}
        \left< \hat{\alpha}_t \right| &= \frac{\left< \alpha_t \right|}{p(y_1,...,y_t)} \\
        \left| \hat{\beta}_t \right> &= \frac{\left| \beta_t \right>}{p(y_t,...,y_{N_p}|y_1,...,y_t)}
\end{align}
so
\begin{equation}
        Q(\theta) = \sum_{t=1}^{N_p} \frac{\left<\hat{\alpha}^k_{t-1} | e^{-\textbf{H}\Delta t} \textbf{y}_{t} | \hat{\beta}_{t}\right> }{c_t}
\end{equation}
because
\begin{equation}
        p(y_1,...,y_{t-1})p(y_t|y_1,...,y_{t-1})p(y_t,...,y_{N_p}|y_1,...,y_t) = P(Y(t))
\end{equation}
\end{definition}

\section{The decomposition of likelihood function}
\begin{definition}[$\left< \hat{\alpha}_{t} | \hat{\beta}_{t}\right>$]
\begin{equation}
        \left< \hat{\alpha}_{t} | \hat{\beta}_{t}\right> = \frac{\left< \alpha_{t} | \beta_{t}\right>}{P(Y)} = 1
\end{equation}
where $\left< \alpha_{t} | \beta_{t}\right>=P(Y)$
\end{definition}

\begin{definition}[Likelihood Function 1]
\begin{equation}
        P(Y) = \prod_{t=1}^{N_p} p(y_t|y_1,...,y_{t-1})=\prod_{t=1}^{N_p}c_t
\end{equation}
\begin{equation}
        \left< \hat{\alpha}_{t-1} | e^{-\textbf{H}\Delta t} \textbf{y}_t| \hat{\beta}_{t}\right> = c_t \left< \hat{\alpha}_{t} | \hat{\beta}_{t}\right> = c_t
\end{equation}
where $\left< \hat{\alpha}_{t} \right| = \frac{\left< \hat{\alpha}_{t-1} \right| e^{-\textbf{H}\Delta t} \textbf{y}_t}{c_t}$.
\end{definition}

\begin{definition}[Likelihood Function 2]
\begin{equation}
        \left< \hat{\alpha}_{t-1} | e^{-\textbf{H}\Delta t} \textbf{y}_t| \psi_1\right> = c_t = p(y_t|y_1,...,y_{t-1})
\end{equation}
Because
\begin{align*}
        &\int \rho_{\text{eq}}(x) p(y_{t-1}|x_{t-1},y_1,...,y_{t-2}) p(x_t|x_{t-1}) p(y_t|x_t) \rho_{\text{eq}}(x) dx_{t-1}dx_{t} \\
        & = \int p_{\text{eq}}(x_{t-1}) p(y_{t-1}|x_{t-1},y_1,...,y_{t-2}) p(x_t|x_{t-1}) p(y_t|x_t)  dx_{t-1}dx_{t}\\
        &= p(y_t|y_1,...,y_{t-1})
\end{align*}
\end{definition}

\begin{definition}[Decomposition of $\left< \hat{\alpha}_{t-1} | e^{-\textbf{H}\Delta t} \textbf{y}_t| \hat{\beta}_{t}\right>$]
\begin{align*}
        \left< \hat{\alpha}_{t-1} | e^{-\textbf{H}\Delta t} \textbf{y}_t| \hat{\beta}_{t}\right> &= \sum_{\{i,j\}} \left< \hat{\alpha}_{t-1} | \psi_i \right> \left< \psi_i |  e^{-\textbf{H}\Delta t}| \psi_j \right> \left< \psi_j | \textbf{y}_t | \hat{\beta}_{t} \right> \\
        &= \sum_{i=1}^{Nv}  e^{-D_i \lambda'_i \Delta t} \left< \hat{\alpha}_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}_{t} \right>
\end{align*}
\end{definition}

\begin{definition}[Decomposition of $\tilde{Q}$ and $\ln{P(Y)}$]
\begin{align}
        \tilde{Q} &= \sum_{t=1}^{N_p} \left< \hat{\alpha}_{t-1} | e^{-\textbf{H}\Delta t} \textbf{y}_t| \hat{\beta}_{t}\right> = \sum_{t=1}^{N_p} \sum_{i=1}^{Nv}  e^{-D_i \lambda'_i \Delta t} \left< \hat{\alpha}_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}_{t} \right>\\
        \ln{P(Y)} &= \sum_{t=1}^{N_p} \ln{\sum_{i=1}^{Nv}  e^{-D_i \lambda'_i \Delta t} \left< \hat{\alpha}_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}_{t} \right>}
\end{align}
\begin{center}
        \includegraphics[scale=0.45]{ch2/lnpy_Q.pdf}
\end{center}
\end{definition}

\section{Optimize D}
\begin{definition}[$\tilde{Q}^k(\textbf{D})$]
\begin{align}
        \textbf{D}&=\begin{bmatrix}D_1, D_2, D_3, D_4,...,D_{N_v}\end{bmatrix} \\
        \tilde{Q}^k(\textbf{D}) &= \sum_{t=1}^{N_p} \left< \hat{\alpha}^k_{t-1} | e^{-\textbf{H}\Delta t} \textbf{y}_t| \hat{\beta}^k_{t}\right> = \sum_{t=1}^{N_p} \sum_{i=1}^{Nv}  e^{-D_i \lambda'_i \Delta t} \left< \hat{\alpha}^k_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}^k_{t} \right> \\
        \tilde{Q}^k(\textbf{D}) &= \sum_{t=1}^{N_p} \left< \hat{\alpha}^k_{t-1} | \psi_1 \right> \left< \psi_1 | \textbf{y}_t | \hat{\beta}^k_{t} \right> + \sum_{t=1}^{N_p} \sum_{i=2}^{Nv}  e^{-D_i \lambda'_i \Delta t} \left< \hat{\alpha}^k_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}^k_{t} \right>
\end{align}
As you can see, $\left< \hat{\alpha}^k_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}^k_{t} \right>$ are calculated by using previous $\textbf{D}^k$, this is the E-step. Note $D_1$ is redundant. For all new $\textbf{D}$, $\sum_{t=1}^{N_p} \left< \hat{\alpha}^k_{t-1} | \psi_1 \right> \left< \psi_1 | \textbf{y}_t | \hat{\beta}^k_{t} \right>$ is the same, so the first eigenvector do not have contribution.
\end{definition}

\begin{definition}[$\frac{d\tilde{Q}}{dD_i}$]
\begin{equation}
        \frac{d\tilde{Q}^k}{dD_i} = -\lambda'_i \Delta t e^{-D_i \lambda'_i \Delta t} \sum_{t=1}^{N_p}  \left< \hat{\alpha}^k_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}^k_{t} \right>
\end{equation}
Only when $D_i$ is very large or $\Delta t$ very large, we can get $\frac{d\tilde{Q}}{dD_i}=0$.
\end{definition}

\begin{definition}[$\ln{P(Y)}$: Constant D]
\begin{align}
        \ln{P(Y)} &= \sum_{t=1}^{N_p} \ln{\left( \sum_{i=1}^{Nv}  e^{-D \lambda'_i \Delta t} \left< \hat{\alpha}^k_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}^k_{t} \right> \right)} \\
        &= \sum_{t=1}^{N_p} \ln{\left( \left< \hat{\alpha}^k_{t-1} | \psi_1 \right> \left< \psi_1 | \textbf{y}_t | \hat{\beta}^k_{t} \right> + \sum_{i=2}^{Nv}  e^{-D \lambda'_i \Delta t} \left< \hat{\alpha}^k_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}^k_{t} \right> \right)}
\end{align}
\begin{align}
        \frac{d \ln{P(Y)}}{dD} &=\sum_{t=1}^{N_p} \frac{\sum_{i=1}^{Nv} -\lambda'_i \Delta t e^{-D \lambda'_i \Delta t} \left< \hat{\alpha}^k_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}^k_{t} \right>}{\sum_{i=1}^{Nv}  e^{-D \lambda'_i \Delta t} \left< \hat{\alpha}^k_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}^k_{t} \right>}
\end{align}
\begin{center}
        \includegraphics[scale=0.45]{ch2/dldD_analytical_numerical.pdf}
\end{center}
\end{definition}

\begin{definition}[$\frac{d \ln{P(Y)}}{dD}$: Detail 1]
\begin{align*}
        \left< \hat{\alpha}_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}_{t} \right>
\end{align*}
\begin{center}
        \includegraphics[scale=0.45]{ch2/alpha_beta_in_dldD.pdf}
\end{center}
\begin{align*}
        e^{-D \lambda'_i \Delta t} \left< \hat{\alpha}_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}_{t} \right>
\end{align*}
where $D=500$
\begin{center}
        \includegraphics[scale=0.45]{ch2/edt_alpha_beta_in_dldD.pdf}
\end{center}
\begin{align*}
        -\lambda'_i \Delta t e^{-D \lambda'_i \Delta t} \left< \hat{\alpha}_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}_{t} \right>
\end{align*}
\begin{center}
        \includegraphics[scale=0.45]{ch2/LQ_edt_alpha_beta_in_dldD.pdf}
\end{center}
\end{definition}

\begin{definition}[$\frac{d \ln{P(Y)}}{dD}$: Detail 2]
The contribution from each time stamp can be split into the numerator
\begin{align*}
        \sum_{i=1}^{Nv} -\lambda'_i \Delta t e^{-D \lambda'_i \Delta t} \left< \hat{\alpha}^k_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}^k_{t} \right>
\end{align*}
and the denominator
\begin{align*}
        \sum_{i=1}^{Nv}  e^{-D \lambda'_i \Delta t} \left< \hat{\alpha}^k_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t | \hat{\beta}^k_{t} \right>
\end{align*}
\begin{center}
        \includegraphics[scale=0.35]{ch2/D_300.pdf}
\end{center}
When $D=300$

\begin{center}
        \includegraphics[scale=0.35]{ch2/D_500.pdf}
\end{center}
When $D=500$

\begin{center}
        \includegraphics[scale=0.35]{ch2/D_800.pdf}
\end{center}
When $D=800$
\end{definition}

\section{Position-Dependent Diffusion}
\begin{definition}[D(x)]
We express the position dependent diffusity as
\begin{equation}
        D(x) = D + \eta(x)
\end{equation}
The domain $x \in [a,b]$, and the integration of $\eta(x)$ over whole domain is 0
\begin{align*}
        \int_a^b \eta(x) dx = 0
\end{align*}
and
\begin{align*}
        \int_a^b D(x) dx &= \int_a^b D dx + \int_a^b \eta(x) dx \\
        &= D \int_a^b dx = (b-a)D
\end{align*}
\end{definition}

\begin{definition}[Goal]
The goal for position-dependent diffusion coefficient is
\begin{itemize}
        \item Given $p_{\text{eq}}(x)$
        \item Learn constant $D$
        \item Try to learn $\eta(x)$?
\end{itemize}
\begin{center}
        \includegraphics[scale=0.35]{ch2/learn_position_D.pdf}
\end{center}
\end{definition}

\begin{definition}[Example]
Test
\begin{center}
        \includegraphics[scale=0.4]{ch2/posi_depe_learn_1.pdf}
\end{center}
When $\zeta$ is greater 1800, the numerical instability occurs.
\begin{center}
        \includegraphics[scale=0.4]{ch2/zeta_2000_numerical_instability.pdf}
\end{center}
\end{definition}

\begin{definition}[Time Propagation]
\begin{equation}
        \frac{\partial \rho(x,t)}{\partial t} = -\textbf{H} \rho(x,t) = -(\textbf{H}^0 +\textbf{H}^D) \rho(x,t)
\end{equation}
where
\begin{align*}
        \textbf{H} &= -D(x) \frac{\partial^2}{\partial x^2} + \frac{1}{2}D(x)\frac{d F(x)}{dx} + \frac{1}{4} D(x) F^{2}(x) -\frac{\partial D(x)}{\partial x}\frac{\partial }{\partial x} + \frac{1}{2} \frac{\partial D(x)}{\partial x} F(x)\\
        \textbf{H}^0 &= -D\frac{\partial^2}{\partial x^2} + \frac{1}{2}D\frac{d F(x)}{dx} + \frac{1}{4} D F^{2}(x) \\
        \textbf{H}^D &= - \eta(x)\frac{\partial^2}{\partial x^2} + \frac{1}{2}\eta(x)\frac{d F(x)}{dx} + \frac{1}{4} \eta(x) F^{2}(x) -\frac{\partial \eta(x)}{\partial x}\frac{\partial }{\partial x} + \frac{1}{2} \frac{\partial \eta(x)}{\partial x} F(x)
\end{align*}
\end{definition}

\begin{definition}[Eigenvector and Eigenvalue]
\begin{align}
        \textbf{H} \left| \psi_i \right> &= \lambda_i \left| \psi_i \right> \\
        \left( \textbf{H}^0 + \textbf{H}^D \right) \left| \psi_i \right> &= \lambda_i \left| \psi_i \right> \\
        \textbf{H}^0 \left| \psi_i \right> + \textbf{H}^D \left| \psi_i \right> &= \lambda_i \left| \psi_i \right>
\end{align}
Recall that
\begin{equation}
        \left| \psi_i \right> = c_{i1}| \psi_1^0 \rangle + c_{i2}| \psi_2^0 \rangle +  c_{i3}| \psi_3^0 \rangle+\cdots + c_{i,N_v}| \psi_{N_v}^0 \rangle
\end{equation}
and
\begin{align}
        \textbf{H}^0 \left| \psi_i \right> &= c_{i1} \lambda^0_1| \psi_1^0 \rangle + c_{i2}\lambda^0_2| \psi_2^0 \rangle +  c_{i3}\lambda^0_3| \psi_3^0 \rangle+\cdots + c_{i,N_v}\lambda^0_{N_v}| \psi_{N_v}^0 \rangle \\
        \textbf{H}^D \left| \psi_i \right> &= c_{i1} \textbf{H}^D| \psi_1^0 \rangle + c_{i2}\textbf{H}^D| \psi_2^0 \rangle +  c_{i3}\textbf{H}^D| \psi_3^0 \rangle+\cdots + c_{i,N_v}\textbf{H}^D| \psi_{N_v}^0 \rangle \\
        \lambda_i \left| \psi_i \right> &= c_{i1} \lambda_i| \psi_1^0 \rangle + c_{i2}\lambda_i| \psi_2^0 \rangle +  c_{i3}\lambda_i| \psi_3^0 \rangle+\cdots + c_{i,N_v}\lambda_i| \psi_{N_v}^0 \rangle
\end{align}
then
\begin{align*}
        \textbf{H}^D \left| \psi_i \right> &=  \lambda_i \left| \psi_i \right> - \textbf{H}^0 \left| \psi_i \right> \\
        &= \left(\lambda_i - \lambda_1^0 \right) \left| \psi_1^0 \right> + \left(\lambda_i - \lambda_2^0 \right) \left| \psi_2^0 \right> + ... + \left(\lambda_i - \lambda_{N_v}^0 \right) \left| \psi_{N_v}^0 \right>
\end{align*}
It becomes
\begin{align*}
        \textbf{H}^D \left| \psi_i \right> &= c_{i1} \textbf{H}^D| \psi_1^0 \rangle + c_{i2}\textbf{H}^D| \psi_2^0 \rangle +  c_{i3}\textbf{H}^D| \psi_3^0 \rangle+\cdots + c_{i,N_v}\textbf{H}^D| \psi_{N_v}^0 \rangle \\
        &= c_{i1} \left(\lambda_i - \lambda_1^0 \right) \left| \psi_1^0 \right> + c_{i2} \left(\lambda_i - \lambda_2^0 \right) \left| \psi_2^0 \right> + ... + c_{i,N_v} \left(\lambda_i - \lambda_{N_v}^0 \right) \left| \psi_{N_v}^0 \right>
\end{align*}
so
\begin{equation}
        \textbf{H}^D \left| \psi_j^0 \right> = \left(\lambda_i - \lambda_j^0 \right) \left| \psi_j^0 \right>
\end{equation}
and $\textbf{H}^D$ can be written as a matrix whose element is
\begin{equation}
        \left(\lambda_i - \lambda_j^0 \right)
\end{equation}
\end{definition}

\begin{definition}[Likelihood Function 1]
\begin{align}
        \left< \hat{\alpha}_{t-1} | e^{-\textbf{H}\Delta t} \textbf{y}_t| \hat{\beta}_{t}\right> &= c_t \\
        \sum_{\{i,j\}} \left< \hat{\alpha}_{t-1} | \psi_i \right>\left< \psi_i | e^{-\textbf{H}\Delta t} | \psi_j \right>\left< \psi_j | \textbf{y}_t| \hat{\beta}_{t}\right> &= c_t \\
        \sum_{i} e^{-\lambda_i \Delta t} \left< \hat{\alpha}_{t-1} | \psi_i \right> \left< \psi_i | \textbf{y}_t| \hat{\beta}_{t}\right> &= c_t
\end{align}
\end{definition}