%&latex

\documentclass[10pt,landscape]{article}
%\documentclass[10pt]{article}

%
% Bring in the AMS math environment
%
\usepackage{amsmath}

%
% Bring in the common page setup
%
\usepackage{trickhlaenv}

%
% Define vector and quaternion macros
%
\usepackage{trickhlamath}

%
% Bring in the hyper ref environment
%
\usepackage[colorlinks,bookmarks]{hyperref}
\hypersetup{
   pdftitle={TrickHLA Package Mathematical Nomenclature},
   pdfauthor={David Hammen}}

\newcommand{\trickhlamath}{\texorpdfstring{\tt trickhlamath}{trickhlamath}}

\newcommand{\symhdr}[1]{%
\multicolumn{4}{l}{\rule{0pt}{3ex}\parbox[c]{0.6\textwidth}{#1}} \\
\cline{2-4} \rule{3em}{0pt} & {\bf Symbol} & {\bf Represents} & {\bf Units (MKS)}\\ \cline{2-4} \cline{2-4}}

\newcommand{\acchdr}[1]{%
\multicolumn{4}{l}{\rule{0pt}{3ex}\parbox[c]{0.6\textwidth}{#1}} \\
\cline{2-4} \rule{3em}{0pt} & {\bf Description} & {\bf Remark} & {\bf Example}\\ \cline{2-4} \cline{2-4}}

\def\purpwidth{0.35\textwidth}
\def\argswidth{0.15\textwidth}

\newcommand{\mlentry}[2]{%
  \minipage[t]{#1}{\flushleft{#2}\endflushleft}\endminipage}

\newcommand{\slinethree}{\rule{0pt}{0.2ex}&\rule{0pt}{0.2ex}&\rule{0pt}{0.2ex}&\rule{0pt}{0.2ex}}
\newcommand{\slinefour}{\rule{0pt}{1.2ex}&&&&}

\begin{document}

\title{TrickHLA Package Mathematical Nomenclature}
\author{David Hammen}
\date{12/21/05}

\pdfbookmark{Title Page}{titlepage}
\maketitle

\tableofcontents

\section*{Introduction}
This note describes the preferred nomenclature for the TrickHLA package
documentation and describes a set of \LaTeX\ macros that implement
the nomenclature.

\pagebreak
\section{Nomenclature}
This section describes the preferred nomenclature for the TrickHLA package
documentation. The preferred nomenclature
\begin{itemize}
\item Uses International System (SI) units with standard SI abbreviations.
\item Represents scalars in plain math font, vectors and matrices in bold math font,
and quaternions in caligraphy font.
\item Represents vectors, matrices, and quaternions in a standard, intuitive style
with an obvious translation to variable names. For example,
$\framerelvect A x a b$ is the vector from point $a$ to point $b$ as expressed in
reference frame $A$. This is denoted as the ``arrow-separated'' format.
Alternative respresentations such as
\trickhlamathcommamode $\relvect x a b$ \trickhlamatharrowmode (``comma-separated'')
and
\trickhlamathstackedmode $\relvect x a b$ \trickhlamatharrowmode (``stacked'')
were
discussed but discarded for various reasons.
\item Quaternions that represent a transformation from one frame to another
are represented as unit left transformation quaternions,
consistent with the quaternion functions defined in the Trick core.
\end{itemize}

The following tables depict the preferred nomenclature.
In cases where alternative representation schemes exist,
the preferred form is listed first followed by alternatives

 \pagebreak
\subsection{Basics}\label{sec:nomen_basics}

\begin{tabular}{l||l|l|l|}
\acchdr{{\bf{Display style for vectors, matrices, and quaternions}}}
\rule{0pt}{3ex} & Scalar &
  in plain math font & $s$ \\
\rule{0pt}{1.5ex} & Vector &
  in bold math font& $\vect x$ \\
\rule{0pt}{1.5ex} & Matrix &
  in bold math font& $\mat T$ \\
\rule{0pt}{1.5ex} & Quaternion&
  in caligraphy font, uppercase & $\quat Q$ \\
\cline{2-4}
\symhdr{{\bf{Scalar Symbols}}}
\rule{0pt}{3ex} & $\alpha$ & Angular acceleration & $r/s^2$ \\
\rule{0pt}{1.5ex} & $\alpha, \beta, \theta, \phi, \psi$ & Angle & $r$ \\
\rule{0pt}{1.5ex} & $\omega$ & Angular rate & $r/s$ \\
\rule{0pt}{1.5ex} & $d, l, s$ & Distance or length & $m$ \\
\rule{0pt}{1.5ex} & $m$ & Mass & $kg$ \\
\rule{0pt}{1.5ex} & $v, s$ & Speed & $m/s$ \\
\cline{2-4}
\symhdr{{\bf{Vector Symbols}}}
\rule{0pt}{3ex} & $\vect \alpha$ & Angular acceleration & $r/s^2$ \\
\rule{0pt}{1.5ex} & $\vect \tau$ & Torque & $Nm$ \\
\rule{0pt}{1.5ex} & $\vect \omega$ & Angular velocity & $r/s$ \\
\rule{0pt}{1.5ex} & $\vect a$ & Acceleration & $m/s^2$ \\
\rule{0pt}{1.5ex} & $\vect L$ & Angular momentum & $Nms \; (kg \, m^2/s)$ \\
\rule{0pt}{1.5ex} & $\vect v$ & Velocity & $m/s$ \\
\rule{0pt}{1.5ex} & $\vect x$ & Position & $m$ \\
\cline{2-4}
\symhdr{{\bf{Matrix Symbols}}}
\rule{0pt}{3ex} & $\mat I$ & Inertia tensor & $kg \, m^2$ \\
\rule{0pt}{1.5ex} & $\mat T$ & Transformation matrix & $--$ \\
\cline{2-4}
\symhdr{{\bf{Quaternion Symbols}}}
\rule{0pt}{3ex} & $\quat Q$ & Left transformation quaternion & $--$ \\
\cline{2-4}
\end{tabular}

\pagebreak

\subsection{Adornments}\label{sec:nomen_addorn}

\begin{tabular}{l||l|l|l|}
\acchdr{{\bf{Vectors}}}
\rule{0pt}{3ex} & Vector from origin to $b$ &Subscripted $b$& $\absvect x b$ \\
%\cline{2-4}
\rule{0pt}{3ex} & Vector from $a$ to $b$ &
 Arrow, comma, stacked formats &
\makebox[1.5cm]{$\relvect x a b$}
\makebox[1.5cm]{\trickhlamathcommamode $\relvect x a b$ \trickhlamatharrowmode}
\makebox[1.5cm]{\trickhlamathstackedmode $\relvect x a b$ \trickhlamatharrowmode} \\
%\cline{2-4}
\rule{0pt}{3ex} & Vector expressed in frame $A$ &&
\makebox[1.5cm]{$\framevect A x$}
\makebox[1.5cm]{\trickhlamathcommamode $\framevect A x$ \trickhlamatharrowmode}
\makebox[1.5cm]{\trickhlamathstackedmode $\framevect A x$ \trickhlamatharrowmode} \\
%\cline{2-4}
\rule{0pt}{3ex} & Vector from $a$ to $b$ expressed in frame $A$ &&
\makebox[1.5cm]{$\framerelvect A x a b$}
\makebox[1.5cm]{\trickhlamathcommamode $\framerelvect A x a b$ \trickhlamatharrowmode}
\makebox[1.5cm]{\trickhlamathstackedmode $\framerelvect A x a b$ \trickhlamatharrowmode} \\
%\cline{2-4}
\rule{0pt}{3ex} & Time derivative of above, observer in frame $A$&&
\makebox[1.5cm]{$\framerelvdot A x a b$}
\makebox[1.5cm]{\trickhlamathcommamode $\framerelvdot A x a b$ \trickhlamatharrowmode}
\makebox[1.5cm]{\trickhlamathstackedmode $\framerelvdot A x a b$ \trickhlamatharrowmode} \\
\rule{0pt}{3ex} & Vector time derivative, observer in frame $A$&
  Dot format & $\framedot A {\vect x}$ \\
\rule{0pt}{3ex} & &
  $or$ d/dt format & $\frac{d}{dt_A}{\vect x}$ \\
\cline{2-4}
\acchdr{{\bf{Matrices}}}
\rule{0pt}{3ex} & Transformation from $A$ to $B$ &
 Arrow, comma, stacked formats &
\makebox[1.5cm]{$\tmat A B$}
\makebox[1.5cm]{\trickhlamathcommamode $\tmat A B$ \trickhlamatharrowmode}
\makebox[1.5cm]{\trickhlamathstackedmode $\tmat A B$ \trickhlamatharrowmode} \\
%\cline{2-4}
\rule{0pt}{3ex} & Matrix product &
  No operator & $\MxM{\tmat B C}{\tmat A B}$ \\
%\cline{2-4}
\rule{0pt}{3ex} & Single matrix transpose &
  Superscript $\top$ & $\matT T$ \\
%\cline{2-4}
\rule{0pt}{3ex} & Matrix product transpose &
  Superscript $\top$ & $\left(\MxM{\tmat B C}{\tmat A B}\right){^\top}$ \\
\cline{2-4}
\acchdr{{\bf{Quaternions}}}
\rule{0pt}{3ex} & Quaternion from $A$ to $B$ &
 Arrow, comma, stacked formats &
\makebox[1.5cm]{$\tquat A B$}
\makebox[1.5cm]{\trickhlamathcommamode $\tquat A B$ \trickhlamatharrowmode}
\makebox[1.5cm]{\trickhlamathstackedmode $\tquat A B$ \trickhlamatharrowmode} \\
%\cline{2-4}
\rule{0pt}{3ex} & Quaternion product &
  No operator & $\QxQ{\tquat B C}{\tquat A B}$ \\
%\cline{2-4}
\rule{0pt}{3ex} & Single quaternion conjugate &
  Superscript $\star$ & $\quatconj Q$ \\
%\cline{2-4}
\rule{0pt}{3ex} & Quaternion product conjugate &
  Superscript $\star $ & $\quatconjlr{\QxQ{\tquat B C}{\tquat A B}}$ \\
%\cline{2-4}
\rule{0pt}{3ex} & Quaternion components &
  Scalar $+$ vector $or$ four-vector&
  $\quatsv{q_s}{\vect{q_v}} \;or\; \bmatrix q_s \\ q_x \\ q_y \\ q_z\endbmatrix$ \\
\cline{2-4}
\end{tabular}

\pagebreak
\section{{\trickhlamath}\ Macros}

This section provides a brief description of the macros defined in
\verb|trickhlamath.sty| in the form of tables that describe the macros,
their arguments, sample usage of the macros, and the displayed math that results.

Note: All of the macros defined in \verb|trickhlamath.sty| assume {\LaTeX} is in math mode.

\pagebreak
\subsection{Vector Macros}

\begin{tabular}{||l|l|l|l|l|} \hline
{\bf Command} & {\bf Purpose} & {\bf Arguments} & {\bf Example} & {\bf Display} \\ \hline \hline
\slinefour \\
\verb|\vect| &
  \mlentry{\purpwidth}{Display a symbol that represents a vector (typically a lowercase letter)} &
  \mlentry{\argswidth}{Symbol} &
  \verb|\vect{x}| & $\vect{x}$ \\ \slinefour \\
\verb|\vhat| &
  \mlentry{\purpwidth}{Display a symbol that represents a unit vector} &
  \mlentry{\argswidth}{Symbol} &
  \verb|\vhat{x}| & $\vhat{x}$ \\ \slinefour \\
\verb|\vdot| &
  \mlentry{\purpwidth}{Time derivative of a vector} &
  \mlentry{\argswidth}{Symbol} &
  \verb|\vdot{x}| & $\vdot{x}$ \\ \slinefour \\
\verb|\framevect| &
  \mlentry{\purpwidth}{Vector represented in a specific frame} &
  \mlentry{\argswidth}{1. Frame\\ 2. Vector} &
  \verb|\framevect B x| & $\framevect B x$  \\ \slinefour \\
  \verb|\framevdot| &
  \mlentry{\purpwidth}{Time derivative of a vector represented in a specific frame} &
  \mlentry{\argswidth}{1. Frame\\ 2. Vector} &
  \verb|\framevdot B x| & $\framevdot B x$  \\ \slinefour \\
\verb|\relvect| &
  \mlentry{\purpwidth}{Vector between two items} &
  \mlentry{\argswidth}{1. Vector\\ 2. Source\\ 3. Destination} &
  \verb|\relvect x a b| & $\relvect x a b$  \\ \slinefour \\
\verb|\relvdot| &
  \mlentry{\purpwidth}{Time derivative of a vector between two items} &
  \mlentry{\argswidth}{1. Vector\\ 2. Source\\ 3. Destination} &
  \verb|\relvdot x a b| & $\relvdot x a b$  \\ \slinefour \\
\verb|\framerelvect| &
  \mlentry{\purpwidth}{Vector between two items} &
  \mlentry{\argswidth}{1. Frame\\ 2. Vector\\ 3. Source\\ 4. Destination} &
  \verb|\framerelvect B x a b| & $\framerelvect B x a b$  \\ \slinefour \\
\verb|\framerelvdot| &
  \mlentry{\purpwidth}{Time derivative of a vector between two items} &
  \mlentry{\argswidth}{1. Frame\\ 2. Vector\\ 3. Source\\ 4. Destination} &
  \verb|\framerelvdot B x a b| & $\framerelvdot B x a b$  \\ \slinefour \\
\verb|\vectxyz| &
  \mlentry{\purpwidth}{Construct a vector from its components} &
  \mlentry{\argswidth}{Three components} &
  \verb|\vectxyz x y z| & $\vectxyz x y z$  \\ \slinefour \\
\hline
\end{tabular}

\pagebreak

\subsection{Quaternion Macros}
\
\begin{tabular}{||l|l|l|l|l|} \hline
{\bf Command} & {\bf Purpose} & {\bf Arguments} & {\bf Example} & {\bf Display} \\ \hline \hline
\slinefour \\
\verb|\quat| &
  \mlentry{\purpwidth}{Display a symbol that represents a quaternion (typically Q)} &
  \mlentry{\argswidth}{Symbol} &
  \verb|\quat{Q}| & $\quat{Q}$ \\ \slinefour \\
\verb|\qdot| &
  \mlentry{\purpwidth}{Time derivative of a quaternion} &
  \mlentry{\argswidth}{Symbol} &
  \verb|\qdot{Q}| & $\qdot{Q}$ \\ \slinefour \\
\verb|\quatconj| &
  \mlentry{\purpwidth}{Conjugate of a quaternion} &
  \mlentry{\argswidth}{Symbol} &
  \verb|\quatconj{Q}| & $\quatconj{Q}$ \\ \slinefour \\
\verb|\quatconjdot| &
  \mlentry{\purpwidth}{Time derivative of the conjugate of a quaternion} &
  \mlentry{\argswidth}{Symbol} &
  \verb|\quatconjdot{Q}| & $\quatconjdot{Q}$ \\ \slinefour \\
\verb|\quatsv| &
  \mlentry{\purpwidth}{Construct a quaternion from a scalar and $3$-vector} &
  \mlentry{\argswidth}{1. Scalar\\ 2. Vector} &
  \minipage[t]{0pt}{\verb|\quatsv| \\ \verb|  {q_s}| \\ \verb|  {\vect{q_v}}|}\endminipage &
  \raisebox{-1.5ex}{$\quatsv{q_s}{\vect{q_v}}$} \\ \slinefour \\
\verb|\quattrot| &
  \mlentry{\purpwidth}{Construct a transformation quaternion from an angle and unit vector} &
  \mlentry{\argswidth}{1. Angle\\ 2. Unit vector} &
  \minipage[t]{0pt}{\verb|\quattrot| \\ \verb|  \theta| \\ \verb|  {\vhat u}|}\endminipage &
  \raisebox{-2.5ex}{$\quattrot{\theta}{\vhat u}$} \\ \slinefour \\
\verb|\conjop| &
  \mlentry{\purpwidth}{Conjugate operator} &
  \mlentry{\argswidth}{None} &
  \verb|\conjop \quat Q| & $\conjop \quat Q$ \\ \slinefour \\
\verb|\scalarpart| &
  \mlentry{\purpwidth}{Scalar part operator} &
  \mlentry{\argswidth}{None} &
  \verb|\scalarpart \quat Q| & $\scalarpart \quat Q$ \\ \slinefour \\
\verb|\vectorpart| &
  \mlentry{\purpwidth}{Vector part operator} &
  \mlentry{\argswidth}{None} &
  \verb|\vectorpart \quat Q| & $\vectorpart \quat Q$ \\ \slinefour \\
\verb|\QBI| &
  \mlentry{\purpwidth}{Inertial-to-body quaternion} &
  \mlentry{\argswidth}{None} &
  \verb|\QBI| & $\QBI$ \\ \slinefour \\
\hline
\end{tabular}

\pagebreak

\subsection{Matrix Macros}

\begin{tabular}{||l|l|l|l|l|} \hline
{\bf Command} & {\bf Purpose} & {\bf Arguments} & {\bf Example} & {\bf Display} \\ \hline \hline
\slinefour \\
\verb|\mat| &
  \mlentry{\purpwidth}{Display a symbol that represents a matrix (typically an uppercase letter)} &
  \mlentry{\argswidth}{Symbol} &
  \verb|\mat{T}| & $\mat{T}$ \\ \slinefour \\
\verb|\framemat| &
  \mlentry{\purpwidth}{Matrix represented in some reference frame} &
  \mlentry{\argswidth}{1. Frame\\ 2. Matrix} &
  \verb|\framemat B \inertia| & $\framemat B \inertia$ \\ \slinefour \\
\verb|\diagmatrix| &
  \mlentry{\purpwidth}{$3\times3$ diagonal matrix} &
  \mlentry{\argswidth}{Three diagonal elements} &
  \verb|\diagmatrix 1 2 3| & $\diagmatrix 1 2 3$ \\ \slinefour \\
\verb|\identmatrix| &
  \mlentry{\purpwidth}{$3\times3$ identity matrix} &
  \mlentry{\argswidth}{None} &
  \verb|\identmatrix| & $\identmatrix$ \\ \slinefour \\
\verb|\inertia| &
  \mlentry{\purpwidth}{Inertia matrix} &
  \mlentry{\argswidth}{None} &
  \verb|\inertia| & $\inertia$ \\ \slinefour \\
\hline
\end{tabular}

\pagebreak

\subsection{Multiplication Macros}

The preferred nomenclature for depicting the product of two (or more) composite objects
is ``implicit multiplication'':  The operands are written with no intervening operator.
However,
a small amount of white space between the operands helps to distinguish the operands.
The multiplication macros provide a default amount of white space between operands.

\begin{tabular}{||l|l|l|l|l|} \hline
{\bf Command} & {\bf Purpose} & {\bf Arguments} & {\bf Example} & {\bf Display} \\ \hline \hline
\slinefour \\
\verb|\MxM| &
  \mlentry{\purpwidth}{Product of two matrices} &
  \mlentry{\argswidth}{Matrix 1\\ Matrix 2} &
  \minipage[t]{2.5cm}{\verb|\MxM| \\ %
    \verb| {\tmat B C}| \\ %
    \verb| {\tmat A B}|}\endminipage &
  $\MxM{\tmat B C}{\tmat A B}$ \\[8ex]
\verb|\QxVxQ | &
  \mlentry{\purpwidth}{Product of a quaternion, a vector, and a quaternion} &
  \mlentry{\argswidth}{Quaternion 1\\ Vector\\ Quaternion 2} &
  \minipage[t]{2.5cm}{\verb|\QxVxQ| \\ %
    \verb| {\quat Q}| \\ %
    \verb| {\vect x}| \\ %
    \verb| {\quatconj Q}|}\endminipage &
    \raisebox{-1.5ex}{$\QxVxQ{\quat Q}{\vect x}{\quatconj Q}$} \\ \slinefour \\
\hline
\end{tabular}

Similar macros are defined for
\begin{itemize}
\item the product of three matrices (\verb|MxMxM|)
\item the product of a matrix and vector (\verb|MxV|)
\item the product of two or three quaternions (\verb|QxQ|, \verb|QxQxQ|)
\item the product of a quaternion and a vector (\verb|QxV|, \verb|VxQ|)
\end{itemize}

The multiplication macros take an optional argument via which an explicit multiplication operator can be specified. For example,

\begin{tabular}{||l|l|l|l|l|} \hline
{\bf Command} & {\bf Purpose} & {\bf Option} & {\bf Example} & {\bf Display} \\ \hline \hline
\slinefour \\
\verb|\QxQ| &
  \mlentry{\purpwidth}{Implicit product of two quaternions} &
  \mlentry{\argswidth}{None} &
  \minipage[t]{2.5cm}{\verb|\QxQ| \\ %
    \verb| {\quat{Q}_1}| \\ %
    \verb| {\quat{Q}_2}|}\endminipage &
  $\QxQ{{\quat Q}_1}{{\quat Q}_2}$ \\[8ex]
\verb|\QxQ| &
  \mlentry{\purpwidth}{Explicit product of two quaternions} &
  \mlentry{\argswidth}{\ttfamily \textbackslash circ} &
 \minipage[t]{2.5cm}{\verb|\QxQ[\circ]| \\ %
    \verb| {{\quat Q}_1}| \\ %
    \verb| {{\quat Q}_2}|}\endminipage &
  $\QxQ[\circ]{{\quat Q}_1}{{\quat Q}_2}$ \\[8ex]
\hline
\end{tabular}

\pagebreak

\subsection{Miscellaneous Macros}

\begin{tabular}{||l|l|l|l|l|} \hline
{\bf Command} & {\bf Purpose} & {\bf Arguments} & {\bf Example} & {\bf Display} \\ \hline \hline
\slinefour \\
\verb|\abs| &
  \mlentry{\purpwidth}{Absolute value} &
  \mlentry{\argswidth}{Scalar expression} &
  \verb|\abs{x}| & $\abs{x}$ \\ \slinefour \\
\verb|\norm| &
  \mlentry{\purpwidth}{Euclidian norm} &
  \mlentry{\argswidth}{Vector or quaternion expression} &
  \verb|\norm{\vect x}| & $\norm{\vect x}$ \\ \slinefour \\
\verb|\framedot| &
  \mlentry{\purpwidth}{Frame-dependent time derivative} &
  \mlentry{\argswidth}{1. Frame\\ 2. Expression} &
  \verb|\framedot B {\vect x}| & $\framedot B {\vect x}$ \\ \slinefour \\
\hline
\end{tabular}

\end{document}


% declare document class and geometry
\documentclass[12pt]{article} % use larger type; default would be 10pt
\usepackage[english]{babel} % for hyphenation dictionary
%\setdefaultlanguage{english} % polyglossia command for use with XeTeX / LuaTeX
\usepackage[margin=1in]{geometry} % handle page geometry

% import packages and commands
\input{../header2.tex}

% title information
\title{Phys 221A -- Quantum Mechanics -- Lec09}
\author{UCLA, Fall 2014}
\date{\formatdate{3}{11}{2014}} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed

\begin{document}
\maketitle

\section{Propagator}

The propagator is defined as the time evolution operator evaluated in position space
\begin{eqn}
K(\v r, t; \v r', t') = \matrixel{\v r}{U(t,t')}{\v r'}.
\end{eqn}
When $t=t'$ we have $U(t,t'=t) = 1$, so that the propagator in this case is just
\begin{eqn}
K(\v r, t; \v r', t'=t) = \delta(\v r - \v r').
\label{eq:prop0}
\end{eqn}
Now, the Schroedinger equation can be rewritten
\begin{eqn}
(i \hbar \partial_t - H) \psi(\v r, t) = 0,
\end{eqn}
where $H$ is some differential operator, e.g. $H = -\frac{\hbar^2}{2m} \nabla_r^2$. Let's call the new operator on the left the ``Schroedinger operator''. Furthermore, we can write
\begin{eqn}
\int \dif^3{r} \, \psi(\v r, t) \ket{\v r} = U(t,t') \int \dif^3{r} \, \psi(\v r, t') \ket{\v r}.
\end{eqn}
Applying a position bra on the left, we have
\begin{align}
\psi(\v r, t) &= \bra{\v r} \int \dif^3{r'} \, \psi(\v r', t) \ket{\v r'} \\
	&= \int \dif^3{r'} \, K(\v r, t; \v r', t') \psi(\v r, t').
\end{align}
Next, applying the Schroedinger operator, we have
\begin{align}
0 &= (i \hbar \partial_t - H) \psi(\v r, t) \\
	&= \int \dif^3{r'} \left[ (i \hbar \partial_t - H) K(\v r, t; \v r', t') \right] \psi(\v r', t').
\end{align}
Since $\psi(\v r', t')$ is a completely arbitrary wavefunction, we find that in general the propagator itself obeys the Schroedinger equation,
\begin{eqn}
(i \hbar \partial_t - H) K(\v r, t; \v r', t') = 0.
\end{eqn}
This combined with the initial condition \eqref{eq:prop0} uniquely characterizes the propagator $K$.

\subsection{Causality, Retarded Propagator}

We can impose causality by defining the retarded propagator
\begin{eqn}
K^R (\v r, t; \v r', t') \equiv \theta(t-t') K(\v r, t; \v r', t'),
\end{eqn}
where $\theta(x)$ is the Heaviside step function. This just cuts off information about the past, which is really just redundant because we should always have symmetry under time reversal. What happens when we applying the Schroedinger operator to the retarded propagator? Everything goes away except for the time derivative acting on the step function,
\begin{align}
(i \hbar \partial_t - H) K^R(\v r, t; \v r', t') &= \left[ i \hbar \partial_t \theta(t-t') \right] K(\v r, t; \v r', t') \\
	&= i \hbar \delta (t-t') \delta^3(\v r - \v r').
\end{align}
This uniquely defines the retarded propagator---note also that, due to the step function, the retarded propagator is zero for $t < t'$.

\subsection{Green's function}

Given a differential equation of the form
\begin{eqn}
L(\v x) f(\v x) = \delta(\v x - \v y)
\end{eqn}
with some boundary and/or initial conditions, where $L(\v x)$ is some operator and $\v y$ is an arbitrary point, we call $f(\v x)$ a Green's function of $L$. There is a lot of mathematical technology developed around Green's functions, so it is useful to consider the propagator as a Green's function of the Schroedinger operator. This formulation is very useful for more advanced applications like quantum field theory.

\subsection{Time-independent Hamiltonian}

Let's specialize to the case of time-independent $H$, i.e. we will have energy eigenkets $\ket{\alpha_i}$ with eigenvalues $E_i$,
\begin{eqn}
H \ket{\alpha_i} = E_i \ket{\alpha_i}.
\end{eqn}
Recall that in this eigenbasis we can write
\begin{eqn}
U(t-t') = \sum_i \bra{\alpha_i} \ket{\alpha_i} e^{-i E_i (t-t') / \hbar},
\end{eqn}
thus we find that
\begin{align}
K(\v r, t; \v r', t') &= \sum_i \underbrace{\braket{\v r}{\alpha_i}}_{\psi_i(\v r)} \braket{\alpha_i}{\v r'} e^{-i E_i (t-t') / \hbar} \\
	&= \sum_i \psi_i (\v r) \psi_i^* (\v r') e^{-i E_i (t-t') / \hbar},
\end{align}
where we have
\begin{eqn}
H(\v r) \psi_i(\v r) = E_i \psi(\v r).
\end{eqn}

\begin{remark}
At the end of the day, we can think of the propagator as a wave function for a particle localized at $\v r'$ at time $t'$.
\end{remark}

Now, we can think of the propagator as as a trace over time,
\begin{align}
G(t-t') &= \tr U(t,t') \\
	&= \int \dif^3 r \, K(\v r, t; \v r', t') \\
	&= \sum_i e^{-i E_i (t-t') / \hbar},
\end{align}
since the eigenfunctions are all normalized. Note that here the trace is defined by
\begin{align}
\tr U(t,t') &= \sum_i \matrixel{\beta_i}{U(t,t')}{\beta_i} \\
	&= \sum_i \int \dif^3{r} \braket{\beta_i}{\v r} \matrixel{\v r}{U}{\beta_i} \\
	&= \int \dif^3{r} \matrixel{\v r}{U(t,t')}{\v r}.
\end{align}

Notice that our Green's function
\begin{eqn}
G(t) = \sum_i e^{-i E_i t / \hbar}
\end{eqn}
now is almost exactly the same as the partition function from statistical mechanics
\begin{eqn}
Z = \sum_i e^{-\beta E_i}.
\end{eqn}
This is no trivial remark---it turns out that much of what we do in statistical mechanics is directly applicable to quantum field theory, and vice versa. If we define an ``imaginary time'' $\tau = \hbar \beta$, the connection becomes clearer. We can analytically continue $G(t)$ onto the complex plane defined by $z = t - i \tau$. Then, we find that under the so-called ``Wick rotation'' $t \rightarrow i \tau$ we have
\begin{eqn}
G(-i \tau) = Z.
\end{eqn}

Of course we can also define a retarded Green's function
\begin{eqn}
G^R(t) = \theta(t) G(t),
\end{eqn}
which we can Fourier transform and write
\begin{eqn}
G^R(\omega) = -i \int_{-\infty}^\infty \dif{t} \, e^{i\omega t} G^R(t) = -i \int_0^\infty \dif{t} \, e^{i \omega t} G^R(t).
\end{eqn}
This generally ends up making further calculations intractible but we can ``regularize'' by taking $\omega \rightarrow \omega + i \eta$,
\begin{eqn}
G^R(\omega) \rightarrow -i \int_0^\infty e^{i \omega t - \eta t} G^R(t),
\end{eqn}
or in other words
\begin{eqn}
G^R(t) \rightarrow G^R(t) e^{-\eta t}.
\end{eqn}
Then we find that
\begin{eqn}
G^R(\omega) = -i \sum_i \int_0^\infty \dif{t} e^{i \omega t - \eta t - i E_i t / \hbar},
\end{eqn}
which we can evaluate using
\begin{eqn}
\int_0^\infty \dif{t} \, e^{-\alpha t} = 1 / \alpha, \qquad \text{where $\Re \alpha > 0$},
\end{eqn}
so that
\begin{eqn}
G^R(\omega) = \sum_i \frac{1}{\omega - E_i / \hbar + i \eta}.
\end{eqn}
So we have a function with poles in the complex plane at $E_i / \hbar - i \eta$, which we have pushed just below the real line, into the negative imaginary half-plane.

\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
% \usepackage{graphicx}
%     \DeclareGraphicsExtensions{.png, .jpeg}
% \usepackage{caption}
\usepackage[top=1in, bottom=1in, left=1in, right=1in]{geometry}

\title{STAT 775: Machine Learning \\ HW 06}
\author{Terence Henriod}
\date{\today}

\begin{document}

\clearpage            % All of
\maketitle            % this,
\thispagestyle{empty} % removes the page number from the title page

\begin{abstract}
In this assignment, feed-forwards neural networks using simple stochastic backpropagation were explored.
\end{abstract}

\newpage
\section{Exercise 01}
\subsection{Problem Statement}
Implement a single neural network to perform digit image classification for all ten classes of digits using the zip.data data set from the ESL website. To reduce the model complexity, perform PCA to limit the input features to 20. Use a single hidden layer with varying numbers of units. Use 10 output units. Use sigmoidal units.

\subsection{Results}
A neural net with $20$ inputs, $20$ hidden layer units, and $10$ outputs was constructed and trained. Using a learning rate of $2$ and exposing the network to each training sample observation $500$ times produces the following results, summarized in a confusion matrix:

\begin{tabular}{| c || c | c | c | c | c | c | c | c | c | c | c |}
  \hline
  Actual/Prediction &   0 &   1 &   2 &   3 &   4 &   5 &   6 &   7 &   8 &   9 &\% Correct \\
  \hline
  \hline
  0                 & 341 &   1 &   2 &   2 &   3 &   3 &   6 &   0 &   1 &   0 & 94.9 \\
  \hline
  1                 &   0 & 254 &   0 &   0 &   3 &   2 &   4 &   0 &   0 &   1 & 96.2 \\
  \hline
  2                 &   5 &   0 & 174 &   4 &   7 &   1 &   2 &   2 &   3 &   0 & 87.9 \\
  \hline
  3                 &   3 &   0 &   3 & 139 &   0 &  18 &   0 &   1 &   2 &   0 & 83.7 \\
  \hline
  4                 &   1 &   2 &   7 &   0 & 181 &   2 &   1 &   1 &   1 &   4 & 90.5 \\
  \hline
  5                 &   3 &   1 &   1 &  11 &   3 & 138 &   0 &   0 &   2 &   1 & 86.3 \\
  \hline
  6                 &   6 &   0 &   2 &   0 &   1 &   3 & 155 &   0 &   3 &   0 & 91.2 \\
  \hline
  7                 &   0 &   0 &   0 &   1 &   5 &   0 &   0 & 138 &   1 &   2 & 93.9 \\
  \hline
  0                 &   2 &   2 &   3 &   9 &   2 &   6 &   2 &   2 & 131 &   7 & 78.9 \\
  \hline
  0                 &   0 &   2 &   1 &   1 &   8 &   0 &   0 &   4 &   3 & 158 & 89.3 \\
  \hline
  \hline
  Overall           & & & & & & & & & &                                         & 90.13 \\
  \hline
\end{tabular}

For reference, the neural net did not outperform the naive bayes classification (with 16 principle components) that was done in a previous assignment, although adding more rounds of training or varying the number of nodes in the hidden layer might remedy this. Also, the training time was extraordinarily long, so other back-propagation techniques should be used to improve the training time.

%
% \newpage
\subsection{Code}
The following R code was used to perform the LDA and classification:
\begin{verbatim}
#
# Initial Setup
#
setwd("C:/Users/Terence/Documents/GitHub/STAT775/HW06")

#
# Data Cleaning
#
DATA.PATH <- "../DataSets/zip.data/"
ZIP.TRAIN.FILE.NAME <- paste0(DATA.PATH, "zip.train")
ZIP.TEST.FILE.NAME <- paste0(DATA.PATH, "zip.test")

get.multi.dimensional.label <- function(label) {
#
# Expands a single digit label to a {0, 1} vector-label
#
# Args:
#   label: a single numeric value [0-9]

  multi.dimensional.label <- matrix(0, nrow = 1, ncol = 10)
  multi.dimensional.label[1, as.numeric(label) + 1] <- 1
  return(multi.dimensional.label)
}

squash.multi.dimensional.label <- function(multi.dim.label) {
#
# Collapses a digit's {0, 1} vector-label to a single-value label
#
# Args:
#   mulit.dim.label: a 10 x 1 {0, 1} vector indicating the true class

  label <- -1
  highest.probability <- max(multi.dim.label)
  for (i in 1:nrow(multi.dim.label)) {
    if (multi.dim.label[[i]] == highest.probability) {
      label <- i - 1
    }
  }
  return(label)
}

read.data.tuples <- function(file.path.name) {
  data.fram.e <- read.table(file.path.name)

  data <- data.matrix(data.fram.e[, -1])

  targets <- matrix(nrow = nrow(data.fram.e), ncol = 10)
  for (i in 1:nrow(data.fram.e)) {
    targets[i, ] <- get.multi.dimensional.label(data.fram.e$V1[[i]])
  }

  data.tuple <- list(
    observations = data,
    labels = data.matrix(data.fram.e[, 1]),
    targets = targets
  )
  return(data.tuple)
}

#
# PCA
#

get.pca.summary <- function(data, num.components = 20) {
#
# Args:
#   data: an n x m matrix of n observations of m dimensions
#   num.components: the number of principle components to keep

  num.component.s <- min(ncol(data), num.components)
  n.obs <- nrow(data)
  full.dimensionality <- ncol(data)
  mu <- colMeans(data)

  # get centered data
  x <- data
  for (i in 1:n.obs) {
    x[i, ] <- data[i, ] - mu
  }

  # covariance matrix
  sigma <- t(x) %*% x
  sigma <- sigma * (1.0 / n.obs)

  eigen.decomposition <- eigen(sigma, F)  # TODO: is cov symmetric? Not sure...
  eigen.vectors <- eigen.decomposition$vectors[, 1:num.component.s]

  pca.summary <- list(
    rotation = eigen.vectors,
    mu = matrix(mu, nrow = 1, ncol = full.dimensionality)
  )

  return(pca.summary)
}

predict <- function(pca.summary, data) {
#
# Args:
#   data: an n x d matrix of observations; rows are observations
#   pca.summary: a tuple of the rotation matrix (eigenvectors as columns) and
#                the mean (row vector of column means) computed in the pca
#                computations

  x <- data
  for (i in 1:nrow(data)) {
    x[i, ] <- data[i, ] - pca.summary$mu
  }

  return (t(t(pca.summary$rotation) %*% t(x)))
}

#
# Neural Net
#

sigmoid <- function(x) {
#
# Args:
#   x: a numeric or vector; the function is applied element-wise

  return(1.0 / (1.0 + exp(-x)))
}

sigmoid.derivative <- function(x) {
  #
  # Args:
  #   x: a numeric or vector

  return(sigmoid(x) * (1.0 - sigmoid(x)))
}

construct.neural.net <- function(
  topology = c(2, 2, 1),
  activation = sigmoid,
  activation.derivative = sigmoid.derivative,
  debug = F) {
#
# Args:
#   topology: a list or vector of the dimensions of each layer
#   activation: a function to be used for the activation of each unit
#   activation.derivative: a function that is the derivative of activation

  layer.weights <- list()
  derivative.matrices <- list()
  outputs <- list()

  previous.layer.dim <- 1
  next.layer.dim <- 1
  for (i in 1:(length(topology) - 1)) {
    previous.layer.dim <- topology[[i]] + 1  # +1 for bias
    next.layer.dim <- topology[[i + 1]]
    num.elements <- (previous.layer.dim) * next.layer.dim

    layer.weights[[i]] <- matrix(
      if(debug) {rep(1, num.elements)}
      else {runif(n = num.elements, min = -0.001, max = 0.001)},
      nrow = previous.layer.dim,
      ncol = next.layer.dim
    )

    outputs[[i]] <- matrix(0, nrow = next.layer.dim, ncol = 1)
    derivative.matrices[[i]] <- diag(0, next.layer.dim)
  }

  return(list(
    input.dim = topology[[1]],
    output.dim = next.layer.dim,  # should be dim of last layer
    n.layers = length(layer.weights),
    activation = activation,
    activation.deriv = activation.derivative,
    input = matrix(0, nrow = 1, ncol = topology[[1]]),
    output = matrix(0, nrow = tail(topology, 1)[[1]], 1),
    weights = layer.weights,
    outputs = outputs,
    derivatives = derivative.matrices
  ))
}

apply.inputs <- function(net, x, for.training = T) {
#
# Args:
#   x: a 1 x n vector of inputs; n should be the same as for net
#   net: a structure with all of the appropriate data for a neural network,
#        as created by construct.neural.net()
#   for.training[T]: currently unused

  net$input <- matrix(x, nrow = 1)
  previous.output <- cbind(net$input, 1)
  for (i in 1:net$n.layers) {
    weighted.sums <- previous.output %*% net$weights[[i]]

    net$outputs[[i]] <- net$activation(weighted.sums)

    net$derivatives[[i]] <- diag(
      as.list(net$activation.deriv(weighted.sums)),
      length(net$outputs[[i]])
    )

    previous.output <- cbind(net$outputs[[i]], 1)
  }

  net$output <- t(tail(net$outputs, 1)[[1]])

  return(net)
}

backprop.weight.update <- function(net, target, learning.rate = 0.1) {
#
# Args:
#   net: a neural net object that has had inputs applied and derivatives stored
#   target: a column vector; the target output that should have been observed
#   learning.rate: the learning rate of the network
#                  TODO: refactor learning.rate to be less hacky, allow for
#                        advanced techniques

  last.index <- net$n.layers

  deltas <- list()
  error <- matrix(net$output, ncol = 1) - matrix(target, ncol = 1)
  deltas[[last.index + 1]] <- error
  W <- diag(1, nrow = nrow(error))
  D <- net$derivatives[[last.index]]
  for (i in last.index:1) {
    deltas[[i]] <- D %*% W %*% deltas[[i + 1]]

    if (i > 1) {
      D <- net$derivatives[[i - 1]]
      W <- net$weights[[i]]
      W <- W[1:(nrow(W) - 1), ]
    }
  }

  weight.updates <- list()
  o.hat <- cbind(net$input, 1)
  for (i in 1:last.index) {
    weight.updates[[i]] <- -learning.rate * t(deltas[[i]] %*% o.hat)

    if (i < last.index) {
      o.hat <- cbind(net$outputs[[i]], 1)
    }
  }

  for (i in 1:length(weight.updates)) {
    net$weights[[i]] <- net$weights[[i]] + weight.updates[[i]]
  }

  return(net)
}

#
# Main
#
NUM.PRINCIPLE.COMPONENTS <- 20
NUM.DIGITS <- 10
NUM.EPOCHS <- 500
CONFUSION.LABELS <-
  c(' 0 ', ' 1 ', ' 2 ', ' 3 ', ' 4 ', ' 5 ', ' 6 ', ' 7 ', ' 8 ', ' 9 ')

# training ###################################
train <- read.data.tuples(ZIP.TRAIN.FILE.NAME)

pca.summary <- get.pca.summary(
  data = train$observations,
  num.components = NUM.PRINCIPLE.COMPONENTS
)

train$observations <- predict(pca.summary, train$observations)

k <- 20
digit.net <- construct.neural.net(
  topology = c(NUM.PRINCIPLE.COMPONENTS, k, NUM.DIGITS),
  activation = sigmoid,
  activation.deriv = sigmoid.derivative
)

for (t in 1:NUM.EPOCHS) {
  for (i in 1:nrow(train$targets)) {
    digit.net <- apply.inputs(
      net = digit.net,
      x = matrix(train$observations[i, ], nrow = 1)
    )
    digit.net <- backprop.weight.update(
      net = digit.net,
      target = matrix(train$targets[i, ], ncol = 1),
      learning.rate = 2
    )
  }
}

# testing ###################################
test <- read.data.tuples(ZIP.TEST.FILE.NAME)
test$observations <- predict(pca.summary, test$observations)

num.correct <- 0
confusion.matrix <- matrix(0, nrow = NUM.DIGITS, ncol = NUM.DIGITS + 1)
row.names(confusion.matrix) <- CONFUSION.LABELS
colnames(confusion.matrix) <- c(CONFUSION.LABELS, '% correct')
for (i in 1:nrow(test$observations)) {
  prediction <- squash.multi.dimensional.label(
    apply.inputs(
      net = digit.net,
      x = matrix(test$observations[i, ], nrow = 1),
      for.training = F
    )$output
  )

  if (prediction == test$labels[[i]]) {
    num.correct <- num.correct + 1
  }

  confusion.matrix[test$labels[[i]] + 1, prediction + 1] <-
    confusion.matrix[test$labels[[i]] + 1, prediction + 1] + 1
}

class.totals <- rowSums(confusion.matrix)
for (i in 1:nrow(confusion.matrix)) {
  confusion.matrix[i, 11] <- 100 * confusion.matrix[i, i] / class.totals[[i]]
}

print(100 * num.correct / nrow(test$labels))
print(confusion.matrix)
\end{verbatim}

\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Overview of Uncertainty Quantification}\label{sec:intro}

In the last several decades, there has been an increasing reliance on quantitative predictions from computational, simulation-based models of physical systems to inform engineering design, predict the behavior of physical systems, and even shape public policy, e.g., see \cite{VO14, VO15, BDMV, HV}, for just a few such examples.
It is therefore more important than ever to quantify, and whenever possible, reduce, the uncertainties impacting such models.
Unfortunately, many key characteristics governing system behavior, described as model inputs (referred to here as parameters), are often hidden from direct observation.
When observable model output data are sensitive to variations in these parameters, we formulate and solve inverse problems using the output data to quantify uncertainties in parameters.
Inverse problems therefore play a vital role in the uncertainty quantification (UQ) community.

In UQ, uncertainties are categorized as being either aleatoric (i.e., irreducible) or epistemic (i.e., reducible) in nature, which are often quantitatively described and interpreted in distinct ways.
Below, we use abstractions of conceptual examples to distinguish how both types of uncertainties arise in parameters, and subsequently impact the type of inverse problem that is solved to quantify these uncertainties.
% and their associated inverse problems while simultaneously comparing and contrasting methodologies for solving these problems.
This distinction further serves to highlight the contributions of this thesis.

Consider modeling the manufacturing process of an engineered system involving various electrical or mechanical components.
The intrinsic variability in component properties, e.g., due to impurities in raw materials used in their construction, are aleatoric in nature.
%These are quantitatively characterized as probability measures.
Component properties define a sample space (the set of all possible outcomes), and combining this sample space with a description of measurable events along with a probability measure defines a probability space.
Scalar-valued model parameters associated with component properties defines a random vector (i.e., a measurable function) from this probability space of components into the parameters required by the model.
Subsequently, the mapping from parameters to observable model outputs defines what we refer to as a Quantities of Interest (QoI) map.
Observation of a probability measure on the range of the QoI map leads to the formulation of a stochastic inverse problem (SIP), where the goal is to pullback the observed probability measure onto the space of parameters.
Conceptually, a pullback measure is data-consistent in the sense that its push-forward through the QoI map matches the observed probability measure.

While it is possible to construct explicit approximations to data-consistent measures in terms of estimating measurable events and their probabilities in the parameter space (e.g., see \cite{BET+14}), such ``set-based'' approximations become computationally intractable for high-dimensional parameter spaces or geometrically complex and/or computationally expensive QoI maps.
A recently developed density-based approach \citep{BJW18a, BJW18b, BWY20} solves the SIP in a novel way by first solving a stochastic forward problem (SFP).
Specifically, an {\em initial} probability measure is first specified on the parameters to encode any prior knowledge of parameter variability.
Then, a SFP is solved where the push-forward of the initial probability measure is used to define a {\em predicted} probability measure on the QoI.
The discrepancy between the predicted and {\em observed} probability measures on the QoI, expressed as a ratio of probability density functions (more generally, Radon-Nikodym derivatives), is then used to {\em update} the initial probability density.
The {\em updated} probability measure associated with this density is then data-consistent.
Moreover, the updates to the initial probability measure only occur in directions informed by the QoI.
In other words, the initial probability measure serves to regularize the space of all pullback measures solving the SIP to produce a unique solution.

The SIP and its solution methodologies are based on rigorous measure theory using the Disintegration Theorem \citep{Dellacherie_Meyer_book, Chang_Pollard} as the central tool in establishing existence, uniqueness, and stability of solutions.
Updated probability measures often have complex structures that are not well approximated by a family of parametrically defined distributions (e.g., Gaussian).
This attribute of the solution further distinguishes this measure-theoretic approach from typical Bayesian-inspired approaches, e.g., Hierarchical Bayesian methods \citep{Smith, Tarantola_book, Wikle1998}, that specify prior distributions from a parametric family of distributions along with additional prior distributions on the so-called hyper-parameters introduced by this parametric family (e.g., the means and variances of a Gaussian).
Subsequently, solutions to the SIP using Bayesian approaches will not, in general, produce solutions (defined as posterior distributions) whose push-forward matches the observed distribution.
In fact, the push-forward of the posterior is not even of general interest in most Bayesian paradigms.
Instead, the posterior predictive, which defines the distribution of possible unobserved values is of central interest \citep{Smith}.
The posterior predictive is constructed as a conditional distribution on the observations but makes practical use of the posterior through a marginalization.
These differences are actually not surprising when one considers that the Bayesian inverse problem that is perhaps most familiar in the UQ community solves an inverse problem involving epistemic uncertainty, as we describe below and expand upon in Section~\ref{sec:compare}.

In a typical Bayesian framework \citep{0266-5611-7-5-003,
  Kennedy_O_JRSSSB_2001, Tarantola_book, MNR07, CDS10, starktenorio,
  AlexanderianPetraStadlerEtAl14, Bui-ThanhGhattas14, Ernst2014,
  0266-5611-30-11-110301, ROM:CMW_2016, Stuart10,
  cockayneoatessullivangirolami}, one of the initial assumptions is that data obtained on a QoI are polluted by measurement error, i.e., the data are ``noisy.''
  Measurement errors can theoretically be reduced using improved measurement instruments (i.e., they are epistemic in nature).
  A data-likelihood function is used to express the relative likelihoods that all of the data came from a particular choice of the parameter.
  Encoding any initial assumptions about which parameters are more likely than others as a prior density allows the formal construction of a posterior density as a conditional density that describes the difference in relative likelihoods of any parameter value given the data.

It is common to use specific point estimators such as the maximum a posteriori (MAP) point given by the mode of the posterior as the actual solution to the inverse problem.
The posterior is then re-interpreted as providing descriptions of uncertainty in that specific point estimate.
The Bernstein-von Mises theorem \citep{vonmises} provides conditions under which the posterior will become concentrated around the single true parameter in the limit of infinite data \citep{Smith}.

Returning to the hypothetical example of modeling a manufacturing process, the typical Bayesian paradigm described above is most applicable to a specific instance of the manufactured system.
In other words, suppose a single system is extracted from the end of the production line.
We subject this system to experiments for which we collect data on the system response, and we are interested in using this data to determine the precise parameter values associated with this single system.
The Bayesian framework is fundamentally designed to address such a problem while the measure-theoretic framework as presented in \cite{BJW18a, BJW18b, BWY20} is not.
The SIP is concerned with modeling the variability in the outputs of the production line as a collection, which is of particular interest to quality control.

The main contributions of this thesis are the extension of the SIP framework to address the reduction of epistemic uncertainty.
This is accomplished by formulating parameter identification problems as ones involving pullbacks of distributions of residuals.
In the following section we provide background and history for the SIP and subsequently define the Deterministic Inverse Problem (DIP), which is the term we use to refer to the problem addressed by the Bayesian framework.
We then compare the two frameworks and provide some illustrative examples to draw attention to the key differences between them.
The chapter will conclude with a summary of the assumptions, properties, and stability of the solutions to the SIP which will be considered throughout this thesis.

\vfill


%\documentclass[aps,prb,twocolumn,groupedaddress,nofootinbib,floatfix]{revtex4}
\documentclass{article}
\usepackage{bm}
\usepackage{mathtools}
\usepackage{mathrsfs}
\usepackage[parfill]{parskip}

\usepackage[margin=1in]{geometry}

\usepackage{color}
\newcommand\editremark[1]{{\color{red}#1}}

\begin{document}

\title{EM PE Implementation and Usage}

%
\author{Ben Champion}

\date{\today}

\maketitle

\section{Models}

\subsection{Simple analytic kilonova model}

\textbf{Note:} some of this text comes directly from \cite{Villar_2017}.

The single-component, two-component, and three-component analytic kilonova models provided with the code are an implementation of the model described in \cite{Villar_2017}.
In the following equations, $M$ is the $r$-process ejecta mass and $v$ is the ejecta velocity.
Note that for now we assume the ejecta consists entirely of $r$-process material, so $M$ is the full ejecta mass.
The radioactive heating rate at time $t$ is given by \cite{Korobkin_2012}:
\begin{equation}
    L_\text{in}(t) = 4 \times 10^{18} M \times \left [ 0.5 - \pi^{-1} \arctan \left ( \frac {t - t_0} {\sigma} \right ) \right ]^{1.3} \text{ erg s}^{-1}
\end{equation}
where $t_0 = 1.3$ s and $\sigma = 0.11$ s are constants.

Only a fraction of $L_\text{in}$ powers the kilonova, given by the thermalization efficiency $\epsilon_\text{th}$.
This is approximated analytically in \cite{Barnes_2016}:
\begin{equation}
    \epsilon_\text{th}(t) = 0.36 \left [ e^{-a t} + \frac {\ln(1 + 2 b t^d)} {2 b t^d} \right ]
\end{equation}
The parameters $a$, $b$, and $d$ are constants that depend on the ejecta mass and velocity; an interpolation of Table 1 in \cite{Barnes_2016} is used in the model.

The bolometric luminosity is calculated as
\footnote{\cite{Villar_2017} is missing a factor of time in the denominator (this can be seen from the units -- our $L_\text{bol}$ has the correct units of erg s$^{-1}$).
The factor of 1/$t_d$ seems to produce reasonable results with the correct units, and better aligns with e.g. \cite{Chatzopoulos_2012}.}
\begin{equation}
    L_\text{bol}(t) = \frac {2} {t_d} \exp{\left ( \frac {-t^2} {t_d^2} \right ) }
                    \int_0^t L_\text{in} \epsilon_\text{th} \exp{\left ( \frac {t^2} {t_d^2} \right ) } \frac {t} {t_d} dt
\end{equation}
where $t_d$ is the diffusion timescale, $t_d = \sqrt{2 \kappa M / \beta v c}$, $\kappa$ is the opacity, and $\beta = 13.7$ is a dimensionless constant related to the ejecta's geometry.

Lightcurves are calculated by assuming the kilonova behaves as a blackbody photosphere that expands at a velocity $v$.
The blackbody temperature is generally defined by its bolometric luminosity; however, once it cools to a critical temperature $T_c$, the photosphere recedes into the ejecta and the temperature remains fixed.
The photosphere temperature is
\footnote{\cite{Villar_2017} has a mistake here: $\sigma_\text{SB}$ should not be squared.}
\begin{equation} \label{T_c}
    T_\text{phot}(t) = \max \left [ \left ( \frac {L_\text{bol}(t)} {4 \pi \sigma_\text{SB} v^2 t^2} \right )^{1/4}, T_c \right]
\end{equation}

When $T_\text{phot} > T_c$, the photosphere radius is simply $R_\text{phot} = v t$.
When $T_\text{phot} = T_c$ (i.e. the photosphere has receded into the ejecta), the photosphere radius is
\begin{equation}
    R_\text{phot}(t) = \left ( \frac {L_\text{bol}(t)} {4 \pi \sigma_\text{SB} T_c^4} \right )^{1/2}
\end{equation}

The flux density at frequency $\nu$ is given in \cite{Metzger_2017}:
\begin{equation}
    F_\nu(t) = \frac {2 \pi h \nu^3} {c^2} \frac {1} {\exp{(h \nu / k T_\text{photo}(t))} - 1} \frac {R_\text{photo}^2(t)} {D^2}
\end{equation}
where $D$ is the source distance.
We use a fixed fiducial distance of $D = 10$ pc to calculate $F_\nu(t)$, then calculate AB magnitude with a distance modulus if necessary.

To compute multi-component lightcurves we assume each component has a photosphere that evolves independently of the others.
The total flux density is the sum of the flux densities of the individual components.
The version of the model implemented in the code uses three components with fixed opacities (a blue component with $\kappa = 0.5$ cm$^2$ g$^{-1}$, a purple component with $\kappa = 3$, cm$^2$ g$^{-1}$ and a red component with $\kappa = 10$ cm$^2$ g$^{-1}$).

\section{Parameter Estimation}

\subsection{Likelihood Function}

For lightcurve magnitudes $x_i$, model values $m_i(\bm{\theta})$ computed with a parameter vector $\bm{\theta}$, data uncertainties $\sigma_i$, and a scatter term $\sigma$, fit as a parameter, that accounts for additional uncertainty in the data and model, the log-likelihood is
\begin{equation}
    \ln \mathcal{L}(\bm{\theta}) = -0.5 \sum_{i=1}^n \left [ \frac {(x_i - m_i(\bm{\theta}))^2} {\sigma_i^2 + \sigma^2} + \ln(2 \pi (\sigma_i^2 + \sigma^2)) \right ]
\end{equation}
where the sum is taken over every data point in every band used in the analysis.

This log-likelihood is implemented as a method in \texttt{EM\_PE}'s \texttt{sampler} class, which can be imported and used in other codes.
The joint gravitational wave and EM log-likelihood is the sum of the individual log-likelihoods.

\subsection{Injection/Recovery Example}

A simple parameter estimation test using fake data can be run after installing the \texttt{EM\_PE} code to ensure it functions properly.
The repository contains a \texttt{Makefile} that generates reproducable parameter estimation runs.
To run the injection/recovery test:
\begin{verbatim}
$ make test_kilonova_3c
$ cd pe_runs/test_kilonova_3c/
$ ./sample.sh
\end{verbatim}
This series of commands will generate a posterior sample file, \texttt{samples.txt}.
Note that the default settings assume the code is being run on a machine with many CPU cores available, so it computes the likelihood in 8 parallel processes.
It will run without issue on a less powerful computer, but \texttt{sample.sh} should be modified to use fewer cores (e.g. set \texttt{--nprocs 2} to use two cores).

To produce a corner plot and lightcurve plot, run
\begin{verbatim}
$ ./plot_corner.sh
$ ./plot_lc.sh
\end{verbatim}
This will produce plots similar to Figure \ref{fig:corner} and Figure \ref{fig:lightcurves}.

\begin{figure}[!ht]
    \centering
    \includegraphics[width=6.5in]{corner.png}
    \caption{Posterior distribution for the injection/recovery test (the blue lines show the true parameter values). For this example, $T_c$ was fixed for each component and the distance was fixed to 40.0 Mpc.}
    \label{fig:corner}
\end{figure}

\begin{figure}[!ht]
    \centering
    \includegraphics[width=6.5in]{lc.png}
    \caption{Fake photometry data and lightcurve models for the injection/recovery test. The solid line shows the lightcurve model evaluated at the maximum-likelihood parameters.}
    \label{fig:lightcurves}
\end{figure}

\subsection{GW170817 Example Analysis}

\textit{Coming soon}

\clearpage

\bibliographystyle{h-physrev5.bst}
\bibliography{bibliography}

\end{document}


\chapter{OpenMath/MathML Translation} \label{analysis}

MathML and OpenMath are closely related, serving a similar purpose of
conveying mathematics across different applications. The aim of this
analysis is to relate MathML and OpenMath to illustrate their
similarities and differences. We intend it to be application
independent, highlighting the problems arising when developing programs
translating MathML to OpenMath and vice versa.

As is stated in the OpenMath standard \cite{openmathspec}, OpenMath
objects have the expressive power to cover all areas of computational
mathematics. This is certainly not the case with MathML. However,
MathML was designed to be displayed on any MathML compliant renderer.
The possibility to translate between them would allow OpenMath objects
to be displayed, and MathML objects to have a wider semantic scope. But is
a translation possible?

OpenMath and MathML have many common aspects. Some features of the
standards help facilitate the translation, mainly that the structure of
both standards is very similar.  They both use prefix operators and are
XML \cite{xml}\index{XML} based. They both construct their objects by
applying certain rules recursively. Such similarities facilitate
mapping across both standards.

Because both standards are XML based, their syntax is governed by the
rules of XML syntax. In other words, the details of using tags,
attributes, entity references and so on are defined in the XML language
specification. By complying with the XML standard it is possible to use
generic XML generators and validators. These can be programmed for the
application being developed, or existing ones can be used.

Finally, OpenMath has specific content dictionaries\index{content
dictionaries} mirroring MathML's semantic scope, which permit a
straightforward mapping between both recommendations. Since both
standards are simply different ways of representing mathematics,
designed with translation in mind, mapping one to the other is
certainly possible.

We shall look at all the areas of both recommendations where
differences occur and how they pose difficulties to designing a
translator. It is important to understand how objects are constructed
and what they represent. We will then discuss how functions and
operators are applied on their arguments. There are various specific
structural differences between both standards which need to be properly
understood; we will attempt to explain these differences and offer a
method of translation for each one. We will also discuss how MathML
supports extensibility and to what extent it is possible to implement
such extensibility to accept new OpenMath symbols. To finish we will
give an explanation of how to handle the translation problem.

Before we start our analysis, it is important that we define a few
terms related to our analysis. We also encourage the reader to have a
look at the standards in order to better appreciate this analysis.
MathML and OpenMath {\it
objects}\index{MathML!objects|textbf}\index{OpenMath!objects|textbf}
convey the meaning of a mathematical expression and are represented as
labelled trees. An object can also be called an {\it expression}. A
{\it symbol}\index{OpenMath!symbol|textbf} in OpenMath is used to
represent a mathematical concept. For instance {\it plus} or {\it max}
are considered symbols. We call {\it elements} the words enclosed
within \texttt{$<$$>$} such as \texttt{$<$apply$>$} or
\texttt{$<$OMA$>$}. Elements enclose other XML data called their
`content' between a `start tag' (sometimes called a `begin tag') and an
`end tag', much like in HTML\index{Html}. There are also `empty
elements' such as \texttt{$<$plus/$>$}, whose start tag ends with /$>$
to indicate that the element has no content or end tag.

\section{Constructing Objects} \label{constructors}

Constructing objects in MathML and OpenMath is done in similar ways.
MathML uses elements termed {\it containers} and OpenMath uses elements
called {\it constructs}.  They are both closely related, and most of
them are easily interchangeable. The nature of the
constructors\index{constructors} in both standards is rather different,
but their usage is the same.

OpenMath objects can be created by applying a symbol onto a series of
arguments.  These are the objects created by {\it application} and are
surrounded by \texttt{$<$OMA$>$\ldots$<$/OMA$>$} elements. In MathML
the approach is different. MathML possesses more constructors and they
are more specific.  It is important to note that OpenMath objects
constructed with the \texttt{$<$OMA$>$} element may translate to
various constructors in MathML.

In OpenMath for instance, defining a list or a matrix would be done by
applying the application constructor on the {\it list} or {\it matrix}
symbol followed by the contents of the list or matrix. In MathML
however, a list would require the \texttt{$<$list$>$$\ldots<$/list$>$}
constructor, and a matrix would need the
\texttt{$<$matrix$>$\ldots$<$/matrix$>$} constructor.

Most OpenMath symbols constructed by application are constructed in
MathML using the {\tt $<$apply$>$} constructor. But there are
exceptions which do not map to {\tt $<$apply$>$} tags. It is important
that all exceptions such as \verb|matrix|, \verb|list|, \verb|set| and
others are determined and that the appropriate MathML constructor is
used when translating. Table \ref{const} shows what possible MathML
constructors {\tt $<$OMA$>$} can map to.

OpenMath objects can also be constructed using the
\texttt{$<$OMBIND$>$} element.  This consists in binding a symbol to a
function with zero or more bound variables.  MathML does not have an
equivalent, and so symbols which use the {\it binding} construct in
OpenMath, like {\tt lambda} or {\tt forall}, may have different ways of
being constructed in MathML. {\tt lambda} uses a specific constructor
in MathML, whereas {\tt forall} uses the {\tt $<$apply$>$} construct.
It is very important in order to ensure proper translation, to
determine which OpenMath symbols use the binding constructor and what
their MathML equivalent is.

There are objects constructed by attributing a value to an object.
These are objects constructed by {\it attribution} and employ the {\tt
$<$OMATTR$>$} elements. MathML also allows objects to possess
attributed values called attributes. The translation is
straightforward.

There are other constructors which we do not mention in more detail
because there exists a direct mapping between both standards. This is
the case of \texttt{$<$OMI$>$, $<$OMF$>$}, \texttt{$<$OMV$>$}
\texttt{$<$cn$>$} and \texttt{$<$ci$>$}. Table \ref{const} shows the
relation between them.

\begin{table}

\begin{center}

\begin{tabular}{|l|l|}
\hline
\label{const}

{\bf OpenMath} 			&	{\bf MathML} \\ \hline

\texttt{$<$OMA$>$}		&	\texttt{$<$interval$>$, $<$set$>$, $<$list$>$, $<$matrix$>$,}\\
				&	\texttt{$<$vector$>$, $<$apply$>$, $<$lambda$>$, $<$reln$>$}. \\
\texttt{$<$OMATTR$>$}		&	{\it attributes associated to a tag} \\
\texttt{$<$OMI$>$, $<$OMF$>$}	&	\texttt{$<$cn$>$} \\
\texttt{$<$OMV$>$}		&	\texttt{$<$ci$>$} \\
\texttt{$<$OMSTR$>$}		&	{\it not supported} \\
\texttt{$<$OMBIND$>$}		&	{\it not supported} \\
{\it not supported}		&	\texttt{$<$declare$>$} \\

\hline

\end{tabular}

\end{center}
\caption{Relation between constructors}

\end{table}

\section{Elements and Functions}
\label{funcs}

MathML has a classification\footnote{MathML standard section 4.2.3}
which categorises elements according to the number of arguments they
accept and the types of these arguments.  This classification can be
summarised for our purpose into the following:

\begin{description}

\item[unary elements] accepting 1 argument

\item [binary elements] accepting 2 arguments

\item [nary elements] accepting 3 or more arguments

\item [operators:] elements whose arguments are given following a
specific syntax. This includes symbols such as {\tt int, sum, diff,
limit, forall} and a few others.

\end{description}

This classification is not explicitly stated in the OpenMath standard
but can also be used since OpenMath symbols fit well into these
categories. By gathering OpenMath and MathML symbols into these defined
groups according to their syntax, it is possible to define specific
translating procedures which deal with all symbols in one group in the
same way.

For instance one procedure could parse through any unary function by
reading in the symbol and then the one argument. Printing out unary
functions would be done by one procedure which would output the symbol
in MathML or OpenMath followed by that one argument.

The advantages of this classification are that it greatly simplifies
the translation.  Parsing and generation of all symbols would then be
the task of a few generic procedures. However, symbols contained in the
{\it operators} group require more attention, since they have different
ways of reading in arguments. Specific procedures need to be
implemented for such cases.  We will discuss these in more detail
later.

\subsection{The Scope of Symbols} \label{scope}

When dealing with a function or an operator in mathematics, it is
important that its scope is well defined. MathML and OpenMath both
specify the scope\index{scope} of an operator by enclosing it with its
arguments inside opening and closing tags. In MathML, the opening and
closing tags \texttt{$<$apply$>$} are employed, and in OpenMath one
uses the opening and closing tags \texttt{$<$OMA$>$}.

However, OpenMath's grammar as it is defined in the OpenMath standard
in section 4.1.2 can produce OpenMath objects where the scope of an
operator is ambiguous, in which case a parser would have great
difficulties validating the syntax for translation. Let us illustrate
this problem with the two OpenMath expressions in figure \ref{omscope} which are grammatically
correct.

\begin{figure}[h]

\begin{tabular}{ l l }

{\bf Example 1}					& {\bf Example 2}\\
						& \\
\verb|<OMOBJ>|					&\verb|<OMOBJ>|\\
\verb| <OMA>|					&\verb| <OMA>|\\
\verb|   <OMS cd=arith1 name=plus/>|		&\verb|   <OMS cd=arith1 name=plus/>| \\
\verb|   <OMS cd=arith1 name=times/>|		&\verb|   <OMA>| \\
\verb|   <OMV name=x/>|				&\verb|     <OMS cd=arith1 name=times/>| \\
\verb|   <OMV name=y/>|				&\verb|     <OMV name=x/>| \\
\verb|   <OMV name=z/>|				&\verb|     <OMV name=y/>| \\
\verb|   <OMI>6</OMI>|				&\verb|   </OMA>| \\
\verb| </OMA>|					&\verb|   <OMV name=y/>| \\
\verb|</OMOBJ>|					&\verb|   <OMI>6</OMI>| \\
						&\verb| </OMA>| \\
						&\verb|</OMOBJ>| \\

\end{tabular}

\caption{The importance of defining scopes}
\label{omscope}
\end{figure}

Example 2 demonstrates how the use of \verb|<OMA>| tags help define
clearly the scope of each operator. A parser can then without
difficulty interpret the expression and translate it correctly. Example
1, on the other side, shows how insufficient use of \verb|<OMA>| tags
can lead to ambiguous expressions both for automatic parsers and
humans.

MathML is stricter when defining the scopes of operators. Every
operator must be enclosed with its own \verb|<apply>| tags. This
difference between both standards is source of problems. The expression
in Example 1 does not allow the scopes of the operators to be
determined with accuracy, and so an equivalent MathML expression cannot
be produced.

When developing an OpenMath/MathML translator, it is important to
specify that operator scopes in OpenMath must be accurately defined, or
else translation to MathML is not possible. The use of \verb|<OMA>|
tags must be imposed.

\section{Differences in Structure}

There are MathML and OpenMath elements which require special attention.
Mainly because there are elements constructed differently in MathML as
they are in OpenMath and because some elements have no equivalent in
both standards. Such cases must be well understood before starting to
implement any translator. We shall look at these cases and propose a
reliable method for overcoming the differences and implementing an
efficient solution. We will mention bound variables, element attributes
and constants representation.

There exist elements in both standards which represent the same
mathematical concept, but where the syntactical structure is different.
The following list shows these elements: {\it matrices, limits,
integrals, definite integrals, differentiation, partial
differentiation, sums, products, intervals, selection from a vector}
and {\it
selection from a matrix}.

\subsection{Selector functions and Matrices}

Let us first look at: {\it matrices, selection from a matrix} and {\it
selection from a vector}.  These elements exist in both
recommendations, but differ syntactically.

Selection from a matrix and from a vector is done by the
\verb|<selector/>| element in MathML and by the symbols {\it
vector\_selector} and {\it matrix\_selector} in OpenMath. Because
MathML uses the same element to deal both with matrices and vectors, it
is necessary for the parser to determine what the arguments of the
expression are before finding the correct equivalent OpenMath. If the
expression has a matrix as argument, then {\it matrix\_selector} is the
correct corresponding symbol. If the argument is a vector then the
corresponding symbol is {\it vector\_selector}.

It is important to note as well the order of arguments. The MathML
\verb|<selector/>| tag first takes the vector or matrix object, and
then the indices of selection. In OpenMath it is the other way around.
First the indices of selection are given and then the object.

Another element where differences in structure are important is the
matrix element.  OpenMath has two ways of representing matrices. One
representation defined in the \verb|"linalg1"| CD and the other defined
in the \verb|"linalg2"| CD. A matrix is defined as a series of
matrixrows in \verb|"linalg1"|, exactly as in MathML. For such
matrices, translation is straightforward.

However, \verb|"linalg2"| defines a matrix as a series of matrix
columns. This representation has no equivalent in MathML. It is
important that a translator is capable of understanding both
representations in order to offer correct translation.

When dealing with a \verb|"linalg2"| matrix, a procedure can be
implemented which given the matrix columns of a matrix, returns a
series of matrix rows representing the same matrix. From these matrix
rows, a MathML expression can be generated.

\subsection{Bound Variables} \label{boundvars}

The remaining elements {\it limit, integrals, definite integrals,
differentiation, partial differentiation, sums, and products} have a
similar structure and can be treated in a similar way when translating.
Following the classification in section \ref{funcs} these elements go
in the {\it operators} group.

What characterises these elements is that in MathML they all specify
their bound variables explicitly using the \verb|<bvar>| construct.
However, in OpenMath, the bound variables are not explicitly stated.
OpenMath expressions are the result of applying the symbol on a lambda
expression. In order to determine the bound variable the parser must
retrieve it from the lambda expression. Let us illustrate this problem
by contrasting two equivalent expressions on figure \ref{bound}

\begin{figure}[h]
\begin{tabular}{ l l }

{\bf OpenMath} 						& {\bf MathML}\\
							& \\
\verb|<OMOBJ>| 						&\verb|<math>| \\
\verb| <OMA>| 						&\verb| <apply><sum/>| \\
\verb|  <OMS cd="arith1" name="sum"/>|    		&\verb|   <bvar>| \\
\verb|  <OMA>|						&\verb|     <ci>i</ci>| \\
\verb|   <OMS cd="interval1" name="interval"/>| 	&\verb|   </bvar>| \\
\verb|   <OMI> 1 </OMI>|				&\verb|   <lowlimit>| \\
\verb|   <OMI> 10 </OMI>|				&\verb|     <cn>0</cn>| \\
\verb|  </OMA>|						&\verb|   </lowlimit>| \\
\verb|  <OMBIND>|					&\verb|   <uplimit>| \\
\verb|   <OMS cd="fns1" name="lambda"/>|		&\verb|     <cn>100</cn>| \\
\verb|   <OMBVAR>|					&\verb|   </uplimit>| \\
\verb|     <OMV name="x"/>|				&\verb|   <apply><power/>| \\
\verb|   </OMBVAR>|					&\verb|       <ci>x</ci>| \\
\verb|   <OMA>|						&\verb|       <ci>i</ci>| \\
\verb|    <OMS cd="arith1" name="divide"/>|		&\verb|   </apply>| \\
\verb|    <OMI> 1 </OMI>|				&\verb| </apply>|  \\
\verb|    <OMV name="x"/>|				&\verb|</math>|   \\
\verb|   </OMA>|					& \\
\verb|  </OMBIND>|					& \\
\verb| </OMA>|						& \\
\verb|</OMOBJ>|						& \\

\end{tabular}

\caption{Use of bound variables}
\label{bound}

\end{figure}

In MathML, the index variable is explicitly stated within the
\verb|<bvar>| tags. It is part of the \verb|<sum/>| syntax and is
obligatory.  In OpenMath, the {\it sum} symbol takes as arguments an
interval giving the range of summation and a function. Specifying the
bound variable is not part of the syntax. It is contained inside the
lambda expression. This same difference in structure exists with the
other operators mentioned above.

When translating any of these elements, it is necessary to support
automatic generation and decoding of lambda expressions. Thus when
going from OpenMath to MathML, the bound variable and the function
described by the lambda expression need to be extracted to generate
valid MathML.

When passing from MathML to OpenMath, the variable contained inside the
\verb|<bvar>| tags and the function given as argument would have to be
encoded as a lambda expression. This is possible for all MathML
expressions of this type, and correct OpenMath is simple to produce.

Thus by retrieving bound variable information from OpenMath lambda
expressions, it is possible to translate to MathML. But OpenMath
grammar does not impose the use of lambda expressions to define bound
variables. Because of this flexibility, it is possible to construct
OpenMath expressions which cannot be translated to MathML by an
automatic translator. If one looks at the \verb|"calculus1"| CD, the
OpenMath examples of {\it int} and {\it defint} do not specify their
variable of integration. A parser would not determine the variables of
integration and an equivalent MathML expression would not be possible.

This is a problem for an OpenMath/MathML translator with no easy
solution. A parser intelligent enough to extract the correct bound
variables of an expression is very difficult to implement. We recommend
that OpenMath expressions which do not specify all the necessary
information for translation are ignored. The use of lambda expressions
should be required.

\subsection{Intervals}

Some operators require an interval to be given specifying the range
within which a variable ranges. The {\it sum} or {\it product} operator
are some good examples. They both take as argument the interval giving
the range of summation or multiplication. Other operators accepting
intervals in some cases are {\it int} and {\it condition}.

Both in MathML and OpenMath these operators define ranges with
intervals, but differently. OpenMath defines intervals using specific
interval defining symbols found in the {\tt interval1} CD. MathML can
use either the interval element or the tags \verb|<lowlimit>| and
\verb|<upperlimit>|. These two tags do not have an OpenMath equivalent
and so when encountered must be transformed into an interval. This is
not difficult since one must simply merge the lower and upper limits
into the edges of an interval.

\subsection{MathML attributes}

There are OpenMath symbols which map to the same MathML element, and
are only distinguished by the attributes characterising the MathML
element. A MathML element which illustrates this is \verb|<interval>|.
The interval element in MathML has a \verb|closure| attribute which
specifies the type of interval being represented. This attribute takes
the following values:  \verb|open|, \verb|closed|, \verb|open_closed|,
\verb|closed_open|.  Depending on the attribute value, a different
OpenMath symbol will be used in the translation. The following example
illustrates how one element with different attribute values maps to
different OpenMath symbols.

\begin{center}

\begin{verbatim}
      <interval type="closed">

      <OMS cd="interval1" name="interval_cc"/>
\end{verbatim}

\end{center}

\noindent are equivalent and so are

\begin{center}

\begin{verbatim}
      <interval type="open">

      <OMS cd="interval1" name="interval_oo"/>
\end{verbatim}

\end{center}

When a translator encounters such elements, it is necessary that the
MathML elements generated posses these attributes, or else semantic
value is lost. Table \ref{allatts} shows the relation between all
MathML elements whose attributes are of importance and their equivalent
OpenMath symbols.

\begin{table}[h]
\begin{center}

\begin{tabular}{|l|l|l|} \hline
\label{allatts}

{\bf MathML element}  		&{\bf Attribute values}			& {\bf
OpenMath symbol} \\ \hline
\verb|<interval>|		&{\it default}				& {\it interval} \\
				&\verb|closure="open_closed"|		& {\it interval\_oc}	\\
				&\verb|closure="closed_open"|		& {\it interval\_co}	\\
				&\verb|closure="closed"|		& {\it interval\_cc}	\\
				&\verb|closure="open"|			& {\it interval\_oo}	\\ \hline
\verb|<tendsto>|		&{\it default}				& {\it above} 	\\
\verb||				&\verb|type="above"|			& {\it above}	\\
\verb||				&\verb|type="below"|			& {\it below}	\\
\verb||				&\verb|type="both_sides"|		& {\it null}	\\ \hline
\verb|<set>|			&{\it default}				& {\it set}	\\
				&\verb|type="normal"|			& {\it set}	\\
				&\verb|type="multiset"|			& {\it multiset}	\\
\hline

\end{tabular}

\end{center}
\caption{Equivalent OpenMath symbols to the different attribute values of MathML
elements }

\end{table}

\subsection{MathML constants}

In MathML, constants are defined as being any of the following:
\verb|e|, \verb|i|, \verb|pi|, \verb|gamma|, \verb|infinity|,
\verb|true|, \verb|false| or \verb|not a number (NaN)|. They appear
within \verb|<cn>| tags when the attribute \verb|type| is set to
\verb|constant|. For instance $\pi$ would be represented in MathML as:

\begin{verbatim}
        <cn type="constant">pi</cn>
\end{verbatim}

In OpenMath, these constants all appear as different symbols and from
different CDs.  Hence, we face a similar problem as we did with MathML
attributes. The \verb|<cn>| tag with the attribute set to
\verb|constant| can map to different OpenMath symbols.

It is important that the translator detects the use of the
\verb|constant| attribute value and maps the constant expressed to the
correct OpenMath symbol.

MathML also allows to define Cartesian complex numbers and polar
complex numbers.  A complex number is of the form two real point
numbers separated by the \verb|<sep/>| tag. For instance $3+4i$ is
represented as:

\begin{verbatim}
        <cn type="cartesian_complex"> 3 <sep/> 4 </cn>
\end{verbatim}

OpenMath is more flexible in its definition of complex numbers. The
real and imaginary parts, or the magnitude and argument of a complex
number do not have to be only real numbers. They may be variables. This
allows OpenMath to represent numbers such as $x+iy$ or $re^{i\theta}$
which cannot be done in MathML.

So how should one map such an OpenMath expression to MathML? Because
there is no specific construct for such complex numbers, the easiest
way is to generate a MathML representation using simple operators. The
two expressions in figure \ref{compls} are equivalent and illustrate how a
translator should perform:

\begin{figure}[h]

\begin{verbatim}
      <OMOBJ>
        <OMA>
          <OMS cd="nums1" name="complex_polar"/>
          <OMV name="x"/>
          <OMV name="y"/>
        </OMA>
      </OMOBJ>
\end{verbatim}

\begin{verbatim}
       <math>
          <apply><times/>
             <ci> x </ci>
             <apply><exp/>
                <apply><times/>
                   <ci> y </ci>
                   <cn type="constant"> &imaginaryi; </cn>
                </apply>
             </apply>
          </apply>
       </math>
\end{verbatim}

\caption{How to translate complex numbers}
\label{compls}
\end{figure}

The problem is the same when representing rationals, since OpenMath
allows variables to be used as elements of a rational number, whereas
MathML only allows real numbers.

\subsection{{\tt partialdiff} and {\tt diff}}

In both standards it is possible to represent normal and partial
differentiations. But the structures are different. Let us first look
at {\tt diff}. In MathML, it is possible to specify the order of the
derivative. In OpenMath, differentiation is always of first order.
The trouble here is translating MathML expressions where the order of
derivation is higher than one. There is no equivalent representation in
OpenMath.

What can be done to overcome this discrepancy is to construct an
OpenMath expression differentiated as many times as is specified by the
MathML derivation order. For instance, when dealing with a MathML
second order derivative, the equivalent OpenMath expression could be a
first order derivative of a first order derivative.  This will surely
generate very verbose OpenMath in cases where the order of derivation
is high, but at least will convey the same semantic meaning and
surmounts OpenMath's limitation.

The case of partial differentiation is complicated. The representations
in both standards are very different. In MathML one specifies all the
variables of integration and the order of derivation of each variable.
In OpenMath one specifies a list of integers which index the variables
of the function. Suppose a function has bound variables $x$, $y$ and
$z$. If we give as argument the integer list $\{1,3\}$ then we are
differentiating with respect to $x$ and $z$. The differentiation is of
first order for each variable.

Translating partial differentials from OpenMath to MathML is simple,
because the information conveyed by the OpenMath expression can be
represented without difficulty by MathML syntax. However the other way
around is difficult. Given OpenMath's limitation of only allowing first
order differentiation for each variable, many MathML expressions which
differentiate with respect to various variables and each at a different
degree cannot be translated. We recommend that such MathML expressions
are discarded by the translator.

\section{Elements not Supported by both Standards}

There are some elements which have no equivalent in both standards.
These are mainly the MathML elements \verb|<condition>| and
\verb|<declare>|and the OpenMath {\it matrixrow} and {\it matrixcolumn}
symbols.

\subsection{{\tt $<$condition$>$}}

The \verb|<condition>| element is used often throughout MathML and is
necessary to convey certain mathematical concepts. There is no direct
equivalent in OpenMath, making translation impossible for certain
expressions.

The \verb|<condition>| element is used to define the `such that'
construct in mathematical expressions.  Condition elements are used in
a number of contexts in MathML. They are used to construct objects like
sets and lists by rule instead of by enumeration. They can be used with
the {\tt forall} and {\tt exists} operators to form logical
expressions. And finally, they can be used in various ways in
conjunction with certain operators. For example, they can be used with
an int element to specify domains of integration, or to specify
argument lists for operators like min and max.

The example in figure \ref{forall} represents $\{\forall x | x<9:
x<10\}$ and shows how the \verb|<condition>| tags can be used in a
MathML expression. This MathML expression has no OpenMath equivalent
because OpenMath does not allow to specify any conditions on bound
variables.

\begin{figure}[h]

\begin{verbatim}
      <math>
        <apply><forall/>
          <bvar>
            <ci> x </ci>
          </bvar>
          <condition>
            <apply><lt/>
              <ci> x </ci>
              <cn> 9 </cn>
            </apply>
          </condition>
          <apply><lt/>
            <ci> x </ci>
            <cn> 10 </cn>
          </apply>
        </apply>
      </math>
\end{verbatim}

\caption{Use of {\tt $<$condition$>$}}
\label{forall}

\end{figure}

The \verb|<condition>| tags are used in the following MathML elements:
{\it set, forall, exists, int, sum, product, limit, min} and {\it max}.
In all of these elements except {\it limit}, the use of
\verb|<condition>| tags makes translation impossible.

The case of {\it limit} is different because OpenMath does allow
constraints to be placed on the bound variable; mainly to define the
limit point and the direction from which the limit point is approached.

\subsection{{\tt $<$declare$>$}}

The \verb|<declare>| construct is used to associate specific properties
or meanings with an object. It was designed with computer algebra
packages in mind. OpenMath's philosophy is to leave the application
deal with the object once it has received it. It is not intended to be
a query or programming language. This is why such a construct was not
defined. A translator should deny such MathML expressions.

\subsection{{\it matrixrow, matrixcolumn}}

In the MathML specification it is stated that {\it `The matrixrow
elements must always be contained inside of a matrix'}. This is not the
case in OpenMath where the {\it matrixrow} symbol can appear on its
own.  A matrix row encountered on its own has no MathML equivalent.
However, when it is encountered within a matrix object, then
translation is possible.

As we mentioned earlier, it is possible to translate a matrix defined
with matrixcolumns to MathML. However, if a matrixcolumn is found on
its own it does not have a MathML equivalent.

\section{Extensibility}

OpenMath already possesses a set of CDs covering all of MathML's
semantic scope. These CDs belong to the MathML CD Group.  It is clear
that these CDs must be understood by an OpenMath/MathML interface.
There are as well a few other symbols from other CDs which are not in
the MathML CD Group but can be mapped such as matrices defined in {\tt
"linalg2"}.

But OpenMath has the capability of extending its semantic scope by
defining new symbols within new content dictionaries\index{content
dictionaries}. This facility affects the design of any OpenMath
compliant application. When it comes to translating to MathML, it is
necessary that newly defined symbols are properly dealt with. A
translator should have the ability to recognise any symbol with no
mapping to MathML.

But how do we deal with most symbols outside the MathML CD Group? Or
with new symbols which will continue to appear as OpenMath evolves? How
do we map them to MathML?

MathML, as any system of content markup\index{content markup}, requires
an extension mechanism which combines notation with semantics.
Extensibility in MathML is not as efficient as in OpenMath, but it is
possible to define and use functions which are not part of the MathML
specification. MathML content markup specifies several ways of
attaching an external semantic definition to content objects.

Because OpenMath contains many elements which have no equivalent in
MathML, and because OpenMath can have new CDs amended to it, we will
need to use these mechanisms of extension. The \verb|<semantic>|
element is used in MathML to bind a semantic definition with a symbol.
An example taken from the MathML specification~\cite{mathml} section
5.2.1\footnote{CHECK!!!!{\tt
http://www.w3.org/WD$-$MathML2$-$19991222/chapter5.html\#mixing:parallel}}
shows how the OpenMath `rank' operator (non existent in MathML) can be
encoded using MathML. The MathML encoding of rank is shown in figure
\ref{rank}:

\begin{figure}[h]
\begin{verbatim}
      <math>
        <apply><eq/>
          <apply><fn>
              <semantics>
                <ci><mo>rank</mo></ci>
                <annotation-xml encoding="OpenMath">
                  <OMS cd="linalg3" name="rank"/>
                </annotation-xml>
              </semantics>
            </fn>
            <apply><times/>
              <apply><transpose/>
                <ci>u</ci>
              </apply>
              <ci>v</ci>
            </apply>
          </apply>
          <cn>1</cn>
        </apply>
      </math>
\end{verbatim}
\caption{Encoding of OpenMath symbol `rank' in MathML}
\label{rank}
\end{figure}

It shows that an OpenMath operator without MathML equivalent is easily
contained within \verb|<semantic>| tags and can be applied on any
number of arguments.

This method works well when dealing with operators constructed by {\it
application} (between \verb|<OMA>| tags), because MathML also
constructs expressions by application (between \verb|<apply>| tags). It
is assumed they take any number of arguments. However, OpenMath can
also construct expressions by binding symbols to their arguments. As we
described earlier (section \ref{constructors}), this method has no
equivalent in MathML.

So what happens when a new symbol is encountered which is constructed
by binding in OpenMath? Enveloping the new symbol inside
\verb|<semantic>| tags will produce an incorrect translation.

It is first necessary to determine if the new symbol encountered is
constructed by binding or not. In order to do so, a file describing the
new symbol specifying these details could be read in by the translator.
This file could be the CD where the symbol is defined. But
unfortunately CDs are written in a human readable way, and there is no
way a program could determine the construction method of a particular
symbol or the number and type of arguments it takes.

One would need to read in the STS file of a symbol. But the best way
would be by checking the tag preceding the new symbol given by the
OpenMath input. If it was \verb|<OMBIND>| then we are sure this symbol
is constructed by binding. Nonetheless, accurate mapping would be
impossible. As we have seen before, MathML only offers extensibility
constructing operators by application. It is not possible to define new
containers, new types, or new operators constructed differently such as
those constructed by binding.

While it is possible to define certain new symbols in MathML, the
advantages of OpenMath extensibility would create problems for a
translator to MathML. This is why it is stated in the OpenMath standard
in section 2.5 that {\it `it is envisioned that a software application
dealing with a specific area of mathematics declares which content
dictionaries\index{content dictionaries} it understands'}. A MathML
translator deals with the area of mathematics defined by MathML and
should understand all CDs within the MathML CD Group. Any other symbols
will be properly translated if they are enclosed inside \verb|<OMA>|
tags.

Extensibility is limited by the extension mechanisms offered by MathML.

\section{How to Handle the Translation problem}

Although there are surely many ways to tackle the translation problem,
there are a few requirements which must be respected by any
OpenMath/MathML translator. Mainly that content dictionaries and
symbols are dealt with correctly during translation in both directions.

In OpenMath, symbols always appear next to the content
dictionary\index{content dictionary} they belong to. The \verb|<OMS>|
element always takes two attributes: the symbol's name and the
symbol's corresponding CD. Two symbols with the same name coming from
different CDs are considered to be different.

When parsing OpenMath, a translator must ensure that the symbols read
belong to the correct CDs, if not it should conclude the symbol has a
meaning it does not understand and deal with it accordingly. Because
an OpenMath/MathML translator will understand all MathML related CDs,
symbols encountered are considered valid if they come from this CD
group. Symbols with the same name, but from unknown CDs should be
enclosed within \verb|<semantic>| tags when possible.

We face the same requirement when generating OpenMath. All OpenMath
symbols output from the translator must appear next to their correct
CDs. If we are translating the MathML element \verb|<plus/>|, the
corresponding OpenMath symbol {\it plus} must appear next to the
\verb|arith1| CD.

This requires a translator to keep a database relating each understood
symbol with its CD. This database must allow the translator to detect
unknown symbols, or to accept some symbols from different CDs with the
same name which have MathML equivalents. This is the case of {\it
matrix} which belongs to various CDs (\verb|linalg1|, \verb|linalg2|) as do
the symbols {\it in}, {\it inverse}, {\it setdiff}, {\it vector}, {\it
arcsinh} to name a few.

These symbols belonging to various CDs pose a problem when
translating from MathML to OpenMath. Which CD do we choose? {\it
inverse} for instance belongs to {\it fns1} and {\it arith2}.
Priority should be given to the CD belonging to the MathML CD group.
If both CDs belong to the MathML then common sense should guide which
CD to place. It is up to the designer.

An OpenMath/MathML interface must be very rigorous when dealing with
content dictionaries. Any mistake may produce invalid OpenMath or
reject valid OpenMath expressions.

\section{Conclusion}

It is clear now that a translation is possible. Putting apart the
difficulties described in this analysis, their are many similarities
between both standards. As we have seen, expressions are constructed
similarly and the application of functions is practically identical.

However, the various differences of structure can limit the power of a
translator in some situations. Mainly when translating partial
differentiations or applying conditions to bound variables.

The design of any translator requires a good understanding of both
standards and how they represent mathematical concepts. The
information described in this document will guide the design of an
OpenMath/MathML translator.