\section{Describing  and Measuring Electricity}

The concepts surrounding electricity and its behavior in a circuit are rich and complex; fortunately, we can limit what we need to understand to a handful of core terms and ideas. A full study of electric charge and electromagnetism is beyond the scope of this book and certainly well beyond what you will need in order to understand the circuits presented and to design circuits of your own. We will focus first on the three key terms we need - \textit{electro-motive force}, also known as emf and measured in \textbf{volts}, \textit{electric current}, a measure of quantity represented as \textbf{amps}, and \textit{impedance}, the resistance of an electric circuit as measured in \textbf{ohms}.

These terms - \textbf{volts}, \textbf{amps}, and \textbf{ohms} - are likely familiar, and volts in particular are used frequently and colloquially. They are also often used \textit{incorrectly} (again, volts in particular, which is often incorrectly used as a measure of quantity, such as "Be careful around that outlet, 110 volts is a lot of electricity!"). We'll define the basic terms here, then move on to some examples:

\begin{itemize}
\item \textbf{Electro-Motive Force} - think of electro-motive force, or emf, as a measure of strength (technically it is a measure of \textit{potential}). When we compare sources of electricity, such as different batteries or an electrical outlet in our home, one of the things we are chiefly interested in is how much work or force can be done by the electricity coming from that source. We measure this force or potential in volts, which can be positive or negative, and we work with volts as a relative measure.
\item \textbf{Electric Current} - think of this item as a measure of quantity, as it is a measure of the number of electrons that move through a point in our circuit over a given time. Our unit here is the amp. While an amp represents a specific quantity of electrons (known as a \textit{joule}), that specific quantity is not really important to us. We'll be using amps to understand the  requirements and limits of the components we use and the circuits we build.
\item \textbf{Impedance} - impedance, or resistance\footnote{This is one of these areas where we are simplifying concepts. Technically, the terms \textit{resistance} and \textit{impedance} represent two different things, with one being a more general case of the other. For our purposes, we can treat them as synonyms.}, tells us how much effort or force is required to be overcome in order for a circuit we build or a component we use to perform it's work. You can almost think of impedance as a measure of difficulty. Impedance is measured in ohms and is also referred to as \textit{load}.
\end{itemize}

These are the three basic ideas that we will leverage throughout all of our work, and are three concepts that balance each other out. We construct a circuit that presents a load, and provide that circuit with a source of energy whose electro-motive force is sufficient to "overcome" that load. By overcoming and powering that load, we cause electricity to move through the circuit; we can measure the amount of electricity moving through the circuit by measuring our current in amps.

When referring to the above three terms in formulas, we use the letter \textit{I} to represent current (amps), the letter \textit{R} to represent resistance (ohms), and \textit{V} to represent volts or electro-motive force. The relationship between these three measures is beautifully elegant, and expressed in something known as Ohm's Law:

\begin{equation}
I = \frac{V}{R}
\end{equation}

This law or equation states that current is equal to force divided by resistance; put another way, the current in a circuit is equal to the force of the power source divided by the resistance of the load we are powering. If \textit{V = 1} and \textit{R = 1} - meaning that we have a circuit that presents 1 ohm of resistance and a battery that can produce 1 volt of electro-motive force, then we can plug in to our equation:

\begin{equation}
I = \frac{1}{1}
\end{equation}

and, in solving for I, see that 1 / 1 = 1 or 1 amp. So a circuit with a resistance of 1 ohm, powered by a battery that can produce 1 volt of force, we succeed in moving 1 amp through that circuit.

Hang tight, we will get to some practical examples of this soon. First, let's try making some changes. Let's assume that we our battery, instead of producing 1 volt, can actually produce 9 volts. Also, for brevity, we will switch here to using V for volts, A for amps, and  $\Omega$ for ohms.

\begin{equation}
I = \frac{9}{1}
\end{equation}

Dividing 9 by 1 shows that our current \textit{I} is now 9A. By increasing the potential of the battery but keeping the needs of the circuit the same, we are causing more electricity to flow through the circuit. Lets say that we add some more components to our circuit, and the load goes up:

\begin{equation}
I = \frac{9}{2}
\end{equation}

By doubling the load, we cut the amount of current \textit{I} in half to 4.5A.

Hopefully this is fairly clear - for any given circuit or load, the more potential created by the battery the more current will flow. This is why the potential or force of our power source will become critical - we need to ensure that we are matching the potential we apply to a load with what that load can tolerate. Let's try a simple example with a batter and a LED\footnote{A LED, or \textit{light-emitting diode}. In this example, we are referring to a simple LED light, not an LED display or anything complicated like that.}

\begin{itemize}
\item We have a battery that can produce 9V of potential.
\item We have an LED that presents 50$\Omega$ of resistance.
\end{itemize}

\begin{equation}
I = \frac{9}{50}
\end{equation}

Solving for \textit{I}, we see we will be passing 0.18A of current. However, if we were to look at the datasheet for our LED\footnote{Pretty much every electronic component we will work with will have a datasheet available that describes its operation and its requirements. They are generally freely available from the seller and/or the manufacturer.} we would see that 0.18A of current is \textit{significantly more} than the LED can handle. This much electricty moving through the LED will literally burn it up! In our example, or datasheet tells us that the LED has a \textit{maximum current} of 0.02A.

We can't change the resistance of the LED - that's fixed. We know the LED has a maximum current rating that it can tolerate of 0.2A. It also has a minimum current rating, under which no damage will occur but the LED won't light, so we can't just arbitrarily reduce the current or we may have an useless circuit. But somehow, we have to limit the amount of current to 0.2A so we can get the LED to light up, not fry.

This brings us to alternate forms of Ohm's law. We can rearrange the formula two different ways, depending on what we want to solve for. If we needed to calculate resistance, we could use this form:

\begin{equation}
R = \frac{V}{I}
\end{equation}

If we know the potential and we already know the amount of current, we can determine what the resistance of our circuit must be. In this example, we know what the resistance is, and we know how much current we can allow, so we need to calculate the property battery to use:

\begin{equation}
V = IR
\end{equation}

If we multiply current by resistance, we will get volts. Plugging in the values from our example:

\begin{equation}
V = (0.02)(50)
\end{equation}

and solving for V, we get 1V. Thats it - in our fictional\footnote{The values for this example were chosen to make nice round numbers, but in reality aren't far off - a typical red LED expects around 0.02A of current and provides somewhere between 40-50$\Omega$ of resistance.} example, we need to find a 1V battery to properly power our LED.


\documentclass{article}

\title{A \LaTeX Workshop \\ for All Skill Levels}
\author{D. Zack G.}
\date{\today}

\usepackage{hyperref}
\usepackage{amsmath, amssymb, amsthm}
\usepackage{graphicx}
\usepackage{parskip}
\usepackage{float}

\newcommand{\RR}[1]{\mathbb{R}^{#1}}

\begin{document}

\maketitle
\newpage
\tableofcontents
\newpage

\section{Introduction}
    Document content goes here!

    We can't just say y = mx + b.

    Use inline math with $y = mx + b$.

    Equation math is $$y = mx + b$$

\section{Content}
	\subsection{Mathematics}
        \subsubsection{Inline Math}
        	Example: $f(x)=ax^2 + c_1$
        \subsubsection{Block-Level Math}
        	Example: $$ \sum_{i=0}^\infty i = -\frac{1}{12} $$

\section{More Content}
    More unrelated text

    $\sum_{i=0}^\infty i = -\frac{1}{12}$

    The function $f: \RR{n} \rightarrow \RR{n}$ is \textit{continuous} if
    $$\lim_{n\rightarrow\infty} x_n = x \Rightarrow \lim_{n\rightarrow\infty} f(x_n) = f(x)$$

$$
M=
  \left[ {
      \begin{array}{ccc}
       1 & \alpha_1 & \alpha_1^2 \\
       1 & \alpha_2 & \alpha_2^2 \\
       1 & \alpha_3 & \alpha_3^2 \\
      \end{array}
  } \right]
$$

$\Box, \blacksquare$

\section{Even More Content}

\begin{figure}[H]
    \centering
    \includegraphics[width=4cm]{homotopy}
    \label{fig:myname}
    \caption{Delicious!}
\end{figure}

\begin{equation} \label{eu_eqn}
    e^{\pi i} + 1 = 0
\end{equation}

See Equation~\ref{eu_eqn}

\begin{equation*}
	\boxed{e^{\pi i} + 1 = 0}
\end{equation*}

\end{document}


\documentclass[]{article}

\usepackage{natbib}
\usepackage{subfiles}
\usepackage{multicol}
%%%%% AUTHORS - PLACE YOUR OWN PACKAGES HERE %%%%%

% Only include extra packages if you really need them. Common packages are:
\usepackage{graphicx}	% Including figure files
\usepackage{amsmath}	% Advanced maths commands
\usepackage{amssymb}	% Extra maths symbols
\usepackage{subcaption}
\usepackage{float}
\usepackage{color}
\bibliographystyle{apj}

\newcommand{\sukhdeep}[1]{{\textcolor{magenta}{Sukhdeep: #1}}}

\begin{document}

\section{Correlation Function}
	We begin by writing the angular space observable, $X$, in terms of the harmonic counterpart
	\begin{align}\label{eq:X_harmonic}
		X(\Omega)=&\sum_{lm}\tilde X_{lm}Y_{lm}(\Omega)
	\end{align}
	where $\Omega$ refers to the angular coordinates on the sky.
	The angular cross correlation function of two (scalar) tracers, $X,Z$ of large scale structure can be written in terms
	of their harmonic space counter parts, $\tilde X, \tilde Z$ as
	\begin{align}
		\langle XZ \rangle(\theta)=&\left\langle\sum_{\ell,m}\sum_{\ell', m'}\tilde X_{\ell m}\tilde Z_{\ell' m'}
									Y_{\ell m}(\Omega)
									Y_{\ell'm'}(\Omega+\theta)\right\rangle\\
									=&\sum_{\ell,m}C_{\ell}Y_{\ell m}(\Omega)
									Y_{\ell m}(\Omega+\theta)\\
		\langle XZ \rangle(\theta)=&\frac{1}{4\pi}\sum_{\ell}(2\ell+1)C_{\ell}P_{\ell}(\cos\theta)\label{eq:xi_pl0}
	\end{align}
	We used the identities
	\begin{align}
		\langle\tilde X_{\ell m}\tilde Z_{\ell' m'}\rangle=&C_{\ell}\delta_D(m,m')\delta_D(\ell,\ell')\\
		\sum_{m=-\ell}^{m=\ell}Y_{\ell m}(\Omega)Y_{\ell m}(\Omega+\theta)=&\frac{2\ell+1}{4\pi}\label{eq:Ylm_Pl}
	\end{align}

	For the case of shear, since it is a spin-2 object, eq.\ref{eq:X_harmonic} is written in terms of spin harmonics
	\citep[see for ex.][]{Castro2005,Kilbinger2017}. Rest of the analysis proceeds similarly, using the relation for spin
	harmonics,
	analogous to eq.~\ref{eq:Ylm_Pl} \citep[see for ex. ][]{Hu1997}.

	Expression of $\xi_+$ is same as eq.~\ref{eq:xi_pl0}. Expressions for galaxy lensing cross correlation
	\cite{Putter2010} and $\xi_-$ is given by
	\begin{align}
		\langle g\gamma_T\rangle(\theta)&=\frac{1}{4\pi}\sum_{\ell}\frac{(2\ell+1)}{\ell(\ell+1)}C_{\ell}^{g\kappa}
		P_{\ell}^2(\cos\theta)\label{eq:xi_g_gamma}\\
		\xi_+(\theta)&=\frac{1}{4\pi}\sum_{\ell}{(2\ell+1)}C_{\ell}^{\kappa\kappa}
		P_{\ell}(\cos\theta)\label{eq:xi_p}\\
		\xi_-(\theta)&=\frac{1}{4\pi}\sum_{\ell}\frac{(\ell-4)!}{(\ell+4)!}\ell^4{(2\ell+1)}C_{\ell}^{\kappa\kappa}
		P_{\ell}^4(\cos\theta)\label{eq:xi_m}
	\end{align}
\sukhdeep{$\xi_-$ is a bit of cheating. I'm not familiar with spin harmonics, so I simply took the relation between
$P_\ell^m$ and $J_m(\ell \theta)$ to get this expression from the Hankel transform for $\xi_-$. It may not be very accurate at large scale (low $\ell$) as is evident from $g\gamma_T$ expression. I can sort this out later. If somebody knows correct expression already, please feel free to put it in.}

	The ccl\_tracer\_corr\_legendre routine computes these transform to convert $C_\ell$ to correlation functions. We
	use the associated Legendre function implementation from gsl library.
	ccl\_tracer\_corr\_legendre routine evaluations can be slow, especially for $P_\ell^m$ with $m>0$. Note that
	$P_\ell^m$ evaluations need to be done only once and can then be saved as long as $\ell,\theta$ values do not change.
	This is not yet implemented, but will be done soon.

	\subsection{Hankel Transform}
		Expression in eq.~\ref{eq:xi_g_gamma}--\ref{eq:xi_m} can be written as Hankel transforms using the relation
		between $P_{\ell}^m$ and bessel functions $J_m$
		\begin{align}
			P_{\ell}^m(\cos\theta)=(-1)^m\frac{(\ell+m)!}{(\ell-m)!}\ell^{-m}J_m(\ell\theta)
		\end{align}

		We get the following analogous expressions (flat-sky limit)
		\begin{align}
			\langle g\gamma_T\rangle(\theta)&=\frac{1}{2\pi}\int d\ell\ell C_\ell J_2(\ell\theta)\\
			\xi_+(\theta)&=\frac{1}{2\pi}\int d\ell\ell C_\ell J_0(\ell\theta)\\
			\xi_-(\theta)&=\frac{1}{2\pi}\int d\ell\ell C_\ell J_4(\ell\theta)
		\end{align}
		To evaluate Hankel transform, we use the fast FFTlog routine\citep{Hamilton2000,Talman2009}. In brief, FFTlog
		works on functions periodic in log space, by writing the Hankel Transform as a convolution between bessel function
		and the function of interest (in this case $C_\ell$). The convolution can
		then be evaluated using Fourier transforms, with Fourier transform of bessel function evaluated using analytical
		functions while Fourier transform of $C_\ell$ and the inverse Fourier transform of the product
		evaluated using fast fourier transform routines.
	\bibliography{correlation}
\end{document}


\documentclass[twoside]{MATH77}
\usepackage[\graphtype]{mfpic}
\usepackage{multicol}
\usepackage[fleqn,reqno,centertags]{amsmath}
\begin{document}
\opengraphsfile{pl02-18}
\begmath 2.18 Digamma or $\psi $ Function

\silentfootnote{$^\copyright$1997 Calif. Inst. of Technology, \thisyear \ Math \`a la Carte, Inc.}

\subsection{Purpose}

The procedures described here compute the digamma or $\psi $ function
defined by $\psi (z) = d[\ln \Gamma (z)]/dz = \Gamma ^{\prime}(z)/\Gamma (z)$%
. Additional procedures, not necessary for simplest usage, are provided to
specify unusual options or retrieve error estimates.

\subsection{Usage}

\subsubsection{Program Prototype, Single Precision}

\begin{description}
\item[REAL]  \ {\bf U , X, SPSI}

\item[EXTERNAL]  \ {\bf SPSI}
\end{description}

Assign a value $x$ to X, and obtain U $= \psi (x)$ by using
$$
\fbox{{\bf U = SPSI(X)}}
$$

\subsubsection{Argument Definitions}

\begin{description}
\item[X]  \ [in] The argument of the function, $x$ above.
\end{description}

\subsubsection{Program Prototype, Single Precision, Specify Unusual Options}

\begin{description}
\item[REAL]  \ {\bf TOL, XERR}

\item[INTEGER]  \ {\bf MSGOFF}
\end{description}

Assign values to TOL, XERR and MSGOFF, and specify options for SPSI by using
$$
\fbox{{\bf CALL SPSIK (TOL, XERR, MSGOFF)}}
$$

\subsubsection{Argument Definitions}

\begin{description}
\item[TOL]  \ [in] Relative error tolerance for $\psi (x)$. When positive,
it indicates relative error tolerance. When negative or zero, it indicates
the default, equal to the square root of the round-off level, $\sqrt{\rho }$%
, should be used. $\rho $ is the smallest number such that the floating
point representation of $1.0+\rho \neq 1.0$. TOL only affects the threshold
for producing error messages; it does not affect the accuracy of computation.

\item[XERR]  \ [in] If non-negative, XERR provides the estimated relative
error in X. If negative, XERR indicates the default error estimate for X,
the round-off level, $\rho $, should be used.

\item[MSGOFF]  \ [in] MSGOFF is added onto the error message level before an
error message is produced by using the error message processor described in
Chapter~19.2.
\end{description}

If SPSIK is not called, the effect is as though CALL SPSIK~(0.0, $-$1.0,~0)
had been executed.
\vspace{10pt}

\hspace{5pt}\mbox{\input pl02-18 }

\subsubsection{Program Prototype, Single Precision, Determine Error Estimate}

\begin{description}
\item[REAL]  \ {\bf ERR, IERFLG}
\end{description}

Retrieve the relative error committed by the last call to SPSI, and the
internal error indicator, by using
$$
\fbox{{\bf CALL SPSIE (ERR,IERFLG)}}
$$

\subsubsection{Argument Definitions}

\begin{description}
\item[ERR]  \ [out] reports the relative error committed by the last call to
SPSI. If SPSI has not been called, ERR is returned with the value $-$1.

\item[IERFLG]  \ [out] reports the value of the internal error flag. If SPSI
has not been called, or if computation on the last call to SPSI was
completely successful, IERFLG is returned with the value zero. See Section
E below for discussion of non-zero values of IERFLG.
\end{description}

\subsubsection{Modifications for Double Precision}

Change the REAL statements to DOUBLE PRECISION and change the first letter
of the procedure names from S to D. One must declare DPSI to be DOUBLE
PRECISION, as its default type would be REAL.

\subsection{Examples and Remarks}

See DRDPSI and ODDPSI for an example of the usage of DPSIK, DPSI and
DPSIE.

For $x \geq 0$, $\psi (x)$ is a monotone increasing function of $x$. $\psi
(x)$ has singularities for $x = 0$ or $x$ a negative integer. See
\cite{ams55} for a discussion of other properties of $\psi (x).$

\subsection{Functional Description}

The computational methods used in SPSI were developed by L. Wayne Fullerton
of Los Alamos National Scientific Laboratory. The methods include rational
Chebyshev expansions, recurrences, and a reflection formula.

When X $>$ 0 SPSI estimates the relative error in $\psi ($X) is the
round-off level.  Otherwise, SPSI computes an estimate for the
relative error in $\psi ($X$) = ($relative error in X$) \times |\text{X}|
\times |d\psi /d\text{X}|\ /\ |\psi (\text{X})|$, with $d\psi /d\text{X}$
approximated by $\pi ^2 \cot ^2 \pi \text{X}$.  When X $<$ 0 and X is
approximately an integer, or $\psi ($X) is approximately zero, the
error will be large.   When X $< < $ 0 and $2\text{X}$ is not approximately an
integer, SPSI may underestimate the error it committed.

\bibliography{math77}
\bibliographystyle{math77}

\subsection{Error Procedures and Restrictions}

If SPSI estimates the specified error tolerance is not satisfied (because $x$
is too near a negative integer), $\psi (x)$ is computed, but an error
message is issued by using the error message processor described in Chapter
19.2, with LEVEL = 1 + MSGOFF, where MSGOFF is zero unless specified by a
call to SPSIK at some time before calling SPSI. The IERFLG output from SPSIE
will be $-$3.

If SPSI is called with X zero or a negative integer, $\psi (x)$ is not
defined, and an error message is issued by using the error message processor
described in Chapter~19.2, with LEVEL = 2 + MSGOFF. If error termination is
suppressed by calling SPSIK with a negative value of MSGOFF, or by calling
the ERMSET procedure described in Chapter~19.2, the function value will be
zero, and the IERFLG output from SPSIE will be~1 if X is zero, or~2 if X is
a negative integer.

\subsection{Supporting Information}

All code is written in ANSI Standard Fortran~77.

The program units SPSI, SPSIB, SPSIE and SPSIK communicate by way of a
common block /SPSIC/. The program units DPSI, DPSIB, DPSIE and DPSIK
communicate by way of a common block /DPSIC/.

\begin{tabular}{@{\bf}l@{\hspace{5pt}}l}
\bf Entry & \hspace{.35in} {\bf Required Files}\vspace{2pt} \\
DPSI & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, DCSEVL, DERM1, DERV1, DINITS, DPSI, ERFIN, ERMSG, IERM1,
IERV1\rule[-5pt]{0pt}{8pt}\rule[-5pt]{0pt}{8pt}}\\
DPSIE & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, DCSEVL, DERM1, DERV1, DINITS, DPSI, ERFIN, ERMSG, IERM1,
IERV1\rule[-5pt]{0pt}{8pt}\rule[-5pt]{0pt}{8pt}}\\
DPSIK & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, DCSEVL, DERM1, DERV1, DINITS, DPSI, ERFIN, ERMSG, IERM1,
IERV1\rule[-5pt]{0pt}{8pt}\rule[-5pt]{0pt}{8pt}}\\
SPSI & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, ERFIN, ERMSG, IERM1, IERV1, SCSEVL, SERM1, SERV1,
SINITS, SPSI\rule[-5pt]{0pt}{8pt}\rule[-5pt]{0pt}{8pt}}\\
SPSIE & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, ERFIN, ERMSG, IERM1, IERV1, SCSEVL, SERM1, SERV1,
SINITS, SPSI\rule[-5pt]{0pt}{8pt}\rule[-5pt]{0pt}{8pt}}\\
SPSIK & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, ERFIN, ERMSG, IERM1, IERV1, SCSEVL, SERM1, SERV1,
SINITS, SPSI}\\
\end{tabular}

Designed and programmed by L. Wayne Fullerton, Los Alamos National
Scientific Laboratory, 1977. Revised and adapted to MATH77 by W. V. Snyder,
1993, 1994.

\begcodenp
\lstset{language=[77]Fortran,showstringspaces=false}
\lstset{xleftmargin=.8in}

\centerline{\bf \large DRDPSI}\vspace{10pt}
\lstinputlisting{\codeloc{dpsi}}

\vspace{30pt}\centerline{\bf \large ODDPSI}\vspace{10pt}
\lstset{language={}}
\lstinputlisting{\outputloc{dpsi}}

\closegraphsfile
\end{document}


% Chapter X

\chapter{Learning model parameters from data} % Chapter title

\label{Learning model parameters from data} % For referencing the chapter elsewhere, use

Figaro provides support for learning model parameters from data. In this section, a special type of compound element will be presented which allows a distribution to be learned from observed evidence. Details are given about the algorithm Figaro provides for learning parameters. Lastly, an example using parameters and learning algorithms to determine the fairness of a set of die rolls is presented.

\section{Parameters and parameterized elements}

This section discusses elements which are learnable parameters. For clarity, a distinction should be made on the meaning of the word \emph{parameter} in this context. This is different from a method parameter or Scala type parameter. In this section, we use \emph{parameter} to refer to a Figaro element which can be learned from data. There are currently two such types of parameters in Figaro - \texttt{BetaParameter} and \texttt{DirichletParameter}.

A customary illustration of parameter learning is to consider the outcomes of a coin flip and determine whether or not the coin is fair. In the case of a \texttt{Flip} element (which is a Bernoulli distribution), the conjugate prior distribution is a Beta distribution. If the coin is not fair, we would expect a prior distribution to have a higher value of alpha or beta (the shape variables of a Beta). First, we will create the conjugate prior distribution of a \texttt{Flip}

\begin{flushleft}
\texttt{val fairness = BetaParameter(1,1)}
\end{flushleft}

The element \texttt{fairness} is the parameter we will use to model the bias of our coin. The behavior of a \texttt{BetaParameter} is similar to a normal \texttt{Beta} element, with a few extra methods and properties. Most importantly, we later use it to create a model learned from parameterized elements. Creation of a parameterized element is accomplished in exactly the same way as creating a compound element.

\begin{flushleft}
\texttt{val f = Flip(fairness)}
\end{flushleft}

This element models a flip of a coin having the fairness specified by the beta parameter, using a value of \texttt{true} to represent heads and \texttt{false} to represent tails. We have actually created an instance of \texttt{ParameterizedFlip}, which is a special type of compound element. A  \texttt{ParameterizedFlip} is created simply by providing a \texttt{BetaParameter} as the argument to \texttt{Flip}.

By using a \texttt{ParameterizedFlip}, the evidence we observe on f can be used to learn the value of \texttt{fairness}. Thus, the next step is to provide the data observed from flips of the coin. Values can be observed just as with other elements, by using \texttt{f.observe(true)} or \texttt{f.observe(false)}. We could also use conditions or constraints.

\marginpar{This example is found in FairCoin.Scala}
As a more detailed example, suppose we have seen 24 heads and 62 tails. One way to represent this data is in a Scala sequence. Note that for readability, the sequence is truncated here.

\begin{flushleft}
\texttt{val data = Seq('H', 'H', 'H', 'T', 'H', 'H', 'T', 'H', ...}
\end{flushleft}

The following block of Scala code will iterate through each of the items in the sequence, create a Flip element using the parameter, and observe true or false based on the side of the coin:

\begin{flushleft}
\texttt{data.foreach \{ d =>
\newline val f = Flip(fairness)
\newline \tab if (d == 'H') \{
\newline \tab f.observe(true)
\newline \} else if (d == 'T') \{
\newline \tab f.observe(false)
\newline \}
\newline \}
}
\end{flushleft}

We have created a parameter, parameterized elements and considered a set of data. Note that each time a parameterize flip is created, it is using the same \texttt{BetaParameter}. It is now desirable to employ a learning mechanism to determine the fairness of the coin, and to create a new element corresponding to the learned value. This is possible by using a learning algorithm.

\section{Expectation maximization}

A learning algorithm can be used to determine the values of parameterized elements. At the end of the process, a parameter targeted by the learning algorithm can provide a new element according to the observed data. Presently, Figaro provides one learning algorithm, expectation maximization, based on variable elimination to compute sufficient statistics. Recall that expectation maximization is an iterative algorithm consisting of an expectation step and a maximization step. During the expectation step, an estimate is produced for the sufficient statistics of the parameter. The estimates are then used in the maximization step to find the most likely value of the parameter. This continues for a set number of iterations and converges toward the true parameter value.

From a practical standpoint, learning a parameter with expectation maximization is very simple. We need only provide the target parameter and, optionally, the number of iterations to the algorithm. The default number of iterations is 10. This can be done in the following way:

\begin{flushleft}
\texttt{val learningAlgorithm = ExpectationMaximization(fairness)
\newline learningAlgorithm.start
\newline learningAlgorithm.kill
\newline
\newline val coin = Flip(fairness.MAPValue)
\newline println("The probability of a coin with this fairness showing
'heads' is: ")
\newline println(coin.prob)
}
\end{flushleft}

After the algorithm has finished running, we can retrieve an element learned from the parameter by using \texttt{Flip(fairness.MAPValue)}. The element \texttt{coin} is a \texttt{Flip}, where the probability of producing true is determined from the data we observed above. It is important to make a distinction between parameterized elements and learned elements. Parameterized elements are compound elements and serve as our means of supplying data about a parameter. A learned element is an atomic element with arguments learned from the data we have observed.

After running the program, we see:

\begin{flushleft}
\texttt{The probability of a coin with this fairness showing 'heads' is:
0.7159090909090909}
\end{flushleft}

We may want to make further explorations about the learned model. For instance, if we wanted to know the probability that two flips of this coin show the same side, we could use

\begin{flushleft}
\texttt{val t1 = Flip(fairness.MAPValue)
\newline val t2 = Flip(fairness.MAPValue)
\newline val equal = t1 === t2}
\end{flushleft}

We can then use variable elimination to determine the probability that the coins show the same side:

\begin{flushleft}
\texttt{val alg = VariableElimination(equal)
\newline alg.start()
\newline println("The probability of two coins which exhibit this fairness showing the same side is: " + alg.probability(equal, \textbf{true}))
\newline alg.kill()
}
\end{flushleft}

This results in the following output:

\begin{flushleft}
\texttt{The probability of two coins which exhibit this fairness showing the same side is: 0.5932334710743803}
\end{flushleft}

\section{A second example}

In the previous sections, parameter learning was discussed using a Beta parameter. This section will explain the use of Dirichlet parameters. The Dirichlet distribution is a multidimensional generalization of the Beta with a variable number of concentration parameters or alpha values. These values correspond to the weight of each possible outcome in the posterior distribution. In a Dirichlet parameter with two dimensions, the alpha values might again correspond to the outcome of heads and tails, or true and false. Using a higher number of dimensions, we can model a number of different categories or outcomes.

Suppose we are given a set of data in which each record represents a roll of two die out of three possible die. The sum of the die is available, as well as which die were selected for the roll. However, the individual outcome of each die is not available. Our task is to learn the fairness of each die.

The first step is to define the possible outcomes from a dice roll. This is easily accomplished by using a Scala list:

\begin{flushleft}
\marginpar{This example is found in FairDice.Scala}
\texttt{val outcomes = List(1, 2, 3, 4, 5, 6)}
\end{flushleft}

Next, we create a parameter representing the fairness of each die:

\begin{flushleft}
\texttt{val fairness1 = DirichletParameter(1,1,1,1,1,1)
\newline val fairness2 = DirichletParameter(1,1,1,1,1,1)
\newline val fairness3 = DirichletParameter(1,1,1,1,1,1)
\newline val parameters = Seq(fairness1, fairness2, fairness3)
}
\end{flushleft}

Each die is initially assumed to be fair. For convenience, we can place all three parameters in a Scala sequence named \texttt{parameters}.  An item this sequence at index i can be accessed with \texttt{parameters(i)}. This sequence will help us to concisely observe the data from which the parameters are learned. We can represent the data in another sequence:

\begin{flushleft}
\texttt{val data = Seq((2, 3, 8), (1, 3, 7), (1, 2, 3), (1, 2, 3), ...}
\end{flushleft}

\texttt{data} is a sequence of 50 Scala tuples. The first two values in each tuple indicate which two die were chosen to roll. The third value is the sum of the two die.

To model the outcome of the sum, we can use an \texttt{Apply} element with a function which sums the outcome of its arguments.

\begin{flushleft}
\texttt{def trial(p1: AtomicDirichlet, p2: AtomicDirichlet, result: Int) = \{
\newline \tab val sum = (i1: Int, i2: Int) => i1 + i2
\newline \tab val die1 = Select(p1, outcomes: \_*)
\newline \tab val die2 = Select(p2, outcomes: \_*)
\newline \tab val t = Apply(die1, die2, sum)
\newline \tab t.observe(result)
\newline \}
}
\end{flushleft}

The code section above defines a Scala function which accepts two Dirichlet parameters and an integer value. \texttt{val sum = (i1: Int, i2: Int) => i1 + i2} defines a Scala function which accepts two integer values and returns their sum.  Next, two Select elements are created and parameterized by the input parameters. Note that the arguments to \texttt{Select} are different from what has been presented previously. Instead of directly enumerating each probability and outcome, we specify a Dirichlet parameter and the list of possible outcomes. The last two lines of \texttt{trial} apply the sum function to the die and observe the result. By calling the \texttt{trial} function for each tuple in the sequence, we can create a model learned from the data.

\begin{flushleft}
\texttt{data.foreach \{ d =>
\newline if (d.\_1 == 1 \&\& d.\_2 == 2) \{
\newline \tab trial(parameters(0), parameters(1), d.\_3)
\newline \tab \} else if (d.\_1 == 1 \&\& d.\_2 == 3) \{
\newline \tab trial(parameters(0), parameters(2), d.\_3)
\newline \} else \{
\newline \tab trial(parameters(1), parameters(2), d.\_3)
\newline \}
\newline \}
}
\end{flushleft}

Just as in the fair coin example, we create an expectation maximization algorithm and use the list of parameters as input. Additionally, this time we have chosen to raise the number of iterations to 100.

\begin{flushleft}
\texttt{val numberOfIterations = 100
\newline val algorithm = ExpectationMaximization(numberOfIterations, parameters: \_*)
\newline algorithm.start
\newline
\newline val die1 = Select(fairness1.MAPValue.view(0,fairness1.MAPValue.size).toList, outcomes)
\newline val die2 = Select(fairness2.MAPValue.view(0,fairness2.MAPValue.size).toList, outcomes)
\newline val die3 = Select(fairness3.MAPValue.view(0,fairness3.MAPValue.size).toList, outcomes)
}
\end{flushleft}

The code block above will retrieve learned elements from the parameters. Note that for a Select, a list of outcomes must be supplied as an argument to along with their corresponding probabilities, which is the array returned by \texttt{MAPValue}. This is because the number of concentration parameters is may vary, and the type of the outcomes is not fixed. Running this code results in the following output, in which we see the model has estimated the probabilities of each value for each die. If one examines the full data declaration in the example code, it is quite easy to see that there are only three observed values of the sum of the die (3, 7 and 8), so the learning algorithm has correctly inferred that the most likely values of the die are 1, 2 and 6, respectively.

\begin{flushleft}
\texttt{The probabilities of seeing each side of d\_1 are:
\newline \tab 0.906250000442371 -> 1
\newline \tab 0.0 -> 2
\newline \tab 0.0 -> 3
\newline \tab 0.0 -> 4
\newline \tab 0.09374999955762903 -> 5
\newline \tab 0.0 -> 6
}
\end{flushleft}

\begin{flushleft}
\texttt{The probabilities of seeing each side of d\_2 are:
\newline \tab 0.0 -> 1
\newline \tab 0.9999999996067813 -> 2
\newline \tab 0.0 -> 3
\newline \tab 0.0 -> 4
\newline \tab 0.0 -> 5
\newline \tab 3.9321864899990694E-10 -> 6
}
\end{flushleft}

\begin{flushleft}
\texttt{The probabilities of seeing each side of d\_3 are:
\newline \tab 0.0 -> 1
\newline \tab 0.0 -> 2
\newline \tab 0.0 -> 3
\newline \tab 0.0 -> 4
\newline \tab 0.0 -> 5
\newline \tab 1.0 -> 6
}
\end{flushleft}


% declare document class and geometry
\documentclass[12pt]{article} % use larger type; default would be 10pt
\usepackage[english]{babel} % for hyphenation dictionary
%\setdefaultlanguage{english} % polyglossia command for use with XeTeX / LuaTeX
\usepackage[margin=1in]{geometry} % handle page geometry

% import packages and commands
\input{../header2.tex}

% title information
\title{Phys 221A -- Quantum Mechanics -- Lec18}
\author{UCLA, Fall 2014}
\date{\formatdate{3}{12}{2014}} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed
         % format: formatdate{dd}{mm}{yyyy}

\begin{document}
\maketitle

\section{Spin-1/2 and Total Angular Momentum}

The total angular momentum $\v J = \v L + \v S$ is the generator of rotation in the total/global Hilbert space $L_2 \otimes (\text{spin space})$, i.e. the total rotation
\begin{eqn}
R_J (\phi \uv n) = e^{-i \phi \uv n \cdot \v J / \hbar}
\end{eqn}
maps to rotated states
\begin{eqn}
\ket{\psi} \otimes \ket{\sigma} \mapsto
	R_J \left( \ket{\psi} \otimes \ket{\sigma} \right)
	= R_L \ket{\psi} \otimes R_S \ket{\sigma}
	= \ket{\psi'} \otimes \ket{\sigma'}
\end{eqn}
The components $J_i$ of total angular momentum are dynamical, observables variables which obey the usual angular momentum commutation relations
\begin{eqn}
[J_i, J_j] = i \hbar \epsilon_{ijk} J_k.
\end{eqn}

For a spin-$\frac{1}{2}$ particle, localized in real space so that $\v L = 0$, the Hilbert space is just the two-dimensional spin space. We can write the spin angular momentum vector as $\v S = \hbar \v \sigma / 2$ where $\v \sigma$ are the Pauli matrices
\begin{eqn}
\v{\sigma} = \set{ \pmat{0 & 1 \\ 1 & 0}, \pmat{0 & -i \\ i & 0}, \pmat{1 & 0 \\ 0 & -1} }
\end{eqn}
which span $SU(2)$.

Writing out the spin-space rotation matrix we have
\begin{align}
R(\phi \uv n) &= e^{-i \phi \uv n \cdot \v S / \hbar} = e^{-i \phi \uv n \cdot \v \sigma / 2} \\ &=
\begin{pmatrix}
\cos(\phi/2) - i n_z \sin(\phi/2) & -i n_- \sin(\phi/2) \\
-i n_+ \sin(\phi/2) & \cos(\phi/2) + i n_z \sin(\phi/2)
\end{pmatrix}
\end{align}
where $n_\pm = n_x \pm i n_y$. We can derive this matrix expression from the Taylor expansion
\begin{eqn}
e^{-\frac{i}{2}\phi \uv n \cdot \v \sigma} = \sum_n \frac{(-i \phi \uv n \cdot \v \sigma / 2)^n}{n!}
\end{eqn}
which, from the identity\footnote{More generally, $(\v a \cdot \v \sigma)(\v b \cdot \v \sigma = \v a \cdot \v b + i (\v a \times \v b) \cdot \v \sigma$} $(\uv n \cdot \v \sigma)^2 = 1$, we find
\begin{eqn}
R(\phi \uv n) = \cos(\phi/2) - i \uv n \cdot \v \sigma \sin(\phi/2),
\end{eqn}
which is equivalent to the matrix written above.

\subsection{Something funny---rotations by $2\pi$}

From our expression for the spin rotation we find that something funny happens when we rotate by $2\pi$---notice that $R(2\pi \uv n) = -1$. So $R$ is a ``projective'' representation of $SO(3)$; it's a 2--1 homomorphism, i.e. a double covering. This turns out to be true for all half-integer spins (fermions).

[Missing: something about Zeemann something or other and Larmor frequencies. He erased the board right after he wrote it..]

\begin{example}[Boson vs fermion interference experiment]
Suppose we set up an interference experiment where a particle can go around one side of a loop entering a black box with a magnetic field $\v B$ passing through it, or the other side of the loop with no field. From the Larmor effect we get a spin rotation of $\delta \phi = \omega_L t$ from the black box, where $t$ is the time spent in the box. If we set up the experiment so that $\delta \phi = 2\pi$, we get constructive interference for bosons and destructive interference for fermions.
\end{example}

[Explained other interesting effects: weak localization and weak anti-localization]

\subsection{Basis spinors}

It is customary to write spin states as column vectors in the basis of normalized eigenstates of $\sigma_z$,
\begin{eqn}
\ket{\uparrow} = \ket{+} = \pmat{1 \\ 0}, \qquad
\ket{\downarrow} = \ket{-} = \pmat{0 \\ 1},
\end{eqn}
with eigenvalues $+1$ and $-1$ respectively. These are thus also eigenstates of $S_z$ with eigenvalues $\pm \hbar / 2$. We can write the closure relation in this space as
\begin{eqn}
1 = \ket{\uparrow} \bra{\uparrow} + \ket{\downarrow} \bra{\downarrow}.
\end{eqn}
Then we have another complete set of eigenstates for $S_x$
\begin{eqn}
\ket{\rightarrow} = \frac{1}{\sqrt 2} \pmat{1 \\ 1}, \qquad
\ket{\leftarrow} = \frac{1}{\sqrt 2} \pmat{1 \\ -1}
\end{eqn}
with eigenvalues of $S_x$ given by $+\hbar / 2$ and $-\hbar / 2$ respectively. So we have another closure relation
\begin{eqn}
1 = \ket{\rightarrow} \bra{\rightarrow} + \ket{\leftarrow} \bra{\leftarrow}.
\end{eqn}

Now, we can write the square angular momentum
\begin{eqn}
\v J^2 = \frac{1}{2} (J_+ J_- + J_- J_+) + J_z^2
\end{eqn}
where $J_\pm = J_x \pm i J_y$ are so-called ladder operators. Notice the ladder operators are non-Hermitian, in fact they are the adjoint of either is the other. We have a new set of commutation relations for the ladder operators
\begin{eqn}
[J_+, J_-] = 2 \hbar J_z, \quad
[J_z, J_\pm] = \pm \hbar J_\pm, \quad
[\v J^2, J_\pm] = [\v J^2, J_z] = 0.
\end{eqn}
So we have two compatible operators $\v J^2$ and $J_z$ with quantum numbers $j$ and $m$ respectively. Furthermore, denoting
\begin{eqn}
\ket{\psi_\pm} = J_\pm \ket{j,m},
\end{eqn}
we find
\begin{eqn}
J_z \ket{\psi_\pm} = \left( [J_z, J_\pm] + J_\pm J_z \right) \ket{j,m} = (j \pm \hbar) \ket{\psi_\pm}
\end{eqn}
Furthermore
\begin{eqn}
\v J^2 \quad \text{something [missed this]}
\end{eqn}

\end{document}


\section{Security Analysis for $\mathcal{TPA}$}
\label{tpa}
\subsection{Correctness}
Assume each $\{Y_i\}_{\{i \in \mathcal{T}\}}$ is correctly calculated,
$G_S$ will be:
\[
  G_S = \prod_{j \in \mathcal{T}}Y_i^{\lambda_j} = g_{\pi}^{a \sum_{j
      \in \mathcal{T}} f(j) \lambda_j} = g_{\pi}^{aS} \pmod p
\]
and by raising $s_i = H(pw, u_i)$ to $G_S$, we get:
\[
  X_i = (G_S^{s_i})^{a'} = (g_{\pi}^{Ss_i})^{aa'} \pmod p
\]
which must be the same as $v_i^{aa'} \bmod p$ for each $i \in
\mathcal{T}$ iff the correct password ($g_{\pi}$) is given at the
step 1. With each $B_i = v_i^{b_i} \bmod p$, the client and servers
share DH keys $K_i = g_{\pi}^{Ss_iaa'b_i} \bmod p$.

\begin{lemma}
\label{tpa1}
From $\{Y_i\}_{i \in \mathcal{L}}$ where $\mathcal{L} \subset \{1..n\}$ and $|\mathcal{L}| <
t$, correct $G_S$ cannot be obtained.
\end{lemma}

\begin{proof}
$S$ cannot be reconstructed from $|\mathcal{L}|$ servers, from the SSS
  property, thus $g_{\pi}^{aS}$ cannot be obtained whatever $a$ is.
\end{proof}

\subsection{Protocol Analysis}
We look into the $\mathcal{TPA}$ protocols by dividing it into two
parts: the first roundtrip (step 1 and 2) is to recover the shared
secret $S$, and the second roundtrip (step 3 and 4) is for key
exchange.

$g_{\pi} = \pi^2 \bmod p$, where $\pi$ is a secure hash value over the
password, is a safe generator in $\mathbb{Z}^*_q$ with high
probability. Raise a random exponent $a$ to mask the
generator. Servers blindly raise its own share $y_i$ to whatever $X =
g_{\pi}^a$ is. As long as $p$ is a safe prime $X^{y_i} \bmod p$ does
not reveal $y_i$ whatever $X$ is.

The second roundtrip can be seen as the Elgamal encryption with a
generator $G_S^{s_i}$ where $s_i := H(pw, u_i)$. $u_i$ is a salt
generated uniquely for each server. The encrypted data is sent in the
next roundtrip.  Another random exponent $a'$ is applied to make it
difficult to do offline dictionary attacks over $s_i$.

The last roundtrip (step 4 and 5) is to check if both parties have
shared a random session key and to obtain a proof. The client receives
the encrypted data iff the MAC is verified at the server, otherwise it
will receive $\bot$.

\subsection{Reduction to Elgamal Encryption}
Since $\mathbb{Z}_p^*$ is a cyclic group, there exist $g \in
\mathbb{Z}_p^*$ and $x \in \mathbb{Z}_p$ such that $g_{\pi} = g^x
\pmod p$ therefore in the step 3, $X_i = G_S^{s_ia'}$ can be
seen as a $DH$ term: $g^{x'}$ where $x' = xSs_iaa' \bmod q$. The
session key on the server side is the same as the standard Elgamal
encryption, i.e., $(g^{x'})^{b_i}$.
At the step 4, the server returns $B_i = v_i^{b_i}$ where $v_i =
g^{x'/aa'} \pmod p$. For any $aa' \in \mathbb{Z}_q$ there exist a
multiplicative inverse in $\mathbb{Z}_q$ such that $v_i^{aa'} = g^{x'}
\pmod p$. Therefore we get the standard Elgamal key such as $B_i^{aa'}
= (g^{x'})^{b_i} \pmod p$ on the client side.

\subsection{Resistance to offline dictionary attacks}
Assume we have $|\mathcal{L}| < t$ compromised servers and try to
reconstruct $g_{\pi}^S$ from $\{y_i\}_{i \in \mathcal{L}}$.
From the Lemma ~\ref{tpa1} we need at least one $Y_i$, $i \notin
\mathcal{L}$, and unless the server $i$ is compromised the only
way to get $Y_i$ is to issue the protocol $X = g_{\pi}^a \bmod
p$ in the step 1.
Unless the attacker knows $\pi$ all he can do is to guess the
password $\pi'$ and to keep asking the server to calculate $Y'_i =
g_{\pi'}^a \bmod p$. If the guess is wrong $Y'_i$ will not
calculate $g_{\pi}^S$ correctly with the compromised data $\{f(j)\}_{j
\in \mathcal{L}}$.
Unless the attacker can calculate $S$ from $g^S \bmod p$ with a
fabricated $g$, he cannot calculate $g_{\pi'}^S \bmod p$ in
order to brute force for $g_{\pi'} \in \{g^e | e = 1..p-1\}$.

Next, we look into dictionary attacks for {\em share} that is
revealed from compromised servers, i.e.,
\begin{align*}
  y_i &= f(i) \\
  v_i &= g_{\pi}^{Ss_i} \bmod p \\
\end{align*}
for $i \in \mathcal{L}$. We also get $G_S = g_{\pi}^{aS} \bmod p$ from
the step 2. $B_i$ will not contribute anything to the attack as they already
have $v_i$.
To brute force on $v_i$ there are two possible ways:
\begin{enumerate}
\item $\exists S \in \mathbb{Z}_q$, check if
  $v_i \stackrel{\text{\tiny ?}}{=} (g_{\pi}^{s_i})^S \pmod p$
  for each $g_{\pi}^{s_i}$ \\
\item $\exists g_{\pi} \in \mathbb{Z}^*_q$, check if
  $v_i \stackrel{\text{\tiny ?}}{=} (g_{\pi}^S)^{s_i} \pmod p$
  for each $s_i$
\end{enumerate}
For the former case we need $S$ and for the latter case we need
$g_{\pi}^S$. From the protocol 1, we can obtain either $g^S \bmod p$
for $\forall g \in \mathbb{Z}$ or $g_{\pi}^{aS} \bmod p$. Obtaining
$S$ is simply the {\sf DLog} problem. $g_{\pi}^S \bmod p$ can be
obtained iff $g$ happens to be $g_{\pi}$ or a legitimate client
chooses $a = 1$ which must be excluded.
If $g_{\pi}^S$ is obtained, the dictionary attack is possible on
$s_i$, however, it is difficult to obtain $g_{\pi}^S$ from $G_S$ which
is calcualted by a legitimate client as $a$ has been randomly
chosen. Attackers can choose $a = 1$ at the step 1 and they can obtain
$g_{\pi'}^S$ but they need to start over the protocol to get the
correct $g_{\pi}^S$.


\subsection{Finite-Difference Time-Domain Method}

The finite-difference time-domain method (also called Yee's method after the mathematician Kane Yee) describes an algorithm to calculate time dependent electromagnetic fields based on Maxwell's equations. It has first been published in a paper by Kane Yee in 1966~\cite{yee}.

To describe a system, space and time are broken down into a grid (which can but does not have to have equidistant points) which contains information about the electric and magnetic fields and the permittivity at each point. After the grid is initialized, the neighboring electric field values can be calculated from the magnetic field and, in a second step, the future electric field can be calculated from the magnetic field. By taking the permittivity into account this method is capable of describing inhomogenous complex systems that consist of different materials and field sources.

While there are other often more efficient methods to numerically simulate and calculate surface plasmons, the finite-difference time-domain method is way more flexible. Because it is basically just a solver for Maxwell's equations without any assumptions on the geometry it can calculate various experiments from wave guides to all kinds of light interaction with wavelength-scale 3D metal geometries~\cite{numel}.


\subsection{meda}

Matrix Exploratory Data Analysis (meda) is a package being developed to
allow for easy generation of modern summary statistics effective for
high-dimensional data analysis.

\begin{compactitem}
  \item Source code: \href{https://github.com/neurodata/meda}{https://github.com/neurodata/meda}
  \item Example output generated from Fisher's Iris data is here:
    \href{http://docs.neurodata.io/meda}{http://docs.neurodata.io/meda}
\end{compactitem}

The goal of this package is to realize the following checklist: Given a new set of n samples of vectors in $\mathbb{R}^d$

\begin{compactenum}
  \item histogram of feature types (binary, integer, non-negative, character, string etc.)
  \item \# NaNs per row? Per column? Infs per row? Per column? "Zero" variance rows? columns?
  \item Heat map of raw data that fits on screen (k-means++ to select 1000 samples, CUR to select 100 dimensions)
  \item 1st moment statistics
  \begin{compactenum}
    \item mean (line plot + heatmap)
    \item median (line plot + heatmap)
  \end{compactenum}
  \item 2nd moment statistics
  \begin{compactenum}
    \item correlation matrix (heatmap)
    \item matrix of energy distances (heatmap)
  \end{compactenum}
  \item density estimate
  \begin{compactenum}
    \item 1D marginals (Violin + jittered scatter plot of each dimension,  if n > 1000 or d>10, density heatmaps)
    \item 2D marginals (Pairs plots for top ~8 dimensions, if n*d>8000, 2D heatmaps)
  \end{compactenum}
  \item Outlier plot
  \item cluster analysis (IDT++)
  \begin{compactenum}
    \item BIC curves
    \item mean line plot
    \item covariance matrix heatmaps
  \end{compactenum}
  \item spectral analysis
  \begin{compactenum}
    \item cumulative variance (with elbows) of data matrix
    \item eigenvectors (pairs plot + heatmap)
  \end{compactenum}
\end{compactenum}

\begin{compactitem}
\item To rescale the data in case of differently scaled features, we will implement the following options:
\begin{compactitem}
  \item raw
  \item linear options
  \begin{compactitem}
    \item linear squash between 0 \& 1
    \item mean subtract and standard deviation divide
    \item median subtract and median absolute deviation divide
    \item make unit norm
  \end{compactitem}
  \item nonlinear
  \begin{compactitem}
    \item rank
    \item sigmoid squash
  \end{compactitem}
\end{compactitem}

\item To robustify in the face of outliers, we will utilize
 \href{http://projecteuclid.org/euclid.bj/1438777595}{Geometric median and robust estimation in Banach spaces}

\item { if features have categories}
\begin{compactenum}
  \item sort by category
  \item color code labels by category
\end{compactenum}

\item { if points have categories}:
   label points in scatter plots by symbol
\end{compactitem}

% \end{document}


\documentclass[simplex.tex]{subfiles}
% NO NEED TO INPUT PREAMBLES HERE
% packages are inherited; you can compile this on its own
\providecommand{\mb}[1]{\boldsymbol{#1}}
\providecommand{\mv}[1]{\vec{#1}}
\providecommand{\ve}[1]{\boldsymbol{#1}}
\newcommand{\bpsi}{\ve{\psi}}
\newcommand{\bv}{\mb{v}}
\newcommand{\bx}{\mb{x}}
\newcommand{\bA}{\mathbf{A}}
\newcommand{\bH}{\mathbf{H}}
\newcommand{\bX}{\mathbf{X}}
\newcommand{\bP}{\mathbf{P}}
\newcommand{\bB}{\mathbf{B}}
\newcommand{\bD}{\mathbf{D}}
\newcommand{\bS}{\mathbf{S}}
\newcommand{\bR}{\mathbf{R}}
\newcommand{\bJ}{\mathbf{J}}
\newcommand{\bLambda}{\mathbf{\Lambda}}
\begin{document}
\subsection{Vertex Screening}
We are developing a vertex screening method to recover the signal subgraph. Specifically, we have $m$ pairs of graph and label sampled independently from some distribution $F_{G,Y}$,
\[(G_1,Y_1),(G_2,Y_2),(G_3,Y_3),...,(G_m,Y_m) \overset{i.i.d.}{\sim} F_{G,Y}. \]
It is often the case that the signal is sparse in $G$. That is to say, there is a small subgraph $G[S]$ induced by signal vertices $S$ which contains all information about $Y$. The vertex screening method wants to recover the signal vertices $S$. It consists of three steps: feature extraction, computing distance correlations, and thresholding. \\

The first step is to extract a feature vector for each vertex in a graph. We use notation $\hat{X}_{i}[u,\cdot]$ to denote the feature extracted for vertex $u$ in graph $i$ where $i \in [m]$ and $u \in [n]$. A simple approach to obtain a feature vector is to set $\hat{X}_{i}[u]$ to the $u$th row of adjacency matrix $A_i$, that is $\hat{X}_{i}[u,\cdot]=A_i[u,\cdot]$. In this case,  $\hat{X}_{i}[u,\cdot]$ is a vector in $\mathbb{R}^n$ which can be a high dimensional space. Alternatively, Adjacency Spectral Embedding could also extract a feature vector $\hat{X}_{i}[u,\cdot]$ which lies in $\mathbb{R}^d$. The second step computes a correlation between  $\{\hat{X}_{i}[u,\cdot]\}_{i=1}^m$ and $\{Y_i\}_{i=1}^m$ for each $u \in V$. The correlation could be distance correlation (Dcorr) or multiscale generalized correlation (MGC). Let $c_u$ be the correlation, that is
\[ c_u = Dcorr(\{\hat{X}_{i}[u,\cdot]\}_{i=1}^m, \{Y_i\}_{i=1}^m ) \text{ or } MGC(\{\hat{X}_{i}[u,\cdot]\}_{i=1}^m, \{Y_i\}_{i=1}^m). \]
The last step order $c_u$s by their magnitudes. Then, we threshold the correlations by a critical value $c$. The vertices survive thresholding will be the estimated signal vertices $\hat{S}$, that is
\[\hat{S} = \{u \in V | c_u > c\}.\]
The estimated signal subgraph and the corresponding adjacency matrix $G[\hat{S}]$ and $A[\hat{S}]$. The Algorithm \ref{alg:vs} describes the general procedure of vertex screening using adjacency vector as feature and MGC. A simulation experiment is shown in the Figure \ref{fig:vs} which demonstrates that the vertex screening can significantly improve the graph classification performance for various sample size. We are currently working on theories of vertex screening and finishing the first draft.
\clearpage
\begin{algorithm}
	\caption{Vertex Screening. Find the signal vertex estimate $\hat{S}$.}
	\label{alg:vs}
	\begin{algorithmic}[1]
		\Procedure{Input $\{(A_i,Y_i)\}_{i=1}^m$ and $c \in [0,1]$}{}
		\For{$u=1:n$ }
		\State $c_u = MGC(\{\hat{X}_{i}[u,\cdot]\}_{i=1}^m, \{Y_i\}_{i=1}^m )$
		\EndFor
		\State Output $\hat{S} = \{u| c_u > c\}$
		\EndProcedure
	\end{algorithmic}
\end{algorithm}

%%%  FIGURE BLOCK
\begin{figure}[!h]
	\begin{cframed}
		\centering
		\includegraphics[scale=0.35, clip = true]{../../figs/Bayes_plugin_IER_3n.png}
		\caption{The classification accuracies of five approaches with their standard errors are shown. We generate graphs from $3$ different inhomogeneous Erdos-Renyi model, then apply $5$ classifiers to classify these graphs: Bayes optimal classifier (green), Bayes plugin on $G[S]$ (purple), Bayes plugin on $G[\hat{S}]$ with $\hat{S}$ estimated by Dcorr (blue), Bayes plugin on $G[\hat{S}]$ with $\hat{S}$ estimated by MGC (yellow), and Bayes plugin on $G$ (red). The classifiers with vertex screening (blue and yellow) have significantly better classification performance compared to without screening (red), and are close to Bayes optimal (green) when given $600$ graphs. }
		\label{fig:vs}
	\end{cframed}
\end{figure}
%
\clearpage

\subsection{Joint Embedding}
The latest draft is posted on \href{https://arxiv.org/abs/1703.03862}{arXiv} and submitted for publication. The code can be found \href{https://github.com/shangsiwang/Joint-Embedding}{here}. We are also working on making a R package with joint embedding included.

\clearpage

\end{document}


\section{Results and Discussion}

We present results with different types of flow field data sets to evaluate our approach, including synthetic data and spatially aggregated data sets. To demonstrate the effectiveness of the proposed algorithm, we compared it with the Monte Carlo (MC) method, which is the general approach to stochastically trace particles in uncertain flow fields. We performed quantitative comparisons on the resulting streamlines generated by our approach and the MC method with different settings and distance measurements. We also qualitatively compare the most likely streamlines as well as the distributions of possible traces produced by our approach and the MC method by visualizing sample traces on different data sets.

\subsection{Synthetic Data}

We first evaluate the proposed algorithm on the analytical static double-gyre data set proposed by Shadden et al. in~\cite{Shadden2005271}. Gaussian noise is added into the vector field to synthesize the uncertainty. In order to quantitatively evaluate the robustness of the proposed algorithm under the influence of noise, we generate streamlines starting from regularly sampled seed positions for the certain double-gyre data set and use those streamlines as our ground truth. Then, a set of sample traces are generated by the MC method and the Bayesian approach starting from the same seed position presented above for the uncertain double-gyre data set with different noise level, which is controlled by the standard deviation $\sigma$ of the Gaussian noise. All the streamlines were generated with a step size of $0.005$ and a maximum step number of $100$. For the Monte Carlo method and the proposed algorithm, a critical parameter is the number of particles used for each seed. Indeed, more particles will give a more accurate presentation of the target distribution but will take more time to generate the results. Hence, a particle count that balances the accuracy and the computation time need to be studied. Based on~\cite{journals/mia/PontabryROSKD13}, we use $100$ particles for both of the methods in the experiments. To compare the accuracy of the resulting traces, the Hausdorff distance~\cite{Roessl:2012:TVCG} and the Mean of the closest point distance~\cite{Corouge04towardsa} between streamlines were used. Figure~\ref{gerror} gives the average of the distances presented above between the most likely streamlines generated from each method and the ground truth, with increasing $\sigma$ values for the noise in the vector field. The figure reveals that our method can produce most likely traces that are closer to the ground truth and the average of the distances increases more slowly than the MC method as the noise increases.

Besides comparing the accuracy of the most likely traces, it is also important to compare all possible traces generated by each method. We evaluate the accuracy of uncertain streamlines starting from a given seed position by measuring the distance between each individual trace and the ground truth, then we compute the weighted sum of all the distances. For the MC method, all traces are equally weighted. For the proposed method, the weights of the traces described above are used. Figure~\ref{gerror_r} shows that the proposed method can generate more accurate traces.

\begin{figure}[!htb]
  \centering
  \begin{subfigure}[b]{0.24\textwidth}
    \centering
    \includegraphics[height=0.8in]{../figures/doublegyre_h.eps}
  \end{subfigure}~
  \begin{subfigure}[b]{0.24\textwidth}
    \centering
    \includegraphics[height=0.8in]{../figures/doublegyre_m.eps}
  \end{subfigure}
  \caption{Comparison of the distance between the most likely traces and the ground truth for our method and the MC method.}
  \label{gerror}
\end{figure}

\begin{figure}[!htb]
  \centering
  \begin{subfigure}[b]{0.24\textwidth}
    \centering
    \includegraphics[height=0.8in]{../figures/doublegyre_hr.eps}
  \end{subfigure}~
  \begin{subfigure}[b]{0.24\textwidth}
    \centering
    \includegraphics[height=0.8in]{../figures/doublegyre_mr.eps}
  \end{subfigure}
  \caption{Comparison of overall trace accuracy. For each method, distances of all sample traces to the ground truth are measured and summed by their weights.}
  \label{gerror_r}
\end{figure}

Figure~\ref{case_1} shows sample traces generated by the MC method and our method at a given seed location in the double-gyre flow field. As we can see in the figure, our method can generate more concentrated traces which are also closer to the ground truth compared with the MC method. The most likely trace generated by our method is also closer to the ground truth, as shown in~\ref{case_1} (c).

\begin{figure}[!htb]
   \centering
  \small
  (a) \vcenterbox{\includegraphics[height=0.5in]{../figures/double_gyre_mc35.eps} } \hfill
  (b) \vcenterbox{\includegraphics[height=0.5in]{../figures/double_gyre_smc35.eps} } \hfill
  (c) \vcenterbox{\includegraphics[height=0.5in]{../figures/double_gyre_opt35.eps} }
  \caption{(a): Sampled streamlines computed by the MC method starting from seeding position $x=0.3, y=0.5$ in the analytical double-gyre data set. (b): Sampled streamlines computed by our method from the same seeding position in (a). (c): The most likely traces generated by both methods compared with the ground truth.}
  \label{case_1}
\end{figure}

\subsection{Spatially aggregated Data Sets}

\begin{figure*}[!htbp]
  \centering
  \small
  (a) \vcenterbox{\includegraphics[width=1.5in]{../figures/isabel_gt.eps} } \hfill
  (b) \vcenterbox{\includegraphics[width=1.5in]{../figures/isabel_mc.eps} } \hfill
  (c) \vcenterbox{\includegraphics[width=1.5in]{../figures/isabel_smc.eps} }

  \caption{Streamlines generated on the Hurricane Isabel data sets. The color is used to enhance the contrast among streamlines. (a): The ground truth streamlines generated on the raw data. (b): Results produced by the Monte Carlo method on the distribution data with block size $16^3$. (c): Streamlines generated by our method on the same data in (b).}
  \label{data_overview}
\end{figure*}

\begin{figure}[!htbp]
  \centering
  \small
  (a) \vcenterbox{\includegraphics[height=1.0in]{../figures/isabel_mc1.eps} } \hfill
  (b) \vcenterbox{\includegraphics[height=1.0in]{../figures/isabel_smc1.eps} }
  \caption{(a): Sampled streamlines computed by the MC method starting from seeding position $x=250, y=150, z=45$ in the Isabel data set. (b): Sampled streamlines computed by our method from the same seeding position in (a).}
  \label{case_4}
\end{figure}

In this section, experiments were done on the Hurricane Isabel data set. Hurricane Isabel is a data set with a resolution of $500 \times 500 \times 100$ that models a strong hurricane in the west Atlantic region in September 2003. In order to test the performance of our algorithm on uncertain data which is represented by non-Gaussian distributions, we decompose the data into small cubic blocks and construct a histogram for each block. For the test data set, distribution-based data are generated with three different block size $8^3$, $16^3$, and $32^3$ to evaluate the performance of the proposed algorithm under the influence of uncertainty.

As presented above, we regularly sample a set of seed locations and compute the streamlines for both the raw data and the spatially down sampled data. To perform the quantitative analysis, we treat the streamlines computed from the raw data as the ground truth and compute the distance between the stochastic particle traces with the ground truth. $100$ particles were used with an integration step size $1.0$ and a maximum step number of $1000$ for the streamline computation. Figure~\ref{berror_r} gives the mean of the distances' weighted sum between sample streamline bundles generated from the test methods and the ground truth on the test data set with different block sizes. The figure reveals that the proposed method can produce traces that are closer to the ground truth.

\begin{figure}[!htb]
  \centering
  \begin{subfigure}[b]{0.24\textwidth}
    \centering
    \includegraphics[height=0.9in]{../figures/isabel_h.eps}
  \end{subfigure}~
  \begin{subfigure}[b]{0.24\textwidth}
    \centering
    \includegraphics[height=0.9in]{../figures/isabel_m.eps}
  \end{subfigure}
  \caption{Distances between the ground truth and sample traces generated by our method and the MC method for the Isabel data set.}
  \label{berror_r}
\end{figure}

Figure~\ref{data_overview} shows the streamlines generated from the test data set on the seed positions presented above. The most likely streamlines generated by the MC method on the data set with block size $16^3$ are given in Figure~\ref{data_overview} (b); as expected, streamlines generated by the MC method are generally not as smooth as the ground truth and some flow features looks quiet different compare with the ground truth. Figure~\ref{data_overview} (c) show the streamlines produced by our method with the same block size, which give more accurate and smoother results. Figure~\ref{case_4} gives sample traces generated by the MC method and the proposed method at a given seed location in the Isabel data set. Figure~\ref{case_4} shows that our algorithm can produce more concentrated and accurate results than the basic MC method, because the correlation between consecutive integration steps are exploited.

\subsection{Performance}

All the experiments were performed on a desktop computer with an Intel(R) Core(TM) i7-4790K CPU 4.0GHz processor, 16GB memory, and an NVIDIA GTX 970 GPU. In Table~\ref{timing}, we compare the performance measurements between the proposed algorithm and the Monte Carlo method for streamlines estimated for a given seed position with $100$ sample points for all the test data sets used in this paper. In all the datasets, our approach is almost as fast as the Monte Carlo method.

\begin{table}[!htb]
\centering
\begin{tabular}{|c|c|c|c|c|}
\hline
\multirow{2}{*}{Data Set}    & \multirow{2}{*}{Method}     & \multicolumn{3}{c|}{Timing(sec)}  \\ \cline{3-5}
                             &                             & 40 Steps  & 80 Steps & 120 Steps  \\ \hline
\multirow{2}{*}{Double Gyre} & MC                          & 0.0026    & 0.005    & 0.008      \\ \cline{2-5}
                             & Bayesian             & 0.0035    & 0.007    & 0.01       \\ \hline
\multirow{2}{*}{Isabel}      & MC                          & 3.3       & 6.7      & 10.1       \\ \cline{2-5}
                             & Bayesian             & 3.4       & 6.8      & 10.7       \\ \hline

\end{tabular}
\caption{Overview of the performance for the proposed algorithm and the Monte Carlo method.}
\label{timing}
\end{table}


\section{Evaluation and Error Calculation}
\label{sec:Evaluation}
% --------------
% Speed of Sound
% --------------
\subsection{Speed of Sound}
\label{subsec:Speed_of_Sound}
To determine the speed of sound, the propagation time of a sound pulse over the distance $s=(2.561\pm0.003)\ m$ was measured 20 times (see table \ref{tab:Speed_of_Sound_Measurements}). The ambient temperature was $\vartheta=23$ °C.

To calculate the speed of sound from the measured values, one has to calculate the mean value of the duration $t_i$ and its error. To do this in Excel, the function "AVERAGE()" is used to calculate the mean value. This function implements equation \ref{eq:Arithmetic_Mean}. To calculate the error of the mean value, the function "STDEV.S()" is used, which implements equation \ref{eq:Arithmetic_Mean_Error}. The same thing can be achieved using the software QtiPlot. The measured values are shown in the following figure \ref{fig:Sound_Propagation_Time}:
\begin{figure}[H]
	\centering
	\includegraphics[scale=1]{Sound_Propagation_Time}
	\caption{Sound Propagation Time}
	\label{fig:Sound_Propagation_Time}
\end{figure}
Equation \ref{eq:Experimental_Standard_Deviation} is used to calculate the experimental standard deviation. This can be done by using the Excel function "STDEV.S()" again. If the equation \ref{eq:Arithmetic_Mean} for the mean value error is compared to the equation \ref{eq:Experimental_Standard_Deviation} for the experimental standard deviation, it becomes apparent that only an $\sqrt{n}$ is missing in the denominator. This can easily be fixed by dividing the function by $\sqrt{n}$. QtiPlot allows for direct calculation of the experimental standard deviation for an entire column of values.
\newpage
The comparison between Excel and QtiPlot is shown in the following table \ref{tab:Comparison_Speed_of_Sound_Measurements}:
\begin{table}[H]
	\centering
	\renewcommand{\arraystretch}{1.2}
	\begin{tabular}{c c c c}
		\hline
		\textbf{Software} & \textbf{Mean Value $\overline t$} & \textbf{Mean Value Error $s_{\overline{t}}$} & \textbf{Standard Deviation $s$} \\
		\hline
		Excel & $7.3225\cdot10^{-3}$ s & $7.35308\cdot10^{-5}$ s & $3.28840\cdot10^{-4}$ s \\
		QtiPlot & $7.3225\cdot10^{-3}$ s & $7.35308\cdot10^{-5}$ s & $3.28840\cdot10^{-4}$ s \\ \hline
	\end{tabular}
	\caption{Comparison between the calculated values}
	\label{tab:Comparison_Speed_of_Sound_Measurements}
\end{table}
The following equation is used to calculate the mean value of the speed of sound:
\begin{equation}
\overline{v}=\frac{\overline s}{\overline t}=\frac{2.561\ m}{7.3225\cdot10^{-3}\ s}=349.744\ \frac{m}{s}
\label{eq:Speed_of_Sound}
\end{equation}
\\
Equation \ref{eq:Error_Propagation} is used to calculate the uncertainty:
\begin{equation}
s_{\overline{v}}=\sqrt{\left(\frac{\partial v}{\partial s}\Biggr|_{\overline v}\cdot s_{\overline{s}}\right)^2 + \left(\frac{\partial v}{\partial t}\Biggr|_{\overline v}\cdot s_{\overline{t}}\right)^2}=\sqrt{\left(\frac{1}{\overline t}\cdot s_{\overline{s}}\right)^2 + \left(-\frac{\overline{s}}{\overline{t}^2}\cdot s_{\overline{t}}\right)^2}=3.53586\ \frac{m}{s}
\end{equation}
\\
The derived speed of sound is as follows:
\begingroup
\Large
\begin{equation}
v=\overline{v}\pm s_{\overline v}=(350\pm4)\ \frac{m}{s}
\end{equation}
\endgroup
\newpage
% ------------
% Iron Content
% ------------
\subsection{Iron Content}
\label{subsec:Iron_Content}
The iron content in an alloy was determined by various methods. Therefore, the accuracy varies between the different measurements.

In Excel, the mean value and its error are calculated the same way as in section \ref{subsec:Speed_of_Sound}. The weighted mean value can be calculated by using the "SUMPRODUCT()" function. The formula is as follows:
\[
=\text{SUMPRODUCT}(\overline{x_1}:\overline{x_n};1/(s_{\overline{x1}}:s_{\overline{xn}})\textasciicircum 2)/\text{SUM}(s_{\overline{x1}}:s_{\overline{xn}})
\]
\\
The measured values (see table \ref{tab:Iron_Content_Measurements}) and their respective error are shown in figure \ref{fig:Iron_Content}. Furthermore, the weighted mean value was calculated by a weighted regression fit in QtiPlot (also shown in fig. \ref{fig:Iron_Content}).
\begin{figure}[H]
	\centering
	\includegraphics[scale=1]{Iron_Content}
	\caption{Iron Content}
	\label{fig:Iron_Content}
\end{figure}
The comparison for the mean value and its error:
\begin{table}[H]
	\centering
	\renewcommand{\arraystretch}{1.2}
	\begin{tabular}{c c c}
		\hline
		\textbf{Software} & \textbf{Mean Value} & \textbf{Mean Value Error} \\
		\hline
		Excel & $20.5556$ \% & $0.519912$ \% \\
		QtiPlot & $20.5556$ \% & $0.519912$ \% \\ \hline
	\end{tabular}
	\caption{Comparison between the calculated values (Mean Value)}
	\label{tab:Comparison_Iron_Content_Mean_Value}
\end{table}
\newpage
The comparison for the weighted mean value and its error:
\begin{table}[H]
	\centering
	\renewcommand{\arraystretch}{1.2}
	\begin{tabular}{c c c}
		\hline
		\textbf{Software} & \textbf{Weighted Mean Value} & \textbf{Weighted Mean Value Error} \\
		\hline
		Excel & $20.4007$ \% & $0.361055$ \% \\
		QtiPlot & $20.4007$ \% & $0.361055$ \% \\ \hline
	\end{tabular}
	\caption{Comparison between the calculated values (Weighted Mean Value)}
	\label{tab:Comparison_Iron_Content_Weighted_Mean_Value}
\end{table}
\newpage
% ---------------
% Spring Constant
% ---------------
\subsection{Spring Constant}
\label{subsec:Spring_Constant}
To determine the spring constant of a pretensioned steel spring from the measured values (see table \ref{tab:Spring_Constant_Measurements}) the following equation is used to create a linear regression fit:
\begin{equation}
F=k\cdot z+F_0
\end{equation}
The measured values and the linear regression fit are shown in the following figure \ref{fig:Spring_Constant}:
\begin{figure}[H]
	\centering
	\includegraphics[scale=1]{Spring_Constant}
	\caption{Spring Constant}
	\label{fig:Spring_Constant}
\end{figure}
The values obtained from QtiPlot are shown in the following table \ref{tab:Calculated_Spring_Force_Parameters}:
\begin{table}[H]
	\centering
	\renewcommand{\arraystretch}{1.3}
	\begin{tabular}{c c}
		\hline
		\textbf{Spring Constant $k$} & \textbf{Pretension Force $F_0$} \\
		\hline
		$(22.5\pm0.9)\ \,^\text{N}\!/_\text{m}$ & $(-0.8\pm0.5)$\ N \\ \hline
	\end{tabular}
	\caption{Calculated Spring Force Parameters}
	\label{tab:Calculated_Spring_Force_Parameters}
\end{table}
The spring force can now be derived from the following equation:
\begin{equation}
F=22.5\ \,^\text{N}\!/_\text{m}\cdot z-0.8\ \text{N}
\end{equation}
\newpage
% --------
% Pendulum
% --------
\subsection{Pendulum}
\label{subsec:Pendulum}
The damped oscillation of a pendulum can be described by the following equation:
\begin{equation}
y=A\cdot \text{exp}(-\Gamma\cdot t)\cdot\text{sin}(2\cdot\pi\cdot f\cdot t-\delta)+y_0
\end{equation}
The values in table \ref{tab:Pendulum_Measurements} were measured with an ultrasonic sensor. The following figure \ref{fig:Pendulum} shows the measured distances as a function of time as well as the nonlinear fitted curve.
\begin{figure}[H]
	\centering
	\includegraphics[scale=1]{Pendulum}
	\caption{Pendulum}
	\label{fig:Pendulum}
\end{figure}
The values obtained from QtiPlot are shown in table \ref{tab:Calculated_Pendulum_Parameters}:
\begin{table}[H]
	\centering
	\renewcommand{\arraystretch}{1.3}
	\begin{tabular}{r|c c c}
		& \textbf{Value} & & \textbf{Uncertainty} \\
		\hline\hline
		\textbf{Amplitude $A$} & $-1.22$\ m & $\pm$ & $0.03$\ m \\
		\textbf{Damping Constant $\Gamma$} & $52\cdot10^{-3}\ \text{s}^{-1}$ & $\pm$ & $2\cdot10^{-3}\ \text{s}^{-1}$ \\
		\textbf{Frequency $f$} & $55.0\cdot10^{-3}$\ Hz & $\pm$ & $0.2\cdot10^{-3}$\ Hz \\
		\textbf{Phase $\delta$} & $-2.63$\ rad & $\pm$ & $0.02$\ rad \\
		\textbf{Offset $y_0$} & $49\cdot10^{-3}$\ m & $\pm$  & $5\cdot10^{-3}$\ m \\
	\end{tabular}
	\caption{Calculated Pendulum Parameters}
	\label{tab:Calculated_Pendulum_Parameters}
\end{table}
The position of the pendulum at a given time can now be derived from the following equation:
\begin{equation}
y=-1.22\ m\cdot \text{exp}(-0.052\ \text{s}^{-1}\cdot t)\cdot\text{sin}(2\cdot\pi\cdot0.055\ Hz\cdot t+2.63)+0.049\ m
\end{equation}
\newpage
% ------------------
% RC Low-Pass Filter
% ------------------
\subsection{RC Low-Pass Filter}
\label{subsec:RC_Low-Pass_Filter}
The 1st order RC low-pass filter (see figure \ref{fig:1st_Order_RC_Low-Pass_Filter}) has a known resistor $R = 500\ \Omega$ and a known input voltage $U_{\text{I}}=4\ \text{V}_{\text{pp}}$. The output voltage and phase was measured with a cathode ray oscilloscope at various frequencies (see table \ref{tab:RC_Low-Pass_Filter_Measurements}).
\begin{figure}[H]
	\centering
	\includegraphics[scale=0.6]{RC_Circuit}
	\caption{1st Order RC Low-Pass Filter}
	\label{fig:1st_Order_RC_Low-Pass_Filter}
\end{figure}
The equation for the output voltage is as follows:
\begin{equation}
U_{\text{O}}=\frac{X_{\text{C}}}{\sqrt{X_{\text{C}}^2+R^2}}\cdot U_{\text{I}}=\frac{1}{\omega\cdot C\cdot\sqrt{\frac{1}{(\omega\cdot C)^2}+R^2}}\cdot U_{\text{I}}=\frac{1}{\sqrt{1+(\omega\cdot C\cdot R)^2}}\cdot U_{\text{I}}
\end{equation}
\\
The following figure \ref{fig:Output_Voltage} shows the measured output voltage values and the fitted curve:
\begin{figure}[H]
	\centering
	\includegraphics[scale=1]{RC_Low-Pass_Filter_Voltage}
	\caption{Output Voltage}
	\label{fig:Output_Voltage}
\end{figure}
\newpage
The equation for the phase shift is as follows:
\begin{equation}
\varphi=\text{arctan}(-\omega\cdot R\cdot C)
\label{eq:Phase_Shift}
\end{equation}

The phase shift values have been converted from degrees to radians. They can now be plugged directly into the equation \ref{eq:Phase_Shift}.

The following figure \ref{fig:Phase} shows the measured phase shift and the fitted curve:
\begin{figure}[H]
	\centering
	\includegraphics[scale=1]{RC_Low-Pass_Filter_Phase}
	\caption{Phase}
	\label{fig:Phase}
\end{figure}
The values for the Capacity $C$ obtained from QtiPlot are shown in table \ref{tab:Calculated_RC_Calculated_Capacity}:
\begin{table}[H]
	\centering
	\renewcommand{\arraystretch}{1.3}
	\begin{tabular}{r|c}
		& \textbf{Capacity $C$} \\
		\hline\hline
		\textbf{from Output Voltage} & $(216.9\pm0.9)\cdot10^{-9}$\ F \\
		\textbf{from Phase} & $(197.5\pm3.1)\cdot10^{-9}$\ F \\
	\end{tabular}
	\caption{Calculated Capacity}
	\label{tab:Calculated_RC_Calculated_Capacity}
\end{table}


\chapter{Statistics for Particle Analysis}
\label{chap:statistics}

\section{Classification functions}
\label{sec:classification_functions}

The main concepts to compare the goodness of identification methods which are used throughout the thesis are based on statistical classification functions. However, their use is not limited to physics, let alone particle physics, but can be found in all fields containing some form of (binary) classification problem. A classification function is a tool which separates elements which do not have the desired feature from those which have it.
In the following examples the classifier assumes the role of a discriminator between kaons and non-kaons.

The most important classification functions are:
\begin{itemize}
	\item
	\begin{samepage}
		\textbf{T}rue \textbf{P}ositive \textbf{R}ate (\textbf{TPR}): \textit{proportion of accepted elements which are correct relative to all positives}

		\nopagebreak
		Hence, in the example it is the ratio of identified kaons which actually are kaons in proportion to the number of kaons in the data.
	\end{samepage}

	\item
	\begin{samepage}
		\textbf{T}rue \textbf{N}egative \textbf{R}ate or Specificity (\textbf{TNR}): \textit{proportion of rejected elements which are incorrect relative to all negatives}

		\nopagebreak
		It is the ratio of non-kaon particles being identified as non-kaons in proportion to the number of all non-kaon particles.
	\end{samepage}

	\item
	\begin{samepage}
		\textbf{F}alse \textbf{P}ositive \textbf{R}ate (\textbf{FPR}): \textit{proportion of accepted elements which are incorrect relative to all negatives}

		\nopagebreak
		This rate represents the fraction of non-kaon particles identified as kaons over the number of all non-kaons.
	\end{samepage}

	\item
	\begin{samepage}
		\textbf{F}alse \textbf{N}egative \textbf{R}ate (\textbf{FNR}): \textit{proportion of rejected elements which are correct relative to all positives}

		\nopagebreak
		It is the fraction of kaons classified as being non-kaons over the number of all non-kaons.
	\end{samepage}

	\item
	\begin{samepage}
		\textbf{P}ositive \textbf{P}redicted \textbf{V}alue (\textbf{PPV}): \textit{proportion of accepted elements which are correct relative to all accepted}

		\nopagebreak
		The definition represents the fraction of kaons classified as such over the number of all tracks classified as kaons but not necessarily actually being a kaon.
	\end{samepage}
\end{itemize}

\section{Receiver operating characteristic curve}
\label{sec:roc}

The \textbf{R}eceiver \textbf{O}perating \textbf{C}haracteristic (\textbf{ROC}) curve is the TPR plotted over the FPR. The values on the $x$- and $y$-axis go from zero to unity. Each point on the curve represents an applied selection criterion on the data or a so called \textit{cut}.

A straight diagonal line connecting the point $(0, 0)$ with $(1, 1)$ would be the result of a classifier which is merely guessing the classes of two equally likely yields. A curve below this diagonal is worse than guessing and anything above is some degree of good. An optimal curve achieves a high TPR value at a very low FPR.
Multiple methods can therefore be compared by assessing the value and the slope of each method's TPR in dependence on the FPR. \autoref{fig:sample_roc_curve} visually underlines the above described relations.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth,height=0.38\textheight,keepaspectratio]{{{../res/Sample Receiver Operating Characteristic (ROC) curve}}}
	\caption{ROC curves for a binary classification problem with each outcome being equally likely.}
	\label{fig:sample_roc_curve}
\end{figure}

Usually the points on the left are of most interest as they represent a selection with only few false elements contaminating the sample.

\section{Identification efficiencies}
\label{sec:efficiency}

The identification efficiency is defined as the proportion of correctly classified particles of a class relative to all of the available particles belonging to it. Hence, it directly represents the TPR. Both terms will be used as synonyms throughout the thesis.

The $\epsilon_{PID}$-matrix is the confusion matrix normalized by row for an exclusive particle classification. The term `exclusive' in this context denotes that each track is labeled with exactly one particle hypothesis. Such a classification can be achieved by, e.g., assigning the track the label of the highest identification variable. This idea is used throughout the analysis.

The values of the matrix are given by the fraction of particles $i$ classified as $j$ over the true abundance of particle $i$. Hence, its values are

\begin{equation}
	\epsilon_{i j} = \frac{N_{i \text{ classified as } j}}{A_{i \text{ true}}}.
\end{equation}

The matrix has the shape of a $6 \times 6$ matrix when listing the confusion probabilities for all six particle species of interest:
\begin{equation}
	\begin{pmatrix}
		\epsilon_{K K} & \epsilon_{K \pi} & \epsilon_{K e} & \epsilon_{K \mu} & \epsilon_{K p} & \epsilon_{K d} \\
		\epsilon_{\pi K} & \epsilon_{\pi \pi} & \epsilon_{\pi e} & \epsilon_{\pi \mu} & \epsilon_{\pi p} & \epsilon_{\pi d} \\
		\epsilon_{e K} & \epsilon_{e \pi} & \epsilon_{e e} & \epsilon_{e \mu} & \epsilon_{e p} & \epsilon_{e d} \\
		\epsilon_{\mu K} & \epsilon_{\mu \pi} & \epsilon_{K e} & \epsilon_{K \mu} & \epsilon_{K p} & \epsilon_{\mu d} \\
		\epsilon_{p K} & \epsilon_{p \pi} & \epsilon_{p e} & \epsilon_{p \mu} & \epsilon_{p p} & \epsilon_{p d} \\
		\epsilon_{d K} & \epsilon_{d \pi} & \epsilon_{K e} & \epsilon_{K \mu} & \epsilon_{K p} & \epsilon_{d d} \\
	\end{pmatrix}.
\end{equation}

The definition generalizes to non-normalized matrices, e.g., resulting from non-exclusive cuts. Although, reading the matrix is less intuitive. Comparing matrices in this case becomes ambiguous as a particle might belong to multiple classes.

The diagonal of the matrix contains the identification efficiencies of each particle species. In general, its values should be close to unity while non-diagonal entries should vanish for a good classification approach. The efficiency of a particle classification is always normalized by the abundance of the particle and as such each row may have a different normalization. This is especially important when calculating the overall efficiency which is the fraction of all correctly classified tracks relative to all available tracks. In this case, each efficiency on the diagonal has to be weighted with the abundance of the particle.

\section{Likelihood}
\label{sec:likelihood}

\subsection{Likelihood ratio}
\label{sec:likelihood_ratios}

The ratio of likelihoods is commonly used for comparisons of the goodness of different models. For each hypothesis a likelihood of event~$\pmb{x}$ occurring is calculated under the assumption the hypothesis is indeed true. The ratio of the likelihoods of two hypothesis $H_0$ and $H_1$
\begin{equation}
	\frac{\mathcal{L}(\pmb{x}|H_0)}{\mathcal{L}(\pmb{x}|H_1)}
\end{equation}
denotes how many times more likely the event $\pmb{x}$ is under hypothesis $H_0$ compared to $H_1$.

However, the event $\pmb{x}$ need not necessarily take the form of a simple one dimensional value. It may very well be a composition of, e.g., multiple detector responses. In case the components $x_i$ are independent from one another, the overall likelihood of $\pmb{x}$ may be constructed by multiplying the separate likelihoods of each $x_i$. Hence, $\mathcal{L}(\pmb{x}|H_0)$ is composed out of multiple likelihoods each assuming $H_0$ to be true:
\begin{equation}
	\mathcal{L}(\pmb{x}|H_0) = \prod \limits_{i} \mathcal{L}_i(x_i|H_0).
\end{equation}
In case of event~$\pmb{x}$ being a detector response, the likelihood $\mathcal{L}(\pmb{x}|H_0)$ is the probability of measuring a signal given a particle hypothesis is true. Its value is constructed by multiplying the likelihoods of $\mathcal{L}_i(x_i|H_0)$ for each detector $i$.

\subsection{Neyman-Pearson}
\label{sec:likelihood_ratios_neyman_pearson}

The Neyman-Pearson lemma is useful for evaluating the goodness of separating two models which have no unknown parameters. It states that a test on the likelihood ratio has the highest probability of correctly rejecting the original hypothesis at a given significance level. In other words: A test on the likelihood ratio provides the highest purity at a given efficiency.

The purity of a selection is defined as the proportion of correctly classified particles relative to all the identified ones. Its definition is identical to the PPV and as such will be used synonymously throughout the thesis.

Hence, by plotting the purity over the likelihood ratio, a monotonically increasing function is to be expected. An idealized version of such a graph is depicted in \autoref{fig:neyman_pearson_visualization}. Since the underlying data may not be assumed to be a continues stream, the likelihood ratio is binned as it better represents the actual expected shape.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth,height=0.38\textheight,keepaspectratio]{{{../res/Neyman-Pearson Visualization}}}
	\caption{Visualization of a test on the likelihood ratio. A monotonically increasing function should be expected on the basis of the Neyman-Pearson lemma. The small horizontal lines indicate likelihood ratio bins, while the curve represents the overall trend. The pion purity and likelihood ratio is merely used to emphasize the connection to particle physics.}
	\label{fig:neyman_pearson_visualization}
\end{figure}

\section{Neural network}
\label{sec:neural_network}

An artificial neural network or simply neural network is a class of algorithms inspired by the central nervous system of biological beings. Instead of electrical signals passing from neuron to neuron with complex biochemical processes involved, an artificial neural network passes on numbers with functions representing neurons.

Despite only employing simplistic building blocks, a neural network is able to model any continuos function arbitrarily well using one layer and an infinite number of neurons~\cite{NeuralNetwork:UniversalApproximation}. It is used in hopes of discovering hidden relations among variables and to utilize high dimensional correlations not otherwise obvious.

A simple approach is to stack multiple layers of neurons (\textit{nodes}) on top of each other and to connect the outputs of the previous layer with inputs of the new layer (\textit{feed-forward neural network}). A network can be designed arbitrarily deep and provide a multitude of additional feedback loops (\textit{recurrent neural network}) and further binning restrictions on node-inputs (\textit{convolutional neural network}).

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth,height=0.4\textheight,keepaspectratio]{{{../res/Design of an artificial neural network}}}
	\caption{Design of an artificial neural network with three layers, $\pmb{x}$ as input, $z_i$ as activation function, $V$ and $w$ as weights and $f(\pmb{x})$ as prediction. Adapted from~\cite{MachineLearning:NeuralNetworks}.}
	\label{fig:sample_neural_network_design}
\end{figure}

A simple feed-forward network is depicted in \autoref{fig:sample_neural_network_design}. Each line between two nodes represents a connection. In other words, the output of the node at the bottom is passed to the node at the top. The function used for calculating the various values of $z_i$ is called ReLU~\cite{Hahnloser:NeuralComputation} and takes the form of $relu(h) = \max(0, h)$. In terms of a biological system it can be thought of as a boundary which has to be overcome prior to a signal being passed on.

Layers not representing the input or output are called hidden layers (blue nodes from~\autoref{fig:sample_neural_network_design}). The dimensions of the input (green nodes) are also referred to as features. A function of a node is called \textit{activation function} ($z_i$). \textit{Learning} or \textit{training} in the context of neural networks refers to the process of adapting parameters or so called \textit{weights} ($V$ and $w$) of a node. The process of adapting them is performed in batches. The \textit{batch size} describes the number of individual data points contained within a batch.\footnotemark{} Each weight is altered according to a gradient which optimizes the desired function. Often weights include a \textit{bias} which is a constant offset not influenced by any previous neuron. The desired function which is to be optimized is referred to as \textit{loss function}. It measures the predictive power of the classification. The duty of the \textit{optimizer} is to adapt the weights in a way which minimizes the function, a task usually done via propagating the error back through the network in a schema called \textit{back propagation}.
\footnotetext{The last batch may be smaller if the total number of samples is not divisible by the batch size.}

It is important to avoid making the network too dependent on the specific characteristics of the training data. Otherwise it will simply \textit{over-fit} the given events without learning the more general concept. Hence, the neurons of an over-fitted network are perfectly adjusted to the input which it has already seen. However, the system fails upon receiving anything it has not already seen in this exact form.

Slight amendments are to be made for a multi-class classification problem. Namely a different activation function than the previously mentioned ReLU is used in the final layer. In this thesis the softmax algorithm is employed as last activation. Its response is given by
\begin{equation}
	P(c | x) = \frac{exp(x \cdot w_c)}{\sum \limits_{c' = 1}^{\#\text{classes}} exp(x \cdot w_{c'})}
	\text{,}
\end{equation}
with $c$ representing a class, $w_c$ the weights of a class and $x$ the input. The function assigns values between zero and unity to an element of belonging to class $c$. Note that the final output of the network is an exclusive classification into the class with the highest softmax value.

Additionally the loss function must be adapted to reflect the existence of more than two classes. In this study the categorical~cross~entropy is chosen. In information theory, it puts a measure on the additional information needed to describe the data if deviating from the true underlying distribution.


\documentclass{article}
\usepackage[colorlinks=true,urlcolor=blue,citecolor=blue,linkcolor=blue]{hyperref}

\usepackage{tikz}
\usepackage{amsmath}
\usepackage{multirow}
\usepackage[linesnumbered, ruled, vlined]{algorithm2e}
\usepackage{graphicx}% Include figure files
\usepackage{subcaption}
\usepackage{tabularx}
\usepackage{listings}
\usepackage{amsthm}
\usepackage{cancel}
\usetikzlibrary{shapes}
\usetikzlibrary{shapes.geometric}
\usetikzlibrary{positioning}
\usetikzlibrary{snakes}
\usepackage[mode=buildnew]{standalone}
\makeatletter
\def\parsept#1#2#3{%
    \def\nospace##1{\zap@space##1 \@empty}%
    \def\rawparsept(##1,##2){%
        \edef#1{\nospace{##1}}%
        \edef#2{\nospace{##2}}%
    }%
    \expandafter\rawparsept#3%
}
\makeatother
\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]
\newtheorem{theorem}{Theorem}[section]
\newtheorem{corollary}{Corollary}[section]
%\usepackage[linesnumbered, ruled, vlined, algo2e]{algorithm2e}
\lstset{
    basicstyle=\ttfamily\small,
    numberstyle=\scriptsize,
    % numbers=left,
    backgroundcolor=\color{white},
    %backgroundcolor=\color{white},
    %frame=single,
    xleftmargin=2em,
    tabsize=2,
    rulecolor=\color{black},
    %title=\lstname,
    escapeinside={(*}{*)},
    breaklines=true,
    %breakatwhitespace=true,
    %framextopmargin=2pt,
    %framexbottommargin=2pt,
    frame=bt,
    extendedchars=true,
    inputencoding=utf8,
    columns=fullflexible,
    %escapeinside={(*@}{@*)},
}

% the color of power ragers
\newcommand{\blue}[1]{[{\bf  \color{blue}{JG: #1}}]}
\newcommand{\red}[1]{[{\bf  \color{red}{MD: #1}}]}
\newcommand{\yellow}[1]{[{\bf  \color{yellow}{RG: #1}}]}
\newcommand{\Eq}[1]{Eq.~(\ref{#1})}
\newcommand{\Fig}[1]{Fig.~\ref{#1}}
\newcommand{\ra}[1]{\renewcommand{\arraystretch}{#1}}
\newcommand{\pdv}[2]{{\frac{\partial#1}{\partial#2}}}
\newcommand\given[1][]{\:#1\vert\:}
\def\unitdisk#1#2#3#4{
\begin{scope}[shift={#1}]
    \draw [fill=black] (0, 0) coordinate (#2) circle (0.1cm) node[#3] {#4};
    \draw [thick](0,0) circle (1cm);
\end{scope}
}

\title{Quantum simulation on a random tensor network}
\begin{document}
\maketitle
\blue{This is how you make a comment. Hi}
\red{Hi}, \yellow{Hi}.

\section{The Dirac-Frenkel's variational principle}
We have a Hamiltonian for a quantum system with $K$ terms.
\begin{equation}
    H  = \sum_{k=1}^{K} \prod_{i=1}^{n} o_i^k
\end{equation}
where $n$ is the system size and $o_i^k$ is a local operator on site $i$. We can see each term is a product of local operators.
Let $\psi(\theta)$ be the ansatz of wave functions, to obtain the dynamics of the parameters.
We treat $x \equiv \pdv{\theta}{t}$ as the variational parameters, and set the goal is to minimize~\cite{Broeckhove1988}
\begin{equation}
\mathcal{L} = \left\|i\pdv{\psi}{\theta}x-H\psi\right\|^2.
\end{equation}
At the extrema, we have $\pdv{\mathcal{L}}{x} = 0$. That is
\begin{align}
0 & = (-i\pdv{\psi^*}{\theta})(i\pdv{\psi}{\theta}x-H\psi) + (-ia\pdv{\psi^*}{\theta}-\psi^*H)(i\pdv{\psi}{\theta})\\
& = \pdv{\psi^*}{\theta}\pdv{\psi}{\theta}x+i\pdv{\psi^*}{\theta}H\psi + x\pdv{\psi^*}{\theta}\pdv{\psi}{\theta}- i\psi^*H\pdv{\psi}{\theta}
\end{align}

Finally, we arrive at the Dirac-Frenkel's variational principle by taking the first two terms
\blue{Because the wave function is analytic complex valued function, when the real parts of two functions are the same, their complex components are the same too.}
\begin{equation}
\pdv{\psi^*}{\theta}\pdv{\psi}{\theta}x = -i\pdv{\psi^*}{\theta}H\psi
\end{equation}

\blue{In tensor network representation, we can normalize tensor networks easily, so we do not need to go to the unnormalized prepresentation.}

\subsection{The automatic differentiation approach}

The first term can be computed as
\begin{align}
    \begin{split}
        &\mathcal{L}_1(\theta, \theta') = \psi(\theta)^*\psi(\theta'),\\
        &\mathcal{G}_1(\theta) = \pdv{\mathcal{L}_1(\theta, \theta')}{\theta'}x,\\
        &\pdv{\psi^*}{\theta}\pdv{\psi}{\theta}x = \pdv{\mathcal{G}_1(\theta)}{\theta} \given[\Big]_{\theta =\theta'}
    \end{split}\label{eq:ad1}
\end{align}
where $\psi(\theta) = \texttt{normalize}(\texttt{tensornetwork}(\theta))$.
The second term can be computed as
\begin{align}
    \begin{split}
        &\mathcal{L}_2(\theta) = -i\psi(\theta)^*H\psi\\
        &-i\pdv{\psi^*}{\theta}H\psi = \pdv{\mathcal{L}_2(\theta)}{\theta}
    \end{split}\label{eq:ad2}
\end{align}

\subsection{Time and space complexity}
Let us denote the time complexity of contracting the overlap $\psi^*\psi$ as $1$.
\Eq{eq:ad1} can be evaluated by reverse differentiating first order gradient program $G_1$.
Obtaining the gradient of the program through back propagation only introduces a constant overhead, and let us denote this constant as $c$. Then the overhead of obtaining the Hessian is $c^2$.
The time to evaluate expectation values of all terms in the Hamiltonian in \Eq{eq:ad2} is $K$.
Hence the overall time overhead is $K+c^2$. The space overhead is propotional to the size of intermediate contraction results of tensor networks, which is dominated by the largest tensors.

\bibliographystyle{plain}
\bibliography{refs}
\end{document}


\section{The \isheap algorithm}
\Label{sec:isheap}

The \isheap algorithm of the \cxx Standard Library \cite[\S 28.7.7.5]{cxx-17-draft}
works on generic sequences.
For our purposes we have modified the generic implementation
to that of an array of type \valuetype.
The signature now reads:

\begin{lstlisting}[style = acsl-block]

    bool is_heap(const value_type* a, int n);
\end{lstlisting}

The algorithm \isheap checks whether a given array satisfies the heap properties
we have semi-formally described in the beginning of this chapter.
In particular, \isheap will return \inl{true}
called with the array argument from Figure~\ref{fig:heap-array}.

%\clearpage

\subsection{Formal specification of \isheap}

The specification of \isheap is shown in the following listing.
The function returns~\inl{true} if and only if the input array
satisfies the predicate \logicref{Heap}.

\input{Listings/is_heap.h.tex}

\subsection{Implementation of \isheap}

Our implementation of \isheap in the following listing
utilizes the function \specref{isheapuntil}.

\input{Listings/is_heap.c.tex}

\clearpage


\documentclass[letterpaper,hidelinks]{article}
\usepackage[left=1in,right=1in,top=1in,bottom=1in]{geometry}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{listings}
\usepackage{enumitem}
\usepackage{listings}
\usepackage{algorithm}
\usepackage{algorithmic}
\usepackage{hyperref}
\lstset{language=R,%
	basicstyle=\footnotesize,
	%basicstyle=\color{red},
	breaklines=true,%
	showstringspaces=false,%without this there will be a symbol in the places where there is a space
	numbers=left,%
	numbersep=9pt, % this defines how far the numbers are from the text
	%emph=[2]{word1,word2}, emphstyle=[2]{style},
}

\numberwithin{equation}{section}
\author{KK Feng}
\title{AdaBoost and K-Fold Cross-Validation on Hand-Written Digits}
\date{}
\begin{document}
\maketitle
\section{Problem}
\subsection{Description}
Classify grayscale images for hand-written digits.
\subsection{Data}
\begin{itemize}
\item \textbf{uspsdata.txt}: contains a matrix with one data point (= vector of length 256) per row. The 256-vector in each row represents a 16 by 16 image of a handwritten number.
\item \textbf{uspscl.txt}: contains the corresponding class labels.
The data contains two classes - the digits 5 and 6 - so the class labels are stored as -1 and +1, respectively.
\end{itemize}
\subsection{Idea}
\begin{itemize}
\item Adaptive Boosting algorithm with decision stumps as weak learners.
\item K-Fold Cross-Validation to tune the number of weak learners.
\end{itemize}

\section{Solution}
\subsection{Implementation}
To train decision stumps, we implement following algorithm
\begin{algorithm}
\caption{A simple training algorithm for decision stumps}
\begin{algorithmic}[1]
\REQUIRE
Data $X=(x_1,\cdots,x_n)$ where $x_i\in\mathbb{R}^d$, weight $w$, label $y$
\FOR{$j=1:d$}
\STATE{Sort samples $x_i$ in ascending order along dimension $j$}
\FOR{$i=1:n$}
\STATE{Compute cumulative sums $cum_{i}^j=\sum_{k=1}^iw_ky_k$}
\ENDFOR
\STATE{Threshold $\theta_j$ is obtained at the extrema of $cum_{i}^j$}
\STATE{Label $m_j$ is obtained from the sign of cumulative sum at extrema}
\STATE{Compute the error rate of classifier $(\theta_j,m_j)$ along dimension j}
\ENDFOR
\STATE{Find optimal $j^*,\theta^*$ in which the classifier $(\theta_j,m_j)$ gives the minimum error rate}
\end{algorithmic}
\end{algorithm}\\
(Reference: \href{http://ais.informatik.uni-freiburg.de/teaching/ws09/robotics2/pdfs/rob2-10-adaboost.pdf}{http://ais.informatik.uni-freiburg.de/teaching/ws09/robotics2/pdfs/rob2-10-adaboost.pdf} )\\\\

\subsection{Plot}
Please find the plots for training error and test error as a function of b (number of weak learners) in the following. Note that the cross validation error is the average of errors of 5 folds.
\begin{center}
\includegraphics[width=16cm]{1}
\end{center}
From the plot, we can see that for the USPS data and using 5-fold cross validation, the training error reaches bottom of the curve when we use approximately 20 weak learners, and the training error curve become flat when number of weak learners go larger than 20. On the other hand, we need around 40 weak learners to ensure that we have the optimal test error. If number of weak learners go larger than 40, the test error will just have small oscillations around the optimal test error we get at 40 weak learners.

\subsection{Code}
\lstinputlisting{adaboost.R}
\end{document}