% Chapter X

\chapter{Learning model parameters from data} % Chapter title

\label{Learning model parameters from data} % For referencing the chapter elsewhere, use

Figaro provides support for learning model parameters from data. In this section, a special type of compound element will be presented which allows a distribution to be learned from observed evidence. Details are given about the algorithm Figaro provides for learning parameters. Lastly, an example using parameters and learning algorithms to determine the fairness of a set of die rolls is presented.

\section{Parameters and parameterized elements}

This section discusses elements which are learnable parameters. For clarity, a distinction should be made on the meaning of the word \emph{parameter} in this context. This is different from a method parameter or Scala type parameter. In this section, we use \emph{parameter} to refer to a Figaro element which can be learned from data. There are currently two such types of parameters in Figaro: (1) \texttt{Beta}, and (2) \texttt{Dirichlet}.

A customary illustration of parameter learning is to consider the outcomes of a coin flip and determine whether or not the coin is fair. In the case of a \texttt{Flip} element (which is a Bernoulli distribution), the conjugate prior distribution is a Beta distribution. If the coin is not fair, we would expect a prior distribution to have a higher value of alpha or beta (the shape variables of a Beta). First, we will create the conjugate prior distribution of a \texttt{Flip}:

\begin{flushleft}
\texttt{val fairness = Beta(1,1)}
\end{flushleft}

The element \texttt{fairness} is the parameter we will use to model the bias of our coin. The creation of the parameter is no different than  Most importantly, we later use it to create a model learned from parameterized elements. Creation of a parameterized element is accomplished in exactly the same way as creating a compound element.

\begin{flushleft}
\texttt{val f = Flip(fairness)}
\end{flushleft}

This element models a flip of a coin having the fairness specified by the beta parameter, using a value of \texttt{true} to represent heads and \texttt{false} to represent tails. We have actually created an instance of \texttt{ParameterizedFlip}, which is a special type of compound element. A  \texttt{ParameterizedFlip} is created simply by providing a \texttt{Beta} as the argument to \texttt{Flip}.

By using a \texttt{ParameterizedFlip}, the evidence we observe on f can be used to learn the value of \texttt{fairness}. Thus, the next step is to provide the data observed from flips of the coin. Values can be observed just as with other elements, by using \texttt{f.observe(true)} or \texttt{f.observe(false)}. We could also use conditions or constraints.

\marginpar{This example is found in FairCoin.Scala}
As a more detailed example, suppose we have seen 24 heads and 62 tails. One way to represent this data is in a Scala sequence. Note that for readability, the sequence is truncated here.

\begin{flushleft}
\texttt{val data = Seq('H', 'H', 'H', 'T', 'H', 'H', 'T', 'H', ...}
\end{flushleft}

The following block of Scala code will iterate through each of the items in the sequence, create a Flip element using the parameter, and observe true or false based on the side of the coin:

\begin{flushleft}
\texttt{data zip model.trials foreach \{
\newline \tab (datum: (Char, Flip)) => if (datum.\_1 == 'H')
\newline \tab datum.\_2.observe(true) else datum.\_2.observe(false)
\newline \}
}
\end{flushleft}

We have created a parameter, parameterized elements and considered a set of data. Note that each time a parameterize flip is created, it is using the same \texttt{Beta}. It is now desirable to employ a learning mechanism to determine the fairness of the coin, and to create a new element corresponding to the learned value. This is possible by using a learning algorithm.

\section{Expectation maximization}

A learning algorithm can be used to determine the maximum a posteriori estimate for parameter elements. Parameter elements have a \texttt{MAPValue} which is set when the parameter is used as a target in a learning algorithm. Presently, Figaro provides one learning algorithm, expectation maximization, which uses existing Figaro algorithms to estimate sufficient statistics. Recall that expectation maximization is an iterative algorithm consisting of an expectation step and a maximization step. During the expectation step, an estimate is produced for the sufficient statistics of the parameter. The estimates are then used in the maximization step to find the most likely value of the parameter. This continues for a set number of iterations and converges toward the true MAP value.

From a practical standpoint, learning a parameter with expectation maximization is very simple. We need only provide the target parameter and, optionally, the number of iterations to the algorithm. The default number of iterations is 10. We can also choose an inference algorithm for estimating the sufficient statistics of the target parameters. Currently, Metropolis Hastings, importance sampling, variable elimination or belief propagation can be used for this purpose. Figaro's Generalized EM algorithm is used in the following way:

\begin{flushleft}
\texttt{val learningAlgorithm = EMwithMH(fairness)
\newline learningAlgorithm.start
\newline learningAlgorithm.kill
\newline
\newline val coin = Flip(fairness.MAPValue)
\newline println("The probability of a coin with this fairness showing
'heads' is: ")
\newline println(coin.prob)
}
\end{flushleft}

The line \texttt{val learningAlgorithm = EMwithMH(fairness)} creates an EM algorithm which uses Metropolis Hastings to estimate sufficient statistics. We could also have used \texttt{EMwithBP(fairness)} or
\texttt{EMwithImp\-ortance(fairness)}.

After the algorithm has finished running, we can create an element learned from the parameter by using \texttt{Flip(fairness.MAPValue)}. The element \texttt{coin} is a \texttt{Flip}, where the probability of producing true is determined from the data we observed above.

After running the program, we see:

\begin{flushleft}
\texttt{The probability of a coin with this fairness showing 'heads' is:
0.7159090909090909}
\end{flushleft}

We may want to make further explorations about the learned model. For instance, if we wanted to know the probability that two flips of this coin show the same side, we could use:

\begin{flushleft}
\texttt{val t1 = Flip(fairness.MAPValue)
\newline val t2 = Flip(fairness.MAPValue)
\newline val equal = t1 === t2}
\end{flushleft}

We can then use an algorithm like variable elimination to determine the probability that the coins show the same side:

\begin{flushleft}
\texttt{val alg = VariableElimination(equal)
\newline alg.start()
\newline println("The probability of two coins which exhibit this fairness showing the same side is: " + alg.probability(equal, \textbf{true}))
\newline alg.kill()
}
\end{flushleft}

This results in the following output:

\begin{flushleft}
\texttt{The probability of two coins which exhibit this fairness showing the same side is: 0.5932334710743803}
\end{flushleft}

\section{Parameter collections}

In the previous sections, parameter learning was discussed using a Beta parameter. The Beta parameters were supplied individually to the learning algorithm, and the MAP value for each parameter was retrieved individually. For more complicated models, it is often useful to define a model structure with parameters which can be learned, and then use the values of the learned parameters in the same structure. The \texttt{ModelParameters} pattern is a simple way of accomplishing this. By using \texttt{ParameterCollection}, the model structure only needs to be defined once. Parameters are added to the collection in a similar fashion to \texttt{Element} collections. We specify the parameter name and add it to the collection when we create the parameters.

This section will also explain the use of Dirichlet parameters. The Dirichlet distribution is a multidimensional generalization of the Beta with a variable number of concentration parameters or alpha values. These values correspond to the weight of each possible outcome in the posterior distribution. In a Dirichlet parameter with two dimensions, the alpha values might again correspond to the outcome of heads and tails, or true and false. Using a higher number of dimensions, we can model a number of different categories or outcomes.

Suppose we are given a set of data in which each record represents a roll of two die out of three possible die. The sum of the die is available, as well as which die were selected for the roll. However, the individual outcome of each die is not available. Our task is to learn the fairness of each die.

The first step is to define the possible outcomes from a dice roll. This is easily accomplished by using a Scala list:

\begin{flushleft}
\marginpar{This example is found in FairDice.Scala}
\texttt{val outcomes = List(1, 2, 3, 4, 5, 6)}
\end{flushleft}

Next, we create a set of model parameters representing the parameters of the fair dice model. We create a parameter representing the fairness of each die and add it to the collection of model parameters.

\begin{flushleft}
\texttt{val params = ModelParameters()
\newline val fairness1 = Dirichlet(2.0, 2.0, 2.0, 2.0, 2.0, 2.0)("fairness1", params)
\newline val fairness2 = Dirichlet(2.0, 2.0, 2.0, 2.0, 2.0, 2.0)("fairness1", params)
\newline val fairness3 = Dirichlet(2.0, 2.0, 2.0, 2.0, 2.0, 2.0)("fairness1", params)
}
\end{flushleft}

Each die is initially assumed to be fair. For convenience, the data which we will learn the parameters from is represented in Scala sequence:

\begin{flushleft}
\texttt{val data = Seq((2, 3, 8), (1, 3, 7), (1, 2, 3), (1, 2, 3), ...}
\end{flushleft}

\texttt{data} is a sequence of 50 Scala tuples. The first two values in each tuple indicate which two die were chosen to roll. The third value is the sum of the two die.

The next step is to define a class representing the model structure.

\begin{flushleft}
\texttt{class DiceModel(val parameters: ParameterCollection, val data: Seq[(Int, Int, Int)], val outcomes: List[Int])}
\end{flushleft}

This defines a class which accepts a \texttt{ParameterCollection}, a set of data, and a list of outcomes as its arguments. As we will see, the values which are retrieved from the \texttt{ParameterCollection} depend on whether we are working with the prior or posterior parameters. To model the outcome of the sum, we can use an \texttt{Apply} element with a function which sums the outcome of its arguments. We place the following loop inside the \texttt{DiceModel} class:

\begin{flushleft}
\texttt{val sum = (i1: Int, i2: Int) => i1 + i2
	\newline val trials = for (datum <- data) yield \{
    \newline \tab val die1 = Select(parameters.get("fairness" + datum.\_1), outcomes: \_*)
    \newline \tab val die2 = Select(parameters.get("fairness" + datum.\_2), outcomes: \_*)
    \newline \tab Apply(die1, die2, sum)
	\newline \}
  }
\end{flushleft}

The code section above defines a Scala function which accepts two Dirichlet parameters and an integer value. \texttt{val sum = (i1: Int, i2: Int) => i1 + i2} defines a Scala function which accepts two integer values and returns their sum.  Next, two Select elements are created and parameterized by the input parameters. We retrieve the parameters by using the \texttt{get} method from the input \texttt{ParameterCollection}.

Note that the arguments to \texttt{Select} are different from what has been presented previously. Instead of directly enumerating each probability and outcome, we specify a Dirichlet parameter and the list of possible outcomes. The last two lines of \texttt{trial} apply the sum function to the die and observe the result. By calling the \texttt{trial} function for each tuple in the sequence, we can create a model learned from the data.

We can now create an instance of the \texttt{DiceModel} class, using the prior parameters from the parameter collection.

\begin{flushleft}
 \texttt{val model = new DiceModel(params.priorParameters, data, outcomes)
}
\end{flushleft}

To apply evidence to the model, we can write another loop over the contents of the data and the trials defined inside the model class.

\begin{flushleft}
\texttt{for ((datum,trial) <- data zip model.trials) \{
    \newline \tab trial.observe(datum.\_3)
    \}
}
\end{flushleft}

Just as in the fair coin example, we create an expectation maximization algorithm. This time, instead of passing the parameters in a list or sequence, we can simply use the collection of parameters as an input argument.

\begin{flushleft}
\texttt{val numberOfBPIterations = 10
	\newline val numberOfEMIterations = 10
    \newline val algorithm = EMWithBP(numberOfEMIterations, numberOfBPIterations, params)
    \newline algorithm.start
    \newline algorithm.stop
    \newline val d1 = Select(params.posteriorParameters.get("fairness1"), outcomes:\_*)
    \newline val d2 = Select(params.posteriorParameters.get("fairness2"), outcomes:\_*)
    \newline val d3 = Select(params.posteriorParameters.get("fairness3"), outcomes:\_*)
}
\end{flushleft}

The code block above will create \texttt{Select} elements using to the MAP value of the learned parameters. We retrieve the MAP value of the parameters by using the \texttt{posteriorParameters.get} method of our parameter collection. If we wanted to create another set of 50 trials using the learned parameter values, we could simply use:

\begin{flushleft}
 \texttt{val model = new DiceModel(params.posteriorParameters, data, outcomes)
}
\end{flushleft}

Note that for a \texttt{Select}, a list of outcomes must be supplied as an argument to along with their corresponding probabilities. This is because the number of concentration parameters is may vary, and the type of the outcomes is not fixed. Running this code results in the following output, in which we see the model has estimated the probabilities of each value for each die. If one examines the full data declaration in the example code, it is quite easy to see that there are only three observed values of the sum of the die (3, 7 and 8), so the learning algorithm has correctly inferred that the most likely values of the die are 1, 2 and 6, respectively.

\begin{flushleft}
\texttt{The probabilities of seeing each side of d\_1 are:
\newline \tab 0.906250000442371 -> 1
\newline \tab 0.0 -> 2
\newline \tab 0.0 -> 3
\newline \tab 0.0 -> 4
\newline \tab 0.09374999955762903 -> 5
\newline \tab 0.0 -> 6
}
\end{flushleft}

\begin{flushleft}
\texttt{The probabilities of seeing each side of d\_2 are:
\newline \tab 0.0 -> 1
\newline \tab 0.9999999996067813 -> 2
\newline \tab 0.0 -> 3
\newline \tab 0.0 -> 4
\newline \tab 0.0 -> 5
\newline \tab 3.9321864899990694E-10 -> 6
}
\end{flushleft}

\begin{flushleft}
\texttt{The probabilities of seeing each side of d\_3 are:
\newline \tab 0.0 -> 1
\newline \tab 0.0 -> 2
\newline \tab 0.0 -> 3
\newline \tab 0.0 -> 4
\newline \tab 0.0 -> 5
\newline \tab 1.0 -> 6
}
\end{flushleft}


\documentclass[11pt]{article}
\usepackage[top=1.5cm,bottom=1.5cm]{geometry}
\usepackage[T1]{fontenc}
\usepackage{url}
\usepackage{hyperref}
\usepackage{xcolor}
%
\usepackage{amsthm}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{ifthen}
\usepackage{mathpartir}
\usepackage{mathtools}
\input{macros}
%
\begin{document}

\title{CO663: Language $\lggeT$ --- summary\footnote{Adapted from:
    Harper, Robert. Practical foundations for programming
    languages. Cambridge University Press, 2016.}}

\date{\vspace{-15ex}}
\maketitle

% \pagenumbering{gobble}

\newcommand{\lazyeval}[1]{\framebox{\parbox[c][#1]{\textwidth}{
      \color{white}{h}%
    }}}

\section{Syntax}

\[
\begin{array}{llclll}
  \TYPES & \tau & \Coloneqq & \tynat  & \tynat & \text{natural}
  \\
         &&& \tyarr{\tau_1}{\tau_2}  & \tau_1 \rightarrow \tau_2 & \text{function}
  \\
  \\
  \EXPS & e & \Coloneqq & \var{x} & \var{x} & \text{variable}
  \\
         &&& \ez &  \ez & \text{zero}
  \\
         &&& \esucc{e} &  \esucc{e} & \text{successor}
  \\
         &&& \elam{\tau}{x}{e} & \clam{\tau}{x}{e} & \text{abstraction}
  \\
         &&& \eapp{e_1}{e_2} & \capp{e_1}{e_2} & \text{application}
  \\
         &&& \eiter{e_0}{x}{e_1}{e} & \multicolumn{2}{c}{\citer{e_0}{x}{e_1}{e}}
\end{array}
\]

\section{Typing rules / Statics}

\[
\tyrule
{\ttyrulename{var}}
{\,}
{\tyjudge{\Gamma_1, \var{x}: \tau, \Gamma_2}{\var{x}}{\tau} }
\qquad\qquad\qquad
\tyrule
{\ttyrulename{nat}}
{\,}
{\tyjudge{\Gamma}{\ez}{\tynat}}
\qquad\qquad\qquad
\tyrule
{\ttyrulename{num}}
{\tyjudge{\Gamma}{{e}}{\tynat{}}}
{\tyjudge{\Gamma}{\esucc{e}}{\tynat{}}}
\]
%
\[
\tyrule
{\ttyrulename{lam}}
{\tyjudge{\Gamma, \var{x} : \tau_1}{e}{\tau_2}}
{\tyjudge{\Gamma}{\elam{\tau_1}{x}{e}}{\tyarr{\tau_1}{\tau_2}}}
\qquad\qquad\qquad
\tyrule
{\ttyrulename{ap}}
{
  \tyjudge{\Gamma}{e_1}{\tyarr{\tau_1}{\tau}}
  \and
  \tyjudge{\Gamma}{e_2}{\tau_1}
}
{
  \tyjudge{\Gamma}{\eapp{e_1}{e_2}}{\tau}
}
\]
\[
\tyrule
{\ttyrulename{ite}}
{
  \tyjudge{\Gamma}{e}{\tynat}
  \and
  \tyjudge{\Gamma}{e_0}{\tau}
  \and
  \tyjudge{\Gamma, \var{x} : \tau}{e_1}{\tau}
}
{
  \tyjudge{\Gamma}{\eiter{e_0}{x}{e_1}{e}}{\tau}
}
\]

\section{Semantics / Dynamics}

% \subsection{Values}
\[
\semrule{\tsemrulename{z}}
{\,}
{\valjudge{\ez}}
\qquad \qquad  \qquad
\semrule{\tsemrulename{s}}
{\valjudge{e}}
{\valjudge{\esucc{e}}}
\qquad \qquad  \qquad
\semrule
{\tsemrulename{lam}}
{\,}
{\valjudge{\elam{\tau}{x}{e}}}
\]

% \subsection{Reductions}

\[
\semrule
{\tsemrulename{ss}}
{\jtrans{e}{e'}}
{
  \jtrans{\esucc{e}}{\esucc{e'}}
}
\qquad \qquad  \qquad
%
\semrule
{\tsemrulename{ap}}
{\jtrans{e_1}{e'_1}}
{
  \jtrans{\eapp{e_1}{e_2}}{\eapp{e'_1}{e_2}}
}
\]
%
%
\[
\semrule
{\tsemrulename{lan}}
{\valjudge{e_1}
  \and \jtrans{e_2}{e'_2}
}
{\jtrans{\eapp{e_1}{e_2}}{\eapp{e_1}{e'_2}}}
\qquad \qquad  \qquad
\semrule
{\tsemrulename{lav}}
{\valjudge{e_2}}
{\jtrans{\eapp{\elam{\tau}{x}{e_1}}{e_2}}{\subs{e_1}{e_2}{x}}}
\]
%
\[
\semrule
{\tsemrulename{rin}}
{\jtrans{e}{e'}}
{
  \jtrans{\eiter{e_0}{x}{e_1}{e}}{\eiter{e_0}{x}{e_1}{e'}}
}
\qquad \qquad  \qquad
%
\semrule
{\tsemrulename{r0}}
{\,}
{
  \jtrans{\eiter{e_0}{x}{e_1}{\ez}}{e_0}
}
%
\]
\[
%
\semrule
{\tsemrulename{rs}}
{\valjudge{{e}}}
{
  \jtrans{\eiter{e_0}{x}{e_1}{\esucc{e}}}{\subs{e_1}{{\eiter{e_0}{x}{e_1}{{e}}}}{x}}
}
\]

\section*{\LaTeX\ template}
The \LaTeX\ sources of this document are available in:
\url{https://github.com/ukc-co663/pl-design-latex-template}.

\end{document}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:


\section{Complex Numbers}

\subsection{Arithmetic Operations}

\subsubsection{Exercise 3}
This exercise is much easier if we represent the complex numbers in polar form (which has not been introduced
yet). We see that
\begin{align*}
        \frac{-1 \pm i \sqrt{3}}{2} &= e^{i \frac{2\pi}{3}}, \: e^{i \frac{4 \pi}{3}} \\
        \frac{\pm 1 \pm i \sqrt{3}}{2} &= e^{i \frac{2\pi}{3}}, \: e^{i \frac{4 \pi}{3}}, \:
        e^{i \frac{\pi}{3}}, \: e^{i \frac{5 \pi}{3}} \\
\end{align*}
which gives us the desired equality since $e^{i 2\pi} = 1$.

\subsection{Square Roots}
Once again, all of these computational exercises are made much easier with polar form.
They seem more tedious than instructive.

\subsection{Justification}
Going to come back to this after finishing chapter 3 of Birkhoff/MacLane (which I've been neglecting...).

\subsection{Conjugation, Absolute Value}

\subsubsection{Exercise 3}
We can manipulate the equality to get
\begin{align*}
        \norm{\frac{a - b}{1 - \bar{a}b}} &= 1 \\
        \norm{a - b}^2 &= \norm{1 - \bar{a}b}^2 \\
        \norm{a}^2 + \norm{b}^2 - 2\Re(a\bar{b}) &= 1 + \norm{a}^2 \norm{b}^2 - 2\Re(a\bar{b})
\end{align*}
Thus we see that equality holds if either $\norm{a} = 1$ or $\norm{b} = 1$, excepting the case where
$a = b = 1$ as that makes the denominator in the equality 0.

\subsubsection{Exercise 4}
Let $z = \alpha + \beta i$. Then we have
\begin{align*}
        \big((a + b) \alpha + c\big) + (a - b)\beta i &= 0 \\
        \implies (a + b) \alpha + c &= 0 \\
        \implies (a - b) \beta = 0
\end{align*}
If $a - b = 0$, $\beta$ could be anything, so we must have $a - b \neq 0$ for the solution to be unique.
Similarly, if $a + b = 0$ then $\alpha$ can either be anything or there is no solution for $\alpha$
(if $c \neq 0$). Thus, the two conditions we need are  $a + b \neq 0$ and $a - b \neq 0$.

\subsubsection{Exercise 5}
We can write $\abs{\sum_{i = 1}^n a_i b_i}^2$ as $(\sum_{i = 1}^n a_i b_i) (\sum_{j = 1}^n \overline{a_j b_j})$
to see that it can be expanded as a sum whose terms consist of $\abs{a_i}^2 \abs{b_i}^2$ and
$a_i b_i \overline{a_j b_j}$ over all $1 \leq i, j \leq n$. The $a_i b_i \overline{a_j b_j}$ terms can be
paired with the $a_j b_j \overline{a_i b_i}$ terms to get that
\begin{align*}
        \abs{\sum_{i = 1}^n a_i b_i}^2 &= \sum_{i = 1}^n \abs{a_i}^2 \abs{b_i}^2 + \sum_{1 \leq i < j \leq n} 2 \Re a_i b_i \overline{a_j b_j} \\
                                       &= \sum_{i = 1}^n \abs{a_i}^2 \sum_{i = 1}^n \abs{b_i}^2 - \sum_{1 \leq i < j \leq n} \abs{a_i \bar{b}_j - a_j \bar{b}_i}^2
\end{align*}

\subsection{Inequalities}

\subsubsection{Exercise 3}
We apply the triangle inequality and then Cauchy-Schwarz to get
\begin{align*}
        \abs{\sum_{i} \lambda_i a_i} \leq \sum_{i} \abs{\lambda_i a_i} \leq \sum_{i} \abs{\lambda_i} \abs{a_i} < \sum_{i} \abs{\lambda_i} = 1
\end{align*}
Where the final strict inequality follows from $\abs{a_i} < 1$.

\subsubsection{Exercise 4}
We have that $\abs{z - a} = \abs{z} - \abs{a}$ when $\frac{z}{a} \geq 1$ (applying our criterion for equality
to $\abs{(z - a) + a}$). Thus, if $\abs{a} \leq \abs{c}$, we can choose $z$ such that $\frac{z}{a} \geq 1$
and $\abs{z} = \abs{c}$, so a solution to the equation exists. Now suppose a solution exists to
 $\abs{z - a} + \abs{z + a} = 2\abs{c}$. Then by applying the triangle inequality, we have
 \begin{align*}
         2\abs{c} \geq \abs{z - a} + \abs{z + a} \geq \abs{z - a - z - a} = 2\abs{a}
 \end{align*}
 so $\abs{a} \leq \abs{c}$.

\subsection{Geometric Addition and Multiplication}

\subsubsection{Exercise 2}
We will only show one direction, since the other direction just reverses the steps. Suppose that
$a_1, a_2, a_3$ indicate the vertices of an equilateral triangle. Then the angles between the edges
$a_1 - a_3, a_3 - a_2, a_2 - a_1$ must be equal. Using the fact that $\text{arg} \frac{a_2}{a_1} = \text{arg} a_2 - \text{arg} a_1$, we get
\begin{align*}
        \frac{a_1 - a_3}{a_3 - a_2} &= \frac{a_3 - a_2}{a_2 - a_1} \\
        a_1 a_2 - a_1^2 - a_2 a_3 + a_3 a_1 &= a_3^2 - 2 a_2 a_3 + a_2^2 \\
        a_1 a_2 + a_2 a_3 + a_3 a_1 &= a_1^2 + a_2^2 + a_3^2
\end{align*}

\subsubsection{Exercise 4}
Let the center of the circle be $c$. Then we can proceed similarly to Exercise 2 by noting that since
$\abs{c - a_1} = \abs{c - a_2} = \abs{c - a_3}$, we have that the angle between $c - a_1$ and $a_1 - a_3$
is the same as the angle between $a_1 - a_3$ and $a_3 - c$. We can then solve for $c$ and use $c$ to compute
the radius. The result, though, is messy and not fun to typeset.

\subsection{The Binomial Equation}

\subsubsection{Exercise 4}
Let $s = 1 + w^h + ... + w^{(n - 1)h}$. Then we have that $w^h s = w^h + w^{2h} + ... + w^{nh}$, which implies
that $w^h s = s$ since $w^{nh} = 1$. If $h$ is not a multiple of $n$, then $w^h \neq 1$, so $s = 0$.

\subsubsection{Exercise 5}
The approach is the same as Exercise 4. Letting $s$ denote the sum, we multiply by $-w^h$ to get that
$-w^h s = (-1)^n - w^h + ... + (-1)^{n - 1} w^{(n - 1)h}$. Thus, $s = 0$ if $n$ is even (even if $w^h = -1$).
Otherwise, we solve $-w^h s = s - 2$ to get that $s = \frac{2}{1 + w^h}$ ($w^h \neq -1$ since $n$ is odd).

\subsection{Analytic Geometry}

\subsubsection{Exercise 1}
We refer back to Exercise 4 from the section ``Conjugation, Absolute Value''. If $z = \alpha + \beta i$, then
\begin{align*}
        (a + b) \alpha + c &= 0 \implies \alpha = -\frac{c}{a + b}\\
        (a - b) \beta &= 0
\end{align*}
If $a - b = 0$, then $z = -\frac{c}{a + b} + ti$ for all $t \in \mathbb{R}$, which is a line.

\subsubsection{Exercise 2}
We have the following:
\begin{itemize}
        \item Ellipse: Given foci $f_1, f_2$, we can write the equation as $\abs{z - f_1} + \abs{z - f_2} = 2a$.
        \item Hyperbola: Given foci $f_1, f_2$, we can write the equation as
                $\abs{\abs{z - f_1} - \abs{z - f_2}} = 2a$.
        \item Parabola is a little trickier, since now we're concerned with distance to a point (focus) as
                well as the distance to a line (directrix). One can compute the minimum distance to the
                directrix using a number of methods (calculus, complex inner product, etc.). Out of laziness
                I'm just going to call the distance to the directrix $\abs{z - d}$. Then we have that the
                equation for a parabola looks like $\abs{z - f} = \abs{z - d}$.
\end{itemize}

\subsection{The Spherical Representation}

\subsubsection{Exercise 1}
If the stereographic projections of $z$ and $z'$ are diametrically opposite, the distance between the
projections is 2. Thus
\begin{align*}
        \frac{\abs{z - z'}^2}{(1 + \abs{z}^2) (1 + \abs{z'}^2} &= 1 \\
        \abs{z}^2 -z\bar{z}' - \bar{z}z' + \abs{z'}^2 &= 1 + \abs{z}^2 + \abs{z'}^2 + \abs{z}^2 \abs{z'}^2 \\
        z\bar{z}' + \bar{z}z' + z\bar{z}' \bar{z}z' &= -1 \\
        (z\bar{z}' + 1)(\bar{z}z' + 1) &= 0 \implies z\bar{z}' = -1
\end{align*}
The other direction follows from reversing the steps above.

\subsubsection{Exercise 2}
Using the fact that the diameter of the sphere is 2, we can solve for the inscribed cube's sidelength by
applying the Pythagorean theorem to the triangle whose hypotenuse joins the bottom left and top right
vertices of the cube. This gives us that $2^2 - 2s^2 = s^2 \implies s = \sqrt{\frac{4}{3}}$. Now if we
label the bottom left corner of the cube as $(x_1, x_2, x_3)$ and the top right corner of the cube as
$(x_1', x_2', x_3')$, we can use the fact that $x_i = -x_i'$ to get that $x_1 = x_2 = x_3 = \sqrt{\frac{1}{3}}$
. Thus, the vertices of the cube are $(\pm \sqrt{\frac{1}{3}}, \pm \sqrt{\frac{1}{3}}, \pm \sqrt{\frac{1}{3}})$
from which it is then straightforward to compute the stereographic projections.


% Copyright 2011-2015 David Hadka.  All Rights Reserved.
%
% This file is part of the MOEA Framework User Manual.
%
% Permission is granted to copy, distribute and/or modify this document under
% the terms of the GNU Free Documentation License, Version 1.3 or any later
% version published by the Free Software Foundation; with the Invariant Section
% being the section entitled "Preface", no Front-Cover Texts, and no Back-Cover
% Texts.  A copy of the license is included in the section entitled "GNU Free
% Documentation License".

\chapter{Representing Decision Variables}
\label{chpt:representations}

In \chptref{chpt:problems} we saw various ways to define new problems using real-valued (floating-point) decision variables.  In addition to floating-point values, the MOEA Framework allows problems to be encoded using integers, bit strings, permutations, programs (expression trees), and grammars.  This chapter details the use of each of these decision variables and their supported variation operators.  This chapter also details the use of the \java{EncodingUtils} class, which provides many helper methods for creating, reading and modifying different types of decision variables.

\section{Floating-Point Values}
Floating-point values, also known as real-valued decision variables, provide a natural way to represent numeric values.  Floating-point decision variables are represented using \java{RealVariable} decision variables.  When creating a new real-valued decision variable, one must specify the lower and upper bounds that the value can represent.

To create real-valued decision variables, use the \java{EncodingUtils.newReal(lowerBound, upperBound)} method.  Note how the lower and upper bounds must be defined.  The example code below demonstrates creating a solution with three different real-valued decision variables.
\begin{lstlisting}[language=Java]
public Solution newSolution() {
    Solution solution = new Solution(3, 2);
    solution.setVariable(0, EncodingUtils.newReal(-1.0, 1.0));
    solution.setVariable(1, EncodingUtils.newReal(0, Math.PI));
    solution.setVariable(2, EncodingUtils.newReal(10.0, 100.0));
    return solution;
}
\end{lstlisting}

Inside the \java{evaluate} method, we can extract the decision variable values from the solution using the \java{EncodingUtils.getReal(...)} method.  Continuing the previous code example, we extract the values of the three decision variables below.
\begin{lstlisting}[language=Java]
public void evaluate(Solution solution) {
    double x = EncodingUtils.getReal(solution.getVariable(0));
    double y = EncodingUtils.getReal(solution.getVariable(1));
    double z = EncodingUtils.getReal(solution.getVariable(2));

    // TODO: evaluate the solution given the values of x, y,
    // and z
}
\end{lstlisting}

Alternatively, if the solution contains exclusively floating-point values, then we can read out all of the variables into an array using a single call.  Note that we pass the entire solution to the \java{EncodingUtils.getReal(...)} method below.
\begin{lstlisting}[language=Java]
public void evaluate(Solution solution) {
    double[] x = EncodingUtils.getReal(solution);

    // TODO: evaluate the solution given the values of x[0],
    // x[1], and x[2]
}
\end{lstlisting}

The \java{EncodingUtils} class handles all the type checking and casting needed to ensure variables are read properly.  Attempting to read or write a decision variable that is not the correct type will result in an \java{IllegalArgumentException}.  If you see this exception, check all your decision variables to ensure they are the types you expect.

\section{Integers}

Integer-valued decision variables can be constructed in a similar way as floating-point values.  For instance, below we construct the solution using calls to \java{EncodingUtils.newInt(lowerBound, upperBound)}.  As we saw with floating-point values, we must specify the lower and upper bounds of the decision variables.
\begin{lstlisting}[language=Java]
public Solution newSolution() {
    Solution solution = new Solution(3, 2);
    solution.setVariable(0, EncodingUtils.newInt(-1, 1));
    solution.setVariable(1, EncodingUtils.newInt(0, 100));
    solution.setVariable(2, EncodingUtils.newInt(-10, 10));
    return solution;
}
\end{lstlisting}

Similarly, the values stored in the decision variables can be read using the \java{EncodingUtils.getInt(...)} method, as demonstrated below.
\begin{lstlisting}[language=Java]
public void evaluate(Solution solution) {
    int x = EncodingUtils.getInt(solution.getVariable(0));
    int y = EncodingUtils.getInt(solution.getVariable(1));
    int z = EncodingUtils.getInt(solution.getVariable(2));

    // TODO: evaluate the solution given the values of x, y,
    // and z
}
\end{lstlisting}

And as we saw with floating-point values, if the solution is exclusively represented by integer-valued decision variables, we can likewise extract all values with a single call to \java{EncodingUtils.getInt(...)}.  Note again that this method is passed the entire solution instead of the individual decision variables as before.
\begin{lstlisting}[language=Java]
public void evaluate(Solution solution) {
    int[] x = EncodingUtils.getInt(solution);

    // TODO: evaluate the solution given the values of x[0],
    // x[1], and x[2]
}
\end{lstlisting}

The integer representation can be used to represent any other kind of discrete value.  For example, suppose we wanted to represent all even numbers between $0$ and $100$.  We can accomplish this using \java{EncodingUtils.newInt(0, 50)} and reading the value with \java{2*EncodingUtils.getInt(variable)}.  Integers are also useful for selecting a single item from a group.  In this scenario, the integer-valued decision variable represents the index of the item in an array.

\begin{tip}
Internally, integers are stored as floating-point values.  This allows the same variation operators to be applied to both real-valued and integer-valued decision variables.  When working with integers, always use the \java{EncodingUtils.newInt(...)} and \java{EncodingUtils.getInt(...)} methods.  This will ensure the internal floating-point representation is correctly converted into an integer.
\end{tip}

\section{Boolean Values}
Boolean values represent simple binary choices, such as ``yes / no'' or ``on / off''.  Use the \java{EncodingUtils.newBoolean()} method to create boolean decision variables, as shown below.  Note also how we can combine multiple decision variable types in a single solution.
\begin{lstlisting}[language=Java]
public Solution newSolution() {
    Solution solution = new Solution(2, 2);
    solution.setVariable(0, EncodingUtils.newBoolean());
    solution.setVariable(1, EncodingUtils.newInt(0, 100));
    return solution;
}
\end{lstlisting}

Boolean values can be read using \java{EncodingUtils.getBoolean(...)}, as demonstrated below.
\begin{lstlisting}[language=Java]
public void evaluate(Solution solution) {
    boolean b = EncodingUtils.getBoolean(
        solution.getVariable(0));
    int x = EncodingUtils.getInt(solution.getVariable(1));

    // TODO: evaluate the solution given the values of b and x
}
\end{lstlisting}

The boolean decision variable works well when the problem has a single choice.  If the problem involves more than one choice, it is more convenient and efficient to use bit strings (an array of booleans) instead.  Bit strings are introduced in the following section.

\section{Bit Strings}
Many problems involve making choices.  For example, the famous knapsack problem involves choosing which items to place in a knapsack to maximize the value of the items carried without exceeding the weight capacity of the knapsack.  If $N$ items are available, we can represent the decision to include each item using a bit string with $N$ bits.  Each bit in the string corresponds to an item, and is set to \java{1} if the item is included and \java{0} if the item is excluded.  For instance, the bit string \java{00110} would place items 3 and 4 inside the knapsack, excluding the rest.

The MOEA Framework supports fixed-length bit strings.  The example code below produces a solution with a single decision variable representing a bit string with length $100$.  Again, note how the entire bit string is stored within a single decision variable.
\begin{lstlisting}[language=Java]
public Solution newSolution() {
    Solution solution = new Solution(1, 2);
    solution.setVariable(0, EncodingUtils.newBinary(100));
    return solution;
}
\end{lstlisting}

When evaluating the solution, the bit string can be read into an array of \java{boolean} values, as demonstrated below.
\begin{lstlisting}[language=Java]
public void evaluate(Solution solution) {
    boolean[] values = EncodingUtils.getBinary(
        solution.getVariable(0));

    //TODO: evaluate the solution given the boolean values
}
\end{lstlisting}

\section{Permutations}
Permutation decision variables appear in many combinatorial and job scheduling problems.  In the famous traveling salesman problem (TSP), a salesman must travel to every city with the condition that they visit each city exactly once.  The order in which the salesman visits each city is conveniently represented as a permutation.  For example, the permutation \java{0,3,1,2} states that the salesman visits the first city first (0 represents the first city), travels to the fourth city (3), then travels to the second city (1), and finally arrives at the third city (2).

The code example below demonstrates the creation of a permutation of 25 elements.
\begin{lstlisting}[language=Java]
public Solution newSolution() {
    Solution solution = new Solution(1, 2);
    solution.setVariable(0, EncodingUtils.newPermutation(25));
    return solution;
}
\end{lstlisting}

The permutation is read out into an array of \java{int} values.  If the permutation is over $N$ elements, the array length will be $N$ and the values stored will range from $0$ to $N-1$.  Each distinct value will appear only once in the array (by definition of a permutation).
\begin{lstlisting}[language=Java]
public void evaluate(Solution solution) {
    int[] permutation = EncodingUtils.getPermutation(
        solution.getVariable(0));

    //TODO: evaluate the solution given the permutation
}
\end{lstlisting}

\section{Programs (Expression Trees)}
The first step towards evolving programs using evolutionary algorithms involves defining the rules for the program (i.e., the syntax and semantics).  The MOEA Framework comes enabled with over $45$ pre-defined program elements for defining constants, variables, arithmetic operators, control structures, functions, etc.  When defining the rules, two important properties should be kept in mind: \emph{closure} and \emph{sufficiency}.

The closure property requires all program element to be able to accept as arguments any value and data type that could possibly be returned by any other function or terminal.  All programs generated or evolved by the MOEA Framework are strongly typed.  No program produced by the MOEA Framework will pass an argument to a function that is an incorrect type.  Furthermore, all functions guard against invalid inputs.  For example, the \java{log} of a negative number is undefined.  Rather then causing an error, the \java{log} method will guard itself and return \java{0.0}.  This allows the rest of the calculation to continue unabated.  With these two behaviors built into the MOEA Framework, the closure property is guaranteed.

The sufficiency property states that the rule set must contain all the necessary functions and terminals necessary to produce a solution to the problem.  Ensuring this property holds is more challenging as it will depend on the problem domain.  For instance, the operators \java{And}, \java{Or} and \java{Not} are sufficient to produce all boolean expressions.  It may not be so obvious in other problem domains which program elements are required to ensure sufficiency.  Additionally, it is often helpful to restrict the rule set to those program elements that are sufficient, thus reducing the search space for the evolutionary algorithm.

Below, we construct a rule set using several arithmetic operators.  One terminal is included, the variable \java{x}.  We will assign this variable later when evaluating the program.  The last setting required is the return type of the program.  In this case, the program will return a number.
\begin{lstlisting}[language=Java]
    //first, establish the rules for the program
		Rules rules = new Rules();
		rules.add(new Add());
		rules.add(new Multiply());
		rules.add(new Subtract());
		rules.add(new Divide());
		rules.add(new Sin());
		rules.add(new Cos());
		rules.add(new Exp());
		rules.add(new Log());
		rules.add(new Get(Number.class, "x"));
		rules.setReturnType(Number.class);
\end{lstlisting}

The second step is constructing the solution used by the evolutionary algorithm.  Here, we define one decision variable that is a program following the rule set we previously defined.
\begin{lstlisting}[language=Java]
public Solution newSolution() {
    Solution solution = new Solution(1, 1);
    solution.setVariable(0, new Program(rules));
    return solution;
}
\end{lstlisting}

Lastly, we evaluate the program.  The program executes inside an environment.  The environment holds all of the variables and other identifiers that the program can access throughout its execution.  Since we previously defined the variable \java{x} (with the \java{Get} node), we want to initialize the value of \java{x} in the environment.  Once the environment is initialized, we evaluate the program.  Since we set the return type to be a number in the rule set, we cast the output from the program's evaluation to a number.
\begin{lstlisting}[language=Java]
public void evaluate(Solution solution) {
    Program program = (Program)solution.getVariable(0);

    // initialize the variables used by the program
    Environment environment = new Environment();
    environment.set("x", 15);

    // evaluate the program
    double result = (Number)program.evaluate(
        environment)).doubleValue();

    // TODO: use the result to set the objective value
}
\end{lstlisting}

\section{Grammars}
Grammars are very similar to programs, but differ slightly in their definition and how the derived programs are generated.  Whereas the program required us to define a set of program elements (the rules) used for constructing the program, the grammar defines these rules using a context free grammar.  The text below shows an example grammar.  The format of this grammar is Backus-Naur form.
\begin{lstlisting}[language=Plaintext]
<expr> ::= <func> | (<expr> <op> <expr>) | <value>
<func> ::= <func-name> ( <expr> )
<func-name> ::= Math.sin | Math.cos | Math.exp | Math.log
<op> ::= + | * | - | /
<value> ::= x
\end{lstlisting}

You should note that this grammar defines the same functions and terminals as the example in the previous section.  This also demonstrates an important difference between programs and grammars in the MOEA Framework.  The grammar explicitly defines where each problem element can appear.  This is in contrast to programs, whose structure is determined by the type system.  As a result, grammars require more setup time but offer more control over programs.  We will now demonstrate the use of grammars in the MOEA Framework.

First, we must parse the context free grammar.  In the example below, the grammar is read from a file.  It is also possible to pass a string containing the grammar using a \java{StringReader} in place of the \java{FileReader}.
\begin{lstlisting}[language=Java]
    ContextFreeGrammar grammar = Parser.load(
        new FileReader("grammar.bnf"));
\end{lstlisting}

Second, we construct the grammar variable that will be evolved by the evolutionary algorithm.  Note how the \java{Grammar} object is passed an integer.  Grammatical evolution uses a novel representation of the decision variable.  Internally, it uses an integer array called a \emph{codon}.  The codon does not define the program itself, but provides instructions for deriving the program using the grammar.  The integer argument to \java{Grammar} specifies the length of the codon.  We defer a detailed explanation of this derivation to the grammatical evolution literature.
\begin{lstlisting}[language=Java]
public Solution newSolution() {
    Solution solution = new Solution(1, 1);
    solution.setVariable(0, new Grammar(10));
    return solution;
}
\end{lstlisting}

Finally, we can evaluate a solution by first extracting the codon and deriving the program.  Unlike programs that can be evaluated directly, the grammar produces a string (the derivation).  While it is common for grammars to produce program code, this is not a requirement.  This is the second major difference between grammars and programs in the MOEA Framework --- the behavior of programs is defined explicitly, whereas the behavior of grammars depend on how the grammar is interpreted.  In this case, we are producing program code and will need a scripting language to evaluate the program.  Using Java's Scripting API and having defined the grammar so that it produces a valid Groovy program, we can evaluate the derivation using the Groovy scripting language.  In the code below, we instantiate a \java{ScriptEngine} for Groovy, initialize the variable \java{x}, and evaluate the program.
\begin{lstlisting}[language=Java]
public void evaluate(Solution solution) {
    int[] codon = ((Grammar)solution.getVariable(0)).toArray();

    // derive the program using the codon
    String program = grammar.build(codon);

    if (program == null) {
        // if null, the codon did not produce a valid grammar
        // TODO: penalize the objective value
    } else {
        ScriptEngineManager sem = new ScriptEngineManager();
        ScriptEngine engine = sem.getEngineByName("groovy");

        // initialize the variables used by the program
        Bindings b = new SimpleBindings();
        b.put("x", 15);

        double result = ((Number)engine.eval(program, b))
            .doubleValue();

        // TODO: use the result to set the objective value
    }
}
\end{lstlisting}

In order to compile and run this example, the Groovy scripting language must be installed.  To install Groovy, download the binary release from \webpage{http://groovy.codehaus.org/}, extract the \file{embeddable\\groovy-all-2.0.1.jar} file into the \folder{lib} folder in your MOEA Framework installation, and add this jar file onto the Java classpath when launching this example.

\section{Variation Operators}
The MOEA Framework contains a number of variation operators (initialization, mutation, crossover, etc.) tailored for each representation type.  This section provides a brief overview of the available operators and details their use.

\subsection{Initialization}
The start of all evolutionary algorithms is the construction of an initial population.  This population is important since, in general, all future solutions are derived from members of this initial population.  Ensuring this initial population provides a diverse and representative set of individuals is paramount.

The floating-point, integer, binary, permutation and grammar variables are all initialized uniformly at random.  This ensures the values, bit strings, etc. are distributed uniformly throughout the search space.

Programs require a slightly more complicated initialization to ensure the initial population contains a diverse sampling of potential programs.  The MOEA Framework provides the ramped half-and-half initialization method, which is one of the most popular initialization techniques for programs.  We refer readers to the genetic programming literature for a detailed description of ramped half-and-half initialization.

\subsection{Variation (Mutation \& Crossover)}
After the initial population is generated, an evolutionary algorithm evolves the population using individual or combinations of variation operators.  Variation operators are classified into two forms: crossover and mutation.  Crossover involves combining two or more parents to create an offspring.  Mutation involves a single parent.  Mutations generally produce only small changes, but this is not mandatory.

\tblref{tbl:operators} lists the supported variation operators in the MOEA Framework.  The table highlights the decision variable representation and type of each variation operator.

\begin{table}
  \centering
  \caption{List of Supported Variation Operators}
  \label{tbl:operators}
  \begin{tabular}{llll}
    \hline
    Representation & Type & Abbr. \\
    \hline
    Real / Integer & Simulated Binary Crossover & SBX \\
    Real / Integer & Polynomial Mutation & PM \\
    Real / Integer & Differential Evolution & DE \\
    Real / Integer & Parent-Centric Crossover & PCX \\
    Real / Integer & Simplex Crossover & SPX \\
    Real / Integer & Unimodal Normal Distribution Crossover & UNDX \\
    Real / Integer & Uniform Mutation & UM \\
    Real / Integer & Adaptive Metropolis & AM \\
    Binary & Half-Uniform Crossover & HUX \\
    Binary & Bit Flip Mutation & BF \\
    Permutation & Partially-Mapped Crossover & PMX \\
    Permutation & Element Insertion & Insertion \\
    Permutation & Element Swap & Swap \\
    Grammar & Single-Point Crossover for Grammars & GX \\
    Grammar & Uniform Mutation for Grammars & GM \\
    Program & Branch (Subtree) Crossover & BX \\
    Program & Point Mutation & PTM \\
    Any & Single-Point Crossover & 1X \\
    Any & Two-Point Crossover & 2X \\
    Any & Uniform Crossover & UX \\
    \hline
  \end{tabular}
\end{table}

The abbreviation column lists the keyword used in the MOEA Framework for referencing each operator.  The example code below shows how we can specify the operator used by an algorithm and also any parameters for the operator.  In this example, we are using parent-centric crossover (PCX) and setting two of its parameters, \java{pcx.eta} and \java{pcx.zeta}.  Refer to the \java{OperatorFactory} class documentation for a complete list of the operators and their parameters.
\begin{lstlisting}[language=Java]
NondominatedPopulation result = new Executor()
    .withProblem("UF1")
    .withAlgorithm("NSGAII")
    .withProperty("operator", "pcx")
    .withProperty("pcx.eta", 0.1)
    .withProperty("pcx.zeta", 0.1)
    .withMaxEvaluations(10000)
    .run();
\end{lstlisting}

It is also possible to combine certain variation operators using the \java{+} symbol.  In the example below, we combine differential evolution with polynomial mutation (\java{de+pm}), and we can set the parameters for both of these operators as shown.
\begin{lstlisting}[language=Java]
NondominatedPopulation result = new Executor()
    .withProblem("UF1")
    .withAlgorithm("NSGAII")
    .withProperty("operator", "de+pm")
    .withProperty("de.rate", 0.5)
    .withProperty("pm.rate", 0.1)
    .withMaxEvaluations(10000)
    .run();
\end{lstlisting}

\begin{important}
Not all combinations of operators are supported.  In general, combining a crossover operator with a mutation operator is ok.  If you request an invalid operator combination, you will see an exception with the message \emph{invalid number of parents}.  See the \java{CompoundVariation} class documentation for more details on what operators can be combined.
\end{important}

\section{Conclusion}
This chapter introduced the various decision variable representations supported by the MOEA Framework.  Look at these different representations as the building blocks for your problem.  If you can construct your problem using these building blocks, the problem will seamlessly integrate with the MOEA Framework.

We close this chapter by commenting that the MOEA Framework is not limited to these representations.  New representations are periodically introduced in the literature.  This fact influenced the design of the MOEA Framework to allow new representations and variation operators.  Interested readers should stay turned for future updates to this user manual that will discuss such extensions in detail.


% $Id$
%
% Author: David Fournier
% Copyright (c) 2008 Regents of the University of California
%

\section{Simultaneous equations models}

For each $t$, $1\le t\le T$ let $y_t$ be an $n$-dimensional vector
and $x_t$ be an $n$-dimensional vector. Let $B$ and~$\Gamma$
be~$(n \times n)$  and~$(n \times m)$ matrices, and suppose that the relationship
$$By_t + \Gamma x_t=u_t$$
holds, where the $u_t$ are $n$-dimensional random vectors of disturbances.
The~$y_t$ are the endogenous variables in the system. The~$x_t$ are
predetermined variables in the sense that they are independent of~$u_t$.
Note that for autoregressive models, the~$x_t$ may contain values
of~$y_j$ for~$j<i$. In general, not all of the coefficients of~$B$
and~$\Gamma$ are estimable. Interesting cases have special structure that
are determined by the particular parameterization of
of~$B$, $\Gamma$, and~$D$. In particular, it is generally assumed
that~$B_{ii}=1$ for~$1 \le i \le n$ and that~$B^{-1}$ exists.

\section{Full information maximum likelihood (\textsc{fiml})}

Assume that for each~$t$, $u_t$ has a multivariate normal distribution
with mean~$0$ and covariance matrix~$D$.
The log-likelihood function for~$B,\Gamma$, and~$D$ is given
by
\begin{equation}
  L(B,\Gamma,D)=T/2\log\big(|B|^2\big)-T/2\log\big(|D|\big)-1/2\sum_{t=1}^T
  [By_t + \Gamma x_t]^\prime D^{-1} [By_t + \Gamma x_t] \label{se:1}
\end{equation}

\section{Concentrating out $D$ for the \textsc{fiml}}

If there are no constraints on $D$, the value of $D$ that maximizes
(\ref{se:1}) can be solved for in terms of the other parameters and
observations. This value, $\hat D$, is given by
\begin{equation}
  \hat D =
  1/T\sum_{t=1}^T [By_t + \Gamma x_t]^\prime [By_t + \Gamma x_t] \label{se:2}
\end{equation}
By
substituting this value into equation~(\ref{se:1}), %xx $(\number\mychapno.1)$
it can be shown that
$$1/2\sum_{t=1}^T
  [By_t + \Gamma x_t]^\prime \hat D^{-1} [By_t + \Gamma x_t] $$
is a constant that can be ignored for the maximization, so equation~(\ref{se:2}), this:
\begin{equation}
  \tilde L(B,\Gamma)=T/2\log \big(|B|^2 \big)-T/2\log \big(|\hat D| \big)\label{se:3}
\end{equation}
plus the \textsc{fiml} estimates for~$B$ and~$\Gamma$ can be found by maximizing
$\tilde L(B,\Gamma)$.

When there are constraints on the parameters of~$D$, then~$\tilde D$ is
no longer the maximum likelihood estimate for~$D$.  So,
 it is necessary to
maximize equation~(\ref{se:1}), which is, in general, a numerically unstable problem.
To successfully carry out the optimization, it is necessary to obtain
reasonable initial estimates for the parameters of~$B$ and~$\Gamma$,
and to use a good method for parameterizing~$D$.
Initial estimates for~$B$ and~$\Gamma$ can be obtained from ordinary
least squares (\textsc{ols}), that is, finding the values
of~$B$ and~$\Gamma$ that minimize
$$\sum_{t=1}^T \|y_t - B^{-1}\Gamma x_t\|^2$$

To parameterize~$D$, note that $\hat D$ is an estimate of~$D$,
so we can parameterize~$D$ by
$$ D = A \hat D A^\prime$$
where~$A$ is a lower triangular matrix. If~$U$ is the Choleski
decomposition of~$D$ and~$\hat U$ is the Choleski decomposition
of~$\hat D$, then $A=\hat U^{-1}U$.
It follows that~$A$ should be close to the identity matrix, which
is a good initial estimate for~$A$.

\section{Evaluating the model's performance}

To evaluate the model's performance, simulated data were generated.
The form of the model~is
%$$\eqalign{
 \begin{align}
   \nonumber y_{t1}+y_{t4}+y_{t5}-2+0.45y_{t-1,1}&=u_{t1}\\
   \nonumber 0.1y_{t1}+y_{t2}+2.0y_{t5}-1-0.6y_{t-1,1}+0.25y_{t-1,2}&=u_{t2}\\
   \nonumber 0.3y_{t1}-0.2y_{t2}+y_{t3}+1&=u_{t3}\\
   \nonumber 1.4y_{t2}-3.1y_{t3}+y_{t4}+1&=u_{t4}\\
   y_{t3}+y_{t4}+y_{t5}&=u_{t5}
 \end{align}
%}
%$$
with a covariance matrix
$$D=\begin{pmatrix}
0.512&0.32&0.256&-1.28&0\cr
0.32&0.328&-0.16&-0.8&0\cr
0.256&-0.16&1.728&0.16&0.8\cr
-1.28&-0.8&0.16&4.8&0.8\cr
0&0&0.8&0.8&0.928
\end{pmatrix}
$$

\bigskip
$$B=\begin{pmatrix}
    1&0&0&1&1\cr
    0.1&1&0&0&2\cr
      0.3&-0.2&1&0&0\cr
      0&1.4&-3.1&1&0\cr
      0&0&1&1&1
  \end{pmatrix}
$$

\bigskip
$$\Gamma=\begin{pmatrix}-2&0.45&0\cr
         -1&-0.6&0.25\cr
          1&0&0\cr
          1&0&0\cr
          0&0&0
   \end{pmatrix}
$$

\bigskip
For this model, $n=5$, $m=3$, and $x_t=(1,y_{t-1,1},y_{t-1,2})$.

The eigenvalues of~$D$ are $(0.00661 0.13583 0.4962 2.21162 5.44573)$.
Having a small eigenvalue tends to produce simulated data that are
difficult to analyze.

Forty time periods of data were generated by the simulator.
The simulated~$y$ values are:
\begin{lstlisting}
 1.63252 3.00223 1.70246 1.34813 -1.12202
 -2.87857 -7.72402 -3.88482 -1.24196 4.71024
 1.12975 -7.92719 -3.85188 -2.33007 4.2984
 1.48112 -2.25692 -1.15585 -2.26166 3.01061
 -2.91887 -5.65015 -2.74198 -0.695815 4.51346
 2.29715 0.524946 0.0268777 1.07624 -0.0898846
 1.32854 6.17993 2.51613 1.67248 -2.39914
 0.5661 -2.53219 -0.966376 0.00820516 1.76543
 0.353591 -3.81146 -2.04431 -1.48574 2.67208
 -2.22887 -2.33436 -1.66284 0.646399 2.26448
 2.29896 -6.42238 -2.41106 -1.70633 3.50028
 0.145878 1.85161 0.646578 -0.380955 0.709761
 -0.779376 -7.60611 -3.79636 -1.63017 4.02658
 -0.107371 -5.61361 -2.35816 -1.77719 3.67762
 0.662221 -5.78832 -2.19632 -2.03071 4.37961
 -0.570661 -7.42505 -3.28544 -2.94125 5.19422
 0.0953742 -1.80617 -1.06915 -0.0320784 2.00018
 -0.406986 -4.96143 -2.8084 -0.948902 3.1811
 1.07219 -7.92608 -2.95484 -3.17022 5.12702
 -0.495144 1.33611 -0.357291 -0.0260083 0.360653
 -0.637878 -8.76117 -3.81638 -1.77116 4.82796
 1.59717 -3.18571 -1.72708 -1.93975 2.79462
 -1.13013 -2.20942 -1.30198 -0.603895 2.29486
 -1.0103 -7.90106 -3.65303 -1.07367 4.66283
 -1.02985 -3.00268 -1.63388 0.309992 2.97876
 0.176882 -7.96282 -3.60299 -1.86289 4.86943
 1.16904 -1.07952 -0.0969977 -0.74563 2.38399
 -0.636119 -2.84841 -1.43676 -0.38474 2.51142
 -1.72929 -5.39866 -2.51289 0.0978131 3.786
 3.56302 3.79343 2.05613 1.43836 -1.2029
 0.15806 -0.863882 -0.302119 -1.19212 1.38518
 1.37323 -1.94413 -0.537631 -0.751294 1.42083
 -0.404075 -8.53817 -3.58618 -3.33976 5.69071
 0.362091 -5.78568 -2.46635 -2.33359 4.21899
 -2.26158 -12.7075 -6.07426 -3.62455 8.49292
 1.20438 -5.44629 -2.30249 -2.02905 3.82742
 1.41463 -1.71734 -0.788698 -1.90306 2.2595
 0.897156 1.28039 0.693579 0.318737 0.385857
 -0.0330384 -1.55642 -0.189474 0.312385 1.57168
 1.5747 0.827181 1.26032 0.813312 0.270432
\end{lstlisting}

For the~$x$ values, the first time periods data $x_0 =(1,1,2)$
were supplied.  The simulated~$x$ values are:
\begin{lstlisting}
 1 1 2
 1 1.63252 3.00223
 1 -2.87857 -7.72402
 1 1.12975 -7.92719
 1 1.48112 -2.25692
 1 -2.91887 -5.65015
 1 2.29715 0.524946
 1 1.32854 6.17993
 1 0.5661 -2.53219
 1 0.353591 -3.81146
 1 -2.22887 -2.33436
 1 2.29896 -6.42238
 1 0.145878 1.85161
 1 -0.779376 -7.60611
 1 -0.107371 -5.61361
 1 0.662221 -5.78832
 1 -0.570661 -7.42505
 1 0.0953742 -1.80617
 1 -0.406986 -4.96143
 1 1.07219 -7.92608
 1 -0.495144 1.33611
 1 -0.637878 -8.76117
 1 1.59717 -3.18571
 1 -1.13013 -2.20942
 1 -1.0103 -7.90106
 1 -1.02985 -3.00268
 1 0.176882 -7.96282
 1 1.16904 -1.07952
 1 -0.636119 -2.84841
 1 -1.72929 -5.39866
 1 3.56302 3.79343
 1 0.15806 -0.863882
 1 1.37323 -1.94413
 1 -0.404075 -8.53817
 1 0.362091 -5.78568
 1 -2.26158 -12.7075
 1 1.20438 -5.44629
 1 1.41463 -1.71734
 1 0.897156 1.28039
 1 -0.0330384 -1.55642
\end{lstlisting}

\section{Results of (\textsc{fiml}) for unconstrained $D$}

For the estimation process, all the elements of the matrices
$B$ and $\Gamma$ with value $0$ were fixed at their correct value.
The \textsc{fiml} estimates for unconstrained covariance matrix $D$ are given
below.

$$B=\begin{pmatrix}
1&0&0&0.364395&0.364395\cr
0.238411&1&0&0&1.37593\cr
0.042875&-0.330484&1&0&0\cr
0&1.93026&-4.90367&1&0\cr
0&0&1.06339&1.09851&1
 \end{pmatrix}
$$

\bigskip
$$\Gamma=\begin{pmatrix}
-0.917978&0.431377&0\cr
0.473915&-0.491422&0.19981\cr
0.441089&0&0\cr
-0.126701&0&0\cr
0&0&0
\end{pmatrix}
$$

\bigskip
$$D=\begin{pmatrix}
0.938236&0.843743&0.506127&-1.38383&0.314262\cr
0.843743&1.55844&0.732235&-0.591771&1.37195\cr
0.506127&0.732235&0.430091&-0.805866&0.62244\cr
-1.38383&-0.591771&-0.805866&4.18591&-0.0363513\cr
0.314262&1.37195&0.62244&-0.0363513&1.75939
\end{pmatrix}
$$

\section{Results of (\textsc{fiml}) for constrained $D$}

Since $D_{51}=0$ and $D_{52}=0$, these values were not well
estimated by the unconstrained \textsc{fiml} procedure. Suppose that
we know that their values should be $0$ and that the
value of $D_{55}\le1.0$. We incorporate this knowledge
into the model by using penalty functions.

$$B=\begin{pmatrix}
1&0&0&0.594218&0.594218\cr
-0.321482&1&0&0&1.97809\cr
0.0416175&-0.365572&1&0&0\cr
0&2.48468&-6.10763&1&0\cr
0&0&0.968057&1.02738&1
\end{pmatrix}
$$

\bigskip
$$\Gamma=\begin{pmatrix}
-1.3418&0.448342&0\cr
-1.14717&-0.593754&0.183788\cr
0.372114&0&0\cr
-0.0640792&0&0\cr
0&0&0
\end{pmatrix}
$$

\bigskip
$$D=\begin{pmatrix}
0.593424&-0.325467&0.298836&-1.43231&0.00401209\cr
-0.325467&0.650371&-0.234747&0.441841&4.48456\e{-06}\cr
0.298836&-0.234747&0.258147&-0.900105&0.255262\cr
-1.43231&0.441841&-0.900105&6.13619&0.180376\cr
0.00401209&4.48456\e{-06}&0.255262&0.180376&1.00262
\end{pmatrix}
$$

\section{Code for (\textsc{fiml}) for constrained $D$}

This is the code in the \textsc{tpl} file, split up by sections and
commented on. The \texttt{DATA\_SECTION} defines the data and some size
aspects of the model structure. Objects that are prefixed by \texttt{init\_}
will be read in from the data file.
\begin{lstlisting}
// This version incorporates constraints via penalty functions.
//This is sample code to determine the parameters of a
// simultaneous equations model. The notation follows
// that of Hamilton, Times Series Analysis, chapter 9.
// the general form of the model is
//
//    By_t + Gamma x_t =u_t
//
//for t=1,...,T. The u_t have covariance matrix D.

DATA_SECTION
  init_int T // the number of observations
  init_int dimy // dimension of the vector of
                // endogenous variables
  init_int dimx // dimension of the vector of
                // predetermined variables
  init_int num_Bpar // the number of parameters in
                // the elements of B to be estimated
  init_int num_Gpar // the number of parameters in
                // the elements of Gamma to be estimated
  init_matrix y(1,T,1,dimy) // the y_t
  init_matrix x(1,T,1,dimx) // the x_t
  int dimy1
 !! dimy1=dimy*(dimy+1)/2;  // size of symmetric matrix
\end{lstlisting}

The \texttt{PARAMETER\_SECTION} describes the model's parameters.
Objects which are prefixed by \texttt{init\_} are the independent
variables of the model.  For example, \texttt{Bpar} is used to
parameterize the nonzero elements of \texttt{B}.
\texttt{ch\_Dpar} is used to parameterize the lower triangular matrix
of the correction from \texttt{emp\_D} to the covariance matrix
\texttt{D}. The minimization is done in a number of phases.
The parameter \texttt{kx} is used to have a parameter that becomes active
in phase~4, so that the minimization will take place in four stages.
This parameter does not enter into the ``real'' part of the model.

\begin{lstlisting}
PARAMETER_SECTION
  init_vector Bpar(1,num_Bpar)
  init_vector Gpar(1,num_Gpar)
  init_vector ch_Dpar(1,dimy1,2)
  matrix B(1,dimy,1,dimy)
  matrix D(1,dimy,1,dimy)  // the covariance matrix for the
                           // disturbances u_t
  matrix emp_D(1,dimy,1,dimy)  // the covariance matrix for the
  matrix Gamma(1,dimy,1,dimx)
  matrix ch_D(1,dimy,1,dimy)
  matrix z(1,T,1,dimy);
  objective_function_value f
  init_number kx(4);
\end{lstlisting}

The \texttt{PROCEDURE\_SECTION} is where the model's calculations
are carried out. It is split up into a set of functions where the
model-specific pieces of code (different code for different models)
are located. Finally, the optimization for parameter estimation is
calculated. This depends on the phase of the optimization procedure.
A \texttt{switch} statement is used to vary the form of the objective
function depending on the phase. The function \texttt{current\_phase()}
returns the number of the current phase of the optimization.
The function \texttt{last\_phase()}
returns~1 (``true'') if the current phase is the last phase of the
optimization. Quadratic penalty functions are put on the model's parameters, and these penalty weights are decreased in subsequent phases.
This procedure helps to stabilize the optimization when several
model parameters are highly correlated.

\begin{lstlisting}
PROCEDURE_SECTION
  fill_B();    // this will vary from model to model
  fill_Gamma();  // this will vary from model to model

  calculate_empirical_copvariance_matrix();

  fill_D();  // this will vary from model to model

  calculate_constraints();  // this will vary from model to model

  int sgn;
  switch (current_phase())
  {
  case 1:
    {
      f+=0.1*norm2(Bpar);
      f+=0.1*norm2(Gpar);
      f+=0.1*norm2(ch_Dpar);
      dvar_matrix Binv=inv(B);
      for (int t=1;t<=T;t++)
      {
        dvar_vector z=y(t)+Binv*Gamma*x(t);
        f+=z*z;
      }
      break;
    }
  default:
    {
      f+= -0.5*T*log(square(det(B)))
       +0.5*T*ln_det(D,sgn);

      dvar_matrix Dinv=inv(D);
      dvariable f1=0.0;
      for (int t=1;t<=T;t++)
      {
        dvar_vector z=B*y(t)+Gamma*x(t);
        f1+=z*(Dinv*z);
      }
      f+=0.5*f1;
      if (!last_phase())
      {
        f+=0.1*norm2(Bpar);
        f+=0.1*norm2(Gpar);
        f+=0.1*norm2(ch_Dpar);
      }
      else
      {
        f+=0.001*norm2(Bpar);
        f+=0.001*norm2(Gpar);
        f+=0.001*norm2(ch_Dpar);
      }
    }
  }

  f+=square(kx);

FUNCTION fill_B
  B.initialize();
  for (int i=1;i<=dimy;i++)
    B(i,i)=1.0;

  // this is part of the special structure of the model
  int ii=1;
  B(2,1)=Bpar(1);
  B(3,1)=Bpar(2);

  B(3,2)=Bpar(3);
  B(4,2)=Bpar(4);

  B(4,3)=Bpar(5);
  B(5,3)=Bpar(6);

  B(5,4)=Bpar(7);
  B(1,4)=Bpar(8);

  B(1,5)=Bpar(8);
  B(2,5)=Bpar(9);

FUNCTION fill_Gamma
  Gamma.initialize();

  // this is the part of special structure of the model
  Gamma(1,1)=Gpar(1);
  Gamma(2,1)=Gpar(2);
  Gamma(3,1)=Gpar(3);
  Gamma(4,1)=Gpar(4);

  Gamma(1,2)=Gpar(5);
  Gamma(2,2)=Gpar(6);

  Gamma(2,3)=Gpar(7);

FUNCTION fill_D

  ch_D.initialize();
  // this is the special structure of the model
  int ii=1;
  for (int i=1;i<=dimy;i++)
  {
    for (int j=1;j<=i;j++)
      ch_D(i,j)=ch_Dpar(ii++);
    ch_D(i,i)+=1;
  }

  D=ch_D*emp_D*trans(ch_D); // so Ch_D is the Choleski
                      // decomposition of D
FUNCTION calculate_empirical_covariance_matrix

  for (int t=1;t<=T;t++)
    z(t)=B*y(t)+Gamma*x(t);

  emp_D=empirical_covariance(z);

FUNCTION  calculate_constraints

  double wt=1.0;
  switch (current_phase())
  {
  case 1:
    wt=1.0;
    break;
  case 2:
    wt=10.0;
    break;
  case 3:
    wt=100.0;
    break;
  default:
    wt=1000.0;
    break;
  }
  if (D(5,5)>1.0)
    f+=wt*square(D(5,5)-1.00);

  f+=wt*square(D(5,1));
  f+=wt*square(D(5,2));
\end{lstlisting}

The \texttt{REPORT\_SECTION} prints out a report of the model's results.

\begin{lstlisting}
REPORT_SECTION
  report << "B" << endl;
  report << B << endl;
  report << "Gamma" << endl;
  report << Gamma << endl;
  report << "D" << endl;
  report << D << endl;
  report << "eigenvalues of D" << endl;
  report << eigenvalues(D) << endl;
  report << "y" << endl;
  report << y << endl;
  report << "x" << endl;
  report << x << endl;
\end{lstlisting}


\chapter*{Nomenclature}

\section*{Units}
All units of measurement throughout this thesis conform to the \emph{Syst\`{e}me Internationale}, with deviations from this rule noted where appropriate.

\section*{Notation}
This section describes the general form of notation for properties such as scalars, vectors and matrices and their derivatives.

\subsection*{Time Derivatives}
\begin{tabular}{@{}l l}
$\dot{x}$		& first derivative of $x$ with respect to time \\
$\ddot{x}$		& second derivative of $x$ with respect to time \\
$x^{(n)}$		& $n$th derivative of $x$ with respect to time \\
\end{tabular}

\subsection*{Scalars, Vectors and Matrices}
\begin{tabular}{@{}l l}
$x$				& scalar \\
$\vc{x}$		& vector or matrix \\
$\vc{x}^T$		& transpose of vector or matrix \\
$x_i$			& $i$th element of vector $\vc{x}$ \\
$f(x)$			& function of scalar $x$ \\
$f(\vc{x})$		& function of vector or matrix $\vc{x}$ \\
$f_{\vc{x}}$		& Jacobian of $f(\vc{x})$ with respect to $\vc{x}$ \\
\end{tabular}

\section*{Symbols}
The following symbols are used throughout this thesis.  Where a symbol is used only briefly, it is defined at the appropriate point in the text.

\subsection*{Latin}
\begin{tabular}{@{}l l}
	$ C_d $	& aerodynamic drag coefficient [--] \\
	$ F $, $ \vc{F} $	& force [\si{\newton}] \\
	$ g $	& acceleration due to gravity [\si{\metre\per\square\second}] \\
	$ i $	& current [\si{\ampere}] \\
	$ L $, $ M $, $ N $ & rotational forces [\si{\newton\metre}] \\
	$ \vc{M} $	& moment [\si{\newton\metre}] \\
	$ m $	& mass [\si{\kilogram}] \\
	$ R $	& resistance [\si{\ohm}] \\
	$ u $, $ v $, $ w $	& surge, sway and heave velocities [\si{\metre\per\second}] \\
	$ V $	& magnitude of velocity [\si{\metre\per\second}] \\
	$ V_a $	& voltage applied to circuit [\si{\volt}] \\
	$ X $, $ Y $, $ Z $	& linear forces [\si{\newton}] \\
	$x$, $y$, $z$ 		& components of position [\si{\metre}] \\
\end{tabular}

\subsection*{Greek}
\begin{tabular}{@{}l l}
	$ \beta $	& slip angle [\si{\radian}] \\
	$\phi$, $\theta$, $\psi$	& roll, pitch and yaw displacements [\si{\radian}] \\
	$ \rho $	& atmospheric density [\si{\kilogram\per\cubic\metre}] \\
	$ \sigma $	& friction coefficient [--] \\
	$ \omega $	& rotational speed [\si{\radian\per\second}] \\
	$ \vc{\omega} $	& angular velocity vector [\si{\radian\per\second}]
\end{tabular}

\subsection*{Subscripts}
\begin{tabular}{@{}l l}
	$ k $	& iteration of inverse simulation \\
	$ n $	& iteration of Newton-Raphson method \\
\end{tabular}

\section*{Abbreviations}
\begin{tabular}{@{}l l}
FDIR	& fault detection, isolation and reconfiguration \\
InvSim	& inverse simulation \\
NED		& north-east-down \\
\end{tabular}


\begin{frame}

	\frametitle{Empirical evaluation}
	\framesubtitle{Evaluation Metrics}

	\begin{itemize}
		\item<only@1>  $\mathit{Precision} =$ $ \mathit{\frac{\#\ of\ correct\ predictions}{\#\ of\ total\ predictions}}$. The fraction of the predictions that are correct.
		\item<only@1> $\mathit{Spread}\ $=$\ end(I) -start(I)$. The width of the prediction interval $I$, which represents the number of events between the start and the end of $I$.

		\item<only@1> $\mathit{PS-}$score = $\alpha * precision + (1 - \alpha ) * ( 1- \frac{spread}{max\  spread})$. In analogy with the $F-score$ we introduce this new score to assess the performance of the system in terms of both the $\mathit{precision}$ and the $\mathit{spread}$ combined.
		\item<only@1> $\mathit{Distance}\ $=$\ start(I) - now$. The distance between the start of the prediction interval $I$ and the time of prediction is produced (now).
		\item<only@2>  $\mathit{Cumulative\ communication}$. The number of messages, which are required to perform the distributed online learning modes to synchronize the prediction models.
		\item<only@2> $Throughput$. The number of events processed per unit time (second).
	\end{itemize}

\end{frame}

\begin{frame}

	\frametitle{Empirical evaluation}
	\framesubtitle{Synchronization Schemes}

	Our proposed system can operate in three different modes of synchronization schemes:
	\begin{itemize}
		\item continuous full synchronization for each incoming event (hypothetical).
		\item static scheme based on synchronizing the prediction models periodically every $b$ of input events in each stream.
		\item dynamic synchronization protocol based on making the predictors communicate their local prediction models periodically but only under condition that the divergence of the local models from a reference model exceeds a variance threshold $\Delta$ (recommended).

	\end{itemize}

\end{frame}

\subsection{Real-word Event Streams\\}

\begin{frame}

	\frametitle{Empirical evaluation }
	\framesubtitle{Over Real-word Event Streams}
\begin{itemize}

	\item<only@1> Event streams describe critical points (i.e., synopses) of moving vessels trajectories, which are derived from raw AIS messages as described in \cite{synopses1}.
	\item<only@1> $4,684,444$ critical points  in Brest, France: October 2015 to March 2016 ($\approx5000$ vessels).
	\item<only@1>  $\mathcal{P}_1=Sailing$ with $m=2$ over $\Sigma_1=$$\{$\textit{VerySlow, Slow, Moving,  Sailing, Stopping}$\}$.
	\item<only@2> $\mathcal{P}_2=$\textit{changeInHeading; gapStart; gapEnd; changeInHeading} with $m=1$ over $\Sigma_2=$
	$\{$\textit{stopStart, stopEnd, changeInSpeedStart, changeInSpeedEnd,  slowMotionStart, slowMotionEnd, gapStart, gapEnd, changeInHeading}$\}$.
	\item<only@2> All experiments are performed with setting the batch size of 100  ($b=100$), the variance threshold of 2 ($\Delta=2$), $80\%$ as PMC prediction threshold ($\theta_{p}=80\%$), and 200 for the maximum spread ($\theta_{s}=200$)
\end{itemize}

\end{frame}

\begin{frame}

	\frametitle{Results on Real-word Event Streams }
	\framesubtitle{Precision scores with respect to the number of input events over time for $\mathcal{P}_1$.}

	\begin{figure}[H]
		\centering
		\includegraphics[width=.7\textwidth,height=.65\textheight]{../chapters/figures/synopses/new/precision_p1.png}\linebreak\\
	Methods of distributed learning outperform the isolated method.

	\end{figure}

\end{frame}

\begin{frame}

	\frametitle{Results on Real-word Event Streams }
	\framesubtitle{Average spread for $\mathcal{P}_1$.}

	\begin{center}

		\begin{figure}[]
			\centering
			\includegraphics[width=.7\textwidth,height=.65\textheight]{../chapters/figures/synopses/new/spread_p1.png}

		\end{figure}
	\end{center}

\end{frame}

\begin{frame}

	\frametitle{Results on Real-word Event Streams }
	\framesubtitle{$PS$-score for $\mathcal{P}_1$ with $\alpha = .5$.}

\begin{center}
	\centering
	\begin{figure}[]

		\includegraphics[width=.7\textwidth,height=.65\textheight]{../chapters/figures/synopses/new/ps_score_p1.png}

	\end{figure}
\end{center}
\end{frame}

\begin{frame}

	\frametitle{Results on Real-word Event Streams }
	\framesubtitle{Distance for $\mathcal{P}_1$.}

	\begin{center}
		\centering
		\begin{figure}[]

			\includegraphics[width=.9\textwidth,height=.8\textheight]{../chapters/figures/synopses/new/distance_p1.png}

		\end{figure}
	\end{center}

\end{frame}

\begin{frame}

	\frametitle{Results on Real-word Event Streams }
	\framesubtitle{Cumulative communication with respect to the number of input events over time for $\mathcal{P}_1$.}

	\begin{center}

		\begin{figure}[]
			\centering
			\includegraphics[width=.9\textwidth,height=.8\textheight]{../chapters/figures/synopses/new/messages_p1.png}

		\end{figure}
	\end{center}

\end{frame}

\begin{frame}

	\frametitle{Results on Real-word Event Streams }
	\framesubtitle{Precision scores of $\mathcal{P}_2$  for vessels of \textit{pleasure craft} type.}

\begin{figure}[]
	\centering
	\includegraphics[width=.9\textwidth,height=.8\textheight]{../chapters/figures/synopses/new/precision_p2.png}

\end{figure}

\end{frame}

\subsection{Synthetic Event Streams}
\begin{frame}

	\frametitle{Empirical evaluation }
	\framesubtitle{Over Synthetic Event Streams}
	\begin{itemize}
		\item<1-> We generate $20$ streams of size 10,000 events from a simulated 1-order Markov process over $\Sigma=\{a, b, c, d\}$.

		\item<1->The used pattern $\mathcal{P}=a ; d ; c$ with $m=1$.

		\item<1-> We set the batch size to 20 ($b=20$), the variance threshold to .0001 ($\Delta=.0001$), the  \pmcmr prediction threshold to $50\%$ ($\theta_{p}=50\%$), and the maximum spread to 10 ($\theta_{s}=10$).
	\end{itemize}

\end{frame}

\begin{frame}

	\frametitle{Results on Synthetic Event Streams }
	\framesubtitle{Precision scores with respect to the number of input events over time for $\mathcal{P}=a;d;c$.}

	\begin{figure}[H]
		\centering
		\includegraphics[width=.7\textwidth,height=.6\textheight]{../chapters/figures/synthetic/new/precision_synthetic.png}\linebreak\\
		Methods of distributed learning converge faster than the isolated method.

	\end{figure}

\end{frame}

\begin{frame}

	\frametitle{Results on Synthetic Event Streams }
	\framesubtitle{Average spread  with respect to the number of input events over time for $\mathcal{P}=a;d;c$.}

\begin{figure}[]
	\centering
	\includegraphics[width=.9\textwidth,height=.8\textheight]{../chapters/figures/synthetic/new/spread_synthetic.png}

\end{figure}

\end{frame}

\begin{frame}

	\frametitle{Results on Synthetic Event Streams }
	\framesubtitle{$\mathit{PS}$-score for $\mathcal{P}=a;d;c$ with $\alpha = .5$.}

\begin{figure}[H]
	\centering
	\includegraphics[width=.9\textwidth,height=.8\textheight]{../chapters/figures/synthetic/new/ps_score_synthetic.png}

\end{figure}

\end{frame}

\begin{frame}

	\frametitle{Results on Synthetic Event Streams }
	\framesubtitle{The error  $\sum_{i,j} |\hat{p}_{i,j} - {p}_{i,j}|$) of estimating the transition probabilities  for $\mathcal{P}=a;d;c$.}

	\begin{figure}[H]
		\centering
		\includegraphics[width=.9\textwidth,height=.8\textheight]{../chapters/figures/synthetic/new/error_synthetic.png}

	\end{figure}

\end{frame}

\begin{frame}

	\frametitle{Throughput Results}
	\framesubtitle{Throughput of the system on YARN cluster in in terms of number of events processed per second with respect to the parallelism level over $\mathcal{P}_1$ with batch size $b=100$ and divergence threshold $\Delta=2$.}

	\begin{figure}[H]

		\includegraphics[width=.8\textwidth,height=.75\textheight]{../chapters/figures/throughput/temp.png}

	\end{figure}

\end{frame}


\subsection{y* cut Optimization}
\label{section: y* cut optimization}

\todo[inline]{Logical flow... Q/G selection optimisation, then y* optimisation....\\ Clean up formulae. }

In QCD two-to-two scattering, $t$-channel is dominant. The QCD dijet production is proportional to $(1-cos\theta^{*})^{-2}$, while $H'$ and $String$ production is expected to be flat in $cos\theta^{*}$.
This means that $y^{*}$($y^{*} = \frac{y_{1}-y_{2}}{2}$) of QCD background will minimize at 0 and that of $H'$ and $String$ will peak at 0.

Figure \ref{fig: hprime significance as a function of y* cut} shows the significance of $H'$($Significance = \sqrt{\sum_{i}{2\cdot((S_{i}+B_{i})\cdot ln(1+\frac{S_{i}}{B_{i}})-S_{i})}}$) for the $m_{jj}$ distribution as a function of $y^{*}$ cut. The significance peaks at around 0.6, so the optimal $y^{*}$ cut for the $H'$ search is $|y^{*}| < 0.6$.

Figure \ref{fig: string significance as a function of y* cut} shows the significance of $String$ for the $m_{jj}$ distribution as a function of $y^{*}$ cut. The significance peaks at around 0.8, so the optimal $y^{*}$ cut for the $String$ search is $|y^{*}| < 0.8$.

\begin{figure}[htbp]
        \centering
        \subfigure[$\geq$1 g-tag]{\includegraphics[width=0.48\columnwidth]{figures/yStarOptimization/Significance_Hprime_gj.pdf}}
        \subfigure[2 g-tag]{\includegraphics[width=0.48\columnwidth]{figures/yStarOptimization/Significance_Hprime_gg.pdf}}
        \caption{$H'$ significance as a function y* cut in the case of (a) $\geq$1 g-tag, (b) 2 g-tag.}
        \label{fig: hprime significance as a function of y* cut}
\end{figure}

\begin{figure}[htbp]
        \centering
        \subfigure[$\geq$1 g-tag]{\includegraphics[width=0.48\columnwidth]{figures/yStarOptimization/Significance_String_gj.pdf}}
        \subfigure[2 g-tag]{\includegraphics[width=0.48\columnwidth]{figures/yStarOptimization/Significance_String_gg.pdf}}
        \caption{$String$ significance as a function y* cut in the case of (a) $\geq$1 g-tag, (b) 2 g-tag.}
        \label{fig: string significance as a function of y* cut}
\end{figure}


\section{Conclusion}

This paper broadly follows three parts: First a structural causal model defining the data generating process of the returns of stocks was formulated. Using daily returns for 11 stock for the last 30 years, relevant statistics was calculated, and used for tuning the parameters of the DGP. Using the structural causal model a data set was simulated. 2) An ensemble of three algorithms that predicts the Sharpe ratio of individual stocks was presented. Using these three models, I tune, train and select an algorithm on the simulated data set. I find the best performing algorithm being a model that estimates the sharp ratio for individual stocks by storing a rolling set of observations and calculating the relevant moments using this memory. 3) Using the tuned algorithm we apply it out-of-sample to the real data. Using the rolling Sharpe algorithm I find (comparing it to 4) other benchmarks, the algorithm performs extremely well, having a Sharpe ratio at $0.1256$ comparing to a tangency portfolio with a Sharpe ratio of $0.05545$. I discuss the different challenges of tuning an LSTM, and the value of using a structural causal model as test environment for trading algorithms. Finally I point out the challenges of using machine learning models in-sample, and express a word of caution of doing so.


The aim of this Section is to explain: (i) how to initialize the hypercubes set at the beginning of the analysis, (ii) how to select the interval widths in the hypercubes, and (iii) how to improve the precision of the analysis by using offsets to identify sub-volumes inside the hypercubes.

\subsection{Initialization of the analysis} \label{sec:initialization}
Before starting the analysis we have to determine the hypercubes dimensionality, i.e., the number of sides each hypercube will have. To do this, we must find all the variables of the code which are not constants. %To avoid scanning the entire code before starting the analysis,
We require the program to initialize all non-constant variables at the beginning of the program. Note that physics simulations, like our case study, satisfy this requirement because they are made up by an initialization of all variables, followed by a \statement{while} loop which contains the core of the program (i.e., the update of the simulated world). Otherwise, we can consider a dummy initialization (i.e., $0.0$) for all variables which are not initialized at the beginning of the program. The actual initialization of the variables will be treated as a normal assignment, without any loss of precision.
The initialization of the analysis is made in two steps. First, for each initialized variable, we compute its abstraction in the non-relational domain chosen to represent the single variables. The resulting set of abstract values could contain more than one element. Let us call $\alpha(V)$ the set of abstract values associated to the initialization of the variable $V \in Vars$. Then we compute the Cartesian product of all sets of abstracted values (one for each variable). The resulting set of tuples (where each tuple has the same cardinality as $Vars$) is the initial set of hypercubes of the analysis.

Formally, $\adomain = \sbigtimes_{V \in Vars} {\alpha(V)}$.

As an example, consider the initialization in Listing \ref{lst:casestudy} of our case study. First of all, we must identify the variables which are not constants: $dt,g,k$ are assigned only once (during the initialization), so we do not include them in $Vars$. The set of not-constant variables is then $Vars = \{ V_1 = px, V_2 = py, V_3 = vx, V_4 = vy \}$. The cardinality of the hypercubes will be $|Vars| = 4$. Suppose that the widths associated to the variables are  $w_1 = 10.0, w_2 = 25.0, w_3 = 30.0, w_4 = 5.0$. Then, the abstraction of each variable is $\alpha(V_1) = \{ 0 \}$, $\alpha(V_2) = \{ 0, 1 \}$, $\alpha(V_3) = \{ 0, 1 \}$, and $\alpha(V_4) = \{ -6 \}$.
The Cartesian product of these abstractions brings us to the following initial set of hypercubes:
$$\adomain = \{ (0,0,0,-6) , (0,0,1,-6) , (0,1,0,-6) , (0,1,1,-6) \}$$

\subsection{Width choice} \label{sec:widths}
The choice of the interval widths is quite important, because it influences both the precision and efficiency of the analysis. The widths influence the granularity of the space partitioning with respect to each variable. On the one hand, if we use smaller widths we certainly obtain more precision, but the analysis risks to be too slow. On the other hand, with bigger widths the analysis will be surely faster, but we could not be able to verify the desired property.

The width selection can be automatized. We implemented a recursive algorithm which adjusts the widths automatically, starting with bigger ones and then decreasing them only in the portions of space where it is really needed, to avoid compromising the performance.

We start with wide intervals (i.e., coarse precision, but fast results) and we run the analysis for the first time. We track, for each hypercube of the final abstract state, the initial hypercubes (\textit{origins}) from which it is derived.

At the end of the analysis, we check, for each hypercube of the final set, if the desired property is verified. We associate to each origin its final result by merging the results of its derived final hypercubes: some origins will certainly verify the property (i.e., they produce only final hypercubes which satisfy the property), some will not, and some will not be able to give us a definite answer. We can partition the starting hypercubes set with respect to this criterion.

We run the analysis again, but \emph{only} on the origins which did not give a definite answer. To obtain more precise results in this specific space portion, the analysis is now run with halved widths. Note that this step is only performed until we reach a specific threshold, i.e., the \textit{minimum width} allowed for the analysis. This parameter can be specified by the user (together with the starting widths). The smaller this threshold is, the more precise (but slower) the analysis becomes.

At the end of this recursive process, we obtain three partitions of the variable space: a set of starting hypercubes which certainly verify the property (\emph{yes} set), a set of starting hypercubes which certainly do not verify the property (\emph{no} set), and a set of starting hypercubes which, at the minimum width allowed for the analysis, still do not give a definite answer (\emph{maybe} set). This means that the analysis tells us which initial values of the variables bring us to verify the property and which do not. Thanks to these results, the user can modify the initial values of the program, and run the analysis again, until the answer is that the property is verified for all initial values. In our case study, for example, we can adjust the possible initial positions and velocities until we are sure that the ball will exit the screen in a certain time frame.

The formalization of this recursive algorithm is presented in Algorithm \ref{alg:widthAdjusting}.

\begin{algorithm}
\caption{The width adjusting recursive algorithm} \label{alg:widthAdjusting}
\begin{algorithmic}
\Function {Analysis} {$currWidth, minWidth, startingHypercubes$}
  \State \Return $(yes \cup yes', no \cup no', maybe')$
  \State \textbf{where}
    \State $(yes, no, maybe) = hypercubesAnalysis(currWidth, startingHypercubes )$
    \If {$currWidth/2.0 \geq minWidth$}
      \State $(yes', no', maybe') = Analysis(currWidth/2.0, minWidth, maybe)$
    \Else
      \State $(yes', no', maybe') = (Set.empty, Set.empty, maybe)$
    \EndIf
\EndFunction
\end{algorithmic}
\end{algorithm}

The overall analysis takes as input the starting width, the minimum width allowed and the set of starting hypercubes (obtained from the initialization of the program as described in Section \ref{sec:initialization}). It executes the analysis on such data with the function \statement{hypercubesAnalysis}, which returns three sets of hypercubes (\statement{yes,no,maybe}) with the meaning explained above. Then, if the halved width is still greater than the minimum one allowed, the algorithm performs a recursive step by repeating the analysis function only on the \statement{maybe} hypercubes set (with halved width).

Note that %the initial hypercubes sets
the three final hypercubes sets (the \emph{yes,no,maybe} partitions)
will contain hypercubes of different sizes: this happens because each hypercube can come from a different iteration of the analysis, and each iteration is associated to a specific hypercube size. A certain portion of the variable space could give a definite answer even at coarse precision (for example, when the horizontal velocity of the ball is sufficiently high, the values of other variables do not matter so much), while another portion could need to be split in much smaller hypercubes to give interesting results.

\subsection{Offsets}
There is still space for improving accuracy in the domain introduced so far. A loss of precision may occur due to the fact that hypercubes proliferate too much, even using small widths. Consider, for example, the statement \statement{x = x + 0.01} (which is repeated at each iteration of the \statement{while} loop) with $1.0$ as the width associated to \statement{x}. If the initial interval associated to \statement{x} was $[0.0,1.0]$, after the first iteration we would obtain two intervals ($[0.0,1.0]$ and $[1.0,2.0]$) because the resulting interval would be $[0.01,1.01]$, which spans over two fixed-width intervals. For the same reason, after the second iteration we would obtain three intervals ($[0.0,1.0],[1.0,2.0]$ and $[2.0,3.0]$) and so on: at each iteration we add one interval.

In order to overcome these situations, we may further improve the definition of our domain: each hypercube is augmented with additional information, which limits the admissible values inside it. In particular, each variable $v_i$ (associated to width $w_i$) is related to (other than an integer index $i$ representing the fixed-width interval $[i \times w_i, (i+1) \times w_i]$) a specific offset $(o_m,o_M)$ \emph{inside} such interval. In this way, we use a sub-interval (of arbitrary width) inside the fixed-interval width, thereby restricting the possible values that the variable can assume. Both $o_m$ and $o_M$ must be smaller than $w_i$, greater than or equal to $0$ and $o_m \leq o_M$. Then, if $i$ and $(o_m,o_M)$ are associated to $v_i$, this means that the possible values of $v_i$ belong to the interval $[(i \times w_i) + o_m, (i \times w_i) + o_M]$. If $o_m=0$ and $o_M=w_i$, the sub-interval corresponds to the whole fixed-width interval.

An element of our abstract domain is now stored (instead as a set of hypercubes) as a map from hypercubes to tuples of offsets. In this way, we can keep the original definition of a hypercube as a tuple of integers, but we also map each hypercube to a tuple of offsets (one for each variable). Before, an abstract state was defined by $ H = \{ h : h \in \integer^{|Vars|}\} $. Now an abstract state is defined by $M : \funzione{\integer^{|Vars|}}{{(\real \times \real)}^{|Vars|}}$, i.e., a map where the domain is the set of hypercubes, and the codomain is the set of tuples of offsets. Each hypercube is associated to a tuple of offsets and each tuple of offsets has one offset for each program variable.

The least upper bound between two abstract states ($M=M_1 \sqcup M_2$) is then defined as follows: $dom(M) = dom(M_1) \cup dom(M_2)$, and
$$
\forall h \in dom(M) : M(h) = \begin{cases}
M_1(h) & \mbox{if } h \in dom(M_1) \wedge h \notin dom(M_2) \\
M_2(h) & \mbox{if } h \in dom(M_2) \wedge h \notin dom(M_1) \\
merge(M_1(h),M_2(h)) & \mbox{otherwise }
\end{cases}$$
where $merge(o_1,o_2)$ creates a new tuple of offsets by merging the two tuples of offsets in input: for each pair of corresponding offsets (for example $(m_1,M_1)$ and $(m_2,M_2)$), the new offset is the widest combination possible (i.e., $(\min(m_1,m_2)$ and $\max(M_1,M_2))$). Note that this definition corresponds to the pointwise application of the least upper bound operator over intervals. The widening operator is extended in the same way: it applies the standard widening operators over intervals pointwisely to the elements of the vector representing the offsets.

We also have to modify the abstract semantics to accommodate this change. As the expression semantics $\mathbb{I}$ returns intervals of arbitrary widths, we can use such exact result to update the offsets of the abstract state. Formally, the semantics of the assignment becomes
\begin{align*}
assign(h, V_i, [a..b]) & = \{ \\
& h[i\mapsto (m,o_m,o_M)] : [m\times w_i .. (m+1)\times w_i] \cap [a..b] \neq \emptyset \wedge \\
& o_m = \begin{cases}
0 & \mbox{if } a \leq (m \times w_i) \\
a - (m \times w_i) & \mbox{otherwise} \\
\end{cases} \wedge \\
& o_M = \begin{cases}
w_i & \mbox{if } b \geq ((m+1) \times w_i) \\
b - (m \times w_i) & \mbox{otherwise} \\
\end{cases} \\
& \}
\end{align*}
where $h$ is a hypercube, $V_i$ is the assigned variable, and $[a..b]$ is the interval we are assigning. When we extract from a hypercube the interval associated to a variable, we use the interval delimited by the offsets, so that abstract operations can be much more precise.

Consider again the previous example (\statement{x = x + 0.01}, with $1.0$ as width of \statement{x} and $[0..1]$ as initial value of \statement{x}): after the first iteration we would return the two intervals $[0.0,1.0]$ and $[1.0,2.0]$ with offsets $[0.01,1.0]$ and $[1.0,1.01]$, respectively. In this way, at the following iteration we would obtain again the same two intervals (instead of three), with the offsets changed to $[0.02,1.0]$ and $[1.0,1.02]$. You can see that hypercubes proliferation is not an issue any more. When an offset moves out of the fixed interval inside which it should reside, we remove the corresponding hypercube from the set. In our example, after 100 iterations the first of the two intervals ($[0.0,1.0]$) would be removed from the hypercubes because the offset would start at $1.0$.

Offsets give us much more precision in verifying properties (as we will see in the next Section), without affecting too much the performances, since we just need to store more data for each hypercube, but the cost of the single operations remains almost the same. Indeed, the overall performance is even improved in practice because in the previous approach the number of hypercubes increased exponentially at each iteration of the \statement{while} loop, making the analysis unfeasible.

12. Hypercubes Domain/HC/Sections/versione SAS2013/4_tuning.tex

12. Hypercubes Domain/Sections/versione SAS2013/4_tuning.tex