\section{PGM to CNF}

\subsection{ENC 1}
Our ENC1 encoding for the Cancer Bayesian network can be found in appendix~\ref{ENC1}. The CNF in dimacs format can be found under report/encodings/cancer/.

\subsection{ENC 2}
Our ENC2 encoding for the Cancer Bayesian network can be found in appendix~\ref{ENC2}. The CNF in dimacs format can be found under report/encodings/cancer/.

\section{SRL to CNF}
\subsection{Encoding of Monty Hall as CNF}
An encoding of problog programs can be generated by our program as follows:
\begin{lstlisting}
python3 scripts/inference.py --problog files/problog/monty_hall.pl
\end{lstlisting}
The CNF will be shown using the program's predicates. A version of the CNF in dimacs format will be shown as well.
See \texttt{README.MD} for more information.

Our CNF encoding for the given Monty Hall ProbLog program is:
\begin{align*}
    \land & (open\_door(2) \lor prize(2) \lor prize(3) \lor \neg p\_open\_door(2)\_0) \\
    \land & (open\_door(2) \lor prize(2) \lor \neg prize(3))                          \\
    \land & (\neg open\_door(2) \lor \neg prize(2) \lor \neg prize(2))                \\
    \land & (\neg open\_door(2) \lor \neg prize(2) \lor prize(3))                     \\
    \land & (\neg open\_door(2) \lor \neg prize(3) \lor \neg prize(2))                \\
    \land & (\neg open\_door(2) \lor \neg prize(3) \lor prize(3))                     \\
    \land & (\neg open\_door(2) \lor p\_open\_door(2)\_0 \lor \neg prize(2))          \\
    \land & (\neg open\_door(2) \lor p\_open\_door(2)\_0 \lor prize(3))               \\
    \land & (open\_door(3) \lor prize(2) \lor prize(3) \lor \neg p\_open\_door(3)\_0) \\
    \land & (open\_door(3) \lor prize(3) \lor \neg prize(2))                          \\
    \land & (\neg open\_door(3) \lor \neg prize(2) \lor \neg prize(3))                \\
    \land & (\neg open\_door(3) \lor \neg prize(2) \lor prize(2))                     \\
    \land & (\neg open\_door(3) \lor \neg prize(3) \lor \neg prize(3))                \\
    \land & (\neg open\_door(3) \lor \neg prize(3) \lor prize(2))                     \\
    \land & (\neg open\_door(3) \lor p\_open\_door(3)\_0 \lor \neg prize(3))          \\
    \land & (\neg open\_door(3) \lor p\_open\_door(3)\_0 \lor prize(2))               \\
    \land & (win\_keep \lor \neg prize(1))                                            \\
    \land & (\neg win\_keep \lor prize(1))                                            \\
    \land & (win\_switch \lor \neg prize(2) \lor open\_door(2))                       \\
    \land & (win\_switch \lor \neg prize(3) \lor open\_door(3))                       \\
    \land & (\neg win\_switch \lor prize(2) \lor prize(3))                            \\
    \land & (\neg win\_switch \lor prize(2) \lor \neg open\_door(3))                  \\
    \land & (\neg win\_switch \lor \neg open\_door(2) \lor prize(3))                  \\
    \land & (\neg win\_switch \lor \neg open\_door(2) \lor \neg open\_door(3))        \\
    \land & (\neg prize(1) \lor \neg prize(2))                                        \\
    \land & (\neg prize(1) \lor \neg prize(3))                                        \\
    \land & (\neg prize(2) \lor \neg prize(3))                                        \\
    \land & (prize(1) \lor prize(2) \lor prize(3))\\
    & \textbf{Weights:}             \\
    & W(p\_open\_door(2)\_0) = 0.5 & W &(\neg p\_open\_door(2)\_0) = 0.5 \\
    & W(p\_open\_door(3)\_0) = 0.5 & W &(\neg p\_open\_door(3)\_0) = 0.5 \\
    & W(select\_door(1)) = 1.00    & W &(\neg select\_door(1)) = 0.00    \\
    & W(prize(1)) = 0.33           & W &(\neg prize(1)) = 1.00           \\
    & W(prize(2)) = 0.33           & W &(\neg prize(2)) = 1.00           \\
    & W(prize(3)) = 0.33           & W &(\neg prize(3)) = 1.00           \\
    & W(open\_door(2)) = 1.00      & W &(\neg open\_door(2)) = 1.00      \\
    & W(open\_door(3)) = 1.00      & W &(\neg open\_door(3)) = 1.00      \\
    & W(win\_keep) = 1.00          & W &(\neg win\_keep) = 1.00          \\
    & W(win\_switch) = 1.00        & W &(\neg win\_switch) = 1.00        \\
\end{align*}

\section{Weighted Model Counting}
\subsection{Weighted model counters on above CNFs}
We have selected MiniC2D and Cachet as weighted model counters and have executed them on DIMACS versions of the CNFs of the previous tasks. The DIMACS files can be found under report/encodings.
The output of the model counters is listed below.

\subsubsection{MiniC2D}
MiniC2D needs to be executed with the $-W$ flag in order for it to do weighted model counting.
The resulting probability can be read next to ``Count''.

\begin{lstlisting}[caption={MiniC2D on ENC1 encoding of Cancer network}]
Constructing CNF... DONE
CNF stats:
  Vars=30 / Clauses=74
  CNF Time	0.000s
Constructing vtree (from primal graph)... DONE
Vtree stats:
  Vtree widths: con<=5, c_con=48 v_con=5
  Vtree Time	0.001s
Counting... DONE
  Learned clauses      	0
Cache stats:
  hit rate   	75.0%
  lookups    	16
  ent count  	4
  ent memory 	0.2 KB
  ht  memory 	152.6 MB
  clists     	1.0 ave, 1 max
  keys       	3.0b ave, 3.0b max, 3.0b min
Count stats:
  Count Time	0.000s
  Count 	0.9999999999999999
Total Time: 0.012s
\end{lstlisting}

\begin{lstlisting}[caption={MiniC2D on ENC2 encoding of Cancer network}]
Constructing CNF... DONE
CNF stats:
  Vars=20 / Clauses=30
  CNF Time	0.000s
Constructing vtree (from primal graph)... DONE
Vtree stats:
  Vtree widths: con<=6, c_con=16 v_con=6
  Vtree Time	0.000s
Counting... DONE
  Learned clauses      	0
Cache stats:
  hit rate   	23.1%
  lookups    	26
  ent count  	20
  ent memory 	1.0 KB
  ht  memory 	152.6 MB
  clists     	1.0 ave, 1 max
  keys       	1.8b ave, 3.0b max, 1.0b min
Count stats:
  Count Time	0.000s
  Count 	1.0000000000000000
Total Time: 0.012s
\end{lstlisting}

\begin{lstlisting}[caption={MiniC2D on CNF encoding of Monty Hall}]
Constructing CNF... DONE
CNF stats:
  Vars=10 / Clauses=26
  CNF Time	0.000s
Constructing vtree (from primal graph)... DONE
Vtree stats:
  Vtree widths: con<=4, c_con=22 v_con=4
  Vtree Time	0.000s
Counting... DONE
  Learned clauses      	0
Cache stats:
  hit rate   	20.0%
  lookups    	5
  ent count  	4
  ent memory 	0.2 KB
  ht  memory 	152.6 MB
  clists     	1.0 ave, 1 max
  keys       	3.2b ave, 4.0b max, 3.0b min
Count stats:
  Count Time	0.000s
  Count 	1.0000000000000000
Total Time: 0.011s
\end{lstlisting}

\subsubsection{Cachet}
For Cachet, there is no need to use extra parameters to get a probability.
It is reported next to ``Satisfying probability''.

\begin{lstlisting}[caption={Cachet on ENC1 encoding of Cancer network}]
Number of total components		11
Number of split components		2
Number of non-split components		5
Number of SAT residual formula		12
Number of trivial components		0
Number of changed components		0
Number of adjusted components		0
First component split level		1

Number of Decisions			11
Max Decision Level			5
Number of Variables			30
Original Num Clauses			74
Original Num Literals			172
Added Conflict Clauses			0
Added Conflict Literals			0
Deleted Unrelevant clauses		0
Deleted Unrelevant literals		0
Number of Implications			124
Total Run Time				0.0163

Satisfying probability			8.72319e-08
Number of solutions			93.6645

\end{lstlisting}

\begin{lstlisting}[caption={Cachet on ENC2 encoding of Cancer network}]
Number of total components		11
Number of split components		2
Number of non-split components		5
Number of SAT residual formula		12
Number of trivial components		0
Number of changed components		0
Number of adjusted components		0
First component split level		1

Number of Decisions			11
Max Decision Level			5
Number of Variables			20
Original Num Clauses			30
Original Num Literals			84
Added Conflict Clauses			0
Added Conflict Literals			0
Deleted Unrelevant clauses		0
Deleted Unrelevant literals		0
Number of Implications			72
Total Run Time				0.017372

Satisfying probability			1
Number of solutions			1.04858e+06

\end{lstlisting}

\begin{lstlisting}[caption={Cachet on WCNF encoding of Monty Hall}]
Number of total components		4
Number of split components		1
Number of non-split components		2
Number of SAT residual formula		5
Number of trivial components		0
Number of changed components		0
Number of adjusted components		0
First component split level		2

Number of Decisions			4
Max Decision Level			4
Number of Variables			10
Original Num Clauses			26
Original Num Literals			73
Added Conflict Clauses			0
Added Conflict Literals			0
Deleted Unrelevant clauses		0
Deleted Unrelevant literals		0
Number of Implications			26
Total Run Time				0.016062

Satisfying probability			0.444444
Number of solutions			455.111
\end{lstlisting}

For ENC1, we see that with Cachet reports a satisfying probability of almost 0. Similarly, for Monty Hall, we see that we get a probability of 0.44. This is due to the fact that with ENC1, the weights of negated literals are 1, but Cachet expects that $weight(x) + weight(-x) = 1$. In the Monty Hall encoding, we also have weights of negated literals equalling 1, which gives the same problem as with ENC1.

\subsection{Difference between the selected WMCs}
%The differences between the different WMCs come from \cite{CHAVIRA2008772}.
%\subsubsection{Cachet vs jointree and recursive conditioning}
%Jointree and recursive conditioning only exploit topological structure thus they take no advantage of the massive determinism available in networks whilst Cachet does this.

\subsubsection{MiniC2D Vs Cachet}
MiniC2D and Cachet are weighted model counters that work in different ways. In short, MiniC2D is a top down compiler that compiles CNFs into SDDs, while Cachet uses formula caching combined with clause learning and component analysis [\cite{MiniC2D},\cite{Cachet}].

Both weighted model counters use concepts from the SAT literature. They both use clause learning and component caching in order to reuse components that later appear again during search. \\
Cachet also uses other methods from SAT literature, like an explicit on the fly calculation of connected components. This is different in MiniC2D, as it relies on vtrees to identify disconnected CNF components. MiniC2d creates vtrees for CNFs and then creates SDDs based on the created vtrees.
%The way that the compilation is done with MiniC2D is as follows:
%The SDD compilation is driven by the vtree, it uses this to identify disconnected CNF components and it uses a component caching scheme to prevent compiling the same component multiple times.

\subsection{Overview of computational requirements}
We have executed the model counters with various CNFs to build an overview of computational requirements. The files we used for testing can be found under report/encodings. We have used scripts to convert the ``.dsc'' files to ENC1 and ENC2 encodings in DIMACS format. We downloaded the ``.dsc'' files from \url{http://www.bnlearn.com/bnrepository/}.

\subsubsection{Cancer network (small)}
\begin{table}[H]
    \centering
    \begin{tabular}{c|c|c|c|c|c|c|}
    \cline{2-7}
            & \multicolumn{3}{c|}{ENC1} & \multicolumn{3}{c|}{ENC2} \\ \cline{2-7}
      & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} \\ \cline{1-7}
      \textbf{Minic2d} & 1.0  & 0.3 KB    & 0.053s   & 1.0    & 1.0 KB    & 0.050s \\
      \hline
    \textbf{Cachet}  & 0.0  & ?    & 0.016s       & 1.0     & ?    & 0.016s    \\ \cline{1-7}
    \end{tabular}
\end{table}
% updated cancer with total time

\subsubsection{Asia network (small)}
\begin{table}[H]
    \centering
    \begin{tabular}{c|c|c|c|c|c|c|}
    \cline{2-7}
            & \multicolumn{3}{c|}{ENC1} & \multicolumn{3}{c|}{ENC2} \\ \cline{2-7}
      & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} \\ \cline{1-7}
      \textbf{Minic2d} & 1.0  & 1.2 KB    & 0.049s   & 1.0    & 1.9 KB    & 	 0.05s \\
      \hline
    \textbf{Cachet}  & 0.0  & ?    & 0.018s       & 1.0     & ?    & 0.017s    \\ \cline{1-7}
    \end{tabular}
\end{table}

\subsubsection{Sachs network (small)}
\begin{table}[H]
    \centering
    \begin{tabular}{c|c|c|c|c|c|c|}
    \cline{2-7}
            & \multicolumn{3}{c|}{ENC1} & \multicolumn{3}{c|}{ENC2} \\ \cline{2-7}
      & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} \\ \cline{1-7}
      \textbf{Minic2d} & 0.99707  & 16.8 KB    & 0.075s   & 1.0    & 13.4 KB   & 	0.07s \\
      \hline
    \textbf{Cachet}  & 0.0  & ?    & 0.019s       & 1.0     & ?    & 0.017s    \\ \cline{1-7}
    \end{tabular}
\end{table}
%sachs also updated

\subsubsection{Earthquake network (small)}
\begin{table}[H]
    \centering
    \begin{tabular}{c|c|c|c|c|c|c|}
    \cline{2-7}
            & \multicolumn{3}{c|}{ENC1} & \multicolumn{3}{c|}{ENC2} \\ \cline{2-7}
      & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} \\ \cline{1-7}
      \textbf{Minic2d} & 1.0  & 0.6 KB & 0.051s   & 1.0    & 1.0 KB  & 	0.05s \\
      \hline
    \textbf{Cachet}  & 0.0  & ?    & 0.016s       & 1.0     & ?    & 0.017s    \\ \cline{1-7}
    \end{tabular}
\end{table}

\subsubsection{Survey network (small)}
\begin{table}[H]
\centering
    \begin{tabular}{c|c|c|c|c|c|c|}
    \cline{2-7}
            & \multicolumn{3}{c|}{ENC1} & \multicolumn{3}{c|}{ENC2} \\ \cline{2-7}
      & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} \\ \cline{1-7}
      \textbf{Minic2d} & 1.0  & 0.5 KB    & 0.035s   & 1.0    & 2.0 KB    & 	0.052s \\
      \hline
    \textbf{Cachet}  & 0.0  & ?    & 0.016s       & 1.0     & ?    & 0.016s    \\ \cline{1-7}
    \end{tabular}
\end{table}

\subsubsection{Alarm network (medium)}
\begin{table}[H]
    \centering
    \begin{tabular}{c|c|c|c|c|c|c|}
    \cline{2-7}
            & \multicolumn{3}{c|}{ENC1} & \multicolumn{3}{c|}{ENC2} \\ \cline{2-7}
      & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} \\ \cline{1-7}
      \textbf{Minic2d} & 0.999  & 451.1 KB    & 0.217s   & 0.999    & 143.4 KB    & 	0.093s \\
      \hline
    \textbf{Cachet}  & 0.0  & ?    & 0.176s       & 1.0     & ?    & 0.222s    \\ \cline{1-7}
\end{tabular}
\end{table}

\subsubsection{Child network (medium)}
\begin{table}[H]
    \centering
    \begin{tabular}{c|c|c|c|c|c|c|}
    \cline{2-7}
            & \multicolumn{3}{c|}{ENC1} & \multicolumn{3}{c|}{ENC2} \\ \cline{2-7}
      & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} \\ \cline{1-7}
      \textbf{Minic2d} & 1.0  & 45.8KB    & 0.076s   & 1.0    & 30.8 KB    & 	0.059s \\
      \hline
    \textbf{Cachet}  & 0.0  & ?    & 0.03s      & 1.0     & ?    & 0.03s    \\ \cline{1-7}
    \end{tabular}
\end{table}

\subsubsection{Hailfinder network (large)}
\begin{table}[H]
    \centering
    \begin{tabular}{c|c|c|c|c|c|c|}
    \cline{2-7}
            & \multicolumn{3}{c|}{ENC1} & \multicolumn{3}{c|}{ENC2} \\ \cline{2-7}
      & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} \\ \cline{1-7}
      \textbf{Minic2d} & 0.999  & 591.8 MB & 46.065s   & 1.0    & 25.1MB    & 	2.73s \\
      \hline
    \textbf{Cachet}  & 0  & ?    & 58.86s       & 1     & ?    & 15.21s    \\ \cline{1-7}
    \end{tabular}
\end{table}

\subsubsection{Andes network (very large)}
\begin{table}[H]
    \centering
    \begin{tabular}{c|c|c|c|c|c|c|}
    \cline{2-7}
            & \multicolumn{3}{c|}{ENC1} & \multicolumn{3}{c|}{ENC2} \\ \cline{2-7}
      & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} & \textbf{Prob}  & \textbf{Memory}  & \textbf{Runtime} \\ \cline{1-7}
      \textbf{Minic2d} & 1.0  & 5.5GB    & 266.605s   & 1.0    & 122.8 MB    & 	5.646s \\
      \hline
    \textbf{Cachet}  & ?  & ?    & $ > 4h (killed) $      & ?     & ?    & $ > 4h (killed) $     \\ \cline{1-7}
    \end{tabular}
\end{table}

For Cachet, we didn't find a way to output memory usage, so we cannot compare that aspect to MiniC2D.
From the results listed above, we can conclude that ENC2 is a better encoding than ENC1 when it comes to computational requirements. In most of the cases, it has lower memory usage and a lower runtime.
Concerning the weighted model counters, we clearly see that Cachet is faster than MiniC2D on small to medium networks. When we use larger networks, Cachet is a lot slower than MiniC2D, and for the very large network, Cachet couldn't even finish in 4 hours!

\section{Knowledge compilation}
\subsubsection{Vtree with the most compact circuit}
For this task, we used both the SDD and MiniC2D packages to generate vtrees and SDDs. The resulting vtrees and their corresponding SDDs can be found under report/1.4/. Subdirectories have been made for each WMC's output for each of the CNFs. The file name in each of these directories follow the following naming convention: (SDD\_vtree\_type $\vert$ MiniC2D\_vtree\_method)[\_SDD\_minimize\_cardinality]\_(vtree $\vert$ sdd)
\begin{itemize}
    \item \textbf{ENC1}: SDD built the best vtree for the most compact circuit with the ``-t balanced -m'' flags. The next best circuit can be created via the vtree created by MiniC2D with ``-m 3'' (reverse elimination order).

    \item \textbf{ENC2}: For this encoding, we have different options that result in the best circuits. The options ``-t balanced'' for SDD and ``-m 3'' for MiniC2D are again one of the best. Like for ENC1, the vtree created by SDD is better than the one from MiniC2D.

    \item \textbf{Monty Hall}: Here we see that by using the option ``-t balanced -m'', we receive the most compact circuit and yet again closely followed by ``-m 3'' from MiniC2D.
\end{itemize}

\subsubsection{Pattern for a good vtree}
From our tests, we've noticed that vtrees that are more shallow result in the best circuits. This seems logical to us, since a vtree is a binary search tree, and with such a tree you also prefer it to be shallow.


\chapter{Morphological Component Analysis}
\label{sect_ctm}
\section{Introduction}
The content of an image is often complex, and there is not a single
transform which is optimal to represent all the contained features. For
example, the Fourier transform better represents some textures, while
the wavelet transform better represents singularities. Even if we limit
our class of transforms to the wavelet one, decision have to
be taken between an isotropic wavelet transform which produce good
results for isotropic objects (such stars and galaxies in astronomical
images, cells in biological images, etc), or an orthogonal wavelet
transform, which is better for images with edges.
This has motivated
the development of different methods \cite{wave:donoho98,wave:meyer98,cur:huo99},
and the two most frequently discussed approaches are the Matching Pursuit (MP)
\cite{wave:mallat93} and the Basis pursuit (BP) \cite{wave:donoho98}.
A dictionary ${\cal D}$ being defined as a collection of waveforms
$(\varphi_{\gamma})_{\gamma \in \Gamma}$,
the general principe consists in representing a signal $s$ as a ``sparse''
linear combination  of a small number of basis such that:
\begin{eqnarray}
 s = \sum_{\gamma} a_{\gamma} \varphi_{\gamma}
\end{eqnarray}
or an approximate decomposition
\begin{eqnarray}
 s = \sum_{i=1}^m a_{\gamma_i} \varphi_{\gamma_i} + R^{(m)} .
\end{eqnarray}

Matching pursuit \cite{wave:mallat93,ima:mallat98} method (MP) uses a greedy
algorithm which adaptively refines the signal approximation with an
iterative procedure:
\begin{itemize}
\item Set $s^0 = 0$ and $R^0 = 0$.
\item Find the element $\alpha_k \varphi_{\gamma_k}$ which best correlates with the
residual.
\item Update $s$ and $R$:
\begin{eqnarray}
s^{k+1} & = & s^k + \alpha_k \varphi_{\gamma_k} \nonumber \\
R^{k+1} & = & s -  s^k .
\end{eqnarray}
\end{itemize}
In case of non orthogonal dictionaries, it has been shown \cite{wave:donoho98}
 that MP may
spend most of the time correcting mistakes made in the first few terms,
and therefore is suboptimal in term of sparsity.

\bigskip
Basis pursuit method  \cite{wave:donoho98} (BP) is a global procedure which
synthesizes an approximation $\tilde{s}$ to $s$ by minimizing a
functional of the type
\begin{equation}
\|s - \tilde{s}\|_{\ell_2} ^2 + \lambda \cdot \|\alpha\|_{\ell_1},
\quad \tilde{s} = \Phi \alpha.
\label{eqn_mp}
\end{equation}
Between all possible solutions, the chosen one has the minimum $l^1$ norm.
This choice of $l_1$ norm is very important. A $l_2$ norm, as
used in the method of frames \cite{wave:daube88b},
 does not preserve the sparsity \cite{wave:donoho98}.

In many cases, BP or MP synthesis algorithms are computationally very
expensive. We present in the following an alternative approach, that we call
{\em Morphological Component Analysis} (MCA), which combines
 the different available transforms
in order to benefit of the advantages of each of them.

\section{The Combined Transformation}

Depending on the content of the data, several transforms
can be combined in order to get an optimal representation of all
features contained in our data set. In addition to the ridgelet
and the curvelet transform, we may want to use the \`a trous algorithm
which is very well suited to astronomical data, or the undecimated
wavelet transform which is commonly used in the signal processing domain.

Other transform such wavelet packets, the Fourier transform,
the Pyramidal median transform \cite{starck:book98},
or other multiscale morphological transforms,
could also be considered. However, we found that in practice,
these four transforms (i.e. curvelet, ridgelet, \`a trous algorithm,
and undecimated wavelet transform)
furnishes a very large panel of waveforms
which is generally large enough to well
represents all features contained in the data.

In general, suppose that we are given $K$ linear transforms $T_1,
\ldots, T_K$ and let $\alpha_k$ be the coefficient sequence of an
object $x$ after applying the transform $T_k$, i.e. $\alpha_k = T_k
x$. We will suppose that for each transform $T_k$ we have available a
reconstruction rule that we will denote by $T^{-1}_k$ although this is
clearly an abuse of notations.

Therefore, we search a vector
${\bf \alpha} = {\alpha_1, \dots, \alpha_{K}}$ such that
\begin{equation}
s =  \Phi {\bf \alpha}
\end{equation}
where $\Phi {\bf \alpha} = \sum_{k=1}^K T_k^{-1} \alpha_k$.
As our dictionary is overcomplete, there is an infinity of vectors
verifing this condition, and we need to solve the following optimization
problem:
\begin{equation}
min \parallel s - \phi {\bf \alpha} \parallel^2 + {\cal C}(\bf \alpha)
\end{equation}
where ${\cal C}$ is a penalty term. We easily see that chosing
${\cal C}({\bf \alpha}) =  \parallel {\bf \alpha} \parallel_{l_1}$ leads to the
BP method, where the dictionary ${\cal D}$ is only composed of the
basis elements of the chosen transforms.

Two iterative methods, {\em soft-MCA} and {\em hard-MCA}, allowing us
to realize such a combined transform, are described in this section.

\section{Soft-MCA}
\label{sect_l1}
% $l^1$ optimization by soft thresholding

Noting $T_1, ..., T_{K}$ the $K$ transform operators,
a solution ${\bf \alpha}$  is obtained by minimizing a functional of the form:
\begin{eqnarray}
J({\bf \alpha}) = \parallel s - \sum_{k=1}^{K} T_k^{-1} \alpha_k  \parallel_2^2 + \lambda \sum_k \parallel \alpha_k \parallel_1
\label{eqn_min_l1}
\end{eqnarray}
where $s$ is the original signal, and $\alpha_k$ are the coefficients
obtained with the transform $T_k$.

An simple algorithm to achieve such an solution is \cite{starck:spie01b,starck:sta02_3}:
\begin{enumerate}
\item Initialize $L_{\max}$, the number of iterations $N_i$,
$\lambda = L_{\max}$, and $\delta_{\lambda} = \frac{L_{\max}}{N_i}$.
\item While $\lambda >= 0$ do
\item For k = 1, .., $K$ do
\begin{itemize}
\item Calculate the residual $R = s - \sum_k T_k^{-1} \alpha_k$.
\item Calculate the  transform $T_k$ of the residual:
$r_k = T_k R$.
\item Add the residual to  $\alpha_k$:  $\alpha_k = \alpha_k + r_k$.
\item Soft threshold the coefficient $\alpha_k$ with the $\lambda$ threshold.
\end{itemize}
\item $\lambda = \lambda - \delta$, and goto 2.
\end{enumerate}

Figure~\ref{fig_cb1_synt} illustrates the result in the case where the
input image contains only lines and Gaussians. In this experiment, we have
initialized  $L_{\max}$ to $20$, and $\delta$ to $2$ (10 iterations).
Two transform operators
were used, the \`a trous wavelet transform and the ridgelet transform. The
first is well adapted to the detection of Gaussian due to the isotropy of
the wavelet function~\cite{starck:book98}, while the second is optimal
to represent lines \cite{cur:candes99_1}. Figure~\ref{fig_cb1_synt} top,
bottom left, and bottom right represents respectively the
original image, the reconstructed image from the \`a trous wavelet
coefficient, and the reconstructed image from the ridgelet
coefficient. The addition of both reconstructed images reproduces the
original one.

\begin{figure}[htb]
\vbox{
\centerline{
\hbox{
\psfig{figure=fig_cb2_line_g.ps,bbllx=2cm,bblly=13.5cm,bburx=13cm,bbury=24.5cm,width=5.5cm,height=5.5cm,clip=}}
}
\centerline{
\hbox{
\psfig{figure=fig_cb2_line_g_atrou.ps,bbllx=2cm,bblly=13.5cm,bburx=13cm,bbury=24.5cm,width=5.5cm,height=5.5cm,clip=}
\psfig{figure=fig_cb2_line_g_rid.ps,bbllx=2cm,bblly=13.5cm,bburx=13cm,bbury=24.5cm,width=5.5cm,height=5.5cm,clip=}
}}}
\caption{Top, original image containing lines and gaussians. Botton left,
reconstructed image for the \`a trous wavelet coefficient, bottom right,
reconstructed image from the ridgelet coefficients.}
\label{fig_cb1_synt}
\end{figure}

In some specific cases where the data are sparse in all bases,
it has been shown \cite{cur:huo99,cur:donoho01}
that the solution is identical to the solution when using a
$\parallel . \parallel_0$ penalty term. This is however generally not the
case.
The problem we met in image restoration applications,
when minimizing equation~\ref{eqn_min_l1}, is
that both the signal and noise are split into the bases. The way the noise
is distributed in the coefficients $\alpha_k$ is not known, and leads to the problem
that we do not know at which level we should threshold the coefficients. Using
the threshold we would have used with a single transform
makes a strong over-filtering of the data. Using the $l^1$ optimization
for data restoration implies to first study how the noise is
distributed in the coefficients. The hard-MCA method does not present
this drawback.

\section{Hard-MCA}
\label{sect_l0}
%  $l^0$ optimization by hard thresholding
The following algorithm consists in hard thresholding the residual
successively on the different bases \cite{starck:spie01b,starck:sta02_3}.
\begin{enumerate}
\item For noise filtering, estimate the noise standard deviation $\sigma$,
and set $L_{\min} = k_{\sigma}$.
Otherwise, set $\sigma=1$ and $L_{\min} = 0$.
\item Initialize $L_{\max}$, the number of iterations $N_i$,
$\lambda = L_{\max}$ and $\delta_{\lambda} = \frac{L_{\max} - L_{\min}}{N_i}$.
\item Set all coefficients $\alpha_k$ to 0.
\item While $\lambda >= L_{\min}$ do
\item for k = 1, .., $K$ do
\begin{itemize}
\item Calculate the residual $R = s - \sum_k T_k^{-1} \alpha_k$.
\item Calculate the transform $T_k$ of the residual:
$r_k = T_k R$.
\item For all coefficients $\alpha_{k,i}$ do
\begin{itemize}
\item  Update the coefficients:
 if $\alpha_{k,i} \ne 0$ or $\mid r_{k,i}  \mid > \lambda \sigma$
then $\alpha_{k,i}  =  \alpha_{k,i} + r_{k,i}$.
\end{itemize}
\end{itemize}
\item $\lambda = \lambda - \delta_{\lambda} $, and goto 5.
\end{enumerate}
For an exact representation of the data, $k_{\sigma}$ must be set to 0.
Choosing $k_{\sigma} > 0$ introduces a filtering. If a single transform is used,
it corresponds to the standard  $k$-sigma hard thresholding.

It seems that starting with a high enough $L_{\max}$ and a high
number of iterations would lead to the $l^0$ optimization solution,
but this remains to be proved.

\section{Experiments}

\subsubsection{Experiment 1: Infrared Gemini Data}
\begin{figure}[htb]
\centerline{
\vbox{
\hbox{
\psfig{figure=fig_gemini.ps,bbllx=1.8cm,bblly=12.5cm,bburx=14.8cm,bbury=25.5cm,width=5cm,height=5cm,clip=}
\psfig{figure=fig_gem_cb_rid.ps,bbllx=1.8cm,bblly=12.5cm,bburx=14.8cm,bbury=25.5cm,width=5cm,height=5cm,clip=}
\psfig{figure=fig_gem_cb_cur.ps,bbllx=1.8cm,bblly=12.5cm,bburx=14.8cm,bbury=25.5cm,width=5cm,height=5cm,clip=}
}
\hbox{
\psfig{figure=fig_gem_cb_atrou.ps,bbllx=1.8cm,bblly=12.5cm,bburx=14.8cm,bbury=25.5cm,width=5cm,height=5cm,clip=}
\psfig{figure=fig_gem_cb_resi_cur_rid_atrou.ps,bbllx=1.8cm,bblly=12.5cm,bburx=14.8cm,bbury=25.5cm,width=5cm,height=5cm,clip=}
\psfig{figure=fig_gem_cb_resi_cur_rid.ps,bbllx=1.8cm,bblly=12.5cm,bburx=14.8cm,bbury=25.5cm,width=5cm,height=5cm,clip=}
}
}}
\caption{Upper left, galaxy SBS 0335-052 (10 $\mu$m), upper middle, upper middle,
and bottom left, reconstruction respectively from the ridgelet, the curvelet
and wavelet coefficients. Bottom middle, residual image. Bottom right,
artifact free image.}
\label{fig_ctm_gemini1}
\end{figure}
$ $
\begin{figure}[htb]
\centerline{
\vbox{
\hbox{
\psfig{figure=fig_gemini2_bw.ps,bbllx=1.8cm,bblly=12.5cm,bburx=14.8cm,bbury=25.5cm,width=7cm,height=7cm,clip=}
\psfig{figure=fig_gem2_cb_cur_rid_bw.ps,bbllx=1.8cm,bblly=12.5cm,bburx=14.8cm,bbury=25.5cm,width=7cm,height=7cm,clip=}
}
\hbox{
\psfig{figure=fig_gem2_cb_atrou_bw.ps,bbllx=1.8cm,bblly=12.5cm,bburx=14.8cm,bbury=25.5cm,width=7cm,height=7cm,clip=}
\psfig{figure=fig_gem2_cb_resi_cur_rid_atrou_bw.ps,bbllx=1.8cm,bblly=12.5cm,bburx=14.8cm,bbury=25.5cm,width=7cm,height=7cm,clip=}
}
}}
\caption{Upper left, galaxy SBS 0335-052 (20 $\mu$m), upper right, addition
of the reconstructed images from both the ridgelet and the curvelet coefficients,
bottom left, reconstruction from the wavelet coefficients, and bottom right,
residual image.}
\label{fig_ctm_gemini2}
\end{figure}
\clearpage
Fig.~\ref{fig_ctm_gemini1} upper left shows
a compact blue galaxy located at 53 Mpc. The data have been
obtained on ground with the GEMINI-OSCIR instrument at $10$ $\mu$m.
The pixel field of
view is $0.089^{\prime\prime}$/pix, and the  source was observed during 1500s.
The data are contaminated by a noise and a stripping artifact due to the
instrument electronic. The same kind of artifact pattern were observed with
the ISOCAM instrument \cite{starck:sta99_1}.

This image, noted $D_{10}$, has been decomposed using wavelets, ridgelets, and curvelets.
Fig.~\ref{fig_ctm_gemini1} upper middle,  upper  right, and bottom left
show the three images $R_{10}$, $C_{10}$, $W_{10}$
reconstructed respectively from the ridgelets, the curvelets, and the wavelets.
Image in Fig.~\ref{fig_ctm_gemini1} bottom middle shows the residual, i.e.
$e_{10} = D_{10} - (R_{10} + C_{10} + W_{10})$. Another interesting
image is the artifact free one, obtained by subtracting $R_{10}$ and
$C_{10}$ from the input data (see Fig.~\ref{fig_ctm_gemini1} bottom right).
The galaxy has well been detected in the wavelet space, while all stripping
artifact have been capted by the ridgelets and curvelets.

Fig.~\ref{fig_ctm_gemini2} upper left shows the same galaxy, but at
$20$ $\mu$m. We have applied the same decomposition on $D_{20}$.
Fig.~\ref{fig_ctm_gemini2} upper right shows the coadded
image $R_{20} + C_{20}$, and we can see bottom left and right the
wavelet reconstruction $W_{20}$ and the residudal
$e_{20} = D_{20} - (R_{20} + C_{20} + W_{20})$.

\subsubsection{Experiment 2: A370}

\begin{figure}[htb]
\centerline{
\vbox{
\hbox{
\psfig{figure=a370.ps,bbllx=1.5cm,bblly=6cm,bburx=19.5cm,bbury=24cm,width=7cm,height=7cm,clip=}
\psfig{figure=a370_ridcur.ps,bbllx=1.5cm,bblly=6cm,bburx=19.5cm,bbury=24cm,width=7cm,height=7cm,clip=}
}
\hbox{
\psfig{figure=a370_atrou.ps,bbllx=1.5cm,bblly=6cm,bburx=19.5cm,bbury=24cm,width=7cm,height=7cm,clip=}
\psfig{figure=a370_comb.ps,bbllx=1.5cm,bblly=6cm,bburx=19.5cm,bbury=24cm,width=7cm,height=7cm,clip=}
}
}}
\caption{Top left, HST image of A370, top right coadded image from the
reconstructions from the ridgelet and the curvelet coefficients, bottom
left  reconstruction from  the \`a trous wavelet coefficients,
and bottom right addition of the three reconstructed images. }
\label{fig_a370}
\end{figure}

Figure~\ref{fig_a370} upper left shows the HST A370 image. It contains many
anisotropic features such the gravitationnal arc, and the arclets. The
image has been decomposed using three transforms: the ridgelet transform,
the curvelet transform, and the \`a trous wavelet transform. Three images
have then been reconstructed from the coefficients of the three basis.
Figure~\ref{fig_a370} upper right shows the coaddition of the ridgelet
and curvelet reconstructed images. The \`a trous reconstructed image
is displayed in Figure~\ref{fig_a370} lower left, and the coaddition
of the three images can be seen in Figure~\ref{fig_a370} lower right.
The gravitational arc and the arclets are all represented in the
ridgelet and the curvelet basis, while all isotropic features are
better represented in the wavelet basis.

We can see that this Morphological Component Analysis (MCA) allows
us to separate automatically features in an image which have different
morphological aspects. It is very different from other techniques such as
Principal Component Analysis or
Independent Component Analysis \cite{mc:cardoso98} where the separation
is performed via statistical properties.

\clearpage
\newpage


\documentclass{article}
\usepackage{fullpage}
\usepackage{nopageno}
\usepackage{amsmath}
\allowdisplaybreaks

\newcommand{\abs}[1]{\left\lvert #1 \right\rvert}

\begin{document}
\title{Notes}
\date{December 6, 2013}
\maketitle
quiz in 6.2-6.4 fri?

\section*{section 6.3}
\begin{align*}
  x'&=3x+3y\\
  y'&=3x+8y \intertext{both x and y are functions of t}
\end{align*}
\subsection*{example}
show that $\left(\begin{array}{c}-2e^{2t}\\e^t\end{array}\right)$ is a solution of $\mathbb{X}'\left(\begin{array}{cc}1&-2\\2&6 \end{array}\right)\mathbb{x}$
\begin{align*}
  \mathbb{X}'&=\left(\begin{array}{c}
      -4e^{2t}\\2e^{2t}
      \end{array}\right)
  &=\left(\begin{array}{cc}
      1&-2\\2&6
      \end{array}\right)
  \left(\begin{array}{c}
      -2e^{2t}\\e^{2t}
      \end{array}\right)
  &=\left(\begin{array}{c}
      -2e^{2t}-2e^{2t}\\
      -4e^{2t}+6e^{2t}
      \end{array}\right)
  &=\left(\begin{array}{c}
      -4e^{2t}\\2e^{2t}
      \end{array}\right)
\end{align*}
\begin{align*}
  W&=\left\lvert\begin{array}{cc}y_1&y_2\\y_1'&y_2'\end{array}\right\rvert\\
  \phi_1&=\left(\begin{array}{c}\alpha_1(t)\\\alpha_2(t)\end{array}\right)\\
  \phi_2&=\left(\begin{array}{c}\beta_1(t)\\\beta_2(t)\end{array}\right)\\
  W(\phi_1,\phi_2)&=\left(\begin{array}{cc}\alpha_1&\beta_1\\ \alpha_2&\beta_2 \end{array}\right)
\end{align*}
\subsection*{example}
check if $\phi$'s are independant solutions
\begin{align*}
  \mathbb{X}'&=\left(\begin{array}{cc}4&-3\\-2&-1 \end{array}\right)\mathbb{X}\\
  \phi_1&=\left(\begin{array}{c}e^{-2t}\\2e^{-2t} \end{array}\right)\\
  \phi_2&=\left(\begin{array}{c}3e^{5t}\\e^{5t} \end{array}\right)\\
  W(\phi_1,\phi_2)&=e^{3t}+6e^{3t}=7e^{3t}\neq0
\end{align*}
they are independant

\subsection*{eigenvalues}
\begin{align*}
  A&=\left(\begin{array}{cc}2&3\\-4&-5\end{array}\right)
\end{align*}
find the eigenvalue of A
\begin{align*}
  \left\lvert\righ\left(\begin{array}{cc}2-\lambda&3\\-4&-5-\lambda\end{array}\right)t\rvert&=0\\
    (2-\lambda)(-5-\lambda)+12&=0\\
    -10-2\lambda+5\lambda+\lambda^2+12&=0\\
    \lambda^2+3\lambda+2&=0\\
    (\lambda+1)(\lambda+2)&=0\\
    \lambda&=-1,-2
\end{align*}
eigenvector corresponding to $\lambda=-1$
\begin{align*}
  A\left(\begin{array}{c}x_1\\x_2\end{array}\right)&=-1\left(\begin{array}{c}x_1&x_2\end{array}\right)\\
  \left(\begin{array}{cc}2&3\\-4&-5\end{array}\right)\left(\begin{array}{c}x_1&x_2\end{array}\right)&=-\left(\begin{array}{c}x_1&x_2\end{array}\right)\\
  \left(\begin{array}{c}2x_1+3x_2\\-4x_1-5x_2\end{array}\right)=-\left(\begin{array}{c}x_1&x_2\end{array}\right)\\
  2x_1+3x_2&=-x_1\\
  -4x_1-5x_2&=x_2\\
  3x_1+3x_2&=0\\
  -4x_1-4x_2&=0\\
  x_1+x_2&=0\\
  x_1&=x_2 \intertext{choose}
  x_1&=1,x_2=-1\\
  \text{eigenvector}&=\left(\begin{array}{c}1\\-1\end{array}\right)
\end{align*}
eigenvector corresponding to $\lambda=-2$ is $\left(\begin{array}{c}3\\-4\end{array}\right)$

quiz show that eigenvector
\end{document}


\documentclass[letterpaper]{article}

\usepackage{fullpage}
\usepackage{nopageno}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{tikz}
\usepackage[utf8]{inputenc}
\usepackage{aeguill}
\usepackage{setspace}

\usetikzlibrary{graphs,graphdrawing}
\usegdlibrary{trees}

\allowdisplaybreaks

\newcommand{\abs}[1]{\left\lvert #1 \right\rvert}

\begin{document}
\title{Notes}
\date{6 février, 2015}
\maketitle
wednesday was eulerian graphs (bridges of königsberg).

cycles are not circuits and trails are not paths

\section*{3.2 hamiltonian paths}
Euler is E, edges is E. Hamiltonian graphs is H, vertices is...not

a {\bfseries Hamiltonian path} or {\bfseries cycle} is a path or cycle that meets every vertex of $G$ exactly once.

a graph with hamiltonian cycle is called {\bfseries hamiltonian}
\subsection*{example}
\begin{enumerate}
\item
Hamiltonian:

\begin{tikzpicture}[main_node/.style={circle,draw,text=black,inner sep=1pt,outer sep=0pt]}]
  \node[main_node] (1) at (-1,-1) {};
  \node[main_node] (2) at (1,-1) {};
  \node[main_node] (3) at (1,1) {};
  \node[main_node] (4) at (-1,1) {};
  \draw (1) -- (2) -- (3) -- (4)--(1);
\end{tikzpicture}
\item
Not hamiltonian

\tikz\path [graphs/.cd, nodes={shape=circle, draw, text=black,inner sep=1pt,outer sep=0pt}]
  graph [tree layout] { 1 -- {2 -- 3};2--4 }
  [shift=(0:1)];
\item
is hamiltonian
\tikz\path [graphs/.cd, nodes={shape=circle, draw, text=black,inner sep=1pt,outer sep=0pt}]
  graph [tree layout] { 1 -- 2;1--3;1--4;1--5;1--6;2--3--4--5--6 }
  [shift=(0:1)];
\end{enumerate}

\subsection*{theorem (Ore)}
if $G$ is a graph of order $n\ge 3$ and $\forall u,v$ vertices $\deg(u)+\deg(v)\ge n$ then $G$ is hamiltonian

{\scshape note:} completely nonconstructive proof

\subsubsection*{proof}
assume for a contradiction that for all $u,v\in V(G)$, $\deg(u)+\deg(v)\ge n$ but $G$ is not hamiltonian.

without loss of generality, we can assume that $G$ is ``maximal'' with this property. why?

$G$ is finite therefore $G$ is a subgraph  of some complete graph $G\le K_n$. But $K_n$ is hamiltonian. Since $G$ is not hamiltonian and $K_n$ is then. somewhere added enough edges to $G$ to make it hamiltonian.

Add edge $xy$ to $G$. The $G\cup \{xy\}$ is hamiltonian, so there is an $x-y$ path in $G$. We have to use the $xy$ edge in the hamiltonian cycle, else the graph would already be hamiltonian. In fact the $x-y$ path is hamiltonian.

let the $x-y$ path be $x=v_1,\dots,v_n=y$. If $x$ is adjacent to $v_i$ then $y$ cannot be adjacent to $v_{i-1}$. why? because then you would have a hamiltonian cycle. $v_1v_i\dots v_nv_{i-1}v_1$

Therefore, for every neighbor of $x$ we can eliminate a possible neighbor of $y$. This means that $\deg(y)\le(n-1)-\deg(x)$. because $n-1$ is max possible deg and $\deg(x)$ are the things that can't be adjacent to $y$. Now $\deg(y)+\deg(x)\le n-1$ which is a contradiction because $\deg(y)+\deg(x)\ge n$

\subsection*{corollary}
if $G$ of order $n\ge 3$ has the property that for all $v\in V(G)$ then $\deg(v)\ge \frac{n}{2}$, then $G$ is hamiltonian.
\subsubsection*{proof}
$\forall u,v\in V(G)$ then $\deg(u)+\deg(v)\ge n$

\section*{independent sets}
a subset of $V(G)$ is called {\bfseries independent} if no vertices are adjacent to one another.

the maximal cardinality of independent sets is called the {\bfseries independent number}. it is denoted $\alpha(G)$

what is $\alpha(K_n)$? 1

\section*{theorem chvátal-erdös}
let $G$ be a graph of order $n\ge 3$. if connectedness$\kappa(G)\ge \alpha(G)$, then $G$ is Hamiltonian.

\section*{homework}
1,7,14
\end{document}


%!TEX root = ../main.tex
\section{Pure random attachment}\label{section:pure-random-attachment}

\subsection{Degree distribution: Theory}
The pure random attachment model can be seen as a limiting case of the BA model, where all existing vertices are chosen with equal probability, i.e. $\Pi = \Pi_{rnd} \propto 1$. This preserves growth but removes preferential attachment.

We start from the master equation in \autoref{eq:master}, but instead of $\Pi = k/ 2E(t)$, we use $\Pi_{rnd} = 1 / N(t)$. Again, we consider the long-time ansatz $n(k, t) \rightarrow N(t) p_{\infty}(k)$. Substituting these terms into \autoref{eq:master}, we have

\begin{equation}
	p_{\infty}(k) = m p_{\infty}(k-1) - m p_{\infty}(k) + \delta_{k, m}.
	\label{eq:ra-degree-distribution-p-infinity}
\end{equation}

Considering the case of $k > m$, we obtain the recurrence relation
\begin{equation}
	p_{\infty}(k) = \left ( \frac{m}{m+1} \right ) p_{\infty}(k-1)=...= \left ( \frac{m}{m+1} \right )^{k-m} p_{\infty}(m)
	\label{eq:ra-degree-recurrence-relation}
\end{equation}

Now we consider $k=m$. Substituting $k=m$ into \autoref{eq:ra-degree-distribution-p-infinity} and remembering that $p_{\infty}(k < m) = 0$, we get
\begin{equation}
	p_{\infty}(m) = -mp_{\infty} + 1,
	\label{eq:ra-degree-k-equal-m}
\end{equation}
giving us
\begin{equation}
	p_{\infty}(m) = \frac{1}{m+1}.
	\label{eq:ra-degree-p-infinity-m}
\end{equation}

Combining this result with \autoref{eq:ra-degree-recurrence-relation}, we get the following formula for $p_{\infty}(k)$:
\begin{equation}
	p_{\infty}(k) = \frac{1}{m+1} \left ( \frac{m}{m+1}\right )^{k-m}.
	\label{eq:p-infinity-solution-ra}
\end{equation}

For normalization, we need to check that
\begin{equation}
	\sum_{k=m}^{\infty}p_{\infty}(k) = \frac{1}{m+1} \sum_{k=m}^{\infty} \left ( \frac{m}{m+1}\right )^{k-m} = 1.
	\label{eq:ra-check-normalization}
\end{equation}
The terms in the summation form a converging geometric series, with the starting term being zero and common ratio being $m / (m+1)$. Hence we have
\begin{equation}
	\sum_{k=m}^{\infty} \left ( \frac{m}{m+1} \right )^{k-m} = \frac{1}{1 - [m / (m+1)]}.
	\label{eq:ra-geom-series}
\end{equation}

By substituting this back into the \autoref{eq:ra-check-normalization}, we can see that normalization is satisfied.

As we can see from \autoref{eq:p-infinity-solution-ra}, the resulting degree distribution in this limit is geometric \citep{Pekoz2013}, indicating that growth alone is not sufficient to produce a scale free structure.

\subsection{Degree distribution: Numerical analysis}\label{subsection:ra-numerical-analysis}
Numerical simulations confirmed that growth alone is not sufficient to produce a scale free structure. \autoref{fig:ra-fixed-n-degree-dist} shows the simulated raw degree distributions for $N=10^6$ and different $m$. The fat tail was reduced by taking the average of multiple simulations. Already, we can see that it does not follow a power law. After log binning, it follows the theoretical geometric simulation very well for small $k$, as can be seen in \autoref{fig:ra-fixed-n-logbin}.

\begin{figure}
    \centering
    \includegraphics[height=0.5\linewidth]{img/ra-fixed-n-degree-dist}
    \caption{Raw degree distribution for random attachment for $N = 10^6$ and $m = 1, 2, 4, 8, 16, 32$. }
    \label{fig:ra-fixed-n-degree-dist}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[height=0.5\linewidth]{img/ra-fixed-n-logbin}
    \caption{Data collapse of the degree distribution for networks of size $N=10^2, 10^3, 10^4, 10^5, 10^6, 10^7$}
    \label{fig:ra-fixed-n-logbin}
\end{figure}

Goodness of fit was tested in the same way as in the previous section. $p$-values obtained from testing against the null hypothesis of a geometric distribution governed by \autoref{eq:p-infinity-solution-ra} are listed below, showing that it is plausible that the simulated data follow the theoretical distribution. While the $p$-values fluctuate between 0.4 and 0.7, they are all safely above the threshold of $0.1$.

\begin{center}
\begin{tabular}{ c | c }
m & $p$-value \\
\hline
1  & 0.714 \\
2  & 0.446 \\
4  & 0.502 \\
8  & 0.706 \\
16 & 0.456 \\
\end{tabular}
\label{table:ra-ks-test}
\captionof{table}{The list of $p$-values for each $m$ for random attachment when compared with synthetic datasets.}
\end{center}

To demonstrate the test's effectiveness, random attachment simulation data was also tested against the power law distribution, and we obtained result of $p = 0$, to an accuracy of $\pm 0.05$, for all $m$. This shows that we can reject the power law hypothesis, as expected.

\subsection{Largest expected degree: Theory}\label{subsection:largest-expected-degree}
To calculate the largest expected degree, we need

\begin{equation}
	N \sum_{k=k_1}^\infty p_{\infty}(k) = N \sum_{k=k_1}^\infty \frac{1}{m+1} \left (\frac{m}{m+1} \right )^{k-m} = 1.
	\label{eq:largest-expected-degree-ra-criteria}
\end{equation}

The summation is similar to the previous \autoref{eq:ra-geom-series} with a different lower limit. Applying the geometric series summation formula to the terms in the summation, we get
\begin{equation}
	\frac{N}{m+1} \left ( \frac{m}{m+1} \right )^{k_1 - m} \frac{1}{1 - (m / (m+1))} = 1
\end{equation}

Rearranging this and taking logarithm of both sides, we obtain an expression for $k_1$:
\begin{equation}
	k_1 = \frac{\ln N}{\ln (m+1) - \ln m} + m
	\label{eq:largest-degree-ra}
\end{equation}
From this, we know that the largest degree grow logarithmically with $N$, as opposed to a power law like for preferential attachment.

\subsection{Largest expected degree: Numerical analysis}
The numerical largest degree was calculated in the same way as described in the previous section. The ratio between $k_1^{\text{theory}}$ and $k_1^{\text{numerical}}$ is largely constant. From \autoref{fig:ra-numerical-theoretical-k1} we can see that the difference between the numerical and theoretical values decrease as $N$ increases, as expected.

\begin{figure}
    \centering
    \includegraphics[height=0.5\linewidth]{img/ra-numerical-theoretical-k1}
    \caption{This shows the difference in $k_1$ for different values of $N$ ranging from $100$ to $10^7$ for random attachment. From the graph we can see that the difference between the theoretical and numerical values decrease as $N$ increases.}
    \label{fig:ra-numerical-theoretical-k1}
\end{figure}

From the following tables of values, we can also see that the values get closer to 1 for larger $N$. Similarly, the standard error was estimated from the sample standard deviation.

\begin{center}
\begin{tabular}{ ||c | c | c | c ||}
\hline
N & $k_1^{\text{theory}}$ & $k_1^{\text{numerical}}$ & $k_1^{\text{numerical}} / k_1^{\text{theory}} $\\
\hline
$10^2$ & 25    & 21.85  $\pm$ 0.01 & 0.887 \\
$10^3$ & 35   & 32.15 $\pm$ 0.03 & 0.920 \\
$10^4$ & 46   & 42.05   $\pm$  0.03  & 0.929 \\
$10^5$ & 56  & 52.75  $\pm$  0.02  & 0.949 \\
$10^6$ & 66  & 63.55  $\pm$  0.02  & 0.964 \\
$10^7$ & 76 & 73.55 $\pm$ 0.03  & 0.965 \\
\hline
\end{tabular}
\label{table:ra-numerical-theoretical-ratio}
\captionof{table}{This table shows the theoretical and numerical values for the largest expected degree as defined in \autoref{eq:largest-degree-ra} for random attachment. The errors on $k_1$ are rounded off to 1 significant figure. }
\end{center}


{"Hp2_version":2, "Hp2_stencil":"asst",
 "duedate": "Thursday 24 May 2018", "studentid":"200878696",
 "semester": "S1 2018", "course":"MATHS 326", "upi": "\\MakeLowercase{aelz176}",
 "asstno":"4", "Buc_tags": ["homework", "auckland", "mathematics", "combinatorics", "326","test"]}
===
\section*{Question One}
\paragraph{(a)}
The symmetry group of the bracelet is $ C_6 $.
\begin{center}\begin{tabular}{|c|c|c|}\hline
  \textbf{Cycle type} & \textbf{Count} & \textbf{Fixed colourings}\\\hline
  $[1^6]$ & 1 & $ 2^6 $\\
  $[2^3]$ & 1 & $ 2^3 $\\
  $[3^2]$ & 2 & $ 2^2 $\\
  $[6]$ & 2 & $ 2^1 $\\\hline
   & 6 &\\\hline
\end{tabular}\end{center}

So the number of orbits of $ C_6 $ on the colourings is
\begin{displaymath}
  \frac{1 \cdot 2^6 + 1 \cdot 2^3 + 2 \cdot 2^2 + 2 \cdot 2^1}{6} = 14.
\end{displaymath}

There are 14 distinct colourings up to rotation.

\paragraph{(b)}
If we also include reflections, then the symmetry group of the bracelet is $ D_6 $ (of order 12).
\begin{center}\begin{tabular}{|c|c|c|}\hline
  \textbf{Cycle type} & \textbf{Count} & \textbf{Fixed colourings}\\\hline
  $[1^6]$ & 1 & $ 2^6 $\\
  $[2^3]$ & 1 & $ 2^3 $\\
  $[3^2]$ & 2 & $ 2^2 $\\
  $[6]$ & 2 & $ 2^1 $\\\hline
  $[1^2,2^2] $ & 3 & $ 2^4 $\\
  $[2^3] $ & 3 & $ 2^3 $\\\hline
   & 12 &\\\hline
\end{tabular}\end{center}

So the number of orbits of $ D_6 $ on the colourings is
\begin{displaymath}
  \frac{1 \cdot 2^6 + 1 \cdot 2^3 + 2 \cdot 2^2 + 2 \cdot 2^1 + 3 \cdot 2^4 + 3 \cdot 2^3}{12} = 13.
\end{displaymath}

There are 13 distinct colourings up to rotation and reflection.

\section*{Question Two}
The cube's symmetry group has order 24.
\begin{center}\begin{tabular}{|c|c|c|}\hline
  \textbf{Cycle type on edges} & \textbf{Count} & \textbf{Fixed colourings}\\\hline
  $[1^{12}]$ & 1 & $ 2^{12} $\\
  $[4^3]$ & 6 & $ 2^3 $\\
  $[2^6]$ & 3 & $ 2^6 $\\
  $[3^4]$ & 8 & $ 2^4 $\\
  $[1^2,2^5]$ & 6 & $ 2^7 $\\\hline
   & 24 &\\\hline
\end{tabular}\end{center}

So the number of orbits of the group on the edge colourings is
\begin{displaymath}
  \frac{1 \cdot 2^{12} + 6 \cdot 2^3 + 3 \cdot 2^6 + 8 \cdot 2^4 + 6 \cdot 2^7}{24} = 218.
\end{displaymath}

There are 218 distinct colourings up to rotation.

\section*{Question Thre}
\paragraph{(a)}
Labelling the vertices with the numbers 1 to 5, and writing the edge between vertex 1 and vertex 2 by 12, we have the following:
\begin{gather*}
  12 \mapsto 12\\
  13 \mapsto 24 \mapsto 15 \mapsto 23 \mapsto 14 \mapsto 25 \mapsto 13\\
  34 \mapsto 45 \mapsto 53.
\end{gather*}

\paragraph{(b)}
The symmetry group of $ K_5 $ is $ S_5 $. We have the following:
\begin{center}\begin{tabular}{|c|c|c|c|}\hline
  \textbf{Cycle type on vertices} & \textbf{Count} & \textbf{Cycle type on edges} &\textbf{Fixed colourings}\\\hline
  $ [1^5] $ & 1 & $ [1^{10}] $ & $ k^{10} $\\
  $ [1^3,2] $ & 10 & $ [1^4,2^3] $ & $ k^7 $\\
  $ [1^2,3] $ & 20 & $ [1,3^3] $ & $ k^4 $\\
  $ [1, 4] $ & 30 & $ [2,4^2] $ & $ k^3 $\\
  $ [1,2,2] $ & 15 & $ [1^2,4^2] $ & $ k^4 $\\
  $ [2,3] $ & 20 & $ [1,3,6] $ & $ k^3 $\\
  $ [5] $ & 24 & $ [5^2] $ & $ k^2 $\\\hline
  & 120 &&\\\hline
\end{tabular}\end{center}

Hence the cycle index of the symmetry group is
\begin{displaymath}
  \frac{1}{120}(a_1^{10} + 10a_1^2 a_2^3 + 20a_1 a_3^3 + 30a_2 a_4^2 + 15a_1^2 a_4^2 + 20 a_1 a_3 a_6 + 24a_5^2).
\end{displaymath}

\paragraph{(c)}
Counting the number of non-isomorphic graphs on five vertices is equivalent to counting the number of 2-colourings of $ K^5 $.
Hence we need only calculate
\begin{displaymath}
  \frac{1}{120}(2^{10} + 10(2^5) + 20(2^4) + 30(2^3) + 15(2^4) + 20(2^3) + 24(2^2)) = 20.
\end{displaymath}

There are 20 distinct graphs on five vertices up to isomorphism.

\section*{Question Four}
The symmetry group of the pentagonal prism is $ C_2 \times C_5 $.
\begin{center}\begin{tabular}{|c|c|c|}\hline
  \textbf{Cycle type} & \textbf{Count} & \textbf{Cycle monomial}\\\hline
  $ [1^7] $ & 1 & $ a_1^7 $\\
  $ [1^2,5] $ & 4 & $ a_1^2 a_5 $\\
  $ [1^5,2] $ & 1 & $ a_1^5 a_2 $\\
  $ [2,5] $ & 4 & $ a_2 a_5 $\\\hline
   & 10 &\\\hline
\end{tabular}\end{center}

So the cycle index on the face colourings is
\begin{displaymath}
  \frac{1}{10}(a_1^7 + 4a_1^2 a_5 + a_1^5 a_2 + 4a_2 a_5).
\end{displaymath}
Let $ x $, $ y $, and $ z $ be the number of green, blue, and red edges respectively; so the generating function for
the colorings of a particular edge is $ f(x,y,z) = x + y + z $. Hence, we want the coefficient of $ x^2 y^2 z^2 $ in
\begin{displaymath}
  \begin{split}
    \frac{1}{10}((x + y + z)^7 + 4(x + y + z)^2 (x^5 + y^5 + z^5) &+ (x + y + z)^5 (x^2 + y^2 + z^2)\\
    &{}+ 4(x^2 + y^2 + z^2)(x^5 + y^5 + z^5)).
  \end{split}
\end{displaymath}
This coefficient is
\begin{displaymath}
  \frac{1}{10}\left(\frac{7!}{2!2!2!} + 0 + 3 \cdot \frac{5!}{2!2!0!} + 0\right) = 72.
\end{displaymath}

There are 72 distinct colourings such that each colour appears precisely twice, up to rotation.

\section*{Question Five}
The symmetry group of the dodecahedron has order sixty.
\begin{center}\begin{tabular}{|c|c|c|}\hline
  \textbf{Cycle type} & \textbf{Count} & \textbf{Cycle monomial}\\\hline
  $ [1^{12}] $ & 1 & $ a_1^{12} $\\
  $ [1^2,5^2] $ & 24 & $ a_1^2a_5^2 $\\
  $ [2^6] $ & 15 & $ a_2^6 $\\
  $ [3^4] $ & 20 & $ a_3^4 $\\\hline
   & 60 &\\\hline
\end{tabular}\end{center}

So the cycle index on the face colourings is
\begin{displaymath}
  \frac{1}{60}(a_1^{12} + 24a_1^2a_5^2 + 15a_2^6 + 20a_3^4).
\end{displaymath}
Let $ x $, $ y $, and $ z $ be the number of red, green, and white faces respectively; so the generating function for
the colorings of a particular face is $ f(x,y,z) = x + y + z $. Hence, we want the coefficient of $ x^3y^3z^6 $ in
\begin{displaymath}
  \frac{1}{60}\left((x+y+z)^{12} + 24(x+y+z)^2(x^5+y^5+z^5)^2 + 15(x^2+y^2+z^2)^6 + 20(x^3+y^3+z^3)^4\right).
\end{displaymath}
This coefficient is
\begin{displaymath}
  \frac{1}{60}\left(\frac{12!}{3!3!6!} + 0 + 0 + 20\cdot\binom{4}{1}\cdot\binom{3}{1}\cdot\binom{2}{2}\right) = 312.
\end{displaymath}