\chapter{Sparse Component Analysis, data restoration and inpainting on the sphere}
\label{ch_sca_datarest}

\section{Morphological Component Analysis on the sphere}

A usual task  in processing signals, images as well as spherical data maps, is to decompose the data
into its elementary building blocks. This can be formulated as an inverse problem where the data is
assumed to have been generated according to the following model :
\begin{equation}
 y = \sum_{i} \alpha_i \phi_i + \eta
\end{equation}
that is a linear combination of relevant waveforms $\phi_i \in \mathbb{R}^n$ with weights $\alpha_i$.
Here $\eta$ represents possible contamination by additive, typically Gaussian white noise. Given data
$y\in \mathbb{R}^n$, one then wants to recover the underlying structures that is to say estimate a set
of waveforms $\phi_i$ that build the data and their corresponding weights $\tilde{\alpha}_i$. The solution
to this estimation problem will depend heavily on the available prior information. Of interest here is
the case where one is given \emph{a priori} a set a waveforms from which to select a good subset. This set
may be a basis, a frame or several bases or frames grouped into a large redundant dictionary.\\
%
Possible dictionaries in 1D and 2D include Fourier and related bases, wavelet bases, as well as
other more recent multiscale systems such as the ridgelet~\cite{cur:candes99_1} and curvelet
frames~\cite{cur:donoho99,starck:sta01_3}, etc. Depending on the morphology of the data, each of
these dictionaries will have different performance characteristics
%perform more or less efficiently
in a non-linear approximation scheme. For instance, sparse approximations of piecewise smooth
signals or images with point singularities are easily obtained using wavelets. However these
are no longer optimal in the case of piecewise smooth images with singularities along smooth
curves or edges. Such images are more efficiently approximated using curvelets which are highly
anisotropic and thus exhibit high directional selectivity. Digital implementations of both ridgelet
and curvelet transforms and their application to image denoising are described in~\cite{starck:sta01_3}.\\
%
Available transforms in the spherical topology include the spherical harmonics and several
wavelet transforms. Software packages such as Healpix\footnote {http://www.eso.org/science/healpix}~\cite{pixel:healpix}
or Glesp~\cite{pixel:glesp} provide approximate digital spherical harmonic transform routines
based on their specific pixelization schemes. Schr{\"o}der and Sweldens~\cite{wave:sweldens95a}
have developed an orthogonal wavelet transform on the sphere based on the Haar wavelet function
which then suffers from the poor frequency domain localization properties of the primitive Haar function
%properties of the Haar function
and from the problems inherent in orthogonal decomposition (\emph{e.g.} lack of translation invariance).
A few papers describe continuous wavelet transforms on the sphere~\cite{wave:antoine99bis,wave:cayon01,wave:holschneider96,wave:wiaux,bogdanova}
which have been extended to directional wavelet transforms~\cite{wave:wiaux2005,wave:hobson04}.
Although useful for data analysis, these continuous transforms lack an inverse transform and hence
are clearly not suitable for restoration or synthesis purposes.\\
%
In their pioneering  work, Freeden and Maier~\cite{freeden02,freeden03} gave a wavelet transform
and reconstruction scheme on the sphere which is based on the spherical harmonic transform.
Following this idea, Starck \emph{et al.}~\cite{starck:sta05_2} have proposed a new invertible
isotropic undecimated wavelet transform (UWT) on the sphere which preserves the same desirable
properties as the standard isotropic UWT for flat 2D maps~\cite{starck:book98}: the reconstruction
is simple and immediate since it is just the addition of all the wavelet bands with the coarsest scale.
Based on this new decomposition, other multiscale transforms such as the pyramidal wavelet transform,
the ridgelet transform and the curvelet transform have been successfully constructed on the sphere \cite{starck:sta05_2}.
%
Each of these decompositions on the sphere will sparsely represent parts of the image based
on their morphological properties. Wavelets will easily detect more or less isotropic localized
structures, while curvelets are better suited for efficiently detecting highly anisotropic objects.\\
%
% ,  based on the spherical harmonics decomposition, is to our knowledge the only one to have an inverse transform. This set of spherical transforms has recently been enriched with a new invertible  isotropic undecimated wavelet transform~\cite{starck:sta05_2} the properties of which are similar to those of the \emph{\`a trous} algorithm and with digital ridgelet and curvelet transforms on the sphere. These tools make it possible to detect both point singularities and  edges in spherical maps.
%
%
A data set $y$ has an exact representation over any complete basis of the data space, or several
such exact representations in the case of redundant overcomplete dictionaries. However, these
representations are not equally interesting in terms of data modeling or feature detection. In fact,
a strong \emph{a priori} is to favor representations of $y$ that use only a small number of waveforms
leading to a more concise and possibly more interpretable representation of the data. In fact,
building sparse representations or approximations is the (he)art of structured data processing:
the design of good detection, denoising, restoration and compression algorithms relies on the
availability of good dictionaries and good selection algorithms. Indeed, selecting the smallest
subset of waveforms from a large dictionary, that will linearly combine to reproduce the salient
features of a given signal or image, is a hard combinatorial problem. Several \emph{pursuit} algorithms
have been proposed that can help build very sparse decompositions such as the greedy Matching Pursuit (MP)~\cite{wave:mallat93}
algorithm which refines the signal approximation by picking at each iteration the one waveform
which best correlates with the current approximation error. Basis Pursuit (BP) ~\cite{wave:donoho98}
is a global procedure which seeks an approximation $\tilde{y}$ to $y$ by solving the linear programming problem:
\begin{equation}
\min_{ \alpha } ~ \|\alpha\|_{{ \ell_1}} \mbox{ subject to } y = \Phi \alpha.
\label{eqn_bp}
\end{equation}
where the ${\ell_1}$ norm measures sparsity in place of the ${\ell_0}$ counting norm.
%
In the presence of noise, a noise-aware variant of BP, known as BPDN (for BP denoising), can be
stated as a convex quadratic programming problem and solved using the Interior Point method
\cite{wave:donoho98}. The BPDN problem can also be written in the augmented Lagrangian form:
\begin{equation}
\min_{ \alpha } ~  \|y - \Phi\alpha\|_{{ \ell_2}} ^2 + \lambda \cdot \|\alpha\|_{ \ell_1}
\label{eqn_mp}
\end{equation}
Among all possible solutions, the chosen one has the minimum ${\ell_1}$ norm. This choice of
$\ell_1$ norm is very important. An $\ell_2$ norm, as used in the method of frames \cite{wave:daube88b},
does not favor sparsity \cite{wave:donoho98}. A number of recent results prove that these
algorithms will recover the unique maximally sparse decomposition provided this solution is
sparse enough and the dictionnary is sufficiently incoherent~\cite{Donoho-Elad,cur:elad02,miki:Gribonval-Nielsen,miki:Temlyakov,miki:fuchs}.
Nevertheless, in problems involving large data sets~(\emph{e.g.} images, spherical maps),
BP or MP synthesis algorithms are computationally prohibitive.
%
Morphological Component Analysis (MCA) is a recent faster alternative described in~\cite{starck:sta04}
%\cite{starck:elad05,starck:sta04_1,starck:sta04,SPIE2005a}
that  constructs a sparse representation of a signal or an image assuming that it is a combination
of morphologically distinct features which are sparsely represented in different dictionaries
associated with fast transform algorithms. For instance, images commonly combine contours and textures:
the former are well accounted for using curvelets, while the latter may be well represented using
local cosine functions. In searching for a sparse decomposition of a signal or image $y$, it is
assumed that $y$ is a sum of $K$ components $ (s_k)_{1,\ldots,K}$, where each can be described as
$s_k = {\bf \Phi}_k \alpha_k$ with a possibly over-complete dictionary ${\bf \Phi}_k$ and a sparse
vector of coefficients $\alpha_k$. It is further assumed that for any given component the sparsest
decomposition over the proper dictionary yields a highly sparse description, while its decomposition
over the other dictionaries, ${\bf \Phi}_{k'\ne k}$, is non sparse. Thus, the different ${\bf \Phi}_k$
can be seen as discriminating between the different components of the initial signal. MCA achieves its
sparse decomposition relying on an iterative thresholding algorithm with a successively decreasing
threshold~\cite{starck:bobin06_tip} thus refining the current approximation by including finer structures
alternatingly in the different morphological components. Based on MCA, it has also been shown that we can
derive a very efficient inpainting method \cite{starck:elad05}.\\
%
{\em This paper: } Motivated by the success of MCA in signal and image processing, the purpose of this contribution
is to take advantage of the variety of transforms on the sphere recently made available~\cite{starck:sta05_2} to extend
the applicability of MCA to the analysis of spherical maps which are commonly recorded in a number of areas such as
geophysics, astrophysics or medical imaging. As in the case of Euclidean 2D images, we further extend the MCA algorithm
on the sphere in order to perform inpainting tasks on the sphere. The proposed numerical tools are shown to be valuable
in several selected applications in physics and astrophysics. The construction of the undecimated isotropic wavelet and
curvelet transforms on the sphere is reviewed in the next section. Sections~\ref{sect_mca} and~\ref{sect_inpaint} describe
the extension to the sphere of the MCA algorithm and of its modification for inpainting purposes.

\section{MCA on the Sphere}
\label{sect_mca}
%---------------------------------------------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------------------------------------------
%a short introduction to this section?
%%---------------------------------------------------------------------------------------------------------------------------------------
%	\subsection{Introduction}
%%---------------------------------------------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------------------------------------------
 \subsection{Principle and algorithm}
 %---------------------------------------------------------------------------------------------------------------------------------------
For a given spherical map $y$ modeled as a linear { combination} of $K$ spherical maps $s_k$, $y = \sum_{k=1}^K s_k$,
having different morphologies, MCA assumes that a dictionary of bases $\{ {\bf \Phi}_1,\cdots,{\bf \Phi}_K\}$ exists
such that, for each $ ~ k$, $ s_k$ is sparse in $ ~ {\bf \Phi}_k$ while its representation in the other
$ ~ {\bf \Phi}_{k'}$ ($ ~ k' \ne k$) is not sparse : $ ~ \forall k' \neq k, ||{\bf \Phi}_k^T s_k||_0 < ||{\bf \Phi}_{k'}^T s_k||_0$,
where $||x||_0$ denotes the $\ell_0$ pseudo-norm of the vector $x$
%  (\textit{i.e.}  the number of non-zero coefficients of $x$)
The problem is to separate the mixture $y$ into its constitutive morphological components $ (s_k)_{k=1,\cdots,K}$ relying
on the discriminating power of the different dictionaries ${\bf \Phi}_k$. Ideally, the $\alpha_k$ are the solutions { to} :
\begin{equation}\label{eq:Separation1}
 ~
\min_{ {\alpha}_1, \dots, ~{\alpha}_K }~\sum_{k=1}^K\|{\alpha}_k\|_0 \quad \mbox{subject to } \quad y= \sum_{k=1}^K  {\bf \Phi}_k {\alpha}_k
\end{equation}
While sensible from the point of view of the desired solution, the problem formulated in Equation (\ref{eq:Separation1})
is non-convex and combinatorial by nature. Its complexity grows exponentially with the number of columns in the overall
dictionary { (NP-hard problem)}. Motivated by recent equivalence results \emph{e.g.} in~\cite{Donoho-Elad}, the MCA
algorithm seeks a solution to the following minimization problem:
\begin{equation}\label{mca:model1}
 ~
\min_{s_1,\ldots,s_K} \lambda \sum_{k=1}^K  \|\alpha_k\|_1 + \left\|y-\sum_{k=1}^K s_k\right\|_2^2\,\,\,\textrm{with}\,\,\,s_k ={\bf \Phi}_k \alpha_k
\end{equation}
where an ${ \ell_1}$ sparsity measure is substituted to the $\ell_0$ counting norm following a prescription of
the Basis Pursuit algorithm \cite{wave:donoho98}. In the above, the equality constraint was relaxed and again
$ ~ s_k = {\bf \Phi}_k \alpha_k $. In the case where each ${\bf \Phi}_k$ is an orthonormal basis,
{ a block-coordinate solution to the above problem is given by} the following set of coupled equations:
\begin{equation}
 ~
\forall k, s_k = r_k - \frac{\lambda_k}{2 }  {\bf \Phi}_k  \mbox{sign}( {\bf \Phi}_k^{T} s_k ) \mbox{  with  } r_k = s - \sum_{k' \neq k} s_{k'}
\end{equation}
This can be solved efficiently using the iterative \textit{Block-Coordinate Relaxation Method} \cite{text:Bruce98}
in conjunction with, at a given $k$, a soft-thresholding of the decomposition of $r_k$ over ${\bf {\bf \Phi}_k}$.
However, when non-unitary or redundant transforms are used, the above is no longer strictly valid. Nevertheless,
simple shrinkage still gives satisfactory results as explained in~\cite{miki:shrinkage}. Finally, denoting by
${\bf T}_k$ and ${\bf R}_k$ the forward and inverse transforms associated with the redundant dictionary
${\bf {\bf \Phi}_k}$, MCA seeks a solution to problem~(\ref{mca:model1}) with the following algorithm:
\begin{center}
\begin{minipage}[b]{0.9\linewidth}
\vspace{0.1in}
\footnotesize{\textsf{1. Set the number of iterations $I_{\max}$ and the initial thresholds $ ~ \left(\lambda_k^{(0)}\right)_{k}$}

\textsf{2. While  $\lambda^{(t)}_{k}$ is greater than a given lower bound $\lambda_{\min}$ (e.g. can depend on the noise standard deviation), }

\hspace{0.15in} \textsf{-- Proceed with the following iteration to estimate components $ (s_k)_{k=1,\ldots,K}$ at iteration $t$:}

\hspace{0.25in} \textsf{For $ ~ k=1,\cdots,K$ }

\hspace{0.5in} \textsf{$\bullet$ Compute the residual term $ ~ r_k^{(t)}$ assuming the current estimates $\tilde{s}_{k' \neq k}^{(t-1)}$ of $ ~ s_{k' \neq k}$, are
fixed:}

\hspace{0.75in} \textsf{ $ ~ r_k^{(t)} = y - \sum_{k' \neq k} \tilde{s}_{k'}^{(t-1)}$}

\hspace{0.5in} \textsf{$\bullet$ Estimate the current coefficients of $ ~ \tilde{s}_k^{(t)}$ by thresholding with threshold $ ~ \lambda_k^{(t)}$:}

\hspace{0.75in} \textsf{$ \tilde{\alpha}_k^{(t)} = \delta_{\lambda_k^{(t)}}\left( {\bf T}_k r_k^{(t)}\right)$}

\hspace{0.5in} \textsf{$\bullet$ Get the new estimate of $ s_k$ by reconstructing from the selected coeffcients $ \tilde{\alpha}_k^{(t)}$ :}

\hspace{0.75in} \textsf{$ ~ \tilde{s}_k^{(t)} = {\bf R}_k \tilde{\alpha}_k^{(t)}$}

\hspace{0.15in} \textsf{-- Decrease the thresholds $\lambda_{k}$ following a given strategy.}
}
\vspace{0.05in}
\end{minipage}
\end{center}
%---------------------------------------------------------------------------------------------------------------------------------------
	\subsection{Thresholding strategy}
%---------------------------------------------------------------------------------------------------------------------------------------
The operator $\delta$ in the above algorithm is a soft thresholding operator as a result of the use of an
${ \ell_1}$ sparsity measure in approximation to the ideal $ ~ \ell_0$ norm. In practice, hard thresholding
leads generally to better results~\cite{starck:sta04}. The final threshold should vanish in the { noiseless}
case or it may be set to a multiple of the noise standard deviation in the presence of noise as in common
detection or denoising methods. The way the threshold is decreased along the iterations of the proposed
iterative thresholding scheme is paramount in terms of performance of the MCA separation mechanism.
The original algorithm~\cite{starck:sta04} used a linear strategy :
\begin{equation}
\lambda^{(t)} = \lambda^{(0)} - (t-1)\frac{\lambda^{(0)} -\lambda_{\min}}{I_{\max}-1}
\end{equation}
where $\lambda^{(0)}$ is the initial threshold, and $I_{\max}$ is the number of iterations. The first threshold
can be set automatically to a large enough value such as the maximum of all coefficients { $\lambda^{(0)}=\max_k~\|{\bf T}_k y\|_\infty$}.
But there is no way to estimate the minimum number of iterations yielding a successful separation. Too small a number
of iterations leads to bad separation while too large a number is computationally costly. Further, experiments have
clearly shown that the optimal number of iterations depends on the data. We recently focused on devising some new data
adaptive thresholding strategies to speed up the MCA decomposition preserving the quality of the component separation.
Hereafter we describe two promising strategies, namely MAD and MOM, in the case where $K = 2$ ;
generalizing to $K\geq2$ is straightforward.
%~\cite{starck:bobin06_tip,starck:dave06}.
\paragraph{MAD} Consider a map $y$ such that $ ~ y = s_1 + s_2 = {\bf \Phi}_1\alpha_1 + {\bf \Phi}_2\alpha_2$
where $ ~ s_1$ and $ ~ s_2$ have similar ${ \ell_2}$ norm and $ \alpha_{k=1,2} = {\bf \Phi}_{k=1,2}^T s_{k=1,2}$ are sparse.
When both $ ~ {\bf \Phi}_{k=1,2}$ are orthonormal bases, decomposing $y$ in ${\bf \Phi}_{1}$ leads to
$ y {\bf \Phi}_{1}^T = \alpha_1 + {\bf \Phi}_1^T{\bf \Phi}_2\alpha_2 $. Provided the mutual coherence~\cite{Elad-Bruckstein,miki:Gribonval-Nielsen,Donoho-Elad}
of ${\bf \Phi}_1$ and ${\bf \Phi}_2$ is low, $y_2$ has no particular structure in ${\bf \Phi}_{1}$ and hence
it is tempting to model $ ~ {\bf \Phi}_1^Ts_2$ as a Gaussian \emph{noise}. Its { standard deviation} can be
estimated using a robust estimator such as the Median Absolute Deviation (MAD)~\cite{rest:donoho93_1}. It follows that
estimating the significant entries $\tilde{\alpha_1}$ in $\alpha_1$ is a denoising problem readily solved
by thresholding ${\bf \Phi}_{1}^T y$ with { a} threshold $k\sigma$ { (typically $k$ is in the range 3 to 4)}.
The next step is to project the residual $ ~†y - \tilde{s_1} = y - {\bf \Phi}_{1}\tilde{\alpha_1}$ on ${\bf \Phi}_2$ and so on.
Clearly, the variance of the residual decreases along iterations and so this provides a simple strategy to adaptively
control the threshold in the MCA algorithm. In practice, this strategy remains fruitful in the case of redundant dictionaries.
Donoho \emph{et al.} in \cite{starck:dave06} have recently focused on an iterative thresholding scheme applied to solving
under-determined linear sparse problems in which they use a similar rule to manage their decreasing threshold.
%
\paragraph{MOM}  Let $ ~ \tilde{s}_1^{(t)}$ and $ ~ \tilde{s}_2^{(t)}$ denote the current estimates of
components $ ~ s_1$ and $ ~ s_2$ at the $t^{th}$ iteration of the MCA decomposition of $y$. The current
residual is $ ~ r^{(t)} = y -  \tilde{s}_1^{(t)} - \tilde{s}_2^{(t)} $. In the strategy coined MOM as in
"Mean of Max", the value of the threshold at iteration $ ~ t$ is given by :
\begin{equation}
\label{MOM_threshold}
 ~
\lambda^{(t)} = \frac{1}{2}\left[ ||{\bf \Phi}_1^T\left( y- \tilde{s}_1^{(t-1)} - \tilde{s}_2^{(t-1)} \right)||_\infty + ||{\bf \Phi}_2^T\left( y- \tilde{s}_1^{(t-1)} - \tilde{s}_2^{(t-1)} \right)||_\infty\right]
\end{equation}
which is easily computed at each step of the iterative process. When one considers more than two dictionaries,
one should take the mean of the two largest decomposition coefficients of the full residual over two distinct dictionaries.
The intuition underlying this strategy is that the next significant coefficients to be selected should be attached
to the dictionary in which the projection of the full residual has coefficients of largest amplitudes. Assuming the
coefficients selected at iteration $ ~ t$ are in ${\bf \Phi}_1$, it can be shown, under some conditions on the sparsity
of the components and the mutual coherence of the dictionary~\cite{starck:bobin06_tip}, that the proposed strategy
fixes the threshold so that :
\begin{equation}
\label{intuitive_choice}
||{\bf \Phi}_1^T{\bf \Phi}_2\bar{\alpha}_2^{(t-1)} ||_\infty < \lambda_1^{(t)} < ||\bar{\alpha}_1^{(t-1)} ||_\infty, \bar{\alpha}_{k=1,2}^{(t-1)}=\alpha_{k=1,2}-\tilde{\alpha}_{k=1,2}^{(t-1)}
\end{equation}
hence avoiding false detections (upper bound) and ensuring that at least one coefficient is selected (lower bound).
This thresholding strategy can easily be made more or less conservative depending on the desired decomposition speed.
With these new thresholding strategies, MCA is a fast and robust algorithm to achieve sparse decompositions in redundant
dictionaries and a practical alternative to other well-known sparse decomposition algorithms~\cite{starck:bobin06_tip}.
\subsection*{Example}
\begin{figure}
\vbox{
\centerline{
\hbox{
%\psfig{figure=test_gauss_alm_bw.ps,bbllx=0.cm,bblly=6.5cm,bburx=22.cm,bbury=21.5cm,height=6.5cm,width=8cm,clip=}
\psfig{figure=test_gauss_alm_bw.ps,bbllx=0.cm,bblly=8.5cm,bburx=22.cm,bbury=19.5cm,height=4cm,width=6.5cm,clip=}
}
}
\centerline{
\hbox{
%\psfig{figure=test_gauss_alm_mca_alm_bw.ps,bbllx=0.cm,bblly=8.5cm,bburx=22cm,bbury=19.5cm,height=4.5cm,width=8cm,clip=}
%\psfig{figure=test_gauss_alm_mca_wt_bw.ps,bbllx=0.cm,bblly=8.5cm,bburx=22cm,bbury=19.5cm,height=4.5cm,width=8cm,clip=}
\psfig{figure=test_gauss_alm_mca_alm_bw.ps,bbllx=0.cm,bblly=8.5cm,bburx=22cm,bbury=19.5cm,height=4cm,width=6.5cm,clip=}
\psfig{figure=test_gauss_alm_mca_wt_bw.ps,bbllx=0.cm,bblly=8.5cm,bburx=22cm,bbury=19.5cm,height=4cm,width=6.5cm,clip=}
}
}
}
\caption{ Simple toy experiment with MCA on the sphere - The top map shows a linear combination of a spherical harmonic function and a localized Gaussian-like function on the sphere. The bottom maps show the resulting separated components that were obtained using the proposed Morphological Component Analysis on the sphere.}
\label{Figure:mcatoy}
\end{figure}
The spherical maps shown on { figure~\ref{Figure:mcatoy}} illustrate a simple numerical experiment.
We applied the proposed Morphological Component Analysis on the Sphere to synthetic data resulting
from the linear mixture of components respectively sparse in the spherical harmonics and the isotropic
wavelet representations. The method was able to separate the data back into its original constituents.
A more involved application is described in the next section.
%---------------------------------------------------------------------------------------------------------------------------------------
\subsection{Application in Physics}
\label{section:bedros}
In Inertial Confinement Fusion (ICF) a spherical shell is irradiated by laser energy directly or after the
laser energy has been converted to soft X-rays~\cite{bedros:atzeni}. Either way, the aim is to implode the
capsule which contains a shell of nuclear fusion fuel (deuterium and tritium) ready to ignite if, after it
has been imploded, its density is high enough and a hot spot in its center becomes hot enough to cause a
propagating nuclear burn wave to travel through the rest of the fuel. This ultimate energy source will not
work if during the implosion hydrodynamic instabilities develop which can break apart the shell before it
assembles at the center and a hot spot forms~\cite{bedros:lindl}. Hydrodynamic instabilities such as Rayleigh-Taylor
occur due to nonuniformities in the laser spatial profile or imperfections in the composition of multiple
surfaces which make up the layers of thin material that surround the nuclear  fuel. Very small amplitude imperfections
initially can result in the ultimate failure of the target due to the large compression ratios involved in ICF.
\begin{figure}
\centerline{
\hbox{
\psfig{figure=bed_mca_orig_data.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
\psfig{figure=bed_mca_large_scale.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
}
}
\caption{ \textbf{left : }Surface structures of ICF spherical shells measured on the nanometer scale are a superposition of global scale variations, isolated bumps and scratches as well as artifacts which look like interference patterns on intermediate scales. \textbf{right :} Coarsest scale of the undecimated isotropic wavelet transform of the surface measurements of an ICF target.}
\label{bedros_mca_data}
\end{figure}
It is therefore extremely important to characterize the inner and outer surfaces of ICF shell targets
so as to know whether they are worthy of consideration for ICF implosions. One day in a reactor setting
tens of thousands of targets will have to be imploded daily so that checking each one is totally
out of the question. Instead, very good target fabrication quality control processes have to be adopted so that
confidence levels in proper performance will be high. A major step along this path to fusion energy then is to
understand why imperfections occur and to correct the systematic elements and control the harm done by random sources.
%
Fine structures on the surfaces of spherical shells can be measured on the nanometer scale, among others,
by atomic force microscopy or phase shifting spherical diffractive optical interferometry. An example of
such measurements is shown on figure~\ref{bedros_mca_data}. As can be seen from the figure, there appears
to be a superposition of global scale variations, isolated bumps and scratches as well as artifacts which
look like interference patterns on intermediate scales of localization. The latter must be isolated and
eliminated from consideration when deciding the readiness of the target for implosion. We have achieved
the morphological feature separation by first doing an isotropic wavelet transform on the spherical data
and subtracting the coarsest scale information. MCA on the sphere was used on the rest of the image using
the undecimated wavelet and the local cosine transforms on the sphere. The isolated bumps were thus identified
and the measurement technique caused artifacts were removed easily. The resulting bumps added to the coarsest scale,
is the clean data with the interference patterns and artifacts removed as shown in figure~\ref{bedros_mca_result}.
The spherical harmonic decomposition of the cleaned image gives rise to coefficients of various $\ell$ modes
which will be amplified by the implosion process which can now be assessed correctly using numerical hydrodynamics
simulation generated growth factors. If the bumps are clustered and not randomly distributed, then systematic errors
in the manufacturing process can be tracked down. A code called MODEM has been put together to study such
target surface data and extract the localized bump statistics including their correlations in height,
size and relative location. For more details see~\cite{bedros:modem}.
\begin{figure}
\vbox{
\centerline{
\hbox{
\psfig{figure=bed_mca_in_data.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
}
}
\centerline{
\hbox{
\psfig{figure=bed_mca_out_dct.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
\psfig{figure=bed_mca_out_wt.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
}
}
}
\caption{\textbf{top :} Spherical map obtained by subtracting the coarse scale map on the right of figure~\ref{bedros_mca_data} from the initial map on the left of figure~\ref{bedros_mca_data}. \textbf{ bottom :} Component maps separated by the MCA method on the sphere : interference patterns and measurement artifacts were grabbed by the local cosine functions on the sphere (left) while  the isolated bumps were caught using the undecimated wavelet on the sphere (right). Adding back the coarse scale on the right of figure~\ref{bedros_mca_data} to the latter map results in a clean map of the surface structures of an ICF spherical shell with the interference patterns and artifacts removed.}
\label{bedros_mca_result}
\end{figure}
%---------------------------------------------------------------------------------------------------------------------------------------
\section{Inpainting on the Sphere}
\label{sect_inpaint}
%---------------------------------------------------------------------------------------------------------------------------------------
\subsection{Algorithm}
%---------------------------------------------------------------------------------------------------------------------------------------
Named after the expert recovery process used for the restoration of deteriorated masterpieces, inpainting refers to a set
of techniques used to alter images in a way that is undetectable to people who are unaware of the original images. There are
numerous applications among which removing scratches or objects in digitized photographs, removing overlayed text or graphics,
filling-in missing blocks in unreliably transmitted images, predicting values in images for better compression or image upsampling.
Inpainting algorithms strive to interpolate through the gaps in the image relying on the available pixels, the continuation of edges,
the periodicity of textures, etc. The preservation of edges and texture, in other words discontinuities, across gaps has attracted
much interest, and many contributions have been proposed to solve this interpolation task. Non-texture image inpainting has received
considerable interest and excitement since the pioneering paper by Masnou and Morel \cite{Masnou98, Masnou02} who proposed variational
principles for image disocclusion. A recent wave of interest in inpainting has started from the recent contributions of Sapiro
\emph{et al.}~\cite{text:sapiro1,text:sapiro2,text:sapiro3}, followed by Chan and Shen~\cite{text:chan1}. In these works, authors point
to the importance of geometry and design anisotropic diffusion PDEs to fill in gaps by smooth continuation of isophotes.
PDE methods have been shown to perform well on piecewise smooth functions.
%
A very different approach is the inpainting algorithm based on MCA described in~\cite{starck:elad05} which has proved capable
of filling in holes in either texture or cartoon content in 2D images. To make the link between building sparse representations
and inpainting, consider the effect of a rectangular gap on the set of Fourier coefficients of a monochromatic sinewave :
because of the non-locality of the Fourier basis functions it takes a large number of coefficients to account for the gap,
which is known as the Gibbs effect. Seeking a sparse representation of the incomplete sine-wave outside the gap, that is without
fitting the gap, enables the recovery of the complete monochromatic sinewave.
%
Following~\cite{starck:elad05}, an inpainting algorithm on the sphere is readily built from the Morphological Component Analysis
on the sphere described in the previous section. Consider a discrete spherical data map $y$ and a binary map $M$ such that ones
in $M$ indicate that the corresponding pixels in $y$ are valid data while zeros indicate invalid data. The objective function of
MCA~(eq.~\ref{mca:model1}) can be modified as follows :
 \begin{equation}\label{inp:model}
\min_{s_1,\ldots,s_n} \lambda \sum_{k=1}^K  \|\alpha_k\|_1 + \left\|M  \odot (y-\sum_{k=1}^K s_k) \right\|_2^2\,\,\,\textrm{with}\,\,\,s_k ={\bf \Phi}_k \alpha_k .
\end{equation}
where $\odot$ stands for entry-wise multiplication. Thus we are preventing the sparse model under construction from attempting to fit
the invalid data. Other constraints can be easily imposed on the interpolated sparse components. For instance, in~\cite{starck:elad05},
a total variation penalty is shown to enhance the recovery of piece-wise smooth components. Asking for the regularity across the gaps of
some localized statistics (~\emph{e.g.} enforcing that the empirical variance of a given inpainted sparse component be \emph{nearly equal}
outside and inside the masked areas) are other possible constraints. In practice, because of the lack of accuracy of some digital
transformations we used in the spherical topology, additional constraints, which may be relaxed close to convergence, were also found
useful in some cases to stabilize the described iterative algorithms.
%
It is proposed that a solution to the above minimization problem can be reached using the same iterative thresholding process as
in the MCA algorithm detailed in the previous section, with the only required modification consisting in \emph{masking} the full
residual using $M$ after each residual estimation. The MCA-inpainting algorithm is as follows :
\begin{center}
\begin{minipage}[b]{0.9\linewidth}
\vspace{0.1in}
\footnotesize{\textsf{1. Set the number of iterations $I_{\max}$ and the initial thresholds $ \lambda^{(0)} $}

\textsf{2. While  $ ~†\lambda^{(t)}_{k}$ is greater than a given lower bound $\lambda_{\min}$ (e.g. can depend on the noise standard deviation), }

\hspace{0.15in} \textsf{-- Proceed with the following iteration to estimate components $ (s_k)_{k=1,\ldots,K}$ at iteration $t$:}

\hspace{0.25in} \textsf{For $ ~ k=1,\cdots,K$ }

\hspace{0.5in} \textsf{$\bullet$ Compute the residual term $  ~ r^{(t)}$ :}

\hspace{0.75in} \textsf{ $  ~ r^{(t)} = y - \sum_{k} \tilde{s}_{k}^{(t-1)}$}

\hspace{0.5in} \textsf{$\bullet$ Estimate the current coefficients of $  ~ \tilde{s}_k^{(t)}$ by thresholding with threshold $ ~ \lambda_k^{(t)}$:}

\hspace{0.75in} \textsf{$  \tilde{\alpha}_k^{(t)} = \delta_{\lambda_k^{(t)}}\left( {\bf T}_k \left(M \odot  r^{(t)}+\tilde{s}_k^{(t-1)}\right)\right)$}

\hspace{0.5in} \textsf{$\bullet$ Get the new estimate of $s_k$ by reconstructing from the selected coeffcients $  \tilde{\alpha}_k^{(t)}$ :}

\hspace{0.75in} \textsf{$  ~ \tilde{s}_k^{(t)} = {\bf R}_k \tilde{\alpha}_k^{(t)}$}

\hspace{0.15in} \textsf{-- Decrease the thresholds $\lambda_{k}$ following a given strategy.}
}
\vspace{0.05in}
\end{minipage}
\end{center}
The different thresholding strategies described in the previous section can be used in the proposed MCA inpainting iterative thresholding algorithm.
%
\subsection*{Example}
\begin{figure*}
\vbox{
\centerline{
\hbox{
\psfig{figure=earth_ori.ps,bbllx=0.cm,bblly=8.8cm,bburx=22cm,bbury=19.75cm,height=3.6cm,width=6.3cm,clip=}
\psfig{figure=earth_mask.ps,bbllx=0.cm,bblly=8.8cm,bburx=22cm,bbury=19.75cm,height=3.6cm,width=6.3cm,clip=}
}
}
\centerline{
\hbox{
\psfig{figure=earth_recons.ps,bbllx=0.cm,bblly=8.8cm,bburx=22cm,bbury=19.75cm,height=3.6cm,width=6.3cm,clip=}
\psfig{figure=earth_diff.ps,bbllx=0.cm,bblly=8.8cm,bburx=22cm,bbury=19.75cm,height=3.6cm,width=6.3cm,clip=}
}
}
}
\caption{ Application of the proposed MCA-inpainting algorithm on the sphere. \textbf{top left :} original satellite view of the Earth (~$mean= 76.9$, $\sigma =  47.7$~).  \textbf{top right :} incomplete map retaining 40 percent of the original pixels.  \textbf{bottom left : } inpainted map.  \textbf{bottom right :} map of reconstruction errors (~$mean= 0.0$, $\sigma =  2.86$ empirically estimated from the reconstructed pixels only~).}
\label{Figure:mcaearth}
\end{figure*}
A simple numerical experiment is shown on figure~\ref{Figure:mcaearth}. Starting with a full satellite view
of the Earth\footnote{availbale from : http://www.nasa.gov/vision/earth/features/bmng\_gallery\_4.html},
an incomplete spherical map was obtained by randomly masking some of the pixels. In fact, as much as sixty percent
of the pixels were masked. Using both the spherical harmonics transform and the curvelet transform on the sphere
within the proposed MCA inpainting algorithm, it is possible to fill in the missing pixels in a visually undetectable way.
The residual map is shown at the bottom right of figure~\ref{Figure:mcaearth}.
%---------------------------------------------------------------------------------------------------------------------------------------
\subsection{Application in Astrophysics}
%---------------------------------------------------------------------------------------------------------------------------------------
A major issue in modern cosmology is the measurement and the statistical characterization (spatial power spectrum, Gaussianity)
of the slight fluctuations in the Cosmic Microwave Background radiation field. These are indeed strongly related to the cosmological
scenarios describing the properties and evolution of our Universe. Some 370 000 years after the 'Big Bang', when the temperature
of the Universe was around 3000~K, thermal energy was no longer sufficient to keep electrons and positively charged particles apart
so they combined. Photons were then set free in a nearly transparent Universe. Since the Universe further expanded, these photons
are now in the microwave range but they should still be distributed according to a Black Body emission law. Indeed, before recombination,
the Universe was a highly homogeneous opaque plasma in near thermal equilibrium in which photons and charged particles were highly interacting.
Hence the slight fluctuations in matter density from which such large scale structures as galaxies or clusters of galaxies have evolved,
are also imprinted on the distribution of photons.
\begin{figure}
\vbox{
\centerline{
\hbox{
\psfig{figure=ima_wmap3y.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
\psfig{figure=wmap3y_inp.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
}
}
}
\caption{ \textbf{left :} CMB data map provided by the WMAP team. Areas of significant foreground contamination in the galactic region and at the locations of strong radio point sources have been masked out. \textbf{right :} Map obtained by applying the proposed MCA-inpainting algorithm on the sphere to the former incomplete WMAP CMB data map.}
\label{Figure:cmb_wmap_inpainting}
\end{figure}
The Cosmic Microwave Background~(CMB) was first observed in 1965 by Penzias and Wilson confirming a prediction
made by Gamow in the late 1940's. But it was not until the early 1990's that evidence for small fluctuations
in the CMB sky could finally be found thanks to the observations made by COBE~\cite{astro:COBE}. This was confirmed
by several subsequent observations and recently by NASA's Wilkinson Microwave Anisotropie Probe\footnote{The WMAP data and mask we used here are available online at http://map.gsfc.nasa.gov/}.
Full-sky multi-spectral observations with unprecedented sensitivity and angular resolution are expected from
the ESA's PLANCK\footnote{http://astro.estec.esa.nl/Planck} mission, which is to be launched in 2008. The statistical
analysis of this data set will help set tighter bounds on major cosmological parameters.
%
\begin{figure}
\vbox{
\centerline{
\hbox{
\psfig{figure=tpjls_mask_kp0.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
\psfig{figure=wt_wmap_inp_scale_1.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
}
}

\centerline{
\hbox{
\psfig{figure=wt_wmap_inp_scale_2.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
\psfig{figure=wt_wmap_inp_scale_3.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
}
}

\centerline{
\hbox{
\psfig{figure=wt_wmap_inp_scale_4.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
\psfig{figure=wt_wmap_inp_scale_5.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
}
}
\centerline{
\hbox{
\psfig{figure=wt_wmap_inp_scale_6.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
\psfig{figure=wt_wmap_inp_scale_7.ps,bbllx=1.5cm,bblly=10cm,bburx=19cm,bbury=19.cm,height=4cm,width=6.5cm,clip=}
}
}
}
\caption{ \textbf{left :} Mask provided by the WMAP team. The dark blue pixels indicate areas of high level foreground contamination in the WMAP CMB data map.  \textbf{From top to bottom and left to right :} Maps of the wavelet decomposition on seven scales of the inpainted WMAP CMB map shown on the right of figure~\ref{Figure:cmb_wmap_inpainting}. From the visual point of view, the masked area cannot be distinguished anymore in the wavelet scales of the inpainted map.}
\label{Figure:cmb_scale_wmap_inpainting}
\end{figure}
There are nonetheless a few practical issues and notably that several other astrophysical sources also emit radiation
in the frequency range used for CMB observations~\cite{fb-rg99}. Separating back the observed mixtures into maps of
the different astrophysical contributions in order to isolate the CMB properly is a difficult inverse problem for which
methods and algorithms are being actively designed (see \emph{e.g.}~\cite{astro:2005MNRAS.364.1185P,starck:bobin06,starck:yassir05,wlens:pires06}
and references therein). The estimated spherical CMB maps will inevitably be contaminated by some level of residual
contributions, most significantly in the galactic region and at the locations of strong radio point sources. Therefore, it is
common practice to mask out that part of the data (\emph{e.g.} using the mask shown on figure~\ref{Figure:cmb_scale_wmap_inpainting}
upper left, provided by the WMAP team) in order to reliably assess the non-gaussianity of the CMB field through estimated
higher order statistics (\emph{e.g.} skewness, kurtosis ) in various representations (\emph{e.g. wavelet, curvelet, etc.})
\cite{starck:sta03_1,starck:jin05}. But the gaps in the data thus created need to be handled properly as the detection
of non-gaussianity in CMB would have a major scientific impact. The proposed MCA-inpainting on the sphere was used here
successfully to fill in the masked regions in order to restore the stationarity of the observed CMB field and lower the impact
of the incompleteness of the data set on the estimated measures of non-gaussianity or any other non-local statistical test.
The experiment was conducted on several simulations of full-sky Gaussian CMB maps. A typical CMB map (the CMB data map disclosed
by the WMAP consortium) is shown on figure~\ref{Figure:cmb_wmap_inpainting} along with the map obtained as a result of
the inpainting process allowing for a first visual assessment of the quality of the proposed method. Figure~\ref{Figure:cmb_scale_wmap_inpainting}
shows the wavelet decomposition of the inpainted map. We can see that the mask is not visible at all in the different scales.
Here we have applied the MCA-Inpainting algorithm with 200 iterations and a single transform which was the Spherical
Harmonic Decomposition. A more quantitative evaluation of the proposed inpainting algorithm is reported on figure ~\ref{Figure:mca_skew_kur}
where plots of the estimated measures of non-Gaussianity on both the original map and the inpainted map are given.
These reveal no significant discrepancy: we believe that the proposed method will help discriminate between truly non-Gaussian CMB
and non-Gaussianity related to the non-stationarity of incomplete maps. This will be further investigated in the future.
\begin{figure}
\vbox{
\centerline{
\hbox{
\psfig{figure=skewness.ps,bbllx=1cm,bblly=1cm,bburx=17cm,bbury=14cm,height=4.5cm,width=6.5cm,clip=}
\psfig{figure=kurtosis.ps,bbllx=1cm,bblly=1cm,bburx=17cm,bbury=14cm,height=4.5cm,width=6.5cm,clip=}
}
}
}
\caption{ Horizontally is the scale number increasing for lower frequencies. \textbf{Left :} skewness of the wavelet coefficients in a given scale of the original complete simulated spherical CMB map ($\times$) and of the inpainted map ($\lozenge $). \textbf{Right :} kurtosis of the wavelet coefficients in a given scale of the original complete simulated spherical CMB map ($\times$) and of the inpainted map ($\lozenge $). Error bars were estimated on a small set of fifteen simulated complete CMB maps.}
\label{Figure:mca_skew_kur}
\end{figure}

\section{Generalized Morphological Component Analysis on the sphere}

\label{sec:gmca}
\subsection{The GMCA model}
\label{sec:model}

The observation with detector $i$ is then a noisy linear mixture of $n$ independent sources $\{s_j\}_{j=1,\cdots,n}$ :
$x_i = \sum_{j=1}^n a_{ij} s_j + n_i$. The coefficient $a_{ij}$ reflects the emission law of source $s_j$ in the
frequency band of the $i$-th sensor; $n_i$ models instrumental noise. When $m$ sensors provide observations at
different frequencies, this linear mixture model can be rewritten in a more convenient matrix formulation :
\begin{equation}
\label{eq:lm_model}
{\bf X} = {\bf AS} + {\bf N}
\end{equation}
where ${\bf X}$ is the $m \times t$ data matrix the rows of which are the observed data maps in each channel, ${\bf A}$
is the $m \times n$ mixing matrix, ${\bf S}$ is the $n \times t$ source matrix the rows of which are the sources $s_j$,
and ${\bf N}$ is the $m \times t$ noise matrix.

We further assume that all the protagonists of the model in Equation~\ref{eq:lm_model} are random components (variables
or vectors). More particularly, the entries of the noise matrix ${\bf N}$ are assumed to be \textit{independently}
distributed according to a zero mean Gaussian distribution with variance $\sigma_i^2$ depending on the detector.
From physical considerations, ${\bf N}$ models instrumental noise the level of which varies independently from one
detector to another. ${\bf N}$ is thus a random Gaussian variable with zero mean and covariance matrix
${\bf \Gamma_N} = \mbox{diag}(\sigma_1^2,\cdots,\sigma_m^2)$. In practice, as the detectors are assumed to be accurately calibrated,
${\bf \Gamma_N}$ is known with high precision. The log-likelihood function is then the following one :
\begin{equation}
\label{eq:ll}
\log P({\bf X} \big| {\bf A},{\bf S},{\bf \Gamma_N}) = -\frac{1}{2} \|{\bf X} - {\bf AS}\|_{2,{\bf \Gamma_N}}^2 + C
\end{equation}
where $C$ is a constant. The notation $\| . \|_{2,{\bf \Gamma_N}}^2$ stands for the Frobenius norm of ${\bf Y}$ in the noise
covariance metric : $\| Y \|_{2,{\bf \Gamma_N}}^2 = \mbox{ Trace}\left( {\bf Y}^T {\bf \Gamma_N}^{-1} {\bf Y}\right)$.
From a Bayesian point of view, adding physical priors should help the separation task. We first assume no particular knowledge
about the emission laws of the components modeled by ${\bf A}$. For simplicity, we consider that each entry of the mixing
matrix ${\bf A}$ is \textit{i.i.d.}\footnote{Independently and identically distributed.} from a uniform zero mean distribution.
Note that it would be possible to add some physical constraint on the emission laws reflected in ${\bf A}$.

In the general case, source separation is merely a question of diversity and contrast between the sources (see \cite{Cardo1}).
For instance, on the one hand JADE relies on non-gaussianity to distinguish between the sources. On the other, SMICA takes advantage
of the diversity of the mixed components' power spectra to achieve the separation task. ``Non-gaussianity" and ``power spectra diversity"
are contrasts between the sources. A combination of both characteristics, ``Non-gaussianity" and ``power spectra diversity",
was also proposed to separate CMB from kinetic SZ signal which are otherwise undistinguishable \cite{forni}. Recent work has
already emphasized on sparsity as a source of diversity to improve component separation (see \cite{Zibu} and \cite{MMCA}).
In that setting, each source $\{s_j\}_{j=1,\cdots,n}$ is assumed to be sparse in a representation (potentially overcomplete) $\mathcal{D}$.
Formally, $\mathcal{D}$ is a fixed dictionary of signal waveforms written as a $T \times t$ matrix. We define the set of projection
coefficients $\alpha_j$ such that : $\forall j \in \{1,\cdots,n\}, \quad s_j = \alpha_j \mathcal{D}$. Any source $s_j$ is said
to be sparse in $\mathcal{D}$ if most of the entries of $\alpha_j$ are nearly zero and only a few have ``significant" amplitudes.
When $\mathcal{D}$ is overcomplete ($T > t$), $\mathcal{D}$ is called a dictionary. Overcomplete representations attractiveness
in image processing theory leans on their potential to generate very sparse representations of data based on their morphological
content (see e.g. \cite{DH} and references therein).

In the field of basic source separation we showed in \cite{MMCA} that morphological diversity and sparsity are key properties
leading to better separation. We noticed that the gist of sparsity-based source separation methods leans on the rationale :
``\textit{independent sources are distinctly sparse in a dictionary $\mathcal{D}$}". In that study, we considered the simple
case of morphologically different sources : components were assumed to be sparsely represented in different sub-dictionaries.
We illustrated that such sparsity prior provides a very effective way to distinguish between sources. In the present paper,
we focus on a more general setting : the sources can have similar morphologies (\textit{i.e.} all the sources are sparsely
represented over the whole $\mathcal{D}$). When the overcomplete dictionary $\mathcal{D}$ is made of the union of $D$ orthonormal
bases (\textit{i.e.} $\mathcal{D} = \left[\Phi_1,\cdots,\Phi_D\right]$) then each source is modeled as the linear combination
of $D$ so-called morphological components (see \cite{SED} for details on Morphological Component Analysis) - each morphological
component being sparse in a different orthonormal basis $\{\Phi_1,\cdots,\Phi_D\}$:
\begin{eqnarray}
\forall  j\in \{1,\cdots,n\}, \quad s_j  & = &  \sum_{k=1}^D \varphi_{jk} = \sum_{k=1}^D \alpha_{jk} \Phi_k
\end{eqnarray}
From a statistical viewpoint, we assume that the entries of $\alpha_{jk} = \varphi_{jk}\Phi_k^T$ are \textit{i.i.d} from
a Laplacian probability distribution with scale parameter $1/\mu$:
\begin{equation}
\label{eq:source_prior}
P(\varphi_{jk}) \propto \exp\left(- \mu \|\varphi_{jk}\Phi_k^T\|_1\right)
\end{equation}
where the $\ell_1$-norm $\|.\|_1$ stands for $\|x\|_1 = \sum_{p=1}^t |x[p]|$ in which $x[p]$ is the $p$-th entry of $x$. In practice,
the Laplacian prior is well adapted to model leptokurtic sparse signals. We classically assume that the morphological components are
statistically mutually independent : $P({\bf S}) = \prod_{j,k} P(\varphi_{jk})$. Estimating the sources ${\bf S}$ is then equivalent
to estimating the set of morphological components $\{\varphi_{jk}\}_{j=1,\cdots,n;k=1,\cdots,D}$. In this Bayesian context, we propose
to estimate those morphological components $\{\varphi_{jk}\}$ and the mixing matrix ${\bf A}$ from a \textit{maximum a posteriori} (MAP)
leading to the following optimization problem:
\begin{equation}
\left\{\{\hat{\varphi}_{jk}\},{\bf \hat{A}}\right\} = {\arg\max}_{\{\varphi_{jk}\},{\bf A}} P({\bf X}|{\bf A},\{\varphi_{jk}\},{\bf \Gamma_N}) \prod_{j,k}P(\varphi_{jk}) P({\bf A})
\end{equation}
where we further assumed that the morphological components $\{\varphi_{jk}\}$ are independent of ${\bf A}$. Owing to Equations~\ref{eq:ll}
and \ref{eq:source_prior}, the mixing matrix ${\bf A}$ and the morphological components $\{\varphi_{jk}\}$ are obtained by minimizing
the following negative log \textit{a posteriori}:
\begin{equation}
\label{eq:optim}
\left\{\{\hat{\varphi}_{jk}\},{\bf \hat{A}}\right\} = {\arg\min}_{\{\varphi_{jk}\},{\bf A}}\|{\bf X} - {\bf AS}\|_{2,{\bf \Gamma_N}}^2 + 2 \mu \sum_{j=1}^n \sum_{k=1}^D \|\varphi_{jk}\Phi_k^T\|_1
\end{equation}
where $\forall j \in \{1,\cdots,n\},\quad s_j = \sum_{k=1}^D \varphi_{jk}$. Equation~\ref{eq:optim} leads to the GMCA estimates
of the sources and the mixing matrix in a general sparse component separation context. Interestingly, in the case of CMB data,
the sources we look for (CMB, galactic dust and SZ) are quite sparse in the same unique orthonormal wavelet basis. The dictionary
$\mathcal{D}$ then reduces to a single orthonormal basis $\Phi$. In that case, since $\Phi$ is unitary, Equation~\ref{eq:optim}
can be rewritten as follows :
\begin{eqnarray}
\label{eq:foptim}
\left\{{\bf \hat{\alpha}},{\bf \hat{A}}\right\} &=& {\arg\min}_{{\boldsymbol \alpha},{\bf A}} \|{\bf X}\Phi^T - {\bf A\alpha}\|_{2,{\bf \Gamma_N}}^2 + 2 \mu \|{\boldsymbol \alpha}\|_1 \nonumber \\
						&=& {\arg\min}_{{\boldsymbol \alpha},{\bf A}} f_\mu({\boldsymbol \alpha},{\bf A}) = {\arg\min}_{{\boldsymbol \alpha},{\bf A}} f_0({\bf A}, {\boldsymbol \alpha}) + 2\mu f_1({\boldsymbol \alpha})
\end{eqnarray}
where ${\boldsymbol \alpha} = {\bf S}\Phi^T$. Note that the estimation is done in the sparse representation $\Phi$ requiring a single
transform of the data ${\bf X}\Phi^T$. To remain computationally efficient, GMCA relies on practical transforms which generally involve
fast implicit operators (typical complexity of $\mathcal{O}\left(t\right)$ or $\mathcal{O}\left(t \log t \right)$). In \cite{Zibu},
the authors also used a unique orthonormal wavelet basis. While a gradient descent is used in \cite{Zibu}, we use a fast and efficient
iterative thresholding optimization scheme which we describe in the next section.

\subsection{Solving the optimization problem}
\label{sec:algo}
The \textit{maximum a posteriori} estimates of the coefficients ${\boldsymbol \alpha}$ and the mixing matrix in Equation~\ref{eq:foptim}
lead to a non-convex minimization problem. Note that in Equation~\ref{eq:foptim} the functional to be minimized suffers from several
invariances : any permutation or rescaling of the sources and the mixing matrix leaves the product $\bf A{\boldsymbol \alpha}$ unaltered.
The scale invariance is computationally alleviated by forcing the columns of ${\bf A}$ to have unit $\ell_2$ norm~: $\forall i\in{1,\cdots,n},\quad a^{i^T}a^i = 1$
where $a^i$ is the $i$-th column of ${\bf A}$.

As solutions of problem~(\ref{eq:foptim}) have no explicit formulation, we propose solving it by means of a block-coordinate relaxation
iterative algorithm such that each iteration $(h)$ is decomposed into two steps : (i) estimation of the sources ${\bf S}$ assuming the
mixing matrix is fixed to its current estimate ${\bf \hat{A}}^{(h-1)}$ and (ii) estimation of the mixing matrix assuming the sources are
fixed to their current estimates ${\bf \hat{S}}^{(h)}$. It is not difficult to see that the objective MAP functional in (\ref{eq:foptim})
is continuous on its effective domain and has compact level sets. Moreover, this objective function is convex in the source coefficient
vectors $(\alpha_1,\ldots,\alpha_n)$, and $f_0$ has an open domain, is continuous and G\^ateaux differentiable. Thus by \cite[Theorem 4.1]{Tseng2001},
the iterates generated by our alternating algorithm are defined and bounded, and each accumulation point is a stationary point of the MAP
functional. In other words, our iterative algorithm will converge. Hence, at iteration $(h)$, the sources are estimated from a \textit{maximum a posteriori}
assuming ${\bf A} = {\bf \hat{A}}^{(h-1)}$. By classical ideas in convex analysis, a necessary condition for ${\boldsymbol \alpha}$ to be
a minimizer is that the zero is an element of the subdifferential of the objective at ${\boldsymbol \alpha}$. We calculate\footnote{For clarity,
we drop the upper script $(h-1)$ and write $\hat{\bf A} = \hat{\bf A}^{(h-1)}$.}:
\begin{equation}
\label{eq:subdiff}
\partial_{\boldsymbol \alpha} f_\mu({\boldsymbol \alpha},{\bf A})= -2{\bf {\bf A}}^T{\bf \Gamma_N}^{-1}({\bf X}\Phi^T - {\bf A}{\boldsymbol \alpha}) + 2\mu \partial_{\boldsymbol \alpha} \|{\boldsymbol \alpha}\|_1
\end{equation}
where $\partial_{\boldsymbol \alpha} \|{\boldsymbol \alpha}\|_1$ is defined as (owing to the separability of the prior):
\[
\partial_{\boldsymbol \alpha} \|{\boldsymbol \alpha}\|_1 = \left\{U \in \mathbb{R}^{n \times t} \Bigg|
\begin{array}{ccc}
U{j,k} & = \mbox{ sign}(\alpha_{j,k}), & ~ \alpha_{j,k} \neq 0 \\
U{j,k} & \in [-1,1], & ~ \alpha_{j,k} = 0
\end{array} \right\}.
\]
Hence, Equation \ref{eq:subdiff} can be rewritten equivalently as two conditions leading to the following (proximal) fixed point equation:
\begin{equation}
\label{eq:it_estim1}
\begin{array}{cc}
\hat{\alpha}_{j,k} = 0, & \text{if} ~ \left|{\left({\bf A}^T{\bf \Gamma_N}^{-1}{\bf X}\Phi^T\right)}_{j,k} \right| \leq \mu \\
{\bf {\bf A}}^T{\bf \Gamma_N}^{-1}({\bf X}\Phi^T - {\bf A}\hat{\boldsymbol \alpha}) = \mu \mbox{ sign}\left(\hat{\boldsymbol \alpha}\right), & \text{otherwise}.
\end{array}
\end{equation}
Unfortunately, Equation~\ref{eq:it_estim1} has no closed-form solution in general. It must be iterated and is thus computationally demanding.
Fortunately, it can be simplified when ${\bf A}$ has nearly orthogonal columns in the noise covariance matrix (\textit{i.e.}
${\bf \hat{A}}^T{\bf \Gamma_N}^{-1}{\bf \hat{A}} \simeq \mbox{diag}\left({\bf \hat{A}}^T{\bf \Gamma_N}^{-1}{\bf \hat{A}}\right)$).
Let ${\bf C} = {\left({\bf \hat{A}}^T{\bf \Gamma_N}^{-1}{\bf \hat{A}}\right)}^{-1}{\bf A}^T{\bf \Gamma_N}^{-1}{\bf X}\Phi^T$, Equation~\ref{eq:it_estim1}
boils down to the following set of equations $\forall j\in\{1,\cdots,n\}$:
\begin{equation}
\label{eq:it_estim}
\begin{array}{ccc}
\hat{\alpha}_{j,k} & = 0, \quad \text{if} ~ \left|{{\bf C}}_{j,k} \right| \leq \mu^{(h)} \sigma_j^2 \\
\hat{\alpha}_j & = {\left[{\bf C}\right]}_j - \mu \sigma_j^2  \mbox{ sign}\left(\hat{\alpha}_j\right), \quad \text{otherwise}.
\end{array}
\end{equation}
where $[{\bf Y}]_j$ is the $j$-th row of ${\bf Y}$. In practice, even if the approximation we make is not strictly valid, such a simplification
leads to good computational results. These equations are known as soft-thresholding with threshold $\mu^{(h)} \sigma_j^2$. We define $\mathrm{ST}_{\delta}(.)$,
the soft-thresholding operator with threshold $\delta$. At iteration $(h)$, the sources are thus estimated such that:
\begin{equation}
\hat{\alpha}_j^{(h)} = \mathrm{ST}_{\mu^{(h)} \sigma_j^2}\left(\left[{\bf C}\right]_j\right)
\end{equation}
The $j$th source is reconstructed as $\hat{s}_j^{(h)} = \hat{\alpha}_j^{(h)}\Phi$. The mixing matrix ${\bf A}$ is then estimated by a maximum
likelihood estimate amounting to a simple least-squares update assuming ${\bf S}$ is fixed. The GMCA algorithm is then described as follows :
\begin{flushleft}
\vspace{0.15in}
\centering
\begin{tabular}{|c|} \hline
\begin{minipage}[h]{0.95\linewidth}
\vspace{0.025in} \footnotesize{\textsf{1. Set the number of iterations $I_{\max}$ and thresholds $\delta_j^{(0)} = \mu^{(0)}\sigma_j^2$\\}
\textsf{2. While each $\mu^{(h)}$ is higher than a given lower bound $\mu_{min}$ (e.g. can depend on the noise variance), \\}
\hspace{0.1in} \textsf{-- Proceed with the following iteration to estimate source coefficients ${\boldsymbol \alpha}$ at iteration $h$ assuming ${\bf A}$ is fixed:}
\hspace{0.2in} \textsf{$\hat{\alpha}_j^{(h)} = \mathrm{ST}_{\mu^{(h)} \sigma_j^2}\left(\left[{\left({\bf \hat{A}}^T{\bf \Gamma_N}^{-1}{\bf \hat{A}}\right)}^{-1}{\bf \hat{A}}^T{\bf \Gamma_N}^{-1}{\bf X}\Phi^T\right]_j\right)$:\\}
\hspace{0.1in} \textsf{-- Update $\bf A$ assuming ${\boldsymbol \alpha}$ is fixed :}
\hspace{0.2in} \textsf{${\bf \hat{A}}^{(h)} = {\bf X}\Phi^T{\bf \hat{\boldsymbol \alpha}}^T\left({\bf \hat{\boldsymbol \alpha}}{\bf \hat{\boldsymbol \alpha}}^T\right)^{-1}$\\}
\textsf{-- Decrease the threshold $\mu^{(h)}$ following a given strategy}}
\vspace{0.05in}
\end{minipage}
\\\hline
\end{tabular}
\vspace{0.15in}
\end{flushleft}
Note that the overall optimization scheme is based on an iterative and alternate thresholding algorithm involving a
\textit{coarse to fine} estimation process. Indeed, \textit{coarse} versions of the sources (\textit{i.e.} containing
the most ``significant" features of the sources) are first computed with high values of $\mu^{(h)}$.
%\begin{equation}
%\forall j\in\{1,\cdots,n\},\quad \hat{s}_j = \mathrm{ST}_{\mu^{(h)} \sigma_j^2}\left({\left({\bf \hat{A}}^T{\bf \Gamma_N}^{-1}{\bf \hat{A}}\right)}^{-1}{\bf \hat{A}}^T{\bf \Gamma_N}^{-1}{\bf X}\Phi^T\right)\Phi
%\end{equation}
In the early stages of the algorithm, the mixing matrix is then estimated from the most ``significant" features of the sources
which are less perturbed by noise. The estimation of ${\bf A}$ and ${\bf S}$ is then refined at each iteration as $\mu^{(h)}$
(and thus the thresholds $\{\mu^{(h)}\sigma_j^2\}_{j=1,\cdots,n}$) decreases towards a final value $\mu_{min}$. We already used
this minimization scheme in \cite{MMCA} where this optimization process provided robustness and helped convergence even in a
noisy context. Experiments in Section~\ref{sec:results} illustrate that it achieves good results with GMCA as well.


\problemname{Better Productivity}

ACME Inc.~is reorganizing their factory, in order to maximize their
productivity of useless trinkets.  The new factory design consists of
$p$ independent and identical production lines.  Each production line
can be assigned some number of workers.

The actual work is of course all done by machines, so the production
rate of a production line is independent of the actual number of
workers assigned to it.  However, the workers work in shifts, and due
to local safety regulations, a production line can only be active
while all of its workers are present (any worker who arrives before
the last person to arrive, or leaves after the first person leaves,
will be spending the inactive time having a coffee break).  In other
words, the productivity of a production line equals the length of the
timespan during which all of the workers assigned to this production line are present.  Crucially, the
productivity of each line \emph{must} be positive (i.e., the last worker to arrive for a line must
arrive strictly before the first worker for that line leaves), since otherwise the
workers feel that their jobs are meaningless.

Unfortunately, due to some pesky labor laws, ACME are not allowed to
fire any of their workers, which means that each of their $n$ workers
must be assigned to some production line, even though this may
actually decrease the overall productivity of the factory.

All these rules and regulations are making a headache for ACME
management.  Can you help them figure out the maximum possible total
productivity (sum of productivities of the $p$ production lines) of
their new factory?

\section*{Input}

The input consists of:
\begin{itemize}
\item one line containing two integers $n$ and $p$ ($1 \le p \le n \le 200$), the number of employees and the number of production lines;
\item $n$ lines each containing two integers $a$ and $b$ ($0 \le a < b \le 100\,000$), representing a worker that arrives at time $a$ and leaves at time $b$.
\end{itemize}

You may assume that there exists at least one valid assignment of
workers to production lines.

\section*{Output}

Output the maximum productivity level possible for the
factory.


\section{Complexity Analysis\label{sec:complexity}}
In this section, we analyze the complexity of the algorithms required by \qbkix.
The input to our overall algorithm is a domain boundary $\Gamma$ with $\Ninit$ patches and boundary condition $f$.
We begin with a summary of algorithm parameters that impact complexity:

\begin{itemize}
\item The number of patches $N$ \emph{after} admissibility refinement.
  This is a function of $\Ninit$, the geometry of $\Gamma$, the definition of $f$, and the choices of parameters $a$ and $b$ in check point construction.
\item Quadrature order $q$ and the degree of smoothness $k$ of $\Gamma$ and $f$.
  We assume that $k$ is sufficiently high to obtain optimal error behavior for a given $q$ by letting $k=2q$ in \cref{eq:error_quad_gen_target}.
\item \qbkix interpolation order $p$.
\item The numbers of evaluation points in different zones $\Nf$, $\Ni$, and $\Nn$, with $\Nt = \Nf+\Ni+\Nn$.
%\item Maximum upsampling ratio $m = \max_i m_i$, where $m_i$ is the upsampling ratio for the $i$-th patch.
%\item Target error $\etrg$ used to determine check point location.
\end{itemize}

%The complexity is also affected by the geometric characteristics of the surface. These include maximum patch size $\Lmx$, relative minimal patch size $\Lmn = \beta_0 \Lmx$, $\beta_0 \leq 1$, as well as \emph{minimal feature size} $\ellm = \alpha_0\Lmx$ and \emph{the variation of area distortion} of the parametrization $C_J$, which we now define more precisely.
%The local feature size $\ellm$ is defined in terms of the \emph{medial axis} of the surface.  The medial axis of a surface $\Gamma$, $M(\Gamma)$, is the set of points in $\mathbb{R}^3$ with more than one closest point on $\Gamma$. For $\vx \in\Gamma$, the local feature size  $\ell(\vx)$ is the distance from $\vx$ to $M(\Gamma)$. We assume that the minimal feature size \emph{relative to $\Lmx$} is bounded from below by $\alpha_0$, i.e., $\ell(\vx) \geq \alpha_0 \Lmx$.
%The variation of the area distortion $C_J(P)$ is $\max_{(u,v)} |J|/\min_{(u,v)} |J|$, where $J$ is the Jacobian of the patch parametrization for an initial patch $P$, and $C_J$ is the maximum of $C_J(P)$ over all patches; this constant indicates how nonuniform the parametrization is, and allows us to estimate how the patch size decreases with refinement \note[MJM]{add a sentence with example: high $C_J$ vs. low $C_J$}.
%We assume that the $\alpha_0$, $\beta_0$ and $C_J$ are independent of $N^{init}$. We also assume that surface principal curvatures are bounded, independently of $N^{init}$.
The complexity is also affected by the geometric characteristics of $\Gamma$.
These include:
\begin{itemize}
  \item The \textit{maximum patch length} $\Lmx = \max_P L(P)$
  \item The \textit{relative minimal patch length} $\Lmn = \beta_0 \Lmx$, $\beta_0 \leq 1$.
  \item The \textit{minimal feature size relative to $\Lmx$}, $\ellm = \alpha_0\Lmx$, which is defined in terms of the \textit{local feature size} and the \textit{medial axis} of $\Gamma$.
The medial axis of $\Gamma$, denoted $M(\Gamma)$, is the set of points in $\mathbb{R}^3$ with more than one closest point on $\Gamma$.
For $\vy \in\Gamma$, the local feature size  $\ell(\vy)$ is the distance from $\vy$ to $M(\Gamma)$.
We assume that the local feature size is bounded below by $\alpha_0\Lmx$, i.e., $\ell(\vy) \geq \alpha_0 \Lmx = \ellm$ for $\vy \in \Gamma$.
  \item The \emph{maximum variation of area distortion} of the parametrization $C_J$.
The variation of the area distortion of a patch $P$ is $C_J(P) = \max_{(u,v)} |J_P(u,v)|/\min_{(u,v)} |J_P(u,v)|$, where $J_P(u,v)$ is the Jacobian of $P$ at the point $(u,v)$.
We define $C_J = \max_{P\in\Gamma} C_J(P)$.
This value is an indicator of how non-uniform the parametrization of $P$ is and allows us to estimate how the patch length decreases with refinement.
\end{itemize}

We assume that the $\alpha_0$, $\beta_0$ and $C_J$ are independent of $\Ninit$.
We also assume that principal curvatures are bounded globally on $\Gamma$ and independent of $\Ninit$.
We now briefly summarize the results of this section:

\begin{itemize}
\item \emph{Admissibility.} (\cref{sec:complexity_admissibility}) The complexity of this step is $O(\Ninit \log \Ninit)$,
  with constants dependent on $\alpha_0$, $\beta_0$ and $C_J$.
  The logarithmic factor is due to use of an \aabb tree for closest surface point queries.

\item \emph{Upsampling.} (\cref{sec:complexity_upsampling}) The complexity of upsampling is $ O(m  N \log(N))$, where $m$ is the upsampling ratio.
  The logarithmic factor appears for similar reason to admissibility,
  with constants that depend on geometric parameters and the boundary condition through the error estimate of \cref{sec:error}.
  We show that the upsampling ratio is independent of $N$.
  %The upsampling ratio is $m = O(\etrg^{-1/q})$.

\item \emph{Point marking.} (\cref{sec:complexity-point-marking}) Identifying which zone an evaluation point belongs to ($\Omega_F, \Omega_I$ or $\Omega_N$) depends on $N$ and the total number of points to be classified $\Nt = \Nf + \Ni + \Nn$.
  The complexity is $O(\Nt \log N)$ with constants dependent on geometric parameters, due to the cost of closest surface point queries.

\item\emph{ Far, intermediate and near zone integral evaluation.} (\cref{sec:complexity_matvec}) The complexity of these components depends on $N$ and  $\Nf$, $\Ni$ and $\Nn$ respectively, with the general form $O(s_1 N + s_2 N')$, where $N'$ is the number of evaluation points in the corresponding class.  For the far field, $s_1 = s_2 = 1$.  For the intermediate evaluation,
$s_2 =1$, and $s_1 = m q^2$; finally, for the near zone, $s_2 = p$, and $s_1 = mq^2$, the same as in the intermediate zone.
If $b$ is chosen appropriately, the intermediate and near zone error is $\etrg$.

  %\note[DZ]{This is not quite true: we should probably modify the upsampling alg description, so that we actually get
  %  this in the near zone, not just intermediate; otherwise it is a pain to explain, as there is also the extra dependence on the extrapolation order, so smth like $(\Lmx*2^\eta_1)^q$}
  %  \note[MJM]{fix}
  \item \emph{\gmres solve.} Due to the favorable conditioning of the double-layer formulation in \cref{eq:int-eq}, \gmres converges rapidly to a solution in a constant number of iterations for a given $\Gamma$ that is independent of $N$.
    This means that the complexity to solve \cref{eq:int-eq} is asymptotically equal (up to a constant dependent on $\Gamma$) to the complexity equal to a near-zone evaluation with $\Nn=N(q+1)^2$.
\end{itemize}

\subsection{Admissibility \label{sec:complexity_admissibility}}

The patch refinement procedure \cref{sec:admissible} to enforce \cref{criteria:1,criteria:2} of admissibility and achieve given approximation errors of the geometry $\err{g}$ and boundary data $\err{f}$ is a local operation on each patch.
If we assume that $\Lmn$, $\Lmx$, the partial derivatives of all patches composing $\Gammah$, and the partial derivatives of $f$ are bounded, then
 errors $\err{g}$ and $\err{f}$ can always be achieved after a fixed number of refinement steps. As a consequence, this stage must have complexity $O(\Ninit)$.

We focus on the additional refinement needed to satisfy \cref{criteria:3}: ensuring that each check center $\vhc$ is closest to its corresponding quadrature point $\vy$.
This can be restated in terms of local feature size: for a quadrature patch $P \in \Gamma$ and quadrature node $\vx \in P$ with check center $\vhc$, $\|\vx - \vhc\|_2 \leq \ell(\vx) \leq \alpha_0 L_0$.
We will first relate the number of required refinement steps $\eta$ to satisfy \cref{criteria:3} to the shape parameters $\alpha_0$ and $C_J$, then we will show that this number does not depend on $N$ under our assumptions.

Recall that the distance from a check center to the surface  for a patch $P$ is given by
$R + r(p+1)/2 = (a+ (p+1)b/2)L(P)= K L(P)$. %, where $L(P)$ is the square root of the area of $P$.
After $\eta$ refinement steps, the area of each child of $P$ relative to $P$ itself will have decreased by at least by $C_J(P)(1/4)^\eta$.
Since the distance from $\vhc$ to the surface is proportional to $L(P)$, we can estimate the required level
of uniform refinement to satisfy \cref{criteria:3} by requiring that the check center distance is less than the minimal local feature size, then taking the maximum value of $L(P)$ over all patches:
\[
K \Lmx \sqrt{C_J} (1/2)^\eta \leq \ellm = \alpha_0 \Lmx
\]
This yields

\begin{equation}
  \eta   =  \lceil -\log_2 \frac{\alpha_0}{K \sqrt{C_J}}\rceil,
\label{eq:eta-estimate}
\end{equation}
which we note depends only on nondimensional quantities $\alpha_0$, $K$ and $C_J$ characterizing the shape of the surface and its parametrization.
If we assume these to be independent of $N$, then
the number of required levels of refinement $\eta$ are also independent of $N$.
This means that the number of patches $N$ generated \cref{alg:admissibility} is a linear function of $\Ninit$, bounded by $4^{\eta} \Ninit$.

Next, we estimate the complexity of work per patch in \cref{alg:admissibility} to determine if a given patch requires refinement.
As described in \cref{sec:admissible}, for each patch, we query the \aabb tree $T_B$ for patches that are at the distance $R + r(p+1)/2 = K L(P)$ from a check center $\vhc$.
The cost of the query is logarithmic in the number of patches $\Ninit$ and proportional to the number of patches $N(\vhc)$ returned.
This means that we need to estimate the number of patches that can be within the distance $K L(P)$ from $\vhc$.

Consider an area element $dA$ of $\Gammah$ at a point $\vx_0$. The parallel surface of $dA$,
given by $\vx_0 + h \vn(\vx_0)$ does not have self-intersections when $|h| \leq \ellm$ and has a corresponding area element given by $dA^{h} = (1+ h \kappa_1)(1+h \kappa_2)dA$ \cite[Section 6.2]{K}, where $\kappa_1$ and $\kappa_2$ are the principal curvatures of $\Gammah$ at $\vx_0$.
The volume of the truncated cone bounded by $dA$ and $dA^{h}$ of height $\ellm$ can be computed directly from the integral $\int_0^{\ellm} dA^h dh$:
\[
  dV = dA \ellm (1 + \frac{1}{2} (\kappa_1 + \kappa_2) \ellm + \frac{1}{3} \kappa_1 \kappa_2 \ellm^2)   = dA\ellm(1 + \frac{1}{2} H \ellm + \frac{1}{3} K \ellm^2)
\]
where $K$ and $H$ are Gaussian and mean curvatures respectively. As principal curvatures satisfy
$\kappa_i \geq -1/\ellm$,  this expression has minimal value for $\kappa_1 = \kappa_2 = -1/\ellm$:
\begin{equation}
dV \geq \frac{1}{3}\ellm dA
\label{eq:vol-lower-bound}
\end{equation}
In other words, each surface element $dA$ has (at least) a volume $\frac{1}{3}\ellm dA$ with no other surface elements inside associated with it.  From this, we can estimate the total area of surface contained within distance $K L(P)$ from $\vhc$ by equating \cref{eq:vol-lower-bound} with the volume of a sphere of raidus $KL(P)$, producing $4 \pi K^3 L(P)^3/\ellm$.
Since the area of each patch is at least $L_{\min}^2$, the number of patches $KL(P)$ from $\vhc$ is bounded by

\begin{equation}
  N(\vhc) \leq  4 \pi K^3 \frac{L(P)^3}{\ellm \Lmn^2} \leq 4 \pi K^3 \frac{\Lmx^3}{\ellm \Lmn^2} =
  \frac{4 \pi K^3}{\alpha_0 \beta_0^2}
 \label{eq:numpatches-estimate}
 \end{equation}

This is independent of $\Ninit$, which means that the complexity of nearest patch retrieval is $O( \Ninit \log \Ninit)$, with constant given by the product of
\eqref{eq:numpatches-estimate} and $4^\eta$, with $\eta$ given by \eqref{eq:eta-estimate}.

To complete the complexity estimate of the admissibility refinement, we need to estimate the cost of computing the closest point on each patch.
The complexity of the Newton's method for finding roots of polynomials  in \cref{app:closest_point} depends only on the polynomial degree and the desired accuracy of the optimization, which we can assume to be bounded by floating-point precision \cite{schleicher2017newton}.
We conclude that the overall complexity of admissibility refinement is $O( \Ninit \log \Ninit)$ with constants proportional to the patch degree and optimization accuracy.

\subsection{Upsampling\label{sec:complexity_upsampling}}
We estimate the complexity of the upsampling algorithm in \cref{sec:adaptive_upsampling} in terms of $N$, the number of patches produced by admissibility refinement, and a parameter $\eps$, which is the desired accuracy achieved by the final upsampled patches at the check points.
As the distance from the surface to the check points $\vc_i$ is bounded from below by
$a \Lmn$,  the $\tilde{V}$ term
in \cref{eq:error_quad_gen_target} is bounded from above by $C \Lmn^{-2q-1}$, for a constant $C$ independent of $q$.
Furthermore, since $\Gammah$ and $f$ are assumed to be smooth, the density and its derivatives can also be assumed to be bounded.
The overall form of the estimate in \cref{eq:error_quad_gen_target} can then be bounded and written as $\tilde{C}(q) \Lmn^{-2q-1} \tilde{L}^{2q}$ for some constant $\tilde{C}(q)$.
The maximum patch length obtained by refinement $\tilde{L}$ is
\begin{equation}
  \tilde{L} = \Lmx^\lbl{fine} \leq \Lmx 2^{-\tilde{\eta}},
  \label{eq:Ltilde}
\end{equation}
where $\tilde{\eta}$ is the maximum amount of required patch refinement.
By setting $C(q) \Lmn^{-2q-1} \tilde{L}^{2q} \leq \eps$ and using \cref{eq:Ltilde}, we can
obtain an upper bound for $\tilde{\eta}$ as a function of $\Lmn$, $\Lmx$, and $\eps$:
\begin{equation}
  \tilde{\eta} \leq -\frac{1}{2q}\log_2 \left( \frac{\eps}{\Lmn^{-2q-1} \Lmx^{2q} C(q)}\right)= \log_2 \eps^{-1/(2q)} + \bar{C}(q,\Lmn,\Lmx),
  \label{eq:tildeeta}
\end{equation}
for some constant $\bar{C}(q,\Lmn,\Lmx)$.

The number of points generated by upsampling is $O(4^{\tilde{\eta}} N )$.
Taking powers of both sides of \cref{eq:tildeeta} yields an
estimate in terms of $\err{target}$:  $O( (2^{\tilde{\eta}})^2N ) \leq O(\eps^{-2/(2q)}N ) =  O(\eps^{-1/q}N )$.
As discussed in \cref{sec:complexity_admissibility}, the closest point computation needed to determine if a checkpoint is in $\Omega_I$ has $\log (N)$ cost per point, leading to $O(\eps^{-1/q} N \log(N))$ overall complexity and an upsampling factor of $\eps^{-1/q}$.
Since we desire upsampled quadrature with an accuracy of $10^{-12}$, we set $\eps$ as such to arrive at the desired complexity.

%Note that both $C$ and $\tilde{C}(q)$ above depend on higher-order derivatives of the patch $P$ and therefore implicitly on principal curvatures $\kappa_1$ and $\kappa_2$.
%This partial explains the difference in accuracy between \cref{eq:error_quad_gen_target} and \cref{eq:rem2d_heuristic}.
%In practice, we use the more precise error bound \cref{eq:rem2d_heuristic}, so the actual refinement is less than \cref{eq:error_quad_gen_target} might suggest in this analysis

\subsection{Point marking\label{sec:complexity-point-marking}}
In the point marking algorithm of \cref{app:point_marking}, we first use the Laplace \fmm to cull points far from $\Gamma$, which requires $O(N+\Nt)$ time.
Let $\bar{L} = \frac{1}{M}\sum_{P \in \Pcoarse}L(P)$ be the average patch length.
After \fmm culling, the remaining unmarked evaluation points are those whose distances from $\Gamma$ are approximately $\bar{L}$ or less.  For each unmarked point $\vx$,  we query the \aabb tree $T_T$ for the nearest triangle in the linear approximation of $\Pcoarse$.

Since there are $O(N)$ such triangles in $T_T$, we can perform this query in $O(\log N)$ time \cite{samet2006foundations}.
This triangle provides a candidate closest patch that is distance $d_0$ from $\vx$.
We then use to query $T_B$ for all bounding boxes at distance $d_0$ from $\vx$.
This query too can be performed in $O(\log N)$ time \cite{samet2006foundations} and returns a bounded number of boxes and that each is processed in constant time, as discussed in \cref{sec:complexity_admissibility}.
As the number of unmarked points after culling is bounded above by $\Nt$, the overall complexity of our marking scheme is $O(\Nt \log N)$.

\subsection{Integral evaluation complexity \label{sec:complexity_matvec}}
We assume that geometric admissibility criteria are already satisfied.
All integral evaluation is accelerated using an \fmm with complexity $O(N + \Nt)$.
\paragraph{Far zone}  The complexity of far evaluation is just the complexity of computing the integrals on $\Pcoarse$ using standard quadrature and \fmm acceleration, i.e.,  $O(q^2N + \Nf)$.% \sim O(n^2N + \Nf)$.

\paragraph{Intermediate zone} The complexity of the intermediate zone evaluation is similar to that of the far zone.
However the computation is performed on $\Pfine$ rather than $\Pcoarse$, which is up to $m$ times finer than $\Pcoarse$,  with $m = O(\eps^{-1/q})$ and $\eps=10^{-12}$.
The density values must be interpolated from points in $\Pcoarse$ to points in $\Pfine$: this can be computed in $O(mq^4N)$ time using a \twod version of the barycentric interpolation formula \cite{BT}.
This yields an overall complexity of  $O(mq^4N + m q^2N + \Ni)$.
Although not asymptotically dominant, for all practical target errors, the quadrature evaluation is the dominant cost in practice due to suppressed \fmm-related constants, as demonstrated in \cref{sec:results-compare}.

\paragraph{Near zone} %In our algorithm, the near zone includes points on the surface itself.
\cref{sec:singular-eval} requires a closest point computation, an intermediate-zone evaluation at $p$ check points and an extrapolation for each target point in $\Omega_N$.
The intermediate zone calculation is the dominant cost, resulting in a complexity of $O(mq^4N + m q^2 N + p\Nn)$.

\paragraph{\gmres solve} As a result of the second-kind integral formulation in \cref{sec:formulation}, the cost of solving \cref{eq:int-eq} via \gmres is asymptotically equal to the cost of a single singular integral evaluation, since the low number of iterations are independent of $N$.
In our algorithm, this is a special case of near-zone evaluation with $\Nn = q^2N$, producing a complexity of $O(mq^4N + mq^2N + pq^2N) =O( (m+p+mq^2) q^2 N)$.

\paragraph{Overall complexity for uniform point distribution}
We now suppose that we wish to evaluate the solution $u$ determined by a density $\phi$ at a set of uniformly distributed points throughout $\Omega$.
We also assume that $\Gammah$ is discretized uniformly by $N$ patches, i.e., $\Lmx = O(N^{-1/2})$ and that the distances between samples in $\Omega$ and from samples to $\Gammah$ are also $O(N^{-1/2})$.
Since the total number of evaluation points is proportional to $1/\Lmx^3$, this implies that $\Nt = O(N^{3/2})$.

The size of the intermediate zone $\Omega_I$ is bounded by the estimate discussed in \cref{sec:complexity_upsampling}.
Letting $d_I$ be the shortest distance along a normal vector of $\Gammah$ which is contained in $\Omega_I$, following the discussion in \cref{sec:complexity_upsampling} yields the following relation:
\begin{equation}
  \tilde{C}(n) d_I^{-2q-1} \Lmx^{2q} \leq \eps.
\end{equation}
Solving for $d_I$ gives us
\begin{equation}
  d_I \leq \left(\frac{\eps}{C(n)}\right)^{-\frac{1}{2q-1}} \left(\Lmx\right)^{\frac{2q}{2q-1}}.
\end{equation}
We are interested in the regime as $N \to \infty$, or $\Lmx \to 0$.
Since $\Lmx^{\frac{2q}{2q-1}} \leq \sqrt{\Lmx} = O(N^{-1/4})$, this gives us
\begin{equation}
  d_I \leq \left(\frac{\eps}{C(n)}\right)^{-\frac{1}{2q-1}} N^{-1/4} = O(\eps^{-1/2q}N^{-1/4}) = O(\sqrt{m}N^{-1/4}),
\end{equation}
after recalling from above that $m = O(\eps^{-1/q})$ is the average upsampling rate to produce $\Pfine$ from $\Pcoarse$.
The size of the near zone is, by construction, of the order $\Lmx$.
It follows that $\Ni =O(  \sqrt{m} N^{5/4})$, and $\Nn =O(N)$.

The overall complexity for this evaluation is the sum of the cost of each separate evaluation:
\begin{align*}
  &O(q^2N + \Nf + mq^4 N + mq^2 N + \Ni + mq^4 N + mq^2N + p\Nn)\\
  %&= O((m + mq^2)q^2N + \Nf + \Ni  + p\Nn)\\
  &= O\left( (m+mq^2)q^2N + \Nt + (p-1)\Nn \right)
\end{align*}
Using the estimates for $\Nt$ and $\Nn$ and dropping dominated terms, we obtain  $O( (m + m q^2)q^2 N + N^{3/2})$ for the overall complexity.
This suggests that for a given $q$ and $\eps$, the minimal cost is obtained from choosing the number of discretization points
$N =O(m^2)$, i.e., $ N = O(\eps^{-2/q})$.

%The overall complexity, $O(n^2N + \Nf + mn^2 N + \Ni + n\Nn  + (m+n) n^2 N)$,
%i.e., $O( (m + n)n^2 N  + \Ni + n \Nn + \Nf) = O( (m+n)n^2 N + \Nt + (n-1)\Nn)$.
%Using the estimates for $\Nt$ and $\Nn$,
%we obtain, dropping  dominated terms,  $O( m n^2 N + N^{3/2})$ for the overall complexity.
%This suggests that for a given target error $\etrg$, the minimal cost is obtained from choosing the number of surface points
%$N \sim m^2$, i.e., $\etrg^{-4/n}$.

\iffalse
\subsection{AABB Tree complexity}
It is a non-trivial task to prove that an AABB tree query has logarithmic complexity with respect to the number of contained boxes for general geometries in 3D.
Since the location of bounding boxes are a function of $\Gamma$, they have no well-defined spatial partition in general; this makes it challenging to make claims about the tree depth.
This is a research topic in its own right \note[MJM]{list two recent-ish papers}.
However, Proposition 1 of \cite{RKO} similarly applies to AABB trees:
\begin{proposition}
  Any query of an AABB tree requires in $O(d+n_\lbl{box})$ work, where $d$ is the number of levels in the tree, and $n_\lbl{box}$ is the number of bounding boxes returned from the query.
  \label{}
\end{proposition}
\note[MJM]{we can comment about how no closed boundary can have the worst case complexity of \cite{hammer2002box} globally, but i don't think we can prove $log(N)$ time}.

\subsection{Complexity comparison with \cite{YBZ}}
\note[MJM]{some comments about why we're only looking at one solver, not sure how to justify}
We compare \qbkix with the singular quadrature scheme of \cite{YBZ}, which we will refer to as \textit{partition of unity quadrature}, or \pou quadrature.
This scheme is quite different from the \qbkix method presented in this work; we summarize the key differences to avoid confusion.
\note[MJM]{add image highlighting difference in discretization, overlapping vs non overlapping}
In \cite{YBZ}, $\Gamma$ is defined as a set of compactly supported overlapping patches, detailed in \cite{ying2004simple}.
Each patch is discretized with a periodic tensor-product trapezoidal rule with spacing $h$.
The potential is computed at all discretization points, then a new partition of unity function $\eta(\vx)$, centered at the target point, is used to subtract the inaccurate near contribution.
$\eta(\vx)$ is then integrated with a periodic polar trapezoidal rule, which cancels the singularity by centering the polar grid at the target, and added to the accumulated potential.

The complexity of this algorithm is $O(N^{3/2})$, because the number of points used in the polar trapezoidal rule for each target point is $O(\sqrt{N})$.
Asymptotically, \qbkix will outperform \pou quadrature.
However, consider that the dominant cost of \qbkix is the \fmm evaluation of size $O(kN + pN)$; meanwhile, the \fmm evaluation of \pou quadrature is simply $O(N+N)$, since the source and targets are equal.
This also implies that the size of the \fmm tree for \pou quadrature is smaller than for \qbkix, whose source and target points are more distributed in space.
For small values of $N$ and low target accuracies, we can expect \pou quadrature to have a shorter \fmm setup and evaluation time than \qbkix, and therefore a lower overall runtime.

\fi


\subsection{Test 1}

\begin{enumerate}[label=\arabic*.]
	\item
		Determine if $y - \ln{y} = x^2 + 1$ is an implicit solution to $\dd{y}{x} = \frac{2xy}{y-1}$. Give reasons for your answer.
	\item
		Find the function $y(x)$ that satisfies the differential equation
		\begin{equation*}
			\dd{y}{x} + 4y - e^{-x} = 0 \text{ and } y(0) = \frac{4}{3}
		\end{equation*}
	\item
		Find the solution to the initial value problem $\dd{x}{t} = x\frac{(t-3)^2}{x+2} \text{, } x(3) = -1$. You may leave the solution in implicit form.
	\item
		Find the general solution to the differential equation $y''' + y'' - 2y = 0$.
	\item
		A tank contains 90kg of salt and 2000L of water. Pure water enters the tank at a rate of 3L/min. The solutions is mixed and drains from the tank at a rate of 6L/min.
		\begin{enumerate}[label=(\alph*)]
			\item
				What is the amount of salt in the tank initially?
			\item
				Find the amount of salt in the tank after 2 hours.
		\end{enumerate}
	\item
		A detective arrives at a murder scene at noon that has an ambient temperature of $16^{\circ}$C. He measures the temperature of the body as $34^{\circ}$C. He then returns an hour later and measures the temperature of the body as $32^{\circ}$C.
		Assuming the normal body temperature is $37^{\circ}$C, write a differential equation using Newton's law of cooling that models the situation. Describe, without actually solving the equation, how one would solve the equation to find then the murder took place.
\end{enumerate}


\documentclass{article}
\usepackage[elliptic]{qed}

\begin{document}

\section*{Elliptic integrals}

The complete elliptic integrals of the first and second kinds are defined as
%
\begin{qed}
    \ellipk(k^2) = \int_0^{\frac{\pi}{2}}
    \frac{1}{\sqrt{1 - k^2 \sin^2\theta}} \, d\theta
\end{qed}
%
and
%
\begin{qed}
    \ellipe(k^2) = \int_0^{\frac{\pi}{2}}
    \sqrt{1 - k^2 \sin^2\theta} \, d\theta
\end{qed}
%
respectively. They have the following properties:
%
\begin{qed}
    \ellipk(0) = \frac{\pi}{2}
\end{qed}
%
\begin{qed}
    \ellipe(0) = \frac{\pi}{2}
\end{qed}
%
\begin{qed}
    \ellipk(1) = \cinfty
\end{qed}
%
\begin{qed}
    \ellipe(1) = 1
\end{qed}
%
\begin{qed}
    \frac{d}{d k}\ellipk(k^2) = -\frac{\ellipk(k^2)
        + \frac{\ellipe(k^2)}{k^2 - 1}}{k}
\end{qed}
%
\begin{qed}
    \frac{d}{d k}\ellipe(k^2) = \frac{\ellipe(k^2) - \ellipk(k^2)}{k}
\end{qed}
%
They also satisfy the Legendre relation:
%
\begin{qed}
    \ellipe(k^2) \ellipk(1-k^2) + \ellipe(1 - k^2) \ellipk(k^2)
    - \ellipk(k^2) \ellipk(1 - k^2) = \frac{\pi}{2}
\end{qed}
%
\end{document}