\vspace{-5mm}
This chapter gives routines for computing translation matrices that operate on the spherical wave function expansions given in Chapter \ref{chap:wavefunctions}. Translation matrices, which are derived from addition theorems, allow us to represent a scalar or vector field in two different but parallel reference frames. The frame of reference is being translated, not the field. The addition theorems and matrix diagonalization are first outlined. Next, algorithms and routines for the scalar axial translation matrices (i.e., translation along the $z$-axis) are given which are computed on full or sparse matrices. Last, algorithms and routines for full and sparse vector axial translation matrices are derived from the scalar versions.

\vspace{-2mm}
\section{Translation Addition Theorem}

The translation addition theorems for spherical wave functions allow the same field to be expanded in two different, but parallel, frames. These are given for scalar waves as \cite{chew1995waves},
\begin{eqnarray}
\psi_{l'm'}(k,\br_i) &=& \sum_{l=0}^{\infty}\sum_{m=-l}^{l} \alpha^{ji}_{lm,l'm'} \textit{Rg}\psi_{lm}(k,\br_j), \quad \quad r_j < r_{ji} \label{adthmscal1} \\
\psi_{l'm'}(k,\br_i) &=& \sum_{l=0}^{\infty}\sum_{m=-l}^{l} \beta^{ji}_{lm,l'm'}\psi_{lm}(k,\br_j), \quad \quad r_j > r_{ji} \label{adthmscal2}\\
\textit{Rg}\psi_{l'm'}(k,\br_i) &=& \sum_{l=0}^{\infty}\sum_{m=-l}^{l} \beta^{ji}_{lm,l'm'}\textit{Rg}\psi_{lm}(k,\br_j), \quad \quad \forall r_j \label{adthmscal3}
\end{eqnarray}

\noindent where $\beta^{ji}_{lm,l'm'} = \textit{Rg} \alpha^{ji}_{lm,l'm'}$ are translation matrices and $\textit{Rg}$ is the regular part. The position vectors, $\br_i$ and $\br_j$, are measured from the origins of each frame. The first relation expands radiating waves from frame $i$ as regular or incoming waves in frame $j$. The second expands radiating waves in frame $i$ as radiating waves in frame $j$. The last expresses regular waves in frame $i$ and regular waves in frame $j$. Each addition theorem has a different region of validity: \eqref{adthmscal1} is valid within a sphere of radius $r_{ji} = \vert \bb{r}_{ji} \vert = \vert \br_j- \br_i\vert$ centered on frame $j$; \eqref{adthmscal2} is valid outside a radius $r_{ji}$ centered on frame $j$; \eqref{adthmscal3}  is valid everywhere.

Translation addition theorems for vector spherical wave functions have similar structures, \cite{chew1995waves}. The first is
\begin{eqnarray}
\bb{M}_{l'm'}(k,\br_i) &=& \sum_{l=1}^{\infty}\sum_{m=-l}^{l} A^{ji}_{lm,l'm'} \textit{Rg}\bb{M}_{lm}(k,\br_j),  + B^{ji}_{lm,l'm'} \textit{Rg}\bb{N}_{lm}(\br_j) \\
\bb{N}_{l'm'}(k,\br_i) &=& \sum_{l=1}^{\infty}\sum_{m=-l}^{l} B^{ji}_{lm,l'm'} \textit{Rg}\bb{M}_{lm}(k,\br_j),  + A^{ji}_{lm,l'm'} \textit{Rg}\bb{N}_{lm}(\br_j)
\end{eqnarray}

\noindent where $A^{ji}_{lm,l'm'}$ and $B^{ji}_{lm,l'm'}$ are vector translation matrices and is valid for $r_j < r_{ji}$. The remaining two relations are analogous to the scalar case and can be obtained with appropriate Hankel or Bessel functions in $A^{ji}_{lm,l'm'}$ and $B^{ji}_{lm,l'm'}$. These show that vector modes mix when translated. The spherical vector components are also transformed and are expressed relative to the origin of the new frame. This means that the vector components in each frame need to be converted to Cartesian components before comparing.

\begin{figure}[H]
   \centering
   \includegraphics[width=3.5in]{Translation/Figures/transdiagram}
   \caption{Coordinate frames $i$ and $j$.}
   \label{translationadd}
\end{figure}

Translation matrices allow us to manipulate field using only the expansion coefficients.  For example, a radiating scalar field is expanded in frame $i$ as
\begin{equation}
\phi(\br_i) = \sum_{l'=0}^{\infty}\sum_{m'=-l'}^{l'} a_{l'm'} \psi_{l'm'}(\br_i)
\end{equation}

The same field expressed as incoming waves in frame $j$ comes from substituting \eqref{adthmscal1} and regrouping terms as
\begin{eqnarray}
\phi(\br_j) &=& \sum_{l=0}^{\infty}\sum_{m=-l}^{l} b_{lm}  \textit{Rg}\psi_{lm}(k,\br_j) \\
b_{lm} &=& \sum_{l'=0}^{\infty}\sum_{m'=-l'}^{l'} \alpha^{ji}_{lm,l'm'}a_{l'm'}
\end{eqnarray}
%
%\begin{eqnarray}
%\phi(\br_j) &=& \sum_{l'=0}^{\infty}\sum_{m'=-l'}^{l'} a_{l'm'} \sum_{l=0}^{\infty}\sum_{m=-l}^{l} \alpha^{ji}_{lm,l'm'} \textit{Rg}\psi_{lm}(k,\br_j) \nonumber \\
%\ &=& \sum_{l=0}^{\infty}\sum_{m=-l}^{l} b_{lm}  \textit{Rg}\psi_{lm}(k,\br_j) \\
%b_{lm} &=& \sum_{l'=0}^{\infty}\sum_{m'=-l'}^{l'} \alpha^{ji}_{lm,l'm'}a_{l'm'}
%\end{eqnarray}

Likewise, for the other relations. In matrix notation this is
\begin{eqnarray}
\phi(k,\br_i) &=&  \boldsymbol{\psi}^t(k,\br_i)\cdot \bb{a} \\
\phi(k,\br_j) &=&  \textit{Rg}\boldsymbol{\psi}^t(k,\br_j)\cdot \bb{b} \\
\bb{b} &=& \boldsymbol{\alpha}_{ji} \bb{a}
\end{eqnarray}

The analogous matrix notation for vector waves is
\begin{eqnarray}
\bb{E}(k,\br_i) &=& \rvectwo{\bb{M}^t(k,\br_i)}{\bb{N}^t(k,\br_i)}\cvectwo{\bb{a}}{\bb{b}} \\
\bb{E}(k,\br_j)&=& \rvectwo{\Re\bb{M}^t(k,\br_j)}{\Re\bb{N}^t(k,\br_j)}\cvectwo{\bb{c}}{\bb{d}} \\
\cvectwo{\bb{c}}{\bb{d}}  &=&\tbt{\bb{A}_{ji}}{\bb{B}_{ji}}{\bb{B}_{ji}}{\bb{A}_{ji}}\cvectwo{\bb{a}}{\bb{b}}
\end{eqnarray}

In both scalar and vector cases the transpose, $^t$, means align the harmonic indices along columns. Technically the sums are infinite, but the matrices are always truncated in practice.

\section{Diagonalization}

Explicit expressions for the matrix elements of the scalar and vector translation matrices are given in \cite{chew1995waves}, but are given in terms of Wiger 3-j symbols, which are slow and inaccurate to compute. Fast recursions for quickly computing the scalar and vector translation matrices were developed in \cite{mackowski1991analysis, chew1992recurrence, chew1993efficient}.

The standard way to compute the translation matrix is to break it into three matrices. The local frame, $i$, is first rotated to point its $z$-axis toward the destination frame, $j$. The local frame is then axially translated. Finally, the frame is then rotated back. This effectively diagonalizes the translation matrix. Specifically, if the vector $\br_{ji}$ points from the origin of frame $i$ to the origin of frame $j$, with magnitude $r_{ji}$ and spherical angles $\theta_{ji}$, $\phi_{ji}$, shown in Figure \ref{translationadd}, then the scalar translation matrix is expanded (diagonalized) as
\begin{equation}
\boldsymbol{\alpha}_{ji} = \bb{D}^*(\phi_{ji}+\pi/2,\theta_{ji},0) \cdot \boldsymbol{\alpha}_z(r_{ji}) \cdot \bb{D}(\phi_{ji}+\pi/2,\theta_{ji},0)
\label{alphazdiag}
\end{equation}

\noindent where $\bb{D}$ is the rotation matrix with ZXZ convention, and $\boldsymbol{\alpha}_z(r_{ji})$ is the axial translation matrix in the $+z$ direction. The factor of $\pi/2$ ensures that $\theta_{ji}$ tips the $z$-axis toward the destination frame with a right-handed X rotation. The sequence starts with an inverse rotation on the right size because we first view the field, which is fixed in the global frame, from the point of view of the rotated frame. Next, this point of view is translated along the $z$-axis of the rotated frame. Finally, the reverse rotation (in this case a forward rotation) is applied to bring the frame back to be parallel with the global frame. This procedure also applies to the vector translation matrices $\bb{A}_{ji}$ and $\bb{B}_{ji}$.

\begin{figure}[H]
   \centering
   \includegraphics[width=5in]{Translation/Figures/alphaz}
   \caption{Diagonalization of the scalar translation matrix.  L = 6.  Left is the full translation matrix.  Right shows the two rotation matrices bookending the z-axial translation matrix.}
   \label{figdiag}
\end{figure}

While one can reduce the recurrence relations for the full translation matrices given in \cite{chew1992recurrence, chew1993efficient} to the axial case, equations for the axial scalar translation are derived directly in \cite{mackowski1991analysis}, from which the axial vector translations follow. These are given succinctly in \cite{duan2015experimental}.

The reason for the dramatic speed up of the matrix-vector multiplication in the case of diagonalization over the full matrix is because the rotation matrix is block diagonal and the $z$-axial translation requires the fewest non-zero terms of all possible translation directions. Together these yield many fewer overall multiplications than when using the full matrix, and the computational advantage grows as the matrix grows. In addition, the rotation matrix only needs to be computed once (the inverse is just conjugate). If the translation matrix is non-square, as will be the case when the number of harmonics between two frames is different, then the rotation matrix can be computed once for the larger of $L$ and $L'$, which are the largest degree along row and column, respectively, and then truncated.  Figure \ref{figdiag} illustrates the diagonalization of the translation matrix. The matrices must be applied individually to the expansion coefficients from right to left, otherwise the benefits of diagonalization are lost.

%[10] W. C. Chew, ?Recurrence Relations for Three-Dimensional Scalar Addi- tion Theorem?, Journal of Electromagnetic Waves and Applications, Vol. 6, No. 2, pp. 133-142, 1992.
%[11] Daniel W. Mackowski, ?Analysis of Radiative Scattering for Multiple Sphere Configurations?, Proceedings: Mathematical and Physical Sciences, Vol. 433, No. 1889, pp. 599-614, Jun. 1991.

\clearpage
\newpage
\section{Scalar Translation Matrix}

In this section we give a routine for the scalar axial translation matrix. We then derive a routine for indexing and computing the inherently sparse scalar axial translation matrix on a 1D array. We also give a routine that accomplishes the diagonalized matrix-vector multiplication and operates on the expansion coefficients directly. Finally, we provide a routine that computes the full scalar translation matrix directly.

\subsection{Scalar Axial Translation Matrix}

The recurrence equations for the scalar axial translation matrix, \eqref{alphazdiag}, follow \cite{mackowski1991analysis}.  The matrix elements are zero except for $m = m'$.  Let $(l,m)$ and $(l',m')$ correspond to the rows and columns, respectively.  While the sums are technically infinite, they are always truncated at maximum orders $L$ and $L'$, respectively.

The calculation is initialized with
\begin{equation}
\alpha_{l,0,0,0} = (-1)^l\sqrt{2l+1}h_l^{(1)}(kr_{ji})
\end{equation}

Next, $\alpha_{l,m,l',\vert l'\vert}$ is computed from
\eq{\alpha_{l,l'+1, l'+1,l'+1} = \sqrt{\dfrac{2l'+3}{2(l'+1)}} \left[\sqrt{\dfrac{(l+l'+1)(l+l')}{(2l-1)(2l+1)}} \alpha_{l-1,l',l',l'}  +   \sqrt{\dfrac{(l-l'+1)(l-l')}{(2l+3)(2l+1)}} \alpha_{l+1,l',l',l'}    \right]
}

%\begin{eqnarray}
%\alpha_{l,l'+1, l'+1,l'+1} &=& \sqrt{\dfrac{2l'+3}{2(l'+1)}} \left[\sqrt{\dfrac{(l+l'+1)(l+l')}{(2l-1)(2l+1)}} \alpha_{l-1,l',l'l'} \right. \nonumber \\
%\ & \ & \left. +   \sqrt{\dfrac{(l-l'+1)(l-l')}{(2l+3)(2l+1)}} \alpha_{l+1,l',l'l'}    \right]
%\end{eqnarray}

and $\alpha_{l,-l', l',-l'} = \alpha_{l,l',l',l'}$.  The remaining coefficients are obtained for $m = \pm l'$ using
\eq{\alpha_{l,m,l'+1,m} = \sqrt{2l'+3}\left[\sqrt{\dfrac{(l+l')(l-l')}{(2l-1)(2l+1)}} \alpha_{l-1,m,l',m}  -  \sqrt{\dfrac{(l+l'+1)(l-l'+1)}{(2l+3)(2l+1)}} \alpha_{l+1,m,l',m}    \right]
}

%
%\begin{eqnarray}
%\alpha_{lm,l'+1,m} &=& \sqrt{2l'+3}\left[\sqrt{\dfrac{(l+l')(l-l')}{(2l-1)(2l+1)}} \alpha_{l-1,m,l'm}  \right.\nonumber \\
%\ & \ & - \left.  \sqrt{\dfrac{(l+l'+1)(l-l'+1)}{(2l+3)(2l+1)}} \alpha_{l+1,m,l'm}    \right] \nonumber \\
%\end{eqnarray}

and $m \ne \pm l'$ using
\begin{eqnarray}
\alpha_{l,m,l'+1,m} &=& \sqrt{\dfrac{(2l'+3)(2l'+1)}{(l'+m+1)(l'-m+1)}}\left[  \sqrt{\dfrac{(l'+m)(l'-m)}{(2l'+1)(2l'-1)}} \alpha_{l,m,l'-1,m}   \right. \nonumber \\
\ & \ & \left. + \sqrt{\dfrac{(l+m)(l-m)}{(2l+1)(2l-1)}}\alpha_{l-1,m,l',m} - \sqrt{\dfrac{(l+m+1)(l-m+1)}{(2l+3)(2l+1)}}\alpha_{l+1,m,l',m} \right] \nonumber \\
\end{eqnarray}

That's about as confusing as it gets.  Some comments on computation:

\begin{enumerate}
\item The equations in \cite{mackowski1991analysis} allow for translation in $\pm z$ directions.  We only translate in $+z$.

\item The algorithm iterates on $l+1$ for an element at $l'+1$. This has a cascading effect that, in order to compute the lower right block at $l = L$ and $l' = L'$, an additional element at $L+1$ is required, and so on.  For example, if the translation matrix is square, we must fill a triangular region to $l = L + L'$ at $l'=0$.  This is explained in detail in \cite{chew1992recurrence}.  Once the computation is finished, the matrix is cropped to the intended size.

\item We need to accommodate non-square matrices for translation of expansions of different harmonic content.  Because of the filling requirement above, it is advantageous to create the fill region in the direction of the larger of $L$ and $L'$.  Rather than rewrite the routine to fill along rows or columns, we can make use of the follow transpose relation
\begin{equation}
\alpha^{ij}_{lm,l'm'} = (-1)^{l+l'}\alpha^{ji}_{lm,l'm'} \label{azflip}
\end{equation}

The triangular fill region is appended to the bottom of the matrix, along the rows up to $\textrm{max}(L,L')$. The matrix is then cropped and transposed if $L' > L$.

\item The computation is the same for $\alpha$ or $\beta$, only the Bessel function changes at the initialization.

\end{enumerate}

\begin{figure}[H]
   \centering
   \includegraphics[width=3.5in]{Translation/Figures/trans1}
   \caption{Scalar axial translation matrix, $L = L' = 8$, $k = 1$, $r_{ji} = 100$. }
   \label{fig4}
\end{figure}

The routine \texttt{alphaz} takes as input the largest row degree, $L$, the largest column degree $L'$, wavenumber $k$, axial translation distance $r_{ji}$, and returns the translation matrix in full, including zeros.  Use string switch \texttt{'rg'} for the regular form. This routine is a good starting point for visualization, but the output matrix should not be used as is for matrix-vector multiplication due to the zeros, or it should be converted to a sparse matrix. The routine is written with the variable $n$ in place of $l'$ for clarity.

{\footnotesize
\VerbatimInput{\code/Translation/alphaz.m}
}

\clearpage
\subsection{Sparse Scalar Axial Translation Matrix}

We want to compute the inherently sparse axial translation matrix on a 1D array, to avoid preallocating a potentially massive, mostly zero, matrix. Indexing is more complicated than for the sparse rotation matrix due to different sized diagonal subblocks as well as the triangular fill region requirement. Analytic formulas for sparse indexing are derived and used below. As implemented, these add a little computation time relative to indexing on the full preallocated matrix.

\begin{figure}[H]
  \centering
 \includegraphics[width=5.5in]{Translation/Figures/sparseindex_2}
   \caption{Indexing the diagonal subblocks of the sparse scalar axial translation matrix. Blue arrows indicate the direction of the linear indexing.  }
   \label{fig2translation}
\end{figure}

The total number of non-zero matrix elements in the scalar axial translation matrix, but not including the required triangular fill region, is counted as
\begin{eqnarray}
N(L,L') &=& \sum_{l' = 0}^{L'} \sum_{l=0}^{L} \sum_{m = -\textrm{min}(l,l')}^{\textrm{min}(l,l')} 1 \label{eq:NLL}\\
\ & = & \sum_{l' = 0}^{L'} \sum_{l=0}^{L} \left( \textrm{2 min}(l,l') + 1 \right) \label{crazysparse}
\end{eqnarray}

This counts the total number of elements in a matrix that has $L \times L'$ subblocks. The number of non-zero elements along the diagonal of a subblock is determined by the smaller of the $l$ and $l'$. See Figure \ref{fig2translation}.  Plugging \eqref{crazysparse} into Wolfram Alpha we get
\begin{equation}
N(L,L')
= \left\{
\begin{array}{ccc}
1 & \ & L =  L' = 0 \\
L'+1 &\ & L = 0, L' >0\\
L + 1 & \ & L' = 0 , L>0\\
-L'^3/3+L'^2 L+L' (2 L+4/3)+L+1 & \ & L'<L, L>0,L'>0 \\
-L^3/3+L^2 L'+L (2 L'+4/3)+L'+1 & \ & L'>L, L>0,L'>0 \\
2/3L^3 + 2L^2 + 7/3L + 1 & \ & L' = L, L>0,L'>0 \\
\end{array}
\right.
 \label{eq:a1}
\end{equation}

Next, the total number of nonzero elements in the triangular fill region, see Figure \ref{fig2translation2}, assuming $L \ge L'$, and assuming that the original matrix is extend along rows, is given by
\begin{equation}
N_{\textrm{fill}}(L,L') = \sum_{l' = 0}^{L'-1} \sum_{l=L+1}^{L + L' - l'} \left( 2 l' + 1 \right) = L'(2L'^2 + 3L' + 1)/6  \label{eq:a2}\end{equation}

\begin{figure}[H]
   \centering
      \includegraphics[width=3.5in]{Translation/Figures/sparseindex_3}
   \caption{Indexing the diagonal subblocks of the triangular fill region of the sparse scalar axial translation matrix. Blue arrows indicate the direction of the linear indexing. }
   \label{fig2translation2}
\end{figure}

Because we have defined the fill region as additional matrix rows, the 1D crop is easiest if we index the diagonal $(l,l')$ subblocks as "row"-major (left-right, top-down), and where we index the diagonals of the subblocks from upper left to lower right. See again Figure \ref{fig2translation}. Once the computation is complete we keep the first $N(L,L')$ elements and apply the transpose if $L' > L$.

The "row"-major 1D linear index for the axial translation matrix, but not including the triangular fill region, is given by
\begin{equation}
I(l,l',m,L') = N(l-1,L') + \sum_{i = 0}^{l'-1}  (2\textrm{min}(l,i) + 1)  + (\textrm{min}(l,l') + m + 1)   \label{eq:a3}
\end{equation}

\noindent where
\begin{equation}
\sum_{i = 0}^{l'-1}  (2\textrm{min}(l,i) + 1) =
\left\{\begin{array}{cc}
l'^2, & l' = 1 \lor ( l+1>l' \land l' > 1) \\
-l^2 - l + 2ll' + l' & \textrm{otherwise} \\
\end{array}
\right\}
\end{equation}

$N(l-1,L')$ counts the submatrix containing all diagonal subblocks up to but not including the desired $l$ row of diagonal blocks. The second term counts completed column blocks in the current row block up that go up to but do not include the desired $l'$. The last term counts the desired diagonal. This counting requires knowledge of $L'$.

Finally, the linear index of an element in the triangular fill region under the conditions that $L \ge L'$, $l > L$, and $0 \le l' \le L'-1$ is
\begin{eqnarray}
I_{\textrm{fill}}(l,l',m,L,L') &=& N(L,L') + \sum_{i= L+1}^{l-1} \left(\sum_{j=0}^{L'-1-(i-L-1)}(2j+1)\right) \nonumber \\
\ & \ & + \left(\sum_{j=0}^{l'-1} (2j + 1)\right) + (l' + m + 1)  \label{eq:a4} \\
\ & =& N(L,L') + \left( 1/6(l-L-1)(2l^2-l(4L+6Lp+7)+2L^2\right. \nonumber \\
\ & \ & \left. +L(6Lp+7)+6(Lp+1)^2)\right) + l'^2 + (l' + m + 1)
\end{eqnarray}

The first term counts the number of elements in the actual matrix.  The second term counts the completed row blocks up to but not including the current row block in the fill triangle.  The third term counts the column blocks in the current row block to go up to but do not include the active diagonal (this is made simpler by the fact that $l' < l$ in the fill region).  The last term indexes the active diagonal.

The routine \texttt{alphazSparse} returns three column arrays of the row index, column index, and matrix entries of the sparse axial translation matrix.  It is the same as \texttt{alphaz} but computed and output on a 1D array. Helper functions are given in Table \ref{sparsetranshelp}. \texttt{Nalphaz} and \texttt{Nalphazfill} count the number of nonzero elements in the matrix and fill region, respectively. \texttt{indAlphazFill} is used to index the sparse matrix including the fill regions; it calls \texttt{indAlphaz} which indexes the primary sparse matrix.  When $L' > L$, the transpose is accomplished by applying \eqref{azflip} and reordering row, column, and array elements to be consistent with the row-major indexing of \texttt{indAlphaz}.

\begin{table}[htbp]
\caption{\texttt{alphazSparse} Helper Functions}
\begin{center}
\begin{tabular}{|c|c|c|}
\hline
Routine & Equation & Description \\
\hline
\texttt{Nalphaz} &   \eqref{eq:a1} & Number of non-zero elements in $L \times L'$ sparse $\alpha_{lm,l'm'}$ \\
\hline
\texttt{Nalphazfill} &  \eqref{eq:a2} & Number of non-zero elements in fill region \\
\hline
\texttt{indAlphaz} &   \eqref{eq:a3} & Row-major 1D linear index in sparse $\alpha_{lm,l'm'}$ \\
\hline
\texttt{indAlphazFill} &  \eqref{eq:a4} & Row-major 1D linear index in sparse fill region \\
\hline
\end{tabular}
\end{center}
\label{sparsetranshelp}
\end{table}%

{\footnotesize
\VerbatimInput{\code/Translation/alphazSparse.m}
}

{\footnotesize
\VerbatimInput{\code/Translation/Nalphaz.m}
}

{\footnotesize
\VerbatimInput{\code/Translation/Nalphazfill.m}
}

{\footnotesize
\VerbatimInput{\code/Translation/indAlphaz.m}
}

{\footnotesize
\VerbatimInput{\code/Translation/indAlphazFill.m}
}

\clearpage
\subsection{Diagonalized Scalar Translation Matrix}

After creating sparse rotation and axial translation matrices, the matrix-vector multiplication should be carried out right to left in sequence otherwise the benefit of diagonalization is lost. For example, if \texttt{a} is a vector of expansion coefficients, and the matrices are in Matlab's sparse matrix format, then the multiplication should be carried out as follows \texttt{b = conj(D)*(Alphaz*(D*a))}.

The routine \texttt{scalarTranslation} takes as input a vector of expansion coefficients in frame $i$, $\bb{a}$, the maximum degrees $L$ and $L'$ of the translation matrix and the translation vector $\br_{ji}$ and returns expansion coefficients in frame $j$, $\bb{b}$. The length of $\bb{a}$ should be $L'^2 + 2L' + 1$.  Use \texttt{'rg'} for the regular form of the translation.  This uses the routines for the sparse forms of the rotation matrix and axial translation matrix to save memory. If the entire matrix is desired then include the optional second output.

{\footnotesize
\VerbatimInput{\code/Translation/scalarTranslation.m}
}

\clearpage
\subsection{Full Scalar Translation Matrix}

Recursion relations to compute the full scalar translation matrix were derived in \cite{chew1992recurrence}. While diagonalization is efficient for large matrices, using the full-matrix recursion is more efficient when the number of rows or columns in the matrix are small. The sums of the additional theorem are technically infinite, however, if the field being translated is known to have low harmonic content, then there is no need to compute the matrix beyond the maximum harmonic. For example, the field expansion of a pure scalar dipole only contains harmonics up to degree $l = 1$ (i.e., $(l,m)$ = (0,0),(1,-1),(1,0),(1,1)). Translating this field as incoming waves in the frame of a large scattering object requires only four matrix columns, but a large number rows. The same computation done with diagonalization needs a square rotation matrix to match the number of harmonics of the rows, which is more computation than necessary. Finally, being able to compute the full matrix is useful for cross checking.

\begin{figure}[H]
\centering
   \includegraphics[width=3.5in]{Translation/Figures/transfull}
   \caption{Scalar translation matrix, $L = L' = 8$, $k = 1$, $r_{ji} = [12, 5, 15]$.}
\end{figure}

Adopting the notation from \cite{chew1992recurrence}, which uses the pairings $(\nu\mu,nm)$ for our $(lm,l'm')$, the recurrence relations for the full scalar translation matrix are initialized with
\eq{\alpha_{\nu\mu,00} = \sqrt{4\pi} (-1)^{\nu}Y^*_{\nu\mu}(\theta,\phi)h_{\nu}^{(1)}(k r)}

\noindent where $(r,\theta,\phi)$ are the spherical coordinates of the vector, $\br_{ji}$, that points from the originating frame to the translated frame. The first recursion relation is
\eq{b_{nn}^{+} \alpha_{\nu\mu,n+1,n+1} = b_{\nu-1,\mu-1}^{+} \alpha_{\nu-1,\mu-1,nn}  + b_{\nu+1,\mu-1}^{-} \alpha_{\nu+1,\mu-1,nn}}

This is also used to iterated on $b_{n,-n}^{+} \alpha_{\nu\mu,n+1,-(n+1)} $. The second relation is
\eq{a_{nm}^{+} \alpha_{\nu\mu,n+1,m}  = -a_{nm}^{-} \alpha_{\nu\mu,n-1,m} + a_{\nu-1,\mu}^{+} \alpha_{\nu-1,\mu,nm} + a_{\nu+1,\mu}^{-} \alpha_{\nu+1,\mu,nm} }

The coefficients are
\ea{a_{nm}^{+} &=& -\sqrt{ \dfrac{(n+m+1)(n-m+1)}{(2 n+1)(2 n+3)}} \\
a_{nm}^{-} &=& \sqrt{\dfrac{(n+m)(n-m)}{(2n+1)(2n-1)}} }
\ea{b_{nm}^{+} &=& \sqrt{\dfrac{(n+m+2)(n+m+1)}{(2n+1)(2n+3)}} \\
b_{nm}^{-} &=& \sqrt{\dfrac{(n-m)(n-m-1)}{(2n+1)(2n-1)} }}
Like the axial translation matrix, the computation requires a triangular fill region. As before, the fill region is placed along the dimension of the matrix with the highest degree harmonic, because we only need to fill to the smaller of $L$ and $L'$. We always fill along rows, so if the matrix needs to be flipped, the following transpose relation is used
\eq{\alpha_{mn,\nu\mu} = (-1)^{\nu+\mu+m+n} \alpha_{\nu,-\mu,n,-m}}

The routine \texttt{alpha} computes the full scalar translation matrix using the recursion relations above.  It takes as input the largest row degree, $L$, the largest column degree $L'$, wavenumber $k$, axial translation vector $\br_{ji}$, and returns the translation matrix linearly indexed along rows and columns. Use string switch \texttt{'rg'} for the regular form. The computation fills along rows for the larger of $L$ and $L'$, then transposes the matrix if necessary. This routine matches the outputs of the diagonalized version to machine precision.

{\footnotesize
\VerbatimInput{\code/Translation/alpha.m}
}

\clearpage
\newpage

\section{Vector Translation Matrix}

This section gives routines for the vector axial translation matrix, sparse vector axial translation matrix, and an implementation of the diagonalized matrix-vector multiplication.
\vspace{-1mm}
\subsection{Vector Axial Translation Matrix}

The vector axial translation matrices are found directly from the scalar axial translation matrix with the following relations:
\begin{eqnarray}
A_{l,m,l',m} &=& k r_{ji}\left[\dfrac{1}{(l+1)}\sqrt{\dfrac{(l+m+1)(l-m+1)}{(2l+3)(2l+1)}} \alpha_{l+1,m,l',m} \right. \nonumber \\
\ & \ & + \left. \dfrac{1}{l}\sqrt{\dfrac{(l+m)(l-m)}{(2l+1)(2l-1)}}\alpha_{l-1,m,l',m} \right] + \alpha_{l,m,l',m} \\
B_{l,m,l',m} &=& \dfrac{imkr_{ji}}{l(l+1)}\alpha_{l,m,l',m}
\end{eqnarray}
\begin{equation}
A_{l,m,l',m'} = B_{l,m,l',m'} = \alpha_{l,m,l',m'} = 0, \quad \quad m \ne m'
\end{equation}

Because the first equation uses $l+1$, the scalar axial matrix needs one extra degree $l = L+1$.  The equations in \cite{mackowski1991analysis} appear to be missing several scaling factors, but \cite{chew1993efficient} gives correct equations.  Both \cite{mackowski1991analysis, chew1993efficient} derived translation matrices for partially normalized vector spherical waves functions.  For fully normalized vector spherical wave functions, the vector translation matrices need to be modified as
\begin{eqnarray}
\widetilde{A}_{l,m,l',m'} &=& \sqrt{\dfrac{l(l+1)}{l'(l'+1)}} A_{l,m,l',m'}\\
\widetilde{B}_{l,m,l',m'} &=& \sqrt{\dfrac{l(l+1)}{l'(l'+1)}}B_{l,m,l',m'}
\end{eqnarray}

The routine \texttt{AzBz} takes as input the maximum harmonic degrees $L$ and $L'$, wavenumber $k$, and magnitude of the z-axis translation, $r_{ji}$, and returns vector translation matrices $A_{l,m,l',m}$ and $B_{l,m,l',m}$. Use string switch \texttt{'rg'} for the regular form of the matrices, and \texttt{'norm'} for fully normalized vector wave functions.

{\footnotesize
\VerbatimInput{\code/Translation/AzBz.m}
}

\subsection{Sparse Vector Axial Translation Matrix}

Because the vector axial translation matrices are computed directly from the scalar functions, they are sparse and can be computed on a 1D array.  We only need to count the number of elements in the vector axial translation matrix and determine its sparse linear index. The number of elements in the sparse vector axial translation matrix is counted the same as the scalar case starting at $l=1$ and $l'=1$.
\begin{eqnarray}
N(L,L') &=& \sum_{l' = 1}^{L'} \sum_{l=1}^{L} \sum_{m = -\textrm{min}(l,l')}^{\textrm{min}(l,l')} 1 \label{eq:NLL}\\
\ & = & \sum_{l' = 1}^{L'} \sum_{l=1}^{L} \left[ 2 \textrm{min}(l,l') + 1 \right]
\end{eqnarray}

Plugging this into Wolfram Alpha
\begin{equation}
N(L,L')
= \left\{
\begin{array}{ccc}
3 & \ & L =  L' = 1 \\
3L &\ & L' = 1, L >1\\
3L' & \ & L = 1 , L' >1\\
- 1/3L'^3  + L L'^2 + 2 LL' + 1/3L' & \ & L > 1, L' > 1, L > L'\\
-1/3L^3  + L'L^2 + 2L'L + 1/3L& \ &  L > 1, L' > 1, L'>L \\
L^2 + 5L + 2/3L'^3 + L'^2 - 14/3L' & \ & L > 1, L' > 1, L = L' \\
\end{array}
\right.
\end{equation}

Counting row-major, the sparse linear index is
\begin{equation}
I(l,l',m,L') = N(l-1,L') + \sum_{i = 1}^{l'-1}  (2\textrm{min}(l,i) + 1)  + (\textrm{min}(l,l') + m + 1)
\end{equation}

\noindent where
\begin{equation}
\sum_{i = 1}^{l'-1}  (2\textrm{min}(l,i) + 1) =
\left\{\begin{array}{ccc}
3 & \ & l + 1 \ge l' \land ((l=1 \land l' \ge 2) \lor (l' = 2 \land l > 1 )) \\
2l(l'-2) + l' + 1 & \ & l + 1 < l' \land ((l=1 \land l' \ge 2) \lor (l' = 2 \land l > 1 )) \\
l'^2 - 1  & \ & l > 1 \land l+1 \ge l' \land l > 2 \\
-l^2 + l + (2l + 1)(l'-1)& \ & \textrm{otherwise} \\
\end{array}
\right\}
\end{equation}

The routine \texttt{AzBzSparse} returns four column arrays of the row index, column index, and matrix entries of the sparse axial translation matrices.  It is the same as \texttt{AzBz} but computed and output on a 1D array.  The helper functions are \texttt{NAzBz.m} and \texttt{indAzBz.m}.

{\scriptsize
\VerbatimInput{\code/Translation/AzBzSparse.m}
}

{\scriptsize
\VerbatimInput{\code/Translation/NAzBz.m}
}

{\scriptsize
\VerbatimInput{\code/Translation/indAzBz.m}
}

\clearpage
\subsection{Diagonalized Vector Translation Matrix}

The routine \texttt{vectorTranslation} takes as input a vector of expansion coefficients in frame $i$, $\bb{a}$ and $\bb{b}$, the maximum degrees $L$ and $L'$ of the translation matrix and the translation vector $\br_{ji}$ in Cartesian coordinates and returns expansion coefficients in frame $j$, $\bb{c}$ and $\bb{d}$. The lengths of $\bb{a}$ and $\bb{b}$ should be $L'^2 + 2L' $.  Use \texttt{'rg'} for the regular form and \texttt{'norm'} for fully normalized vector wave functions. This uses the routines for the sparse versions of the rotation matrix and axial translation matrices to save memory.  If the pair of matrices is desired, then include the optional third and fourth outputs.

{\scriptsize
\VerbatimInput{\code/Translation/vectorTranslation.m}
}
\clearpage

\subsection{Full Vector Translation Matrix}

Like the full scalar translation matrix, recursion relations to compute the full vector translation matrices were derived in \cite{chew1993efficient}. Two versions are available: the first iterates on the vector matrices from scratch, the second iterates on the full scalar translation matrix. We implement the second version, which requires computing the scalar translation matrix at a maximum harmonic degree one higher than that of the vector matrices.

\begin{figure}[H]
   \centering
   \subfigure{\includegraphics[width=3in]{Translation/Figures/transfullA1}}
   \subfigure{\includegraphics[width=3in]{Translation/Figures/transfullB1}} \\
   \subfigure{\includegraphics[width=3in]{Translation/Figures/transfullA2}}
   \subfigure{\includegraphics[width=3in]{Translation/Figures/transfullB2}}
   \caption{Vector translation matrices for $L = L' = 8$, $k = 1$, $r_{ji} = [12, 5, 15]$. Top row: matrices for partially normalized vector wave functions. Bottom row: matrices for fully normalized vector wave functions.}
   \label{transfullAB}
\end{figure}

Adopting the notation from \cite{chew1993efficient}, which uses the pairings $(\nu\mu,nm)$ for our $(lm,l'm')$, the recurrence relations for the two vector translation matrices derived from the scalar translation matrix are
\ea{A_{\nu\mu,nm} &=& \alpha_{\nu\mu,nm} +
c_{1,\nu\mu} \alpha_{\nu+1,\mu-1,nm} +
c_{2,\nu\mu} \alpha_{\nu-1,\mu-1,nm} +
c_{3,\nu\mu} \alpha_{\nu+1,\mu+1,nm} + \nonumber \\
\ & \ &
c_{4,\nu\mu} \alpha_{\nu-1,\mu+1,nm} +
c_{5,\nu\mu} \alpha_{\nu+1,\mu,nm} +
c_{6,\nu\mu} \alpha_{\nu-1,\mu,nm}
\label{Anumu} \\
B_{\nu\mu,nm} &=& c_{7,\nu\mu} \alpha_{\nu\mu,nm} +
c_{8,\nu\mu} \alpha_{\nu,\mu+1,nm} +
c_{9,\nu\mu} \alpha_{\nu,\mu-1,nm} \label{Bnumu} }

Note, \eqref{Anumu} and \eqref{Bnumu} only operate on the row indices $(\nu,\mu)$. The coefficients are
\ea{
c_{1,\nu\mu} &=& \dfrac{kr\sin\theta e^{-i\phi}}{2(\nu + 1)}\sqrt{\dfrac{(\nu-\mu+2)(\nu-\mu+1)}{(2\nu + 1)(2\nu + 3)}} \\
c_{2,\nu\mu} &=&  -\dfrac{kr\sin\theta e^{-i\phi}}{2\nu}\sqrt{\dfrac{(\nu+\mu-1)(\nu+\mu)}{(2\nu - 1)(2\nu + 1)}} \\
c_{3,\nu\mu} &=& -\dfrac{kr\sin\theta e^{i\phi}}{2(\nu + 1)}\sqrt{\dfrac{(\nu+\mu+2)(\nu+\mu+1)}{(2\nu + 1)(2\nu + 3)}} \\
c_{4,\nu\mu} &=&  \dfrac{kr\sin\theta e^{i\phi}}{2\nu}\sqrt{\dfrac{(\nu-\mu)(\nu-\mu-1)}{(2\nu - 1)(2\nu + 1)}}\\
c_{5,\nu\mu} &=& \dfrac{kr\cos\theta}{\nu+1}\sqrt{\dfrac{(\nu+\mu+1)(\nu-\mu+1)}{(2\nu + 1)(2\nu + 3)}} \\
c_{6,\nu\mu} &=&  \dfrac{kr\cos\theta}{\nu}\sqrt{\dfrac{(\nu+\mu)(\nu-\mu)}{(2\nu - 1)(2\nu + 1)}}  \\
c_{7,\nu\mu} &=& \dfrac{i\mu kr\cos\theta }{\nu(\nu+1)} \\
c_{8,\nu\mu} &=& \dfrac{i kr\sin\theta e^{i\phi} }{2\nu(\nu+1)} \sqrt{(\nu-\mu)(\nu+\mu+1)} \\
c_{9,\nu\mu} &=& \dfrac{i kr\sin\theta e^{-i\phi} }{2\nu(\nu+1)} \sqrt{(\nu+\mu)(\nu-\mu+1)}  }

\noindent where $(r,\theta,\phi)$ are the spherical coordinates of the vector $\br_{ji}$ that points from the originating frame to the translated frame.

The routine \texttt{AB} takes as input the maximum harmonic degrees $L$ and $L'$, wavenumber $k$, and translation vector, $r_{ji}$, and returns full vector translation matrices $A$ and $B$. It calls \texttt{alpha} to compute the full scalar translation matrix. Use string switch \texttt{'rg'} for the regular form of the matrices, and \texttt{'norm'} for fully normalized vector wave functions.

{\footnotesize
\VerbatimInput{\code/Translation/AB.m}
}


% declare document class and geometry
%\documentclass[12pt]{article} % use larger type; default would be 10pt
\input{../plaheader.tex}
\usepackage[margin=1in]{geometry} % handle page geometry

\title{Phys 220A -- Classical Mechanics -- Lec06}
\author{UCLA, Fall 2014}
\date{\formatdate{21}{10}{2014}} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed

\begin{document}
\setlength{\unitlength}{1mm}
\maketitle

\section{Small oscillations}

Generally, in classical mechanics, the dynamics may often by too complicated to solve exactly, or even numerically. Often we will have a Lagrangian of the form
\begin{equation}
L = \frac{1}{2} \sum_{ij} M_{ij}(q) \dot{q}_i \dot{q}_j + \sum_i N_i (q) \dot{q}_i - V(q),
\end{equation}
or we will be able to approximate it in this form to order $\dot{q}^2$, assuming no time dependence. This gives us stability around equilibrium, which is of course important physically. An example where $N_i$ comes into play would be the potential for a particle coupled to the EM field. For driven systems we will have different stability properties. In condensed matter, small energy excitations around equilibrium are called phonons.

Equilibrium points occur at the extrema of the potential, that is at solutions $q_i^{(0)}$ to $\pd{V}{q_i} = 0$. The Euler-Lagrange equations are then solved by $q_i(t) = q_i^{(0)}$ for all $t$, i.e. $\dot{q}_i = 0$. Under small oscillations we will have
\begin{equation}
q_i = q_i^{(0)} + \eta_i(t) + \bigo(\eta^2)
\end{equation}
and we must expand the Lagrangian to quadratic order (i.e. linearize),
\begin{align}
L &= \frac{1}{2} \sum_{ij} M_{ij} (q^{0}) \dot{\eta}_i \dot{\eta}_j + \sum_i N_i (q^{0}) \dot{\eta}_i + \sum_{ij} \eval{\pd{N_i}{q_j}}_{q=q^0} \eta_j \dot{\eta}_i - V(q^0) \\
	& \qquad - \sum_i \eval{\pd{V}{q_i}}_{q = q^0} \eta_i - \frac{1}{2} \eval{\md{V}{2}{q_i}{}{q_j}{}}_{q = q^0} \eta_i \eta_j + \bigo(n^3). \\
	&\sim \frac{1}{2} \sum_{ij} m_{ij} \dot \eta_i \dot \eta_j - \frac{1}{2} \sum_{ij} v_{ij} \eta_i \eta_j + \sum_{ij} p_{ij} \eta_i \dot \eta_j + \bigo(\eta^3),
\end{align}
where in the second expression we've thrown out the $\sum_i N_i (q^0) \dot{\eta}_i$ and $V(q^0)$ terms since they are just a total derivative and an overall constant, and
\begin{equation}
m_{ij} = M_{ij}(q^0), \qquad p_{ij} = \eval{\pd{N_i}{q_j}}_{q = q^0}, \text{and} \qquad v_{ij} = \eval{\md{V}{2}{q_i}{}{q_j}{}}_{q = q^0}
\end{equation}
are constant matrices. Now, if $p_{ij}$ has a symmetric part $p_{(ij)}$ we can throw it out and just consider it antisymmetric since
\begin{equation}
p_{(ij)} \eta_i \dot{\eta}_j = \frac{1}{2} (p_{ij} + p_{ji}) \eta_i \dot{\eta}_j = p_{ij} (\eta_i \dot{\eta}_j + \eta_j \dot{\eta}_i) = \od{}{t} (\frac{1}{2} p_{ij} \eta_i \eta_j)
\end{equation}
is just a total derivative.

\subsection{First analysis}

As a first analysis, let's set $p_{ij} = 0$. Then we can write
\begin{equation}
L = \frac{1}{2} \dot{\eta}^\top m \dot{\eta} - \frac{1}{2} \eta^\top v \eta + \bigo(\eta^3),
\end{equation}
for which we can write the equation of motion in terms of the vector $\bar{\eta} = (\eta_1, \dots, \eta_n)$,
\begin{equation}
m \ddot{\bar{\eta}} + v \bar{\eta} = 0.
\end{equation}
This has ansatz $\bar{\eta} = \bar{\gamma} e^{i \omega t}$ which, plugging back in to the equation of motion, we have
\begin{equation}
(-\omega^2 m + v) \bar{\gamma} = 0.
\end{equation}
Thus $\omega$ is determined by the vanishing of the determinant
\begin{equation}
\det (-\omega^2 m + v) = 0.
\end{equation}

However are all directions stable? What does our eigenvector look like? Recall some facts about linear algebra. Both matrices $m_{ij}$ and $v_{ij}$ are symmetric and can hence be diagonalized by orthogonal transformations we have
\begin{equation}
\bigo^T m \bigo = I \begin{array}{c} m_1 \\ m_2 \\ ... \\ m_n\end{array} = \bar{m}
\end{equation}
We'll assume that all $\bar{m_i}$ are positive (i.e. all masses are in general positive). Then we have a well defined square root of m. We can write
\begin{equation}
\bar{m}^{1/2} \bigo^T m \bigo \bar{m}^{-1/2} = 1
\end{equation}
Defining
\begin{equation}
n = \bigo \bar{m}^{1/2} \bar{n}
\end{equation}
we have
\begin{equation}
L = \frac{1}{2} \bar{\dot{n}}^T \bar{n} - \frac{1}{2} \bar{n}^T \bar{V} \bar{n}
\end{equation}
where
\begin{equation}
\bar{m}^{1/2} \bigo^T v \bigo \bar{m}^{-1/2} = \bar{v}
\end{equation}
is still a symmetric matrix

Our characteristic equation then becomes
\begin{equation}
\ddot{\bar{n}} + \bar{v} \bar{n} = 0
\end{equation}
An ansatz of
\begin{equation}
\bar{n} = e^{i\omega t} \gamma
\end{equation}
gives
\begin{equation}
(-\omega^2 1 + \bar{v})\gamma = 0
\end{equation}
If the eigenvalues of $\bar{v}$ are all real and nonzero then the oscillations are stable. Certain directions can be stable depending on the sign of the eigenvalues. Given the eigenvalues the equations of motion are oscillatory with frequencies $\omega =  \sqrt{\lambda_i}$

Thus our equatons of motion are given by
\begin{equation}
\ddot{\bbar{\eta}}_i + \lambda_i \bbar{\eta}_i = 0, \qquad i = 1, \dots, N.
\end{equation}
where in the stable case we have $\lambda_i = \omega_i^2$, the characteristic frequencies. So the normal modes are decoupled 1-dimensional oscillators along $\bbar{\eta}_i$ with coordinate relationship
\begin{equation}
\eta = O \bar{m}^{-1/2} \bar{O} \bar{\bar{\eta}}.
\end{equation}

In the more general case with $\eta_{ij} \neq 0$, we have
\begin{equation}
L = \frac{1}{2} \dot{\bar{\eta}}^\top \bar{\eta} + \bar{\eta}^\top \bar{p} \dot{\bar{\eta}} - \frac{1}{2} \bar{\eta}^\top \bar{v} \bar{\eta}
\end{equation}
where
\begin{equation}
\bar{p} = \bar{m}^{-1/2} O^\top p O \bar{m}^{-1/2}.
\end{equation}
So the Euler-Lagrange equations are
\begin{equation}
\ddot{\bar{\eta}} + \bar{v} \bar{\eta} - 2 \bar{p} \dot{\bar{\eta}} = 0
\end{equation}
which has a new characteristic equation for the ansatz $\bar{\eta} = \gamma e^{i\omega t}$,
\begin{equation}
\det (-\omega^2 + \bar{v} - 2i \omega \bar{p}) = 0
\end{equation}
which determines $\omega$ and $\gamma$ in general. These cases are important for a particle in an EM field and motion in non-inertial frames.

\subsection{Double pendulum}

Let's take the simple case that
\begin{equation}
m_1 = m_2 = m
\end{equation}
\begin{equation}
l_1 = l_2 = l
\end{equation}
Our Lagrangian in this case is
\begin{equation}
L = ml^2 \dot{\theta_1}^2 + \frac{1}{2}ml^2 \dot{\theta_2}^2 + ml^2 \dot{\theta_1}\dot{\theta_2} \cos(\theta_1 - \theta_2) + 2mgl \cos \theta_1 + mgl\cos\theta_2
\end{equation}
In equilibrium
\begin{equation}
\theta_1 = \theta_2 = 0
\end{equation}
Let's expand around  this equilibrium and drop the small terms and constants
\begin{equation}
L = ml^2 \dot{\theta_1}^2 + \frac{1}{2} ml^2 \dot{\theta_2}^2  + ml^2 \dot{\theta_1} \dot{\theta_2} - mgl \theta_1^2 - \frac{1}{2} mgl\theta_2^2
\end{equation}

\begin{equation}
m_ij = \left( \begin{array}{cc} 2ml^2 & ml^2 \\ ml^2 & ml^2 \end{array}\right)
\end{equation}
\begin{equation}
v_{ij} = \left(\begin{array}{cc} 2mgl & 0\\ 0 & mgl \end{array}\right)
\end{equation}
Our Euler-Lagrange equations are then
\begin{equation}
\left(\begin{array}{cc} 2ml^2 & ml^2 \\ ml^2 & ml^2 \end{array}\right) \left( \begin{array}{c} \ddot{\theta_1} \\ \ddot{\theta_2} \end{array}\right) + \left( \begin{array}{cc} 2mgl & 0 \\ 0 & mgl \end{array}\right) \left(\begin{array}{c} \theta_1 \\ \theta_2\end{array}\right) = 0
\end{equation}
Dividing by $ml^2$
\begin{equation}
\left(\begin{array}{cc} 2 & 1 \\ 1 & 1 \end{array}\right) \left( \begin{array}{c} \ddot{\theta_1} \\ \ddot{\theta_2} \end{array}\right) + \left( \begin{array}{cc} \frac{2g}{l} & 0 \\ 0 & \frac{g}{l} \end{array}\right) \left(\begin{array}{c} \theta_1 \\ \theta_2\end{array}\right) = 0
\end{equation}
Multiplying on both sides by the inverse of the first term
\begin{equation}
 \left( \begin{array}{c} \ddot{\theta_1} \\ \ddot{\theta_2} \end{array}\right) + \frac{g}{l}\left( \begin{array}{cc} 2 & -1 \\ -2 & 2 \end{array}\right) \left(\begin{array}{c} \theta_1 \\ \theta_2\end{array}\right) = 0
\end{equation}
We then obtain the frequencies from the eigenvalue equation
\begin{equation}
det \left( -\omega^2 + \frac{g}{l} \left( \begin{array}{cc} 2 & -1 \\ -2 & 2\end{array} \right)\right ) = 0
\end{equation}
This results in the equation
\begin{equation}
\omega^4 - \frac{4g}{l} \omega^2 + \frac{2g^2}{l^2} = 0
\end{equation}
\begin{equation}
\omega^2_\pm = \frac{2g}{l} \left( 1 \pm \frac{1}{\sqrt{2}}\right)
\end{equation}
The eigenvector for the positive frequency is
\begin{equation}
v_+ = \left(\begin{array}{c} -\frac{1}{\sqrt{2}} \\ 1 \end{array} \right)
\end{equation}
This corresponds to the segments oscillating in opposite directions. For the negative frequencies
\begin{equation}
v_- = \left(\begin{array}{c} \frac{1}{\sqrt{2}} \\ 1 \end{array} \right)
\end{equation}
which corresponds to the segments oscillating in the same direction

\section{Lagrange Points}
Consider a two-body planet system. The two masses move in circular
orbits about their common center of mass, with equal velocities. We
can transform into this frame by rotating our frame by the angular
velocity. The Lagrange points are points where you can put a test
mass and it will be stationary \textbf{in the rotating frame}. In the
rest frame, it has the same angular velocity as the other
masses. From the inertial frame the points have positions
\begin{equation}
\v{x_1} = \left(\begin{array} r_1 \cos\omega_t \\ r_1 \sin \omega_t \end{array}\right)
\end{equation}
\begin{equation}
\v{x_2} = \left(\begin{array}{c} -r_2 \cos\omega_t \\ -r_2 \sin \omega_t \end{array}\right)
\end{equation}
where d is the separation between the two, and $r_1$ are and $r_2$ are the distances of each of the particles from the center of mass, such that
\begin{equation}
d = r_1 + r_2
\end{equation}
\begin{equation}
M = m_1 + m_2
\end{equation}
\begin{equation}
m_1 r_1 = m_2 r_2
\end{equation}
This results in the expressions
\begin{equation}
r_1 = \frac{m_2}{m_1 + m_2} d
\end{equation}
and
\begin{equation}
r_2 = \frac{m_1}{m_1 + m_2} d
\end{equation}
This frame rotates with a frequency determined by Kepler's law and is unaffected by the third mass.
\begin{equation}
G (m_1 + m_2) = d^3 \omega^2
\end{equation}
In the inertial frame we thus have the Lagrangian
\begin{equation}
\frac{1}{2} \dot{x}^2 + \frac{Gm m_1}{|\v{x} - \v{x_1}|}
+ \frac{Gmm_2}{|\v{x} - \v{x_2}|}
\end{equation}
In general this is dificult since everything depends on t. Let's transform into a rotating frame with angular velocity $\omega$ such that
\begin{equation}
\v{x} = \left( \begin{array}{c} x\cos\omega t - y\sin \omega t \\
x \sin \omega t + y \cos \omega t \\ z\end{array} \right)
\end{equation}

\begin{equation}
\v{x} - \v{x_1} =\left( \begin{array}{c} x\cos\omega t - y\sin \omega
t - r_1 \cos\omega t \\
x \sin \omega t + y \cos \omega t - r_1 \sin\omega t \\z\end{array} \right)
\end{equation}
So we find that the magnitude by multiplying and skipping the gross algebra
\begin{equation}
|\v{x} - \v{x_1}| = \sqrt{(x - r_1)^2 + y^2 + z^2}
\end{equation}
$R_2$ is more or less the same except we flip the sign on r
\begin{equation}
|\v{x} - \v{x_2}| = \sqrt{(x - r_2^2 + y^2 + z^2}
\end{equation}
Now, when we take all of this into our Lagrangian, we can get the
kinetic energy from our normal motion in a rotational frame. We have a
centripetal term (the first) and a centrifugal term (the second)
\begin{equation}
L = \frac{1}{2} m [ \dot{x} + \dot{y} + \dot{z} + \omega^2 (x^2 +
y^2)+ 2\omega (x\dot{y} - y\dot{x})] + Gm [\frac{m_1}{\sqrt{(x -
r_1)^2 + y^2 + z^2}} + \frac{m_2}{\sqrt{(x - r_2^2 + y^2 + z^2)}}]
\end{equation}
Okay, so from common sense, we know that the stability points cannot
be anywhere except for the z plane, there are no forces to stabilize
them in any other plane. The name of the game now is to have a balance
of these forces while minimizing the effective potential. Our first
Lagrange point is simply a balance between the gravitational
forces. This point is stationary in both the inertial frame and the
non-inertial frame so we don't have to consider coriolis forces or
centripetal.

Now let's consider the other two forces. We have two points that move
in circular orbits in the inertial frame. These points only experience
centripetal forces which balance the gravitational forces. From the
math, these points form equilateral triangles with the two masses.

The third point is behind each of the masses. These
points only have inward pointing forces from the gravitational forces,
but if we combine the centripetal and centrifugal forces (which point
in the same direction) they cancel the gravitational forces.

Let's consider the stability of the noncolinear points
The effective potential for is
\begin{align}
\phi_{eff} &= \frac{G\mu}{2R^3}(x^2 + y^2) - \frac{Gm_1}{\sqrt{(r_1 + x)^2 + y^2}} + \frac{Gm_2}{\sqrt{(r_2 - x)^2 + y^2}}
\end{align}
Now we take the first derivative. For our Lagrange points this derivative is equal to zero
\begin{align}
\d{\phi_{eff}}{x} &= \frac{G\mu}{R^3}x - \frac{Gm_1 (r_1-x)}{((r_1 - x)^2 + y^2)^{3/2}} + \frac{Gm_2(r_2 + x)}{((r_2 + x)^2 + y^2)^{3/2}}
\end{align}
Since $r_1 - x = r_2 + x$
\begin{align}
\d{\phi_{eff}}{x} &= \frac{G\mu}{R^3}x - G\frac{m_1  r_1- m_1x - m_2 r_2 - m_2 x}{((r_1 - x)^2 + y^2)^{3/2}} \\
&= \frac{G\mu}{R^3}x + G\frac{ x(m_1 + m_2)}{((r_1 - x)^2 + y^2)^{3/2}} &= 0 \\
\end{align}
For y we have
\begin{align}
\d{\phi_{eff}}{y} &= \frac{G\mu}{R^3} y - \frac{Gm_1y}{((r_1 -x)^2 + y^2)^{3/2}} + \frac{Gm_2 y }{((r_2 + x)^2 + y^2)^{3/2}}\\
&= \frac{G\mu}{R^3} y + \frac{G(m_1 + m_2)y}{((r_1 -x)^2 + y^2)^{3/2}}
\end{align}

Okay, now to do the stability analysis, let's perturb the Lagrangian by
\begin{equation}
x = x_L + \epsilon_x
\end{equation}
and
\begin{equation}
y = y_L + \epsilon_y
\end{equation}
\begin{equation}
\d{}{t} \pd{L}{\dot{\epsilon}} = \pd{L}{\epsilon}
\end{equation}
\begin{equation}
\d{}{t} ( m\dot{\epsilon_x} - 2\omega \epsilon_y) = m\ddot{\epsilon_x} - 2\omega \dot{\epsilon_y} =  \epsilon_x \pdd{\phi_{eff}}{x} + \epsilon_y \frac{\partial^2 \phi_{eff}}{\partial x\partial y}
\end{equation}
Where the partial derivatives are all evaluated at the Lagrange points
\begin{equation}
\ddot{\epsilon_y} + 2\omega\dot{\epsilon_x} = \epsilon_y \pdd{\phi_{eff}}{y} + \epsilon_x \frac{\partial^2 \phi_{eff}}{\partial x\partial y}
\end{equation}
Let's take these partial derivatives
\begin{equation}
\pdd{\phi_{eff}}{x} = \frac{G\mu}{R^2} + \frac{G(m_1 + m_2)}{((r_1 - x)^2 + y^2)^{3/2}} + \frac{G(m_1 + m_2)x}{((r_1 - x)^2 + y^2)^{3/2}}
\end{equation}
The first and second term are zero due to the first derivative.
\begin{equation}
\pdd{\phi_{eff}}{x} = \frac{\frac{3}{2}G(m_1 + m_2)x(r_1 - x)}{((r_1 - x)^2 + y^2)^{5/2}}
\end{equation}
We found above that
\begin{equation}
x = \frac{1}{2}(r_1 - r_2)
\end{equation}
\begin{equation}
y = \frac{\sqrt{3}}{2} (r_1 + r_2)
\end{equation}
and
\begin{equation}
(r_1 - x)^2 + y^2 = d^2
\end{equation}
\begin{equation}
\pdd{\phi_{eff}}{x} = \frac{\frac{3}{2}G(m_1 + m_2)x((\frac{1}{2}r_1 + \frac{1}{2}r_2)}{d^5}
\end{equation}
\begin{equation}
\pdd{\phi_{eff}}{x} = \frac{3}{4} x_l\frac{G(m_1 + m_2)}{R^3}= -\frac{3}{4} \frac{G(m_1 + m_2)}{d^3} = -\frac{3}{4} \omega^2
\end{equation}
You get the idea. The rest are derived in the textbook so I'll skip to the punchline
\begin{equation}
\ddot{\epsilon_x} - 2\omega \dot{\epsilon_y} - \frac{3}{4} \omega^2 \epsilon_x + \frac{3\sqrt{3}}{4} \frac{m_1 - m_2}{m_1 + m_2} \omega^2 \epsilon_y = 0
\end{equation}
\begin{equation}
\ddot{\epsilon_x} +2\omega \dot{\epsilon_y} - \frac{9}{4} \omega^2 \epsilon_x + \frac{3\sqrt{4}}{4}\frac{m_1 - m_2}{m_1 + m_2}\omega^2 \epsilon_y = 0
\end{equation}
We make the ansatz
\begin{equation}
\epsilon_x = c_1 e^{i\omega_s t}
\end{equation}
\begin{equation}
\epsilon_y = c_2 e^{i\omega_s t}
\end{equation}
Which gives us two equations
\begin{equation}
-\omega_s^2 c_1 - 2i\omega \omega_s c_2 - \frac{3}{4} \omega^2 c_1 + \alpha \omega^2 c_2 = 0
\end{equation}
where we have defined
\begin{equation}
\alpha = \frac{3\sqrt{4}}{4}\frac{m_1 - m_2}{m_1 + m_2}
\end{equation}
\begin{equation}
-c_2 \omega_s^2 + 2i\omega \omega_sc_1 - \frac{9}{4} \omega^2 c_2 + \alpha \omega^2 c_1 = 0
\end{equation}
Eliminating the constants we get
\begin{equation}
- 16\omega_s^4 + 16 \omega^2\omega_s^2 + (-27+16\alpha^2)\omega^4  = 0
\end{equation}
Using quadratic formula
\begin{equation}
\omega_s^2 = \frac{1}{32}\left[ 16\omega^2 \pm \sqrt{16^2 \omega^4 - 64 (-27 + 16 \alpha^2) \omega^4}\right]
\end{equation}
\begin{equation}
\omega_s^2 = \omega^2\left[ \frac{1}{2} \pm \sqrt{\frac{1}{4}  - \frac{1}{16} (-27 + 27 \frac{m_1 - m_2}{m_1 + m_2}) \omega^4}\right] = \omega^2\left[ \frac{1}{2} \pm \sqrt{-\frac{23}{16} + \frac{27}{16} \frac{m_1 - m_2}{m_1 + m_2}) \omega^4}\right]
\end{equation}
Ourstability condition is
\begin{equation}
\frac{1}{2} > \sqrt{-\frac{23}{16} + \frac{26}{16}( (\frac{m_1}{m_1 + m_2})^2 + \frac{m_1}{m_1 + m_2})}
\end{equation}
Solving we get
\begin{equation}
\mu < \frac{27 - \sqrt{621}}{54} \approx .038
\end{equation}

\end{document}


\chapter{Outline of lectures in PKU summer school}

Lecture 1:

Lecturer: Jinchao
TA:	  Qingguo

Introduction to finite element spaces
Interpolation error estimates for 1D

Lecture 2:  Jun Hu

N-term approximation [for Haar basis: Lin)
Interpolation error estimates for adaptive grids [Juncai discuss with Chen Long]

Lecture 3:

Error estimates for sparse grids

Introduction to DNN
Approximation property for general activation function (non-polynomial)
Jones's approximation result for cosine activation function
Barron's result in L2 (homework)
(Notes: Chunyue)

Lecture 4:

DNN with ReLU (paper)
Approximation of CNN (for 1d data: curves:  Juncai)
Sprecher representation theorem (no proof, Xuhao)

Lecture 5: Spectral approximation for ReLU DNN
1. $x^2$ approximation and $x*y$
2. abstract function class properties
3. spectral approximation properties


% !TeX root = ./apxthy.tex

\section{Tensor Products and Sparse Grids}
%
\label{sec:sparse}

\subsection{Introduction and Motivation}
%
\label{sec:sparse:intro}
%
We now consider the approximation of functions $f : [a,b]^d \to \R$. We will
explore (1) how to construct ``good'' approximations and (2) what effect the
dimension $d$ has on the rates of approximation.

We will build multi-variate approximations from tensor products of uni-variate
basis functions. For example, a polynomial in two dimensions can be written
as
\[
  p(x_1, x_2) = \sum_{k_1 = 0}^N \sum_{k_2 = 0}^N c_{k_1k_2} x_1^{k_1} x_2^{k_2}.
\]
The function $(x_1, x_2) \mapsto x_1^{k_1}x_2^{k_2}$ is called the tensor
product between the two functions $x_j \mapsto x_j^{k_j}$.

We know of course from our univariate theory that the Chebyshev basis has
many advantageous algorithmic and approximation properties, thus we may
prefer to write $p(x_1, x_2)$ in the form
\[
    p(x_1, x_2) =
    \sum_{k_1 = 0}^N \sum_{k_2 = 0}^N
    \tilde{c}_{k_1k_2} T_{k_1}(x_1) T_{k_2}(x_2).
\]
In a general dimension $d > 1$ we will write
\begin{align*}
    \bx &= (x_1, \dots, x_d), \\
    \bk &= (k_1, \dots, k_d), \\
    T_{\bk}(\bx) &= \prod_{\alpha = 1}^d T_{k_\alpha}(x_\alpha),
\end{align*}
and consider polynomials
\[
    p(\bx) = \sum_{\bk \in \mathcal{K}} c_{\bk} T_{\bk}(\bx),
\]
where $\calK \subset \N^d$ is a suitable index set.

The definition of $T_{\bk}$ can also equivalently be written as
\[
    T_{\bk} = T_{k_1} \otimes T_{k_2} \otimes \cdots \otimes T_{k_d}
\]

However we write these multi-variate polynomials we can see the ``curse of
dimensionality'' creep in already: the number of coefficients to represent a
$d$-variate polynomial of degree $N$ is clearly $(1+N)^d$. Suppose our
computational budget is one million coefficients (quite a lot!) and we are
working in 10 dimensions. What is the maximal degree we may choose? But we will
see that the notion of degree has no unique extension to higher dimensions and
this will sometimes save us.

Before we proceed, we need to introduce some minimal amount of additional
notation.

\subsection{The Curse of Dimensionality}
%
\label{sec:sparse:curse}
%
As a first attempt at constructing high-dimensional approximations we
apply the one-dimensional techniques in each coordinate direction. For
example, let us assume that $f : [-1,1]^2 \to \R$ has any regularity
we may later need, then we can first apply a Chebyshev interpolant
in the $x_1$-direction which we denote by $I_N^{(1)}$
which becomes a polynomial in $x_1$ but a general function in $x_2$, i.e.,
\[
    I^{(1)}_N f(x_1, x_2) =
    \sum_{k_1 = 0}^N c_{k_1}(x_2) T_{k_1}(x_1).
\]
We then apply an interpolant in the $x_2$-direction to obtain
\[
    I^{(2)}_N I^{(1)}_N f(x_1, x_2)
    = \sum_{k_1 = 0}^N I^{(2)}_N c_{k_1}(x_2) T_{k_1}(x_1)
    = \sum_{k_1, k_2 = 0}^N c_{k_1k_2} T_{k_1(x_1)} T_{k_2}(x_2)
    = \sum_{\bk \in \{0\dots N\}^2} c_\bk T_\bk(\bx).
\]
More generally, we define the $d$-dimensional Chebyshev interpolation
operator to be
\[
    I^{(1..d)}_N :=  I^{(d)}_N I^{(d-1)}_N \cdots I^{(1)}_N
\]

To estimate the error committed, let us assume that $x_\alpha \mapsto f(\bx) \in
C^j([-1,1])$ for all $\alpha$, then we can bound
\begin{align*}
    \b\|f -  I^{(d)}_N I^{(d-1)}_N \cdots I^{(1)}_N f\b\|_\infty
    &\leq
    \b\|f -  I^{(d)}_N f\b\|_\infty
    + \b\| I^{(d)}_N f - I^{(d)}_N I^{(d-1)}_N \cdots I^{(1)}_N f \b\|_\infty  \\
    &\leq
    C (\log N) N^{-j} \| \partial_{x_d}^j f \|_\infty
    + \b\| I^{(d)}_N \big[ f - I^{(d-1)}_N \cdots I^{(1)}_N f\big] \b\|_\infty
    \\
    &\leq
    C (\log N) \Big\{ N^{-j} \| \partial_{x_d}^j f \|_\infty
    + \b\| f - I^{(d-1)}_N \cdots I^{(1)}_N f \b\|_\infty \Big\}.
\end{align*}
Arguing by induction we obtain the following result.

\begin{theorem} \label{th:sparse:curse}
    (1) Let $x_\alpha \mapsto f(\bx) \in C^j([-1,1])$ for $\alpha = 1, \dots,
    d$, then
    \[
        \b\|f -  I^{(1..d)}_N f\b\|_\infty
        \leq C (\log N)^d N^{-j} \sum_{\alpha = 1}^d \| \partial_{x_\alpha}^j f \|_\infty
    \]
    (2) Let $x_\alpha \mapsto f(\bx) \in A(E_\rho)$ for some $\rho > 1$ and for
    all $\bx \in [-1,1]^d-1$, then
    \[
        \b\|f -  I^{(1..d)}_N f\b\|_\infty
        \leq
        C  M_f (\rho-1)^{-d} (\log N)^d \rho^{-N} \sum_{\alpha = 1}^d M^{(\alpha)},
    \]
    where $M^{(\alpha)} = \|f\|_{L^\infty(E_\rho^{(\alpha)})}$, where
    $E_\rho^{(\alpha)} = [-1,1] \times \cdots \times [-1,1] \times E_\rho
    \times [-1,1] \times \cdots \times [-1,1]$ and the position of the
    Bernstein ellipse is in the $\alpha$th coordinate.
\end{theorem}

Let us translate these estimates into cost-error relations. The number of basis
functios for a $d$-dimensional tensor product basis of degree $N$ is $(1+N)^d$
and this is directly proportional to the associated computational cost. Further,
even though the terms $(\log N)^d$ are significiant they arise from sub-optimal
estimates of the interpolation error (cf. \cite{Trefethen2013-rg} for sharp
estimates) hence we ignore them.
Letting $\epsilon := \b\|f -  I^{(1..d)}_N f\b\|_\infty$, we therefore obtain
\[
    {\rm Cost} \approx N^d \lesssim
        \left\{\begin{array}{ll}
            \epsilon^{d/j}, & \text{case (1)}; \\
            (\log \epsilon)^{d}, & \text{case (2)}.
        \end{array}\right.
\]
The exponential dependence of these estimates on $d$ is what we call the ``curse
of dimensionality'': without new ideas and new information it becomes
exponentially harder to approximate functions in high dimension.

\subsection{Chebyshev series and greedy approximation}
%
\label{sec:sparse:chebseries}
We begin by making precise the notion of a multi-variate Chebyshev series,
which has already been at the back of our mind since the beginning of
\S~\ref{sec:sparse}. To that end, we define the $d$-dimensional Chebvyshev
space
\begin{align*}
    L^2_{\rm C}([-1,1]^d) &:= \b\{ f : [-1,1]^d \text{measurable, and }
                                   \| f\|_{L^2_{\rm C}} < \infty \b\},
                                        \qquad \text{where} \\
    &\|f\|_{L^2_{\rm C}}^2
        := \int_{[-1,1]^d}  |f(x)|^2 \, \prod_{\alpha = 1}^d (1 - x_\alpha^2)^{-1/2} \, d\bx.
\end{align*}

Note how the weight in this integral is simply the tensor product of the
univariate weights. Because of this, we have the orthogonality (Exercise: check
this!)
\begin{equation}
    \label{eq:sparse:dDorth}
    \b\< T_\bk, T_{\bk'} \b\>_{L^2_{\rm C}} = \delta_{\bk\bk'}.
\end{equation}
That is, $\{ T_\bk \}_{\bk \in \N^d}$ is an ortho-normal subset of $L^2_{\rm C}$.
The result from the previous section moreover indicates density of
their linear combinations (polynomials) and after making this precise we will
obtain the following result:

\begin{theorem} \label{th:sparse:mvchebseries}
    (i) Let $f \in L^2_{\rm C}([-1,1]^d)$, then there exist coefficients
    $\tilde{f}_\bk, \bk \in \N^d$ such that
    \[
        f = \sum_{\bk \in \N^d} \tilde{f}_\bk T_\bk,
    \]
    where the convergence of the series in is the $L^2_{\rm C}$-norm.

    (ii) Moreover, we have a multi-variate Plancherel Theorem
    \begin{equation}
        \label{eq:sparse:plancherel}
        \|f\|_{L^2_{\rm C}}^2 = \sum_{\bk \in \N^d}
            \b| \tilde{f}_\bk \b|^2.
    \end{equation}

    (iii) If $(\tilde{f}_\bk)_{\bk \in \N^d} \in \ell^1(\N^d)$, then $f$ is continuous
    and the convergence is in the max-norm (i.e., uniform in $[-1,1]^d$).
\end{theorem}

We can now translate the  Hilbert-space best approximation results
to the multivariate setting. For every finite set $\mathcal{K} \subset \N^d$
we have a Chebyshev series truncation
\[
    p_\calK := \tilde\Pi_{\calK} f := \sum_{\bk \in \calK} \tilde{f}_\bk T_\bk
\]
The resulting error is
\[
    \| f - p_\calK \|_{L^2_{\rm C}}^2
        = \sum_{\bk \in \N^d \setminus \calK}
        |\tilde{f}_\bk|^2
\]
which we can minimise as follows:
\begin{enumerate}
\item Compute the coefficients $\tilde{f}_\bk$ and order them by magnitude,
\[
    |\tilde{f}_{\bk_1}| \geq |\tilde{f}_{\bk_2}| \geq \dots
\]
\item Given a ``budget'' $M$, let $\calK := \{\bk_1, \dots, \bk_M\}$
\item Then, $p_\calK$ is the {\em best $M$-term approximation} to $f$
        in the $L^2_{\rm C}$-norm.
\end{enumerate}

\begin{remark}
    It is not clear at all that minimising the number of terms in $\calK$
    optimises the computational cost to evaluate $p_\calK$ for a given target
    error, due to the fact that the basis functions are computed via a recursion
    formula, but there are far more signifiicant issues with computing with best
    $M$-term approximations, hence we will not explore this any further.
\end{remark}

By a similar mechanism we can also get an $L^\infty$-error bound,
\[
    \|f-p_\calK\|_{\infty}
    \leq \sum_{\bk \in \N^d \setminus \calK}
    \b|\tilde{f}_\bk\b|,
\]
using the fact that $|T_\bk(\bx)| \leq 1$. However, it is not at all obvious yet
whether this bound is close to optimal.

\begin{proof}[Proof of Theorem~\ref{th:sparse:mvchebseries}]
    We will use, without proof, the fact that continuously differentiable
    functions are dense in $L^2_{\rm C}$. This can be proven e.g. by using a
    multi-variate Jackson-type approximation. Then Theorem~\ref{sec:sparse:curse}
    implies that polynomials are dense in $L^2_{\rm C}$. The remaining
    statements are straightforward.
\end{proof}

\subsection{Sparse Grids}
%
\label{sec:sparse:sparse}
%
In practise it is rare (though not impossible) to have the explicit coefficients
$\tilde{f}_\bk$ available, which makes it exceedingly difficult to develop
``greedy algorithms''; however, see \cite{DeVore1998-do} for an extensive review
article. However, one may have some more generic qualitative information,
such as genuine multi-variate versions of $C^j$ regularity or analyticity.

Before we motivate the next definition let us assume that $f$ has not only
univariate partial derivatives $\partial_{x_\alpha}^j f$ but also mixed
derivatives, e.g., $\partial_{x_1} \partial_{x_2} f$. Unfortunately, this
idea is difficult to motivate in the Chebyshev basis, so we temporarily
switch to multi-variate trigonometric polynomials. That is,
\[
    f \in L^2(\TT^d)
\]
and the multi-variate Fourier series is given by
\[
    f(\bx) = \sum_{\bk \in \Z^d} \hat{f}_\bk
        e^{i \bk\cdot \bx},
\]
where we note that
\[
    e^{i \bk\cdot \bx} =
    e^{i \sum_{\alpha = 1}^d k_\alpha x_\alpha}
    \prod_{\alpha = 1}^d e^{i k_\alpha x_\alpha},
\]
i.e. this is again precisely the same setting as before.

Assuming that $f : \TT^2 \to \R$ has two mixed derivatives, then just
calculating formally,
\begin{align*}
    \partial_{x_1} \partial_{x_2} f
    &=
    \sum_{\bk \in \Z^2} \hat{f}_\bk
    \partial_{x_1} e^{ik_1x_1} \partial_{x_2} e^{ik_2x_2} \\
    &=
    \sum_{\bk \in \Z^2} - k_1 k_2 \hat{f}_\bk
    e^{i \bk \cdot \bx}.
\end{align*}
Thus, if $\partial_{x_1} \partial_{x_2} f  \in L^2(\TT^2)$, then we have
that
\[
    \sum_{\bk \in \Z^2} |k_1|^2 |k_2|^2 |\hat{f}_{\bk}|^2 < \infty.
\]
This gives us some first information about the decay of $\hat{f}$ that we
can exploit. Specifically, the best information we have is that we should choose
index sets of the form
\[
    \mathcal{K}_N^{\rm hc} =  \b\{
            |k_1| |k_2| \leq N \b\}.
\]
This is an example of the ``hyperbolic cross'' approximation. We will
return to this idea again in a moment.

The foregoing example shows how genuine multi-variate regularity
relates directly to decay of Fourier coefficients, and one can show more
for $f: [-1,1]^d \to \R$ that it relates to the decay of Chebyshev
coefficients. It is therefore expedient to define the following function classes:
\begin{align*}
    \mathcal{A}_\omega^{(2)}
    &:=
        \B\{ f \in L^2_{\rm C} \Bsep
             \sum_{\bk \in \N^d} \omega(\bk)^2 |\tilde{f}_\bk|^2 < \infty \B\} \\
             %
    \mathcal{A}_\omega^{(\infty)}
    &:=
    \B\{ f \in L^2_{\rm C} \Bsep
    \sum_{\bk \in \N^d} \omega(\bk) |\tilde{f}_\bk| < \infty \B\}.
\end{align*}
Informally, the ``faster'' $\omega(\bk)$ grows as $|\bk| \to \infty$ the
``smoother'' is $f$. In the following we will focus on
$\mathcal{A}_\omega^{(\infty)}$ but analogous results also hold for
$\mathcal{A}_\omega^{(2)}$.

There are or course limitless possibilities, we will consider two specific
cases:
\[
    (1) \quad \omega(\bk) = \prod_{\alpha = 1}^d (1+k_\alpha)^j,
    \qquad \text{and} \qquad
    (2) \quad \omega(\bk) = \prod_{\alpha = 1}^d \rho^{k_\alpha} = \rho^{\sum_\alpha k_\alpha},
\]
where $j \geq 1$ and $\rho > 1$. The case (1) is related to mixed $C^j$
regularity, while the case (2) corresponds to analyticity of $f$ in
$E_{\rho}^d$. Both cases can be extended in obvious ways to more anisotropic
conditions, e.g., $\omega(\bk) = \prod_{\alpha = 1}^d \rho_\alpha^{k_\alpha}$
and similar.

{\bf Warning: } Case (1) is in fact difficult to obtain practise except with very strongly dimension-dependent constants. In particular the hyperbolic cross approximation we derive from this below is not how it is used in practise.
However, (1) is similar in spirit to $\omega(\bk) = \prod_\alpha \max(1, k_\alpha)^j$, which leads to the {\em true} hyperbolic cross approximation.

As Chebyshev coefficients we choose the indices that are in the
sublevel of $\omega$, i.e.,
\begin{align*}
    \calK_N^{\rm hc} &:= \B\{ \bk \in \N^d \Bsep \prod_{\alpha=1}^d (1+k_\alpha) \leq N \B\},
    \qquad \text{in Case (1);} \\
    \calK_N^{\rm tot} &:= \B\{ \bk  \in \N^d \Bsep \sum_{\alpha = 1}^d k_\alpha \leq N \B\},
    \qquad \text{in Case (2).}
\end{align*}
Case (1) is called the {\em hyperbolic cross approximation} scheme, while Case
(2) is called the {\em sparse grid approximation}. $\sum_\alpha k_\alpha$ is also called the {\em total degree} of the polynomial $T_{\bk}$.

\begin{theorem} \label{th:sparse:grids}
    (1) Suppose that $f \in \mathcal{A}_\omega^{(\infty)}$ with
    $\omega(\bk) = \prod_\alpha (1+k_\alpha)^j$, then
    \begin{align}
        \label{eq:sparse:hc_error}
        \| f - \tilde\Pi_{\calK_N^{\rm hc}} f \|_\infty
            &\leq M_f N^{-j}, \qquad \text{while} \\
        \label{eq:sparse:hc_nbasis}
        \#\calK_N^{\rm hc} &\leq N \log^{d-1} N.
    \end{align}
    where $M_f := \sum_{\bk \in \N^d} \omega(\bk) |\tilde{f}_\bk|$.

    (2) Suppose that $f \in \mathcal{A}_\omega^{(\infty)}$ with
    $\omega(\bk) = \rho^{|\bk|_1}$ where $\rho > 1$, then
    \begin{align}
        \label{eq:sparse:sp_error}
        \| f - \tilde\Pi_{\calK_N^{\rm hc}} f \|_\infty
            &\leq M_f \rho^{-N}, \qquad \text{while} \\
        \label{eq:sparse:sp_nbasis}
        \#\calK_N^{\rm tot} &= {N+d \choose d},
    \end{align}
    where $M_f := \sum_{\bk \in \N^d} \omega(\bk) |\tilde{f}_\bk|$.
\end{theorem}

Let $\epsilon := \| f - \tilde\Pi_{\calK_N^{*}} f \|_\infty$, then  in
case (1), hyperbolic cross, we obtain
\[
    {\rm Cost} \approx \#\calK_N^{\rm hc}
    \lesssim \epsilon^{1/j} \log^{d-1} \epsilon^{1/j}.
\]

In case (2), sparse grid, the cost-error estimate is a little more involved.
First, we use Stirling's formula to estimate
\begin{align*}
    {N+d \choose d} &\approx \sqrt{\frac{2\pi (N+d)}{2\pi N 2\pi d}}
            \frac{ ((N+d)/e)^{N+d}}{(N/e)^N (d/e)^d} \\
    &\lesssim
    \B(1+\frac{d}{N}\B)^N \B(1+\frac{N}{d}\B)^d.
\end{align*}
We distinguish two cases, $d \ll N$ and $N \ll d$:
\[
    \#\calK_N^{\rm tot} \lesssim
    \left\{\begin{array}{rl}
        e^d (1+N/d)^d, & N \gg d, \\
        e^N (1+d/N)^N, & N \ll d.
    \end{array} \right.
\]
In the $N \gg d$ case, which is the one more relevant for moderately high
dimension we can now readily obtain
\[
    {\rm Cost} \approx \#\calK_N^{\rm tot}  \lesssim
    \B( \frac{c |\log \epsilon|}{d}\B)^d,
\]
which does not entirely remove the curse of dimensionality, but it substantially
ameliorates it. Contrast this with the estimate for a tensor product
basis, ${\rm Cost} \approx |\log \epsilon|^d$.

% The case $d \gg N$ is more difficult.

It is somewhat striking that the ``good'' case when $f$ is analytic uses less
agressive sparsification and also ameliorates the curse of dimensionality to a
lesser degree. That said, the rate of approximation is still better of course,
and this can be seen in the occurance of $\log \epsilon$ instead of
$\epsilon^{1/j}$ in the estimate.

\subsubsection{Proofs}

Throughout the following proofs we write $p_N := \tilde\Pi_{\calK}f$ where
$\calK = \calK_N^{\rm tot}$ or $\calK = \calK_N^{\rm hc}$. Further, we define Let
$M_{Nd} := \#\calK$ in dimension $d$.

\begin{proof}[Proof of \eqref{eq:sparse:hc_error}]
    \begin{align*}
        \|f - p_N\|_\infty
        &\leq
        \sum_{\prod (1+k_\alpha)>N} |\tilde{f}_\bk| \\
        &\leq
        N^{-j}
        \sum_{\prod (1+k_\alpha)>N}  \prod_\alpha (1+k_\alpha)^j |\tilde{f}_\bk| \\
        &\leq
        M_f N^{-j}. \qedhere
    \end{align*}
\end{proof}

\begin{proof}[Proof of~\eqref{eq:sparse:hc_nbasis}]
    For $d = 1$ we have $M_{N,1} = N+1$. For $d > 1$ we can create a
    recursion
    \begin{align*}
        M_{N,d}
        &= \sum_{k_{d} = 0}^N M_{\lfloor N/(1+k_d) \rfloor, d-1} \\
        &\leq \sum_{k_d = 0}^N \log^{d-1} \b[ (N+2)/(1+k_d) \b] \frac{N+1}{1+k_d} \\
        &\leq (N+1) \log^{d-2} (N+2) \sum_{k_d = 0}^N \frac{1}{1+k_d} \\
        &\leq (N+1) \log^{d-1} (N+2).  \qedhere
    \end{align*}
\end{proof}

\begin{proof}[Proof of~\ref{eq:sparse:sp_error}]
    This is analogous as  \eqref{eq:sparse:hc_error},
    \begin{align*}
        \|f - p_N\|_\infty
        &\leq \sum_{\sum k_\alpha > N} |\tilde{f}_\bk| \\
        &\leq \rho^{-N} \sum_{\sum k_\alpha > N} \rho^{|\bk|_1} |\tilde{f}_\bk| \\
        &\leq M_f \rho^{-N}. \qedhere
    \end{align*}
\end{proof}
\begin{proof}[Proof of~\eqref{eq:sparse:sp_nbasis}]
    This is a simple combinatorics problem: The set $\calK_N^{\rm tot}$ can be
    interpreted as the set of all $d$-element multi-sets from
    $\{0, \dots, N\}$, which gives the stated expression.
    % or it can be proven by induction:
    % for $d = 1$ the result is obvious. For the induction step, we have
    % \begin{align*}
    %     M_{N,d+1} &= \sum_{k_{d+1} = 0}^N M_{N-k_{d+1}, d} \\
    %     &= \sum_{k_{d+1} = 0}^N \frac{(N-k_{d+1}+1)\cdots(N-k_{d+1}+d)}{d!} \\
    %     &= \sum_{k = 0}^N \frac{(k+1)\cdots(k+d)}{d!} \\
    %     &= \frac{(N+1) \cdots (N+d+1)}{(d+1)!}.
    % \end{align*}
    % The final step needs further comment: we are claiming that
    % \[
    %     {{N+d+1} \choose {d+1}} = \sum_{k = 0}^N {k+d \choose d}
    % \]
\end{proof}


\documentclass[12pt]{article}
\usepackage{graphics}
\begin{document}
\newcounter{problem}
\thispagestyle{empty}

\section*{random extra problems}

\paragraph{Problem~\theproblem}\refstepcounter{problem}%
Consider a slab of air with density $\rho$ of height $\Delta y$ and
perpendicular (that is, horizontal) area $A$, with volume $A\,\Delta
y$.  If this air is in equilibrium with its surroundings, it neither
rises nor falls, so its weight is supported by a pressure difference
$\Delta P$.

\textsl{(a)}~Draw a free-body diagram for the slab and find an
expression for the pressure difference $\Delta P$ in terms of $\Delta
y$ and whatever else you need.

\textsl{(b)}~In the ideal gas law, there is a relationship between
$P$, $V$, $N$, and $T$.  Convert this into a relationship between $P$
and $\rho$.  \emph{Hint:} Recall that $\rho$ is a mass per unit
volume; you might need to use the average mass of an air molecule.

\textsl{(c)}~Rearrange your result from part (a) using your result
from part (b) to get a differential equation that relates
$\mathrm{d}\rho/\mathrm{d}y$ to $\rho$.  Integrate this equation to
get the density $\rho$ as a function of $y$ for the atmosphere,
assuming that it is \emph{isothermal} (it isn't).

\textsl{(d)}~Comment on the relationship between your answer and the
Boltzmann factor discussed in your textbook.

\paragraph{Problem~\theproblem}\refstepcounter{problem}%
In practice problem 14.58, you will show that in one period of
oscillation, a weakly damped harmonic oscillator loses energy $\Delta
E = 2\pi\,E/Q$, where $E$ is the mechanical energy in the oscillator
and $Q$ is the quality factor, defined to be the ratio
$Q\equiv\omega_0/\gamma$.  If this oscillator is driven by a driving
force so that instead of decaying, its amplitude is held constant, how
much average mechanical power does the driving force have to supply to
the oscillator?  A typical mechanical clock has a pendulum with a mass
of 2~kg, a period of 1~s, and a quality factor of 20.  What is the
power in W required to keep this pendulum swinging at an amplitude of
4~cm?

\paragraph{Problem~\theproblem}\refstepcounter{problem}%
Consider a one-dimensional wave on a string of the form
\begin{equation}
y = y_0 \cos(k\,x + \omega\,t)
\end{equation}
with $y_0= 1.0\times 10^{-2}~\mathrm{m}$, $k= 1.0~\mathrm{m^{-1}}$,
and $\omega= 2.0~\mathrm{s^{-1}}$.  What is the velocity $\vec{v}$
(speed and direction) of the wave?  Draw three quantitative diagrams
showing (a)~the displacement $y$ as a function of time $t$ for a fixed
point at $x= +1.047~\mathrm{m}$, (b)~the displacement $y$ as a
function of position $x$ for a fixed time $t=0$, and (c)~the
displacement $y$ as a function of position $x$ for a fixed time $t=
1.047~\mathrm{s}$.

\paragraph{Problem~\theproblem}\refstepcounter{problem}%
Two wave pulses, labeled A and B, are approaching each other on a rope
as shown.  The rope has mass per unit length $\mu= 0.1~{\rm
kg\,m^{-1}}$ and is held under constant tension $T= 10~{\rm N}$.
Recall that although the waves travel in the $+x$ and $-x$ directions,
each piece of the rope is only moving in the $y$ direction.
\\ \rule{0.1\textwidth}{0pt}
\resizebox{0.8\textwidth}{!}{\includegraphics{../pro/twopulse.eps}}

(a) What is the wave speed $c$ in the rope?

(b) Draw a graph of the instantaneous $y$-direction speed $v_y$ of the
rope as a function of $x$ for the instant shown above.  Be
quantitative; carefully label all interesting positions and speeds.
Also, pay attention to the {\em sign} (positive or negative) of the
speed.

(c) On your graph, point out the locations where the rope feels a $y$
component of force and indicate the sign (positive or negative) of the
force at each location.

\paragraph{Problem~\theproblem}\refstepcounter{problem}%
A continuation of Problem 3: Draw a graph of the rope configuration
at the time at which the two wave pulses are exactly coincident.  In
what form is the ``energy'' of the wave pulses at this moment?

\paragraph{Problem~\theproblem}\refstepcounter{problem}%
In a mechanical watch, the oscillator is a metal wheel with moment of
inertia $I$ attached to a torsional spring.  Model the torsional
spring as making a restoring torque $\tau$ which is proportional to
its angular displacement $\theta$ from its equilibrium orientation by
the equation
\begin{equation}
\tau = -\kappa\,\theta \nonumber
\end{equation}
Derive the differential equation relating $\theta$ to its second
derivative (with respect to time).  What is the period $T$ of
oscillation?  As the watch gets warmer, the metal wheel expands.  If
it expands (in radius) by a factor of $1.0001$ (ie, $1+10^{-4}$), does
the period go up or down?  If the watch originally kept perfect time,
how many seconds per day does it gain or lose after the expansion?
Assume (incorrectly) that the torsional spring is unaffected by the
thermal expansion.

\end{document}


\documentclass[journal,onecolumn]{IEEEtran}
%\usepackage[left=2.2cm,right=2.2cm,top=2.5cm,bottom=2.5cm]{geometry}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{minted}
\usepackage{booktabs}
%\usepackage{commath}
\usepackage{float}
\usepackage{mathtools}
\usepackage{color}
\usepackage{amsthm}
\usepackage{parskip}
\usepackage{bm}

\usepackage[binary-units=true]{siunitx}

\newcommand{\py}[1]{\mintinline{python}{#1}}

\title{Artificial Intelligence (\texttt{LINGI2261}) \\ Assignment 4 --- Group 13}
\author{Martin Braquet, Gilles Peiffer}
\date{December 11, 2019}

\begin{document}

\maketitle

\section{The Bin Packing Problem}
\begin{enumerate}
	\item The bin packing problem can be formulated as a local search problem as follows:
	\begin{itemize}
		\item The \emph{problem} is to find a partition of a finite set \(U\) of items of rational size \(0 \le s(u) \le 1\) for each \(u \in U\) into disjoint subsets \(U_1, \ldots, U_k\) such that the sum of the item sizes in each \(U_i\) is no more than 1, and such that \(k\) is as small as possible.

		This is equivalent to the problem of packing a finite set of items of integer size (\(s(u) \in [0, C]\)) into bins of capacity \(C\), while minimizing the number of bins.

		A state is represented by a list of bins, each containing the indexes of the blocks inside this bin.
		\item The \emph{cost function}, which we want to maximize, is
		\[
		f(k, \bm{\mathrm{fullness}}) = -\mathrm{Fitness} =  \left(\frac{\sum_{i=1}^{k} \left(\frac{\mathrm{fullness}_i}{C}\right)^2}{k}\right) - 1,
		\]
		where \(k\) is the number of bins, \(\bm{\mathrm{fullness}}\) is a vector of size \(k\) containing the total weight of each bin and \(C\) is the capacity of the bins.
		This is the opposite of the Fitness coefficient, where the sign originates from a desire to have a maximization problem.
		\item A solution is feasible if every item is in a bin and
		\[
		\mathrm{fullness}_i \le C \quad \textnormal{for} \quad i = 1, \ldots, k.
		\]
		We call the set of feasible bin packings \(\mathcal{S}\).
		\item A solution \(\sigma\) is optimal if
		\[
		s \in \mathcal{S} \implies f(\sigma) \ge f(s).
		\]
		Equivalently, one can write the set of optimal solutions as \(\{s \in \mathcal{S} : f(s) = \max_{p \in \mathcal{S}} f(p)\}\).
	\end{itemize}
	\item The initial solution is constructed by taking the various items in the order in which they are given, and to try, for each one, to add it to the current bin.
	If this is possible (i.e., the bin's capacity is not exceeded), then add it and move to the next bin.
	If not, create a new bin and add the item to that bin, then move to the next item.
	This is the strategy which was initially implemented in the code template.

	The successor functions works by generating the two kinds of swap moves, and yielding the resulting states.
	That is, given an input state, it tries to find all possible item-item and item-blank space swaps which do not violate the feasibility constraints and yields a successor state where the bins have been updated to reflect the change.

	Both functions have been submitted and verified on INGInious, obtaining a score of 15/15.
	\item  The results for the three strategies on each of the ten given instances are detailed in Table~\ref{time1}.\footnote{Experiments were run on an Early 2015 MacBook Pro, running macOS Sierra 10.12.6, using a \SI{2.9}{\giga\hertz} Intel Core i5 processor, with \SI{8}{\giga\byte} of \SI{1867}{\mega\hertz} DDR3 RAM and an Intel Iris Graphics 6100 GPU.}
	When applicable, instances were tested multiple times and the average values were taken.
	The optimality value is given using the cost function detailed higher up in this document.
	When the sign is reversed, this becomes the fitness measure defined in the assignment's statement.

	\begin{table}[H]
		\centering
		\begin{tabular}{c@{\hspace{0.7cm}}ccc|ccc|ccc}
			\toprule
			Inst. & \multicolumn{3}{c}{\py{maxvalue}} & \multicolumn{3}{c}{\py{randomized_maxvalue}} & \multicolumn{3}{c}{\py{random_walk}}  \\
			\midrule
			& Time (ms) & Opt. & Steps & Time (ms) & Opt. & Steps & Time (ms) & Opt. & Steps \\
			\midrule
			1 & 213 & \(-0.0256572\) & 9 & 214 & \(-0.0487638\) & 13.1 & 401 & \(-0.314751\) & 12.7 \\
			2 & 329 & \(-0.0316317\) & 10 & 242 & \(-0.0316621\) & 32.6 & 430 & \(-0.296054\) & 2.3 \\
			3 & 252 & \(-0.231149\) & 5 & 261 & \(-0.231456\) & 15.6 & 409 & \(-0.291955\) & 4.8 \\
			4 & 251 & \(-0.233704\) & 6 & 260 & \(-0.234356\) & 31.5 & 450 & \(-0.295092\) & 1.6 \\
			5 & 546 & \(-0.247307\) & 2 & 357 & \(-0.247385\) & 2.7 & 386 & \(-0.247858\) & 0 \\
			6 & 183 & \(-0.0278449\) & 6 & 219 & \(-0.0278968\) & 28 & 375 & \(-0.259277\) & 2.2 \\
			7 & 197 & \(-0.0417206\) & 4 & 223 & \(-0.0417037\) & 27.4 & 389 & \(-0.272147\) & 1.3 \\
			8 & 240 & \(-0.0370099\) & 4 & 232 & \(-0.037048\) & 5.7 & 385 & \(-0.267996\) & 2.6 \\
			9 & 177 & \(-0.0143129\) & 5 & 186 & \(-0.0143188\) & 14.2 & 398 & \(-0.252326\) & 0.6 \\
			10 & 237 & \(-0.0432446\) & 5 & 222 & \(-0.0432216\) & 25.3 & 434 & \(-0.2732\) & 1.9 \\
			\bottomrule
			\\
		\end{tabular}
		\caption{Comparison of execution time, optimal value and number of steps for the three local search strategies.}
		\label{time1}
	\end{table}
	\item \begin{enumerate}
		\item It is hard to say which strategy is best, since the results for the \py{maxvalue} and \py{randomized_maxvalue} strategies are very similar, with \py{maxvalue} obtaining its optimal value after fewer steps.
		This is slightly counterintuitive, as one expects that the \py{randomized_maxvalue} strategy would outperform the \py{maxvalue} strategy, which is likely to get stuck in local maxima.
		This is probably a byproduct of the low limit on the number of steps.

		There is however no doubt about the fact that both strategies outperform the \py{random_walk} strategy.
		\item The \py{maxvalue} strategy is a simple hill-climbing algorithm, and chooses the best neighbour at each step.
		The other strategies, while less likely to get stuck in local optima, do not necessarily go for the neighbours with the highest value, and thus often end up with worse solutions.
		\item The \py{maxvalue} strategy focuses entirely on intensification, that is, searching for the neighbour with the highest value, but does not take into account diversification (deviating from optimality in order to avoid getting stuck in local optima).

		The \py{random_walk} strategy, on the other hand, focuses entirely on diversification, while not paying attention to intensification: the next neighbour is chosen randomly, regardless of its value.

		Finally, the \py{randomized_maxvalue} strategy tries to find a balance between looking at the values of the neighbours (by taking the best five) and diversifying between those neighbours (by randomizing its choice).
		\item Since the \py{maxvalue} strategy does not diversify its choices, it has little to no chance of escaping local maxima.
		Unless the best neighbour of the local maximum has another neighbour which has an even higher value, the algorithm is going to start jumping back and forth between the maximum and its best neighbour, until the step limit is reached.

		On the other hand, the \py{random_walk} strategy is more likely to escape local maxima since it has a nonzero probability of ``randomly walking'' down the hill it climbed, finding another hill to climb.

		Finally, the \py{randomized_maxvalue} strategy is somewhere in between.
		It has a possibility to escape local maxima, given that the local maxima are not too isolated (so that the random part of the algorithm has a chance of jumping to a new hill).
	\end{enumerate}
\end{enumerate}

\section{Propositional Logic}
\subsection{Models and Logical Connectives}
\begin{enumerate}
	% TODO perhaps we should use a slightly less exhaustive algorithm to answer this question.
	\item The first sentence's truth table is given in Table~\ref{tab:tt1}.
	Since it does not appear in the expression, we omit \(D\).
	\begin{table}[H]
		\centering
		\(
		\begin{array}{ccc@{\quad}ccc}
		\toprule
		A & B & C & \lnot A  \lor C & \lnot B \lor C & (\lnot A  \lor C) \land (\lnot B \lor C)\\
		\midrule
		0 & 0 & 0 & 1 & 1 & 1 \\
		0 & 0 & 1 & 1 & 1 & 1 \\
		0 & 1 & 0 & 1 & 0 & 0 \\
		0 & 1 & 1 & 1 & 1 & 1 \\
		1 & 0 & 0 & 0 & 1 & 0 \\
		1 & 0 & 1 & 1 & 1 & 1 \\
		1 & 1 & 0 & 0 & 0 & 0 \\
		1 & 1 & 1 & 1 & 1 & 1 \\
		\bottomrule\\
		\end{array}
		\)
	\caption{Truth table for \((\lnot A  \lor C) \land (\lnot B \lor C)\).}
	\label{tab:tt1}
	\end{table}
	As one can see, there are ten valid interpretations (five for each value of \(D\)).

	The second sentence's truth table is given in Table~\ref{tab:tt2}.
	Since it does not appear in the expression, we omit \(D\).
	\begin{table}[H]
		\centering
		\(
		\begin{array}{ccc@{\quad}ccc}
		\toprule
		A & B & C & C \implies \lnot A & \lnot(B \lor C) & (C \implies \lnot A) \land \lnot(B \lor C)\\
		\midrule
		0 & 0 & 0 & 1 & 1 & 1 \\
		0 & 0 & 1 & 1 & 0 & 0 \\
		0 & 1 & 0 & 1 & 0 & 0 \\
		0 & 1 & 1 & 1 & 0 & 0 \\
		1 & 0 & 0 & 1 & 1 & 1 \\
		1 & 0 & 1 & 0 & 0 & 0 \\
		1 & 1 & 0 & 1 & 0 & 0 \\
		1 & 1 & 1 & 0 & 0 & 0 \\
		\bottomrule\\
		\end{array}
		\)
		\caption{Truth table for \((C \implies \lnot A) \land \lnot(B \lor C)\).}
		\label{tab:tt2}
	\end{table}
	As one can see, there are four valid interpretations (two for each value of \(D\)).

	The third sentence's truth table is given in Table~\ref{tab:tt3}.
	\begin{table}[H]
		\centering
		\(
		\begin{array}{cccc@{\quad}cccc}
		\toprule
		A & B & C & D & \lnot A \lor B & \lnot(B \implies \lnot C) & \lnot(\lnot D \implies A) & (\lnot A \lor B) \land \lnot(B \implies \lnot C) \land \lnot(\lnot D \implies A)\\
		\midrule
		0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 \\
		0 & 0 & 0 & 1 & 1 & 0 & 0 & 0 \\
		0 & 0 & 1 & 0 & 1 & 0 & 1 & 0 \\
		0 & 0 & 1 & 1 & 1 & 0 & 0 & 0 \\
		0 & 1 & 0 & 0 & 1 & 0 & 1 & 0 \\
		0 & 1 & 0 & 1 & 1 & 0 & 0 & 0 \\
		0 & 1 & 1 & 0 & 1 & 1 & 1 & 1 \\
		0 & 1 & 1 & 1 & 1 & 1 & 0 & 0 \\
		1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
		1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 \\
		1 & 0 & 1 & 0 & 0 & 0 & 0 & 0 \\
		1 & 0 & 1 & 1 & 0 & 0 & 0 & 0 \\
		1 & 1 & 0 & 0 & 1 & 0 & 0 & 0 \\
		1 & 1 & 0 & 1 & 1 & 0 & 0 & 0 \\
		1 & 1 & 1 & 0 & 1 & 1 & 0 & 0 \\
		1 & 1 & 1 & 1 & 1 & 1 & 0 & 0 \\
		\bottomrule\\
		\end{array}
		\)
		\caption{Truth table for \((\lnot A \lor B) \land \lnot(B \implies \lnot C) \land \lnot(\lnot D \implies A)\).}
		\label{tab:tt3}
	\end{table}
	As one can see, there is only one valid interpretation.
\end{enumerate}

\subsection{Color Grid Problem}
\begin{enumerate}
	\item Using propositional logic, one can express the three types of constraints fairly easily.
	We write \(n\) to denote the number of rows/columns/colors.

	To express the fact that a color can only appear once in a given row and column, we write
	\[
	C_{i, j, k} \implies \Bigg(\bigwedge_{0 \le \alpha < n, \alpha \ne j} \lnot C_{i, \alpha, k}\Bigg) \land \Bigg(\bigwedge_{0 \le \alpha < n, \alpha \ne i} \lnot C_{\alpha, j, k}\Bigg).
	\]

	For the diagonal constraints, one should notice that diagonals can be defined by the properties of the indices of the elements contained in them.
	The first type, which we call ``constant-difference'' diagonals, has the property that the difference between the elements of the diagonal is a constant value.
	The second type, which we call ``constant-sum'' diagonals, has the property that the sum of the elements of the diagonal is a constant value.

	To express that a color can only appear once in a given constant-difference diagonal, we write
	\[
	C_{i, j, k} \implies \bigwedge_{0 \le i+\alpha, j+\alpha < n, \alpha \ne 0} \lnot C_{i+\alpha, j+\alpha, k}.
	\]

	To express that a color can only appear once in a given constant-sum diagonal, we write
	\[
	C_{i, j, k} \implies \bigwedge_{0 \le i+\alpha, j-\alpha < n, \alpha \ne 0} \lnot C_{i+\alpha, j-\alpha, k}.
	\]

	One must also assert that for a given square, at least one color must be used:
	\[
	\bigwedge_{\substack{0 \le i < n \\ 0 \le j < n}} \Bigg(\bigvee_{0 \le k < n} C_{i, j, k}\Bigg).
	\]

	All of these constraints must be repeated for all possible values of \((i, j, k)\).

	Additionally, all input values must be asserted as well:
	\[
	\bigwedge_{(i, j, k) \in \textnormal{Inputs}} C_{i, j, k}.
	\]

	Since the number of rows, columns and colors is identical, it is worth noting that the unicity of the color for each case is automatically verified by the row, column and color conditions stated above. Indeed, it is impossible for a cell to have 2 different colors if all the $n$ cells in a row (or column) need to pick a different color among the $n$ colors available.
	\item To put the constraints given in the previous part into conjunctive normal form, one proceeds as follows:
	\begin{align*}
	&\bigwedge_{\substack{0 \le i < n \\ 0 \le j < n \\ 0 \le k < n}} \left(C_{i, j, k} \implies {\underbrace{\bigwedge_{0 \le \alpha < n, \alpha \ne j} \lnot C_{i, \alpha, k}}_{\textnormal{row}}} \land {\underbrace{\bigwedge_{0 \le \alpha < n, \alpha \ne i} \lnot C_{\alpha, j, k}}_{\textnormal{column}}}\right. \\
	&\left. \qquad \qquad \! \! {} \land {\underbrace{\bigwedge_{0 \le i+\alpha, j+\alpha < n, \alpha \ne 0} \lnot C_{i+\alpha, j+\alpha, k}}_{\textnormal{constant-difference diagonal}}} \land {\underbrace{\bigwedge_{0 \le i+\alpha, j-\alpha < n, \alpha \ne 0} \lnot C_{i+\alpha, j-\alpha, k}}_{\textnormal{constant-sum diagonal}}}\right) \\
	&\qquad \qquad \! \! {} \land {\underbrace{\bigwedge_{\substack{0 \le i < n \\ 0 \le j < n}} \Bigg(\bigvee_{0 \le k < n} C_{i, j, k}\Bigg)}_{\textnormal{color}}} \land {\underbrace{\bigwedge_{(i, j, k) \in \textnormal{Inputs}} C_{i, j, k} \vphantom{\bigwedge_{\substack{0 \le j < n \\ 0 \le k < n}}}}_{\textnormal{inputs}}} \\
	=&\bigwedge_{\substack{0 \le i < n \\ 0 \le j < n \\ 0 \le k < n}} \left({\underbrace{\bigwedge_{0 \le \alpha < n, \alpha \ne j} \Big(\lnot C_{i, j, k} \lor \lnot C_{i, \alpha, k} \Big)}_{\textnormal{row}}} \land {\underbrace{\bigwedge_{0 \le \alpha < n, \alpha \ne i} \Big(\lnot C_{i, j, k} \lor \lnot C_{\alpha, j, k}\Big)}_{\textnormal{column}}} \right.\\
	&\left. \qquad \qquad \! \! {}  \land {\underbrace{\bigwedge_{0 \le i+\alpha, j+\alpha < n, \alpha \ne 0} \Big( \lnot C_{i, j, k} \lor \lnot C_{i + \alpha, j+\alpha, k}\Big)}_{\textnormal{constant-difference diagonal}}} \land {\underbrace{\bigwedge_{0 \le i+\alpha, j-\alpha < n, \alpha \ne 0} \Big( \lnot C_{i, j, k} \lor \lnot C_{i-\alpha, j+\alpha, k} \Big)}_{\textnormal{constant-sum diagonal}}}\right)\\
	&\qquad \qquad \! \! {} \land {\underbrace{\bigwedge_{\substack{0 \le i < n \\ 0 \le j < n}} \Bigg(\bigvee_{0 \le k < n} C_{i, j, k}\Bigg)}_{\textnormal{color}}} \land {\underbrace{\bigwedge_{(i, j, k) \in \textnormal{Inputs}} C_{i, j, k} \vphantom{\bigwedge_{\substack{0 \le j < n \\ 0 \le k < n}}}}_{\textnormal{inputs}}},
	\end{align*}
	where the last line is obtained using the distributivity and implication properties.
	As required, the final form is a conjunction of disjunctions of literals, i.e., a conjunctive normal form.
	\item The \py{cgp_solver.py} program has been submitted and verified on INGInious, obtaining a score of 10/10.
\end{enumerate}

\end{document}