\section*{Problem 1 Solution}

\begin{enumerate}[a)]

\item

A macroscopic cross section for a homogeneous mixture of materials can be found by summing the cross sections of those components, weighted by their volume fractions:
\begin{equation}
\label{gentotmacroXS}
\Sigma = \sum_{i} \frac{V_i}{V_t}\Sigma_i
\end{equation}
where $\Sigma_i$ is the macroscopic cross section of the $i^{\text{th}}$ material, $V_i$ is the volume of the $i^{\text{th}}$ material, and $V_t$ is the total volume. For this problem, we are told that the fraction $\frac{V_{\text{UO2}}}{V_t} = 0.15$ and $\frac{V_{\text{H2O}}}{V_t} = 0.85$. Using these in equation (\ref{gentotmacroXS}) the total macroscopic cross section of our reactor can be expressed just in terms of the macroscopic cross sections of UO$_2$ and H$_2$O.
\begin{equation}
\label{totmacroXS}
\Sigma = 0.15\Sigma_{\text{UO2}} + 0.85\Sigma_{\text{H2O}}
\end{equation}
Calculating these two macroscopic cross sections requires knowledge of each of the individual microscopic cross sections of the compounds, since the macroscopic cross section is defined as
\begin{equation}
\label{macroXSi}
\Sigma_i = n_i\sigma_i ,
\end{equation}
where $n_i$ and $\sigma_i$ are respectively the number density and total microscopic cross sections of the compounds. While neither of these quantities are given explicitly, we can calculate each from the information provided.
\-\\

We can find a compound's number density by dividing the weight density by the mass of one molecule of the compound.
$$ n_i = \frac{\rho_i}{m_i} $$
The densities for UO$_2$ and H$_2$O are provided, and we can find the mass of a molecule by dividing the molar mass of the material by Avogadro's number, $N_A = 6.022\times10^{23}\text{ molecules/mol}$.
$$ m_i = \frac{M_i}{N_A}, $$
and so
$$ n_i = \frac{\rho_i N_A}{M_i} .$$
For our two compounds, UO$_2$ and H$_2$O these number densities are
$$ n_{\text{UO}_2} = \frac{\rho_{\text{UO}_2} N_A}{M_{\text{UO}_2}} \qquad\text{and}\qquad n_{\text{H}_2\text{O}} = \frac{\rho_{\text{H}_2\text{O}} N_A}{M_{\text{H}_2\text{O}}} .$$
Before moving on to finding the microscopic cross sections, we can note that the molar mass of the compounds can be broken into the sum of the molar masses of their constituents according to their fraction in the compound.
$$ n_{\text{UO}_2} = \frac{\rho_{\text{UO}_2} N_A}{f_{\text{U}5}M_{\text{U235}} + f_{\text{U}8}M_{\text{U238}} + 2M_{\text{O}}} \qquad\text{and}\qquad n_{\text{H}_2\text{O}} = \frac{\rho_{\text{H}_2\text{O}} N_A}{2M_{\text{H}} + M_{\text{O}}} .$$
\-\\

Next, we look at microscopic cross sections. The microscopic cross section of a compound is given by the sum of the microscopic cross sections of it's elemental components. The complete microscopic cross section for water can be found using this fact.
$$ \sigma_{\text{H2O}} = 2\sigma_{\text{H}} + \sigma_{\text{O}} $$
The microscopic cross section for UO$_2$ can be found similarly, however the calculation is slightly more complicated due to its enrichment in $^{235}$U. First, we find the average microscopic cross section for uranium by weighting the cross sections of $^{235}$U and $^{238}$U by their abundance.
$$ \sigma_{\text{U}} = f_{\text{U}5} \sigma_{\text{U}5} + f_{\text{U}8} \sigma_{\text{U}8} $$
where $f_{i}$ is the atomic fraction of material $i$ in the compound (for uranium in this case, this fraction is just the enrichment in atom \%). Then, we can complete the process exactly as we did for H$_2$O,
\begin{align*}
\sigma_{\text{UO}_2}	&= \sigma_{\text{U}} + 2\sigma_{\text{O}} \\
						&= f_{\text{U}5} \sigma_{\text{U}5} + f_{\text{U}8} \sigma_{\text{U}8}  + 2\sigma_{\text{O}}.
\end{align*}
\-\\

Combining these number densities and macroscopic cross sections, we can expand and rewrite the macroscopic cross sections described by equation (\ref{macroXSi}) as
%$$ \Sigma_{\text{UO}_2} = \frac{\rho_{\text{UO}_2} \left( f_{\text{U}5} \sigma_{\text{U}5} + f_{\text{U}8} \sigma_{\text{U}8}  + 2\sigma_{\text{O}}\right) N_A}{f_{\text{U}5}M_{\text{U235}} + f_{\text{U}8}M_{\text{U238}} + 2M_{\text{O}}} \qquad\text{and}\qquad \Sigma_{\text{H}_2\text{O}} = \frac{\rho_{\text{H}_2\text{O}} \left(2\sigma_{\text{H}} + \sigma_{\text{O}}\right) N_A}{2M_{\text{H}} + M_\text{O}} $$
Now we can substitute the given values.
$$ \Sigma_{\text{UO}_2} = \frac{\left(10.4\text{ g/cm}^3\right) \left(0.05(607.5\text{ b}) + 0.95(11.8\text{ b})  + 2(3.5\text{ b})\right)\left(6.022\times10^{23}\text{ mol}^{-1}\right)}{0.05(235.044\text{ g/mol}) + 0.95(238.050\text{ g/mol}) + 2(15.995\text{ g/mol})} $$
$$ \Sigma_{\text{UO}_2} = 112.7\text{ m}^{-1} $$

$$ \Sigma_{\text{H}_2\text{O}} = \frac{\left(1.0\text{ g/cm}^3\right) \left(2(20.8\text{ b}) + 3.5\text{ b}\right) \left(6.022\times10^{23}\text{ mol}^{-1}\right)}{2(1.008\text{ g/mol}) + 15.995\text{ g/mol}} $$
$$ \Sigma_{\text{H}_2\text{O}} = 150.8\text{ m}^{-1} $$

and plug these into equation (\ref{totmacroXS}) for the total cross section of the core.
$$ \Sigma = 0.15(112.7\text{ m}^{-1}) + 0.85(150.8\text{ m}^{-1})
 $$
$$\boxed{ \Sigma = 145.085\text{ m}^{-1} }$$

\item

Neutron attentuation follows a decaying exponential according to the equation
$$ N = N(0) \, e^{-\Sigma x} $$
where $N$ is the number of neutrons (you may be more familiar with this equation in terms of intensity; we have just eliminated the area and rate components from both sides).

We let $N(0)$ be our initial number of incident particles, $10^{15}$, $\Sigma$ be the macroscopic cross section calculated in part (a), and $x$ be the distance traveled by the neutrons, 4 m.
$$ N = 10^{15} e^{(-145.085\text{ m}^{-1})(4\text{ m})} $$
This works out to be about $9.153\times10^{-238}$, or \underline{zero particles}.
$$ N = 0 $$
Note that this number only indicates the quantity of particles which make it through the core uncollided. Since many of these interactions are scattering collisions, and not absorptive events, it is quite possible that some neutrons will eventually make it through the reactor. A more complicated derivation would be required in that case.
\end{enumerate}


\chapter{Resampling methods: Bias, Variance, and their trade-off}
We have defined various smoothers and nonparametric estimation
techniques. In classical statistical theory we usually assume that the
underlying model generating the data is in the family of models we
are considering. For nonparametrics this assumptions is relaxed and
asymptotic and finite sample bias and variance
estimates are not always easy to find in closed form. In this Chapter
we discuss some resampling methods that are commonly used to get
approximations of bias, variance, confidence intervals, etc...

In particular we will look at the problem of choosing smoothing
parameters. Remember how most of the smoothers we have defined have
some parameter that controls the smoothness of the final smooth or
curve estimate. For kernel smoothers we defined the scale parameter, for
local regression we defined the span or bandwidth, and for smoothing
splines we had the penalty term. We will call all of these {\it
the  smoothing parameter} and denote it with $\lambda$. It should be
clear from the context which of the specific smoothing parameters we
are referring to.

\input{section-05-01.tex}
\input{section-05-02.tex}
\input{section-05-03.tex}


\subsection{The Single-Vector Lanczos Algorithm}

We are armed now with all the required tools to present the main
algorithm that is used for SVD, in the context of LSI (at least in the
serial form). As mentioned on the introductory section of this
chapter, the algorithm exists thanks to Berry
(\cite{berry92},\cite{berry95}); we should probably present our
respects to Berry in this moment, as he worked for more than a decade
around the particular problem of solving efficiently the SVD/LSI
problem (and in essence, today's applications still use his
contributions). \\

The \cref{alg:lasvd} we present below is not the final one, as there are more
practical considerations to cover at the end of this subsection; but it is
easier to present this simplified version, as it has all the main
ingredients. We can see in particular, how it combines Lanczos
Tridiagonalization Step (\cref{alg:lanczos-step}) (which implicitly
uses the Power Method \cref{alg:power-method}), with the Rayleigh-Ritz
Method (\cref{alg:ritz}). \\

We also take the opportunity to come back to our original context,
where $A$ is a large and sparse matrix coming from an LSI problem. The
pseudocode is based on \cite{berry92}, filling additional details
from the C code of the implemented routine (LAS2) \footnote{Originally
the routine was coded in Fortran77, but we found more comfortable to
check the C port instead; both made by Berry, by the way.}.

\begin{algorithm}
  \label{alg:lasvd}
  \caption{The Single-Vector Lanczos Algorithm}
%
  \setstretch{1}
  \SetKwInOut{Input}{Input}
  \SetKwInOut{Output}{Output}
  \DontPrintSemicolon
%
    \Input{A matrix $A^{m \times n}$ and a truncation factor
    $k$}
    \BlankLine
%
    \Output{The $k$ singular values and its associated right singular
      vectors of $A$ \footnote{The eigenvectors of matrix $\trans{A}A$ are called
        right singular vectors of $A$, while those of $A\trans{A}$ are the
        left singular vectors.} (which are the first $k$ eigenpairs of
      symmetric matrix $\trans{A}A$). Both are numeric approximations. }
    \BlankLine
    \BlankLine
%
    Use Lanczos Tridiagonalization step \cref{alg:lanczos-step} to
    generate a family of symmetric tridiagonal matrices, $\{ T_j \} (j
    = 1,2, \dots, c) \suchthat c > k$. Note that these matrices
    approximate the eigenvalues of symmetric matrix
    $\trans{A}A$\ (which happen to be the singular values of $A$). \;
    \BlankLine
    \BlankLine
%
    \strut Compute the eigenvalues and eigenvectors of $T_k$ using the
    (implicit) QL Method. \;
    \BlankLine
    \BlankLine
%
    For each computed eigenvalue $\lambda_i$ of $T_k$ (hence of gramian matrix
    $\trans{A}A$), calculate the associated unit eigenvector $\vec{z_i}$
    such that $T_k\vec{z_i} = \lambda_i\vec{z_i}$. \;
    \BlankLine
    \BlankLine
%
    For each calculated eigenvector $\vec{z_i}$ of $T_k$, compute the Ritz
    vectors $v_i = Q_c\vec{z_i}$ as an approximation to the
    $i$-th eigenvector of $\trans{A}A$ (hence, to the right singular
    vectors of $A$). Note that the matrix $Q_c$ is a side product of
    the first step. \;
    \BlankLine
    \BlankLine
%
    return $(\{\lambda_1,\lambda_2,\cdots,\lambda_k\},
            \{\vec{v_1},\vec{v_2},\cdots,\vec{v_k}\})$
\end{algorithm}
\hfill

Although complete in appearance, \cref{alg:lasvd} still has a serious
numerical issue: the potential loss of orthogonality in the vectors of
matrix $Q_c$. To solve that problem, we could reorthogonalize all
vectors at every execution of \cref{alg:lanczos-step}; but that would
be kind of brute force, and eliminate the advantages of the whole
proposal. A clever approach, selective reorthogonalization, was
selected by Berry in order to complete his master-piece: the LASVD/LAS2
routine \footnote{Berry actually proposed four different methods of
  calculating SVD, for the LSI problem; LAS2 (descendant of LASVD)
  routine is just one of them. But it seems the fastest, and it was the
  only one ported to the new skin of Berry's SVDPACKC,
  which is SVDLIBC \cite{svdlibc}. Interestingly though, Berry
  mentions in \cite{berry91} that LASVD is suitable only for low to
  medium precision in the singular values. A pending task then, is to
  confirm of modern incarnations still have such limitation.}. \\

The selective reorthogonalization approach, as explained by Golub in
\cite{golub13}, is inspired on the error analysis made by Paige
  \cite{paige71}. Peige shows that the most recently computed vector
  \vec{\tilde{q_{k+1}}}, tends to have a non trivial and unwanted
  component in the direction of the already converged Ritz vectors
  \footnote{Recall that the Ritz vectors approximate the eigenvectors
    of the gramian matrix of $A$, hence the singular vectors of
    $A$}. Therefore, we do not need to re-orthogonalize against all
  the previously calculated vectors, rather use only the already
  converged ones. \\

Such adjustment is done during the Lanczos step
(\cref{alg:lanczos-step}), using a criteria devised by Parlett et al
\cite{parlett79}, which allows one to: know when a Ritz vector is
converged. \\

Berry does not include a final pseudocode of his LASVD
routine (inspired on the LANSOS routine from Parlett, Simon et
al). The routine eventually got renamed as LAS2 and made its way into
the famous SVDPACK (Fortran77) and SVDPACKC libraries; and more
recently in the modern version called SVDLIBC. It is the latest, which
is currently used by several LSI applications.


%!TEX root = /home/renaud/Documents/EPL/tfe/latex/tfe.tex
\section{Mean-square limit} \label{app:mslim}
Suppose that we have a probability space $\Omega$, and a sequence of random variables $X_n$ defined on $\Omega$. We say that $X_n$ converges to $X$ in the mean-square sense if
\begin{equation}
	\lim_{n\rightarrow \infty} \esp{(X_n-X)^2} = 0,
\end{equation}
and we note
\begin{equation}
	\mslim_{n\rightarrow \infty} X_n = X.
\end{equation}


\section{Lines \& Planes as VVFs}
\noindent
VVFs are parametric equations that take one input value and return one or more output values as a vector. We can draw curves in space by defining the tail of the output of the VVF be at the origin and have the tip trace out the curve. We will look at some simple VVFs that you need to be able to recognize.\\
\input{./vectorValuedFunctions/lines}
\input{./vectorValuedFunctions/planes}


\section*{Latent: Missing Data}
\subsection*{Mixture modeling}
Model each cluster as probability distr. $P(x|\theta_j)$\\
%Assuming data iid, likelihood is
data iid, likelih.: $P(D|\theta) = \prod_{i=1}^n \sum_{j=1}^k w_j P(x_i|\theta_j)$\\
%Choose parameters to minimize negative log l.\\
$\underset{min}{argmin}L(D;\theta) = \underset{min}{argmin} - \sum_i log \sum_j w_j P(x_i| \theta_j)$

\subsection*{Gaussian-Mixture Bayes classifiers}
Estimate class prior $P(y)$; Est. cond. distr. for each class:
$P(x|y) = \sum_{j=1}^{k_y} w_j^{(y)} \mathcal{N}(x; \mu_j^{(y)}, \Sigma_j^{(y)})$\\
$P(y|x) = \frac{1}{P(x)} p(y) \sum_{j=1}^{k_y} w_j^{(y)} \mathcal{N}(x; \mu_j^{(y)}, \Sigma_j^{(y)})$

\subsection*{Hard-EM}
Initialize parameters $\theta^{(0)}$; For $t=1,2,..$ (predict most likely class for each data point):\\
$z_i^{(t)} = \underset{z}{\operatorname{argmax}} P(z|x_i, \theta^{(t-1)})\\
= \underset{z}{\operatorname{argmax}} P(z|\theta^{(t-1)}) P(x_i|z,\theta^{(t-1)})$;\\
Compute the MLE (see Gauss Bayes Classifier):\\
$\theta^{(t)} = \underset{\theta}{\operatorname{argmax}} P(D^{(t)}|\theta)$

\subsection*{Soft-EM: "While not converged repeat"}
E-step: For each i and j calculate $\gamma_j^{(t)}(x_i)$\\
M-step: Fit clusters to weighted data points:\\

$w_j^{(t)} \leftarrow \frac{1}{n} \sum_{i=1}^n \gamma_j^{(t)} (x_i)$;  %\text{|semi-supervised}\\
$\mu_j^{(t)} \leftarrow \frac{\sum_{i=1}^n \gamma_j^{(t)} (x_i) x_i}{\sum_{i=1}^n \gamma_j^{(t)} (x_i)}\\ %\text{|learning with GMMs } ^t = ^*\\
\Sigma_j^{(t)} \leftarrow \frac{\sum_{i=1}^n \gamma_j^{(t)}(x_i) (x_i - \mu_j^{(t)}) (x_i - \mu_j^{(t)})^T}{\sum_{i=1}^n \gamma_j^{(t)}(x_i)}$\\ %\text{|}\gamma^{(t) = \gamma}$
EM for semi-supervised learning with GMMs:\\
%unlabeled points
unl. p.: $\gamma_j^{(t)}(x_i) = P(Z=j|x_i, \mu^{(t-1)}, \Sigma^{(t-1)}, w^{(t-1)})$\\
labeled points $y_i$: $\gamma_j^{(t)}(x_i) = [j = y_i]$

%If enough space put this on summary.
%\subsection*{Log-likelihood}
%$l(\theta) = log P(\mathcal{D})$\\
%$=\sum_{\overset{i=1}{y_i=\times}}^n log P(x_i;\theta) + \sum_{\overset{i=1}{y_i\not=\times}}^n log P(x_i,y_i;\theta)$\\
%$=\sum_{\overset{i=1}{y_i=\times}}^n log \sum_{j=1}^m P(x_i, Y=j;\theta) +$\\
%$ \sum_{\overset{i=1}{y_i\not=\times}}^n log P(x_i,y_i;\theta)$\\
%$=\sum_{\overset{i=1}{y_i=\times}}^n log \sum_{j=1}^m P(x_i|Y=j;\theta)P(Y=j;\theta) +$\\
%$ \sum_{\overset{i=1}{y_i\not=\times}}^n log P(x_i,y_i;\theta)$

%Only until here.
\iffalse
\subsection*{Log-likelihood}
$l(\theta) = log P(\mathcal{D})$ \\
$=\sum_{\overset{i=1}{y_i=\times}}^n log P(x_i;\theta) + \sum_{\overset{i=1}{y_i\not=\times}}^n log P(x_i,y_i;\theta)$\\
$=\sum_{\overset{i=1}{y_i=\times}}^n log \sum_{i=1}^m P(x_i, Y=j;\theta) +$\\
$ \sum_{\overset{i=1}{y_i\not=\times}}^n log P(x_i,y_i;\theta)$\\
$=\sum_{\overset{i=1}{y_i=\times}}^n log \sum_{i=1}^m P(x_i|Y=j;\theta)P(Y=j|\theta) +$\\
$ \sum_{\overset{i=1}{y_i\not=\times}}^n log P(x_i,y_i;\theta)$

\subsection*{Latent variable}
We denote the latent variable indicating the component the point is sampled from by Z, which takes on values in $\{1,...,k\}$.

\subsection*{E-step: Posterior probabilities}
$\gamma_j^t(x_i) = P(Z=j|x_i, \theta_t) = \frac{P(x_i|Z=j, \theta_t) P(Z=j|\theta_t)}{P(x_i;\theta_t)}$

\subsection*{M-step: maximizing expected log likelihood}
$\mathbb{E}_{\gamma^t}[\log P(\mathcal{D;\theta})] =
\mathbb{E}_{\gamma^t}[\log \Pi_{i=1}^nP(x_i,z_i;\theta)] = $ \\
$\sum_{i=1}^n \mathbb{E}_{\gamma^t}[\log P(x_i,z_i;\theta)] = $ \\
$\sum_{i=1}^n \sum_{j=1}^k \gamma_j^t(x_i) \log (P(x_i|z_i=j;\theta) P(z_i=j;\theta))$ \\
$\theta_{t+1} = \underset{\theta}{\operatorname{argmax}} \mathbb{E}_{\gamma^t}[\log P(\mathcal{D;\theta})]$
\fi


\newpage

 \section{Linear neural networks}

\subsection{Linear NN experiments}
Our main goal of this experiment is to test if deeper linear neural network can converge faster. we use linear MNIST with totally 60000 data points as our dataset. The first experiment randomly split this dataset into training set with 48000 training data points and 12000 test data points. The second experiment we use the full data as our training set.\\
Generally, we use a linear neural network model.  A linear NN with $H$ hidden layers can be modeled as
\begin{equation}
	\bm f(x,\bm\theta) = W_{H+1}(W_H(\cdots(W_1 x + b_1)\cdots)+b_H)+b_{H+1}
\end{equation}
where $W_1\in \mathbb{R}^{d_1\times d_x}$, $b_1 \in \mathbb{R}^{d_1}$, $W_i\in \mathbb{R}^{d_{i}\times d_{i-1}}$, $b_i \in \mathbb{R}^{d_i}$, $i = 2,\cdots,H$, $W_{H+1}\in \mathbb{R}^{d_y\times d_{H+1}}$, $b_{H+1} \in \mathbb{R}^{d_y}$. Notice that $d_x$ represents the input size which is 500 in this experiment, while $d_y$ represents the output size , namely, the number of classes, which is 10 in this experiment. So from now on, we may only  use an array $(d_x,d_1,\cdots,d_H,d_y)$ to represent our model.\\
We use cross-entropy with $L_2$ regularization as our loss function.
\begin{equation}
	L(\bm\theta,D) = \frac{1}{|D|} \sum_{i = 1}^k \sum_{x\in D_i} (\log ({\bm 1}^T e^{\bm f(x,\bm\theta)}) - f_i(x,\bm\theta)) + \lambda \sum_{j = 0}^{p+1} (\|W_j\|_F^2 + \|b_j\|_2^2)
\end{equation}

\begin{itemize}
	\item Model.\\
    \begin{itemize}
    	\item Net0 (500,10).
    	\item Net1 (500,100,10)
    	\item Net2 (500,200,100,10)
    	\item Net3 (500,400,200,100,10)
    	\item Net4 (500,800,200,100,10)
    \end{itemize}
    \item Training algorithm.
    \begin{itemize}
    	\item Full gradient descent(GD).
    	    \begin{equation}
    	    	\bm\theta = \bm\theta - lr \frac{\partial L}{\partial \bm\theta}
    	    \end{equation}
    	\item Coordinate descent(CD).
    	\begin{align}
    		&\bm\theta_1^+ = \bm\theta_1 - lr \frac{\partial L}{\partial \bm\theta_1}(\bm\theta_1,\bm\theta_2,\cdots,\bm,\bm\theta_p,\theta_{p+1}),\\
    		&\bm\theta_2^+ = \bm\theta_2 - lr \frac{\partial L}{\partial \bm\theta_2}(\bm\theta_1^+,\bm\theta_2,\cdots,\bm\theta_p,\bm\theta_{p+1}),\\
    		&\bm\theta_3^+ = \bm\theta_3 - lr \frac{\partial L}{\partial \bm\theta_3}(\bm\theta_1^+, \bm\theta_2^+,\cdots,\bm\theta_p,\bm\theta_{p+1}),\\
    		&\cdots,\\
    		&\bm\theta_{p+1}^+ = \bm\theta_{p+1} - lr \frac{\partial L}{\partial \bm\theta_{p+1}}(\bm\theta_1^+, \bm\theta_2^+,\cdots,\bm\theta_p^+,\bm\theta_{p+1})
    	\end{align}
    	where $\bm\theta_j = (W_j,b_j), j = 1,2,\cdots,p+1$.
    \end{itemize}
    \item Stopping criterion.\\
    We stop iteration when training accuracy first obtain 100\%.
\end{itemize}

\begin{enumerate}
	\item Training Net0-4 using gradient descent.\\
	We set $\lambda = 10^{-5}$, learning rate $= 0.1$.
	\begin{itemize}
		\item Experiment 1: Using ranomly chosen 48000 data points from linear MNIST as training set.\\
		\begin{table}[H]
			\centering
			\caption{Randomly chosen 80\% full dataset as training set}
			\label{tab:1}
			\begin{tabular}{lllllll}
				\hline\noalign{\smallskip}
				Model &  Optim & Iterations & Time  & TrainL  & TestL & TestA \\
				\noalign{\smallskip}\hline\noalign{\smallskip}

				Net0 & GD & 3190 & 3743 & 0.0062 & 0.0077 & 99.94   \\

				Net1 & GD & 2009 & 2475 & 0.0046 & 0.0063 & 99.95 \\

				Net2 & GD & 2322 & 3245 & 0.0034 & 0.0049 & 99.94 \\

				Net3 & GD & 1767 & 2705 & 0.0048 & 0.0066 & 99.93 \\

				Net4 & GD & 1805 & 3346 &0.0059 & 0.0075 & 99.93 \\

				\noalign{\smallskip}\hline

			\end{tabular}
		\end{table}

%		\begin{figure}[H]
%			\centering
%			\includegraphics[width=3in]{logGDtrainL.png}
%			\caption{log(Train Loss)}
%		\end{figure}

		\item Experiment 2: Using whole linear MNIST dataset as training set.
		\begin{table}[H]
			\centering
			\caption{Full dataset}
			\label{tab:2}
			\begin{tabular}{lllll}
				\hline\noalign{\smallskip}
				Model &  Optim & Iterations & Time  & TrainL \\
				\noalign{\smallskip}\hline\noalign{\smallskip}

				Net0 & GD & 4218 & 6168 & 0.0051  \\

				Net1 & GD & 2546 & 3937 & 0.0038  \\

				Net2 & GD & 2269 & 4010 & 0.0035  \\

				Net3 & GD & 2099 & 4046 & 0.0044  \\

				Net4 & GD & 1578 & 3815 &0.0065  \\

				\noalign{\smallskip}\hline

			\end{tabular}
		\end{table}

%		\begin{figure}[H]
%			\centering
%			\includegraphics[width=3in]{logGDfullL.png}
%			\caption{log(Train Loss) for full dataset}
%		\end{figure}
		We can see very clearly in the two figures that the loss of Net0 always decrease slower than other deeper linear NN. Or we can see in the table that Net0 use most iterations to reach 100\% training accuracy.

	    \item Experiment 3: Training '0' and '1' data in MNIST.\\
	     First I use Adam algorithm to verify that '0' and '1' are linearly separable in the training set of MNIST. Then I use full GD to iterate 50 epoches for each model and record their loss history.
%	    \begin{figure}[H]
%		  \centering
%		  \includegraphics[width=3in]{log_MNIST_50.png}
%		  \caption{log(Train Loss) for MNIST dataset}
%	      \end{figure}
		\end{itemize}
	\end{enumerate}


%
% 125
%
\chapter{The Expansion of Functions in Infinite Series}

\Section{7}{1}{A formula due to Darboux*TODO.  Journal de Math. (3), ii. (1876), p. 271.}

Let $f(z)$ be analytic at all points of the straight line joining $a$
to $z$, and let $\phi(t)$ be any polynomial of degree $n$ in $t$.

Then if $0 \leq t \leq 1$, we have by differentiation
\begin{align*}
  \frac{\dd }{\dd t}
  \sum_{m=1}^{n} (-)^{m} (z-a)^{m} \phi^{(n-m)}(t) f^{(m)}(a + t(z-a))
  \\
  =
  -(z-a) \phi^{(n)}(t) f'(a + t(z-a))
  +
  (-)^{n} (z-a)^{n+1} \phi(t) f^{(n+1)}(a + t (z-a)).
\end{align*}

Noting that $\phi^{(n)}(t)$ is constant $= \phi^{(n)}(0)$, and integrating
between the limits $0$ and $1$ of $t$, we get
\begin{align*}
  \phi^{(n)}(0) \thebrace{ f(z) - f(a) }
  \\
  =&
  \sum_{m=1}^{n} (-)^{m-1} (z-a)^{m}
  \thebrace{
    \phi^{(n-m)}(1) f^{(m)}(z) - \phi^{(n-m)}(0) f^{(m)}(a)
  }
  \\
  &
  + (-)^{n} (z-a)^{n+1}
  \int_{0}^{1} \phi(t) f^{(n+1)}(a + t(z-a)) \dmeasure t,
\end{align*}
which is the formula in question.

Taylor's series may be obtained as a special case of this by writing
$\phi(t) = (t-1)^{n}$ and making $n\rightarrow\infty$.

%\begin{smalltext}
Example. By substituting $2n$ for $n$ in the formula of Darboux, and
taking $\phi(t) = t^{n} (t-1)^{n}$, obtain the expansion (supposed convergent)
$$
f(z) - f(a)
=
\sum_{n=1}^{\infty}
\frac{ (-)^{n-1} (z-a)^{n} }{2^{n} n!}
\thebrace{ f^{(n)}(z) + (-)^{n-1} f^{(n)}(a) },
$$
and find the expression for the remainder after $n$ terms in this
series.
%\end{smalltext}
\Section{7}{2}{The \Bernoulli an numbers and the \Bernoulli an polynomials.}

The function $\half z \cot \half z$ is analytic when $\absval{z} <
2\pi$,
and, since it is an even function of $z$, it can be expanded into a
Maclaurin series, thus
$$
\half z \cot \half z
=
1
- B_{1} \frac{z^{2}}{2!}
- B_{2} \frac{z^{4}}{4!}
- B_{3} \frac{z^{6}}{6!}
\cdots
;
$$
then $B_{n}$ is called the $n$th \emph{\Bernoulli an
  number}\footnote{TODO These numbers were introduced by Jakob BernouUi in bis Ars
Conjectandi, p. 97 (published posthumously, 1713).}.
It is found that\footnote{TODO Tables of the first sixty-two \Bernoulli an numbers have been given by
Adams, Brit. A.is. ReiJorts, 1877.}
$$
B_{1} = \frac{1}{6},
\quad
B_{2} = \frac{1}{30},
\quad
B_{3} = \frac{1}{42},
\quad
B_{4} = \frac{1}{30},
\quad
B_{5} = \frac{5}{66},
\quad
\ldots.
$$

%
% 126
%
These numbers can be expressed as definite integrals as follows:

We have, by example TODO:2 (p. TODO:122) of Chapter TODO:VI,
\begin{align*}
  \int_{0}^{\infty}
  \frac{\sin px \dmeasure x}{e^{\pi x} - 1}
  =&
  -\frac{1}{2p} + \frac{i}{2} \cot ip
  \\
  =&
  -\frac{1}{2p}
  +
  \frac{1}{2p} \thebrace{
    1
    + B_{1} \frac{(2p)^{2}}{2!}
    - B_{2} \frac{(2p)^{4}}{4!}
    + \cdots
    }.
\end{align*}

Since
$$
\int_{0}^{\infty}
\frac{x^{n} \sin \theparen{px + \half n \pi}}{e^{\pi x} - 1}
\dmeasure x
$$
converges uniformly (by de la Vall\'ee Poussin's test) near
$p=0$ we
may, by \hardsubsectionref{4}{4}{4} corollary, differentiate both
sides of this equation any number of times and then put $p = 0$; doing
so and writing $2t$ for $x$, we obtain
$$
B_{n}
=
4n
\int_{0}^{\infty}
\frac{t^{2n-1} \dmeasure t}{e^{2\pi t} - 1}.
$$
%\begin{remark}
A proof of this result, depending on contour integration, is given by
Carda, Monatshefte fur Math, v.nd Phys. v. (1894), pp. 321-4.
%\end{remark}

TODO:fixexample
Example. Shew that
$$
B_{n}
=
\frac{2n}{\pi^{2n} (2^{2n}-1)}
\int_{0}^{\infty}
\frac{x^{2n-1} \dmeasure x}{\sinh x}
> 0.
$$

Now consider the function $t \frac{e^{zt}-1}{e^{t}-1}$, which may be
expanded into a
Maclaurin series in powers of $t$ valid when $\absval{t} < 2\pi$.

\emph{The \Bernoulli an polynomial\footnote{TODO The name was given by
    Raabe, Journal filr Math. xlii. (1851), p.348.} of order $n$} is
defined to be the coefficient of $\frac{t^{n}}{n!}$
in this expansion. It is denoted by $\phi_{n}(z)$, so that
$$
t \frac{e^{zt}-1}{e^{t}-1}
=
\sum_{n=1}^{\infty}
\frac{\phi
_{n}(z) t^{n}}{n!}.
$$

This polynomial possesses several important properties. Writing $z+1$
for $z$ in the preceding equation and subtracting, we find that
$$
t e^{zt}
=
\sum_{n=1}^{\infty}
\thebrace{
  \phi_{n}(z+1) - \phi_{n}(z)
}
\frac{t^{n}}{n!}.
$$

On equating coefficients of $t^{n}$ on both
sides of this equation we obtain
$$
n z^{n-1}
=
\phi_{n}(z+1) - \phi_{n}(z),
$$
which is a difference-equation satisfied by the function $\phi_{n}(z)$.

%
% 127
%

An explicit expression for the \Bernoulli an polynomials can be obtained
as follows. We have
$$
e^{zt} - 1
=
zt
+ \frac{z^{2}t^{2}}{2!}
+ \frac{z^{3}t^{3}}{3!}
+ \cdots,
$$
and
$$
\frac{t}{e^{t}-1}
=
\frac{t}{2i} \cot \frac{t}{2i} - \frac{t}{2}
=
1
- \frac{t}{2}
+ \frac{B_{1} t^{2}}{2!}
- \frac{B_{2} t^{4}}{4!}
+ \cdots.
$$

Hence
$$
\sum_{n=1}^{\infty}
\frac{\phi_{n}(z) t^{n}}{n!}
=
\thebrace{
  zt
  + \frac{z^{2} t^{2}}{2!}
  + \frac{z^{3} t^{3}}{3!}
  + \cdots
}
\thebrace{
  1
  - \frac{t}{2}
  + \frac{B_{1} t^{2}}{2!}
  - \frac{B_{2} t^{4}}{4!}
  + \cdots
}.
$$

From this, by equating coefficients of $t^{n}$
(\hardsubsectionref{3}{7}{3}), we have
$$
\phi_{n}(z)
=
z^{n}
- \half n z^{n-1}
+ \binomialcoeff{n}{2} B_{1} z^{n-2}
- \binomialcoeff{n}{4} B_{2} z^{n-4}
+ \binomialcoeff{n}{6} B_{3} z^{n-6}
- \cdots,
$$
the last term being that in $z$ or $z^{2}$ and
$\binomialcoeff{n}{2}, \binomialcoeff{n}{4},\ldots$  being the
binomial coefficients; this is the Maclaurin series for the $n$th
\Bernoulli an polynomial.
%\begin{Remark}
When $z$ is an integer, it may be seen from the difference-equation that
$$
\phi_{n}(z)/n
=
1^{n-1}
+ 2^{n-1}
+ \cdots + (z-1)^{n-1}.
$$

The Maclaurin series for the
expression on the right was given by \Bernoulli.

%\begin{Remark}
\begin{wandwexample}
  Shew that, when $n > 1$,
  $$
  \phi_{n}(z) = (-)^{n} \phi_{n}(1-z).
  $$
\end{wandwexample}
%\end{Remark}
\Subsection{7}{2}{1}{The Euler-Maclaurin expansion.}

In the formula of Darboux \hardsectionref{7}{1}) write $\phi_{n}(t)$
for $\phi(t)$, where $\phi_{n}(t)$ is the $n$th \Bernoulli an polynomial.

Differentiating the equation
$$
\phi_{n}(t+1) - \phi_{n}(t) = n t^{n-1}
$$
$n - k$ times, we have
$$
\phi_{n}^{(n-k)}(t+1) - \phi_{n}^{(n-k)}(t)
=
n (n-t) \cdots k t^{k-1}
$$
Putting $t=0$ in this, we have
$\phi_{n}^{(n-k)}(1) = \phi_{n}^{(n-k)}(0).$

Now, from the Maclaurin
series for $\phi_{n}(z)$, we have if $k > 0$
\begin{align*}
  \phi_{n}^{(n-2k-1)}(0) = 0,
  &
  \quad
  \phi_{n}^{(n-2k)}(0) = \frac{n!}{(2k)!} (-)^{k-1} B_{k},
  \\
  \phi_{n}^{(n-1)}(0) = -\half n!,
  &
  \quad
  \phi_{n}^{(n)}(0) = n!.
\end{align*}

Substituting these values of $\phi_{n}^{(n-k)}(1)$ and
$\phi_{n}^{(n-k)}(0)$ in Darboux's
result, we obtain the Euler-Maclaurin sum
formula\footnote{TODO A history of the formula is given by Barnes, Proc. London Math. Soc.
(2), iii. (1905), p. 253. It was discovered by Euler (1732), but was
not published at the time. Euler communicated it (June 9, 1736) to
Stirling who replied (April 16, 1738) that it included his own theorem
(see \hardsubsectionref{12}{3}{3}) as a particular case, and also that the more general
theorem had been discovered by Maclaurin; and Euler, in a lengthy
reply, waived his claims to priority. The theorem was published by
Euler, Comm. Acad. Imp. Petrop. vi. (1732), [Published 1738], pp.
68-97, and by Maclaurin in 1742, Treatise on Fluxions, p. 672. For
information concerning the correspondence between Euler and Stirling,
we are indebted to Mr C. Tweedie.},

%
% 128
%
\begin{align*}
(z-a) f'(a)
=&
f(z) - f(a)
- \frac{z-a}{2} \thebrace{ f'(z) - f'(a) }
\\
&
+ \sum_{m=1}^{n-1} \frac{ (-)^{m-1} B_{m} (z-a)^{2m}}{(2m)!}
\thebrace{f^{(2m)}(z) - f^{(2m)}(a)}
\\
&
-\frac{(z-a)^{2n+1}}{(2n)!}
\int_{0}^{1} \phi_{2n}(t) f^{(2n+1)}\thebrace{ a + (z-a) t }
\dmeasure t.
\end{align*}

In certain cases the last term tends to zero as
$n \rightarrow \infty$, and we can
thus obtain an infinite series for $f(z) - f(a)$.

If we write $\omega$ for $z - a$ and $F(x)$ for $f'(x)$, the last formula becomes
\begin{align*}
  \int_{a}^{a+\omega} F(x) \dmeasure x
  =
  &
  \half \omega \thebrace{ F(a) + F(a + \omega) }
  \\
  &
  + \sum_{m=1}^{n-1}
  \frac{(-)^{m} B_{m} \omega^{2m}}{(2m)!}
  \thebrace{F^{(2m-1)}(a+\omega) - F^{(2m-1)}(a)}
  \\
  &
  + \frac{\omega^{2n+1}}{(2n)!}
  \int_{0}^{1} \phi_{2n}(t) F^{(2n)}(a + \omega t) \dmeasure t.
\end{align*}

Writing $a + \omega, a + 2\omega, \ldots, a + (r-1) \omega$
for $a$ in this result and adding up, we get
\begin{align*}
\int_{a}^{a + r\omega} F(x) \dmeasure x
=
\omega
 &
\thebrace{
  \half F(a) + F(a+\omega) + F(a+2\omega)
  + \cdots + \half F(a + r\omega)
}
\\
&
+ \sum_{m=1}^{n-1}
\frac{(-)^{m} B_{m} \omega^{2m}}{(2m)!}
\thebrace{
  F^{(2m-1)}(a + r\omega)
  -
  F^{(2m-1)}(a)
}
+ R_{n},
\end{align*}
where
$$
R_{n}
=
\frac{\omega^{2n+1}}{(2n)!}
\int_{0}^{1} \phi_{2n}(t)
\thebrace{ \sum_{m=0}^{r-1} F^{(2n)}(a + m\omega + \omega t)}
\dmeasure t.
$$

This last formula is of the utmost importance in connexion with the
numerical evaluation of definite integrals. It is valid if $F(x)$ is
analytic at all points of the straight line joining
$a$ to $a + r \omega$.

%\begin{Remark}
\begin{wandwexample}
  If $f(z)$ be an odd function of $z$, shew that
  $$
  z f'(z)
  =
  f(z)
  +
  \sum_{m=2}^{n}
  (-)^{m}
  \frac{B_{m-1} (2z)^{2m-2}}{(2m-2)!}
  f^{(2m-2)}(z)
  -
  \frac{2^{2n} z^{2n+1}}{(2n)!}
  \int_{0}^{1}
  \phi_{2n}(t)
  f^{(2n+1)}(-z + 2zt)
  \dmeasure t.
  $$
\end{wandwexample}
\begin{wandwexample}
  Shew, by integrating by parts, that the remainder after $n$
  terms of the expansion of $\half z \cot \half z$ may be written in the form
  $$
  \frac{ (-)^{n+1} z^{2n+1} }{ (2n)! \sin z }
  \int_{0}^{1} \phi_{2n}(t) \cos (zt) \dmeasure t.
  $$
\addexamplecitation{Math. Trip. 1904.}
\end{wandwexample}
%\end{Remark}
\Section{7}{3}{\Burmann's theorem*TODO * Memoires de VInstitut, ii. (1799), p. 13. See also Dixon, Proc.
London Math. Soc. xxxiv. (1902), pp. 151-153..}

We shall next consider several theorems which have for their object
\emph{the expansion of one function in powers of another function.}

%
% 129
%

Let $\phi(z)$ be a function of $z$ which is analytic in a closed
region $S$ of which $a$ is an interior point; and let
$$
\phi(a) = b.
$$

Suppose also that $\phi'(a) \neq 0$. Then Taylor's theorem furnishes the
expansion
$$
\phi(z) - b
=
\phi'(a) (z-a)
+ \half \phi''(a) (z-a)^{2}
+ \cdots,
$$
and if it is legitimate to revert this series we obtain
$$
z - a
=
\frac{1}{\phi'(a)}
\thebrace{ \phi(z) - b }
-
\half \frac{\phi''(a)}{\thebrace{\phi'(a)}^{3}}
\thebrace{ \phi(z) - b }^{2}
+ \cdots,
$$
which expresses $z$ as an analytic function of the variable
$\thebrace{ \phi(z) - b }$,
for sufficiently small values of $\absval{z-a}$. If then $f(z)$ be
analytic near $z = a$, it follows that $f(z)$ is an analytic function
of $\thebrace{ \phi(z) - b }$
when $\absval{z - a}$ is sufficiently small, and so there will be an
expansion of the form
$$
f(z)
=
f(a)
+ a_{1} \thebrace{ \phi(z) - b }
+ \frac{a_{2}}{2!} \thebrace{ \phi(z) - b }^{2}
+ \frac{a_{3}}{3!} \thebrace{ \phi(z) - b }^{3}
+ \cdots.
$$

The actual coefficients in the expansion are given by the following
theorem, which is generally known as \emph{\Burmann's theorem}.

\emph{Let $\psi(z)$ be a function of $z$ defined by the equation
$$
\psi(z) = \frac{z-a}{ \phi(z) - b };
$$
then an analytic function $f(z)$ can, in a certain domain of values of
$z$, be expanded in the form
$$
f(z)
=
f(a)
+
\sum_{m=1}^{n-1}
\frac{ \thebrace{\phi(z)-b}^{m} }{m!}
\frac{\dd^{m-1}}{\dd a^{m-1}}
\thebracket{
  f'(a) \thebrace{\psi(a)}^{m}
}
+
R_{n},
$$
where
$$
R_{n}
=
\frac{1}{2 \pi i}
\int_{a}^{z}
\int_{\gamma}
\thebracket{
  \frac{\phi(z) - b}{\phi(t) - b}
}^{n-1}
\frac{ f'(t) \phi'(z) }{\phi(t) - \phi(z)}
\dmeasure t \dmeasure z,
$$
and $\gamma$ is a contour in the $t$-plane, enclosing the points $a$
and $z$ and such that, if $\zeta$ be any point inside it, the equation
$\phi(t) = \phi(\zeta)$ has no roots on or inside the contour
except\footnote{It is assumed that such a contour can be chosen if
  $\absval{z - a}$ be sufficiently small;
  see \hardsubsectionref{7}{3}{1}.} a simple root $t=\zeta$.
}

To prove this, we have
\begin{align*}
  f(z) - f(a)
  = &
  \int_{a}^{z} f'(\zeta) \dmeasure \zeta
  = \frac{1}{2 \pi i} \int_{a}^{z} \int_{\gamma} TODO
\end{align*}

%
% 130
%

But, by \hardsectionref{4}{3},
\begin{align*}
  &
  \frac{1}{1 \pi i}
  \int_{a}^{z} \int_{\gamma}
  \thebracket{ \frac{\phi(\zeta) - b}{\phi(t) - b} }^{m}
  \frac{f'(t)\phi'(\zeta)}{\phi(t) - b}
  \dmeasure t \dmeasure \zeta
  \\
  &=
  \frac{ [\phi(z) - b]^{m+1} }{2 \pi i (m+1)}
  \int_{\gamma} \frac{f'(t)}{[\phi(t) - b]^{m+1}} \dmeasure t
  \\
  &=
  \frac{ [\phi(z) - b]^{m+1} }{2 \pi i (m+1)}
  \int_{\gamma}
  \frac{ f'(t) \thebrace{\psi(t)}^{m+1} }{ (t-a)^{m+1} }
  \dmeasure t
  \\
  &=
  \frac{ [\phi(z) - b ]^{m+1} }{ (m+1)! }
  \frac{ \dd^{m} }{ \dd a^{m} }
  \thebracket{ f'(a) \thebrace{\psi(a)}^{m+1} }.
\end{align*}
Therefore, writing
$m - 1$ for $m$,
\begin{align*}
  f(z) = f(a) +
  &
  \sum_{m=1}^{n-1}
  \frac{ [\phi(z) - b]^{m} }{m!}
  \frac{\dd^{m-1}}{\dd a^{m-1}}
  [ f'(a) \thebrace{ \psi(a) }^{m} ]
  \\
  &
 + \frac{1}{2 \pi i}
  \int_{a}^{z} \int_{\gamma}
  \thebracket{
    \frac{\phi(\zeta) - b}{\phi(t) - b}
  }^{n-1}
  \frac{f'(t) \phi'(\zeta)}{\phi(t) - \phi(\zeta)}
  \dmeasure t \dmeasure \zeta.
\end{align*}

If the last integral tends to zero as $n \rightarrow \infty$, we may write the
right-hand side of this equation as an infinite series.
\begin{wandwexample}
Prove that
$$
z
=
a
+ \sum_{n=1}^{\infty}
\frac{(-)^{n-1} C_{n} (z-a)^{n} e^{n (z^{2} - a^{2})}}{n!},
$$
where
$$
C_{n}
=
(2na)^{n-1}
- \frac{n(n-1)(n-2)}{1!} (2na)^{n-3}
+ \frac{n^{2}(n-1)(n-2)(n-3)(n-4)}{2!} (2na)^{n-5}
- \cdots.
$$
To obtain this expansion, write
$$
f(z) = z,
\quad
\phi(z) - b = (z-a) e^{z^{2} - a^{2}},
\quad
\psi(z) = e^{a^{2} - z^{2}}
$$
in the above expression
of \Burmann's theorem; we thus have
$$
z
=
a
+ \sum_{n=1}^{\infty}
\frac{1}{n!}
(z-a)^{n}
e^{n(z^{2} - a^{2})}
\thebrace{
  \frac{\dd^{n-1}}{\dd z^{n-1}}
  e^{n(a^{2} - z^{2})}
}_{z=a}.
$$

But, putting $z = a + t$,
\begin{align*}
  \thebrace{
    \frac{\dd^{n-1}}{\dd z^{n-1}}
    e^{n(a^{2} - z^{2})}
  }_{z=a}
  =&
  \thebrace{
    \frac{\dd^{n-1}}{\dd t^{n-1}}
    e^{-n(2at + t^{2})}
  }_{t=0}
  \\
  =&
  TOOD
\end{align*}
The highest value of $r$ which gives a term in the summation is
$r = n-1$.
Arranging therefore the summation in descending indices $r$,
beginning with $r = n-1$, we have
\begin{align*}
  \thebrace{
    \frac{\dd^{n-1}}{\dd z^{n-1}}
    e^{n(a^{2} - z^{2})}
  }_{z=a}
  =&
  TODO
  \\
  =& (-)^{n-1} C_{n},
\end{align*}
which gives the required result.
\end{wandwexample}
\begin{wandwexample}
  Obtain the expansion
  $$
  z^{2}
  =
  \sin^{2} z
  + \frac{2}{3} \half \sin^{4} z
  + \frac{2 \cdot 4}{3 \cdot 5} \frac{1}{3} \sin^{6} z
  + \cdots.
  $$
\end{wandwexample}
%
% 131
%
\begin{wandwexample}
Let a line $p$ be drawn through the origin in the $z$-plane,
perpendicular to the line which joins the origin to any point $a$. If
$z$ be any point on the $z$-plane which is on the same side of the line
$p$ as the point $a$ is, shew that
$$
\log z
=
\log a
+ 2 \sum_{m=1}^{\infty}
\frac{1}{2m+1} \theparen{\frac{z-a}{z+a}}^{2m+1}.
$$
\end{wandwexample}
\Subsection{7}{3}{1}{\Teixeira's extended form of \Burmann's theorem.}

In the last section we have not investigated closely the conditions of
convergence of \Burmann's series, for the reason that a much more
general form of the theorem will next be stated; this generalisation
bears the same relation to the theorem just given that Laurent's
theorem bears to Taylor's theorem: viz., in the last paragraph we
were concerned only with the expansion of a function in \emph{positive}
powers of another function, whereas we shall now discuss the expansion
of a function in \emph{positive and negative} powers of the second
function.

The general statement of the theorem is due to
\Teixeira\footnote{TODO Journal f\"ur Math, cxxii. (1900), pp.
  97-123.}, whose exposition we shall follow in this section.

Suppose (i) that $f(z)$ is a function of $z$ analytic in a ring-shaped
region $A$, bounded by an outer curve $C$ and an inner curve $c$;
(ii) that $\theta(z)$ is a function analytic on and inside $C$,
and has only one zero a
within this contour, the zero being a simple one;
(iii) that $x$ is a given point within $A$;
(iv) that for all points $z$ of $C$ we have
$$
\absval{\theta(x)} < \absval{\theta(z)},
$$
and for all points $z$ of $c$ we have
$$
\absval{\theta(x)} > \absval{\theta(z)}.
$$

The equation
$$
\theta(z) - \theta(x) = 0
$$
has, in this case, a single root $z = x$ in the interior of $C$, as is
seen from the equation\footnote{The expansion is justified
  by \hardsectionref{4}{7}, since
  $\sum_{n=1}^{\infty} \thebrace{\theta(x)/\theta(z)}^{n}$
  converges uniformly when $z$
  is on $C$.}
\begin{align*}
  \frac{1}{2 \pi i}
  \int_{C} \frac{\theta'(z)}{\theta(z) - \theta(x)} \dmeasure z
  =&
  \frac{1}{2 \pi i}
  \thebracket{
    \int_{C} \frac{\theta'(z)}{\theta(z)} \dmeasure z
    +
    \theta(x)
    \int_{C} \frac{\theta'(z)}{ \thebrace{\theta(z)}^{2}}
    \dmeasure z
    + \cdots
  }
  \\
  =&
  \frac{1}{2 \pi i}
  \int_{C} \frac{\theta'(z)}{\theta(z)} \dmeasure z
\end{align*}
of which the left-hand and right-hand members represent respectively
the number of roots of the equation considered
(\hardsubsectionref{6}{3}{1}) and the
number of the roots of the equation $\theta(z) = 0$ contained within
$C$.

Cauchy's theorem therefore gives
$$
f(x)
=
\frac{1}{2 \pi i}
\thebracket{
  \int_{C} \frac{f(z) \theta'(z)}{\theta(z) - \theta(x)} \dmeasure z
  -
  \int_{c} \frac{f(z) \theta'(z)}{\theta(z) - \theta(x)} \dmeasure z
}.
$$

%
% 132
%
The integrals in this formula can be expanded, as in Laurent's
theorem, in powers of $\theta(x)$, by the formulae
\begin{align*}
  TODO
\end{align*}

We thus have the formula
$$
TODO
$$
where
$$
TODO
$$

Integrating by parts, we get, if $n \neq 0$,
$$
TODO
$$

This gives a development of $f(x)$ in positive and negative powers of
$\theta(x)$, valid for all points $x$; within the ring-shaped space $A$.

If the zeros and poles of $f(z)$ and $\theta(z)$ inside $C$ are known,
$A_{n}$ and $B_{n}$ can be evaluated by \hardsubsectionref{5}{2}{2} or
by \hardsectionref{6}{1}.

\begin{wandwexample}
  Shew that, if $\absval{x} < 1$, then
  $$
  x
  =
  \half \theparen{ \frac{2x}{1+x^{2}} }
  +
  \frac{1}{2 \cdot 4} \theparen{ \frac{2x}{1+x^{2}} }^{3}
  +
  \frac{1\cdot 3}{2 \cdot 4 \cdot 6} \theparen{ \frac{2x}{1+x^{2}} }^{5}
  +
  \cdots.
  $$

  Shew that, when $\absval{x} > 1$, the second member represents $x^{-1}$.
\end{wandwexample}
\begin{wandwexample}
If $S^{(m)}_{2n}$ denote the sum of all combinations of the numbers
$$
2^{2}, 4^{2}, 6^{2}, \ldots, (2n-2)^{2},
$$
taken $m$ at a time, shew that
$$
\frac{1}{z}
=
\frac{1}{\sin z}
+
\sum_{n=0}^{\infty}
\frac{(-)^{n+1}}{ (2n+2)! }
\theparen{
  \frac{1}{2n+3}
  - \frac{S^{(1)}_{2(n+1)}}{2n+1}
  + \cdots
  + \frac{ (-)^{n} S^{(n)}_{2(n+1)}}{3}
}
(\sin z)^{2n+1}
$$
the expansion being valid for all values of $z$ represented by points
within the oval whose equation is $\absval{\sin z} = 1$ and which contains the
point $z = 0$. \addexamplecitation{\Teixeira.}
\end{wandwexample}

\Subsection{7}{3}{2}{Lagrange's theorem.}

Suppose now that the function $f(z)$ of \hardsubsectionref{7}{3}{1}
is analytic at all points in the interior of $C$, and let
$\theta(x) = (x - a) \theta_{1}(x)$. Then $\theta_{1}(x)$ is
analytic and not zero on or inside $C$ and the contour $c$ can be
dispensed with; therefore the formulae which give $A_{n}$ and
$B_{n}$ now become, by \hardsubsectionref{5}{2}{2} and \hardsectionref{6}{1},
\begin{align*} % TODO: multiline?
  A_{n}
  =&
  \frac{1}{2\pi in} \!
  \int_{C} \frac{f'(z)}{(z-a)^{n} \theta_{1}^{n}(z)} \dmeasure z
  = \frac{1}{n!} \frac{\dd^{n-1}}{\dd a^{n-1}}
  \thebrace{
    \frac{f'(a)}{\theta_{1}^{n}(a)
    }
  }
  \quad (n \geq 1),
  \\
  A_{0}
  =&
  \frac{1}{2\pi i} \!
  \int_{C} \frac{f(z) \theta'(z)}{\theta_{1}(z)}
  \frac{\dmeasure z}{z-a}
  =
  f(a),
  \\
  B_{n}
  =&
  0.
\end{align*}

%
% 133
%

The theorem of the last section accordingly takes the following form,
if we write $\theta_{1}(z) = 1 / \phi(z)$:

\emph{Let $f(z)$ and $\phi(z)$ be functions of $z$ analytic on and inside a
contour $C$ surrounding a point $a$, and let $t$ be such that the inequality
$$
\absval{t \phi(z) } < \absval{z - a}
$$
is satisfied at all points $z$ on the perimeter of $C$;
then the equation
$$
\zeta = a + t \phi(\zeta),
$$
regarded as an equation in $\zeta$, has one root in the interior
of $C$; and further any function of $\zeta$ analytic on and inside
$C$ can be expanded as a power series in $t$ by the formula
$$
f(\zeta)
=
f(a)
+
\sum_{n=1}^{\infty}
\frac{t^{n}}{n!}
\frac{\dd^{n-1}}{\dd a^{n-1}}
\thebracket{
  f'(a) \phi^{n}(a)
}.
$$
}
This result was published by Lagrange\footnote{TODO Mem. de VAcad. de Berlin, xxiv.; Oeuvres, iii. p. 25.} in 1770.
\begin{wandwexample}
Within the contour surrounding $a$ defined by the inequality
$\absval{z (z - a)} > \absval{a}$, where
$\absval{a} < \half \absval{ a }$, %TODO: verify
the equation
$$
z - a - \frac{a}{z} = 0
$$
has one root $\zeta$, the expansion of which is given by Lagrange's theorem
in the form
$$
\zeta
=
a
+
\sum_{n=1}^{\infty}
\frac{(-)^{n-1} (2n-2)!}{n! (n-1)! a^{2n-1}} a^{n}
%TODO: verify
$$

Now, from the elementary theory of quadratic equations, we know that
the equation
$$
z - a - \frac{a}{z} = 0
%TODO: verify
$$
has two roots, namely $TODO$ and $TODO$; and our
expansion
\emph{represents the former\footnote{The latter is outside the given
    contour.} of these only}---an example of the need for
care in the discussion of these series.
\end{wandwexample}
\begin{wandwexample}
  If $y$ be that one of the roots of the equation
  $$
  TODO
  $$
  which tends to $1$ when $z \rightarrow 0$, shew that
  $$
  TODO
  $$
  so long as $\absval{z} < \frac{1}{4}$.
\end{wandwexample}
\begin{wandwexample}
If $x$ be that one of the roots of the equation
$$
x = 1 + y x^{a}
$$
which tends to $1$ when $y \rightarrow 0$, shew that
$$
TODO
$$
the expansion being valid so long as
$$
\absval{y}
<
\absval{
  (a-1)^{a-1} a^{-a}
}.
$$
\addexamplecitation{McClintock.}
\end{wandwexample}

%
% 134
%

\Section{7}{4}{The expansion of a class of functions in rational fractions*'.}
Consider a function $f(z)$, whose only singularities in the finite
part of the plane are simple poles $a_{1},a_{2},a_{3},\ldots$, where
$\absval{a_{1}} \leq \absval{a_{2}} \leq \absval{a_{3}} \leq \cdots$;
let $b_{1},b_{2},b_{3},\ldots$, be the residues at these
poles, and let it be possible to choose a sequence of circles $C_{m}$ (the
radius of $C_{m}$ being $R_{m}$) with centre at $O$, not passing through any
poles, such that $\absval{f(z)}$ is bounded on $C_{m}$. (The function
$\cosec z$ may
be cited as an example of the class of functions considered, and we
take $R_{m} = (m + \half)\pi$.) Suppose further that
$R_{m} \rightarrow \infty$ as $m \rightarrow \infty$ and that
the upper bound\footnote{Which is a function of $m$.}
of $\absval{f(z)}$ on $C_{m}$ is itself bounded
as\footnote{Of course $R_{m}$ need not (and frequently must not) tend to infinity
  continuously; e.g. in the example taken
  $R_{m} = (m+\half)z$, where $m$ assumes only integer values.}
$m\rightarrow\infty$; so
that, for all points on the circle $C_{m}$, $\absval{f(z)} < M$, where $M$ is
independent of $m$.

Then, if $x$ be not a pole of $f(z)$, since the only poles of the
integrand are the poles of $f(z)$ and the point $z = x$, we have, by \hardsectionref{6}{1},
$$
\frac{1}{2 \pi i} \int_{C_{m}} \frac{f(z)}{z-x} \dmeasure z
=
f(x) + \sum_{r} \frac{b_{r}}{a_{r}-x}.
$$
where the summation extends over all poles in the interior of $C_{m}$.

But
\begin{align*}
  \frac{1}{2 \pi i}
  \int_{C_{m}} \frac{f(z)}{z-x} \dmeasure z
  =&
  \frac{1}{2 \pi i}
  \int_{C_{m}} \frac{f(z)}{z} \dmeasure z
  +
  \frac{x}{2 \pi i}
  \int_{C_{m}} \frac{f(x)}{z(z-x)} \dmeasure z
  \\
  =&
  f(0) + \sum_{r} \frac{b_{r}}{a_{r}}
  +
  \frac{x}{2 \pi i}
  \int_{C_{m}} \frac{f(z)}{z(z-x)} \dmeasure z,
\end{align*}
if we suppose the function $f(z)$ to be analytic at the origin.

Now as $m \rightarrow \infty$,
$\int_{C_{m}} \frac{f(z)}{z(z-x)} \dmeasure z$ is
$\bigo(R_{m}^{-1})$, and so tends to zero as
$m$ tends to infinity.

Therefore, making $m \rightarrow \infty$, we have
$$
0
=
f(x) - f(0)
+
\sum_{n=1}^{\infty}
b_{n} \theparen{
  \frac{1}{a_{n}-x} - \frac{1}{a_{n}}
}
-
\lim_{m\rightarrow\infty}
\frac{x}{2 \pi i}
\int_{C_{m}} \frac{f(z)}{z(z-x)} \dmeasure x,
$$
\ie
$$
f(x) = f(0)
+
\sum_{n=1}^{\infty}
b_{n}
\thebrace{
  \frac{1}{x-a_{n}} + \frac{1}{a_{n}}
  %TODO:consistent with above?
}
$$
which is an expansion of $f(x)$ in rational fractions of $x$; and the
summation extends over \emph{all} the poles of $f(x)$.

%\begin{smalltext}
If $\absval{a_{n}} < \absval{a_{n+1}}$ this series converges uniformly
throughout the region given by $\absval{x} < a$, where $a$ is any constant
(except near the points $a_{n}$).
For if $R_{m}$ be the radius of the circle which encloses the points
$\absval{a_{1}}, \ldots, \absval{a_{n}}$,
the modulus of the remainder of the terms of the series after the first $n$ is
$$
\absval{ \frac{x}{2 \pi i}
  \int_{C_{m}} \frac{f(z)}{z(z-x)} \dmeasure z
}
< \frac{Ma}{R_{m}-a},
$$
by \hardsubsectionref{4}{6}{2}; and, given
$\eps$, we can choose $n$ \emph{independent} of $x$
such that $Ma/(R_{m}-a) < \eps$.

* Mittag-Leffler, Acta Soc. Scient. Fennicae, xi. (1880), pp. 273-293.
See also Acta Math. iv. (1884), pp. 1-79.

%
% 135
%

The convergence is obviously still uniform even if
$\absval{a_{n}} \leq \absval{a_{n+1}}$
provided the terms of the series are grouped so as to combine the
terms corresponding to poles of equal moduli.

If, instead of the condition $\absval{f(z)} < M$, we have the
condition $\absval{ z^{-p} f(z) } < M$,
where $M$ is independent of $m$ when $z$ is on $C_{m}$, and $p$ is
a positive integer, then we should have to
expand $\int_{C} \frac{f(z)}{z-x} \dmeasure z$ by writing
$$
\frac{1}{z-x}
=
\frac{1}{z}
+ \frac{x}{z^{2}}
+ \cdots
+ \frac{x^{p+1}}{z^{p+1}(z-x)},
$$
and should obtain a similar but somewhat more complicated expansion.
\begin{wandwexample}
Prove that
$$
\cosec z
=
\frac{1}{z}
+
\sum (-)^{n}
\theparen{\frac{1}{z-n\pi} + \frac{1}{n\pi}}
$$
the summation extending to all positive and negative values of $n$.

To obtain this result, let $\cosec z - \frac{1}{z} = f(z)$.
The singularities of this function are at the
points $z=n\pi$, where $n$ is any positive or negative integer.

The residue of $f(z)$ at the singularity $n\pi$ is therefore
$(-)^{n}$, and the reader will easily see that $\absval{f(z)}$ is
bounded on the circle $\absval{z} = (n + \half) \pi$ as
$n \rightarrow \infty$.

Applying now the general theorem
$$
f(z)
=
f(0)
+
\sum c_{n} \thebracket{ \frac{1}{z-a_{n}} + \frac{1}{a_{n}}  },
$$
where $c_{n}$ is the residue at the singularity $a_{n}$, we have
$$
f(z)
=
f(0)
+
\sum (-)^{n} \thebrace{ \frac{1}{z-n\pi} + \frac{1}{n\pi}  }.
$$

But
$$
f(0)
=
\lim_{z \rightarrow 0} \frac{z - \sin z}{z \sin z} = 0.
$$

Therefore
$$
\cosec z
=
\frac{1}{z}
+
\sum (-)^{n} \thebracket{ \frac{1}{z-n\pi} + \frac{1}{n\pi}  },
$$
which is the required result.
\end{wandwexample}
\begin{wandwexample}
If $0<a<1$, shew that
$$
\frac{e^{az}}{e^{z}-1}
=
\frac{1}{z}
+
\sum_{n=1}^{\infty}
\frac{2z \cos 2na\pi - 4n\pi \sin 2na\pi}{z^{2}+4n^{2}\pi^{2}}.
$$
\end{wandwexample}
\begin{wandwexample}
Prove that
$$
\frac{1}{2 \pi x^{2} (\cosh x - \cos x)}
=
\frac{1}{2 \pi x^{4}}
-
\frac{1}{e^{\pi}-e^{-\pi}}
\frac{1}{\pi^{4} + \frac{1}{4} x^{4}}
+
\frac{2}{e^{2 \pi}-e^{-2 \pi}}
\frac{1}{(2 \pi)^{4} + \frac{1}{4} x^{4}}
-
\frac{3}{e^{3 \pi}-e^{-3 \pi}}
\frac{1}{(3 \pi)^{4} + \frac{1}{4} x^{4}}
+
\cdots.
$$

The general term of the series on the right is
$$
\frac{(-)^{r} r}{(e^{r \pi}-e^{-r \pi})
  \thebrace{(r \pi)^{4} + \frac{1}{4} x^{4}}},
$$
which is the residue at each of the four singularities
$r, -r, ri, -ri$ of the function
$$
\frac{\pi z}{(\pi^{4}z^{4} + \frac{1}{4} x^{4}) (e^{\pi z} - e^{-\pi
    z}) \sin \pi z}.
$$

%
% 136
%

The singularities of this latter function which are not of the type
$r, -r, ri, -ri$ are at the five points
$$
0,
\frac{(\pm 1 \pm i) x}{2 \pi}.
$$
At $z=0$ the residue is
$$
\frac{2}{\pi x^{4}};
$$
at each of the four points
$\frac{(\pm 1 \pm i) x}{2 \pi}$, the residue is
$$
\thebrace{
  2 \pi x^{2} (\cos x - \cosh x)
}^{-1}
$$

Therefore
\begin{align*}
  4
  \sum_{r=1}^{\infty}
  \frac{(-)^{r} r}{e^{r\pi} - e^{-r\pi}}
  \frac{1}{(r\pi)^{4} + \frac{1}{4} x^{4}}
  +
  \frac{2}{\pi x^{4}}
  -&
  \frac{2}{ \pi x^{2} (\cosh x - \cos x)}
  \\
  &
  =
  \frac{1}{2 \pi i}
  \lim_{n \rightarrow \infty}
  \int_{C}
  \frac{\pi z}{
    (\pi^{4}z^{4} + \frac{1}{x^{4}})
    (e^{\pi z}-e^{-\pi z})
    \sin \pi z
  }
  \dmeasure z,
\end{align*}
where $C$ is the circle whose radius is $n + \half$, ($n$ an integer),
and whose centre is the origin. But, at points on $C$, this integrand is
$\bigo( \absval{z}^{-3} )$; the limit of the integral round $C$ is
therefore zero.

From the last equation the required result is now obvious.
\end{wandwexample}
\begin{wandwexample}
  Prove that
  $$
  \sec x
  =
  4 \pi
  \theparen{
    \frac{1}{\pi^{2} - 4 x^{2}}
    -
    \frac{3}{9 \pi^{2} - 4 x^{2}}
    +
    \frac{5}{25 \pi^{2} - 4 x^{2}}
    -
    \cdots
  }.
  $$
\end{wandwexample}
\begin{wandwexample}
  Prove that
  $$
  \cosech x
  =
  \frac{1}{x}
  -
  2x
  \theparen{
    \frac{1}{\pi^{2} + x^{2}}
    -
    \frac{1}{4\pi^{2} + x^{2}}
    +
    \frac{1}{9 \pi^{2} + x^{2}}
    -
    \cdots
  }.
  $$
\end{wandwexample}
\begin{wandwexample}
  Prove that
  $$
  \sec x
  =
  4 \pi
  \theparen{
    \frac{1}{\pi^{2} + 4 x^{2}}
    -
    \frac{3}{9\pi^{2} + 4 x^{2}}
    +
    \frac{5}{25 \pi^{2} + 4 x^{2}}
    -
    \cdots
  }.
  $$
\end{wandwexample}
\begin{wandwexample}
  Prove that
  $$
  \coth x
  =
  \frac{1}{x}
  +
  2x
  \theparen{
    \frac{1}{\pi^{2} + x^{2}}
    +
    \frac{1}{4 \pi^{2} + x^{2}}
    +
    \frac{1}{9 \pi^{2} + x^{2}}
    +
    \cdots
  }.
  $$
\end{wandwexample}
\begin{wandwexample}
  Prove that
  $$
  \sum_{m=-\infty}^{\infty}
  \sum_{n=-\infty}^{\infty}
  \frac{1}{ (m^{2}+a^{2}) (n^{2}+b^{2}) }
  =
  \frac{\pi^{2}}{ab} \coth \pi a \coth \pi b.
  $$
\addexamplecitation{Math. Trip. 1899.}
\end{wandwexample}
\Section{7}{5}{The expansion of a class of functions as
infinite products.}

The theorem of the last article can be applied to the expansion of a
certain class of functions as infinite products.

For let $f(z)$ be a function which has simple zeros at the
points\footnote{These being the only zeros of $f(z)$; and $a_{n} \neq 0$.}
$a_{1}, a_{2}, a_{3}, \ldots$, where
$\lim_{n \rightarrow \infty} \absval{a_{n}}$ is infinite; and let
$f(z)$ be analytic for all values of $z$.

Then $f'(z)$ is analytic for all values of $z$
(\hardsubsectionref{5}{2}{2}), and so $\frac{f'(z)}{f(z)}$ can have
singularities only at the points $a_{1}, a_{2}, a_{3}, \ldots$.

Consequently, by Taylor's theorem,
$$
f(z)
=
(z-a_{r}) f'(a_{r})
+
\frac{ (z-a_{r})^{2} }{2} f''(a_{r})
+
\cdots
$$
and
$$
f'(z)
=
f'(a_{r})
+
(z-a_{r}) f''(a_{r})
+
\cdots.
$$

/ /
%
% 137
%
It follows immediately that at each of the points $a_{r}$, the
function
$\frac{f'(z)}{f(z)}$ has a simple pole, with residue $+1$.

If then we can find a sequence of circles $C_{m}$ of the nature described
in \hardsectionref{7}{4}, such that $\frac{f'(z)}{f(z)}$ is bounded on
$C_{m}$ as $m \rightarrow \infty$, it follows, from the
expansion given in \hardsectionref{7}{4}, that
$$
\frac{f'(z)}{f(z)}
=
\frac{f'(0)}{f(0)}
+
\sum_{n=1}^{\infty}
\thebrace{
  \frac{1}{z-a_{n}}
  -
  \frac{1}{a_{n}}
}.
$$

Since this series converges uniformly when the terms are suitably
grouped (\hardsectionref{7}{4}), we may integrate term-by-term
(\hardsectionref{4}{7}). Doing so, and taking the exponential of each
side, we get
$$
f(z)
=
c
e^{ z \frac{f'(0)}{f(0)} }
\prod_{n=1}^{\infty}
\thebrace{
  \theparen{ 1 - \frac{z}{a_{n}} }
  e^{ \frac{z}{a_{n}} }
},
$$
where $c$ is independent of $z$.

Putting $z = 0$, we see that $f(0) = c$, and thus the general result
becomes
$$
f(z)
=
f(0)
e^{ z \frac{f'(0)}{f(0)} }
\prod_{n=1}^{\infty}
\thebrace{
  \theparen{ 1 - \frac{z}{a_{n}} }
  e^{ \frac{z}{a_{n}} }
}.
$$

This furnishes the expansion, in the form of an infinite product, of
any function $f(z)$ which fulfils the conditions stated.
\begin{wandwexample}
  Consider the function
  $f(z) = \frac{\sin z}{z}$, which has simple zeros at
  the points $r \pi$, where $r$ is any positive or negative integer.

  In this case we have
  $$
  f(0) = 1,
  \quad
  f'(0) = 0,
  $$
  and so the theorem gives immediately
  $$
  \frac{\sin z}{z}
  =
  \prod_{n=1}^{\infty}
  \thebrace{
    \theparen{1 - \frac{z}{n \pi}}
    e^{ \frac{z}{n \pi} }
  }
  \thebrace{
    \theparen{1 + \frac{z}{n \pi}}
    e^{ -\frac{z}{n \pi} }
  };
  $$
  for it is easily seen that the condition concerning the behaviour of
  $\frac{f'(z)}{f(z)}$ as $\absval{z} \rightarrow \infty$ is fulfilled.
\end{wandwexample}
\begin{wandwexample}
  Prove that
  \begin{align*}
  \thebrace{
    1 + \theparen{ \frac{k}{x}  }^{2}
  }
  \thebrace{
    1 + \theparen{ \frac{k}{2\pi - x}  }^{2}
  }
  &
  \thebrace{
    1 + \theparen{ \frac{k}{2\pi + x}  }^{2}
  }
  \thebrace{
    1 + \theparen{ \frac{k}{4\pi - x}  }^{2}
  }
  \thebrace{
    1 + \theparen{ \frac{k}{4\pi + x}  }^{2}
  } \cdots
  \\
  =&
  \frac{\cosh x - \cos x}{1-\cos x}.
  \end{align*}
\addexamplecitation{Trinity, 1899.}
\end{wandwexample}
\Section{7}{6}{The factor theorem of Weierstrass*.}

The theorem of \hardsectionref{7}{5} is very similar to a more general theorem in
which the character of the function $f(z)$, as
$\absval{z} \rightarrow \infty$, is not so
narrowly restricted.

* Berliner Alh. (1876), pp. 11-60; Math. Werke, 11. (1895), pp.
77-124.
%
% 138
%
Let $f(z)$ be a function of $z$ with no essential singularities (except at
`the point infinity'); and let the zeros and poles of $f(z)$ be at
$a_{1}, a_{2}, a_{3}, \ldots$, where
$0 < \absval{a_{1}} \leq \absval{a_{2}} \leq \absval{a_{3}} \ldots$.
Let the zero\footnote{We here regard a pole as being a zero of
  negative order.} at $a_{n}$ be of (integer) order $m_{n}$.

If the number of zeros and poles is unlimited, it is necessary that
$\absval{a_{n}} \rightarrow \infty$, as $n \rightarrow \infty$;
for, if not, the points $a_{n}$ would have a limit
point\footnote{From the two-dimensional analogue
  of \hardsubsectionref{2}{2}{1}.},
which would be an essential singularity of $f(z)$.

We proceed to shew first of all that it is possible to find
polynomials $g(z)$ such that
$$
\prod_{n=1}^{\infty}
\thebracket{
  \thebrace{
    \theparen{
      1
      -
      \frac{z}{a_{n}}
    }
    e^{g_{n}(z)}
  }^{m_{n}}
}
$$
converges for all\footnote{Provided that $z$ is not at one of the points $a_{m}$
  for which $m$ is negative.} finite values of $z$.

Let $K$ be any constant, and let $\absval{z} < K$; then, since
$\absval{a_{n}} \rightarrow \infty$, we can
find $N$ such that, when $n > N$, $\absval{a_{n}} > 2K$.

The first $N$ factors of the product do not affect its
convergence\footnote{Provided that $z$ is not at one of the points $a_{m}$
  for which $m$ is negative.}; % TODO:duplicated-footnote
consider any value of $n$ greater than $N$, and let
$$
g_{n}(z)
=
\frac{z}{a_{n}}
+
\half \theparen{ \frac{z}{a_{n}} }^{2}
+
\cdots
+
\frac{1}{k_{n}-1}
\theparen{ \frac{z}{a_{n}} }^{k_{n} - 1}.
$$
Then
\begin{align*}
  \absval{
    -
    \sum_{m=1}^{\infty}
    \frac{1}{m}
    \theparen{
      \frac{z}{a_{n}}
    }^{m}
    +
    g_{n}
  }
  =&
  \absval{
    \sum_{m=k_{n}}^{\infty}
    \frac{1}{m}
    \theparen{
      \frac{z}{a_{n}}
    }^{m}
  }
  \\
  <&
  \absval{ \frac{z}{a_{n}} }^{k_{n}}
  \sum_{m=0}^{\infty}
  \absval{ \frac{z}{a_{n}} }^{m}
  \\
  <&
  2
  \absval{ (K a_{n}^{-1})^{k_{n}}  },
\end{align*}
since
$ \absval{ z_{n} a_{n}^{-1} } < \half$.

Hence
$$
\thebrace{
  \theparen{
    1
    -
    \frac{z}{a_{n}}
  }
  e^{g_{n}(z)}
}^{m_{n}}
=
e^{u_{n}(z)},
$$
where
$$
\absval{ u_{n}(z) }
\leq
2
\absval{
  m_{n}
  (K a_{n}^{-1})^{k_{n}}
}.
$$

Now $m_{n}$ and $a_{n}$ are given, but $k_{n}$ is at our disposal;
since $K a_{n}^{-1} < \half$, we
choose $k_{n}$ to be the smallest number such that
$2 \absval{m_{n} (K a_{n}^{-1})^{k_{n}}} < b_{n}$,
where
$\sum_{n=1}^{\infty} b_{n}$ is any convergent series\footnote{E.g. we
  might take $b_{n} = 2^{-n}$.} of positive terms.

Hence
$$
\prod_{n = N+1}^{\infty}
\thebracket{
  \thebrace{
    \theparen{
      1
      -
      \frac{z}{a_{n}}
    }
    e^{g_{n}(z)}
  }^{m_{n}}
}
=
\prod_{n = N+1}^{\infty}
e^{u_{n}(z)},
$$
where
$ \absval{ u_{n}(z) } < b_{n} $;
and therefore, since $b_{n}$ is independent of $z$, the
product converges absolutely and uniformly when
$\absval{z} < K$, except near the points $a_{n}$.
%
% 139
%

Now let
$$
F(z)
=
\prod_{n=1}^{\infty}
\thebracket{
  \thebrace{
    \theparen{
      1
      -
      \frac{z}{a_{n}}
    }
    e^{g_{n}(z)}
  }^{m_{n}}
}.
$$

Then, if $f(z) \div F(z) = G_{1}(z)$, $G_{1}(z)$ is an integral
function (\hardsubsectionref{5}{6}{4}) of $z$ and has no zeros.

It follows that
$\frac{1}{G_{1}(z)} \frac{\dd}{\dd z} G_{1}(z)$
is analytic for all finite values of $z$; and so, by Taylor's theorem,
this function can be expressed as a series $\sum_{n=1}^{\infty} n
b_{n} z^{n-1}$ converging everywhere; integrating, it follows that
$$
G_{1}(z) = c e^{G(z)},
$$
where $G(z) = \sum_{n=1}^{\infty} b_{n} z^{n}$ and $c$ is a constant;
this series converges everywhere, and so $G(z)$ is an integral
function.

Therefore, finally,
$$
f(z)
=
f(0)
e^{G(z)}
\prod_{n=1}^{\infty}
\thebracket{
  \thebrace{
    \theparen{
      1
      -
      \frac{z}{a_{n}}
    }
    e^{g_{n}(z)}
  }^{m_{n}}
},
$$
where $G(z)$ is some integral function such that $G(0) = 0$.
%\begin{smalltext}
[Note. The presence of the arbitrary element $G(z)$ which occurs in
this formula for $f(z)$ is due to the lack of conditions as to the
behaviour of $f(z)$ as $\absval{z} \rightarrow \infty$.]

\corollary. If $m_{n} =1$, it is
sufficient to take $k_{n} = n$, by \hardsubsectionref{2}{3}{6}.
%\end{smalltext}
\Section{7}{7}{The expansion of a class of periodic functions in a series of
cotangents.}

Let $f(z)$ be a periodic function of $z$, analytic except at a certain
number of simple poles; for convenience, let $\pi$ be the period of
$f(z)$ so that $f(z) = f(z + \pi)$.

Let $z = x + iy$ and let $f(z) \rightarrow l$ uniformly with respect
to $x$ as $y \rightarrow +\infty$,
when $0 \leq x \leq \pi$; similarly let $f(z) \rightarrow l'$
uniformly as $y \rightarrow -\infty$.

Let the poles of $f(z)$ in the strip $0 < x \leq \pi$ be at
$a_{1}, a_{2}, \ldots, a_{n}$; and
let the residues at them be $c_{1}, c_{2}, \ldots, c_{n}$.

Further, let $ABCD$ be a rectangle whose corners are\footnote{If any
  of the poles are on $x = \pi$, shift the rectangle slightly to
  the right; $\rho, \rho'$ are to be taken so large that
  $a_{1}, a_{2}, \ldots, a_{n}$ are
  inside the rectangle.}
$-i\rho$, $\pi - i\rho$, $\pi + i\rho'$, and $i\rho'$ in order.

Consider
$$
\frac{1}{2 \pi i}
\int f(t) \cot (t-z) \dmeasure t
$$
taken round this rectangle; the residue of the integrand at $a_{r}$ is
$c_{r} \cot (a_{r}-z)$, and the residue at $z$ is $f(z)$.

Also the integrals along $DA$ and $CB$ cancel on account of the
periodicity of the integrand; and as $\rho \rightarrow \infty$,
the integrand on $AB$ tends
uniformly to $l' i$, while as $\rho' \rightarrow \infty$
the integrand on $CD$ tends uniformly
to $-li$; therefore
$$
\half (l' + l)
=
f(z)
+
\sum_{r=1}^{n}
c_{r}
\cot (a_{r} - z).
$$
%
% 140
%

That is to say, we have the expansion
$$
f(z)
=
\half (l' + l)
+
\sum_{r=1}^{n}
c_{r}
\cot (z - a_{r}).
$$
\begin{wandwexample}
  \begin{align*}
    \cot (x - a_{1})
    \cot (x - a_{2})
    \cdots
    \cot (x - a_{n})
    =&
    \sum_{r=1}^{n}
    \cot (a_{r} - a_{1})
    \cdots
    *
    \cdots
    \cot (a_{r} - a_{n})
    \cot (x - a_{r})
    +
    (-)^{\half n},
    \\
    \textrm{or}
    =&
    \sum_{r=1}^{n}
    \cot (a_{r} - a_{1})
    \cdots
    *
    \cdots
    \cot (a_{r} - a_{n})
    \cot (x - a_{r}),
  \end{align*}
according as $n$ is even or odd; the `$*$' means that the factor
$\cot (a_{r} - a_{r})$ is omitted.
\end{wandwexample}
\begin{wandwexample}
Prove that
\begin{align*}
  \frac{
    \sin (x - b_{1})
    \sin (x - b_{2})
    \cdots
    \sin (x - b_{n})
  }{
    \sin (x - a_{1})
    \sin (x - a_{2})
    \cdots
    \sin (x - a_{n})
  }
  =&
  \frac{
    \sin (a_{1} - b_{1})
    \cdots
    \sin (a_{1} - b_{n})
  }{
    \sin (a_{1} - a_{2})
    \cdots
    \sin (a_{1} - a_{n})
  }
  \cot (x - a_{1})
  \\
  &
  +
  \frac{
    \sin (a_{2} - b_{1})
    \cdots
    \sin (a_{2} - b_{n})
  }{
    \sin (a_{2} - a_{1})
    \cdots
    \sin (a_{2} - a_{n})
  }
  \cot (x - a_{2})
  \\
  &
  +
  \cdots
  \\
  &
  +
  \cos (a_{1} + a_{2} + \cdots + a_{n}
  - b_{1} - b_{2} - \cdots - b_{n}).
\end{align*}
\end{wandwexample}
\Section{7}{8}{Borel's theorem.}
\footnote{TODO Lemons sur les series divergentes (1901), p. 94. See also the
memoirs there cited.}

Let $f(z) = \sum_{n=0}^{\infty} a_{n} z^{n}$ be analytic when
$\absval{z} \leq r$, so that, by
\hardsubsectionref{5}{2}{3},
$\absval{ a_{n} r^{n} } < M$
where $M$ is independent of $n$.

Hence, if
$\phi(z) = \sum_{n=0}^{\infty} \frac{a_{n} z^{n}}{n!}$,
$\phi(z)$ is an integral function, and
$$
\absval{ \phi(z) }
<
\sum_{n=0}^{\infty} \frac{M \absval{z^{n}} }{ r^{n} \cdot n!}
=
M e^{\absval{z}/r},
$$
and similarly
$\absval{ \phi^{(n)}(z) } < M e^{\absval{z}/r}/r^{n}$.

Now consider
$f_{1}(z) = \int_{0}^{\infty} e^{-t} \phi(zt) \dmeasure t$;
this integral is an analytic function
of $z$ when $\absval{z} < r$, by \hardsubsectionref{5}{3}{2}.

Also, if we integrate by parts,
\begin{align*}
  f_{1}(z)
  =&
  \thebracket{- e^{-t} \phi(zt) }_{0}^{\infty}
  +
  z \int_{0}^{\infty} e^{-t} \phi'(zt) \dmeasure t
  \\
  =&
  \sum_{m=0}^{n}
  z^{m}
  \thebracket{ - e^{-t} \phi^{(m)}(zt) }_{0}^{\infty}
  +
  z^{n+1}
  \int_{0}^{\infty}
  e^{-t} \phi^{(n+1)}(zt) \dmeasure t.
\end{align*}

But $\lim_{t \rightarrow 0} e^{-t} \phi^{(m)}(zt) = a_{m}$; and,
when $\absval{z} < r$,
$\lim_{t \rightarrow \infty} e^{-t} \phi^{(m)}(zt) = 0$.

Therefore
$$
f_{1}(z) = \sum_{m=0}^{n} a_{m} z^{m} + R_{n},
$$
%
% 141
%
where
\begin{align*}
  TODO
\end{align*}

Consequently, when $\absval{z} < r$,
$$
f_{1}(z) = \sum_{m=0}^{\infty} a_{m} z^{m} = f(z);
$$
and so
$$
f(z) = \int_{0}^{\infty} e^{-t} \phi(zt) \dmeasure t,
$$
where
$
\phi(z) = \sum_{n=0}^{\infty} \frac{a_{n} z^{n}}{n!};
$
is called \emph{Borel's function} associated with
$\sum_{n=0}^{\infty} a_{n} z^{n}$.

If
$S = \sum_{n=0}^{\infty} a_{n}$
and
$\phi(z) = \sum_{n=0}^{\infty} \frac{a_{n} z^{n}}{n!}$
and if we can establish the relation
$S = \int_{0}^{\infty} e^{-t} \phi(t) \dmeasure t$,
the series $S$ is said \hardsubsectionref{8}{4}{1}) to be
'\emph{summable (B)}; so that the
theorem just proved shews that a Taylor's series representing an
analytic function is summable (B).

\Subsection{7}{8}{1}{Borel's integral and analytic continuation.}
We next obtain Borel's result that his integral represents an analytic
function in a more extended region than the interior of the circle
$\absval{z} = r$.

TODO:figure

This extended region is obtained as follows: take the singularities
$a,b,c,\ldots$ of $f(z)$ and through each of them draw a line perpendicular
to the line joining that singularity to the origin. The lines so drawn
will divide the plane into regions of which one is a polygon with the
origin inside it.

\emph{Then Borel's integral represents an analytic function}
(which, by \hardsectionref{5}{5}
and \hardsectionref{7}{8}, is obviously that defined by $f(z)$
and its continuations)
\emph{throughout the interior of this polygon.} The reader will observe that
this is the first actual formula obtained for the analytic
continuation of a function, except the trivial one of
\hardsectionref{5}{5}, example.

For, take any point $P$ with affix $\zeta$ inside the polygon; then
the circle on $OP$ as diameter has no singularity on or inside
it\footnote{The reader will see this from the figure; for if there were such a
singularity the corresponding side of the polygon would pass between
$O$ and $P$; \ie,
 $P$ would be outside the polygon.}; and
consequently we can draw a slightly
%
% 142
%
larger concentric circle\footnote{The differeuce of the radii of the
  circles being, say, $\delta$.} $C$ with no singularity on or inside
it. Then, by \hardsectionref{5}{4},
$$
a_{n}
=
\frac{1}{2 \pi i}
\int_{C} \frac{f(z)}{z^{n+1}} \dmeasure z,
$$
and so
$$
\phi(\zeta t)
=
\frac{1}{2 \pi i}
\sum_{n=0}^{\infty}
\frac{\zeta^{n} t^{n}}{n!}
\int_{C} \frac{f(z)}{z^{n+1}} \dmeasure z;
$$
but
$\sum_{n=0}^{\infty} \frac{\zeta^{n} t^{n}}{n!} \frac{f(z)}{z^{n+1}}$
converges uniformly (\hardsubsectionref{3}{3}{4}) on $C$ since
$f(z)$ is bounded and $\absval{z} \geq \delta > 0$, where
$\delta$ is independent of $z$; therefore, by \hardsectionref{4}{7},
$$
\phi(\zeta t)
=
\frac{1}{2 \pi i}
\int_{C} z^{-1} f(z) \exp(\zeta t z^{-1}) \dmeasure z,
$$
and so, when $t$ is real,
$\absval{\phi(\zeta t)} < F(\zeta) e^{\lambda t}$,
where $F(\zeta)$ is bounded in any closed region lying wholly
\emph{inside} the polygon and is independent of $t$;
and $\lambda$ is the greatest value of the real part of
$\zeta / z$ on $C$.

If we draw the circle traced out by the point $z/\zeta$, we see that
the real part of $\zeta/z$ is greatest when $z$ is at the extremity of the
diameter through $\zeta$, and so the value of $\lambda$ is
$ \absval{\zeta} \cdot \thebrace{\absval{\zeta} + \delta}^{-1} < 1$.

We can get a similar inequality for $\phi'(\zeta t)$ and hence,
by \hardsubsectionref{5}{3}{2},
$\int_{0}^{\infty} e^{-t} \phi(\zeta t) \dmeasure t$
is analytic at $\zeta$ and is obviously a one-valued function of
$\zeta$.

This is the result stated above.

\Subsection{7}{8}{2}{Expansions in series of inverse factorials.}

A mode of development of functions, which, after being used by
Nicole\footnote{TODO Mem de VAcad. des Sci. (Paris, 1717); see Tweedie, Proc. Edin. Math.
  Soc. xxxvi. (1918).}
and
Stirling\footnote{TODO Methodus Dijferentialis (Londou, 1730).}
in the eighteenth century, was systematically
investigated by
\Schlomilch\footnote{TODO Compendium der h'dheren AnalysU. More recent investigations are due
to Kluyver, Nielsen and Pincherle. See Comptes liendiis, cxxxiii.
(1901), cxxxiv. (1902), Annales de I'Ecole norm, sup. (3), XIX.,
XIII., xxiii., JRendiconti del Lincei, (5), xi. (1902), and Palermo
Rendiconti, xxxiv. (1912). Properties of functions defined by series
of inverse factorials have been studied in an important memoir by
Norlund, Acta Math, xxxvii. (1914), pp. 327-H87.}
in 1863, is that of expansion in a series
of inverse factorials.

To obtain such an expansion of a function analytic when
$\absval{z} > r$, we let
the function be
$f(z) = \sum_{n=0}^{\infty} a_{n} z^{-n}$, and use the formula
$f(z) = \int_{0}^{\infty} z e^{-tz} \phi(t) \dmeasure t$,
where $\phi(t) = \sum_{n=0}^{\infty} a_{n} t^{n} / n!$;
this result may be obtained in the same way as
that of \hardsectionref{7}{8}.
Modify this by writing
$e^{-t} = 1 - \xi$, $\phi(t) = F(\xi)$;
then
$$
f(z)
=
\int_{0}^{1}
z (1 - \xi)^{z-1} F(\xi) \dmeasure \xi.
$$

Now if $t = u + iv$ and if $t$ be confined to the strip
$-\pi < v < \pi$, $t$ is a
one-valued function of $\xi$ and $F(\xi)$ is an analytic function of
$\xi$; and $\xi$ is
restricted so that $-\pi < \arg (1-\xi) < \pi$. Also the interior of the
circle $\absval{\xi} = 1$ corresponds
%
% 143
%
to the interior of the curve traced out by the point
$t = - \log \theparen{2 \cos \half \theta} + \half i \theta$,
(writing $\xi= \exp \thebrace{i (\theta + \pi) }$ ); and inside this curve
$$
\absval{t} - \Re(t)
\leq
\sqrt{ \thebrace{\Re(t)}^{2} + \pi^{2} }
-
\Re(t)
\rightarrow
0,
$$
as $\Re(t) \rightarrow \infty$.

It follows that, when $\absval{\xi} \leq 1$,
$\absval{F(\xi)} < M e^{r\absval{t}} < M_{1} \absval{e^{rt}}$,
where $M_{1}$ is independent of $t$; and so
$F(\xi) < M_{1} \absval{(1-\xi)^{-r}}$.

Now suppose that $0 \leq \xi < 1$; then, by
\hardsubsectionref{5}{2}{3},
$\absval{F^{(n)}(\xi)} < M_{2} n! \rho^{-n}$
where $M_{2}$ is the upper bound of
$\absval{F(z)}$ on a circle with centre $\xi$ and
radius $\rho < 1 - \xi$.

Taking $\rho = \frac{n}{n+1} (1-\xi)$ and observing
that\footnote{ $(1 + x^{-1})^{x}$ increases with $x$;
  for $\frac{1}{1-y} > e^{y}$, when $y < 1$, and so
  $\log \theparen{\frac{1}{1-y}} > y$. That is to
  say, putting $y^{-1} = 1+x$,
  $
  \frac{\dd}{\dd x} x \log (1 + x^{-1})
  =
  \log (1 + x^{-1})
  -
  \frac{1}{1+x}
  >
  0
  $.
}
$(1 + n^{-1})^{n} < e$
we find that
\begin{align*}
  \absval{F^{(n)}(\xi)}
  <&
  M_{1}
  \thebracket{
    1
    -
    \thebrace{
      \xi
      +
      \frac{n}{n+1} \xi
    }
  }^{-r}
  \cdot
  n!
  \thebrace{
    \frac{n (1-\xi)}{n+1}
  }^{-n}
  \\
  <&
  M_{1} e (n+1)^{r} n! (1-\xi)^{-r-n}.
\end{align*}

Remembering that, by \hardsectionref{4}{5}, $\int_{0}^{1}$ means
$\lim_{\epsilon \rightarrow +0} \int_{0}^{1-\epsilon}$, we
have, by repeated integrations by parts,
\begin{align*}
  f(z)
  =&
  \lim_{\epsilon \rightarrow +0}
  \thebracket{
    -(1 - \xi)^{z} F(\xi)
  }_{0}^{1-\epsilon}
  +
  \int_{0}^{1 - \epsilon}
  (1 - \xi)^{z} F'(\xi) \dmeasure \xi
  \\
  =&
  \lim_{\epsilon \rightarrow +0}
  \thebracket{
    -(1 - \xi)^{z} F(\xi)
  }_{0}^{1 - \epsilon}
  +
  \frac{1}{z + 1}
  \thebracket{
    -(1 - \xi)^{z+1} F'(\xi)
  }_{0}^{1 - \epsilon}
  \\
  &
  \quad
  \quad
  +
  \frac{1}{z + 1}
  \int_{0}^{1 - \epsilon}
  (1 - \xi)^{z+1} F''(\xi) \dmeasure \xi
  \\
  =&
  \cdots
  \\
  =&
  b_{0}
  +
  \frac{b_{1}}{z+1}
  +
  \frac{b_{2}}{(z+1)(z+2)}
  +
  \cdots
  +
  \frac{b_{n}}{(z+1)(z+2)\cdots(z+n)}
  +
  R_{n},
\end{align*}
where
\begin{align*}
  b_{n}
  =&
  \lim_{\epsilon \rightarrow 0}
  \thebracket{
    -(1-\xi)^{z+n} F^{(n)}(\xi)
  }_{0}^{1 - \epsilon}
  \\
  =&
  F^{(n)}(0),
\end{align*}
if the real part of $z+n-r-n>0$, \ie if $\Re(z) > r$;
further
\begin{align*}
  \absval{R_{n}}
  \leq &
  \frac{1}{ \absval{ (z+1)(z+2)\cdots(z+n)  }}
  \lim_{\epsilon \rightarrow 0}
  \int_{0}^{1 - \epsilon}
  \absval{ (1-\xi)^{z+n} F^{(n+1)}(\xi) }
  \dmeasure \xi
  \\
  <&
  \frac{ M_{1} e (n+2)^{r} n!}{
    \absval{(z+1)(z+2)\cdots(z+n)} \Re(z-r)}
  \\
  <&
  \frac{ M_{1} e (n+2)^{r} n!}{
    \absval{ (r+1+\delta)(r+2+\delta) \cdots (r+n+\delta) \delta }
    },
\end{align*}
where $\delta = \Re(z - r)$.
%
% 144
%

Now
$$
\prod_{m=1}^{n}
\thebrace{
  \theparen{
    1
    +
    \frac{r + \delta}{m}
  }
  e^{-\frac{r+\delta}{m}}
}
$$
tends to a limit \hardsubsectionref{2}{7}{1}) as
$n \rightarrow \infty$, and so $\absval{R_{n}} \rightarrow 0$
if
$ (n+2)^{r} e^{-(r+\delta) \sum_{1}^{n} 1/m} $
tends to zero; but
$$
\sum_{m=1}^{n} 1/m
>
\int_{1}^{n+1} \frac{\dmeasure x}{x}
=
\log(n+1),
$$
by \hardsubsectionref{4}{4}{3} (II), % TODO:insertref
and $(n + 2)^{r} (n+1)^{-r-\delta} \rightarrow 0$ when
$\delta > 0$; therefore $R_{n} \rightarrow 0$ as
$n \rightarrow \infty$, and so, when $\Re(z) > r$,
we have the convergent expansion
$$
f(z)
=
b_{0}
+
\frac{b_{1}}{z+1}
+
\frac{b_{2}}{(z+1)(z+2)}
+
\cdots +
\frac{b_{n}}{(z+1)(z+2)\cdots(z+n)}
+
\cdots.
$$
\begin{wandwexample}
Obtain the same expansion by using the results
$$
\frac{1}{(z+1)(z+2)\cdots(z+n+1)}
=
\frac{1}{n!}
\int_{0}^{1} u^{n} (1-u)^{z} \dmeasure u,
$$
$$
\int_{C}
\frac{ f(t) \dmeasure t }{z - t}
=
\int_{C} \dmeasure t
\int_{0}^{1}
f(t) (1-u)^{z-t-1} \dmeasure u.
$$
\end{wandwexample}
\begin{wandwexample}
Obtain the expansion
$$
\log\theparen{1 + \frac{1}{z}}
=
\frac{1}{z}
-
\frac{a_{1}}{z(z+1)}
-
\frac{a_{2}}{z(z+1)(z+2)}
-
\cdots,
$$
where
$$
a_{n}
=
\int_{0}^{1}
t (1-t) (2-t) \cdots (n-1-t) \dmeasure t,
$$
and discuss the region in which it converges.
\addexamplecitation{Schlomilch.}
\end{wandwexample}

%TODO
REFERENCES. E. Goursat, Cours d' Analyse (Paris, 1911), Chs. xv, xvi.
E. BoREL, Lecons sur les series divergentes (Paris, 1901). T. J. I'a.
Bromwich\footnote{The expansions considered by Eromwich are obtained by elementary
methods, i.e. without the use of Cauchy's theorem.} Theory of Infinite
Series (1908), Chs. viii, x, xi. 0.
Schlomilch, Compendium der hoheren Analysis, ii. (Dresden, 1874).

\begin{wandwmiscexamples}
  \begin{wandwmiscexample}
    If $y - x - \phi(y) = 0$, where $\phi$ is a given function of
    its argument, obtain the expansion
    $$
    f(y)
    =
    f(x)
    +
    \sum_{m=1}^{\infty}
    \frac{1}{m!}
    \thebrace{\phi(x)}^{m}
    \theparen{
      \frac{1}{1-\phi'(x)}
      \frac{\dd}{\dd x}
    }^{m}
    f(x),
    $$
    where $f$ denotes any analytic function of its argument, and discuss
    the range of its validity.
    \addexamplecitation{TODO:Levi-Civitk, Bertd. dei Lincei, (5), xvl
      (1907), p. 3.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    Obtain (from the formula of Darboux or otherwise) the expansion
    $$
    f(z) - f(a)
    =
    \sum_{n=1}^{\infty}
    \frac{(-)^{n-1} (z-a)^{n}}{n! (1-r)^{n}}
    \thebrace{
      f^{(n)}(z) - r^{n} f^{(n)}(a)
    };
    $$
    find the remainder after $n$ terms, and discuss the
    convergence of the series.
  \end{wandwmiscexample}
  % 145
  %
  \begin{wandwmiscexample}
    Shew that
    \begin{align*}
      f(x+h) - f(x)
      =&
      \sum_{m=1}^{n}
      (-)^{m-1}
      \frac{1 \cdot 3 \cdot 5 \cdots (2m-1)}{(m!)^{2}}
      \frac{h^{m}}{2^{m}}
      \thebrace{
        f^{(m)}(x+h)
        -
        (-)^{m} f^{(m)}(x)
      }
      \\
      & \quad
      +
      (-)^{n} h^{n+1}
      \int_{0}^{1}
      \gamma_{n}(t)
      f^{(n+1)}(x + ht) \dmeasure t,
    \end{align*}
    where
    $$
    \gamma_{n}(t)
    =
    \frac{x^{n+\half} (1-x)^{n + \half}}{(n!)^{2}}
    \frac{\dd^{n}}{\dd x^{n}}
    \thebrace{
      x^{-\half} (1-x)^{-\half}
    }
    =
    \frac{1}{\pi n!}
    \int_{0}^{1}
    (x-z)^{n}
    z^{-\half}
    (1-z)^{-\half}
    \dmeasure z,
    $$
    and shew that $\gamma_{n}(x)$ is the coefficient of
    $n! t^{n}$ in the expansion of
    $\thebrace{ (1-tx)(1+t-tx) }^{-\half}$
    in ascending powers of $t$.
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    By taking
    $$
    \phi(x+1)
    =
    \frac{1}{n!}
    \thebracket{
      \frac{\dd^{n}}{\dd u^{n}}
      \thebrace{
        \frac{(1-r)e^{xu}}{1 - r e^{-u}}
      }
    }_{n=0}
    $$
    in the formula of Darboux, shew that
    \begin{align*}
      f(x+h) - f(x)
      =&
      -
      \sum_{m=1}^{n}
      a_{m}
      \frac{h^{m}}{m!}
      \thebrace{
        f^{(m)}(x+h) - \frac{1}{r} f^{(m)}(x)
      }
      \\
      & \quad
      +
      (-)^{n} h^{n+1}
      \int_{0}^{1}
      \phi(t) f^{(n+1)}(x+ht) \dmeasure t,
    \end{align*}
    where
    $$
    \frac{1-r}{1 - r e^{-u}}
    =
    1
    -
    a_{1} \frac{u}{1!}
    +
    a_{2} \frac{u^{2}}{2!}
    -
    a_{3} \frac{u^{3}}{3!}
    +
    \cdots.
    $$
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    Shew that
    \begin{align*}
      f(z) - f(a)
      =&
      \sum_{m=1}^{n}
      (-)^{m-1}
      \frac{2 B_{m} (2^{2n} - 1)(z-a)^{2m-1}}{2m!}
      \thebrace{
        f^{(2m-1)}(a)
        +
        f^{(2m-1)}(z)
      }
      \\
      &
      \hfill +
      \frac{(z-a)^{2n+1}}{2n!}
      \int_{0}^{1}
      \psi_{2n}(t)
      f^{(2n+1)}\thebrace{
        a + t(z-a)
      }
      \dmeasure t,
    \end{align*}
    where
    $$
    \psi_{n}(t)
    =
    \frac{2}{n+1}
    \thebracket{
      \frac{\dd^{n+1}}{\dd u^{n+1}}
      \theparen{
        \frac{u e^{tu}}{e^{u} + 1}
      }
    }_{u=0}.
    $$
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    Prove that
    \begin{align*}
      &
      f(z_{2}) - f(z_{1})
      =
      C_{1} (z_{2} - z_{1}) f'(z_{2})
      +
      C_{2} (z_{2} - z_{1})^{2} f''(z_{1})
      -
      C_{3} (z_{2} - z_{1})^{3} f'''(z_{2})
      \\
      &
      -C_{4} (z_{2}-z_{1})^{4} f^{\textrm{iv}}(z_{1})
      +
      \cdots
      +
      (-)^{n} (z_{2} - z_{1})^{n+1}
      \int_{0}^{1}
      \thebrace{
        \frac{\dd^{n}}{\dd u^{n}}
        \theparen{
          e^{tu} \sech u
        }
      }_{u=0}
      f^{(n+1)}(z_{1} + t z_{2} - t z_{1})
      \dmeasure t;
    \end{align*}
    in the series plus signs and minus signs occur in pairs, and the last
    term before the integral is that involving
    $(z_{2}-z_{1})^{n}$, also $C_{n}$ is the
    coefficient of $z^{n}$ in the expansion of
    $\cot\theparen{\frac{\pi}{4} - \frac{z}{2}}$
    in ascending powers of $z$. \addexamplecitation{Trinity, 1899.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    If $x_{1}$ and $x_{2}$ are integers, and $\phi(z)$ is a function
    which is analytic and bounded for all values of $z$ such that
    $x_{1} \leq \Re(z) \leq x_{2}$, shew (by integrating
    $$
    \int \frac{\phi(z) \dmeasure z}{ e^{\pm 2 \pi i z} - 1 }
    $$
    round indented rectangles whose corners are
    $x_{1}$, $x_{2}$, $x_{2} \pm \infty i$, $x_{1} \pm \infty i$)
    that
    \begin{align*}
      &
      \half \phi(x_{1})
      + \phi(x_{1} + 1)
      + \phi(x_{1} + 2)
      + \cdots
      + \phi(x_{2} - 1)
      + \half \phi(x_{2})
      \hfill
      \\
      &
      \hfill =
      \int_{x_{1}}^{x_{2}} \phi(z) \dmeasure z
      +
      \frac{1}{i}
      \int_{0}^{\infty}
      \frac{ \phi(x_{2}+iy) - \phi(x_{1}+iy)
        - \phi(x_{2}-iy) + \phi(x_{1}-iy)}{ e^{2 \pi y} - 1 }
      \dmeasure y.
    \end{align*}
    Hence, by applying the theorem
    $$
    4n
    \int_{0}^{\infty}
    \frac{y^{2n-1}}{e^{2 \pi y} - 1}
    \dmeasure y
    =
    B_{n},
    $$
    %
    % 146
    %
    where $B_{1}, B_{2}, \ldots$ are \Bernoulli's numbers, shew that
    $$
    \phi(1) + \phi(2) + \cdots + \phi(n)
    =
    C
    + \half \phi(n)
    + \int^{n} \phi(z) \dmeasure z
    +
    \sum_{r=1}^{\infty}
    \frac{(-)^{r-1} B_{r}}{2r!} \phi^{(2r-1)}(n),
    $$
    (where $C$ is a constant not involving $n$), provided that the
    last series converges.

    (This important formula is due to TODO Plana, Mem. della R, Accad. di
    Torino, xxv. (1820), pp. 403-418; a proof by means of contour
    integration was published by Kronecker, Journal fur Math. cv. (1889),
    pp. 345-348. For a detailed history, see Lindelof, Le Calcul des
    Residus. Some applications of the formula are given in Chapter xii.)
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    Obtain the expansion
    $$
    u
    =
    \frac{x}{2}
    +
    \sum_{n=2}^{\infty}
    (-)^{n-1}
    \frac{1 \cdot 3 \cdots (2n-3)}{n!}
    \frac{x^{n}}{2^{n}}
    $$
    for one root of the equation
    $x = 2u + u^{2}$ and shew that it converges so
    long as $\absval{x} < 1$.
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    If $S^{(m)}_{2n+1}$ denote the sum of all combinations of the numbers
    $$
    1^{2}, 3^{2}, 5^{2}, \ldots (2n-1)^{2},
    $$
    taken $m$ at a time, shew that
    $$
    \frac{\cos z}{z}
    =
    \frac{1}{\sin z}
    +
    \sum_{n=0}^{\infty}
    \frac{(-)^{n+1}}{(2n+2)!}
    \thebrace{
      \frac{2^{2(n+1)}}{2n+3}
      -
      S^{(1)}_{2(n+1)}
      \frac{2^{2n}}{2n+1}
      +
      \cdots
      +
      (-)^{n}
      S^{(n)}_{2(n+1)}
      \frac{2^{2}}{3}
    }
    \sin^{2n+1} z.
    $$
    \addexamplecitation{Teixeira.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    If the function $f(z)$ is analytic in the interior of that one of
    the ovals whose equation is $\absval{\sin z} = C$
    (where $C \leq 1$), which includes the origin, shew that $f(z)$
    can, for all points $z$ within this oval, be
    expanded in the form
    \begin{align*}
      f(z)
      =&
      f(0)
      +
      \sum_{n=1}^{\infty}
      \frac{ f^{(2n)}(0)
        + S^{(1)}_{2n} f^{(2n-2)}(0)
        + \cdots
        S^{(n-1)}_{2n} f''(0)
      }{2n!}
      \sin^{2n} z
      \\
      &
      \quad +
      \sum_{n=0}^{\infty}
      \frac{
        f^{(2n+1)}(0)
        + S^{(1)}_{2n+1} f^{(2n-1)}(0)
        + \cdots
        + S^{(n)}_{2n+1} f'(0)
      }{(2n+1)!}
      \sin^{2n+1} z,
    \end{align*}
    where $S^{(m)}_{2n}$ is the sum of all combinations of the numbers
    $$
    2^{2}, 4^{2}, 6^{2}, \ldots, (2n-2)^{2},
    $$
    taken $m$ at a time, and $S^{(m)}_{2n+1}$ denotes the
    sum of all combinations of the numbers
    $$
    1^{2}, 3^{2}, 5^{2}, \ldots, (2n-1)^{2},
    $$
    taken $m$ at a time.
    \addexamplecitation{Teixeira.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    Shew that the two series
    $$
    2z
    + \frac{2 z^{3}}{3^{2}}
    + \frac{2 z^{5}}{5^{2}}
    + \cdots
    $$
    and
    $$
    \frac{2z}{1 - z^{2}}
    -
    \frac{2}{1 \cdot 3^{2}}
    \theparen{
      \frac{2z}{1 - z^{2}}
    }^{3}
    +
    \frac{2 \cdot 4}{3 \cdot 5^{2}}
    \theparen{
      \frac{2z}{1 - z^{2}}
    }^{5}
    -
    \cdots
    $$
    represent the same function in a certain region of the $z$ plane,
    and can be transformed into each other by \Burmann's theorem.

    \addexamplecitation{TODO Kapteyn, Nieuw Archief, (2), iii. (1897), p. 225.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    If a function $f(z)$ is periodic, of period $2 \pi$, and is
    analytic at all points in the infinite strip of the plane,
    included between the two branches of the curve
    $\absval{\sin z} = C$ (where $C > 1$),
    shew that at all points in the strip it can be expanded in
    an infinite series of the form
    \begin{align*}
      f(z)
      =&
      A_{0} + A_{1} \sin z + \cdots + A_{n} \sin^{n} z + \cdots
      \\
      &
      \hfill
      + \cos z
      ( B_{1} + B_{2} \sin z + \cdots + B_{n} \sin^{n-1} z + \cdots );
    \end{align*}
    and find the coefficients
    $A_{n}$ and $B_{n}$.
  \end{wandwmiscexample}
  %
  % 147
  %
  \begin{wandwmiscexample}
    If $\phi$ and $f$ are connected by the equation
    $$
    \phi(x) + \lambda f(x) = 0,
    $$
    of which one root is $a$,
    shew that
    $$
    TODO
    $$
    the general term being
    $
    (-)^{m}
    \frac{\lambda^{m}}{1! 2! \cdots m! (\phi')^{\half m(m+1)}}
    $
    multiplied by a determinant in which
    the elements of the first row are
    $\phi', (\phi^{2})', (\phi^{3})', \ldots, (\phi^{m-1})', (f^{m} F')$
    and each row is the differential coefficient of the preceding
    one with respect to $a$; and
    $F, f, F', \ldots$ denote
    $F(a), f(a), F'(a), \ldots$.

    (TODOWronski, Philosophie de la Technie, Section ii. p. 381. For proofs of
    the theorem see Cayley, Quarterly Journal, xil. (1873), Transon, Nouv.
    Ann. de Math. xill. (1874j, and C Lagrange, Brux. Mem. Couronnes, 4",
    xlvii. (1886), no. 2.)
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    If the function $W(a, b, x)$ be defined by the series
    $$
    W(a,b,x)
    =
    x
    + \frac{a-b}{2!} x^{2}
    + \frac{(a-b)(a-2b)}{3!} x^{3}
    + \cdots,
    $$
    which converges so long as
    $$
    \absval{x} < \frac{1}{\absval{b}},
    $$
    shew that
    $$
    \frac{\dd}{\dd x} W(a,b,x)
    =
    1
    +
    (a-b) W(a-b,b,x);
    $$
    and shew that if
    $$
    y = W(a,b,x),
    $$
    then
    $$
    x = W(b,a,y).
    $$

    Examples of this function are
    \begin{align*}
      W(1,0,x) =& e^{x} - 1, \\
      W(0,1,x) =& \log (1+x), \\
      W(a,1,x) =& \frac{(1+x)^{a} - 1}{a}.
    \end{align*}
    \addexamplecitation{\Jezek}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    Prove that
    $$
    \frac{1}{ \sum_{n=0}^{\infty} a_{n} x^{n}}
    =
    \frac{1}{a_{0}}
    +
    \sum_{1}^{\infty}
    \frac{ (-)^{n} x^{n} }{ n! a_{0}^{n+1} } G_{n},
    $$
    where
    $$
    TODO
    $$
    and obtain a similar expression for
    $$
    \thebrace{
      \sum_{n=0}^{\infty} a_{n} x^{n}
    }^{\half}.
    $$
    \addexamplecitation{TODO Mangeot, Ann. de VEcole norm. sup. (3), xiv.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    Shew that
    $$
    \frac{1}{ \sum_{r=0}^{n} a_{r} x^{r} }
    =
    -
    \sum_{r=0}^{\infty}
    \frac{1}{r+1}
    \frac{\partial S_{r+1}}{\partial a_{1}} x^{r},
    $$
    %
    % 148
    %
    where $S_{r}$ is the sum of the $r$-th powers of the reciprocals of
    the roots of the equation
    $$
    \sum_{r=0}^{n} a_{r} x^{r} = 0.
    $$
    \addexamplecitation{TODO Gambioli, Bologna Memorie, 1892.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    If $f_{n}(z)$ denote the $n$th derivate of $f(z)$, and if
    $f_{-n}(z)$ denote that one of the $n$th integrals of
    $f(z)$ which has an $n$-ple zero at $z=0$,
    shew that if the series
    $$
    \sum_{n=-\infty}^{\infty} f_{n}(z) g_{-n}(x)
    $$
    is convergent it represents a function of $z + x$;
    and if the domain of convergence includes the origin in the
    $x$-plane, the series is equal to
    $$
    \sum_{n=0}^{\infty} f_{-n}(z+x) g_{n}(0).
    $$
    Obtain Taylor's series from this result, by putting $g(z) = 1$.
    \addexamplecitation{Guichard.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    Shew that, if $x$ be not an integer,
    $$
    TODO
    $$
    as $\nu \rightarrow \infty$, provided that all terms for which
    $m = n$ are omitted from the summation.
    \addexamplecitation{Math. Trip. 1895.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    Sum the series
    $$
    \sum_{n=-q}^{p}
    \theparen{
      \frac{1}{(-)^{n} x-a-n}
      +
      \frac{1}{n}
    },
    $$
    where the value $n = 0$ is omitted, and $p,q$ are
    positive integers to be increased without
    limit.
    \addexamplecitation{Math. Trip. 1896.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    If
    $
    F(x)
    =
    e^{\int_{0}^{x} x \pi \cot (x \pi) \dmeasure x}
    $, shew that
    $$
    F(x)
    =
    e^{x}
    \frac{
      \prod\limits_{n=1}^{\infty} \thebrace{
        \theparen{1 - \frac{x}{n}}^{n}
        e^{x + \half \frac{x^{2}}{n}}
      }
    }{
    \prod\limits_{n=1}^{\infty} \thebrace{
      \theparen{1 + \frac{x}{n}}^{n}
      e^{-x + \half \frac{x^{2}}{n}}
      }
    },
    $$
    and that the function thus defined satisfies the relations
    $$
    F(-x) = \frac{1}{F(x)},
    \quad
    F(x) F(1-x) = 2 \sin x \pi.
    $$
    Further, if
    $$
    \psi(z)
    =
    z
    + \frac{z^{2}}{2^{2}}
    + \frac{z^{3}}{3^{2}}
    + \cdots
    =
    - \int_{0}^{z} \log (1-t) \frac{\dmeasure t}{t},
    $$
    shew that
    $$
    F(x)
    =
    e^{\half \pi i x^{2}
      -
      \frac{1}{2 \pi i}
      \psi( 1 - e^{-2 \pi i x} )
    }
    $$
    when
    $$
    \absval{ 1 - e^{-2 \pi i x} } < 1.
    $$
    \addexamplecitation{Trinity, 1898.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    Shew that
    \begin{align*}
      &
      \thebracket{
        1 + \theparen{\frac{k}{x}}^{n}
      }
      \thebracket{
        1 + \theparen{\frac{k}{2 \pi - x}}^{n}
      }
      \thebracket{
        1 + \theparen{\frac{k}{2 \pi + x}}^{n}
      }
      \thebracket{
        1 + \theparen{\frac{k}{4 \pi - x}}^{n}
      }
      \thebracket{
        1 + \theparen{\frac{k}{4 \pi + x}}^{n}
      } \cdots
      \\
      &
      \hfill =
      \frac{
        \prod_{g=1}^{\leq \half n}
        \sqrt{1 - 2 e^{-\alpha_{g}} \cos(x + \beta_{g}) + e^{-2\alpha_{g}}}
        \sqrt{1 - 2 e^{-\alpha_{g}} \cos(x - \beta_{g}) + e^{-2\alpha_{g}}}
      }{
        2^{\half n}
        (1 - \cos x)^{\half n}
        e^{-k \cos \pi / n}
      },
    \end{align*}
    where
    $$
    \alpha_{g} = k \sin \frac{2g-1}{n} \pi,
    \quad
    \beta_{g} = k \cos \frac{2g-1}{n} \pi,
    $$
    and
    $$
    0 < x < 2 \pi.
    $$
    \addexamplecitation{Mildner.}
  \end{wandwmiscexample}
  %
  % 149
  %
  \begin{wandwmiscexample}
    If $\absval{x} < 1$ and $a$ is not a positive integer, shew that
    $$
    \sum_{n=1}^{\infty}
    \frac{x^{n}}{n - a}
    =
    \frac{2 \pi i x^{a}}{1 - e^{2 a \pi i}}
    +
    \frac{x}{1 - e^{2 a \pi i}}
    \int_{C} \frac{t^{a-1} - x^{a-1}}{t - x} \dmeasure t,
    $$
    where $C$ is a contour in the plane enclosing the points $0,x$.
    \addexamplecitation{TODO Lerch, Casopis, xxi. (1892), pp. 65-68.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    If $\phi_{1}(z), \phi_{2}(z), \ldots$ are any polynomials in $z$,
    and if $F(z)$ be any integrable function, and if
    $\psi_{1}(z), \psi_{2}(z), \ldots$ be polynomials
    defined by the equations
    \begin{align*}
      &
      \int_{a}^{b}
      F(x) \frac{ \phi_{1}(z) - \phi_{TODO}(x) }{z - x}
      \dmeasure x
      =
      \psi_{1}(z),
      \\
      &
      \int_{a}^{b}
      F(x) \phi_{1}(x)
      \frac{ \phi_{2}(z) - \phi_{2}(x) }{z - x}
      \dmeasure x
      =
      \psi_{2}(z),
      \\
      &
      \int_{a}^{b}
      F(x)
      \phi_{1}(x) \phi_{2}(x) \cdots \phi_{m-1}(x)
      \frac{\phi_{m}(z) - \phi_{m}(x)}{z-x}
      \dmeasure x
      =
      \psi_{m}(z),
    \end{align*}
    Shew that
    \begin{align*}
      &
      \hfill
      \int_{a}^{b}
      \frac{F(x) \dmeasure x}{z - x}
      =
      \frac{\psi_{1}(z)}{\phi_{1}(z)}
      +
      \frac{\psi_{2}(z)}{\phi_{1}(z) \phi_{2}(z)}
      +
      \frac{\psi_{3}(z)}{\phi_{1}(z) \phi_{2}(z) \phi_{3}(z)}
      +
      \cdots
      \hfill
      \\
      &
      +
      \frac{\psi_{m}(z)}{\phi_{1}(z) \phi_{2}(z) \cdots \phi_{m}(z)}
      +
      \frac{1}{\phi_{1}(z) \phi_{2}(z) \cdots \phi_{m}(z)}
      \int_{a}^{b}
      F(x) \phi_{1}(x) \phi_{2}(x) \cdots \phi_{m}(x)
      \frac{ \dmeasure x }{z - x}.
    \end{align*}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    A system of functions $p_{0}(z), p_{1}(z), p_{2}(z), \ldots$
    is defined by the equations
    $$
    p_{0}(z) = 1,
    \quad
    p_{n+1}(z) = (z^{2} + a_{n} z + b_{n}) p_{n}(z),
    $$
    where $a_{n}$ and $b_{n}$ are given functions of $n$, which tend
    respectively to the limits $0$ and $-1$ as $n \rightarrow \infty$.

    Shew that the region of convergence of a series of the form
    $\sum e_{n} p_{n}(z)$ where
    $e_{1}, e_{2}, \ldots$ are independent of $z$, is a Cassini's oval
    with the foci $+1, -1$.

    Shew that every function $f(z)$, which is analytic on and inside the
    oval, can, for points inside the oval, be expanded in a series
    $$
    f(z) = \sum (c_{n} + z c'_{n}) p_{n}(z) % TODO: verify
    $$
    where
    $$
    c_{n}
    =
    \frac{1}{2 \pi i} \int (a_{n}+z) q_{n}(z) f(x) \dmeasure z,
    \quad
    c'_{n}
    =
    \frac{1}{2 \pi i} \int q_{n}(z) f(z) \dmeasure z,
    $$
    the integrals being taken round the boundary of the region, and the
    functions $q_{n}(z)$ being defined by the equations
    $$
    q_{0} = \frac{1}{z^{2} + a_{0} z + b_{0}},
    \quad
    q_{n+1}(z)
    =
    \frac{1}{z^{2} + a_{n+1} z + b_{n+1}}
    q_{n}(z).
    $$
    \addexamplecitation{TODO Pincherle, Rend, dei Lincei, (4), v. (1889), p. 8.}
  \end{wandwmiscexample}
  \begin{wandwmiscexample}
    Let $C$ be a contour enclosing the point $a$, and let $\phi(z)$ and
    $f(z)$ be analytic when $z$ is on or inside $C$. Let $\absval{t}$ be so small that
    $$
    \absval{ t \phi(z) } < \absval{ z - a }
    $$
    when $z$ is on the periphery of $C$. By expanding
    $$
    \frac{1}{2 \pi i}
    \int_{C}
    f(z)
    \frac{1 - t \phi'(z)}{z - a - t \phi(z)}
    \dmeasure z
    $$
    in ascending powers of $t$, shew that it is equal to
    $$
    f(a)
    +
    \sum_{n=1}^{\infty}
    \frac{ t^{n} }{n!}
    \frac{ \dd^{n-1} }{ \dd a^{n-1} }
    \thebracket{ f'(a) \thebrace{\phi(a)}^{n}
    }.
    $$
    Hence, by using \hardsectionref{6}{3}, \hardsubsectionref{6}{3}{1}, obtain Lagrange's theorem.
  \end{wandwmiscexample}
\end{wandwmiscexamples}