\documentclass[a4paper]{article}

\def\npart{IB}

\def\ntitle{Complex Methods}
\def\nlecturer{R.\ E.\ Hunt}

\def\nterm{Lent}
\def\nyear{2018}

\input{header}

\usepackage[normalem]{ulem}%strikethrough

\begin{document}

\input{titlepage}

\tableofcontents

\section{Analytic Functions}

\subsection{The Complex Plane and the Riemann Sphere}

Any \(z \in \C\) can be written in the form \(x + iy\) where \(x = \Re z\) and \(y = \Im z\), \(x, y \in \R\) or \(re^{i\theta}\) where the \emph{modulus} \(|z| = r = \sqrt{x^2 + y^2}\) and the argument \(\theta = \arg z\) satisfies \(y = x\tan \theta\). The argument is defined only up to multiples of \(2\pi\). The \emph{principal value of the argument} is the value of \(\theta\) in the range \((-\pi, \pi]\). Note that the formula \(\tan^{-1} \frac{y}{x}\) gives the correct value for the principal value of \(\theta\) only if \(x > 0\). If \(x \leq 0\) then it might be out by \(\pm \pi\) (consider \(1 + i\) and \(1 - i\)).

An \emph{open set} \(D\) is a subset of \(\C\) which does not include its boundary\footnote{Hint: this is an applied course.}.

A \emph{neighbourhood} of a point \(z\) is an open set containing \(z\).

A \emph{domain} is an open set that is connected (i.e.\ cannot be split into wto disjoint open subsets). A \emph{simply-connected domain} is one with no holes (i.e.\ any curve lying in the domain can be shrunk continuouly to a point without leaving the domain).

Note that a hole could be caused merely be a particular function under consideration being undefined at a single point, e.g.\ \(\frac{1}{z}\).

The \emph{extended complex plane} is \(\C^* = \C \cup \{\infty\}\). We can reach the ``point at infinity'' be going off in any direction in the plane, and all are equivalent. Conceptually we may use the \emph{Riemann sphere}, which is a sphere resting on the complex plane with its ``south pole'' \(S\) at \(z = 0\). For any point \(z \in \C\), drawing a line through the ``north pole'' \(N\) of the sphere to \(z\) and noting where this line intersects the sphere specifies an equivalent point \(P\) on the sphere. Then \(\infty\) is equivalent to the ``north pole'' itself.

To investigate properties of \(\infty\) we use the substitution \(\zeta = \frac{1}{z}\). A function \(f(z)\) is said to have a particular property at infinity if \(f(\frac{1}{z})\) has the same property at \(0\).

\subsection{Complex Differentiation}

Recall the definition of differentiation for a real function \(f(x)\):
\[
  f'(x) = \lim_{\delta x \to 0} \frac{f(x + \delta x) - f(x)}{\delta x}.
\]
It is implicit that the limit must be the same whichever direction we approach from. Consider \(|x|\) at \(x = 0\) for example: if we approach from the right (\(\delta x \to 0^+\)) then the limit is \(+1\), whereas from the left (\(\delta x \to 0^-\)) it is \(-1\). Because these limits are different we say that \(|x|\) is not differentiable at \(x = 0\).

Now extend the defintion to complex function \(f(z)\). \(f\) is \emph{differentiable} at \(z\) if
\[
  f'(z) = \lim_{\delta z \to 0} \frac{f(z + \delta z) - f(z)}{\delta z}
\]
exists (and is therefore independent of direction of approach --- but now there is an infinity of possible directions).

We say that \(f\) is \emph{analytic} at a point \(z\) if there exists a neighbourhood of \(z\) throughtout which \(f'\) exists. The term \emph{regular} and \emph{holomorphic} are also used. A function which is analytic throughtout \(\C\) is called \emph{entire}.

A \emph{singularity} of \(f\) is a point at which it is not analytic, or not even defined.

The property of analyticity is in fact a surprisingly strong one. For example, two consequenses include
\begin{enumerate}
\item if a function is analytic then it is differentiable infinitely many times (c.f.\ the existence of real functions which can be differentiated \(N\) times but no more, for any given \(N\));
\item a bounded entire function is constant (c.f.\ \(\tanh x\) for \(x \in \R\), which is bounded but not constant).
\end{enumerate}

\subsection{Cauchy-Riemann Equations}

Separate \(f\) and \(z\) into real and imaginary parts:
\[
  f(z) = u(x, y) + iv(x, y)
\]
where \(z = x + iy\) and \(u, v\) are real functions. Suppose that \(f\) is differentiable at \(z\). We may take \(\delta z\) in any direction. First take it to be real, \(\delta x = \delta x\). Then
\begin{align*}
  f'(z) &= \lim_{\delta x \to 0} \frac{f(z + \delta x) - f(z)}{\delta x} \\
        &= \lim_{\delta x \to 0} \frac{u(x + \delta x, y) + iv(x + \delta x, y) - u(x, y) - iv(x, y)}{\delta x} \\
        &= \frac{\partial u}{\partial x} + i \frac{\partial v}{\partial x}
\end{align*}
Now take \(\delta z\) to be pure imaginary, \(\delta z = i\delta y\). Then
\begin{align*}
  f'(z) &= \lim_{\delta y \to 0} \frac{f(z + i\delta y) - f(z)}{i\delta y} \\
        &= \lim_{\delta y \to 0} \frac{u(x, y + \delta y) + iv(x, y + \delta y) - u(x, y) - iv(x, y)}{i\delta y} \\
        &= -i\frac{\p u}{\p y} + \frac{\p v}{\p y}.
\end{align*}
The two values for \(f'(z)\) are the same since \(f\) is differentiable. Thus
\begin{align*}
  \frac{\p u}{\p x} &= \frac{\p v}{\p u} \\
  \frac{\p u}{\p y} &= -\frac{\p v}{\p x}
\end{align*}
This is know as the \emph{Cauchy-Riemann equations}. The converse (that a function satisfying the Cauchy-Riemann equaitons is differentiable) is also true as long as we impose additional requirements, for example that the partial derivative \(u_x, u_y, v_x, v_y\) are continuous functions of \(x\) and \(y\), in the sense described in IB Analysis II.

\begin{eg}\leavevmode
  \begin{enumerate}
  \item \(f(z) = z\) is entire. Here \(u = x, v = y\) and the Cauchy-Riemann equations are satisfied.
  \item \(f(z) = e^z = e^x(\cos y + i \sin y)\) is entire since
    \begin{align*}
      \frac{\p u}{\p x} &= e^x \cos y = \frac{\p v}{\p y} \\
      \frac{\p u}{\p y} &= -e^x \sin y = -\frac{\p v}{\p x}
    \end{align*}
    The derivative is
    \[
      f'(z) = e^x \cos y + ie^x \sin y = e^x
    \]
    as expected.
  \item \(f(z) = z^n\), where \(n\) is a positive integer, is entire. Write \(z = r(\cos \theta + i \sin \theta)\) we obtain \(u = r^n \cos n\theta, v = r^n \sin n\theta\). We can check the Cauchy-Riemann equaitions using \(r = \sqrt{x^2 + y^2}\) and \(\tan \theta = \frac{y}{x}\). The derivative is \(nz^{n - 1}\) as we would expect!
  \item Any rational function, i.e.\ \(f(z) = \frac{P(z)}{Q(z)}\) where \(P\) and \(Q\) are polynomials, is analytic except at points where \(Q(z) = 0\). For instance \(f(z) = \frac{z}{z^2 + 1}\) is analytic except at \(\pm i\).
  \item Many standard real functions can be extended naturally to complex functions and obey the usual rule for their derivatives. For example \(f(z) = \sin z = \frac{e^{iz} - e^{-iz}}{2i}\) has derivative \(f'(z) = \cos z\). We can also write
    \begin{align*}
      \sin z &= \sin (x + iy) \\
             &= \sin x \cos iy + \cos x \sin iy \\
             &= \sin x \cosh y + i \cos x \sinh y
    \end{align*}
    This applies to other trigonometric functions.

    \(\log z = \log |z| + i \arg z\) has derivative \(\frac{1}{z}\).

    The product, quotient and chain rules hold in exactly the same way as for real function.
  \end{enumerate}
\end{eg}

\begin{eg}\leavevmode
  \begin{enumerate}
  \item \(f(z) = \Re z\) has \(u = x, v = 0\) but \(\frac{\p u}{\p x} = 1 \neq 0 = \frac{\p u}{\p y}\) so \(\Re z\) is nowhere analytic.
  \item \(f(z) = |z|\) has \(u = \sqrt{x^2 + y^2}, v = 0\) and is nowhere analytic.
  \item \(f(z) = \conj z = x - iy\) is nowhere analytic.
  \item \(f(z) = |z|^2 = x^2 + y^2\). The Cauchy-Riemann equaitions are satisfied only at the origin, so \(f\) is only differentiable at \(z = 0\). However it is not analytic there because there is no neighbourhood of \(0\) throughout which \(f\) is differentiable.
  \end{enumerate}
\end{eg}

\subsection{Analytic Continuation*}

If we are given the values of an analytic function in some restricted region --- which could be rather small, such as a short curve somewhere in the complex plane --- then there is a unique extension of the function to the rest of \(\C\) that is still analytic. (No proof given here.) The extension might have some singularities, and might be multivalued.

This fact can be useful in extending the domain of definition of a function. We shall see an example in Section 5.2.

\subsection{Harmonic Functions}

If \(f(z) = u + iv\) is analytic, then
\[
  \frac{\p^2 u}{\p x^2} = \frac{\p}{\p x} \frac{\p u}{\p x} = \frac{\p}{\p x} \frac{\p v}{\p y} = -\frac{\p^2 u}{\p y^2}
\]
so \(u\) satisfies Laplace's equaiotn in two dimension, \(\gradient^2 u = 0\). Similarly so does \(v\). A function satisfying Laplace's equation in an open set is said to be \emph{harmonic} there.

Functions \(u\) and \(v\) satisfying the Cauchy-Riemann equations are called \emph{harmonic conjugates}. If we know one then we can find the other, up to a constant. For example, consider \(u(x, y) = x^2 - y^2\), which is easily verified to be harmonic. Its harmonic conjugate \(v\) satisfies
\begin{align*}
  \frac{\p v}{\p y} &= 2x \implies v = 2xy + g(x) \\
  \frac{\p v}{\p x} &= 2y \implies 2y + g'(x) = 2y
\end{align*}
so \(g = \alpha\) for some constant \(\alpha\). The corresponding analytic function whose real part is \(u\) is therefore
\[
  f(z) = x^2 - y^2 + 2ixy + i\alpha = (x + iy)^2 + i\alpha = z^2 + i\alpha.
\]

If the domain is not simply-connected then this method might give a solution that is multi-valued. For example, if \(u = \frac{1}{2} \log(x^2 + y^2)\), which is harmonic in the domain \(|z| > 0\), the corresponding \(f(z)\) is \(\log z\), which is multi-valued (see \Cref{sec:multi-valued functions}).

We end this section by a geometric obseravation:

\begin{proposition}
  Contours of harmonic conjugate functions are perpendicular to each other:
\end{proposition}

\begin{proof}
  \(\gradient u\) is perpendicular to contours of \(u\) (i.e.\ curves \(u = \) constant), using a result from IA Vector Calculus. Similarly \(\gradient v\) is perpendicular to contour of \(v\). But
  \begin{align*}
    \gradient u \cdot \gradient v &= \frac{\p u}{\p x}\frac{\p v}{\p x} + \frac{\p u}{\p y}\frac{\p v}{\p y} \\
                                  &= \frac{\p u}{\p x}\left(-\frac{\p u}{\p y}\right) + \frac{\p u}{\p y}\frac{\p u}{\p x} \\
                                  &= 0
  \end{align*}
  and the result follows.
\end{proof}

\subsection{Multi-valued functions}\label{sec:multi-valued functions}

For \(z = re^{i\theta}\) we define \(\log z = \log r + i\theta\). There are thus infinitely many values of \(\log z\), for \(\theta\) may take an infinity of values. For example,
\[
  \log i = \frac{\pi i}{2} \text{ or } \frac{5\pi i}{2} \text{ or } -\frac{3\pi i}{2} \text{ or } \dots
\]
depending on which choice of \(\theta\) we make.

Missed a lecture

Note that a branch cut alone does not specify a branch (compare (b) above, with the principal branch which is a different branch even though it has the same branch cut) nor is a single value of the fucntion sufficient by itself (compare (a) and (c) above).

\subsubsection{Riemann Surfaces*}

Riemann imagined different branches as separate copies of \(\C\), stacked on top of each other but each one joined to the next at the branch cut. This structure is a \emph{Riemann surface}.

\subsubsection{Multiple Branch Cuts}

When there is more than one branch point we may need more than one branch cut. FOr \(f(z) = (z(z - 1))^{1/3}\) there are branch points at \(0\) and \(1\), so we need two branch cuts. A possibility is shown below. Then no curve can wrap round either \(0\) or \(1\).

For any \(z\) write \(z = re^{i\theta}\) and \(z - 1 = r_1e^{i\theta_1}\) with \(\theta \in (-\pi, \pi], \theta_1 \in [0, 2\pi)\) and define
\[
  f(z) = \sqrt[3]{rr_1} e^{i(\theta + \theta_1)/3}
\]
This is continuous so long as we don't cross either cut. Sometimes we need fewer branch cuts than we might think. See the worked example.

\subsection{Möbius Maps}

The Möbius map \(z \mapsto w = \frac{az + b}{cz + d}\) where \(ad - bc \neq 0\), is analytic except at \(z = -\frac{d}{c}\). It is useful to consider it as a map \(\C^* \to \C^*\) with
\begin{align*}
  -\frac{d}{c} &\mapsto \infty \\
  \infty &\mapsto \frac{a}{c}
\end{align*}
It is then bijective, the inverse being \(w \mapsto z = \frac{-dw + b}{cw - a}\), another Möbius map.

\begin{definition}[Circline]
  A \emph{circline} is either a circle or a line.
\end{definition}

Möbius maps take circlines to circlines.

\begin{proof}
  Any circline can be expressed as a circle of Apollonius
  \[
    |z - z_1| = \lambda|z - z_2|
  \]
  where \(z_1, z_2  \in \C, \lambda \in \R_{> 0}\). This is a result from IA Vectors and Matrices. The case \(\lambda = 1\) corresponds to a line and \(\lambda \neq 1\) to a circle. Apply a Möbius map,
  \begin{align*}
    \left| \frac{-dw + b}{cw - a} - z_1 \right| &= \lambda \left| \frac{-dw + b}{cw - a} - z_2 \right| \\
    |(cz_1 + d) - (az_1 + b)| &= \lambda|(cz_1 + d)w - (az_2 + b)| \\
    |w - w_1| &= \lambda \left| \frac{cz_2 + d}{cz_1 + d} \right| |w - w_2|
  \end{align*}
  where \(w_1 = \frac{az_1 + b}{cz_1 + d}, w_2 = \frac{az_2 + b}{cz_2 + d}\), which is another circle of Apollonius.

  The proof failsif either \(cz_1 + d = 0\) or \(cz_2 + d = 0\), but in either of these cases the equation trivially defines a circle.
\end{proof}

Geometrically it is clear that choosing three distinct points in \(\C^*\) uniquely specifies a circline (if one of the points is \(\infty\) then we have specified the straight line through the other two points).

Given \(\alpha, \beta, \eta, \alpha', \beta', \gamma' \in \C^*\) we can find a Möbius map which sends \(\alpha \mapsto \alpha'\) etc.

\begin{proof}
  The map
  \[
    f_1(z) = \frac{\beta - \gamma}{\beta - \alpha}\frac{z - \alpha}{z - \gamma}
  \]
  sends \(\alpha \mapsto 0, \beta \mapsto 1, \gamma \mapsto \infty\). Let \(f_2(z)\) be defined analogously for the primed version. Then \(f_2^{-1} \compose f_1\) is the required mapping. It is also a Möbius map as they are closed under composition.
\end{proof}

Putting all these results together, we conclude that we can find a Möbius map taking any given circline to any other.

\subsection{Conformal Maps}

\begin{definition}[Conformal map]
  A \emph{conformal map} \(f: U \to V\), where \(U, V\) are \emph{open} subsets of \(\C\), is one which is analytic with non-zero derivative in \(U\).
\end{definition}

Though not part of the definition, it is usual (and helpful) to require that \(f\) be one-to-one from \(U\) to \(V\).

An alternative definition is that a conformal map is one that preserves the angle (in both magnitude and orientation) between intersecting curves. We shall show that our definition implies this. The converse is also true (proof omittted) so the two definitions are equivalent.

Suppose that \(z_1(t)\) is a curve in \(\C\) parameterised by \(t \in \R\), which passes through a point \(z_0\) when \(t = t_1\). Suppose further that its tangent there, \(z_1'(t_1)\), has a wel-defined direction. Then \(z_1'(t) \neq 0\) and the curve makes an angle \(\phi = \arg z_1'(t_1)\) to the \(x\)-axis at \(z_0\). Consider the image of the curve, \(Z_1(t) = f(z_1(t))\). Its tangent direction at \(t = t_1\) is
\[
  Z_1'(t_1) = z_1'(t_1)f'(z_1(t_1)) = z_1'(t_1)f'(z_0)
\]
and therefore makes an angle with the \(x\)-axis of
\[
  \arg Z_1'(t_1) = \phi + \arg f'(z_0).
\]
Note that \(\arg f'(z_0)\) exists since \(f\) is conformal so \(f'(z_0) = 0\). In other words, the trangent direction is rotated by \(\arg f'(z_0)\).

Now if \(z_2(t)\) is another curve passing throught \(z_0\) then its tangent direction will also be rotated by \(\arg f'(z_0)\). The result follows.

Sometimes we do not know what \(V\), the image set of \(f\) acting on  \(U\), is in advance. Often the easiest way to find it is first to find the iamge of the boundary \(\p U\), which will form the boundary \(\p V\) of \(V\); but, since this does not reveal upon which side of \(\p V\) \(V\) lies, to then find the image of a single point of our choice within \(U\), which will lie within \(V\).

\begin{eg}\leavevmode
  \begin{enumerate}
  \item The map \(z \mapsto az + b\) where \(a, b \in \C, a \neq 0\), rotates by \(\arg a\), enlarges by \(|a|\) and translates by \(n\) and is conformal everywhere.
  \item \(f(z) = z^2\) is a conformal map from
    \[
      U = \{z: 0 < |z| < 1, 0 < \arg z < \frac{\pi}{2}\}
    \]
    to
    \[
      V = \{w: 0 < |w| < 1, 0 < \arg w < \pi\}.
    \]
    Note that the right angle between the two boundary curves at \(z = 1\) is preserved because \(f\) is conformal there. Similarly at \(z = 1\). But the right angle at \(z = 0\) is not preserved because \(f\) is not conformal there (\(f'(0) = 0\)). Fortunately this does not matter since \(U\) is an open set so does not include \(0\).
  \item How would we map the lef-hand half-plane
    \[
      U = \{z: \Re z < 0\}
    \]
    to a wedge
    \[
      V = \{w: -\frac{\pi}{4} < \arg w < \frac{\pi}{4}\}?
    \]
    We need to halve the angle, so by using \(z^{1/2}\), for which we need to choose a branch. The branch cut must not lie in \(U\) (since \(z^{1/2}\) is not analytic on the branch cut) so choose a cut along the negative imaginary axis: \(re^{i\theta} \mapsto \sqrt{r} e^{i\theta/2}\) where \(\theta \in (-\pi/2, 3\pi/2]\). Having defined this branch, we now apply \(z^{1/2}\) to \(U\) to produce the wedge \(\{z': \pi/4 < \arg z' < 3\pi/4\}\); so we just need to rotate through \(-\pi/2\). The final map is \(f(z) = -iz^{1/2}\).
  \item \(e^z\) takes rectangles conformally to sectors of annuli. With an appropriate choice of branch, \(\log z\) does the reverse.
  \item Möbius maps (which are conformal everywhere except at the point that it is sent to \(\infty\) are very useful in taking circles, or parts of them, to straight lines, or vice versa. Consider \(f(z) = \frac{z - 1}{z + 1}\) acting on the unit disc \(U = \{z: |z| < 1\}\). The boundary of \(U\) is a cricle; the three points \(-1, i\) and \(1\) lie on the circle and are mapped to \(\infty, i\) and \(0\) respectively. Therefore the image of \(\p U\) is the imaginary axis; since \(f(0) = -1\) we see that the image of \(U\) is the left-hand half-plane.

    The inverse map, which is \(z \mapsto \frac{1 + z}{1 - z}\), maps \(V\) to \(U\) conformally.

    Alternatively, \(w = \frac{z - 1}{z + 1}\) if and only if \(z = \frac{1 + w}{1 - w}\). So \(|z| < 1\) if and only if \(|w + 1| < |w - 1|\), i.e.\ \(w\) is close to \(-1\) than ti is to \(1\), which describges precisely the left-hand half-plane.

    In fact this particular map can usefully be depoloyed more generally on \emph{quadrants} of the unit disc or of the complex plane.
  \item \(f(z) = \frac{1}{z}\) is a simple Möbius map useful for acting on vertical or horizontal lines, which maps to circles passing through the origin with centres one of the axes, or for mapping sectors within the unit circle to sectors outside the circle, or vice versa.
  \end{enumerate}
\end{eg}

In practice, complicated conformal maps are usually built up from individual building blocks, each a simple conformal map; the required map is the composition of these (note that the composition of conformal maps is conformal, by the chain rule). Seee the worked examples.

\sout{Hungry. Skipped a lecture.}

Hungry. Skipped two lectures.

\section{Laurent series and Cauchy's theorem}

\subsection{Taylor and Laurent series}

\subsection{Zeroes}

\begin{eg}\leavevmode
  \begin{enumerate}
  \item \(z^3 + iz^2 + z + i = (z - i)(z + i)^2\) has a simple zero at \(z = i\) and a zero of order two at \(z = -i\).
  \item \(\sinh x\) has zero where
    \[
      \frac{e^z - e^{-z}}{2} = 0
    \]
    so \(z \in \pi i\Z\). The zeros are all simple since \(\cosh n\pi i = \cos n\pi \neq = 0\).
  \item Since \(\sinh z\) has a simple zero at \(z = \pi i\), \(\sinh^3 z\) has a zero of order \(3\) there. If needed, we can find its Taylor series about \(\pi i\) by write \(\zeta = z - \pi i\).
    \begin{align*}
      \sinh^3 z
      &= \sinh^3 (\zeta + \pi i) \\
      &= (-\sinh \zeta)^3 \\
      &= -(\zeta + \frac{1}{3!}\zeta^3 + \dots) ^3 \\
      &= -\zeta^3 - \frac{1}{2}\zeta^5 - \dots \\
      &= -(z - \pi i)^3 - \frac{1}{2}(a - \pi i)^5 + \dots
    \end{align*}
  \end{enumerate}
\end{eg}

\subsection{Laurent Series}

If \(f\) has a singularity at \(z_0\) then we cannot expect it to have a Taylor series there. Instead, if \(f\) is analytic in an annulus \(R_1 < |z - z_0| < R_2\) then it has a \emph{Laurent series} about \(z_0\)
\[
  f(z) = \sum_{n = \infty}^\infty a_n(z - z_0)^n
\]
convergent within the annulus. See the proof in the separate sheet.

It can be shown that the Laurent series for \(f\) about a particular \(z_0\) is unique within any given radius. Note that Taylor series are just a special case of Laurent series, with \(a_n = 0\) for all \(n < 0\).

\begin{eg}\leavevmode
  \begin{enumerate}
  \item \(\frac{e^z}{z^3}\) has a Laurent series about \(z_0 = 0\) given by
    \[
      \frac{e^z}{z^3} = \sum_{m = 0}^\infty \frac{z^{m - 3}}{m!} = \sum_{n = -3}^\infty \frac{z^n}{(n + 3)!}
    \]
    so \(a_n = \frac{1}{(n + 3)!}\) for \(n \geq -3\).
  \item \(e^{1/z}\) about \(z_0 = 0\) has
    \[
      e^{1/z} = 1 + \frac{1}{z} + \frac{1}{2!z^2} + \frac{1}{3!z^3} + \dots
    \]
    so \(a_n = \frac{1}{(-n)!}\) for \(n \leq 0\).
  \item If \(f(z) = \frac{1}{z - a}\) where \(a \in \C\) then \(f\) is analytic in \(|z| < |a|\) so it has a Taylor series about \(z = 0\) given by
      \[
        \frac{1}{z - a} = -\frac{1}{a} \left( 1 - \frac{z}{a} \right)^{-1} = -\sum_{n = 0}^\infty a^{-n - 1}z^n.
      \]
      The binomial expansion is valid since \(\left| \frac{z}{a} \right| < 1\). In \(|z| > |a|\) it has a Laurent series (in the annulus \(|a| < |z| < \infty\)) given by
      \[
        \frac{1}{z - a} = \frac{1}{z} \left( 1 - \frac{a}{z} \right)^{-1} = \sum_{m = 0}^\infty \frac{a^m}{z^{m + 1}}.
      \]
  \end{enumerate}
\end{eg}

\end{document}


%#################### 7.1 ####################
\section{Simple pie chart}
	\begin{table}[h!]
	\begin{tabular}{c | c}
	\begin{minipage}[m]{0.4\textwidth}
	\enum{
	\begin{tikzpicture}[thick,scale=0.6, every node/.style={transform shape}]
\pie{22.97/Los Angeles Lakers,
22.97/Boston,
8.11/Golden State,
8.11/Chicago,
6.76/San ,
31.07/Other Teams}
\end{tikzpicture}}{7.1}
	\end{minipage}
	&
	\begin{minipage}[m]{0.55\textwidth}
	\renewcommand\textminus{\mbox{-}}%<<<<<<<<<<<
	\begin{lstlisting}[numberstyle=\zebra{red!15}{yellow!15},numbers=left,basicstyle=\footnotesize]{tex}
\documentclass[border=0.2cm]{standalone}
\usepackage{pgf-pie}

\begin{document}
\begin{tikzpicture}
\pie{22.97/Los Angeles Lakers,
22.97/Boston Celtics,
8.11/Golden State Warriors,
8.11/Chicago Bulls,
6.76/San Antonio Spurs,
31.07/Other Teams}
\end{tikzpicture}
\end{document}
	\end{lstlisting}
	\end{minipage}
	\end{tabular}
	\end{table}

	%#################### 7.2 ####################
\section{Circled arrows with text}
\begin{table}[h!]
\begin{tabular}{c | c}
\begin{minipage}[m]{0.4\textwidth}
\enum{
\begin{center}
\begin{tikzpicture}[->,scale=.9]
\node (i) at (90:1cm)  {$T$};
\node (j) at (-30:1cm) {$D$};
\node (k) at (210:1cm) {$R$};
\draw (70:1cm)  arc (70:-10:1cm) node[midway, right] {{\footnotesize 1}};
\draw (-50:1cm) arc (-50:-130:1cm) node[midway, below] {{\footnotesize 1}};
\draw (190:1cm) arc (190:110:1cm) node[midway, left] {{\footnotesize -1}};
\end{tikzpicture}\end{center} }{7.2}
\end{minipage}
&
\begin{minipage}[m]{0.55\textwidth}
\renewcommand\textminus{\mbox{-}}%<<<<<<<<<<<
\begin{lstlisting}[numberstyle=\zebra{red!15}{yellow!15},numbers=left,basicstyle=\scriptsize]{tex}
\documentclass{article}
\usepackage{tikz}

\begin{document}
\begin{tikzpicture}[->,scale=.7]
\node (i) at (90:1cm)  {$T$};
\node (j) at (-30:1cm) {$D$};
\node (k) at (210:1cm) {$R$};
\draw (70:1cm)  arc (70:-10:1cm) node[midway, right] {{\footnotesize 1}};
\draw (-50:1cm) arc (-50:-130:1cm) node[midway, below] {{\footnotesize 1}};
\draw (190:1cm) arc (190:110:1cm) node[midway, left] {{\footnotesize -1}};
\end{tikzpicture}
\end{document}
\end{lstlisting}
\end{minipage}
\end{tabular}
\end{table}
\newpage
%#################### 7.3 ####################
\section{Diamond with text}
	\begin{table}[h!]
	\begin{tabular}{c | c}
	\begin{minipage}[m]{0.4\textwidth}
	\enum{   \includegraphics[width=1\linewidth]{7.3.pdf}   }{7.3}
	\end{minipage}
	&
	\begin{minipage}[m]{0.55\textwidth}
	\renewcommand\textminus{\mbox{-}}%<<<<<<<<<<<
	\begin{lstlisting}[numberstyle=\zebra{red!15}{yellow!15},numbers=left,basicstyle=\scriptsize]{tex}
\documentclass[a4paper,14pt]{extreport}
\usepackage[left=1.5cm,right=1.5cm,top=1.5cm,bottom=2cm,bindingoffset=0cm]{geometry}
\usepackage{amsmath}
\usepackage{tikz}
\usetikzlibrary{shapes.geometric}

\begin{document}
\begin{tikzpicture}
\node[diamond,font=\small,
line width=0.4mm,scale=0.7,
    draw = cyan, minimum width = 7.5cm, %text = red,
    minimum height = 9cm] (d) at (0,0) { };
      \node [above=0.5cm] (a) at (d.90) {$w = f(x,y)$};
      \node [above=0.5cm,right=0.1cm] (b) at (d.45) {$\dfrac{\partial w}{\partial y}$};
      \node [above=0.5cm,left=0.1cm] (c) at (d.135) {$\dfrac{\partial w}{\partial x}$};
      \node [left=0.1cm] (dd) at (d.180) {$x$};
      \node [right=0.1cm] (e) at (d.0) {$y$};
      \node [below=0.1cm] (f) at (d.270) {$t$};
      \node [below=0.9cm,right=-0.3cm] (g) at (d.-30) {$\dfrac{\partial y}{\partial t}$};
      \node [below=0.5cm,left=0.1cm] (h) at (d.220) {$\dfrac{\partial x}{\partial t}$};
      \node at (d.90) [cyan,circle,fill,inner sep=3pt]{};
      \node at (d.180) [cyan,circle,fill,inner sep=3pt]{};
      \node at (d.0) [cyan,circle,fill,inner sep=3pt]{};
      \node at (d.270) [cyan,circle,fill,inner sep=3pt]{};
\end{tikzpicture}
\end{document}
	\end{lstlisting}
	\end{minipage}
	\end{tabular}
	\end{table}


\lab{Policy Function Iteration}{Policy Function Iteration}
\objective{This section teaches how to improve dynamic programming convergence using policy function iteration.}

Now that we have covered how to solve simple dynamic programming problems by value function iteration, we consider
the convergence of the algorithm.  We demonstrate two additional methods known as policy function iteration, or Howard's
Improvement, and modified policy function iteration.

\section*{Policy Function Iteration}
For infinite horizon dynamic programming problems, it can be shown that value function iteration converges at the
rate $\beta$, where $\beta$ is the discount factor.  In practice, $\beta$ is usually close to one, which means this
algorithm often converges slowly.

In order to examine the value function iteration algorithm, it is helpful to see which functions take the most runtime.
We can perform this analysis using the profiling tools in IPython.
\begin{problem}
\label{prob:profile}
Import your function \li{eatCake} from the Value Function Iteration lab into an IPython environment.
Profile this function with arguments $\beta = .9$, $N = 100$, \li{finite=False}, \li{plot=False}.
Now change the arguments $\beta = .95$ and $N = 1000$.
How does your runtime change? What parts of the code take up the most time?

Recall that we can access the profiling tool in IPython as follows:
\begin{lstlisting}
>>> %prun function_call
\end{lstlisting}
where \li{function_call} is a placeholder for whatever function you wish to profile.
\end{problem}

In Problem \ref{prob:profile} you should have noticed that runtime was significantly longer to run for larger $N$ or
$\beta$ closer to 1.  The profiler gives more detailed information than just the overall runtime, however.
The results of Problem \ref{prob:profile} should look something like the following (we have removed a few
columns and several rows from the full output):
\begin{lstlisting}
     503 function calls in 2.101 seconds

Ordered by: internal time

ncalls  tottime  filename:lineno(function)
    1    1.759   <ipython-input-4-1487b3d35ebf>:1(eatCake)
   59    0.244   {method 'argmax' of 'numpy.ndarray' objects}
    1    0.033   twodim_base.py:427(triu)
    1    0.018   {numpy.core.multiarray.where}
    1    0.009   method 'repeat' of 'numpy.ndarray' objects}
    1    0.009   {method 'outer' of 'numpy.ufunc' objects}
    1    0.005   twodim_base.py:349(tri)
    1    0.004   twodim_base.py:657(mask_indices)
    1    0.004   {method 'astype' of 'numpy.ndarray' objects}
   59    0.004   {method 'reduce' of 'numpy.ufunc' objects}
\end{lstlisting}
We notice that the most time was spent in the maximization step (\li{method 'argmax'}).

Recall that the value function iteration method for the infinite horizon problem
approximates the value function $V$ iteratively, producing a sequence of
approximations $V_1, V_2, V_3, \ldots$ that converge to $V$.
At each step, the algorithm also determines the policy function $\psi_k$
corresponding to the approximated value function $V_k$.
This process can also be viewed as applying the approximate policy function $\psi_k$ once to obtain
the approximate value function $V_k$.
Unfortunately, this gives us a rather crude approximation to the value function,
resulting in slow convergence.

What if instead of iteratively approximating the value function in this manner, we instead iteratively approximate
the policy function, and calculate the exact value function associated with each new policy function?
This is the idea behind the policy function iteration algorithm.
In this way we iterate on the policy functions rather than the value functions.
The algorithm for the policy function iteration can be summarized as follows:
\begin{enumerate}
\item Set an initial policy rule $W' = \psi_0(W)$ and a tolerance $\delta$.

\item \label{item:step2_PFI} Compute the value function assuming this rule is used forever:
\begin{equation*}
V_k(W) = \sum_{t=0}^\infty \beta^t u(W_t - \psi_k(W_t)).
\end{equation*}

\item Determine a new policy $\psi_{k+1}$ so that
\begin{equation*}
\psi_{k+1}(W) = \underset{W'}{\text{argmax}} \{u(W-W') + \beta V_k(W')\}.
\end{equation*}

\item If $\delta_k = ||\psi_{k+1} - \psi_k|| < \delta$, stop, otherwise go back to step \ref{item:step2_PFI} with subscript $k+1$.
\end{enumerate}
In order to compute the value function, $V_k$ corresponding to a given policy $\psi_k$, we must solve
\begin{equation}
\label{Val_Fun}
V_k(W) = u(W-W') + \beta V_k(W')
\end{equation}
for $V_k$.

As always, we take a discrete approximation to $W$, obtaining a length $N$ vector
\[
w = (w_1, w_2, \ldots, w_N)
\]
giving the possible cake quantities.
The variable $W'$ becomes $w' = \psi_k(w)$.
Once we have done this, equation \eqref{Val_Fun} is a linear system which we can rewrite as
\begin{equation*}
V_k(w) = u(w-w') + \beta QV_k(w)
\end{equation*}
where $Q$ is the $N\times N$ matrix
\begin{equation*}
Q_{ij} = \left\{
     \begin{array}{ll}
       1 & \text{if} \quad  w_i' = w_j\\
       0 & \text{otherwise.}
     \end{array}
   \right.
\end{equation*}
Solving this system of equations, we have $V_k(w) = (I-\beta Q)^{-1}u(w-w')$.
Although $Q$ may be large, we can take advantage of the fact that it is sparse, containing only $N$ nonzero entries out of $N^2$ total entries.

\begin{problem}
\label{prob:cake_eating_policyfun}
Solve the infinite horizon cake eating problem from the Value Function Iteration lab again, this time using policy function iteration.
Write a function \li{policyIter} that takes arguments $\beta$ (discount factor), $N$ (size of discrete approximation), and
$W_{max}$ (size of original cake, set to default value of 1). Take the function $u$ to be the square root function, as before.
Return the converged value function and policy function, both of which should be arrays of length $N$.

As in the value function iteration, first create your discrete approximation of the $W$ values:
\begin{lstlisting}
>>> import numpy as np
>>> w = np.linspace(0, Wmax, N)
\end{lstlisting}

You will still need to pre-compute all values of $u(W-W')$, storing these in an $N \times N$ array. Make sure, as before,
that the upper triangular entries are set to large negative values, so that we don't choose to consume more cake than is
available. In the code snippets below, we will refer to this array as \li{U}.

You may also find it convenient to not track the approximated policy function $\psi_k$ directly during the iteration, but
rather to have a length $N$ array \li{psi_ind}, whose $i$-th entry gives the index of $w_i'$ relative to the discrete approximation
$w$. Thus, rather than initializing a policy function $\psi_0$ directly, you can initialize it indirectly by setting, for example,
\begin{lstlisting}
>>> psi_ind = np.arange(N)
\end{lstlisting}
which corresponds to an initial policy $\psi_0(W) = W$.
The policy function vector can be obtained from \li{psi_ind} by simply slicing \li{w} as follows:
\begin{lstlisting}
>>> psi = w[psi_ind]
\end{lstlisting}

In order to take advantage of the sparse matrices $I$ and $Q$, use the following imports from the SciPy \li{sparse} library.
\begin{lstlisting}
>>> from scipy import sparse
>>> from scipy.sparse import linalg
\end{lstlisting}
and the following code to initialize $I$ (outside the loop)
\begin{lstlisting}
>>> I = sparse.identity(N, format='csr')
>>> rows = np.arange(0, N)
\end{lstlisting}
and $Q$ (inside the loop)
\begin{lstlisting}
>>> columns = psi_ind
>>> data = np.ones(N)
>>> Q = sparse.coo_matrix((data, (rows,columns)), shape=(N,N))
>>> Q = Q.tocsr()
\end{lstlisting}
Rather than compute $(I-\beta Q)^{-1}$ directly, use Scipy's sparse solver
\begin{lstlisting}
V = linalg.spsolve(I-beta*Q, u(w-w[psi_ind]))
\end{lstlisting}
where \li{u} is the square root function.

In each iteration, we update \li{psi_indices} much as we did in the value function iteration algorithm:
\begin{lstlisting}
>>> psi_ind = np.argmax(U + beta*V, axis=1)
\end{lstlisting}
where we assume \li{V} has shape \li{(1,N)}.

Use the 2-norm when computing $\|\psi_{k+1} - \psi_k\|$, and set the tolerance to $\delta = 1e-9$.

Take $N = 1000$ and $\beta = .95$.
Plot the policy function and compare with your policy function from the Value Function Iteration Lab, using
the same inputs. You should obtain the same answer, but in less runtime.
\end{problem}

\section*{Modified Policy Function Iteration}
While policy function iteration converges in fewer iterations, solving the linear system can be slow,
especially for problems with a large state space.  There is an alternative to this called modified policy
function iteration.

In modified policy function iteration, we don't compute the exact value function corresponding to a policy.
Instead, at step \ref{item:step2_PFI} of the policy iteration algorithm we iterate $m$ times on the value function
equation \eqref{Val_Fun} to get an approximation of the new value function.  This is faster than solving for the
exact value function for large state spaces.  There is no strict rule on the value of $m$, the number of value function
iterations.  In practice values such as $m=10$ or $m=15$ often work well.

Note that our methods for solving dynamic programs boil down to some combination of two things: iterating on the value
function and iterating on the policy function.  Modified policy function does a combination of the two, taking advantage
of the advantages of both methods.  Because modified policy iteration takes only slightly more work to code than value
function iteration, it is often preferred in practice.  Whether policy or modified policy iteration will perform better
may depend on the problem.

\begin{problem}
Solve the infinite horizon cake eating problem again, this time using the modified policy function
iteration method.

Write a function \li{modPolicyIter} that takes the same arguments as your \li{policyIter} function, along
with an additional keyword argument $m$ (set to default value $m=15$), which gives the number of iterations used
to approximate the value function at each step. Return the converged value function and policy function.

Much of the code in your \li{policyIter} will be unchanged when implementing this function. The key differences are
as follows.
First, you do not need to initialize the sparse arrays $I$ and $Q$ in the modified policy function iteration.
Secondly, rather than solving a linear system of equations to obtain the value function, you will loop
the equation
\begin{lstlisting}
>>> V = u(w - w[psi_ind]) + beta*V[psi_ind]
\end{lstlisting}
$m$ times in each iteration. Notice how this line of code corresponds with Equation \eqref{Val_Fun}, where
\li{w} represents $W$, \li{w[psi_ind]} represents $W'$, and \li{V[psi_ind]} represents $V(W')$.

The remainder of the code should be unchanged.

\end{problem}

\begin{problem}
Solve the cake eating problem with each of the three methods and report how many iterations each takes.  Use $N= 1000$ as the
number of grid points for $W$ and $\beta = 0.95$.  It is important that you use the same initial guess in each case in order to
make the results comparable.  The accuracy of the initial guess greatly affects the number of iterations to convergence.  Take
your initial guess as $V = 0$, which corresponds to an initial guess of the policy function with indices $[0,1,2,\ldots, N-1]$
(meaning $\psi(W) = W$, i.e. we eat no cake, leaving it all for the next period).
\end{problem}

In general we should see that value function iteration takes more iterations than modified policy function iteration which in turn
takes more iterations than policy function iteration.  It is important to note that this does not directly say anything about runtime.
Each iteration of policy iteration may take longer than an iteration of value function iteration.

\section*{Discrete Choice (Threshold) Problems}\label{SecDiscrChoice}

One powerful application of dynamic programming is that we can make
models that have both continuous and discrete state variables. These models are sometimes referred to as
discrete choice problems or optimal stopping problems.  Examples include models of employment that involve both
the choice of whether to work and how much to work, models of firm entry and exit that involve the choice of both
whether to produce and how much to produce, and models of marriage that involve the choice of whether to date
(get married or keep dating) and how much to date.
This application illustrates the versatility of dynamic programming as a dynamic solution method

In this lab, we follow a simple version of a standard job search model.
Assume that workers live infinitely long.
We will split a worker's life into discrete time periods, and in each period the worker is either
employed or unemployed, and receives a job offer. The worker must make a choice between discrete actions
(such as accepting or rejecting a job offer), with the goal of maximizing some utility function (hence,
this is a \emph{discrete choice} problem).

We can state this problem in terms of dynamic programming by defining an appropriate value function.
Let the value of entering a period with most recent wage $w$,
current job offer wage $w'$, and employment status $s$ be given by the following value function,
\begin{equation}\label{EqV}
   V(w,w',s) = \begin{cases}
                  V^E(w)    \quad&\text{if}\quad s = E \\
                  V^U(w,w') \quad&\text{if}\quad s = U \\
               \end{cases}
\end{equation}
where employment status is a binary variable $s\in\{E,U\}$ ($E$ indicates ``employed" and $U$ indicates ``unemployed");
a person can be either employed or unemployed.

As in the cake eating problem, the value function is calculated as the sum of some reward (based on the
current state) and the discounted value of entering the next period in some particular state.
The reward function, denoted (as usual) by $u$, gives the utility of spending available funds.
Assuming that a worker receives some wage $x$ in a given period and spends all available money in the period,
the utility of consumption is given by
\[
u(x).
\]
Calculating the value for the next period depends on the employment status in the current period, so we address
this separately for each case.

Let us first consider the case where the individual is unemployed ($s = U$). As is customary, let $s'$ denote the
employment status of the worker in the next period.
In this unemployed state, the worker receives unemployment benefits equal to a fraction of her most recent wage,
i.e. $\alpha w$, where $\alpha \in (0, 1)$.
Hence, the utility of consumption in the current unemployed state is given by
\[
u(\alpha w).
\]

The worker also receives one wage offer ($w'$) per period, and will obtain
this wage in the next period provided that she chooses to accept employment, i.e. provided $s' = E$.
The worker must decide whether to accept the current wage offer $w'$ or to remain unemployed in the next period,
i.e. she must decide on the value of $s'$. How
does she make this choice? She must weigh the value of entering the next period as an employed worker with wage
$w'$ (given by $V^E(w')$) versus the value of entering the next period as an unemployed worker with
previous wage $w$ and unknown wage offer $w''$ (given by $V^U(w,w'')$). Because the worker cannot know
what the future wage offer $w''$ will be, it is treated as a random variable with a particular probability
distribution. Hence, the worker must actually compute the \emph{expected} value of entering the next
period unemployed. This term is simply
\[
\mathbb{E}_{w''}V^U(w,w''),
\]
where $\mathbb{E}_{w''}$ denotes the expectation operator with respect to the probability distribution of future
wage offers $w''$.
To sum up, the worker chooses to accept the wage offer $w'$ or remain unemployed in the next period based on
which option gives the greater expected value, and the value of this decision is given by
\[
\max\Bigl\{V^E(w'), \,\, \mathbb{E}_{w''}V^U(w,w'')\Bigr\}.
\]

The overall value of the current unemployed state with previous wage $w$ and current wage offer $w'$ is
just the utility of consumption plus the discounted value of the next period, i.e.
\begin{equation}\label{EqVu}
V^U(w,w') = u(\alpha w) + \beta \max\Bigl\{V^E(w'), \,\, \mathbb{E}_{w''}V^U(w,w'')\Bigr\},
\end{equation}
where $\beta$ is the discount factor.

Now we turn to the case where the job status is employed ($s = E$).
In this case, the worker receives a wage $w$ in the current period, and so the utility of consumption is
just
\[
u(w).
\]
In the next period, the worker will have most recent wage $w$, she will receive wage offer $w''$, and will
have employment status $s'$. As in the unemployed case, $w''$ is unknown and treated as a random variable.
Unlike the unemployed case, however, the worker's future employment status $s'$ is not under her control,
but rather is also a random variable. The reason for this is that the worker will remain employed
until she loses the job, a random event that occurs with some fixed probability in each time period.
Hence, we must calculate the expected value of the next period with respect to both $w''$ and $s'$.
We may write the entire value function for the employed case as
\begin{equation}\label{EqVe1}
   V^E(w) = u(w) + \beta \mathbb{E}_{w'',s'}V(w,w'',s').
\end{equation}

To calculate the expectation term, we need to know the joint probability distribution over $w''$ and $s'$.
This can be characterized in the following way.
We assume that $s'$ and $w''$ are independent. Hence, we can split the joint expectation
operator into the composition of the two individual expectation operators:
\[
\mathbb{E}_{w'',s'} = \mathbb{E}_{w''}\mathbb{E}_{s'}.
\]
Let $\gamma$ represent the probability that an employed worker becomes unemployed in the next period,
so that $1-\gamma$ is the probability of remaining employed in the next period.
If the worker stays employed in the next period ($s' = E$), then next period's wage equals the current
period's wage, and the term inside the expectation is
\[
V(w,w'',E) = V^E(w).
\]
We then have
\begin{align*}
\mathbb{E}_{s'}V(w,w'',s') &= (1-\gamma)V(w,w'',E) + \gamma V(w,w'',U)\\
&= (1-\gamma)V^E(w) + \gamma V^U(w,w'').
\end{align*}
Notice that the term $(1-\gamma)V^E(w)$ is constant with respect to $w''$. Then
\begin{align*}
\mathbb{E}_{w''}\mathbb{E}_{s'}V(w,w'',s') &= \mathbb{E}_{w''}\left[(1-\gamma)V^E(w) + \gamma V^U(w,w'')\right]\\
&= \mathbb{E}_{w''}(1-\gamma)V^E(w) + \mathbb{E}_{w''}\gamma V^U(w,w'')\\
&= (1-\gamma)V^E(w) + \gamma \mathbb{E}_{w''}V^U(w,w'').
\end{align*}
Hence, we can rewrite \eqref{EqVe1} as follows:
\begin{equation}\label{EqVe2}
   V^E(w) = u(w) + \beta \Bigl[(1-\gamma)V^E(w) + \gamma \mathbb{E}_{w''}V^U(w,w'')\Bigr].
\end{equation}

We have now completely described the value function. What about the policy function?
The policy function for the unemployed worker gives her decision on whether to accept the job $s'=E$
or to reject the job $s'= U$.
This will be a function of both the most recent wage $w$ and the current wage offer $w'$.
The employment status $s'$ in the next period is determined by the policy function $\psi$:
\[
s' = \psi(w,w').
\]

These discrete choice problems are often called threshold
problems because the policy choice depends on whether the state variable is greater than or less than
some threshold level. That is, an unemployed worker will accept a job if and only if the offer wage is
above some set amount that depends on the most recent wage $w$. In the labor search model,
the threshold level is called the ``reservation wage'' $w_R'$. The reservation wage $w_R'$ is defined as
the wage offer such that the worker is indifferent between accepting the job $s' = E$ and
staying unemployed $s' = U$. Hence, this reservation wage satisfies the equation
\begin{equation}\label{EqWR}
   V^E(w_R') = E_{w''}\left[V^U(w,w'')\right].
\end{equation}
The policy function will then take the form of accepting the job if $w' \geq w_R'$ or
rejecting the job offer and remaining unemployed if $w' < w_R'$:
\begin{equation}\label{EqSprime}
   s' = \psi(w,w') = \begin{cases}
                      E \quad\text{if}\quad w' \geq w_R' \\
                      U \quad\text{if}\quad w' < w_R'.
                   \end{cases}
\end{equation}
Figure \ref{fig:disc_policy} shows an example of the discrete policy function.

\begin{figure}
\includegraphics[width=\textwidth]{disc_policy.pdf}
\caption{Here is the policy function for fixed $w = 50$.  Numerically we let 0 represent unemployment, $U$,
and 1 represent employment, $E$.  Thus we see that an individual will choose to take a new job, given
their old wage was 50, at a wage of roughly 35.  Thus for a previous wage of 50, we say the reservation wage is 35.}
\label{fig:disc_policy}
\end{figure}

In summary, the labor search discrete choice problem is characterized by the value functions \eqref{EqV}, \eqref{EqVu},
and \eqref{EqVe2}, the reservation wage \eqref{EqWR}, and the policy function \eqref{EqSprime}. Because wage offers
are distributed according to some given probability distribution (denote the cdf by $F(w')$),
and because the policy function takes the form of \eqref{EqSprime},
the probability that the unemployed worker receives a wage offer that she will reject is $F(w_R')$ and the probability
that she receives a wage offer that she will accept is $1 - F(w_R')$. Just like the continuous-choice cake eating
problems, this problem can be solved by value function, policy function, or modified policy function iteration.

The value function iteration solution method for the equilibrium in the labor search problem is analogous to the
value function iteration from the previous labs. The only difference is that two value functions ($V^E$ and $V^U$)
must converge to a fixed point in this problem instead of just one value function converging in the previous problems.
Although there are two value functions to consider, there is only one policy function, since decisions are only made
in the unemployed state.  Thus, there is only one policy function on which to iterate in the case of policy or modified
policy iteration.

In the following problems, you will solve the job search problem using value function iteration and modified policy
function iteration. Assume that the consumption utility function $u$ is given by
\[
u(w) = \sqrt{w}.
\]
Assume that the probability of becoming unemployed in a given period is $\gamma = 0.10$, the fraction of wages paid
in unemployment benefits is $\alpha = 0.5$, and the discount factor is $\beta = 0.9$.

Assume that the log of wage offers are distributed normally.  We then say that offers are distributed
lognormally and write
\[
w'\sim \text{LogN}(\mu,\sigma).
\]
This is a convenient choice for the distribution of
wage offers.  Among other things, it guarantees that wage offers will be positive.
A mean of $20$ and variance of $200$ are typical parameters of such a wage distribution,
and we will use these parameters in the following problems.

As usual when dealing with continuous variables, we form a discrete approximation of
the possible wage values. In particular, approximate the wage values by a vector of
length $N = 500$ of equally-spaced values from $w_{min} = 0$ to $w_{max} = 100$, inclusive.
We then form a corresponding discrete approximation of the probability density function
$f(w')$ for the lognormal wage offers using code provided in the file \li{discretelognorm.py},
as follows, where \li{w} is the length-$N$ vector of wage values, $m$ is the mean, and $v$ is
the variance as specified above:
\begin{lstlisting}
>>> from discretelognorm import discretelognorm
>>> f = discretelognorm(w, m, v)
\end{lstlisting}
The function \li{discretelognorm} computes the discrete pdf of the specified lognormal distribution in much
the same way that you calculate the discrete normal pdf when solving the stochastic cake eating problem.

\begin{problem}
Solve the job search problem using value function iteration. Return the converged value functions
$V^E$ and $V^U$, as well as the converged policy function $\psi$. The following steps provide detailed
instructions. Note that there are multiple ways to proceed, and the following is simply one (fairly good)
possibility.
\begin{enumerate}

   \item As described above, represent the possible wage values by an array \li{w} of length
   $N$. Denote this array entrywise by
   \[
   w = (w_1,w_2,\ldots,w_N).
   \]
   Calculate the corresponding discrete lognormal pdf \li{f}, exactly as shown above.

   \item Note that $u(w)$ and $u(\alpha w)$ are needed when computing the value functions.
   Since these quantities do not change from one iteration to another, it is smart to
   compute them once at the outset. Denote $u(w)$ by \li{uw} and $u(\alpha w)$ by \li{uaw}.
   These are easily calculated as follows:
\begin{lstlisting}
>>> uw = u(w)
>>> uaw = u(alpha*w).reshape((N,1))
\end{lstlisting}
    where \li{u} is the square root function. We must reshape \li{uaw} because of array broadcasting
    issues that arise in the code snippets below.

   \item Since $V^E$ is a function of only $w$, it will be represented by a vector of length $N$, where
   the $i$-th entry gives $V^E(w_i)$. The unemployed value function $V^U$, however,
   is a function of both $w$ and $w'$, so it will be represented by an $N \times N$ array,
   where the $(i,j)$-th entry gives $V^U(w_i,w_j)$. Initialize the entries of these arrays to 0:
\begin{lstlisting}
>>> VE = np.zeros(N)        #employed value function
>>> VU = np.zeros((N,N))    #unemployed value function
\end{lstlisting}

   \item Note that $\mathbb{E}_{w''}V^U(w,w'')$  is needed to calculate both $V^E$ and $V^U$.
   This expectation depends on $w$, and so can be represented by a length $N$ array, where the
   $i$-th entry is $\mathbb{E}_{w''}V^U(w_i,w'')$.
   It is convenient to assign a variable to this array to keep track of it throughout the iterations.
   We denote the expectation by \li{EVU}, and initialize it to zeros:
\begin{lstlisting}
>>> EVU = np.zeros(N)
\end{lstlisting}

   \item For reasons that will soon become apparent, we will need to create an $N\times N$ helper array
   whose rows are equal to \li{VE} (call this array \li{MVE}), and a $N \times N$ helper array whose columns
   are equal to \li{EVU} (call this array \li{MEVU}).
   At the outset, simply initialize these arrays to zeros.

   \item Because job status is a binary variable, the policy function returns one of two possible values. It is
   convenient to represent ``employed" by $1$ and ``unemployed" by $0$. Now the policy function depends
   on $w$ and $w'$, so it will also be represented by an $N\times N$ array \li{PSI} of zeros and ones,
   where the $(i,j)$-th entry gives $\psi(w_i, w_j)$.

   \item Now we are ready to begin the iteration.
   A single iteration involves computing the updated value functions $V^E$ and $V^U$ from
   equations \eqref{EqVe2} and \eqref{EqVu} and then calculating the $2$-norm distance between
   both pairs of old and updated value functions to test for convergence. If both of these
   $2$-norm distances are less than $10^{-9}$, terminate the iteration.

   Before calculating the updated value functions, we first update our helper arrays \li{MVE} and
   \li{MEVU}. The rows of \li{MVE} need to equal \li{VE}. We can use array broadcasting:
\begin{lstlisting}
>>> MVE[:,:] = VE.reshape((1,N))
\end{lstlisting}
   The columns of \li{MEVU} need to equal \li{EVU}, so use a similar technique:
\begin{lstlisting}
>>> MEVU[:,:] = EVU.reshape((N,1))
\end{lstlisting}

   Now let us address how to compute the updated $V^U$, which we denote by \li{VU1}.
   Equation \eqref{EqVu} shows that it is the sum of
   two terms. The first, $u(\alpha w)$, we have already computed and stored in the variable \li{uaw}.
   The second term involves a maximization between two alternatives. One can imagine writing
   a double for loop ranging over the values of $w'$ and $w$ to compute each individual
   $\max\{V^E(w'), \mathbb{E}_{w''}V^U(w,w'')\}$, but we can take advantage of the helper
   arrays \li{MVE} and \li{MEVU} to do this computation in one efficient line of code.
   Note that the $(i,j)$-th entry of \li{MVE} is just $V^E(w_j)$ and the $(i,j)$-th
   entry of \li{MEVU} is $\mathbb{E}_{w''}(w_i, w'')$, and
   \[
   V^U(w_i,w_j) = u(\alpha w_i) + \beta\max\{V^E(w_j), \mathbb{E}_{w''}V^U(w_i,w'')\}.
   \]
   Hence, taking the entrywise maximum of the arrays \li{MVE} and \li{MEVU} gives us the appropriate
   max term for $V^U$. To get the entrywise maximum of two arrays, stack the arrays along a new
   axis using \li{np.dstack}, and maximize along that axis. The computation for \li{VU}, then, is
   \begin{lstlisting}
>>> VU1 = uaw + beta*np.max(np.dstack([MEVU, MVE]), axis=2)
   \end{lstlisting}

   Calculating the updated $V^E$, denoted by \li{VE1}, is more straightforward.
   Equation \eqref{EqVe2} shows that it is just
   a particular linear combination of the arrays \li{uw}, \li{VE}, and \li{EVU}:
   \begin{lstlisting}
>>> VE1 = uw + beta*((1-gamma)*VE + gamma*EVU)
   \end{lstlisting}

   We can now calculate the 2-norm distances between old and updated value functions.
   It remains to update \li{VE}, \li{VU}, and \li{EVU}.
   The first two updates are trivial, and calculating \li{EVU} is equivalent to the
   matrix-vector multiplication of \li{VU} with \li{f}. This is similar to how we computed
   expectations in previous labs:
   \begin{lstlisting}
>>> EVU = np.dot(VU,f).ravel()
   \end{lstlisting}
   We use the \li{ravel} function to ensure that \li{EVU} is a flat array.

   \item Notice that it is not necessary to iteratively update the policy function, as it is not needed
   to update the value functions. Thus, we need only compute the policy function once, after convergence
   of the value functions has been achieved. This is done in a manner similar to calculating
   $\max\{V^E(w'), \mathbb{E}_{w''}V^U(w,w'')\}$ as described above, except we need to take the \emph{argmax}:
\begin{lstlisting}
>>> PSI = np.argmax(np.dstack([MEVU,MVE]), axis=2)
\end{lstlisting}

   \item Compute the reservation wage $w_R'$ as a function of the current wage $w$. It will be represented
   by a length $N$ array called \li{wr}. The reservation wage is the
   value of $w'$ where the policy function changes from zeros to ones (the optimal choice changes from remaining
   unemployed to accepting the job offer). We can calculate this as follows:
   \begin{lstlisting}
>>> wr_ind = np.argmax(np.diff(PSI), axis = 1)
>>> wr = w[wr_ind]
   \end{lstlisting}

   \item Plot the equilibrium reservation wage $w_R'$ of the converged problem as a function of the current
   wage $w$ with the current wage on the $x$-axis and the reservation wage $w_R'$ on the $y$-axis. This is
   the most common way to plot discrete choice policy functions. The reservation wage represents the wage
   that makes the unemployed worker indifferent between taking a job offer and rejecting it. So any wage
   above the reservation wage line represents $s' = E$ and any wage below the reservation wage line represents
   $s' = U$. Your plot should resemble that in Figure \ref{fig:res_wage}.

\end{enumerate}
\end{problem}

\begin{figure}
\includegraphics[width=\textwidth]{reservation_wage.pdf}
\caption{The reservation wage as a function of previous wage for the job search problem.}
\label{fig:res_wage}
\end{figure}

In the previous problem, it was necessary to iterate on two value functions.
Consequently the convergence is relatively slow.
We can improve upon this situation by using modified policy function iteration.

\begin{problem}
Solve the same problem, this time using modified policy function iteration with $15$ value function iterations
within each policy iteration.  You should be able to re-use much of your code from the previous problem.

Start off by initializing all of the same variables. Additionally, initialize your policy function array \li{PSI},
say
\begin{lstlisting}
>>> PSI = 2*np.ones((N,N))
\end{lstlisting}

Next comes the iteration.
Essentially, the iteration will consist of an outer while-loop (which terminates once the 2-norm distance
between successive policy functions passes below $10^{-9}$), and an inner for loop (with $15$ loops).

The first step in the while-loop is to calculate the new policy function \li{PSI1}, just as in the previous
problem. Next, perform the inner for-loop, which consists simply of the value function iteration, but this
time using the current policy function. This means the line of code
\begin{lstlisting}
>>> VU = uaw + beta*np.max(np.dstack([MEVU, MVE]), axis=2)
\end{lstlisting}
is no longer valid, as it does not use the policy function. We must instead have
\begin{lstlisting}
>>> VU = uaw + beta*(MVE*PSI1 + MEVU*(1 - PSI1))
\end{lstlisting}
Why is this code correct?

Finally, after exiting the for loop, calculate the 2-norm distance between the old and the new policy function,
and then update your old policy function, i.e.
\begin{lstlisting}
>>> PSI = PSI1
\end{lstlisting}

After convergence is achieved, once again compute the reservation wage array, and plot it as in the previous
problem. Then return the converged policy function.
\end{problem}

\begin{problem}
How many iterations did the value function iteration method take?
How many iterations did the modified policy function iteration method take?
Which was faster?
\end{problem}


\epigraph{``Since Newton, mankind has come to realise that the law of physics are always expressed in the language of differential equations"}{Steven Strogatz}

\section{RNN}
\subsection{Introduction}
Weather forecasting has traditionally been done by physical models of the atmosphere, which are unstable to perturbations, and thus are inaccurate for large periods of time\cite{why_rnn}. Since machine learning techniques are more robust to perturbations, it would be logical to combine a neural network with a physical model. Weather forecasting is a sequential data problem, therefore, a recurrent neural network is the most suitable option for this task.

\begin{definition}
A recurrent neural network is a class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence.
\end{definition}

Before, we delve into the specific example of using a recurrent neural network to predict the future state of the atmosphere, it is necessary to review what a recurrent neural network is. Recurrent Neural Networks (RNNs) are neural networks that are used in situations where data is presented in a sequence. For example, let's say you want to predict the future position of a fast-moving ball. Without information on the previous position of the ball, it is only possible to make an inaccurate guess. If you had, however, a large number of snapshots of the previous position, you are then able to predict the future position of the ball with some certainty. RNNs excel at modelling sequential data such as this. This is due to sequential memory.

In order to intuitively understand sequential memory, the prime example would be the alphabet. While it is easy to say the alphabet from A-Z, it is much harder to go from Z-A. There is a logical reason why this is difficult. As a child, you learn the alphabet in a sequence. Sequential memory is a mechanism that makes it easier for your brain to recognise sequence patterns.

In a traditional neural network, there is an input layer, hidden layer, and an output layer. In a recurrent neural network, a loop is added to pass information forward as seen in the diagram below (provided by Towards Data Science)\cite{intro_rnn}:

\begin{figure}[H]
    \centering
    \includegraphics[width=.2\linewidth]{Images/rnn.png}
    \caption{Visualisation of a Recurrent Neural Network}
\end{figure}

The information that is forwarded is the hidden layer, which is a representation of previous inputs. How this works in practise is that you initialise your network layers and the initial hidden state. The shape and dimension of the hidden state will be dependent on the shape and dimension of your recurrent neural network. Then you loop through your inputs, pass the relevant parameter and hidden state into the RNN. The RNN returns the output and a modified hidden state. Last you pass the output to the output layer, and it returns a prediction.

There is, however, a major problem known as short-term memory. Short-term memory is caused by something known as the vanishing gradient problem, which is also prevalent in other neural network architectures. As the RNN processes more steps, it has troubles retaining information from previous steps. Short-Term memory and the vanishing gradient is due to the nature of back-propagation. This can be comprehended through understanding how a neural network is trained\cite{intro_rnn}.

\begin{definition}
Back-propagation is an algorithm used to train and optimise neural networks.
\end{definition}

To train a recurrent neural network, you use an application of back-propagation called back-propagation through time. Training a neural network has three major steps. First, the relevant data vector is normalised between 0 and 1, the vector is fed into the RNN, and it goes through an activation function. The activation function utilised in the software is the rectified linear activation function\cite{lstm_rnn}.

\begin{definition}
The rectified linear activation function is a piece-wise linear function that will output the input directly if is positive, otherwise, it will output zero.
\end{definition}

The function is linear for values greater than zero, meaning it has a lot of the desirable properties of a linear activation function when training a neural network using back-propagation. Yet, it is a nonlinear function as negative values are always output as zero. As a result, the rectified function is linear for half of the input domain and nonlinear for the other half, it is referred to as a piece-wise linear function\cite{relu}. This nonlinear element is extremely important if the system has a nonlinear component, for example in predicting the evolution of the future state of the atmosphere.

\begin{figure}[H]
    \centering
    \includegraphics[width=.65\linewidth]{Images/relu.png}
    \caption{Sketch of the Rectified Linear Activation Function}
\end{figure}

Second, it outputs the results. Third, it compares the prediction to the ground truth using a loss function.

\begin{definition}
A loss function outputs an error value which is an estimate of how poorly the network is performing.
\end{definition}

The lost function that will be utilised in the software will be the function for mean squared error. The reason for choosing this particular function is that it heavily penalises large errors, as it squares the difference between the predicted and actual value. A large error in a weather forecast is highly undesirable, hence, the use of this function. The function is represented below:

\begin{equation}
    MSE = \frac{1}{n}\sum_{i=1}^n(Y_i-\hat{Y_i})^2
\end{equation}

If  a vector of $n$ predictions is generated from a sample of $n$ data points on all variables, and $Y$ is the vector of observed values of the variable being predicted, with $\hat{Y_i}$ being the predicted values.

\begin{definition}
Mean squared error is the average squared difference between the estimated values and the actual value.
\end{definition}

Returning to the training of the RNN, it uses that error value from the loss function. to do back propagation which calculates the gradients for each time step in the network. The gradient is the value used to adjust the networks internal weights, allowing the network to learn. The bigger the gradient, the bigger the adjustments and vice versa. Here is where the problem lies. When doing back propagation, the gradient of the current time step is calculated with respect to the effects of the gradients, in the time step before it. So if the adjustments to the time step before it is small, then adjustments to the current time step will be even smaller.  The gradient values will exponentially shrink as it propagates through each time step. That causes gradients to exponentially shrink as it back propagates down. The earlier layers fail to do any learning as the internal weights are barely being adjusted due to extremely small gradients.

Because of vanishing gradients, the RNN doesn’t learn the long-range dependencies across time steps. So not being able to learn on earlier time steps causes the network to have a short-term memory. In order to combat this, a long short-term memory is used\cite{intro_rnn}.

\subsection{LSTM}
LSTM's were created as a solution to the short-term memory problem. They have internal mechanisms called gates that can regulate the flow of information. These gates can learn which data in a sequence is important to keep or throw away. By doing that, it can pass relevant information down the long chain of sequences to make predictions. For example, if you were interested in buying a particular product, you might read a review in order to determine if the purchase of the product is a good decision. When you read a review, your brain subconsciously only remembers important keywords. You pick up words like ``amazing", ``superb", or ``awful", you don't remember words such as "the", "as", or "because". This is what an LSTM does, it learns to keep only the relevant information to make predictions.

An LSTM has a similar control flow as a recurrent neural network. It processes data passing on information as it propagates forward. The differences are the operations within the LSTM’s cells. The core concept of LSTM’s are the cell state, and it’s various gates. The cell state is the method by which information is transferred down the sequence chain. The cell state, in theory, can carry relevant information throughout the processing of the sequence. So even information from the earlier time steps can make its way to later time steps, reducing the effects of short-term memory. As the cell state goes on its journey, information gets added or removed to the cell state via gates\cite{lstm_rnn}.

\begin{definition}
A gate is an electric circuit with an output which depends on the combination of several inputs.
\end{definition}

Gates contain the sigmoid activation function. The sigmoid activation function squishes values between 0 and 1. That is helpful to update or forget data because any number getting multiplied by 0 is 0, causing values to disappears or be ``forgotten". Any number multiplied by 1 is the same value therefore that value stay’s the same or is ``kept".

\begin{figure}[H]
    \centering
    \includegraphics[width=.65\linewidth]{Images/sigmoid.png}
    \caption{Sketch of the Sigmoid Activation Function}
\end{figure}

There are three types of gates utilised within a neural network: a forget gate, an input gate, and an output gate. A forget gate decides what information should be thrown away or kept. Information from the previous hidden state and information from the current input is passed through the sigmoid function. An input gate is where the previous hidden state and current input are passed into a sigmoid function. The output gate decides what the next hidden state should be. The hidden state is also used for predictions. First, we pass the previous hidden state and the current input into a sigmoid function. Then we pass the newly modified cell state to the rectified linear activation function. We multiply the rectified linear activation function output with the sigmoid output to decide what information the hidden state should carry. The output is the hidden state. The new cell state and the new hidden is then carried over to the next time step\cite{lstm_rnn}.

\subsection{Principal Component Analysis}\label{pca_section}
The number of features in a dataset is referred to as its dimensionality. Dimensionality reduction refers to techniques that reduce the number of input variables in a dataset. More input features often make a predictive modelling task more challenging to model, and can be computational expensive. Considering dimensionality reduction is a data preparation technique performed on data prior to modelling, it might be performed after data cleaning and data scaling and before training a predictive model\cite{dimension_reduction}.

For a given training window, there is 22,032,000 points. Attempting such a dataset would be computational burdensome, hence, the attractiveness of dimensionality reduction. Dimensionality reduction can reduce the number of data points without severely impacting accuracy. Fewer input dimensions often mean correspondingly fewer parameters or a simpler structure in the machine learning model, referred to as degrees of freedom. A model with too many degrees of freedom is likely to overfit the training dataset and therefore may not perform well on new data. It is desirable to have simple models that generalise well, and in turn, input data with few input variables. The problem is high-dimensional functions have the potential to be much more complicated than low-dimensional ones, and that those complications are harder to discern. The only way to beat this problem is to incorporate knowledge about the data that is correct\cite{dimension_reduction}. A popular choice for dimensionality reduction is principal component analysis, and was the technique selected for this project.

\begin{definition}
Principal Component Analysis, is a dimensionality reduction method that is often used to reduce the dimensionality of large data sets, by transforming a large set of variables into a smaller one that still contains most of the information in the large set.
\end{definition}

Reducing the number of variables of a data set naturally comes at the expense of accuracy, but the trick in dimensionality reduction is to trade a little accuracy for simplicity. Because smaller data sets are easier to explore and visualise and make analysing data much easier and faster for machine learning algorithms without extraneous variables to process\cite{pca}.

First, the dataset is standardised to the range of the continuous initial variables so that each one of them contributes equally to the analysis. Second, to see if there is any relationship between the variables of the input dataset, covariance matrix computation is done. The covariance matrix is not more than a table that summaries the correlations between all the possible pairs of variable. The eigenvectors and eigenvalues of the covariance matrix is computed in order to identify the principal components. Principal components are new variables that are constructed as linear combinations or mixtures of the initial variables. PCA tries to put maximum possible information in the first component, then maximum remaining information in the second and so on. Organizing information in principal components this way, will allow for the reduction of dimensionality without losing much information, and  by discarding the components with low information and considering the remaining components as your new variables\cite{pca}. For the purposes of this project, 99\% of the variance was retained in order to balance accuracy with reducing the number of features.

\section{Dataset}
\subsection{ERA5 Atmospheric Reanalysis Dataset}\label{era5_dataset}
ERA5 provides hourly estimates of a large number of atmospheric, land and oceanic climate variables. The data covers the Earth on a 30km grid and resolves the atmosphere using 137 levels from the surface up to a height of 80km. ERA5 includes information about uncertainties for all variables at reduced spatial and temporal resolutions. Quality-assured monthly updates of ERA5 are published within 3 months of real time. Preliminary daily updates of the dataset are available to users within 5 days of real time. ERA5 combines vast amounts of historical observations into global estimates using advanced modelling and data assimilation systems\cite{era5}.

The ERA5 reanalysis dataset was used for training, validating and testing the performance of the neural network architecture. Reanalysis datasets provide the best guess of the atmospheric state at any point in time by combining a forecast model with the available observations. The raw data is available hourly for 40 years from 1979 to 2019 on a $0.25^{\circ}$ latitude-longitude grid ($721 \times 1440$ grid points) with 37 vertical levels. Since this raw dataset is quite significant, it is necessary to regrid the dataset to a lower resolution and use a smaller fraction of the available dataset\cite{rasp2020weatherbench}.  The poles were excluded from the dataset in order to avoid a singularity, and the potential negative impact that could have on predictive ability of the neural network.

It was ultimately decided to use a spatial resolution of $3^{\circ}$ ($60 \times 120$ grid points) and a temporal resolution of 2 hours. In regards to pressure surfaces, 17 vertical pressure levels were ultimately chosen:  1, 2, 5, 10, 20, 50, 100, 200, 300, 400, 500, 600, 700, 800, 900,  950, 1000 hPa. Note that it is common to use pressure in hectopascals as a vertical coordinate instead of physical height. The pressure at sea level is approximately 1000 hPa and decreases roughly exponentially with height. 850 hPa is at around 1.5 km height. 500 hPa is at around 5.5 km height. The data is split into yearly NetCDF files for each variable. The entire dataset at $3^{\circ}$ resolution has a size of 75GB. The available variables were chosen based on meteorological consideration. Geopotential, temperature, humidity and wind are prognostic state variables in most physical NWP and climate models\cite{rasp2020weatherbench}.

\subsection{Integrated Forecasting System}\label{ifs_section}
The Integrated Forecast System is a global numerical weather prediction system jointly developed and maintained by the European Centre for Medium-Range Weather Forecasts. The version of the IFS run at ECMWF is often referred to as the "ECMWF" or the "European model" in North America, to distinguish it from the American GFS. It comprises a spectral atmospheric model with a terrain-following vertical coordinate system coupled to a 4D variational data assimilation system. In 1997 the IFS became the first operational forecasting system to use a 4D variational, data assimilation system

\begin{definition}
4D dimensional variational data assimilation system adjusts a short-range forecast, called the background, in space and time to bring it into closer agreement with meteorological observations.
\end{definition}

It is one of the predominant global medium-range models in general use worldwide; its most prominent rival is in the 6–10 day medium range include the American Global Forecast System, the Canadian Global Environmental Multiscale Model and the UK Met Office Unified Model. It is the gold standard of medium-range numerical weather prediction. The current IFS deterministic forecast is computed on a cluster with 11,664 cores. One 10 day forecast at 10 km resolution takes around 1 hour of real time to compute\cite{rasp2020weatherbench}. The Integrated Forecasting System will be used as a comparison against the neural network.

\begin{figure}[H]
    \centering
    \includegraphics[width=.8\linewidth]{Images/ifs.jpg}
    \caption{ECMWF Integrated Forecast System}
\end{figure}

To provide physical baselines more in line with the current resolution of the neural network, it was compared against the IFS model at two coarser horizontal resolutions, T42 (approximately $2.8^{\circ}$) with 62 vertical levels and T63 (approximately $1.9^{\circ}$) with 137 vertical levels. It must be noted that I personally did not generate such forecasts, or perform the analysis on said forecasts. It acquired the results from Weather Bench, a benchmark dataset for data-driven weather forecasting. According to said source; computationally, a single forecast takes 270 seconds for the T42 model and 503 seconds for the T64 model on a single XC40 node with 36 cores.

\section{Implementation}\label{implement_rnn}
The dataset consists of five features: geopotential, zonal wind, meridional wind, air temperature, and relative humidity. For a single day, there will be twelve observations. The goal for this project will be to predict the relevant atmospheric parameter in 6 hours time given the last fifteen days of data. In order to make such predictions, it is necessary to create a window of the last 180 ($15 \times 12$) observations to train the model\cite{time_series}. The neural network was trained on observational data from 2005 to 2016. The remainder of the dataset, 2017 to 2019, was preserved for validation, testing and benchmarking the neural network against physics-based models.

At the start, a seed is set in order to ensure reproducibility. As mentioned previously, it is important to scale features before training a neural network. Normalisation is a common way of doing this scaling by subtracting the mean and dividing by the standard deviation of each feature. In order for the most optimal performance, the method ``MinMaxScaler" from the library, scikit-learn, is utilised within the software\cite{scikit-learn}. An LSTM requires a 1-dimensional sequence, however, the atmosphere is a 3-dimensional system. Hence, it is necessary to flatten the 3-dimensional vector that represents the state of the atmosphere. This is done in order to avoid the need of repeatably running the RNN. Batches are then created to split the data into manageable sequences. A large batch size is used in order to greater emphasise values at the extremes, as such observations are particularly important in numerical weather predictions in tracking hurricanes and tornadoes. The diagram on the following page shows how the data is represented after flattening the data and batching it (provided by Tensorflow)\cite{time_series}.

\begin{figure}[H]
    \centering
    \includegraphics[width=.5\linewidth]{Images/data_rnn.png}
    \caption{Visualisation of how the data is represented after flattening and batching.}
\end{figure}

Following this process, the data is fed into the RNN. The LSTM model is built using Keras in TensorFlow, which is an free and open-source software library for machine learning. It was developed by the Google Brain Team\cite{tensorflow}. It is apparent that a multi-step model is needed as the model needs to learn to predict a range of future values. The source code for the LSTM model developed for the software is shown below:

\begin{minted}[mathescape,linenos,frame=lines]{python}
# Prepossessed historical data, which has been flatten and batched.
x_data, y_data, x_val, y_val = prepossessing_function(input_data)
# Prepossessed initial conditions, which has been flatten and batched.
initial_conditions = prepossessing_function(input_initialconditions)

# The network is shown data from the last 15 days.
past_history = 15 * 12

# The network predicts the next 6 hours worth of steps.
future_target = 0.5 * 12

# Create, and train models.
# Optimiser.
opt = Adam(lr=1e-3, decay=1e-5)
# Create model.
model = Sequential()
model.add(
    LSTM(
        480, activation='relu', input_shape=(past_history, features)
    )
)
model.add(RepeatVector(future_target))
model.add(LSTM(480, activation='relu', return_sequences=True))
model.add(TimeDistributed(Dense(features)))
model.compile(
    optimizer=opt, loss='mse', metrics=['mean_absolute_error']
)

# Train.
model.fit(
    x_data, y_data, validation=(x_val, y_val), epochs=epochs, batch_size=120
)

# Predict (this function is iterated to the forecast length required).
future_state = model.predict(initial_conditions)
# Invert normalisation, and flattening.
future_state = inverse_prepossessing(future_state)
\end{minted}

The model consists of four LSTM layers, which in combination are able to produce a more accurate and reliable prediction than a single LSTM layer. As is evident, the activation function for each LSTM is the rectified linear activation function, which is built into Keras. The number of epochs can be specified by the end user depending on the computational resources they have and what they need. More epochs will evidently lead to a more accurate neural network.

\begin{definition}
An epoch is one forward pass and one backward pass of all the training examples.
\end{definition}


\documentclass[11pt]{amsart}
\usepackage{geometry}                % See geometry.pdf to learn the layout options. There are lots.
\geometry{letterpaper}                   % ... or a4paper or a5paper or ...
%\geometry{landscape}                % Activate for for rotated page geometry
%\usepackage[parfill]{parskip}    % Activate to begin paragraphs with an empty line rather than an indent
\usepackage{graphicx}
\usepackage{amssymb}
\usepackage{epstopdf}
\DeclareGraphicsRule{.tif}{png}{.png}{`convert #1 `dirname #1`/`basename #1 .tif`.png}

\title{}
\author{}
%\date{}                                           % Activate to display a given date or no date

\begin{document}
\maketitle
%\section{}
%\subsection{}

The set of equations we are solving are:

\begin{align}
\frac{dV_i}{dt} &= rV_i\Big(1 - \frac{V_i}{K}\Big) - \frac{\alpha V_i}{V_i + B}H_i, \\
\frac{dH_i}{dt} &= H_i\Big(\frac{\alpha \beta V_i}{V_i + B} - m\Big) + f_i(H),
\end{align}

\noindent where $V$ and $H$ are the vegetation and herbivore density interacting on $i = 1, 2, ... 100$ discrete nodes of a network, and where $r$ is intrinsic growth rate, $K$ is the carrying capacity of the vegetation, $\alpha$ is the maximum predation rate, $\beta$ is the herbivore efficiency, $B$ is the half-saturation constant, $m$ is the mortality rate of the herbivore, and $f$ is a function describing the coupling interaction between different nodes. An example $f$ is given below:

\begin{equation}
f_i(H) = \frac{\sigma}{2P} \sum_{k = i - P}^{i + P} (H_k - H_i)
\end{equation}

\noindent where $\sigma$ controls the coupling strength and $1 \leq P \leq N/2$ is the coupling range of the topology. This is linear coupling and is the coupling used in Dutta and Banerjee (2015) which we are trying to reproduce and extend. Our extension will be changing the form of $f$.

\begin{equation}
D_n(H)=d_n\frac{H^\alpha}{S+H^\alpha}
\end{equation}
Where $d_n$ is the maximum dispersal rate per capita, varying this has the physical interpretation of varying the distance between nodes or the ability of animal to traverse that distance. S is the half saturation and $\alpha$ varies the shape of the distribution. We took this function evaluated at each node to gain the dispersal contribution (effectively limiting the maximum dispersal between nodes vs the linear case where the greater the difference, the greater the contribution).
\end{document}


% Preamble.
\documentclass[12pt]{article}
\usepackage[margin=1.25in]{geometry}
\usepackage[fleqn]{amsmath}
\usepackage{textcomp}
\usepackage{gensymb}
\usepackage{amsfonts}
\usepackage{enumitem}
%\usepackage{tikz}  % Include for figures.
%\usepackage{subfiles}  % Include for subfiles.

%% Title macros.
\newcommand{\HOMEWORKNUM}{22}
\newcommand{\NAME}{D. Choi}
\newcommand{\DATE}{2020-06-25}

\title{\vspace{-2\baselineskip}MATH 225 - Homework \#\HOMEWORKNUM}
\author{\NAME}
\date{\DATE}

%% Formatting options.
%\pagenumbering{gobble}  % Include for single-page document.

% Document.
\begin{document}
\maketitle

\section*{1.}
\textit{For the following matrices, find the eigenvectors and associated
eigenvalues by thinking geometrically about the corresponding matrix
transformation.} \\[\baselineskip]
\begin{enumerate}[label=(\alph*)]
	\item $\begin{pmatrix} 3 & 0 \\ 0 & 3 \end{pmatrix}$. \\[\baselineskip]
	This matrix transformation is equivalent to scaling a vector by $3$:
	\begin{equation*}
		\begin{pmatrix} 3 & 0 \\ 0 & 3 \end{pmatrix}
		\vec{v}
		=
		3 \vec{v}
		.
	\end{equation*}
	Thus, every nonzero vector is an eigenvector of this matrix. Their
	corresponding eigenvalue is \boxed{3}.

	\item $\begin{pmatrix} -2 & 0 \\ 0 & 4 \end{pmatrix}$. \\[\baselineskip]
	This matrix transformation is equivalent to scaling a vector's first
	component by $-2$ and its second component by $4$.
	\begin{equation*}
		\begin{pmatrix} -2 & 0 \\ 0 & 4 \end{pmatrix}
		\begin{pmatrix} x \\ y \end{pmatrix}
		=
		\begin{pmatrix} -2x \\ 4y \end{pmatrix}
		.
	\end{equation*}
	Thus, nonzero vectors in the form of
	$\begin{pmatrix} n \\ 0 \end{pmatrix}$ are eigenvectors of this matrix,
	with a corresponding eigenvalue of \boxed{-2}, and nonzero vectors in the
	form of $\begin{pmatrix} 0 \\ n \end{pmatrix}$ are eigenvectors of this
	matrix, with a corresponding eigenvalue of \boxed{4}.

	\item \textit{The identity matrix.} \\[\baselineskip]
	The identity transformation is equivalent to scaling a vector by $1$:
	\begin{equation*}
		I \vec{v} = 1 \vec{v}.
	\end{equation*}
	Thus, every nonzero vector is an eigenvector of the identity matrix. Their
	corresponding eigenvalue is \boxed{1}.

	\item \textit{A diagonal matrix with distinct diagonal entries.}
	\\[\baselineskip]
	This matrix transformation is equivalent to scaling a vector's entries
	each by their corresponding diagonal entry. \\
	Thus, such a matrix's eigenvectors are scalar multiples of the standard
	basis vectors. Each eigenvector's corresponding diagonal entry is their
	corresponding eigenvalue.
\end{enumerate}

\section*{2.}
\textit{Suppose that $A$ is a $2 \times 2$ matrix having eigenvectors}
\begin{equation*}
	\vec{v}_1 = \begin{pmatrix} 2 \\ 1 \end{pmatrix}, \quad
	\vec{v}_2 = \begin{pmatrix} -1 \\ 2 \end{pmatrix}
\end{equation*}
\textit{and associated eigenvalues $\lambda_1 = 2$, $\lambda_2 = -3$. If
$\vec{x} = \begin{pmatrix} 5 \\ 0 \end{pmatrix}$, find the vector
$A^4 \vec{x}$.}

\section*{3.}
\textit{Determine whether the following statements are true or false and
provide a justification for your response.}
\begin{enumerate}[label=(\alph*)]
	\item \textit{The eigenvalues of a diagonal matrix are equal to the
	entries on the diagonal.}
	\item \textit{If $A v = \lambda v$, then $A^2 v = \lambda v$ as well.}
	\item \textit{Every vector is an eigenvector of the identity matrix.}
	\item \textit{If $\lambda = 0$ is an eigenvalue of $A$, then $A$ is
	invertible.}
	\item \textit{For every $n \times n$ matrix $A$, it is possible to find
	a basis of $\mathbb{R}^n$ consisting of eigenvectors of $A$.}
\end{enumerate}

\end{document}


\section{Theoretical Foundations}
\label{sec:Theoretical_Foundations}
This section summarizes the theory and formulas necessary to understand the following experiments on interference and diffraction in section \ref{sec:Evaluation}.

\subsection{Wave Interference}
\label{subsec:Interference}
Interference is caused, when two waves superpose to form a wave of greater, lower or the same amplitude. The interfering waves are usually coherent with each other, either because they come from the same source or because they have the same frequency \cite{diffraction}.

\subsection{Diffraction Slit / Anti-Slit}
\label{subsec:Diffraction_Slit}
The minima of the diffraction pattern of a slit or an anti-slit $\varphi_m$ can be derived by using the Huygens–Fresnel principle. It states that every point on a slit is itself the source of spherical wavelets. These wavelets have the same amplitude and phase \cite{diffraction}.

The order $m$, the wavelength $\lambda$ and the width of the slit $w$ is used to calculate the minima $\varphi_m$:
\begin{equation}
\sin\varphi_m = \frac{m\cdot\lambda}{w} \qquad \text{with} \qquad m = \pm 1, \pm 2, ...
\label{eq:slit_minima}
\end{equation}
To obtain the maxima $\varphi_m$, the following equation is used \cite{uni_hamburg}:
\begin{equation}
\sin\varphi_m =
\begin{cases}
	0 & \qquad \text{for} \qquad m = 0\\
	\left(m + \frac{1}{2}\right)\cdot\frac{\lambda}{w} & \qquad \text{for} \qquad m = 1, 2, 3, ...\\[6pt]
	\left(m - \frac{1}{2}\right)\cdot\frac{\lambda}{w} & \qquad \text{for} \qquad m = -1, -2, -3, ...
\end{cases}
\label{eq:slit_maxima}
\end{equation}
where:
\begin{conditions}
	\varphi_m & angle of the interference of order $m$ \\
	m & order of the interference \\
	\lambda & wavelength of the laser \\
	w & width of the slit or the anti-slit
\end{conditions}
\begin{figure}[H]
	\centering
	\includegraphics[scale=1.5]{slit_theory}
	\caption{Diffraction of light on a slit \cite{diffraction} - partially modified}
	\label{fig:diff_slit}
\end{figure}
Figure \ref{fig:diff_slit} shows the diffraction of light on a slit. The partial wave 1 (red) and the partial wave 2 (green) differ by $\lambda/2$ and thus interfere destructively, which results in a minimum. The width of the slit is denoted as $w$ and the angle of interference is denoted as $\varphi$ \cite{diffraction}.

\subsection{Diffraction Circular Aperture}
\label{subsec:Diffraction_Circular_Aperture}
When diffraction occurs at a circular aperture, interference rings can be observed. The angles at which the minima occur can be derived by using the following equation \cite{diffraction}:
\begin{equation}
\sin\varphi_c = \frac{c_k\cdot\lambda}{d} \qquad \text{with} \qquad c_k = \frac{j_{\text{1,k}}}{\pi}
\label{eq:circular_aperture}
\end{equation}
The first five Bessel coefficients $c_k$ are the following (used in this laboratory report):
\begin{equation}
c_k = \{1.220,~2.233,~3.238,~4.241,~5.243,~...\}
\label{eq:coeffs}
\end{equation}
where:
\begin{conditions}
	\varphi_c & angle of the interference \\
	c_k & order of the interference \\
	j_{\text{1,k}} & $k$th positive zero of the Bessel function $J_1(x)$ \\
	\lambda & wavelength of the laser \\
	d & diameter of the circular aperture
\end{conditions}
The Bessel coefficients $c_k$ shown above were calculated with MATLAB (see appendix \ref{sec:MATLAB_Error_Calculation}).

\subsection{Diffraction Cross-Grid Aperture}
\label{subsec:Diffraction_Cross-Grid_Aperture}
To calculate the maximas of a cross-grid aperture on either the x-axis or the y-axis, the equation for a line-grid can be used \cite{diffraction}.
\begin{equation}
\sin\varphi_m = \frac{m\cdot\lambda}{g}
\label{eq:cross-grid}
\end{equation}
where:
\begin{conditions}
	\varphi_m & angle of the interference of order $m$ \\
	m & order of the interference \\
	\lambda & wavelength of the laser \\
	g & grid constant
\end{conditions}
For the above equation \ref{eq:cross-grid} to be valid, the distance between the cross-grid aperture and the projection surface ($x$) must be large \cite{diffraction}.
\begin{figure}[H]
	\centering
	\includegraphics[scale=1.5]{line-grid}
	\caption{Diffraction of light on a line-grid \cite{diffraction} - partially modified}
	\label{fig:line-grid}
\end{figure}
Figure \ref{fig:line-grid} shows the diffraction of light on a line-grid, which is a simplified version of the cross-grid. The angle of interference is denoted as $\varphi$.

\subsection{Calculation of the Angle $\varphi$}
\label{subsec:Calculation_of_the_Angle}
The angle of the interference $\varphi$ is only measured indirectly by measuring the distance from the aperture to the projection surface ($x$) and the position of the maxima or minima ($y$).
\begin{figure}[H]
	\centering
	\includegraphics[scale=1]{angle}
	\caption{Calculation of the angle $\varphi$}
	\label{fig:angle}
\end{figure}
Figure \ref{fig:angle} shows the relationship between the angle $\varphi$ and the distances $x$ and $y$. To calculate the angle of interference the following equation \ref{eq:angle} is used:
\begin{equation}
\varphi = \arctan\left(\frac{y}{x}\right)
\label{eq:angle}
\end{equation}
where:
\begin{conditions}
	\varphi & angle of the interference \\
	$x$ & distance between the aperture and the projection surface (see figure \ref{fig:experimental_arrangement}) \\
	$y$ & position of the maxima or the minima
\end{conditions}