\documentclass[11pt]{amsart}
\usepackage{amsmath,amsfonts,amsthm,amssymb, amsaddr}

\title{Distributions (Generalised Functions)}

\author{Joe Bentley}

\date{\today}

\begin{document}

\maketitle

\newpage

\section{Dirac Delta Function}

Consider a vector function defined as $\mathbf{A} = \frac{\mathbf{r}}{r^3}$. In spherical polar coordinates this can be written as $\mathbf{A} = \frac{1}{r^2} \mathbf{e_r}$. Next we calculate its divergence using the formula proved in previous notes,

\begin{align*}
  \nabla\cdot\mathbf{A} = \frac{1}{r^2}\frac{\partial}{\partial r}\left(r^2 \frac{1}{r^2}\right) = 0
\end{align*}

Consider next the divergence theorem for the same vector field $\mathbf{A}$, also known as Gauss' theorem, which was shown to be,

\begin{align*}
  \int_V \nabla\cdot\mathbf{A} dV = \oint_S \mathbf{A}\cdot d\mathbf{S}
\end{align*}

By integrating over a sphere of radius $a$ centered at the origin we see,

\begin{align*}
  \oint_S \mathbf{A}\cdot d\mathbf{S} = \frac{1}{a^2} 4\pi a^2 = 4\pi
\end{align*}

There appears to be a contradiction as we have just proved that $\nabla\cdot\mathbf{A} = 0$, but this integral tells us that since the surface integral is non-zero, then so is the volume integral. It turns out however that we have only really proved that $\nabla\cdot\mathbf{A} = 0$ when $\mathbf{r} \neq \mathbf{0}$. Therefore we must have that $\nabla\cdot\mathbf{A}$ suddenly blows up at $\mathbf{r} = \mathbf{0}$ in such a way that the integral over it has the value $4\pi$. This result is given by a function called the Dirac delta function, in this case in 3 dimensions,

\begin{align*}
  \nabla\cdot\mathbf{A} = 4\pi \delta^3(\mathbf{r})
\end{align*}

where

\begin{align*}
  \delta^3(\mathbf{r}) =
  \begin{cases}
    0 & \mathbf{r} \neq \mathbf{0} \\
    \infty & \mathbf{r} = \mathbf{0}
  \end{cases}
\end{align*}

which also has the constraint,

\begin{align*}
  \int_V \delta^3(\mathbf{r}) d^3\mathbf{r} = 1
\end{align*}

For example, the electric field due to a point charge at the origin is given by,

\begin{align*}
  \mathbf{E} = \frac{q}{4\pi\epsilon_0} \frac{\mathbf{r}}{r^3}
\end{align*}

The divergence of this can thus be written as,

\begin{align*}
  \nabla\cdot\mathbf{E} = \frac{q}{\epsilon_0}\delta^3(\mathbf{r}) = \frac{\rho(\mathbf{r})}{\epsilon_0}
\end{align*}

Here, $\rho(\mathbf{r}) = q\delta^3(\mathbf{r})$ is the charge density of a point charge $q$ at the origin. It is zero at $\mathbf{r} \neq \mathbf{0}$ and infinite at $\mathbf{r} = \mathbf{0}$ such that the integral over its volume gives $q$.

The Dirac delta function shows up every time we mix discrete and continuous objects.

\section{The 1D Dirac Delta Function}

The one-dimensional Dirac delta function $\delta(x)$ is defined,

\begin{align*}
  \delta(x) =
  \begin{cases}
    0 & x \neq 0 \\
    \infty & x = 0
  \end{cases}
\end{align*}

Under the constraint,

\begin{align*}
  \int_{-\infty}^{\infty} \delta(x) dx = 1
\end{align*}

As in the three-dimensional case, it may be defined as the limit of a sequence of functions that become increasingly peaked, getting more and more narrow. A good function to choose for this is a Gaussian distribution,

\begin{align*}
  \delta_n(x) = \sqrt{\frac{n}{\pi}} e^{-nx^2}
\end{align*}

We divide by $\sqrt{\frac{n}{\pi}}$ to normalise the distribution so that if we integrate over it we get one regardless of the value of $n$. We can see that the limits give us back the Dirac delta function,

\begin{alignat*}{2}
  &\lim_{n \to \infty} \delta_n(x) = 0 \qquad &&\text{for } x\neq0 \\
  &\lim_{n \to \infty} \delta_n(x) = \infty \qquad &&\text{for } x = 0 \\
  &\lim_{n \to \infty} \int_{-\infty}^{\infty} \delta_n(x) = 1
\end{alignat*}

We can therefore think of the Dirac delta function $\delta(x)$ as the limit of $\delta_n(x)$ as $n \to \infty$.

The key property of the delta function is that,

\begin{align*}
  \int_a^b f(x) \delta(x - x_0) dx =
  \begin{cases}
    f(x_0) & a < x_0 < b \\
    0 & \text{otherwise}
  \end{cases}
\end{align*}

This is because the delta function effectively picks out the value of $f(x)$ where $x = x_0$. This is because the only place where the delta function isn't zero, is when $x = x_0$. Since the integral over the entire Dirac delta function is one, that means we just get the function $f(x_0)$ back.

We may also write this as,

\begin{align*}
  f(x)\delta(x - x_0) = f(x_0)\delta(x - x_0)
\end{align*}

Using some intuition it is clear to see that they are equal, as the product will be zero for all $x \neq x_0$, so we can just replace $f(x)$ with $f(x_0)$

We can relate the one-dimensional and three-dimensional Dirac delta functions by,

\begin{align*}
  \delta^3(\mathbf{r}) = \delta(x)\delta(y)\delta(z)
\end{align*}

Or for a given offset $\mathbf{r_0}$,

\begin{align*}
  \delta^3(\mathbf{r} - \mathbf{r_0}) = \delta(x - x_0)\delta(y - y_0)\delta(z - z_0)
\end{align*}

\section{The Heaviside (Step) Function}

The Heaviside function, also known as the step function, is defined as the integral of the Dirac delta function,

\begin{align}
  \label{eq:heavisideintegral}
  \Theta(x) = \int_{-\infty}^x\delta(t)dt
\end{align}

By considering the definition of the Dirac delta function, and looking at the cases where $x < 0$ and $x > 0$ we can find a numerical definition of the Heaviside function,

\begin{align*}
  \Theta(x) =
  \begin{cases}
    0 & x < 0 \\
    1 & x > 0
  \end{cases}
\end{align*}

The function, when plotted, looks like an instantaneous step from zero to one at $x = 0$. By considering what happens when we have a negative $x$ as the argument to the function we can also define,

\begin{align*}
  \Theta(-x) =
  \begin{cases}
    1 & x < 0 \\
    0 & x > 0
  \end{cases}
\end{align*}

This allows us to write the expression,

\begin{align*}
  \Theta(x) + \Theta(-x) = 1
\end{align*}

which holds for all $x$ not equal to zero.

By considering the definition of the Heaviside function we can define its derivative by the fundamental theorem of calculus. That is, the derivative is the opposite operation of the integral. Therefore by taking the derivative with respect to $x$ of both sides of eq.~\ref{eq:heavisideintegral} we obtain,

\begin{align*}
  \frac{d}{dx}\Theta(x) = \delta(x)
\end{align*}

So far the Heaviside function is not defined for $x = 0$, so we want to explore a way that we can define this as if it is a continuous function. Intuitively we expect that $\Theta(x = 0) = \frac{1}{2}$ which we will show by considering the integral,

\begin{align*}
  \int_{-\infty}^{\infty} \frac{\sin x}{x} dx = \pi
\end{align*}

We will not prove this result here as it is long, but the proof can be found on the internet or in the notes on Canvas. This integral can be further generalized by considering the value of,

\begin{align*}
  \int_{-\infty}^{\infty} \frac{\sin ax}{x} dx
\end{align*}

for different values of $a$.

Firstly, we will consider the case when $a = 0$. First we define a placeholder value,

\begin{align*}
  y &= ax \\
  dy &= adx
\end{align*}

By dividing the second line by the first line,

\begin{align*}
  \frac{dy}{y} = \frac{dx}{x}
\end{align*}

Therefore by substituting this back into our integral, we see that the result is independent of the value of $a$,

\begin{align*}
   \int_{-\infty}^{\infty} \frac{\sin ax}{x} dx = \int_{-\infty}^{\infty} \frac{\sin y}{y} dy = \pi
\end{align*}

It is this property of being independent of $a$ which makes this integral a good function for representing the Heaviside step function.

Next, we need to consider the case where $a = 0$, which is simply

\begin{align*}
  \int_{-\infty}^{\infty} \frac{0}{x} dx = 0
\end{align*}

Finally, for the case where $a < 0$ we let $a = -b$ where $b > 0$, such that,

\begin{align*}
  \sin ax = \sin(-bx) = -\sin bx
\end{align*}

The integral can thus be rewritten in the form,

\begin{align*}
  -\int_{-\infty}^{\infty} \frac{\sin bx}{x} dx = -\pi
\end{align*}

Therefore, we have that,

\begin{align*}
  \int_{-\infty}^{\infty} \frac{\sin ax}{x} dx =
  \begin{cases}
    \pi & a > 0 \\
    0 & a = 0 \\
    -\pi & a < 0
  \end{cases}
\end{align*}

By adding a term and dividing by a constant we can bring this in line with the definition of the Heaviside step function,

\begin{align*}
  \Theta(a) = \frac{1}{2} + \frac{1}{2\pi}\int_{-\infty}^{\infty}\frac{\sin ax}{x} dx=
  \begin{cases}
    1 & a > 0 \\
    \frac{1}{2} & a = 0 \\
    0 & a < 0
  \end{cases}
\end{align*}

Or, by change of variable we can make this a function of $x$,

\begin{align*}
  \Theta(x) = \frac{1}{2} + \frac{1}{2\pi}\int_{-\infty}^{\infty}\frac{\sin xt}{t} dt=
  \begin{cases}
    1 & x > 0 \\
    \frac{1}{2} & x = 0 \\
    0 & x < 0
  \end{cases}
\end{align*}

We can see that this lends itself to a continuous representation of the step function which allows us to define a value for $\Theta(x = 0)$. This is also useful as it is directly differentiable to give a trigonometric form of the Dirac delta function,

\begin{align*}
  \delta(x) &= \frac{d}{dx}\Theta(x) \\
  \delta(x) &= \frac{1}{2\pi} \int_{-\infty}^{\infty} \frac{t \cos xt}{t} dt \\
  \delta(x) &= \frac{1}{2\pi} \int_{-\infty}^{\infty} \cos xt dt
\end{align*}

This can also be expressed in an exponential form by noting that,

\begin{align*}
  \int_{-\infty}^{\infty} e^{ixt} dt &= \int_{-\infty}^{\infty} \left[\cos xt + i\sin xt\right] dt \\
                                     &= \int_{-\infty}^{\infty} \cos xt dt
\end{align*}

In the second line we used the fact that sine is an odd function, so the integral over it from $-\infty$ to $\infty$ is zero. We can now express the Dirac delta function in the exponential form,

\begin{align*}
  \delta(x) = \frac{1}{2\pi} \int_{-\infty}^{\infty} e^{ixt} dt
\end{align*}

\section{Schr\"{o}dinger Equation in One Dimension}

The time independent Schr\"{o}dinger equation in one dimension can be written as,

\begin{align*}
  -\frac{\hbar^2}{2m}\frac{d^2\psi}{dx^2} + V(x)\psi = E\psi
\end{align*}

where $V(x)$ is the position-dependent potential energy. The Schr\"{o}dinger equation is only possible to solve for discrete values of energy $E$. We are interested in the eigenfunctions $\psi_n(x)$ and their corresponding energy eigenvalues (energy levels) $E_n$.

Consider an attractive $\delta$-function potential. In this case, it is a potential of the form $V(x) = -\lambda\delta(x)$, a sharp spike potential that goes to $-\infty$ at $x = 0$. The Schr\"{o}dinger equation can be written as,

\begin{align*}
  -\frac{\hbar^2}{2m}\frac{d^2\psi}{dx^2} - \lambda\delta(x)\psi = E\psi
\end{align*}

We want to look for the bound state with an energy,

\begin{align*}
  E = -\frac{\hbar^2q^2}{2m}
\end{align*}

For the case where $x \neq 0$, the potential function $V(x) = 0$, and we have,

\begin{align*}
  -\frac{\hbar^2}{2m}\frac{d^2\psi}{dx^2} = E\psi = -\frac{\hbar^2q^2}{2m}\psi
\end{align*}

From this we can see that,

\begin{align*}
  \frac{d^2\psi}{dx^2} = q^2\psi
\end{align*}

The solution therefore has the form,

\begin{align*}
  \psi(x) =
  \begin{cases}
    Ae^{qx} & x < 0 \\
    Be^{-qx} & x > 0
  \end{cases}
\end{align*}

showing us that the solution has the form of an exponential decay either side of $x = 0$.

By considering the Heaviside function, we can also write this in the equivalent form,

\begin{align*}
  \psi(x) = Ae^{qx}\Theta(-x) + Be^{-qx}\Theta(x)
\end{align*}

To substitute this into the Schr\"{o}dinger equation we calculate the derivatives,

\begin{align*}
  \frac{d\psi}{dx} &= Aqe^{qx}\Theta(-x) - Bqe^{-qx}\Theta{x} - Ae^{qx}\delta(x) + Be^{-qx}\delta(x) \\
                   &= Aqe^{qx}\Theta(-x) - Bqe^{-qx}\Theta{x} + (B - A)\delta(x)
\end{align*}

In the second line we used the fact that the Dirac delta function is zero every except where $x = 0$, so we can write $e^{qx}\delta(x) = \delta(x)$ as $e^{qx}$ is zero at $x = 0$, and $\delta(x)$ is zero at $x \neq 0$. We can now compute the second derivative,

\begin{align*}
  \frac{d^2\psi}{dx^2} &= Aq^2e^{qx}\Theta(-x) + Bq^2e^{-qx}\Theta(x) \\
                       &- Aqe^{qx}\delta(x) - Bqe^{-qx}\delta(x) \\
                       &+ (B-A)\delta'(x)
\end{align*}

Using $e^{qx}\delta(x) = \delta(x)$ from before, as well as noting that the first line is just the wavefunction multiplied by $q^2$,

\begin{align*}
  \frac{d^2\psi}{dx^2} = q^2\psi - (A+B)q\delta(x) + (B-A)\delta'(x)
\end{align*}

By substituting this back into the time independent Schr\"{o}dinger equation,

\begin{align*}
  -\frac{\hbar^2}{2m}q^2\psi+\frac{\hbar^2q\left(A+B\right)}{2m}\delta(x) - \frac{\hbar^2}{2m}\left(B - A\right)\delta'(x) - \lambda\delta(x)\psi = E\psi
\end{align*}

We do not yet have a suitable expression for the third term that isn't in terms of $\psi$. Note that we can write the third term in the form,

\begin{align*}
  \delta(x)\psi=\delta(x)\left[A\Theta(-x)e^{qx}+B\Theta(x)e^{-qx}\right]
\end{align*}

Remember that $\delta(x) = 0$ for all $x \neq 0$, which means that we can write this as,

\begin{align*}
  \delta(x)\psi=\delta(x)\Theta(0)\left[A+B\right] = \frac{1}{2}(A+B)\delta(x)
\end{align*}

Subtituting this into the Schr\"{o}dinger equation we then have,

\begin{align*}
  -\frac{\hbar^2}{2m}q^2\psi+\frac{\hbar^2q\left(A+B\right)}{2m}\delta(x) - \frac{\hbar^2}{2m}\left(B - A\right)\delta'(x) - \frac{\lambda}{2}\left(A+B\right)\delta(x) = E\psi
\end{align*}

The coefficients of each term on the left hand side can be compared with the right hand side.

Firstly, comparing $\delta'(x)$,

\begin{align*}
  -\frac{\hbar^2}{2m}(B-A)=0
\end{align*}

and therefore,

\begin{align*}
  B = A
\end{align*}

This is intuitive, as the wavefunction $\psi$ is clearly symmetric about $x = 0$.

Next we compare $\delta(x)$,

\begin{align*}
  \frac{\hbar^2q}{2m}(A+B)-\frac{\lambda}{2}(A+B) = 0
\end{align*}

Therefore can find an expression for $q$,

\begin{align*}
  \frac{\hbar^2q}{2m} &= \frac{\lambda}{2} \\
  q = \frac{m\lambda}{\hbar^2}
\end{align*}

Finally, comparing $\psi$ terms,

\begin{align*}
  E = -\frac{\hbar^2q}{2m} = -\frac{\hbar^2}{2m} {\left(\frac{m\lambda}{\hbar^2}\right)}^2
\end{align*}

and therefore the energy $E$ is given by,

\begin{align*}
  E = -\frac{m\lambda^2}{2\hbar^2}
\end{align*}

Since $A = B$, we can write our expression for the wavefunction as,

\begin{align*}
  \psi(x) =
  \begin{cases}
    Ae^{qx} & x \leq 0 \\
    Ae^{-qx} & x \geq 0
  \end{cases}
\end{align*}

An alternative form using the absolute value function,

\begin{align*}
  \psi(x) = Ae^{-q\|x\|}
\end{align*}

To find the constant $A$ we can normalize the wavefunction by noting that the integral of the probability density over all space must be $1$,

\begin{align*}
  \int_{-\infty}^{\infty}\psi(x)^2 dx &= A^2\int_{-\infty}^{\infty} e^{-2q\|x\|} dx \\
                                      &= 2A^2\int_{-\infty}^{\infty} e^{-2qx} dx \\
                                      &= {\left[-\frac{A^2}{q}e^{-2qx}\right]}^{\infty}_0 e^{-2qx} dx \\
                                      &= \frac{A^2}{q} = 1
\end{align*}

Therefore we see that the normalization constant $A$ is given by,

\begin{align*}
  A = \sqrt{q}
\end{align*}

The wavefunction can then be written as,

\begin{align*}
  \psi(x) = \sqrt{q}e^{-q\|x\|}
\end{align*}

\section{Alternative Boundary Condition Approach}

In this section we will take a boundary condition approach to solving the same problem. First consider the Schr\"{o}dinger equation for the potential $V(x) = -\lambda\delta(x)$,

\begin{align*}
  -\frac{\hbar^2}{2m}\frac{d^2\psi}{dx^2} - \lambda\delta(x)\psi = E\psi
\end{align*}

Rearranging this for the second derivative we have,

\begin{align}
  \label{eq:second}
  \frac{d^2\psi}{dx^2} = -\frac{2m\lambda}{\hbar^2}\delta(x)\psi - \frac{2mE}{\hbar^2}\psi
\end{align}

From here on we will introduce a new notation: $\psi(0+)$ will mean $\psi(x)$ as $x \to 0$ from the \textit{positive} $x$-direction, and $\psi(0-)$ will mean $\psi(x)$ as $x to 0$ from the \textit{negative} $x$-direction.

At $x = 0$ there is a sharp delta potential as we have defined. We can integrate over this potential from $x = -\epsilon$ to $x = \epsilon$ where $\epsilon$ is very small. On the left side of the potential we have the approaching $\psi(\epsilon-)$, and on the right side we have $\psi(\epsilon+)$. By integrating both sides of eq.~\ref{eq:second} with respect to $dx$,

\begin{align*}
  \psi'(\epsilon+) - \psi'(\epsilon-) = -\frac{2m\lambda}{\hbar^2}\psi(0) - \frac{2mE}{\hbar^2}\int_{-\epsilon}^{\epsilon}\psi(x) dx
\end{align*}

By letting $\epsilon \to 0$ we get a discontinuous step in $\psi'(x)$ where the gradient suddenly becomes negative at $x = 0$,

\begin{align}
  \label{eq:wfderiv}
  \psi'(0+)-\psi'(0-)=-\frac{2m\lambda}{\hbar^2}\psi(0)
\end{align}

Since this is not equal to zero, it shows us that there is a discontinuous change in the gradient at $x = 0$. If it were continuous at $x = 0$, the two gradients would be the same at $x = 0$, and thus the difference would be zero. Hence for a delta function potential, the gradient of the wavefunction is discontinuous at $x = 0$.

To find the solutions to the Schr\"{o}dinger equation, consider how the Schr\"{o}dinger equation for this problem behaves in different regions. First look at the region $x \neq 0$. At $x \neq 0$, the delta function $\delta(x) = 0$, and therefore the Schr\"{o}dinger equation becomes,

\begin{align*}
  -\frac{\hbar^2}{2m}\frac{d^2\psi}{dx^2} = E\psi
\end{align*}

Since we have defined the energy $E$ as,

\begin{align*}
  E = -\frac{\hbar^2q^2}{2m}
\end{align*}

The solution has the form,

\begin{align*}
  \psi(x) =
  \begin{cases}
    Ae^{qx} & x < 0 \\
    Be^{-qx} & x > 0
  \end{cases}
\end{align*}

Next we look at the boundary where $x = 0$. Here the wavefunction is continuous, that is, $\psi(0+) = \psi(0-)$, unlike the derivative of the wavefunction, where can therefore write that $A = B$ and thus the solution now takes the form,

\begin{align*}
  \psi(x) =
  \begin{cases}
    Ae^{qx} & x < 0 \\
    Ae^{-qx} & x > 0
  \end{cases}
\end{align*}

To use eq.~\ref{eq:wfderiv} we also need the first derivative,

\begin{align*}
  \psi'(x) =
  \begin{cases}
    Aqe^{qx} & x < 0 \\
    -Aqe^{-qx} & x > 0
  \end{cases}
\end{align*}

Substituting into eq.~\ref{eq:wfderiv} we obtain,

\begin{align*}
  \psi'(0+)-\psi'(0-) &= -\frac{2m\lambda}{\hbar^2}\psi(0) \\
  -Aq - (+Aq) &= -\frac{2m\lambda}{\hbar^2}A \\
  -2Aq &= -\frac{2m\lambda}{\hbar} A \\
  q &= \frac{m\lambda}{\hbar^2}
\end{align*}

We can substitute this into our equation for the energy $E$,

\begin{align*}
  E &= -\frac{\hbar^2q^2}{2m} \\
  E &= -\frac{m\lambda^2}{2\hbar^2}
\end{align*}

which is the result we obtained last time, but we arrived at it much more easily using boundary conditions.

\section{Attractive Double Delta Potential}

Consider the Schr\"{o}dinger equation for an attractive double delta potential. That is, two infinitely negative potentials with strength $\lambda$, both spaced a distance $a$ away from the origin $x = 0$. We then have the potential function,

\begin{align*}
  V(x) = -\lambda\left[\delta(x - a) + \delta(x + a)\right]
\end{align*}

Since the potential function $V(x)$ is even, due to us choosing symmetrical delta potentials about $x = 0$, the solutions $\psi(x)$ will be either even or odd functions.

The solutions in $x < -a$ and $x > a$ are the same as before but what in the region in between? Since it is the superposition of the two other states we can just write it as a hyperbolic $\cosh(x)$ function, as $\cosh(x) = \frac{e^x + e^{-x}}{2}$. The solutions therefore take the form,

\begin{align*}
  \psi(x) =
  \begin{cases}
    Ce^{qx} & x < -a \\
    D\cosh{qx} & -a < x < a \\
    Ce^{-qx} & x > a
  \end{cases}
\end{align*}

We know that the two constants $C$ must be the same in both $x < -a$ and $x > a$ as the potential function is symmetric about $x = 0$. The derivative is given by,

\begin{align*}
  \psi'(x) =
  \begin{cases}
    qCe^{qx} & x < -a \\
    Dq\sinh{qx} & -a < x < a \\
    -qCe^{-qx} & x > a
  \end{cases}
\end{align*}

Next we consider the boundary conditions.

At $x = a$, we know that the wavefunction is continuous,

\begin{align}
  \label{eq:doubleone}
  \psi(a+) &= \psi(a-) \notag\\
  Ce^{-qa} &= D\cosh{qa}
\end{align}

The derivative has the same boundary condition as before,

\begin{align}
  \label{eq:doubletwo}
  \psi'(a+) - \psi'(a-) &= -\frac{2m\lambda}{\hbar^2}\psi(a) \notag\\
  -qCe^{-qs} - Dq\sinh{qa} &= -\frac{2m\lambda}{\hbar^2}Ce^{-qa} \notag\\
  \left(\frac{2m\lambda}{\hbar^2} - q\right)Ce^{-qa} &= Dq\sinh{qa}
\end{align}

Dividing eq.~\ref{eq:doubletwo} by eq.~\ref{eq:doubleone} we obtain,

\begin{align*}
  \frac{2m\lambda}{\hbar^2} - q &= q\tanh{qa} \\
  \frac{2m\lambda a}{\hbar^2} = qa\left[1 + \tanh{qa}\right]
\end{align*}

The quantity $qa$ must be dimensionless, as it is inside the hyperbolic tangent function. By letting $y = qa$ we can write this equation as,

\begin{align*}
  \frac{2m\lambda a}{\hbar^2} = y\left[1 + \tanh{y}\right]
\end{align*}

This is a transcendental equation, so cannot be solved analytically, although we can solve this numerically.

We can rewrite this in an exponential form by noting,

\begin{align*}
  \tanh{y} = \frac{\sinh{y}}{\cosh{y}} = \frac{e^y - e^{-y}}{e^y + e^{-y}}
\end{align*}

and by adding one,

\begin{align*}
  \tanh y + 1 = \frac{e^y - e^{-y}}{e^y + e^{-y}} + \frac{e^y + e^{-y}}{e^y + e^{-y}} = \frac{2e^y}{e^y + e^{-y}} = \frac{2}{1 + e^{-2y}}
\end{align*}a

The condition thus becomes,

\begin{align*}
  \frac{2y}{1 + e^{-2y}} &= \frac{2m\lambda a}{\hbar^2} \\
  \frac{z}{1 + e^{-z}} &= \frac{2m\lambda a}{\hbar^2}
\end{align*}

where $z = 2y = 2qa$.

Let the function $f(z)$ be equal to the left hand side of this equation. Taking the derivative,

\begin{align*}
  f'(z) &= \frac{1 + e^{-z} + ze^{-z}}{{\left(1 + e^{-z}\right)}^2} \\
        &= \frac{e^z\left(e^z + 1 + z\right)}{{\left(e^z + 1\right)}^2}
\end{align*}

This is clearly greater than zero for all $z$. This means that $f(z)$ is \textit{always} increasing from $f(0) = 0$ to $f(\infty) = \infty$.

Therefore the function $f(z) = \frac{2m\lambda a}{\hbar^2}$ has only one solution, which is the $z$ at which $f(z) = \frac{2m\lambda a}{\hbar^2}$. There is therefore an even solution for any $\lambda$ where $0 < \lambda < \infty$.

Now let us look at the odd solution, $\psi_0(x)$, where $\psi_0(-x) = -\psi_0(x)$,

\begin{align*}
  \psi(x) =
  \begin{cases}
    -Ce^{qx} & x < -a \\
    D\sinh{qx} & -a < x < a \\
    Ce^{-qx} & a < x
  \end{cases}
\end{align*}

and its derivative is,

\begin{align*}
  \psi'(x) =
  \begin{cases}
    -qCe^{qx} & x < -a \\
    qD\cosh{qx} & -a < x < a \\
    -qCe^{-qx} & a < x
  \end{cases}
\end{align*}

Now we apply our boundary conditions. First we have continuity of $\psi(x)$ at $x = a$,

\begin{align}
  \label{eq:evenone}
  Ce^{-qa} = D\sinh{qa}
\end{align}

We have the same discontinuity in $\psi'(x)$ at $x = a$,

\begin{align}
  \label{eq:eventwo}
  \psi'(a+) - \psi'(a-) &= -\frac{2m\lambda}{\hbar^2}\psi(a) \notag\\
  -qCe^{-qa} - qD\cosh{qa} &= -\frac{2m\lambda}{\hbar^2}Ce^{-qa} \notag\\
  \left(\frac{2m\lambda}{\hbar^2} - q\right)Ce^{-qa} &= qD\cosh{qa}
\end{align}

Diving eq.~\ref{eq:eventwo} by eq.~\ref{eq:evenone} we obtain,

\begin{align*}
  \frac{2m\lambda}{\hbar^2} - q &= q\coth{qa} \\
  \frac{2m\lambda a}{\hbar^2} &= qa\left[1 + \coth{qa}\right]
\end{align*}

In step two, as well as rearranging the $q$ term we multiply both sides by $a$. Now let $y = qa$ as before. We can therefore write this as,

\begin{align*}
  y\left[1 + \coth{y}\right] = \frac{2m\lambda a}{\hbar^2}
\end{align*}

We can write the hyperbolic cot term in exponential form as before,

\begin{align*}
  \coth{y} = \frac{1}{\tanh{y}} = \frac{e^y + e^{-y}}{e^y - e^{-y}}
\end{align*}

and then by adding one,

\begin{align*}
  1 + \coth{y} &= \frac{e^y + e^{-y}}{e^y - e^{-y}}+\frac{e^y - e^{-y}}{e^y - e^{-y}} = \frac{2e^y}{e^y - e^{-y}} \\
               &= \frac{2}{1 - e^{-2y}}
\end{align*}

Our equation therefore becomes,

\begin{align*}
  \frac{z}{1 - e^{-z}} = \frac{2m\lambda a}{\hbar^2}
\end{align*}

where $z = 2y = 2qa$.

Consider the function $g(z) = \frac{z}{1 - e^{-z}}$. By taking the limit of $g(z)$ as $z \to 0$,

\begin{align*}
  g(0) = \lim_{z\to 0} \frac{z}{1 - e^{-z}} = \lim_{z\to 0} \frac{1}{e^-z} = 1
\end{align*}

and, as before, $g(\infty) = \infty$. By taking the first derivative of the function we can see that,

\begin{align*}
  g'(z) = \frac{1-e^{-z}-ze^{-z}}{{\left(1 - e^{-z}\right)}^2} = \frac{e^z \left[e^z - 1 - z\right]}{{\left(e^z - 1\right)}^2}
\end{align*}

We can see by using the maclaurin series that this is greater than zero for all $z$, and therefore the function $g(z)$ is increasing from $g(0) = 1$ to $g(\infty) = \infty$. This time it is possible to have no solution, as there are only solutions if $\frac{2m\lambda a}{\hbar^2} > 1$ since there is a lower limit on the function, $g(0) = 1$. This means that there is not always an odd solution to the Schr\"{o}dinger equation, specifically there isn't when $\frac{2m\lambda a}{\hbar^2} < 1$.

\section{The Top Hat Function}

The top hat function $T(x; a, b)$ is defined such that,

\begin{align*}
  T(x; a, b) = \Theta(x - a) - \Theta(x - b)
\end{align*}

By considering the definition of the heaviside step function, we can see that,

\begin{align*}
  T(x; a, b) =
  \begin{cases}
    0 & x < a \\
    1 & a < x < b \\
    0 & x > b
  \end{cases}
\end{align*}

We can use the top hat function to describe a function that is defined piecewise, that is, it is described by different functions over different regions,

\begin{align*}
  f(x) =
  \begin{cases}
    f_1(x) & x_0 < x < x_1 \\
    f_2(x) & x_1 < x < x_1 \\
    \cdots & \cdots \\
    f_n(x) & x_{n-1} < x < x_n \\
  \end{cases}
\end{align*}

We can write this more succinctly using the top hat notation,

\begin{align*}
  f(x) &= \sum\limits_{i = 1}^n f_i(x) T(x; x_{i - 1}, x) \\
       &= \sum\limits_{i = 1}^n f_i(x) \left[\Theta(x - x_{i - 1}) - \Theta(x - x_i)\right]
\end{align*}

The top hat function can be used to select specific regions of integration,

\begin{align*}
  \int_{-\infty}^{\infty} f(x) dx &= \sum\limits_{i = 1}^n \int_{-\infty}^{\infty} f_i(x) T(x; x_{i - 1}, x_i) dx \\
                                  &= \sum\limits_{i = 1}^n \int_{x_i - 1}^{x_i} f_i(x) dx
\end{align*}

The top hat function here simply selects the limits of the integration region, since the integral will be zero everywhere outside of the top hat function's width.

What if $f(x)$ is defined all the way out to infinity? In this case $f(x)$ can be written as,

\begin{align*}
  f(x) =
  \begin{cases}
    f_0(x) & x < x_0 \\
    f_1(x) & x_0 < x < x_1 \\
    \cdots & \cdots \\
    f_n(x) & x_{n-1} < x < x_n \\
    f_{n+1}(x) & x_n < x
  \end{cases}
\end{align*}

Note the added first and last terms, in the regions $x < x_0$ and $x_n < x$. We can writ this using the heaviside step function, such that $f(x)$ can be written as before plus the two extra terms,

\begin{align*}
  f(x) &= \sum\limits_{i=1}^n f_i(x) \left[\Theta(x - x_{i-1}) - \Theta(x - x_i)\right] \\
       &+ f_0(x) \Theta(x_0 - x) \\
       &+ f_{n+1}(x) \Theta(x - x_n)
\end{align*}

By expanding the summation and grouping each heaviside term $\Theta(x_i - x_{i-1})$ we can see that this can be written,

\begin{align*}
  f(x) = f_0(x) + \sum\limits_{i=0}^n \left[f_{i+1}(x) - f_i(x)\right] \Theta(x - x_i)
\end{align*}

\section{Derivatives of Delta Functions}

Derivatives of the delta function can be defined using integration by parts,

\begin{align*}
  \int_{-\infty}^{\infty} \frac{d\delta(x)}{dx} f(x) dx &= {\left[\delta(x)f(x)\right]}_{-\infty}^{\infty} - \int_{-\infty}^{\infty} \delta(x) \frac{d f(x)}{dx} dx \\
                                                        &= -f'(0)
\end{align*}

Similar analysis gives an expression for the $n$-th derivative,

\begin{align*}
  \int_{-\infty}^{\infty} \delta^n(x) f(x) dx = {(-1)}^n f^n(0)
\end{align*}

\section{Delta Function of Nontrivial Arguments}

Consider the function $\delta\left(f\left(x\right)\right)$. This will only be non-zero when $f(x) = 0$, that is, only at the zeroes $x_n$ of the function $f(n)$.

Let us evaluate the integral

\begin{align*}
  \int_{-\infty}^{\infty} \delta\left(f(x)\right) dx
\end{align*}

We want to split the integration into a number of monotonic regions, such that there is only one solution in each region. First we let $y = f(x)$, and then to evaluate the integral,

\begin{align*}
  \int_{-\infty}^{\infty} \delta\left(f(x)\right) dx = \int \delta(y) \frac{dx}{dy} dy
\end{align*}

Functions only have inverses if they are monotonic. That is, only one-to-one functions have inverses, for example, $f(x) = x^2$ is not one-to-one, as for one $f(x)$ there are two values of $x$ that could correspond to that given $f(x)$, and hence there is no single inverse of this function. Therefore we will take the integral over monotonic regions instead,

\begin{align*}
  \int_{-\infty}^{\infty} \delta\left(f(x)\right) &= \int \delta(y) \left|\frac{dx}{dy}\right| dy \\
                                                  &= \int \delta(y) \frac{1}{|f'(x)|} dy \\
                                                  &= \sum\limits_n \frac{1}{|f'(x)|}
\end{align*}

We take the summation over $x_n$ as each $x_n$ gives a contribution. It therefore follows that,

\begin{align*}
  \delta\left(f(x)\right) = \sum\limits_n \frac{1}{|f'(x)|} \delta(x - x_n)
\end{align*}

For example, evaluating $\delta(x^2 - a^2)$, where $a > 0$. First define $f(x) = x^2 - a^2$, which has zeroes at $+a$ and $-a$. The derivative is trivially calculated as $f'(x) = 2x$, so $f'(a) = 2a$ and $f'(-a) = -2a$. Therefore is follows,

\begin{align*}
  |f'(a)| = |f'(-a)| = 2a
\end{align*}

and therefore,

\begin{align*}
  \delta(x^2 - a^2) = \frac{1}{2a}\left[\delta(x - a) + \delta(x + a)\right]
\end{align*}

Next, consider the function $\delta(\sin{x})$. The function $f(x) = \sin{x}$ has zeroes when $x = n\pi$ where $n$ is any integer. Differentiating $f(x)= \sin{x}$ gives us $f'(x) = \cos{x}$. Our zeroes thus give us $f'(n\pi) = \cos n\pi = {(-1)}^n$, and therefore we can write the delta function as,

\begin{align*}
  \delta(\sin{x}) = \sum\limits_{n=-\infty}^{\infty} \delta(x - n\pi)
\end{align*}

\end{document}


%% -*- coding:utf-8 -*-
\chapter{Functors}

\section{Definitions}

\begin{definition}[Functor]
  \label{def:functor}
  Let $\cat{C}$ and $\cat{D}$ are 2 categories. A mapping $F: \cat{C}
  \tof \cat{D}$ between the categories is called \textit{functor} if it
  preserves the internal structure (see \cref{fig:functor}):
  \begin{itemize}
  \item $\forall a_C \in \catob{C}, \exists a_D \in \catob{D}$ such that
    $a_D = F( a_C )$
  \item $\forall f_C \in \cathom{C}, \exists f_D \in \cathom{D}$ such
    that $\dom f_D = F (\dom f_C), \cod f_D = F (\cod f_C)$. We will use
    the following notation later: $f_D = F(f_C)$.
  \item $\forall f_C, g_C$ the following equation holds:
    \[
    F\left(f_C \circ
    g_C\right) = F\left(f_C\right) \circ F\left(g_C\right) = f_D \circ
    g_D.
    \]
  \item $\forall x \in \catob{C}: F(\idm{x}) = \idm{F(x)}$.
  \end{itemize}

  \begin{figure}
    \centering
    \begin{tikzpicture}[ele/.style={fill=black,circle,minimum
          width=.8pt,inner sep=1pt},every fit/.style={ellipse,draw,inner
          sep=-2pt}]

      % the texts

      \node at (0,3) {$C$};
      \node at (4,3) {$D$};

      \node[ele,label=above:$a_C$] (ac) at (0,2) {};
      \node[ele,label=below:$b_C$] (bc) at (0,0) {};
      \node[ele,label=above:$a_D$] (ad) at (4,2) {};
      \node[ele,label=below:$b_D$] (bd) at (4,0) {};

      \node[draw,fit= (ac) (bc),minimum width=2cm, minimum
        height=3.5cm] {} ;
      \node[draw,fit= (ad) (bd),minimum width=2cm, minimum
        height=3.5cm] {} ;

      \draw[->,thick,shorten <=2pt,shorten >=2pt] (ac) to
      node[left]{$f_C$} (bc);
      \draw[->,thick,shorten <=2pt,shorten >=2pt] (ad) to
      node[right]{$f_D$} (bd);
      \draw[->,thick,shorten <=2pt,shorten >=2pt] (ac) to
      node[sloped,above]{$a_D = F(a_C)$} (ad);
      \draw[->,thick,shorten <=2pt,shorten >=2pt] (bc) to
      node[sloped,above]{$b_D = F(b_C)$} (bd);
    \end{tikzpicture}
    \caption{Functor $F: \cat{C} \tof \cat{D}$ definition}
    \label{fig:functor}
  \end{figure}
\end{definition}

\begin{remark}[Functor]
  When we say that functor preserve internal structure we assume that
  the functor is not just mapping between \mynameref{def:object}s but
  also between \mynameref{def:morphism}s.

  Thus functor is something that allows map one category into another.
  The initial category can be considered as a pattern thus the mapping
  is some kind of searching of the pattern inside another category.
\end{remark}

Programming languages can be considered as a good platform for the
functor examples.
The functor can be defined in Haskell as follows
\footnote{the real definition is quite different from the current one}
\begin{example}[Functor][\textbf{Hask}]
  \label{ex:functor_haskell}
  \begin{minted}{haskell}
    class Functor f where
    fmap :: (a -> b) -> f a -> f b
  \end{minted}
\end{example}

In Scala it can be defined in the same way
\begin{example}[Functor][\textbf{Scala}]
  \label{ex:functor_scala}
  \begin{minted}{scala}
    trait Functor[F[_]] {
      def fmap[A, B](f: A => B): F[A] => F[B]
  }\end{minted}
\end{example}

In C++ the definition differs
\begin{example}[Functor][\textbf{C++}]
  \label{ex:functor_cpp}
  In C++ templates can be considered as type constructors in Haskell
  and therefore can convert one type for another. For instance the
  list of strings can be got with the following construction:

  \begin{minted}{c++}
    using StringList = std::list<std::string>;
    StringList a = {"1", "2", "3"};
  \end{minted}
  i.e. we have \mynameref{def:object}s mapping out of the box.
  Therefore we need to define fmap
  operation for \mynameref{def:morphism}s mapping to complete the
  \mynameref{def:functor} definition. It can be declared as
  follows
  \begin{minted}{c++}
    template < template< class ...> class F, class A, class B>
    F<B> fmap(std::function<B(A)>, F<A>);
  \end{minted}
  The template specialization for the \mintinline{c++}{std::list} can be
  written as follows
  \begin{minted}{c++}
    // file: functor.h
    template <class A, class B>
    std::list<B> fmap(std::function<B(A)> f, std::list<A> a) {
      std::list<B> res;
      std::transform(a.begin(), a.end(), back_inserter(res), f);
      return res;
    }
  \end{minted}

  The simple usage example is the following
  \begin{minted}{c++}
    StringList a = {"1", "2", "3"};
    std::function<int(std::string)> f = [](std::string s) {
      return 2 * atoi(s.c_str());
    };
    auto res = fmap<>(f, a);
  \end{minted}
\end{example}

\begin{definition}[Endofunctor]
  \label{def:endofunctor}
  Let $\cat{C}$ is a \mynameref{def:category}. The
  \mynameref{def:functor} $E: \cat{C} \tof \cat{C}$ i.e.
  the functor from a category to the same category is called
  \textit{endofunctor}.
\end{definition}

\begin{definition}[Identity functor]
  \label{def:idfunctor}
  Let $\cat{C}$ is a \mynameref{def:category}. The
  \mynameref{def:functor} $\idf{C}: \cat{C} \tof \cat{C}$ is called \textit{identity
    functor} if for every object $a \in \catob{C}$
  \[
  \idf{C}(a) = a
  \]
  and for every \mynameref{def:morphism} $f \in \cathom{C}$
  \[
  \idf{C}(f) = f
  \]
\end{definition}

\begin{remark}[Identity functor]
  \label{rem:idfunctor}
  First of all notice that \mynameref{def:idfunctor} is an
  \mynameref{def:endofunctor}.

  There is difference between identity functor and \mynameref{def:id}
  because the first one has deal with both \mynameref{def:object}s and
  \mynameref{def:morphism}s while the second one with the objects
  only.
\end{remark}

\begin{definition}[Functor composition]
  \label{def:functor_composition}
  If we have 3 categories $\cat{C}, \cat{D}, \cat{E}$ and 2 functors
  between them: $F: \cat{C} \tof \cat{D}$ and $G: \cat{D} \tof \cat{E}$
  then we can construct a new functor $H: \cat{C} \tof \cat{E}$ that is
  called \textit{functor composition} and denoted as $H = G \circ F$.
  TBD
\end{definition}

\section{\textbf{Cat} category}
The \mynameref{def:functor_composition} is associative by definition.
Therefore \mynameref{def:idfunctor} with the associative composition
allow us to define a category where other categories are considered as
objects and functors as morphisms:
\begin{definition}[\textbf{Cat} category]
  \label{def:cat_category}
  The category of small categories (see \mynameref{def:small_category})
  denoted as \textbf{Cat} is the \mynameref{def:category} where objects
  are small categories and morphisms are \mynameref{def:functor}s
  between them.
\end{definition}

We can construct an extension of Cartesian product as follows
\begin{definition}[Category Product]
  \label{def:category_product}
  If we have 2 categories $\cat{C}$ and $\cat{D}$ then we can construct
  a new category $\cat{C} \times \cat{D}$ with the following components:
  \begin{itemize}
  \item \mynameref{def:object}s are the pairs $(c,d)$ where $c \in
    \catob{C}$ and $d \in \catob{D}$
  \item \mynameref{def:morphism}s are the pair $(f,g)$ where $f \in
    \cathom{C}$ and $g \in \cathom{D}$
  \item \mynameref{axm:composition} is defined as follows
    \(
    (f_1, g_1) \circ (f_2, g_2) = (f_1 \circ f_2, g_1 \circ g_2)
    \)
  \item Identity is defined as follows: $\idm{C \times D} =
    \left(\idm{C}, \idm{D}\right)$
  \end{itemize}
\end{definition}

\begin{definition}[Terminal object in \textbf{Cat} category]
  \label{def:terminal_object_cat}
  \index{Terminal object!\textbf{Cat} category}
  Let consider $\Delta_c$ is a trivial functor from \mynameref{def:category}
  $\cat{A}$ to category $\cat{C}$ such that $\forall a \in \catob{A}:
  \Delta_c a = c$ -fixed object in $\cat{C}$ and
  $\forall f \in \cathom{A}: \Delta_c f = \idm{c}$.
\end{definition}

The good example can be found in $\cat{Hask}$ category.
\begin{example}[Terminal object in \textbf{Cat} category][\textbf{Hask}]
  \label{ex:terminal_object_cat_hask}
  \begin{minted}{haskell}
    data Const c a = Const c
    fmap :: (a -> b) -> Const c a -> Const c b
    fmap f (Const c a) = Const c
  \end{minted}
\end{example}

\section{Bifunctors}

\begin{definition}[Bifunctor]
  \label{def:bifunctor}
  Bifunctor is a \mynameref{def:functor} whose \mynameref{def:domain} is
  a \mynameref{def:category_product}. I.e. if $\cat{C_1}, \cat{C_2},
  \cat{D}$ are 3 categories then the \mynameref{def:functor}
  \(
  F: \cat{C_1} \times \cat{C_2} \tof \cat{D}
  \) is called \textit{bifunctor}.
\end{definition}

\begin{example}[Bifunctor][\textbf{Set}]
  \label{ex:product_bifunctor}
  Lets $A,B,C$ and $D$ are sets and $f: A \to C, g: B \to D$ are two
  \mynameref{def:function}s. Then the \mynameref{def:cartesian_product}
  with \mynameref{def:product_of_morphisms} form a
  \mynameref{def:bifunctor} $\times$.
\end{example}

\begin{example}[Maybe as a bifunctor][\textbf{Hask}]
  \label{ex:maybe_functor}
  Lets show how the \mintinline{haskell}{Maybe a} type can be
  constructed from different
  \mynameref{def:functor}s and as result show that the
  \mintinline{haskell}{Maybe a} is also a
  \mynameref{def:functor}.
  \begin{minted}{haskell}
    data Maybe a = Nothing | Just a
    -- This is equivalent to
    data Maybe a = Either () (Identity a)
    -- Either is a bifunctor and () == Const () a
    -- Thus Maybe is a composition of 2 functors
  \end{minted}
\end{example}

\begin{definition}[Contravariant functor]
  \label{def:contravariant_functor}
  If we have categories $\cat{C}$ and $\cat{D}$ then the
  \mynameref{def:functor} $\cat{C^{op}} \tof \cat{D}$ is called
  \textit{contravariant functor}.
\end{definition}

\begin{example}[Contravariant functor][\textbf{Hask}]
  \label{ex:contravariant_functor_hask}
  TBD
  \begin{minted}{haskell}
    class Contravariant f where
    contramap :: (a -> b) -> f b -> f a
  \end{minted}
\end{example}

\begin{definition}[Profunctor]
  \label{def:profunctor}
  If we have a category $\cat{C}$ then the \mynameref{def:bifunctor}
  $\cat{C^{op}} \times \cat{C} \tof \cat{C}$ is called
  \textit{profunctor}.
\end{definition}

\begin{example}[Profunctor][\textbf{Hask}]
  \label{ex:contravariant_functor_hask}
  TBD
  \begin{minted}{haskell}
    class Profunctor p where
    dimap :: (a' -> a) -> ( b -> b' ) -> p a b -> p a' b'
    -- p a b == a -> b
    dimap f g h = g . h . f
  \end{minted}
\end{example}


\documentclass{article}

\title{\textbf{Second laboratory work}}
\author{Kondratev Egor 206}

\usepackage{sagetex}
\setlength{\sagetexindent}{10ex}

\usepackage{geometry}
\geometry{a4paper, total={170mm,257mm}, left=20mm, right=20mm, top=20mm, bottom=20mm}

\usepackage{xcolor}
\usepackage{titlesec}

\definecolor{myred1}{RGB}{107, 14, 14}
\definecolor{mygreen1}{RGB}{0, 102, 0}

\begin{document}

\maketitle
\titleformat{\section}[block]{\color{red}\Large\bfseries\filcenter}{}{1em}{}

\section*{Task 3 variant 1}

\textbf{Create some constants which will be useful in future for plot 3d graphics.}
\begin{sageblock}
xmin, xmax = -10, 10
ymin, ymax = -10, 10
zmin, zmax = -10, 10
\end{sageblock}
\textbf{Say for sage that \textcolor{mygreen1}{x},  \textcolor{mygreen1}{y}, \textcolor{mygreen1}{z} is variables, and create function \textcolor{myred1}{f}.}
\begin{sageblock}
var("x y z")
f(x, y, z) = 7*x^2 + 3*y^2 + 3*z^2 +  8*x*y + 8*x*z + 6*y*z + 6*x + y + 7
\end{sageblock}
\textbf{Display the function formula.}

\textcolor{myred1}{$\sage{f(x, y, z)}$}\\
\textbf{Plot graphic of unchanged figure.}
\begin{sageblock}
p = implicit_plot3d(f(x=x, y=y, z=z), (x, xmin, xmax), (y, ymin, ymax), (z, zmin, zmax))
\end{sageblock}

\begin{center}
\sageplot{p}
\end{center}
\textbf{Function that, according to a given algorithm, converts the function passed as an argument to the canonical form.
Return tuple (\textcolor{mygreen1}{lambdas}, \textcolor{mygreen1}{a}, \textcolor{mygreen1}{a0}).}
\begin{sageblock}
def kanonic_coeffs(fun):
    try:
        var("l l1 l2 l3")
        lvcts = []
        svcts = []
        tmp_fun = fun
        a = vector(RR, 3)
        tmp_vct = vector(RR, 9)
        var_combs_tmp = (x^2, y^2, z^2, x*y, x*z, y*z)
        var_combs_0 = (x^2, x*y, x*z, x*y, y^2, y*z, x*z, y*z, z^2)
        var_combs_1 = (x, y, z)
        for i, var_comb in enumerate(var_combs_0):
            if i == 0 or i == 4 or i == 8:
                tmp_vct[i] = fun.coefficient(var_comb)
            else:
                tmp_vct[i] = fun.coefficient(var_comb) / 2
        for var_comb in var_combs_tmp:
            tmp_fun -= fun.coefficient(var_comb)*var_comb
        for i, var_comb in enumerate(var_combs_1):
            a[i] = tmp_fun.coefficient(var_comb) / 2
            tmp_fun -= tmp_fun.coefficient(var_comb)*var_comb
        a0 = tmp_fun.n()
        A = matrix(SR, 3, tmp_vct)
        L = matrix(SR, 3, 3, var('l'))
        E = matrix(SR, 3, 3, 1)
        lvct = vector([l1, l2, l3])
        lambdas = solve([(A-L).determinant() == 0], l)
        for i, el in enumerate(lambdas):
            nums = []
            lhs = (A-el.rhs()*E)*lvct
            res = solve([lhs[0] == 0, lhs[1] == 0, lhs[2] == 0], l1, l2, l3)[0]
            for i in range(len(res)):
                if len(res[i].rhs().variables()) == 0:
                    nums.append(res[i].rhs())
                else:
                    nums.append(res[i].rhs()(1))
            lvcts.append(vector(nums))
        for el in lvcts:
            norm_lvct = (el / sqrt((el*el).n())).n()
            svcts.append(norm_lvct)
        ST = matrix(RR, 3)
        for i in range(len(svcts)):
            ST[i] = svcts[i]
        a_ = ST*a
        for i in range(len(lambdas)):
            lambdas[i] = lambdas[i].rhs().n()
        return (lambdas, a_, a0)
    except:
        print("Something gone wrong\n")
        return (None, None, None)
\end{sageblock}
\textbf{Obtain from the function a tuple of the required coefficients.}
\begin{sageblock}
(lambdas, a, a0) = kanonic_coeffs(f)
\end{sageblock}
\textbf{Display the obtained coefficients.}
\begin{sageblock}
print("Lambdas:", tuple(lambdas))
print("a:", a)
print("a0:", a0)
\end{sageblock}

\textbf{\textcolor{mygreen1}{Lambdas:}} $\sage{tuple(lambdas)}$

\textbf{\textcolor{mygreen1}{a:}} $\sage{a}$

\textbf{\textcolor{mygreen1}{a0:}} $\sage{a0}$\\
\textbf{Create a function with obtained coefficients.}
\begin{sageblock}
var("kx ky kz")
first_part = lambdas[0]*kx^2 + lambdas[1]*ky^2 + lambdas[2]*kz^2
second_part = 2*a[0]*kx + 2*a[1]*ky + 2*a[2]*kz + a0
kanonic_func(kx, ky, kz) = first_part + second_part
\end{sageblock}
\textbf{Display the obtained function formula.}

\textcolor{myred1}{$\sage{kanonic_func(kx, ky, kz)}$}\\
\textbf{Plot graphic of changed figure.}
\begin{sageblock}
p = implicit_plot3d(kanonic_func(kx=kx, ky=ky, kz=kz), (kx, xmin, xmax),\
(ky, ymin, ymax), (kz, zmin, zmax))
\end{sageblock}
\begin{center}
\sageplot{p}
\end{center}

\section*{Initial function.}
\begin{center}
\textcolor{myred1}{$7*x^2 + 3*y^2 + 3*z^2 +  8*x*y + 8*x*z + 6*y*z + 6*x + y + 7$}
\end{center}

\section*{Canonical function.}

\begin{center}
\textcolor{myred1}{$\sage{lambdas[0]}*kx^2+\sage{lambdas[1]}*ky^2 \sage{a[2]}*kz$}
\end{center}

\end{document}


\documentclass[11pt, oneside]{article}

\usepackage{../../shared/preamble}
\addbibresource{../../shared/references.bib}

\usepackage{../sets/sets}
\usepackage{topological-spaces}

\title{Topological Spaces}
\author{Arthur Ryman, {\tt arthur.ryman@gmail.com}}
\date{\today}

\begin{document}

\maketitle

\begin{abstract}
This article defines topological spaces and related concepts.
\end{abstract}

\section{Topological Spaces}

\subsection{$Topology$}

A {\it topology} $\tau$ on $X$ is a family of subsets of $X$, referred to as the {\it open} subsets of $X$, that satisfy the following axioms.

\begin{schema}{Topology}[X]
	\tau: \family X
\where
	\emptyset \in \tau
\also
	X \in \tau
\also
	\forall F: \finset \tau @ \bigcap F \in \tau
\also
	\forall F: \power \tau @ \bigcup F \in \tau
\end{schema}

\begin{itemize}
\item The empty set is open.
\item The whole set is open.
\item The intersection of a finite family of open sets is open.
\item The union of any family of open sets is open.
\end{itemize}

\subsection{$top$ and $tops$}

Let $top[X]$ denote the set of all topologies on $X$.

\begin{gendef}[X]
	top: \power(\family X)
\where
	top = \{~ Topology[X] @ \tau ~\}
\end{gendef}

Let $tops[X]$ denote the set of all topologies on subsets $U \subseteq X$.

\begin{gendef}[X]
	tops: \power(\family X)
\where
	tops = \bigcup \{~ U: \power X @ top[U] ~\}
\end{gendef}

\subsection{$discrete$ and $indiscrete$}

The {\it discrete} topology on $X$ consists of all subsets of $X$.
The {\it indiscrete} topology on $X$ consists of just $X$ and $\emptyset$.
Let $discrete[X]$ and $indiscrete[X]$ denote the discrete and indiscrete topologies on $X$.

\begin{gendef}[X]
	discrete, indiscrete: \family X
\where
	discrete = \power X
\also
	indiscrete =  \{ \emptyset, X \}
\end{gendef}

\begin{example}
Let $\setX$ be an arbitrary set.
Then $discrete[\setX]$ and $indiscrete[\setX]$ are topologies on $\setX$.

\begin{zed}
	discrete[\setX] \in top[\setX]
\also
	indiscrete[\setX] \in top[\setX]
\end{zed}

\end{example}

\subsection{$topGen$}

\begin{remark}

The intersection of a set of topologies on $X$ is also a topology on $X$.

\end{remark}

Given a family $B$ of subsets of $X$, the topology {\it generated by} $B$ is the intersection of all
topologies that contain $B$.
The set $B$ is referred to as a {\it basis} for the topology it generates.
Let $topGen[X]~B$ denote the topology on $X$ generated by the basis $B$.

\begin{gendef}[X]
	topGen: \family X \fun top[X]
\where
	\forall B: \family X @ \\
	\t1	topGen~B = \bigcap \{~ \tau: top[X] | B \subseteq \tau ~\}
\end{gendef}

\begin{example}
Let $\setX$ be an arbitrary set.

\begin{zed}
	topGen[\setX] \emptyset = indiscrete[\setX]
\also
	topGen[\setX] \{ \emptyset \} = indiscrete[\setX]
\also
	topGen[\setX] \{ \setX \} = indiscrete[\setX]
\end{zed}

\end{example}

\subsection{$topSpace$}

Let $X$ be a set.
A {\it topological space} is a pair $(X, \tau)$ where $\tau$ is a topology on $X$.
Let $topSpace[X]$ denote the set of all topological spaces $(X,\tau)$.

\begin{zed}
	topSpace[X] == \{~ \tau: top[X] @ (X, \tau) ~\}
\end{zed}

\begin{example}
Let $\setX$ be an arbitrary set.

\begin{zed}
	(\setX, indiscrete[\setX]) \in topSpace[\setX]
\also
	(\setX, discrete[\setX]) \in topSpace[\setX]
\end{zed}

\end{example}

\subsection{$topSpaces$}

Let $topSpaces[t]$ dentote the set of all topological spaces $(X,\tau)$ where $X$ is a subset of $t$.

\begin{gendef}[t]
	topSpaces: \power t \rel \family t
\where
	topSpaces = \{~ X: \power t; \tau: \family t | \tau \in top[X] ~\}
\end{gendef}

\begin{remark}

\begin{zed}
	topSpace[\setX] \subseteq topSpaces[\setX]
\end{zed}

\end{remark}

\section{Continuous Mappings}

Let $(X,\tau)$ and $(Y,\sigma)$ be topological spaces.

\subsection{$Continuous$}

A mapping $f \in X \fun Y$ is said to be {\it continuous} if the inverse image of every open set is open.

\begin{schema}{Continuous}[X,Y]
	f: X \fun Y \\
	\tau: top[X] \\
	\sigma: top[Y]
\where
	\forall U: \sigma @ \\
	\t1	f\inv\limg U \rimg \in \tau
\end{schema}

\subsection{\zcmd{CzeroTT}}

Let $A$ and $B$ be topological spaces, and
let $\CzeroTT(A,B)$ denote the set of continuous mappings from $A$ to $B$.

\begin{gendef}[X,Y]
	\CzeroTT: topSpace[X] \cross topSpace[Y] \fun \power (X \fun Y)
\where
	\forall \tau: top[X]; \sigma: top[Y] @ \\
	\t1	\LET A == (X, \tau); B == (Y, \sigma) @ \\
	\t2		\CzeroTT(A,B) = \{~ f: X \fun Y | Continuous[X,Y] ~\}
\end{gendef}

\subsection{The Identity Mapping}

\begin{remark}
The identity mapping is continuous.

\begin{zed}
	\forall \tau: top[\setX] @ \\
	\t1	\LET A == (\setX, \tau) @ \\
	\t2		\id \setX \in \CzeroTT(A, A)
\end{zed}

\end{remark}

\begin{remark}
The constant mapping is continuous.

\begin{zed}
	\forall \tau: top[\setX]; \sigma: top[\setY]; c: \setY @ \\
	\t1	\LET A == (\setX, \tau); B == (\setY, \sigma) @ \\
	\t2		\const[\setX,\setY] c \in \CzeroTT(A,B)
\end{zed}

\end{remark}

\subsection{Composition of Continuous Mapping}

\begin{remark}
Let $\setX$, $\setY$, and $\setZ$ be arbitrary sets.
The composition of continuous mappings is a continuous mapping.

\begin{zed}
	\forall A: topSpace[\setX]; B: topSpace[\setY]; C: topSpace[\setZ] @ \\
	\t1	\forall f: \CzeroTT(A, B); g: \CzeroTT(B, C) @ \\
	\t2		g \circ f \in \CzeroTT(A, C)
\end{zed}

\end{remark}

\section{Induced Topology}

Let $A = (X, \tau)$ be a topological space and let $U \subseteq X$ be a subset.
The topology on $X$ {\it induces} a topology on $U$.
This topology is variously referred to as the {\it induced}, {\it relative}, or {\it subspace} topology on $U$.

\subsection{\zcmd{inducedFam}}

Let $\phi$ be a family of subsets of $X$ and let $U$ be a subset of $X$.
The family of subsets of $U$ {\it induced} by $\phi$ is the set of intersections of the members of $\phi$ with $U$.
Let $\phi \inducedFam U$ denote the family on $U$ induced by $\phi$.

\begin{gendef}[X]
	\_ \inducedFam \_:  \family X \cross \power X \fun \family X
\where
	\forall \phi: \family X; U: \power X @ \\
	\t1	\phi \inducedFam U = \{~ Y: \phi @ Y \cap U ~\}
\end{gendef}

\begin{remark}
If $\tau$ is a topology on $X$ then $\tau \inducedFam U$ is a topology on $U$.

\begin{zed}
	\forall \tau: top[\setX]; U: \power \setX @ \\
	\t1	\tau \inducedFam U \in top[U]
\end{zed}

\end{remark}

\subsection{\zcmd{inducedTopSp}}

Let $(X, \tau) \inducedTopSp U$ denote the corresponding induced topological space.

\begin{gendef}[X]
	\_ \inducedTopSp \_: topSpace[X] \cross \power X \fun topSpaces[X]
\where
	\forall \tau: top[X]; U: \power X @ \\
	\t1	(X, \tau) \inducedTopSp U = (U, \tau \inducedFam U)
\end{gendef}

\section{Product Topology}

Let $(X, \tau)$ and $(Y, \sigma) $ be topological spaces.
There is a natural topology on $X \cross Y$ generated by the products of the sets in $\tau$ and $\sigma$.

\subsection{\zcmd{prodFam}}

Let $X$ and $Y$ be sets and let $\phi$ and $\psi$ be families on them.
The product of these families is the family that consists of the products of the sets in them and is a family on $X \cross Y$.
Let $\phi \prodFam \psi$ denote the product of the families.

\begin{gendef}[X,Y]
	\_ \prodFam \_ : \family X \cross \family Y \fun \family(X \cross Y)
\where
	\forall \phi: \family~X; \psi: \family~Y @ \\
	\t1	\phi \prodFam \psi = \{~ U: \phi; V: \psi @ U \cross V ~\}
\end{gendef}

\begin{remark}

If $\tau$ and $sigma$ are topologies then $\tau \prodFam \sigma$ is not, in general, a topology.
However, we can use it to generate a topology.

\end{remark}

\subsection{\zcmd{prodTop}}

Let $\tau \prodTop \sigma$ denote the topology generated by $\tau \prodFam \sigma$.

\begin{gendef}[X,Y]
	\_ \prodTop \_: top[X] \cross top[Y] \fun top[X \cross Y]
\where
	\forall \tau: top[X]; \sigma: top[Y] @ \\
	\t1	\tau \prodTop \sigma = topGen(\tau \prodFam \sigma)
\end{gendef}

\subsection{\zcmd{prodTopSp}}

Let $(X, \tau) \prodTop (Y, \sigma)$ denote the product topological space.

\begin{gendef}[X,Y]
	\_ \prodTopSp \_: topSpace[X] \cross topSpace[Y] \fun topSpace[X \cross Y]
\where
	\forall \tau: top[X]; \sigma: top[Y] @ \\
	\t1	(X, \tau) \prodTopSp (Y, \sigma) = (X \cross Y, \tau \prodTop \sigma)
\end{gendef}

\printbibliography

\end{document}


\chapter{Distance Oracles}

In this chapter, we learn about distance oracles as presented in the seminal paper \cite{thorup2005approximate}. Distance Oracles are data structures that allow for any undirected graph $G =(V,E)$ to be stored compactly in a format that allows to query for the (approximate) distance between any two vertices $u,v$ in the graph. The main result of this chapter is the following data structure.

\begin{theorem}\label{thm:mainDistanceOracle}
There is an algorithm that, for any integer $k \geq 1$ and undirected graph $G=(V,E)$, computes a data structure that can be stored using $\tilde{O}(kn^{1+1/k})$ bits such that on querying any two vertices $u,v \in V$ returns in $O(k)$ time a distance estimate $\widetilde{\mathbf{dist}}(u,v)$ such that
\[
\mathbf{dist}(u,v) \leq \widetilde{\mathbf{dist}}(u,v) \leq (2k-1) \cdot \mathbf{dist}(u,v).
\]
The algorithm computes the data structure in expected time $\tilde{O}(kmn^{1/k})$.
\end{theorem}

\begin{remark}Note that for $k=1$, the theorem above is trivial: it can be solved by computing APSP and storing the distance matrix of $G$.
\end{remark}

\begin{remark}
We point out that given space $O(n^{1+1/k})$, approximation $(2k-1)$ is the best that we can hope for according to a popular and widely believed conjecture that essentially says that there are unweighted graphs that have no cycle of length $(2k+1)$ but have $\tilde{\Omega}(n^{1+1/k})$ edges. A more careful analysis than we will carry out allows to shave all logarithmic factors from \Cref{thm:mainDistanceOracle} and therefore the data structure is only a factor $k$ off in space from optimal while also answering queries \emph{extremely} efficiently. It turns out that the factor $k$ can also be removed in space and query time (although currently preprocessing is quite expensive), see therefore the following (really involved) articles \cite{chechik2014approximate, chechik2015approximate}.
\end{remark}

\begin{remark}
Also note that in directed graphs no such distance oracle is possible. Even maintaining the transitive closure (the information of who reaches who) can only be preserved if one stores $\tilde{\Omega}(n^2)$ bits.
\end{remark}

\section{Warm-up: A Distance Oracle for $k=2$}

Let us first describe the data structure for the case where $k=2$. See therefore the pseudo-code below. Here we use the convention that $\mathbf{dist}(x,X)$ for some vertex $x \in V$ and some subset $X \subseteq V$ is the minimum distance from $x$ to any $y \in X$, formally $\mathbf{dist}(x,X) = \min_{y \in X} \mathbf{dist}(x,y)$.

\begin{algorithm}
  \SetAlgoLined
  Obtain $S$ by sampling every vertex $v \in V$ i.i.d. with probability $n^{-1/2}$\;
  \ForEach{$s \in S$}{
    Compute all distances from $s$ to any other vertex $v \in V$\;
    In a hash table $\mathcal{H}_s$ store for each $v \in V$ an entry with key $v$ and value $\mathbf{dist}_G(s,v)$.
  }

  \ForEach{$u \in V \setminus S$}{
    Find the \emph{pivot} $p(u)$ of $u$ to be some vertex in $S$ that minimizes the distance to $u$\;
    Store $p(u)$ along with $\mathbf{dist}_G(u,p(u)) = \mathbf{dist}_G(u,S)$\;
    Find the \emph{bunch} $B(u) = \{ v \in V \;|\; \mathbf{dist}_G(u,v) < \mathbf{dist}_G(u,S)\}$\;
    In a hash table $\mathcal{H}_u$ store for each $v \in B(v)$ an entry with key $v$ and value $\mathbf{dist}_G(u,v)$.
  }
  \caption{\textsc{Preprocess}(G)}
\end{algorithm}

The key to the algorithm is the definition of \emph{pivots} and \emph{bunches}. Below is an example that illustrates their definitions.

\begin{figure}[!ht]
    \centering
    \includegraphics[scale=0.2]{./fig/lecture_DistanceOracles_Bunch.jpeg}
    \caption{Graph $G$ (without the edges) where vertices are drawn according to their distance from $u$. The blue and red vertices are in $S$. The red vertex is chosen to be the pivot $p(u)$. Note that another vertex in $S$ could have been chosen to become the pivot. The bunch of $u$ is the vertices that are strictly withing the red circle. In particular, both blue vertices, the red vertex and also the white vertex on the boundary of the red circle are \emph{not} in the bunch $B(u)$.}
\end{figure}

Without further due, let us discuss the query procedure which is depicted below. It essentially consists of checking whether the vertex $v$ is already in the bunch of $u$ in which case we have stored the distance $\mathbf{dist}_G(u,v)$ explicitly. Otherwise, it uses a detour via its pivot.

\begin{algorithm}
  \SetAlgoLined
  \lIf{$v \in \mathcal{H}_u$}{
    \Return value $\mathbf{dist}_G(u,v)$
  }
  \Return $\mathbf{dist}_G(u,p(u)) + \mathbf{dist}_G(p(u),v)$ (the latter from $\mathcal{H}_{p(u)}$)
  \caption{\textsc{Query}(u,v)}
\end{algorithm}

The second case is illustrated below.

\begin{figure}[!ht]
    \centering
    \includegraphics[scale=0.25]{./fig/lecture_DistanceOracles_Query.jpeg}
\end{figure}

\paragraph{Approximation Analysis.} It is straight-forward to see that if we return in line 1 of the query algorithm, then we return the exact distance.

If we return in the second line, then we know that $v \not\in B(u)$. By definition of a bunch this implies that
\[
 \mathbf{dist}(u,v) \geq \mathbf{dist}(u,S) = \mathbf{dist}(u,p(u)).
\]
We can further use the triangle inequality to conclude that
\[
\mathbf{dist}_G(p(u),v) \leq \mathbf{dist}(u,p(u)) + \mathbf{dist}(u,v)
\]
Combining these two inequalities, we obtain
\[
\mathbf{dist}_G(u,p(u)) + \mathbf{dist}_G(p(u),v) \leq 2 \cdot \mathbf{dist}(u,p(u)) + \mathbf{dist}(u,v) \leq 3 \cdot \mathbf{dist}(u,v).
\]
We conclude that we obtain a $3$-approximation.

\paragraph{Space Analysis.} For each vertex $s \in S$, we store a hash-table with one entry for each vertex $v$. This can be stored with space $O(|S|n)$. We have by a standard Chernoff-bound that $|S| = \tilde{O}(\sqrt{n})$ w.h.p. so this becomes $\tilde{O}(n^{3/2})$.

Next, fix some $u \in V \setminus S$, and let us argue about the size of $B(u)$ (which asymptotically matches $|\mathcal{H}_u|$). We order the vertices $v_1, v_2, \dots, v_n$ in $V$ by their distance from $u$. Since we sample uniformly at random, by a simple Chernoff bound, we obtain that the first vertex $v_i$ in $v_1, v_2, \dots, v_n$ that is in $S$ has $i = \tilde{O}(\sqrt{n})$ w.h.p.. But note that since only vertices that are \emph{strictly} closer to $u$ than $v_i$ are in $B(u)$, this implies that $|B(u)| = \tilde{O}(\sqrt{n})$.

It remains to take a careful union bound over all bad events at every vertex $u \in V \setminus S$ to conclude that with high probability, the hash tables $\mathcal{H}_u$ for all $u \in V \setminus S$ combined take total space $\tilde{O}(|V \setminus S| \sqrt{n}) = \tilde{O}(n^{3/2})$. For the rest of the section, we condition on the event that each $|B(v)| =\tilde{O}(\sqrt{n})$ to use it as if it was a deterministic guarantee.

\paragraph{Preprocessing and Query Time.} In order to find the distances stored in the hash tables $\mathcal{H}_s$ for $s \in S$, we can simply run Dijkstra from each $s \in S$ on the graph in total time $\tilde{O}(m|S|) = \tilde{O}(mn^{1/2})$.

To compute the \emph{pivots} for each vertex $u$, we can insert a super-vertex $s'$ and add an edge from $s'$ to each $s \in S$ of weight $0$. We can then run Dijkstra from $s'$ on $G \cup \{s'\}$. Note that for each $u$, we have $\mathbf{dist}_{G \cup \{s'\}}(s', u) = \mathbf{dist}_G(p(u),u)$ and that $p(u)$ can be chosen to be the closest vertex on the path from $s'$ to $u$ that Dijkstra outputs along with the distances (recall that Dijkstra can output a shortest path tree). This takes $\tilde{O}(m)$ time.

It remains to compute the bunches $B(u)$. Here, we use duality: we define the cluster $C(w)$ for every vertex $w \in V \setminus S$ to be the set
\[
    C(w) = \{ v \in V \;|\; \mathbf{dist}_G(v,w) < \mathbf{dist}_G(v,p(v))\}.
\]
Note the subtle difference to the bunches in that membership of $v$ now depends on $p(v)$ and \emph{not} on $p(u)$! It is not hard to see that $u \in C(w) \iff w \in B(u)$. And it is straight-forward to compute the bunches from the clusters in time $O(\sum_v |B(v)|) = O(\sum_w |C(w)|) = \tilde{O}(n^{3/2})$.

Finally, it turns out that we can compute each $C(w)$ by running Dijkstra with a small modification.

\begin{lemma}[Lemma 4.2 in \cite{thorup2005approximate}]
Consider running Dijkstra from a vertex $w$ but only relaxing edges incident to vertices $v$ that satisfy $\mathbf{dist}_G(v,w) < \mathbf{dist}_G(v,p(v))$. Then, the algorithm computes $C(w)$ and all distances $\mathbf{dist}(v,w)$ for $v \in C(w)$ in time $\tilde{O}(|E(C(w))|)$ where $E(C(w))$ are the edges that touch a vertex in $C(w)$.
\end{lemma}

It remains to observe that the total time required to compute all clusters is
\[
    \tilde{O}(\sum_{w} |E(C(w))|) = \tilde{O}(\sum_{w, v \in C(w)} |E(v)|) = \tilde{O}(\sum_{v, w \in B(v)} |E(v)|) = \tilde{O}(\sum_{v} |E(v)| |B(v)|).
\]
But we have upper bounded $|B(v)| = \tilde{O}(\sqrt{n})$ for all $v$, thus each vertex just pays its degree $\tilde{O}(\sqrt{n})$ times and we get running time $\tilde{O}(m\sqrt{n})$.

\paragraph{Monte Carlo vs. Las Vegas.} Note that the analysis above only guarantees that the algorithm works well with high probability. However, it is not hard to see that the algorithm can be transformed into a Las Vegas algorithm: whenever we find a bunch $B(v)$ whose size exceeds our $\tilde{O}(\sqrt{n})$ bound, we simply re-run the algorithm. This guarantees that the final data structure that we output indeed satisfies the guarantees stipulated in the theorem.

\section{Distance Oracles for any $k \geq 2$}

The generalization for all $k$'s is rather straight-forward except for the query operation which works a bit magically. Let's first define the data structure by giving our new pre-processing algorithm.

\begin{algorithm}
  \SetAlgoLined
  $S_1 = V$; $S_{k+1} = \emptyset$\;
  \lForEach{$i \in [1,k]$}{
    Obtain $S_{i+1}$ by sampling every $v \in S_{i}$ i.i.d. with prob. $n^{-1/k}$
  }

    \ForEach{$u \in V$}{
        \ForEach{$i \in [1,k]$}{Let $p_{i}(u)$ be some vertex in $S_{i}$ that minimizes the distance to $u$\;
        Store $p_{i}(u)$ along with $\mathbf{dist}_G(u,p_{i}(u))$. }
        Let \emph{bunch} $B(u) = \bigcup_i B_i(u)$ where $B_i(u) = \{ v \in S_i \;|\; \mathbf{dist}_G(u,v) < \mathbf{dist}_G(u,S_{i+1})\}$\;
        In a hash table $\mathcal{H}_u$ store for each $v \in B(v)$ an entry with key $v$ and value $\mathbf{dist}_G(u,v)$.
     }
  \caption{\textsc{Preprocess}(G)}
\end{algorithm}

Note that in a sense this algorithm is almost easier than the one for $k=2$ since it treats each level in the same fashion. Here we ensure that the last set $S_{k+1}$ is empty (which would happen with constant probability otherwise).

We make the implicit assumption throughout that $S_k \neq \emptyset$ so that $p_k(u)$ is well-defined for each $u$. We also define $\mathbf{dist}(x,X) = \infty$ if $X$ is the empty set.

The drawing below illustrates the new definition of a bunch where we have chosen $k=3$ to keep things simple.

\begin{figure}[!ht]
    \centering
    \includegraphics[scale=0.25]{./fig/lecture_DistanceOracles_MultiLevelBunch.jpg}
    \caption{Graph $G$ (without the edges) where vertices are drawn according to their distance from $u$. All vertices are in $S_1$. The blue and red vertices are in $S_2$. The dark blue and dark red vertices are also in $S_3$. Finally, we have $S_4 = \emptyset$.
    The light red vertex is chosen as pivot $p_2(u)$; the dark red vertex is chosen as pivot $p_3(u)$.
    The bunch $B(u)$ includes all vertices that have a black edge in this graph to $u$; in particular these are the white vertices within the circle drawn in light red ($B_1(u)$); the light blue vertices encircled by the dark red circle ($B_2(u)$); and all dark blue vertices ($B_3(u)$).}
\end{figure}

Before we explain the query procedure, let us give a (rather informal) analysis of the space required by our new data structure.

\paragraph{Space Analysis.} We have for each vertex $u \in V$, and each $1 \leq i \leq k$ that the bunch $B_i(u)$ consists of all vertices in $S_i$ that are closer to $u$ than the closest vertex in $S_{i+1}$.

Now, order the vertices $x_1, x_2, \dots$ in $S_i$ by their distance from $u$. Since each vertex in $S_i$ is sampled into $S_{i+1}$ with probability $n^{-1/k}$, we have that with high probability some vertex $x_i$ with $i = O(n^{1/k} \log n)$ is sampled into $S_{i+1}$. This ensures that $|B_i(u)| = \tilde{O}(n^{1/k})$ with high probability.

Applying this argument for all $i$, we have that $|B(u)| = \tilde{O}(k \cdot n^{1/k})$ for each $u$ w.h.p. and therefore our space bound follows.

\paragraph{Preprocessing Time.} Much like in the $k=2$ construction, we can define for each $u \in S_i \setminus S_{i+1}$ the cluster $C(u) = \{ v \in V \;|\; \mathbf{dist}_G(u,v) < \mathbf{dist}_G(u,S_{i+1})\}$. Extending our analysis from before using this new definition, we get construction time $\tilde{O}(kmn^{1/k})$.

\paragraph{Query Operation.} A straight-forward way to query our new data structure for a tuple $(u,v)$ would be to search for the smallest $i$ such that $v \in B(p_i(u))$ and then return $\mathbf{dist}_G(u, p_i(u)) + \mathbf{dist}_G(p_i(u), v)$. This can be analyzed in the same way as we did for $k=2$ to obtain stretch $4k-3$\footnote{One can actually prove that this strategy gives an $4k-5$ stretch with a little trick.}.

However, we aim for stretch approximation $2k-1$. We state below the pseudo-code to achieve this guarantee.

\begin{algorithm}
  \SetAlgoLined
  $w \gets u$; $i \gets 1$\;
  \While{$w \not\in B(v)$}{
    $i \gets i+1$\;
    $(u,v) \gets (v,u)$\;
    $w \gets p_i(u)$
  }
  \Return $\mathbf{dist}_G(u,w) + \mathbf{dist}_G(w,v)$
  \caption{\textsc{Query}(u,v)}
\end{algorithm}

Our main tool in the analysis is the claim below where we define $\Delta = \mathbf{dist}_G(u,v)$.

\begin{claim}\label{claim:DistanceOracleSwapping} After the $i^{th}$ iteration of the while-loop, we have $\mathbf{dist}_G(u,w) \leq i\Delta$.
\end{claim}

This implies our theorem, since we have at most $k-1$ iterations, then $w$ is a vertex in $S_k$ and $S_k \subseteq B(x)$ for all vertices $x \in V$. Therefore we have that for the final $w$, we have $\mathbf{dist}_G(u,w) \leq (k-1)\Delta$. It remains to conclude by the triangle inequality that
\[
\mathbf{dist}_G(u,w) + \mathbf{dist}_G(w,v) \leq 2\mathbf{dist}_G(u,w) + \Delta \leq (2k-1)\mathbf{dist}_G(u,v).
\]

\begin{proof}[Proof of \Cref{claim:DistanceOracleSwapping}]
Let $w_i, u_i, v_i$ denote the variables $w,u,v$ after the $(i+1)^{th}$ while-loop iteration (or right before for $w_0,u_0,v_0$).

For $i = 0$, we have that $w_0 = u_0$; thus, $\mathbf{dist}_G(u_0,w_0) = 0$.

For $i \geq 1$, we want to prove that if the $i^{th}$ while-loop iteration is executed then $\mathbf{dist}_G(u_i,w_i) \leq \mathbf{dist}_G(u_{i-1},w_{i-1}) + \Delta$ (if it is not executed then the statement follows trivially).

In order to prove this, observe that by the while-loop condition, we must have had $w_{i-1} \not\in B(v_{i-1})$, thus  $\mathbf{dist}_G(v_{i-1}, w_{i-1}) \geq \mathbf{dist}_G(v_{i-1}, p_i(v_{i-1}))$.

But the while-iteration sets $u_i = v_{i-1}$ and $w_i = p_i(v_{i-1})$, and therefore we have
\begin{align*}
\mathbf{dist}_G(u_{i}, w_i) &= \mathbf{dist}_G(v_{i-1}, p_i(v_{i-1})) \leq
\mathbf{dist}_G(v_{i-1}, w_{i-1}) = \mathbf{dist}_G(v_{i-1}, p_{i-1}(u_{i-1})) \\
&\leq \mathbf{dist}_G(u_{i-1}, p_{i-1}(u_{i-1})) + \mathbf{dist}_G(v_{i-1}, u_{i-1}) = \mathbf{dist}_G(u_{i-1}, w_{i-1}) + \Delta.
\end{align*}
\end{proof}


\subsection{part b}
$$\vec X_0 = \begin{bmatrix}
	-1\\1
\end{bmatrix}$$
Tolerance is: $10^{-7}$

$$\vec X_{ans} = \begin{bmatrix}
	-1.7556\\
	0.3655
\end{bmatrix}$$
\subsubsection{figures}
\newpage
\begin{itemize}
	\item Steepest Descent
	\begin{itemize}
		\item Quadratic Interpolation
		\begin{figure}[H]
			\caption{Steepest Descent and Quadratic Interpolation}
			\centering
			\includegraphics[width=11.5cm]{../Figure/Q1/part b Steepest Descent + Quadratic Interpolation.png}
		\end{figure}
		\item Golden Section
		\begin{figure}[H]
			\caption{Steepest Descent and Golden Section}
			\centering
			\includegraphics[width=11.5cm]{../Figure/Q1/part b Steepest Descent + Golden Section.png}
		\end{figure}
	\end{itemize}
	\item BFGS
	\begin{itemize}
		\item Quadratic Interpolation
		\begin{figure}[H]
			\caption{BFGS and Quadratic Interpolation}
			\centering
			\includegraphics[width=11.5cm]{../Figure/Q1/part b BFGS + Quadratic Interpolation.png}
		\end{figure}
		\item Golden Section
		\begin{figure}[H]
			\caption{BFGS and Golden Section}
			\centering
			\includegraphics[width=11.5cm]{../Figure/Q1/part b BFGS + Golden Section.png}
		\end{figure}
	\end{itemize}
\end{itemize}
\subsubsection{result}
\begin{itemize}
	\item Time
	\begin{table}[H]
		\caption {Time compare between four methods}
		\begin{center}
			\begin{tabular}{ |c|c|c|c| }
				\hline
				\multicolumn{2}{|c|}{Steepest Descent} &
				\multicolumn{2}{|c|}{BFGS} \Tstrut\\
				\hline
				Quadratic Interpolation & Golden Section & Quadratic Interpolation &
				Golden Section \Tstrut\\
				\hline
				$0.208\sec$ & $0.146\sec$ & $0.106\sec$ & $0.142\sec$\Tstrut\\
				\hline
			\end{tabular}
		\end{center}
	\end{table}
	\item Number of Cost calculation
	\begin{table}[H]
		\caption {Number of Cost calculation compare between four methods}
		\begin{center}
			\begin{tabular}{ |c|c|c|c| }
				\hline
				\multicolumn{2}{|c|}{Steepest Descent} &
				\multicolumn{2}{|c|}{BFGS} \Tstrut\\
				\hline
				Quadratic Interpolation & Golden Section & Quadratic Interpolation &
				Golden Section \Tstrut\\
				\hline
				$246$ & $285$ & $142$ & $142$\Tstrut\\
				\hline
			\end{tabular}
		\end{center}
	\end{table}
	\item Number of Gradient calculation
	\begin{table}[H]
		\caption {Number of Gradient calculation compare between four methods}
		\begin{center}
			\begin{tabular}{ |c|c|c|c| }
				\hline
				\multicolumn{2}{|c|}{Steepest Descent} &
				\multicolumn{2}{|c|}{BFGS} \Tstrut\\
				\hline
				Quadratic Interpolation & Golden Section & Quadratic Interpolation &
				Golden Section \Tstrut\\
				\hline
				$14$ & $12$ & $7$ & $7$\Tstrut\\
				\hline
			\end{tabular}
		\end{center}
	\end{table}
\end{itemize}


\documentclass[12pt]{article}

\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{float}
\usepackage{ulem}
\usepackage{bm}

\usepackage{anysize}

\marginsize{2cm}{2cm}{0.9cm}{1.8cm}
\usepackage[framed,numbered,autolinebreaks,useliterate]{mcode}
\usepackage{listings}
\lstset{language=Matlab}
\lstset{breaklines}
\lstset{extendedchars=false}

\title{Machine Learning/Pattern Recognition\\  \begin{Large} Homework \#1 \end{Large} }
\author{Jiyu Tian}
\date{}

\begin{document}

\maketitle
%%---------------------------------------------------------------
%% Problem 1
%%---------------------------------------------------------------
\section{Problem 1}
(a) Since $\int_{-\infty}^{+\infty} p(x|w_i) \text{ d}x =1$, then we have
\begin{equation*}
\begin{aligned}
\int_{-\infty}^{+\infty} p(x|w_i) \text{ d}x &=\int_{-\infty}^{a_i} A e^{(x-a_i)/b_i} \text{ d}x \ + \int_{a_i}^{+\infty} A e^{-(x-a_i)/b_i} \text{ d}x \\
&=Ab_i[\left.e^{(x-a_i)/b_i} \right|^{a_i} _{-\infty} \left.-e^{-(x-a_i)/b_i} \right|^{+\infty} _{a_i} \ ]\\
&=Ab_i[1-(-1)]=2Ab_i=1
\end{aligned}
\end{equation*}
So $A=\frac{1}{2b_i}$. Then the normalized analytic expression is:
\begin{equation*}
p(x|w_i)=\frac{1}{2b_i} e^{-|x-a_i|/b_i}\ \ \  \text{for}\ \  i=1,2 \ \  \text{and} \ \ b\geq 0
\end{equation*}
(b) The likelihood ratio is given by:
\begin{equation*}
\begin{aligned}
\frac {p(x|w_1)}{p(x|w_2)} &=(\frac{1}{2b_1} e^{-|x-a_1|/b_1})/(\frac{1}{2b_2} e^{-|x-a_2|/b_2})\\
&= \ \frac{b_2}{b_1} e^{-|x-a_1|/b_1+|x-a_2|/b_2}
\end{aligned}
\end{equation*}
(c) When $a_1=1,b_1=1,a_2=2,b_2=2$, we have:
\begin{equation*}
\frac {p(x|w_1)}{p(x|w_2)}=\ 2e^{-|x-1|+\frac{|x-2|}{2}}
\end{equation*}
Using the following MATLAB code, the likelihood ratio is plotted as Figure \ref{fig1}:
\begin{lstlisting}
clear;clc

x = -2:0.00001:5;
y = 2 * exp( abs(x-2) / 2 - abs(x-1) );

hold on
plot(x, y, 'k')
plot(2, 2 * exp (-1), 'ro')
plot(1, 2 * exp (0.5), 'ro')
hold off
title('$Likelihood\ Ratio\ as\ a\ function\ of\ x$', 'Interpreter', 'latex')
xlabel('$x$', 'Interpreter', 'latex')
ylabel('$Likelihood\ Ratio$', 'Interpreter', 'latex')
grid on
\end{lstlisting}

\begin{figure}[H]
\centering
\includegraphics[width=0.9\textwidth]{11c.jpg}
\caption{\label{fig1}Likelihood Ratio.}
\end{figure}

%%---------------------------------------------------------------
%% Problem 2
%%---------------------------------------------------------------
\section{Problem 2}
(a) Given $P(w_1)=P(w_2)$, we have $P(w_1)=P(w_2)=\frac{1}{2}$. So, solving for the Bayes optimal decision boundary is a matter of solving for the roots of the equation below:\\
\begin{equation*}
\begin{aligned}
p(w_1|x)&=p(w_2|x)\\
\frac{p(x|w_1)P(w_1)}{P(x)} & =\frac{p(x|w_2)P(w_2)}{P(x)}\\
p(x|w_1)&=p(x|w_2)\\
\frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2} x^2} & = \frac{1}{\sqrt{2\pi}\sigma} e^{-\frac{1}{2\sigma^2} (x-\mu)^2} \\
\sigma e^{-\frac{1}{2} x^2} & = e^{-\frac{1}{2\sigma^2} (x-\mu)^2} \\
ln(\sigma)-\frac{1}{2} x^2 & =-\frac{1}{2\sigma^2} (x-\mu)^2 \\
2\sigma^2ln(\sigma)-\sigma^2 x^2 & = -(x-\mu)^2 \\
\end{aligned}
\end{equation*}
\begin{equation*}
(\sigma^2-1)x^2+2\mu x-\mu^2-2\sigma^2ln\sigma =0
\end{equation*}

\noindent The decision boundary is found by solving for the roots of the quadratic:
\begin{equation*}
x_{1,2}=\frac{-\mu \pm \sqrt{\mu^2\sigma^2+2\sigma^2(\sigma^2-1)ln\sigma}}{\sigma^2-1}
\end{equation*}
\vfill
\clearpage
% NEW page----------------------------------------------------------------
\noindent (b) With $\mu_1=0$, $\sigma_1=1$, $\mu_2=2$ and $\sigma_2^2=2$, we have the class conditional pdfs:\\
\begin{equation*}
\begin{aligned}
& p(x|w_1)=\frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2} x^2} \\
& p(x|w_2)=\frac{1}{2\sqrt{\pi}} e^{-\frac{1}{4} (x-2)^2}
\end{aligned}
\end{equation*}

\noindent And the posterior probabilities:
\begin{equation*}
\begin{aligned}
& p(w_1|x)=\frac{p(x|w_1)P(w_1)}{P(x)}=\frac{1}{2\sqrt{2\pi}P(x)}    \ e^{-\frac{1}{2} x^2}\\
& p(w_2|x)=\frac{p(x|w_2)P(w_2)}{P(x)}=\frac{1}{4\sqrt{\pi}P(x)}    \ e^{-\frac{1}{4} (x-2)^2}\\
\end{aligned}
\end{equation*}
where $P(x)$ is given by $P(x)=\sum^2_{i=1}P(x|w_i)P(w_i)=\frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2} x^2} + \frac{1}{2\sqrt{\pi}} e^{-\frac{1}{4} (x-2)^2}$, and $x_1=-5.0637,\ x_2=1.0637$.\\
\noindent Using the MATLAB code below we can plot Figure \ref{fig2}:
\begin{lstlisting}
clear; clc; close all
x = -10 : 0.0001 : 8;
y1 = exp (-0.5 * x.^2) / sqrt (2 * pi);
y2 = exp (-0.25 * (x-2).^2) / (2 * sqrt (pi) );
k1 = y1./ (y1 + y2);
k2 = y2./ (y1 + y2);
mu = 2;
sigma = sqrt (2);
delt = mu^2 + ( sigma^2 - 1 ) * ( mu^2 + 2 * sigma^2 * log (sigma) );
xopt1 = ( - mu - sqrt ( delt ) ) / ( sigma^2 - 1 );
xopt2 = ( - mu + sqrt ( delt ) ) / ( sigma^2 - 1 );
figure
subplot(2, 1, 1)
hold on
plot (x, y1, 'r', x, y2, 'k');
plot([xopt1 xopt1], [-1 2], 'k-.', [xopt2 xopt2], [-1 2], 'k-.')
hold off
axis ( [-6 8 -0.05 0.5] );
title ('$Class\ Conditional\ PDFs$', 'Interpreter', 'latex')
xlabel ('$x$', 'Interpreter', 'latex')
ylabel ('$pdf$', 'Interpreter', 'latex')
h = legend('$P(x|w_1)$', '$P(x|w_2)$', 'Boundary');
set (h, 'Interpreter', 'latex')
grid on
subplot(2, 1, 2)
hold on
plot (x, k1, 'r', x, k2, 'k')
plot([xopt1 xopt1], [-1 2], 'k-.', [xopt2 xopt2], [-1 2], 'k-.')
hold off
axis( [-10 6 -0.05 1.05] );
title('$Posterior\ Proabilities\ with\ Optimal\ Decision\ Regions$', 'Interpreter', 'latex')
xlabel ('$x$', 'Interpreter', 'latex')
ylabel ('$Posterior\ Proabilities$', 'Interpreter', 'latex')
h = legend ('$P(w_1|x)$', '$P(w_2|x)$', 'Boundary');
set (h, 'Interpreter', 'latex');
grid on
\end{lstlisting}
\vfill
\clearpage

\begin{figure}
\centering
\includegraphics[width=0.9\textwidth]{12b.jpg}\caption{\label{fig2}Conditional PDFs and Posterior Probabilities}
\end{figure}

\noindent (c) With the decision regions $C_1:(x_1,x_2]\ ,\ C_2:(-\infty, x_1]\ \bigcup\ [x_2,+\infty)$, we can calculate the Bayes error rate:
\begin{equation*}
\begin{aligned}
p_e&=\int^{+\infty}_{-\infty}p(err|x)p(x)\text{ d}x\\
&=\int_{C_2}p(w_1|x)p(x)\text{ d}x+\int_{C_1}p(w_2|x)p(x)\text{ d}x\\
&=\int_{C_2}p(x|w_1)P(w_1)\text{ d}x+\int_{C_1}p(x|w_2)P(w_2)\text{ d}x\\
&=\frac{1}{2} \int^{x_1}_{-\infty} \frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2} x^2} \text{ d}x + \frac{1}{2} \int^{+\infty}_{x_2} \frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2} x^2} \text{ d}x +   \frac{1}{2} \int^{x_2}_{x_1}\frac{1}{2\sqrt{\pi}} e^{-\frac{1}{4} (x-2)^2} \text{ d}x\\
&=\frac{1}{2} (\ Q(-x_1)+Q(x_2)+Q(\frac{x_1-2}{\sqrt{2}})-Q(\frac{x_2-2}{\sqrt{2}})\ )
\end{aligned}
\end{equation*}
where $Q(x)=\int^{+\infty}_x \frac{1}{\sqrt{2\pi}} e^{ -\frac{1}{2} x^2} \text{ d}x,\ x_1\approx-5.0637,\ x_2\approx1.0637$.\\
\\With MATLAB the $p_e$ can be calculated:
\begin{lstlisting}
clear;clc
mu = 2;
sigma = sqrt (2);
delta = mu^2 + ( sigma^2 - 1 ) * ( mu^2 + 2 * sigma^2 * log (sigma) );
x1 = ( - mu - sqrt ( delta ) ) / ( sigma^2 - 1 );
x2 = ( - mu + sqrt ( delta ) ) / ( sigma^2 - 1 );
pe = ( qfunc (-x1) + qfunc (x2) + qfunc ( (x1 - 2) / sigma ) -...
    qfunc ( ( x2 - 2 ) / sigma ) ) /2
\end{lstlisting}
and we have $p_e\approx 0.1988$.\\
\\(d) The likelihood $p(x|w_1)$ is centralized within a small region, while the other one, $p(x|w_2)$, is nearly uniformly distributed on a large region, it occurs to me that the classification of the image of nebula and star could be a sample of this. Another situation is classifying zero mean signals of different variance.\\

%%---------------------------------------------------------------
%% Problem 3
%%---------------------------------------------------------------
\section{Problem 3}
The function can be realized with the following code (2-CLASS, 2-DMENSIONAL) \footnote{Actually, in order to use the same dataset generated by the function in Problem 3 and 4, all six figures in this section are sketched in Problem 4.}:
\begin{lstlisting}
function [data, classIndex] = generateGaussianSamples(mu, sigma, nSamples, prior);
%
% Function to simulate data from k Gaussian densities
% (1 for each class) in d dimensions.
%
% INPUTS:
% mu - k-by-1 cell with the class dependent d-dimensional mean vector
% sigma - k-by-1 cell with the class dependent d-dimensional conv matrix
% nSamples - scalar indicaitng number of samples to be generated
% prior - k-by-1 vector with class dependent mean
%
% OUTPUTS:
% data - nSamples-by-d array with the simulated data distributed along the rows
% classIndex - vector of length nSamples with the class index for each datapoint

% Generating datas and outputs.
nuclas1 = binornd(nSamples, prior(1));
data1 = mvnrnd ( mu{1}, sigma{1}, nuclas1);
data2 = mvnrnd ( mu{2}, sigma{2}, nSamples - nuclas1 );
sample = [data1, ones(nuclas1, 1); data2, 2 * ones(nSamples - nuclas1, 1)];
data = sample(:, 1:2);
classIndex = sample(:, 3);
% Plots.
figure
hold on
plot(data1(:,1), data1(:,2), 'r.')
plot(data2(:,1), data2(:,2), 'k.')
xlabel('$x_1$', 'Interpreter', 'latex')
ylabel('$x_2$', 'Interpreter', 'latex')
hold off
title('$I.I.D\ Gaussian\ Samples\ of\ Two-classes.$', 'Interpreter', 'latex')
h=legend('$Class\ 1$', '$Class\ 2$');
set(h, 'Interpreter', 'latex');
grid on
end
\end{lstlisting}

\noindent To simulate the datasets, we first sample the respective class indexes with binomial distribution, $b(nSamples,\ 0.5)$. Then with the class dependent parameters we can sample the gaussian dataset using function $mvnrnd()$. MATLAB code to generate and plot are attached at the end.\\

\noindent (a) The given parameters are:
\begin{lstlisting}
mu = {[0,0];[-3,-3]}; sigma = {eye(2) ; eye(2)};
nSamples = 400; prior = [0.5, 0.5];
\end{lstlisting}

\noindent (b) The given parameters are:
\begin{lstlisting}
mu = {[0,0];[-3,-3]}; sigma = {[3 1; 1 0.8] ; [3 1; 1 0.8]};
nSamples = 400; prior = [0.5, 0.5];
\end{lstlisting}

\noindent (c) The given parameters are:
\begin{lstlisting}
mu = {[0,0];[-3,-3]}; sigma = {[2 0.5; 0.5 1] ; [2 -1.9; -1.9 5]};
nSamples = 400; prior = [0.5, 0.5];
\end{lstlisting}

\noindent (d) The given parameters are:
\begin{lstlisting}
mu = {[0,0];[-3,-3]}; sigma = {[3 1; 1 0.8] ; [3 1; 1 0.8]};
nSamples = 400; prior = [0.05, 0.95];
\end{lstlisting}

\noindent (e) The given parameters are:
\begin{lstlisting}
mu = {[0,0];[-3,-3]}; sigma = {[3 1; 1 0.8] ; [3 1; 1 0.8]};
nSamples = 400; prior = [0.05, 0.95];
\end{lstlisting}

\noindent (f) The given parameters are:
\begin{lstlisting}
mu = {[0,0];[-3,-3]}; sigma = {[2 0.5; 0.5 1] ; [2 -1.9; -1.9 5]};
nSamples = 400; prior = [0.05, 0.95];
\end{lstlisting}

\begin{figure}
\centering
\subfigure[]{\includegraphics[width=0.45\textwidth]{13a.jpg}}
%\mbox{\hspace{0.5cm}}
\subfigure[]{\includegraphics[width=0.45\textwidth]{13b.jpg}}
\\
\subfigure[]{\includegraphics[width=0.45\textwidth]{13c.jpg}}
%\mbox{\hspace{0.5cm}}
\subfigure[]{\includegraphics[width=0.45\textwidth]{13d.jpg}}
\\
\subfigure[]{\includegraphics[width=0.45\textwidth]{13e.jpg}}
%\mbox{\hspace{0.5cm}}
\subfigure[]{\includegraphics[width=0.45\textwidth]{13f.jpg}}
\caption{\label{fig3}Dataset generated in Problem 1.3}
\end{figure}

\vfill
\clearpage

%%---------------------------------------------------------------
%% Problem 4
%%---------------------------------------------------------------
\section{Problem 4}
\textbf{CASE 1} : $\bm{\Sigma}_i=\sigma^2\bm{\text{I}}$\\
The discriminant function is given by  $g_i(\textbf{x})=\textbf{w}_i^T\textbf{x}+w_{i0}$, where
\begin{equation*}
\textbf{w}_i=\frac{1}{\sigma^2} \bm{\mu}_i,\ \
w_{i0}=-\frac{1}{2\sigma^2} \bm{\mu}_i^T \bm{\mu}_i+\text{ln } P(w_i)
\end{equation*}
And the boundary is $\textbf{w}^T(\textbf{x}-\textbf{x}_0)=0$, where
\begin{equation*}
\textbf{w}=\bm{\mu}_i-\bm{\mu}_j,\ \ \bm{x}_0=\frac{1}{2}(\bm{\mu}_i+\bm{\mu}_j)-\frac{\sigma^2}{||\bm{\mu}_i-\bm{\mu}_j||_2^2}\text{ln }\frac{P(w_i)}{P(w_j)}(\bm{\mu}_i-\bm{\mu}_j)
\end{equation*}
\\
\noindent \textbf{CASE 2} : $\bm{\Sigma}_i=\bm{\Sigma}$\\
The discriminant function is then  $g_i(\textbf{x})=\textbf{w}_i^T\textbf{x}+w_{i0}$, where
\begin{equation*}
\textbf{w}_i=\bm{\Sigma}^{-1}\bm{\mu}_i,\ \
w_{i0}=-\frac{1}{2}  \bm{\mu}_i^T \bm{\Sigma}^{-1} \bm{\mu}_i+\text{ln }P(w_i).
\end{equation*}
And the boundary is $\textbf{w}^T(\textbf{x}-\textbf{x}_0)=0$, where
\begin{equation*}
\textbf{w}=\bm{\Sigma}^{-1}(\bm{\mu}_i-\bm{\mu}_j),\ \ \bm{x}_0=\frac{1}{2}(\bm{\mu}_i+\bm{\mu}_j)-\frac{\text{ln }P(w_i)/P(w_j)}{(\bm{\mu}_i-\bm{\mu}_j)^T\bm{\Sigma}^{-1}(\bm{\mu}_i-\bm{\mu}_j)}(\bm{\mu}_i-\bm{\mu}_j).
\end{equation*}
\\
\noindent \textbf{CASE 3} : $\bm{\Sigma}_i=$ \textbf{arbitrary}\\
The linear discriminant function is given by  $g_i(\textbf{x})=\textbf{x}^T\bm{W}_i\textbf{x}+\textbf{w}_i^T\textbf{x}+w_{i0}$, where
\begin{equation*}
\bm{W}_i=-\frac{1}{2}\bm{\Sigma}_i^{-1},\ \
\textbf{w}_i=\bm{\Sigma}_i^{-1}\bm{\mu}_i,\ \
w_{i0}=-\frac{1}{2}  \bm{\mu}_i^T \bm{\Sigma}_i^{-1} \bm{\mu}_i -\frac{1}{2}\text{ln } |\bm{\Sigma}_i|+\text{ln }P(w_i)
\end{equation*}

The function can be coded as follows:
\begin{lstlisting}
function g = discric(x, mu, sigma, nSamples, prior, cas)
g = zeros(nSamples ,2);
if cas == 1
    for ii = 1 : 2
        w = mu{ii} / det(sigma);
        w0 = - mu{ii} * mu{ii}' / (2 * det(sigma)) + log (prior(ii));
        g(:, ii) = (w * x' + w0)';
    end
elseif cas == 2
    for ii = 1 : 2
        w = sigma \mu{ii}';
        w0 = - mu{ii}* (sigma\ mu{ii}') /2 + log ( prior(ii) );
        g(:, ii) = (w' * x' + w0)';
    end
elseif cas == 3
    for ii = 1 : 2
        w2 = sigma{ii} \mu{ii}';
        w0 = - mu{ii}* (sigma{ii}\ mu{ii}') /2 - log( det (sigma{ii}) )/2 + log ( prior(ii) );
        for jj= 1 : nSamples
            w1 = - inv (sigma{ii})/2;
            g(jj , ii) = (x(jj,:) * w1 * x(jj, :)' + x(jj, :) * w2 + w0);
        end
    end
end
end
\end{lstlisting}

\noindent (a) Using MATLAB code 1.4(a) we have

\begin{figure}[H]
\centering
\includegraphics[width=0.6\textwidth]{14a.jpg}
\caption{\label{14a}Classification Result for 1.3(a)}
\end{figure}

\text{ }\\
\\
\noindent(b) Using MATLAB code 1.4(b) we have

\begin{figure}[H]
\centering
\includegraphics[width=1\textwidth]{14b.jpg}
\caption{\label{14b}Classification Result for 1.3(b)}
\end{figure}

\noindent As we can see from the two figures, the decision boundaries are different, and with the second boundary there is a higher accuracy. I also repeated the classification for several times and the accuracy of case II is always higher than that of case I. This is because when we apply the classifier of case I onto the datasets of case II, we utilize primarily differences in means for classification while the differences in their variance be discarded.\\

\vfill
\clearpage

(c) Using MATLAB code 1.4(c) we have
\begin{figure}[H]
\centering
\subfigure[Case 1 \& 2 Classifier]{
\begin{minipage}{1\textwidth}
\centering
\includegraphics[width=0.9\textwidth]{14c1.jpg}\\
%\hspace{2cm}
\end{minipage}%
}%
%

\subfigure[Case 3 Classifier]{
\begin{minipage}{1\textwidth}
\centering
\includegraphics[width=0.9\textwidth]{14c2.jpg}\\
%\hspace{2cm}
\end{minipage}
}
\caption{Classification Result for 1.3(c)}
\end{figure}
\noindent From the result figure we can see that the decision boundary is a straight line only when two classes share the same covariance matrix. Generally speaking, classifier of case 3 performs better because when we apply case 1 or case 2 on dataset generated with case 3, some information are discarded. Simply averaging the individual class covariances makes sense only when two priors are equal, and computing a weighted average of covariances will be better.

\vfill
\clearpage

\noindent(d) Using MATLAB code 1.4(d) we have

\begin{figure}[H]
\centering
\includegraphics[width=0.9\textwidth]{14d.jpg}
\caption{\label{14d}Classification Result for 1.3(d)}
\end{figure}

\noindent(e) Using MATLAB code 1.4(e) we have

\begin{figure}[H]
\centering
\includegraphics[width=1\textwidth]{14e.jpg}
\caption{\label{14e}Classification Result for 1.3(e)}
\end{figure}

\vfill
\clearpage

\noindent(f) Using MATLAB code 1.4(f) we have
\begin{figure}[H]
\centering
\subfigure[Case 1 \& 2 Classifier]{
\begin{minipage}{1\textwidth}
\centering
\includegraphics[width=1\textwidth]{14f1.jpg}\\
%\hspace{2cm}
\end{minipage}%
}%
%

\subfigure[Case 3 Classifier]{
\begin{minipage}{1\textwidth}
\centering
\includegraphics[width=0.9\textwidth]{14f2.jpg}\\
%\hspace{2cm}
\end{minipage}
}
\caption{Classification Result for 1.3(f)}
\end{figure}

\noindent Classifying all data points as the class with the highest prior can lead to high accuracy in some extreme cases, such as $P(w_1)=0.05,\ P(w_2)=0.95$ in homework. In general, accuracy can be a poor metric depending on the case. Important information, such as covariance, cost, etc, should also be taken into account when classifying.

\vfill
\clearpage

\section{MATLAB Code for Problem 1.3 and 1.4}

(a) MATLAB code for 1.3 \& 1.4 (a)
\begin{lstlisting}
clear; clc; close all
mu = {[0,0];[-3 -3]};
sigma = {eye(2),eye(2)};
nSamples = 400;
prior = [0.5, 0.5];
[data, classIndex] = generateGaussianSamples(mu, sigma, nSamples, prior);
% Case 1 classifier
g = discric(data, mu, sigma{1}, nSamples, prior, 1);
clas = 2 * ones(400,1);
clas(g(:,1) >= g(:,2))=1;
accu = 1 - sum(abs(clas-classIndex))/ nSamples;
clsdata = sortrows([data, clas] , 3);
q = 400 - sum(clsdata(:,3)-1);
figure
hold on
plot( clsdata(1:q, 1), clsdata(1:q, 2), 'g.')
plot( clsdata(q + 1:400, 1), clsdata(q + 1:400, 2), 'b.')
db = ezplot(@(x, y) diff(score([x;y], mu, sigma, prior)), [-15 10 -14 16]);
set(db, 'color', 'm','LineStyle','--');
hold off
xlabel('$x_1$', 'Interpreter', 'latex')
ylabel('$x_2$', 'Interpreter', 'latex')
title(['$Classification\ Result\ with\ Accuracy=$',num2str(accu)], 'Interpreter', 'latex')
axis([-7 3 -7 3])
h = legend('$Class\ 1$', '$Class\ 2$','Decision\ Boundary');
set(h, 'Interpreter', 'latex');
grid on
\end{lstlisting}

\noindent (b) MATLAB code for 1.3 \& 1.4 (b)
\begin{lstlisting}
clear; clc; close all
mu = {[0,0];[-3 -3]};
sigma = {[3 1; 1 0.8], [3 1; 1 0.8]};
nSamples = 400;
prior = [0.05, 0.95];
[data, classIndex] = generateGaussianSamples(mu, sigma, nSamples, prior);
% Case 1 classifier
g1 = discric(data, mu, eye(2), nSamples, prior, 1);
clas1 = 2 * ones(400,1);
clas1(g1(:,1)>g1(:,2)) = 1;
accu1 = 1 - sum(abs(clas1-classIndex))/ nSamples;
clsdata = sortrows([data, clas1] , 3);
q = 400 - sum(clsdata(:,3)-1);
figure
subplot(1, 2, 1)
hold on
plot( clsdata(1:q, 1), clsdata(1:q, 2), 'g.')
plot( clsdata(q + 1:400, 1), clsdata(q + 1:400, 2), 'b.')
db = ezplot(@(x, y) diff(score([x;y], mu, eye(2), prior)), [-15 10 -14 10]);
set(db, 'color', 'm','LineStyle','--');
hold off
xlabel('$x_1$', 'Interpreter', 'latex')
ylabel('$x_2$', 'Interpreter', 'latex')
title(['$Classification\ Result\ of\ Case\ I\ \ Accuracy=$',num2str(accu1)], 'Interpreter', 'latex')
axis([-10 6 -6 3])
h = legend('$Class\ 1$', '$Class\ 2$','Decision\ Boundary');
set(h, 'Interpreter', 'latex');
grid on
% Case 2 classifier
g2 = discric(data, mu, sigma{1}, nSamples, prior, 2);
clas2 = 2 * ones(400,1);
clas2(g2(:,1)>g2(:,2))=1;
accu2 = 1 - sum(abs(clas2-classIndex))/ nSamples;
clsdata = sortrows([data, clas2] , 3);
q = 400 - sum(clsdata(:,3)-1);
subplot(1, 2, 2)
hold on
plot( clsdata(1:q, 1), clsdata(1:q, 2), 'g.')
plot( clsdata(q + 1:400, 1), clsdata(q + 1:400, 2), 'b.')
dba = ezplot(@(x, y) diff(score([x;y], mu, sigma, prior)), [-15 10 -14 10]);
set(dba, 'color', 'm','LineStyle','--');
hold off
xlabel('$x_1$', 'Interpreter', 'latex')
ylabel('$x_2$', 'Interpreter', 'latex')
title(['$Classification\ Result\ of\ Case\ II\ \ Accuracy=$',num2str(accu2)], 'Interpreter', 'latex')
axis([-10 6 -6 3])
h = legend('$Class\ 1$', '$Class\ 2$','Decision\ Boundary');
set(h, 'Interpreter', 'latex');
grid on
\end{lstlisting}

\noindent(c) MATLAB code for 1.3 \& 1.4 (c)
\begin{lstlisting}
clear;clc;close all
mu = {[0,0];[2 2]};
sigma = {[2 0.5; 0.5 1], [2 -1.9; -1.9 5]};
nSamples = 400;
prior = [0.05, 0.95];
[data, classIndex] = generateGaussianSamples(mu, sigma, nSamples, prior);
% Case 1 classifier
g1 = discric(data, mu, eye(2), nSamples, prior, 1);
clas1 = 2 * ones(400,1);
clas1(g1(:,1)>g1(:,2))=1;
accu1 = 1 - sum(abs(clas1 - classIndex))/ nSamples;
clsdata = sortrows([data, clas1] , 3);
q = 400 - sum(clsdata(:,3)-1);
figure
subplot(1, 2, 1)
hold on
plot( clsdata(1:q, 1), clsdata(1:q, 2), 'g.')
plot( clsdata(q + 1:400, 1), clsdata(q + 1:400, 2), 'b.')
dba = ezplot(@(x, y) diff(score([x;y], mu, eye(2), prior)), [-15 10 -14 10]);
set(dba, 'color', 'm','LineStyle','--');
axis([-5 8 -6 10])
hold off
xlabel('$x_1$', 'Interpreter', 'latex')
ylabel('$x_2$', 'Interpreter', 'latex')
title(['$Classification\ Result\ of\ Case\ I\ \ Accuracy=$',num2str(accu1)], 'Interpreter', 'latex')
h = legend('$Class\ 1$','$Class\ 2$','Decision\ Boundary');
set(h, 'Interpreter', 'latex');
grid on
% Case 2 classifier
g2 = discric(data, mu, (sigma{1}+sigma{2})/2, nSamples, prior, 2);
clas2 = 2 * ones(400,1);
clas2(g2(:,1)>g2(:,2))=1;
accu2 = 1 - sum(abs(clas2 - classIndex))/ nSamples;
clsdata = sortrows([data, clas2] , 3);
q = 400 - sum(clsdata(:,3)-1);
subplot(1, 2, 2)
hold on
plot( clsdata(1:q, 1), clsdata(1:q, 2), 'g.')
plot( clsdata(q + 1:400, 1), clsdata(q + 1:400, 2), 'b.')
dbb = ezplot(@(x, y) diff(score([x;y], mu, (sigma{1} + sigma{2})/2, prior)), [-15 10 -14 10]);
set(dbb, 'color', 'm','LineStyle','--');
axis([-5 8 -6 10])
hold off
xlabel('$x_1$', 'Interpreter', 'latex')
ylabel('$x_2$', 'Interpreter', 'latex')
title(['$Classification\ Result\ of\ Case\ II\ \ Accuracy=$',num2str(accu2)], 'Interpreter', 'latex')
h = legend('$Class\ 1$', '$Class\ 2$','Decision\ Boundary');
set(h, 'Interpreter', 'latex');
grid on
% Case 3 classifier
g3 = discric(data, mu, sigma, nSamples, prior, 3);
clas3 = 2 * ones(400,1);
clas3(g3(:,1)>g3(:,2))=1;
accu3 = 1 - sum(abs(clas3 - classIndex))/ nSamples;
clsdata = sortrows([data, clas3] , 3);
q = 400 - sum(clsdata(:,3)-1);
figure
hold on
plot( clsdata(1:q, 1), clsdata(1:q, 2), 'g.')
plot( clsdata(q + 1:400, 1), clsdata(q + 1:400, 2), 'b.')
db = ezplot(@(x, y) diff(score([x;y], mu ,sigma, prior)), [-15 10 -14 10]);
set(db, 'color', 'm','LineStyle','--');
axis([-5 8 -6 10])
hold off
xlabel('$x_1$', 'Interpreter', 'latex')
ylabel('$x_2$', 'Interpreter', 'latex')
title(['$Classification\ Result\ with\ Accuracy=$',num2str(accu3)], 'Interpreter', 'latex')
h = legend('$Class\ 1$', '$Class\ 2$','Decision\ Boundary');
set(h, 'Interpreter', 'latex');
grid on
\end{lstlisting}

\text{ }\\

\noindent The MATLAB code for (d) (e) and (f) can be obtained by changing priors in (a) (b) and (c) from $[0.5\ 0.5]$ to $[0.05\ 0.95]$.

\end{document}

\noindent MATLAB code 1.4(a)
\begin{lstlisting}
clear;clc;close all

mu = {[0,0];[-3 -3]};
sigma = {eye(2),eye(2)};
nSamples = 400;
prior = [0.5, 0.5];
[data, classIndex] = generateGaussianSamples(mu, sigma, nSamples, prior);
g = discric1(data, mu, sigma, nSamples, prior);

% Compute the accuracy.
clas = 2 * ones(400,1);
clas(g(:,1)>g(:,2))=1;
accu = 1 - sum(abs(clas-classIndex))/ nSamples;
ww = mu{1} - mu{2};
x0 = (mu{1} + mu{2})/2;

% Plotting
figure
hold on
x = -8 : 0.001 : 6;
y = ( -ww(1)*x + ww * x0')/ww(2);
clsdata = sortrows([data, clas] , 3);
q = 400 - sum(clsdata(:,3)-1);
plot( clsdata(1:q, 1), clsdata(1:q, 2), 'g.')
plot( clsdata(q + 1:400, 1), clsdata(q + 1:400, 2), 'b.')
plot(x, y, 'm--')
hold off
xlabel('$x_1$', 'Interpreter', 'latex')
ylabel('$x_2$', 'Interpreter', 'latex')
title(['$Classification\ Result\ with\ Accuracy=$',num2str(accu)], 'Interpreter', 'latex')
axis([-10 6 -6 3])
h = legend('$Class\ 1$', '$Class\ 2$','Decision\ Boundary');
set(h, 'Interpreter', 'latex');
grid on
\end{lstlisting}

\noindent MATLAB code 1.4(b)
\begin{lstlisting}
clear;clc;close all

mu = {[0,0];[-3 -3]};
sigma = {[3 1; 1 0.8], [3 1; 1 0.8]};
nSamples = 400;
prior = [0.5, 0.5];
[data, classIndex] = generateGaussianSamples(mu, sigma, nSamples, prior);

g1 = discric1(data, mu, sigma, nSamples, prior);
clas1 = 2 * ones(400,1);
clas1(g1(:,1)>g1(:,2))=1;
accu1 = 1 - sum(abs(clas1-classIndex))/ nSamples;
ww1 = mu{1} - mu{2};
x10 = (mu{1} + mu{2})/2;
figure
hold on
x = -8 : 0.001 : 6;
y = ( -ww1(1)*x + ww1 * x10')/ww1(2);
clsdata = sortrows([data, clas1] , 3);
q = 400 - sum(clsdata(:,3)-1);
plot( clsdata(1:q, 1), clsdata(1:q, 2), 'g.')
plot( clsdata(q + 1:400, 1), clsdata(q + 1:400, 2), 'b.')
plot(x, y, 'm--')
hold off
xlabel('$x_1$', 'Interpreter', 'latex')
ylabel('$x_2$', 'Interpreter', 'latex')
title(['$Classification\ Result\ with\ Accuracy=$',num2str(accu1)], 'Interpreter', 'latex')
axis([-10 6 -6 3])
h = legend('$Class\ 1$', '$Class\ 2$','Decision\ Boundary');
set(h, 'Interpreter', 'latex');
grid on

g2 = discric2(data, mu, sigma, nSamples, prior);
clas2 = 2 * ones(400,1);
clas2(g2(:,1)>g2(:,2))=1;
accu2 = 1 - sum(abs(clas2-classIndex))/ nSamples;
ww2 = sigma{1}\(mu{1} - mu{2})';
x20 = (mu{1} + mu{2})/2;
figure
hold on
x = -8 : 0.001 : 6;
y = ( -ww2(1)*x + x20 * ww2)/ww2(2);
clsdata = sortrows([data, clas2] , 3);
q = 400 - sum(clsdata(:,3)-1);
plot( clsdata(1:q, 1), clsdata(1:q, 2), 'g.')
plot( clsdata(q + 1:400, 1), clsdata(q + 1:400, 2), 'b.')
plot(x, y, 'm--')
hold off
xlabel('$x_1$', 'Interpreter', 'latex')
ylabel('$x_2$', 'Interpreter', 'latex')
title(['$Classification\ Result\ with\ Accuracy=$',num2str(accu2)], 'Interpreter', 'latex')
axis([-10 6 -6 3])
h = legend('$Class\ 1$', '$Class\ 2$','Decision\ Boundary');
set(h, 'Interpreter', 'latex');
grid on
\end{lstlisting}


\subsection{AdaIN}
Another quite known method of style transfer is AdaIN, short for Adaptive Instance Normalization. It was implemented in the paper by Huang \textit{et al.} \cite{Huang:1}. AdaIN is an extension of Instance Normalization, which we have described more in detail in Section \ref{sec:instance_normalization}. With AdaIN you need two normalized inputs: $x$, the image input and $y$, the style input. Then the method simply aligns the channel-wise variance and mean of $x$ to match $y$. The formula is as following:
\begin{equation}
\begin{aligned}
\label{eq:batch_normalization}
    AdaIN(x,y)&=\sigma(y) \frac{x-\mu(x)}{\sigma(x)}+\mu(y),
\end{aligned}
\end{equation}
where $\sigma(y)$ is used to scale the content input and $\mu(y)$ is used to shift it. The purpose of the AdaIN layer is to perform style transfer by using the mean and variance of the features. Because AdaIN is just a simple layer on top of an IN, it requires almost no extra computation. This makes AdaIN style transfer very fast compared to other tradional methods such as Gatys. This is the reason it is used for real-time style transfer in the paper of Huang \textit{et al.} mentioned above.