\chapter{Theories and Specifications}
\label{cha:theor-spec}

In this chapter we review the notions of a mathematical theory on one
side, and a program specification on the other.

%----------------------------------------------------------------------
%----------------------------------------------------------------------
\section{Theories}
\label{sec:theories}

Speaking informally, a (mathematical) theory is any mathematical topic
that we might want to study. For example, if we are interested in the
properties of a general group, our theory, call it $\thy{Group}$ would
consist of an unspecified group~$G$, equipped with unit and the usual
operations. All we know about~$G$ is that it satisfies the group
axioms. We might also wish to think about a particular group, say the
symmetric group $S_6$. This is again a theory $\thy{S6}$, but whatever
we prove in it applies only to one specific group. A third possibility
is to study the theory $\thy{Groups}$ of \emph{all} groups, which
would also speak about \emph{homomorphisms} between groups and
constructions of new groups from old ones.

\vspace{2cm}

In RZ the theory $\thy{Group}$ would be written as follows:
%
\sourcefile{group.thy}

%----------------------------------------------------------------------
\subsection{Simple Theories}
\label{sec:simple-theories}

Theories consist of sets, constants, relations and axioms.

Models.

The first-order language of theories.

Future extensions.

%----------------------------------------------------------------------
\subsection{Parametrized Theories}
\label{sec:param-theor}

Simple examples.

Advanced examples.

%----------------------------------------------------------------------
%----------------------------------------------------------------------
\section{Specifications}
\label{sec:specifications}

%----------------------------------------------------------------------
\subsection{ML Structures and Signatures}
\label{sec:ml-signatures}

%----------------------------------------------------------------------
\subsection{Parametrized Signatures (Functors)}
\label{sec:param-sign}

%----------------------------------------------------------------------
\subsection{Assertions}
\label{sec:assertions}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "userman"
%%% End:


\documentclass[a4page, twocolumn, twoside, 11pt]{article}

\input{pre-amble.tex}

\title{Finite Difference of the 1D Explicit Heat Equation}
\author{Dr J.\ A.\ Gopsill}
\date{2019}

\begin{document}

\maketitle

%\section{Introduction}

The one-dimensional transient heat conduction equation without heat generating sources is given by:

\begin{equation}
  \rho c_p \frac{\partial T}{\partial t} = \frac{\partial}{\partial x}\left(k\frac{\partial T}{\partial x}\right)
\end{equation}

\noindent where $\rho$ is the density, $c_p$ heat capacity, $k$ thermal conductivity, $T$ temperature, $x$ distance, and $t$ time.

If $\rho$, $c_p$ and $k$ are constant then the equation can be simplified to:

\begin{equation}
  \frac{\partial T}{\partial t} = \kappa\frac{\partial^2 T}{\partial x^2}
  \label{equ:1d-equation}
\end{equation}

\noindent where:

\begin{equation}
  \kappa = \frac{k}{\rho c_p}
\end{equation}

An example of the heat equation in practice is where a thin body with thermal conductivity $\kappa$ (e.g.\ rod or laminate) is at a starting temperature $T_s$ and is heated at both ends at a constant temperature $T_b$ while being insulated along its length $L$ (\cref{fig:model}). In this situation, one might be interested in how long it will take for the body to reach $T_b$.

\begin{figure}
  \centering
  \begin{tikzpicture}
    \draw[] (0,0) -- (5,0) node[pos=0.5, anchor=south] {$T_s$} -- (5,1) node[pos=0.5, anchor=west] {$T_b$} -- (0,1) -- (0,0) node[pos=0.5, anchor=east] {$T_b$};

    \draw[fill] (0,0) -- (5,0) -- (5,-0.1) -- (0,-0.1) -- (0,0);
    \draw[fill] (0,1) -- (5,1) -- (5,1.1) -- (0,1.1) -- (0,0);

    \draw[<->] (0,-0.3) -- (5,-0.3) node[pos=0.5, anchor=north] {$L$};

    \draw[dotted] (0.4,0) -- (0.4,1);
    \draw[dotted] (0.8,0) -- (0.8,1);
    \draw[dotted] (1.2,0) -- (1.2,1);
    \draw[<->] (1.2, 0.25) -- (1.6, 0.25) node[pos=0.5, anchor=south] {\footnotesize $\Delta x$};
    \draw[dotted] (1.6,0) -- (1.6,1);
    \draw[dotted] (2.0,0) -- (2.0,1);
  \end{tikzpicture}
  \caption{1D Heat Model}\label{fig:model}
\end{figure}

We cen solve this numerically by discretising the continuous derivatives ($\partial t, \partial x$) in \cref{equ:1d-equation} using finite difference methods. Lets first look at the time derivative $\frac{\partial T}{\partial t}$, which can be approximated using the forward finite difference approximation.

\begin{equation}
  \frac{\partial T}{\partial t} \approx \frac{ T^{n+1}_{i}-T^{n}_{i} }{ T^{n+1}-t^{n} } = \frac{ T^{n+1}_{i}-T^{n}_{i} }{ \Delta t }
  \label{equ:forward}
\end{equation}

\noindent where $n$ is a step in time and $i$ is a point along the discretised domain.

Second, we turn our attention to the spatial derivative $\frac{\partial^2 T}{\partial x^2}$, which can be approximated using the central difference approximation.

\begin{equation}
  \begin{split}
    \frac{\partial T}{\partial x^2} = \frac{\partial}{\partial x}\left(\frac{\partial T}{\partial x}\right) & \approx \frac{ \frac{ T^{n}_{i+1}-T^{n}_{i} }{ \Delta x } - \frac{ T^{n}_{i}-T^{n}_{i-1} }{ \Delta x } }{\Delta x} \\ & \approx \frac{T^n_{i+1}-2T^n_i+T^n_{i-1}}{\Delta x^2}
  \end{split}
  \label{equ:central}
\end{equation}

Substituting \cref{equ:forward,equ:central} into \cref{equ:1d-equation} gives:

\begin{equation}
  \frac{T^{n+1}_{i}-T^{n}_{i}}{\Delta t} = \kappa \left(\frac{T^n_{i+1}-2T^n_i+T^n_{i-1}}{\Delta x^2}\right)
\end{equation}

Rearranging for $T^{n+1}_{i}$ gives:

\begin{equation}
  T^{n+1}_{i} = T^{n}_{i} + \kappa\Delta t \left(\frac{T^n_{i+1}-2T^n_i+T^n_{i-1}}{\Delta x^2}\right)
\end{equation}

\end{document}


\section{Linear Classification
\hfill $\color{section-text-color} y, \bm z \in\brace{\pm 1}, \enskip \bm z \equiv c(\bm x)$}

% ===
\begin{tabular}{@{}>{\bfseries}l @{\enskip}l @{\enskip+}l}
    (1) Prob. gener. & $p(\bm x,y)$ & outlier det. \\
    (2) Prob. discr. & $p(y\mid\bm x)$ & deg. of belief \\
    (3) Purely discr. & $c: \bm X \to \bm y$ & easiest
\end{tabular}

\emph{Loss Functions:}
\enskip $\mathcal L(y, \bm z)$
\enskip{$\color{gray} \bm z \coloneqq \bm w^\top \bm x$}\\
\begin{tabular}{>{$\mathcal L\!$}l @{$\:=\:$}l}
    $~\ap{CE}$ & $- \brack{ y' \log\bm z' + (1-y') \log (1-\bm z') }$ \\
    $~\ap{0/1}$ & $\mathbb I \brace{\mathrm{sign}(\bm z) \neq y}$ \\
    $~\ap{hinge}$ & $\max(0, 1-y\bm z)$ \enskip \textcolor{gray}{for SVM's}\\
    $~\ap{percep}$ & $\max(0, -y\bm z)$ \\
    $~\ap{logistic}$ & $\log(1 + \exp(-y\bm z))$ \\
    $~\ap{exp}$ & $\exp(-y\bm z)$ \enskip \textcolor{gray}{for AdaBoost} \\
\end{tabular}
for CE (log loss):\; $y' {=} (1 {+} y) / 2$,\; $\bm z' {=} (1 {+} \bm z) / 2$

% ===
\subsection{Linear Discriminant Analysis \hfill(1)}

Assume $Y \sim \mathrm{Ber}(\beta)$,\; $P(X\vert Y{=}i) = \Gauss{\mu_i, \Sigma_i}$.

$\Rightarrow P(y_i\mid \bm x_i) = \sigma( \cancel{\bm x_i^\top \bm W \bm x_i} + \bm w\!^\top \bm x_i + w_0)$
\hfill{$\scriptstyle\color{gray}\text{if }\Sigma_0=\Sigma_1$}
%$\Rightarrow P(y_i\mid \bm x_i) = \sigma( \underbracket[.7pt][.7pt]{\cancel{\bm x_i^\top \bm W \bm x_i}}_{=0 \text{, if } \Sigma_0=\Sigma_1} + \bm w^\top \bm x_i + w_0)$

\textbf{Min. gener. error.:}\enskip
$\min_f \E[X,Y]{\mathcal L(y, c(\bm x))}$\\
\enskip $\rightsquigarrow$
$c^\ast(\bm x) = \argmin_c \sum_y p(y\mid\bm x) \mathcal L(y, c(\bm x))$

% ===
\subsection{Prob. discr. approach \hfill(2)}

Assume $P(y{=}1 \mid \bm x_i, \bm w) = \sigma(\bm w^\top \bm x)$,\enskip
$\implies L(\bm w)$ via
$P(\bm X, \!\bm Y \mid \bm w)
= \prod_i P(y_i \mid \bm x_i , \bm w) \color{gray} {\underbracket[.7pt][.7pt]{\the\everymath P(\bm x_i \mid \bm w)}}_{\:\mathit{const} \text{ w.r.t. } \bm w}$

\enskip $\propto \prod_i \sigma(\bm w^\top \!\bm x_i)^{y_i} (1 {-} \sigma(\bm w^\top \!\bm x_i))^{1-y_i}$

\textit{Note:}\enskip
$\bm w^\ast$ intractable but diff'able $\to$ \textbf{GD}!

% ===
\subsection{Purely discriminative \hfill(3)}
\label{purely-discriminative}

\iffalse
\begin{minipage}{\linewidth}
    \centering
    $\E[X,Y]{\mathcal L(Y, c(X))}$
    \enskip$\rightsquigarrow$\enskip
    $\frac1n \sum_n \mathcal L(y_i, c(\bm x_i))$
\end{minipage}
\fi

\emph{Perceptron:}
$f(x) = \mathrm{sgn}(\bm w^\top \bm x)$\\
\textbf{Loss:} $L(\bm w) = \sum_{i : \text{misclass.}} (-y_i \bm w^\top \bm x_i)$
\enskip use (S)GD

\textbf{Converges if} data is linearly separable,\\\enskip
and\enskip $\eta(k) {\geq} 0$,\; $\sum_k \eta(k) {\to} \infty$,\; $\sum_k \eta^2(k) {<} \infty$.

% ===
\emph{Gradient Descent:}
$N\!L(\bm w) \coloneqq -L(\bm w)$\\
$\highlight*{\bm w^{(k+1)} \leftarrow \bm w^{(k)} - \eta(k) \cdot \nabla_{\bm w} N\!L(\bm w^{(k)})}$

\textit{Opt. learning rate:} $\eta(k) = \argmin_\eta N\!L(\bm w^{(k+1)})$\\\vspace{-3pt}\enskip
\scalebox{.75}{$\textrm{(Taylor \& } \pderiv{}{\eta(k)} \overset!= 0)$}\enskip
$= \frac{\norm{ \nabla N\!L(\bm w^{(k)}) }^2}{ \nabla N\!L(\bm w^{(k)})^\top H_{N\!L}(\bm w^{(k)}) \nabla N\!L(\bm w^{(k)}) }$

\emph{Newton's Method:}
$\bm w^{(k+1)} \leftarrow \argmin_{\bm w} N\!L(\bm w)$\\\enskip
\scalebox{.75}{$\textrm{(Taylor \& } \pderiv{}{\bm w} \overset!= 0)$}\enskip\enskip
$= \bm w^{(k)} - H_{N\!L}^{-1} (\bm w^{(k)}) \nabla N\!L (\bm w^{(k)})$

% ===
\emph{Fisher's LDA:}
% Find projection that max. the distance of the projections AND min. the variance
\\\vspace{-5pt}\hfill
$J(\bm w) = \frac{\bm w\!^\top \bm\Sigma_B \bm w}{\bm w\!^\top \bm\Sigma_W \bm w}$
\enskip$\xrightarrow{\color{gray}(\ast)}$\enskip
$\bm w^\ast \propto \bm\Sigma_W^{-1} (\overline{\bm x}_1 - \overline{\bm x}_2)$

{\everymath\expandafter{\the\everymath\color{gray}}
\enskip $\ast :
    {\scriptstyle \pderiv{J(\bm w)}{\bm w} \overset!= 0}
    \:\rightsquigarrow\:
    (\cancel{\color{gray} \bm w\!^\top \bm\Sigma_B \bm w}) \bm\Sigma_W \bm w =
    (\cancel{\color{gray} \bm w\!^\top \bm\Sigma_W \bm w}) \bm\Sigma_B \bm w
$}

$\bm\Sigma_B = (\overline{\bm x}_1 - \overline{\bm x}_2) (\overline{\bm x}_1 - \overline{\bm x}_2)\!^\top$
    \hfill{\scriptsize between-class covariance}\\
$\bm\Sigma_W \!= \sum_k \sum_{\bm x \in \mathcal C_k} (\bm x - \overline{\bm x}_k) (\bm x - \overline{\bm x}_k)\!^\top$\!\!
    \hfill{\scriptsize within-class covariance}

\iffalse
    \emph{Fisher's linear discriminant:}
    Find projection that max. the distance of the projections \;AND\; min. the variance
    $\Rightarrow \max_w \frac{(\bm w^\top (\overline{\bm x}_0 - \overline{\bm x}_1))^2}{\widetilde{\mathrm{Var}}(\bm w^\top C_0) + \widetilde{\mathrm{Var}}(\bm w^\top C_1)}$
    \\\enskip
    where $\widetilde{\mathrm{Var}}(\bm w^\top C_i) = \sum_{i\in C_i} (\bm w^\top \bm x_i - \bm w^\top \overline{\bm x}_0)^2$.
    \\
    Solution: $\bm w^\ast \propto S_{\bm W}^{-1} (\overline{\bm x}_0 - \overline{\bm x}_1)$
    where $S_{\bm W} = \widetilde{\mathrm{Var}}(C_0) + \widetilde{\mathrm{Var}}(C_1)$
\fi

% ===


\documentclass[11pt]{article}
\usepackage[a4paper,margin= 2cm]{geometry}
\usepackage{graphicx}
\usepackage{caption}
\usepackage{subcaption}

\title{\LARGE{\bf{Task 4 - Quadrature}}}
\author{\Large{\bf{Kirtan Patel - AE19B038}}}
\date{}

\begin{document}

\maketitle

\section{Introduction}
In mathematics, quadrature is a historical term which means the process of determining area. This term is still used nowadays in the context of differential equations, where "solving an equation by quadrature" or "reduction to quadrature" means expressing its solution in terms of integrals.\\

There are various reasons as of why such approximations
can be useful. First, not every function can be analytically integrated. Second, even if a
closed integration formula exists, it might still not be the most efficient way of calculating
the integral. In addition, it can happen that we need to integrate an unknown function,
in which only some samples of the function are known.\\

This report includes working and analysis of 4 methods of quadrature:
\begin{enumerate}
	\item Rectangular Method
	\begin{enumerate}
		\item Left Endpoint Rule
		\item Right Endpoint Rule
		\item Midpoint Rule
	\end{enumerate}
	\item Trapezoid Method
\end{enumerate}

\subsection{Rectangular Method}
In order to gain some insight on numerical integration, we review Riemann integration, a framework that can be viewed as an approach for approximating integrals.\\

We assume that \textit{f(x)} is bounded function defined on [a,b] and that {$x_0, . . . . ,x_n$} is a partition(P) of [a,b]. For each \textit{i} we let
\[ M_i(f) = \mathop{sup}_{x\in[x_{i-1},x_i]} f(x)\] and \[m_i(f) = \mathop{inf}_{x\in[x_{i-1},x_i]} f(x)\]
Thus, $M_i(f)$ represents the Maxima of f(x) in the ith interval [$x_{i-1},x_{i}]$ while $m_i(f)$ represents the Minima of f(x) in the ith interval [$x_{i-1},x_{i}]$.\\
Letting $\delta x_i = x_i - x_{i-1}$ , the \textbf{upper (Darboux) sum} of f(x) with respect to the partition P is defined as :
\[ U(f,P) = \sum_{i=1}^{n}M_i\Delta x_i\]
while the \textbf{lower (Darboux) sum} of f(x) with respect to the partition P is defined as :
\[ L(f,P) = \sum_{i=1}^{n}m_i\Delta x_i\] \\

The upper bound integral of f(x) on [a,b] is defined as :
\[ U(f)= inf(U(f,P))\]
and the lower bound integral of f(x) is defined as
\[ L(f)= sup(L(f,P))\] where both the infimum and the supremum are taken over all possible partitions, P, of the interval [a,b]. \\

As mentioned above, the bound estimation of integral $\int_{a}^{b}f(x) \,dx$ can be done using the extremum values attained by the function in the given range. However, these sums are not defined in the most convenient way for an approximation algorithm. This is because we need to find the extrema of the function in every subinterval. Finding the extrema of the function, may be a complicated task on its own, which we would like to avoid.\\

A simpler approach for the approximation is to compute the product of the value of the function ar either of the end-points of the interval by the length of the interval. For an increasing function, taking the left endpoint would give a lower bound value while the right endpoint would give an upper bound value. The opposite is true if the function is decreasing.\\

A variation on the rectangular rule is the midpoint rule. Similar to the rectangular rule, we approximate the value of the integral by multiplying the length of the interval by the value of the function at one point. Only this time, we use the value of the function at the center point of the interval.\\

\begin{figure} [h!]
	\centering
	\begin{subfigure}{0.5\textwidth}
		\centering
		\includegraphics[width=1.1\linewidth]{Sample1}
		\label{fig1:sub1}
	\end{subfigure}%
	\hspace{1cm}
	\begin{subfigure}{0.4\textwidth}
		\centering
		\includegraphics[width=0.9\linewidth]{Sample2}
		\label{fig1:sub2}
	\end{subfigure}
	\caption{Graphical Representation of Quadrature using Rectangular Rule}
\end{figure}

\subsection{Trapezoid Method}
This technique is a much more accurate way to
approximate area beneath a curve.  To construct the
trapezoids, you mark the height of the function at the beginning and end of the width interval, then connect the two points.
We get a better approximation of graphs which are closer to linear, by assuming each interval to be consisting of trapeziums. Here we use both the end-points and average the values at the endpoints.\\

On dividing the integration interval into sub-intervals, this is termed as the composite trapezoid rule, which is obtained by applying the trapezoid rule in each sub-interval and then adding the resulting areas together\\

The Trapezoidal Rule overestimates a curve that is
concave up and underestimates functions that are
concave down.

\begin{figure} [h!]
	\centering
	\begin{subfigure}{0.5\textwidth}
		\centering
		\includegraphics[width=0.8\linewidth]{Sample3}
		\label{fig2:sub2}
	\end{subfigure}%
	\begin{subfigure}{0.5\textwidth}
		\centering
		\includegraphics[width=0.8\linewidth]{Sample4}
		\label{fig2:sub3}
	\end{subfigure}
	\caption{Graphical Representation of Quadrature using Rectangular Rule}
\end{figure}
\vspace{1cm}
\subsection{Remarks}
There are several other method which use weighted averages of the areas and give a less errorneous result.The following are a few of them:
\begin{itemize}
	\item \textbf{Newton-Cotes Methods}
	Use intepolating polynomials. Trapezoid, Simpson’s 1/3 and 3/8 rules,
	Bode’s are special cases of 1st, 2nd, 3rd and 4th order polynomials are
	used, respectively
	\item \textbf{Romberg Integration(Richardson Extrapolation)}
	uses knowledge of error estimated to build a recursive higher order scheme
	\item \textbf{Gauss Quadrature}
	Like Newton-Cotes, but instead of a regular grid, choose a set that lets you get higher order accuracy
	\item \textbf{Monte Carlo Integration}
	Use randomly selected grid points. Useful for higher dimensional integrals (d>4)
\end{itemize}

\pagebreak

\section{Results and Analysis}
Taking the number of sub-intervals from the user, and finding the error in the quadrature found by each method, we get the following results.
\subsection{Absolute Error vs $\Delta$x}
\begin{figure}[h!]
	\centering
	\centering
	\includegraphics[width=0.8\linewidth]{abs_error}
	\caption{Absolute Error vs $\Delta x$ size of sub-interval}
	\label{fig4}
\end{figure}

\subsection{log(Absolute Error) vs $\Delta$x}
\begin{figure} [h!]
	\centering
	\begin{subfigure}{0.55\textwidth}
		\centering
		\includegraphics[width=\linewidth]{log_abs_error}
		\caption{log(absolute error) vs $\Delta$x}
		\label{fig2:sub1}
	\end{subfigure}%
	\begin{subfigure}{0.55\textwidth}
		\centering
		\includegraphics[width=\linewidth]{log_error_vs_delta_x_Lin_Reg}
		\caption{log(absolute error) vs $\Delta$x - Linear Regression}
		\label{fig2:sub2}
	\end{subfigure}
	\caption{Results obtained by plotting log(Absolute Error) vs $\Delta$x}
\end{figure}
We can see that the Midpoint and Trapezoid Rules give a far more accurate result than the Left and Right Endpoint Rules. This difference is significant in functions with large slopes and large slope changes.\\

Furthermore, the Midpoint Rule gives a slightly better result than the Trapezoid Rule. For more linear functions, this difference negligible while for highly non-linear functions, the difference is large.\\

We can even see that the relation is almost linear between the Absolute error and the size of the sub-interval. Naturally, if we use a finer sub-interval, we go closer and closer to the true value of the integral and hence the error tends to zero.\\

It is important to note that the user inputs the number of intervals the integration interval is to be divided in. The sub-interval size $\Delta$x is given by : \[ \Delta x = \frac{(upper~integration~limit - lower~integration~limit)}{number~of~sub-intervals}\]
\subsection{Absolute Error vs Number of Sub-intervals}

\begin{figure}[h!]
	\centering
	\centering
	\includegraphics[width=0.8\linewidth]{abs_error_vs_num_subintervals}
	\caption{Result obtained by plotting Absolute Error vs number of sub-intervals}
	\label{fig4}
\end{figure}

We can see that the Absolute Error is \textbf{inversely proportional} to the number of sub-intervals. \\

This is even verified by seeing the Absolute Error vs $\Delta$x plot which is linear. This, along with the relation between $\Delta$x and number of sub-intervals confirms that the plot lines with the mathematical relation y $\propto$ $\frac{1}{x}$ \\

As the number of subintervals tends to $\infty$, the Absolute Error tends to 0
\pagebreak

\section{Order of Accuracy}
The 'order of accuracy' is a way to quantify the accuracy of the method i.e. how close the output of the method is to the actual value. Since the value we get from numerical method is an approximation to the actual value, there is an error value which is greater than zero, as the approximate value is not exactly the same as the actual value but close to it.\\

The degree of accuracy of a Quadrature Formula is the largest positive integer n such that the formula is for x$^k$ for each k = 0,1,...n.\\

The integral is given as :
\[ \int_{a}^{b}f(x) \,dx = Numerical~Integral~+~Error\]
In terms of the above defition, the order of accuracy is the highest degree of a polynomial for which the Integration Method gives no error.
\subsection{Rectangular Rule}
The Left Endpoint Rectangular Rule follows :
\[\int_{a}^{b}f(x) \,dx = {b-a}[f(a)]~+~Error\]
The Right Endpoint Rectangular Rule follows :
\[\int_{a}^{b}f(x) \,dx = {b-a}[f(b)]~+~Error\]
The Midpoint Rectangular Rule follows :
\[\int_{a}^{b}f(x) \,dx = {b-a}[f(\frac{a+b}{2})]~+~Error\] \\

Let f(x) = x$^0$\\
For which integral value~~$\int_{a}^{b} x^0 \,dx~=~\int_{a}^{b} 1 \,dx~=~ b-a$\\
And the Rectangular Rule (for all the 3 sub-rules) gives value: $(b-a).[1]~=~b-a$ \\
Hence there is \textbf{No Error} for x$^0$ \\

For f(x) = x$^1$\\
For which integral value~~$\int_{a}^{b} x \,dx~=~\frac{b^2-a^2}{2}$\\
the Left Endpoint Rectagular Rule gives value: $(b-a).[a]~=~ab-a^2$\\
the Right Endpoint Rectagular Rule gives value: $(b-a).[b]~=~b^2-ab$\\
the Midpoint Rectagular Rule gives value: $(b-a).[\frac{a+b}{2}]~=~\frac{b^2-a^2}{2}$\\
Hence there is \textbf{No Error} for x$^1$ in the Midpoint Rule, while there is \textbf{Error} for the Left Endpoint and the Right Endpoint Rules.\\

But for f(x) = x$^2$\\
For which integral value~~$\int_{a}^{b} x^2 \,dx~=~\frac{b^3-a^3}{3}$\\
And the Midpoint Rectangular Rule gives value~= $\frac{b-a}{2}~[(\frac{a~+~b}{2})^2]$\\
Here there is a \textbf{ Error} for f(x) = x$^2$.\\

Therefore, the order of accuracy for the Midpoint Rectangular method is 1,\\
while the order of accuracy for the Left and Right Endpoint Rules is 0.
\subsection{Trapezoid Rule}
The Trapezoid Rule follows that :
\[ \int_{a}^{b}f(x) \,dx = \frac{b-a}{2}[f(a)+f(b)]~+~Error\]

Let f(x) = x$^0$\\
For which integral value~~$\int_{a}^{b} x^0 \,dx~=~\int_{a}^{b} 1 \,dx~=~ b-a$\\
And the Trapezoid Rule gives value~= ~ $\frac{b-a}{2}~[1~+~1]~=~b-a$\\
Hence there is \textbf{No Error} for x$^0$ \\

Similarly Let f(x) = x$^1$\\
For which integral value~~$\int_{a}^{b} x \,dx~=~\frac{b^2-a^2}{2}$\\
And the Trapezoid Rule gives value~= ~ $\frac{b-a}{2}~[a~+~b]~=~\frac{b^2-a^2}{2}$\\
Hence there is \textbf{No Error} for x$^1$ as well.\\

But for f(x) = x$^2$\\
For which integral value~~$\int_{a}^{b} x^2 \,dx~=~\frac{b^3-a^3}{3}$\\
And the Trapezoid Rule gives value~= ~ $\frac{b-a}{2}~[a^2~+~b^2]$\\
Here there is a cubic\textbf{ Error} for f(x) = x$^2$.\\

Therefore, the the order of accuracy for the Trapezoid method is 1.\\

As we begin to use interpolation and other mathematical techniques, we can get a higet order of accuracy. e.g. The Simpson's 1/3 rule has an order of accuracy of 3.\\

Tabulating the order of accuracy(OoA) calculated,\\
\begin{table} [h!]
	\centering
\begin{tabular}{| l || l |}
	\hline
	Quadrature Method & OoA  \\
	\hline \hline
	Left Endpoint Rectangular Method & 0\\
	\hline
	Right Endpoint Rectangular Method & 0 \\
	\hline
	Midpoint Rectangular Method & 1 \\
	\hline
	Trapezoid Method & 1 \\
	\hline
\end{tabular}
\end{table}

This is seen evidently in the Absolute Error vs $\Delta$x plots (Figure 3 and Figure 4)

\section{References}
\begin{enumerate}
	\item https://en.wikipedia.org/wiki/Quadrature$\_$(mathematics)
	\item https://math.stackexchange.com/questions/2873291/what-is-the-intuitive-meaning-of-order-of-accuracy-and-order-of-approximation
	\item http://www.math.pitt.edu/~sussmanm/2070Fall07/lab$\_$10/index.html
	\item https://www.kth.se/social/upload/5287cd2bf27654543869d6c8/Ant-OrderofAccuracy.pdf
	\item https://www3.nd.edu/~zxu2/acms40390F11/sec4-3-Numerical$\_$integration.pdf
\end{enumerate}

\end{document}


\subsection{The continuous constant pH molecular dynamics framework}

In continuous CpHMD, each titratable residue is assigned a titration coordinate, $\lambda_i$ \cite{Lee_Brooks_2004_Proteins,Khandogin_Brooks_2005_Biophys.J.}.
We attribute to this coordinate an implicit function of an underlying coordinate $\theta_i$ such that $\lambda_i$ is bound between 0 and 1:
\begin{equation}
    \lambda_i = \sin^2\theta_i,
\end{equation}
where 0 and 1 represent the protonated and deprotonated states, respectively.
Since $\lambda$ continuously evolves between 0 and 1,
in simulation analysis we apply cutoffs to define
protonated ($\lambda^P < 0.2$) and deprotonated states ($\lambda^U > 0.8$).
Having the titration coordinate in place, we use an extended Hamiltonian to allow the simultaneous propagation of spatial (real) and titration (virtual) coordinates:

\begin{equation}
\begin{split}
\label{eq:Hamiltonian}
\mathcal{H}(\{\bm{r}_a\},\{\theta_i\})=
&\frac{1}{2}\sum_am_a\dot{\bm{r}}_a^2+\frac{1}{2}\sum_im_i\dot{\theta}_i^2+U^{\rm int}(\{\bm{r}_a\})\\
&+U^{\rm hybr}(\{\bm{r}_a\},\{\theta_i\})+\sum_iU^{\ast}(\theta_i),
\end{split}
\end{equation}
where $\bm{r}_a$ refers to the (x,y,z) coordinates
of atom $a$, and $\theta_i$ refers to the titration coordinate of titratable residue $i$.
The two first terms of Eq.~\ref{eq:Hamiltonian} give the kinetic energies of real atoms and virtual $\lambda$ particles.
$U^{\rm int}$ represent the titration-independent bonded and non-bond energies.
The fourth term, $U^{\rm hybr}$, describes the
non-bond energies, which are dependent on both spatial and titration coordinates.
For the particle-mesh Ewald (PME) all-atom CpHMD
\cite{Huang_Shen_2016_J.Chem.TheoryComput.}
$U^{\rm hybr}$ includes only Coulomb and van der Waals (vdW) energies, while for
the implicit- and hybrid-solvent CpHMD methods $U^{\rm hybr}$ also includes the generalized Born (GB) energy:

 \begin{equation}
\label{eq:Uhybr}
\begin{split}
U^{\rm hybr}(\{\bm{r}_a\},\{\theta_i\})=U^{\rm elec}(\{\bm{r}_a\},\{\theta_i\})
+ U^{\rm vdW}(\{\bm{r}_a\},\{\theta_i\})\\
+ U^{\rm GB}(\{\bm{r}_a\},\{\theta_i\})
\end{split}
\end{equation}

The dependence of the electrostatic energy on the titration coordinates arises from the requirement that
the partial atomic charges on the titrating residue
are linearly interpolated
between the values in the deprotonated and protonated states with respect to $\lambda$:

% Thus, we can write the charge on atom $j$, $q_j$, on the titrating residue $i$ as
%
\begin{equation}
q_j(\lambda_i) = (1-\lambda_i) q_j^{\rm prot} +  \lambda_i q_j^{\rm unprot},
\label{eq:qlamb}
\end{equation}
where $q_j$ refers to the partial charge of an atom $j$ in the titrating residue $i$.
As to vdW energies, the
interactions involving a titratable proton are linearly attenuated with respect to $\lambda$:
\begin{equation}
U^{\rm vdW}_{ij}(\bm r_i, \bm r_j, \lambda_i) = (1-\lambda_i)U^{\rm vdW\ast}_{ij}(\bm r_i, \bm r_j),
\end{equation}
where $i$ is the index for the titratable proton, $j$ is the index for all other atoms, and $U^{\rm vdW \ast}_{ij}$ refers to the protonation independent vdW interaction energy. If both $i$ and $j$ are titratable protons,
the interaction is linearly attenuated by the $\lambda$ values of both:
\begin{equation}
U^{\rm vdW}_{ij}(\bm r_i, \bm r_j, \lambda_i, \lambda_j) = (1-\lambda_i)(1-\lambda_j)U^{\rm vdW\ast}_{ij}(\bm r_i, \bm r_j).
\end{equation}
Note, the vdW energy correction is small and typically neglected in constant pH methods except for the CpHMD methods implemented in CHARMM and Amber \cite{Khandogin_Brooks_2005_Biophys.J.,Wallace_Shen_2011_J.Chem.TheoryComput.,Huang_Shen_2016_J.Chem.TheoryComput.,Huang_Shen_2018_J.Chem.Inf.Model.,Harris_Shen_2019_J.Chem.Inf.Model.}.

Finally, the last term in Eq. \ref{eq:Hamiltonian}
only affects titratable groups and is consisted of
three biasing potentials:
\begin{equation}
\label{eq:biasingpotential}
U^{\ast}(\theta_i)=-U^{\rm mod}(\lambda_i)+U^{\rm barr}(\lambda_i)+U^{\rm pH}(\lambda_i).
\end{equation}

The first biasing potential $U^{\rm mod}$ describes a potential of mean force (PMF) of model titration along the $\lambda$-coordinate, where the model represents a fully solvent-exposed amino acid, i.e., a blocked single amino acid or a small peptide in solution.
According to the linear response theory,
model PMF is (in GB solvent) or can be approximated (in explicit solvent) as a quadratic function:
\begin{equation}
\label{eq:Umod_1d}
U^{\rm mod}(\lambda_i)=A(\lambda_i-B)^2.
\end{equation}
Here $A$ and $B$ are parameters that can be determined through free energy calculation methods, such as thermodynamic integration (TI, see section Parameterization).
By analogy, in the case of coupled double-site titration \cite{Khandogin_Brooks_2005_Biophys.J.}, e.g., histidine or carboxylic acid, the model PMF is second order in both $\lambda$ and $x$, where
$x$ is a coordinate that represents the tautomer interconversion.
The two-dimensional PMF can be written as a bivariate polynomial,
%
\begin{equation}
\begin{split}
U^{\rm mod}(\lambda_i,x_i) =
&a_0 \lambda_i^2 x_i^2 + a_1 \lambda_i^2 x_i + a_2 \lambda_i x_i^2 + a_3 \lambda_i^2 \\
&+ a_4 x_i^2 + a_5 \lambda_i x_i + a_6 \lambda_i + a_7 x_i + a_8.
\end{split}
\label{eq:double}
\end{equation}
Here $x_{i}=\sin^{2}\theta^{x}_{i}$ is bound between 0 and 1, where $\theta^{x}_{i}$ is the underlying unbound
variable.
%

$U^{\rm barr}$, the second biasing potential in Eq. \ref{eq:biasingpotential}, describes a barrier or penalty potential in the center of the titration coordinate,
%
\begin{equation}
U^{\rm barr}(\lambda_i) = -4\beta\left(\lambda_i - \frac{1}{2} \right)^2,
\label{eq:Ubarr}
\end{equation}
where $\beta$ is a parameter that determines the height of the barrier.
Adding the barrier potential serves to
increase the sampling time at the end-point states (e.g., $\lambda<0.2$ or $\lambda>0.8$) and therefore
minimizing the unphysical vdW and electrostatic interactions associated with mixed state
(e.g., $0.2 \le \lambda \le 0.8$).

$U^{\rm pH}$, the last term of Eq.~\ref{eq:biasingpotential}
describes the pH dependence of the deprotonation free energy
and is given by
%
\begin{equation}
U^{\rm pH}(\lambda_i)=ln(10)k_BT({\rm pH}-\textrm{p}{K_a}^{\rm mod})\lambda_i,
\label{eq:UpH}
\end{equation}
%
where $k_B$ is the Boltzmann constant, $T$ is the system temperature, and $\textrm{p}{K_a}^{\rm mod}$ is the model {\pka} for the titratable residue $i$.


\section{Diophantine Equations}

	\prob{https://artofproblemsolving.com/community/c6h79786p456605}{APMO 1999 P4}{EM}{Determine all pairs $(a,b)$ of integers with the property that the numbers $a^2+4b$ and $b^2+4a$ are both perfect squares.}

		\solu{Easy case work assuming positive or negative values for $ a, b $.}

	\prob{https://artofproblemsolving.com/community/c6t177f6h1437607_infinitely_many_pairs}{All Squares}{E}{Prove that there are infinitely many pairs of positive integers $(x, y)$ satisfying that $x+y, x-y, xy+1$ are all perfect squares.}\label{problem:construction_1}

		\solu{Just work it out.}

	\prob{https://artofproblemsolving.com/community/c6h418641p2362006}{ISL 2010 N3}{E}{Find the smallest number $n$ such that there exist polynomials $f_1, f_2, \ldots , f_n$ with rational coefficients satisfying \[x^2+7 = f_1\left(x\right)^2 + f_2\left(x\right)^2 + \ldots + f_n\left(x\right)^2.\]}

		\solu{Find the obvious answer, which is very small, so we can probably case work it out. We find the case for $ 3 $ a bit challenging. But we plan to show that $ 7a^2 $ can't be written as a sum of $ 3 $ squares. Which numbers can be written as the sum of $ 3 $ squares? Investigate...}

	\prob{https://artofproblemsolving.com/community/c6h1113199p5083571}{ISL 2014 N5}{E}{Find all triples $(p, x, y)$ consisting of a prime number $p$ and two positive integers $x$ and $y$ such that $x^{p -1} + y$ and $x + y^ {p -1}$ are both powers of $p$.}

		\solu{The old school trick, replacement. Assume $ x<y $ and replace $ y $, and then compare the power of $ p $.}


\chapter{Probability}

\begin{ex}
  Consider $B_i$ and $B_j$ with $i<j$. Suppose $x\in B_i$ and note that then
  $x\not\in B_j$, since by definition
  \[
    B_j
    =\left\{\omega\in\Omega\mid
    \omega\in A_j,\omega\not\in A_{j-1},\ldots,
    \omega\not\in A_{i}\ldots,\omega\not\in A_{1}\right\}.
  \]
  Therefore, $B_i\cap B_j=\emptyset$.

  Since the sequence of $A_i$'s is monotone increasing, it is clear that
  $A_n=\bigcup_{i=1}^n A_i$, since $A_i\subset A_n$ for all $i\leq n$. Note that
  $\bigcup_{i=1}^n B_i\subset A_n$, since $B_i\subset A_i$
  by definition. If $x\in \bigcup_{i=1}^n A_i$, then there exists a $j$ such
  that $x\in A_j$, but $x\not\in A_i$ for any $i<j$, and therefore $x\in B_j$.
  Hence, $\bigcup_{i=1}^n A_i\subset\bigcup_{i=1}^n B_i$.

  Thus, $\bigcup_{i=1}^n A_i=\bigcup_{i=1}^n B_i$ for all $n$, and therefore
  $\bigcup_{i=1}^\infty A_i=\bigcup_{i=1}^\infty B_i$. This completes the proof
  of the monotone increasing case of Theorem 1.8.

  If $A_1\supset A_2\supset\cdots$ is monotone decreasing, it follows that
  $A_1^c\subset A_2^c\subset\cdots$ is monotone increasing. Therefore,
  \[
    \lim_{n\to\infty}\P{A_n}
    =1-\lim_{n\to\infty}\P{A_n^c}
    =1-\P{A^c}
    =\P{A}.
  \]
\end{ex}

\begin{ex}
  Note that since $\emptyset\cap\emptyset=\emptyset$, the empty set is disjoint
  to itself. Therefore,
  \[
    \P{\emptyset}
    =\P{\bigsqcup_{i=1}^\infty \emptyset}
    =\sum_{i=1}^\infty\P{\emptyset}
  \]
  by Axiom 3. However, this series only converges if $\P{\emptyset}=0$.

  Suppose $A\cap B=\emptyset$. Let $C_1=A$, $C_2=B$ and $C_n=\emptyset$ for
  $n\geq 3$, and note that then $\bigcup_{i=1}^\infty C_i$ is a disjoint
  partition of $A\cup B$. Therefore,
  \[
    \P{A\cup B}
    =\P{\bigsqcup_{i=1}^\infty C_i}
    =\sum_{i=1}^\infty \P{C_i}
    =\P{A}+\P{B}+\sum_{i=3}^\infty 0
    =\P{A}+\P{B}.
  \]

  If $A\subset B$, then $B=A\sqcup (B\setminus A)$, and so by the preceding
  result,
  \[
    \P{B}=\P{A\sqcup (B\setminus A)}=\P{A}+\P{B\setminus A}.
  \]
  Therefore, since $\P{B\setminus A}\geq 0$ by Axiom 1, we get
  $\P{A}\leq \P{B}$.

  Note that $0\leq \P{A}$ is just a restatement of Axiom 1. $\P{A}\leq 1$
  follows by the previous result after observing that $A\subset \Omega$ and
  $\P{\Omega}=1$ by Axiom 2.

  Finally, note that $\Omega=A\sqcup A^c$, and therefore
  \[
    1=\P{\Omega}=\P{A}+\P{A^c}.
  \]
  Hence, $\P{A^c}=1-\P{A}$.
\end{ex}

\begin{ex}
  \begin{enumerate}[(a)]
    \item Let $s,t\in\Z_{>0}$ such that $s<t$ and note that then
          \[
            B_s
            =\bigcup_{i=s}^n A_i
            =\bigcup_{i=s}^{t-1} A_i \cup \bigcup_{i=t}^\infty A_i
            =\bigcup_{i=s}^{t-1} A_i\cup B_t.
          \]
          Therefore, $B_t \subset B_s$.

          Now consider
          \[
            C_s
            =\bigcap_{i=s}^n A_i
            =\bigcap_{i=s}^{t-1} A_i \cap \bigcap_{i=t}^\infty A_i
            =\bigcap_{i=s}^{t-1} A_i\cap C_t.
          \]
          Therefore, $C_s\subset C_t$.

    \item Suppose $\omega\in \bigcap_{n=1}^\infty B_n$. Then $\omega\in B_n$ for
          all $n$, and therefore for any $M>0$, we can find an $m>M$ such that
          $\omega\in A_m$. Hence, letting $M_1=1$, $m_i$ be the smallest integer
          greater than $M_i$ such that $\omega\in A_{m_i}$, and $M_i=m_{i-1}$
          for $i>1$, we get an infinite family of events $\{A_{m_i}\}$ such that
          $\omega\in A_{m_i}$ for all $i$.

          Conversely, suppose that $\omega\in A_n$ for infinitely many $n$. Then
          for any $B_n$, since there are only finitely many $A_i$'s with
          $i\leq n$, there must exist some $m>n$ such that $\omega\in A_m$.
          Hence, $\omega\in B_n$, and since $n$ was arbitrary,
          $\omega\in \bigcap_{n=1}^\infty B_n$.

    \item Suppose $\omega\in \bigcup_{n=1}^\infty C_n$. Then there exists some
          $m$ such that $\omega\in C_m$ and therefore
          $\omega\in\bigcap_{i=m}^\infty A_i$. Hence, $\omega\in A_i$ for all
          $i\geq m$, and thus $\omega$ belongs to all $A_i$'s except possibly
          finitely many.

          Conversely, suppose that $\omega$ belongs to all the events except
          possibly finitely many. Let $m-1$ be the largest integer such
          $\omega\not\in A_{m-1}$ and note that then
          $\omega\in \bigcap_{i=m}^\infty A_i=C_m$. Hence,
          $\omega\in \bigcup_{n=1}^\infty C_n$.
  \end{enumerate}
\end{ex}

\begin{ex}
  Suppose $x\in\bigcap_{i\in I}A_i^c$. Then for all $i\in I$, $x\in A_i^c$ and
  therefore $x\not\in A_i$ for all $i\in I$. Hence,
  $x\not\in \bigcup_{i\in I} A_i$ and $x\in\left(\bigcup_{i\in I} A_i\right)^c$.
  Thus, $\bigcap_{i\in I}A_i^c\subset \left(\bigcup_{i\in I} A_i\right)^c$.

  Suppose $x\in\left(\bigcup_{i\in I} A_i\right)^c$. Then
  $x\not\in\bigcup_{i\in I} A_i$, and hence $x\not\in A_i$ for all $i\in I$.
  Thus, $x\in A_i^c$ for all $i\in I$ and so $x\in\bigcap_{i\in I}A_i^c$. Hence,
  $\left(\bigcup_{i\in I} A_i\right)^c\subset\bigcap_{i\in I}A_i^c$, and
  therefore
  \[
    \left(\bigcup_{i\in I} A_i\right)^c=\bigcap_{i\in I}A_i^c.
  \]

  Replacing $A_i$ with $A_i^c$, taking the complement of both sides and using
  the fact that the complement of a complement is the original set, it follows
  that
  \[
    \bigcup_{i\in I} A_i^c=\left(\bigcap_{i\in I}A_i\right)^c.
  \]
\end{ex}

\begin{ex}
  The sample space is
  \[
    S=\left\{
    \underbrace{T\cdots T}_{n_1\text{ times}}
    H\underbrace{T\cdots T}_{n_2\text{ times}}H \mid n_1,n_2\geq0 \right\}.
  \]
  Note that if exactly $k$ tosses are required, the final coin toss is always
  the second heads, but the first heads can occur on any of the preceding $k-1$
  tosses. Hence, for $k\geq 2$ the probability of this event is
  \[
    (k-1)\left(\frac{1}{2}\right)^k.
  \]
\end{ex}

\begin{ex}
  Suppose that there exists such a distribution. Let $p=\P{\{0\}}$ and note that
  $p\neq 0$, since otherwise $1=\P{\Omega}=\sum_{i=0}^n\P{\{i\}}=0$, a
  contradiction. Therefore, we can let
  $N=\left\lceil\frac{1}{p}\right\rceil+1$ and $A=\{0, 1, \ldots, N-1\}$. Then
  \begin{align*}
    \P{A}
    =N\P{\{0\}}
    =\left(\left\lceil\frac{1}{p}\right\rceil+1\right)p
    \geq 1 + p,
  \end{align*}
  contradicting the requirement that probability is between $0$ and $1$.
  Therefore, no such distribution can exist.
\end{ex}

\begin{ex}
  Let $B_n=A_n\setminus \bigcup_{i=1}^{n-1}A_i$. Consider $B_s$ and $B_t$ such
  that $s<t$ and note that if $x$ is an element of $B_s$, by definition $x$ must
  be an element of $A_s$. Hence, since $t>s$,
  $x\in\bigcup_{i=1}^{t-1} A_i$, and thus
  $x\not\in A_t\setminus \bigcup_{i=1}^{t-1}A_i=B_t$. Therefore, $B_s$ and $B_t$
  are disjoint whenever $s\neq t$.

  Since $B_n\subset A_n$ for all $n$, it follows that
  $\bigcup_{n=1}^\infty B_n\subset \bigcup_{n=1}^\infty A_n$. Moreover, if
  $x\in \bigcup_{n=1}^\infty A_n$ there exists some $m$ such that $x\in A_m$,
  but $x\not\in A_s$, for $s<m$. This implies that $x\in B_m$ and therefore
  $\bigcup_{n=1}^\infty A_n\subset \bigcup_{n=1}^\infty B_n$. Hence,
  $\bigcup_{n=1}^\infty A_n= \bigcup_{n=1}^\infty B_n$.

  Thus,
  \[
    \P{\bigcup_{n=1}^\infty A_n}
    =\P{\bigsqcup_{n=1}^\infty B_n}
    =\sum_{n=1}^\infty \P{B_n}
    \leq \sum_{n=1}^\infty \P{A_n},
  \]
  since $B_n\subset A_n$ for all $n$.
\end{ex}

\begin{ex}
  \begin{align*}
    \P{\bigcap_{i=1}^\infty A_i}
     & =1-\P{\bigcup_{i=1}^\infty A_i^c} &  & \text{(by De Morgan's law)}  \\
     & \geq 1-\sum_{i=1}^\infty\P{A_i^c} &  & \text{(by previous problem)} \\
     & =1.
  \end{align*}
\end{ex}

\begin{ex}
  Note that for any event $A$, we have
  \[
    \cP{A}{B}
    =\frac{\P{A\cap B}}{\P{B}}\geq 0,
  \]
  since $\P{B}> 0$ and $\P{A\cap B}\geq 0$ by the first axiom of probability,
  and therefore $\cP{\,\cdot}{B}$ also satisfies the first axiom.

  We also have
  \[
    \cP{\Omega}{B}=\frac{\P{\Omega\cap B}}{\P{B}}=\frac{\P{B}}{\P{B}}=1,
  \]
  and therefore $\cP{\,\cdot\,}{B}$ satisfies the second axiom.

  Finally, suppose that $A_1, A_2, \ldots$ are disjoint. Then
  \begin{align*}
    \cP{\bigcup_{i=1}^\infty A_i}{B}
     & =\frac{\P{\bigcup_{i=1}^\infty A_i\cap B}}{\P{B}} \\
     & =\frac{1}{\P{B}}\sum_{i=1}^\infty \P{A_i \cap B}  \\
     & =\sum_{i=1}^\infty \frac{\P{A_i \cap B}}{\P{B}}   \\
     & =\sum_{i=1}^\infty \cP{A_i}{B},
  \end{align*}
  and therefore $\cP{\,\cdot}{B}$ satisfies the third axiom.
\end{ex}

\begin{ex}
  Without loss of generality, we may assume that the player initially selects
  the first door and the host reveals the second door to be empty. Then, letting
  $D_i$ be the event of the reward being behind door $i$ and $R_2$ being the
  event of the host opening door $2$, we have
  \begin{align*}
    \cP{D_1}{R_2}
    =\frac{\cP{R_2}{D_1}\P{D_1}}{\P{R_2}}
    =\frac{\frac{1}{2}\cdot\frac{1}{3}}{\frac{1}{2}}=\frac{1}{3},
  \end{align*}
  for Door 1 and
  \begin{align*}
    \cP{D_3}{R_2}
    =\frac{\cP{R_2}{D_3}\P{D_3}}{\P{R_2}}
    =\frac{1\cdot\frac{1}{3}}{\frac{1}{2}}=\frac{2}{3},
  \end{align*}
  for Door 3, since if the reward were behind Door 1 the host would have been
  able to reveal either Door 2 or Door 3, but if it were behind Door 3, he
  would be forced to reveal Door 2.
\end{ex}

\begin{ex}
  \begin{align*}
    \P{A^c\cap B^c}
     & =\P{(A^{c^c}\cup B^{c^c})^c}                  &  & \text{(by De Morgan's law)} \\
     & =\P{(A \cup B)^c}                                                              \\
     & =1 - \P{A\cup B}                                                               \\
     & =1 - \left[\P{A} + \P{B} - \P{A\cap B}\right]                                  \\
     & =1-\P{A}-\P{B}+\P{A}\P{B}                     &  & \text{(since $A\amalg B$)}  \\
     & =\left[1-\P{A}\right]\left[1-\P{B}\right]                                      \\
     & =\P{A^c}\P{B^c}
  \end{align*}
\end{ex}

\begin{ex}
  Let $S_g$ be the event of drawing a card showing a green side and $C_g$ be the
  event of drawing the card that is green on both sides. Then, by Bayes'
  theorem,
  \begin{align*}
    \cP{C_g}{S_g}
     & =\frac{\cP{S_g}{C_g}\P{C_g}}{\P{S_g}}
    =\frac{1\cdot \frac{1}{3}}{\frac{1}{2}}
    =\frac{2}{3}.
  \end{align*}
\end{ex}

\begin{ex}
  \begin{enumerate}[(a)]
    \item The sample space is given by
          \[
            S=\left\{
            \underbrace{T\cdots T}_{n\text{ times}}H \mid n\geq 1
            \right\} \bigcup
            \left\{
            \underbrace{H\cdots H}_{n\text{ times}}T \mid n\geq 1
            \right\}.
          \]
    \item There are two possible ways that can happen: by throwing $HHT$ or
          $TTH$. Therefore, the probability is
          \[
            2\cdot \left(\frac{1}{2}\right)^3=\frac{1}{4}.
          \]
  \end{enumerate}
\end{ex}

\begin{ex}
  Let $\P{A}=0$ and let $B$ be an arbitrary event. Then it is clear that
  \[
    0\leq \P{A\cap B}\leq \P{A}=0,
  \]
  and therefore $\P{A\cap B}=\P{A}\P{B}$ trivially. Hence, $A\amalg B$.

  Otherwise, if $\P{A}=1$, note that $\P{A^c}=0$, and therefore $A^c\amalg B^c$
  for any event $B^c$. However, Exercise 11 then implies that $A\amalg B$.

  If $A$ is independent of itself, we have $\P{A}=\P{A\cap A}=[\P{A}]^2$, and
  therefore $\P{A}$ must equal $0$ or $1$.
\end{ex}

\begin{ex}
  \begin{enumerate}[(a)]
    \item Having at least two children with blue eyes corresponds to the event
          $\{BBO, BOB, OBB,$ $BBB\}$. Note that by Bayes' theorem
          \[
            \cP{BBO}{B_{\geq 1}}
            =\frac{\cP{B_{\geq 1}}{BBO}\P{BBO}}{\P{B_{\geq 1}}}
            =\frac{1\cdot \frac{3}{64}}{\frac{37}{64}}
            =\frac{3}{37},
          \]
          and similarly $\cP{BOB}{B_{\geq 1}}=\cP{OBB}{B_{\geq 1}}=\frac{3}{37}$
          by independence between children. Meanwhile,
          \[
            \cP{BBB}{B_{\geq 1}}
            =\frac{\cP{B_{\geq 1}}{BBB}\P{BBB}}{\P{B_{\geq 1}}}
            =\frac{1\cdot \frac{1}{64}}{\frac{37}{64}}
            =\frac{1}{37}.
          \]
          Therefore, the probability at least two children have blue eyes given
          that at least one child has blue eyes is $\frac{10}{37}$.

    \item This is identical to asking what is the probability that at least one
          of the two remaining children has blue eyes. The probability that
          either the middle or eldest child, but not both, have blue eyes is
          $2\cdot \frac{1}{4}\cdot\frac{3}{4}=\frac{3}{8}$, while the
          probability that both have blue eyes is $\frac{1}{16}$. Therefore,
          the probability that at least two children have blue eyes given that
          the youngest child has blue eyes is $\frac{7}{16}$.
  \end{enumerate}
\end{ex}

\begin{ex}
  If $A$ and $B$ are independent events, then
  \[
    \cP{A}{B}=\frac{\P{A\cap B}}{\P{B}}=\frac{\P{A}\P{B}}{\P{B}}=\P{A}.
  \]

  For any events $A$ and $B$, the definition of conditional probability states
  that
  \[
    \cP{A}{B}=\frac{\P{A\cap B}}{\P{B}}\text{, and }
    \cP{B}{A}=\frac{\P{A\cap B}}{\P{A}}.
  \]
  Solving both for $\P{A\cap B}$ gives us
  \[
    \P{A\cap B}=\cP{A}{B}\P{B}=\cP{B}{A}\P{A}.
  \]
\end{ex}

\begin{ex}
  This is immediate from applying the definition of conditional probability
  twice, first to $A\,|\,B\cap C$ and then to $B\,|\,C$,
  \begin{align*}
    \P{A\cap B\cap C}
    =\cP{A}{B\cap C}\P{B\cap C}
    =\cP{A}{B\cap C}\cP{B}{C}\P{C}.
  \end{align*}
\end{ex}

\begin{ex}
  Let $\P{A_1}-\cP{A_1}{B}=\varepsilon$ and suppose that
  $\cP{A_i}{B}\leq \P{A_i}$ for all $i=2,\ldots,k$. Then
  \begin{align*}
    0 & = \sum_{i=1}^k\P{A_i}-\sum_{i=1}^k\cP{A_i}{B}                      \\
      & = \P{A_1}-\cP{A_1}{B}+\sum_{i=2}^k\left(\P{A_i}-\cP{A_i}{B}\right) \\
      & \geq \varepsilon,
  \end{align*}
  a contradiction. Therefore, $\cP{A_i}{B}> \P{A_i}$ for some $i$.
\end{ex}

\begin{ex}
  \begin{align*}
    \cP{W}{+}
     & =\frac{\cP{+}{W}\P{W}}{\P{+}}                                          &  & \text{(by Bayes' theorem)} \\
     & =\frac{0.82 \cdot 0.5}{0.65\cdot 0.3 + 0.82 \cdot 0.5 + 0.5 \cdot 0.2}                                 \\
     & =0.58.
  \end{align*}
\end{ex}

\begin{ex}
  \begin{enumerate}[(a)]
    \item Note that by Bayes' theorem,
          \[
            \cP{C_i}{H}
            =\frac{\cP{H}{C_i}\P{C_i}}{\sum_{j=1}^5 \cP{H}{C_j}\P{C_j}}
          \]
          where
          \[
            \sum_{j=1}^5 \cP{H}{C_j}\P{C_j}
            =\frac{1}{4}\cdot\frac{1}{5}+\frac{1}{2}\cdot\frac{1}{5}
            +\frac{3}{4}\cdot\frac{1}{5}+1\cdot\frac{1}{5}
            =\frac{1+2+3+4}{20}=\frac{1}{2}.
          \]

          Hence,
          \[
            \cP{C_1}{H} = 0,\,
            \cP{C_2}{H} = \frac{1}{10},\,
            \cP{C_3}{H} = \frac{1}{5},\,
            \cP{C_4}{H} = \frac{3}{10}\text{, and }
            \cP{C_5}{H} = \frac{2}{5}.
          \]
    \item
          \[
            \cP{H_2}{H_1}
            =\sum_{j=1}^5\cP{H}{C_j}\cP{C_j}{H}
            =0+\frac{1}{4}\cdot\frac{1}{10}
            +\frac{1}{2}\cdot\frac{1}{5}
            +\frac{3}{4}\cdot\frac{3}{10}
            +1\cdot\frac{2}{5}=\frac{3}{4}.
          \]
    \item Note that
          \[
            \cP{B_4}{C_1}=0,\,
            \cP{B_4}{C_2}=\frac{27}{256},\,
            \cP{B_4}{C_3}=\frac{1}{16},\,
            \cP{B_4}{C_4}=\frac{3}{256}\text{, and }
            \cP{B_4}{C_5}=0,
          \]
          and,
          \[
            \sum_{j=1}^5 \cP{B_4}{C_j}\P{C_j}
            =0+\frac{1}{5}\cdot\frac{27}{256}+\frac{1}{5}\cdot\frac{1}{16}
            +\frac{1}{5}\cdot\frac{3}{256}+0
            =\frac{23}{640}.
          \]
          Therefore, by Bayes' theorem,
          \[
            \cP{C_1}{B_4} = 0,\,
            \cP{C_2}{B_4} = \frac{27}{46},\,
            \cP{C_3}{B_4} = \frac{8}{23},\,
            \cP{C_4}{B_4} = \frac{3}{46}\text{, and }
            \cP{C_5}{B_4} = 0.
          \]
  \end{enumerate}
\end{ex}

\begin{ex}~
  \inputminted{python}{../code/01-21.py}

  \begin{figure}[H]
    \centering
    \includegraphics[scale=0.8]{../images/01-21a}
    \caption{Proportion of heads as a function of the number of coin flips for
      $p_\text{heads}=0.3$.}
  \end{figure}

  \begin{figure}[H]
    \centering
    \includegraphics[scale=0.8]{../images/01-21b}
    \caption{Proportion of heads as a function of the number of coin flips for
      $p_{heads}=0.03$.}
  \end{figure}

\end{ex}

\begin{ex}~
  \inputminted{python}{../code/01-22.py}
  \inputminted{text}{../output/01-22.txt}
\end{ex}

\begin{ex}~
  \inputminted{python}{../code/01-23.py}
  \inputminted{text}{../output/01-23.txt}

  In the case of $A=\{2, 4, 6\}$ and $B=\{1, 2, 3, 4\}$, the theoretical values
  are $\P{A}=\frac{1}{2}$, $\P{B}=\frac{2}{3}$ and $\P{AB}=\frac{1}{3}$.

  For $A=\{1, 2, 3, 4\}$ and $B=\{3, 4, 5, 6\}$, the theoretical values are
  $\P{A}=\frac{2}{3}$, $\P{B}=\frac{2}{3}$ and $\P{AB}=\frac{1}{3}$.
\end{ex}


\lab{Algorithms}{Jordan Forms and Schur Decomposition}{Jordan Forms and Schur Decomposition}
\label{Ch:Jordan}

\objective{Learn about Jordan forms, what we would like to use them for, and why we use Schur decomposition instead.}

\textbf{Outline}
\begin{itemize}
	\item Recall that Jordan forms are (blah blah blah)
	\item Jordan forms are cool because it's good to know the eigenvalues and eigenvectors.
	\item Unfortunately the Jordan form is really hard to compute. Abel's Impossibility Theorem suggests we might have to iterate. Most eigenvalue solvers these days are iterative algorithms that find the Schur decomposition. Like the QR algorithm, which you've already done a lab about.

\end{itemize}

\section*{Schur Decomposition}
Schur decomposition relies on the following theorem.

\begin{theorem}
{\bf Schur's Theorem:} For any complex $n \times n$ matrix $A$, there exists a unitary matrix $Q$ such that $A = Q^\ast U Q$, where $U$ is upper triangular.
\end{theorem}

%\footnotetext{The requirement that $A$ be complex doesn't mean we're in trouble if $A$ only has real entries. It just means we have to allow complex entries in the Schur decomposition for the theorem to hold in general. If $A$ is real and has all real eigenvalues, then the Schur decomposition of $A$ will also be real.}

The eigenvalues of $A$ are the diagonal entries of $U$. (Why?) But it follows from Abel's Impossibility Theorem (Theorem \ref{Theorem:Abel}) that no finite algorithm can compute the eigenvalues of $A$ exactly. (How does this follow?) Therefore, no finite algorithm can directly compute the Schur factorization of A.


\input{../header.tex}
\title{\vspace{-2cm}INF3490/INF4490 Exercises - Simple search algorithms}
\author{Eivind Samuelsen\input{../author_footnote.tex}}
\date{}

% Removing paragraph indents is sometimes useful:
\setlength\parindent{0pt}
% ==============================================================================

% ================================= DOCUMENT ===================================
\begin{document}
    \renewcommand\marginsymbol[1][0pt]{%
  \tabto*{0cm}\makebox[-1cm][c]{$\mathbb{P}$}\tabto*{\TabPrevPos}}

\maketitle
\input{../intro.tex}

\section{Simple search algorithms}

Given the function \(f(x) = -x^4 + 2x^3 + 2x^2 - x\):

\subsection{Derivative}
What is its derivative \(f'(x)\) ?

\subsection{Plotting \marginsymbol}
Plot the function, and its gradient(derivative) from \(x=-2\) to \(x=3\).
Use python, wolfram alpha or another plotting tool of your choice.

\subsection{Gradient Ascent \marginsymbol}
\label{subsec:grada}
Maximize using gradient ascent.
You can try step size 0.1 and start somewhere in the range [-2, 3].
How does the choice of starting point and step size affect the algorithm's performance?
Is there a starting point where the algorithm would not even be able to find a local maximum?

\subsection{Exhaustive Search \marginsymbol}
\label{subsec:exhaust}
Assume that we are only interested in maxima of \(f(x)\),
where \(-2\leq x \leq 3\),
and \(x\) increases in steps of length 0.5 (\(\Delta x = 0.5\)).
Perform an exhaustive search to maximize \(f(x)\) and plot the result.

\subsection{Greedy Search and Hill Climbing}
In what way would greedy search and hill climbing differ for the maximization problem in Problem \ref{subsec:grada}?
Can you identify a starting position where the two algorithms might give different results?

\subsection{Possible improvements}
Gradient ascent, greedy search and hill climbing are quite similar, and are all based almost exclusively on exploitation.
Can you think of any additions to these algorithms in order to do more exploration?

\subsection{Exhaustive search vs. simulated annealing}
Which algorithm do you think is the most efficient at maximizing \(f(x)\) under the conditions in Problem \ref{subsec:exhaust};
exhaustive search or simulated annealing? Explain.

\input{../contact.tex}

\end{document}
% ==============================================================================


%
% CMPT 310: Artificial Intelligence - A Course Overview
% Section: Machine Learning
%
% Author: Jeffrey Leung
%

\section{Machine Learning}
	\label{sec:machine-learning}
\subsection{Introduction}
	\label{subsec:ml-introduction}
\begin{easylist}

& \textbf{Machine learning:} Approximation of a function which detects the classification of a problem given attributes of data

& \textbf{Classification:} Category of data
& \textbf{Attribute:} Characteristics of a problem using a variable (boolean, discrete, continuous, etc.)
& Epoch: Round of training data

& Types of machine learning:
	&& Supervised learning
	&& \textbf{Semi-supervised learning:} Machine learning technique which classifies data through processing a combination of pre-classified data and unclassified data
	&& \textbf{Unsupervised learning:} Machine learning technique which classifies data and finds patterns through processing previously unclassified data
		&&& E.g. Clustering
	&& \textbf{Reinforcement learning:} Machine learning technique which takes actions to obtain a score, and repeats to find the correct actions to obtain the optimal score
		&&& Used for robotics, games

\end{easylist}
\subsection{Supervised Learning}
	\label{subsec:supervised-learning}
\begin{easylist}

& \textbf{Supervised learning:} Machine learning technique which classifies data through processing provided pre-classified data

& \textbf{Episodic problem (ML):} Situation in supervised learning where a dataset is built from discrete data points
	&& E.g. Spam checker

& \textbf{Inductive learning:} Approximation of a function on the attributes of a problem which produces a label
	&& Notation: Ideal function $f(features) \rightarrow label$; find $h \approx f$
	&& \textbf{Label:} Resulting classification
	&& Highly similar to \href{http://onlinestatbook.com/2/regression/intro.html}{linear regression}
	&& E.g. Finding a best-fit line to a set of datapoints
	&& Choose the simplest and most generalizable model $h$ (Occam's razor)

& \textbf{Hypothesis space:} Set of models $h$ which are valid
	&& Given $n$ boolean functions, there are $2^n$ values in the dataset and $2^{2^n}$ unique possible assignments of results to the dataset
	&& E.g. Neural networks, logistic regressions, support vector machines, random forests

& Dataset training:
	&& {Training data:} Dataset which is applied to a neural network to provide neurons with weights and biases
		&&& The greater the data for training, the better the model created
	&& \textbf{Validation data:} Dataset which is applied to a neural network with known optimal results to test accuracy
	&& \textbf{Testing data:} Dataset which is applied to a neural network after training and validation
		&&& Cannot be the same as training data
		&&& The greater the data for testing, the better the estimate of accuracy
	&& Cross-validation: Splitting (`folding') the original dataset into equal subsets, one for testing, one for validation, and the remaining for training
		&&& Strategy: Cross-validate multiple times with each fold to find the most accurate fold
	&& \textbf{Overfitting:} Being overly specific with training data such that testing data is no longer accurate

\end{easylist}
\subsection{Decision Trees}
	\label{subsec:decision-trees}
\begin{easylist}

& \textbf{Decision tree:} Tree where each node represents a decision with multiple choices and each leaf represents a unique choice
	&& Restricting hypothesis space: Limit on depth
	&& Split paths on most informative features (which result in the lowest resulting entropy)
		&&& \textbf{Entropy:} Uncertainty of information (in this context, calculated as similarity of ratio between choices)
			&&&& Notation: $H$ where $0 \leq H \leq 1$
			&&&& Unit: Bits
			&&&& Calculation: $H(p_1, p_2, \dotsc, p_n) = - \sum_i p_i \log_2 p_i$
			&&&& E.g. $H(0.5, 0.5) = - \frac{1}{2} \log \frac{1}{\frac{1}{2}} = 1$
			&&&& E.g. $H(0, 1) = - 0 \cdot \log \frac{1}{0} - 1 \cdot \log \frac{1}{1} = \lim_{x \rightarrow 0} x \log \frac{1}{x} = 0$
			&&&& E.g. $H(\frac{1}{3}, \frac{2}{3}) = 0.918\dots$
		&&& To find the reduction in uncertainty, calculate:

		$$H(root) - \sum_{i \in \textrm{children}} \frac{\textrm{samples of } i}{\textrm{samples of root}} H(i)$$

		&&& See diagram and patrons graph

\end{easylist}
\subsection{Neural Networks}
	\label{subsec:neural-networks}
\begin{easylist}

& \href{https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi}{\textbf{Neural network:}} Combination of neuron structures through which data flows and is manipulated, to create a set of output values
	&& \textbf{Bias:} Value which is added as a standalone neuron input (typically $-1$)
	&& \textbf{Weight:} Value by which any neuron input is multipled
	&& \href{http://playground.tensorflow.org}{Neural network simulation}

& \textbf{Activation function:} Simple function applied to the sum of weighted neuron inputs to create a neuron output value
	&& Notation: $g: \mathbb{R} \rightarrow \mathbb{R}$
	&& Types of functions:
		&&& \textbf{Threshold function:} Activation function which has the output value -1 for negative input values, and the output value 1 for positive input values
		\end{easylist}
		\[
			a = \begin{dcases}
				-1 & \textrm{if } in < 0 \\
				1 & \textrm{otherwise}
			\end{dcases}
		\]
		\begin{easylist}

		&&& \textbf{Sigmoid/logistic function:} Activation function which changes output value gradually from 0 to 1, and has output value 0.5 at input value 0
		\end{easylist}
		\[
			a = \frac{1}{1+exp_e(-in)}
		\]
		\begin{easylist}

		&&& \textbf{Rectifier Linear Unit (ReLU) function:} Activation function which has output value 0 for negative values, and output value equal to the input value for positive values
		\end{easylist}
		\[
			a = max(0, in) = \begin{dcases}
				0 & \textrm{if } in \leq 0 \\
				in & \textrm{otherwise}
			\end{dcases}
		\]
		\begin{easylist}

& Process of training a network:
	&& \textbf{Squared error:}
	\end{easylist}
	\begin{align*}
		Err
		&= y - h_w(x) \\[1em]
		\textrm{Squared error } E
		&= \frac{Err^2}{2} \\
		&= \frac{(y - h_w(x))^2}{2} \\[1em]
		\textrm{where }
		x &= \textrm{ input value} \\
		y &= \textrm{ true/ideal output value}
	\end{align*}
	\begin{easylist}

		&&& Using \href{https://www.youtube.com/watch?v=IHZwWFHWa-w&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi&index=2}{gradient descent} to find the optimal value (one iteration per attribute):
		\end{easylist}
		\begin{align*}
			\frac{\partial E}{\partial W_j}
			&= Err \times \frac{\partial E}{\partial W_j} \\
			&= Err \times \frac{\partial}{\partial W_j} (y - g(\sum_{j=0}^n W_j x_j)) \\
			&= -Err \times g'(in) \times x_j
		\end{align*}
		\begin{easylist}
		&&& Given the error, update the weight by:
		\end{easylist}
		\begin{align*}
			W_j &\leftarrow W_j + (\alpha \times Err \times g'(in) \times x_j) \\[1em]
			\textrm{where }
			\alpha
			&= \textrm{ rate of learning correction}
		\end{align*}
		\begin{easylist}
	&& \href{https://www.youtube.com/watch?v=Ilg3gGewQ5U&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi&index=3}{\textbf{Back propagation:}} Process of training a neural network by altering weights and biases to provide the intended results, using the outputs of training data
		&&& Involves \href{https://www.youtube.com/watch?v=tIeHLnjs5U8&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi&index=4}{calculus}
	\[
		\delta_{out} = (y - a_{out}) \cdot g'(in_{out})
	\]

& \textbf{Single-layer perceptron:} Neural network consisting of a single neuron
	&& \textbf{Linearly separable:} Property of a multidimensional dataset which can be classified using a single linear function
		&&& A function can be represented by a single-layer perceptron if and only if its data is linearly separable

& \textbf{Multi-layer perceptron:} Neural network consisting of sets of multiple interconnected neurons where every node in each layer depends on at least one node in the layer immediately prior, and does not depend on nodes in other layers
	&& \textbf{Hidden layer:} Layer of neurons which is neither the input layer nor the output layer
	&& Acyclic digraph (i.e. has a topological order)
	&& Multi-layer perceptron with 2 hidden layers (3 total layers of processing) can emulate any possible function
	&& \textbf{Multi-task learning:} Model of multi-layer neural network which has consistent hidden layer processing but has multiple unique outputs
		&& E.g. Image content differentiator

& \href{https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/}{\textbf{Convolutional layer:}} Layer component of a neural network which handles prestructured input by abstracting and processing components of the input, often to fewer inputs
	&& E.g. Image consisting of a 2-d grid of pixels, squares of which are taken and compressed
	&& \textbf{Filter:} Structured subset of input values
	&& \textbf{Pooling:} Combination of multiple inputs (often the depth of a 3D matrix) into a single output (e.g. summation, multiplication, averaging, maximization)
& \textbf{Recurrent network:} Acyclic digraph which is structured as a 2D matrix, with a set of input nodes flowing to a set of output nodes
	&& E.g. Conversational bot

\end{easylist}
\clearpage


\section{Model}
\label{sec:model}

\subsection{Distribution network}

In this section, we present the model we used to solve the problem of ANM in DSO grid.
In particular, we will show how this problem can be cast as a MDP and how RL can be included when the distribution of the uncertainty is unknown and we will highlight the modeling assumptions used.

The electrical network can be mathematically modeled as a graph whose edges represent the electrical lines and whose nodes are locations in the network where power can be consumed or injected.
In a typical distribution network, the nodes correspond to households or small factories that behave as prosumers.
In this work, we will make the assumption that the graph is a tree as it is the case of most distribution networks.
Let $\mathcal{N}$ be the set of nodes and $\mathcal{E}$ be the set of edges.
Let $\mathcal{G}$ be the graph representing the network, i.e. $\mathcal{G} = (\mathcal{N}, \mathcal{E})$.

We represent every node by an integer in $\{0, \dots, n\}$ where $n = |\mathcal{N}|-1$ is the cardinality of set $\mathcal{N}$.
The root of the tree is the node that connects the distribution network to the transmission network and is the node $0$ in our model and we denote by $\mathcal{N}_+ = \mathcal{N} \backslash \{0\}$.
Each node $i \in \mathcal{N}_+$ has a unique ancestor denoted by $A_i$ and each node $i \in \mathcal{N}$ has a set of children denoted by $\mathcal{C}_i$.
We choose the orientation of the lines from being from $i$ to $A_i$ so that we can unambiguously represent each line by its origin node.
We thus have $\mathcal{E} = \{1, \dots, n\}$.
We will define the following nodal and branch quantities :

For each node $i \in \mathcal{N}$, we define :
\begin{itemize}
  \item $v_i$ as the square of the norm of the complex voltage at that node,
  \item $s_i = p_i + \mathbf{i} q_i$ as complex net power injection (the net injection being the power consumed minus the power produced)
  \item $d_i$ is the active net power injection demanded at node $i$, i.e. the power that the agent at that node wants to reinject in the network.
  It is thus different than the power that will actually be seen by the network (that is, $p_{i}$) as some curtailment mechanisms can take place.
  This will be further discussed later.
  It is positive when the power is injected and negative if it is retrieved.
\end{itemize}

For each line $i \in \mathcal{E}$, we define :
\begin{itemize}
  \item $z_i = r_i + \mathbf{i}x_i$ as the complex impedance,
  \item $l_i$ as the square of the norm of the complex current,
  \item $S_i = P_i + \mathbf{i}Q_i$ as the sending-end complex power, where $P_i$ denote the active power and $Q_i$ the reactive power.
\end{itemize}

A schematic representation of a line together with the notations we use is given in Figure~\ref{fig:line}.

\begin{figure}
  \begin{center}
    \begin{tikzpicture}[scale=8]
        \draw [->] (0,0.7) -- (0,0.53);
        \draw [->] (0.8,0.7) -- (0.8,0.53);
        \draw [thick] (0,0.5) -- (0.8, 0.5);
        \draw [<-] (0.25, 0.45) -- (0.55, 0.45);
        \node [left] at (0,0.7) {$s_{A_i}$};
        \node [right] at (0.8,0.7) {$s_i$};
        \node [below, left] at (0,0.5) {$v_{A_i}$};
        \node [below, right] at (0.8,0.5) {$v_i$};
        \node [above] at (0.4, 0.5) {$z_i$};
        \node [below] at (0.3, 0.45) {$l_i$};
        \node [below] at (0.5, 0.45) {$S_i$};
    \end{tikzpicture}
  \caption{Schematic representation of a line.}
  \label{fig:line}
  \end{center}
\end{figure}

We consider the problem in a multi-time-step framework.
This means that the variables are defined at each time step.
We consider a horizon of $T$ time steps.
The set of time steps is denoted by $\mathcal{T}$ and each one is represented by a positive integer, i.e. $\mathcal{T} = \{1, \dots, T\}$.

Moreover, we consider the context of a distribution system with a high level of renewable energy production.
We consider that each house has a PV installation and that the overproduction is reinjected in the network.
Therefore, the net power injection at each node can be decomposed into a power produced $p_{i,t}^p$ and a power consumed $p_{i,t}^c$ such that

\[ p_{i,t} = p_{i,t}^p - p_{i,t}^c \]

\subsection{Power flow equations}

For a radial distribution network, the equations that link all nodal and branch electrical quantities by taking \emph{Kirchhoff}'s laws into account are called the \emph{Power Flow equations}.
They can be written as~\cite{Farivar_Relax1}

\begin{align}
  & 0  =  p_{0,t} + \sum_{j\in \mathcal{C}_0} \Big(P_{j,t}-r_{j}l_{j,t} \Big), \hspace{0.5cm} \forall{t} \in \mathcal{T} \label{ACOPF1}\\
  & 0   =  q_{0,t} + \sum_{j\in \mathcal{C}_0} \Big( Q_{j,t}-x_{j}l_{j,t} \Big), \hspace{0.5cm} \forall{t} \in \mathcal{T} \label{ACOPF2}\\
  & v_{\mathcal{A}_i,t}  =   v_{i,t} -2(r_iP_{i,t} + x_iQ_{i,t}) + (r_i^2 + x_i^2)l_{i,t}, \\
  & \hspace{0.5cm} \forall i \in \mathcal{N}_{+}, \hspace{0.2cm} \forall{t} \in \mathcal{T} \label{ACOPF3}\\
  & P_{i,t} =  p_{i,t} + \sum_{j\in \mathcal{C}_i} \Big( P_{j,t}-r_{j}l_{j,t} \Big), \hspace{0.5cm} \forall i \in \mathcal{N}_{+}, \hspace{0.2cm} \forall{t} \in \mathcal{T} \label{ACOPF4}\\
  & Q_{i,t}  =  q_{i,t} + \sum_{j\in \mathcal{C}_i} \Big( Q_{j,t}-x_{j}l_{j,t} \Big), \hspace{0.5cm} \forall i \in \mathcal{N}_{+}, \hspace{0.2cm} \forall{t} \in \mathcal{T} \label{ACOPF5}\\
  & v_{i,t}l_{i,t} = P_{i,t}^2 + Q_{i,t}^2, \hspace{0.5cm} \forall i \in \mathcal{N}_{+}, \hspace{0.2cm} \forall{t} \in \mathcal{T} \label{ACOPF:quad}
\end{align}

With these equations, the power injection at each node is sufficient to obtain the values of all other electrical quantities.

\subsection{Uncertainty}

These power injection, however, is subject to uncertainty.
They depend on the consumption and the production at each node.
We consider that forecast values, denoted by $\tilde{p}_{i,t}^p$ and $\tilde{p}_{i,t}^c$ are available to the DSO and the uncertainty can thus be modeled as the deviation from these forecast values.
Let us denote by $\delta_t^p$ and $\delta_t^c$ the deviation for, respectively, the production and the consumption at time step $t$.
These deviations are common to all nodes.
This is motivated by the fact that this uncertainty usually relies to external factor as solar irradiance, temperature, etc.
When the uncertain deviations are known, the power injections can be written as
\[
   p_{i,t}^p = \tilde{p}_{i,t}^p + \delta_t^p\tilde{p}_{i,t}^p
  \]
and
\[
   p_{i,t}^c = \tilde{p}_{i,t}^c + \delta_t^c\tilde{p}_{i,t}^c
  \]
We consider the following discrete sample space $\mathcal{P}$ for both type of deviation :
\[
  \mathcal{P} = \{0.1, 0, -0.1\}
\]
with an unknown probability distribution.
This means that we suppose that both the consumption and the production can either be $10\%$ more than the forecast value, exactly the forecast value or $-10\%$ of the forecast value.

\subsection{Flexible loads}
In modern distribution networks, some loads are more flexible than others.
Flexible loads can include small factories who sign contracts with the DSO allowing it to disconnect the factory from the network for some time.
In this work, we consider that there is a set $\mathcal{F}$ of flexible loads.
They are either in the state \emph{on} or \emph{off}.
For these loads, we consider that there are no production but only consumption.

\subsection{Set of states}

The state of the system is made of the time step in which we are, the level of the deviation from the forecast values for both the consumption and the production and the state of the flexible loads.
The state is represented by a vector of length $2+|\mathcal{F}|$ where the two first elements correspond to the deviation from respectively the production and the consumption and the $|\mathcal{F}|$ last elements are the binay states of the flexible loads ($0$ or $1$).
We denote by $\mathcal{S}$ the set of states and by $\mathcal{S}_t$ the set of states for a fixed $t$.
We have that
\begin{align*}
  \mathcal{S}_t = \{&(0.1, 0.1, 0, \dots, 0), (0, 0.1,0, \dots, 0), \dots, \\
                    &(-0.1, -0.1,1, \dots, 1)\} \\
\end{align*}
Note that $|\mathcal{S}_t| = 3^2 2^{|\mathcal{F}|} = 9 \cdot 2^{|\mathcal{F}|}$ and that $\mathcal{S}_t$ is independent of $t$.

\subsection{Control actions}

We consider that the action the DSO has to control its network is the curtailment of all or part of the PV production at each node and the choice of activating or not each flexible load, all of these at each time step.
By curtailing excessive solar production, the reliability of the network could be greatly increased.
This curtailment is also expressed as a percentage of the total injection seen by the network.
We denote by $\overline{p}_{i,t}$ the curtailment decided for node $i \in \mathcal{N}$ and $t \in \mathcal{T}$ so that the total power injected after curtailment is written as
\[
  p^{inj}_{i,t} = p_{i,t} - \overline{p}_{i,t}p_{i,t}
\]
We suppose the following set of curtailment actions $\mathcal{U}_{\text{load}}$ independent of the state :
\[
  \mathcal{U}_{\text{load}} = \{0, 0.5, 1\}
\]
We thus have $\overline{p}_{i,t} \in \mathcal{U}_{\text{load}}$ for all node and time step.
This means that we consider that the DSO can curtail either nothing, half of the production or the entire production.

The set of flexible load actions, denoted by $\mathcal{U}_{\text{flex}}$, depends on the state of the flexible loads.
If the flexible load is on, we can either deactivate it or not.
When it is off, we can only let it off.

A diagram representing in a simple fashion how the states of uncertainty
and the control actions should be seen can be found in figure~\ref{fig:model_diag}.

\subsection{Cost}
The cost associated to an action in a certain state is made of the three following elements :
\begin{itemize}
  \item The cost of congestion. It is the cost associated to a power flow that would result in breaking the voltage and current limits.
  \item The cost of curtailment. It is the cost associated to throwing away free green energy.
  \item The cost of flexible load disconnection.
\end{itemize}

We will model both cost by linear functions.
Let us suppose that the following limits on voltage and current have to hold :
\begin{align*}
  \underline{v} \le &v_{i,t} \le \overline{v}\\
  &l_{i,t} \le \overline{l}
\end{align*}

The cost of congestion is computed as
\begin{align*}
  \ccong &= C_c \Big( \sum_{i \in \mathcal{N}} -(\overline{v}-v_{i,t})_+ - (v_{i,t}-\underline{v})_+  \\
  &+ \sum_{j \in \mathcal{N}_+} -(\overline{l}-l_{j,t})_+ \Big)
\end{align*}
while the cost of curtailment is computed as
\[
  \ccurt = \sum_{i \in \mathcal{N},t \in \mathcal{T}} C_e(\overline{p}_{i,t}p_{i,t})
\]
and the cost of flexible load is
\[
  \cflex = \sum_{d \in \mathcal{F},t \in \mathcal{T}} C_f(\text{deact}_{d,t})
\]
where $C_c$ is the cost of curtailment (here, $8300\si{\euro\per\mega\watt\hour}$), where $C_f$ is the cost of deactivating a flexible load (here, $1\si{\euro\per\mega\watt\hour}$) and where $C_e$ is the usual cost of energy, taken as $300\si{\euro\per\mega\watt\hour}$ and where $(\cdot)_+$ denotes the positive part function.

\begin{figure}
  \begin{center}
    \includegraphics[scale=0.43]{./img/model_diagram.png}
  \end{center}
  \caption{Schematic representation of the states and control actions.}
  \label{fig:model_diag}
\end{figure}


\chapter{Statistical Mechanics of Multi-Level Atoms and Molecules}
\label{app:multilevel_atoms}

This appendix provides a full mathematical treatment of the statistics of multi-level atoms and molecules out of thermodynamic equilibrium, including the effects of a background radiation field. This appendix is intended as a reference rather than a full derivation, and so at several points we assert results without proof. Full demonstrations of these results can be found in standard references such as \citet{rybicki86a}, \citet{shu91a}, or \citet{draine11a}.

\section{Matter-Radiation Interaction}

A general radiation field can be specified in terms of the radiation intensity $I(\nu, \mathbf{n})$ at any point in space $\mathbf{x}$ and time $t$; here $\nu$ is the frequency of radiation and $\mathbf{n}$ is a unit vector specifying the direction of radiation propagation. The intensity specifies the amount of radiant energy per unit area per unit frequency per unit solid angle. An alternative representation of the radiation field, which is more useful when dealing with problems in statistical mechanics, is the photon occupation number, defined by
\begin{equation}
n_\gamma(\nu, \mathbf{n}) = \frac{c^2}{2h\nu^3} I(\nu, \mathbf{n}).
\end{equation}
Physically, the photon occupation number is the number of quanta (photons) in a particular mode, and is dimensionless. In local thermodynamic equilibrium (LTE) at temperature $T$, the radiation intensity in all directions $\mathbf{n}$ is given by the Planck function
\begin{equation}
I(\nu,\mathbf{n}) = B_\nu(T) = \frac{2h\nu^3}{c^2}\frac{1}{e^{h\nu/k_B T} - 1}.
\end{equation}
The equivalent photon occupation number is
\begin{equation}
\label{eq:ngamma_LTE}
n_{\gamma,\rm LTE}(\nu,\mathbf{n}) = \frac{1}{e^{h\nu/k_B T} - 1}.
\end{equation}

For non-relativistic problems, the rates at which photons are emitted or absorbed by atoms undergoing a particular quantum mechanical transition does not depend upon the direction of photon propagation, and thus it is convenient to average over the direction $\mathbf{n}$. We define the directionally-averaged photon occupation number by
\begin{equation}
\langle n_\gamma\rangle(\nu) = \frac{1}{4\pi} \int n_\gamma(\nu, \mathbf{n}) \, d\Omega,
\end{equation}
where the integral is over all directions $\mathbf{n}$.

Now consider a particle of species $X$ with two quantum states that we will denote $u$ and $\ell$, with energies $E_u$ and $E_\ell$, ordered so that $E_u > E_\ell$. The states have degeneracies $g_u$ and $g_\ell$, respectively. Particles in state $u$ can spontaneously emit photons and transition to state $\ell$ with an $e$-folding timescale $A_{u\ell}$. Formally, if $n_u$ is the number density of particles in state $u$, then
\begin{equation}
\left(\frac{dn_u}{dt}\right)_{\rm spon.~emiss.} = -n_u A_{u\ell}.
\end{equation}
Particles in state $\ell$ can also absorb photons at frequencies $\nu$ near $\nu_{u\ell} = (E_u-E_\ell)/h$ and transition to state $u$, and the absorption rate is proportional to $\langle n_\gamma\rangle(\nu_{u\ell})$ and $n_\ell$, where $n_\ell$ is the number density of particles in state $\ell$. Finally, the presence of photons with frequencies near $\nu_{u\ell}$ can cause stimulated emission, whereby particles in state $u$ emit a photon and transition to state $\ell$; again, the rate at which this process occurs must be proportional to both $\langle n_\gamma\rangle(\nu_{u\ell})$ and $n_u$. We write the rates of these two processes as $(dn_u/dt)_{\rm abs.} \propto n_\ell \langle n_\gamma\rangle(\nu_{u\ell})$ and $(dn_u/dt)_{\rm stim.~emiss.} \propto -n_u \langle n_\gamma\rangle(\nu_{u\ell})$. Putting these processes together, the total rate of change of $n_u$ is given by
\begin{eqnarray}
\frac{dn_u}{dt} & = & \left(\frac{dn_u}{dt}\right)_{\rm spon.~emiss.} + \left(\frac{dn_u}{dt}\right)_{\rm stim.~emiss.} + \left(\frac{dn_u}{dt}\right)_{\rm abs.} \\
& = & -n_u A_{u\ell} - C_{u\ell} n_u \langle n_\gamma\rangle(\nu_{u\ell}) + C_{\ell u} n_\ell \langle n_\gamma\rangle(\nu_{u\ell}),
\label{eq:dnudt}
\end{eqnarray}
where the two constants of proportionality $C_{u\ell}$ and $C_{\ell u}$ are to be determined.

Consider a region where the number density of particles is so low that collisions occur negligibly often. However, the particles can still be in LTE with the radiation field. Let $n_\ell$ be the number density of particles in state $\ell$. In LTE the values of $n_u$ and $n_\ell$ must be related by the usual Boltzmann factor, so
\begin{equation}
n_u = \frac{g_u}{g_\ell} e^{-h\nu_{u\ell}/k_B T} n_\ell.
\end{equation}
The directionally-averaged photon occupation number must take on its LTE value
\begin{equation}
\label{eq:ngamma_avg_LTE}
\langle n_\gamma\rangle(\nu) = \frac{1}{e^{h\nu/k_B T} - 1}.
\end{equation}
Inserting these values of $n_u$ and $\langle n_\gamma\rangle$ into equation (\ref{eq:dnudt}), and noting that we must have $dn_u/dt = 0$ for a system in LTE, we have
\begin{equation}
-\frac{g_u}{g_\ell} e^{-h\nu_{u\ell}/k_B T} \left(A_{u\ell} + \frac{C_{u\ell}}{e^{h\nu/k_B T} - 1}\right) + \frac{C_{\ell u} }{e^{h\nu_{u\ell}/k_B T} - 1} = 0.
\end{equation}
For temperatures $T$ such that $h\nu_{u\ell} \ll k_B T$, all the exponential terms approach unity, and thus the two terms proportional to $C_{u\ell}$ and $C_{\ell u}$ are far larger than the term proportional to $A_{u\ell}$. Dropping this term, we immediately see that the equation can be satisfied only if
\begin{equation}
C_{\ell u} = \frac{g_u}{g_\ell} C_{u\ell}.
\end{equation} Conversely, for temperatures $T$ such that $h\nu_{u\ell} \gg k_B T$, the terms in the exponentials are large. We can therefore drop the $-1$ terms in the denominators, and neglect $C_{u\ell}/e^{h\nu_{u\ell}/k_B T}$ in comparison to $A_{u\ell}$. Doing so, we immediately obtain
\begin{equation}
C_{\ell u} = \frac{g_u}{g_\ell} A_{u\ell}.
\end{equation}

Inserting these results into our expressions for the rates of stimulated emission and absorption, we finally have
\begin{eqnarray}
\left(\frac{dn_u}{dt}\right)_{\rm stim.~emiss.} & = & n_u \langle n_\gamma\rangle(\nu_{u\ell}) A_{u\ell} \\
\left(\frac{dn_u}{dt}\right)_{\rm abs.} & = & \frac{g_u}{g_\ell} n_\ell \langle n_\gamma\rangle(\nu_{u\ell}) A_{u\ell}.
\end{eqnarray}

\section{Statistical Equilibrium for Multi-Level Systems}

Now let us consider some species with a series of possible quantum states. We number them $0, 1, 2, \ldots$ in order of increasing energy, so state $0$ is the ground state. We denote the energy and degeneracy of state $i$ as $E_i$ and $g_i$ respectively. We write the energy difference between any two states as $E_{ij} = E_i - E_j$,  the corresponding frequency as $\nu_{ij} = E_{ij}/h$, and we write the Einstein spontaneous emission coefficient for transitions from state $i$ to state $j$ as $A_{ij}$. The species of interest has number density $n$, and we let $n_i$ be the number density of that species in state $i$. Finally, the species of interest can undergo collisions with another species or with itself, and these can cause state transitions as well. We let $n_c$ be the number density of colliders, and we let $k_{ij}$ be the collision rate coefficient connecting any two states, so that the rate of collisionally-induced transitions from state $i$ to state $j$ is given by $n_i n_c k_{ij}$.

Given this setup, we can write out the rates of all processes that induce changes in the number density of any quantum state. Specifically, the rates of collisional transitions out of and into state $i$ are
\begin{eqnarray}
\left(\frac{dn_i}{dt}\right)_{\rm coll.~out} & = & -n_i n_c \sum_j k_{ij} \\
\left(\frac{dn_i}{dt}\right)_{\rm coll.~in} & = & n_c \sum_j n_j k_{ji}.
\end{eqnarray}
Here the first expression is a sum over the rate of collisional transitions from state $i$ to all other states, while the second is a sum over the rate of collisional transitions from all other states to state $i$. By convention we take $k_{ii} = 0$, i.e., we set the rate of collisional transitions from a state to itself to zero. The corresponding rates of transition out of and into state $i$ via spontaneous emission are
\begin{eqnarray}
\left(\frac{dn_i}{dt}\right)_{\rm spon.~emiss.~out} & = & -n_i \sum_j A_{ij} \\
\left(\frac{dn_i}{dt}\right)_{\rm spon.~emiss.~in} & = & \sum_j n_j A_{ji},
\end{eqnarray}
where we adopt the convention that $A_{ij} = 0$ for $i \leq j$, i.e., the spontaneous transition rate from a lower energy state to a higher energy one is zero. Finally, the expressions for stimulated emission- and absorption-induced transitions are
\begin{eqnarray}
\left(\frac{dn_i}{dt}\right)_{\rm stim.~emiss.~out} & = & -n_i \sum_j A_{ij} \langle n_{\gamma,ji}\rangle \\
\left(\frac{dn_i}{dt}\right)_{\rm stim.~emiss.~in} & = & \sum_j n_j A_{ji} \langle n_{\gamma,ji}\rangle \\
\left(\frac{dn_i}{dt}\right)_{\rm abs.~out} & = & -n_i \sum_j \frac{g_j}{g_i} A_{ij} \langle n_{\gamma,ij}\rangle \\
\left(\frac{dn_i}{dt}\right)_{\rm abs.~in} & = & \sum_j \frac{g_i}{g_j} n_j A_{ji} \langle n_{\gamma,ij}\rangle,
\end{eqnarray}
where for convenience we have introduced the shorthand $\langle n_{\gamma,ij}\rangle \equiv \langle n_{\gamma}\rangle(\nu_{ij})$. Note that, per our convention that $A_{ij}$ is non-zero only for $i > j$, the terms in the sums for stimulated emission are non-zero only for states $j > i$, while the terms in the sums for absorption are non-zero only for states $j < i$.

Combining all of the above expressions, we can write out the full rate of change for the number density of particles in each state $i$ as
\begin{eqnarray}
\frac{dn_i}{dt} & = & \sum_j n_j\left[n_c k_{ji} + \left(1+\langle n_{\gamma,ji}\rangle\right) A_{ji}\right] +
 \sum_j n_j \frac{g_i}{g_j} \langle n_{\gamma,ij}\rangle A_{ij}
\nonumber \\
& & \qquad {} - n_i \sum_j \left[n_c k_{ij} + \left(1+\langle n_{\gamma,ij}\rangle\right) A_{ij}\right]
\nonumber \\
& & \qquad {} -
 n_i \sum_j \frac{g_j}{g_i} \langle n_{\gamma,ji}\rangle A_{ji}.
\label{eq:stat_eq}
\end{eqnarray}
If the system is in statistical equilibrium (but not necessarily LTE), then $dn_i/dt = 0$ for all states $i$. In this case the set of equations (\ref{eq:stat_eq}) represents a set of linear equations to be solved for the unknown number densities $n_i$. With some algebraic manipulation, one can express this system as a matrix equation
\begin{equation}
\mathbf{M} \cdot \mathbf{n} = \mathbf{n},
\end{equation}
where $\mathbf{n} = (n_0, n_1, n_2, \ldots)$ is the vector of number densities, and the matrix $\mathbf{M}$ has elements
\begin{equation}
M_{ij} = \frac{n_c k_{ji} + \left(1 + \langle n_{\gamma,ji}\rangle\right) A_{ji} + \frac{g_i}{g_j} \langle n_{\gamma,ij}\rangle A_{ij}}
{ \sum_\ell \left[n_c k_{i\ell} + \left(1 + \langle n_{\gamma,i\ell}\rangle\right) A_{i\ell} + \frac{g_\ell}{g_i} \langle n_{\gamma,\ell i}\rangle A_{\ell i}\right] }.
\end{equation}
The matrix $\mathbf{M}$ is therefore specified entirely in terms of the known rate coefficients, degeneracies, and radiation fields, and the problem of finding the level populations $\mathbf{n}$ therefore reduces to that of finding the eigenvector of $\mathbf{M}$ that has an eigenvalue of unity.

\section{Critical Densities for Multi-Level Systems}

Chapter \ref{ch:obscold} gives a derivation of the critical density for two-level systems. Armed with the formalism of the previous section, we can generalize this to many-level systems. Consider some level $i$ which has the property that it is populated primarily from below, meaning that transitions into the state via collisional excitation or radiative absorption from lower levels occur much more often than transitions into the state via radiative decays or collisional de-excitations of higher levels, or transitions out of the state to higher levels via collisions or absorptions. In this case, the time rate of change of the level population reduces to
\begin{eqnarray}
\frac{dn_i}{dt} & = & \sum_{j<i} n_j n_c k_{ji} + \sum_{j<i} n_j \frac{g_i}{g_j} \langle n_{\gamma,ij}\rangle A_{ij}
\nonumber \\
& & {} - n_i \sum_{j<i} \left[n_c k_{ij} + \left(1+\langle n_{\gamma,ij}\rangle\right)A_{ij}\right].
\end{eqnarray}
Here the first term describes collisional excitation into state $i$ from lower levels, the second describes the rate of radiative excitation into state $i$ from lower levels, and the final term describes depopulation of state $i$ via collisions, spontaneous emission, and stimulated emission.

If the system is in steady state, then $dn_i/dt = 0$, and we have
\begin{equation}
\label{eq:nsteady}
n_i = \frac{\sum_{j<i} n_j n_c k_{ji} + \sum_{j<i} n_j \frac{g_i}{g_j} \langle n_{\gamma,ij}\rangle A_{ij}}{\sum_{j<i} \left[n_c k_{ij} + \left(1 + \langle n_{\gamma,ij}\rangle\right)A_{ij}\right]}.
\end{equation}
In analogy with the case of a two-level system, we now define the critical density for state $i$ via
\begin{equation}
n_{{\rm crit},i} = \frac{\sum_{j<i} \left(1 + \langle n_{\gamma,ij}\rangle\right) A_{ij}}{\sum_{j<i} k_{ij}},
\end{equation}
i.e., the critical density is the rate of radiative de-excitation divided by the rate of collisional de-excitation. The sole differences between this and the two-level critical density defined by equation (\ref{eq:ncrit}) are that this expression sums over all states into which radiative and collisional de-excitation can occur, and that it contains an extra factor of $ \left(1 + \langle n_{\gamma,ij}\rangle\right)$ in order to properly account for enhancements in the radiative de-excitation rate due to stimulated emission.

Substituting this definition of $n_{{\rm crit},i}$ into equation (\ref{eq:nsteady}) for the steady state population gives
\begin{equation}
n_i = \left(\frac{n_c}{n_c+n_{{\rm crit},i}}\right) \frac{\sum_{j<i} n_j k_{ji}}{\sum_{j<i} k_{ij}}
+ \left(\frac{n_{{\rm crit},i}}{n_c+n_{{\rm crit},i}}\right) \frac{\sum_{j<i} n_j \frac{g_i}{g_j} \langle n_{\gamma,ij}\rangle A_{ij}}{\sum_{j<i} \left(1+\langle n_{\gamma,ij}\rangle\right) A_{ij}}.
\end{equation}
Examining this expression, one can see that the generalized $n_{{\rm crit},i}$ plays much the same role as $n_{\rm crit}$ for a two-level system. In the limit $n_c \gg n_{{\rm crit},i}$, the first term dominates and the second is negligible. In this case the level population is simply set by collisional effects, and radiative effects become irrelevant. Given the relationships between the various collision rate coefficients $k_{ij}$ (c.f. equation \ref{eq:detailed_balance}), this implies that the level population goes to the usual Boltzmann distribution at the gas temperature $T$. Conversely, if $n_c \ll n_{{\rm crit},i}$, the first term is negligible and the second one dominates, so the level population is determined solely by the radiation field. In the absence of an external radiation field (i.e., $\langle n_{\gamma,ij}\rangle \rightarrow 0$), level $i$ becomes depopulated and thus the excitation is sub-thermal. If the radiation field follows a blackbody distribution (i.e., $\langle n_{\gamma,ij}\rangle$ has the value given by equation \ref{eq:ngamma_avg_LTE}), then one can show that the result is that the levels are populated following a Boltzmann distribution at the radiation field temperature.