\section{Up up and away! Adding More Layers to the Atmosphere}
Up until now we have neglected any vertical movement. This hampers the model, as the rising of warm air that then flows to the poles, cools down and flows back to the tropics is not possible
as the warm air cannot rise. So let us change that, let's add some vertical motion and some more layers to the atmosphere.

Remember \autoref{eq:atmos change}? We need this equation for every layer in the atmosphere. This also means that we have to adjust the main loop of the code, which is described  in
\autoref{alg:temperature with density}. The $T_a$ needs to change, we need to either add a dimension (to indicate which layer of the atmosphere we are talking about) or we need to add different
matrices for each atmosphere layer. Let us define some useful variables in \autoref{alg:more layers}. We opt for adding a dimension as that costs less memory than defining new arrays
\footnote{This has to do with pointers, creating a new object always costs a bit more space than adding a dimension as we need a pointer to the object and what type of object it is whereas with
adding a dimension we do not need this additional information as it has already been defined}. So $T_a$, and all other matrices that have to do with the atmosphere (so not $T_p$ for instance)
are no longer indexed by $lat, lon$ but are indexed by $lat, lon, layer$.

\begin{algorithm}
    $nlevels \leftarrow 4$ \;
    $heights \leftarrow \text{Array with } nlevels \text{ layers, each with a uniform thickness of } \frac{100 \cdot 10^3}{nlevels} m$ \;
    \caption{Definition of variables that are used throughout the code}
    \label{alg:more layers}
\end{algorithm}

We also need to change all the gradient functions (\autoref{alg:gradient x} and \autoref{alg:gradient y}) to incorporate the atmospheric layers. Additionally we need a new gradient function that
calculates the gradient in the $z$ direction (vertical). Let us first change the existing gradient functions to take the atmospheric layer in effect. The changes can be found in
\autoref{alg:gradient x layer} and \autoref{alg:gradient y layer}. Let us improve the gradient in the $y$ direction as well. Since we are using the central difference method (calculating the
gradient by taking the difference of the next grid cell and the previous grid cell) there is no gradient at the poles. What we can do instead of returning $0$ for those cases is forward
differencing (calculating the gradient by taking the difference of the cell and the next/previous cell, multiplied by $2$ to keep it fair). A special change in both functions is checking whether
$k$ is equal to \texttt{NULL}. We do this as sometimes we want to use this function for matrices that does not have the layer dimension. Hence we define a default value for $k$ which is
\texttt{NULL}. \texttt{NULL} is a special value in computer science. It represents nothing. This can be useful sometimes if you declare a variable to be something but it is referring to something
that has been deleted or it is returned when some function fails. It usually indicates that something special is going on. So here we use it in the special case where we do not want to consider
the layer part in the gradient.

\begin{algorithm}[hbt]
    \SetKwInOut{Input}{Input}
    \SetKwInOut{Output}{Output}
    \Input{Matrix (double array) $a$, first index $i$, second index $j$, third index $k$ with default value \texttt{NULL}}
    \Output{Gradient in the $x$ direction}
    \eIf{$k == \texttt{NULL}$}{
        $grad \leftarrow \frac{a[i, (j + 1)\text{ mod } nlon] - a[i, (j - 1) \text{ mod } nlon]}{\delta x[i]}$ \;
    }{
        $grad \leftarrow \frac{a[i, (j + 1)\text{ mod } nlon, k] - a[i, (j - 1) \text{ mod } nlon, k]}{\delta x[i]}$ \;
    }
    \Return{$grad$} \;
    \caption{Calculating the gradient in the $x$ direction}
    \label{alg:gradient x layer}
\end{algorithm}

\begin{algorithm}[hbt]
    \SetKwInOut{Input}{Input}
    \SetKwInOut{Output}{Output}
    \Input{Matrix (double array) $a$, first index $i$, second index $j$, third index $k$ with default value \texttt{NULL}}
    \Output{Gradient in the $y$ direction}
    \eIf{$k == \texttt{NULL}$}{
        \uIf{$i == 0$}{
            $grad \leftarrow 2 \frac{a[i + 1, j] - a[i, j]}{\delta y}$ \;
        }\uElseIf{$i == nlat - 1$}{
            $grad \leftarrow 2 \frac{a[i, j] - a[i - 1, j]}{\delta y}$ \;
        }\uElse{
            $grad \leftarrow \frac{a[i + 1, j] - a[i - 1 j]}{\delta y}$ \;
        }
    }{
        \uIf{$i == 0$}{
            $grad \leftarrow 2 \frac{a[i + 1, j, k] - a[i, j, k]}{\delta y}$ \;
        }\uElseIf{$i == nlat - 1$}{
            $grad \leftarrow 2 \frac{a[i, j, k] - a[i - 1, j, k]}{\delta y}$ \;
        }\uElse{
            $grad \leftarrow \frac{a[i + 1, j] - a[i - 1 j]}{\delta y}$ \;
        }
    }
    \Return $grad$ \;
    \caption{Calculating the gradient in the $y$ direction}
    \label{alg:gradient y layer}
\end{algorithm}

With those changes done, let us define the gradient in the $z$ direction. The function can be found in \autoref{alg:gradient z layer}. Here $a.dimensions$ is the attribute that tells us how
deeply nested the array $a$ is. If the result is $1$ we have just a normal array, if it is $2$ we have a double array (an array at each index of the array) which is also called a matrix and if it
is $3$ we have a triple array. We need this because we have a one-dimensional case, for when we do not use multiple layers and a three-dimensional case for when we do use multiple layers. This
distinction is needed to avoid errors being thrown when running the model with one or multiple layers.

\begin{algorithm}[hbt]
    \SetKwInOut{Input}{Input}
    \SetKwInOut{Output}{Output}
    \Input{Matrix (double array) $a$, first index $i$, second index $j$, third index $k$}
    \Output{Gradient in the $z$ direction}
    \uIf{$a.dimensions == 1$}{
        \uIf{$k == 0$}{
            $grad \leftarrow \frac{a[k + 1] - a[k]}{\delta z[k]}$ \;
        }\uElseIf{$k == nlevels - 1$}{
            $grad \leftarrow \frac{a[k] - a[k - 1]}{\delta z[k]}$ \;
        }\uElse{
            $grad \leftarrow \frac{a[k + 1] - a[k - 1]}{2\delta z[k]}$ \;
        }
    } \uElse {
        \uIf{$k == 0$}{
            $grad \leftarrow \frac{a[i, j, k + 1] - a[i, j, k]}{\delta z[k]}$ \;
        }\uElseIf{$k == nlevels - 1$}{
            $grad \leftarrow \frac{a[i, j, k] - a[i, j, k - 1]}{\delta z[k]}$ \;
        }\uElse{
            $grad \leftarrow \frac{a[i, j, k + 1] - a[i, j, k - 1]}{2\delta z[k]}$ \;
        }
    }

    \Return $grad$ \;
    \caption{Calculating the gradient in the $z$ direction}
    \label{alg:gradient z layer}
\end{algorithm}

As you can see, we have used $\delta z$ however, we have not defined it yet. Let us do that in \autoref{alg:gradient z}.

\begin{algorithm}[hbt]
    \For{$k \in [0, nlevels - 1]$}{
        $\delta z[k] \leftarrow heights[k + 1] - heights[k]$ \;
    }
    $\delta z[nlevels - 1] \leftarrow \delta z [nlevels - 2]$ \;
    \caption{Defining $\delta z$ for later use throughout the code}
    \label{alg:gradient z}
\end{algorithm}

Let's incorporate the changes for the Laplacian operator (\autoref{alg:laplacian}) as well. The new code can be found in \autoref{alg:laplacian layer}.

\begin{algorithm}[hbt]
    \SetKwInOut{Input}{Input}
    \SetKwInOut{Output}{Output}
    \Input{A matrix (double array) a}
    \Output{A matrix (double array) with results for the laplacian operator for each element}
    \eIf{$a.dimensions == 2$}{
        \For{$lat \in [1, nlat - 1]$}{
            \For{$lon \in [0, nlon]$}{
                $output[lat, lon] \leftarrow \frac{\Delta_x(a, lat, (lon + 1) \text{ mod } nlon) - \Delta_x(a, lat, (lon - 1) \text{ mod } nlon)}{\delta x[lat]} + \frac{\Delta_y(a, lat + 1, lon) -
                \Delta_y(a, lat - 1, lon)}{\delta y}$\;
            }
        }
    }{
        \For{$lat \in [1, nlat - 1]$}{
            \For{$lon \in [0, nlon]$}{
                \For{$k \in [0, nlevels - 1]$}{
                    $output[lat, lon, k] \leftarrow \frac{\Delta_x(a, lat, (lon + 1) \text{ mod } nlon, k) - \Delta_x(a, lat, (lon - 1) \text{ mod } nlon, k)}{\delta x[lat]} + \frac{\Delta_y(a,
                    lat + 1, lon, k) - \Delta_y(a, lat - 1, lon, k)}{\delta y} + \frac{\Delta_z(a, lat, lon, k + 1) - \Delta_z(a, lat, lon, k + 1)}{2\delta z[k]}$\;
                }
            }
        }
    }

    \Return{$ouput$} \;
    \caption{Calculate the laplacian operator over a matrix a}
    \label{alg:laplacian layer}
\end{algorithm}

Of course we also need to incorporate the new layers in the divergence operator (\autoref{alg:divergence}). The new changes can be found in \autoref{alg:divergence layer}. Here we use $w$, the
vertical wind velocity. We define $w$ in the same way as $u$ and $v$, it is all zeroes (in the beginning) and has the same dimensions as $u$ and $v$.

\begin{algorithm}[!hbt]
    \SetKwInOut{Input}{Input}
    \SetKwInOut{Output}{Output}
    \Input{A matrix (double array) $a$}
    \Output{A matrix (double array) containing the result of the divergence operator taken over that element}
    $dim_1 \leftarrow \text{ Length of } a \text{ in the first dimension}$ \;
    \For{$i \in [0, dim_1]$}{
        $dim_2 \leftarrow \text{ Length of } a \text{ in the second dimension (i.e. the length of the array stored at index } i)$ \;
        \For{$j \in [0, dim_2]$}{
            $dim_3 \leftarrow \text{ Length of } a \text{ in the third dimension}$ \;
            \For{$k \in [0, dim_3]$}{
                $output[i, j] \leftarrow \Delta_x(au, i, j, k) + \Delta_y(av, i, j, k) + \Delta_z(aw, i, j, k)$ \;
            }
        }
    }
    \Return{$output$} \;
    \caption{Calculate the result of the divergence operator on a vector}
    \label{alg:divergence layer}
\end{algorithm}

With all those changes in the functions done, let us incorporate the changes into the model itself. We now need to account for the temperature change throughout the layers. Let us look at the
atmospheric temperature equation again (\autoref{eq:atmos change}). We need to account for one more thing, the absorbtion of energy from another layer. The new equation is shown in
\autoref{eq:atmos change layer}. Here $k$ is the layer of the atmosphere, $k = -1$ means that you use $T_p$ and $k = nlevels$ means that $T_{a_{nlevels}} = 0$ as that is space. Also, let us
rewrite the equation a bit such that the variables that are repeated are only written once and stuff that is divided out is removed, which is done in \autoref{eq:atmos change layer improved}.
Let us also clean up the equation for the change in the surface temperature (\autoref{eq:surface change}) in \autoref{eq:surface change improved}.

\begin{subequations}
    \begin{equation}
        \Delta T_{a_k} = \frac{\delta t (\sigma \epsilon_{k - 1}T_{a_{k - 1}}^4 + \sigma \epsilon_{k + 1}T_{a_{k + 1}}^4 - 2\epsilon_k\sigma T_{a_k}^4)}{C_a}
        \label{eq:atmos change layer}
    \end{equation}
    \begin{equation}
        \Delta T_{a_k} = \frac{\delta t \sigma (\epsilon_{k - 1}T_{a_{k - 1}}^4 + \epsilon_{k + 1}T_{a_{k + 1}}^4 - 2\epsilon_kT_{a_k}^4)}{C_a}
        \label{eq:atmos change layer improved}
    \end{equation}
    \begin{equation}
        \Delta T_p = \frac{\delta t (S + \sigma(4\epsilon_pT_a^4 - 4T_p^4))}{4C_p}
        \label{eq:surface change improved}
    \end{equation}
\end{subequations}

With the changes made to the equation, we need to make those changes in the code as well. We need to add the new dimension to all matrices except $T_p$ and $a$ as they are unaffected (with
regards to the storage of the values) by the addition of multiple atmospheric layers. Every other matrix is affected. The new code can be found in \autoref{alg:temperature layer}.

\begin{algorithm}[hbt]
    \SetAlgoLined

    \While{\texttt{TRUE}}{
        \For{$lat \in [-nlat, nlat]$}{
            \For{$lon \in [0, nlot]$}{
                \For{$layer \in [0, nlevels]$}{
                    $T_p[lat, lon] \leftarrow T_p[lat, lon] + \frac{\delta t ((1 - a[lat, lon])S + \sigma(4\epsilon[0](T_a[lat, lon, 0])^4 - 4(T_p[lat, lon])^4))}
                    {4C_p[lat, lon]}$ \;
                    \uIf{$layer == 0$}{
                        $T_a[lat, lon, layer] \leftarrow T_a[lat, lon, layer] + \frac{\delta t \sigma((T_p[lat, lon])^4 - 2\epsilon[layer](T_a[lat, lon, layer])^4)}
                        {\rho[lat, lon, layer]C_a\delta z[layer]}$ \;
                    }\uElseIf{$layer == nlevels - 1$}{
                        $T_a[lat, lon, layer] \leftarrow T_a[lat, lon, layer] + \frac{\delta t \sigma(\epsilon[layer - 1](T_a[lat, lon, layer - 1])^4 - 2\epsilon[layer](T_a[lat, lon, layer])^4)}
                        {\rho[lat, lon, layer]C_a\delta z[layer]}$ \;
                    }\uElse{
                        $T_a[lat, lon, layer] \leftarrow T_a[lat, lon, layer] + \frac{\delta t \sigma(\epsilon[layer - 1](T_a[lat, lon, layer - 1])^4 + \epsilon[layer + 1]T_a[lat, lon, layer + 1]
                        - 2\epsilon[layer](T_a[lat, lon, layer])^4)}{\rho[lat, lon, layer]C_a\delta z[layer]}$ \;
                    }
                    $t \leftarrow t + \delta t$ \;
                }
            }
        }
    }
    \caption{The main loop of the temperature calculations}
    \label{alg:temperature layer}
\end{algorithm}

We also need to initialise the $\epsilon$ value for each layer. We do that in \autoref{alg:epsilon}.

\begin{algorithm}
    $\epsilon[0] \leftarrow 0.75$ \;
    \For{$i \in [1, nlevels]$}{
        $\epsilon[i] \leftarrow 0.5\epsilon[i - 1]$
    }
    \caption{Intialisation of the insulation of each layer (also known as $\epsilon$)}
    \label{alg:epsilon}
\end{algorithm}

Now we need to add vertical winds, or in other words add the $w$ component of the velocity vectors. We do that by editing \autoref{alg:stream3}. We change it to \autoref{alg:velocity}. Here we
use gravity ($g$) instead of the coriolis force ($f$) and calculate the change in pressure. Therefore we need to store a copy of the pressure before we do any calculations. This needs to be a
copy due to aliasing \footnote{Aliasing is assigning a different name to a variable, while it remains the same variable. Take for instance that we declare a variable $x$ and set it to be $4$.
Then we say $y \leftarrow x$, which you might think is the same as saying they $y \leftarrow 4$ but behind the screen it is pointing to $x$. So if $x$ changes, then so does $y$.}

\begin{algorithm}
    $S_{xu} \leftarrow \texttt{gradient\_x}(u, lan, lon)$ \;
    $S_{yu} \leftarrow \texttt{gradient\_y}(u, lan, lon)$ \;
    $S_{xv} \leftarrow \texttt{gradient\_x}(v, lan, lon)$ \;
    $S_{yv} \leftarrow \texttt{gradient\_y}(v, lan, lon)$ \;
    $S_{px} \leftarrow \texttt{gradient\_x}(p, lan, lon)$ \;
    $S_{py} \leftarrow \texttt{gradient\_y}(p, lan, lon)$ \;
    \While{\texttt{TRUE}}{
        \For{$lat \in [1, nlat - 1]$}{
            \For{$lon \in [0, nlon]$}{
                \For{$layer \in [0, nlevels]$}{
                    $u[lan, lon, layer] \leftarrow u[lat, lon, layer] + \delta t \frac{-u[lat, lon, layer]S_{xu} - v[lat, lon, layer]S_{yu} + f[lat]v[lat, lon, layer] - S_{px}}{\rho}$ \;
                    $v[lan, lon, layer] \leftarrow v[lat, lon, layer] + \delta t \frac{-u[lat, lon, layer]S_{xv} - v[lat, lon, layer]S_{yv} - f[lat]u[lat, lon, layer] - S_{py}}{\rho}$ \;
                    $w[lan, lon, layer] \leftarrow w[lat, lon, layer] - \frac{p[lat, lon, layer] - p_o[lat, lon, layer]}{\delta t\rho[lat, lon, layer]g}$ \;
                }
            }
        }

        $p_o \leftarrow copy(p)$ \;
    }
    \caption{Calculating the flow of the atmosphere (wind)}
    \label{alg:velocity}
\end{algorithm}

Lastly, we need to add the correct indices to the advection algorithm \autoref{alg:advectionfix}. Let us add it, with \autoref{alg:advection layer} as a result. Here the ':' means all indices
of the 3 dimensional matrix.

\begin{algorithm}
    $\alpha_a \leftarrow 2 \cdot 10^{-5}$ \;
    $\alpha_p \leftarrow 1.5 \cdot 10^{-6}$ \;
    $boundary \leftarrow 7$ \;
    \While{\texttt{TRUE}}{
        $T_{add} \leftarrow T_a + \delta t \alpha_a \nabla^2(T_a) + \nabla(T_a)$ \;
        $T_a \leftarrow T_a - 0.5T_{add}[boundary:-boundary, :, :] \text{ //Only subtract } T_{add} \text{ to } T_a \text{ for indices in the interval } [-nlat + boundary, nlat - boundary]$. \;
        $\rho[boundary: -boundary, :, :] \leftarrow \rho - 0.5(\delta t \nabla \rho) \text{ //Only change the density for indices in the interval } [-nlat + boundary, nlat - boundary]$ \;
        $T_p \leftarrow T_p + \delta t \alpha_p \nabla^2(T_p)$ \;
    }
    \caption{The main loop for calculating the effects of advection}
    \label{alg:advection layer}
\end{algorithm}


\subsection{Maximum Ratio Combining based Macro diversity}
Due to the fact that a transmitted packet theoretically can be simultaneously received by all BS (ignorance of background noise), linear combining of signals at each BS can be leveraged so that the output SINR is maximized. Such a scheme is called Maximum Ratio Combining (MRC) based Macro diversity\qsong{It is better to give a reference to say this is studied by some LPWAN networks}. In this section, the performance of MRC-based Macro diversity in one-shot random access is evaluated.

In the literature, it has been prove that if the weigh factors involved in MRC context is well designed, the output $\text{SINR}$ $\Theta$ of best combiner is expressed as \qsong{Perhaps we also need a reference for this...}:
\begin{align}
	\Theta = \sum_{y_j \in \Phi_{b}}^{} \text{SINR}_{y_j},
\end{align}
where $\text{SINR}_{y_j}$ is the received SINR at BS with label $y_j$.

The Laplace Transform (LT) of $\Theta$ is by definition as follows:
\begin{align}
	\label{eq:lt-sinr-mrc-1}
	\mathcal{L}_{\Theta}\left( s \right) &= \mathbb{E}\left[ e^{-s\Theta}\right] \nonumber\\
	&=\mathbb{E}\left[ \exp( -s \sum_{y_j \in \Phi_{b}}^{} \Theta_{y_j} )\right]
\end{align}

Recall that the received SINR of target BS is defined as:
\begin{align}
	\theta &= \frac{H \exp(\chi) r^{-\gamma}}{I} \nonumber\\
	&= \epsilon r^{-\gamma}
\end{align}
where term $\epsilon = H\exp(\chi)  / I$ is identically independently distributed RV for each BS.
\begin{align}
	\label{eq:lt-sinr-mrc-2}
	\mathcal{L}_{\Theta}\left( s \right) &= \mathbb{E}\left[ \prod_{y_j \in \Phi_b}^{} \mathbb{E}_{\epsilon} \left[ \exp( -s \epsilon r^{-\gamma}) \right] \right]
\end{align}
Applying Campbell theorem to $(\ref{eq:lt-sinr-mrc-1})$,
\begin{align}
	\mathcal{L}_{\Theta}\left( s \right) &= \exp \left\lbrace -\mathbb{E}_{\epsilon}\left[ \int_{0}^{+\infty} \left(1-\exp(-s\epsilon r^{-\gamma} )2 \pi \lambda_{b} r dr\right)  \right]   \right\rbrace
\end{align}

Let us focus on the integral $D = \int_{0}^{+\infty} \left(1-\exp(-s\epsilon r^{-\gamma} ) \right) 2 \pi \lambda_{b} r dr $:
\begin{align}
	\label{itg:D}
	D &= \int_{0}^{+\infty} \left(1-\exp(-s\epsilon r^{-\gamma/2} ) \right) \pi \lambda_{b} dr \nonumber\\
	&= \pi \lambda_{b} \int_{0}^{+\infty} \left(1-\exp(-s\epsilon r^{-\gamma/2} ) \right) \frac{2}{\gamma} x^{\frac{2}{\gamma}  - 1} dx \text{ (substitution $x = r^{\gamma/2}$)}
\end{align}
To further calculate $(\ref{itg:D})$, we note that it is the expected value of RV $ \left(    (X/\epsilon s) ^{-1} \right)^{\frac{2}{\gamma}} $ where $X$  is an exponential RV with unit mean. Since $\mathbf{E}\left[ X^{p}\right] = \Gamma(1+p)$ by the definition of the gamma function. Hence, we have:
\begin{align}
	\mathbf{E} \left[ \left(    (X/\epsilon s) ^{-1} \right)^{\frac{2}{\gamma}}  \right] = (s \epsilon) ^{\frac{2}{\gamma}} \Gamma (1 - \frac{2}{\gamma}) \nonumber.
\end{align}

Finally, $(\ref{eq:lt-sinr-mrc-2})$ is simplified as:
\begin{align}
	\mathcal{L}_{\Theta}\left( s \right) &= \exp(-\lambda_{b} \pi \mathbb{E}\left[ \epsilon ^{\frac{2}{\gamma}} \right]  \Gamma(1-\frac{2}{\gamma}) s^{\frac{2}{\gamma}})
\end{align}

Let us now focus on term $\mathbb{E}\left[ \epsilon ^{\frac{2}{\gamma}} \right] $:
\begin{align}
	\mathbb{E}\left[ \epsilon ^{\frac{2}{\gamma}} \right]  = \mathbb{E}\left[ \left( H \exp(\chi)\right)  ^{\frac{2}{\gamma}} \right] \mathbb{E}\left[ I ^{-\frac{2}{\gamma}}\right]
\end{align}

For term $\mathbb{E}\left[ I ^{-\frac{2}{\gamma}}\right] $, it is a negative fractional moment calculation problem, which is also a research subject in applied mathematical domain. \qsong{Still need to find some numerical technique in the literature to compute this. I think it is the last difficulty to be solved if we want to obtain a general analytical framework...}

With substitution $s = -j\omega$, the characteristic function (CF) of $\Theta$:
\begin{align}
\phi_{\Theta}\left( \omega \right) &= \exp(-\lambda_{b} \pi \mathbb{E}\left[ \epsilon ^{\frac{2}{\gamma}} \right]  \Gamma(1-\frac{2}{\gamma}) \exp(-i\pi/\gamma) \omega^{\frac{2}{\gamma}}),
\end{align}
where $\omega \geq 0$.

As a continuous random variable, the cumulative distribution function $F_{\Theta}\left( x \right)$ of total SINR $\Theta$ can be directly derived from its characteristic function $\phi_{\Theta}\left(\omega\right)$, for example by use of Gil-Pelaez Theorem~\cite{gil1951note}. However, directly using Gil-Pelaez Theorem needs long time. Applying mathematical techniques used in finance domain~\cite{hirsa2012computational}, we seek to calculate the Fourier transform of $e^{-\eta x} F_{\Theta}\left( x \right)$ where term $e^{-\eta x}$ is a damping function with $\eta > 0$.
\begin{align}
\label{eq:intermediate_formula_1}
\int_{-\infty}^{+\infty} e^{i\omega x} e^{-\eta x} F_{Y_k}\left( x \right) dx = \frac{1}{\eta - iw} \phi_{Y_{k}}\left( \omega +i\eta \right)
\end{align}
Applying Fourier inversion for ($\ref{eq:intermediate_formula_1}$), we obtain the expression for $F_{\Theta}\left( x \right)$ as follows:
\begin{align}
\label{eq:pr_c_m_case2}
F_{\theta}\left( x \right)  &= \frac{e^{\eta x}}{2\pi} \int_{-\infty}^{+\infty} e^{-i \omega x} \frac{1}{\eta - i\omega} \phi_{\theta}\left( \omega +i\eta\right) d\omega  \nonumber\\
&= \frac{e^{\eta x}}{\pi} \Re\left\lbrace  \int_{0}^{+\infty} e^{-i \omega x} \frac{1}{\eta - i\omega} \phi_{\theta}\left( \omega +i\eta\right) d\omega\right\rbrace,
\end{align}
The cumulative distribution function $F_{\theta}\left( x \right)$ now can be derived directly from ($\ref{eq:pr_c_m_case2}$) using a single numerical integration.

\subsection{A special case, $\gamma=4$}
In this case, we study a special case where $\gamma=4$.
\begin{align}
	\mathbb{E}\left[ \left( H \exp(\chi)\right)  ^{\frac{1}{2}} \right]  = \Gamma(1+\frac{1}{2}) \exp \left( \frac{\sqrt{2}\sigma}{4}\right) ^2
\end{align}
Reference~\cite{haenggi2009interference} gives the PDF of cumulative interference $I$ suffered by device at the origin:
\begin{align}
	f_{I}(x) = \frac{p\lambda_{m}}{4} (\frac{\pi}{x})^{\frac{3}{2}} \exp(-\frac{\pi^4 p^2\lambda_{m}^2}{16x}), x >= 0
\end{align}
We can get the PDF of cumulative interference $I$ in the presence of Rayleigh fading and log-normal shadowing, just by scaling $\lambda_{m}$ as $ \lambda_{m} \exp(\frac{\sigma^2}{8})$:
\begin{align}
	f_{I}(x) = \frac{    p\lambda_{m}   \exp(\frac{\sigma^2}{8})  }{4} (\frac{\pi}{x})^{\frac{3}{2}} \exp(    -\frac{    \pi^4    p^2\lambda_{m}^2     \exp^2(\frac{\sigma^2}{8})    }{    16x    }    ),
\end{align}
Hence,
\begin{align}
	\mathbb{E}\left[ I ^{-\frac{1}{2}}\right] &= \int_{0}^{+\infty} x^{-\frac{1}{2}} f_{I}(x) dx \nonumber\\
	&= \frac{4}{    \pi^{\frac{5}{2}}  p\lambda_{m} \exp(\frac{\sigma^2}{8}) }
\end{align}

\begin{align}
	\mathbb{E}\left[ \epsilon ^{\frac{1}{2}} \right] & = 4\Gamma(\frac{3}{2})\pi^{-\frac{5}{2}} p^{-1}\lambda_{m}^{-1}\nonumber\\
	&=2\pi^{-2} p^{-1}\lambda_{m}^{-1}
\end{align}
Therefore:
\begin{align}
	\mathcal{L}_{\Theta}\left( s \right) &= \exp(-\lambda_{b} \pi 2\pi^{-2}\lambda_{m}^{-1} \Gamma(\frac{1}{2}) s^{\frac{1}{2}}) \nonumber \\
	&= \exp(-2\lambda_{b} p^{-1}\lambda_{m}^{-1}\pi^{-\frac{1}{2}}  s^{\frac{1}{2}})
\end{align}
Its CDF and CCDF:
\begin{align}
	F_{\Theta} (x) &=1 -\erf \left(
	\frac{\lambda_{b}p^{-1}\lambda_{m}^{-1}}{\sqrt{\pi x}}
	\right) \\
	F^c_{\Theta} (x) &=\erf(\frac{\lambda_{b} p^{-1}\lambda_{m}^{-1}}{\sqrt{\pi x}}) \nonumber\\
	&= \erf(\frac{1}{\frac{p\lambda_{m}}{\lambda_{b}}\sqrt{\pi x}})
\end{align}

The packet loss rate performance in the case of applying maximum ratio combining is illustrated in Fig.~\ref{fig:newpacketlossratemprmrc}. It is not surprisingly to observe that maximum ratio combing bring significant performance improve.
\begin{figure}
	\centering
	\includegraphics[width=1.0\columnwidth]{/Users/qsong/Documents/slotted_aloha_related_project/multiple_recepteur_capacity/new_packet_loss_rate_mpr_mrc}
	\caption{Network packet loss rate with respect to normalized load $p\lambda_{m}/\lambda_{b}$ (ANA=analytical, SIM=simulation)}
	\label{fig:newpacketlossratemprmrc}
\end{figure}


\section{Mathematical Modeling}
\label{sec:mathmodel}

% \begin{itemize}
% \item \st{unstable stratified boundary layers (raleigh number estimate)}
% \item \st{justify incompressible N-S}
% \item \st{justification of far-field eddy-viscosity model (M-O)}
% \item modeling eddy-viscosity in device
% \item vane and turbine representation via penalty function // immersed boundary method
% \item cone representation
% \end{itemize}

%remember that \st{} is strikethrough
%
% should this all be math modeling?
%

The aim of the proposed work is to simulate synthetic
dust devils in the field. This requires a model of the ambient
conditions for a representative case, such as Arizona, where
experimental data is available from tests that have been
performed. Furthermore, for this to be more generally useful in the
prediction of flows in a variety of conditions, we need a model generally
applicable to any flow near the surface of the earth.

This section details an analysis of surface fluid mechanics, and
develops a mathematical model for turbulence in a thermally stratified
medium. We seek to emulate the operation of the apparatus during the
day, when dust devils are observed to form readily.
At these times, the atmospheric surface layer has the following
character. Incident radiation from the Sun does not significantly
interact with the air, which is nearly transparent. Instead, this
radiation is absorbed by the ground, which causes its temperature to
rise. This results in a thermal gradient between the hot ground and the
cooler air. The warm ground conducts heat to the air, causing expansion
and lowering the density of the air. This reduced density air near the
surface is driven upwards by the force of buoyancy.

For sufficiently large temperature gradients, the hot surface layer is
unstable, and as the warm air is driven upwards the flow will transition
to turbulence. For the typical use case we consider, namely Arizona in
summer, the temperature difference can be in excess of 30 Kelvin.
Rayleigh numbers associated with temperature gradients of this magnitude
are between $10^{9} - 10^{11}$ and therefore meet the criterion
% ref thermo book here?
for transition to a turbulent regime. The flow is that of an unstably
stratified fluid.

This section begins by describing the governing equations of the system
of interest. It then proceeds to the development of a viscosity model
used to resolve the large scale features of the solution. Next, models
used to represent the vanes and turbine, as well as the separation of
fluid off of these modeled surfaces, are introduced.  Finally, the
models for the computational domain extent and the boundary conditions
are discussed.

%Note that a complete numerical specification of all the model
%parameters introduced in this section are provided in a table in
%appendix \ref{app:model_param}.

\subsection{Equations of Fluid Motion}
\label{sub_sec:ns_en}
%
% do I need to justify this more? These are pretty critical, after all
%

The equations describing fluid flow with natural convection are,
\begin{align}
  \frac{\partial u}{\partial t} + u \cdot \nabla u =& \,
  -\frac{1}{\rho}\nabla P + \nu \nabla^2 u - g \frac{T'}{T_0}\\
  \nabla \cdot u =& \, 0 \\
  \rho c_p \frac{\partial T}{\partial t} + u \cdot \nabla T =& \, \nabla
 \cdot ( k \nabla T)
\end{align}
under the assumption that the temperature variation is small in
comparison to the mean temperature of the region. These are the
incompressible Navier-Stokes equations with Boussinesq, a representation
of buoyancy coupled with the heat equation.
%
% in full document be sure to mention that neglecting coriolis is legit
% below 50ms
%
%
As discussed above, we anticipate that the flow will be
turbulent. Turbulence significantly alters the character of the flow,
and necessitates either resolving the resulting small scales or
providing a model that emulates their impact. In this case, a
Reynolds Averaged Navier-Stokes (RANS) formulation is used, where the
turbulent viscosity and thermal conductivities are permitted to vary in
space, and the flow is decomposed into constant laminar and varying
turbulent and vane components,

\begin{eqnarray*}
 \nu =& \nu_{l} + \nu_{T}(z) + \nu_{V}(r,z) \\
 K =& K_{l} + K_{T}(z) + K_{V}(r,z).
\end{eqnarray*}

This is an effective eddy viscosity model, and the subsequent two
sections will elaborate on the spatial dependence and character of
$\nu_T$, $K_T$, $\nu_C$ and $K_C$. The laminar, base diffusivities
are $\nu_l$ and $K_l$, which do not vary in space.

\subsection{Viscosity Model}

We use the well-known similarity model of Monin and
Obukhov\cite{monin2007statistical,monin1954basic} as a guide to the
specification of an eddy viscosity model to describe the vertical mixing
in the atmosphere. This formulation is an extension of the mixing-length
model of Prandtl, where the concepts of gradient diffusion and mixing
length were generalized to thermally stratified flow.

%
% justify prandtl assumption here
%

Monin and Obukhov argued that under statistically stationary, horizontally
homogeneous conditions, the dynamics of any mean turbulent quantity
($\bar f$) in a thermally stratified medium depend only on,

\begin{equation}
\bar f = f(z,\frac{g}{T_0},\rho_0,\nu_l,K_l,u^*,q).
\end{equation}
Aside from near the surface, the laminar diffusivities $\nu_l$
and $K_l$ will be
small compared to their turbulent counterparts, $\nu_T$ and $K_T$, and
are therefore negligible.
The remaining five parameters are: the distance from the ground, z; the
buoyancy coefficient, $\frac{g}{T_0}$; the density of the fluid,
$\rho_0$; a velocity scale, $u^*$ (in particular, the freestream
velocity); and the heat flux to the ground, $q$. %\todo{is this right?}
% Likewise, if we define $z-z_0$ as an ``effective roughness
% height'' or displacement distance, we can reasonably neglect $z_0$ from these
% considerations. While the roughness height can be large (for instance in
% a cornfield, where the roughness height could reasonably be several
% meters), for the present study the expectation is that this roughness
% height will be on the order of centimeters\cite{oke1987boundary}, and
% therefore negligible.
%
% add refence to dynamical and physical meteorology
%
These quantities depend on four dimensions: length, time, temperature
and mass. Dimensional analysis implies that this mean turbulent quantity
($\bar f$) should then only be a function of a single dimensionless
group\cite{munson2012fundamentals}. This is chosen to be,
\begin{equation}
 \xi = -\frac{\kappa \frac{g}{T_0} \frac{q}{c_p \rho_0} z}{ {u^*}^3}.
\end{equation}
where $\kappa$ is the (dimensionless) Von-Karman constant.
The physical meaning of this quantity bears some discussion.
The numerator, $\kappa \frac{g}{T_0} \frac{q}{c_p \rho_0} $, is
proportional to the buoyant production of kinetic energy.  The
denominator, $\frac{{u^*}^3}{z}$, is a shear production rate.

The non-dimensional group $\xi$ is typically cast into the following
form,
\begin{equation}
 \xi = \frac{z}{L_{M-O}}
\end{equation}
where $L_{M-O}$ is the famous, ``Monin-Obukhov'' length,
\begin{equation}
 L_{M-O} = -\frac{{u^*}^3}{\kappa \frac{g}{T_0} \frac{q}{c_p \rho_0}}.
  \label{eqn_mo_length}
\end{equation}

This length can be interpreted as the vertical location
where the production of buoyantly generated kinetic energy is
approximately equal to the energy generated by wind shear. When the
magnitude of $L_{M-O}$ is large, the flow is dominated by shear effects,
and the impact of buoyancy is small. Conversely, a small magnitude of
$L_{M-O}$ implies that buoyant effects largely dominate the kinetic
energy production. Notice also that the sign convention in Equation
\ref{eqn_mo_length} is such that for the systems we consider (q>0, heat
flux from the surface to the air), $L_{M-O}$ will always be
negative. This is as expected, as the convection from the high
temperature surface to cooler air is unstable.

%The mean quantity $\bar f$ has a
%functional representation to the effect,
In this case, appropriately normalized mean turbulent quantities should
be functions of only the non-dimensional group
\begin{equation}
 \frac{\bar f}{f_{MO}} = \phi(\frac{z}{L_{M-O}})
\end{equation}
with $f_{MO}$ a normalizing constant with units of $\bar f$, and $\phi$
is a function only of $\xi$. We are interested in the case where
$\xi<0$, which corresponds to heat flux from the ground into the
air. For instance, the mean velocity field would have scaling,
$\frac{u^*}{\kappa}$ and the temperature fields would be scaled as $T^*
= \frac{1}{\kappa u^*} \frac{q}{c_p \rho_0}$. In this way, the mean
velocity and temperature fields would have the form,
\begin{eqnarray}
\bar u(z) = \frac{u^*}{\kappa} \phi_u(\frac{z}{L_{M-O}}) \\
\bar T(z) = T^* \phi_T(\frac{z}{L_{M-O}}).
\end{eqnarray}
As a result, the vertical gradients of the velocity and temperature are
necessarily,
\begin{eqnarray}
\frac{\partial \bar u(z)}{\partial z} = \frac{u^*}{\kappa L_{M-O}}
 \varphi_u(\frac{z}{L_{M-O}}) \label{eq:uz} \\
\frac{\partial \bar T(z)}{\partial z} = \frac{T^*}{L_{M-O}}
 \varphi_T(\frac{z}{L_{M-O}}) \label{eq:tz}.
\end{eqnarray}
Notice that $\phi$ and $\varphi$ are different universal functions. Eddy
viscosity is defined as, $u'v' = \nu_T \frac{\partial
u}{\partial z}$\cite{durbin2001statistical}, in which case, using
equation \ref{eq:uz}, it must scale as
\begin{equation}
 \nu_T = \frac{{u^*}^2}{\frac{\partial \bar u}{\partial z}} = \frac{u^*
  \kappa L_{M-O}}{\varphi_u(\xi)}.
\end{equation}
While eddy thermal diffusivity (defined as, $q = c_p \rho_0 K_T
\frac{\partial T}{\partial z}$) is
\begin{equation}
 K_T = \frac{q/c_p \rho_0}{\frac{\partial \bar T}{\partial z}} = \frac{u^*
  \kappa L_{M-O}}{\varphi_T(\xi)}.
\label{eqn:eddy_kt}
\end{equation}
Note the difference between $\varphi_u$ and
$\varphi_T$, which for turbulent Prandtl numbers near unity (e.g. $Pr_T
\approx 1$) the functions will be identical. The asymptotic behavior of
the $\varphi_T$ and $\varphi_u$ at large and small values of $\xi$
provides guidance to the more general character of the functions.
Our interest lies in the case where $L_{M-O}<0$, which corresponds to heat flux
from the ground into the air.
%

The case where $\xi \to -\infty $ implies $\frac{z}{L_{M-O}} \to
-\infty $ and $z>>L_{M-O}$. This is most readily interpreted as the instance
where $u^* \to 0$, e.g. the buoyancy-dominated case with no wind
(free-convection). For this case, the function $\varphi_T$ must have no
dependence on $u^*$, and will approach a constant. Scaling
analysis implies that the overall function will not depend on $u^*$ only
when the function $\varphi$ scales to the $-\frac{4}{3}$ power. The
function must then be

\begin{equation}
 K_T = \frac{1}{C_T} \left( \frac{q}{c_p \rho_0} \frac{g}{T_0}
		     \right)^\frac{1}{3} z^{\frac{4}{3}}  \text{
for } z \gg L_{M-O}.
\end{equation}

So long as the Prandtl number remains constant in space, then
% todo: provide discussion as to why this is not an unreasonable expectation
identical arguments regarding the asymptotic behaviour at large $\xi$ provide
the analogous result for the eddy viscosity's variation with respect to
distance from the ground,
\begin{equation}
 \nu_T = \frac{1}{C_{\nu_T}} \left( \frac{q}{c_p \rho_0} \frac{g}{T_0}
			     \right)^\frac{1}{3} z^{\frac{4}{3}}  \text{
for } z \gg L_{M-O}.
\end{equation}

These functions have been found to be broadly applicable and
accurate\cite{Foken2006}, and are easily implemented in software.

\subsection{Eddy Viscosity in the Device}

%However, this is also a more
%difficult regime to model.\todo{poor justification}
The validation process identified a refinement to the virtual vane
formulation that results in a better representation of the vane
effects in a broader range of flows. The thermal and momentum
diffusivities are even larger in the device where the flow across the
vanes produces shear and generates turbulence. The model now include an
enhanced turbulent diffusivity in the vortical plume region to account
for the effects of vortex shedding from the trailing edge of the vanes,
which is not represented in the virtual vane representation (discussed in
\ref{subsec:vane}).

% To successfully accomplish this, source terms
% for production of diffusivity were formulated to properly
% account for the generation of turbulence in the region of the
% vanes. This diffusivity would then convect and diffuse through space.
% To avoid modeling a temporal and three-dimensionally spatially
% varying field of diffusivities, we have instead calibrated the field
% based on data provided by our partners at Georgia Tech.

%This calibration
%is detailed in Section \ref{sec:validation}.
The eddy-viscosity in the region of the vanes and interior is set based
on scaling relations for a turbulent self-similar circular jet, as
described in Pope\cite{pope2000turbulent},

\begin{equation}
  \nu_C = U_0 y_{1/2} \bar \nu_C.
  %\nu_C(r) = U_0(r) y_{1/2}(r) \bar \nu_C
\end{equation}

In this equation, $U_0$ is the peak velocity, and $y_{1/2}$ the jet
half-width (taken to be the dust devil half-width).
% In words, we are scaling the
% calibrated viscosity by the velocity and length scale of the
% apparatus. $\bar \nu_C $ is input, measured from the experimental
% laboratory.  The diffusivity here is essentially a top hat filter, which
% radial values interior to the vanes the nominal calibrated value, and
% those outside the vanes zero, e.g. $\nu_C(r>r_{\text{vane}})=0$.
The dimensionless constant $\bar \nu_C $ is calibrated based on
experimental data, and is set to zero outside the device.
The thermal diffusivity inside the device, $K_C$, is then fixed with the
assumption that the Prandtl number is unity.

%The thermal and momentum diffusivities are expected to be even larger in the
%device where the flow across the vanes produces shear and generates
%turbulence. Our model for the diffusivities inside the vanes should
%therefore be higher than the ambient regions outside the vanes.

\subsection{Vane and Turbine Representation}
\label{subsec:vane}
To rapidly prototype general system configurations, the
computations must be able to explore a large space of possible
geometries and settings. This presents a significant meshing and
computational challenge if the detailed flow around the vanes is to be
computed. In the region near the vanes, where a no-slip boundary
condition is imposed, the flow will necessarily form a thin momentum
boundary layer. Resolving this boundary layer requires high resolutions
immediately adjacent to the walls. Changing the vane location requires
that a new mesh be generated.
This is a significant
challenge, as the development of a new mesh often requires significant
human effort and time. Furthermore, the process is error-prone,
and would require that each simulation using a new mesh undergo
detailed solution verification.

% Your text on the virtual vanes does not provide enough information to
% know exactly what we did. It is needlessly vague, and does not
% adequately connect to the real vane geometry. I propose the following
% more direct and more precise text. Further, the penalty nomenclature
% is inappropriate.

Instead, we have developed a modeling formulation that does not require
explicitly meshing the turning vanes, or any surface. These so-called
``virtual vanes'' are implemented as a body force that
is applied in the annular region that contains the vanes. Vane
geometry is specified by the angle $\phi$ a vane makes with a radial
line as a function of the radial coordinate $r$. A unit normal to vane
surfaces ${\bf n}$ is defined as
%
\begin{equation}
 {\bf n}({\bf x}) = \sin \left(\phi(r) \right) \hat{{\bf r}}+ \cos
  \left(\phi(r) \right) \hat{{\bf \theta}}
\end{equation}
%
where $\hat{{\bf r}}$ and $\hat{{\bf \theta}}$ are unit
vectors in the radial and azimuthal directions, respectively.
With this vane-normal vector field specified, a body force ${\bf f_v}$
is defined
that will drive the velocity in the ${\bf n}$ direction toward zero,
effectively turning the flow to be parallel to the vanes. The body
force is defined:
\begin{equation}
 {\bf f}_v= -\frac{1}{\ell_v}|{\bf u}|({\bf u}\cdot{\bf n}){\bf n}
 \label{eqn:body_force}
\end{equation}
with ${\bf u}$ the velocity and $\ell_v$ is a specified length
scale. $\ell_v$ represents the distance over which the
flow evolves under the influence of the body force before the
velocity in the normal direction is reduced by a factor of $1/{\bf
e}$. In other words, this is the length scale over which the
normal component of the velocity decays exponentially.
It is a modeling constant and is specified to be
the same order as the separation distance between neighboring vanes in
the physical vane configuration, since entry lengths in internal flows
scale with the width of the channel.

This virtual vane formulation is similar to the ``actuator disk'' model
commonly used to represent the rotor of a wind turbine \cite{betz}. An actuator
disk model is under development and will be detailed in the dissertation.

\subsection{Solid Surface Representation}
\label{subsec:solid_surface}

In addition to vanes, the SoV device includes impermeable surfaces
such as the wind break (``cone'') on the top of the facility. As with
the turning vanes, this is represented without explicitly meshing the
surface nor imposing  a boundary condition at the surface. This allows
rapid exploration of configurations  with different solid surfaces to
control and manipulate the fluid flow. These solid surfaces are
represented by a body force acting in a region surrounding the wall.
A body force normal to the surface is defined in this region so
that it will drive the normal velocity to zero, resulting in the flow
moving only parallel to the virtual surface.
The body force is defined as in Equation
\ref{eqn:body_force}; however, the length scale $\ell_v$ is specified to
be identical to the width of the surface being represented. This is
typically the width of two or three grid cells. While the actual surface we are
emulating is thinner than this, the numerical method has difficulty
converging for surfaces smaller than the grid size.

Forcing models designed to mimic a surface
are not unique to this project, and the current formulation is
 closely related to (among others)
``immersed boundary methods'' as used by various other
researchers\cite{doi:10.1146/annurev.fluid.37.061903.175743}.
This approach is unique in its use of Babuska's penalty treatment of
constraints\cite{1973fempen,ZAMM:ZAMM19880680925} to enforce the
behavior at the boundary. This method was selected because it is easily
imposed in the FEM context, and the method has been explored in detail
in the literature.
%Typically, the enforcement occurs along a domain
%boundary, but in this work it is used in the interior,
%and is not imposed as a mathematical constraint but rather as a modeling
%approach.

% We add a penalty term to the weak
% form of the Navier-Stokes momentum equation described in the subsequent
% section %\ref{eq:ns_weak}
% that has the form,
% \begin{equation}
% P_\epsilon = \int_\Omega ({\bf f_v} \cdot v) \, dx
% \end{equation}
% where ${\bf f_v}$ is as described in equation \ref{eqn:body_force}.
%% As the system is formulated as a variational problem that seeks to
%% minimize the test function $v$, any velocity that is not aligned with
%% the vane normal will incur a penalty versus one that does. Note that unlike
%% some penalty methods, this does not automatically satisfy
%% continuity. Rather, the velocity field remains divergence free through a
%% separately enforced constraint.

\subsection{Separation Model}

In the presence of wind, it was found that there was a significant flow
out through the vanes in the back of the device. This was obviously
inconsistent with the findings of our colleagues in the field, who
observed no outflows out of the back of the device. Moreover, this resulted
in large inconsistencies between our predictions and the field results,
almost certainly because of the kinetic and thermal energy that our vane
representation was permitting to leave out the back of the device.

This exposed a weakness of the turning vane representation outlined
previously. When the flow entered the virtual vane forcing region it was
always turned to align with the vane angle, even when the forcing was in
the opposite direction of the present velocity.
This is in contrast to the physical situation, in which we
expect the flow to continue along an averaged streamline separating from the
trailing edges of the vanes, instead of turning around it.
The averaged streamline will continue past the trailing edge of the vane
due to the separation of the boundary layer off the edge surface. An
image depicting these two cases in shown in Figure \ref{fig:sep_model}.

\begin{figure}[!htb]
  \begin{center}
    \includegraphics[width = 6 cm]{figs/sep_model}
    \caption{Schematic depicting the separation model that extends past
   the trailing edge of the vanes. In the top case, the flow entering
   the virtual vane region is forced to align with the vane angle despite
   this resulting in a reversal of the flow direction. This is a
   consequence of the forcing function acting on the fluid to ensure the
   velocity vector aligns with the vane.
   The second case depicts the separation
   model, where the flow under certain conditions is not forced and
   continues to move tangent to the vanes due to
   the separation of the boundary layer off the trailing edge.}
    \label{fig:sep_model}
  \end{center}
\end{figure}

Let $\bf n^v$ be the normal vector to the vanes,
and $\bf n^r$ the normal vector pointing out of the vane
region\footnote{\normalsize The superscripts ``v'' and ``r'' stand for
vane and radial, respectively.}.
Then, $\bf t^v$ is the tangential vector to the vanes pointing out of
the vane region and is defined as,

%\begin{equation}
% \bf t^v = \left( {\bf n^v_y},{\bf -n_x^v} \right) \text{sign}\left[
%	    \left( {\bf n^v_y},{\bf -n_x^v} \right) \cdot {\bf n^r} \right].
%\end{equation}
\begin{equation}
 \bf t^v = \left( {\bf n^{v^\perp}} \right) \text{sign}\left(
	    {\bf n^{v^\perp}} \cdot {\bf n^r} \right).
\end{equation}
Here, ${\bf n^{v^\perp}}$ is the vector perpendicular to the normal
vector of the vanes, which is simply,
\begin{equation*}
 {\bf n^{v^\perp}} = \left[ \begin{array}{c}
n_x\\
n_y\\
\end{array}\right]^{\perp} =
 \left[ \begin{array}{c}
  -n_y\\
  n_x\\
	\end{array}\right].
\end{equation*}

%
% example of an algorithm
%
\begin{center}
 \begin{algorithm}
  \caption{The crude separation model. This model identifies if the flow
  is coming into or out of the vane region, and if the velocity vector
  is in the same direction as the tangent line of the vanes. In the case
  of the ``special forcing'' the flow is forced as if it was
  impacting a solid surface. In the
  algorithm below, $r_0$ is the max radius of vanes, $r_i$ the minimum
  radius of vanes, and $\delta$ is the width of the separation region.}
  \label{alg:sep}
  \begin{algorithmic}[1]
   \If{($r_0 > r > r_i$)}
   \If{$(r_0 - r) < \delta$ \textbf{or} $((r - r_i) < \delta)$}
   \State $\overrightarrow{n^r} = \overrightarrow{r}/|r|$
   \If{$(r - r_i) < \delta$}
   \State $\overrightarrow{n^r} = -\overrightarrow{n^r}$
   \EndIf
   \State  $\bf t^v = \left( {\bf n^{v^\perp}} \right) \text{sign}\left(
	    {\bf n^{v^\perp}} \cdot {\bf n^r} \right)$
   \If{$v \cdot t^v > 0$ \textbf{and} $v \cdot n^r < 0 $}
   \State ${\bf n}({\bf x}) = \hat r$ \quad (Special Forcing)
   \Else
   \State  ${\bf n}({\bf x}) = \sin \left(\phi(r) \right) \hat{{\bf r}}+ \cos
  \left(\phi(r) \right) \hat{{\bf \theta}} $
  \quad (Normal Forcing)
   \EndIf
   \Else
   \State ${\bf n}({\bf x}) = \sin \left(\phi(r) \right) \hat{{\bf r}}+ \cos
  \left(\phi(r) \right) \hat{{\bf \theta}} $
  \quad (Normal Forcing)
   \EndIf
   \EndIf
  \end{algorithmic}
 \end{algorithm}
\end{center}
%

The forcing is modified when the velocity vector of the local flow, $\bf
u$ is pointing in to the forcing region: ${\bf u} \cdot {\bf n^r} < 0$, and
when the velocity vector is in the same direction as the tangent line to
the vanes: ${\bf u} \cdot {\bf t^v} > 0 $. In these instances, the
forcing acts as if there was a rigid surface past the vane edge, and
gives the appearance of a special ``no-penetration'' condition for the
velocity for these cases. The pseudo-code for this procedure is shown in
Algorithm~\ref{alg:sep}.

The addition of this simple separation model significantly reduced the
flow that penetrated the back of the vanes, and produces results
consistent with the observations provided by our experimental
colleagues.

\subsection{Effect of Surface Roughness}

%%
%% this does not describe the phenomena being modeled or the precise
%% model -- rewrite
%%
%%
%% this does not say enough about the surface roughness
%% motivate that and explain how it is used, than show your estimate
%% to argue it is small
%%

Surface roughness effects have been shown to play a role in the
formation of dust devils and related atmospheric
phenomena\cite{oke1987boundary}. For the flat and sandy regions we are
simulating, the impact is expected to be a small velocity perturbation
in the vertical direction. This is modeled as a volumetric
forcing in a narrow region above the surface,
\begin{equation}
 F^{'''}_{z_0} = \frac{1}{2}\rho V_z^2/z_{0},
\end{equation}
where $z_{0}$ is the roughness height. We ensure that the energy
introduced into the flow is a small fraction of total flow energy by comparing
this with the energy flux through the top of the vanes. The total energy
added is measured as,
\begin{equation}
 E_{\text{injected}} = \int_0^{2\pi} \int_0^R \int_0^{z_0} F^{'''}_{z_0}
  dz dr d\theta.
\end{equation}
R is the outer diameter of the vanes.
The value of $E_{\text{injected}}$ is typically a few percent of the
total kinetic energy flux measured through the top of the
vanes.

%This general forcing provides additional capabilities including the
%ability to investigate engineering greater surface roughness or
%structures that could provide greater ``kick-up'' of the thin thermal
%layer near the surface into the flowing regime. It can also support more
%general turning configurations than the virtual vanes outlined above.

\subsection{Simulation Geometry and Boundary Conditions}
\label{sec:bc}

In this project, two principle modeling regimes are considered.
One is the ``thermal-only'' scenario, in which there are no ambient
velocities and there is an imposed elevated temperature on the ground.
In the other, there are also ambient winds that contribute to the SoV energy
(``wind'' cases).
The computational domain and boundary conditions for these
two scenarios are described below.

\textbf{Computational Domain}

All simulations are performed in a cuboid domain, with six
faces.  The domain is denoted $\Omega \subset \mathbb{R}^3$.
The domain extents are scaled by the system diameter, D, created by the
outer vane radius. The extents are defined in terms of $\{L_x,L_y,L_z\}$ indicating the
streamwise, spanwise and vertical directions, respectively.
For both simulation regimes, sensitivity analyses
were performed to ensure that the results were not sensitive
to the domain extents. For the thermal-only case, for which $L_x = L_y$,
the system
extents $L_x/D$ and $L_y/D$ are chosen to be 3. The height ($L_z/D$) is
three times the system diameter, which is typically nearly equal to the
height of the vanes. This defines the thermal-only domain $\Omega_T$,
as $\Omega_T = \left[-L_x,L_x \right] \times \left[-L_y,L_y \right]
\times \left[0,L_z \right]$.

For the wind cases, the streamwise extent is no longer equal to
the spanwise length, $L_y$. In these cases, the domain length extends
two diameters in front of the vanes and three behind. The
spanwise direction is symmetric and extends two diameters in each direction
from the center ($L_y/D = 2$). The height is identical to the
thermal-only case, at three system diameters ($L_z/D = 3$). Thus, the
wind domain is defined as $\Omega_W = \left[-2D,3D \right] \times
\left[-L_y,L_y \right] \times \left[0,L_z \right]$.

The boundary for the thermal only case is decomposed as,
$\partial \Omega_T = \Gamma_G \bigcup \Gamma_T \bigcup \Gamma_P $.
$\Gamma_G$ is the boundary along the ``Ground'', $\Gamma_T$
the ``Top'' boundary, and $\Gamma_P$ the four periodic ``Sides''. A 3d
diagram labeling these boundaries is shown in
Figure~\ref{fig:thermal3d}. For this case study (no mean wind),
periodic boundary conditions are used on the four sides , with a modified
``inflow-outflow'' Neumann condition\cite{gunzburger1989finite} on the
top boundary. On the ground, a ``no-slip'' velocity boundary condition is
imposed, and a Dirichlet condition uniformly fixes
the temperature of the surface.
Each of the $\Gamma$ boundary terms are defined in the paragraphs below.
Note that a finite thickness ``Sponge Layer'' is
indicated on the figure along the top boundary and is defined below.

\begin{figure}[!htb]
  \begin{center}
    \includegraphics[width = 14 cm]{figs/thermal_only_3d}
    \caption{Domain for the thermal-only
   scenario. The diagram scale is representative of typical cases. Note
   the SoV apparatus in the center, which provides perspective on the
   extent of the domain with respect to the turning vane diameter. The
   ground, sides and top boundaries are labeled with the discussion the
   precise boundary conditions on each provided in
   section~\ref{sec:bc}. Notice also the finite thickness, high
   viscosity ``sponge layer'' at the top of the domain.}
    \label{fig:thermal3d}
  \end{center}
\end{figure}

The boundary for the wind cases is decomposed as,
$\partial \Omega_W = \Gamma_G \bigcup \Gamma_T \bigcup \Gamma_O \bigcup
\Gamma_I \bigcup \Gamma_S $.
Where $\Gamma_G$ is the boundary along the ``Ground'',
$\Gamma_T$ the ``Top'' boundary, $\Gamma_S$ the two ``Sides'',
$\Gamma_I$ the inflow boundary, and $\Gamma_O$ the ``Outflow''
boundary.
The ``wind'' simulation domain is diagrammed in
Figure~\ref{fig:wind3d}, with the boundaries labeled.
For this particular study (a heated ground with
an ambient wind), the wind case has a proscribed inlet boundary layer
along the upstream streamwise face ($\Gamma_I$) for both the temperature
and the velocity. The ``Ground'' boundary is identical to
the thermal-only case. The ``Sides'', ``Outflow'' and ``Top'' are all
set to modified Neumann boundary conditions. Note that ``Sponge Layers''
are used on both the back boundary and the top.

\begin{figure}[!htb]
  \begin{center}
   %\includegraphics[width = 14 cm]{figs/wind_3d}
   \includegraphics[width = 14 cm]{figs/wind_3d}
    \caption{Domain for the wind and thermal scenario. The diagram scale
   is representative of typical cases. Note the SoV apparatus which
   provides perspective on the extent of the domain with respect to the
   turning vane diameter. The ground, sides, inflow, back and top
   boundaries are labeled with the discussion the precise boundary
   conditions on each provided in section~\ref{sec:bc}. Notice also the
   finite thickness, high viscosity ``sponge layer'' at the top and back
   of the domain.}
    \label{fig:wind3d}
  \end{center}
\end{figure}

\textbf{Ground Boundary Conditions, $\Gamma_G$}

For both the wind and thermal-only cases the ground has a fixed
temperature and no-slip velocity boundary conditions. This boundary
($\Gamma_G$) is modeled with a Dirichlet boundary condition,
\begin{align}
 \overrightarrow{u} &= 0 \quad \text{ on } \Gamma_G \\
 T &= T_g.
\end{align}
Where $\Gamma_G = \{(x,y,0) \subset \partial \Omega \} $.

%
% http://fenicsproject.org/documentation/dolfin/dev/python/demo/documented/periodic/python/documentation.html
%
%
\textbf{Periodic Boundary Condition, $\Gamma_P$}

A periodic boundary condition is used in the thermal only cases,
along the streamwise and spanwise boundary faces
(denoted $\Gamma_{P,x}$ and $\Gamma_{P,y}$, respectively). In these
cases the state variables
are contrained to have the same value on the opposite faces of the domain,
for instance in the streamwise direction the boundary conditions are,
\begin{align}
 \overrightarrow{u}(-L_x,y,z) &= \overrightarrow{u}(L_x,y,z) \quad \text{ on } \Gamma_{P,x} \\
 T(-L_x,y,z) &= T(L_x,y,z)
\end{align}
and in the spanwise direction,
\begin{align}
 \overrightarrow{u}(x,-L_y,z) &= \overrightarrow{u}(x,L_y,z) \quad \text{ on } \Gamma_{P,y} \\
 T(x,-L_y,z) &= T(x,L_y,z).
\end{align}
Where $\Gamma_{P,x} = \{(-L_x,y,z) \bigcup (L_x,y,z) \subset \partial
\Omega \}$
and $\Gamma_{P,y} = \{(x,-L_y,z) \bigcup (x,L_y,z) \subset \partial
\Omega \}$.

\textbf{Inflow Boundary Condition, $\Gamma_I$}

On the inflow boundary $(\Gamma_I$), dirichlet conditions are used for both
velocity and temperature. The boundary-normal, or streamwise component
is a function of the surface normal coordinate (z), representing a boundary
layer below a uniform velocity, U.
The common 7th power model of a turbulent boundary layer is used,
\begin{equation*}
  u_{\text{in}}(z) = U \text{ min }\left(\left(\frac{z}{\delta}\right)^7,1\right)
  \label{eq:bl_u}
\end{equation*}
where $\delta$, the boundary layer thickness, is set based on data
measured by our experimental partners in the field.
The thermal boundary layer is assumed to have a similar boundary layer,
but, as observed in real atmospheric flows, there remains a vertical
temperature gradient outside the thin boundary layer. Based on
results in the literature a $2/3$ Kelvin per meter gradient is
imposed\cite{Blocken2007238}. The thermal inflow is then
\begin{equation*}
  T_{\text{in}}(z) = \Delta T \left(1- \text{ min
			}\left(\left(\frac{z}{\delta}\right)^7,1\right)\right)
  + T_0 - 2z/3.
  \label{eq:bl_t}
\end{equation*}
% 335+18*tanh(-z/0.1)-z*2/3
The inflow boundary is at the surface $x=-L_x$.

\textbf{Mixed inflow/outflow Boundary Conditions on $\Gamma_T$,
$\Gamma_S$ and $\Gamma_B$}

At outflow boundaries, a homogeneous Neumann condition is appropriate,
\begin{align}
  \frac{\partial u}{\partial n}\bigg|_{\Gamma_T} = 0 \\
  \frac{\partial T}{\partial n}\bigg|_{\Gamma_T} = 0
\end{align}
However, for the cases in this study,
a modified Neumann condition is necessary due to the possibility that there will
be an inflow on these boundaries.
For example, in the region above the vanes, the concentrated hot plume is
lifted by buoyancy upward and out of the simulation domain. However, the
radial inflow towards the apparatus is drawn in by large scale
convection cells larger than the system diameter. Thus, our boundary
conditions must permit inflow along the areas above and external to the
vanes, while simultaneously permitting outflow in the area above the vanes.

% Roy Stogner: Okay, so this DDN is basically no-traction when there's
% outflow and Tn=v when there's inflow?  We have no-traction for v*n,
% no-traction for anything else when there's outflow, and Dirichlet
% v.cross.n = 0 when there's inflow.

To accomplish this, the boundary condition is,
\begin{align}
  \frac{\partial u_n}{\partial n}\bigg|_{\Gamma_T} = 0 \\
  \text{if } (w<0) \text{ then}& \begin{cases}
    u_t = 0,\\
    T = T_{\text{in}}
  \end{cases} \\
  \text{ else}& \begin{cases}
    \frac{\partial u_t}{\partial n}\bigg|_{\Gamma_T} = 0, \\
    \frac{\partial T}{\partial n}\bigg|_{\Gamma_T} = 0
  \end{cases}
\end{align}

where $u_n$ and $u_t$ are normal and tangential components of the velocity,
respectively. This boundary condition is applied on the top boundary
$\Gamma_T$ ($z=L_z$) and downstream side boundary  $\Gamma_B$ in the wind case.

\textbf{Sponge Layer}

Finally, a finite thickness ``sponge layer'' is used in the region adjacent
to the mixed inflow/outflow boundaries $\Gamma_T$ and $\Gamma_B$.
This layer artificially increases the momentum diffusivity by
up to a factor of ten times the nominal value. This was designed to stabilize
the modified Neumann boundary conditions which can exhibit an instability
when there is a compact jet of fluid leaving the domain.
These regions are referred to by many names in the
literature\cite{doi:10.1146/annurev.fluid.36.050802.121930}, such as
absorbing layers, fringe regions, buffer zones, sponges, etc.

%%  This
%% would create small high velocity inflows, and the feedback loop would
%% result in instabilities and numerical blow-up. Mindful of the fact that
%% the character of solution not important in this region, and that our
%% physical interest remains focused on the region inside and in immediate
%% proximity to the vanes, we introduced a higher diffusivity ``sponge''
%% region that would diffuse the high velocity exiting jets sufficiently to
%% prevent numerically un-physical behavior. No results are quoted from
%% this ``sacrificial'' region, as it is not considered physically
%% meaningful.


\chapter{Visual Exploration of Time Series Datasets}
In data science, it is very rare to get a well-defined and detailed problem description. A much more common task is to analyze the data and to detect interesting patterns, outliers, or other phenomena that are fulfilling a complex set of criteria, which results in multiple processing steps. As we cannot comprehensively represent all necessary information only by numbers, we use data visualization.

This chapter looks at methods for comprehensible and effective visualization of comprehensibly and effectively visualize a time series datasets. These methods enable to study these datasets in detail.

\section{Dimensionality Reduction}
Usual datasets that we encounter in computer science have often tens or hundreds of dimensions, far beyond our imagination's capabilities. It makes it unintuitive for us to examine the dataset while also increases computational requirements for its analysis and visualization. For a visual exploration of the dataset, it is necessary to project our data into a lower-dimensional space. In this section, we will focus on the dimensionality reduction techniques, that are specifically used in data visualization.

\subsection{Principal Component Analysis}
The oldest and probably the most used dimensionality reduction technique in computer science is the \textit{Principal Component Analysis} (PCA) \cite{vis:pca}. PCA is a linear transformation that determines the orthogonal axes in which the dataset has the largest variances and then projects it onto these axes. They are represented as orthogonal eigenvectors with eigenvalues that correspond to the variance on the vector. Formally we define the PCA transformation as:
\begin{equation}
    PCA(X) = X \times W_L^T = M
\end{equation}
where $X$ ($m \times n$) is the original dataset with $m$ data points, each having $n$ features, $W^T_L$ ($n \times l$) is the transposed matrix of $l$ eigenvectors, and $M$ ($m \times l;~ l \leq n$) is the lower-dimensional representation of $X$. Usually we choose eigenvectors with the largest eigenvalues that are corresponding to directions with the largest variances.

Because PCA comes from statistical data analysis, its primary purpose is to capture maximal statistical information in lower-dimensional representation, not data visualization. We usually use eigenvectors with the largest variance to project data into two or three-dimensional embeddings when we want to use it for data visualization. It means that we mainly show global structures without enough detail to see the data's local behavior (Fig.~\ref{fig:pca-emb}). In combination with incredible speed and efficiency of modern PCA implementations, it is a perfect first step for every visual exploration of multi-dimensional datasets.

\subsection{t-SNE}
t-SNE or \textit{t-Distributed Stochastic Neighbor Embedding} \cite{vis:tsne} is a popular manifold representation learning technique that is particularly efficient for visualizing high dimensional datasets \cite{vis:repre-learning}. It transforms the multi-dimensional data into a low-dimensional space, emphasizing the preservation of local similarities and distances from the high-dimensional space. It does so by converting affinities of data points into probabilities. Gaussian joint probabilities represent the higher-dimensional space's affinities, and Student's t-distributions represent the embedded space's affinities. Because it very well preserves the local structures in a dataset (Fig.~\ref{fig:tsne}), it is one of the most popular techniques for data visualization \cite{vis:tsne-analysis}.
\begin{figure}[h]
    \centering
    \begin{subfigure}[b]{0.33\textwidth}
        \centering
        \includesvg[width=\textwidth]{img/s_dataset.svg}
        \caption{S-shape dataset}
        \label{fig:s-shape-sub}
    \end{subfigure}
    \begin{subfigure}[b]{0.3\textwidth}
        \centering
        \includesvg[width=\textwidth]{img/s_dataset_pca.svg}
        \caption{PCA embedding}
        \label{fig:pca-emb}
    \end{subfigure}
    \begin{subfigure}[b]{0.3\textwidth}
        \centering
        \includesvg[width=\textwidth]{img/s_dataset_tsne.svg}
        \caption{t-SNE embedding}
        \label{fig:tsne}
    \end{subfigure}

    \caption{Comparison of PCA (b) and t-SNE (c) applied to the S-shape dataset (a), commonly used for benchmarking the dimensionality reduction techniques.}
    \label{fig:s-shape}
\end{figure}

However, the usage of t-SNE comes with several disadvantages:
\begin{itemize}
    \item It is quite computationally expensive. The common practice uses some other fast dimensionality reduction methods, such as PCA, to reduce the dataset into a reasonable number of parameters and use t-SNE afterwards. Another option is to use optimized approximate Barnes-Hut t-SNE \cite{vis:barnes-hut-tsne}, which is only available for two or three-dimensional embeddings.
    \item As it is focusing on preserving mainly the local structure of the data, it does not guarantee to preserve the global structure correctly. For example, the number and distances between clusters are not always reliable.
    \item t-SNE is a stochastic algorithm, and each its run every run could return slightly different results.
\end{itemize}

\subsection{UMAP and densMAP}
\textit{Uniform Manifold Approximation and Projection} (UMAP) \cite{vis:umap} is one of the more recent manifold dimension reduction techniques. Similar to t-SNE, it is excellent for visualization but also as a general non-linear dimensionality reduction transformation. The underlying idea of UMAP comes from the Riemannian geometry and algebraic topology. It makes three assumptions about the data:
\begin{enumerate}
    \item Dataset has a uniform distribution on the Riemannian manifold.
    \item The Riemannian metric is locally approximately constant.
    \item The manifold is locally connected.
\end{enumerate}
Using these assumptions, UMAP models the manifold as a fuzzy topological structure, and the resulting low-dimensional representation is the closest possible representation with an equivalent topological structure. In other words, it captures the local similarities of data with respect to their global structure (Fig.~\ref{fig:fas-umap}).

Usage of UMAP over the t-SNE comes with several advantages:
\begin{itemize}
    \item It scales way better than t-SNE. It is faster, more efficient, and not restricted to two or three-dimensional embeddings so that it can process even high-dimensional sparse data.
    \item Because UMAP is a general dimensionality reduction technique, it is possible to use it in a preprocessing step.
    \item Although both methods mainly capture the local similarities in data, UMAP shows better results in preserving the global structure.
    \item UMAP can use non-metric distance measures.
\end{itemize}

Despite their visualization qualities, both t-SNE and UMAP neglect information about the local density of the original dataset. As they mainly look at the $n$ closest neighbors but not at their density, it could lead to misleading visualizations where the cluster's size and shape are primarily representing how many points are in it, rather than the underlying distribution.
\begin{figure}[h]
    \centering
    \begin{subfigure}[b]{0.33\textwidth}
        \centering
        \includegraphics[width=\textwidth]{img/fashion-pca.png}
        \caption{PCA}
        \label{fig:fas-pca}
    \end{subfigure}
    \begin{subfigure}[b]{0.3\textwidth}
        \centering
        \includegraphics[width=\textwidth]{img/fashion-umap.png}
        \caption{UMAP}
        \label{fig:fas-umap}
    \end{subfigure}
    \begin{subfigure}[b]{0.3\textwidth}
        \centering
        \includegraphics[width=\textwidth]{img/fashion-densmap.png}
        \caption{densMAP}
        \label{fig:fas-densmap}
    \end{subfigure}
    \captionsetup{font={footnotesize}}
    \caption{Comparison of PCA (a), UMAP (b), and densMAP (c) on the Fashion MNIST dataset \cite{vis:fashion-mnist}.}
    \label{fig:fashion}
\end{figure}

Narayan et al. \cite{vis:densMAP} introduced density-preserving data visualization derivatives called \textit{den-SNE} and \textit{densMAP}. Since both methods converge by iteratively optimizing their objective functions, they added a new term called \textit{local radius} to represent local densities. In other words, this term represents an average distance to the closest neighbors. We can see the difference in Fig.~\ref{fig:fashion}, where we applied the UMAP and densMAP on the Fashion MNIST dataset \cite{vis:fashion-mnist}. The densMAP's embedding is more spread out due to density preservation.

\section{Time Series Downsampling for Visualization}
Plotting time series containing many points comes with several pitfalls. We cannot draw all data points in time series without overplotting due to restricted space, making it almost impossible for us to examine them in detail. The second pitfall is that with the rising number of objects to render, we need more power. Because we do not want to create a misleading plot, finding the closest representation with visible structures while minimizing information loss is essential.

\subsection{Simple Downsampling and Piecewise Aggregate Approximation}
The most straightforward solution to overplotting in spatial data is sub-sampling the dataset, where its time series equivalent is called downsampling. If we try to sub-sample the data in the same way as in the spatial domain, we quickly encounter problems with the temporal aspect.

As we cannot randomly select points, the simplest downsampling algorithm picks every $n$-th data point. This algorithm is very fast and we can use it on specific types of time series because we are losing $\frac{1}{n}$ of the original information.

\textit{Piecewise Aggregate Approximation} (PAA) is a simple downsampling algorithm that, instead of removing data points, aggregates them into smaller representations \cite{vis:paa}. The basic idea is to split time series into approximately equal-sized buckets and aggregate their data points. We can use any aggregation function like average, mode, or median, based on the application. As the aggregation function combines all data points in the bucket, PAA captures much more information than selecting every $n$-th data point. Because of the simple nature of PAA, the whole process is fast and scalable.

\subsection{Largest Triangle Downsampling}
Already mentioned algorithms have excellent properties from a computational perspective but not from a visualization standpoint. As they remove or aggregate data points by a fixed value or range, we can lose some visually important information.

Because of this issue, Steinarsson proposed the Largest Triangle downsampling algorithms designed for time series downsampling for visual representation \cite{vis:lttb}. These algorithms select data points by their effective area, similarly to the \textit{Visvalingam–Whyatt algorithm}. Having data points $X_a$, $X_b$, $X_c$ such as $a < b < c$ and indices $a$, $b$, $c$ are the positions in time series, the effective area of the $X_b$ is the area of a triangle $X_a X_b X_c$ (Fig.~\ref{fig:effective-area}). The indices $a$, $b$, $c$ are chosen differently in every algorithm, and for the final downsampling, we are using the data points with the largest effective area.

\begin{figure}[h]
    \centering
    \includesvg[width=\textwidth]{img/effective-area.svg}
    \caption{Effective area when $X_a$, $X_b$, and $X_c$ are directly successive data points (there are no data points between them). The color of the effective area corresponds to the color of a data point, first and last datapoints are different as they are part of the downsampled result.}
    \label{fig:effective-area}
\end{figure}

\subsubsection{Largest Triangle One Bucket Algorithm}
The simplest algorithm proposed by Steinarsson is \textit{the Largest Triangle One Bucket} (LTOB) algorithm. First, all points get rank by their effective area, which we compute using directly successive data points. Afterwards, it removes data points with zero effective areas and splits points into buckets based on the number of data points we want to have in the downsampled time series. For every bucket, it selects point with the highest rank (Fig.~\ref{fig:ltob}). This method's disadvantage is that it only uses the effective area computed from two adjacent points, which can potentially lead to misleading representations.

\begin{figure}[h]
    \centering
    \includesvg[width=\textwidth]{img/ltob.svg}
    \caption{For every bucket, the LTOB downsampling algorithm selects data points with the largest effective area within the bucket.}
    \label{fig:ltob}
\end{figure}

\subsubsection{Largest Triangle Three Buckets Algorithm}
\textit{The Largest Triangle Three Buckets}~(LTTB) algorithm partially solves the problem of the LTOB and searches a much larger area for point exclusion. First it separates data points into equally-sized buckets, where the first and the last data point get their own buckets, as we do not want to exclude them. Second, it computes the effective area for every point by iterating through the buckets from left to right, taking three directly successive buckets $A$, $B$, $C$ at a time. For every point $X_b \in B$, it computes the effective area as an area $S$ of a triangle $X_a X_b X_c$, where $X_a \in A$, $X_c \in C$, such as:
\begin{equation}
    S = \max_{X_a \in A,~X_c \in C} S_{X_a X_b X_c}
\end{equation}
Finally, for every bucket, it selects the point with the largest effective area (Fig.~\ref{fig:lttb}).

This algorithm is robust, reasonably efficient, and solves the problems of LTOB. The only problem could be the bucket selection when working with data with values non-uniformly spread over time.

\begin{figure}[h]
    \centering
    \includesvg[width=\textwidth]{img/lttb.svg}
    \caption{LTTB downsampling algorithm.}
    \label{fig:lttb}
\end{figure}

\subsubsection{Largest Triangle Dynamic Algorithm}
Because both methods above rely on equally-sized buckets, the last algorithm Steinarsson proposed is the \textit{Largest Triangle Dynamic}~(LTD) algorithm. First, it uses the predefined number of equally-sized buckets. Then it computes the interval \textit{calmness} as the mean square error of linear regression on that bucket, including the last point from the previous interval and the first point in the next one. Afterwards, it joins two adjacent intervals with the smallest error and divides the one with the largest error, so the number of buckets remains the same. This iterative process converges to the optimal bucket sizes. The number of iterations is empirically determined, but in the original paper, the author recommends starting with one-tenth of the original time series's size. Afterwards, it uses the LTTB algorithm to select one point from each bucket.

TLD solves the problem of equally-sized buckets but requires a lot of computational time, which makes it hard to apply it to long time series.

\subsubsection{Datashader}
All of the approaches mentioned so far transform time series into a representation with fewer points, which we then can use for visualization. But as we downsample time series, we lose information based on the number of points we eventually plot. For example, it is not uncommon to have time series consisting of hundred thousand data points, and usually, we are downsampling to less than one thousand data points for visualization. It is impossible to capture the information precisely and even with the best possible downsampling algorithms, we lose a lot of information, especially the initial data density. If our task requires to study these underlying properties, we must logically plot all our data points.
\begin{figure}[tbh]
    \centering
     \includegraphics[width=\textwidth]{img/datashader.png}
    \caption{Datashader and LTTB demonstrated on one milion points.}
    \label{fig:my_label}
\end{figure}

The \textit{Datashader} renderer \cite{vis:datashader} is a complete graphical pipeline from the original data to final graph. It breaks plotting into multiple computations on intermediate representations.
That allows multiple fast and efficient aggregations, such as value counts and averaging. Then Datashader renders the results to the final image's pixels as color saturations. The resulting representation is a raster image that accurately represents the aggregated information. Accuracy is limited only by the resolution of the final image and, due to its effectiveness, it can process millions of data points in a reasonable time. As the raster image loses detail as we zoom in, this method gives us an excellent overview of the whole time series (or dataset), but if we want to explore details, we need to repeat the processing pipeline all over again.

\section{Chapter Summary}
In this chapter, we showed several dimensionality reduction techniques used for visualization. Generally, it is advisable to start with PCA to capture the global data structure and then combine it with either t-SNE, UMAP, or densMAP to examine the details. In our case, we will primarily use UMAP and densMAP as they have several advantages over t-SNE.

For time series plotting, we can use the LTTB downsampling algorithm if we want a smaller representation for visualization or use the Datashader renderer to render the original data accurately.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% PROBLEM 2 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Problem 2}

Consider a bare sphere composed of a homogeneous multiplying medium.
\begin{enumerate}[a)]
\item Give the steady-state, continuous energy diffusion equation. Assume that the diffusion coefficient ($D$) and the average neutrons produced from fission ($\nu$) are constant for all energies.
\item Derive the multigroup equation corresponding to the case where there are three energy groups. Assume that there is no upscattering, all groups are directly coupled, and fission is only induced by the slowest group while only producing neutrons in the fastest group.
\item Write the multigroup equation you found as a matrix-equation.
\end{enumerate}


\section{Method}
\label{sec:method}

In the following, we introduce the mathematical formulation of the weakly-supervised 3D shape completion problem. Subsequently, we briefly discuss denoising variational auto-encoders (\DVAEs) \citep{Kingma2014ICLR,Im2017AAAI} which we use to learn a strong shape prior \red{that embeds a set of reference shapes in a low-dimensional latent space.} Then, we formally derive our proposed amortized maximum likelihood (\AML) approach. \red{Here, we use maximum likelihood to learn an embedding of the observations within the same latent space -- thereby allowing to perform shape completion.} The overall approach is also illustrated in \figref{fig:method}.

\subsection{Problem Formulation}
\label{subsec:method-problem}

\input{fig_method_problem}

In a supervised setting, the task of 3D shape completion can be described as follows: Given a set of incomplete observations $\mathcal{X} = \{x_n\}_{n = 1}^N \subseteq \mathbb{R}^R$ and corresponding ground truth shapes $\mathcal{Y}^* = \{y_n^*\}_{n = 1}^N \subseteq \mathbb{R}^R$, learn a mapping $x_n \mapsto y_n^*$ that is able to generalize to previously unseen observations and possibly across object categories. We assume $\mathbb{R}^R$ to be a suitable representation of observations and shapes; in practice, we resort to occupancy grids and signed distance functions (SDFs) defined on regular grids, \ie, $x_n, y_n^* \in \mathbb{R}^{H \ntimes W \ntimes D} \simeq \mathbb{R}^R$. Specifically, occupancy grids indicate occupied space, \ie, voxel $y_{n,i}^* = 1$ if and only if the voxel lies on or inside the shape's surface. To represent shapes with sub-voxel accuracy, SDFs hold the distance of each voxel's center to the surface; for voxels inside the shape's surface, we use negative sign.
Finally, for the (incomplete) observations, we write $x_n \in \{0, 1, \uk\}^R$ to make missing information explicit; in particular, $x_{n,i} = \uk$ corresponds to unobserved voxels, while $x_{n,i} = 1$ and $x_{n,i} = 0$ correspond to occupied and unoccupied voxels, respectively.

On real data, \eg, KITTI \citep{Geiger2012CVPR}, supervised learning is often not possible as obtaining ground truth annotations is labor intensive, \cf \citep{Menze2015CVPR,Xie2016CVPR}. Therefore, we target a weakly-supervised variant of the problem instead: Given observations $\mathcal{X}$ and reference shapes $\mathcal{Y} = \{y_m\}_{m = 1}^M \subseteq \mathbb{R}^R$ both of the same, known object category, learn a mapping $x_n \mapsto \tilde{y}(x_n)$ such that the predicted shape $\tilde{y}(x_n)$ matches the unknown ground truth shape $y_n^*$ as close as possible -- or, in practice, the sparse observation $x_n$ while being plausible considering the set of reference shapes, \cf \figref{fig:method-problem}. Here, supervision is provided in the form of the known object category. Alternatively, the reference shapes $\mathcal{Y}$ can also include multiple object categories resulting in an even weaker notion of supervision as the correspondence between observations and object categories is unknown. \red{Except for the object categories, however, the set of reference shapes $\mathcal{Y}$, and its size $M$, is completely independent of the set of observations $\mathcal{X}$, and its size $N$, as also highlighted in \figref{fig:method}.} On real data, \eg, KITTI, we additionally assume the object locations to be given in the form of 3D bounding boxes in order to extract the corresponding observations $\mathcal{X}$. In practice, the reference shapes $\mathcal{Y}$ are derived from watertight, triangular meshes, \eg, from ShapeNet \citep{Chang2015ARXIV} or ModelNet \citep{Wu2015CVPR}.

\subsection{Shape Prior}
\label{subsec:method-prior}

\red{We approach the weakly-supervised shape completion problem by first learning a shape prior using a denoising variational auto-encoder (\DVAE). Later, this prior constrains shape inference (see \secref{subsec:method-inference}) to predict reasonable shapes. In the following, we briefly discuss the standard variational auto-encoder (\VAE), as introduced by \cite{Kingma2014ICLR}, as well as its denoising extension, as proposed by \cite{Im2017AAAI}.}

\boldparagraph{Variational Auto-Encoder (\VAE)}
%
We propose to use the provided reference shapes $\mathcal{Y}$ to learn a \red{generative} model of possible 3D shapes over a low-dimensional latent space $\mathcal{Z} = \mathbb{R}^Q$, \ie,  $Q \ll R$. \red{In the framework of \VAEs, the joint distribution $p(y, z)$ of shapes $y$ and latent codes $z$} decomposes into $p(y | z)p(z)$ with $p(z)$ being a unit Gaussian, \ie, $\mathcal{N}(z;0, I_Q)$ and $I_Q \in \mathbb{R}^{R \times R}$ being the identity matrix. \red{This decomposition allows to sample $z \sim p(z)$ and $y \sim p(y|z)$ to generate random shapes.}
\red{For training, however, we additionally need to approximate the posterior $p(z | y)$.} \red{To this end, the so-called recognition model $q(z | y) \approx p(z | y)$} takes the form
%
\begin{align}
q(z | y) &= \mathcal{N}(z; \mu(y), \text{diag}(\sigma^2(y)))\label{eq:encoder-decoder}
\end{align}
%
where $\mu(y), \sigma^2(y) \in \mathbb{R}^Q$ are predicted using the encoder neural network. \red{The generative model $p(y|z)$ decomposes over voxels $y_i$; the corresponding probabilities $p(y_i | z)$ are represented using Bernoulli distributions for occupancy grids or Gaussian distributions for SDFs:}
%
\begin{align}
    \begin{split}
        p(y_i | z) &= \text{Ber}(y_i ; \theta_i(z))\quad\text{or}\\
        p(y_i | z) &= \mathcal{N}(y_i ; \mu_i(z), \sigma^2).\label{eq:decoder}
    \end{split}
\end{align}
%
In both cases, the parameters, \ie, $\theta_i(z)$ or $\mu_i(z)$, are predicted using the decoder neural network. \red{For SDFs, we explicitly set $\sigma^2$ to be constant (see \secref{sec:training}). Then, $\sigma^2$ merely scales the corresponding loss, thereby implicitly defining the importance of accurate SDFs relative to occupancy grids as described below.}

In the framework of variational inference, the parameters of the encoder and the decoder \red{neural networks} are found by maximizing the likelihood $p(y)$. \red{In practice, the likelihood is usually intractable and the evidence lower bound is maximized instead, see \citep{Kingma2014ICLR,Blei2016ARXIV}. This results in the following loss to be minimized:}
%
\begin{align}
\mathcal{L}_{\text{VAE}}(w) = - \mathbb{E}_{q(z |y)}[\ln p(y|z)] + \text{KL}(q(z | y)| p(z)).\label{eq:vae}
\end{align}
%
\red{Here,} $w$ are the weights of the encoder and decoder \red{hidden in the recognition model $q(z | y)$ and the generative model $p(y | z)$, respectively}. \red{The Kullback-Leibler divergence $\text{KL}$ can be computed analytically as described in the appendix of \citep{Kingma2014ICLR}.} The negative log-likelihood $-\ln p(y|z)$ corresponds to a binary cross-entropy error for occupancy grids \red{and} a scaled sum-of-squared error for SDFs. The loss $\mathcal{L}_{\text{VAE}}$ is minimized using stochastic gradient descent (SGD) by approximating the expectation \red{using samples}:
%
\begin{align}
- \mathbb{E}_{q(z |y)}[\ln p(y|z)] \approx - \sum_{l = 1}^L \ln p(y | z^{(l)})
\end{align}
%
\red{The required samples $z^{(l)} \sim q(z|y)$ are computed using the so-called reparameterization trick,
\begin{align}
z^{(l)} = \mu(y) + \epsilon^{(l)} \sigma(y)\quad\text{with}\quad\epsilon^{(l)} \sim \mathcal{N}(\epsilon; 0, I_Q),\label{eq:repa}
\end{align}
in order to make $\mathcal{L}_{\text{VAE}}$, specifically the sampling process, differentiable.} \red{In practice, we found $L = 1$ samples to be sufficient -- which conforms with results by \cite{Kingma2014ICLR}. At test time, the sampling process $z\sim q(z|y)$ is replaced by the predicted mean $\mu(y)$.}
\red{Overall, the standard VAE allows us to embed the reference shapes in a low-dimensional latent space. In practice, however, the learned prior might still include unreasonable shapes.}

\boldparagraph{Denoising \VAE (\DVAE)}
%
\red{In order to avoid inappropriate shapes to be included in our shape prior, we consider a denoising variant of the \VAE allowing to obtain a tighter bound on the likelihood $p(y)$.} More specifically, a corruption process $y' \sim p(y' | y)$ is considered and the corresponding evidence lower bound results in the following loss:
%
\begin{align}
    \begin{split}
        \mathcal{L}_{\text{DVAE}}(w) = &- \mathbb{E}_{q(z | y')}[\ln p(y|z)]\\
        &+ \text{KL}(q(z | y')| p(z)).
    \end{split}
\end{align}
%
\red{Note that the reconstruction error $-\ln p(y|z)$ is still computed with respect to the uncorrupted shape $y$ while $z$, in contrast to \eqnref{eq:vae}, is sampled conditioned on the corrupted shape $y'$.} In practice, the corruption process $p(y' | y)$ is modeled using Bernoulli noise for occupancy grids and Gaussian noise for SDFs.
In experiments, we found \DVAEs to learn more robust latent spaces \red{-- meaning the prior is less likely to contain unreasonable shapes. In the following, we always use \DVAEs as shape priors.}

\subsection{Shape Inference}
\label{subsec:method-inference}

After learning the shape prior, \red{defining the joint distribution $p(y, z)$ of shapes $y$ and latent codes $z$ as product of generative model $p(y|z)$ and prior $p(z)$}, shape completion can be formulated as a maximum likelihood (\ML) problem \red{for $p(y, z)$} over the lower-dimensional latent space $\mathcal{Z} = \mathbb{R}^Q$. The corresponding negative log-likelihood $-\ln p(y, z)$ to be minimized can be written as
%
\begin{align}
\mathcal{L}_{\text{ML}}(z) &= - \sum_{x_i \neq \uk} \ln p(y_i = x_i | z) - \ln p(z).\label{eq:ml}
\end{align}
%
As the prior $p(z)$ is Gaussian, the negative log-probability $- \ln p(z)$ is proportional to $\|z\|_2^2$ and \red{constrains the problem to likely, \ie, reasonable, shapes with respect to the shape prior}. As before, the generative model $p(y | z)$ decomposes over voxels; here, we can only consider actually observed voxels $x_i \neq \uk$. \red{We assume that the learned shape prior can complete the remaining, unobserved voxels $x_i = \uk$.} Instead of solving \eqnref{eq:ml} for each observation $x \in \mathcal{X}$ independently, however, we follow the idea of amortized inference \citep{Gersham2014COGSCI} and train a \red{new} encoder $z(x;w)$ to \emph{learn} \ML. To this end, we keep the generative model $p(y|z)$ fixed and train \red{only} the weights $w$ of the \red{new} encoder $z(x;w)$ using the \ML objective as loss:
%
\begin{align}
    \begin{split}
        \mathcal{L}_{\text{dAML}}(w) =& - \sum _{x_i \neq \uk} \ln p(y_i = x_i | z(x; w))\\
        &- \lambda \ln p(z(x; w)).\label{eq:aml}
    \end{split}
\end{align}
%
\red{Here,} $\lambda$ controls the importance of the shape prior. The exact form of the probabilities $p(y_i = x_i | z)$ depends on the used shape representation. For occupancy grids, this term results in a cross-entropy error as \red{both the predicted voxels $y_i$ and the observations $x_i$} are, for $x_i \neq \uk$, binary. For SDFs, however, the term is not well-defined as $p(y_i | z)$ is modeled with a continuous Gaussian distribution, while the observations $x_i$ are binary. As solution, we could compute (signed) distance values along the rays corresponding to observed points (\eg, following \citep{Steinbrucker2013ICCV}) in order to obtain continuous observations $x_i \in\mathbb{R}$ for $x_i \neq \uk$. However, as illustrated in \figref{fig:method-sdf}, noisy observations cause the distance values along the whole ray to be invalid. This can partly be avoided when relying \red{only} on occupancy to represent the observations; in this case, free space (\cf \figref{fig:method-problem}) observations are partly correct even though observed points may lie within the corresponding shapes.

For making SDFs tractable (\ie, to predict sub-voxel accurate, visually smooth and appealing shapes, see \secref{sec:experiments}) \red{while using binary observations}, we propose to define $p(y_i = x_i | z)$ through a simple transformation. In particular, as $p(y_i | z)$ is modeled using a Gaussian distribution $\mathcal{N}(y_i ; \mu_i(z), \sigma^2)$ where $\mu_i(z)$ is predicted using the fixed decoder ($\sigma^2$ is constant), and $x_i$ is binary (for $x_i \neq \uk$), we introduce a mapping $\theta_i(\mu_i(z))$ transforming the predicted \red{mean SDF value to an occupancy probability} $\theta_i(\mu_i(z))$:
%
\begin{align}
p(y_i = x_i | z) = \text{Ber}(y_i = x_i; \theta_i(\mu_i(z)))
\end{align}
%
As, \red{by construction (see \secref{subsec:method-problem}), occupied voxels have negative sign or value zero in the SDF}, we can derive the occupancy probability $\theta_i(\mu_i(z))$ as the probability of a non-positive distance:
%
\begin{align}
\theta_i(\mu_i(z)) &= \mathcal{N}(y_i \leq 0; \mu_i(z), \sigma^2)\\[3px]
&= \frac{1}{2} \left(1 + \text{erf}\left(\frac{- \mu_i(z)}{\sigma \sqrt{\pi}}\right)\right).\label{eq:sdf}
\end{align}
%
Here, $\text{erf}$ is the error function which, in practice, can be approximated following~\citep{Abramowitz1974}. \eqnref{eq:sdf} is illustrated in \figref{fig:method-sdf} where the occupancy probability $\theta_i(\mu_i(z))$ is computed as the area under the Gaussian bell curve for $y_i \leq 0$. This per-voxel transformation can easily be implemented as non-linear layer and its derivative \wrt $\mu_i(z)$
is, by construction, a Gaussian. \red{Note that the transformation is correct, not approximate, based on our model assumptions and the definitions in \secref{subsec:method-problem}.} \red{Overall, this transformation allows us to easily minimize \eqnref{eq:aml} for both occupancy grids and SDFs using binary observations. The obtained encoder embeds the observations in the latent shape space to perform shape completion.}

\begin{figure}[t]
	\vspace*{-\figskipabove px}
	\centering
	\hfill
	\begin{subfigure}[t]{0.25\linewidth}
		\vspace{0px}
		\centering
		\includegraphics[height=4.5cm]{fig_method_sdf_2}
	\end{subfigure}
	\begin{subfigure}[t]{0.6\linewidth}
		\vspace{3px}
		\centering
		\hspace*{-12px}
		\includegraphics[height=4.5cm]{fig_method_sdf_1}
	\end{subfigure}
	\hfill
	\vspace*{-8px}
	\caption{{{\bf Left: Problem with SDF Observations.} Illustration of a ray ({\color{red}red line}) correctly hitting a surface ({\color{blue}blue line}) causing the (signed) distance values and occupancy values computed for voxels along the ray to be correct (\cf (a)). A noisy ray, however, causes all voxels along the ray to be assigned incorrect distance values (marked {\colorbox{red!25}{red}}) \wrt to the true surface ({\color{blue}blue line}) because the ray ends far behind the actual surface (\cf (b)). When using occupancy only, in contrast, only the voxels behind the surface are assigned invalid occupancy states (marked {\colorbox{red!25}{red}}); the remaining voxels are labeled correctly (marked {\colorbox{green!25}{green}}; \cf (c)).
			{\bf Right: Proposed Gaussian-to-Bernoulli Transformation.} For $p(y_i) := p(y_i | z) = \mathcal{N}(y_i;\mu_i(z), \sigma^2)$ ({\color{blue}blue}), we illustrate the transformation discussed in \secref{subsec:method-inference} allowing to use the binary observations $x_i$ (for $x_i \neq \uk$) to supervise the SDF predictions. This is achieved by transforming the predicted Gaussian distribution to a Bernoulli distribution with occupancy probability $\theta_i(\mu_i(z)) = p(y_i \leq 0)$ ({\color{blue}blue area}).}}
	\label{fig:method-sdf}
	\vspace*{-\figskipbelow px}
\end{figure}

\subsection{Practical Considerations}

\input{fig_data_synthetic}

\boldparagraph{Encouraging Variety}
%
So far, our \AML formulation assumes a deterministic encoder $z(x,w)$ which predicts, given the observation $x$, a single code $z$ corresponding to a completed shape. A closer look at \eqnref{eq:aml}, however, reveals an unwanted problem: the data term scales with the number of observations, \ie, $|\{x_i \neq \uk\}|$, while the regularization term stays constant -- with less observations, the regularizer gains in importance leading to limited variety in the predicted shapes because $z(x; w)$ tends towards zero.

In order to encourage variety, we draw inspiration from the \VAE shape prior. Specifically, we use a probabilistic recognition model
%
\begin{align}
q(z|x) = \mathcal{N}(z; \mu(x), \text{diag}(\sigma^2(x)))
\end{align}
%
(\cf see \eqnref{eq:encoder-decoder}) and replace the negative log-likelihood $-\ln p(z)$ with the corresponding Kullback-Leibler divergence $\text{KL}(q(z|x)|p(z))$ with $p(z) = \mathcal{N}(z; 0, I_Q)$. Intuitively, this makes sure that the encoder's predictions ``cover'' the prior distribution -- thereby enforcing variety. Mathematically, the resulting loss, \ie,
%
\begin{align}
\begin{split}
    \mathcal{L}_{\text{AML}}(w) =& - \red{\mathbb{E}_{q(z|x)}\left[\sum_{x_i \neq \uk} \ln p(y_i = x_i | z)\right]}\\
    &+ \lambda \text{KL}(q(z|x)p(z)),\label{eq:daml}
\end{split}
\end{align}
%
can be interpreted as the result of maximizing the evidence lower bound of a model with observation process $p(x | y)$ (analogously to the corruption process $p(y'|y)$ for \DVAEs in \citep{Im2017AAAI} and \secref{subsec:method-prior}). \red{The expectation is approximated using samples (following the reparameterization trick in \eqnref{eq:repa}) and, during testing, the sampling process $z \sim q(z|x)$ is replaced by the mean prediction $\mu(x)$.} In practice, we find that \eqnref{eq:daml} improves visual quality of the completed shapes. We compare this \AML model to its deterministic variant \dAML in \secref{sec:experiments}.

\boldparagraph{Handling Noise}
%
Another problem of our \AML formulation concerns noise. On KITTI, for example, specular or transparent surfaces cause invalid observations -- laser rays traversing through these surfaces cause observations to lie within shapes or not get reflected. However, our \AML framework assumes deterministic, \ie, trustworthy, observations \red{-- as can be seen in the reconstruction error in \eqnref{eq:daml}.} Therefore, we introduce per-voxel weights $\kappa_i$ computed using the reference shapes $\mathcal{Y} = \{y_m\}_{m=1}^M$:
%
\begin{align}
\kappa_i = 1 - \left(\frac{1}{M} \sum_{m = 1}^M y_{m,i}\right) \in [0,1]
\end{align}
%
where $y_{m,i} = 1$ if and only if the corresponding voxel is occupied. Applied to observations $x_i = 0$, these are trusted less if they are unlikely under the shape prior. Note that for point observations, \ie, $x_i = 1$, this is not necessary as we explicitly consider ``filled'' shapes (see \secref{sec:data}). This can also be interpreted as imposing an additional \red{mean shape prior on the predicted shapes with respect to the observed free space}. In addition, we use a corruption process $p(x' | x)$ consisting of Bernoulli and Gaussian noise during training (analogously to the \DVAE shape prior).