\chapter{Backgrounds}
\label{sec:background}

In this chapter, we briefly review related backgrounds.

\section{E-graphs and e-matching}

\paragraph{Terms.}
Let $\Sigma$ be a set of function symbols with associated arities.
A function symbol is called a \textit{constant} if it has a zero arity.
Let $V$ be the set of variables. We define $T(\Sigma, V)$ to be the set of terms constructed using function symbols from $\Sigma$ and variables from $V$.
More formally, $T(\Sigma, V)$ is the smallest set that (1) all variables and constants are in $T(\Sigma,V)$ and (2) $t_1,\dots,t_k\in T(\Sigma,V)$ implies $f(t_1,\dots,t_k)\in T(\Sigma,V)$, where $f\in \Sigma$ has arity $k$.
A \textit{ground term} is a term in $T(\Sigma,V)$ that contains no variables. A non-ground term is also called a \textit{pattern}. We call a term of the form $f(t_1,\ldots,t_k)$ an $f$-application term.

\paragraph{Congruence relation.}
An \textit{equivalence relation} $\equiv_\Sigma$ is a binary relation over $T(\Sigma,\emptyset)$ that is reflexive, symmetric, and transitive. A \textit{congruence relation} $\cong_\Sigma$ is an equivalence relation satisfying that if $t_i\cong t'_i$ for all $i=1,\dots,n$ hold, $f(t_1,\dots,t_n)\cong f(t'_1,\dots,t'_n)$ holds as well, where $f$ is a $n$-ary function symbol. We wrote $\cong$ when $\Sigma$ is clear from the context.

\paragraph{\Egraph.}
Intuitively, an \egraph $E$ is a set of \eclasses $\{c_1, \ldots, c_k\}$, where each \eclass $c$ is a set of \enodes $\{n_1,\ldots,n_k\}$. Each \enode consists of a function symbol $f$ and a list of children \eclasses.
Similar to patterns, we call an \enode of the form $f(c_1, \ldots c_n)$ an $f$-application \enode.
More formally, we define an \egraph $E$ to be an tuple $(\Sigma,N,C,\textit{symbol}, \textit{lookup}, \textit{child})$ where
\begin{itemize}
    \item $\Sigma$ is a set of function symbols with associated arities,
    \item $N$ is the set of \enodes,
    \item $C$ is the set of \eclasses,
    \item \textit{symbol} is a function that maps each \enode in $N$ to a function symbol in $\Sigma$,
    \item \textit{lookup} is a function that maps every \enode in $N$ to the \eclass in $C$ that contains it, and
    \item \textit{child} is a function that maps an $\enode$ to a list of \eclass that are the children of this \enode.
\end{itemize}
For convenience, we use $n.\sym$, $n.\textit{id}$, and $n.\child_i$ to denote $\textit{symbol}\,(n)$, $\textit{lookup}\,(n)$, and the $i$th element of $\child(n)$.

An \egraph $E$ efficiently represents sets of ground terms in a congruence relation.
An \egraph is said to \textit{represent} a ground term $t$ if its \eclasses represent it.
An \eclass $c$ represents a ground term if $c$ contains an \enode $a$ that represents it.
An \enode $f(c_1,\dots,c_k)$ represents a ground term $f(t_1,\dots,t_k)$ if they have the same function symbol $f$ and each e-class $c_i$ represents term $t_i$.
Terms represented by an \eclass are equivalent to each other.
Consequently, an \egraph forms a congruence relation.
Consider \autoref{fig:egraph} for example. $g(f(c,c))$, $f(c, g(b))$, and $f(a, g(a))$ are three ground terms represented by \eclass $c_1$ in this \egraph. Therefore, they are congruent with each other by definition.

\begin{figure}[!t]
    \hspace{-2em}
  \begin{tabular}[b]{cc}
    \begin{subfigure}[b]{0.45\linewidth}
        \includegraphics[width=\linewidth]{figures/egraph.png}
      \caption{}
      \label{fig:egraph}
    \end{subfigure}
    \begin{tabular}[b]{c}
      \begin{subfigure}[t]{0.45\columnwidth}
      \centering
        \begin{tabular}{|ccc|}
            \hline
            eclass-id & $\text{child}_1$ & $\text{child}_2$ \\
            \hline
            1         & 4        & 3        \\
            2         & 4        & 4       \\
            \hline
        \end{tabular}
        \caption{}
        \label{fig:repr-f}
      \end{subfigure}\\
      \begin{subfigure}[b]{0.5\columnwidth}
      \centering
        \begin{tabular}{|cc|}
            \hline
            eclass-id & $\text{child}_1$ \\
            \hline
            1         & 2         \\
            3         & 4         \\
            3         & 5         \\
            \hline
        \end{tabular}
        \caption{}
        \label{fig:repr-g}
      \end{subfigure}
    \end{tabular}
  \end{tabular}
  \caption{(a) an e-graph over $T(\Sigma, \emptyset)$ and  $\cong_\Sigma$ where $\Sigma=\{f,g,a,b,c\}$ and $a,b,c$ are nullary functions. Each solid box denotes an e-node and each dashed box denotes an e-class. Every term represented by an e-class is mutually equivalent. For example, $a\cong_\Sigma c$, $g(a)\cong_\Sigma g(b)$, and $f(a, g(a)) \cong_\Sigma g(f(a, a))$. The labels of e-classes are at bottom left.\quad (b) relation representing $f$. \quad(c) relation representing $g$.}
\end{figure}

\paragraph{\Ematching.}
\Ematching is the task of finding \ematching substitutions that instantiate patterns to set of terms represented in the \egraph.
An \textit{\ematching substitution} $\sigma$ is a function that maps every variable in a pattern to e-classes.
For convenience, we use $\sigma(p)$ to denote the set of terms obtained by replacing every occurrence of variable $v_i$ in $p$ with terms in $\sigma(v_i)$.
Formally, given an e-graph $E$ and a pattern $p$,  \ematching finds the set of all possible pairs $(\sigma, r)$ such that every term in $\sigma(p)$ is represented in the e-class $r$.
Terms in $\sigma(p)$ are said to be matched by pattern~$p$. $r$ is said to be the root of matched terms.
For instance, pattern $f(\alpha, g(\alpha))$
 matches four terms in e-class $c_1$: $f(a, g(a))$, $f(a, g(c))$, $f(c,g(c))$, and $f(c, g(a))$;
 all of which are witnessed by the substitution $\{\alpha \mapsto c_4\}$.

Existing approaches to e-matching rely on backtracking \citep{efficient-ematching,simplify,egg}.
For example, \citet{efficient-ematching} proposed a backtracking-based e-matching algorithm that is used by \textsc{Z3} \citep{Z3} and \egg \citep{egg}, two state-of-the-art e-graph implementations.
To match the pattern $f(\alpha, g(\alpha))$ on the e-graph in Figure \ref{fig:egraph}, their algorithm does a depth-first search over the e-graph:
% Conceptually, it recursively searches for the pattern $f(\alpha, g(\beta))$ and filters out substitutions that map $\alpha$ and $\beta$ to different e-classes.
it searches for all $f$-application e-nodes $n_f$, adds $\alpha\mapsto n_f.\textit{child}_1$ to substitution $\sigma$, iterates through all $g$-application e-nodes $n_g$ in e-class $n_f.\textit{child}_2$, and only yield $\sigma$ if $n_g.\textit{child}_1=\sigma(\alpha)$.
In general, this procedure runs in time that is quadratic of the \egraph size.
In a large e-graph, there may be thousands of pairs of $n_f$ and $n_g$ where $n_g$ is in e-class $n_f.\textit{child}_2$, but only a few satisfy the constraint $n_f.\textit{child}_1=n_g.\textit{child}_1$. Therefore, backtracking-based \ematching enumerates an unnecessarily large pool of candidates.
Even worse, complex query patterns may involve many variables that occur at several places, which makes na\"ive backtracking enumerates an excessive number of terms and therefore extremely slow.
This inefficiency is due to the fact that na\"ive backtracking does not use the equality constraints to prune the search space \textit{globally}.
This is in contrast to our approach, which exploits the equality constraints during query planning for greater performance and guarantees worst-case optimality with respect to the output size.

\section{Conjunctive queries}

\paragraph{Relational schema.}
A relational schema $S_D$ over domain $D$ is a set of relation symbols with associated arities. An atom under a schema $S_D$ is an expression $R(t_1,\ldots,t_k)$, where $k$ is the arity of $R$ in $S_D$ and $t_i$ is an element in $D$. An instance of $S_D$ is a set of atoms over $S_D$.

\paragraph{Conjunctive queries.}
A conjunctive query over schema $S_D$ and a set of variables $V$ is a formula of the form
\[
    \textit{ans}(x_1,\ldots x_k)\textit{ :- } R_1(v_{1,1},\ldots,v_{1,k_1}),\ldots, R_n(v_{n,1},\ldots,v_{n,k_n}),
\]
where $n\geq 0$, $R_1\ldots R_n$ are relation names in $S_D$ with arities $k_1,\ldots k_n$, \textit{ans} is the name of the resulting relation not in $S_D$, $v_{i,j}$ are variables in $V$\footnote{Some definitions of conjunctive queries allow both variables and constants, but we only allow variables in conjunctive queries for simplicity.}, and $x_1,\ldots x_k$ are variables occurring in $v_{i,j}$. We call such $R_i(v_{i,1},\ldots,v_{i,k_1})$ an atom of the conjunctive query.

\paragraph{Semantics of conjunctive queries.}
Similar to \ematching substitutions, we define a conjunctive query substitution as a function that maps every variable occurring in $v_{i,j}$ to elements in $D$. The semantics of conjunctive queries is defined as follows: Let $q$ be a conjunctive query, $I$ be an instance of $S_D$, conjunctive queries yield the set of all $\textit{ans}(\sigma(x_1),\ldots,\sigma(x_k))$ where $\sigma$ is a substitution satisfying that $R_i(\sigma(t_1),\ldots \sigma(t_k))$ are atoms in $I$ for $i=1,\ldots,n$. We denote the result set as $q(I)$.

We make the observation that the definition of conjunctive query and \ematching
are definitionally similar to each other: both are defined as finding
substitutions whose instantiations are present in a (relational or graph)
database. In fact, the \ematching problem can be viewed as a ``nested''
conjunctive queries on a database where the atoms are nested. Therefore, it is
tempting to reduce the \ematching problem to a conjunctive query over the
relational database, thereby benefiting from well-studied techniques from the
database community, including join algorithms, query optimization, and query
evaluations.

\section[The AGM bound]{The AGM bound and Generic Join \footnote{This section is inspired by Remy Wang's \href{https://gitlab.com/remywang/blog/-/blob/master/posts/wcoj.md}{introduction to generic join algorithms}.}}

\algrenewcomment[1]{\(\triangleright\) #1}
\algnewcommand{\LineComment}[1]{\State \(\triangleright\) {\it #1}}
\paragraph{The AGM Bound}
\begin{figure}
  \centering
\begin{algorithmic}[1]
\Procedure{SolveTriangleQuery}{$R,S,T$}
\State $A := R(x,y).x \cap T(z,x).x$\;
  \LineComment{Compute $Q(\alpha,y,z) \textit{ :- } R(\alpha,y),S(y,z),T(z,\alpha)$}
  \For{$\alpha\in A$}
  \State $B := R(\alpha,y).y \cap S(y,z).y$\;
    \LineComment{Compute $Q(\alpha,\beta,z) \textit{ :- } R(\alpha,\beta),S(\beta,z),T(z,\alpha)$}
    \For{$\beta \in B$}
        \State $C := S(\beta,z).z \cap T(z,\alpha).z$\;
        \LineComment{Yield join results $Q(\alpha, \beta, \gamma)$}
        \For{$\gamma \in C$}
        \State {\bf output } $Q(\alpha,\beta,\gamma)$
        \EndFor
    \EndFor
  \EndFor
\EndProcedure
  \caption{Generic join algorithm for the triangle query $Q(x,y,z) \textit{ :- } R(x,y),S(y,z),T(z,x)$, with ordering $[x, y, z]$.}\label{alg:gj}
\end{algorithmic}
\end{figure}

Given a database and a conjunctive query, the AGM bound \cite{agm} is a bound on how large the query output could be. A simple bound just multiplies the cardinally of each relation, which is the size of the Cartesian product of all relations.
For example, for triangle query $ Q(x,y,z) \textit{ :- } R(x,y), S(y,z), T(x,z)$, this  bound computes to $|Q| \leq |R| \times |S| \times |T|$.
If $|R|=|S|=|T|=N$, then $|Q| \leq N^3$.
However, we can do better: we observe that $Q$ contains fewer atoms than the query
$ Q’(x,y,z)\textit{ :- } R(x,y), S(y,z),$ as $Q$ is further filtered by joining with $T$. Therefore, we know $|Q| \leq |R| \times |S| = N^2$.
In fact, the theoretical bound of conjunctive query output size is given as the AGM bound \citep{agm}, which is $N^{3/2}$ in this case.

The AGM bound proposes a question: is there an algorithm that always achieves the performance described by the AGM bound? In particular, \textit{cyclic queries} is a kind of conjunctive queries whose query hypergraph contains cycles, such as the triangle query $Q$ above, and they are known to make traditional join plans (e.g., join plans using two-way joins like hash joins or merge joins) suboptimal. Most of real-world database systems therefore have bad performance on cyclic queries.

\paragraph{Generic join.} Fortunately, theoretical progress has been made in the database theory community to achieve optimal performance on conjunctive queries and on cyclic queries in particualr. In particular, the generic join algorithm for conjunctive query is developed that runs in time linear to the worst-case output size with a log factor \citep{wcoj}.
The generic join algorithm has great performance in practice when the query is cyclic, and is competitive to traditional join algorithms in other cases.
Given a conjunctive query and a relational database, generic join is parameterized over a ordering of the set of variables in the query, say $[x,y,z]$ for the query above. The ordering does not matter for the optimality guarantee, but may impact practical performance greatly.
Given an ordering,
we assume the input relations are stored in tries sorted by the ordering.
That is, given the ordering $[x,y,z]$,
$R(x,y)$ is sorted by $x$ and then $y$,
and the first-level trie nodes are the $x$'s.
% Even more concretely,
% Figrue~\ref{fig:trie} illustrates the trie
% representing $R=\{(3, 4), (2, 1), (2, 5)\}$.
We can very efficiently intersect (join) relations on their
first (according to the variable ordering) variable given such tries.
Algorithm~\ref{alg:gj} shows the generic join algorithm for computing $Q$.
Note that selection, e.g. $R(a, y)$ is very fast with the built tries.
$A \cap B$ can also be done in $\tilde{O}(\min(|A|, |B|))$ time
($\tilde{O}$ means $O$ with a log factor).
For general queries we may have to intersect more than two relations,
in which case the intersection should be performed in
$\tilde{O}(\min_i|A_i|)$ time (using the merge in merge-sort).


\section{Discretization of the Electrophysiology Models}\label{sec:discretization}

The partial and ordinary differential equations described in the last section contain spatial and temporal derivatives that have to be discretized to be solved numerically. For temporal derivatives, we use timestepping schemes, for spatial derivatives, we employ the finite element method.

In this section, we describe the discretization of the subcellular and electrophysiology models that were presented in the last section. A description of the discretization of the solid mechanics model follows in \cref{sec:discretization_mechanics}.

We begin with the discretization in time in \cref{sec:discretization_monodomain}, followed by the spatial discretization for the monodomain (\cref{sec:discretization_diffusion,sec:mass_lumping}) and multidomain models (\cref{sec:discretization_multidomain,sec:discretization_body_domain}).

\subsection{Discretization of the Monodomain Model}\label{sec:discretization_monodomain}

Electrophysiology models typically consist of a reaction-diffusion equation. The diffusion term describes the electric conduction in the tissue and the reaction term includes the subcellular model. In our model, the monodomain equation  \cref{eq:monodomain} used in the fiber based description and the second multidomain equation \cref{eq:multidomain2} are equations of this type.

This type of partial differential equation is often solved using operator splitting schemes. A first order operator splitting scheme is Godunov splitting \cite{Godunov2003}. It was used for the solution of the chemo-electro-mechanical model in \cite{Roehrle2012}. In addition to Godunov splitting, we also employ the second order accurate Strang splitting scheme \cite{Strang1968}.

In the following, the application of these two schemes is illustrated for the monodomain equation \cref{eq:monodomain}. The right-hand sides of the diffusion and reaction terms are denoted in short as $\mathcal{L}_1$ and $\mathcal{L}_2$:%
\begin{align*}
  \mathcal{L}_1(V_m) := \dfrac{1}{A_m\,C_m} \sigma_\text{eff}\dfrac{\partial^2 V_m}{\partial x^2}, &&
  \mathcal{L}_2(V_m) := -\dfrac{1}{C_m} I_\text{ion}(V_m,\bfy).
\end{align*}
%
Then, the monodomain equation takes the form:
%
\begin{align}\label{eq:monodomain_operator_splitting}
  \p{V_m}{t} = \mathcal{L}_1(V_m) + \mathcal{L}_2(V_m).
\end{align}

A timestepping scheme is constructed that starts with a given initial value $V_m^{(0)}$ and computes solution values $V_m^{(i)}$ at discrete points in time $t^{(i)} = i\cdot\dt$ with a fixed timestep width $\dt$.
Godunov splitting proceeds by alternatingly performing steps in the two directions of the right-hand sides $\mathcal{L}_1$ and $\mathcal{L}_2$. In the first substep per iteration, an intermediate value $V_m^{\ast}$ is calculated, which is used as starting point for the second substep. Each of the substeps are performed using independent timestepping scheme, e.g., the explicit Euler scheme:
%
\begin{subequations}\label{eq:monodomain_godunov}
  \begin{align}
    V_m^{\ast} &= V_m^{(i)} + \dt \mathcal{L}_1(V_m^{(i)},t^{(i)}),\\[4mm]
    V_m^{(i+1)} &= V_m^{\ast} + \dt \mathcal{L}_2(V_m^{\ast},t^{(i)})
  \end{align}
\end{subequations}

Strang splitting uses a similar approach with three substeps per timestep and two intermediate values $V_m^{\ast}$ and $V_m^{\ast\ast}$:
%
\begin{subequations}\label{eq:monodomain_strang}
  \begin{align}
    V_m^{\ast} &= V_m^{(i)} + \dfrac{\dt}{2} \mathcal{L}_1(V_m^{(i)},t^{(i)}),\\[4mm]
    V_m^{\ast\ast} &= V_m^{\ast} + \dt \mathcal{L}_2(V_m^{\ast},t^{(i)}),\\[4mm]
    V_m^{(i+1)} &= V_m^{\ast\ast} + \dfrac{\dt}{2} \mathcal{L}_1(V_m^{\ast\ast},t_{(i)}+\dfrac12 \dt).
  \end{align}
\end{subequations}
%
% stuff needed to create the plot
%\definecolor{vred}{RGB}{170,0,0}
%\definecolor{vyellow}{RGB}{212,170,0}
%\definecolor{vgreen}{RGB}{0,170,0}
%\begin{equation*}
%  \begin{array}{lll}
%    \frac{\partial}{\partial t} V_m  = \textcolor{vred}{c_1 \frac{\partial^2}{\partial x^2} V_m }\\[4mm]
%    \frac{\partial}{\partial t} \bfy = \textcolor{vyellow}{G(V_m,\bfy)}\\[4mm]
%    \frac{\partial}{\partial t} V_m = \textcolor{vyellow}{c_2\,I_\text{ion}(V_m,\bfy)}\\[4mm]
%    \textcolor{vyellow}{\dt_\text{0D}}\quad
%    \textcolor{vred}{\dt_\text{1D}}\quad
%    \dt_\text{splitting}\quad
%    \textcolor{vgreen}{\dt_\text{3D}}\quad
%    t^{(i)}, t^{(i)} + \dt_\text{splitting}/2, t^{(i)} + \dt_\text{splitting} ...\\[4mm]
%    t^{(i)} + \textcolor{vgreen}{\dt_\text{3D}}\\[4mm]
%    t^{(i+1)}
%  \end{array}
%\end{equation*}
%\textcolor{vyellow}{0D:}
%\textcolor{vred}{1D:}
%\textcolor{vgreen}{3D:}

Note that each substep can either be executed as a single timestep of the chosen method as in \cref{eq:monodomain_godunov,eq:monodomain_strang} or divided into several steps with timestep widths $\dt_{0D}$ (for the 0D subcellular model represented by $\mathcal{L}_1$) and $\dt_{1D}$ (for the diffusion equation represented by $\mathcal{L}_2$).

\Cref{fig:splitting_schemes} visualizes both splitting schemes applied to the monodomain equation.
The yellow arrows correspond to the solution of the 0D subcellular model. The red arrows correspond to the solution of the 1D diffusion equation. The timestep width of one splitting step is $\dt_\text{splitting}$. Depending on how the timestep widths are chosen in relation to each other, different numbers of subcycles are used in the solution of the 0D and 1D problems.

\begin{figure}%
  \centering%
  \begin{subfigure}[t]{0.48\textwidth}%
    \centering%
    \def\svgwidth{\textwidth}
    \includegraphics[width=\textwidth]{images/theory/godunov_splitting.pdf}
    \caption{The Godunov splitting uses two substeps: 0D, 1D.}%
    \label{fig:godunov_splitting}%
  \end{subfigure}
  \quad
  \begin{subfigure}[t]{0.48\textwidth}%
    \centering%
    \includegraphics[width=\textwidth]{images/theory/strang_splitting.pdf}
    \caption{The Strang splitting uses three substeps: 0D, 1D, 0D.}%
    \label{fig:strang_splitting}%
  \end{subfigure}
  \caption{Godunov and Strang splitting schemes that are used to solve the monodomain equation. The equation is split into a reaction part (0D,yellow) and a diffusion part (1D,red) and these parts are solved alternatingly. The visualizations show one splitting timestep starting at the left circle and completing at the right circle.}%
  \label{fig:splitting_schemes}%
\end{figure}%

Instead of the explicit Euler method in \cref{eq:monodomain_godunov,eq:monodomain_strang}, other timestepping methods can be used for the substeps.
We use the following schemes, which are listed as single steps for the generic ODE ${\p V_m / \p t = \mathcal{L}(V_m,t)}$:
\begin{subequations}\label{eq:ode_solver_schemes}
  \begin{align}
    V_m^{(i+1)} &= V_m^{(i)} + \dt \mathcal{L}(V_m^{(i)},t^{(i)}), \label{eq:explicit_euler}\\[4mm]
    V_m^{(i+1)} &= V_m^{(i)} + \dfrac{\dt}{2}\Big(
      \mathcal{L}(V_m^{(i)},t^{(i)}) + \mathcal{L}\big(V_m^{(i)} + \dt \mathcal{L}(V_m^{(i)},t^{(i)}),t^{(i+1)}\big)\Big), \label{eq:heun}\\[4mm]
    V_m^{(i+1)} &= V_m^{(i)} + \dt \mathcal{L}(V_m^{(i+1)},t^{(i+1)}), \label{eq:implicit_euler}\\[4mm]
    V_m^{(i+1)} &= V_m^{(i)} + \dfrac{\dt}{2}\Big(
      \theta\,\mathcal{L}(V_m^{(i+1)},t^{(i+1)}) + (1-\theta)\,\mathcal{L}(V_m^{(i)},t^{(i)})\Big).\label{eq:crank_nicolson}
  \end{align}
\end{subequations}
%
Here, \cref{eq:explicit_euler} is the first-order accurate explicit Euler scheme, \cref{eq:heun} is the second-order accurate Heun scheme, \cref{eq:implicit_euler} is the first order accurate implicit Euler scheme, and \cref{eq:crank_nicolson} is the Crank-Nicolson scheme \cite{CrankNicolson1947}, which for $\theta=0$ equals the explicit Euler and for $\theta=1$ equals the implicit Euler scheme. For $\theta=\frac12$, it is second order accurate. An advantage of the implicit schemes in \cref{eq:implicit_euler,eq:crank_nicolson} is, that, for our considered diffusion problems, they are unconditionally stable. A disadvantage is, that a linear equation has to be solved in every timestep.

A second order accurate timestepping scheme yields a faster decrease of the numerical error with decreasing step size and, thus, in many cases allows a larger step size than a first order scheme.
To obtain a second order scheme for the monodomain equation, we use Strang splitting (\cref{eq:monodomain_strang}) with the Crank-Nicolson scheme (\cref{eq:crank_nicolson}) for the diffusion term $\mathcal{L}_1$ and Heun's method (\cref{eq:heun}) for the reaction term $\mathcal{L}_2$. In the subcellular model, the system of ODEs with state vector $\bfy$ given in \cref{eq:subcellular} is solved with Heun's method along with the equation in terms of $V_m$.

Next, the spatial derivatives in the diffusion part $\mathcal{L}_2$ of the split equation have to be discretized. Then, both the multidomain and the fiber based models can be solved using the splitting scheme.

\subsection{Discretization of the Diffusion and Laplace Equations}\label{sec:discretization_diffusion}

For the spatial discretization, we first derive the finite element formulation for a generic parabolic diffusion equation in a domain $\Omega\subset\R^d$ of arbitrary dimensionality $d$. Then, specialization to 1D yields the formulation for the monodomain equation. Considering a 3D domain, the formulation is an important building block for the discretization of the multidomain model. This is shown in more detail in a later section, \cref{sec:discretization_multidomain}

We consider the following diffusion problem in the variable $u: \Omega \times [0,t_\text{end}] \to \R$ with Neumann boundary conditions on a part of the boundary $\Gamma_f \subset ∂\Omega$ with normal vector $\bfn$:
\begin{align*}
  \p{u}{t} &= \div(\bfsigma \grad u), &(\bfsigma\,\grad u) \cdot \bfn &= f \quad \text{on }\Gamma_f, & (\bfsigma\,\grad u) \cdot \bfn &= 0 \quad \text{on } ∂Ω\backslash \Gamma_f.
\end{align*}
We discretize the temporal derivative using the Crank-Nicolson scheme as in \cref{eq:crank_nicolson}. Following the procedure of the Galerkin finite element formulation with the Hilbert space $H^1_0(\Omega)$ of test functions $\phi$ that are zero on the boundary, we arrive at the following weak form:
%
\begin{align*}
  \ds\int_Ω \big(\theta\,∇\cdot(\bfsigma ∇ \bfu^{(i+1)})  + (1-\theta)\,∇\cdot(\bfsigma ∇ u^{(i)})\big)\,\phi \,\d\bfx &&\\
    \qquad = \dfrac{1}{\dt} \ds\int_Ω(u^{(i+1)} - u^{(i)})\,\phi\,\d\bfx, &&\qquad \forall \phi \in H^1_0(\Omega).
\end{align*}
For brevity, we express divergence and gradient using the nabla operator.

To discretize the weak form in space, we choose a function space $V_h = \text{span}\{\varphi_j \mid j = 1, \dots, N\}$ to represent the solution as $u = \sum_{j=1}^N u_j \phi_j$. Applying the divergence theorem, we obtain:
\begin{equation}\label{eq:diffusion_helper1}
  \begin{array}{l}
    \ds\sum\limits_{j=1}^{N} \big(\theta\,u_j^{(i+1)} + (1-\theta)\,u_j^{(i)}\big)
    \left(-\ds\int_Ω \bfsigma\,∇\varphi_j\cdot ∇\varphi_k \,\d\bfx + \ds\int_{∂Ω} \big(\bfsigma\,∇\varphi_j\cdot \bfn\big)\,\varphi_k \,\d\bfx  \right) \\
      \quad = \dfrac{1}{\dt} \sum\limits_{j=1}^{N} \big(u_j^{(i+1)} - u_j^{(i)}\big) \ds\int_Ω \varphi_j\,\varphi_k\,\d\bfx, \qquad \forall k = 1,\dots, N.
  \end{array}
\end{equation}
This iteration step can be written in matrix notation in terms of the vectors of unknowns $\bfu^{(i)}=(u^{(i)}_0,\dots,u^{(i)}_N)^\top$ at timestep $i$:%
\begin{align*}
  \bfA\,\bfu^{(i+1)} = \bfb(\bfu^{(i)}).
\end{align*}
The system matrix $\bfA$ and the right-hand side $\bfb$ are given by:
\begin{align*}
  \bfA &= \theta\,(\bfK_{\bfsigma} + \bfB_{\bfsigma}) -\dfrac{1}{\dt}\bfM, &
  \bfb &= \big((\theta-1)\,(\bfK_{\bfsigma} + \bfB_{\bfsigma}) - \dfrac{1}{\dt} \bfM \big)\,\bfu^{(i)}.
\end{align*}
The formulation uses the standard stiffness matrix $\bfK_{\bfsigma}$, the matrix $\bfB_{\bfsigma}$ of the boundary integral and the mass matrix $\bfM$, whose components are defined as%
\begin{align}\label{eq:diffusion_matrices}
  \bfK_{\bfsigma,kj} &= -\ds\int_Ω (\bfsigma\, ∇\varphi_j)\cdot ∇\varphi_k \,\d\bfx,&
     \bfB_{\bfsigma,kj} &= \ds\int_{\Gamma_f} \big((\bfsigma\,∇\varphi_j)\cdot \bfn\big)\,\varphi_k \,\d\bfx,&
     \bfM_{kj} &= \ds\int_Ω \varphi_j\,\varphi_k\,\d\bfx.
\end{align}
Note that, after applying the divergence theorem, the definition of the stiffness matrix has a minus sign.

Next, we take into account the Neumann boundary condition $\bfsigma∇u\cdot \bfn = f$ on the boundary $\Gamma_f$. The flux $f$ over the boundary is discretized by $M$ separate ansatz functions $\psi_j$ on $\Gamma_f$ as $f = \sum_{j=1}^M f_j\, \psi_j$.
The flux values are summarized in a vector $\bff=(f_1,\dots,f_M)^\top$.
Plugging this into \cref{eq:diffusion_helper1} yields the following equation in matrix notation:%
\begin{align}\label{eq:diffusion_helper2}
  \tilde{\bfA}\,\bfu^{(i+1)} = \tilde{\bfb}(\bfu^{(i)}),
\end{align}
with the system matrix $\tilde{\bfA}$ and right-hand side $\tilde{\bfb}$:
\begin{align*}
  \tilde{\bfA} &= \theta\,\bfK_{\bfsigma} -\dfrac{1}{\dt}\bfM, &
    \tilde{\bfb} &= \big((\theta-1)\,\bfK_{\bfsigma} - \dfrac{1}{\dt} \bfM \big)\,\bfu^{(i)} - \bfB_{\Gamma_f}\,\big(\theta\,\bff^{(i+1)} + (1-\theta)\,\bff^{(i)}\big),
\end{align*}
and the boundary matrix $\bfB_{\Gamma_f}$ given by:
\begin{align}\label{eq:definition_boundary_matrix}
  \bfB_{\Gamma_f,kj} &= \ds\int_{\Gamma_f} \psi_j\,\varphi_k \,\d\bfx.
\end{align}
Note that incorporating the Neumann boundary conditions in the weak form corresponds to the following exchange of the boundary matrices $\bfB_\sigma$ and $\bfB_{\Gamma_f}$:%
\begin{align}\label{eq:boundary_relation}
  \bfB_\sigma\,\bfu = \bfB_{\Gamma_f}\,\bff.
\end{align}
%

\Cref{eq:diffusion_helper2} is used to solve the diffusion part of the monodomain equation given in \cref{eq:monodomain} after inserting the corresponding constant prefactors.

When deriving or implementing new models or optimizing solver code, it is often beneficial to study certain effects in isolation. It can help to use a toy problem such as the simple Laplace problem $Δu = 0$, possibly with Neumann boundary condition $\partial u/\partial \bfn = f$.
By specializing the formulation in \cref{eq:diffusion_helper2} accordingly, we obtain the system
%
\begin{align*}
  (\bfK_\bfI + \bfB_\bfI)\,\bfu = \bfzero
\end{align*}
for the case without boundary condition (set $\bfB_\bfI$ to zero to assume homogeneous Neumann boundaries) or
\begin{align}\label{eq:discretization_laplace}
  \bfK_\bfI\,\bfu = -\bfB_{\Gamma_f}\,\bff
\end{align}
to include the formulated Neumann boundary condition.

\subsection{Using Mass Lumping for Implicit Timestepping}\label{sec:mass_lumping}
Implicit timestepping schemes such as implicit Euler or the Crank-Nicolson scheme for $\theta=\frac12$ need to solve a linear equation in every timestep.
Assuming homogeneous Neumann boundary conditions for simplicity, the iteration step of the canonical Crank-Nicolson scheme follows from \cref{eq:diffusion_helper2}:
\begin{subequations}\label{eq:lumping_crank_nicolson}
  \begin{align}
    \big(\dfrac1{2}\bfK-\dfrac{1}{\dt}\bfM\big)\, \bfu^{(i+1)} &= \big(-\dfrac12{\bfK} - \dfrac{1}{\dt}\bfM\big)\, \bfu^{(i)}\label{eq:lumping_crank_nicolson_1}\\[4mm]
    \Leftrightarrow \quad (\bfI - \dfrac{\dt}{2}\,\bfM^{-1}\bfK)\,\bfu^{(i+1)}&= (\bfI + \dfrac{\dt}{2}\,\bfM^{-1}\bfK)\,\bfu^{(i)}.\label{eq:lumping_crank_nicolson_2}
  \end{align}
\end{subequations}
For the implicit Euler method, we obtain:%
\begin{subequations}\label{eq:lumping_implicit_euler}
  \begin{align}
    \ds(\bfK-\frac{\bfM}{\dt})\,\bfu^{(i+1)} &=\,\ds -\frac{\bfM}{\dt}\bfu^{(i)}\label{eq:lumping_implicit_euler_1}\\[4mm]
    \Leftrightarrow \quad (\bfI - \dt\,\bfM^{-1}\bfK)\,\bfu^{(i+1)}&= \,\bfu^{(i)}.\label{eq:lumping_implicit_euler_2}
  \end{align}
\end{subequations}
Both iteration steps in \cref{eq:lumping_crank_nicolson_1,eq:lumping_crank_nicolson_2} and in \cref{eq:lumping_implicit_euler_1,eq:lumping_implicit_euler} are equivalent, as the second equation follows from the first one by left multiplication of $(-\dt \bfM^{-1})$. In the second equations, the matrices to be multiplied are created by a sum of the unity matrix $\bfI$ and another matrix term that is scaled by the potentially small timestep width $\dt$. For the implicit Euler in \cref{eq:lumping_implicit_euler_2}, the matrix on the right-hand side even reduces to the identity matrix. This is preferred over the first iteration steps in \cref{eq:lumping_crank_nicolson_1,eq:lumping_implicit_euler_1} as it leads to better conditioned matrix-vector multiplications.

The required inversions of the mass matrix cannot be carried out explicitly as the inversion would fill in numerous matrix entries and eliminate the sparse structure. This is not feasible for highly resolved meshes with many degrees of freedom. Instead, \emph{mass lumping} is used, where the mass matrix $\bfM$ is approximated by a diagonal matrix with diagonal entries equal to the row sums in $\bfM$ \cite{Hinton1976}. Thus, multiplication with the inverse mass matrix corresponds to a rescaling of columns by the inverse lumped diagonal entries.

\subsection{Discretization of the Multidomain Model}\label{sec:discretization_multidomain}
With the prerequisites of temporal discretization in \cref{sec:discretization_monodomain} and the finite element formulation of a diffusion equation in \cref{sec:discretization_diffusion}, we can now discretize the multidomain model. Since this has not been previously done in literature using the finite element method, the subsequent derivation is more detailed.

The first multidomain equation given in \cref{eq:multidomain1} yields the following form after applying the finite element derivation in \cref{eq:diffusion_helper2}:
%
\begin{align}\label{eq:multidomain_discretization_helper_multidomain1}
  \big(\bfK_{\bfsigma_e + \bfsigma_i} + \bfB_{\bfsigma_e + \bfsigma_i}\big)\,\bfphi_{e} +  \s{k=1}{N_\text{MU}} f_r^k \big(\bfK_{\bfsigma_i^k} + \bfB_{\bfsigma_i^k}\big)\,\bfV_m^k = 0.
\end{align}
Here, $\bfphi_{e}$ and $\bfV_m^k$ are the vectors of degrees of freedom for the extracellular potential $\phi_e$ and membrane voltage $V_m^k$ of compartment $k$. The matrices are defined by \cref{eq:diffusion_matrices} and do not yet include the boundary conditions.
The subscripts of the stiffness matrices $\bfK$ and boundary integral matrices $\bfB$ refer to the anisotropy tensors that occur in their definitions.

The diffusion part of the second multidomain equation, \cref{eq:multidomain2}, discretized with Crank-Nicolson, yields the system%
\begin{align}\label{eq:multidomain_discretization_helper_multidomain2}
  \bfA\,\mat{\bfV_m^{k,(i+1)}\\ \bfphi_{e}^{(i+1)}} = \bfb,
\end{align}
%
with the $1 \times 2$ block system matrix $\bfA$ and right-hand side vector $\bfb$ given by:%
\begin{subequations}\label{eq:multidomain_discretization_helper_multidomain3}
\begin{align}
   \bfA &= \matt{
      \dfrac{\theta}{A_m^k\,C_m^k}(\bfK_{\bfsigma_i^k} + \bfB_{\bfsigma_i^k}) -\dfrac{1}{\dt}\bfM & \quad
      \dfrac{\theta}{A_m^k\,C_m^k}(\bfK_{\bfsigma_i^k} + \bfB_{\bfsigma_i^k})
    }, \\[4mm]
    \bfb &= \Big( \dfrac{\theta-1}{A_m^k\,C_m^k}(\bfK_{\bfsigma_i^k} + \bfB_{\bfsigma_i^k}) - \dfrac{1}{\dt}\bfM\Big)\,\bfV_m^{k,(i)}
      + \dfrac{\theta - 1}{A_m^k\,C_m^k}(\bfK_{\bfsigma_i^k} + \bfB_{\bfsigma_i^k})\,\bfphi_e^{(i)}.
\end{align}
\end{subequations}

A separate instance of this equation holds for every compartment $k$. Again, the integrals over the boundary are still present in the $\bfB_{\bfsigma_i^k}$ matrices.
To resolve this and to close the formulation, we have to consider the fluxes over the boundary of all involved unknowns and to replace them using the boundary conditions.

One required boundary conditions to solve the multidomain model without body domain is given in \cref{eq:multidomain_bc1}. The boundary condition for compartment $k$ in terms of the intracellular potential $\phi_i^k$,
%
\begin{align}\label{eq:multidomain_discretization_helper1}
  (\bfsigma_i^k\,∇\phi_i^k) \cdot \bfn_m = 0 \qquad \text{on } ∂\Omega_M,
\end{align}
%
is expressed in terms of the unknowns $V_m^k$ and $\phi_e$ to yield the condition
%
\begin{align}\label{eq:multidomain_discretization_helper2}
  (\bfsigma_i^k\,∇V_m^k) \cdot \bfn_m &= -(\bfsigma_i^k\,∇\phi_e)\cdot \bfn_m =: p^k \qquad \text{on }∂\Omega_M.
\end{align}
%
We define the value of this flux to be equal to a helper variable $p^k$.
A second flux is formulated for the extracellular potential $\phi_e$. We assign its value to the helper variable $q$:
\begin{align}\label{eq:definition_q}
  (\bfsigma_e ∇ \phi_e)\cdot \bfn_m =:q \qquad \text{on }∂\Omega_M.
\end{align}

We can now express the flux value $\big((\bfsigma_e + \bfsigma_i)\,∇\phi_e\big) \cdot \bfn_m$, which occurs in the discretized first multidomain equation, \cref{eq:multidomain_discretization_helper_multidomain1}, in terms of the variables $p^k$ and $q$. Using \cref{eq:multidomain_discretization_helper1,eq:multidomain_discretization_helper2} and the relation $\phi_e = \phi^k_i - V_m^k$, we derive:
\begin{align}\label{eq:multidomain_discretization_helper3}
   \big((\bfsigma_e + \bfsigma_i)\,∇\phi_e\big) \cdot \bfn_m
    &= (\bfsigma_e\,∇\phi_e)\cdot \bfn_m + (\bfsigma_i\,∇\phi_e)\cdot \bfn_m = q - \s{k=1}{N_\text{MU}} f_r^k\,p^k.
\end{align}

We discretize the flux values $p^k$ and $q$ analogously to the Neumann boundary condition flux $f$ in \cref{sec:discretization_diffusion} and summarize the degrees of freedoms in vectors $\bfp^k$ and $\bfq$.

Next, we combine the flux values with the first and second multidomain equation.
Plugging the generic relation \cref{eq:boundary_relation} for boundary integral terms into the discretization of the first multidomain equation, \cref{eq:multidomain_discretization_helper_multidomain1}, and using the derived flux values in \cref{eq:multidomain_discretization_helper2,eq:multidomain_discretization_helper3} leads in a first step to the following equation:
\begin{align*}
  \bfK_{\bfsigma_e + \bfsigma_i}\,\bfphi_{e} + \bfB_{\Gamma_M}\,\big(\bfq - \s{k=1}{N_\text{MU}} f_r^k\,\bfp^k\big) +  \s{k=1}{N_\text{MU}} f_r^k \big(\bfK_{\bfsigma_i^k}\,\bfV_m^k + \bfB_{\Gamma_M}\,\bfp^k\big) = 0.
\end{align*}

It can be seen that the terms involving $\bfp^k$ cancel out, such that we get:
\begin{align}\label{eq:multidomain_discretization_helper4}
    \bfK_{\bfsigma_e + \bfsigma_i}\,\bfphi_{e} + \s{k=1}{N_\text{MU}} f_r^k \bfK_{\bfsigma_i^k}\,\bfV_m^k = -\bfB_{\Gamma_M}\,\bfq.
\end{align}

If the multidomain description is used without body fat domain, the boundary condition in \cref{eq:multidomain_bc2} is used and the right-hand side in \cref{eq:multidomain_discretization_helper4} vanishes. If a body domain is considered, the right-hand side interacts with the body domain model, which is considered in the next section.

Adding boundary conditions to the discretization of the second multidomain equation proceeds using \cref{eq:multidomain_discretization_helper_multidomain2,eq:multidomain_discretization_helper_multidomain3}.
Carrying out the analog procedure to the first multidomain equation, we plug in \cref{eq:boundary_relation} to yield the matrix equation
\begin{align}\label{eq:multidomain_discretization1}
  \bfA\,\mat{\bfV_m^{k,(i+1)}\\ \bfphi_{e}^{(i+1)}} = \bfb
\end{align}
%
with system matrix $\bfA$ and right-hand side vector $\bfb$ given by
%
\begin{align}
 \bfA &= \matt{
    \dfrac{\theta}{A_m^k\,C_m^k}\bfK_{\bfsigma_i^k} -\dfrac{1}{\dt}\bfM & \quad
    \dfrac{\theta}{A_m^k\,C_m^k}\bfK_{\bfsigma_i^k},
  },\label{eq:multidomain_discretization2} \\[4mm]
  \bfb &= \Big( \dfrac{\theta-1}{A_m^k\,C_m^k}\bfK_{\bfsigma_i^k} - \dfrac{1}{\dt}\bfM\Big)\,\bfV_m^{k,(i)}
    + \dfrac{\theta - 1}{A_m^k\,C_m^k}\bfK_{\bfsigma_i^k}\,\bfphi_e^{(i)}\nonumber \\[4mm]
  & +\dfrac{\theta-1}{A_m^k\,C_m^k}\bfB_{\Gamma_M}\,\bfp^{k,(i)} - \dfrac{\theta-1}{A_m^k\,C_m^k}\bfB_{\Gamma_M}\,\bfp^{k,(i)}
  -\dfrac{\theta}{A_m^k\,C_m^k}\bfB_{\Gamma_M}\,\bfp^{k,(i+1)} + \dfrac{\theta}{A_m^k\,C_m^k}\bfB_{\Gamma_M}\,\bfp^{k,(i+1)}.\nonumber
\end{align}
Again, the boundary terms involving $\bfp^k$ vanish to yield the following expression for $\bfb$:%
\begin{align}\label{eq:multidomain_discretization3}
    \bfb &= \Big( \dfrac{\theta-1}{A_m^k\,C_m^k}\bfK_{\bfsigma_i^k} - \dfrac{1}{\dt}\bfM\Big)\,\bfV_m^{k,(i)}
      + \dfrac{\theta - 1}{A_m^k\,C_m^k}\bfK_{\bfsigma_i^k}\,\bfphi_e^{(i)}.
\end{align}
In summary, \cref{eq:multidomain_discretization_helper4} with $\bfq=\bfzero$ coupled with $N_\text{MU}$  instances of \cref{eq:multidomain_discretization1,eq:multidomain_discretization2,eq:multidomain_discretization3} comprises the discretization for the multidomain model without body domain. Definitions of the involved stiffness and mass matrices are given in \cref{eq:diffusion_matrices}.

\subsection{Discretization of the Multidomain Model for Surface EMG}\label{sec:discretization_body_domain}

To discretize the multidomain model with the electric potential $\phi_b$ in the body domain, we extend the formulation without body domain in \cref{sec:discretization_multidomain}.
The body domain adds the electric potential $\phi_b$ to the vector of unknowns, for which the system has to be solved. As before, we discretize the field using finite element ansatz functions and solve for the vector $\bfphi_b$ of degrees of freedom.

The model for $\phi_b$ is the Laplace equation given in \cref{eq:body} with homogeneous Neumann boundary conditions given in \cref{eq:body_domain_bc3}. According to \cref{eq:discretization_laplace}, the discretized equation is given by
\begin{align}\label{eq:discretized_body}
  \bfK_{\bfsigma_b}\,\bfphi_b = 0.
\end{align}

In addition, the value of the body potential $\phi_b$ is coupled to the extracellular potential $\phi_e$ in the muscle domain $\Omega_M$ via the coupling conditions on the boundary $\Gamma_M$ given in \cref{eq:body_domain_coupling}.

We write the discretized and coupled multidomain equations as a linear system of equations in generic block-matrix form:
\begin{align}\label{eq:discretized_multidomain_body}
  \left[
  \begin{array}{@{}c|c|c|c@{}}
    \bfA_{V_m,V_m}^k & \bfB_{V_m,\phi_e}^k & &\\[2mm]
    \bfB_{\phi_e,V_m}^k & \bfB_{\phi_e,\phi_e} & &\bfB_{\Gamma_M} \\[2mm] \hline
    &&\bfC_{\phi_b,\phi_b} & -\bfB_{\Gamma_M}\\[2mm]\hline
    & \bfI_{\Gamma_M,\phi_e} & -\bfI_{\Gamma_M,\phi_b} &\\[2mm]
  \end{array}
  \right]
  \left[
  \begin{array}{@{}c@{}}
    \bfV_{m}^{k,(i+1)}  \\[2mm]\hline
    \bfphi_{e}^{(i+1)} \\[2mm]\hline
    \bfphi_{b}^{(i+1)}  \\[2mm]\hline
    \bfq^{(i+1)}
  \end{array}\right]
  =
  \left[\begin{array}{@{}c@{}}
    \bfb_{V_m}^{k,(i)} \\[2mm]
    \bfzero\\\hline
    \bfzero\\\hline
    \bfzero
  \end{array}\right].
\end{align}

The vector of unknowns consists of the degrees of freedom in the finite element formulation at the next timestep $(i+1)$ of the transmembrane voltage $\bfV_m^{k,(i+1)}$, the extracellular potential $\bfphi_{e}^{(i+1)}$, the body potential $\bfphi_{b}^{(i+1)}$, and additionally the flux $\bfq^{(i+1)}$ over the shared boundary $\Gamma_M$ of the muscle and the body domain, which was defined in \cref{eq:definition_q}. For illustration purposes, only one compartment, $k=1$, for one MU, $N_\text{MU}=1$, is considered.

We refer to parts of the matrix in \cref{eq:discretized_multidomain_body} as block rows and block columns according to the given block-structure.

The first block row in the matrix equation is given by the discretized second multidomain equation. Following \cref{eq:multidomain_discretization2,eq:multidomain_discretization3}, the matrices and the right-hand side are given by
\begin{align*}
  &\bfA^k_{V_m,V_m} = \dfrac{\theta}{A_m^k\,C_m^k}\bfK_{\bfsigma_i^k} -\dfrac{1}{\dt}\bfM, \qquad
  \bfB^k_{V_m,\phi_e} = \dfrac{\theta}{A_m^k\,C_m^k}\bfK_{\bfsigma_i^k},\\[4mm]
  &\bfb_{V_m}^{k,(i)} = \Big( \dfrac{\theta-1}{A_m^k\,C_m^k}\bfK_{\bfsigma_i^k} - \dfrac{1}{\dt}\bfM\Big)\,\bfV_m^{k,(i)}
      + \dfrac{\theta - 1}{A_m^k\,C_m^k}\bfK_{\bfsigma_i^k}\,\bfphi_e^{(i)}.
\end{align*}
%

The second block row describes the first multidomain equation that was derived in \cref{eq:multidomain_discretization_helper4}. The flux term $\bfq$ has been brought to the left-hand side and is incorporated by the boundary matrix $\bfB_{\Gamma_M}$ defined in \cref{eq:definition_boundary_matrix}. The other matrices are formulated as follows:
%
\begin{align*}
  \bfB_{\phi_e,V_m}^k &= f_r^k \bfK_{\bfsigma_i^k}, &
  \bfB_{\phi_e,\phi_e} &= \bfK_{\bfsigma_e + \bfsigma_i}.
\end{align*}
%

The third block row is the formulation of the harmonic body potential $\phi_b$ and the matrix $\bfC_{\phi_b,\phi_b}$ equals the system matrix $\bfK_{\bfsigma_b}$ in \cref{eq:discretized_body}. The coupling condition on the flux $q$ in  \cref{eq:body_domain_bc2} is accounted for by including the boundary matrix $\bfB_{\Gamma_M}$ in the last column. The minus sign comes from the fact that the outward normal vector on $\Gamma_M$ as the boundary of $\Omega_B$ has the opposite direction to the normal vector on $\Gamma_M$ that is used for the models in the muscle domain $\Omega_M$. Using the helper variable $\bfq^{(i+1)}$, the second and third row of \cref{eq:discretized_multidomain_body} are coupled according to the prescribed condition in \cref{eq:body_domain_bc2}.

The other coupling condition, \cref{eq:body_domain_bc1}, is accounted for by the last block row in \cref{eq:discretized_body}. The degrees of freedom for the extracellular potential $\bfphi_e^{(i+1)}$ and the body potential $\bfphi_b^{(i+1)}$ have equal values on the boundary $\Gamma_M$. The matrices $\bfI_{\Gamma_M,\phi_e}$ and $\bfI_{\Gamma_M,\phi_b}$ are identity matrices that only have nonzero entries on the diagonal for the boundary degrees of freedom in the meshes of muscle domain and body domain, respectively.

Because the vector $\bfq^{(i+1)}$ is not an unknown in the system, the respective values in \cref{eq:discretized_multidomain_body} have to be eliminated.
As a result, we obtain the following system, which is formulated for a generic number $N_\text{MU}$ of MUs:
%
\begin{align}\label{eq:discretized_multidomain_body2}
  \left[\begin{array}{@{}ccc|c|c@{}}
    \bfA_{V_m,V_m}^1 &&& \bfB_{V_m,\phi_e}^1 &\\[2mm]
    &\ddots&&\vdots&\\[2mm]
    &&\bfA_{\phi_e,V_m}^{N_\text{MU}} & \bfB_{V_m,\phi_e}^{N_\text{MU}}&\\[2mm]
    \bfB_{\phi_e,V_m}^1 & \dots & \bfB_{\phi_e,V_m}^{N_\text{MU}} & \bfB_{\phi_e,\phi_e} & \bfD \\[2mm] \hline
    &&&\bfE & \tilde{\bfC}_{\phi_b,\phi_b}
  \end{array}\right]
  \left[\begin{array}{@{}c@{}}
    \bfV_{m}^{1,(i+1)}  \\[2mm]
    \vdots\\[2mm]
    \bfV_{m}^{N_\text{MU},(i+1)}\\[2mm]\hline
    \bfphi_{e}^{(i+1)} \\[2mm]\hline
    \tilde{\bfphi}_{b}^{(i+1)}
  \end{array}\right]
  =
  \left[\begin{array}{@{}c@{}}
    \bfb_{V_m}^{1,(i)} \\[2mm]
    \vdots \\[2mm]
    \bfb_{V_m}^{N_\text{MU},(i)}\\[2mm]
    \bfzero\\[2mm]\hline
    \bfzero
  \end{array}\right].
\end{align}

Formally, the elimination step is carried out by adding the equations of the third block row in \cref{eq:discretized_multidomain_body}, that correspond to the boundary degrees of freedom on $\Gamma_M$, to the corresponding equations of the same degrees of freedom in the second block row. This eliminates the last block column, which corresponds to $\bfq^{(i+1)}$. Next, the duplicate boundary degrees of freedom, that appear in both the $\Omega_M$ and $\Omega_B$ meshes, get unified. The corresponding matrix columns in the third block column are removed. To preserve the entries in the third block row, they are added in the sub matrix of block row three and block column two.

Now considering the updated matrix equation in \cref{eq:discretized_multidomain_body2}, all sub blocks are equal to \cref{eq:discretized_multidomain_body}, except for the former matrix $\bfC_{\phi_b,\phi_b}$ and the new matrices $\bfD$ and $\bfE$. The new matrix $\tilde{\bfC}_{\phi_b,\phi_b}$ is obtained from $\bfC_{\phi_b,\phi_b}$ by removing all rows and columns of boundary degrees of freedom. The removed entries are contained in the new matrices $\bfD$ and $\bfE$.

The size of the system matrix in \cref{eq:discretized_multidomain_body2} equals $a\times a$, where the number $a$ is composed of $N_\text{MU}+1$ times the number of degrees of freedom in the muscle mesh plus the number of degrees of freedom in the fat layer mesh without the boundary degrees of freedom on $\Gamma_M$. Accordingly, the vector $\tilde{\bfphi}_b^{(i+1)}$ is the same as $\bfphi_b^{(i+1)}$ except that it does not contain the boundary degrees of freedom, which are already included in $\bfphi_e^{(i+1)}$.

\Cref{eq:discretized_multidomain_body2} describes one iteration of the Crank-Nicolson scheme that is used to solve the multidomain model. This iteration is carried out alternatingly with the subcellular model according to the chosen operator splitting scheme.

The first $N_\text{MU}$ block rows in \cref{eq:discretized_multidomain_body2} contain the second multidomain equation for every MU. The second-to-last block row contains the first multidomain equation and the last block row  contains the body fat layer model.

Because of the implicit formulation, electric conduction in the intracellular and extracellular space and the body domain are bidirectionally coupled. Therefore, the model can be used to simulate the effects of natural activation in the muscle on EMG signals on the skin surface as well as the reverse effect of external stimulation on the surface on the electrophysiology.

\subsection{Discretization of the Fiber Based Electrophysiology Model}

% Nach zwei Abschnitten über das Multidomain-Modell wäre es hier glaub ich gut, noch einen Abschnitt über das fiber-based Modell einzufügen, mit entsprechender Überschrift. Der Abschnitt darf gern kurz sein und Gemeinsamkeiten und Unterschiede zu den vorausgehenden Abschnitten erklären. Eine kurze Zusammenfassung in generischer Matrix-Block-Schreibweise wie oben mit Erklärung uu den Blöcken wäre sehr hilfreich. Ich denke, man versteht dann auch die Parallelisierung und Implementierung besser, wenn man hier schonmal ganz klar die unabhängigen Blöcke für die Fasern gesehen hat.

The fiber based electrophysiology model consists of multiple independent 1D fiber domains, where the monodomain equation \cref{eq:monodomain} is solved. The transmembrane voltage $V_m$ is then mapped to a 3D mesh of the muscle domain and unidirectionally coupled to the first bidomain equation \cref{eq:bidomain1}. The first bidomain equation is solved for the extracellular potential $\phi_e$ and possibly the electric potential $\phi_b$ in the body fat domain, which corresponds to EMG signals on the skin surface.

The temporal discretization of the monodomain equation was described in \cref{sec:discretization_monodomain}. The diffusion term within the operator splitting requires a spatial discretization for which we use the finite element method. This 1D diffusion equation is given as
\begin{align}\label{eq:discretization_diffusion_term}
  \p{V_m}{t} = \dfrac{\sigma_\text{eff}}{A_m\,C_m} \p{V_m}{x}{2}.
\end{align}
It can be solved using a timestepping scheme such as the implicit Euler method to obtain time-discrete values $V_m^{(i)}, i=1,2,\dots$ for the transmembrane potential. The discretization leads to the matrix equation given in \cref{eq:diffusion_helper2} and to the variants presented in \cref{sec:mass_lumping} if mass lumping is used. In the stiffness and mass matrices, the anisotropic conduction tensor is replaced by the constant scalar prefactor $c := \sigma_\text{eff}/(A_m\,C_m)$ of the spatial second derivative in \cref{eq:discretization_diffusion_term}.

The first bidomain equation \cref{eq:bidomain1} is a 3D Poisson problem in terms of the unknown extracellular potential $\phi_e$. According to \cref{eq:discretization_laplace}, the finite element discretization is given by%
\begin{align*}
  \bfK_{\bfsigma_i + \bfsigma_e} \bfphi_e^{(i+1)} &= - \bfB_{\Gamma_f}\bff + \textbf{rhs},
\end{align*}
where the right-hand side $\textbf{rhs}$ of the Poisson problem is the transmembrane flow and is given by
\begin{align}\label{eq:static_bidomain_rhs}
  \textbf{rhs} &= -\bfK_{\bfsigma_i} \bfV_{m,3D}^{(i+1)}.
\end{align}
Here, $\bfphi_e^{(i+1)}$ and $\bfV_{m,3D}^{(i+1)}$ are the vectors of degrees of freedom on the 3D mesh for the extracellular potential $\phi_e$ and the membrane potential $V_m$ at timestep $(i+1)$. With the homogeneous Neumann boundary conditions for $V_m$ and $\phi_e$ given in \cref{eq:monodomain_bc}, the boundary term $\bfB_{\Gamma_f}$ vanishes.

In summary, the following matrix equations are solved for the fiber based electrophysiology model with $n$ fibers:
\begin{subequations}\label{eq:discretized_fibers}
  \begin{align}
    \left[\begin{array}{@{}ccc@{}}
      \bfA &&\\
      &\ddots&\\
      &&\bfA
    \end{array}\right]
    \left[\begin{array}{@{}c@{}}
      \bfV_{m}^{1,(i+1)}  \\
      \vdots\\
      \bfV_{m}^{n,(i+1)}
    \end{array}\right]
    &=
    \left[\begin{array}{@{}c@{}}
      \bfV_{m}^{1,(i)}  \\
      \vdots\\
      \bfV_{m}^{n,(i)}
    \end{array}\right],\label{eq:discretized_fibers_1} \\[4mm]
    \bfV_{m,3D}^{(i+1)} &= \bfP \left[\begin{array}{@{}c@{}}
      \bfV_{m}^{1,(i+1)} \label{eq:discretized_fibers_2} \\
      \vdots\\
      \bfV_{m}^{n,(i+1)}
    \end{array}\right],\\[4mm]
    \bfK_{\bfsigma_i + \bfsigma_e} \bfphi_e^{(i+1)} &= -\bfK_{\bfsigma_i} \bfV_{m,3D}^{(i+1)} \label{eq:discretized_fibers_3}
  \end{align}
\end{subequations}
with the system matrix $\bfA$ for a single fiber given according to \cref{eq:lumping_implicit_euler_2} by
\begin{align*}
  \bfA &= \bfI - \dt\,\bfM_c^{-1}\bfK_c.
\end{align*}
\Cref{eq:discretized_fibers_1} solves the diffusion part of the operator splitting in \cref{eq:monodomain_operator_splitting}. After the values $\bfV_{m}^{j,(i+1)}$ for the timestep $(i+1)$ are computed on the 1D fiber meshes, the homogenized vector $\bfV_{m,3D}^{(i+1)}$ in the 3D mesh of the muscle domain $\Omega_M$ is obtained by the prolongation operation $\bfP$ in \cref{eq:discretized_fibers_2}. The homogenized vector is used in the right-hand side of the bidomain model in \cref{eq:discretized_fibers_3}, which computes the discretized extracellular potential $\bfphi_e^{(i+1)}$.

\Cref{eq:discretized_fibers_3} can be extended by adding a body fat layer $\Omega_B$ and the corresponding model for the electric potential $\phi_b^{(i+1)}$. Then, the vector of unknowns contains the degrees of freedom for both $\phi_e^{(i+1)}$ and $\phi_b^{(i+1)}$. The stiffness matrix $\bfK_{\bfsigma_i + \bfsigma_e}$ is obtained by integrating over both meshes in $\Omega_M \cup \Omega_B$. Only in the elements of the finite element mesh for $\Omega_B$, the conduction tensors are redefined as $\bfsigma_i = \bfzero$ and $\bfsigma_e = \bfsigma_b$. This sets the right-hand side of \cref{eq:discretized_fibers_3} to zero in $\Omega_B$ and the solution $\phi_b$ in harmonic according to the model in \cref{eq:body}. The coupling conditions \cref{eq:body_domain_coupling} between $\phi_e$ and $\phi_b$ and the outer Neumann boundary conditions \cref{eq:body_domain_bc3} for $\phi_b$ are satisfied automatically by this approach.

The comparison between the discretized multidomain model in \cref{eq:discretized_multidomain_body2} with the discretized fiber based model in \cref{eq:discretized_fibers} reveals several differences. Whereas the multidomain description consists of a single coupled linear system for electric conduction in the intracellular, extracellular and body domains, the formulations are only unidirectionally coupled in the fiber based description. While the multidomain model always computes the EMG signals on the skin surface in every timestep, the corresponding model in the fiber based description can be solved with larger timestep widths, using subcycling for the action potential propagation model.

As can be seen in \cref{eq:discretized_fibers_1}, the system matrix is decoupled and contains independent problems for every fiber. This is an advantage compared to the multidomain model, where a system describing the whole muscle domain has to be solved. On the downside, separate representations of the transmembrane voltage $V_m$ exist in the fiber based description. The representation in the 3D mesh has to be computed by interpolation from the representation on the fibers. The multidomain description has a single vector of degrees of freedom for $V_m$ with fewer entries than in the fiber-based description.

\subsection{Summary of Domains and Meshes}

Various finite element meshes occur in the formulation of the multi-scale model.
If the fiber based description is used, the description requires finite element meshes for the 1D fiber domains $\Omega_f^j$ for ${j=1,\dots,n}$. Further meshes are needed for the 3D muscle domain $\Omega_M$ and for the 3D body domain $\Omega_B$. The meshes for $\Omega_M$ and $\Omega_B$ share nodes on their common boundary $\Gamma_M$. The fiber meshes are embedded in the muscle domain. Their nodes do not necessarily have to coincide with the nodes of the muscle mesh.

The subcellular model is solved at locations $\Omega_s^i$ for $i=1,\dots,m$. These locations are the nodes of the fiber meshes for the fiber based description and the nodes of the muscle mesh for the multidomain description. We therefore have the inclusion $\Omega_s^i \subset \Omega_f^j \subset \Omega_M$.

For the solid mechanics model, the unified 3D domain $\Omega = \Omega_M \cup \Omega_B$ is used. The mesh for the continuum mechanics formulation can be different from the meshes used for the electrophysiology model. In fact, the continuum mechanics mesh has special requirements in order to yield a consistent formulation. Our implementation uses two overlaid meshes of quadratic and linear hexahedral elements for displacements and the hydrostatic pressure.

Often, the required accuracy of the electrophysiology model is higher than for the continuum mechanics model, such that differently resolved meshes can be used. To facilitate data mapping, the nodes of the mechanics mesh should be chosen as subset of the nodes of the electrophysiology meshes.

%We spatially discretize the variables in \cref{eq:multi-domain1,eq:multi-domain2,eq:body,eq:bc1,eq:bc2,eq:bc3,eq:monodomain,eq:bidomain,eq:subcellular} using the finite element method with linear ansatz functions. The subcellular points $\Omega_s^i$ are placed at the nodes of the muscle domain mesh $\Omega_M$ for the multi-domain model and at the nodes of the fiber domain meshes $\Omega_f^j$ for the fiber based model. The fibers $\Omega_f^j$ are embedded in the muscle domain $\Omega_M$, however, the nodes of their meshes do not necessarily coincide. The nodes on the internal boundary $\Gamma_M$ between $\Omega_M$ and $\Omega_B$ are shared between the meshes of $\Omega_M$ and $\Omega_B$.

% ---


\lab{Algorithms}{Scientific Visualization}{Scientific Visualization}

\objective{Go over techniques for making publication ready figures and making other pretty things.}

Python and SciPy have no builtin plotting capbilities.  These capabilites are provided by Matplotlib.  In this lab we will be focusing on producing publication quality figures in Matplotlib using Python and SciPy.  Thus far we have produced many different plots for several applications.  Here we will focus on different ways of presenting those plots so that they are more informative and aesthetically pleasing.

\section{The \li{pyplot.plot} Command}
Several of the past labs have required the use of the \li{pyplot.plot} command.  While what we have done so far as been fairly straightforward as far as plotting is concerned, a quick glance at the Matplotlib documentation for \li{pyplot.plot} reveals a plethora of options that can be set to adjust the final plot.

How we use the \li{pyplot.plot} command will vary with our application.  Plotting data requires a very different presentation approach than graphing straight lines.  In the sequel we examine several different options.

\begin{table}[h!]
\begin{center}
	\begin{tabular}{|c|c|}

	\hline

	Usage & Property Options\\

	\hline

	\li{color} & \li{yellow, green, blue} etc., or user defined colors \\& given by $[r g b]$, a three element red, green, \\& blue vector.  See documentation for full list. \\

	\hline

	\li{linestyle} & \li{-, --, :, -., 'None'} \\

	\hline

	\li{linewidth} & Width in points.  $1$ point is $\frac{1}{72}$ inch.  Default is $1.0$ points. \\

	\hline

	\li{marker} & \li{+, *, x, .} etc. See documentation for full list. \\

	\hline
	\end{tabular}
\end{center}
\caption{Common Plot commands in MATLAB}
\end{table}

For example, if we wanted to plot $f(x) = x^2$, we don't need to extend our typical plotting knowledge much, except maybe we like green better than blue:

\begin{lstlisting}[style=python]
: import matplotlib.pyplot as plt
: import scipy as sp
: x = sp.linspace(-1,5,50)
: f = lambda x: x**2
: plt.plot(x, map(f, x), color='green')
: plt.show()
\end{lstlisting}

See figure 37.1.

\begin{figure}
\begin{center}
\includegraphics[scale=0.5]{./FiguresMAT/plot1}
\caption{Graph of $f(x) = x^2$}
\end{center}
\end{figure}

However, if we wanted to plot the data drawn from a certain distribution, we would want to use the plot command in a very different way.  For example:

%Explain the code?
\begin{lstlisting}[style=python]
>> X = random('Normal',0,1,[100,1]);
>> plot(X,'marker','*','LineStyle','none');
\end{lstlisting}

See figure 37.2.

\begin{figure}
\begin{center}
\includegraphics[scale=0.5]{./FiguresMAT/plot3}
\caption{100 data points pulled from a normal distribution}
\end{center}
\end{figure}

The \li{hold} command will also be useful for when plotting more than one graph on a single plot.  For example, if we wanted to graph $f(x) = \sin{x}$ and it's derivative $f'(x) = \cos{x}$, we would do the following:

\begin{lstlisting}[style=python]
: f = lambda x: sp.sin(x)
: df = lambda x: sp.cos(x)
: x = sp.linspace(0,2*sp.pi, 50)
: plt.hold(True)
: plot(x, map(f, x), color='blue')
: plot(x, map(df, x), color='green')
: plt.hold(False)
: plt.show()
\end{lstlisting}

See figure 37.3.

\begin{figure}
\begin{center}
\includegraphics[scale=0.5]{./FiguresMAT/plot2}
\caption{$f(x) = \sin{x}$ and $f'(x) = \cos{x}$. Note the color choice makes for better presentation.}
\end{center}
\end{figure}

\begin{problem}  Using the MATLAB documentation and the aforementioned commands, do the following:
\begin{itemize}
\item Generate 25 data points from a distribution of your choosing.  Plot the data points and the least squares regression line on the same plot.  Use different colors and symbols for the data and the regression line.
\item
\end{itemize}
\end{problem}

\subsection{Changing the Plot window}

Within in MATLAB we have several options for changing the \li{plot} window.  This includes adding captions and other labels, legends, and altering the axes (among other things).  For example, we plot 3 trajectories of a projectile shot from a cannon at 30 meters per second.

\begin{lstlisting}[style=python]
: g = 9.8
: v = 30.0
: t = linspace(0,6,100)
:
: h1 = lambda t: v*sp.math.sin(sp.pi/6)*t-0.5*g*t**2
: h2 = lambda t: v*sp.math.sin(sp.pi/4)*t-0.5*g*t**2
: h3 = lambda t: v*sp.math.sin(sp.pi/3)*t-0.5*g*t**2
:
: plt.hold(True)
: plt.plot(v*sp.math.cos(sp.pi/6)*t, map(h1, t), color='green')
: plt.plot(v*sp.math.cos(sp.pi/4)*t, map(h2, t), color='blue')
: plt.plot(v*sp.math.cos(sp.pi/3)*t, map(h3, t), color='red')
: plt.axis([0, 100, 0, 40])
: plt.xlabel('Distance')
: plt.ylabel('Height')
: plt.hold(False)
: plt.show()
\end{lstlisting}

See figure 37.4

\begin{figure}
\begin{center}
\includegraphics[scale=0.5]{./FiguresMAT/plot4}
\caption{Trajectory of a projectile fired at three different angles.}
\end{center}
\end{figure}

\begin{problem}
\end{problem}

\section{The \li{subplot} command}
Often times we want to include two or more plots in a single figure.  This is done using the \li{subplot} command.  This command will break the plot window into an $m\times n$ matrix of plots.  The subplot windows are numbered starting at $1$ in the top left cell and then ascending from left to right, top to bottom.  For example, instead of using different colors on the previous example, we could do 3 separate plots in a single window.  We could also include a graph of, for example, vertical velocity vs. time, which is informative but unable to be included in the same plot since it requires different units.  Let's see how this is done.

\begin{lstlisting}[style=python]
: g = 9.8
: v = 30.0
: t = linspace(0,6,100)
:
: h1 = lambda t: v*sp.math.sin(sp.pi/6)*t-0.5*g*t**2
: h2 = lambda t: v*sp.math.sin(sp.pi/4)*t-0.5*g*t**2
: h3 = lambda t: v*sp.math.sin(sp.pi/3)*t-0.5*g*t**2
:
: plt.subplot(221)
: plt.plot(v*cos(pi/6)*t,h1(t),'color','green')
: plt.axis([0, 100, 0, 40])
: plt.xlabel('Distance')
: plt.ylabel('Height')
: plt.title('30 degrees')
:
: plt.subplot(222)
: plt.plot(t, v*sin(pi/6) - g*t,'color','green')
: plt.xlabel('Time')
: plt.ylabel('Vertical Velocity')
:
: plt.subplot(223)
: plt.plot(v*cos(pi/4)*t,h2(t),'color','blue')
: plt.axis([0, 100, 0, 40])
: plt.xlabel('Distance')
: plt.ylabel('Height')
: plt.title('45 degrees')
:
: plt.subplot(224)
: plt.plot(t, v*sin(pi/4) - g*t, 'color','blue')
: plt.xlabel('Time')
: plt.ylabel('Vertical Velocity')
\end{lstlisting}

See figure 37.5.

\begin{figure}
\begin{center}
%\includegraphics[scale=0.5]{./Figures/plot5}
\caption{Trajectory of a projectile fired at two different angles and the projectiles corresponding vertical velocity at time $t$.}
\end{center}
\end{figure}

\section{Time graphics?}


\section{Moment Generating Functions}
\subsection{Moment Generating Function of One Random Variable}
\begin{definition}
    Let $X$ have density $f$, then the moment generating function (MGF) is defined by
    $$m(\theta)=\mathbb E[e^{\theta X}]=\int_{-\infty}^\infty e^{\theta x}f(x)\,\mathrm dx$$
    wherever it is finite.
\end{definition}
Note that $m(0)=1$.
\begin{theorem}
    The MGF uniquely determines the distribution of random variable provided it is defined on some open interval around $0$.
\end{theorem}
\begin{proof}
    Omitted.
\end{proof}
\begin{theorem}
    Suppose the MGF is defined for some open interval around $0$, then $m^{(r)}(0)=\mathbb E[X^r]$.
\end{theorem}
\begin{proof}
    Omitted as well.
\end{proof}
\begin{example}
    1. Gamma distribution $X\sim\Gamma(n,\lambda)$ for $n\in\mathbb N,\lambda\ge 0$.
    $$f(x)=e^{-\lambda x}\lambda^n\frac{x^{n-1}}{(n-1)!}$$
    defined for $x\ge 0$.
    For $n=1$, we get $\operatorname{Exp}(\lambda)$.
    One can show by reduction formula that this is indeed a density.\\
    In this case, we have
    \begin{align*}
        m(\theta)&=\mathbb E[e^{\theta X}]\\
        &=\int_0^\infty e^{-(\lambda-\theta)x}\frac{\lambda^n}{(\lambda-\theta)^n}(\lambda-\theta)^n\frac{x^{n-1}}{(n-1)!}\,\mathrm dx\\
        &=\left(\frac{\lambda}{\lambda-\theta} \right)^n
    \end{align*}
    taking $n=1$ then gives the MGF of the exponential.
    Suppose now that $X_1,\cdots,X_n$ are independent, then the MGF of the sum
    $$m(\theta)=\mathbb E[e^{\theta(X_1+\cdots+X_n)}]=\mathbb E[e^{\theta X_1}]\cdots\mathbb E[e^{\theta X_n}]$$
    So if $X\sim\Gamma(n,\lambda),Y\sim\Gamma(m,\lambda)$ are independent, then
    $$\mathbb E[e^{\theta(X+Y)}]=\left(\frac{\lambda}{\lambda-\theta} \right)^n\left(\frac{\lambda}{\lambda-\theta} \right)^m=\left(\frac{\lambda}{\lambda-\theta} \right)^{m+n}\sim\Gamma(n+m,\lambda)$$
    In particular, if $X_1,\ldots,X_n$ are i.i.d. $\operatorname{Exp}(\lambda)$, then $X_1+\cdots+X_n\sim\Gamma(n,\lambda)$.\\
    2. Suppose $X\sim\mathcal N(\mu,\sigma^2)$, then the MGF of $X$ is
    \begin{align*}
        m(\theta)&=\mathbb E[e^{\theta X}]\\
        &=\int_{-\infty}^\infty e^{\theta x}\frac{1}{\sqrt{2\pi \sigma^2}}e^{-(x-\mu)^2/(2\sigma^2)}\,\mathrm dx\\
        &=e^{\mu\theta+\theta^2\sigma^2/2}
    \end{align*}
    After some completing square and calculation.
    Now if $X\sim\mathcal N(\mu,\sigma^2)$, then we know that $aX+b\sim\mathcal N(a\mu+b,a^2\sigma^2)$.
    We can prove this again using MGF, indeed,
    $$\mathbb E[e^{\theta(aX+b)}]=e^{\theta b}e^{a\theta\mu+(a\theta)^2\sigma^2/2}=e^{\theta(a\mu +b)+\theta^2(a^2\sigma^2)/2}$$
    So $aX+b\sim\mathcal N(a\mu+b,a^2\sigma^2)$.
    Now let $Y=\mathcal N(\nu,\tau^2)$ independent of $X$, then by using the same trick (MGF), we get $\mathbb E[e^{\theta(X+Y)}]=e^{\theta(\mu+\nu)+\theta^2(\sigma^2+\tau^2)/2}$ so $X+Y\sim\mathcal N(\mu+\nu,\sigma^2+\tau^2)$.\\
    3. (non-example) Cauchy's Distribition is obtained by $f(x)=1/(\pi(1+x^2))$ for $x\in\mathbb R$.
    Then we can get $m(\theta)=\infty$ for any $\theta\neq 0$, so if $X\sim f$ then $X,2X,\ldots$ all have the same MGF but not the same distribution.
    So the assumption on $m(\theta)$ being finite on an open interval is essential.
\end{example}
\subsection{Multivariate Moment generating Function}
\begin{definition}
    Let $X=(X_1,\ldots,X_n)$ be a random variable in $\mathbb R^n$, then the MGF of $X$ is defined to be a function $m:\mathbb R^n\to\mathbb R$ with
    $$m(\theta)=\mathbb E[e^{\theta^\top X}]=\mathbb E[e^{\theta_1X_1+\cdots+\theta_nX_n}]$$
    where $\theta=(\theta_1,\ldots,\theta_n)^\top$.
\end{definition}
Provided that $m(\theta)$ is finite for a certain range (which is out of the scope of this course) of values of $\theta$, it uniquely characterizes the distribution of $X$.
Also
$$\left.\frac{\partial^rm}{\partial\theta_i^r}\right|_{\theta=0}=\mathbb E[X_i^r],\left.\frac{\partial^{r+s}m}{\partial\theta_i^r\partial\theta_j^s}\right|_{\theta=0}=\mathbb E[X_i^rX_j^s]$$
Also $X_1,\ldots,X_n$ are independent iff
$$m(\theta)=\prod_{i=1}^n\mathbb E[e^{\theta_iX_i}]$$
\begin{definition}
    Let $(X_n,n\in\mathbb N)$ be a sequence of random variables, and let $X$ be a random variable.
    We say $X_n\to X$ in distribution if
    $$F_{X_n}(x)\to F_X(x),F_{X_n}(x)=\mathbb P(X_n\le x),F_X(x)=\mathbb P(X\le x)$$
    For each $x\in\mathbb R$ such that $F_X$ is continuous at $x$.
\end{definition}
\begin{theorem}[Continuity Theorem for MGFs]
    Let $X$ be a random variable with MGF $m$ and $m(\theta)<\infty$ for some $\theta\neq 0$.
    Suppose we have
    $$m_n(\theta)=\mathbb E[e^{\theta X_n}]\to m(\theta),\forall\theta\in\mathbb R$$
    Then $X_n\to X$ in distribution.
\end{theorem}
\begin{proof}
    Omitted.
\end{proof}


\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

% Required to call the command \operatorname{name} and \DeclareMathOperator{command}{definition}
\usepackage{amsmath}

% Required to render the clickable url
\usepackage[hyphens]{url}
\usepackage{hyperref}

% Declare a new math function
\DeclareMathOperator{\abc}{abc}

\begin{document}

\section*{Existing functions}

\LaTeX{} comes with a lot of built-in mathematical functions, such as $\cos x$, $\ln x$ or $\max x$.
A complete list of symbols can be found in the \emph{The Comprehensive LATEX Symbol List} hosted in the \emph{Best Practices} part of this repository (\url{https://github.com/ZenLulz/LatexCompendium/blob/master/best-practices/the-comprehensive-latex-symbol-list.pdf}).

\section*{Custom functions}
\subsection*{The command \emph{operatorname}}

The following formula uses a custom function called \emph{abc}.

\[\operatorname{abc} x\]

\subsection*{The command \emph{DeclareMathOperator}}

This command enables to define math functions or operators in the preamble so they can be reused everywhere.

\[\abc x\]

\subsection*{The command \emph{mathrm}}

Many people use the command \emph{mathrm}, nevertheless this leads to space issue, as illustrated below.

\[\mathrm{abc}x\]

\end{document}


%!TEX root = ../notes.tex
\section{March 18, 2022}
\subsection{Elliptic Curves over Finite Fields \emph{continued}}
\recall from last class, we had our toy example
\begin{example*}
    We take elliptic curve
    \[y^2 = x^3 + x + 2\quad \text{ over }\ZZ/5\ZZ.\]
    with group law
    \[\begin{array}{c|cccc}
            +     & \cO   & (1,2) & (4,0) & (1,3) \\ \hline
            \cO   & \cO   & (1,2) & (4,0) & (1,3) \\
            (1,2) & (1,2) & (4,0) & (1,3) & \cO   \\
            (4,0) & (4,0) & (1,3) & \cO   & (1,2) \\
            (1,3) & (1,3) & \cO   & (1,2) & (4,0)
        \end{array}\]
\end{example*}
We define some more useful functions:
\begin{lstlisting}
def minus(P, p):
    if P == O:
        return O
    else:
        x, y = P
        return (x, (-y) % p)
\end{lstlisting}
What about multiplication? We can repeatedly add:
\begin{lstlisting}
def multiply(P, n, a, p):
    S = O
    for _ in range(n):
        S = add(P, S, a, p)
    return S
\end{lstlisting}
but this might be very slow (it does $n$ iterations). We can do something similar to fast powering for exponentiation, except we repeatedly double our point:
\begin{lstlisting}
def multiply(P, n, a, p):
    S = 0
    while n != 0:
        if n % 2 == 1:
            S = add(S, P, a, p)
        n = n // 2
        P = add(P, P, a, p)
    return S
\end{lstlisting}

\begin{ques*}
    What is the order of $E(\FF_p)$? That is, how many points are there on the elliptic curve?
\end{ques*}
Let's say we take $x, y, \dots$. We want to solve
\[y^2\overset{?}{=}x^3 + ax + b\]
There are $p^2$ different $(x, y)$. The probability that this equality holds is \emph{like} $\frac{1}{p}$. So there are about $p^2\cdot \frac{1}{p} + 1\approx p + 1$ (added one for the point $\cO$) elements in $E(\FF_p)$.

\begin{theorem}
    \[|(p+1) - \#E(\FF_p)| \leq 2\sqrt{p}.\] That is, the difference between our estimate $p+1$ and the actual number of points in $E(\FF_p)$ is bounded by $2\sqrt{p}$.
\end{theorem}
\begin{proof}
    \emph{(Beyond the scope of this class.)}
\end{proof}

\begin{remark}
    ~\begin{enumerate}
        \item We note that this number can be efficiently computed (in polynomial time in the digits of $p$). \emph{(Again, beyond the scope of this class.)}
        \item We call this number $|(p+1) - \#E(\FF_p)|$ the \ul{trace of Frobenius}. \emph{(You guessed it: again, beyond the scope of this class.)}
    \end{enumerate}
\end{remark}

Just for funsies, we can compute the \emph{trace of Frobenius}\footnote{Using \emph{Sage}.}:
\begin{lstlisting}
p = next_prime(92834712736591432)
E = EllipticCurve([34123498, GF(p)(2349182347)])
E.trace_of_frobenius()
\end{lstlisting}

\subsection{Elliptic Diffe-Hellman Key Exchange}
\subsubsection{Elliptic Discrete Log Problem}
This is based on the \emph{Elliptic Discrete Log Problem}:
Given $E$ an elliptic curve over $\FF_p$ with $P$ a point on $E$. We take $P, n$ and calculate
\[n\cdot P\]
The \ul{elliptic curve discrete log problem} is given point $n\cdot P$, computing $n$.

The best known algorithm for \emph{ECDLP} is Babystep-Giantstep, which runs in $\mathcal{O}(\sqrt{p})$ and $\mathcal{O}(\sqrt{p})$ memory. There is \emph{no} analog of index calculus. This could be good or bad... This could mean a lack of knowledge about elliptic curves. We could also create greater security at smaller key sizes.

\subsubsection{Sharing Secrets}
\emph{Public information:} we have some $p$ prime and $E$ an elliptic curve over $\FF_p$. We have $P$ a point in the elliptic curve $E$.

Alice and Bob do the following:

\begin{enumerate}
    \item Alice and Bob each generate $a$ and $b$. Alice computes $a\cdot P$ and Bob computes $b\cdot P$. This is shared to each other (and public).
    \item Alice now computes $a\cdot (b\cdot P)$ and Bob computes $b\cdot (a\cdot P)$. These are all equal to $(a\cdot b)\cdot P$ which is a shared secret.
\end{enumerate}


\section{Planes}
In this section we will discuss how to represent planes, and many ways we can use them. We will see that they play a very similar role to the role lines play in 2D, and many of the operations we defined in section~\ref{s:lines} have a direct equivalent here. Since the explanations are nearly identical, we make them a bit shorter here; please refer to section~\ref{s:lines} if you want more details.

\subsection{Defining planes}
Planes are sets of points $(x,y,z)$ which obey an equation of the form $ax+by+cz=d$. Here, $a,b,c$ determine the orientation of the plane, while $d$ determines its position relative to the origin.

\centerFig{plane0}

Vector $\vv{n}=(a,b,c)$ is perpendicular to the plane and is called a \emph{normal} of the plane. The equation can be rewritten as $\vv{n}\cdot(x,y,z) = d$: that is, the plane is formed by all points whose dot product with $\vv{n}$ is equal to constant $d$. This makes sense, because we know that the dot product doesn't change when one vector (here, the point $(x,y,z)$) moves perpendicularly to the other (here, the normal $\vv{n}$).

\centerFig{plane1}

Here are some other ways to define a plane, and how to find $\vv{n}$ and $d$ in each case:
\begin{itemize}
\item if we know the normal $\vv{n}$ and a point $P$ belonging to the plane: we can find $d$ as $\vv{n}\cdot P$;
\item if we know a point $P$ and two (non-parallel) vectors $\vv{v}$ and $\vv{w}$ that are parallel to the plane: we can define $\vv{n}=\crossv{v}{w}$ then find $d$ as above;
\item if we know 3 non-collinear points $P,Q,R$ on the plane: we first find two vectors $\vv{v}=\vv{PQ}$ and $\vv{w} = \vv{PR}$ that are parallel from the plane, then find $\vv{n}$ and $d$ as above.
\end{itemize}

We implement this with the following structure and constructors:
\begin{lstlisting}
struct plane {
    p3 n; T d;
    // From normal n and offset d
    plane(p3 n, T d) : n(n), d(d) {}
    // From normal n and point P
    plane(p3 n, p3 p) : n(n), d(n|p) {}
    // From three non-collinear points P,Q,R
    plane(p3 p, p3 q, p3 r) : plane((q-p)*(r-p), p) {}

    // Will be defined later:
    // - these work with T = int
    T side(p3 p);
    double dist(p3 p);
    plane translate(p3 t);
    // - these require T = double
    plane shiftUp(double dist);
    p3 proj(p3 p);
    p3 refl(p3 p);
};
\end{lstlisting}

\subsection{Side and distance}
The first thing we are interested for a plane $\Pi$ in is the value of $ax+by+cz-d$. If it's zero, the point is on $\Pi$, and otherwise it tells us which side of $\Pi$ the point lies. We name it \lstinline|side()|, like its 2D line equivalent:
\begin{lstlisting}
T side(p3 p) {return (n|p)-d;}
\end{lstlisting}
which we will denote $\side_{\Pi}(P)$.

$\side_{\Pi}(P)$ is positive if $P$ is on the side of $\Pi$ pointed by $\vv{n}$, and negative for the other side. In fact, for given points $P,Q,R,S$, \lstinline|plane(p,q,r).side(s)| gives the same value as \lstinline|orient(p,q,r,s)|.

\centerFig{plane2}

And just like the \lstinline|side()| for 2D lines, we can get the distance from it, if we compensate for the norm of $\vv{n}$:
\begin{lstlisting}
double dist(p3 p) {return abs(side(p))/abs(n);}
\end{lstlisting}

\subsection{Translating a plane}
If we translate a plane by a vector $\vv{t}$, the normal $\vv{n}$ remains unchanged, but the offset $d$ changes.

\centerFig{plane3}

To find the new value $d'$, we use the same argument as for 2D lines: suppose point $P$ is on the old plane, that is, $\vv{n}\cdot P = d$. Then $P+\vv{t}$ should be in the new plane:
\[d' = \vv{n}\cdot(P+\vv{t}) = \vv{n}\cdot P + \dotv{n}{t} = d + \dotv{n}{t}\]

\begin{lstlisting}
plane translate(p3 t) {return {n, d+(n|t)};}
\end{lstlisting}

And if we want to shift perpendicularly (in the direction of $\vv{n}$) by some distance $\delta$, then $\dotv{n}{t}$ becomes $\delta\normv{n}$, which gives the following code:
\begin{lstlisting}
plane shiftUp(double dist) {return {n, d + dist*abs(n)};}
\end{lstlisting}

\subsection{Orthogonal projection and reflection}
The orthogonal projection of a point $P$ on a plane $\Pi$ is the point on $\Pi$ that is closest to $P$. The reflection of point $P$ by plane $\Pi$ is the point on the other side of $\Pi$ that is at the same distance and has the same orthogonal projection.

\centerFig{plane4}

We use a similar reasoning as in section~\ref{ss:proj-refl}. Clearly, to go from $P$ to its projection on $\Pi$, we need to move perpendicularly to $\Pi$, that is, we need to move by $k\vv{n}$ for some $k$, so that the resulting point $P+k\vv{n}$ is on $\Pi$.

From this we find $k$:
\begin{align*}
\vv{n}\cdot(P+k\vv{n}) = d
&\Leftrightarrow \vv{n}\cdot P + k(\dotv{n}{n}) = d \\
&\Leftrightarrow k\normv{n}^2 = -(\vv{n}\cdot P - d) \\
&\Leftrightarrow k = -\frac{\side_{\Pi}(P)}{\normv{n}^2}
\end{align*}

And to find the reflection, we need to move $P$ twice as far to $P+2k\vv{n}$, so we get the following implementation for both:
\begin{lstlisting}
p3 proj(p3 p) {return p - n*side(p)/sq(n);}
p3 refl(p3 p) {return p - n*2*side(p)/sq(n);}
\end{lstlisting}

\subsection{Coordinate system based on a plane}
When we have a plane $\Pi$, sometimes we will want to know what are the coordinates of a point in $\Pi$. That is, suppose we have a few points that we know are coplanar, and we want to use some 2D algorithm on them. How do we get the 2D coordinates of those points?

To do this, we need to chose an origin point $O$ on $\Pi$ and two vectors $\vv{d_x}$ and $\vv{d_y}$ indicating the desired $x$ and $y$ directions. Let's first assume $\vv{d_x}$ and $\vv{d_y}$ are perpendicular and their norms are 1. Then, to find the $x$- and $y$-coordinate of a point $P$ on $\Pi$, we simply need to compute
\begin{align*}
x &= \dotv{OP}{d_x} \\
y &= \dotv{OP}{d_y}
\end{align*}
and if we also have vector $\vv{d_z} = \crossv{d_x}{d_y}$ perpendicular to the first two, we can find the ``height'' of $P$ respective to $\Pi$:
\[z = \dotv{OP}{d_z}\]

\centerFig{plane5}

If we have three non-collinear points $P,Q,R$ that form plane $\Pi$, how can we choose $O$, $\vv{d_x}$, $\vv{d_y}$ and $\vv{d_z}$?
\begin{enumerate}
\item First, we choose the origin $O$ to be $P$.
\item Then, we choose $\vv{d_x}$ to be $\vv{PQ}$, and scale it to have a norm of 1.
\item Next, we compute $\vv{d_z} = \vv{d_x}\times\vv{PR}$, and scale it to have a norm of 1. It is perpendicular to $\Pi$ because it is perpendicular to two non-parallel vectors in it.
\item Finally, we find $\vv{d_y}$ as $\crossv{d_z}{d_x}$.
\end{enumerate}

This gives the following code. Method \lstinline|pos2d()| gives the position on the plane as a 2D point, and method \lstinline|pos3d()| gives the position and height as a 3D point (so in a way this structure represents a change of coordinate system).
\begin{lstlisting}
struct coords {
    p3 o, dx, dy, dz;
    // From three points P,Q,R on the plane:
    // build an orthonormal 3D basis
    coords(p3 p, p3 q, p3 r) : o(p) {
        dx = unit(q-p);
        dz = unit(dx*(r-p));
        dy = dz*dx;
    }
    // From four points P,Q,R,S:
    // take directions PQ, PR, PS as is
    coords(p3 p, p3 q, p3 r, p3 s) :
        o(p), dx(q-p), dy(r-p), dz(s-p) {}

    pt pos2d(p3 p) {return {(p-o)|dx, (p-o)|dy};}
    p3 pos3d(p3 p) {return {(p-o)|dx, (p-o)|dy, (p-o)|dz};}
};
\end{lstlisting}

\begin{mathy}
The second constructor allows us specify points $P,Q,R,S$ and choose $\vv{d_x}=\vv{PQ}$, $\vv{d_y}=\vv{PR}$ and $\vv{d_z}=\vv{PS}$ directly. This can be useful if we don't care that the 2D coordinate system $(\vv{d_x},\vv{d_y})$ is not orthonormal (perpendicular and of norm 1), because it allows us to keep using integer coordinates.

If $\vv{d_x}$ and $\vv{d_y}$ are not perpendicular or do not have a norm of 1, the computed distances and angles will not be correct. But if we are only interested in the relative positions of lines and points, and finding the intersections of lines or segments, then everything works fine. Computing the 2D convex hull of a set of points is an example of such a problem, because it only requires that the sign of $\orient()$ is correct.
\end{mathy}


\documentclass[]{BasiliskReportMemo}

\newcommand{\submiterInstitute}{Autonomous Vehicle Simulation (AVS) Laboratory,\\ University of Colorado}

\newcommand{\ModuleName}{RWJitterBackSub}
\newcommand{\subject}{Derivation of Reaction Wheel Jitter Back-Substitution}
\newcommand{\status}{Draft}
\newcommand{\preparer}{J. Alcorn, C. Allard}
\newcommand{\summary}{This document derives the back substitution method for reaction wheel jitter to conform with Basilisk's dynamics architecture.}

\begin{document}

\makeCover

%
%	enter the revision documentation here
%	to add more lines, copy the table entry and the \hline, and paste after the current entry.
%
\pagestyle{empty}
{\renewcommand{\arraystretch}{1.1}
\noindent
\begin{longtable}{|p{0.5in}|p{4.0in}|p{1.64in}|}
\hline
{\bfseries Rev}: & {\bfseries Change Description} & {\bfseries By} \\
\hline
Draft & Initial draft. & J. Alcorn, C. Allard \\
\hline

\end{longtable}
}

\newpage
\setcounter{page}{1}
\pagestyle{fancy}

\tableofcontents
~\\ \hrule ~\\

\section{Introduction}
The goal is to manipulate the reaction wheel equations of motion to conform to the following form
\begin{equation}
\begin{bmatrix}
[A] & [B]\\
[C] & [D]
\end{bmatrix} \begin{bmatrix}
\ddot{\bm r}_{B/N}\\
\dot{\bm\omega}_{\cal B/N}
\end{bmatrix} = \begin{bmatrix}
\bm v_{\text{trans}}\\
\bm v_{\text{rot}}
\end{bmatrix}
\end{equation}
Solving the system-of-equations by
\begin{equation}
\dot{\bm\omega}_{\cal B/N} = \Big([D] - [C]][A]^{-1}[B]\Big)^{-1}(\bm v_{\text{rot}} - [C][A]^{-1}\bm v_{\text{trans}})
\end{equation}
\begin{equation}
\ddot{\bm r}_{B/N} = [A]^{-1} (\bm v_{\text{trans}} - [B]\dot{\bm\omega}_{\cal B/N})
\end{equation}

\section{Balanced Reaction Wheel Back-Substitution}

\subsection{Equations of Motion}
The translational equation of motion is not coupled with $\dot{\Omega}$ as seen in the equation below.
\begin{equation}
m_{\text{sc}} [I_{3 \times 3}]\ddot{\bm r}_{B/N}
-m_{\text{sc}} [\tilde{\bm{c}}] \dot{\bm\omega}_{\cal B/N}
= \bm F_{\text{ext}}- 2 m_{\text{sc}} [\tilde{\bm\omega}_{\cal B/N}] \bm c'
-m_{\text{sc}} [\tilde{\bm\omega}_{\cal B/N}][\tilde{\bm\omega}_{\cal B/N}]\bm{c}
\label{eq:Rbddot35}
\end{equation}
The rotational equation of motion includes $\dot{\Omega}$ terms, and is thus coupled with wheel motion as seen below.
\begin{multline}
m_{\text{sc}}[\tilde{\bm{c}}] \ddot{\bm r}_{B/N}
+[I_{\text{sc},B}] \dot{\bm\omega}_{\cal B/N}
+\sum_{i=1}^{N}J_{\text{s}_i} \hat{\bm g}_{\text{s}_i}\dot{\Omega}_i
= -[\bm{\tilde{\omega}}_{\cal B/N}] [I_{\text{sc},B}] \bm\omega_{\cal B/N} - \sum_{i=1}^{N}(\bm\omega_{\cal B/N}\times J_{\text{s}_i}\Omega_i\hat{\bm g}_{\text{s}_i})
+ \bm{L}_B
\label{eq:Final7}
\end{multline}
The motor torque equation can be seen below.
\begin{equation}
\dot{\Omega}_i = \frac{u_{\text{s}_i}}{J_{\text{s}_i}}-\hat{\bm g}_{\text{s}_i}^T \dot{\bm{\omega}}_{\cal B/N}
\label{eq:mtorque}
\end{equation}

\subsection{Back-Substitution Derivation}
Plugging Eq.~\eqref{eq:mtorque} into Eq.~\eqref{eq:Final7}
\begin{multline}
m_{\text{sc}}[\tilde{\bm{c}}] \ddot{\bm r}_{B/N}
+([I_{\text{sc},B}]-\sum_{i=1}^{N}J_{\text{s}_i} \hat{\bm g}_{\text{s}_i} \hat{\bm g}_{\text{s}_i}^T) \dot{\bm\omega}_{\cal B/N}
= -[\bm{\tilde{\omega}}_{\cal B/N}] [I_{\text{sc},B}] \bm\omega_{\cal B/N} - \sum_{i=1}^{N}(\hat{\bm g}_{\text{s}_i} u_{\text{s}_i} + \bm\omega_{\cal B/N}\times J_{\text{s}_i}\Omega_i\hat{\bm g}_{\text{s}_i})
\\- [I'_{\text{sc},B}] \bm\omega_{\cal B/N}
+ \bm{L}_B
\label{eq:Final72}
\end{multline}

\subsection{Back-Substitution Contribution Matrices}
The following can be defined:
\begin{equation}
[A_\text{contr}] = [0_{3 \times 3}]
\end{equation}

\begin{equation}
[B_\text{contr}] = [0_{3 \times 3}]
\end{equation}

\begin{equation}
[C_\text{contr}] = [0_{3 \times 3}]
\end{equation}

\begin{equation}
[D_\text{contr}] = -\sum_{i=1}^{N}J_{\text{s}_i} \hat{\bm g}_{\text{s}_i} \hat{\bm g}_{\text{s}_i}^T
\end{equation}

\begin{equation}
\bm v_{\text{trans,contr}} = \bm 0
\end{equation}

\begin{equation}
\bm v_{\text{rot,contr}} =  - \sum_{i=1}^{N}(\hat{\bm g}_{\text{s}_i} u_{\text{s}_i} + \bm\omega_{\cal B/N}\times J_{\text{s}_i}\Omega_i\hat{\bm g}_{\text{s}_i})
\end{equation}

\section{Imbalanced Reaction Wheel Back-Substitution}
\subsection{Equations of Motion}
The translational equation of motion is
\begin{equation}
	\ddot{\bm r}_{B/N} -[\tilde{\bm{c}}]\dot{\bm\omega}_{\cal B/N} +\frac{1}{m_{\text{sc}}}\sum\limits_{i=1}^{N}m_{\text{rw}_i}d_i\hat{\bm{w}}_{3_i}\dot{\Omega}_i = \ddot{\bm r}_{C/N}   - 2[\tilde{\bm\omega}_{\cal B/N}]\bm{c}'-[\tilde{\bm\omega}_{\cal B/N}][\tilde{\bm\omega}_{\cal B/N}]\bm{c} + \frac{1}{m_{\text{sc}}}\sum\limits_{i=1}^{N}m_{\text{rw}_i}d_i\Omega_i^2\hat{\bm{w}}_{2_i}
	\label{eq:rBddot}
\end{equation}
The rotational equation of motion is
\begin{equation}
	\begin{split}
		m_{\text{sc}}[\tilde{\bm c}]\ddot{\bm r}_{B/N} +& [I_{\text{sc},B}]\dot{\bm\omega}_{\cal B/N} +\sum\limits_{i=1}^{N}\Big([I_{\text{rw}_i,W_{c_i}}]\hat{\bm{g}}_{s_i} + m_{\text{rw}_i}d_i[\tilde{\bm{r}}_{W_{c_i}/B}]\hat{\bm{w}}_{3_i}\Big)\dot{\Omega}_i
		\\=&
		\sum\limits_{i=1}^{N}\Big[ m_{\text{rw}_i}[\tilde{\bm{r}}_{W_{c_i}/B}]d_i \Omega_i^2\hat{\bm{w}}_{2_i}-[I_{\text{rw}_i,W_{c_i}}]'\Omega_i \hat{\bm{g}}_{s_i} -[\tilde{\bm\omega}_{\cal B/N}]\Big([I_{\text{rw}_i,W_{c_i}}]\Omega_i \hat{\bm{g}}_{s_i}+ m_{\text{rw}_i}[\tilde{\bm{r}}_{W_{c_i}/B}]\bm{r}'_{W_{c_i}/B}\Big)\Big]
		\\&  -[\tilde{\bm\omega}_{\cal B/N}][I_{\text{sc},B}]\bm\omega_{\cal B/N}-  [I_{\text{sc},B}]'\bm\omega_{\cal B/N} + \bm{L}_B
	\end{split}
	\label{eq:rotationalEOM}
\end{equation}
The motor torque equation is (note that $J_{12_i} = J_{23_i} = 0$)
\begin{multline}
	\big[m_{\text{rw}_i} d_i \hat{\bm{w}}_{3_i}^T\big]\ddot{\bm{r}}_{B/N} + \big[(J_{11_i} + m_{\text{rw}_i} d_i^2)\hat{\bm g}_{\text{s}_i}^T  + J_{13_i}\hat{\bm w}_{3_i}^T -m_{\text{rw}_i} d_i \hat{\bm{w}}_{3_i}^T [\tilde{\bm r}_{W_i/B}]\big]\dot{\bm\omega}_{\cal B/N} + \big[J_{11_i} + m_{\text{rw}_i} d_i^2\big]\dot{\Omega}_i
	\\=   - J_{13_i} \omega_{w_{2_i}}\omega_{s_i}
	+ \omega_{w_{2_i}} \omega_{w_{3_i}} (J_{22_i} - J_{33_i} - m_{\text{rw}_i} d_i^2
	)
	 -m_{\text{rw}_i} d_i \hat{\bm{w}}_{3_i}^T [\tilde{\bm\omega}_{\cal B/N}] [\tilde{\bm\omega}_{\cal B/N}] \bm r_{W_i/B} + u_{s_i}
	\label{eq:motorTorqueFinal}
\end{multline}

\subsection{Derivation of Back-Substitution}

Solve motor torque equation for $\dot{\Omega}_i$ in terms of $\ddot{\bm{r}}_{B/N}$ and $\dot{\bm\omega}_{\cal B/N}$
\begin{multline}
\dot{\Omega}_i
= \frac{-m_{\text{rw}_i} d_i \hat{\bm{w}}_{3_i}^T }{J_{11_i} + m_{\text{rw}_i} d_i^2}\ddot{\bm{r}}_{B/N} + \frac{-\big[(J_{11_i} + m_{\text{rw}_i} d_i^2)\hat{\bm g}_{\text{s}_i}^T  + J_{13_i}\hat{\bm w}_{3_i}^T -m_{\text{rw}_i} d_i \hat{\bm{w}}_{3_i}^T [\tilde{\bm r}_{W_i/B}]\big]}{J_{11_i} + m_{\text{rw}_i} d_i^2}\dot{\bm\omega}_{\cal B/N}
\\
+\frac{1}{J_{11_i} + m_{\text{rw}_i} d_i^2}\left[\omega_{w_{2_i}} \omega_{w_{3_i}} (J_{22_i} - J_{33_i} - m_{\text{rw}_i} d_i^2
)-J_{13_i} \omega_{w_{2_i}}\omega_{s_i} -m_{\text{rw}_i} d_i \hat{\bm{w}}_{3_i}^T [\tilde{\bm\omega}_{\cal B/N}] [\tilde{\bm\omega}_{\cal B/N}] \bm r_{W_i/B} + u_{s_i} \right]
\label{eq:motorTorqueFinal2}
\end{multline}

\begin{equation}
\bm{a}_{\Omega_i} = -\frac{m_{\text{rw}_i} d_i \hat{\bm{w}}_{3_i} }{J_{11_i} + m_{\text{rw}_i} d_i^2}
\end{equation}

\begin{equation}
\bm{b}_{\Omega_i} = -\frac{(J_{11_i} + m_{\text{rw}_i} d_i^2)\hat{\bm g}_{\text{s}_i}  + J_{13_i}\hat{\bm w}_{3_i} +m_{\text{rw}_i} d_i  [\tilde{\bm r}_{W_i/B}]\hat{\bm{w}}_{3_i}}{J_{11_i} + m_{\text{rw}_i} d_i^2}
\end{equation}

\begin{equation}
c_{\Omega_i} = \frac{1}{J_{11_i} + m_{\text{rw}_i} d_i^2}\left[\omega_{w_{2_i}} \omega_{w_{3_i}} (J_{22_i} - J_{33_i} - m_{\text{rw}_i} d_i^2
) - J_{13_i} \omega_{w_{2_i}}\omega_{s_i} -m_{\text{rw}_i} d_i \hat{\bm{w}}_{3_i}^T [\tilde{\bm\omega}_{\cal B/N}] [\tilde{\bm\omega}_{\cal B/N}] \bm r_{W_i/B} + u_{s_i} \right]
\end{equation}

\begin{equation}
\dot{\Omega}_i = \bm{a}_{\Omega_i}^T\ddot{\bm r}_{B/N} + \bm{b}_{\Omega_i}^T\dot{\bm\omega}_{\cal B/N} + c_{\Omega_i}
\end{equation}
Plugging the equation above into Eq.~\eqref{eq:rBddot} and multiplying both sides by $m_\text{sc}$, (plug $\dot{\Omega}_i$ into translation)
\begin{multline}
\left[ [I_{3\times3}] +\frac{1}{m_{\text{sc}}}\sum\limits_{i=1}^{N}m_{\text{rw}_i}d_i\hat{\bm{w}}_{3_i}\bm{a}_{\Omega_i}^T \right] \ddot{\bm r}_{B/N} +\left[ -[\tilde{\bm{c}}] + \frac{1}{m_{\text{sc}}}\sum\limits_{i=1}^{N}m_{\text{rw}_i}d_i\hat{\bm{w}}_{3_i}\bm{b}_{\Omega_i}^T \right] \dot{\bm\omega}_{\cal B/N}
\\= \ddot{\bm r}_{C/N}   - 2[\tilde{\bm\omega}_{\cal B/N}]\bm{c}'-[\tilde{\bm\omega}_{\cal B/N}][\tilde{\bm\omega}_{\cal B/N}]\bm{c} + \frac{1}{m_{\text{sc}}}\sum\limits_{i=1}^{N}m_{\text{rw}_i}d_i\left(\Omega_i^2\hat{\bm{w}}_{2_i}-c_{\Omega_i}\hat{\bm{w}}_{3_i}\right)
\end{multline}
Moving on to rotation, (plug $\dot{\Omega}_i$ into rotation)
\begin{multline}
\left[ m_{\text{sc}}[\tilde{\bm c}]+ \sum\limits_{i=1}^{N}\Big([I_{\text{rw}_i,W_{c_i}}]\hat{\bm{g}}_{s_i} + m_{\text{rw}_i}d_i[\tilde{\bm{r}}_{W_{c_i}/B}]\hat{\bm{w}}_{3_i}\Big)\bm{a}_{\Omega_i}^T \right] \ddot{\bm r}_{B/N}
\\
+ \left[ [I_{\text{sc},B}] + \sum\limits_{i=1}^{N}\Big([I_{\text{rw}_i,W_{c_i}}]\hat{\bm{g}}_{s_i} + m_{\text{rw}_i}d_i[\tilde{\bm{r}}_{W_{c_i}/B}]\hat{\bm{w}}_{3_i}\Big)\bm{b}_{\Omega_i}^T \right] \dot{\bm\omega}_{\cal B/N}
\\=
\sum\limits_{i=1}^{N}\Big[ m_{\text{rw}_i}[\tilde{\bm{r}}_{W_{c_i}/B}]d_i \Omega_i^2\hat{\bm{w}}_{2_i}-[I_{\text{rw}_i,W_{c_i}}]'\Omega_i \hat{\bm{g}}_{s_i} -[\tilde{\bm\omega}_{\cal B/N}]\Big([I_{\text{rw}_i,W_{c_i}}]\Omega_i \hat{\bm{g}}_{s_i}+ m_{\text{rw}_i}[\tilde{\bm{r}}_{W_{c_i}/B}]\bm{r}'_{W_{c_i}/B}\Big) \\
-\Big([I_{\text{rw}_i,W_{c_i}}]\hat{\bm{g}}_{s_i} + m_{\text{rw}_i}d_i[\tilde{\bm{r}}_{W_{c_i}/B}]\hat{\bm{w}}_{3_i}\Big)c_{\Omega_i}\Big]
\\  -[\tilde{\bm\omega}_{\cal B/N}][I_{\text{sc},B}]\bm\omega_{\cal B/N}-  [I_{\text{sc},B}]'\bm\omega_{\cal B/N} + \bm{L}_B
\end{multline}
Now we have two equations containing $\ddot{\bm{r}}_{B/N}$ and $\dot{\bm\omega}_{\cal B/N}$

\subsection{Back-Substitution Contribution Matrices}

\begin{equation}
[A_\text{contr}] = \frac{1}{m_{\text{sc}}}\sum\limits_{i=1}^{N}m_{\text{rw}_i}d_i\hat{\bm{w}}_{3_i}\bm{a}_{\Omega_i}^T
\end{equation}

\begin{equation}
[B_\text{contr}] = \frac{1}{m_{\text{sc}}}\sum\limits_{i=1}^{N}m_{\text{rw}_i}d_i\hat{\bm{w}}_{3_i}\bm{b}_{\Omega_i}^T
\end{equation}

\begin{equation}
[C_\text{contr}] = \sum\limits_{i=1}^{N}\Big([I_{\text{rw}_i,W_{c_i}}]\hat{\bm{g}}_{s_i} + m_{\text{rw}_i}d_i[\tilde{\bm{r}}_{W_{c_i}/B}]\hat{\bm{w}}_{3_i}\Big)\bm{a}_{\Omega_i}^T
\end{equation}

\begin{equation}
[D_\text{contr}] = \sum\limits_{i=1}^{N}\Big([I_{\text{rw}_i,W_{c_i}}]\hat{\bm{g}}_{s_i} + m_{\text{rw}_i}d_i[\tilde{\bm{r}}_{W_{c_i}/B}]\hat{\bm{w}}_{3_i}\Big)\bm{b}_{\Omega_i}^T
\end{equation}

\begin{equation}
\bm{v}_\text{trans,contr} = \frac{1}{m_{\text{sc}}}\sum\limits_{i=1}^{N}m_{\text{rw}_i}d_i\left(\Omega_i^2\hat{\bm{w}}_{2_i}-c_{\Omega_i}\hat{\bm{w}}_{3_i}\right)
\end{equation}

\begin{multline}
\bm{v}_\text{rot,contr} = \sum\limits_{i=1}^{N} \Big[ m_{\text{rw}_i}[\tilde{\bm{r}}_{W_{c_i}/B}]d_i \Omega_i^2\hat{\bm{w}}_{2_i}-[I_{\text{rw}_i,W_{c_i}}]'\Omega_i \hat{\bm{g}}_{s_i} -[\tilde{\bm\omega}_{\cal B/N}]\Big([I_{\text{rw}_i,W_{c_i}}]\Omega_i \hat{\bm{g}}_{s_i}+ m_{\text{rw}_i}[\tilde{\bm{r}}_{W_{c_i}/B}]\bm{r}'_{W_{c_i}/B}\Big) \\
-\Big([I_{\text{rw}_i,W_{c_i}}]\hat{\bm{g}}_{s_i} + m_{\text{rw}_i}d_i[\tilde{\bm{r}}_{W_{c_i}/B}]\hat{\bm{w}}_{3_i}\Big)c_{\Omega_i} \Big]
\end{multline}

\end{document}


\documentclass{notes}

  \title{Heap Sort}
  \author{ian.mcloughlin@gmit.ie}
  \date{\today}

\begin{document}

  \section*{Sorting}
    \[l = (l_0, l_1, l_2, \ldots, l_{n-1})\]
    \[l_0 \leq l_1 \leq \cdots \leq l_{n-1}\]

  \section*{On a tree}

  \begin{center}
    \begin{forest}
      for tree={circle, draw}
      [\(l_0\)
        [\(l_1\)
          [\(l_3\)
            [\(l_7\)]
            [\(l_8\)]
          ]
          [\(l_4\)
            [\(l_9\)]
          ]
        ]
        [\(l_2\)
          [\(l_5\)]
          [\(l_6\)]
        ]
      ]
    \end{forest}
  \end{center}

  \section*{Heaps}
  Almost complete binary tree with heap property.
    \begin{description}
      \item[Max heap:] each parent bigger than children.
      \item[Min heap:] each parent smaller than children.
    \end{description}

  \section*{To min or max heap}
    \begin{enumerate}
      \item Start with last node, moving backwards.
      \item Compare node to children, swap if needed.
      \item Swap parent down tree until we have a heap.
    \end{enumerate}

    \section*{Example heap}
    Sort ascending \(\rightarrow\) use max heap.

    \begin{center}
      \begin{forest}
        for tree={circle, draw}
        [\(1\)
          [\(3\)
            [\(2\)
              [\(7\)]
              [\(5\)]
            ]
            [\(8\)
              [\(9\)]
            ]
          ]
          [\(10\)
            [\(4\)]
            [\(6\)]
          ]
        ]
      \end{forest}
    \end{center}

    Last five nodes have no children.
    Sixth-last has one child, is smaller so swap.
    Now have a heap from sixth-last.
    Same for seventh-last: swap 2 for 7.
    Third node is a heap.
    Second node swaps 9 for 3, and filters down swapping 3 for 8.
    Finally, the root is swapped with 10 and then 6.

    \begin{center}
      \begin{forest}
        for tree={circle, draw}
        [\(10\)
          [\(9\)
            [\(7\)
              [\(2\)]
              [\(5\)]
            ]
            [\(8\)
              [\(3\)]
            ]
          ]
          [\(6\)
            [\(4\)]
            [\(1\)]
          ]
        ]
      \end{forest}
    \end{center}

  \section*{As an array}

  \begin{center}
    \begin{forest}
      for tree={circle, draw}
      [\(l_i\)
        [\(l_{2i + 1}\)]
        [\(l_{2i + 2}\)]
      ]
    \end{forest}
  \end{center}

  \begin{align*}
  &(&l_0&,&l_1&,&l_2&,&l_3&,&l_4&,&l_5&,&l_6&,&l_7&,&l_8&,&l_9\ &)&\\
  &(&1&,&3&,&10&,&2&,&{\color{gmitred}8}&,&4&,&6&,&7&,&5&,&{\color{gmitred}9}\ &)&\\
  &(&1&,&3&,&10&,&{\color{gmitred}2}&,&9&,&4&,&6&,&{\color{gmitred}7}&,&{\color{gmitred}5}&,&8\ &)&\\
  &(&1&,&3&,&{\color{gmitred}10}&,&7&,&9&,&{\color{gmitred}4}&,&{\color{gmitred}6}&,&2&,&5&,&8\ &)&\\
  &(&1&,&{\color{gmitred}3}&,&10&,&{\color{gmitred}7}&,&{\color{gmitred}9}&,&4&,&6&,&2&,&5&,&8\ &)&\\
  &(&1&,&9&,&10&,&7&,&{\color{gmitred}3}&,&4&,&6&,&2&,&5&,&{\color{gmitred}8}\ &)&\\
  &(&{\color{gmitred}1}&,&{\color{gmitred}9}&,&{\color{gmitred}10}&,&7&,&8&,&4&,&6&,&2&,&5&,&3\ &)&\\
  &(&10&,&9&,&{\color{gmitred}1}&,&7&,&8&,&{\color{gmitred}4}&,&{\color{gmitred}6}&,&2&,&5&,&3\ &)&\\
  &(&10&,&9&,&6&,&7&,&8&,&4&,&1&,&2&,&5&,&3\ &)&\\
  \end{align*}

  \section*{Heap Sort}
    \begin{enumerate}
      \item Convert complete binary tree to heap.
      \item Swap root for last child, ignore last child.
      \item Repeat \(n-1\) times.
    \end{enumerate}

    \begin{align*}
      &(&l_0&,&l_1&,&l_2&,&l_3&,&l_4&,&l_5&,&l_6&,&l_7&,&l_8&,&l_9\ &)&\\
      &(&10&,&9&,&6&,&7&,&8&,&4&,&1&,&2&,&5&,&3\ &)&\\
      &(&3&,&9&,&6&,&7&,&8&,&4&,&1&,&2&,&5&,&{\color{gmitblue}10}\ &)&\\
      &(&9&,&8&,&6&,&7&,&3&,&4&,&1&,&2&,&5&,&{\color{gmitblue}10}\ &)&\\
      &(&5&,&8&,&6&,&7&,&3&,&4&,&1&,&2&,&{\color{gmitblue}9}&,&{\color{gmitblue}10}\ &)&\\
      &(&8&,&7&,&6&,&5&,&3&,&4&,&1&,&2&,&{\color{gmitblue}9}&,&{\color{gmitblue}10}\ &)&\\
      &(&2&,&7&,&6&,&5&,&3&,&4&,&1&,&{\color{gmitblue}8}&,&{\color{gmitblue}9}&,&{\color{gmitblue}10}\ &)&\\
    \end{align*}

  \section*{Comparisons}

    \begin{description}
      \item[To heap:] \(O(n \log n)\)
      \item[Replace root:] \(O(\log n)\) but \( O(n) \) times.
      \item[Check heap:] \(O(n)\)
    \end{description}

  %\bibliography{bibliography}
\end{document}