%Chapter "Computational aspects"
%
\graphicspath{ {./img/Computational/} }
\chapter{FEM formulation of the elasticity BVP}
\label{chap: Computational Aspects}

\section*{Preliminary}
The previous chapter covered the boundary value problem for the model of theory of elasticity. It was shown that the model could equivalently be written in strong form (i.e., governing equations and boundary conditions) or in weak form. It was also shown that a particular interpretation of the weak form was that of the principle of virtual displacements. In this chapter we start from this principle and use interpolation theory to approximate both physical and virtual functions. As a result, this weak form of the BVP becomes a system of linear equations in the interpolation parameters that artificially enforce the principle of virtual displacements. The resulting finite element equations are shown to be equivalent to those governing the mechanical equilibrium of a discrete system of particles. To facilitate conceptual understanding we find first the equations for a single element and then proceed to consider the full mesh after invoking equilibrium and displacement compatibility conditions along element interfaces.

At the end of this chapter\footnote{{\bf This chapter, together with theoretical and computational learning activities is complemented by Jupyter Notebooks 8 through 12 available at the course REPO.}} the student should be able to:

\begin{itemize}
\item[•] Use interpolation theory to write the finite element based discrete version of the principle of virtual work.

\item[•] Recognize the finite element equilibrium equations of the elasticity BVP as physically analogous to those of a discrete system of particles.

\item[•] Understand the computational steps required for the implementation of the finite element. equilibrium equations in a computer language like Python.

\item[•] Understand the approximate nature of finite element solutions.

\end{itemize}

\section{FE algorithm starting from the weak formulation of the BVP}
In the previous chapter it was shown that the elasticity BVP could equivalently be formulated in differential and integral form in a so-called weak formulation. In this section we show that a simple finite element algorithm can be obtained if one introduces discretization, through the use of interpolation theory, into this weak form. Discretization appears in two forms: first, when one assumes that a primary variable is known at discrete (nodal) points and second when the interpolation functions exist over finite sub-domains or elements. For completeness we start by recalling the weak formulation introduced previously and then use our combined index notation for scalar components of tensorial functions and for componentes of nodal variables. The weak form is then given as follows:

Given the body forces $f_i$, the surface tractions $t_i^{\hat n}$ prescribed over the part of the boundary $S_t$ and the surface displacements ${\bar u_i}$ prescribed over the part of the boundary $S_u$ find the displacement field ${u_i}:V \to \mathbb{R}$ and such $\forall {w_i} \in V$ it satisfies:
\[\intL_V \sigma_{ij} w_{i,j}\, \dd{V} - \intL_V f_i w_i\, dV  - \intL_{S_t} t_i^{\hat n} w_i\, \dd{S} = 0.\]

In the expression above $V$ represents the complete problem domain while $S_t$ is that part of the boundary over which tractions are prescribed. Assume now that the full domain $V$ is divided into $NUMEL$ non-overlapping sub-domains (or finite elements) as shown schematically in \cref{fig:blow2}

\begin{figure}[h]
\centering
\includegraphics[width=8cm]{meshblow.pdf}
\caption{Meshed blow}
\label{fig:blow2}
\end{figure}

and in such a way that mathematically this can be expressed like:

\[V=\bigcup \Omega _e\]

and where $\Omega _e$ represents the sub-domain occupied by one such an element.

\begin{tcolorbox}
In the original strong form of the BVP the partial differential equations result from equilibrium arguments valid over infinitesimal material points occupying the domain $V$. In the FEM the operation of dividing the full domain $V$ into {\bf NUMEL} subdomains is equivalent to rendering the continuous problem with infinite material points into a discrete problem with a finite number of elements. In conclusion, a finite element is the discrete approximation of an infinitesimal material point.

\end{tcolorbox}

Note that each element is defined by an ordered set of nodal points establishing a prescribed interpolation space and used to conduct function approximations. In this element partition adjacent elements are connected through the nodal points in such a way that interpolation functions exist only within its element but are continuous through element interfaces. Using this partition in the weak form of equilibrium and using a summation symbol yields;

\[\sum_e\int_{\Omega_e}\sigma_{ij}w_{i,j}\operatorname d\Omega_e-\sum_e\int_{\Omega_e}f_iw_i\operatorname d\Omega_e-\sum_e\int_{S_e}t_iw_i\operatorname dS_e = 0\]

which is still the same form of the weak equilibrium expression.

In a second discretization operation we use interpolation methods to approximate both, the displacement field $u_i$ and the weighting function $w_i$ over each element as discussed previously. In particular assume that inside every element the fields can be approximated like:

\begin{align*}
u_i(\overrightarrow x)& =N_i^Q(\overrightarrow x)\widehat u^Q \\
w_i(\overrightarrow x)& =N_i^Q(\overrightarrow x)\widehat w^Q
\end{align*}

and where $u^Q$ represent the scalar components of the displacement vector at the nodal point $Q$, $N_i^Q(\overrightarrow x)$ is the shape function for the nodal point $Q$ evaluated at the field point $\overrightarrow x$ and $u_i(\overrightarrow x)$ is the interpolated approximation to the displacement vector at the field point $\overrightarrow x$.

\begin{tcolorbox}
Recall that in expressions like

\[u_i(\overrightarrow x) =N_i^Q(\overrightarrow x)\widehat u^Q\]

the repeated super-script $Q$ implies summation over $Q= 1,..,Nnodes$ where $Nnodes$ is the number of nodes of the element.

\end{tcolorbox}

Considering the displacement field as the primary variable we can write in terms of the elastic constitutive tensor:

\[\sigma_{ij}=C_{ijkl}\varepsilon_{kl}\equiv C_{ijkl}B_{kl}^Q(\overrightarrow x)\widehat u^Q\]

after using

\begin{align*}
\varepsilon_{ij}(\overrightarrow x)=B_{ij}^Q(\overrightarrow x)\widehat u^Q \\
w_{(i,j)}(\overrightarrow x)=B_{ij}^Q(\overrightarrow x)\widehat w^Q
\end{align*}

and where

\[ B_{ij}^Q(\overrightarrow x)=\frac12\left[\frac{\partial N_i^Q}{\partial x_j}+\frac{\partial N_j^Q}{\partial x_i}\right].\]

Substitution of the above in the discretized weak form gives us for an arbitrary $e$-element of domain $\Omega_e$

\[\widehat w^P\int_{\Omega_e}B_{ij}^PC_{ijkl}B_{kl}^Q\operatorname d\Omega_e\widehat u^Q-\widehat w^P\int_{\Omega_e}N_i^Pf_i\operatorname d\Omega_e-\widehat w^P\int_{S_e}N_i^Pt_i^{(n)}\operatorname ds_e=0.\]

Note that $w_i$ in the above expression is an arbitrary function and it is convenient to assign values like

\[\widehat w^T=\begin{bmatrix}0\cdots&1\cdots&0\end{bmatrix}\]

in turn  for every node in order to cancel the common term $\widehat w^P$ in such a way that we can write for an arbitrary $P$ degree of freedom:

\[\int_{\Omega_e}B_{ij}^PC_{ijkl}B_{kl}^Qd\Omega_e\widehat u^Q=\int_{\Omega_e}N_i^Pf_id\Omega_e+\int_{S_e}N_i^Pt_idS_e.\]

In this expression the dummy superscripts imply a summation over the number of nodes of the element in such a way that the only free index is $P$ which in this compact notation implies also that we have a system of equations corresponding to $P= 1,..,Nnodes$, which is the number of nodes of the element. Note also that the term $K_e^{PQ}$ defined by

\[K_e^{PQ}=\int_{\Omega_e}B_{ij}^PC_{ijkl}B_{kl}^Q\operatorname d\Omega_e\]

represents the force along degree of freedom $P$ due to a unit displacement along degree of freedom $Q$. In this sense we can define the following set of nodal forces along the $P$-th degree of freedom:

\begin{align*}
f_\sigma^P & =K_e^{PQ}\widehat u^Q \\
f_B^P & =\int_{\Omega_e}N_i^Pf_i\operatorname d\Omega_e \\
f_c^P & =\int_{S_e}N_i^Pt_i^{(n)}\operatorname dS_e
\end{align*}

satisfying

\begin{equation}
f_\sigma^P=f_B^P+f_C^P
\label{nodal_forces}
\end{equation}

and where $f_\sigma^P$, $f_B^P$ and $f_C^P$ are forces due to the internal element stresses, body forces and surface tractions respectively.

\Cref{nodal_forces} which can also be written like:

\[K_e^{PQ}\widehat u^Q =f_B^P+f_C^P\]

is an equation in the nodal displacement $u^Q$ . However it must be observed that this equation has been derived for a single element $\Omega_e$ and the possible contribution in terms of forces and stiffness terms from additional elements is yet to be considered. This final step, leading to the assembly of the global system of equilibrium equations (to be discussed later) gives:

\begin{equation}
\left[K^G\right]\left\{U^G\right\}=\left\{RHS^G\right\}
\label{globalsys}
\end{equation}

and where the term $RHS^G$ is a vector of assembled global forces of the type $f_B^P+f_C^P$ storing the contribution from body forces and external surface tractions.

\begin{tcolorbox}
The details of the Python implementation of the stiffness matrix:

\[K_e^{PQ}=\int_{\Omega_e}B_{ij}^PC_{ijkl}B_{kl}^Q\operatorname d\Omega_e\]

are presented as an in-class activity in Notebook 6. This activity requires prior knowledge of interpolation (see Notebook 3) and numerical integration.

\end{tcolorbox}

\section{FE algorithm starting from the principle of virtual work}

As shown in the previous chapter a valid solution $(u_i , \sigma_{ij} , \epsilon_{ij})$ to the well-posed boundary value problem in theory of elasticity and where $u_i={\overline u}_i \quad \text{in} \quad  S_u$

satisfies

\begin{equation}
\int\limits_V \sigma _{ij}\var{\varepsilon_{ij}}\dd{V}  - \int\limits_V f_i\delta u_i\dd{V}  - \int\limits_{S_t} t_i^{(n)}\var{u_i}\dd{S}  = 0
\label{PVDs}
\end{equation}

for arbitrary functions $\delta u_i$ such that  $\delta u_i =0 \quad \text{in} \quad  S_u$ and where:

\[\delta\varepsilon_{ij}=\frac12\left(\frac{\partial\delta u_i}{\partial x_j}+\frac{\partial\delta u_j}{\partial x_i}\right).\]

After using the same arguments and interpolation scheme as in the weak form discretization \cref{PVDs} leads to:

\[\int_{\Omega_e}B_{ij}^PC_{ijkl}B_{kl}^Qd\Omega_e\widehat u^Q=\int_{\Omega_e}N_i^Pf_id\Omega_e+\int_{S_e}N_i^Pt_idS_e\]

which is the same equation resulting from the discretization of the weak form.

Note that the PVW statement is an scalar (energy) equation relating the strain energy,

\[\delta E_s=\int_{\Omega_e}\sigma_{ij}\delta\varepsilon_{ij}\operatorname d\Omega_e\]

accumulated in the elastic solid as a result of the imposed virtual strain field $\delta\varepsilon_{ij}$ to the work of the external loads

\[\delta W=\int_{\Omega_e}f_i\delta u_i\operatorname d\Omega_e+\int_{S_e}t_i^{(n)}\delta u_i\operatorname dS_e\]

due to the imposition of the virtual displacement $\delta u_i$.

Using the same notation as in the previous algorithm the discrete version can also be written like:

\[\delta\widehat u^Pf_\sigma^P-\delta\widehat u^Pf_B^P-\delta\widehat u^Pf_c^P=0\]

and after using the arbitrariness in $\delta u_i$

\begin{equation}
f_\sigma^P-f_B^P-f_c^P=0.
\label{PVW_force}
\end{equation}

In a sense the FEM scheme makes use of interpolation theory to convert the continuous BVP governed by a set of partial differential equations into a discrete problem governed by a set of algebraic equations. Within this context \cref{PVW_force} can be interpreted as the force equilibrium equation governing the response of a particle $P$ subject to the action of:

\begin{itemize}
\item[•] External nodal forces $f_B^P$ consistent with body forces $f_i$.
\item[•] External nodal forces $f_c^P$ consistent with surface tractions $t_i^{(n)}$
\item[•] Internal forces $f_\sigma^P$ consistent with element stresses and equilibrating the external forces.
\end{itemize}

On the other hand, recall  that the PVW statement only holds if the set $(u_i , \sigma_{ij} , \epsilon_{ij})$ is the actual solution to the BVP. This fact implies that in the FE formulation where we only have an approximate solution $u_i(\overrightarrow x) =N_i^Q(\overrightarrow x)\widehat u^Q $ the principle does not hold as this is not the actual solution. However, the nodal displacements found after solving \cref{PVW_force} enforces such condition.

\begin{tcolorbox}

Since the finite element equation written in the form the term:

\[f_\sigma^P  =K_e^{PQ}\widehat u^Q\]

represents nodal forces equivalent to element stresses the coefficients $K_e^{PQ}$ represent the nodal force along degree of freedom $P$ due to a unit displacement along degree of freedom $Q.$

\end{tcolorbox}

\section{Global assembly}

The fundamental equation

\begin{equation}
K_e^{PQ}\widehat u^Q =f_B^P+f_c^P
\label{oneelement}
\end{equation}

resulting from the discretization of the PVW through the introduction of interpolation functions was derived over a single element of domain $\omega_e$. In order to consider the contribution from all the elements in a finite element discretized version of a solid we use Newton's third law of motion (action and reaction principle). For that purpose recall that the term $f_C^P$ in \cref{oneelement} corresponds to the contact forces resulting from surface tractions and it is computed according to:

\begin{equation}
f_c^P=\int_{S_t}N_i^Pt_i^n\operatorname dS
\label{traction}
\end{equation}

where it is evident that they depend on the external outward normal vector $\widehat n$ over $S_t$. Clearly, the mechanical interaction between two elements in contact takes place through these contact forces. To explain this interaction consider the schematic mesh shown in \cref{fig:4assembled1} below:

\begin{figure}[H]
\centering
\includegraphics[width=8cm]{assembled}
\caption{4-elements mesh}
\label{fig:4assembled1}
\end{figure}

Focusing attention in elements $1$ and $2$ let the interacting surface be labelled as $S_b$ and extend this notation to the degrees of freedom and forces in such a way that the degrees of freedom of nodal points along $S_b$ would be named $U_b$ while those at other parts of the element would be named $U_c$. Accordingly the finite element equilibrium equations for each element can be written like:

\begin{equation}
\begin{Bmatrix}F_a\\F_b(\widehat n)\end{Bmatrix} = \begin{bmatrix}K_{aa}^1&K_{ab}^1\\K_{ba}^1&K_{bb}^1\end{bmatrix}\begin{Bmatrix}U_a\\U_b\end{Bmatrix}
\end{equation}

and

\begin{equation}
\begin{Bmatrix}-F_b(\widehat n\ast)\\F_C\end{Bmatrix}=\begin{bmatrix}K_{bb}^2&K_{bc}^2\\K_{cb}^2&K_{cc}^2\end{bmatrix}\begin{Bmatrix}U_b\\U_c\end{Bmatrix}
\end{equation}

The dependency of the contact forces along the common interface $S_b$ upon the normal outward vector $\widehat{n}$ has been made explicit in the equations. In particular this vector reads $\widehat n$ for element $1$ and ${\widehat n}^*$ for element $2$ and they satisfy:

\[\widehat n = - {\widehat n}^* . \]

Using this relation between the normal vectors at the contact interface we can write for the nodal forces:

\[
F_b(\widehat{n})+F_b({\widehat{n} }^*)=0
\]

or equivalently

\begin{equation}
F_b(\widehat n)-F_b(\widehat n)=0.
\label{coupling}
\end{equation}

This condition is shown for completeness in \cref{fig:coupled1}

\begin{figure}[H]
\centering
\includegraphics[width=12cm]{coupled1}
\caption{Equilibrated nodal forces along the contact interface $S_b$ of elements 1 and 2.}
\label{fig:coupled1}
\end{figure}

where it is evident that the contact nodal forces along element interfaces are equal and opposite in accordance with Newton's third law. On the other hand, from displacements continuity we have the condition:

\begin{equation}
U_B = U_B^1 = U_B^2.
\label{continuity}
\end{equation}

Using \cref{coupling} and \cref{continuity} in the equilibrium relationships yields the partial assemblage of the global stiffness matrix resulting after considering the interaction between elements $1$ and $2$:

\begin{equation}
\begin{bmatrix}K_{aa}^1&K_{ab}^1&0\\K_{ba}^1&K_{bb}^1+K_{bb}^2&K_{bc}^2\\0&K_{cb}^2&K_{cc}^2\end{bmatrix}\begin{Bmatrix}U_a\\U_b\\U_c\end{Bmatrix}=\begin{Bmatrix}F_a\\0\\F_c\end{Bmatrix}.
\label{finaleq}
\end{equation}

Note that in the right hand side vector from the above assembled global equilibrium equations only contact forces have been included. This process of adding elemental matrices through contact nodal forces is termed {\bf element assembly} and it is symbolically written like:

\begin{equation}\label{eq:assem}
{K^G}=\assem_{i=1}^{Numel} k^i
\end{equation}

and where $K^G$ is the global coefficient matrix; $k^i$ is the local elemental matrix for the $i$-th element; $\assem_{i=1}^{Numel}$ is the assembly operator and $i$ is an element index ranging between $1$ and the total number of elements $Numel$ that conform the finite element model. Notice that the assembly operator is analogous to a summation operator commonly used in the representation of series of finite or infinite number of terms, however in the finite element algorithm the assembly operator contains information indicating the position of each single term from the element coefficient matrix within the global system.

\begin{tcolorbox}

In most finite element codes the assembly operator is an array of integer values indicating the position (row and column) from each coefficient of the stiffness matrix of a given element. In {\bf SolidsPy} this operator is termed the {\bf DME()} operator and it is computed in the assembly module {\bf ASSEMUTIL}.

\end{tcolorbox}

\section{The assembly algorithm}

In this section we will discuss the fundamental algorithmic steps required for building the system of algebraic equations through the addition or assembly of elemental coefficient matrices as described by \cref{eq:assem}. This process of assembly of the elemental coefficient matrices into the global system involves:

\begin{itemize}
\item (i) The identification of the active degrees of freedom (or equation numbers) assigned to each node in the mesh.
\item (ii) The identification of the relationship between the elemental degrees of freedom and the global degrees of freedom.
\item (iii) The computation of the coefficient matrix for each element in the model.
\end{itemize}

\begin{tcolorbox}

In the finite element jargon the word {\bf elemental} commonly refers to variable or computations performed at the element level.

\end{tcolorbox}

Step (i) is easily accomplished by assigning a boundary condition flag to the degrees of freedom existing at each node indicating if the degree of freedom is active, in which case it contributes with an equation, or if it is prescribed, in which case it contributes with a specified displacement boundary condition. In our notation we use a $0$ value to indicate an active degree of freedom and a $-1$ value to indicate a prescribed degree of freedom. In the Python based code {\bf SolidsPy} this information is stored in a boundary conditions array termed $IBC$ and with dimensions $nn \times MDIM$, where $nn$ corresponds to the  total number of nodal points and $MDIM$ represents the problem dimensionality. The $IBC$ array is first given to the program through an input data file and later translated from $0$s and $-1$s to equation numbers. Thus during this process the boundary condition flag is read and equations are counted and numbers assigned according to the result. If the boundary condition flag is equal to $0$ the program assigns an equation number while it retains a $-1$ value for a $-1$ flag. In summary, the $IBC$ array exists in two instances. In its first instance it just contains integer flags indicating active or restrained degrees of freedom at each node while in a second instance it indicates the actual active equations existing at each node.

In parallel and also with data given through an input file, the nodes conforming each element are stored in a {\bf connectivity} array termed $IELCON$ and of dimension $Numel \times MxNNel$ where $Numel$ is the number of elements in the model and $MxNNel$ is the maximum number of nodes in a given element. Thus each row in the $IELCON$ array stores the nodal point data for the element. Each entry in the $IELCON$ array can now be directly translated into equation numbers using the processed version of the $IBC$ array. This process results in the discrete version of the assembly operator $\assem_{i=1}^{Numel}$ also called the assembly list or the $DME$ operator in {\bf SolidsPy}.

In the final step the mesh is looped one element at a time and each elemental coefficient matrix is assembled into the global matrix using the information (equation numbers) stored in the $DME$ operator. The actual computation of the elemental matrix is conducted by an element based subroutine, called $UEL$ in {\bf SolidsPy}. These subroutines may be different for each element in the mesh according to different kinematic o material assumptions. The assembly process is further illustrated by the following sample problem.

\paragraph*{Example}
For the mesh shown in \cref{fig:quad} where the elements are numbered from left to right and bottom to top write:

\begin{itemize}
\item (i) The boundary conditions array {\bf IBC()} in its two instances.
\item (ii) The connectivities array for the mesh.
\item (iii) The assembly {\bf DME()} operator.
\end{itemize}

\begin{figure}[h]
\centering
\includegraphics[width=8cm]{mesh2.pdf}
\caption{Mesh of 4 bi-linear finite elements.}
\label{fig:quad}
\end{figure}

(i) In a first instance the boundary conditions array {\bf IBC()} is filled with data assigned by the user in the input file. Since all the nodes in the bottom and top faces of the mesh are restrained in the horizontal and vertical directions they are assigned the flag $-1$ in each column. Similarly, node 4 is restrained only in the horizontal direction so it is assigned a $-1$ in the first position and a $0$ in the second position, while nodes 5 and 6 are completely free along both directions and they are assigned values of $0$. The resulting array is given by:

\[IBC_1 = \begin{bmatrix}
-1 & -1\\
-1 & -1\\
-1 & -1\\
-1 & 0\\
0 & 0\\
0 & 0\\
-1 & -1\\
-1 & -1\\
-1 & -1
\end{bmatrix}\]

In a second instance the boundary conditions array {\bf IBC()} is modified by the program which reads every value retaining those corresponding to $-1$ and assigning sequential numbers to those corresponding to $0$. Accordingly the first equation identified as $U_0$ corresponds to the vertical displacement of nodal point 4 while the remaining 4 equations associated to degrees of freedom $U_1 , U_2 , U_3 , U_4$ are the horizontal and vertical displacements of nodes 5 and 6. The final form of the array is given like:

\[IBC_2 = \begin{bmatrix}
-1 & -1\\
-1 & -1\\
-1 & -1\\
-1 & 0\\
1 & 2\\
3 & 4\\
-1 & -1\\
-1 & -1\\
-1 & -1
\end{bmatrix}\]

(ii) The next step corresponds to the creation of the connectivities array {\bf IELCON()} storing the nodal points defining each element. In this case each element is defined by a list of 4 nodal point numbers. Recall that for the computation of the elemental stiffness matrix each element in the physical mesh is mapped to the canonical element in the natural space. In the natural space the shape functions are associated to a fixed node numbering and element orientation. If one follows a counter-clockwise orientation the node numbering must follow the same counter-clockwise orientation in the physical space. However the selection of the first node in the sequence is arbitrary and the mapping into the natural space will take care of the corresponding rotation. In this case a valid connectivities array is that given by:

\[IELCON = \begin{bmatrix}
0 &1 &4 &3\\
1 &2 &5 &4\\
3 &4 &7 &6\\
4 &5 &8 &7
\end{bmatrix}\]

(iii) The final ingredient required for the assembly of the global system of equations is the so-called {\bf DME()} operator. This is actually the same {\bf IELCON()} array but translated into equation numbers assigned at each element. Computationally this involves a subroutine that runs over the {\bf IELCON()} array, reads the value of the nodal identifier and then extracts the assigned equation numbers from the boundary conditions array {\bf IBC()}. In this example the assembly operator consistent with the definition of the connectivities array corresponds to:

\[DME = \begin{bmatrix}
-1 &-1 &-1 &-1 &1 &2 &-1 &0\\
-1 &-1 &-1&-1 &{\bf 3} &4 &1 &2\\
-1 &0 &1 &2 &-1 &-1 &-1 &-1\\
1 &2 &3 &4 &-1 &-1 &-1 &-1
\end{bmatrix}\]

\paragraph*{A note on the assembly operator:}
The definition of the element connectivities and its subsequent translation into degrees of freedom is carried out according to the local or canonical element definition in the natural space(\cref{fig:locdof})

\begin{figure}[H]
\centering
\includegraphics[width=6cm]{localdof.pdf}
\caption{Local definition for a 4-noded element.}
\label{fig:locdof}
\end{figure}

As a result the $(i,j)$ entry in the $DME$ array corresponds to the global equation number associated with the local degree of freedom $j$ of the $i$ element. For instance the value of $3$ stored at position $(1,4)$ in the current {\bf DME()} operator indicates that the global equation $3$ corresponds to the local equation $4$ (column index) in element $1$ (row index).

The assembly process is then conducted by identifying the relation between the entries in each row of the {\bf DME()} operator and the list of local degrees of freedom for the reference element shown in \cref{fig:locdof}. Accordingly, for element 1 it follows that the assembly of row $4$ of the local stiffness matrix proceeds as follows:
\begin{align*}
K_{3,3}^G & \leftarrow  K_{3,3}^G + k_{4,4}^1 \\
K_{3,4}^G & \leftarrow  K_{3,4}^G + k_{4,5}^1 \\
K_{3,1}^G & \leftarrow  K_{3,1}^G + k_{4,6}^1
\end{align*}

\section{Summary: The finite element algorithm}
At this point it becomes evident that the algorithm for the displacements based finite element method reduces to the solution of a linear system of algebraic equations in the unknown nodal displacements $U^G$ resulting after assembling the contribution to the stiffness matrix $K^G$ and loads vector $RHS^G$ from all the elements in the mesh. This process can be summarized in the 4-steps pseudo-code described in \cref{algo:overall1}:

\begin{algorithm}[H]
\SetAlgoLined
\KwData{Finite element model}
\KwResult{Field function}
\BlankLine
PREPROCESSING (Reads the model and computes {\bf IBC()}, {\bf IELCON()} and {\bf DME()} arrays);\\
ASSEMBLY (Forms the system of equations $\left[K^G\right]\left\{U^G\right\}=\left\{RHS^G\right\}$);\\
SOLUTION (Finds $U^G$);\\
POSTPROCESSING (Scatter the global solution to the elements and computes element forces, strains, and stresses);\\
\caption{Global steps involved in the finite element algorithm}
\label{algo:overall1}
\end{algorithm}

In the pre-processing stage the code reads the input file and forms the arrays {\bf IBC()}, {\bf IELCON()} and {\bf DME()} which are required for the assembly process. Once the global system of equations is assembled and solved, the values of the nodal displacements are distributed to the corresponding elements in a process commonly referred to like scattering. This process is somehow inverse to the assembly operation. With the nodal displacements for each element at hand it is now possible to compute nodal forces together with strain and stress distributions inside the element. The assembly and solution process is given in \cref{algo:overall2}. In this algorithm it must be noticed that the local stiffness matrix for each $i$ element is obtained by the call to the local element subroutine {\bf UEL()}.

\begin{algorithm}[H]
\SetAlgoLined
\KwData{Finite element model}
\KwResult{Field function}
\BlankLine
READ {\bf IBC()} and {\bf IELCON()} arrays from input files ;\\
COMPUTE modified {\bf IBC()} array ;\\
COMPUTE {\bf DME()} operator (using {\bf IBC()} and {\bf IELCON()} arrays);\\
INITIALIZE Global stiffness matrix $K^G \leftarrow 0.0$;\\
(Start assembly)
\BlankLine
\For{$i \leftarrow 1$ to $Numel$}{
    Call UEL(element parameters;$k^i$) ;\\
    $K^G \leftarrow K^G+^i$ (Assemble each $k^i$ into $K^G$ according to the {\bf DME()} operator);\\
    $RHS^G \leftarrow RHS^G+rhs^i$ (Assemble each $rhs^i$ into $RHS^G$);\\
	\BlankLine
	}
\BlankLine
SOLVE ${K^G}{U^G} = RH{S^G}$;\\
Scatter nodal displacements solution to the elements
\caption{Details of the assembly and system solution process}
\label{algo:overall2}
\end{algorithm}

The details of the elemental subroutine {\bf UEL()} are given in \cref{algo:uel}. This process basically involves looping through the Gauss points and computing the required terms. The computation of the the strain-displacements matrix $B(r_j , s_j )$ and Jacobian determinant $\left\|J_j\right\|$ at each Gauss point is conducted by additional subroutines.

\newpage
\begin{algorithm}[H]
\SetAlgoLined
\KwData{Material parameters, nodal coordinates, applied loads}
\KwResult{$k^l$ and $rhs^l$}
\BlankLine
INITIALIZE $k^l \leftarrow 0.0$ ;\\
COMPUTE constitutive tensor $C_{ijkl}$ ;\\
FORM Gauss quadrature arrays $w()$ and $r() , s()$;\\
(Start loop through the Gauss points)
\BlankLine
\For{$j \leftarrow 1$ to $Ngpts$}{
    RETRIEVE $r_j$ , $s_j$ , $w_j$   ;\\
    COMPUTE $\left\|J_j\right\|$, $B(r_j , s_j )$ ;\\
    $k^l \leftarrow k^l+B^T C B \left\|J_j\right\| \left\|J_j\right\|$ ;\\
	\BlankLine
	}
\BlankLine
\caption{Details of the elemental {\bf UEL} subroutine}
\label{algo:uel}
\end{algorithm}

%\section{Sparse assembly}
%In Finite Elements is common to have stiffness and mass matrices that are sparse, i.e., matrices in which most of the elements are zero.
%
%
%For instance, in a regular mesh formed with bilinear quadrilaterals the number of nonzero entries is given by the expression
%\[\text{storage} = 9 n_x n_y - 9 n_x - 3 n_y + 4\, ,\]
%where $n_x$ is the number of nodes in the $x$ coordinate and $n_y$ is the number of nodes in the $y$ coordinate. Figure \ref{fig:sparse_storage} presents the needed storage for a mesh with $n_x=n_y$ for different sizes.
%\begin{figure}[H]
%\centering
%\includegraphics[width=6 in]{sparse_storage.pdf}
%\caption{Nonzero entries in a sparse matrix for a structured mesh of bilinear elements.}
%\label{fig:sparse_storage}
%\end{figure}

\paragraph*{Proposed problems}
\begin{enumerate}

\item \label{punto01} In the bi-linear element shown in \cref{fig:onemesh} the nodal displacements resulting from a finite element solution are indicated by the blue arrows. For the material parameters given in the figure find the nodal forces consistent with the element stresses. Assume plane strain behaviour.

\begin{figure}[H]
\centering
\includegraphics[height=6cm]{post.png}
\caption{One-element mesh for problem 1.}
\label{fig:onemesh}
\end{figure}

\item \label{punto02} For the mesh shown in the figure, with internal surfaces between elements 1-3 and 3-2 labelled $S_b$ and $S_c$ respectively, write the form of the global stiffness matrix resulting from the physical assembly. Explicitly formulate the force and displacement compatibility equations along both boundaries

\begin{figure}[H]
\centering
\includegraphics[height=6cm]{long.png}
\caption{Long mesh for problem 2.}
\label{fig:longmesh}
\end{figure}

\item \label{punto03} For the mesh shown in the figure propose different node numbering schemes and identify the resulting changes in the size of the half-band in the stiffness matrix. Assume that each element subroutine is full of $1$s.

\begin{figure}[H]
\centering
\includegraphics[height=6cm]{halfband.png}
\caption{Long mesh for problem 3.}
\label{fig:halfmesh}
\end{figure}

\item \label{punto04} A finite element model is conformed by an assemblage of three elements of the type shown in \cref{fig:cercha}

\begin{figure}[H]
\centering
\includegraphics[height=3cm]{truss.png}
\caption{Fundamental element}
\label{fig:cercha}
\end{figure}

and with a stiffness matrix (in the global coordinate system) given by:

\[
k = \begin{bmatrix}1&1&1&1\\1&1&1&1\\1&1&1&1\\1&1&1&1.
\end{bmatrix}
\]

On the other hand, it is known that global stiffness matrix has been assemble using the following operator

\[
DME = \begin{bmatrix}-1&-1&2&3\\-1&-1&1&-1\\1&-1&2&3\end{bmatrix}
\]

in which a value of $-1$ refers to a restrained degree of freedom and thus equal to zero. It is required to:

\begin{itemize}
\item Draw the complete model including its displacement boundary conditions.
\item Indicate the order of the global stiffness matrix for the complete structure.
\item Find the global stiffness matrix for the structure.
\end{itemize}

%%%%%
\item \label{punto05} Using SolidsPy compute the stress concentration factor defined according to:

\[SCF = \frac{\max\, \sigma_{yy}}{\text{promedio}\, \sigma_{yy}}\, \]

and introduced after drilling a hole in the following plates:

\begin{figure}[H]
	\centering
	\includegraphics[height=5 cm]{tema1.pdf}
	\includegraphics[height=5 cm]{tema2.pdf}
	\caption{Plates with holes.}
	\label{fig:tema1}
\end{figure}

In both cases the material parameters are defined in \cref{tab:comun}

\begin{table}[H]
    \centering
    \begin{tabular}{cc}
        \hline
        Parameter & Value \\
        \hline
        Young's modulus (Pa)    & $E =200 \times 10^9$ \\
        Poisson's ratio    & $\nu = 0.285$   \\
        Load (N/m)   & $S = 10^8$  \\
        \hline
    \end{tabular}
    \caption{Material parameters for the analysis defined in problem 2.}
    \label{tab:comun}
\end{table}

while the geometric parameters are those of \cref{tab:tema1}

\begin{table}[H]
	\centering
	\begin{tabular}{cccc}
		\hline
		$a$ & $b$ & $h$ & $w$ \\
		\hline
		20      & 40      & 50      & 100  \\
		\hline
	\end{tabular}
	\caption{Geometric parameters for the analysis defined in problem 2.}
	\label{tab:tema1}
\end{table}

%\inputminted[]{python}{src/engine.py}

%En este caso el factor de concentración de esfuerzos ($SCF$) se calcula como:
%
%
%\subsection*{Tema 2}
%\begin{figure}[H]
%	\centering
%	\includegraphics[height=5 cm]{tema2.pdf}
%	\caption{Geometrías y condiciones del tema 2.}
%	\label{fig:tema2}
%\end{figure}
%\begin{table}[H]
%	\centering
%	\begin{tabular}{ccccc}
%		\hline
%		Equipo & $a$ & $b$ & $h$ & $w$ \\
%		\hline
%        3    & 40      & 40      & 50      & 100  \\
%        10   & 20      & 40      & 50      & 200  \\
%        14   & 10      & 20      & 50      & 100  \\
%        16   & 20      & 40      & 50      & 100  \\
%        21   & 10      & 40      & 100     & 200  \\
%        24   & 10      & 40      & 100     & 300  \\
%        28   & 10      & 30      & 100     & 200  \\
%		\hline
%	\end{tabular}
%	\caption{Equipos y parámetros para el tema 2. Longitudes en milímetros. Considere las simetrías del problema para que esté cinemáticamente determinado.}
%	\label{tab:tema2}
%\end{table}
%
%En este caso el factor de concentración de esfuerzos ($SCF$) se calcula como:
%\[SCF = \frac{\max\, \sigma_{yy}}{\text{promedio}\, \sigma_{yy}}\, .\]
%
%\subsection*{Tema 3}
%\begin{figure}[H]
%	\centering
%	\includegraphics[height=4 cm]{tema3.pdf}
%	\caption{Geometrías y condiciones del tema 3.}
%	\label{fig:tema3}
%\end{figure}
%\begin{table}[H]
%	\centering
%	\begin{tabular}{ccccc}
%		\hline
%		Equipo & $a$ & $b$ & $h$ & $w$ \\
%		\hline
%		2   & 20      & 10      & 50      & 200  \\
%		6   & 10      & 10      & 50      & 200  \\
%		8   & 20      & 20      & 50      & 200  \\
%		13  & 10      & 20      & 50      & 200  \\
%		19  & 20      & 30      & 60      & 240  \\
%		25  & 20      & 40      & 60      & 240  \\
%		27  & 40      & 20      & 60      & 240  \\
%		\hline
%	\end{tabular}
%	\caption{Equipos y parámetros para el tema 3. Longitudes en milímetros.}
%	\label{tab:tema3}
%\end{table}
%
%En este caso el factor de concentración de esfuerzos ($SCF$) se calcula como:
%\[SCF = \frac{\max|\sigma_{xx}|}{\text{promedio}\, |\sigma_{xx}|}\, .\]
%
%\subsection*{Tema 4}
%\begin{figure}[H]
%	\centering
%	\includegraphics[height=4 cm]{tema4.pdf}
%	\caption{Geometrías y condiciones del tema 4.}
%	\label{fig:tema4}
%\end{figure}
%\begin{table}[H]
%	\centering
%	\begin{tabular}{ccccc}
%		\hline
%		Equipo & $a$ & $b$ & $h$ & $w$ \\
%        \hline
%        1   & 20      & 10      & 50      & 100  \\
%		7   & 20      & 10      & 50      & 200  \\
%		9   & 20      & 10      & 50      & 300  \\
%		15  & 10      & 10      & 50      & 100  \\
%		18  & 10      & 10      & 50      & 200  \\
%		20  & 10      & 10      & 50      & 300  \\
%		26  & 20      & 20      & 60      & 200  \\
%		\hline
%	\end{tabular}
%	\caption{Equipos y parámetros para el tema 4. Longitudes en milímetros.}
%	\label{tab:tema4}
%\end{table}
%
%En este caso el factor de concentración de esfuerzos ($SCF$) se calcula como:
%\[SCF = \frac{\max|\sigma_{xx}|}{\text{promedio}\, |\sigma_{xx}|}\, .\]

%%%%%

\end{enumerate}


\subsection{Final Score}

After retrieving the sets of \emph{PageRank} and \emph{TF-IDF} scores
for all users that discuss any query term, they are normalized to zero-average
and unitary variance so that they can be mixed together by means of a parameter, $\alpha$.
Finally both terms are combined, and the resulting score can be seen in Equation \ref{eq:finalscore}, where $s_t$ is the TF-IDF score for user $u$ and query $q$ and $s_p$ the PageRank score of $u$.

\begin{equation}
s(u,q) = \alpha * s_t(u,q) + (1 - \alpha) * s_p(u)
\label{eq:finalscore}
\end{equation}


\section{Constants}

We can prove that all the above constants belong to the
interpretations of their types.
%
\begin{theorem}{[Constants]}\label{thm:constant}
$c \in \interp{\constty{c}}$.
\end{theorem}
%
The Theorem trivially holds for more of the constants.
For example,
\newcommand\eqtype{\ensuremath{
	\tfun{x}{b^\lfinite}{
	\tfun{y}{b^\lfinite}{
	\tlref{v}{\tbool}{\lfinite}{v \Leftrightarrow x = y}
	}}
}}
$$= \in \interp{\eqtype}$$
as $\forall e_1, e_2, \evals{e_1}{d_1}, \evals{e_2}{d_2}\Rightarrow
			\evals{(e_1=e_2 \Leftrightarrow e_1= e_2)}{\etrue}
			\land \exists d. \evals{(e_1 = e_2)}{d}$

Here we prove that for any type $\tau$,
\efix{\tau} and \etfix{\tau} satisfy Theorem~\ref{thm:constant}.

Given the families of constants:
\begin{align*}
\ceval{\etfix{\tau}}{f} & \doteq \efun{n}{}{\efun{f}{}{\etfixn{\tau}{}{n}}}\\
%\ceval{\etfixf{\tau}{f}{??}}{n} &\doteq
%f\ n\ (\etfixn{\tau}{f}{n}\ f) \\
\ceval{\etfixn{\tau}{}{n}}{m} &\doteq
\efun{f}{}{
f\ m\ (\etfixn{\tau}{f}{m}\ f)} \\
\end{align*}
%
and their types
%
\begin{align*}
%\decr{\tau}{n} & \doteq \decrtypefull{\tau}{n}\\
\constty{\etfix{\tau}} &\doteq
	(\decrty{\tau})
	 \rightarrow
	\tfun{m}{\tnat^\lfinite}{\tau\sub{x}{m}}\\
%\constty{\etfixf{\tau}{f}{n}} &\doteq
%	 \etfixfty{\tau}{f}{n}\\
\constty{\etfixn{\tau}{f}{n}} &\doteq
	(\decrty{\tau})
	\rightarrow\adecrty{\tau}{n}\\
%\constty{\etfixfn{\tau}{f}{n}} &\doteq
%	 \etfixfty{\tau}{f}{n}\\
\end{align*}
we prove that the constants belong to the
meanings of their types:
%
\begin{theorem}{[Terminating Fixpoint]}\label{thm:fixpoint}
\begin{enumerate}
\item\label{nfix}$\forall n. \etfixn{\tau}{f}{n} \in \constty{\etfixn{\tau}{f}{n}}$
\item\label{tfix}$\etfix{\tau} \in \constty{\etfix{\tau}}$
\item\label{fix}$\efix{\tau} \in \constty{\efix{\tau}}$, if the result of $\tau$ is a \Div type.
\end{enumerate}
\end{theorem}
\begin{proof}
\begin{itemize}
\item \ref{nfix}.
We prove that for all
%$f \in \interp{\decrty{\tau}}$
appropriate $f$
and $ m \in \interp{\tref{v}{\tnat}{\lfinite}{v < n}}$,
$e \equiv \etfixn{\tau}{f}{n}\ f \ m \in \interp{\tau\sub{x}{m}}$
%
by induction on $n$.

For $n=0$,
it is trivial, as
there is no $m$ such that
$m \in \interp{\tref{v}{\tnat}{\lfinite}{v < 0}}$.

For the inductive step, $e$ reduces to
$$
\etfixn{\tau}{f}{n}\ f\ m
\hookrightarrow
\etfixfn{\tau}{f}{n}\ m
\hookrightarrow
f\ m\ (\etfixn{\tau}{f}{m}\ f)\\
$$
By IH, since $m < n$,
$\etfixn{\tau}{f}{m} \in \constty{\etfixn{\tau}{f}{m}}$,
so $f$ receives the appropriate arguments,
and returns the appropriate result that proves the theorem.
%
\item \ref{tfix}.
We prove that
for all appropriate $f$
% \\$f \in \interp{\decrty{\tau}}$
and     $ m \in \interp{\tnat^\lfinite}$,
$\etfix{\tau}\ f \ m \in \interp{\tau\sub{x}{m}}$.
%
Since $m \in \interp{\tref{v}{\tnat}{\lfinite}{v < m+1}}$
$$\etfixn{\tau}{f}{m+1}\ f \ m \in \interp{\tau\sub{x}{m}}$$
%
But operationally,
$\etfixn{\tau}{f}{m+1}\ f \ m$
and
$\etfix{\tau}\ f \ m$
behave equivalently, which proves the theorem.
\item \ref{fix}. The prove for
$\efix{\tau} \in \constty{\efix{\tau}}$.
is similar.
%
The only difference is that for the base case
\efixn{\tau}{0} should be defined to belong
into the interpretation of any type.
%
Thus, it is defined as a diverging expression
and the type of \efix{\tau} is constrainted
to $\tau$'s with potentially diverging result.
%
With refinement types we prove that the basic
\etfixn{\tau}{f}{0} operator
cannot be called, so we omit
the definition of this basic case.
\end{itemize}
\end{proof}
%


\section{Group operations, revisited}

In this section we revise \emph{Riordan group} concepts described in
\autoref{subsection:back:to:the:basics:riordan:group}, looking at them under
the light of $h$-characterization. In particular, we show how a Riordan arrays
written using the proposed characterization, allows us to build new arrays
respect to group operation $\cdot$ and to find their inverses.

\subsection{Inverting an array}

Let $\mathcal{R}\left(d(t),h(t)\right)$ be a Riordan array and let
$\mathcal{R}_{h(t)}\left(g(h(t)),h(t)\right)$ its $h$-characterization, for some
function $g$. %in the variable $h(t)$.

Using rules for inversion in the Riordan group,
\marginpar{using $\mathcal{R}_{h(t)}$
polymorphically as an array, remind \autoref{par:h:characterization:is:an:array:polymorphism}}
proceed as follow:
\begin{displaymath}
    \begin{split}
        \mathcal{R}_{h(t)}^{-1}\left(\frac{1}{g(h(\hat{h}(t)))},\hat{h}(t)\right)&=
        \left[\mathcal{R}^{-1}\left(\left.\frac{1}{g(h(y))},y\right) \right| y = \hat{h}(t) \right]\\
        &= \mathcal{R}_{\hat{h}(t)}^{-1}\left(k(\hat{h}(t)),\hat{h}(t)\right)
    \end{split}
\end{displaymath}
where function $k$ is defined as $k(y)=\frac{1}{g(h(y))}$,
therefore we got a new array $\mathcal{T}_{\hat{h}(t)}$ which is the $\hat{h}$-characterization
of $\mathcal{R}_{h(t)}^{-1}$ and, as the same time, of $\mathcal{R}^{-1}$.
\marginpar{again, $\hat{h}(t)$ plays the role of a \emph{variable},
so don't be tempted to say $h(\hat{h}(t))=t$ as in the normal course of things \ldots}
\\\\
For the sake of clarity we apply the previous derivation to build the inverse of $\mathcal{F}$,
the Fibonacci array.  To set the stage we need function $h$, take it from the definition:
\begin{displaymath}
    \begin{split}
        h(t)&=\frac{1-\sqrt{1-4t}}{2}\\
    \end{split}
\end{displaymath}
we need also function $g$, take it from the $h$-characterization $\mathcal{F}_{h(t)}$:
\begin{displaymath}
    \left[g(y)=\left.\frac{1}{1-y+2y^3-y^4} \right| y=h(t)\right]
\end{displaymath}
we're ready to apply:
\begin{displaymath}
    \left[\mathcal{F}^{-1}\left.\left(1-y-y^2,y\right) \right| y = \hat{h}(t) \right]=
    \mathcal{F}_{\hat{h}(t)}^{-1}\left(1-\hat{h}(t)-\hat{h}(t)^2,\hat{h}(t)\right)
\end{displaymath}
where function $\hat{h}$ is the compositional inverse of function $h$. Observe
that $1-y-y^2$ in the left array under variable constraint $y=\hat{h}(t)$, is
obtained by evaluating $\frac{1}{g(h(y))}$, considering $y$ as a variable:
using this abstraction, function $\hat{h}(t)$ cannot be used
\emph{functionally}, otherwise $g(h(\hat{h}(k)))=g(k)$, where $k=h(t)$
according to constraint under $g$ is defined. If that would be the case, it
yields a factorization in $h(t)$, not in $\hat{h}(t)$, as we would like to
have.

If the explicit definition for $\mathcal{F}^{-1}$ is desired, just plug $\hat{h}(t)=t-t^2$:
\begin{displaymath}
    \left(\mathcal{F}_{\hat{h}(t)}^{-1}\right)^{\stackrel{\hat{h}(t)}{\rightarrow}} =
        \mathcal{F}^{-1}\left(1-t+2t^3-t^4,t-t^2\right)
\end{displaymath}

\subsection{Multiplying two arrays}

Finally, let us tackle the product of two Riordan arrays.
Multiplication, the action of the Riordan group, between two elements
$\mathcal{A}(d_{\mathcal{A}}(t), h_{\mathcal{A}}(t))$ and
$\mathcal{B}(d_{\mathcal{B}}(t), h_{\mathcal{B}}(t))$ belonging to the group, is defined as:
\begin{displaymath}
    \mathcal{A}\mathcal{B} = \left(d_{\mathcal{A}}(t)d_{\mathcal{B}}(h_{\mathcal{A}}(t)),
        h_{\mathcal{B}}(h_{\mathcal{A}}(t))\right)
\end{displaymath}
Consider the $h_{\mathcal{A}}$-characterization of $\mathcal{A}$:
\begin{displaymath}
    \mathcal{A}_{h_\mathcal{A}(t)} \left(\gamma(h_{\mathcal{A}}(t)), h_{\mathcal{A}}(t)  \right)
\end{displaymath}
for some function $\gamma$, and the $h_{\mathcal{B}}$-characterization of $\mathcal{B}$:
\begin{displaymath}
    \mathcal{B}_{h_\mathcal{B}(t)} \left(\eta(h_{\mathcal{B}}(t)), h_{\mathcal{B}}(t)  \right)
\end{displaymath}
for some function $\eta$, respectively. Apply now the multiplication rule:
\begin{displaymath}
    \begin{split}
        & \left(\gamma(h_{\mathcal{A}}(t)), h_{\mathcal{A}}(t)  \right)
            \left(\eta(h_{\mathcal{B}}(t)), h_{\mathcal{B}}(t)  \right) \\
        &=\left[\left.\left(\gamma(y), y  \right)
            \left(\eta(h_{\mathcal{B}}(t)), h_{\mathcal{B}}(t)  \right) \right| y=h_{\mathcal{A}}(t) \right]\\
        &=\left[\left.\left(\gamma(y)\eta(h_{\mathcal{B}}(y)), h_{\mathcal{B}}(y)  \right) \right|
             y=h_{\mathcal{A}}(t) \right]\\
    \end{split}
\end{displaymath}
Stop here, it's enough to build the $h_{\mathcal{A}}$-characterization
of the product $\mathcal{A}\mathcal{B}$:
\begin{displaymath}
        \left[\left.\left(\Omega(y), h_{\mathcal{B}}(y)  \right) \right| y=h_{\mathcal{A}}(t) \right]
        =\big(\mathcal{A}\mathcal{B}\big)_{h_{\mathcal{A}}(t)}\left(
            \Omega(h_{\mathcal{A}}(t)), h_{\mathcal{B}}(h_{\mathcal{A}}(t))  \right)
\end{displaymath}
where $\left[\left.\Omega(y)= \gamma(y)\eta(h_{\mathcal{B}}(y))\right| y=h_{\mathcal{A}}(t) \right]$.
\\\\
Nonetheless,
\marginpar{The following can be skipped on a first reading}
from were we stopped, it's possible to factor $\mathcal{A}\mathcal{B}$ respect to $h_{\mathcal{B}}(t)$ in
a similar way, this time abstracting over $h_{\mathcal{B}}(y)$, remembering to track $y=h_{\mathcal{A}}(t)$:
\begin{displaymath}
    \begin{split}
        &\left[\left.\left(\gamma(y)\eta(h_{\mathcal{B}}(y)), h_{\mathcal{B}}(y)  \right) \right|
             y=h_{\mathcal{A}}(t) \right]\\
        &=\left.\left[\left.\left(\gamma(\hat{h}_{\mathcal{B}}(k))\eta(k), k  \right) \right|
             k=h_{\mathcal{B}}(y) \right]\right|_{y=h_{\mathcal{A}}(t)}\\
        &=\left.\left[\left.\left(\Theta(k), k  \right) \right| k=h_{\mathcal{B}}(y) \right]\right|_{y=h_{\mathcal{A}}(t)}\\
        &=\left.\big(\mathcal{A}\mathcal{B}\big)_{h_{\mathcal{B}}(y)}\left(
            \Theta(h_{\mathcal{B}}(y)), h_{\mathcal{B}}(y)  \right)\right|_{y=h_{\mathcal{A}}(t)}
    \end{split}
\end{displaymath}
\marginpar{in order to stop tracking $y=h_{\mathcal{A}}(t)$, we should write $(\Theta(k), k)$,
    where $k$ solves
    $h_{\mathcal{B}}(\hat{h}_{\mathcal{A}}(\hat{h}_{\mathcal{B}}(k)))=h_{\mathcal{B}}(t)$,
    but it doesn't supply an explicit variable substitution}
where $\left.\left[\left.\Theta(k)=\gamma(\hat{h}_{\mathcal{B}}(k))\eta(k) \right|
    k=h_{\mathcal{B}}(y) \right]\right|_{y=h_{\mathcal{A}}(t)}$.

Observe that for the latter factorization, it is necessary to compute function $\hat{h}_{\mathcal{B}}$,
the compositional inverse of function $h_{\mathcal{B}}$.
\\\\
For the sake of clarity we apply the previous derivation to the product $\mathcal{P}\mathcal{F}$, namely
we'll multiply Pascal and Fibonacci arrays, \emph{in the given order}, providing
both $\big(\mathcal{P}\mathcal{F}\big)_{h_{\mathcal{P}}(t)}$
and $\left.\big(\mathcal{P}\mathcal{F}\big)_{h_{\mathcal{F}}(y)}\right|_{y=h_{\mathcal{P}}(t)}$.

Recall $\mathcal{P}$ and $\mathcal{F}$ $h$-characterizations, they are respectively:
\begin{displaymath}
    \begin{split}
        &\mathcal{P}_{h_{\mathcal{P}}(t)}\left( 1+h_{\mathcal{P}}(t), h_{\mathcal{P}}(t) \right)
            \text{ where } h_{\mathcal{P}}(t) = \frac{t}{1-t}\\
        &\mathcal{F}_{h_{\mathcal{F}}(t)}\left( \frac{1}{1-h_{\mathcal{F}}(t)+
            2h_{\mathcal{F}}(t)^3-h_{\mathcal{F}}(t)^4}, h_{\mathcal{F}}(t) \right)
                \text{ where } h_{\mathcal{F}}(t) = \frac{1-\sqrt{1-4t}}{2}\\
    \end{split}
\end{displaymath}
and recognize the following functions under variable constraint (in the following, $y$
is a ``local'' variable, \emph{it's not shared} among the two definitions):
\begin{displaymath}
    \begin{split}
        &\left.\left[\gamma(y) = 1+y \right| y=h_{\mathcal{P}}(t) \right]\\
        &\left.\left[\eta(y) = \frac{1}{1-y+2y^3-y^4} \right| y=h_{\mathcal{F}}(t) \right]\\
    \end{split}
\end{displaymath}

Toward $\big(\mathcal{P}\mathcal{F}\big)_{h_{\mathcal{P}}(t)}$, compute $\Omega$ function:
\begin{displaymath}
    \left.\left[\Omega(y)= \gamma(y)\eta(h_{\mathcal{F}}(y))\right| y=h_{\mathcal{P}}(t) \right]
        = \left.\left[\Omega(y)= \frac{1+y}{1-y-y^2}\right| y=h_{\mathcal{P}}(t) \right]
\end{displaymath}
\marginpar{$\big(\mathcal{P}\mathcal{F}\big)_{h_{\mathcal{P}}(t)}$}
therefore answer the first question:
\begin{displaymath}
    \big(\mathcal{P}\mathcal{F}\big)_{h_{\mathcal{P}}(t)} \left(\frac{1+
        h_{\mathcal{P}}(t)}{1-h_{\mathcal{P}}(t)-h_{\mathcal{P}}(t)^2}, \frac{1-\sqrt{1-4h_{\mathcal{P}}(t)}}{2} \right)
\end{displaymath}

\marginpar{$\left.\big(\mathcal{P}\mathcal{F}\big)_{h_{\mathcal{F}}(y)}\right|_{y=h_{\mathcal{P}}(t)}$}
Toward $\left.\big(\mathcal{P}\mathcal{F}\big)_{h_{\mathcal{F}}(y)}\right|_{y=h_{\mathcal{P}}(t)}$, compute $\Theta$ function:

\begin{displaymath}
    \begin{split}
        &\left.\left[\left.\Theta(k)=\gamma(\hat{h}_{\mathcal{F}}(k))\eta(k) \right| k=h_{\mathcal{F}}(y) \right]\right|_{y=h_{\mathcal{P}}(t)}\\
        &= \left.\left[\left.\Theta(k)=\frac{1+k-k^2}{1-k+ 2k^3 - k^4} \right| k=h_{\mathcal{F}}(y) \right]\right|_{y=h_{\mathcal{P}}(t)}\\
    \end{split}
\end{displaymath}
where $\hat{h}_{\mathcal{F}}(y)=y-y^2$, therefore answer the second question:
\begin{displaymath}
    \left.\big(\mathcal{P}\mathcal{F}\big)_{h_{\mathcal{F}}(y)} \left(
        \frac{1+h_{\mathcal{F}}(y)-h_{\mathcal{F}}(y)^2}{1-h_{\mathcal{F}}(y)+ 2h_{\mathcal{F}}(y)^3 - h_{\mathcal{F}}(y)^4} ,
        h_{\mathcal{F}}(y) \right)\right|_{y=h_{\mathcal{P}}(t)}
\end{displaymath}

\marginpar{check substituting functions $h_{\mathcal{P}}$ and $h_{\mathcal{F}}$}
Little check plugging in function $h_{\mathcal{P}}(t)$:
\begin{displaymath}
    \left(\big(\mathcal{P}\mathcal{F}\big)_{h_{\mathcal{P}}(t)}\right)^{\stackrel{h_{\mathcal{P}}(t)}{\rightarrow}}
        = \big(\mathcal{P}\mathcal{F}\big)\left(\frac{1-t}{1-3t+t^2}, \frac{1}{2}-\frac{1}{2}\sqrt{\frac{1-5t}{1-t}} \right)
\end{displaymath}
on the other hand, plugging in function $h_{\mathcal{F}}(y)$, where $y=h_{\mathcal{P}}(t)$:
\begin{displaymath}
    \left.\left(\big(\mathcal{P}\mathcal{F}\big)_{h_{\mathcal{F}}(y)}\right)^{\stackrel{h_{\mathcal{F}}(y)}{\rightarrow}}\right|_{y=h_{\mathcal{P}}(t)}
        = \left.\big(\mathcal{P}\mathcal{F}\big)\left(\frac{1+y}{1-y-y^2}, \frac{1-\sqrt{1-4y}}{2} \right)\right|_{y=h_{\mathcal{P}}(t)}
\end{displaymath}


\section{From regular expressions to \eNFA{s}}

We left behind the regular expressions when we informally introduced
the transition diagrams for token recognition. Now let us show that
regular expressions when used in lexers to specify tokens can be
converted to \eNFA{s}, and therefore to DFAs. This proves that regular
languages are recognisable languages. Actually, it is possible to
prove that any \eNFA can be converted to a regular expression denoting
the same language, but we will not do so. Therefore, keep in mind that
the regular languages are the same as the recognisable languages. In
other words, the choice of using a regular expression or a finite
automaton is only a matter of convenience.

The construction we present here to build an \eNFA from a regular
expression is called \emph{Thompson's construction}. Let us first
associate an \eNFA to the basic regular expressions.
\begin{itemize}

  \item For the expression \(\epsilon\), construct the following NFA,
    where~\(i\) and~\(f\) are new states:
  \begin{center}
    \includegraphics[bb=48 710 135 730]{thompson_epsilon}
  \end{center}

  \item For \(a \in \Sigma\), construct the following NFA, where~\(i\)
    and~\(f\) are new states:
  \begin{center}
    \includegraphics[bb=48 710 135 730]{thompson_symbol}
  \end{center}

\end{itemize}
Now let us associate NFAs to complex regular expressions. In the
following, let us assume that~\(N(s)\) and~\(N(t)\) are the NFAs for
regular expressions \(s\)~and~\(t\).
\begin{itemize}

  \item For the regular expression \(st\), construct the following NFA
    \(N(st)\), where no new state is created:
\begin{center}
\includegraphics[bb=65 660 295 714]{thompson_conc}
\end{center}
  The final state of \(N(s)\) becomes a normal state, as well as the
  initial state of \(N(t)\). This way only remains a unique initial
  state~\(i\) and a unique final state~\(f\).

  \item For the regular expression \(s\) \disj \(t\), construct the
  following NFA \(N(s \, \text{\disj} \, t)\)
\begin{center}
\includegraphics[bb=65 590 272 715,scale=0.9]{thompson_disj}
\end{center}
where \(i\) and \(f\) are new states. Initial and final
states of \(N(s)\) and \(N(t)\) become normal.

  \item For the regular expression \(s\)\kleene, construct the following
    NFA \(N(s\text{\kleene})\), where~\(i\) and~\(f\) are new
    states:
\begin{center}
\includegraphics[bb=50 620 255 718]{thompson_kleene}
\end{center}
Note that we added two \(\epsilon\) transitions and that the initial
and final states of \(N(s)\) become normal states.

\end{itemize}
How do we apply these simple rules when we have a complex regular
expression, having many level of nested parentheses and other
constructs? Actually, the abstract syntax tree of the regular
expression directs the application of the rules. If the syntax tree
has the shape shown in \fig~\vref{fig:re_ast_conc},
\begin{figure}[b]
\centering
\subfloat[\(s \cdot t\)\label{fig:re_ast_conc}]{
  \includegraphics{re_ast_conc}
}
\qquad
\subfloat[\(s \,\text{\disj}\, t\)\label{fig:re_ast_disj}]{
\includegraphics{re_ast_disj}
}
\qquad
\subfloat[\(s\text{\kleene}\)\label{fig:re_ast_kleene}]{
\includegraphics{re_ast_kleene}
}
\caption{Three tree patterns for three regular expressions}
\end{figure}
then we construct first \(N(s)\), \(N(t)\) and finally \(N(s \cdot
t)\). If the syntax tree has the shape found in
\fig~\vref{fig:re_ast_disj}, then we construct first \(N(s)\),
\(N(t)\) and finally \(N (s \, \text{\disj} \, t)\). If the syntax
tree has the shape shown in \fig~\vref{fig:re_ast_kleene}, then we
construct first \(N(s)\) and finally \(N(s\text{\kleene})\). These
pattern matchings are applied first at the root of the abstract syntax
tree of the regular expression.


% proposal
%
% PROPOSAL: A brief plan of intention, strategy and accomplishments (about ½-1 page).
%
%----------------------------------------------------------------------------------------
%	PACKAGES AND OTHER DOCUMENT CONFIGURATIONS
%----------------------------------------------------------------------------------------
% none

\section{Proposal}
In a research study, a university medical center urology group was interested in the association between prostate-specific antigen (PSA) and a number of prognostic clinical measurements in men with advanced prostate cancer. Data were collected on 97 men who were about to undergo radical prostectomies. The data given has identifications numbers, and provides information on 8 other variables on each person. The 8 variables being: PSA Level, Cancer Volume, Weight, Age, Benign Prostatic Hyperplasia, Seminal Vesicle Invasion, Capsular Penetration, and Gleason Score. \par
With this available data set, I will carry out a complete logistic regression analysis by first creating a binary response variable Y, called high-grade-cancer, by letting Y=1 if Gleason Score equals 8, and Y=0 otherwise (i.e., if Gleason Score equals 6 or 7). Thus, the response of interest is high-grade-cancer (Y), and the pool of predictors include those previously mentioned. \par
My analysis will consider transformations of predictors, the inclusion of second-order predictors, analysis of residuals and influential observations, model selection, goodness of fit evaluation, and the development of an ROC curve. Additionally, I will discuss the determination of a prediction rule for determining whether the grade of disease is predicted to be high grade or not, model validation, and finally assess the strengths and weaknesses of my final model. \\


\section{The \findii algorithm---reuse of specification elements}
\Label{sec:findii}

In this section we specify \find in a slightly different way.
Our approach is motivated by a considerable number of closely related \acsl formulas
in the contract \specref{find} and the implementation \implref{find}.

\begin{lstlisting}[style=acsl-block]

    \exists integer i; 0 <= i < n        &&   a[i] == v;

    \forall integer i; 0 <= i < \result  ==>  a[i] != v;

    \forall integer i; 0 <= i < n        ==>  a[i] != v;

    \forall integer k; 0 <= k < i        ==>  a[k] != v;
\end{lstlisting}

Note that the first formula is the negation of the third one.

\subsection{The predicates \SomeEqual and \NoneEqual}

In order to be more explicit about the commonalities of these formulas
we define a predicate, called
\logicref{SomeEqual},
which describes the situation that there is a valid index \inl{i}
where~\inl{a[i]} equals~\inl{v}.

\input{Listings/SomeNone.acsl.tex}

We first remark that the \SomeEqual, its negation \NoneEqual
and the lemmas \NotSomeEqualNoneEqual and \NoneEqualNotSomeEqual are encapsulated
in the \emph{axiomatic block}
\logicref{SomeNone}.
This is a \emph{feeble} attempt to establish some modularization for the various predicates,
logic functions and lemmas.
We say \emph{feeble} because axiomatic blocks are, in contrast to \acsl \inl{module}s,
\emph{not} name spaces.
\acsl modules, however, are not yet implemented by \framac.

We also remark that both predicates come in overloaded versions.
The first of theses versions is a definition for array sections while the
second definition is for the case of complete arrays.

Note that we have provided a label, viz.\ \inl{A}, to the
predicate \SomeEqual.
Its purposes to express that the evaluation of the predicate depends on a memory state,
viz.\ the contents of \inl{a[0..n-1]}.
In general, we have to write

\begin{lstlisting}[style=acsl-block]

    \exists integer i; 0 <= i < n && \at(a[i],A) == v;
\end{lstlisting}

in order to express that we refer to the value \inl{a[i]} in
the program state~\inl{A}.
However, \acsl allows to abbreviate \inl{\\at(a[i],A)} by \inl{a[i]} if, as in
\SomeEqual or \NoneEqual, the label~\inl{A} is the only available label.
In particular, we have omitted the label in the overloaded versions for complete arrays.

%\clearpage

\subsection{Formal specification of \findii}

With the predicates \logicref{SomeEqual}
and \logicref{NoneEqual}
we are able to encapsulate all uses of the universal and existential
quantifiers in both the specification and implementation of \findii.

As a result, the revised contract \specref{findii} is more concise
than that of \specref{find}.
%
In particular, it can be seen immediately that the conditions in the
assumes clauses
of the two behaviors \inl{some} and \inl{none} are mutually
exclusive since
one is the literal negation of the other.
Moreover, the requirement that \find returns the smallest index can
also be expressed
using the \logicref{NoneEqual} predicate, as depicted with the last postcondition of
behavior \inl{some}.

\input{Listings/find2.h.tex}

We also enriched the specification of \find by user-defined names
(sometimes called \emph{labels}, too, the distinction to program state identifiers
being obvious)  to refer to
the \inl{requires} and \inl{ensures} clauses.
We highly recommend this practice in particular for more complex annotations.
For example, \framac can be instructed to verify only clauses with a
given name.

\clearpage

\subsection{Implementation of \findii}

The predicate \NoneEqual is also used in the loop annotation inside
the implementation of \implref{findii}.
Note that, as in the case of the specification, we use labels to name individual annotations.

\input{Listings/find2.c.tex}

%\clearpage


\chapter{Pairwise Graph Convolutional Networks for Interface Prediction}
\label{chap:methods}

The methods presented in this thesis were motivated by a desire to exploit the local structure around a residue when performing interface prediction.
The biological reasoning for this is that a residue's neighborhood influences its propensity to participate in an interface.
It was noted in Chapter \ref{chap:neuralnetworks} that convolutional neural networks are one way of detecting features in a local neighborhood, but they are limited to regular grids.
Unfortunately, proteins cannot naturally be represented as a regular grid, so convolution must be developed for a more natural representation: graphs.

\section{Proteins As Graphs}

An undirected, unweighted graph $G$ consists of a set of vertices, $V=\{v_1, v_2, ..., v_n\}$, and a set of eges, $E=\{e_1, e_2, ..., e_m\}$ where each edge is incident to two vertices and there is at most one edge between two vertices.
One way of representing proteins as graphs is to let each vertex represent a residue in the protein, and each edge represent the relationship between two residues.
Thus any information pertaining to a particular residue can be associated with the relevant vertex in the form of a feature vector.
The features used in this work are drawn from features used in prior interface prediction work \cite{minhas2014}.
Likewise, any information about the relationship between two residues can be associated with the relevant edge.
The edge features used here describe the distance between and relative orientation of two residues.
These edge features are defined between any two residues in the protein, so the graph is complete.
A detailed explanation of each feature is contained in Appendix \ref{appendix:features}.

This representation is an abstraction of the original protein to a well studied mathematical object, with two notable facts.
First, the graph itself does not rely on a coordinate system, as is the case when working with raw 3D positions.
Second, the features contained on the graph are also not tied to a coordinate system.
This is because they either describe coordinate free attributes of individual residues, or they describe relative spatial relationships between two residues.
These facts make a protein graph invariant to rotations or translations in space.
However, since the graph was constructed from points in 3D space, local neighborhoods of vertices can be defined using spatial proximity.
This is useful when designing convolutions that use a local neighborhood as a receptive field.
We now turn our attention to graph convolution

\section{Graph Convolution}
Recent years have seen increased attention to problems involving graph structured data, prompting developments in graph convolution to perform various tasks on those data~\cite{bronstein2016}.
These developments allow leveraging of deep learning techniques, which have shown great success for problems on regular grids.
Though each variant of graph convolution is tailored to suit the data and problem being addressed, there are some common approaches which merit describing generally.
These approaches generally fall into two categories: \emph{spectral} and \emph{spatial}.

\subsection{Spectral and Spatial Graph Convolution}
Spectral methods of graph convolution are based on linear functions in the frequency domain of a graph, defined using the laplacian operator $\mathcal{L}=I-D^{-1/2}WD^{-1/2}$, where $I$ is the identity matrix, $W$ is a matrix of edge weights, and $D$ is a diagonal matrix containing the degree of each vertex~\cite{bruna2013, henaff2015, kipf2016}.
Each filter in a spectral convolution implies a weighting of each frequency in the spectral decomposition of the graph~\cite{mallat2009}.
In physics, the Laplacian operator is used to model the process of diffusion.
In a similar way, the graph Laplacian can be thought of as modeling diffusion along the edges of the graph.

Spatial approaches instead directly model the diffusion process through a transition matrix, or by defining operations in a localized neighborhood of a central vertex~\cite{henaff2015, atwood2016}.
In the latter, each neighborhood constitutes a receptive field where a convolution operation is performed (see Figure \ref{fig:spatial_graph_conv}).
Spatial convolution commonly involves a vector of weights and takes a weighted sum of neighbors, much like a discrete convolution on a grid can be viewed as taking a weighted sum of pixels within the receptive field.
Spatial convolution is more directly analogous to grid based convolution as described in Chapter \ref{chap:neuralnetworks}.
There is a critical difference, however, in that receptive fields from different parts of a graph have no natural correspondence with one another.

\begin{figure}
	\centering
	%\begin{center}
	\includegraphics[width=\textwidth]{conv_graph.pdf}
	%\end{center}
	\caption{Receptive fields in a graph context, where each receptive field is defined around a central vertex. The result of convolution is applied to the central vertex in the receptive field.}
	\label{fig:spatial_graph_conv}
\end{figure}

\subsection{Receptive Field Correspondence in Spatial Convolution}

Grid receptive fields have defined positions, such as "upper left most" or "bottom middle".
Therefore weights can be applied consistently across receptive fields such that each weight is always applied to the same position.
With graphs, there is often no such correspondence between receptive fields, since vertices are inherently unordered and lack a specific position (recall their coordinate free nature).
The only well defined position is the center, but the problem persists in the neighbors.
See Figure \ref{fig:grid_vs_graph_rf}
In fact, even the number of neighbors may vary from one receptive field to another, depending on the definition of a local neighborhood.
Hence the consistent application of weights across receptive fields is only possible after these issues have been addressed.
This is typically done in one of two ways:

\begin{figure}
\includegraphics[width=0.9\textwidth]{grid_vs_graph_rf.png}
\caption{The difference between grid receptive fields and graph receptive fields with respect to correspondence. Grid receptive fields give rise to positions (denoted by color) which are consistent from one receptive field to another. Graphs have no positions other than the central vertex (in blue).}
\label{fig:grid_vs_graph_rf}
\end{figure}

\begin{enumerate}
	\item \emph{Imposed ordering of neighbors}. This approach establishes a correspondence between two receptive fields by ordering the neighbors in each and associating neighbors that share a common position.
	Ordering methods are either based on vertex characteristics, like degree and betweenness centrality, or domain specific knowledge ~\cite{niepert2016, duvenaud2015}.
	They typically require the number of neighbors in a receptive field to remain the same.
	This approach allows filter weights which are applied to a particular position in the ordering, which, it is assumed, has some significance across all receptive fields.
	If the imposed ordering is arbitrary, this approach has limited utility.
	Methods following this approach can be called \emph{ordered}.

	\item \emph{Identical treatment of neighbors}. This approach ignores the need to establish a correspondence between receptive fields and instead treats all neighbors identically.
	Rather than apply different weights to neighbors depending on their position in an ordering, the same weights are applied to each neighbor.
	This allows for different sizes of receptive fields and avoids choosing an ordering method, but lacks the ability to treat each neighbor uniquely.
	Methods following this approach can be called \emph{order-free}.
\end{enumerate}

Figure \ref{fig:correspondence_approaches} depicts both approaches.
Examples of each were evaluated for this thesis.
In addition, proposed convolutions were examined which attempt to incorporate the advantages of both ordered and order-free methods.
Below is a description of each existing method followed by the proposed methods presented in this thesis.

\begin{figure}
	\centering
	%\begin{center}
	\includegraphics[width=0.8\textwidth]{correspondence_approaches.pdf}
	%\end{center}
	\caption{Two approaches of establishing correspondence between the neighbors of receptive fields A and B. Central vertices are shown in blue and neighbors in green. The central vertices always correspond with one another. Left: neighbors are ordered and placed into correspondence based on position. Unique weights (\emph{$w_2$--$w_4$}) can then be applied to each position in the order. Right: neighbors are left unordered and treated identically. This requires that the same weights (\emph{$w_2$}) be used for all neighbors.}
	\label{fig:correspondence_approaches}
\end{figure}

\subsection{Diffusion Based Method}
As mentioned, spectral methods utilize the Laplacian operator, which is used to model diffusion processes.
Although no spectral methods were examined in this thesis, a spatial diffusion method was.
Atwood \& Towsley~\cite{atwood2016} proposed a Diffusion Convolutional Neural Network (DCNN) which converts a graph's weight matrix to a transition matrix by normalizing its rows.
The transition matrix $P$ is raised to successive exponents, generating a power series, with each power corresponding to all walks of length equivalent to that power.
For a maximum power of $L$, the activation of a vertex is:

\begin{equation}
h(x_i | W) = \sigma \bigg( \sum_{l=0}^L (W_{l\cdot} \odot P^l X ) \bigg),
\label{eq:diffusion}
\end{equation}

\noindent
where $W$ is a matrix of weights, $W_{l\cdot}$ is its $l^{th}$ row, $\odot$ denotes broadcasted elementwise multiplication, and $X$ is a matrix of all vertices, where each row is a vertex and each column a different feature.
For each power $l$, all vertices that have a random walk of length $l$ that end at the vertex of interest are summed together, weighted by the walk probability.
This sum is then multiplied elementwise with a weight vector and summed with the result of all other powers to produce the overall signal.
Unlike other spatial convolutions presented in this thesis, this method does not use a receptive field, instead relying on the similarity matrix to indicate proximity.

\subsection{Ordered Method}

Niepert, Ahmed, \& Kutzkov~\cite{niepert2016} described an ordered graph convolution, \emph{PATCHY-SAN}, which performs classification at the graph level.
This method constructs local neighborhoods and orders neighboring vertices according to a \emph{normalization} procedure.
The ordered vertices serve as the receptive field for convolution at the central vertex, which is also in the ordering.
If $(x_1, x_2, ... , x_k)$ is the ordered list of vertices, then convolution takes the form:

\begin{equation}
h(x_i | \{ W_{j} \}, b)= \sigma \bigg( \frac{1}{k} \sum_{j=1}^{k}(W_{j} x_j) + b \bigg),
\label{eq:patchysan}
\end{equation}

\noindent
where $W_j$ is the weight matrix for position $j$ in the ordering.
A natural ordering technique is one which places the central vertex first and orders its neighbors from least to greatest distance from the center.
In this way, one weight matrix is used for all central vertices, another for all nearest neighbors, etc.
The vertex ordering also imposes a lexicographic ordering on the neighborhood edges as well, so unique weights can be used to generate a signal from each edge.
Adding this term to the convolution generates:

\begin{equation}
h(x_i | \{ W_{j} \}, \{ W_{jk} \}, b)= \sigma \bigg( \frac{1}{k} \sum_{j=1}^{k}(W_{j} x_j) + \frac{1}{k^2} \sum_{j = 1}^{k-1} \sum_{l=j+1}^{k}(W_{jk} A_{jk})  + b \bigg),
\label{eq:patchysan_2e}
\end{equation}

\noindent
where $W_{jl}$ is the weight matrix associated with edge $(j, l)$ in the lexicographic ordering, and $A_{jk}$ is the feature vector associated with edge $(j, k)$.
Originally, Niepert et al. convolved vertices and edges separately and combine them in a subsequent layer.
Classification is then performed on the whole graph level.
For interface prediction however, the edges and vertex signals are averaged together and applied to the central vertex so that repeated graph convolutions may be performed without losing the graph structure.

\subsection{Order-Free Methods}
One of the simplest forms of order-free graph convolution was proposed by Duvenaud \& Maclaurin, et al.~\cite{duvenaud2015}, which was used for generating molecular \emph{fingerprints}.
This method uses a single set of weights for all vertices in the receptive field, including the central vertex:

\begin{equation}
h(x_i | W, b)= \sigma \bigg( W x_i +  \frac{1}{|\mathcal{N}_i|} \sum_{j \in \mathcal{N}_i} (W x_j) + b\bigg),
\label{eq:fingerprint}
\end{equation}

\noindent
where $\mathcal{N}_i$ is the index set of all neighbors of $x_i$.
In the original formulation, no bias term was included, and the sum was not divided by the neighborhood size.
However, when used for interface prediction, including the bias and normalization provided better overall results.

As mentioned, center vertices may be treated separately from the neighbors, even if all neighbors are treated identically.
Schlichtkrull \& Kipf~\cite{schlichtkrull2017} proposed such an approach called \emph{Relational Graph Convolutional Networks}, or R-GCN.
Their methods were developed for use in knowledge bases, graph structures where the vertices are named entities and the edges capture the many binary relationships between the entities.
To convolve a vertex, they take the neighborhood defined by each relation type and sum the signal from all the neighbors in that neighborhood.
The resultant signal is the sum of signals from each relation type.
For protein graphs, spatial proximity is the only method for determining neighborhoods that makes sense biologically, so there is only one neighborhood.
The adapted version of this convolution for use in interface prediction is:

\begin{equation}
h(x_i | W^\textsc{c}, W^\textsc{n}, b)= \sigma \bigg( W^\textsc{c} x_i + \frac{1}{|\mathcal{N}_i|} \sum_{j \in \mathcal{N}_i}(W^\textsc{n} x_j)  + b\bigg),
\label{eq:rgcn}
\end{equation}

\noindent
where separate weight matrices, $W^\textsc{c}$ and $W^\textsc{n}$, are used for the center and neighbors, respectively
In the original version, a different weight matrix was used for each of the many relation types.
To tie learning across all relation types, all weight matrices were simply linear combinations of a set of learnable basis weight matrices:

\begin{equation}
\begin{split}
W^\textsc{n} = \sum_{b} a^\textsc{n}_b V_b \\
W^\textsc{c} = \sum_{b} a^\textsc{c}_b V_b
\end{split}
\label{eq:rgcn_basis}
\end{equation}

\noindent
where $\{V_b\}$ is a set of basis matrices, and $a^\textsc{c}$ and $a^\textsc{n}$ are scalar weights that combine the basis matrices to create $W^\textsc{c}$ and $W^\textsc{n}$, respectively.
For interface prediction, R-GCNs were examined both with and without basis matrices.

The above order-free methods do not incorporate edge information.
Sch{\"u}tt, Arbabzadah, Chmiela, M{\"u}ller, \& Tkatchenko~\cite{schutt2017} proposed a version called \emph{Deep Tensor Neural Networks} (DTNN) which creates a signal for the neighbor vertices as well as the edges which connect them to the central vertex:

\begin{equation}
h(x_i | W, W^\textsc{n}, W^\textsc{e}, b^\textsc{n}, b^\textsc{e})= x_i + \frac{1}{|\mathcal{N}_i|} \sum_{j \in \mathcal{N}_i} \sigma \bigg[ W \bigg( (W^\textsc{n} x_j + b^\textsc{n}) \odot (W^\textsc{e} A_{ij} + b^\textsc{e}) \bigg) \bigg],
\label{eq:deep_tensor}
\end{equation}

\noindent
where $\odot$ denotes the elementwise product.
In this formulation, $W^\textsc{n}$ and $W^\textsc{e}$ transform vertices and edges respectively into a common space, and $W$ transforms their combination to have the same dimensionality as the input.
This convolution does not transform the input by any weight matrix, rather it can be viewed as updating the representation at $x_i$ using information from its neighbors.
This restricts representations to always have the same dimensionality, which is not generally required in convolutions.
Again, the normalization is omitted from the original formulation, but consistently improves performance when performing interface prediction.
Nevertheless, this method uniquely incorporates edge information, so that even while all neighbors use the same weight matrix, the information on their edges is used to differentiate them.
This concept is carried forward into the convolution methods proposed below.

\subsection{Proposed Order-Free Methods}
Like DTNN, these methods incorporate edge information to help differentiate neighbors, but also avoid imposing an arbitrary ordering on the neighbors in a receptive field.
This is accomplished by incorporating information from the edges between each neighbor and the central vertex.
Here are two variants of graph convolution which differ only in how the edge information is incorporated, denoted \emph{Sum Coupling} and \emph{Product Coupling}.
For a central vertex $i$ on the graph and a local neighborhood of vertices $\mathcal{N}_i$, the output of Sum Coupling graph convolution is:
\begin{equation}
h(x_i | W^\textsc{c}, W^\textsc{n}, W^\textsc{e}, b) = \sigma \bigg( W^{\textsc{c}} x_i + \frac{1}{|\mathcal{N}_i|}\sum_{j \in \mathcal{N}_i} (W^{\textsc{n}} x_j + W^{\textsc{e}} A_{ij}) + b \bigg),
\label{eq:sum_coupling}
\end{equation}
where $x_i$ is the feature vector associated with vertex $i$, $W^\textsc{c}$, $W^\textsc{n}$ and $W^\textsc{e}$ are weight matrices, and $b$ is a vector of biases.
Intuitively, this calculates an activation for the central vertex, each neighbor vertex, and each edge between a neighbor and the central vertex separately.
It is the inclusion of edge activations that allows each neighbor to be distinguished from the others on the basis of its relationship to the central vertex.
This variant is called Sum Coupling because the neighbor vertex and edge activations are added together.
Because of this, the direct association between a neighbor its edge is lost.
A variant which maintains the association is Product Coupling, which output is:
\begin{equation}
h(x_i | W^\textsc{c}, W^\textsc{n}, W^\textsc{e}, b) = \sigma \bigg( W^{\textsc{c}} x_i + \frac{1}{|\mathcal{N}_i|}\sum_{j \in \mathcal{N}_i} (W^{\textsc{n}} x_j \odot W^{\textsc{e}} A_{ij}) + b \bigg),
\label{eq:prod_coupling}
\end{equation}
where $\odot$ denotes the elementwise product between two vectors or matrices.
This allows a neighbor's influence on the overall activation to be modulated by its relationship to the central vertex.
For protein graphs, this means neighboring residues will contribute more or less to the overall activation, depending on their distance from and relative orientation to the central vertex, with the precise modulation determined by the edge activations.

In relation to prior methods, Sum Coupling is most similar to Fingerprint and R-GCN convolutions.
Unlike Fingerprint convolutions, it uses separate weights for central and neighbor vertices.
Unlike R-GCN convolutions, it does not use basis functions which tie together all weight matrices.
Also, Fingerprint and R-GCN convolutions do not generate signals from the edges.
Product Coupling is similar to DTNN convolutions in the way vertex and edge information is coupled together, but unlike DTNN it calculates a signal for the central vertex and allows the number of channels/features to change layer by layer.

The receptive fields are always defined around a central vertex, so the results of convolution can be applied to that vertex.
This retains the graph structure after each convolution, so convolutional layers are stackable, just like convolutions on grids.

A note on receptive fields: protein graphs are complete and embedded in a metric space, so we can define a receptive field using a fixed number of closest neighbors to the central vertex.
A receptive field can also be defined using a threshold $\delta>0$ such that all vertices closer to the central vertex than the threshold are included in the receptive field.
All neighbors in a receptive field are guaranteed to share an edge with the central vertex, allowing the application of equations (\ref{eq:sum_coupling}) and (\ref{eq:prod_coupling}).
For incomplete graphs, a receptive field can be defined as all vertices within $k$ hops of the central vertex.
If $k=1$, both Sum and Product Coupling can directly be applied.
If $k>1$, then Product Coupling can't be directly applied to neighbors more than 1 hop away from the center vertex, since they share no edge with the center.
Though there are ways to deal with this issue, they are not the focus of this thesis.

Lastly, to assess the benefit that incorporating information from neighboring residues has on classification performance, the \emph{No Convolution} variant is defined as:

\begin{equation}
h(x_i | W^\textsc{c}, b)= \sigma \bigg( W^{\textsc{c}} x_i + b \bigg),
\label{eq:no_conv}
\end{equation}

\noindent
which excludes all neighbors.
Note the similarity to Equation (), except that in this case the output is just for a single vertex rather than for the entire layer.

\section{Pairwise Neural Network Architecture}
These graph convolution operations allow the detection of local patterns on a single graph, and produce a new representation at each vertex.
Partner specific protein interaction, however, requires classifying pairs of residues in different proteins (vertices in different graphs), which is equivalent to making predictions on vertices in the product graph.
Such predictions are made using a pairwise neural network architecture.

A pairwise architecture consists first of two identical convolutional modules, each responsible for generating the representation for one of the proteins in the pair.
A key requirement for the pairwise architecture is symmetry, since the prediction for a pair of residues should be the same irrespective of which leg is used for which protein.
To ensure symmetry in the convolution layers, weights are shared between layers in the different modules.
The merge layer then combines the vertex representations from one graph with the vertex representations from the other into pairs.
To maintain symmetry, this merge process should also be symmetric.
For example, the elementwise sum, elementwise product, and outer-product are all commutative and therefore produce symmetric output.
Another option is to combine pairs asymmetrically (e.g. concatenate the two representations together), but then average the predictions from each ordering of the pair.
Finally, the combined representation for each pair of residues is passed through a number of fully connected layers.
The data are represented as pairs of residues at this point.
Theoretically, graph convolution could be performed at this stage as well, this time in the product graph.
However, the computational and memory requirements of doing so prove prohibitive, since the number of convolutions and the number of neighbors in each convolution increases quadratically in the graph size.
Hence the work in this thesis performs no convolution after merging.
The final layer has a single output for each pair indicating the prediction for that pair.
See Figure \ref{fig:pairwise_arch1} for a graphical depiction.

This output is converted to a binary class prediction vector using a softmax function,

\begin{equation}
\text{softmax}(x) = \bigg[ \frac{e^{-x}}{e^{x} + e^{-x}} , \frac{e^{x}}{e^{x} + e^{-x}} \bigg],
\label{eq:softmax}
\end{equation}

\noindent
the elements of which can be interpreted as class probabilities for the negative (non-interfacial) and positive (interfacial) classes, respectively.
This output is compared to a one-hot label vector indicating whether \big($[0, 1]$\big) or not\big($[1, 0]$\big) the pair constitute part of the true interface.

\begin{figure}
	\includegraphics[width=\textwidth]{pairwise_network4.pdf}
	\caption{A pairwise neural network architecture that takes two protein graphs as input. Each leg contains one or more convolutional layers. The resultant graphs are then merged to create representations of residue pairs. After more fully connected layers, a final classification is performed for each pair.}
	\label{fig:pairwise_arch1}
\end{figure}

This chapter has presented protein graphs, graph convolution operations, and pairwise neural network architectures, all of which are components in this thesis' approach to partner specific protein interface prediction.
Chapter \ref{chap:experiments} describes the experiments that were performed.


\section{Solution}
\label{sec:solution}

Our goal is now to find the policy $\pi$, which is an action to execute for each state, that will minimize the total cost on the time horizon $\mathcal{T}$.
We now define the Q-values of the MDP for each action $a$, state $k$, and policy $\pi$, which is defined as the expected cost of applying action $a$ at stage $k$ and then relying on policy $\pi$.

The basics of Reinforcement Learning is to apply stochastic approximation on the expectation of the cost to obtain algorithm can thus be written as
\[
  Q(k,a) \leftarrow Q(k,a) + \alpha(t) \left( c(a|k) + \underset{j\in \mathcal{U}(k')}{\text{min}} \{Q(k',j)\} + Q(k,a) \right)
\]
where $t$ represents an iteration variable and where alpha $t$ is a coefficient which decreases with $t$.

As we can see, this algorithm is model free in the sense that it is not required to know the probability distribution in advance.


    \filetitle{gamma}{Create function proportional to log of gamma distribution}{logdist/gamma}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
F = logdist.gamma(Mean,Std)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\item
  \texttt{Mean} {[} numeric {]} - Mean of the gamma distribution.
\item
  \texttt{Std} {[} numeric {]} - Std dev of the gamma distribution.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{F} {[} function\_handle {]} - Function handle returning a
  value proportional to the log of the gamma density.
\end{itemize}

\paragraph{Description}\label{description}

See \href{logdist/Contents}{help on the logdisk package} for details on
using the function handle \texttt{F}.

\paragraph{Example}\label{example}


\subsection{Convolutional Layer}

The convolutional layer - referred to as conv2d - receives an input data tensor $X_{ci} $ of size $[H, W, C_{in}]$ and outputs a data tensor of size $[H,W,C_{out}]$. Here $H$ and $W$ denote the height and width of the convolutional kernel. So every input tensor is transferred to each single convolutional computation channel, which is responsible for computing a \emph{single} output channel. Each convolutional channel is parameterized with its own set of values. Those values are implemented as constants in VHDL, which has the advantages of greater optimization options by the compiler as well as reduced data movement. Therefore template entities are created for the convolution channel- and layer entities where the weights are then populated via a Python script.

\begin{figure}[hb]
	\centering
	\includegraphics[width=0.6\textwidth]{img/convolution.pdf}
	\caption[Convolution Operation]{Convolution Operation. The convolutional kernel (red-dashed-square) is shifted over the input data. The currently processed data is shown in blue and the only three pixels (violett) need to be loaded for the next convolution because of data reuse. Note that for the convolution operation the image must be padded (in our case by zeros) to sustain image dimensions. Depth channels omitted for illustration purposes.}
	\label{fig:hw-conv-operation}
\end{figure}

The principal operation can be shown in Figure~\ref{fig:hw-conv-operation}. Here the convolutional kernel slides over the input data from left to right, top to bottom. It can be seen that most of the $6$ of $9$ input pixels (of each input channel) can be reused. This is done via shift registers and because all weights of the kernel remain unchanged for the operation this results in minimum data movement. Note that for the convolutional operations the input data must be padded in order to sustain the image dimensions. If a new line occurs, two cycles are required to load a new 3x3 kernel instead of one as an additional 3x1 vector needs to be loaded from memory.

\begin{figure}[h]
	\centering
	\begin{subfigure}[t]{0.5\textwidth}
		\centering
		% Constrain the height instead of the widht so the images are equally aligned at the top
		\includesvg[height=2.5in]{img/inkscape/conv2d.svg}
		\caption[Conv2d block diagram.]{Conv2d block diagram. For each output channel a conv\_channel module is used. $k$ indicates the number of output channels.}
		\label{fig:conv2d}
	\end{subfigure}%
	~
	\begin{subfigure}[t]{0.5\textwidth}
		\centering
		\includesvg[height=2.5in]{img/inkscape/conv-channel.svg}
		\caption[conv\_channel block diagram.]{conv\_channel block diagram. For each input channel a kernel\_3x3 module is used. $n$ indicates the number of input channels.}
		\label{fig:conv-channel}
	\end{subfigure}
	\caption{Block diagram of the Convolutional Layer}
	\label{fig:hw-layer-conv}
\end{figure}

In Figure~\ref{fig:conv2d} the block diagram of the top-level conv2d module is shown. It consists of $k$ conv\_channel modules to realise $k$ output channels. All conv\_channel modules recieve the same input vector $X_{c_i}$.
The internal structure of a conv\_channel module is shown in Figure \ref{fig:conv-channel}. It uses $n$ kernel\_3x3 modules to realise $n$ input channels. All kernel\_3x3 modules get a different input vector $X_{c_{i1}}$ to $X_{c_{in}}$ which are $3 \times 3$ input matrices. All kernel outputs are summed up to one final value of length BIT\_WIDTH\_OUT.

\subsubsection{Interface}

\begin{itemize}
	\item Input interface connected to shift register, which consists of a $n \cdot 3 \times 3$ vector of values of length BIT\_WIDTH\_IN, in which $n$ is the number of input channels.
	\item Output interface connected to the pooling layer, which is a vector of $m$ values of length BIT\_WIDTH\_OUT, in which $m$ is the number of output channels.
\end{itemize}
Both input and output interfaces have ready, last and valid signals to control the flow of data.

\subsubsection{Parameter}

\begin{table}[hb]
	\centering
	\begin{tabular}{lcc}
		\toprule
		Parameter & VHDL Datatype & Type \\
		\midrule
		 BIT\_WIDTH\_IN & integer & Generic\\
		 BIT\_WIDTH\_OUT & integer & Generic \\
		 INPUT\_CHANNELS & integer & Generic\\
 	 	 OUTPUT\_CHANNELS & integer & Generic \\
		\bottomrule
	\end{tabular}
\end{table}

\subsubsection*{Convolution Channel}

\textbf{Interface}
\begin{itemize}
	\item Input interface, same as conv2d.
	\item Output interface connected to the pooling layer, which is a value of length BIT\_WIDTH\_OUT.
\end{itemize}

\textbf{Parameter}
\begin{itemize}
 	\item BIT\_WIDTH\_IN : integer
 	\item KERNEL\_WIDTH\_OUT : integer, output bit width of the kernel\_3x3 module
 	\item BIT\_WIDTH\_OUT: integer
 	\item N: integer, number of kernels
 	\item OUTPUT\_MSB: integer, defines which of the $n$=BIT\_WIDTH\_OUT bits is the most significant bit
 	\item BIAS: integer, currently unused as bias seems to not be very important in the convolutional layers
\end{itemize}

\subsubsection*{Kernel-3x3}

This modules performs the convolution of a single kernel with a single input image patch. This is a multiplication of - in our case - 9 values of length BIT\_WIDTH\_IN with their respective weights which are defined in an array that can be set with a generic. The multiplication results are then added up in an adder tree. The weights are specified in a single generic constant array in row-major notation.

\subsubsection{Interface}
\begin{itemize}
	\item Input interface, a vector of 9 values of length BIT\_WIDTH\_IN.
	\item Output interface, same as conv\_channel.
\end{itemize}

\subsubsection{Parameter}
\begin{itemize}
	\item BIT\_WIDTH\_IN: integer
	\item BIT\_WIDTH\_OUT: integer
	\item WEIGHT: array of 9 integers
	\item WEIGHT\_WIDTH: integer
\end{itemize}


\section{Ring theory}\label{sec:ring_theory}

As discussed in \fullref{rem:additive_magma}, commutative and non-commutative groups are quite different despite having similar definitions. Rings are extensions of \hyperref[def:abelian_group]{Abelian groups}, which allow multiplication with more than members of \( \BbbZ \).

For commutative rings, this second operation is often truly an extension of \fullref{def:magma/exponentiation} to arbitrary ring elements. For noncommutative ring, this second operation is usually given by function composition.