\section{Computing moments from the master equation}\label{supp_moments}

In this section we will compute the moment equations for the distribution  $P(m,
p)$. Without lost of generality here we will focus on the three-state
regulated promoter. The computation of the moments for the two-state promoter
follows the exact same procedure, changing only the definition of the matrices
in the master equation.

\subsection{Computing moments of a distribution}

(Note: The Python code used for the calculations presented in this section can
be found in the
\href{https://www.rpgroup.caltech.edu//chann_cap/software/moment_dynamics_system.html}{following
link} as an annotated Jupyter notebook)

To compute any moment of our chemical master equation (\eref{eq_cme_matrix})
let us define a vector
\begin{equation}
  \ee{\bb{m^x p^y}} \equiv (\ee{m^x p^y}_A, \ee{m^x p^y}_I, \ee{m^x p^y}_R)^T,
\end{equation}
where $\ee{m^x p^y}_S$ is the expected value of $m^x p^y$ in state $S \in \{A,
I, R\}$ with $x, y \in \mathbb{N}$. In other words, just as we defined the
vector $\PP(m, p)$, here we define a vector to collect the expected value of
each of the promoter states. By definition, these moments $\ee{m^x p^y}_S$ are
computed as
\begin{equation}
  \ee{m^x p^y}_S \equiv \sum_{m=0}^\infty \sum_{p=0}^\infty m^x p^y P_S(m, p).
  \label{seq_mom_def}
\end{equation}
To simplify the notation, let $\sum_x \equiv \sum_{x=0}^\infty$. Since we are
working with a system of three ODEs, one for each state, let us define the
following operation:
\begin{equation}
  \ee{\bb{m^x p^y}} =
  \smp m^x p^y \PP(m, p) \equiv
  \begin{bmatrix}
    \smp m^x p^y P_A(m, p)\\
    \smp m^x p^y P_I(m, p)\\
    \smp m^x p^y P_R(m, p)\\
  \end{bmatrix}.
\end{equation}

With this in hand we can then apply this sum over $m$ and $p$ to
\eref{eq_cme_matrix}. For the left-hand side we have
\begin{equation}
  \smp m^x p^y \dt{\PP(m, p)} = \dt{}\left[ \smp m^x p^y \PP(m, p) \right],
  \label{seq_sum_mom}
\end{equation}
where we made use of the linearity property of the derivative to switch the
order between the sum and the derivative. Notice that the right-hand side of
\eref{seq_sum_mom} contains the definition of a moment from \eref{seq_mom_def}.
That means that we can rewrite it as
\begin{equation}
  \dt{}\left[ \smp m^x p^y \PP(m, p) \right] = \dt{\bb{\ee{m^x p^y}}}.
\end{equation}

Distributing the sum on the right-hand side of \eref{eq_cme_matrix} gives
\begin{equation}
  \begin{aligned}
    \dt{\bb{\ee{m^x p^y}}} &=
    \Km \smp m^x p^y \PP(m, p)\\
    &- \Rm \smp m^x p^y \PP(m, p) + \Rm \smp m^x p^y \PP(m-1, p)\\
    &- \Gm \smp (m) m^x p^y \PP(m, p) + \Gm \smp (m + 1) m^x p^y \PP(m + 1, p)\\
    &- \Rp \smp (m) m^x p^y \PP(m, p) + \Rp \smp (m) m^x p^y \PP(m, p - 1)\\
    &- \Gp \smp (p) m^x p^y \PP(m, p) + \Gp \smp (p + 1) m^x p^y \PP(m, p + 1).
  \end{aligned}
  \label{seq_master_sum}
\end{equation}

Let's look at each term on the right-hand side individually. For the terms in
\eref{seq_master_sum} involving $\PP(m, p)$ we can again use \eref{seq_mom_def}
to rewrite them in a more compact form. This means that we can rewrite the
state transition term as
\begin{equation}
  \Km \smp m^x p^y \PP(m, p) = \Km \bb{\ee{m^x p^y}}.
\end{equation}
The mRNA production term involving $\PP(m, p)$ can be rewritten as
\begin{equation}
  \Rm \smp m^x p^y \PP(m, p) = \Rm \bb{\ee{m^x p^y}}.
\end{equation}
In the same way the mRNA degradation term gives
\begin{equation}
  \Gm \smp (m) m^x p^y \PP(m, p) = \Gm \bb{\ee{m^{(x + 1)} p^y}}.
\end{equation}
For the protein production and degradation terms involving $\PP(m, p)$ we have
\begin{equation}
  \Rp \smp (m) m^x p^y \PP(m, p) = \Rp \bb{\ee{m^{(x + 1)} p^y}},
\end{equation}
and
\begin{equation}
  \Gp \smp (p) m^x p^y \PP(m, p) = \Gp \bb{\ee{m^x p^{(y + 1)}}},
\end{equation}
respectively.

For the sums terms in \eref{seq_master_sum} involving $\PP(m \pm 1, p)$ or
$\PP(m, p \pm 1)$ we can reindex the sum to work around this mismatch. To be
more specific let's again look at each term case by case. For the mRNA
production term involving $\PP(m-1, p)$ we define $m' \equiv m - 1$. Using this
we write
\begin{equation}
  \Rm \smp m^x p^y \PP(m-1, p) =
  \Rm \sum_{m' = -1}^\infty \sum_p (m' + 1)^x p^y \PP(m', p).
\end{equation}
Since having negative numbers of mRNA or protein doesn't make physical sense
we have that $\PP(-1, p) = 0$. Therefore we can rewrite the sum starting from 0
rather than from -1, obtaining
\begin{equation}
  \Rm \sum_{m' = -1}^\infty \sum_p (m' + 1)^x p^y \PP(m', p) =
  \Rm \sum_{m'=0}^\infty \sum_p (m' + 1)^x p^y \PP(m', p).
  \label{seq_reindex}
\end{equation}
Recall that our distribution $\PP(m, p)$ takes $m$ and $p$ as numerical inputs
and returns a probability associated with such a molecule count.  Nevertheless,
$m$ and $p$ themselves are dimensionless quantities that serve as indices of how
many molecules are in the cell. The distribution is the same whether the
variable is called $m$ or $m'$; for a specific number, let's say $m = 5$, or $m'
= 5$, $\PP(5, p)$ will return the same result. This means that the variable name
is arbitrary, and the right-hand side of
\eref{seq_reindex} can be written as
\begin{equation}
  \Rm \sum_{m'=0}^\infty \sum_p (m' + 1)^x p^y \PP(m', p) =
  \Rm \bb{\ee{(m+1)^x p^y}},
\end{equation}
since the left-hand side corresponds to the definition of a moment.

For the mRNA degradation term involving $\PP(m + 1, p)$ we follow a similar
procedure in which we define $m' = m + 1$ to obtain
\begin{equation}
  \Gm \smp (m + 1) m^x p^y \PP(m + 1, p) =
  \Gm \sum_{m' = 1}^\infty \sum_p m' (m' - 1)^x p^y \PP(m', p).
\end{equation}
In this case since the term on the right-hand side of the equation is multiplied
by $m'$, starting the sum over $m'$ from 0 rather than from 1 will not affect
the result since this factor will not contribute to the total sum. Nevertheless
this is useful since our definition of a moment from \eref{seq_mom_def} requires
the sum to start at zero. This means that we can rewrite this term as
\begin{equation}
  \Gm \sum_{m' = 1}^\infty m' \sum_p (m' - 1)^x p^y \PP(m', p) =
  \Gm \sum_{m' = 0}^\infty m' \sum_p (m' - 1)^x p^y \PP(m', p).
\end{equation}
Here again we can change the arbitrary label $m'$ back to $m$ obtaining
\begin{equation}
  \Gm \sum_{m' = 0}^\infty m' \sum_p (m' - 1)^x p^y \PP(m', p) =
  \Gm \bb{\ee{m (m - 1)^x p^y}}.
\end{equation}

The protein production term involving $\PP(m, p - 1)$ can be reindexed by
defining $p' \equiv p - 1$. This gives
\begin{equation}
  \Rp \smp (m) m^x p^y \PP(m, p - 1) =
  \Rp \sum_m \sum_{p'=-1}^\infty m^{(x + 1)} (p + 1)^y \PP(m, p').
\end{equation}
We again use the fact that negative molecule copy numbers are assigned with
probability zero to begin the sum from 0 rather than -1 and the arbitrary nature
of the label $p'$ to write
\begin{equation}
  \Rp \sum_m \sum_{p'=0}^\infty m^{(x + 1)} (p + 1)^y \PP(m, p') =
  \Rp \bb{\ee{m^{(x + 1)} (p + 1)^y}}.
\end{equation}
Finally, we take care of the protein degradation term involving $\PP(m, p + 1)$.
As before, we define $p' = p + 1$ and substitute this to obtain
\begin{equation}
  \Gp \smp (p + 1) m^x p^y \PP(m, p + 1) =
  \Gp \sum_m \sum_{p'=1}^\infty (p') m^x (p' - 1)^y \PP(m, p').
\end{equation}
Just as with the mRNA degradation term, having a term $p'$  inside the sum
allows us to start the sum over $p'$ from 0 rather than 1. We can therefore
write
\begin{equation}
  \Gp \sum_m \sum_{p'=0}^\infty (p') m^x (p' - 1)^y \PP(m, p') =
  \Gp \bb{\ee{m^x p (p - 1)^y}}.
\end{equation}

Putting all these terms together we can write the general moment ODE. This is
of the form
\begin{equation}
  \begin{aligned}
    \dt{\bb{\ee{m^x p^y}}} &=
    \Km \bb{\ee{m^x p^y}}
    \text{  (promoter state transition)}\\
    &- \Rm \bb{\ee{m^x p^y}} + \Rm \bb{\ee{(m+1)^x p^y}}
    \text{  (mRNA production)}\\
    &- \Gm \bb{\ee{m^{(x + 1)} p^y}} + \Gm \bb{\ee{m (m - 1)^x p^y}}
    \text{  (mRNA degradation)}\\
    &- \Rp \bb{\ee{m^{(x + 1)} p^y}} + \Rp \bb{\ee{m^{(x + 1)} (p + 1)^y}}
    \text{  (protein production)}\\
    &- \Gp \bb{\ee{m^x p^{(y + 1)}}} + \Gp \bb{\ee{m^x p (p - 1)^y}}
    \text{  (protein degradation)}.
  \end{aligned}
  \label{seq_mom_ode}
\end{equation}

\subsection{Moment closure of the simple-repression distribution}

A very interesting and useful feature of \eref{seq_mom_ode} is that for a given
value of $x$ and $y$ the moment $\bb{\ee{m^x p^y}}$ is only a function of lower
moments. Specifically $\bb{\ee{m^x p^y}}$ is a function of moments
$\bb{\ee{m^{x'} p^{y'}}}$ that satisfy two conditions:
\begin{equation}
  \begin{aligned}
    &1) y' \leq y,\\
  &2) x' + y' \leq x + y.
  \end{aligned}
  \label{seq_mom_conditions}
\end{equation}

To prove this we rewrite \eref{seq_mom_ode} as
\begin{equation}
  \begin{aligned}
    \dt{\bb{\ee{m^x p^y}}} &=
    \Km \bb{\ee{m^x p^y}}\\
    &+ \Rm \bb{\ee{p^y \left[ (m + 1)^x -m^x \right]}}\\
    &+ \Gm \bb{\ee{m p^y \left[ (m - 1)^x - m^x \right]}}\\
    &+ \Rp \bb{\ee{m^{(x + 1)} \left[ (p + 1)^y - p^y \right]}}\\
    &+ \Gp \bb{\ee{m^x p \left[ (p - 1)^y - p^y \right]}},
    \label{seq_mom_ode_factorized}
  \end{aligned}
\end{equation}
where the factorization is valid given the linearity of expected values. Now the
objective is to find the highest moment for each term once the relevant
binomial, such as $(m-1)^x$, is expanded. Take, for example, a simple case in
which we want to find the second moment of the mRNA distribution. We then set $x
= 2$ and $y = 0$. \eref{seq_mom_ode_factorized} then becomes
\begin{equation}
  \begin{aligned}
    \dt{\bb{\ee{m^2 p^0}}} &=
    \Km \bb{\ee{m^2 p^0}}\\
    &+ \Rm \bb{\ee{p^0 \left[ (m + 1)^2 - m^2 \right]}}\\
    &+ \Gm \bb{\ee{m p^0 \left[ (m - 1)^2 - m^2 \right]}}\\
    &+ \Rp \bb{\ee{m^{(2 + 1)} \left[ (p + 1)^0 - p^0 \right]}}\\
    &+ \Gp \bb{\ee{m^2 p \left[ (p - 1)^0 - p^0 \right]}}.
  \end{aligned}
\end{equation}
Simplifying this equation gives
\begin{equation}
    \dt{\bb{\ee{m^2}}} =
    \Km \bb{\ee{m^2}}
    + \Rm \bb{\ee{\left[ 2m + 1 \right]}}
    + \Gm \bb{\ee{\left[- 2m^2 + m \right]}}.
    \label{seq_second_mom_mRNA}
\end{equation}

\eref{seq_second_mom_mRNA} satisfies both of our conditions. Since we set $y$ to
be zero, none of the terms depend on any moment that involves the protein number,
therefore $y' \leq y$ is satisfied. Also the highest moment in
\eref{seq_second_mom_mRNA} also satisfies $x' + y' \leq x + y$ since the second
moment of mRNA doesn't depend on any moment higher than $\bb{\ee{m^2}}$. To
demonstrate that this is true for any $x$ and $y$ we now rewrite
\eref{seq_mom_ode_factorized}, making use of the binomial expansion
\begin{equation}
  (z \pm 1)^n = \sum_{k=0}^n {n \choose k} (\pm 1)^{k} z^{n-k}.
\end{equation}
Just as before let's look at each term individually. For the mRNA production
term we have
\begin{equation}
  \Rm \bb{\ee{p^y \left[ (m + 1)^x -m^x \right]}} =
  \Rm \bb{\ee{p^y \left[ \sum_{k=0}^x {x \choose k} m^{x-k} - m^x \right]}}.
\end{equation}
When $k = 0$, the term inside the sum on the right-hand side cancels with the
other $m^x$, so we can simplify to
\begin{equation}
  \Rm \bb{\ee{p^y \left[ (m + 1)^x -m^x \right]}} =
  \Rm \bb{\ee{p^y \left[ \sum_{k=1}^x {x \choose k} m^{x-k} \right]}}.
\end{equation}
Once the sum is expanded we can see that the highest moment in this sum is given
by $\bb{\ee{m^{(x-1)} p^y}}$ which satisfies both of the conditions on
\eref{seq_mom_conditions}.

For the mRNA degradation term we similarly have
\begin{equation}
  \Gm \bb{\ee{m p^y \left[ (m - 1)^x - m^x \right]}} =
  \Gm \bb{\ee{m p^y \left[ \sum_{k=0}^x {x \choose k}(-1)^k m^{x-k} -
                          m^x \right]}}.
\end{equation}
Simplifying terms we obtain
\begin{equation}
  \Gm \bb{\ee{m p^y \left[ \sum_{k=0}^x {x \choose k}(-1)^k m^{x-k} -
                          m^x \right]}} =
  \Gm \bb{\ee{p^y \left[ \sum_{k=1}^x {x \choose k}(-1)^k m^{x+1-k} \right]}}.
\end{equation}
The largest moment in this case is $\bb{\ee{m^x p^y }}$, which again satisfies
the conditions on \eref{seq_mom_conditions}.

The protein production term gives
\begin{equation}
  \Rp \bb{\ee{m^{(x + 1)} \left[ (p + 1)^y - p^y \right]}} =
  \Rp \bb{\ee{m^{(x + 1)} \left[ \sum_{k=0}^y {y \choose k} (-1)^k p^{y-k}
                                - p^y \right]}}.
\end{equation}
Upon simplification we obtain
\begin{equation}
  \Rp \bb{\ee{m^{(x + 1)} \left[ \sum_{k=0}^y {y \choose k} (-1)^k p^{y-k}
                                - p^y \right]}} =
  \Rp \bb{\ee{m^{(x + 1)} \left[ \sum_{k=1}^y {y \choose k}(-1)^k p^{y-k}
  \right]}}.
\end{equation}
Here the largest moment is given by $\bb{\ee{m^{x+1} p^{y-1}}}$, that again
satisfies both of our conditions. For the last term, for protein degradation we
have
\begin{equation}
  Rp \bb{\ee{m^{(x + 1)} \left[ (p + 1)^y - p^y \right]}} =
  Rp \bb{\ee{m^{(x + 1)} \left[ \sum_{k=1}^y {y \choose k} (-1^k) p^{y - k}
  \right]}}.
\end{equation}
The largest moment involved in this term is therefore $\bb{\ee{m^x p^{y-1}}}$.
With this, we show that the four terms involved in our general moment equation
depend only on lower moments that satisfy \eref{seq_mom_conditions}.

As a reminder, what we showed in this section is that the kinetic model
introduced in \fref{fig2_minimal_model}(A) has no moment-closure problem. In
other words, moments of the joint mRNA and protein distribution can be computed
just from knowledge of lower moments. This allows us to cleanly integrate the
moments of the distribution dynamics as cells progress through the cell cycle.

\subsection{Computing single promoter steady-state moments}

(Note: The Python code used for the calculations presented in this section can
be found in the
\href{https://www.rpgroup.caltech.edu//chann_cap/software/chemical_master_steady_state_moments_general.html}{following
link} as an annotated Jupyter notebook)

As discussed in \secref{sec_cell_cycle}, one of the main factors contributing to
cell-to-cell variability in gene expression is the change in gene copy number
during the cell cycle as cells replicate their genome before cell division. Our
minimal model accounts for this variability by considering the time trajectory
of the distribution moments as given by \eref{seq_mom_ode_factorized}. These
predictions will be contrasted with the predictions from a kinetic model that
doesn't account for changes in gene copy number during the cell cycle in
\siref{supp_multi_gene}.

If we do not account for change in gene copy number during the cell cycle or for
the partition of proteins during division, the dynamics of the moments of the
distribution described in this section will reach a steady state. In order to
compute the steady-state moments of the kinetic model with a single gene across
the cell cycle, we use the moment closure property of our master equation. By
equating \eref{seq_mom_ode_factorized} to zero for a given $\bb{x}$ and
$\bb{y}$, we can solve the resulting linear system and obtain a solution for
$\bb{\ee{m^x p^y}}$ at steady state as a function of moments $\bb{\ee{m^{x'}
p^{y'}}}$ that satisfy \eref{seq_mom_conditions}. Then, by solving for the
zero$\th$ moment $\bb{\ee{m^0 p^0}}$ subject to the constraint that the
probability of the promoter being in any state should add up to one, we can
substitute back all of the solutions in terms of moments $\bb{\ee{m^{x'}
p^{y'}}}$ with solutions in terms of the rates shown in
\fref{fig2_minimal_model}. In other words, through an iterative process, we can
get at the value of any moment of the distribution. We start by solving for the
zero$\th$ moment. Since all higher moments, depend on lower moments we can use
the solution of the zero$\th$ moment to compute the first mRNA moment. This
solution is then used for higher moments in a hierarchical iterative process.


\section{Introduction}

The equivalent layer is a well-known technique for processing potential-field data in applied geophysics
since the 1960's.
It comes from potential theory as a mathematical solution of Laplace's equation, in the region above the
sources, by using the Dirichlet boundary condition \citep{kellogg1929}.
This theory states that any potential-field data produced by an arbitrary 3D physical-property distribution can
be exactly reproduced by a fictitious layer located at any depth and having a continuous 2D physical-property
distribution. In practical situations, the layer is approximated by a finite set of sources (e.g., point masses
or dipoles) and their physical properties are estimated by solving a linear system of equations that yields an
acceptable potential-field data fit. These fictitious sources are called equivalent sources.

Many previous works have used the equivalent layer to perform different potential-field data
transformations such as gridding \citep[e.g.,][]{dampney1969, cordell1992, mendonca-silva1994},
upward/downward continuation \citep[e.g.,][]{emilia1973, hansen-miyazaki1984,
li-oldenburg2010}, reduction to the pole \citep[e.g.,][]{silva1986, leao-silva1989, guspi-novara2009,
oliveirajr-etal2013}, combining multiple data sets \citep[e.g.,][]{boggs-dransfield2004} and
gradient data processing \citep[e.g.,][]{barnes-lumley2011}.

Although the use of the equivalent-layer technique increased over the last decades, one of the biggest
problems is still its high computational cost for processing large-data sets.
This problem propelled several studies aimed at improving the computational efficiency of the
equivalent layer technique.
\citet{leao-silva1989} develop a fast method for processing a regular grid of potential-field data.
The method consists in estimating an equivalent layer which exactly reproduces the potential-field data
within a small data window. The data window is shifted over the whole gridded data in a procedure similar
to a discrete convolution.
The equivalent layer extends beyond the moving-data window and is located at a depth between two and six
times the grid spacing of the observations. For each data window, the equivalent layer is estimated by
solving an underdetermined linear system.
After estimating an equivalent layer, the transformed-potential field is computed only at the center of the
moving-data window. The use of a small moving-data window greatly reduces the total number of floating-point
operations (\textit{flops}) and RAM memory storage. The computational efficiency of this method relies on the
strategy of constructing the equivalent layer by successively solving small linear systems instead of solving
just one large linear system for the entire equivalent layer.
\citet{mendonca-silva1994} also follow the strategy of solving successive small linear systems for
constructing an equivalent layer. Their method is based on the  equivalent-data concept, which
consists in determining a subset of all potential-field data (named equivalent-data set), such that the
interpolating surface that fits the chosen subset also automatically fits all remaining data.
The equivalent data set is obtained by iteratively introducing the potential-field observation with the
greatest residual in the preceding iteration. By applying to the interpolation problem, the method is
optimized by approximating dot products by the discrete form of an analytic integration that can be
evaluated with less computational effort. According to the authors, the equivalent-data set is usually
smaller than the total number of potential-field observations, leading to computational savings.
The authors also pointed out that the computational efficiency of the method depends on the number of
equivalent data. If the potential-field anomaly is nonsmooth, the number of equivalent data can be large and
the method will be less efficient than the classical approach.

By following a different strategy, \citet{li-oldenburg2010} develop a rapid method that transforms the
dense sensitivity matrix associated with the linear system into a sparse one by using a wavelet technique.
After obtaining a sparse representation of the sensitivity matrix, those authors estimate the
physical-property distribution within the equivalent layer by using an overdetermined formulation.
Those authors pointed out that, given the sparse representation, their method reduces the computational time
required for solving the linear system by as many as two orders of magnitude if compared with the same
formulation using a dense matrix.
\citet{barnes-lumley2011} follow a similar strategy and transformed the dense sensitivity matrix
into a sparse one. However, differently from \citet{li-oldenburg2010}, their method operates in the space
domain by grouping equivalent sources far from an observation point into blocks with average physical
property. This procedure aims at reducing the memory storage and achieving computational efficiency
by solving the transformed linear system with a weighted-least-squares conjugate-gradient algorithm.
Note that, instead of constructing the equivalent layer by solving successive small linear systems,
these last two methods first transform the large linear system into a sparse one and then take advantage of
this sparseness.

\citet{oliveirajr-etal2013} develop a fast method based on the reparameterization of the physical-property
distribution within the equivalent layer. Those authors divided the equivalent layer into a regular grid of
equivalent-source windows inside which the physical-property distribution is described by bivariate
polynomial functions. By using this polynomial representation, the inverse problem for estimating the
equivalent layer is posed in the space of the total number of polynomial coefficients within all
equivalent-source windows instead of in the space of the total number of equivalent sources. According to
\citet{oliveirajr-etal2013}, the computational efficiency of their method relies on the fact that the total
number of polynomial coefficients needed to describe the physical-property distribution within the
equivalent layer is generally much smaller than the number of equivalent sources, leading to a very smaller
linear system to be solved. Those authors could verify that the total number of \textit{flops} needed for
building and solving the linear inverse problem of estimating the total number of polynomial coefficients can
be reduced by as many as three and four orders of magnitude, respectively, if compared with the same inverse
problem of estimating the physical property of each equivalent source via Cholesky decomposition.

There is another class of methods that iteratively estimates the physical-property distribution within
the equivalent layer without solving linear systems.
The method presented by \citet{cordell1992}, and later generalized by \citet{guspi-novara2009}, updates
the physical property of the sources, which are located below each potential-field data, using a
procedure that removes the maximum residual between the observed and predicted data.
\citet{xia-sprowl1991} and \citet{xia-etal1993} develop
fast iterative schemes for updating the physical-property distribution
within the equivalent layer in the wavenumber and space domains, respectively.
Grounded on excess mass constraint, \cite{siqueira-etal2017} develop an iterative scheme
starting with a mass distribution within the equivalent layer that is proportional to observed gravity data.
Then, their method iteratively adds mass corrections that are proportional to the gravity residuals.
The total number of \textit{flops} required by these iterative methods for estimating the physical-property
distribution within the equivalent layer depends on the total number of iterations, however this number is
generally much smaller than the total number of \textit{flops} required to solve a large-scaled linear system.
Generally, the most computational expensive step in each iteration of these methods is the forward problem
of calculating the potential-field data produced by the equivalent layer.

In the present work, we show that the sensitivity matrix associated with a planar equivalent layer
of point masses has a very well-defined structure called Block-Toeplitz Toeplitz-Block (BTTB) for
the case in which (1) the observed gravity data is located on a regularly spaced grid at constant
height and (2) there is one point mass directly beneath each observation point.
This technique have been successfully used in potential-field methods for
3D gravity inversion \citep{zhang-wong2015}, downward continuation
\citep{zhang-etal2016} and 3D magnetic modeling \citep{qiang_etal2019}.
By using this property, we propose an efficient algorithm based on FFT convolution
\citep[e.g.,][ p. 207]{vanloan1992} for computing the forward problem at each iteration of
the fast equivalent-layer technique proposed by \citet{siqueira-etal2017}.
Our method uses the gravitational effect produced by a single point mass to compute the
effect produced by the whole equivalent layer, which results in a drastic reduction
not only in the number of flops, but also in the RAM memory usage of the fast equivalent-layer technique.
Tests with synthetic and field data illustrate the good performance of our method in processing
large gravity data sets.


\section{Policy Gradient Methods}

In this section we take an approach that is different to the action-value methods that we have considered previously. We continue the function approximation scheme, but attempt to learn a \emph{parameterised policy} $\pi(a \vert{} s, \vec{\theta})$ where $\vec{\theta} \in \R{}^{d'}$ is the policy's \emph{parameter vector}. Our methods might also learn a value function, but the policy will provide a probability distribution of possible actions without directly consulting the value function as we did previously. \\

We will learn the policy parameter by \emph{policy gradient methods}. These are gradient methods based on some scalar performance measure $J(\vec{\theta})$. In particular, performance is maximised by \emph{gradient ascent} using some stochastic estimate of $J$, $\widehat{\grad J(\vec{\theta}_t)}$, whose expectation approximates $\E{}[\grad_{\vec{\theta}} J(\vec{\theta}_t)]$
\[
    \vec{\theta}_{t+1} = \vec{\theta} + \alpha \widehat{\grad J(\vec{\theta}_t)}.
\]
Methods that also learn a value function are called \emph{actor-critic} methods. Actor is in reference to the learn policy, while critic is in reference to the learned (usually state-) value function.

\subsection{Policy Approximation and its Advantages}

In policy gradient methods, the policy can be parameterised by any differentiable function of the parameter $\vec{\theta}$. We generally require $\pi \in (0, 1)$ to be defined on the open interval to ensure exploration. \\

For action-spaces that are discrete and not too large, it is common to learn a preference function $h(s, a, \vec{\theta}) \in R{}$ and then take a soft-max to get the policy
\[
    \pi(a \vert{} s, \vec{\theta}) = \frac{e^{h(s, a, \vec{\theta})}}{\sum_b e^{h(s, b, \vec{\theta})}}.
\]
We call this type of parameterisation \emph{soft-max in action preferences}. (Note the homomorphism: preferences add, while probabilities multiply.) We can learn the preferences any way we like, be it linear or using a deep learning.\\

Some advantages of policy parameterisation:
\begin{itemize}
    \item Action-value methods, such as $\epsilon$-greedy action selection, can result give situations in which an arbitrarily small change in the action-values completely changes the policy.
    \item The soft-max method will approach a deterministic policy over time. If we used action-values then these would approach their (finite) true values, leading to finite probabilities (with the soft-max). Action preferences do not necessarily converge, but instead are driven to produce an optimal stochastic policy.
    \item In some problems, the best policy may be stochastic. Action-value methods have no natural way of approximating this, whereas it is embedded in this scheme.
    \item Often the most important reason for choosing a policy based learning method is that policy parameterisation provides a good way to inject prior knowledge into the system.
\end{itemize}

\subsection{The Policy Gradient Theorem}
The episodic and continuing cases of the policy gradient theorem have different proofs (since they use different formulations of the expected reward). Here we will first focus on the episodic case, but the results carry over the same.\\

In the episodic case the performance function is the true value of the start state under the current policy
\[
    J(\vec{\theta}) = v_{\pi_{\vec{\theta}}}(s_0).
\]
In the following we assume no discounting ($\gamma = 1$), but this can be inserted by making the requisite changes (see exercises).\\

The success of the \emph{policy gradient theorem} is that it gives a gradient of the performance function that does not include derivatives of the state distribution. The result for the episodic case is as follows and is derived in the box shown below
\[
    \grad_{\vec{\theta}} J(\vec{\theta}) \propto \sum_s \mu(s) \sum_a q_\pi(s, a) \grad_{\vec{\theta}} \pi(a \vert{} s, \vec{\theta}).
\]\\

\includegraphics[width=\textwidth]{\NotesImages/proof_policy_gradient_episodic.png}\\

\subsection{REINFORCE: Monte Carlo Policy Gradient}
We now attempt to learn a policy by stochastic gradient ascent on the performance function. To begin, the policy gradient theorem can be stated as
\[
    \grad_{\vec{\theta}}J(\vec{\theta}) = \Epi \left[\sum_a q_\pi(S_t, a) \grad_{\vec{\theta}}\pi(a \vert{} S_t, \vec{\theta}) \right].
\]
The \emph{all-actions} method simply samples this expectation to give the update rule
\[
    \vec{\theta}_{t+1} = \vec{\theta}_t + \alpha \sum_a \hat{q}(S_t, a, \vec{w})\grad_{\vec{\theta}}\pi(a\vert{}S_t, \vec{\theta}).
\]
The classical REINFORCE algorithm involves only $A_t$, rather than a sum over all actions. We proceed
\begin{align}
    \grad_{\vec{\theta}} &= \Epi\left[ \sum_a \pi(a\vert{}S_t, \vec{\theta}) q_\pi(S_t, a)\frac{\grad_{\vec{\theta}}\pi(a \vert{} S_t, \vec{\theta})}{\pi(a \vert{} S_t, \vec{\theta})} \right] \\
  &= \Epi\left[q_\pi(S_t, A_t)\frac{\grad_{\vec{\theta}}\pi(A_t \vert{} S_t, \vec{\theta})}{\pi(A_t \vert{} S_t, \vec{\theta})} \right] \\
  &= \Epi\left[G_t\frac{\grad_{\vec{\theta}}\pi(A_t \vert{} S_t, \vec{\theta})}{\pi(A_t \vert{} S_t, \vec{\theta})} \right],
\end{align}
which yields the REINFORCE update
\begin{equation}
    \vec{\theta}_{t+1} = \vec{\theta}_t + \alpha G_t \grad_{\vec{\theta}} \log\pi(A_t\vert{} S_t, \vec{\theta}).
\end{equation}
This update moves the parameter vector in the direction of increasing the probability of the action taken proportional to the return and inversely proportional to the probability of the action. It uses the complete return from time $t$, so in this sense is a Monte Carlo algorithm. We refer to the quantity
\[
    \grad_{\vec{\theta}} \log\pi(A_t\vert{} S_t, \vec{\theta})
\]
as the \emph{eligibility vector}. Pseudocode is given in the box below (complete with discounting). Convergence to a local optimum is guaranteed under the standard stochastic approximation conditions for decreasing $\alpha$. However, since it is a Monte Carlo method, it will likely have high variance which will slow learning.\\

\includegraphics[width=\textwidth]{\NotesImages/REINFORCE_monte_carlo_algorithm}\\

\subsection{REINFORCE with Baseline}
The policy gradient theorem can be generalised to incorporate a comparison to a \emph{baseline} value $b(s)$ for each state
\begin{equation}
    \grad_{\vec{\theta}} J(\vec{\theta}) \propto \sum_s \mu(s) \sum_a \left( q_\pi(s, a) - b(s) \right) \grad_{\vec{\theta}} \pi(a \vert{} s, \vec{\theta}).
\end{equation}
The baseline can be a random variable, as long as it doesn't depend on $a$. The update rule then becomes
\begin{equation}
    \vec{\theta}_{t+1} = \vec{\theta}_t + \alpha \left( G_t \grad_{\vec{\theta}} - b(S_t) \right) \log\pi(A_t\vert{} S_t, \vec{\theta}).
\end{equation}
The idea of the baseline is to reduce variance -- by construction it has no impact on the expected update.\\

A natural choice for the baseline is a learned state-value function $\hat{v}(S_t, \vec{w})$. Pseudocode for Monte Carlo REINFORCE with this baseline (also learned by MC estimation) is given in the box below.\\

\includegraphics[width=\textwidth]{\NotesImages/REINFORCE_monte_carlo_baseline_algorithm}\\

This algorithm has two step sizes $\alpha^{\vec{\theta}}$ and $\alpha^{\vec{w}}$. Choosing the step size for the value estimates is relatively easy, for instance in the linear case we have the rule of thumb $\alpha^{\vec{w}} = 1/ \E{}\left[ || \grad_{\vec{w}} \hat{v}(S_t, \vec{w}) ||_\mu^2 \right]$. It is much less clear how to set the step size for the policy parameters.

\subsection{Actor-Critic Methods}
Although REINFORCE with baseline can use an estimated value function, it is not an actor-critic method because it does not incorporate value estimates through bootstrapping.\\

We present here a one-step actor-critic method that is an analog of TD(0), Sarsa(0) and Q-learning. We replace the full return of REINFORCE with a bootstrapped one-step return:
\begin{align}
    \vec{\theta} &= \vec{\theta}_t + \alpha \left(  G_{t:t+1} - \hat{v}(S_t, \vec{w})\right) \frac{\grad_{\vec{\theta}} \pi(A_t \vert{} S_t, \vec{\theta}_t)}{\pi(A_t \vert{} S_t, \vec{\theta}_t)}\\
    &= \vec{\theta}_t + \alpha \left(  G_{t:t+1} - \gamma\hat{v}(S_{t+1}, \vec{w}) - \hat{v}(S_{t}, \vec{w})\right) \frac{\grad_{\vec{\theta}} \pi(A_t \vert{} S_t, \vec{\theta}_t)}{\pi(A_t \vert{} S_t, \vec{\theta}_t)}\\
    &= \vec{\theta}_t + \alpha \delta_t \frac{\grad_{\vec{\theta}} \pi(A_t \vert{} S_t, \vec{\theta}_t)}{\pi(A_t \vert{} S_t, \vec{\theta}_t)}
\end{align}
with $\delta_t$ as the one-step TD error. The natural method to learn the state-value function in this case would be semi-gradient TD(0). Pseudocode is given in the boxes below for this algorithm and a sister algorithm using eligibility traces.\\

\includegraphics[width=\textwidth]{\NotesImages/one_step_actor_critic_algorithm.png}\\

\includegraphics[width=\textwidth]{\NotesImages/actor_critic_eligibility_traces_algorithm.png}\\

\subsection{Policy Gradient for Continuing Problems}
For continuing problems we need a different formulation. We choose as our performance measure the average rate of reward per time step:
\begin{align}
    J(\vec{\theta}) = r(\pi) &= \lim_{h \to \infty} \frac{1}{h} \sum_{t=1}^h \E{}[R_t \vert{} S_0, A_{0:t-1} \sim \pi] \\
    &= \lim_{t \to \infty} \E{}[R_t \vert{} S_0, A_{0:t-1} \sim \pi] \\
    &= \sum_s \mu(s) \sum_a \pi(a \vert{} s) \sum_{s', r} p(s', r \vert{} s, a)r,
\end{align}
where $\mu$ is the steady distribution under $\pi$, $\mu(s) = \lim_{t \to \infty} P{}(S_t = s \vert{} A_{0:t} \sim \pi)$ which we assume to exist and be independent of $S_0$ (ergodicity). Recall that this is the distribution that is invariant under action selections according to $\pi$:
\[
    \sum_s \mu(s) \sum_a \pi(a \vert{} s, \vec{\theta}) p(s' \vert{} s, a) = \mu(s').
\]
We also define the values with respect to the differential return:
\[
    G_t \doteq R_{t+1} - r(\pi) + R_{t+2} - r(\pi) + R_{t+3} - r(\pi) + \cdots.
\]
With these changes the policy gradient theorem remains true (proof given in the book). The forward and backward view equations also remain the same. Pseudocode for the actor-critic algorithm in the continuing case is given below.\\

\includegraphics[width=\textwidth]{\NotesImages/actor_critic_eligibility_traces_continuing.png}

\subsection{Policy Parameterisation for Continuous Actions}
We simply define a continuous (parameterised) probability distribution over the actions, then do the gradient ascent as above.


\problemname{Logland}
Two thieves (who strongly ``recommended'' us not to reveal their names) has broken into the central bank of Logland, where the country's cash reserve is stored.
In Logland, the currency has the $k$ denominations $1, 2, 4, 8, \ldots, 2^{k - 1}$, and the bank currently stores $x_i$ coins of denominations $2^i$.

Thieves live by a very strict honor code, which stipulates that any loot grabbed must be divided evenly between the theives.
Unfortunately (for the thieves), this may mean that some loot must be left behind.
For example, if the bank contained a single coin of denomination $8$, and two coins of denomination $2$, there is no way they could bring the $8$ coin.
Instead, they would have to take one coin of value $2$ each, leaving more than half of the possible loot!

Since neither leaving loot nor solving NP-hard problems are things thieves like to do, they have asked you to compute the minimum value of the loot they must leave behind, in order to be able to
split the remaining loot evenly.

\section*{Input}
The first line of input contains the integer $1 \le k \le 10^6$ -- the number of denominations in Logland.
The next line contains the $k$ integers $0 \le x_0, x_1, \dots, x_{k-1} \le 2^{30}$, the number of coins of the denominations $2^0, 2^1, \dots, 2^{k - 1}$.

\section*{Output}
Output a single line, the minimum amount of money the thieves must leave behind.
Since this number may be very large, output it modulo the prime number $10^9 + 7$.


\subsection{Cantilever beam}
\paragraph{}
A two-dimensional cantilever beam subjected to a parabolic shear load at the free end is examined as shown in Fig.~\ref{qdt_fig:ex_cantilever_beam_geo_bc}.
    \begin{figure}[h!]
    \centering
        \scalebox{0.6}{\includegraphics{isogeometric_sbfem/images/cantilever_beam_geo_bc.eps}}
        \caption{ Cantilever beam: Geometry and boundary conditions.}
        \label{qdt_fig:ex_cantilever_beam_geo_bc}
    \end{figure}
%
The geometry is: length $L=\SI{8}{\meter}$, height $D=\SI{4}{\meter}$.
The material properties are: Young’s modulus $E = \SI{3e7}{\newton \per \square \meter}$ , Poisson’s ratio $ \nu =0.25$.
The parabolic shear force is $P = \SI{250}{\newton}$.
The exact solutions for the displacements are given by \citep{Aug2008} as Eq.~\ref{iso_eq:cantilever_beam_displacement_solution}.
where $I=D^3/12$ is the moment of inertia, $\mean{E}=E$, $\mean{\nu}=\nu$ and $\mean{E}=E/(1-\nu^2)$, $\mean{\nu}=nu/(1-nu)$ for plane stress and plane strain condition respectively.
The stress $\sigma$ can be expressed as \citep{Aug2008} as Eq.~\ref{iso_eq:cantilever_beam_stress_solution}.
The strain energy can be derived from Eq.~\ref{iso_eq:cantilever_beam_stress_solution} and Eq.~\ref{iso_eq:cantilever_beam_displacement_solution} as Eq.~\ref{iso_eq:cantilever_beam_energy_solution}.

\paragraph{}
In this example, rigid body motion is constrained by fixing 3 DOF on the left edge of the beam.
$u_x=0$ for points at $(0,-D/2)$ and $(0,D/2)$ and $u_y =0$ for point at $(0,0)$.
Stress from analytical solution in Eq.~\ref{iso_eq:cantilever_beam_stress_solution} are applied on the boundaries.

\paragraph{}
Due to the fact that the geometry of the cantilever beam can be described by four points and four straight lines, drawing in AutoCAD may not be necessary.
As a result, the input geometry is defined manually.
Generated background mesh, coloring and the final result with $res=32$, $s_{max}=16$ and $s_{min}=1$ are shown in Fig.~\ref{qdt_fig:ex_cantilever_beam_background_mesh}, Fig.~\ref{qdt_fig:ex_cantilever_beam_mesh_coloring} and Fig.~\ref{qdt_fig:ex_cantilever_beam_mesh_final}.
%
    \begin{figure}
        \centering
        \scalebox{0.4}{
            \includegraphics{quadtree/ex_images/cantilever_background_mesh.eps}
        }
        \caption[Background mesh of cantilever beam]{Background mesh of cantilever beam : Bold lines represents the input geometry}
        \label{qdt_fig:ex_cantilever_beam_background_mesh}
    \end{figure}
%
    \begin{figure}
        \centering
        \scalebox{0.4}{
            \includegraphics{quadtree/ex_images/cantilever_coloring.eps}
        }
        \caption[Mesh coloring of cantilever beam]{Mesh coloring of cantilever beam : Grey area represents the cantilever beam}
        \label{qdt_fig:ex_cantilever_beam_mesh_coloring}
    \end{figure}
%
    \begin{figure}
        \centering
        \scalebox{0.3}{
            \includegraphics{quadtree/ex_images/cantilever_final_mesh.eps}
        }
        \caption[Final mesh of cantilever beam]{Final mesh of cantilever beam}
        \label{qdt_fig:ex_cantilever_beam_mesh_final}
    \end{figure}
% DOF err
% 146 - 0.019587 (32,16)
% 178 - 0.0082845 (32,2)
% 438 - 0.0027044 (64,2)
% 1658- 0.00066929 (128,2)
Mesh with different parameters are plotted in Fig.~\ref{qdt_fig:ex_cantilever_mesh_all} and the convergence study is plotted in Fig.~\ref{qdt_fig:ex_cantilever_mesh_conv}
%
    \begin{figure}[h!]
        \begin{subfigure}[b]{1\linewidth}
            \centering
            \scalebox{0.4}{
                \includegraphics{quadtree/ex_images/qdt_cantilever_mesh_178.eps}
            }
            \caption{Mesh with $res=32$, $s_{max}=2$, 178 DOFs}
        \end{subfigure}
        \\
        \begin{subfigure}[b]{1\linewidth}
            \centering
            \scalebox{0.4}{
                \includegraphics{quadtree/ex_images/qdt_cantilever_mesh_438.eps}
            }
            \caption{Mesh with $res=64$, $s_{max}=2$, 438 DOFs}
        \end{subfigure}
        \\
        \begin{subfigure}[b]{1\linewidth}
            \centering
            \scalebox{0.4}{
                \includegraphics{quadtree/ex_images/qdt_cantilever_mesh_1658.eps}
            }
            \caption{Mesh with $res=128$, $s_{max}=2$, 1658 DOFs}
        \end{subfigure}
        \caption[Mesh of the cantilever beam]{Mesh of the cantilever beam}
        \label{qdt_fig:ex_cantilever_mesh_all}
    \end{figure}
%
%
    \begin{figure}[H]
        \centering
        \scalebox{0.6}{
            \includegraphics{quadtree/ex_images/qdt_cantilever_conv.eps}
        }
        \caption[Convergence of the cantilever beam]{Convergence of the cantilever beam}
        \label{qdt_fig:ex_cantilever_mesh_conv}
    \end{figure}


\documentclass{article}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}

\newcommand{\ZZ}{\mathbf{Z}}
\newcommand{\Z}{\mathbf{Z}}
\newcommand{\FF}{\mathbf{F}}

\begin{document}

\title{Some notes about an algorithm for computing Bernoulli numbers mod $p$}
\author{David Harvey}
\maketitle

Conceptually the algorithm has two parts. The first part involves an expression for the Bernoulli numbers in terms of distributions on $\Z_p$ from Lang's ``Cyclotomic Fields'' (chapter 2, Theorem 2.3), which in down-to-earth terms says that
    $$ B_k/k = \frac1{1 - g^k} \sum_{x \in \Z/p\Z} x^{k-1} h(x)  \pmod p $$
where $g$ is a generator of $(\Z/p\Z)^*$, and where $h$ is
    $$ h(x) = \left\{ \frac xp \right\} - g \left\{ \frac{g^{-1}x}p \right\} + \frac{g-1}2. $$
($\{ \cdot \}$ denotes fractional part.) Substituting $x = g^j$, using the fact that $h(g^j)/g^j$ has period $(p-1)/2$ as a function of $j$, and since we are only interested in even $k$, we have
    $$ B_{2k} = \frac{4k}{1 - g^{2k}} \sum_{j=0}^{(p-3)/2} g^{2jk}\frac{h(g^j)}{g^j} \pmod p. $$
The sum on the right is a \emph{number-theoretic transform} of length $(p-1)/2$. The second part of the algorithm involves evaluating this transform by using \emph{Bluestein's trick}: any transform of the form
   $$ b_k = \sum_{j=0}^{(p-3)/2} g^{2jk} a_j $$
can be rewritten using the identity $2jk = k^2 + j^2 - (k-j)^2$ as
   $$ b_k = g^{k^2} \sum_{j=0}^{(p-3)/2} c_{k-j} d_j,$$
where $c_j = g^{-j^2}$ and $d_j = g^{j^2} a_j$. This last sum is a convolution, and so is tantamount to computing the product of the polynomials
        $$ F(X) = \sum_{j=-(p-3)/2}^{(p-3)/2} c_j X^j \quad \text{and} \quad  G(X) = \sum_{j=0}^{(p-3)/2} d_j X^j. $$
In fact one checks that $c_{j+(p-1)/2} = (-1)^{(p-1)/2} c_j$, so
   $$ F(X) = 1 + \left(1 + (-1)^{(p-1)/2}X^{-(p-1)/2}\right) \sum_{j=1}^{(p-3)/2} c_j X^j; $$
this observation reduces the problem to multiplying two polynomials of length $(p-1)/2$, which we do using NTL.

This algorithm seems closely related to the one described in ``Irregular primes and cyclotomic invariants to 12 million'' (Buhler et al). What they do can be interpreted as evaluating the same number-theoretic transform that we do here, but they evaluate it using completely different methods. In particular they take advantage of the factorisation of $p-1$ to improve the running time and reduce memory requirements.

\section*{Further improvements?}

Here's an interesting question, which someone (else) might want to think about someday. In the paper by Buhler et al where they do the power series inversion method, they explain how to use something called ``multisectioning'' to reduce to inversion of a series of length $p/8$ (plus a few multiplications), instead of inversion of length $p/2$ which my first piece of code does. The idea is to use some slightly more tricky series, kind of analogous to using $x^2/(\cosh x - 1)$ instead of $x/(e^x - 1)$ to remove all the even index terms, but even trickier. (That should be implemented in SAGE sometime too.) Now this *suggests* that there might be a way to improve my second algorithm to a few multiplications of length $p/8$, by rewriting the number-theoretic transform in some clever way. I haven't thought about how this might work, but I wouldn't be at all surprised if such an improvement was possible, and I bet it would be faster than their power series method.

\end{document}


\subsection*{FGSM}
\paragraph{Targeted:}
$x' := x - \epsilon \cdot \sign(\nabla_x \loss_\text{target}(x))$

\paragraph{Untargeted:}
$x' := x + \epsilon \cdot \sign(\nabla_x \loss_\text{label}(x))$

\subsection*{Carlini-Wagner (Minimize Perturbation)}
\paragraph{Opt. Prob.:} \texttt{find} $\eta$ \texttt{minimize} $\|\eta\|_p$ \texttt{s.t.} $f(x+\eta) =t, x+\eta \in [0,1]^n$
\paragraph{Relaxed:} \texttt{find} $\eta$ \texttt{minimize} $\|\eta\|_p + c \cdot \obj_t(x+\eta)$ \texttt{s.t.} $x+\eta \in [0,1]^n$

With $\obj_t(x+\eta) \le 0 \Rightarrow f(x+\eta) = t$ (e.g., $\loss_t(x) - 1 = -\log_C(p(x)_t) - 1$ or $\max(0, 0.5- p(x)_t)$)

When using $L_\infty$, gradient of $\|\eta\|_\infty$ is zero at all non-max entries $\rightarrow$ use $L(\eta) = \sum_i \max(0, \lvert \eta_i \rvert - \tau)$ instead; Start with $\tau=1$, update $\eta$ $K$ times, if $L(\eta) = 0$ decrease $\tau$ and repeat, otherwise stop and return previous $\eta$.

\subsection*{PGD}
\algrenewcommand{\algorithmicfunction}{\textbf{def}}

\begin{algorithmic}
\Function{PGD}{$x, y, k, \epsilon_\text{step}, \epsilon$}
\State $x' \gets x + \eta$ for random $\eta$ with $\|\eta\|_\infty \le \epsilon$
\For{$i = 1, \ldots, k$}
    \State $g \gets \nabla_{x'} \loss(f(x'), y)$ \Comment{uFGSM($x'$, $y$)}
    \State $x' \gets x' +  \epsilon_\text{step} \cdot \sign(g)$ \Comment{uFGSM($x'$, $y$)}
    \State $x' \gets x + \max(\min(x'-x, \epsilon), -\epsilon)$
    \State Clip $x'$ to input domain \Comment{e.g., $[0,1]^n$}
\EndFor
\State \Return $x'$
\EndFunction
\end{algorithmic}

For general norm $\|\cdot\|$, use
\begin{algorithmic}
\State $x' \gets x' + \epsilon_\text{step} \cdot \frac{g}{\|g\|}$ \Comment{dir of $g$, not sign}
\If{$\|x'-x\|_p > \epsilon$} \State $x' \gets x + \epsilon \frac{x'-x}{\|x'-x\|}$ \EndIf
\end{algorithmic}

\subsection*{Diffing Networks}
$\obj_t(x) := f(x)_t - g(x)_t$ (or abs. diff. of prob.)

\begin{algorithmic}
\Function{Diff\_nets}{$f, g, \epsilon$}
\State Select $x$ classified as $t$ by both $f$ and $g$
\While{$\class(f(x)) = \class(g(x))$}
    \State $x \gets x + \epsilon \cdot \nabla_x \obj_t(x)$ \Comment{Make $f$ more confident about $t$ while making $g$ less confident}
\EndWhile
\State \Return $x$
\EndFunction
\end{algorithmic}


\documentclass[12pt]{article}
\usepackage[english]{babel}
%\usepackage{subcaption}
\usepackage{hyperref}
\usepackage{graphicx}
\usepackage{amsmath}
\graphicspath{{images/}}
\usepackage{geometry}
 \geometry{
 a4paper,
 total={170mm,257mm},
 left=20mm,
 top=20mm,
 }
\begin{document}
\section{Implementation of Lucas-Kanade (LK) template tracker}
\subsection{Introduction}
In this project we tracked an object or a human being through out the video using Lucas-Kanade (LK) algorithm.

\subsection{Lucas Kanade Algorithm}
The goal of the Lucas-Kanade algorithm is to minimize the sum of squared error between two
images, the template T and the image I warped back onto the coordinate frame of the template.

\begin{figure}[h]
    \centering
    \includegraphics[width=4cm]{error1}
    \caption{Mean Square error}
    \label{fig:Mean Square error}
\end{figure}
Warping I back to compute I (W(x; p)) requires interpolating the image I at the sub-pixel locations W(x; p). The minimization in Equation (3) is performed with respect to p and the sum is
performed over all of the pixels x in the template image T (x). Minimizing the expression in Equation (1) is a non-linear optimization task even if W(x; p) is linear in p because the pixel values
I (x) are, in general, non-linear in x. In fact, the pixel values I (x) are essentially un-related to
the pixel coordinates x. To optimize the expression in Equation (3), the Lucas-Kanade algorithm
assumes that a current estimate of p is known and then iteratively solves for increments to the
parameters $\Delta$p; i.e. the following expression is (approximately) minimized:

\begin{figure}[h]
    \centering
    \includegraphics[width=5cm]{error2}
    \caption{Warpped Mean Square error}
    \label{fig:Warpped Mean Square error}
\end{figure}

with respect to $\Delta$p, and then the parameters are update
\begin{equation}
p = p + \Delta p
\end{equation}

These two steps are iterated until the estimates of the parameters p converge. Typically the test for
convergence is whether some norm of the vector $\Delta$p is below a threshold $\varepsilon$; i.e. $||\Delta p|| \leq \varepsilon $
\subsection{Pipeline followed}
\begin{enumerate}
\item First step is to crop the template out of the video which we want to track through out the video. The template ($T(x)$) is extracted from the first frame of every video. For this project the following are the templates considered.

\begin{figure}[h]
    \centering
    \includegraphics[width=2cm]{babytemplate}
    \caption{Dragon Baby template}
    \label{fig:Dragon Baby template}
\end{figure}

\begin{figure}[h]
    \centering
    \includegraphics[width=6cm]{cartemplate}
    \caption{Car template}
    \label{fig:Car template}
\end{figure}

\begin{figure}[h]
    \centering
    \includegraphics[width=1cm]{bolttemplate}
    \caption{Bolt template}
    \label{fig: Bolt template}
\end{figure}

\item The next step is to perfrom affine transform that warps the current frame so that the template in the first frame is aligned with the warped current frame. The affine transform takes care of the change in scale of the template in the current frame. The obtained image is denoted by $I(W(x;p))$

\item The next step is to compute the error between the warped image and the template.

\begin{figure}[h]
    \centering
    \includegraphics[width=4cm]{error}
    \caption{Error}
    \label{fig: Error}
\end{figure}

\item The next step is computing the gradient $\nabla$I of the warpped image $I(W(x;p))$. Gradient of the image is computed both w.r.t x and y using the Sobel filter.
\begin{equation}
 \nabla I = \left(\frac{\partial I}{\partial x}, \frac{\partial I}{\partial y}\right)
\end{equation}
\item Then the jacobian is computed at (x;p). The jacobian is computed for every point in the warpped image The jacobian is given by :
\begin{equation}
\frac{\partial W}{\partial p} =
\begin{pmatrix}
x & 0 & y & 0 & 1 & 0 \\
0 & x & 0 & y & 0 & 1
\end{pmatrix}
\end{equation}
\item The next step is to compute the steepest gradient descent. The steepest gradient is given by $\nabla I*\frac{\partial W}{\partial p}$:

\item Then we need to compute the Hessian matrix. The Hessian matrix is given by
\begin{equation}
H = \displaystyle\sum_{x}\left[\nabla I*\frac{\partial W}{\partial p}\right]^T*\left[\nabla I*\frac{\partial W}{\partial p}\right]
\end{equation}

\item Finally computing $\Delta p$ which is the updated parameters of the affine transformation matrix which shift the bounding box of the template to bound the object in the current frame.
\begin{equation}
\Delta p = H^{-1}\displaystyle\sum_{x}\left[\nabla I*\frac{\partial W}{\partial p}\right]^T*\left[T(x) - I(W(x;p))\right]
\end{equation}
 \item We need to iterate this process till $\Delta p$ converges to a small threshold value.
\end{enumerate}
The figure below summarizes the algorithm:
\begin{figure}[h]
    \centering
    \includegraphics[width=13cm]{lucas}
    \caption{Lucas Kanade Algorithm}
    \label{fig: Lucas Kanade Algorithm}
\end{figure}

\section{Output images by implementing Lucas Kanade}
\subsection{Tracking Bolt}
\begin{figure}[h]
    \centering
    \includegraphics[width=12cm]{trackbolt1}
    \caption{Tracking Bolt}
    \label{fig:Tracking Bolt}
\end{figure}
\newpage
\begin{figure}[h]
    \centering
    \includegraphics[width=12cm]{trackbolt2}
    \caption{Tracking Car}
    \label{fig:Tracking Car}
\end{figure}

\begin{figure}[h]
    \centering
    \includegraphics[width=12cm]{trackcar1}
    \caption{Tracking Car}
    \label{fig:Tracking Car}
\end{figure}
\newpage
\begin{figure}[h]
    \centering
    \includegraphics[width=12cm]{trackcar2}
    \caption{Tracking Car}
    \label{fig:Tracking Cat}
\end{figure}

\begin{figure}[h]
    \centering
    \includegraphics[width=12cm]{trackbaby1}
    \caption{Tracking Baby}
    \label{fig:Tracking Baby}
\end{figure}
\newpage
\begin{figure}[h]
    \centering
    \includegraphics[width=12cm]{trackbaby2}
    \caption{Tracking Baby}
    \label{fig:Tracking Baby}
\end{figure}
\section{Evaluation of the tracker}
Since a taylor approximation is being used, the tracker will work best when
\begin{enumerate}
  \item Approximation made is close to the template- This means when the object moves really fast in successive frames, the errors induced in the taylor approximation becomes huges and the tracker breaks down.
  \item Brightness of the tracked object remains the same as that of the template
\end{enumerate}

\textbf{Car tracker} -
The tracker breaks down in the case of car video because the intensity of the frame in the video changes continuously. In the case of the car video, when the car goes into the shadow, the tracker looses the car because the intensity level changes drastically between frames.
\begin{figure}[h]
    \centering
    \includegraphics[width=12cm]{trackcar3}
    \caption{Tracking error}
    \label{fig:Tracking error}
\end{figure}

\textbf{Baby tracker}-
The tracker breaks down for the baby as the change is successive frames is slightly big.The baby is just as fast or probably
faster than usian bolt. \\

\begin{figure}[h]
    \centering
    \includegraphics[width=12cm]{trackbaby4}
    \caption{Tracking error}
    \label{fig:Tracking error}
\end{figure}

\begin{figure}[h]
    \centering
    \includegraphics[width=12cm]{trackbaby5}
    \caption{Tracking error}
    \label{fig:Tracking error}
\end{figure}

\textbf{Usian tracker}-
The tracker breaks for usian bolt as the template used in the first frame(\textbf{usian bolt in a crouched position})is not the same as the template used in successive frames(\textbf{usian bolt in upright}). Template correction-that is, update template with next frame- can be applied but the downside is that if an error is made, it propagates and affects the remaining frames.

In all Usian bolt tracking is the best compared to the others as the shift in successive frames is fairly small and the brightness remains constant.
\newpage
\begin{figure}[h]
    \centering
    \includegraphics[width=12cm]{trackbolt3}
    \caption{Bolt in crouched position}
    \label{fig:Bolt in crouched position}
\end{figure}
\begin{figure}[h]
    \centering
    \includegraphics[width=12cm]{trackbolt4}
    \caption{Tracking error in upright position}
    \label{fig:Tracking error in upright position}
\end{figure}

\section{Robustness to Illumination}
In order to increase the robustness of the tracker, we need to scale the brightness of pixels in each frame so that the average brightness of pixels in the tracked region stays the same as the average brightness of pixels in the template.

\textbf{Car tracker}: For tracking the car, gamma correction did not give better results. So, instead we employed z-scores method for tracking. In z-scores, the mean of template image and current frame are checked. So, pixels values in the current frame is changed by a certain z-score relative to the mean of the template.
\begin{enumerate}
\item First the standard deviation of the all the frames of the video are computed w.r.t to the mean of the frames.
\newpage
\begin{figure}[h]
    \centering
    \includegraphics[width=6cm]{zscore}
    \caption{Standard Deviation}
    \label{fig:Standard Deviation}
\end{figure}
\item Then the z-scores are computed. Z-scores is a measure of how far is the pixel value from the mean of the template in terms of standard deviation of the frames.
\begin{figure}[h]
    \centering
    \includegraphics[width=8cm]{zscore1}
    \caption{Z-score formula}
    \label{fig:Z- score formula}
\end{figure}
\item Finally, we shift every pixel in the frame based on z-score.
\begin{figure}[h]
    \centering
    \includegraphics[width=6cm]{zscore2}
    \caption{Updated pixels formula}
    \label{fig:Updated pixels formula}
\end{figure}
\end{enumerate}
The following are the improved output after employing z-scores:

\begin{figure}[h]
    \centering
    \includegraphics[width=11cm]{trackcar4}
    \caption{Tracking with z-scores}
    \label{fig:Tracking with z-scores}
\end{figure}
\begin{figure}[h]
    \centering
    \includegraphics[width=11cm]{trackcar5}
    \caption{Tracking with z-scores}
    \label{fig:Tracking with z-scores}
\end{figure}
\newpage
\section{Team Members:}
1. Eashwar Sathyamurthy
2. Akwasi A Obeng
3. Achal P Vyas
\section{Output Videos:}
To access output videos please use this \href{https://drive.google.com/drive/folders/1nbAUx-p-eyOWts9neJgAPA8YATY4ekmI?usp=sharing}{\underline{link}}.

\section{References}
\href{https://www.statisticshowto.com/probability-and-statistics/z-score/}{\underline{Z-score}}.

\end{document}


\documentclass{report}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{listings}
\begin{document}
\section{PCA}
\subsection{Abstract}
In this issue, we begin to learn the algorithm of dimension reduction.\\\\
As we know, dimension reduction is the best way to solve the problem of over fitting, in addition to increasing data and regularization.\\\\
In fact, the predecessors had encountered dimensional disasters earlier. We know that the volume of the $n $ dimensional sphere is $CR^n$\\\\
Therefore, the ratio of the volume of the sphere to the $n$ dimensional hypercube is
$$
\lim_{n \to +\infty}=\frac{CR^n}{2^n R^n}=0
$$
From the formula, we can see that in high-dimensional data, the distribution of samples is quite sparse, and the interior of hypercube is almost hollow, so it is more difficult to model the data. This is the so-called dimensional disaster.

The dimensionality reduction methods are divided into:
\begin{itemize}
	\item Direct dimensionality reduction, feature selection
	\item Linear dimensionality reduction, PCA, MDS etc.
	\item Piecewise linear, manifolds include Isomap, LLE etc.
\end{itemize}
\subsection{Idea}
As for the core idea of PCA, the teacher summarized a line: one center, two basic points
\begin{itemize}
	\item one center:
	\begin{itemize}
	\item to transform each feature that may be linearly related into a set of linearly independent features via an orthogonal transformation.
	\item That is, to reconstruct the original feature space.
	\end{itemize}
	\item Two basic points:
	\begin{itemize}
	\item Maximum Projection Variance
	\begin{itemize}
	\item Make the data more dispersed in the reconstructed feature space (because the original data is clustered together and scattered in corners)
	\end{itemize}
	\item Minimum Reconfiguration Distance
	\begin{itemize}
	\item to minimize the loss of information (i.e., fewer components of the complementary space) after the data has been reconstructed
	\end{itemize}
	\end{itemize}
\end{itemize}
\subsection{Algorithm}
we'll mainly talk about the first basic point: maximum projection variance. In fact, the two basic points mean the same thing, but interpret a center from different angles.\\\\
Firstly,we are to review the projection. We have talked about projection before, and the same is true here. Let's assume sample $x_ i $, a base vector $u_ i $, assuming $u_ i^Tu_ i = 1 $, so you can get the projection of the sample in  this dimension $u_ i $ is
$$
project_i=x_i^Tu_i
$$
After orthogonal transformation, the sample originally has $p $ feature dimensions. Because we need to reduce the dimension, we only take the first $q $ features, and these $q $ features are linearly independent. Therefore, these projections can be directly added to obtain the projection of the sample in the new feature space.\\\\
Note that the data is centered before the projection, so the mean value of the data is changed to zero, and the variance of the projection can be squared directly.\\
To sum up, we get the objective function:
$$
J=\frac{1}{N} \sum_{i=1}^{N} \sum_{j=1}^{q}\left(\left(x_{i}-\bar{x}\right)^{T} u_{j}\right)^{2}
$$
The objective function is derived slightly below:\\
since the shape of $((x_i-\bar{x})^Tu_j)$ is $(1,p)* (p,1)=(1,1)$, therefore, it can be transposed:
$$
\begin{aligned}
J&=\frac{1}{N} \sum_{i=1}^{N} \sum_{j=1}^{q}\left(\left(x_{i}-\bar{x}\right)^{T} u_{j}\right)^{2}\\
&=\frac{1}{N} \sum_{i=1}^{N} \sum_{j=1}^{q}(u_{j}^T(x_{i}-\bar{x}))^{2}\\
&=\frac{1}{N} \sum_{i=1}^{N} \sum_{j=1}^{q}u_{j}^T(x_{i}-\bar{x}))(x_{i}-\bar{x})^T u_j\\
&=\sum_{j=1}^{q} u_{j}^T(\frac{1}{N} \sum_{i=1}^{N} (x_{i}-\bar{x}))(x_{i}-\bar{x})^T) u_j\\
&=\sum_{j=1}^q u_j^T S u_j\\
\end{aligned}
$$
Don't forget that we have another condition: $s.t\ u_j^T u_j=1$\\\\
So you can use Lagrange multiplier:
$$
\underset{u_{j}}{\operatorname{argmax}} L\left(u_{j}, \lambda\right)=\underset{u_{j}}{\operatorname{argmax}} u_{j}^{T} S u_{j}+\lambda\left(1-u_{j}^{T} u_{j}\right)
$$
To derive from the above:
$$
\frac{\partial \Delta}{\partial u_j}=2S u_j -2\lambda u_j=0
$$
we get:
$$
S u_j = \lambda u_j
$$
You can see that the transformed base vector is actually the eigenvector of the covariance matrix, $\lambda$ is the eigenvalue of $S$.\\
In fact, the solution of covariance matrices can also be simplified:
$$
\begin{aligned}
S &=\frac{1}{N} \sum_{i=1}^{N}\left(x_{i}-\bar{x}\right)\left(x_{i}-\bar{x}\right)^{T} \\
&=\frac{1}{N}\left(x_{1}-\bar{x}, x_{2}-\bar{x}, \cdots, x_{N}-\bar{x}\right)\left(x_{1}-\bar{x}, x_{2}-\bar{x}, \cdots, x_{N}-\bar{x}\right)^{T} \\
&=\frac{1}{N}\left(X^{T}-\frac{1}{N} X^{T} I_{N} I_{N}^{T}\right)\left(X^{T}-\frac{1}{N} X^{T} I_{N} I_{N}^{T}\right)^{T} \\
&=\frac{1}{N} X^{T}\left(E_{N}-\frac{1}{N} I_{N} I_{N}^T\right)\left(E_{N}-\frac{1}{N} I_{N} I_{N}^T\right)^{T} X \\
&=\frac{1}{N} X^{T} H_{N} H_{N}^{T} X \\
&=\frac{1}{N} X^{T} H_{N} H_{N} X=\frac{1}{N} X^{T} H X
\end{aligned}
$$
Here, $H$ is a special matrix, called a central matrix.
$$
H=E_N - \frac{1}{N}I_N I_N^T
$$
Therefore, in practice, we only need to find the covariance matrix using the above formula, and then decompose it orthogonally to get the eigenvalues and eigenvectors.
\newpage
\subsection{Implement}
\begin{lstlisting}[language={python}]
import numpy as np
import os
os.chdir("../")
from models.decompose_models import PCA

k, b = 3, 4
x = np.linspace(0, 10, 100)
y = x * k + b
x += np.random.normal(scale=0.3, size=x.shape)
data = np.c_[x, y]

model = PCA()
model.fit(data)
model.draw(data)
\end{lstlisting}
\end{document}