\section{Stereo vision}

Three-dimensional dynamic capture means in general the way of recording a sequence of movements of a real-life target or scene.
This section introduces the methods for seeing three-dimensional structures from stereo setups, consisting of two or more cameras.
The configuration of a proper setup with two is described; it can be extended to the multi-view domain.
Most of the sophisticated algorithms able to recover a dense reconstruction base their work on the same principles, as follows.

\subsection{Coordinate systems and transforms} \label{sec:coord}

The previous section described how to record a view of a scene with a camera. From now on, the term camera refers to a particular camera configuration, which can be a single physical camera moved to different locations.

The camera is a projective object located somewhere in the imaged scene.
Its \textit{intrinsic parameters} model the properties of projection, but do not take into account the camera location in any global coordinate system.
The \textit{extrinsic parameters} contain the camera location and rotation in another global coordinate system, structured as a matrix.
This is especially advantageous when there are more than one cameras and their coordinates must be related.
\cite{hartley03multiview,heyden2005multiple}
This part quickly reviews basic transforms whose results are needed in the later reconstruction steps.

%Calibration is often specified with a camera projection matrix, or several separate matrices.
%It may be convenient to store intrinsics and extrinsics separately if the intrinsic matrix is constant for several pictures, for example.

\simplefig{h}{%
\includegraphics[width=0.7\textwidth]{gfx/pinhole3d}
}{fig:pinhole3d}
{Pinhole camera geometry. Camera coordinate system origin at O, axis X3 points towards the optical axis, Y1 and Y2 point to image plane axes and R is the principal point, at the image center. The point P projects to Q, as well as everything else on the line joining them. The image plane is f units away from camera origin; f is called the focal length.}

In computer graphics and vision, points and directions are usually described in homogeneous coordinates. Translation, perspective projection, rotation, and other operations are conveniently described as matrices by using an additional dimension for points, of which usually the last element is 1: $(x, y, z, 1)$. All points $(xw, yw, zw, w)$ map to the same point $(x, y, z)$. \cite{dubrofsky2009homography,hartley03multiview}

%Homography definition (mapping of points and lines in $P^2$)

The imaging process essentially captures a projection to a flat two-dimensional plane of the camera's view, as described in section \ref{sec:imaging}.
When relating points between different cameras that view the same scene, the cameras' relational positions and rotations must be known.
One of the cameras is often conveniently chosen as the origin of a global coordinate frame, so that its extrinsic parameters become unity transforms (programming libraries often assume this, see e.g. \cite{opencv}).
Each three-dimensional point in the world is transformed to the small sensor or film inside the camera, which is then digitized to a discrete two-dimensional grid of pixels. The size of this pixel array (i.e. image) is referred to as the camera's resolution.

%Figure \ref{fig:TODO} illustrates this transformation chain, which is encoded as the following equations, given a homogeneous point (4-dimensional vector) $X$ representing a 3D location described in physical (e.g. metric) coordinates:

The transformation chain is encoded as follows, given a homogeneous point (4-dimensional vector) $X$ representing a 3D location described in physical (e.g. metric) coordinates:

\begin{align}
	x &= P X\\
	  &= M_i X_s\\ % X_s on the sensor
	  &= M T R X\\
	  &= M_i M_p T R X\\ % R, T camera pose, M_4 to camera sensor, M_3 to pixel coords
\end{align}

$x$ is a 2d pixel in a discrete image, $X_s$ exists on the sensor. $R$, $T$ encode the camera rotation and translation (extrinsics); $M_p$ projects the world coordinates to the camera sensor (film) - still in world coordinates (intrinsics!), and finally the affine $M_i$ transforms the points from the sensor to pixel coordinates on the digital discretized image.

The whole projection $P = M_i M_p T R$ can be used as-is without decomposing it to separate matrices, unless the individual parameters are needed. As the chain consists of several matrices, some of them are defined only up to scale; the coordinate systems' units can be chosen freely. Software packages usually do not decompose the chain, because it is not needed and unique parameters cannot be found because of scaling.

%The external camera parameters are called the extrinsics: camera coordinate system position and rotation (heading) in the global space.
%Camera position sits at the projection center blah.

The internal parameters, intrinsics, encode how the image is formed on the sensor: they consist of focal length, sensor size and principal point: (last column left out, as it's full of zeroes in 3x4)

\begin{equation}
	M =
	\begin{pmatrix}
		m_x & \gamma & u_0\\
		0   &    m_y & v_0\\
		0   &        0 & 1
	\end{pmatrix}
\cdot
	\begin{pmatrix}
		f & 0 & 0\\
		0 & f & 0\\
		0 & 0 & 1
	\end{pmatrix}
	=
	\begin{pmatrix}
		\alpha_x & \gamma   & u_0\\
		0        & \alpha_y & v_0\\
		0        & 0        & 1
	\end{pmatrix}
\end{equation}

For simplicity, it is often denoted $\alpha_x = m_x f$, $\alpha_y = m_y f$. $R = (u_0, v_0)$ is the image center (or principal point). For square pixels, $m_x = m_y$, and for a non-skewed sensor, $\gamma = 0$, which is often the case. \cite{hartley03multiview,szeliski10vision,heyden2005multiple}

%Le image. Horizontal planar triangle, lines between camera origins etc. lecture11.pdf.

\subsection{Camera calibration}

%In order to accurately measure a scene with a camera, the camera's properties must be known.
Reconstruction algorithms need to relate points between images; the camera properties are needed.
Calibrating a camera means to measure its intrinsics and extrinsics in order to map its data to a known coordinate frame.
Calibration has always to be done, but it does not necessary need to be a manual step before scanning; self-calibration attempts to target this convenience problem. \cite{pollefeys1999hand,hartley03multiview}

%Projective calibration only is too general, as it leaves out some assumptions that can be done about a physical world, such as relative angles and sizes; metric calibration something something. \cite{zisserman1995metric}.

Automatic calibration tools rely on an amount of feature pairs of which the best matches are found, or a known pattern, such as a planar checkerboard pattern [chuan; zhang] whose features are also distinguished with a similar algorithm but a priori knowledge of the object structure is used for precise calibration.
These usually need several pictures taken with the same camera from different poses.

The checkerboard calibration step can also measure optical distortion at the same time. \cite{opencv,camcalmatlab}

%TODO Figure: show extrinsic in matlab cam calibs, nice pics (both cam and world centered)

%Single three-dimensional calibration object is also sufficient blbl

One possible way is direct linear transform (DLT)
\cite{hartley03multiview}: the whole matrix $P$ is solved from $x_i = PX_i$ by constructing a system of equations from the projections of some known points $i$, and minimizing an error metric, as the case is usually overconditioned.

%Methods that dig the matrix out of a single image have certain restrictions, and won't work if e.g. seven points lie on the same plane [longuet-higgins etc.]

%XXX see below. Intrinsic, extrinsic. Distortions. Projection matrices. Camera resectioning.

%many single planar chessboard pics vs. a single image of an accurate 3d model.

%The scale of values in the equations above affects the precision [hartley, in defense of .., h,ziss]. A similarity transform can be used to modify the values to a more consistent range; this is called normalization of the data.

\subsection{Binocular disparity}
%Essential, fundamental matrices. Correspondence problem. Rectification, undistortion. Epipolar geometry.

\simplefig{h!}{
\begin{tikzpicture}[scale=0.3]
	% P, Z
	\draw[fill] (0, 20) circle [radius=0.3];
	\node at (0, 21) {$P$};

	\draw [<->] (0, 0.5) -- (0, 19.5);
	\node at (0.5, 10) {$Z$};

	% origins, T
	% TODO: circles, node ends not exactly at those points
	\draw [<->] (-9.5,0) -- (9.5, 0);
	\draw[fill] (-10, 0) circle [radius=0.3];
	\draw[fill] ( 10, 0) circle [radius=0.3];
	\node at (-10, -1) {$O_l$};
	\node at (10, -1) {$O_r$};
	\node at (0, -1) {$T$};

	% headings
	\draw [->] (-10, 0) -- (-10, 10);
	\draw [->] (10, 0) -- (10, 10);

	% image planes, at y=4
	\draw[thick] (-13, 4) -- (-7, 4);
	\draw[thick] (13, 4) -- (7, 4);

	\draw [<->] (-6, 0.5) -- (-6, 3.5);
	\node at (-5.5, 2) {$f$};

	% intersection points at principals and xs
	\draw[fill] (-10, 4) circle [radius=0.3];
	\draw[fill] (10, 4) circle [radius=0.3];

	\node at (-10.5, 3) {$c_l$};
	\node at (10.5, 3) {$c_r$};

	\node at (-9, 5) {$x_l$};
	\node at (9, 5) {$x_r$};

	% O-to-P
	\draw (-10, 0) -- (0, 20);
	\draw (10, 0) -- (0, 20);

	% p
	\draw[fill] (8, 4) circle [radius=0.3];
	\node at (8, 3) {$p_r$};
	\draw[fill] (-8, 4) circle [radius=0.3];
	\node at (-8, 3) {$p_l$};
\end{tikzpicture}
}{fig:simplestereo}
{A very simple stereo setup, picture from above. The image planes (thick lines) are actually imaginary, as a real film in a camera would exist behind the principal point and project the image upside down, as described earlier in \ref{sec:imaging}. The coordinates exist in the world coordinate units. The symbols $O$ are the camera origins ($T$ units between each other); $c$ the principal points; $x$ the image plane coordinates of $p$ w.r.t. the principal points; and $f$ is the focal length. The unknown is $Z$, depth of point $P$.}

%Next, the setup of binocular stereo vision is described. Common stereo vision rigs use the simplest possible case: two identical cameras with a fixed distance, both oriented to the same direction, parallel to the line connecting them, as in figure \ref{fig:simplestereo}.

Assuming known calibration with identical cameras (same focal length and sensor) in a setup described above, visualized in figure \ref{fig:simplestereo}, points can be triangulated as follows:

From similar triangles with a common vertex at $P$, we get (note that $x_r < 0$ as it's to the left, towards to the negative axis, from the corresponding plane's origin)

\begin{align}
	\frac{Z}{T} &= \frac{Z-f}{T - x_l + x_r} \\
	&= \frac{Z-f}{T - d}\\
	ZT - Zd &= ZT - fT\\
	Z &= \frac{fT}{d} \label{eq:z}
\end{align}

The disparity $d$ is the difference of the points in their image planes, $d = x_r - x_l$.
If the image planes would be fixed as being physically correct, in the back side of the camera origins, the focal length should be negated to keep the correct interpretation and sign because the projected physical image is mirrored in both axes. Image processing between the sensor and a picture file usually inverts this.

As the equation \ref{eq:z} shows, depth is directly inversely proportional to disparity in this simple case.
To map the depth to correct units, only focal length $f$ and the baseline $T$ are needed additionally; when using pixel coordinates instead of physical in $d$, also the pixel size should be taken into account.
All of these are encoded in the camera parameters.
Algorithms such as those in OpenCV \cite{opencv} can compute point clouds from disparity images.

\subsection{Epipolar geometry}

\simplefig{h!}{
\begin{tikzpicture}[scale=0.5]
	% cameras
	\draw[fill] (-11,-1) circle [radius=0.2];
	\draw[fill] ( 11,-1) circle [radius=0.2];
	\draw (-11,-1) -- (11, -1);
	\node at (0, -2) { baseline };

	\node at (-11,-2) {$C_l$};
	\node at ( 11,-2) {$C_r$};

	% planes
	\draw (-10,1) -- (-10,7) -- (-5,3) -- (-5,-3) -- cycle;
	\draw ( 10,1) -- ( 10,7) -- ( 5,3) -- ( 5,-3) -- cycle;

	% 3d pts
	\draw[fill] ( 0,3) circle [radius=0.2];
	\draw[fill] ( 0,9) circle [radius=0.2];
	\node at (0,2) {$A$};
	\node at (0,8) {$B$};

	% origins via pts
	\draw (-11,-1) -- (0,3) -- (11,-1);
	\draw (-11,-1) -- (0,9) -- (11,-1);

	% epis
	\draw[fill] (-6,-1) circle [radius=0.1];
	\draw[fill] (6,-1) circle [radius=0.1];
	\node at (-6,-1.5) { $e_l$ };
	\node at (6,-1.5) { $e_r$ };

	% projections
	\draw[fill] (-7.5,0.2727) circle [radius=0.1];
	\draw[fill] (-7.5,2.1818) circle [radius=0.1];
	\node at (-8.5, 0.2727) {$A_l$};
	\node at (-8.5, 2.1818) {$B_l$};
	% lines from epis
	\draw (-6,-1) -- +(-2*1.5,2*1.2727);%(-7.5,0.2727);
	\draw (-6,-1) -- +(-2*1.5,2*3.1818);%(-7.5,2.1818);

	\draw[fill] (7.5,0.2727) circle [radius=0.1];
	\draw[fill] (7.5,2.1818) circle [radius=0.1];
	\node at (8.5, 0.2727) {$A_r$};
	\node at (8.5, 2.1818) {$B_r$};
	\draw (6,-1) -- +(2*1.5,2*1.2727);%(7.5,0.2727);
	\draw (6,-1) -- +(2*1.5,2*3.1818);%(7.5,2.1818);
\end{tikzpicture}
}{fig:epigeom}
{Two camera views on same scene.
World points $A$, $B$ project to planes of different views imaged from $C_l$ and $C_r$ on the left ($A_l$ and $B_l$), and to the right ($A_r$, $B_r$).
When $A_l$ is known, its corresponding point $A_r$ (not initially known in practice) is found on the epipolar line joining $e_r$ and $A_r$ in the right image.
All epipolar lines in a view join in the same point ($e_l$ and $e_r$).}

Triangulation or reconstruction of the scene structure given by image pair(s) is usually done on the base of a known relationship between the cameras.
Such relationship, known as calibrating the cameras, can be automatically determined, given by corresponding points that can be distinguished in each image and matched.
\cite{trucco1998introductory,hartley03multiview}

In stereo vision, the same scene of interest is seen by two or more cameras at the same time.
The cameras are rarely aligned perfectly such as in the disparity setup described above, however.
Epipolar geometry encodes the relations between arbitrarily positioned cameras in a standard way so that coordinates of a 3D point seen in several images can be calculated with the same triangulation.

A point seen by camera $C_l$ at 3D point A could be anywhere on the line between $C_l$'s origin and P, because a certain line passing through the principal point always projects to a point.
This line is seen as a single point $A_l$.
From another viewpoint in camera $C_r$, this line equals to some line on B's image plane.
The real point must be on that line.
The inverse applies for any point on $C_r$ and a line on $C_l$.
The lines on the image planes are called epipolar lines.

Essential matrix defines how the camera poses differ by the something something points seen by both. When $A_l$, $A_r$ encode the points in figure \ref{fig:epigeom} by the corresponding camera coordinates, and the baseline difference (vector from $C_l$ to $C_r$) is marked as $t$, it holds that $(A_l-C_l) \cdot t \times (A_r-C_r) = 0$, as all the vectors are coplanar; the cross product yields a normal to the plane, which is perpendicular to all the vectors, thus the dot product equals 0. \cite{hartley03multiview}

Essential matrix is a matrix form of this relation; it includes the relative rotation and translation of the two cameras.

%\begin{align*} \label{eq:essential}
%	%A_r &= R (A_l - t) \\
%	%A_r^T R T A_l &= 0 \\
%	%A_r^T E A_l &= 0
%	A_l \cdot t \times A_r = 0\\
%	A_l \cdot t \times R A_r = 0\\
%	A_l^T T
%\end{align*}

%where $T$ is the cross-product form of $t$ encoded in a matrix form as below. The essential matrix is obtained as $E = R T$.
%
%Le image. lecture11.pdf. O->p dot (O->O' cross O'->p') = 0
%
%Cross product expressed in a skew-symmetric matrix form is
%\begin{equation}
%\vec a \times \vec b =
%\begin{pmatrix}
%	 0   & -a_z &  a_y\\
%	 a_z &  0   & -a_x\\
%	-a_y &  a_x & 0
%\end{pmatrix}
%\begin{pmatrix}
%	b_x\\b_y\\b_z
%\end{pmatrix}
%= \vec c
%\end{equation}

Fundamental matrix relates the corresponding points in stereo images; it has the same meaning as the essential matrix, but it works in the pixel coordinates of the cameras, which are obtained after the projective transform that takes the intrinsics into account. Inverting the matrix $M_i$ (\ref{sec:coord}) in sensor-to-pixel coordinate transform and using it on pixel coordinates, world coordinates seen by the camera can be obtained.

%\[
%\hat pAl = M_p A_l\\
%\hat A_r = M_p A_r
%\]
%
%and using it on pixel coordinates, the world coords can be obtained, plugging in to the equation \ref{eq:essential}
%
%\[
%A_r^T E A_l = 0\\
%(M_p^-1 \hat A_r)^T E (M_p^-1 \hat A_l) = 0\\
%\hat A_r^T M_p^-T E M_p^-1 A_l = 0\\
%\hat A_r^T F \hat A_l = 0
%\]
%
%the fundamental matrix
%
%\[
%F = M_p^-T E M_p^-1 = M_p^-T R T M_p^-1
%\]

The fundamental matrix relates the pixels and epipolar lines, and as such it is useful in image processing where the images are described as pixels in a color array (image) and not colored physical coordinates.

%Epipole can be interpreted as the location of another camera as seen by other camera.

\subsection{Point matching}

Previously, basics for reconstructing three-dimensional location for a point pair were introduced, assuming known positions for the same point in different images.
To reconstruct a whole scene from a full image, all pairwise points must be matched, i.e. found that what pixel in one view represents the same object as one in other view.

Matching is often also called correspondence searching:
Given a pixel in one image, what is the corresponding pixel in another image taken from the same scene?
Pixels correspond to each other if they represent the same physical point.

To describe the pixel's characteristics, its surrounding environment is encoded as a \textit{feature}, a easily recognizable, unique property vector.
When discussing about features, not every pixel's neighbourhoods are used; \textit{good} features are those that have strongly distinguishable properties, such as edges and corners.

Edges or corners are essentially high-frequency information in the image that can be interpreted as a 2D discrete function; thus, they can be detected by a discrete high-pass or band-pass filter, zeroing all but those pixels where a high difference is found \cite{marr1980theory}

Neighbourhood of a pixel where features are detected is often called a window or a patch.

Matching can be done in sparse or dense mode; sparse matching finds a set of features from each image, and tries to match them. Dense matching runs through each pixel of one image and tries to find the same from another one with e.g. template matching \cite{duda1973pattern}; from the coordinate differences in image space between the two images, a disparity map is built. The disparities are then directly transformed to depth values, yielding a point cloud.

Scale-invariant feature transform (SIFT) \cite{lowe1999object} is a commonly used algorithm for local feature detection. A GPU implementation is also available \cite{changchang2007siftgpu}.  Invariance to scaling, translation and rotation makes SIFT useful in describing features that can be matched between unaligned images. Other similar commonly used means are Speede Up Robust Features (SURF) \cite{bay2006surf} Harris corner detector \cite{harris1988combined}.

\subsection{Correspondence and rectification} \label{sec:rectification}

In order to triangulate a real point from two or more photographs, the location of the point in all images must be known.
Rectification is a process that simplifies this search problem by restricting the search to a single dimension.
By aligning the cameras such that their images are coplanar, the search only has to be performed on a line that is parallel to the line connecting the camera centers.
After rectification, the corresponding lines are axis-aligned (horizontal or vertical) in both images. \cite{hartley03multiview}
% LOL TODO guido gerig image rectification (stereo) slides

\subsection{N-view stereo}

The case of more cameras than a single pair uses the same principles in epipolar geometry.
It brings more restrictions and dimensions; in three dimensions, for example, the fundamental matrix becomes three-dimensional, the trifocal tensor. \cite{hartley03multiview}
It is also more computationally heavy, as more data must be processed; if no information about camera parameters is available, pairwise checks between the images may become expensive. \cite{wu2013towards}

Multiple baseline stereo is a simple special case for many cameras. When all the cameras lie on the same baseline, calibration is easier and points can be selected by using a minimized sum of errors. \cite{okutomi1993multiple}

The cameras that are used in capturing a scene can be fixed or positioned arbitrarily; in fact, the structure from motion technique \cite{snavely2006photo,fitzgibbon1998automatic} enables to use just one camera that is moved around.
Accurate and fast reconstructions are still traditionally done with stereo camera pairs, though.

Another common way is to use pairwise cameras for individual reconstructions to build a point cloud for every pair, and then register them together. \cite{bradley2010high}

%\subsection{Structure from motion}
%
%Structure from motion (SfM) refers usually to recovering the structure of a scene from the motion of a single camera.
%For each view, the pose of the camera is determined and the scene structure is extended with the new information in the image.
%(pollefeys)
%Bundle adjustment is used to refine the camera parameters.

%\subsection{Post-processing}
%Uv mapping. Manual work. 3d noise removal; ignore points that have no close pair in other clouds.
%Rendering: "as usual".
%Post: remodel the mesh (face), see what it would look like. Refine parameters to get a similar output as in the photos (normal map etc.), backproject. Use colors and highpass them; assume uniform lightning and locally uniform texture color (bradley). (Simply a rendering technique, that level of detail in 3D structure might not be needed). Still, structured light and/or shading assumptions [shape from single image cues/shading trucco,verri p.225] done too.

\subsection{Reprojection errors}

The quality of the reconstruction is measured by reprojecting the 3D points back to the cameras with the estimated parameters, and calculating the distance between the projected and original point. \cite{hartley03multiview}
Bundle adjustment \cite{wu2011multicore} seeks to optimize all camera parameters at once with good performance.

A common way to handle feature errors is Random Sample Consensus (RANSAC). Random subsets of the sample space is iterated, and samples that do not fit well to a model that is constructed of a smaller set are ignored. The iteration that matches most samples is selected. \cite{hartley03multiview}


\section*{Appendix B: Cross Entropy Method for Parameter Calibration}

This Appendix describes the pseudocode for the Cross Entropy Method for Normal distribution \citep{rubinstein1999cross}.

\begin{algorithm} [H]
	%\scriptsize
	\SetAlgoLined
	Set $p=(\mu_1,\sigma_1,\mu_2,\sigma_2,...,\mu_K,\sigma_K)$ \quad \%Initial distribution parameters

	Set $M$ \quad     \%Number of stops

	Set $T$  \quad  \% Maximum iteration number

	Set $I$  \quad  \% Maximum iteration number

	Set $\rho$  \quad \% Set selection ratio

	\For {$t$ from 1 to $T$} { \  \%Main CEM loop

	    \For {$i$ from 1 to $I$} {

	Draw $y^{(i)}$ from $\mathcal{N}(\mu,\sigma)$  \quad  \%Draw $I$ samples

	Compute $f^i:=f(y^{(i)}$}

	Sort $f^i$-values \quad  \%Order by decreasing magnitude

	$\gamma \leftarrow f_{\rho.I}$ \quad  \%Set threshold

	$L_\gamma \leftarrow \{ y^{(i)} | f(y^{(i)}) \leq \gamma$  \quad  \%Collect elite samples

	$\mu'_j = \frac{1}{L_\gamma} \sum_{i=1}^{L_\gamma} \mu_{i,j}$  \quad  \%Update $\mu$

	$\sigma'_j = \frac{1}{L_\gamma} \sum_{i=1}^{L_\gamma} \sigma_{i,j}$   \quad  \%Update $\sigma$

	$\mu_j \leftarrow \alpha \mu'_j + (1 - \alpha)  \mu_j$ \quad \%Update with step size $\alpha$

	$\sigma_j \leftarrow \alpha \sigma'_j + (1 - \alpha)  \sigma_j$ \quad \%Update with step size $\alpha$
	}
	\caption{Cross-Entropy Method for Normal distribution}
	\label{algo:CEM}
\end{algorithm}


%---------------------------Volume-----------------------------
\section{Wedge Volume}

The wedge volume metric is simply
\[
q = V.
\]

\othermetrictable{volume}%
{$L^3$}%                                      Dimension
{$[0,DBL\_MAX]$}%                             Acceptable range
{$[-DBL\_MAX,DBL\_MAX]$}%                     Normal range
{$[-DBL\_MAX,DBL\_MAX]$}%                     Full range
{$\frac{\sqrt{3}}{4}$}%                       Unit element
{--}%                                         Citation
{v\_wedge\_volume}%                           Verdict function name


\documentclass[../main.tex]{subfiles}

\begin{document}
\chapter{Limits and Continuity}
\section{Easy Limits}
With easy limits, you can get the answer simply by plugging in the
limiting value:
\[ \lim_{x \to 3} \frac{x^2 + x}{x + 1} = \frac{(3)^2 + (3)}{(3) + 1} = 3 \]
\section{Continuity}
\begin{defn}
    We say $f(x)$ is continuous at $x_0$ when:
    \[ \lim_{x \to x_0} f(x) = f \left( x_0 \right) \]
    \label{def:continuity}
\end{defn}
\begin{exmp}
    Consider the following \emph{piecewise} function:
    \[
        f(x) =
        \begin{cases}
            x + 1   & :\ x > 0\\
            -x      & :\ x \leq 0\\
        \end{cases}
    \]
    It is said to be \emph{discontinuous}
    (see Figure \ref{fig:discontinuous-function}) at $x = 0$
    since we know $f(0) = 0$, but for $x > 0$:
    \[ \lim_{x \to 0} f(x) = 1 \]
    We can say $f$ is continuous from the left at $x = 0$,
    but not the right.
\end{exmp}
\begin{defn}[Right-Hand Limit]
    \[ \lim_{x \to x_0^+} f(x) = \lim_{x \to x_0} f(x)\ :\ x > 0 \]
\end{defn}
\begin{defn}[Left-Hand Limit]
    \[ \lim_{x \to x_0^-} f(x) = \lim_{x \to x_0} f(x)\ :\ x < 0 \]
\end{defn}
\begin{figure}[h]
    \centering
    \includegraphics[width=0.5\textwidth]{discontinuous-function.pdf}
    \caption{Graph of a discontinuous function}
    \label{fig:discontinuous-function}
\end{figure}
\subsection{Discontinuities}
\begin{defn}
    A discontinuity is removable if:
    \[
        \lim_{x \to x_0^+} f(x) =
        \lim_{x \to x_0^-} f(x) \neq
        f \left( x_0 \right)
    \]
\end{defn}
\begin{case*}
    The function $\frac{\sin(x)}{x}$ is undefined for $x = 0$. However,
    its limit as $x \to 0$ still exists:
    \begin{figure}[h]
        \centering
        \includegraphics[width=0.5\textwidth]{removable-discontinuity.pdf}
        \caption
        {
            A removable discontinuity ---
            the function is continuous everywhere except one point
        }
        \label{fig:removable-discontinuity}
    \end{figure}
\end{case*}
\begin{defn}
    A jump discontinuity is when
    $\lim_{x \to x_0^+} f(x) \neq \lim_{x \to x_0^-} f(x)$
    even if they both exist.
\end{defn}
\begin{figure}[h]
    \centering
    \includegraphics[width=0.5\textwidth]{jump-discontinuity.pdf}
    \caption{An example of a jump discontinuity}
    \label{fig:jump-discontinuity}
\end{figure}
\begin{defn}
    There is an infinite discontinuity when the right- and left-hand
    limits are both infinite, but in opposite directions.
\end{defn}
\begin{case*}
    $f(x) = \frac{1}{x}$ has an infinite discontinuity
    (\emph{singularity}) at $x = 0$:
    \[
        \lim_{x \to x_0^+} \frac{1}{x} = -\infty \quad , \quad
        \lim_{x \to x_0^+} \frac{1}{x} = \infty
    \]
\end{case*}
\begin{figure}[h]
    \centering
    \includegraphics[width=0.5\textwidth]{infinite-discontinuity.pdf}
    \caption{$f(x) = \frac{1}{x}$ is an example of an infinite discontinuity}
    \label{fig:infinite-discontinuity}
\end{figure}
\subsubsection{Ugly Discontinuities}
The function shown in Figure \ref{fig:ugly-discontinuity} doesn't even go
to $\pm \infty$ --- it doesn't make sense to say it "goes to" anything.
For something like this, we say the limit does not exist (\textsc{dne}).
\begin{figure}[h]
    \centering
    \includegraphics[width=0.5\textwidth]{ugly-discontinuity.pdf}
    \caption
    {
        An example of an ugly discontinuity:
        a function that oscillates a lot as it approaches the origin
    }
    \label{fig:ugly-discontinuity}
\end{figure}
\section*{}
\begin{thm}
    If $f$ is differentiable at $x_0$, then $f$ is continuous
    at $x_0$
\end{thm}
\begin{proof}
    From Definition \ref{def:continuity}, we need to show:
    \begin{align*}
        &\lim_{x \to x_0} f(x) = f \left( x_0 \right)\\
        \Rightarrow &\lim_{x \to x_0} f(x) - f \left( x_0 \right) = 0
    \end{align*}
    The \textsc{lhs} can be rewritten as:
    \begin{align*}
        \lim_{x \to x_0} f(x) - f \left( x_0 \right)
        &=  \lim_{x \to x_0} f(x) - f \left( x_0 \right)
            \cdot \frac{x - x_0}{x - x_0} \quad \because \quad
            \frac{x - x_0}{x - x_0} = 1\\
        &=  \lim_{x \to x_0} \frac{f(x) - f \left( x_0 \right)}{x - x_0}
            \cdot \left( x - x_0 \right)\\
        &=  \lim_{x \to x_0} \frac{f(x) - f \left( x_0 \right)}{x - x_0}
            \cdot \lim_{x \to x_0} \left( x - x_0 \right)
    \end{align*}
    If $f$ is differentiable, we can use Equation \ref{eqn:derivative}
    and rearrange to get:
    \begin{align*}
        \lim_{x \to x_0} f(x) - f \left( x_0 \right)
        &=  \lim_{x \to x_0} \left( x - x_0 \right)
            \cdot f' \left( x_0 \right)\\
        &=  \left( x_0 - x_0 \right) \cdot f' \left( x_0 \right)\\
        &=  0 \cdot f' \left( x_0 \right)\\
        &=  0
    \end{align*}
\end{proof}
\begin{note}
    You can never divide by zero! The first step was to multiply by
    $\frac{x - x_0}{x - x_0}$. It looks as if this is illegal because
    we are multiplying by $\frac{0}{0}$ when $x = x_0$. But, when computing
    the limit as $x \to x_0$, we always assume $x \neq x_0$. In other words,
    $x - x_0 \neq 0$. Therefore, the proof is valid.
\end{note}
\end{document}


\documentclass{article}
\usepackage{mathtools}
\usepackage{pgfplots}
\usepackage{natbib}
\usepackage{graphicx}

\usepackage[utf8]{inputenc}

\title{CSE546 HW0 A}
\author{Bobby Deng | 1663039 | dengy7 }
\date{March 2020}

\begin{document}

\maketitle

\section{A.1}

\[ P(A|B)=\frac{P(A)*P(B|A)}{P(B)} \]
\[ P(B)=P(A)*P(B|A)+P(A^-)*P(B|A^-) \]
\[ P(A^-)=1-P(A)\]

$A$: You have the disease. $A^-$: You don't have the disease.

$B$: Test is positive. $B^-$: Test is negative.

$P(A)=0.0.0001$, $P(A^-)=0.9999$

\[ P(B)= P(A)*P(B|A)+P(A^-)*P(B|A^-) \]
\[ P(B)= 0.0001*0.99+0.9999*0.01 \]
\[ P(B)= 0.010098 \]

\[ P(A|B)=\frac{P(A)*P(B|A)}{P(B)} \]
\[ P(A|B)=\frac{0.0001*0.99}{0.010098} \]
\[ P(A|B)=0.0098 \]

\section{A.2}
\subsection{a.}
\[ Cov(X,Y)=E[(X-E[X])(Y-E(Y))] \]
\[ Cov(X,Y)=E[XY-XE[Y]-YE[X]+E[X]E[Y]] \]
\[ Cov(X,Y)=E[XY]-E[X]E[X+Y]+E[X]E[Y] \]
\[ Cov(X,Y)=E[XY]-E[X](E[X]+E[Y])+E[X]E[Y] \]
\newline

1. \textbf{When $E[Y|X=x]=x$, then $E[X]=E[Y]$, so we can change all $E[Y]$ to $E[X]$.}
\[ Cov(X,Y)=E[XY]-E[X](E[X]+E[X])+E[X]E[Y] \]
\[ Cov(X,Y)=E[XX]-E[X]E[X+X]+E[X]^2 \]
\[ Cov(X,Y)=E[XX-XE[X]-XE[X]+E[X]^2] \]
\[ Cov(X,Y)=E[(X-E[X])(X-E(X))] \]
\[ Cov(X,Y)=E[(X-E[X])^2] \]

\subsection{b.}
When X,Y are independent:
\[ Cov(X,Y)=E[XY-XE[Y]-YE[X]+E[X]E[Y]] \]
\[ Cov(X,Y)=E[XY]-E[X]E[Y]-E[Y]E[X]+E[X]E[Y] \]\newline

1. We know that $E[XY]=E[X]E[Y]$, Then
\[ Cov(X,Y)=0 \]

\section{A.3}
\subsection{a.}
Since $Z=X+Y$, the probability function for Z should be joint probability of X and Y.

\[ h(z) = P(Z);Z=X+Y \]
\[ Y=0,X=Z;Y=1,X=Z-1;Y=2,X=Z-2;....;Y=Z-1,X=1;Y=0,X=Z \]
Since X,Y are independent, then
\[h(z)=\sum_0^Z P(X=i,Y_k=Z-i)\]
\[h(z)=\sum_0^Z P(X=i)*P(Y=Z-i)\]
\[h(z)=\sum_0^Z f(x)*g(y)\]
Above is showing what happened for discrete variable. It is the same story for continuous variable, however there are infinite many of that variable X,Y combination. So, we need to use integral to represents the infinite many variables and the area under it which representing the probability of a single Z.

\[ h(z)=\int_{-\infty}^{\infty} f(x)*g(y) \]

We know that $Z=X+Y$, so we can replace $Y=Z-X$, then it becomes:
\[ h(z)=\int_{-\infty}^{\infty} f(x)*g(z-x) \]

\subsection{b.}
First of all, Z can not be less than 0 or bigger than 2 since X,Y are on [0,1].

Then we look at Z in [1,2] and Z in [0,1] separately.

$f(x)*g(z-x)$ Could either equal to 0 or 1.

When X and Z-x are both on the designated interval, $f(x)*g(z-x)=1$, 0 otherwise.

Now we only look at situations where it is 1.

When $1\ge x \ge0$ or $1\ge Z-X \ge 0$

Then we get two intervals: $Z\ge X \ge 0$ and $1\ge X \ge Z-1$\newline

We Get:
\[ h(z)=\int_{0}^{Z} 1dx = z\]
\[ h(z)=\int_{Z-1}^{1} 1dx = 2-z\]

\[
h(z) = \begin{cases}
z & \text{for $0 < z < 1$} \\
2-z & \text{for $1 \le z < 2$} \\
0 & \text{otherwise.}
\end{cases}
\]

\section{A.4}
We want $E[Y]=0$ and $var(Y)=\sigma^2=1$
\[ E[Y]=aE[X]+b=0\]
\[ 0=a\mu+b\]
\[ b=-a\mu\]
\[ var(Y) = var(aX+b) =1\]
\[ var(y) = a^2var(X) + var(b)=1\]
\[ var(y) = a^2\sigma^2 + 0=1\]
\[ var(y) = a^2\sigma^2=1\]
\[ a^2 = 1/\sigma^2\]
\[a=1/\pm \sigma\]
So,
\[b=\mu / \pm \sigma\]

\section{A.5}
\[ E[\hat{\mu}_n] = E[\frac{1}{n}\sum_{i=1}^nX_i] = \frac{1}{n}\sum_{i=1}^nE[X_i] = \frac{1}{n}\sum_{i=1}^n\mu = \mu \]

\[ var(\hat{\mu}_n) = var(\frac{1}{n}\sum_{i=1}^n X_i)=\frac{1}{n^2}var(\sum_{i=1}^nX_i)=\frac{1}{n^2}\sum_{i=1}^nvar(X_i)=\frac{1}{n^2}\sum_{i=1}^n\sigma^2=\frac{\sigma^2}{n}\]
Now we know $E[\hat{\mu}_n]=\mu$ and $var(\hat{\mu}_n)=\frac{\sigma^2}{n}$. We can use the above calculation for the equations below.
\[ E[Z] = E[\sqrt{n}(\hat{\mu}_n-\mu)] = E[\sqrt{n}](E[\hat{\mu}_n] - E[\mu]) = \sqrt{n}(\mu - \mu) = 0\]

\[ var(Z)=var(\sqrt{n}(\hat{\mu}_n-\mu)) = n*var(\hat{\mu}_n-\mu)\] \[=n*E[((\hat{\mu}_n-\mu)-E[\hat{\mu}_n-\mu])^2] \]
\[=n*E[((\hat{\mu}_n-\mu)-0)^2] \]
\[=n*E[(\hat{\mu}_n-\mu)^2] \]
\[=n*var(\hat{\mu}_n)\]
\[=\sigma^2\]

\section{A.6}
\subsection{a.}
\[\hat{F_n}(x)= \frac{1}{n}\sum_{i=1}^n1\{X_i\le x\}\]
\[E[\hat{F_n}(x)]= E[\frac{1}{n}\sum_{i=1}^n1\{X_i\le x\}] = \frac{1}{n}\sum_{i=1}^nE[1\{X_i\le x\}] \]

Lets work out what $E[1\{X_i\le x\}]$ is:
\[ E[1\{X_i\le x\}] = \int_{-\infty}^{\infty} 1\{X_i\le x\}*f(x) dx \]

\[ = \int_{-\infty}^{\infty}1*f(x) dx; \;\;\;when X_i\le x\]

\[ = F(x); \;\;\;when X_i\le x \]

Now:
\[ E[\hat{F_n}(x)]= \frac{1}{n}\sum_{i=1}^nE[1\{X_i\le x\}] = \frac{1}{n}\sum_{i=1}^nF(x)=F(x)\]

\subsection{b.}
\[ var(\hat{F_n}(x))=E[(\hat{F_n}(x) -F(x))^2] = E[(\hat{F_n}(x) -F(x))(\hat{F_n}(x) -F(x))] \]
\[ =E[\hat{F_n}(x)^2-2\hat{F_n}(x)F(x)+F(x)^2] \]

\[ var(\hat{F_n}(x))= var(\frac{1}{n}\sum_{i=1}^n1\{X_i\le x\}) = \frac{1}{n^2}\sum_{i=1}^nvar(1\{X_i\le x\}) \]
Lets work out what $var(1\{X_i\le x\})$ (Let A to represent it)is:
\[ var(A) = E[(A-E[A])^2]\]
\[ var(A) = E[A^2 -2AE[A]+(E[A])^2]\]
Since $A$ is 1, so $A^2=A$ :
\[ var(A) = E[A] -2E[A]E[A]+(E[A])^2] \]
\[ var(A) = E[A] -E[A]E[A]] \]
\[ var(A) = F(x) - F(x)^2 \]
\[ var(A) = F(x)(1-F(x)) \]

\[var(\hat{F_n}(x))=\frac{1}{n^2}\sum_{i=1}^nvar(1\{X_i\le x\})=\frac{1}{n^2}\sum_{i=1}^nF(x)(1-F(x))\]
\[ var(\hat{F_n}(x))=\frac{F(x)(1-F(x))}{n}\]

\subsection{c.}

\[ var(\hat{F_n}(x))=E[(\hat{F_n}(x)-F(x))^2] =\frac{F(x)(1-F(x))}{n}\]
Prove if $\frac{F(x)(1-F(x))}{n}\le\frac{1}{4n}$.
Which is same as prove: $F(x)(1-F(x))<\frac{1}{4}$.\newline

\[ F(x)(1-F(x)) = \frac{1}{4}-\frac{1}{4}+F(x)^2-F(x) \]

\[ F(x)(1-F(x)) = \frac{1}{4}-(\frac{1}{2}-F(x))^2 \le \frac{1}{4}\]
Which is same as:
\[\frac{F(x)(1-F(x))}{n}\le\frac{1}{4n}\]

\section{A.7}
\subsection{a.}
Rank for A is: 2.\newline
Rank for B is: 2.
\subsection{b.}
$A^T=\begin{bmatrix}
1 & 1 & 1\\
2 & 0 & 1\\
1 & 3 & 2
\end{bmatrix}$
=
$\begin{bmatrix}
1 & 1 & 1\\
2 & 0 & 1\\
0 & 2 & 1
\end{bmatrix}$
=
$\begin{bmatrix}
1 & 0 & 1/2\\
0 & 1 & 1/2\\
0 & 0 & 0
\end{bmatrix}$ \newline

So the minimal basis for A:
$\begin{bmatrix}
1\\
0\\
1/2
\end{bmatrix}$
and
$\begin{bmatrix}
0\\
1\\
1/2
\end{bmatrix}$ \newline

$B^T=\begin{bmatrix}
1 & 1 & 1\\
2 & 0 & 1\\
1 & 3 & 2
\end{bmatrix}$
=
$\begin{bmatrix}
1 & 1 & 1\\
0 & 2 & 1\\
0 & 0 & 0
\end{bmatrix}$
=
$\begin{bmatrix}
1 & 0 & 1/2\\
0 & 1 & 1/2\\
0 & 0 & 0
\end{bmatrix}$\newline

So the minimal basis for B:
$\begin{bmatrix}
1\\
0\\
1/2
\end{bmatrix}$
and
$\begin{bmatrix}
0\\
1\\
1/2
\end{bmatrix}$

\section{A.8}
\subsection{a.}

$\begin{bmatrix}
0 & 2 & 4\\
2 & 4 & 2\\
3 & 3 & 1
\end{bmatrix}$
$\begin{bmatrix}
1 & 1 & 1
\end{bmatrix}^T$
=
$\begin{bmatrix}
0 & 2 & 4\\
2 & 4 & 2\\
3 & 3 & 1
\end{bmatrix}$
$\begin{bmatrix}
1\\
1\\
1
\end{bmatrix}$
=
$\begin{bmatrix}
6\\
8\\
7
\end{bmatrix}$

\subsection{b.}
$\left[\begin{array}{ccc|c}
0 & 2 & 4 & -1\\
2 & 4 & 2 & -2\\
3 & 3 & 1 & -4
\end{array}\right]$\newline
Solution to this augmented matrix is:
$\begin{bmatrix}
-2\\
1\\
-1
\end{bmatrix}$

\section{A.9}
\subsection{a.}
Based on the relationship, we can get this:
$\begin{bmatrix}
-1 & 2
\end{bmatrix}$
*
$\begin{bmatrix}
x_1\\
x_2
\end{bmatrix}$
+2=0 \newline

Then we can get the relationship $x_1 = 2x_2+2$.

\begin{tikzpicture}
	\begin{axis}[
		xlabel=$x_2$,
		ylabel=$x_1$,
		zlabel=$x_3$,
		xlabel style={sloped like x axis},
		ylabel style={sloped}
	]
	\addplot3[surf] {2*x+2};
	\end{axis}
\end{tikzpicture}

\subsection{b.}
$\begin{bmatrix}
1 & 1 & 1
\end{bmatrix}$
*
$\begin{bmatrix}
x_1\\
x_2\\
x_3
\end{bmatrix}$
+0=0 \newline

Then we can get the relationship $x_1 = -x_2-x_3$.

\begin{tikzpicture}
	\begin{axis}[
		xlabel=$x_2$,
		ylabel=$x_3$,
		zlabel=$x_1$,
		xlabel style={sloped like x axis},
		ylabel style={sloped}
	]
	\addplot3[surf] {-x-y};
	\end{axis}
\end{tikzpicture}

\subsection{c.}

When $\Tilde{x}_0$ is the minimizer, $x=\Tilde{x}_0$
\[\min_{x}\begin{Vmatrix}
$x_0-x$
\end{Vmatrix}^2
=
\begin{Vmatrix}
$x_0-\Tilde{x}_0$
\end{Vmatrix}^2
=
\begin{vmatrix}
\frac{(w^T x_0-w^T\Tilde{x}_0)}{w^T w}
\end{vmatrix}^2
=
(
\frac{w^T x_0-w^T\Tilde{x}_0}{w^T w}
)^2
\]
Since $w^T x + b=0$, then $w^T \Tilde{x}_0 + b=0$,   \newline
Then $b=-w^T \Tilde{x}_0$.  \newline
\[(
\frac{w^T x_0-w^T\Tilde{x}_0}{w^T w}
)^2
=
(\frac{w^T x_0+b}{w^T w}
)^2
\]
So the square distance is $(\frac{w^T x_0+b}{w^T w}
)^2 $.

\section{A.10}
\subsection{a.}
\[x^T Ax
=
\begin{bmatrix}
x_1 & \dots & x_n
\end{bmatrix}
\begin{bmatrix}
A_{1,1} & \dots & A_{1,n}\\
\vdots & \ddots & \vdots\\
A_{n,1} & \dots & A_{n,n}
\end{bmatrix}
\begin{bmatrix}
x_1\\
\vdots\\
x_n
\end{bmatrix}\]

\[
=
\begin{bmatrix}
\sum_{i=1}^n x_i A_{i,1} & \dots & \sum_{i=1}^n x_i A_{i,n}
\end{bmatrix}
\begin{bmatrix}
x_1\\
\vdots\\
x_n
\end{bmatrix}
=
\sum_{j=1}^n (x_j (\sum_{i=1}^n x_i A_{i,j}))
\]

For $y^T Bx$, it is the similar story, \newline
\[ \
y^T Bx
=
\sum_{j=1}^n (x_j (\sum_{i=1}^n y_i B_{i,j})) \]
So,

\[ f(x,y)= x^T Ax + y^T Bx + c
=
\sum_{j=1}^n (x_j (\sum_{i=1}^n x_i A_{i,j}))+\sum_{j=1}^n (x_j (\sum_{i=1}^n y_i B_{i,j}))+c
=
\]

\[
=
\sum_{j=1}^n (x_j (\sum_{i=1}^n x_i A_{i,j})+(\sum_{i=1}^n y_i B_{i,j}))+c
=
\sum_{j=1}^n (x_j \sum_{i=1}^n (x_i A_{i,j}+ y_i B_{i,j}))+c
\]

\subsection{b.}
\[
\nabla_z f(x,y)=
\begin{bmatrix}
\frac{\partial f(x,y)}{\partial z_1} &
\dots &
\frac{\partial f(x,y)}{\partial z_n}
\end{bmatrix}^T
\]

\[
\nabla_x f(x,y)=
\begin{bmatrix}
\frac{\partial f(x,y)}{\partial x_1} &
\dots &
\frac{\partial f(x,y)}{\partial x_n}
\end{bmatrix}^T
\]

Let's look at each $\frac{\partial f(x,y)}{\partial x_k}$ separately, and use k here in order to differentiate the i from matrix.\newline

Since $f(x,y)=\sum_{j=1}^n (x_j \sum_{i=1}^n (x_i A_{i,j}+ y_i B_{i,j}))+c$:

\begin{equation}
\begin{split}
\frac{\partial}{\partial x_k} & = \frac{\partial f(x,y)}{\partial x_k}(\sum_{j=1}^n (x_j \sum_{i=1}^n (x_i A_{i,j}+ y_i B_{i,j}))+c) \\
 & = \sum_{j=1}^n \frac{\partial}{\partial x_k}[(x_j \sum_{i=1}^n (x_i A_{i,j}+ y_i B_{i,j}))+c] \\
 & = \sum_{j=1}^n \frac{\partial}{\partial x_k}(x_j \sum_{i=1}^n (x_i A_{i,j}+ y_i B_{i,j})) + 0 \\
 & = \sum_{j=1}^n \frac{\partial}{\partial x_k}(x_j \sum_{i=1}^n (x_i A_{i,j}+ y_i B_{i,j})) \\
 & = \sum_{j=1}^n [\frac{\partial}{\partial x_k}(x_j)( \sum_{i=1}^n (x_i A_{i,j}+ y_i B_{i,j}))+\frac{\partial}{\partial x_k}(\sum_{i=1}^n (x_i A_{i,j}+ y_i B_{i,j}))(x_j)] \\
 & = \sum_{j=1}^n \frac{\partial}{\partial x_k}(x_j)( \sum_{i=1}^n (x_i A_{i,j}+ y_i B_{i,j}))+ \sum_{j=1}^n \frac{\partial}{\partial x_k}(\sum_{i=1}^n (x_i A_{i,j}+ y_i B_{i,j}))(x_j) \\
 & = [0+0+\frac{\partial}{\partial x_k}(x_k)( \sum_{i=1}^n (x_i A_{i,k}+ y_i B_{i,k}))+\dots+0+0]+[\sum_{j=1}^n(x_j \sum_{i=1}^n \frac{\partial}{\partial x_k}(x_i A_{i,j})+0)] \\
 & = \sum_{i=1}^n (x_i A_{i,k}+ y_i B_{i,k}) + \sum_{j=1}^n x_j (0+0+\frac{\partial}{\partial x_k}(x_k A_{k,j})+0+\dots+0+0) \\
 & = \sum_{i=1}^n (x_i A_{i,k}+ y_i B_{i,k}) + \sum_{j=1}^n x_j (\frac{\partial}{\partial x_k}(x_k A_{k,j})) \\
 & = \sum_{i=1}^n (x_i A_{i,k}+ y_i B_{i,k}) + \sum_{j=1}^n x_k A_{k,j} - They are all from 1-n \\
 & = \sum_{i=1}^n (x_i A_{i,k}+ y_i B_{i,k} + x_k A_{k,i})
\end{split}
\end{equation}

So,

\begin{equation}
\begin{split}
\nabla_x f(x,y) & = \begin{bmatrix}
\frac{\partial f(x,y)}{\partial x_1} &
\dots &
\frac{\partial f(x,y)}{\partial x_n}
\end{bmatrix}^T \\
& = \begin{bmatrix}
\sum_{i=1}^n (x_i A_{i,1}+ y_i B_{i,1} + x_1 A_{1,i}) &
\dots &
\sum_{i=1}^n (x_i A_{i,n}+ y_i B_{i,n} + x_k A_{n,i})
\end{bmatrix}^T
\end{split}
\end{equation}

\subsection{c.}
\[
\nabla_y f(x,y)=
\begin{bmatrix}
\frac{\partial f(x,y)}{\partial y_1} &
\dots &
\frac{\partial f(x,y)}{\partial y_n}
\end{bmatrix}^T
\]

Now, do the same thing as x. \newline

\begin{equation}
\begin{split}
\frac{\partial f(x,y)}{\partial y_k} & = \frac{\partial}{\partial y_k}(\sum_{j=1}^n (x_j \sum_{i=1}^n (x_i A_{i,j}+ y_i B_{i,j}))+c) \\
& = \sum_{j=1}^n x_j(\frac{\partial}{\partial y_k}(\sum_{i=1}^n (x_i A_{i,j}+ y_i B_{i,j}))) \\
& = \sum_{j=1}^n x_j (0+0+[0+\frac{\partial}{\partial y_k}(y_k B_{k,j})]+0+\dots + 0)\\
& = \sum_{j=1}^n x_j B_{k,j}
\end{split}
\end{equation}

So we can get,
\begin{equation}
\begin{split}
\nabla_y f(x,y) & = \begin{bmatrix}
\frac{\partial f(x,y)}{\partial y_1} &
\dots &
\frac{\partial f(x,y)}{\partial y_n}
\end{bmatrix}^T \\
& = \begin{bmatrix}
\sum_{j=1}^n x_j B_{1,j} &
\dots &
\sum_{j=1}^n x_j B_{n,j}
\end{bmatrix}^T
\end{split}
\end{equation}

\section{A.11}
\subsection{a.}
\includegraphics[scale=0.7]{11a.png}

\subsection{b.}
\includegraphics[scale=0.7]{11b.png}

\section{A.12}
\subsection{a.}
According to previous answer: $var(\hat{F_n}(x))\le\frac{1}{4n}$, so:

\begin{equation}
\begin{split}
\sqrt{var(\hat{F_n}(x))}\le 0.0025 \\
var(\hat{F_n}(x)) \le & (0.0025)^2 \\
var(\hat{F_n}(x)) \le & \frac{1}{4n} \\
(0.0025)^2 = & \frac{1}{4n} \\
n = & 4000
\end{split}
\end{equation}

\includegraphics[scale=0.5]{12a.png}

\subsection{b.}
\includegraphics[scale=0.5]{12b.png}

\end{document}


%!TEX root = ../Main.tex
The Green's function and self-energies play the central role when it comes to obtaining the Local Density of States (LDOS) as well as electron transport in a system. In fact, the imaginary part of the Green's function is the LDOS for a specific site in a system. What the Green's function and self-energy actually is and how they come about will here be explained formally, to motivate the practical use in the following sections.
\subsection{Green's functions and self-energy}\label{greensandself}
Some of the concepts in this section will be explained using the simple system as an example (\cref{inlinepointplot}). Imagine a system like the one in \cref{atomrepfig}. It contains a unit cell in the centre, marked by a black border, surrounded by repeated unit cells in all directions. The aim is to explain how electrons move through this region. Suppose all cells surrounding the centre cell are considered "contacts" in the sense that they represent a semi-infinite chain of molecules and that they are the source of electrons (or states) that is injected in to the centre cell. What the Green's function is doing is that it ``takes the states through'' the centre region. It propagates the states in this particular area. In other words, the Green's function is the solution to the Schr\"{o}dinger Equation in this area and the equation has the form
\begin{align}\label{Greensunsolved}
	[(E+i\eta)\mathbf{1}-\mathbf{H}]\mathbf{G}(E) = \mathbf{1}
\end{align}
Where \(\eta\) is a small number ensuring that the equation does not diverge when we use the numerical recursion routine described later on. \textit{E} is the energy for which we are probing the system.
From this equation one can also get the Green's function as
\begin{align}\label{Greenssolved}
	\mathbf{G}(E) & = \mathbf{1}([(E+i\eta)\mathbf{1}-\mathbf{H}])^{-1} \\
	              & = [(E+i\eta)\mathbf{1}-\mathbf{H}]^{-1}
\end{align}
The Green's functions in these equations are represented as matrices that contain all the individual Green's functions for the unit cell as well a the Green's functions for the rest of the chain. As seen in the equations, all that is needed to get the Green's function for a unit cell, in theory, is an energy and the Hamiltonian of the unit cell. Note that the solution to the Green's function matrix is a diagonal matrix with the two first off diagonals. This is because of rules for nearest neighbour interaction dictated by the tight-binding approximation. As the Green's functions for all unit cells in a potentially semi-infinite system are needed, in practice, one has to turn to more sophisticated methods to obtain all the Green's functions, namely recursion. More on that shortly. For now this is the introduction to the Green's function. How it relates to a unit cell in a system and that it is the source of the LDOS in a unit cell.\\
As described  one can use the Green's functions to get the propagation of states through a specific on-site Hamiltonian. However, if the system contains a range of cells, possibly infinitely many, the Hamiltonian would be of infinite size and the inversion in \cref{Greenssolved} would be impossible to do practically. The solution to this, is to model a semi-infinite tight-binding chain of atom/molecules and then use \textit{recursion} on this chain. The way the recursion is done is by removing every second cell in the chain. Because the chain is semi-infinite, the yield would just be a new semi-infinite chain. Continuing this way the system can be reduced to a finite size which can actually be inverted. Say one continues to remove every second element in the chain, then in the end, the cells would be too far apart to interact an no hopping between cells would occur. At this point the recursion should stop. More on how this is done practically later.  For now one just have to keep in mind that the removing of cells in the chain effectively changes to coupling between them and this is where \textit{self-energy} comes in. The self-energy is what describes the effective coupling between a cell and the rest of the semi-infinite chain. And it can be derived by looking at a cell at the very end of the semi-infinite chain and see how it couples to the rest. First one needs the Green's functions. The Green's matrix for this single cell would be given by the equation in \cref{Greenssolved}. This is before when only one cell and thus one matrix had to be considered. But now, there is an semi-infinite amount of cells and an semi-infinite amount of matrices to consider. However, the cell in the end of the chain only interacts with the cell next to it and so on. Considering this one can write op an equation equivalent to that of \cref{Greensunsolved} but as system of matrix equations for the chain.
\begin{align}\label{Greenssystem}
	\begin{pmatrix}
		z\mathbf{1}-\mathbf{H}_c & -\mathbf{V}^{\dagger} \\ -\mathbf{V} & (z-\varepsilon')\mathbf{1}
	\end{pmatrix}
	\begin{pmatrix}
		\mathbf{X}      & \mathbf{G}_{0c} \\
		\mathbf{G}_{c0} & \mathbf{G}_{00}
	\end{pmatrix}
	=
	\begin{pmatrix}
		\mathbf{1} & \mathbf{0} \\
		\mathbf{0} & \mathbf{1}
	\end{pmatrix}
\end{align}
where \(\varepsilon'\) is the on-site Hamiltonian of the first cell, \(z\) is \(E+i\eta\),  \(\mathbf{G}_{0c/c0}\) is the Green's matrices coupling the cell to the rest of the chain and \(\mathbf{X}\) is the Green's matrices for the rest of the chain. This is also assuming one knows the Green's function within the chain \(\mathbf{G}_c\) and that the chain has constant hopping and on-site elements \(\mathbf{H}_c,\mathbf{V},\mathbf{V}^{\dagger}\).
Solving this system for \(\mathbf{G}_{00}\) and eliminating \(\mathbf{G}_{0c}\), which is unknown, one gets
\begin{align}\label{greenszero}
	\mathbf{G}_{00}(z) & = \left[(z-\varepsilon')-\vb{V}(z\vb{1}-\vb{H}_c)\vb{V}^{\dagger}\right]^{-1} \\
	                   & = (z-\varepsilon'-\Sigma(z))^{-1}
\end{align}
where \(\Sigma(z)\) is the self-energy. One can isolate the self energy from the equations above to
\begin{align}\label{selfformal}
	\mathbf{\Sigma}(z) = \mathbf{V}[z\mathbf{1}-\mathbf{H}_c]^{-1}\mathbf{V}^{\dagger}
\end{align}
So from \cref{greenszero} it can be seen that the solution to the system of matrices for the Green's matrix \(\vb{G00}\) is the same as in \cref{Greenssolved} but with a correction (\cref{selfformal}), which is the self-energy. And that is what describes the coupling for the first cell to the rest of the chain. This concludes the formal introduction to Green's functions and self-energy.\subsection{Obtaining first cell self-energy and Green's matrix through programming}\label{recursionroutinesec}
For simplicity and in order to check whether the routine would yield the expected results, the system in \cref{inlinepointplot} is used as an example. The goal is to get the Green's functions for the centre unit cell in the semi-infinite chain and the self-energies coupling to rest of the chain right and left. Specifically for the simple system one should imagine first having one centre unit cell like \cref{inlinepointplot} and then repeating it infinitely in the left and right direction. The fact that there is a left \textit{and} right self energy is that the unit cell lies within the semi-infinite chain and not at the very end as described in \cref{greensandself}. To be assured, this does not conflict with any of the preciously mentioned formalism and the left and right self-energies are quite easily obtained as one shall see shortly. As mentioned the goal is to get the Green's functions of a specific unit cell and the self-energies related to it. If the Green's matrix \(\mathbf{G}\) represents the whole chain, then the equation of the whole system would be equivalent to that of \cref{Greensunsolved}. Considering the Green's functions for specific unit cell in question, it would correspond to one column in the system of equations, say the first. One can define the on-site Hamiltonian \(\mathbf{h}_0\) for the specific unit cell and its hopping matrices \(\mathbf{V},\mathbf{V}^{\dagger}\).  The two hopping matrices correspond to hopping left or right in the chain respectively. These can be obtained using the functions already developed in \cref{hamilsec}. Throughout this section they will be named \(a_0 = \mathbf{V}^{\dagger}, \ b_0 = \mathbf{V}, \ e_{s0} = \mathbf{h}_{s}\). The recursion is an iterative process and so the zero index indicates the starting point of the iterations and the \textit{s} index indicates that it is the Hamiltonian of the specific wanted cell. One can also define a Green's function for a single unit cell as \(g_0 = (z-e_{0})^{-1}\) just like \cref{Greensunsolved} where \(e_{0}=\mathbf{h}\) which is the on-site Hamiltonian of the other cells. With these elements a system of equations, similar to \cref{Greensunsolved} can be setup. The first difference being that the identity matrix is replaced by its first column, because the solution of interest is that one first column in the Green's matrix. The second is that the first element in the Hamiltonian matrix \(\mathbf{H}\) is related to the specific single unit cell \(\mathbf{h}_s\). Next a range of multiplications of the different elements stated so far will be shown, and afterwards it will be explained how these affect the system of equations to give recursion. The multiplications are:
\begin{align}
	a_1    & = a_0 \times g_0 \times a_0                  \nonumber                      \\
	b_1    & = b_0\times g_0\times b_0                   \nonumber                       \\
	e_1    & = e_0 + a_0\times g_0\times b_0 + b_0\times g_0\times a_0 \label{matmulrec} \\
	e_{1s} & = e1_{0s} + a_0\times g_0\times b_0          \nonumber                      \\
	g_1    & = (z - e_1)^{-1} \nonumber
\end{align}
These equations constitutes the first iteration in the recursion and they can be repeated indefinitely. In the matrix system of equations these multiplications effectively shifts all elements in the matrix containing on-site Hamiltonians and hopping matrices by one column. Because the matrix is diagonal, it will leave the first column of the matrix empty. The column can then be removed and this is exactly what corresponds to removing a cell in the semi-infinite chain. Keeping on doing these multiplications, raising the index by +1 every time, one can move through the system as a whole, removing columns (cells) in the system of equations, thus reducing it to a finite size. In \cref{recurfunc} the loop developed to get recursion.
\im{Listings/Functions.py}{92}{104}
\vspace{-.5\baselineskip}
\captionof{listing}{The while loop in the recursion routine. The matrix elements are overwritten with the new variables until the resulting matrix is small enough to invert\label{recurfunc}}\vspace{\baselineskip}
Note that some intermediate multiplications are made e.g. \textit{ag = a0 @ b0}. This is for run-time optimisation only, as these products are used multiple times per iteration.
The recursion is run until a threshold is met. The threshold is determined by the value of the hopping matrix \(a_0\). As it reaches a value close to zero, there is no longer any effective interacting (hopping) between the cells because of removal of cells and the recursion should stop.
In the end one will obtain re-normalised Hamiltonians and hopping matrices which is then used to get the Green's functions and self-energies through these simple equations:
\begin{align}
	\mathbf{\Sigma}_R & = e_s - h                   \nonumber       \\
	\mathbf{\Sigma}_L & = e - h - \mathbf{\Sigma}_R \label{outputs} \\
	\mathbf{G00}      & = (z - e_s)^{-1} \nonumber
\end{align}
These are calculated out of the Python function's while loop as such:
\im{Listings/Functions.py}{106}{109}
\vspace{-.5\baselineskip}
\captionof{listing}{Self-energies from left and right as well as a normalised Green's functions-matrix are calculated at the end of the recursion loop.\label{RecurOutputs}}\vspace{\baselineskip}
This concludes how recursion works and how the first cell Green's function as well as the self-energies are obtained.
\subsection{Plotting the real and imaginary part of the first cell Green's function}
% \begin{wrapfigure}[10]{r}{.45\textwidth}
% 	\vspace{-2em}
% 	\includegraphics[width=0.45\textwidth]{Figures/BetaimrealTE.eps}
% 	\caption{A plot showing the real and imaginary part of Green's function at the zeroth site resulting from the recursion routine on the simple system. Note that the yellow imaginary part is the representation of the local density of states.}\label{LDOSsimple}
% \end{wrapfigure}
One of the results possible to obtain via the recursion routine is the Green's function of the centre unit cell in relation to the rest of the chain. As mentioned the imaginary part of the elements Green's matrix is the LDOS of the different sites in the unit cell. With a relatively simple approach, the Green's matrix elements can be obtained as a function of energy, using a \textit{for loop}, looping over a range of energies which is then used as input in the \textit{RecursionRoutine} function (\cref{recurfunc}), see \cref{plotcode}:
\im{Listings/SelfEnergyByRecursion.py}{64}{68}
\vspace{-1\baselineskip}
\captionof{listing}{Code showing the loop which produces the complex Green's function (or y-values) for a range of energies used in the plot in \cref{LDOSsimple}\label{plotcode}}\vspace{\baselineskip}
\begin{figure}[H]
	\centering
	\includegraphics[width=.45\textwidth]{Figures/BetaimrealTE.eps}
	\caption{A plot showing the real and imaginary part of Green's function at the zeroth site resulting from the recursion routine on the simple system. Note that the yellow imaginary part is the representation of the local density of states.}
	\label{LDOSsimple}
\end{figure}
Taking the imaginary part of the output in \cref{plotcode} gives information about the LDOS at a specific energy and place in space, namely a specific atom in the unit cell. The resulting plot for the simple system (at atom index 4) can be seen in \cref{LDOSsimple}. Note that the plot only represents the LDOS for a specific site on the molecule and that they may change radically from site to site (see \cref{appfigs}, \cref{siteLDOSplot} for an example using the same system as \cref{inlinepointplot}). The site can be changed by choosing another index in \cref{plotcode} line 68, which corresponds to the atom indices in \cref{inlinepointplot}.


\section*{Non-Negative Matrix Factorization}
Want to learn words $w$ in a document $d$. Use topic latent variable:
$\mathbf{X} \in \mathbb{Z}^{N \times M}_{\geq 0}$, NMF: $\mathbf{X} \approx \mathbf{U^\top V}, x_{ij}=\sum_z{u_{zi}v_{zj}}=\langle\mathbf{u}_i \mathbf{v}_j\rangle$
$\mathbf{U},\mathbf{V}$ are non-neg., $L_1$ col normal.
\subsection*{Probabilistic LSA}
\textbf{Context Model:} $p(w | d) = \sum_{i=1}^K p(w | i) p(i | d)$\\
\textbf{Conditional independence assumption ($*$):}\\
$p(w|d) = \sum_i p(w,i|d) = \sum_i p(w|d,i)p(i|d) \stackrel{*}{=}$\\$\sum_i p(w|i)p(i|d)$ \quad Note $p(w|i):=p(w|z=i)$\\
\textbf{Symmetric parameterization:}\\
$p(w, d) = \sum_z p(z)p(w | z) p(d | z)$
\subsection*{EM for MLE for pLSA (NO global opt guarantee)}
Log-Likelihood: $L(\mathbf{U}, \mathbf{V}) = \sum_{w,d} X_{w,d}\log p(w|d)$ \\
$ p(w|i) = u_{wi}$, $p(i|d) = v_{id}$, $\sum_w u_{wi} = \sum_i v_{di} = 1$\\
$q_{iwd} \in \{0,1\}: w \text{ in } d \text{ generated via } z=i$. \\
Lower bound from Jensen: $\sum_{w,d}\log \sum_{z=1}^K q_{iwd} \frac{u_{wi}v_{id}}{q_iwd} \geq \sum_{w,d,i} q_{iwd} [\log u_{wi} + \log v_{id} - \log q_{iwd}]$\\
Don't forget to add the sum over $i,j$ and $X_{ij}$ again.
E-Step (optimal q: posterior $p(z=i|w,d)$):\\
$q_{iwd} = \frac{p(w|i)p(i|d)}{\sum_{k=1}^K p(w|k)p(k|d)} := \frac{u_{wi}v_{id}}{\sum_{k=1}^K u_{wk}v_{kd}}$, $\sum_i q_{iwd}=1$\\
M-Steps:\\
$p(w|i)=u_{wi} = \frac{\sum_d q_{iwd}X_{wd}}{\sum_{w,d}q_{iwd}X_{wd}}, \
p(i|d)=v_{id} = \frac{\sum_w q_{iwd}X_{wd}}{\sum_{w}X_wd}$ \\
\textbf{Derivations:} maximize lower bound w.r.t. $\sum_w u_{wi}=1$ and $\sum_i v_{id}=1$ (no $>=0$ constraint bc. $\log$). $\min_{U,V} \max_{\alpha, \beta}\mathcal{L}$ where $\mathcal{L} = -g(X; U, V) + \sum_i \alpha_i(\sum_w u_{wi} - 1) + \sum_d \beta_d (\sum_i v_{id} - 1)$. Set $\partial \mathcal{L} / \partial u_{wi} = 0$ and get $u_{wi} = \sum_d X_{wd}q_{iwd}/\alpha_i$. Setting $\partial \mathcal{L}/\partial \alpha_i$ gives $\sum_w u_{wi} = 1 \to \sum_{w,d} X_{wd} q_{iwd} / \alpha_i = 1 \to \alpha_i = 1 / (\sum_{w,d} X_{wd} q_{iwd})$ then plug it in. Similar for $v_{id}$ but with extra step: $\sum_w X_{wd} \sum_i q_{iwd} / \beta_d = 1 \to \beta_d = 1/(\sum_w X_{wd})$ since $\sum_i q_{iwd}=1$.

\subsection*{Latent Dirichlet Allocation}
To sample new $d$, need to extend $X$ and $U^T$ (in pLSA matrix dims fixed). For each $d_i$ sample topic weights $\mathbf{u}_i$\textasciitilde Dirichlet($\alpha$): $p(u_i|\alpha) = \prod_{z=1}^K u_{zi}^{\alpha_k-1}$, then topic $z^t$\textasciitilde Multi($u_i$), word $w^t$\textasciitilde Multi($v_{z^t}$)\\
LDA Model: $p(\mathbf{x}|V,u) = \frac{l!}{\prod_j \mathbf{x}_j!}\prod_j \pi_j^{\mathbf{x}_j}$
where $\pi_j=\sum_z v_{zj} u_z$, $l=\sum_j x_j$ and $l = \sum_j x_j$\\
Bayesian averaging over $\mathbf{u}$: \\
$p(\mathbf{x}|\mathbf{V},\alpha)=\int p(\mathbf{x}|\mathbf{V},\mathbf{u})p(\mathbf{u}|\alpha)d\mathbf{u}$
\subsection*{NMF Algorithm for Quadratic Cost Function}
$\min_{\mathbf{U}, \mathbf{V}} J(\mathbf{U}, \mathbf{V}) = \frac{1}{2} \|\mathbf{X} - \mathbf{U}^\top\mathbf{V}\|_F^2$
s.t. $\forall i,j,z:u_{zi},v_{zj} \geq 0 $ (non-negativity)\\
Comparison with pLSA: different sampling model (Gaussian not multinomial), different objective (quadratic not KL divergence), not normalized \\
ALS (not joint convex over (U,V)):\\
1. init: $\mathbf{U}, \mathbf{V} = rand()$ 2. repeat 3\textasciitilde4 for $\mathit{maxIters}$:\\
3. upd. $(\mathbf{VV}^\top)\mathbf{U} = \mathbf{VX}^\top$, proj. $u_{zi} = \max \{ 0, u_{zi} \}$\\
4. update $(\mathbf{UU}^\top)\mathbf{V} = \mathbf{UX}$, proj. $v_{zj} = \max \{ 0, v_{zj} \}$


\documentclass[addpoints]{exam}

\usepackage{subfigure}
\usepackage{caption}
\usepackage{graphbox}
\usepackage{hyperref}
\usepackage{listings}
\usepackage{multirow}
\usepackage{tabularx}
\usepackage{graphicx}
\usepackage{xcolor}
\usepackage{amsmath}

% Header and footer.
\pagestyle{headandfoot}
\runningheadrule
\runningfootrule
\runningheader{Probability and Statistics}{Final Project}{Spring 2021}
\runningfooter{}{Page \thepage\ of \numpages}{}
\firstpageheader{}{}{}

\title{Final Project}
\author{MATH 310/EE 354 Probability and Statistics\\Habib University\\Spring 2021}
\date{Akeel Ather Medina}

\begin{document}
\maketitle

\section{Random Walk}
\begin{questions}
\question
For this part, we need to take input for the probability, and number of walks. The iteration count has been set to 1000, which means we will end up with 1000 expected final positions.\\
I created an array of size 'iterations' to hold these ending positions, and now for the main calculation, we create a nested loop, of which the inside loop runs 'n' times, and the outer loop runs 'iterations' times. \\
The expected final position is reset to 0 after each walk, and inside the loop, based on the input probability, if we generate a random value from 0 to 1, using np.random.random(), we can determine which way to walk by comparing it to the probability input. \\
The calculation for bins was done, and reused several times in this assignment, using code from stackexchange which is linked in the references tab.\\
This bin width and count is ideal for all situations so value of bins does not need to be changed for different iterations and values of p and n.\\
\\
To begin with, if we plot 100 steps, with the probability of moving along the x-axis as 0.5, given iterations is set to 1000, this is the plot we get:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_1_1.png}
\end{center}
Immediately we can see that this plot follows a normal distribution. Although the expected value does not come out to be 0 every time, it is more likely for it to be nearer to 0 than to be farther away from it. \\
Next we will keep the number of walks similar and test out different probabilities.\\
With $p=0.75$:
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_1_2.png}
\end{center}
The main difference we can note here is that the mean of our distribution is now around 50, instead of 0. Intuitively speaking, this is because we now have a higher probability of moving to the right, so in 100 steps, if we take 75 to the right and 25 to the left, we will end up on '50'.\\
With $p=0.25$:
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_1_3.png}
\end{center}
This is a similar situation to the above, but since the probability of moving right is now less likely, out of 100 steps, we will likely move left 75 times, and right 25 times, leaving with an approximate value of 50, as we can see in our plot.\\
With $p=0.99$:
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_1_4.png}
\end{center}

For $p=0.99$ although it does not seem obvious whether this plot follows a normal distribution, the intuitive approach that came to my mind was to increase the number of steps taken. The reason for this is that for such a high probability, with only 100 steps taken a trend cannot be accurately measured because the number of times we will take a step left is very low. Changing the number of steps, although the probability remains the same, gives a chance to observe a normal distribution again because more steps left can be observed. \\Thus this is the plot of $p=0.99$ with $n=1000$.\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_1_5.png}
\end{center}

\newpage
\question

For this part, the function is exactly the same except instead of an if-else condition to move left or right, it is now an if-else-if condition, so that in case we are at the point x=1, we cannot move left.\\
The bin width and count is calculated the same way as in the previous part, with a few adjustments to make the trend more obvious.\\
So to highlight the difference with the previous part, I will plot the values in the same order, with n as 100:\\ \\
With $p=0.5$:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_2_1.png}
\end{center}
We can immediately see this looks like the right half of the graph we got for part 1. \\
With $p=0.75$:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_2_2.png}
\end{center}
Increasing the probability of moving right leads to the same normal distribution from the last part.\\ \\
With $p=0.25$:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_2_3.png}
\end{center}
If we were to only consider the positive values from our part 1 equivalent, this plot shows the few positive results we get. In the part 1 equivalent, the mean was near -50, so that is why this graph has such few points plotted, as final values less than 0 are not included.\\ \\
With $p=0.99$:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_2_4.png}
\end{center}
This is identical to our result for the same values in the previous part, so we will continue like before increasing the number of steps, to see if those observations still hold.\\ \\
With $p=0.99$:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_2_5.png}
\end{center}
This is very similar to our plot in part 1.\\ \\
With $p=0.25$:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_2_6.png}
\end{center}
We observe that there is no obvious change for $p=0.25$ even with an increase in steps. This is to be expected because the number of steps would not matter for a normal random variable, the mean and standard deviation would remain the same.

\newpage
\question

For this part we start our code with taking all 4 inputs, the starting point of the objects and the probability they move right. Proceeding with the same approach as in the previous parts, we iterate 'iterations' number of times, and in each iteration, we set the positions of the objects equal to what we previously input. We start a while loop so that only once the objects meet on the real line will the steps be recorded and the next iteration begins. Inside the loop, both objects each move once, and the steps taken for meeting is also incremented.\\
It is important that our starting points have an even distance between them or else it will loop infinitely.\\
The process for determining a right or left step remains the same. First, taking a random value from 0 to 1, and determining if they are less than the probability input, then we move right, else left. Two seperate random values have to be generated for each object.\\
\\
The first plot I tried to compute was for $d1=1$, $d2=-1$, $p1=0.5$ and $p2=0.5$. It was taking too long to compute, but when I tried it again, it worked in just 20 seconds. The plot, however, looked like this:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_3_1.png}
\end{center}
Even after manually adjusting the bins, it was difficult to see all other points, so I printed the max value of the total steps taken, to get 9398093. This value varies greatly, and since there is a much higher probability of the points converging immediately, rather than divering and then converging again, we can only see this one line on the histogram. So i used lines 5-7 from Question 2 part 2 to observe the graph without that cluster of points in the early stage to see if there was a pattern.
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_3_2.png}
\end{center}
Values lesser than 500 have been removed from our array from which we plot the values. This code has been changed to do the opposite function as that in Question 2 part 2. However, it should be noted that the calculation for bins needs to be commented out when running this, and the argument for the histogram can be arbritrarily assigned, such as to 100. \\
So we can see that for $d1=1$, $d2=-1$, $p1=0.5$ and $p2=0.5$, the steps taken can vary greatly, and the only observation we can make is that for greater total steps, similar results are more sparse. They are clustered together near the beginning. This can be imagined as a graph over time as well. More results are recorded at less 'time' (or steps), and very few the greater the amount of time taken is. There is a lesser probability of steps taken being high.\\ \\
For $d1=1$, $d2=-1$, $p1=0.6$ and $p2=0.6$, the code does not run, and there is a memory error. For different identical values of $p$ as well, such as $0.4$. It is likely that some of the iterations encounter a case where the two objects are chasing after each other and this carries on for too long a time for the computer to compute. \\
\\
For $d1=1$, $d2=-1$, $p1=0.4$ and $p2=0.6$, we get this graph:
\begin{center}
\includegraphics[width=.48\textwidth]{images/p1_3_3.png}
\end{center}
It is obvious that this follows a normal distribution, and so do other plots with a similar nature, such that the probability of the two objects moving together is higher than moving away from each other.
\end{questions}

\newpage
\section{Picking a random point correctly}
\begin{questions}
\question
For this question, I have taken one total input for the radius, which is used by all the parts. Changing the radius has no effect on the graph plotted for all parts.\\
For the first part, since we are now doing a scatter plot, where we need to plot the x and y coordinates of our points, we need 2 different lists to store our values in. To calculate each set of x and y coordinates in the mentioned method asked by the question, in each of 'n' iterations, we randomly calculate a radius, from 0 to the radius limit input by the user, using np.random.uniform. Functionally, this is also identical to the np.random.random function. The angle, $\theta $ is also calculated the same way.\\
To convert this into a plottable set of x and y values we use the formula to convert from polar to cartesian, so $x=r*cos(\theta)$ and $y=r*sin(\theta)$.\\
With iterations set to 50000 for an ideal number of points plotted, each point has a color from the map called 'twilight shifted', which highlights the uniformity of plots, or lack of, which we will see in this part.\\
The code for the plotting of the boundary of the circle was taken from stack exchange.\\
The output of this function looks like:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p3_1.png}
\end{center}
The scatter plot remains similar for all radius, and the aspect ratio is maintained as a square, with boundaries 1.2 times the diameter of the outer circle. \\
We can see that values closer to the center are more clustered together as compared to ones farther away. The reason for this is that if we compare the area of say half the radius of our outer circle, it will not have half the area. This is because area is determined by $\pi r^2$ and thus is not linear. This means that even if there are an equal amount of points in the inner and outer part of the circle, it will not be uniform because the outer circle has more area to fill, so points are more sparse.\\
Even if we randomly choose radius and $\theta$, the number of plots would have to increase for a higher radius.\\
Another way to think of this is if we take some random angles and draw lines from the origin to the outer circle, then draw points on the intersections of the inner and outer circles, it will seem as if the inner circle has more points because the lines move farther apart the further from the origin we go.\\
\\
The variation of this plot was easily calculated using the np.var function, taking in our list of X coordinates which we use to plot. For 3.1, we get a variation of 0.17 for a radius of 1, and a variation of 4.16 for radius 5.

\newpage
\question
The method for plotting remains the same, the only difference in this part lies in the method of choosing the x and y coordinates. This time, we actually calculate the x and y values in a square of 2*'radius limit'. We randomly choose an x and y coordinate from between the positive and negative values of the input radius, seperately. This is effectively choosing values from a square. We add a while loop so that until the distance of these x and y coordinates from the origin is not within an imaginary circle of specified radius, we keep on choosing new x and y coordinates. This also ensures our data has the same number of elements as compared to the other parts of this question.\\
As with the previous part, the plot remains the same for all kinds of input radius, so this is the output we get:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p3_2.png}
\end{center}
We can clearly see the difference with the previous part. There is no clustering in the center, and the points are distributed uniformly across the circle. Since we are now uniformly choosing x and y coordinates, it seems likely that given any space on the cartesian plane, sampling x and y coordinates uniformly will give a uniformly filled space. In the previous part, even if we randomly chose radius and $\theta$, the number of plots would have to increase for a higher radius, but over here there is no relation between that and the x and y coordinates. They are independant.\\
\\
The variation of this part using the same function as the previous part gives 0.25 for a radius of 1, and 6.28 for a radius of 5.

\newpage
\question
For this part, we again do the plotting the same way as in the previous parts. The difference lies with how we pick points. In the last part, we found a way to uniformly pick points from a circle using x and y coordinates. In this part, we want to alter our method in the first part, 3.1, which sampled uniformly from polar coordinates instead of cartesian, such that our circle will have points uniformly spread, not clustered in the center. \\
The question first asks that if we plot a circle with radius 1 and a circle with radius 2, is the probability of a point lying in the larger circle 4 times the smaller one in 3.1. This is not the case, since in that part, the radius and angle were chosen randomly. Since a point being within a circle is independent of angle, we know that approximately 50 percent of the time, the radius will be less than 1, and 100 percent of the time it would be lesser than 2. This proves the above question is not true, because the probability of radius being lesser than 2 should be 4 times the probability of it being lesser than 1. \\ \\
We can prove this by using the code created in question 4 for the same reason (to prove that points chosen using 3.2 and 3.3 adhere to this question states above, while 3.1 does not). Using the approach in 3.1, the number of points randomly plotted in a circle of a certain radius, is two times that of a circle with half this radius. However, our approach in 3.2 always gives an approximate value of 4, meaning that a point being the the outer circle is 4 times more likely, thus our approach in that question uniformly picks a point over a circle.\\
Based on this logic, we can try and develop a way of picking radius and angle such that it will be uniformly spread over the circle.\\
So our main goal is to make sure that we calculate our random point in such a way that it has a greater amount of larger values, given we are choosing it uniformly over a range.\\ \\
My first approach was to square random values and accept only if they lie within the radius of the circle, as in part 2, but this also gave a clustered center. \\
My next approach was to randomly pick a value from 0 to the square of the radius, and then take the square root of the answer, however this also resulted in a graph with a clustered center.\\
\\
Next, i ended up using an online source which is the second link in the References tab, for which I tried and ended up getting a uniform distribution. However no explanation was given, so I tried to solve the distribution given as a bonus because I could not guess how to properly choose r and $\theta$ to get random points uniformly, and ended up with:\\
\begin{center}
$k = \frac{1}{\pi r^2} $\\
$P(r \leq R) = \frac{1}{R^2}*r^2$\\
$\sqrt{P(r \leq R)} = \frac{r}{R}$\\
\end{center}
What I figured from this is that to choose a random point in the circle, we need to take the square root of the probability of it lyring inside a circle fo radius r $\leq$ R, and multiply it with the radius R itself. Since the probability is between 0 and 1, this makes sense as the square root of numbers lesser than 1 gives a value greater than itself (as shown in 4.4), or in other words, further from the center. Since we want a random point inside our circle, the main line of code seperating this approach from 3.1 is:\\
\begin{center}
$np.sqrt(np.random.random()) * radiuslimit$\\
\end{center}
This line of code is similar to the line I saw in the online source. The graph plotted for this formula for choosing a point, similar to 3.2, looks like:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p3_3.png}
\end{center}

The variation of X coordinates for a radius of 1 is 0.25, and 6.25 for radius 5. The variance of X coordinates in 3.2 and 3.3 is approximately the same.

\end{questions}

\newpage
\section{Saying random is not enough}
\begin{questions}
\question
In this question, we need to randomly generate 2 angles between 0 and 2*pi, and the chord length is the distance between the two points of where the angle extends to touch the circle. If we calculate this for any two arbritary angles, we could imagine it as a triangle being formed with two sides being of length 'radius', and the last side being the chord length. To calculate this chord length we can divide the triangle into two halves, and apply basic trigonometry to get the chord length. That is:\\
\begin{equation}
 c = 2 * r * \sin{(|a2-a1|/2)}
\end{equation}
Where c is the length of the chord, a1 and a2 are the angles randomly chosen each iteration, and r is the radius. We divide the angle by 2 because we can only apply $\sin(\theta) = \frac{Opposite}{Hypotenuse}$ if we have a right triangle, which we can do by splitting the triangle we form with angle $|a2-a1|$ into 2 parts. This only gives half the chord length, so multiplying by two gives our final chord length. \\
\\
The plot of chord length generated by this formula looks like: \\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p4_1_1.png}
\end{center}
Note that bin calculation remains the same as in previous parts. The only difference is that we have set the y axis to plot probability of occurrance. \\
We can see that the highest probability of chord length is that for it being approximately 10, then greately decreasing. The radius for all questions was chosen as 5. This means that the angles being 180 degrees apart is the most likely scenario, as when the angles are 180 degrees apart, the chord between them will give us a length of 10 for this circle. We can confirm this, as if for example angle 1 is 180 and angle 2 is 0, we will do sin(90) which gives 1. Thus our final answer will be 10, the maximum chord length.
\newpage
\question
In this second approach, we need to choose a random angle, and randomly pick a point from the angle extended to the boundary of the circle, from the origin. This point will be the perpendicular bisector of some chord, and that is how we choose it. Now one thing to note is that when calculating the chord length, we do not actually need the angle. This is because the chord length can simply be calculated using pythogoras theorem. We know this perpendicular bisector meets two points of the circle, and any point on the circle to the origin is of length radius. Next, we ourselves are choosing this random point from 0 to radius, thus we also know the length of the line from the origin to the point, and the origin to the point on the circle where the bisector cuts. Thus, by manipulating pythogoras theorem we get:\\
\begin{equation}
 c = \sqrt{r^2 - p^2}
\end{equation}
Where r is the radius, and p is the random point on the line. However, as we noticed in the last question, choosing a random point from 0 to the radius (as the questions asks us to), we will not get a uniform distrubution of points on this circle. As proof, I assigned 2 variables as counters for our points generated, if these points (they are not x and y points, but a polar radius) are lesser than radius (as all of them must be, giving a number the same as 'iterations), increment the variable called outer by 1, and if they are lesser than $\frac{radius}{2}$, then increment inner by 1. The reason for doing it this way is it is the example given to us in 3.3. If we have two circles, one of double the radius, it should have 4 times the amount of area (or points plotted inside it). This code works for any radius and any number of iterations, and we always approximately get an output of 2, when we compute $\frac{outer}{inner}$. This shows that the center of the circle is more clustered and has more points than the outer circle.\\
Back to the question, our distribution of chord lengths looks like:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p4_2_1.png}
\end{center}
We can see the difference with this and the last part. There are very few chords with a small length, and many with a higher length. The reason could be as shown before, that points with a location nearer to the origin will have a much larger chord length than to the edge.
\newpage
\question
For this question, to pick a random point uniformly from a circle i re-used my code from 3.2, in which a random x and random y value is chosen from the square of side 'radius', until it is physically within a circle of radius 'radius'.
The code is otherwise identical to the previous part. For our calculation of the point in the circle, we need it in terms of radius so that we can apply the same pythogorean manipulation as before. The formula for this is:\\
\begin{equation}
 p = \sqrt{x^2 + y^2}
\end{equation}
Where p is the 'radius', in cartesian to polar form. \\
I have commented out the approach used in 3.3 for a random point as an alternative approach, it also gives the same plot.\\
The difference in this part and the previous part can be seen by our results of inner and outer at the end of our iterations, this time $\frac{outer}{inner}$ always gives an approximate value of 4, as compared to 2. This is how we know that our point being chosen is uniform over our circle. Our distribution looks like:\\
\begin{center}
\includegraphics[width=.48\textwidth]{images/p4_3_1.png}
\end{center}
This distrubution looks widely different from the previous ones. It is linearly decreasing, with the most chord lengths being of maximum length possible. I will elaborate on why I think this approach is the most accurate in the next part.\\
\newpage
\question
Our approach in 4.3 intuitively should give the most accurate random chord length, the reason being that it is the only part where the points chosen are also uniform over a circle, not biased like in 4.2. A possible explanation for the nature of our plot in 4.3 could be that, for example, if we consider a circle of radius 1. Now lets use our way of calculating chord length. For the points 0.1, 0.2, 0.5, 0.9: \\
\begin{center}

$\sqrt{1^2 - 0.1^2} = 0.995$    \\
$\sqrt{1^2 - 0.2^2} = 0.980   $   \\
$\sqrt{1^2 - 0.5^2} = 0.866     $   \\
$\sqrt{1^2 - 0.9^2} = 0.436 $    \\
\end{center}
The result does not decrease linearly, and thus our final plot, although the chord lengths are random, is not greatest for values in our 'outer circle', although now the outer circle has 4 times the amount of points as the inner. Also we should note that our approach in 3.3 has made sure this holds for all radius, as the distribution is between 0 and 1, multipied by a constant, the max radius.
\end{questions}

\newpage
\section{References}

\noindent\fbox{%
    \parbox{\textwidth}{%
\url{https://stackoverflow.com/questions/33203645/how-to-plot-a-histogram-using-matplotlib-in-python-with-a-list-of-data}{The code for calculation of bins for all histograms was taken from here.}\\
\url{https://blogs.sas.com/content/iml/2016/03/30/generate-uniform-2d-ball.html}{Referred to a line of code on this website for 3.3}
    }%
}

\end{document}


\documentclass{mrl}

\title{Application of Bulletproofs in MKEcoin Transactions}
\authors{Sarang Noether\footnote{\texttt{sarang.noether@protonmail.com}}}
\affiliations{MKEcoin Research Lab}
\date{\today}

\type{TECHNICAL NOTE}
\ident{MRL-XXXX}

\begin{document}

\begin{abstract}
This technical note briefly describes the proposed application of Bulletproofs \cite{bp} in MKEcoin. The proofs are used as a drop-in replacement of the existing Borromean bitwise non-interactive zero-knowledge range proofs used to show that a committed amount is in a specified range. Bulletproofs reduce both proof size and verification time, as well as provide a straightforward method for batch verification of proofs from multiple transactions. We describe our implementation, noting specific areas of optimization from the original paper.
\end{abstract}

\section{Introduction}
The implementation of confidential transaction amounts in MKEcoin is accomplished using homomorphic commitments. Each input and output amount, including fees, is represented by a commitment of the form $vG + \mu H$, where $G$ and $H$ are elliptic curve generators, $v$ is the amount, and $\mu$ is a mask. Without knowledge of the commitment opening, a third party cannot determine the amount; however, it is trivial for the third party to convince itself that a transaction balances (that is, that the difference between inputs and output amounts is zero). The homomorphic property of the commitments is such that the difference in commitments must itself be a commitment to zero.

However, this is not sufficient to ensure a correct and safe transaction model. An adversary could easily construct a combination of positive and negative outputs such that the transaction amounts balance. A third party would still verify that the transaction balances, though the adversary has effectively printed free money in an undetected fashion. To combat this, we require that each amount commitment come equipped with a \textit{range proof} that convinces a verifier that the corresponding output is both positive and does not risk an overflow by being too large. The range proof scheme must be non-interactive and zero-knowledge; that is, the verifier does not need to communicate with the prover once the proof is generated, and the proof itself reveals no information about the amount except that it is within the stated range.

The current range proof style used in MKEcoin confidential transactions is a \textit{Borromean bitwise} range proof. To generate a proof that a commitment $C \equiv vG + \mu H$ represents an amount $v \in [0,2^n-1]$ for some bit length $n > 0$ (in MKEcoin $n = 64$), the prover generates separate commitments for each bit. The prover then generates a Borromean ring signature showing that each commitment is to either $0$ or $2^i$ for appropriate $i$. Any third-party verifier can then convince itself that the bit commitments reconstruct the committed amount, that each commitment is to either $0$ or $2^i$, and therefore that the committed amount lies in the correct range.

However, this comes at a cost. Borromean bitwise proofs scale linearly in size with the number of bits in the range. Further, if multiple outputs are used in a transaction, a separate proof is required for each. Each proof is large, taking up $6.2$ kB of space.

\section{Bulletproofs}
Bulletproofs are a recent general non-interactive zero-knowledge proof construction \cite{bp}. Using a novel inner product argument, they can be used in a variety of applications ranging from range proofs (pun intended) to verifiable shuffles and even proofs of general arithmetic circuit evaluation. For our purposes, they can accomplish the same goal as Borromean bitwise range proofs: convincing a verifier that a committed amount is within a claimed range.

The details of Bulletproof construction, both for prover and verifier, are discussed in the paper \cite{bp}, so we will not duplicate them here. However, several definitions are useful when discussing the scaling. A standard Bulletproof that shows an amount is within the $n$-bit range $[0,2^n-1]$ is called a \textit{single-output proof} or a \textit{1-proof}. However, it is possible for a prover to construct a single proof showing that $m$ separate amounts (with separate random masks) each lie within the range $[0,2^n-1]$, where $m$ is a power of two. Such a proof is called an \textit{aggregate proof} or, more precisely, an $m$\textit{-proof}. The scheme is constructed in such a way that a single-output proof is trivially an $m$-proof with $m=1$ (which simplifies the code). It is important to note that the construction of an aggregate proof requires that the prover know each amount and mask; this means that while it is useful for all outputs in a transaction to be contained within a single aggregate proof for space savings, it is not possible for a third party to take existing proofs and construct an aggregate proof, either within a single transaction or between different transactions.

The size scaling benefits of Bulletproofs occur at two levels:
\begin{enumerate}
\item \textbf{Bit length of range}. The size of a Bulletproof increases logarithmically with the number of bits in the range. In bitwise range proofs, the proof size increased linearly with the number of bits.
\item \textbf{Number of amounts in aggregate proof}. The size of a Bulletproof increases logarithmically with the number of amounts included in a single aggregate proof. In bitwise range proofs, the proof size increased linearly with the number of bits (since a separate proof was needed for each amount).
\end{enumerate}
We discuss efficiency in more detail below.

There is a separate scaling argument that is useful. A new node that comes online will receive many $m$-proofs, at least one per post-Bulletproof transaction in the blockchain. Instead of verifying each of the proofs separately, the node can perform a \textit{batch verification} of as many proofs at a time as it wishes. As described below, this process requires that certain portions of each proof be verified separately, but allows for the remaining parts of the proofs to be batched and verified together. The resulting verification time is linear in the number of proofs, but with a significantly lower time per proof. An existing node that has already verified the transactions in the blockchain can still use batch verification on new transactions it receives, but the benefits are not as great due to the lower number of transactions that must be verified in a short time.

\section{Optimizations}
For the most part, the proposed implementation of Bulletproofs in MKEcoin follows the Bulletproofs paper in scope and notation wherever possible. However, we include several optimizations that have also been discussed for other projects. These optimizations are algebraically equivalent to those in the paper, but reduce the time required for verification. The author understands that some or all of the optimizations may be included in an update to the Bulletproofs paper sometime in the future. However, we document them here for completeness and ease of code review. The reader is encouraged to refer to the paper for the complete context of our changes.

\subsection{Curve group notation}
The paper is written with a general group structure in mind, so scalar-group operations are written multiplicatively (\textit{e.g.} $x = a^bc^d$). In the case of elliptic curve groups, we use additive notation instead (\textit{e.g.} $X = bA + dC$) and use case to differentiate between curve points and scalars for clarity. This is purely a notational convenience.

\subsection{Basepoint notation}
Throughout the paper, amount commitments are expressed as $V \equiv vG + \mu H$, where $G$ and $H$ are distinct (but arbitrary) fixed elliptic curve group generators. We interchange the roles of $G$ and $H$ throughout our implementation to match the use of existing base points used in commitments elsewhere in the MKEcoin codebase. Note that the indexed $\{G_i\}$ and $\{H_i\}$ curve points are not modified in this way.

\subsection{Fiat-Shamir challenges}
To make the Bulletproof scheme non-interactive, we follow the paper by introducing Fiat-Shamir challenges computed by hashing the proof transcript up to the point that a new challenge is needed. This is done by introducing a rolling hash that uses as input the previous challenge and any new proof elements introduced. The prover and verifier compute these challenges identically.

\subsection{Inner product argment}
The inner product argument in Protocol 1 of the Bulletproofs paper uses recursion to shrink the size of its input vectors down to single elements. These inputs include distinct curve group generators $\{G_i\}$ and $\{H_i\}$, which we compute using an indexed hash function. We make several optimizations to this protocol for the verifier.

First, we observe that the curve points in Equation (10) are in fact linear combinations of $\{G_i\}$ and $\{H_i\}$ that use the scalar challenges in Equations (24)-(25). Next, we note that the point $P$ in Equation (62) is passed into Protocol 1 as described in Section 4.2 of the paper. Since this curve point contains a linear combination of the same group generators as Protocol 1, we can take advantage of this and compute a single linear combination, rather than separately compute Equations (62) and (10).

In practice, we replace Equations (62) and (10) with the following check, where $M \equiv |\{L_j\}| = |\{R_j\}|$:
$$A + xS - \mu G + \sum_{j=0}^{M-1}(w_j^2 L_j + w_j^{-2} R_j) + (t - ab)xH - \sum_{i=0}^{mn-1}(g_iG_i + h_iH_i) = 0$$
The symbols are mostly those used in the paper. However, we use $w_j$ to represent the round challenges in Lines (21)-(22), and $x$ to represent the challenge in Lines (32)-(33) to avoid reuse of symbols. The scalars $g_i$ and $h_i$ are computed in the following way. Express the index $i = b_0b_1 \cdots b_{M-1}$ bitwise, where $b_{M-1}$ is the least-significant bit. Then
$$g_i = a\prod_{j=0}^{M-1} w_j^{2b_j-1} + z$$
and
$$h_i = \left(by^{-i}\prod_{j=0}^{M-1} w_j^{-2b_j+1} - zy^i + z^{2+\lfloor i/N \rfloor}2^{i\operatorname{mod}N}\right)y^{-i}$$
This optimization is applied only to the verifier.

\subsection{Batch verification}
Our implementation permits the verifier to take many aggregate proofs and verify them together as a batch. We do not assume that the proofs each have the same number of outputs, nor make any restrictions on the maximum size of a batch. The batch verification we describe will only succeed if each proof is valid, and will fail if one or more proofs are invalid.

Batch verification is split into two checks, performed after iterating over each proof in the batch. During the iteration, the verifier keeps ongoing sums of components from each proof and then performs the first-stage check for Equation (61):
\begin{equation}
\sum_l (\beta_l\tau_{xl}) G + \sum_l \beta_l\left[ t_l - (k_l + z_l \langle \overline{1}^{mn},\overline{y_l}^{mn} \rangle) \right] H - \sum_l \beta_l \left( \sum_j z_l^{j+2} V_{lj} - x_lT_{1l} - x_l^2T_{2l} \right) = 0 \nonumber
\end{equation}
The second-phase check proceeds similarly:
\begin{multline}
\sum_l \beta_l(A_l + x_lS_l) - \sum_l(\beta_l\mu_l) G + \sum_l\left[\beta_l \sum_j(w_{lj}^2 L_{lj} + w_{lj}^{-2} R_{lj})\right] + \sum_l \beta_l x_l(t_l - a_lb_l) H \\
- \sum_i \left[\sum_l(\beta_l g_{li})G_i + \sum_l(\beta_l h_{li})H_i\right] = 0 \nonumber
\end{multline}
Here each $l$-indexed sum is over each proof in the batch, and $\beta_l$ is a weighting factor chosen at random (not deterministically) by the verifier. This ensures that, except with negligible probability, the checks will only succeed if each proof is separately valid; an adversary cannot selectively provide a batch containing invalid proofs in an attempt to fool the verifier. The benefit to this approach is that the sums can be computed as large multi-exponentiation operations after the scalars from all proofs have been assembled.

If the batch fails either check, at least one proof in the batch is invalid. To identify which proofs are at fault, the verifier can either iterate through each proof and perform the checks separately (in linear time), or perform a binary search by successively performing the checks on half-batches until it identifies all faulty proofs (in logarithmic time).

\section{Proof size}
Including the amount commitment $V$, a single Borromean bitwise range proof occupies $6.2$ kB of space; a transaction with $m$ outputs therefore requires $6.2m$ kB of space. An $m$-proof (with a $64$-bit range) requires $2\lg m + 17$ group elements and $5$ scalars, each of which takes up $32$ bytes. Table \ref{table:size} shows the space savings from Bulletproofs for several values of $m$.

\begin{table}[h]
\begin{center}
\begin{tabular}{r|rr|c}
$m$ & Bulletproof & Borromean & Relative size \\
\hline
$1$   & $704$  & $6200$   & $0.114$ \\
$2$   & $768$  & $12400$  & $0.062$ \\
$8$   & $896$  & $49600$  & $0.018$ \\
$16$  & $960$  & $99200$  & $0.010$ \\
$128$ & $1152$ & $793600$ & $0.001$
\end{tabular}
\caption{Size (bytes) of $m$ Borromean proofs versus $m$-proof}
\label{table:size}
\end{center}
\end{table}

Using data from the MKEcoin blockchain\footnote{Data was taken from blocks 1400000 through 1500000} on the distribution of the number of outputs in transactions, the use of Bulletproofs would reduce the total size of range proofs by $94\%$.

\bibliographystyle{plain}
\bibliography{bulletproofs}

\end{document}