%---------------------------------------------------
% Integration
%---------------------------------------------------
\chapter[Integration]{Integration $\,\int$}
In mathematics we are often given a function $f$ and asked to find a function $F$ whose derivative is $f$. $\;F$ in this situation is called the integral of $f$. It is usual to develop a list of integrals by differentiating a range of functions then using those to work backwards. The terms integration and anti-differentiation are synonymous. Generally we will use the term integration, however, both are acceptable.

The process of `reversing' or `undoing' a derivative has its own symbol, the integrand: $\displaystyle\int$
\begin{tcolorbox}
	\[\int f'(x) dx = f(x)+C\]
\end{tcolorbox}

%---------------------------------------------------
% Standard Integrals
%---------------------------------------------------
\section{Standard Integrals}
It is not the intention here to list all of the rules that are required however at this stage let us explore the Power Rule to establish a rule for integrating an expression of the form $y =x^{n}$ \\

\begin{center}
\begin{tabular}[c]{p{1.5cm}p{1.5cm}}\toprule
function $f(x)$  & derivative $f'(x)$  \\
\midrule
$x$  & $1$  \\
$x^{2}$  & $2 x$  \\
$x^{3}$  & $3 x^{2}$  \\
$x^{4}$  & $4 x^{3}$  \\
\bottomrule
\end{tabular}\ or
\
\begin{tabular}[c]{ll}\toprule
 $f$  & $f'$  \\
 &\\
\midrule
$x$  & $1$  \\
$\frac{1}{2} x^{2}$  & $x$  \\
$\frac{1}{3} x^{3}$  & $x^{2}$  \\
$\frac{1}{4} x^{4}$  & $x^{3}$  \\
\bottomrule
\end{tabular}\ now, in reverse \
\begin{tabular}[c]{p{2cm}p{2.5cm}}\toprule
derivative $f'(x)$  &  integral $\int f'(x)=f(x)$  \\\midrule
$1$  & $x +C$  \\
$x$  & $\frac{1}{2} x^{2} +C$  \\
$x^{2}$  & $\frac{1}{3} x^{3} +C$  \\
$x^{3}$  & $\frac{1}{4} x^{4} +C$  \\
\bottomrule
\end{tabular}
\end{center}
\bigskip This establishes the pattern and if you think about the rule for differentiating $y =x^{n}$ you can soon establish the rule for integrating $x^{n}$.
\begin{tcolorbox}
	The \textbf{Power Rule} for integrating polynomials
\[\int x^n dx = \frac{x^{n +1}}{n +1} +c, \text{ where }n \neq  -1
\]
\end{tcolorbox}

\begin{equation*}\text{ If }f (x) =x^{ -1}\text{ then the integral of }f\text{ is }\ln  \left \vert x\right \vert  +c\text{ or }\ln  \left \vert k x\right \vert
\end{equation*}
All of the differentiation rules we have met so far lead to integration rules. For instance we can establish standards for $\sin  x$, $\cos  x$, and $\sec ^{2} x\text{.}$ The standard integrals are summarized in the table below.
\renewcommand\arraystretch{1.2}
\begin{center}
	\begin{tabular}{llcl}
		\toprule
		Function&Integral&&Notes\\
		$f(x)$  &  $\int f(x)dx$ \\ \midrule
		$1$&$x+C$&&constant\\
		$A$&$Ax+C$&&$A$ is constant\\ \midrule
		$x^n, n \neq -1$ & $\displaystyle\frac{x^{n+1}}{n+1}+C$&&power rule general form\\
		$e^x$ & $e^x+C$&&exponential\\
		$\frac{1}{x}$ & $\ln|x|+C$&&special case: $x^{-1}$\\
		$\ln x$&$x\ln |x|-x+C$&&\\\midrule
		$\sin(x)$ & $-\cos(x)+C$&& trigonometric \\
		$\cos(x)$ & $\sin(x)+C$\\
		$\tan (x)$&$\ln |\sec x|+C$\\
		$\sec^2(x)$ & $\tan(x)+C$ \\ \bottomrule
	\end{tabular}
\end{center}

To allow us to combine these integrals and thus extend the range of questions we can tackle we use two important rules for integrals
\begin{tcolorbox}
\textbf{Sum Rule} The integral of the sum of two functions is the sum of the integrals of the functions.
\[\int \left [f (x) +g (x)\right ]\; d x =\int f (x)\; d x +\int g (x)\; d x\]
\end{tcolorbox}
This is easily extended to the sum or difference of a number of functions.
\begin{tcolorbox}
\textbf{Constant Multiple Rule} The integral of a constant times a function is the constant times the integral of the function.
\[\int c f (x)\; d x =c \int f (x)\; d x
\]\end{tcolorbox}

\subsection*{Indefinite Integrals}
Although $\int f (x) d x$ looks very similar to $\int _{a}^{b}f (x)\; d x$ they are quite different and must not be confused or used in place of each other. $\int f (x)\; d x$ is a function of $x$ or a family of functions of $x$ and $\int _{a}^{b}f (x)\; d x$ is a number. They are connected of course, provided $f (x)$ is a continuous function of $x$ on $\left [a ,b\right ]$. In this case the Evaluation Theorem gives the connection between them.
\begin{equation*}\int _{a}^{b}f (x)\; d x =\left .\int f (x)\; d x\right ]_{a}^{b}
\end{equation*}

The indefinite integral represents either a particular integral or a family of integrals. These will use a constant $C$ where $C$ takes a different value for each member of the family. $C$ is called the \emph{constant of integration}.
\clearpage
\example Integrate the following functions; find $\displaystyle\int f(x)$.
\begin{tasks}(2)
	\task $f(x) =3x^{2}$ \medskip\\
	\solution Applying the power rule:\\
\begin{align*}\int 3x^{2}\,dx&=\frac{3x^{2+1}}{2+1}+C\\
	&=x^3+C\end{align*}
\task $f (x) =7$ \medskip \\
\solution Here we are integrating a constant:\\
\[\int 7\,dx=7x+C\]
\task $f(x) =x^{\frac{2}{3}}$ \medskip\\
\solution The power rule still applies to fractional indices:\\
\begin{align*}\int x^{\frac{2}{3}}\,dx&=\frac{x^{\frac{2}{3}+1}}{\frac{2}{3}+1}+C\\
&=\frac{3 x^{5/3}}{5}+C \end{align*}
\task $f (x) =\frac{1}{2 \sqrt{x}} +\frac{1}{\sqrt{2}}$ \medskip\\
\solution Here we need to combine the sum rule and the power rule:\\
\begin{align*}
\int f(x)\,dx &= \int(\frac{1}{2 \sqrt{x}}) \,dx+\int(\frac{1}{\sqrt{2}}) \,dx\\
&=\frac{1}{2}\int x^{-\frac{1}{2}}+\frac{1}{\sqrt{2}}x+C\\
&=\sqrt{x}+\frac{x}{\sqrt{2}}+C
\end{align*}
\end{tasks}
These four examples are all \textit{indefinite} integrals and have an unknown constant in the answer. The following section will introduce definite integrals.\vspace{1cm}\\
\rule{6.8cm}{0.5pt}\\
\example Find $f (x)$ given $f^{ \prime  \prime } (x) =6$\medskip\\
\solution Here we have a \textit{second} derivative, indicated by the double-prime symbol, $f''$. Knowing that $\int f'(x)=f(x)$ we can safely assume that
\begin{tcolorbox}
	\[\int f''(x) \,dx = f'(x).\]
\end{tcolorbox}
So $f'(x)=\int f''(x) = \int 6 \,dx = 6x+C$. Now we need to integrate a second time to get $f$.
\[\int (6x+C)\,dx=6x^2+Cx+D\]
We end up with two unknown values, $C$ and $D$ as opposed to just a single value.

The previous two examples of equations involving derivatives. Any equation involving derivatives of a function is called a \emph{differential equation}. We will look into this subject in the next chapter.

\clearpage
%---------------------------------------------------
% Area
%---------------------------------------------------
\section{Area}\begin{multicols}{2}
In this section we attempt to find the area under a curve. That is the area that lies between a curve and the $x$-axis from $x =a$ to $x =b$. The area is bounded by the $x$-axis, a continuous curve $y =f (x)$ and the two vertical lines $x =a$ and $x =b$. This is shown in the figure with the area shaded in. Note the area stops at the axis. Area as calculated by integration is always in reference to the axis.\\
\includegraphics[width=8cm]{area1}
\end{multicols}

Previously, when we wanted to find the slope of a tangent line we found the slope of a secant line and applied the limiting process: $\Lim{h\to 0}$. A similar procedure will be used to find the area. We first approximate the area with rectangular strips then we take the limit of the areas of these rectangular strips by making the strips narrower and narrower and thus the number of strips between $x =a$ and $x =b$ greater and greater.

\subsection*{Area with Riemann Sums}
\example Given the parabola $y =x^{2}$, use rectangles to find the area under this curve between $0$ and $1$.

\solution
\begin{multicols}{2}
\includegraphics[width=8cm]{Area4}\label{fig:riemann}
Consider $4$ strips by constructing vertical lines at $x =\frac{1}{4}, \frac{1}{2}, \frac{3}{4}, \text{and }1$ as shown in the diagram. Rectangles are constructed using the right-hand boundary and we know from inspection this will be larger than the actual area.
\end{multicols}
\begin{align*}
\text{Right sum} &  = \frac{1}{4} \genfrac{(}{)}{}{}{1}{4}^{2} +\frac{1}{4} \genfrac{(}{)}{}{}{1}{2}^{2} +\frac{1}{4} \genfrac{(}{)}{}{}{3}{4}^{2} +\frac{1}{4} \left (1\right )^{2} \\
 &  = \frac{1}{64} +\frac{1}{16} +\frac{9}{64} +\frac{1}{4} \frac{15}{32} =0.46875\end{align*}

Repeating this process with a larger number of rectangles will improve the accuracy of the method. Using a spreadsheet like Excel shows a convergence on a value of $\frac{1}{3}$ as $n$ rectangles increase. The sum using the left-hand boundary is included for comparison.
\begin{center}
	\begin{tabular}[c]{ccc}\toprule
		$n$  & Left sum  & Right sum  \\\midrule
		10		& 0.2850000  & 0.3850000  \\\midrule
		20		& 0.3087500  & 0.3587500  \\\midrule
		30		& 0.3168519  & 0.3501852  \\\midrule
		100		& 0.3283500  & 0.3383500  \\\midrule
		1000	& 0.3328335  & 0.3338335  \\\midrule
		$\infty$& $\frac{1}{3}$&$\frac{1}{3}$\\\bottomrule
\end{tabular}\end{center}
It can be seen that a very accurate approximation to the area can be obtained as the number of rectangles increases. It should be clear that as $n \rightarrow \infty $ both the left sum and the right sum approach the area under the curve we write
\begin{equation*}A =\Lim{n \to \infty}\text{ Left Sum } =\Lim{n \to \infty}\text{ Right Sum }
\end{equation*}
This process can be generalised by selecting any height within each rectangular strip
and finding the area of each strip using this height. Let there be $n$ strips and consider the $i^{t h}$ strip. Select a value of $x$ in the $i^{t h}$ strip call it $x_{i}$. The height of this rectangle will be $f (x_{i})$. Consider the situation described above where the area is bounded by the $x$-axis, a continuous curve $y =f (x)$ and the two vertical lines $x =a$ and $x =b$.

With $n$ rectangles the length of the base of each rectangle is $ \Delta x =\frac{b -a}{n}$. The area of the $i^{t h}$ rectangle is $f (x_{i})  \Delta x$.

The sum of all the rectangles is
\begin{equation*}\underset{i =1}{\sum ^{n}} f (x_{i})  \Delta x =f \left (x_{1}\right )  \Delta x +f \left (x_{2}\right )  \Delta x +\ldots  +f \left (x_{n}\right )  \Delta x
\end{equation*}

And
\begin{equation*}A =\underset{n \rightarrow \infty }{\lim }\left [\underset{i =1}{\sum ^{n}} f (x_{i})  \Delta x\right ]
\end{equation*}

If $f$ is a continuous function defined on the interval $\left [a ,b\right ]$ then as $n \rightarrow \infty $ the number represented by$\;\underset{i =1}{\sum ^{n}} f (x_{i})  \Delta x \rightarrow A$ the area under the curve $y =f (x)$. This number is called the definite integral of $f$ from $a$ to $b$ and is denoted by $\int _{a}^{b}f$ or $\int _{a}^{b}f (x) d x$
\begin{equation*}\int _{a}^{b}f (x) d x =\underset{n \rightarrow \infty }{\lim }\left [\underset{i =1}{\sum ^{n}} f (x_{i})  \Delta x\right ]
\end{equation*}

This process is called a \emph{Riemann sum} after the German mathematician Bernard Riemann (1826-1866) who defined the integral in this way. The symbol $\int $ was introduced by Leibniz and is called the \emph{integral sign}.

\subsection*{Definite Integrals}
The method of computing Riemann sums is often long and to achieve a result that is accurate enough requires a computer. Both Isaac Newton and Gottfried Leibniz discovered a much simpler way based on the integral. This discovery is called \emph{The Evaluation Theorem}.

\begin{tcolorbox}
\textbf{Evaluating Definite Integrals}\\Given $F$ is an integral of $f$ i.e. $F^{ \prime } =f$, provided $f$ is continuous on the interval $[a ,b]$ then
\[\int _{a}^{b}f (x) d x =F (b) -F (a)\]
\end{tcolorbox}

This is an amazing result in view of the fact that it replaces such a complex procedure as finding
Riemann sums over greater and greater numbers of elementary rectangles.

\example Evaluate $\int _{0}^{1}x^{2} d x$.

\solution Because we know a particular integral of $f (x) =x^{2}$ is $F (x) =\frac{1}{3} x^{3}$ We have from the Evaluation Theorem
\begin{equation*}\int _{0}^{1}x^{2} d x =F (1) -F (0) =\frac{1}{3} \cdot 1^{3} -\frac{1}{3} \cdot 0^{3} =\frac{1}{3}
\end{equation*}
Looking back at the calculation of left sum and right sum above we can now see that the actual area
that we were endeavouring to calculate was in fact $1/3$ or $0. \dot{3}$.

These are some of the different notations for using the Evaluation Theorem
\begin{equation*}F (b) -F (a) =F (x)\vert _{a}^{b} =\left [F (x)\right ]_{a}^{b} =\left .F (x)\right ]_{a}^{b}
\end{equation*}

\subsection*{Area with Definite Integrals}
Areas above the $x$-axis have \emph{positive} definite integrals and areas below the $x$-axis have \emph{negative} definite integrals. If the context of the question is to evaluate a definite integral then use the definition. If the question is about \textbf{area} you must find the parts of the question that have areas above the $x$-axis and those parts that have areas below the $x$-axis and evaluate them separately. The definite integral calculates the result as the net sum of the positive and negative areas. To find the total area take the absolute value of the individual parts.

\example Find the area under the curve $y =x^{3} -x$ between $x = -1$ and $x =1$
\begin{SCfigure}[1][h!]
	\includegraphics[width=0.6\textwidth]{area3}
	\caption*{Figure: A cubic showing how area `under' the curve is evaluated. The area for $-1\leq x\leq 0$ is positive (above the axis), and the area for $0\leq x\leq 1$ is negative.}
\end{SCfigure}

\solution This can be factored to give $y =x (x +1) (x -1)$. This cubic curve crosses the x-axis at $ -1$, $0$, and $1$. Here, this area must be found in two parts.

\begin{tasks}[column-sep=3ex,label-offset=-5em](3)
	\task[]This is expected to be negative because it is below the axis:\\
	$\displaystyle\int _{0}^{1}(x^{3} -x) dx$\\
	$\displaystyle=\frac{x^4}{4}-\frac{x^2}{2}\Big\vert_{0}^1$\\
	$\displaystyle=[\frac{1}{4}-\frac{1}{2}]-0$\\
	$\displaystyle=-\frac{1}{4}$
	\task[]This is expected to be positive:\\
	$\displaystyle\int _{ -1}^{0}(x^{3} -x) d x$ \\
	$\displaystyle=\frac{x^4}{4}-\frac{x^2}{2}\Big\vert_{-1}^0$\\
	$\displaystyle=0-[\frac{1}{4}-\frac{1}{2}]$\\
	$\displaystyle=+\frac{1}{4}$
	\task[]Therefore the total area is:\medskip\\
	 $\displaystyle\Big\vert-\frac{1}{4}\Big\vert+\frac{1}{4}=\frac{1}{2}\text{ units}^2$
\end{tasks}
We will compare with a single integral from $-1$ to $1$.
\[\int _{ -1}^{1}(x^{3} -x) d x=\frac{x^4}{4}-\frac{x^2}{2}\vert_{-1}^1 =-\frac{1}{4}-(-\frac{1}{4})=0\]
Area must be non-negative, and so this result is nonsensical given the context.\\
\rule{6.8cm}{0.5pt}\\

\begin{multicols}{2}
\example The energy, or electrical charge, that a capacitor can discharge is found by taking the integral of the voltage-time function. This can neatly be represented as the area under the voltage-time curve. Find the total discharge from the capacitor after 5 seconds. The units for charge are coulombs.\medskip\\
\solution Integrate the $V(t)$ function to find the area:\\
	\begin{align*}
\mathrm{area}&=\int_{a}^{b}V(t)dt\\
&=8\int_{0}^{5}\left(e^{-t}\right)dt\\
&=8\left[\left(-1e^{-t}\right)\Big|_{0}^5\right]\\
&=8\left[-e^{-5}-(-e^{0})\right]=8.054 \,\, \text{coulombs}\\
\end{align*}
%\columnbreak
\begin{center}
	\begin{tikzpicture}
	\begin{axis}[
	width=8cm,height=10cm,
	axis lines=center,
	axis on top=true,
	ymax=9,ymin=0,
	xmax=6,xmin=-.025,
	xlabel=time($s$),
	ylabel=voltage($V$),
	]
	\addplot [name path =F,->,domain=0:6,thick, samples=100, black] {8*exp(-x)};
	\addplot [name path =G,,domain=0:5,thick, samples=100] {0};
	\addplot[pattern=north west lines, pattern color=black!75]fill between[of=F and G, soft clip={domain=0:5}];
	\addplot[mark=]coordinates{(0.5,4.85)} node[pin=45:{$V(t)=8e^{-t}$}]{};
	\end{axis}
	\end{tikzpicture}
\end{center}
\end{multicols}
The standard integral to calculate area is bounded by the axis as seen in the previous examples. In the following example we see that when finding area between intersecting curves, a new strategy can be applied. \medskip\\
\clearpage
\example Find the area between the two curves: $f(x)=x$ and $g(x)=x^2-2x$. They intersect as shown at $(0,0)$, and $(3,3)$.
\begin{center}\includegraphics[width=0.6\textwidth]{area5}\end{center}
\solution When inspecting the shaded region, the straight line function, $y=x$ is above the parabola. We could handle this with two separate integrals: find $\int f(x)$ and then subtract $\int g(x)$, or we can simplify and find $\int f(x)-g(x)$:
\begin{tasks}[label-offset=-5em](2)
\task[]\begin{align*}
\text{Area}&=\int_a^b[f(x)-g(x)]dx\\
&=\int_0^3[x-(x^2-2x)]dx\\
&=\int_0^3[-x^2+3x]dx\\
\end{align*}
\task[]\begin{align*}
&=-\frac{x^3}{3}+\frac{3x^2}{2}\Big\vert_0^3\\
&=\left(-9+\frac{27}{2}\right)-0=\frac{9}{2}\text{ units}^2\\
\end{align*}
\end{tasks}
\begin{tcolorbox}
\textbf{Area between functions}\\
Let $f(x)$ be the upper function and $g(x)$ be the lower function, then\\
\[\text{Area }=\int_a^b[f(x)-g(x)]dx\]
\end{tcolorbox}

\clearpage
\example A logo is formed by the shaded area between the cubic function  $ f(x)=4x -x^3$ and a parabola $g(x)=2x-x^2$. The two curves intersects at $x=0$ and $x=2$. Find the shaded area.\\
\begin{multicols}{2}
	\begin{tikzpicture}
	\begin{axis}[
	axis lines=center,
	height=8cm,
	axis on top=true,
	ymax=4,ymin=-4,
	xmax=3,xmin=0,
	xlabel=$x$,
	]
	\addplot [name path =F,<->,domain=-2.3:2.3,thick, samples=200, black] {-x^3+4*x};
	\addplot [name path = G,<->,domain=-2.3:2.3,thick, samples=200, red] {-x^2+2*x};
	\addplot[pattern=north west lines, pattern color=blue!50]fill between[of=F and G, soft clip={domain=0:2}];
	\end{axis}
	\end{tikzpicture}
	\columnbreak
\\	\solution Identify the top curve $f(x)$ and subtract the bottom $g(x)$:
	\begin{align*}
	\mathrm{Shaded \hspace{0.2cm}area}&=\int_{a}^{b}(f(x)-g(x))dx\\
	&=\int_{0}^{2}\left((4x-x^3)-(2x-x^2)\right)dx\\
	&=\int_{0}^{2}\left((2x-x^3+x^2)\right)dx\\
	&=\left[\left(2\frac{x^2}{2}-\frac{x^4}{4}+\frac{x^3}{3} \right)\Big|_{0}^2\right]\\
	&=\left[\left(4-\frac{16}{4}+\frac{8}{3}\right)-(0)\right]\\
	&=\frac{8}{3} \,\, \mathrm{units}^2
	\end{align*}
\end{multicols}

%---------------------------------------------------
% Volume
%---------------------------------------------------
\section{Volume}
If a function is revolved around an axis it creates a volume between the axis and the function. Similar to how if we integrate a function, it results in an area --- if we integrate an \textit{area} it results in a volume.

\begin{multicols}{2}
	\example A connector was obtained by revolving the function $f(x)={0.3x+1}$ around the $x$-axis for $0\leq x \leq 5$. Calculate the volume of the connector.
\begin{center}\includegraphics[width=8cm]{solids1}\end{center}
\columnbreak
\solution\\ Given the function of the outer boundary of the shape, we must square the function and integrate this result.\\
\begin{align*}
\mathrm{volume}&=\int_{a}^{b}\pi[f(x)]^2 dx\\
&=\pi\int_0^5\left(0.3x+1\right)^2 dx\end{align*}
Use the power rule and chain rule to integrate:\\
\[=\pi \frac{ (0.3x+1)^3}{0.3(3)} \Big|_0^5\]
Evaluate at the boundary points:\\
\begin{align*}
&=\frac{10\pi}{9}\left[\left(\frac{5}{2}\right)^3-1\right]=\frac{65\pi}{4}\\
&\approx 51.05 \text{ units}^3
\end{align*}
\end{multicols}

\clearpage
The previous example used the disc-method to calculate a volume. Beginning with a rectangular section of width $\Delta x$, and length $f(x)$, just like the Riemann Sum for calculating area on page~\pageref{fig:riemann}. This height becomes a radius when rotated around an axis of revolution, labelled $R$ below. The \textit{volume} of a single disc is $\pi R^2\Delta x$.
\begin{center}\includegraphics[width=\textwidth]{discMethod}\end{center}
Finding the volume of all the discs involves integrating over the length of the axis from $a$ to $b$.\[\text{Volume}=\int \pi R^2 \Delta x\] The radius is the function evaluation, $f(x)$, and lowercase delta, $\delta$, means infinitesimal thin discs as the number of them goes to infinity, $n\rightarrow\infty$,  we can generalize the formula for volume:
\begin{tcolorbox}
	\textbf{Volume rotated around the $x$-axis}
	\[ \text{Volume} = \int_{a}^{b} \pi y^2 dx = \int_{a}^{b} \pi (f(x))^2 dx\]
	\textbf{Volume rotated around the $y$-axis}
	\[ \text{Volume} = \int_{c}^{d} \pi x^2 dy =\int_{c}^{d} \pi (f(y))^2 dy  \]
\end{tcolorbox}

\example Fluorescent and incandescent light bulbs are often filled with the inert gas krypton. Find the volume of krypton gas required to fill the bulb shown.\vspace{0.4cm}\\
\includegraphics[width=15cm]{bulb}\\
The function has been estimated to be:\\
\[f(x)=1-\frac{x^2}{5}\qquad\text{ for } -1.5\leq x \leq 2\text{ cm}\]
\solution You will have to calculate $\left(1-\frac{x^2}{5}\right)^2$ before integrating\\
	\begin{align*}
	\text{volume}&=\int_{a}^{b}\pi[f(x)]^2 dx\\
	&=\pi\int_{-1.5}^{2}\left(1-\frac{x^2}{5}\right)^2 dx\\
	&=\pi\int_{-1.5}^{2}\left(\frac{x^4}{25}-\frac{2x^2}{5}+1\right)dx\\
	&=\pi\left(\frac{x^5}{125}-\frac{2x^3}{15}+x\right)  \Bigg|_{-1.5}^2\\
	&=\pi\left[\left(\frac{32}{125}-\frac{16}{15}+2\right)-\left(-0.06075--0.45-1.5\right)\right]=2.3\pi\approx 7.23\, \text{ cm}^3\end{align*}
\\
\begin{multicols}{2}
\examq Calculate the volume of the container found by rotating the curve $y=\sqrt{x^3}$ around the $y-$axis for $0\leq y \leq 5$.\\

Here the volume is created by rotation about the $y$-axis, and therefore we need to adjust our formula. First we will solve the equation $y=\sqrt{x^3}$ for $x$, and then integrate.
\columnbreak
\begin{center}
		\includegraphics[width=0.5\textwidth]{funnel.png}
\end{center}
\end{multicols}

\solution Volume $=\int_{a}^{b} \pi f(y)^2 dy$\\
\begin{multicols}{2}
Rearrange the function to isolate $x$:
	\begin{align*}
	y&=\sqrt{x^3}\\
	y^2&=x^3\\
	\sqrt[3]{y^2}&=x\end{align*}
Integrate to find volume:\\
	\begin{align*}V&=\pi\int_{0}^{5} (y^{\frac{2}{3}})^2\,dy\\
	&=\pi\int_{0}^{5} y^{\frac{4}{3}}\,dy\\
	&=\pi\frac{y^{\frac{7}{3}}}{\frac{7}{3}}\bigg\vert_{0}^{5}\\
	&=\frac{3\pi}{7}5^{\frac{7}{3}}-0\\
	&\approx 57.6 \,\,\mathrm{units}^3
	\end{align*}
\end{multicols}

%---------------------------------------------------
% Integration by Substitution
%---------------------------------------------------
\section{Integration by Substitution}
Earlier we stated that every differentiation rule leads to an integration rule. The chain rule for differentiation leads to the substitution rule for integration, i.e. the \emph{substitution rule for integration}.

A series of examples will illustrate how integration by substitution works.

\example Find the integral: $\int 2 x \sqrt{1 +x^{2}}\; d x$.\medskip\\
\solution This cannot be integrated using the techniques discussed so far. This is an example of a function that can be integrated using the substitution rule. They are often recognised by noting the presence of a composite function; the $\sqrt{1 +x^{2}}$ can be seen as $\sqrt{u}$ where the $1+x^2$ under the root is replaced by a new variable.

Let $u=1+x^2$. The integral now becomes $\int 2x\sqrt{u}\; dx$. This is not yet ready because there are two different variables: $u$ \textbf{and} $x$. Lets differentiate the new equation:
\begin{align*}
u&=1+x^2\\
\frac{du}{dx}&=2x
\end{align*}
And we can isolate the $dx$ with the intention of replacing it in the original integral, so $\displaystyle dx=\frac{du}{2x}$.

If we now look back at the original question we are ready to substitute expressions containing $u$ for expressions containing $x$.
\begin{align*}
\int 2 x \sqrt{1 +x^{2}}\; d x &  = \int \sqrt{u} \cdot 2 x\; d x \\
 &  = \int \sqrt{u}\cdot \cancel{2x}\; \frac{du}{\cancel{2x}}\\
 &  = \int u^{\frac{1}{2}}\; d u \\
 &  = \frac{u^{\frac{1}{2} +1}}{\frac{1}{2} +1}= \frac{2}{3}\sqrt[2]{u^3}+C
 \end{align*}
 One last step is to switch back to the original variable $x$. Make the final substitution $u=1+x^2$.
 \[ = \frac{2 \sqrt{\left (1 +x^{2}\right )^{3}}}{3} +C \]

The Substitution Rule can be stated formally as follows. If $u =g (x)$ is a differentiable function whose range is an interval $I$ and $f$ is continuous on $I$, then $\int f \left (g \left (x\right )\right ) g^{ \prime } \left (x\right )\; d x =\int f \left (u\right )\; d u$. $d x$ and $d u$ are known as differentials. The Substitution Rule permits us to replace $g^{ \prime } \left (x\right )\; d x$ with $d u$.

\example Find $\int x \sin  \left (x^{2}\right )\; dx$\medskip\\
\solution Using the procedure from the previous example try $u =x^{2}$. This gives $d u =2 x\; d x\text{.}$
\begin{align*}\int x \sin  \left (x^{2}\right )\; d x &  = \int \sin  \left (x^{2}\right ) \cdot x\; d x \\
 &  = \int \sin(u) \cdot \cancel{x}\cdot\frac{du}{2\cancel{x}} \\
 &  = \frac{1}{2} \int \sin(u)\; d u \\
 &  =  -\frac{1}{2} \cos  u +C \\
 &  =  -\frac{1}{2} \cos  \left (x^{2}\right ) +C\end{align*}

It is clear the challenge of the Substitution Rule is to come up with a suitable substitution. Earlier examples often have fairly obvious substitutions but they can quickly become quite complicated and result in many false starts. \\
\rule{6.8cm}{0.5pt}\\
\example Find $\int \frac{x}{\sqrt{1 -x^{2}}}\; d x$\medskip\\
\solution Let $u =1 -x^{2}$. Then $d u = -2 x\; d x$. So $x\; d x = -\frac{1}{2} d u$. These are now substituted into the original expression
$$\int \frac{x}{\sqrt{1 -x^{2}}}\; d x = -\frac{1}{2} \int \frac{1}{\sqrt{u}}\; d u = -\frac{1}{2} \int u^{ -1/2}\; d u = -\frac{1}{2} \left (2 u^{1/2}\right ) +C = -\sqrt{1 -x^{2}} +C$$

In each of the above 3 examples you could use desmos to compare the original function and the integral to
see if the result is reasonable. For example 3 try graphing the original function $y =\frac{x}{\sqrt{1 -x^{2}}}$ and the integral $y = -\sqrt{1 -x^{2}}$ on the same axes and check the original function represents the slope of the tangent lines to the curve $y = -\sqrt{1 -x^{2}}$.

\subsection*{Evaluating Definite Integrals by Substitution}
We will look at two ways to evaluate a definite integral.

\textbf{Method 1} Find the indefinite integral then use the Evaluation Theorem.

\textbf{Method 2} Make the substitution to the integrand and differential (remembering to change the limits to the new variable).

\example Evaluate $\int _{0}^{8}\sqrt{x +1}\; d x$. Graphing provides us with the following:
\begin{SCfigure}[1][h!]
	\includegraphics[width=0.65\textwidth]{L4SZ283J}
	\caption*{Figure: The curve is clearly continuous. If we let $u =x +1$ then $u^{ \prime } =1$, this is also continuous.}
\end{SCfigure}

\solution
\textbf{Method 1}
Substitute $u =x +1$ then $d u =d x$. Substituting these values in the indefinite integral we get
\begin{equation*}\int \sqrt{x +1}\; d x =\int \sqrt{u}\; d u =\frac{2}{3} u^{3/2} +C =\frac{2}{3} \left (x +1\right )^{3/2} +C
\end{equation*}So
\begin{align*}\int _{0}^{8}\sqrt{x +1}\; d x &  = \left .\int \sqrt{x +1}\; d x\right ]_{0}^{8} \\
 &  = \left .\frac{2}{3} \left (x +1\right )^{3/2}\right ]_{0}^{8} \\
 &  = \frac{2}{3} \left (9\right )^{3/2} -\frac{2}{3} \left (1\right )^{3/2} \\
 &  = \frac{2}{3} 27 -\frac{2}{3} 1 \\
 &  = \frac{2}{3} \left (27 -1\right ) =\frac{2}{3} 26 =\frac{52}{3} =17\frac{1}{3}\end{align*}

\textbf{Method 2}
Again we let $u =x +1$ so $d u =d x$. Also we calculate the new limits for $u$ using $u =x +1$.
\begin{align*}x &  = 0\text{ gives }u =0 +1 =1 \\
x &  = 8\text{ gives }u =8 +1 =9\end{align*}

Now the definite integral is transformed into a definite integral in $u$.

\begin{align*}\int _{0}^{8}\sqrt{x +1}\; d x &  = \int _{1}^{9}\sqrt{u}\; d u \\
 &  = \left .\frac{2}{3} u^{3/2}\right ]_{1}^{9} \\
 &  = \frac{2}{3} 9^{3/2} -\frac{2}{3} 1^{3/2} \\
 &  = \frac{2}{3} \left (27 -1\right ) =17\frac{1}{3}\end{align*}

A check on the graph will show that an area of $17\frac{1}{3}$ appears to be reasonable.

Method 2 is usually preferred as the step where the indefinite integral is first calculated has been neatly incorporated into the method. The difficulty with method 2 is that once the values for $u$ have been calculated the integral is completely transformed and we never return to the original question. We answer a different question that, because of the transformation, has the same answer. This concept might cause confusion. However a graph of the situation shows what has happened.
\begin{center}\includegraphics[ width=5.3195in, height=1.9579in]{L4SZ283K}\end{center}
It is clear that the area under the first graph between $0$ and $8$ is the same as the area under the second graph between $1$ and $9$.

%---------------------------------------------------
% Integration by Parts
%---------------------------------------------------
\section{Integration by Parts}
Recall the rule for the differentiation of a product
\begin{equation*}\frac{d}{d x} \left (f (x) \cdot g (x)\right ) =f (x) \cdot g^{ \prime } (x) +g (x) \cdot f^{ \prime } (x)
\end{equation*}

We can integrate each side and write the process as follows
\begin{align*}f (x) \cdot g (x) &  = \int \left [f (x) \cdot g^{ \prime } (x) +g (x) \cdot f^{ \prime } (x)\right ]\; d x \\
 &  = \int f (x) \cdot g^{ \prime } (x)\; d x +\int g (x) \cdot f^{ \prime } (x)\; d x\end{align*}

This is rewritten in a particular way to become the formula for \emph{integration by parts}.
\begin{align}\int f (x) \cdot g^{ \prime } (x)\; d x &  = f (x) \cdot g (x) -\int g (x) \cdot f^{ \prime } (x)\; d x \tag{1} \\
\text{or}\int f (x) \cdot g^{ \prime } (x)\; d x &  = f (x) \cdot g (x) -\int f^{ \prime } (x) \cdot g (x)\; d x \tag{2}\end{align}

This formula is written in a number of different ways in textbooks. Here
are two ways

Let $u =f (x)$ and $v =g (x)$ then $d u =f^{ \prime } (x)\; d x$ and $d v =g^{ \prime } (x)\; d x$ so using the substitution rule equation (1) becomes
\begin{equation}\int u\; d v =u v -\int v\; d u\tag{3}
\end{equation}

Let $\int f (x) \cdot g^{ \prime } (x)\; d x$ be regarded as the the integral of the product of two functions then $g =\int g^{ \prime }$. We can write equation (2) as
\begin{tcolorbox}
	\[\int fg'=fg-\int f'g \tag{4}\]
\end{tcolorbox}
Equations (3) and (4) are the common forms that are best to remember.

The success of this method depends on the discovery that a simpler integral results from this process. Sometimes the process results in a worse situation than you started with so should be abandoned. Sometimes you produce a pattern which leads to a solution after 2 or more applications of the integration by parts rule. The patterns that produce solvable problems can be discovered as different questions are tried.

\example Find $\int x \cos  x\; d x$. This can be seen as the integral of a product. The two functions
are $f (x) =x$ and $g (x) =\cos  x$. So $f^{ \prime } (x) =1$ and $\int g (x) =\sin  x$ are easy to find. Notice though that $f^{ \prime } (x) =1$ gives us a clue that \emph{integration by parts} is going to be a fruitful method. \\
\solution
Using equation (2)
\begin{align*}\int x \cos  x\; d x &  = f (x) \cdot g (x) -\int f^{ \prime } (x) \cdot g (x)\; d x \\
 &  = x \cdot \sin  x -\int 1 \cdot \sin  x\; d x \\
 &  = x \cdot \sin  x -\int \sin  x\; d x \\
 &  = x \cdot \sin  x -\left ( -\cos  x\right ) +C \\
 &  = x\; \sin  x +\cos  x +C\end{align*}
\rule{6.8cm}{0.5pt}\\
\example Use integration by parts to find $\int \ln  x\; d x$. This is a function that has arisen in the course as the integral of $\frac{1}{x}$, however we are now able to use this fact to help with this question. \\
\solution Notice there is only one function here so we have to create two functions by stating $\ln  x =1 \cdot \ln  x$. The two functions are therefore $f (x) =\ln  x$ and $g (x) =1$. Can we find $f^{ \prime }$ and $\int g$? Yes!
\begin{align*}\int \ln  x\; d x &  = \int \ln  x \cdot 1\; d x \\
 &  = \ln  x \cdot x -\int \frac{1}{x} \cdot x\; d x \\
 &  = x\; \ln  x -\int 1\; d x \\
 &  = x\; \ln  x -x +C\end{align*}
\rule{6.8cm}{0.5pt}\\
\example Find $\int e^{x} \cos  x\; d x$ This is an example where a pattern is established and perseverance leads to the solution. \\
\solution Let $f (x) =\cos  x$ and $g (x) =e^{x}$. Can we find $f^{ \prime }$ and $\int g$? Yes!
\begin{align*}\int e^{x} \cos  x\; d x &  = \cos  x \cdot e^{x} -\int  -\sin  x \cdot e^{x}\; d x \\
 &  = e^{x} \cos  x +\int e^{x} \sin  x\; d x\end{align*}

This is an integral that is very similar in appearance to the original question with the $\cos  x$ transformed into $\sin  x$. We continue by repeating the integration by parts process with the new integral
$\int e^{x} \sin  x\; d x$
\begin{align*}\int e^{x} \cos  x\; d x &  = e^{x} \cos  x +\overset{\int e^{x} \sin  x\; d x}{\overbrace{\sin  x \cdot e^{x} -\int \cos  x \cdot e^{x} d x}} \\
 &  = e^{x} \cos  x +e^{x} \sin  x -\int e^{x} \cos  x\; d x\end{align*}

Notice the original integral has now appeared on the right hand side of the equation. We
now simply solve the equation for $\int e^{x} \cos  x\; d x$ and add the constant of integration.
\begin{align*}2 \int e^{x} \cos  x\; d x &  = e^{x} \cos  x +e^{x} \sin  x \\
\int e^{x} \cos  x\; d x &  = \frac{e^{x}}{2} \left (\cos  x +\sin  x\right ) +C\end{align*}

This can easily be verified by differentiating.

\subsection*{Definite Integrals}
Integration by parts can be combined with the Evaluation Theorem to evaluate definite integrals. If
we assume $f^{ \prime }$ and $g^{ \prime }$ are continuous then we can use the Evaluation Theorem and write equation (1) as follows
\begin{equation}\int _{a}^{b}f (x) \cdot g^{ \prime } (x)\; d x =\left .f (x) \cdot g (x)\right ]_{a}^{b} -\int _{a}^{b}g (x) \cdot f^{ \prime } (x)\; d x\tag{5}
\end{equation}

\example Evaluate $\int _{0}^{1}x e^{x}\; d x$. A graph of $y =x e^{x}$ shows the area required and gives us an idea of the answer to expect.
\begin{center}
\includegraphics[ width=4.7374in, height=2.6126in,]{L4SZ283M}
\end{center}
\solution Notice $x$ becomes simpler when differentiated and $e^{x}$ is unchanged when it is integrated
\begin{align*}\int _{0}^{1}x e^{x}\; d x &  = \left .x e^{x}\right ]_{0}^{1} -\int _{0}^{1}1 \cdot e^{x}\; d x \\
 &  = \left .x e^{x}\right ]_{0}^{1} -\left .e^{x}\right ]_{0}^{1} \\
 &  = \left (\left (1 e^{1}\right ) -\left (0 e^{0}\right )\right ) -\left (e^{1} -e^{0}\right ) \\
 &  = e^{1} -e^{1} +e^{0} \\
 &  = e^{0} =1\end{align*}

%---------------------------------------------------
% Applications of Integration
%---------------------------------------------------
\section{Applications of Integration}

\subsection*{Rectilinear Motion}
We will use integration to analyse the motion of an object moving in a straight line. Let
the position function for the object be $s =f (t)$ where $t$ is the time. The velocity function is $v (t) =s^{ \prime } (t)$. Therefore the position function is the integral of the velocity function. Also
the acceleration function is $a (t) =v^{ \prime } (t)$ so the velocity function is the integral of the acceleration function. We can obtain
the position function from the acceleration function by integrating twice. This process will generate
two constants of integration so we need two additional pieces of information to find the particular solution. Usually
$s (0)$ and $v (0)$ are given.

\example A particle moves in a straight line with an acceleration of $a (t) =4 t +2$. If the initial velocity is $ -4 \mbox{cm}$/$\mbox{s}$ and the initial displacement is $5 \mbox{cm}$, find the position function.

\solution \begin{multicols}{2}
As $v^{ \prime } (t) =a (t) =4 t +2$ we can integrate $a (t)$ to obtain $v (t)$
\begin{align*}v(t)&=\int a(t)dt\\
&=\int (4t+2)dt\\
&=4 \frac{t^{2}}{2} +2 t +C_{1} \\
&=2 t^{2} +2 t +C_{1}
\end{align*}

Substitute $t =0$ since we know the initial velocity $v(0)=-4$:
\begin{align*}v (0) &  = 2 \cdot 0^{2} +2 \cdot 0 +C_{1}\\
C_{1} &  =  -4\text{ and therefore} \\
v (t) &  = 2 t^{2} +2 t -4\end{align*}

Integrate $v (t)$ to obtain $s (t)$:
\begin{align*}s(t)&=\int v(t)dt\\
&=2 \frac{t^{3}}{3} +2 \frac{t^{2}}{2} -4 t +C_{2} \\
 &  = \frac{2}{3} t^{3} +t^{2} -4 t +C_{2}\end{align*}

Substitute $t =0$ because we are given the initial displacement i.e. $s (0) =5$
\begin{align*}s (0) &  = \frac{2}{3} \cdot 0^{3} +0^{2} -4 \cdot 0 +C_{2}\\
C_{2} &  = 5\text{ and therefore} \\
s (t) &  = \frac{2}{3} t^{3} +t^{2} -4 t +5\end{align*}
\end{multicols}

\rule{6.8cm}{0.5pt}\\

The gravitational force near the Earth's surface that produces a downwards acceleration is about $-9.8$ m/s$^2$, represented by the vector quantity, $\vec{g}$.\\
\example A ball is thrown vertically upwards with a speed of $24.5 \mbox{m}$/$\mbox{s}$ from the edge of a cliff that is $147 \mbox{m}$ above the ground.
\begin{tasks}[column-sep=30pt](2)
\task Find the position function.
\task Find the time when the ball reaches its maximum height.
\task Find the maximum height above the ground.
\task When does the ball hit the ground? \end{tasks}

\clearpage\solution
\begin{multicols}{2} \textbf{(a)} It is usual to choose the positive direction
to be upwards this means that $\vec{g}=-9.8$ m/s$^2$. Integrating the acceleration function yields the velocity function.
\begin{align*}v(t)&=\int a(t)dt\\
&  =\int -9.8dt\\
&=-9.8t +C_{1}\end{align*}
Substituting $v (0) =24.5$ we get that $C_{1} =24.5$
\begin{equation*}v (t) = -9.8 t +24.5
\end{equation*}
Because $s^{ \prime } (t) =v (t)$ we integrate $v (t)$
\begin{equation*}\int v(t)=s(t)=-9.8 \frac{t^{2}}{2} +24.5 t +C_{2}
\end{equation*}
Substitute $s (0) =147$. We get $C_{2} =147$.
\begin{equation*}\therefore s(t) = -4.9 t^{2} +24.5 t +147
\end{equation*}
This expression will hold true until the ball is acted on by an external force (hits the ground).

\textbf{(b)} The maximum height is reached when\\ $v (t) =0$; for a moment the vertical velocity will be zero before it begins to fall.
\begin{align*} -9.8 t +24.5 &  = 0 \\
24.5 &  = 9.8 t \\
t &  = \frac{24.5}{9.8}=2.5\text{ s}\end{align*}

\textbf{(c)} The maximum height is found by substituting $t =2.5$ into $s(t)$
\begin{align*}s (2.5) &  =  -4.9 \cdot 2.5^{2} +24.5 \cdot 2.5 +147 \\
 &=177.6 \text{ m}\end{align*}

\textbf{(d)} The ball hits the ground when the displacement, $s(t) =0$.
\begin{align*}-4.9 t^{2} +24.5 t +147 &  = 0 \\
t^{2} -5 t -30 &  = 0\end{align*}
Solve for $t$ using the quadratic formula:
\begin{align*}t &  = \frac{5 \pm \sqrt{25 -4 \times 1 \times  -30}}{2} \\
 &  = \frac{5 \pm \sqrt{145}}{2} \\
 &  =8.5\text{ s, or }-3.5\text{s}\end{align*}
The negative result will be discarded, however, represents a solution to the parabola in an algebraic sense. We did not test whether the value of $t$ when $v (t) =0$ gives a maximum or minimum. A maximum can be assumed knowing that $-9.8$ m/s$^2$ has a negative coefficient.
\end{multicols}

\subsection*{Work}
The strategy we use to allow us to apply calculus to a problem in engineering is the same as we used to evaluate areas. The
physical quantity is divided up into a large number of small parts, each one approximately equal to the theoretical quantity it represents. These
are then summed and a limit taken as the number of small parts, $n \rightarrow \infty $. This process allows us to evaluate the resulting integral.

You
will recall that from Newton's Second Law of Motion
\begin{equation}F =m a =m \frac{d^{2} s}{d t^{2}}\tag{1}
\end{equation}

Where $s (t)$ is the position function, $m$ is the mass of an object and $F$ is the force required to produce an acceleration of $a$. (Where $a =\frac{d^{2} s}{d t^{2}}$).

We usually measure mass in kilograms ($\mbox{kg}$), distance in metres ($\mbox{m}$) and force in newtons ($\mbox{N}$) \linebreak\relax ($N =k g \cdot m/s^{2}$). If the acceleration is constant then the force to produce that acceleration will also
be constant.

Work = force $ \times $ distance or
\begin{equation}W =F d\tag{2}
\end{equation}If $F$ is in newtons and $d$ is in metres then $W$ is in newton-metres. One newton-metre is called a\linebreak\relax joule ($\mbox{J}$).

\example A mass of 3.5 kg lifted 0.5 m requires a force of $\displaystyle F =m a =3.5 \times 9.8 =34.3 \mbox{N}$. This is the force required to counter the force exerted by gravity. Calculate the work done.\\
\solution The work done is calculated using equation 2
\begin{equation*}W =F d =34.3 \times 0.5 =17.15 \text{ J}
\end{equation*}

If the force is variable this formula can no longer be applied. Let the force acting on an object as it moves along the $x$-axis in a positive direction from $a$ to $b$ be $f (x)$, where $f$ is a continuous function of $x$. Divide the interval from $a$ to $b$ into $n$ subintervals of width $ \Delta x$ where $ \Delta x =\frac{b -a}{n}$. For simplicity we let the end points of these subintervals be $x_{0}$, $x_{1}$, $x_{2}$, ..., $x_{n}$ We select the $i^{t h}$ subinterval and select a representative $x$-value in this interval $x_{i}^{ \ast }$. The work done when we move the object from $x_{i -1}$ to $x_{i}$ is
\begin{equation*}W_{i} \approx f (x_{i}^{ \ast })  \Delta x
\end{equation*}

The total work done is
\begin{equation}W =\sum _{i =1}^{n}f (x_{i}^{ \ast })  \Delta x\tag{3}
\end{equation}

As we did with area we find the limit as $n \rightarrow \infty $ of this sum. As this is a Riemann sum the limit is a definite integral
\begin{equation}W =\sum _{i =1}^{n}f (x_{i}^{ \ast })  \Delta x =\int _{a}^{b}f (x)\; d x\tag{4}
\end{equation}
\rule{6.8cm}{0.5pt}\\
\example If the force on a particle is given by the equation $f (x) =3 x^{2} -2 x \text{ N}$, how much work is done moving the particle from $x =2$ to $x =3$?

\solution The graph of $f (x) =3 x^{2} -2 x$ shows that for the interval $\left [2 ,3\right ]$ the area is above the $x$-axis.
\begin{align*}W &  = \int _{a}^{b}f (x)\; d x \\
 &  = \int _{2}^{3}\left (3 x^{2} -2 x\right )\; d x \\
 &  = \left .3 \frac{x^{3}}{3} -2 \frac{x^{2}}{2}\right ]_{2}^{3} \\
 &  = \left .x^{3} -x^{2}\right ]_{2}^{3} \\
 &  = \left (3^{3} -3^{2}\right ) -\left (2^{3} -2^{2}\right ) \\
 &  = 18 -4 =14\text{ J}\end{align*}
\clearpage
%\rule{6.8cm}{0.5pt}\\
\examq Hooke's Law states that the force required to maintain a spring stretched $x$ units beyond its natural length is proportional to $x$, the spring displacement in meters, such that $f (x) =k x$ where $k$ is the spring constant and $f$ the force. This law holds provided $x$ does not get too large. \medskip\\
The work required to stretch the spring can be calculated by integrating the force function:
\[W=\int_{a}^{b}f(x) \, dx\,. \]
Find the work done in stretching a spring from 12 cm to 17 cm given that it takes 4 newtons of force to stretch the spring from 10 cm to 12 cm. \\
\solution\medskip\\
First you will need to find the spring constant $k$:
\begin{align*}f(x)&=kx\\
4\,\text{N}&=k(0.12-0.10)\\
k&=\frac{4}{0.02}=200\,\,\frac{\text{newtons}}{\text{meter}}
\end{align*}
Integrate the force function for find the work done in stretching the spring:
\begin{align*}
W&=\int_{0.12}^{0.17}200x\,dx\\
&=100x^2\Big\vert_{0.12}^{0.17}\\
&=100(0.17^2-0.12^2)\\
&=1.45 \text{ J}
\end{align*}
\clearpage
%---------------------------------------------------
% Chapter Exercises in a separate file
%---------------------------------------------------
\section{Chapter Exercises}
\subimport{}{5IntegrationExercises}


	\documentclass[a4paper]{article}
\newcommand{\dd}[1]{\mathrm{d}#1}
%% Language and font encodings
\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage[T1]{fontenc}

%% Sets page size and margins
\usepackage[a4paper,top=3cm,bottom=2cm,left=3cm,right=3cm,marginparwidth=1.75cm]{geometry}

%% Useful packages
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage[colorinlistoftodos]{todonotes}
\usepackage[colorlinks=true, allcolors=blue]{hyperref}
\usepackage{braket}
\usepackage{amssymb}
\usepackage{subcaption}
\usepackage[section]{placeins}
\usepackage{float}
\usepackage{color}
\restylefloat{table}

\title{Bayesian Classification Model}
\author{Shreyas Bapat, Bhavya Bhatt, GaganDeep Tomar}

\begin{document}
\maketitle

\begin{abstract}
The following discussion revolves around the modelling of data classifier in accordance with Bayesian Decision Theory. We then apply the model on three differnt kinds of dataset which are precisly - linearly separable, non-linearly separable and actual data which can be of random nature. The goal is to compare the performance of the model with the above datasets under different conditions on covariance matrix.
\end{abstract}

\section{Introduction}

The Bayesian Decision Theory is a probablistic theory for classifying the data points on the basics of pre-known prior and class conditional probabilities(which in real scenario is not known in any closed form expression). We give below the basics of Bayes rule in the context of pattern recognition.
\subsection{Bayes Rule}
The bayes rule states that if we have prior probabilities of a class and we know class conditional probability for each class then given a sample data point we can find the probability that it belongs to some particular class $i$ as follows
\begin{equation}\label{bayes equation}
P(C_{i}|\bar{x}) = \frac{P(\bar{x}|C_{i})P(C_{i})}{\sum_{j=1}^{m}P(\bar{x}|C_{j})P(C_{j})}
\end{equation} Where $P(C_{i})$ is prior probablity of class $i$ and $P(\bar{x}|C_{i})$ is class conditional probablity. Then we can find to which class the data point belong to as follows
\begin{equation}
C = \{C_{i}, max\{P(C_{i}|\bar{x}), i=1\dots m\}\}
\end{equation} where $m$ is the number of classes.
\subsection{Assumptions}
In the above equation $\ref{bayes equation}$ we have to have $P(C_{i})$ and $P(\bar{x}|C_{i})$ for the classification. We can obtain prior probablity by observing the training dataset. The ad-hoc assumption in our model is that class conditional probablity is considered to be normal distributed with parameters $\mu_{i}$ and $\sigma_{i}^{2}$ which are mean and variance of the dataset belonging to that particular class $i$. Note that the above parameters are scalar in univariate case(dimension of data point is one) but they would be a vector (mean vector)and a matrix(covariance matrix) respectively in bivariate and multivariate case. The expression for multidimensional gaussian distribution is
\begin{equation}
P(\bar{x}|C_{i}) = \frac{1}{\sqrt{det(2\pi\mathbf{\Sigma_{i}})}}exp\{\frac{-1}{2}(\bar{x}-\bar{\mu_{i}})^{\intercal}\mathbf{\Sigma_{i}}^{-1}(\bar{x}-\bar{\mu_{i}})\}
\end{equation}Where $\bar{\mu_{i}}$ is mean vector of class $i$ and $\mathbf{\Sigma_{i}}$ is covariance matrix of class $i$.
\section{Coutour Curves and Covariance Matrix}\label{appendix}
In this section\footnote{for a complete discussion refer to the appendix} we discuss the relation between the shape of the cross section produced by slicing the bivariate gaussian distribution with a hyperplane parallel to the 2D-feature plane and covariance matrix. The bivariate gaussian distribution is a follows
\begin{equation}
P(\bar{x}|C_{i}) = \frac{1}{\sqrt{det(2\pi\mathbf{\Sigma_{i}})}}exp\{\frac{-1}{2}(\bar{x}-\bar{\mu_{i}})^{\intercal}\mathbf{\Sigma_{i}}^{-1}(\bar{x}-\bar{\mu_{i}})\}
\end{equation}with $mu_{i}$ be a $2\times1$ mean column vector and $\Sigma_{i}$ be $2\times2$ covariance matrix. So we assume the covariance matrix in its expanded form as
\[
\mathbf{\Sigma} = \left[ {\begin{array}{cc}
\Sigma_{11} & \Sigma_{12} \\
\Sigma_{12} & \Sigma_{22} \\
\end{array}} \right]
\]
where diagonal terms are variance of the features and off-diagonal terms are covariance bewteen feature-1 and feature-2. The above matrix is symmetric precisly due to the fact that $cov(x_{i}, x_{j})=cov(x_{j}, x_{i})$. Now we set the above distribution function to some constant $k$ and find the resultant curve projected on the feature space which is called constant contour curve.
\[
\frac{1}{\sqrt{det(2\pi\mathbf{\Sigma_{i}})}}exp\{\frac{-1}{2}(\bar{x}-\bar{\mu_{i}})^{\intercal}\mathbf{\Sigma_{i}}^{-1}(\bar{x}-\bar{\mu_{i}})\} = k
\]after some manupilation and taking log both sides we get
\[
(\bar{x}-\bar{\mu})^{\intercal}\mathbf{\Sigma}^{-1}(\bar{x}-\bar{\mu})=-2\ln(\sqrt{2\pi\left|\mathbf{\Sigma}\right|}k)
\]where we have dropped the index $i$ for simplicity and the whole analysis can be done without the loss of generality. Now writing the matrix in full and evaluating the required operation on the column vector we get finally
\begin{equation}\label{cov}
\Sigma_{22}X_{1}^{2} + \Sigma_{11}X_{2}^{2} - 2\Sigma_{12}X_{1}X_{2} + 2\ln(\sqrt{2\pi\left|\mathbf{\Sigma}\right|}k)=0
\end{equation}where $\bar{x}=\left[x_{1}x_{2}\right]^{\intercal}$, $\bar{\mu}=\left[\mu_{1} \mu_{2}\right]^{\intercal}$, $X_{1}=x_{1}-\mu_{1}$ and $X_{2}=x_{2}-\mu_{2}$. This equation is in the form of general equation for conic section
\begin{equation}
ax^{2}+by^{2}+cxy+d=0
\end{equation}Now in our case the coefficients $a$ and $b$ are $\Sigma_{22}$ and $\Sigma_{11}$ respectively. The above equation thus represents an ellipse in our case as variances are always positive values. Now from the elementary analysis of conics we know that the coefficient of $xy$ represent the extend to which the ellipse is titled w.r.t to the axis. Also the coefficients of $x^{2}$ and $y^{2}$ represents the length of major and minor axis respectively. Now we consider $\Sigma_{12}=0$ (covariance matrix is diagonal) then the we recover the familiar equation of ellipse with major and minor axis parallel to the x-y axis. The equation is
\begin{equation}
\frac{X_{1}^{2}}{\left(\frac{-2\ln(\sqrt{2\pi\left|\mathbf{\Sigma}\right|}k)}{\Sigma_{22}}\right)}+\frac{X_{2}^{2}}{\left(\frac{-2\ln(\sqrt{2\pi\left|\mathbf{\Sigma}\right|}k)}{\Sigma_{11}}\right)}=1
\end{equation}which is of the form
\[
\frac{x^{2}}{A^{2}}+\frac{y^{2}}{B^{2}}=1
\]the above is the equation of countour curve projected in the feature space. Now we consider following three cases
\[
\Sigma_{22} < \Sigma_{11}\rightarrow A > B \hspace{0.5cm} ellipse
\]
\[
\Sigma_{22} = \Sigma_{11}\rightarrow A=B \hspace{0.5cm} circle
\]
\[
\Sigma_{22} > \Sigma_{11}\rightarrow A < B \hspace{0.5cm} ellipse
 \]
\section{Training and Testing}
In the presented work we would be taking test size of $0.25$. We would use training data points to train our classifier and testing data points to analyse the performance of the trained model(by performance we mean the ability of the model to correctly classify the unseen data points $i.e$ training examples). Training the model has different meaning for different models, here training means calculating the covariance matrix using training dataset and obtaining the required decision boundary.
\subsection{Discriminant function}
The decision boundary is represented mathematically using discriminant function. For Bayes Classifier, the discriminant function is as follows
\begin{equation}
f(\bar{x}) = g_{i}(\bar{x}) - g_{j}(\bar{x})
\end{equation}where $g_{i}(\bar{x})$ is
\begin{equation}
g_{i}(\bar{x}) = \ln(P(C_{i}|\bar{x}))
\end{equation}We observe that $f(\bar{x})=0$ is the equation for decision boundary. And $f(\bar{x})>0$ the data point belongs to class $i$ and $f(\bar{x})<0$ the data point belongs to class $j$. Now we know that $P(C_{i}|\bar{x})=P(\bar{x}|C_{i})P(C_{i})$ and taking logarithm simplfies the expression for the discriminant function\footnote{for more insight you can refer to the textbook, R.O.Duda, D.G.Stork Pattern Classification-wiley}. We have omitted the total probability term in the denominator because it is same for all the classes and does not contribute to the classifion process.
\section{Performance of the model}
In this section\footnote{Note that in the further discussion class-1 red, class-2 green, class-3 blue} we start our actual task of performance check on three different datasets. The analysis includes for each dataset we have considered four cases on covariance matrix which are as follows
\begin{itemize}
\item Covariance matrix for all the classes is the same and is $\sigma^{2}\mathbf{I}$.
\item Full Covariance matrix for all the classes is the same and is $\mathbf{\Sigma}$.
\item Covariance matric is diagonal and is different for each class
\item Full Covariance matrix for each class is different
\end{itemize}
and for each such case we calculate
\begin{itemize}
\item Classification accuracy, precision for every class, mean precision, recall for
every class, mean recall, F-measure for every class and mean F-measure on
test data
\item Confusion matrix\footnote{Note that entry $C_{ij}$ of confusion matrix represents number of points known to be in class $i$ but classified as class $j$} based on the performance for test data
\item Constant density contour plot for all the classes together with the training data
superposed
\item Decision region plot for every pair of classes together with the training data
superposed
\item Decision region plot for all the classes together with the training data
superposed
\end{itemize}
\newpage
\subsection{Dataset-1}
This dataset contains two types of two dimensional data points - linearly separable and non-linearly separable. Both have classification among three classes.
\subsubsection{Linearly Separable}
First we show the scatter plots of the data points in training dataset.
\begin{figure}[h!]
  \includegraphics[width=0.7\linewidth]{1_12.png}
  \caption{Dataset-1: Linearly Separable}
  \label{fig:Linearly Separable}
\end{figure}

As we can see from the figure \ref{fig:Linearly Separable} the data points are linearly separable. Now we consider the four stated cases on this dataset.
\newpage
\paragraph{Case-1: $\mathbf{\Sigma_{i}}=\sigma^{2}\mathbf{1}$}
According to the theory considered for this model we get a linear\footnote{linear means that if feature is two dimensional then it is a line, if three dimensional it is a hyperplane and similarly for higher dimensions} discriminant function of the form
\begin{equation}
\mathbf{w^{\intercal}}(\bar{x}-\bar{x_{0}})=0
\end{equation}
\[
\mathbf{w} = \bar{\mu_{i}}-\bar{\mu_{j}}
\]
\[
\bar{x_{0}} = \frac{1}{2}(\bar{\mu_{i}}+\bar{\mu_{j}})-\frac{\sigma^{2}}{\mid\mid \bar{\mu_{i}}-\bar{\mu_{j}}\mid\mid^{2}}\ln\frac{P(C_{i})}{P(C_{j})}(\bar{\mu_{i}}-\bar{\mu_{j}})
\]
where $\mathbf{w}$ is weight matrix and $\bar{x_{0}}$ through which the decision boundary(linear) passes. The decision surface between each pair of three classes are shown in figure \ref{fig:1_1}. Now we state the performance parameters obtained by testing the classifier on the test dataset(0.25).

\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{1_12_1.png}
     \caption{case-1: class-1 class-2}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{1_23_1.png}
    \caption{case-1: class-2 class-3}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{1_31_1.png}
    \caption{case-1: class-3 class-1}
  \end{subfigure}
  \caption{Decision Boundary and Training Data points for Linearly Seperable Dataset-1, Case-1}
  \label{fig:1_1}
\end{figure}

\subparagraph{Confusion Matrix}
The confusion matrix obtained is
\[
\mathbf{M} = \left[ {\begin{array}{ccc}
375 & 0 & 0\\
0 & 375 & 0\\
0 & 0 & 375\\
\end{array}} \right]
\]We observe here is that even though the diagonal terms (correctly classified) are appreciable but the cross diagonal terms suggest that this case of covariance matrix cannot be a pratical approximation for the dataset.
\subparagraph{Precision, Recall and F-measure} \textcolor{white}{:}
\begin{table}[H]
  \begin{center}
    \caption{Performance parameters for Linearly Seperable Dataset-1, Case -1}
    \label{tab:table1}
    \begin{tabular}{l|c|c|r} % <-- Alignments: 1st column left, 2nd middle and 3rd right, with vertical lines in between
      \textbf{class} & \textbf{Recall} & \textbf{Precision} & \textbf{F-Measure}\\
      \hline
      class-1 & 0.6639344262 & 0.3875598086 & 0.4894259819\\
      class-2 & 0.72 & 0.4812834225 & 0.5769230769\\
      class-3 & 0.184 & 0.5187165775 & 0.271642518\\
    \end{tabular}
  \end{center}
\end{table}
$\mathbf{Accuracy}$ 43.66$ \%,$ $\mathbf{Mean- Precision}$ 0.4625$ ,$ $\mathbf{Mean- Recall}$ 0.522$.$
\paragraph{Case-2: $\mathbf{\Sigma_{i}}=\mathbf{\Sigma}$}
According to the theory considered for this model we get a linear\footnote{linear means that if feature is two dimensional then it is a line, if three dimensional it is a hyperplane and similarly for higher dimensions} discriminant function of the form
\begin{equation}
\mathbf{w^{\intercal}}(\bar{x}-\bar{x_{0}})=0
\end{equation}
\[
\mathbf{w} = \mathbf{\Sigma}^{-1}(\bar{\mu_{i}}-\bar{\mu_{j}})
\]
\[
\bar{x_{0}} = \frac{1}{2}(\bar{\mu_{i}}+\bar{\mu_{j}})-\frac{\ln(P(C_{i}))-\ln(P(C_{j}))}{(\bar{\mu_{i}}-\bar{\mu_{j}})^{\intercal}\mathbf{\Sigma}^{-1}(\bar{\mu_{i}}-\bar{\mu_{j}})}(\bar{\mu_{i}}-\bar{\mu_{j}})
\]
where $\mathbf{w}$ is weight matrix and $\bar{x_{0}}$ through which the decision boundary(linear) passes. The decision surface between each pair of three classes are shown in figure \ref{fig:1_2}.
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{1_12_2.png}
     \caption{case-2: class-1 class-2}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{1_23_2.png}
    \caption{case-2: class-2 class-3}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{1_31_2.png}
    \caption{case-2: class-3 class-1}
  \end{subfigure}
  \caption{Decision Boundary and Training Data points for Linearly Seperable Dataset-1, Case-2}
  \label{fig:1_2}
\end{figure}
Now we state the performance parameters obtained by testing the classifier on the test dataset(0.25).
\subparagraph{Confusion Matrix}\label{reason}
The confusion matrix obtained is
\[
\mathbf{M} = \left[ {\begin{array}{ccc}
375 & 0 & 0\\
0 & 375 & 0\\
0 & 0 & 375\\
\end{array}} \right]
\]We can observe that this is performing even worst as it has two diagonal terms equal to zero which means it does not correctly classify any point in class $1$ and class $2$. So this also is not a realisitic approximation of the covariance matrix.

\subparagraph{Precision, Recall and F-measure}\textcolor{white}{:}
\begin{table}[H]
  \begin{center}
    \caption{Performance parameters for Linearly Seperable Dataset-1, Case -2}
    \label{tab:table1}
    \begin{tabular}{l|c|c|r} % <-- Alignments: 1st column left, 2nd middle and 3rd right, with vertical lines in between
      \textbf{class} & \textbf{Recall} & \textbf{Precision} & \textbf{F-Measure}\\
      \hline
      class-1 & 0 & 0 & NaN\\
      class-2 & 0 & 0 & NaN\\
      class-3 & 0.816 & 0.449339207 & 0.5795454545\\
    \end{tabular}
  \end{center}
\end{table}
$\mathbf{Accuracy}$ 40.8$\%,$ $\mathbf{Mean -Precision}$ 0.149$,$ $\mathbf{Mean- Recall}$ 0.272$.$
\paragraph{Case-3: $\mathbf{\Sigma_{i}}$ is diagonal and different}
According to the theory considered for this model we get a non-linear discriminant function of the form
\begin{equation}\label{eq:non-linear}
g_{i}(\bar{x}) = \bar{x}^{\intercal}\mathbf{W_{i}}\bar{x}+\bar{w_{i}}^{\intercal}\bar{x}+\omega_{i0}
\end{equation}
\[
\mathbf{W_{i}} = \frac{-1}{2}\mathbf{\Sigma_{i}}^{\intercal}
\]
\[
w_{i} = \mathbf{\Sigma_{i}}^{-1}\bar{\mu_{i}}
\]
\[
\omega_{i0} = \frac{-1}{2}\bar{\mu_{i}}^{\intercal}\mathbf{\Sigma_{i}}^{-1}\bar{\mu_{i}}-\frac{1}{2}\ln(\left|\mathbf{\Sigma_{i}}\right|)+\ln(P(C_{i}))
\]. From equation the decision surface between each pair of three classes would be non-linear as shown in figure \ref{fig:1_3}
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{1_12_3.png}
     \caption{case-3: class-1 class-2}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{1_23_3.png}
    \caption{case-3: class-2 class-3}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{1_31_3.png}
    \caption{case-3: class-3 class-1}
  \end{subfigure}
  \caption{Decision Boundary and Training Data points for Linearly Seperable Dataset-1, Case-3}
  \label{fig:1_3}
\end{figure}
\subparagraph{Confusion Matrix}
The confusion matrix obtained is
\[
\mathbf{M} = \left[ {\begin{array}{ccc}
375 & 0 & 0\\
0 & 375 & 0\\
0 & 0 & 375\\
\end{array}} \right]
\]This shows that off diagonal terms are appreciably decreased which makes this approximation to be reasonable one for this dataset.

\subparagraph{Precision, Recall and F-measure}\textcolor{white}{:}
\begin{table}[h!]
  \begin{center}
    \caption{Performance parameters for Linearly Seperable Dataset-1, Case -3}
    \label{tab:table1}
    \begin{tabular}{l|c|c|r} % <-- Alignments: 1st column left, 2nd middle and 3rd right, with vertical lines in between
      \textbf{class} & \textbf{Recall} & \textbf{Precision} & \textbf{F-Measure}\\
      \hline
      class-1 & 0.96 & 0.9230769231 & 0.9411764706\\
      class-2 & 0.936 & 0.9590163934 & 0.9473684211\\
      class-3 & 0.9959677419 & 0.9959677419 & 0.9959677419\\
    \end{tabular}
  \end{center}
\end{table}
\\$\mathbf{Accuracy}$ 96.8$\%,$ $\mathbf{Mean -Precision}$ 0.959$,$ $\mathbf{Mean- Recall}$ 0.963$.$
\newpage
\paragraph{Case-4: $\mathbf{\Sigma_{i}}$ is arbitary}
In this case also we can use the equation \ref{eq:non-linear}. The decision surface  between each pair of three classes would again be non-linear as shown in figure \ref{fig:1_4}
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{1_12_4.png}
     \caption{case-4: class-1 class-2}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{1_23_4.png}
    \caption{case-4: class-2 class-3}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{1_31_4.png}
    \caption{case-4: class-3 class-1}
  \end{subfigure}
  \caption{Decision Boundary and Training Data points for Linearly Seperable Dataset-1, Case-4}
  \label{fig:1_4}
\end{figure}
\subparagraph{Confusion Matrix}
The confusion matrix obtained is
\[
\mathbf{M} = \left[ {\begin{array}{ccc}
375 & 0 & 0\\
0 & 375 & 0\\
0 & 0 & 375\\
\end{array}} \right]
\]This also performs similar to that of the previous case.

\subparagraph{Precision, Recall and F-measure}\textcolor{white}{:}
\footnote{The discriminant functions discussed above for all the different cases would be same for all the subsequent datasets}
\begin{table}[h!]
  \begin{center}
    \caption{Performance parameters for Linearly Seperable Dataset-1, Case -4}
    \label{tab:table1}
    \begin{tabular}{l|c|c|r} % <-- Alignments: 1st column left, 2nd middle and 3rd right, with vertical lines in between
      \textbf{class} & \textbf{Recall} & \textbf{Precision} & \textbf{F-Measure}\\
      \hline
      class-1 & 0.952 & 0.9224806202 & 0.937007874\\
      class-2 & 0.936 & 0.9512195122 & 0.9435483871\\
      class-3 & 0.9959677419 & 0.9959677419 & 0.9959677419\\
    \end{tabular}
  \end{center}
\end{table}
\\$\mathbf{Accuracy}$ 96.6$\%,$ $\mathbf{Mean- Precision}$ 0.956$, \mathbf{Mean -Recall}$ 0.961$.$
\newpage
\subparagraph{Conclusion}
The best performance is given by case-$3$ and worst performance is given by case-$2$. Now we shown the final decision boundary simultaneously for all the three classes and for all the four cases along with their contour plots superimposed on them in figure \ref{fig:cont_1_4}
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{cont_1_123_1.png}
     \caption{case-1: interclass decision surface and contour plot}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{cont_1_123_2.png}
    \caption{case-2: interclass decision surface and contour plot}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{cont_1_123_3.png}
    \caption{case-3: interclass decision surface and contour plot}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{cont_1_123_4.png}
    \caption{case-4: interclass decision surface and contour plot}
  \end{subfigure}
  \caption{decision boundary for all three classes and contour plots}
  \label{fig:cont_1_4}
\end{figure}
\goodbreak \newpage \newpage
\subsubsection{Non-Linearly separable}
First we show the scatter plots of the data points in training dataset.
\begin{figure}[!h]
  \includegraphics[width=0.7\linewidth]{2.png}
  \caption{Dataset-1: Non-Linearly Separable}
  \label{fig:Non Linearly Separable}
\end{figure}
As we can see from the figure \ref{fig:Non Linearly Separable} the data points are not linearly separable. Now we consider the four stated cases on this dataset.
\newpage
\paragraph{Case-1: $\mathbf{\Sigma_{i}}=\sigma^{2}\mathbf{1}$}
The decision boudary between each pair of classes is shown in figure \ref{fig:2_1}
\begin{figure}[h]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{2_12_1.png}
     \caption{case-1: class-1 class-2}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{2_23_1.png}
    \caption{case-1: class-2 class-3}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{2_31_1.png}
    \caption{case-1: class-3 class-1}
  \end{subfigure}
  \caption{Decision Boundary and Training Data points for Non-Linearly Seperable Dataset-1, Case-1}
  \label{fig:2_1}
\end{figure}
\subparagraph{Confusion Matrix}
The confusion matrix obtained is
\[
\mathbf{M} = \left[ {\begin{array}{ccc}
272 & 0 & 103\\
67 & 263 & 43\\
308 & 300 & 142\\
\end{array}} \right]
\]We can observe here that cross-diagonal terms are exceptionally greater than diagonal terms so this is not a good approximation for the dataset.

\subparagraph{Precision, Recall and F-measure}:
\begin{table}[h!]
  \begin{center}
    \caption{Performance parameters for Non-Linearly Seperable Dataset-1, Case -1}
    \label{tab:table1}
    \begin{tabular}{l|c|c|r} % <-- Alignments: 1st column left, 2nd middle and 3rd right, with vertical lines in between
      \textbf{class} & \textbf{Recall} & \textbf{Precision} & \textbf{F-Measure}\\
      \hline
      class-1 & 0.6639344262 & 0.4358974359 & 0.526275559\\
      class-2 & 0.768 & 0.4571428571 & 0.5731343284\\
      class-3 & 0.192 & 0.5052631579 & 0.2782608696\\
    \end{tabular}
  \end{center}
\end{table}
\\
$\mathbf{Accuracy}$ 45.8$\%,$ $\mathbf{Mean- Precision}$ 0.466$,$ $\mathbf{Mean -Recall}$ 0.541$.$
\newpage
\paragraph{Case-2: $\mathbf{\Sigma_{i}}=\mathbf{\Sigma}$}

The decision boundary between each pair of classes is shown in figure \ref{fig:2_2}
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{2_12_2.png}
     \caption{case-2: class-1 class-2}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{2_23_2.png}
    \caption{case-2: class-2 class-3}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{2_31_2.png}
    \caption{case-2: class-3 class-1}
  \end{subfigure}
  \caption{Decision Boundary and Training Data points for Non-Linearly Seperable Dataset-1, Case-2}
  \label{fig:2_2}
\end{figure}
\subparagraph{Confusion Matrix}
The confusion matrix obtained is
\[
\mathbf{M} = \left[ {\begin{array}{ccc}
0 & 0 & 375\\
0 & 0 & 375\\
114 & 5 & 631\\
\end{array}} \right]
\]This is performing worst\footnote{same reason as given in section \ref{reason}}.\\

\subparagraph{Precision, Recall and F-measure} \textcolor{white}{:}
\begin{table}[h!]
  \begin{center}
    \caption{Performance parameters for Non-Linearly Seperable Dataset-1, Case -2}
    \label{tab:table1}
    \begin{tabular}{l|c|c|r} % <-- Alignments: 1st column left, 2nd middle and 3rd right, with vertical lines in between
      \textbf{class} & \textbf{Recall} & \textbf{Precision} & \textbf{F-Measure}\\
      \hline
      class-1 & 0 & 0 & NaN\\
      class-2 & 0 & 0 & NaN\\
      class-3 & 0.796 & 0.4432071269 & 0.5693848355\\
    \end{tabular}
  \end{center}
\end{table}
\\
$\mathbf{Accuracy}$ 39.8$\%,$ $\mathbf{Mean- Precision}$ 0.147$,$ $\mathbf{Mean- Recall}$ 0.265$.$
\newpage
\paragraph{Case-3: $\mathbf{\Sigma_{i}}$ is diagonal and different}
The decision boundary is shown in figure \ref{fig:2_3}
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{2_12_3.png}
     \caption{case-3: class-1 class-2}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{2_23_3.png}
    \caption{case-3: class-2 class-3}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{2_31_3.png}
    \caption{case-3: class-3 class-1}
  \end{subfigure}
  \caption{Decision Boundary and Training Data points for Non-Linearly Seperable Dataset-1, Case-3}
  \label{fig:2_3}
\end{figure}
\subparagraph{Confusion Matrix}
The confusion matrix obtained is
\[
\mathbf{M} = \left[ {\begin{array}{ccc}
348 & 24 & 3\\
24 & 342 & 9\\
7 & 0 & 743\\
\end{array}} \right]
\]This approximation on covariance matrix is performing fairly good.\\

\subparagraph{Precision, Recall and F-measure} \textcolor{white}{:}
\begin{table}[h!]
  \begin{center}
    \caption{Performance parameters for Non-Linearly Seperable Dataset-1, Case -3}
    \label{tab:table1}
    \begin{tabular}{l|c|c|r} % <-- Alignments: 1st column left, 2nd middle and 3rd right, with vertical lines in between
      \textbf{class} & \textbf{Recall} & \textbf{Precision} & \textbf{F-Measure}\\
      \hline
      class-1 & 0.872 & 0.9396551724 & 0.9045643154\\
      class-2 & 0.936 & 0.9285714286 & 0.9322709163\\
      class-3 & 0.992 & 0.9612403101 & 0.9763779528\\
    \end{tabular}
  \end{center}
\end{table}
\\$\mathbf{Accuracy}$ 94.8$\%,$ $\mathbf{Mean- Precision}$ 0.943$,$ $\mathbf{Mean- Recall}$ 0.933$.$
\newpage
\paragraph{Case-4: $\mathbf{\Sigma_{i}}$ is arbitary}
The decision boundary is shown in figure \ref{fig:2_4}
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{2_12_4.png}
     \caption{case-4: class-1 class-2}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{2_23_4.png}
    \caption{case-4: class-2 class-3}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{2_31_4.png}
    \caption{case-4: class-3 class-1}
  \end{subfigure}
  \caption{case-4 Decision Boundary and Training Data points for Non-Linearly Seperable Dataset-1, Case-4}
  \label{fig:2_4}
\end{figure}
\subparagraph{Confusion Matrix}
The confusion matrix obtained is
\[
\mathbf{M} = \left[ {\begin{array}{ccc}
349 & 26 & 0\\
23 & 342 & 10\\
6 & 0 & 744\\
\end{array}} \right]
\]This also peforms fairly good similar to the above case.\\

\subparagraph{Precision, Recall and F-measure} \textcolor{white}{:}
\begin{table}[h!]
  \begin{center}
    \caption{Performance parameters for Non-Linearly Seperable Dataset-1, Case -4}
    \label{tab:table1}
    \begin{tabular}{l|c|c|r} % <-- Alignments: 1st column left, 2nd middle and 3rd right, with vertical lines in between
      \textbf{class} & \textbf{Recall} & \textbf{Precision} & \textbf{F-Measure}\\
      \hline
      class-1 & 0.904 & 0.9416666667 & 0.9224489796\\
      class-2 & 0.936 & 0.9285714286 & 0.9322709163\\
      class-3 & 0.992 & 0.9763779528 & 0.9841269841\\
    \end{tabular}
  \end{center}
\end{table}
\\$\mathbf{Accuracy}$ 95.6$\%,$ $\mathbf{Mean -Precision}$ 0.948$,$ $\mathbf{Mean- Recall}$ 0.944$.$
\newpage
\subparagraph{Conclusion}
The best performance is given by case-$3$ and worst performance is given by case-$2$. Now we shown the final decision boundary simultaneously for all the three classes and for all the four cases along with their contour plots superimposed on them in figure \ref{fig:cont_2_4}
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{cont_2_123_1.png}
     \caption{case-1: interclass decision surface and contour plot}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{cont_2_123_2.png}
    \caption{case-2: interclass decision surface and contour plot}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{cont_2_123_3.png}
    \caption{case-3: interclass decision surface and contour plot}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{cont_2_123_4.png}
    \caption{case-4: interclass decision surface and contour plot}
  \end{subfigure}
  \caption{decision boundary for all three classes and contour plots}
  \label{fig:cont_2_4}
\end{figure}
\newpage

\subsection{Dataset-2}
This dataset is real dataset and is much more random\footnote{randomness here means that classes can be overlapping also} than the dataset-1 considered above. We carry out similar analysis for this dataset also. First we show the scatter plots of the data points in training dataset in figure \ref{fig:3}
\begin{figure}[!h]
  \includegraphics[width=0.7\linewidth]{3.png}
  \caption{Dataset-2: real dataset}
  \label{fig:3}
\end{figure}
\newpage
\paragraph{Case-1: $\mathbf{\Sigma_{i}}=\sigma^{2}\mathbf{1}$}
The decision boundary is shown in figure \ref{fig:3_1}
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{3_12_1.png}
     \caption{case-1: class-1 class-2}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{3_23_1.png}
    \caption{case-1: class-2 class-3}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{3_31_1.png}
    \caption{case-1: class-3 class-1}
  \end{subfigure}
  \caption{Decision Boundary and Training Data points for Dataset-2, Case-1}
  \label{fig:3_1}
\end{figure}
\subparagraph{Confusion Matrix}
The confusion matrix obtained is
\[
\mathbf{M} = \left[ {\begin{array}{ccc}
1629 & 211 & 0\\
298 & 1568 & 0\\
9 & 8 & 1701\\
\end{array}} \right]
\]We observe here that it is performing good with the approximated covariance matrix.\\

\subparagraph{Precision, Recall and F-measure} \textcolor{white}{:}
\begin{table}[h!]
  \begin{center}
    \caption{Performance parameters for Dataset-2, Case-1}
    \label{tab:table1}
    \begin{tabular}{l|c|c|r} % <-- Alignments: 1st column left, 2nd middle and 3rd right, with vertical lines in between
      \textbf{class} & \textbf{Recall} & \textbf{Precision} & \textbf{F-Measure}\\
      \hline
      class-1 & 0.8713355049 & 0.8546325879 & 0.8629032258\\
      class-2 & 0.8569131833 & 0.8666666667 & 0.8617623282\\
      class-3 & 0.9912739965 & 1 & 0.9956178791\\
    \end{tabular}
  \end{center}
\end{table}
\\$\mathbf{Accuracy}$ 90.4$\%,$ $\mathbf{Mean -Precision}$ 0.907$,$ $\mathbf{Mean- Recall}$ 0.906$.$
\newpage
\paragraph{Case-2: $\mathbf{\Sigma_{i}}=\mathbf{\Sigma}$}
The decision boundary is shown in figure \ref{fig:3_2}
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{3_12_2.png}
     \caption{case-2: class-1 class-2}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{3_23_2.png}
    \caption{case-2: class-2 class-3}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{3_31_2.png}
    \caption{case-2: class-3 class-1}
  \end{subfigure}
  \caption{Decision Boundary and Training Data points for Dataset-2, Case-2}
  \label{fig:3_2}
\end{figure}
\subparagraph{Confusion Matrix}
The confusion matrix obtained is
\[
\mathbf{M} = \left[ {\begin{array}{ccc}
1620 & 220 & 0\\
158 & 1708 & 0\\
7 & 11 & 1700\\
\end{array}} \right]
\]
\\

\subparagraph{Precision, Recall and F-measure} \textcolor{white}{:}
\begin{table}[h!]
  \begin{center}
    \caption{Performance parameters for Dataset-2, Case-2}
    \label{tab:table1}
    \begin{tabular}{l|c|c|r} % <-- Alignments: 1st column left, 2nd middle and 3rd right, with vertical lines in between
      \textbf{class} & \textbf{Recall} & \textbf{Precision} & \textbf{F-Measure}\\
      \hline
      class-1 & 0.8680781759 & 0.9126712329 & 0.8898163606\\
      class-2 & 0.922829582 & 0.896875 & 0.9096671949\\
      class-3 & 0.9895287958 & 1 & 0.9947368421\\
    \end{tabular}
  \end{center}
\end{table}
\\$\mathbf{Accuracy}$ 92.5$\%,$ $\mathbf{Mean -Precision}$ 0.936$,$ $\mathbf{Mean -Recall}$ 0.926$.$
\newpage
\paragraph{Case-3: $\mathbf{\Sigma_{i}}$ is diagonal and different}
The decision boundary is shown in figure \ref{fig:3_3}
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{3_12_3.png}
     \caption{case-3: class-1 class-2}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{3_23_3.png}
    \caption{case-3: class-2 class-3}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{3_31_3.png}
    \caption{case-3: class-3 class-1}
  \end{subfigure}
  \caption{Decision Boundary and Training Data points for Dataset-2, Case-3}
  \label{fig:3_3}
\end{figure}
\subparagraph{Confusion Matrix}
The confusion matrix obtained is
\[
\mathbf{M} = \left[ {\begin{array}{ccc}
1611 & 229 & 0\\
215 & 1651 & 0\\
11 & 3 & 1704\\
\end{array}} \right]
\]
\\

\subparagraph{Precision, Recall and F-measure} \textcolor{white}{:}
\begin{table}[h!]
  \begin{center}
    \caption{Performance parameters for Dataset-2, Case-3}
    \label{tab:table1}
    \begin{tabular}{l|c|c|r} % <-- Alignments: 1st column left, 2nd middle and 3rd right, with vertical lines in between
      \textbf{class} & \textbf{Recall} & \textbf{Precision} & \textbf{F-Measure}\\
      \hline
      class-1 & 0.8631921824 & 0.8833333333 & 0.8731466227\\
      class-2 & 0.8906752412 & 0.865625 & 0.8779714739\\
      class-3 & 0.9930191972 & 1 & 0.996497373\\
    \end{tabular}
  \end{center}
\end{table}
\\$\mathbf{Accuracy}$ 91.3$\%,$ $\mathbf{Mean- Precision}$ 0.916$,$ $\mathbf{Mean- Recall}$ 0.915$.$
\newpage
\paragraph{Case-4: $\mathbf{\Sigma_{i}}$ is arbitary}
The decision boundary is shown in figure \ref{fig:3_4}
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{3_12_4.png}
     \caption{case-4: class-1 class-2}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{3_23_4.png}
    \caption{case-4: class-2 class-3}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{3_31_4.png}
    \caption{case-4: class-3 class-1}
  \end{subfigure}
  \caption{Decision Boundary and Training Data points for Dataset-2, Case-4}
  \label{fig:3_4}
\end{figure}
\subparagraph{Confusion Matrix}
The confusion matrix obtained is
\[
\mathbf{M} = \left[ {\begin{array}{ccc}
1571 & 263 & 6\\
183 & 1683 & 0\\
14 & 3 & 1701\\
\end{array}} \right]
\]
\\

\subparagraph{Precision, Recall and F-measure} \textcolor{white}{:}
\begin{table}[h!]
  \begin{center}
    \caption{Performance parameters for Dataset-2, Case-4}
    \label{tab:table1}
    \begin{tabular}{l|c|c|r} % <-- Alignments: 1st column left, 2nd middle and 3rd right, with vertical lines in between
      \textbf{class} & \textbf{Recall} & \textbf{Precision} & \textbf{F-Measure}\\
      \hline
      class-1 & 0.8575667656 & 0.898911353 & 0.8777524677\\
      class-2 & 0.9019292605 & 0.8538812785 & 0.8772478499\\
      class-3 & 0.9895287958 & 0.9964850615 & 0.9929947461\\
    \end{tabular}
  \end{center}
\end{table}
\\$\mathbf{Accuracy}$ 94.3$\%,$ $\mathbf{Mean- Precision}$ 0.916$,$ $\mathbf{Mean- Recall}$ 0.916$.$
\newpage
\subparagraph{Conclusion}
We can see that all of the above cases on covariance matrix are good approximation for this dataset and any of the approximate form can be used for classification. Now we shown the final decision boundary simultaneously for all the three classes and for all the four cases along with their contour plots superimposed on them in figure \ref{fig:cont_3_4}
\begin{figure}[h!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{cont_3_123_1.png}
     \caption{case-1: interclass decision surface and contour plot}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{cont_3_123_2.png}
    \caption{case-2: interclass decision surface and contour plot}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{cont_3_123_3.png}
    \caption{case-3: interclass decision surface and contour plot}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{cont_3_123_4.png}
    \caption{case-4: interclass decision surface and contour plot}
  \end{subfigure}
  \caption{decision boundary for all three classes and contour plots}
  \label{fig:cont_3_4}
\end{figure}
\newpage
\section{Appendix}
This section includes some of the derivations relating the angle with which the ellipse(constant contour curves) is titled with the axis and the elements of the covariance matrix. This section can be skiped without any loss. The derivation includes all the points discussed in section \ref{appendix}. We first assume general form of elliptical curve in two dimensions as follows\footnote{Note that we are not considering the terms which are because of shift of origin because it would not affect the shape and tilted angle}
\begin{equation}\label{original}
ax^{2}+by^{2}+cxy+d=0
\end{equation}Now we rotate the axis(active rotation) using rotation matrix for two dimensional space as follows

\[
\left( \begin{array}{c}
x^{\prime} \\
y^{\prime}
\end{array} \right) =
\left( \begin{array}{cc}
\cos\theta & -\sin\theta \\
\sin\theta & \cos\theta
\end{array} \right)
%
\left( \begin{array}{c}
x \\
y
\end{array} \right)
\]
Now we find $x$ and $y$ in terms of primed $x^{\prime}$ and $y^{\prime}$. After simple algebraic calculations and manipulations we obtain following equation in $x^{\prime}$-$y^{\prime}$ coordinate system
\begin{equation}\label{main}
\begin{split}
(a\cos^{2}\theta+b\sin^{2}\theta-c\cos\theta\sin\theta)x^{\prime 2} + (a\sin^{2}\theta+b\cos^{2}\theta+c\cos\theta\sin\theta)y^{\prime 2}&\\
+(2a\cos\theta\sin\theta-2b\sin\theta\cos\theta+c(\cos^{2}\theta-\sin^{2}\theta))x^{\prime}y^{\prime}+d=0
\end{split}
\end{equation}Here $\theta$ is the angle with which the coordinate axis are rotated(which we further state that is the angle with which the major and minor axis of the tilted ellipse is making with $x$-$y$ axis). Now we impose the condition that in this coordinate system the above ellipse should be parallel to the axis and is of the standard form
\begin{equation}
\frac{x^{\prime 2}}{A^{2}}+\frac{y^{\prime 2}}{B^{2}}=1
\end{equation}for this we should have coefficient of $x^{\prime}y^{\prime}$ in the equation \ref{main} to be zero which is equivalent as
\[
\begin{split}
2a\cos\theta\sin\theta-2b\sin\theta\cos\theta+c(\cos^{2}\theta-\sin^{2}\theta) &= 0 \\
(a-b)\sin2\theta+c\cos2\theta &= 0\\
\end{split}
\]
\begin{equation}\label{thet}
\tan2\theta = \frac{c}{b-a}
\end{equation}
This precisly gives the dependence of angle on the coefficients in equation
\ref{original}. This can be easily proved that if $c=0$ (angle would then be zero acc. to \ref{thet})then the ellipse given by equation \ref{original} would be parallel to the $x$-$y$ axis. Now for this ellipse to be a circle we equate the coefficients of $x^{\prime 2}$ and $y^{\prime 2}$ in equation \ref{main}
\[
\begin{split}
a\cos^{2}\theta+b\sin^{2}\theta-c\cos\theta\sin\theta &= a\sin^{2}\theta+b\cos^{2}\theta+c\cos\theta\sin\theta \\
a\cos^{2}\theta-a\sin^{2}\theta+b\sin^{2}\theta-b\cos^{2}\theta &=2c\cos\theta\sin\theta \\
(a-b)\cos2\theta &=c\sin2\theta \\
\tan2\theta &= \frac{a-b}{c}
\end{split}
\]but from equation \ref{thet} we can finally write the above condition as
\begin{equation}
-(a-b)^{2}=c^{2}
\end{equation}which is true only when $a=b$ and $c=0$(this condition is not ad-hoc and makes much more sense as there is no such thing as rotated circle so we can safely consider $c=0$ without loss of generality). So for ellipse to be a circle the coefficients of $x^{2}$ and $y^{2}$ should be equal and you recover the standard equation of circle
\begin{equation}
x^{2}+y^{2}=r^{2}
\end{equation}Now this whole analysis can be used to relate the coefficients of equation \ref{original} to that of the section \ref{appendix} which on comparision with equation \ref{cov} gives us
\[
a = \Sigma_{22}
\]
\[
b = \Sigma_{11}
\]
\[
c = -2\Sigma_{12}
\]and now you can correlate the conclusions given in section \ref{appendix} with the above calculations.
\bibliographystyle{alpha}
\bibliography{sample}

\end{document}


\section{Description Logics}
\label{app:dl}
In this section, we recall the expressive description logic $\mathcal{ALC}$ \cite{DBLP:journals/ai/Schmidt-SchaussS91}. We refer to
\cite{DBLP:journals/ws/LukasiewiczS08} for a detailed description of $\mathcal{SHOIN}(\mathbf{D})$ DL, that is at the basis of OWL DL.

%%ALC
Let $\mathbf{A}$, $\mathbf{R}$ and $\mathbf{I}$ be sets of \emph{atomic concepts}, \emph{roles} and \emph{individuals}.
A \emph{role} is an atomic role $R \in \mathbf{R}$.
\emph{Concepts} are defined by induction as follows. Each $C \in \mathbf{A}$, $\bot$ and $\top$
are concepts.
If $C$, $C_1$ and $C_2$ are concepts and $R \in \mathbf{R}$, then $(C_1\sqcap C_2)$, $(C_1\sqcup C_2 )$, $\neg C$,
$\exists R.C$, and $\forall R.C$ are concepts.
Let $C$, $D$ be concepts,  $R \in \mathbf{R}$ and $a, b \in \mathbf{I}$.
An \emph{ABox} $\cA$ is a finite set of \textit{concept membership axioms} $a : C$ and \textit{role membership
	axioms} $(a, b) : R$, while
a \emph{TBox} $\cT$ is a finite set of \textit{concept inclusion axioms} $C\sqsubseteq D$. $C \equiv D$ abbreviates $C \sqsubseteq D$ and $D\sqsubseteq  C$.

A \emph{knowledge base} $\cK = (\cT, \cA)$ consists of a TBox $\cT$ and an ABox $\cA$.
A KB $\cK$ is assigned a semantics in terms of set-theoretic interpretations $\cI = (\Delta^\cI , \cdot^\cI )$, where $\Delta^\cI$ is a non-empty \textit{domain} and $\cdot^\cI$ is the \textit{interpretation function} that assigns  an element in $\Delta ^\cI$ to each $a \in \mathbf{I}$, a subset of $\Delta^\cI$ to each $C \in \mathbf{A}$ and a subset of $\Delta^\cI \times \Delta^\cI$ to each $R \in \mathbf{R}$.

\section{DISPONTE}
\label{app:disponte}
In the field of Probabilistic Logic Programming (PLP for short) many proposals have been presented.
An effective and popular approach is the Distribution Semantics \cite{DBLP:conf/iclp/Sato95}, which underlies many PLP languages such as
PRISM~\cite{DBLP:conf/iclp/Sato95,DBLP:journals/jair/SatoK01},
Independent Choice Logic  \cite{Poo97-ArtInt-IJ}, Logic Programs with Annotated Disjunctions \cite{VenVer04-ICLP04-IC} and ProbLog \cite{DBLP:conf/ijcai/RaedtKT07}.
Along this line, many reserchers proposed to combine probability theory with Description Logics (DLs for short) \cite{DBLP:journals/ws/LukasiewiczS08,DBLP:conf/rweb/Straccia08}.
DLs are at the basis of the Web Ontology Language (OWL for short), a family of knowledge representation formalisms used for modeling information
of the Semantic Web

TRILL follows the DISPONTE \cite{RigBelLamZese12-URSW12,Zese17-SSW-BK} semantics to compute the probability of queries.
DISPONTE applies the distribution semantics \cite{DBLP:conf/iclp/Sato95} of probabilistic logic programming to DLs.
A program following this semantics defines a probability distribution over normal logic programs
called \emph{worlds}. Then the distribution is extended to queries and the probability of a query is obtained by marginalizing the joint distribution of the query and the programs.

In DISPONTE, a \emph{probabilistic knowledge base} $\cK$ is a set of \emph{certain axioms} or \emph{probabilistic axioms} in which each axiom is independent evidence.
Certain axioms take the form of regular DL axioms while probabilistic axioms are
$p::E$
where $p$ is a real number in $[0,1]$ and $E$ is a DL axiom.

The idea of DISPONTE is to associate independent Boolean random variables to the probabilistic axioms.
To obtain a \emph{world}, we include every formula obtained from a certain axiom.
For each probabilistic axiom, we decide whether to include it or not in $w$.
A world therefore is a non probabilistic KB that can be assigned a semantics in the usual way.
A query is entailed by a world if it is true in every  model of the world.

The probability $p$ can be interpreted as an \emph{epistemic probability}, i.e., as the degree of our belief in axiom $E$.
For example, a probabilistic concept membership axiom
$
p::a:C
$
means that we have degree of belief $p$ in $C(a)$.
A probabilistic concept inclusion axiom of the form
$
p::C\sqsubseteq D
$
represents our belief in the truth of $C \sqsubseteq D$ with probability $p$.

Formally, an \emph{atomic choice} is a couple $(E_i,k)$ where $E_i$ is the $i$th probabilistic axiom  and $k\in \{0,1\}$.
$k$ indicates whether $E_i$ is chosen to be included in a world ($k$ = 1) or not ($k$ = 0).
A \emph{composite choice} $\kappa$ is a consistent set of atomic choices, i.e.,  $(E_i,k)\in\kappa, (E_i,m)\in \kappa$ implies $k=m$ (only one decision is taken for each formula).
The probability of a composite choice $\kappa$  is
$P(\kappa)=\prod_{(E_i,1)\in \kappa}p_i\prod_{(E_i, 0)\in \kappa} (1-p_i)$, where $p_i$ is the probability associated with axiom $E_i$.
A \emph{selection} $\sigma$ is a total composite choice, i.e., it contains an atomic choice $(E_i,k)$ for every
probabilistic axiom  of the probabilistic KB.
A selection $\sigma$ identifies a theory $w_\sigma$ called  a \emph{world} in this way:
$w_\sigma=\cC\cup\{E_i|(E_i,1)\in \sigma\}$ where $\cC$ is the set of certain axioms. Let us indicate with $\mathcal{S}_\cK$ the set of all selections and with $\mathcal{W}_\cK$ the set of all worlds.
The probability of a world $w_\sigma$  is
$P(w_\sigma)=P(\sigma)=\prod_{(E_i,1)\in \sigma}p_i\prod_{(E_i, 0)\in \sigma} (1-p_i)$.
$P(w_\sigma)$ is a probability distribution over worlds, i.e., $\sum_{w\in \mathcal{W}_\cK}P(w)=1$.

We can now assign probabilities to queries.
Given a world $w$, the probability of a query $Q$ is defined as $P(Q|w)=1$ if $w\models Q$ and 0 otherwise.
The probability of a query can be defined by marginalizing the joint probability of the query and the worlds, i.e.
$P(Q)=\sum_{w\in \mathcal{W}_\cK}P(Q,w)=\sum_{w\in \mathcal{W}_\cK} P(Q|w)p(w)=\sum_{w\in \mathcal{W}_\cK: w\models Q}P(w)$.

\begin{example}
	\label{people+petsxy}
	\begin{small}
		Consider the following KB, inspired by the \texttt{people+pets} ontology  \cite{ISWC03-tut}:
		{\center $0.5\ \ ::\ \ \exists hasAnimal.Pet \sqsubseteq NatureLover\ \ \ \ \ 0.6\ \ ::\ \ Cat\sqsubseteq Pet$\\
			$(kevin,tom):hasAnimal\ \ \ \ \ (kevin,\fluffy):hasAnimal\ \ \ \ \ tom: Cat\ \ \ \ \ \fluffy: Cat$\\}
		\noindent The KB indicates that the individuals that own an animal which is a pet are nature lovers with a 50\% probability and that $kevin$ has the animals
		$\fluffy$ and $tom$.  Fluffy and $tom$ are cats and cats are pets with probability 60\%.
		We associate a Boolean variable to each axiom as follow
		%\begin{small}
		$F_1 = \exists hasAnimal.Pet \sqsubseteq NatureLover$, $F_2=(kevin,\fluffy):hasAnimal$, $F_3=(kevin,tom):hasAnimal$, $F_4=\fluffy: Cat$, $F_5=tom: Cat$ and $F_6= Cat\sqsubseteq Pet$.
		%\end{small}.

		The KB has four worlds and the query axiom $Q=kevin:NatureLover$ is true in one of them, the one corresponding to the selection
		$
		\{(F_1,1),(F_2,1)\}
		$.
		The probability of the query is $P(Q)=0.5\cdot 0.6=0.3$.
	\end{small}
\end{example}

\begin{example}
	\label{people+pets_comb}
	\begin{small}
		Sometimes we have to combine knowledge from multiple, untrusted sources, each one with a different reliability.
		Consider a KB similar to the one of Example \ref{people+petsxy} but where we have a single cat, $\fluffy$.
		{\center $\exists hasAnimal.Pet \sqsubseteq NatureLover\ \ \ \ \ (kevin,\fluffy):hasAnimal\ \ \ \ \ Cat\sqsubseteq Pet$\\}

		\noindent and there are two sources of information with different reliability that provide the information that $\fluffy$ is a cat.
		On one source the user has a degree of belief of 0.4, i.e., he thinks it is correct with a 40\% probability,
		while on the other source he has a degree of belief 0.3. %, i.e. he thinks it is correct with a 30\% probability.
		The user can reason on this knowledge by adding the following statements to his KB:
		{\center$0.4\ \ ::\ \ \fluffy: Cat\ \ \ \ \ 0.3\ \ ::\ \ \fluffy: Cat$\\}
		The two statements represent independent evidence on $\fluffy$ being a cat. We associate $F_1$ ($F_2$) to the first (second) probabilistic axiom.

		The query axiom $Q=kevin:NatureLover$ is true in 3 out of the 4 worlds, those corresponding to the selections
		$
		\{ \{(F_1,1),(F_2,1)\},
		\{(F_1,1),(F_2,0)\},
		\{(F_1,0),(F_2,1)\}\}
		$.
		So
		$P(Q)=0.4\cdot 0.3+0.4\cdot 0.7+ 0.6\cdot 0.3=0.58.$
		This is reasonable if the two sources can be considered as independent. In fact,  the probability comes from the  disjunction of two
		independent Boolean random variables with probabilities respectively 0.4 and 0.3:
		$
		P(Q) = P(X_1\vee X_2) = P(X_1)+P(X_2)-P(X_1\wedge X_2)
		= P(X_1)+P(X_2)-P(X_1)P(X_2)
		= 0.4+0.3-0.4\cdot 0.3=0.58
		$
	\end{small}
\end{example}

\section{Inference}
\label{app:inf}
Traditionally, a reasoning algorithm decides  whether an axiom is entailed or not by a KB by refutation: the  axiom $E$ is entailed if $\neg E$ has no model
in the KB.
Besides deciding whether an axiom is entailed by a KB, we want to find also explanations for the axiom, in order to compute the probability of the axiom.

\subsection{Computing Queries Probability}
The problem of finding  explanations for a query
has been investigated by various authors \cite{DBLP:conf/ijcai/SchlobachC03,DBLP:journals/ws/KalyanpurPSH05,DBLP:conf/semweb/KalyanpurPHS07,Kalyanpurphd,extended_tracing,Zese17-SSW-BK}.
It was called  \emph{axiom pinpointing} in
\cite{DBLP:conf/ijcai/SchlobachC03}  and considered as a non-standard reasoning service useful for tracing derivations and debugging ontologies.
In particular, in \cite{DBLP:conf/ijcai/SchlobachC03} the authors define \emph{minimal axiom sets}  (\emph{MinAs} for short).
\begin{definition}[MinA]
	Let $\cK$ be a knowledge base and $Q$ an
	axiom that follows from it, i.e.,
	$\cK \models Q$. We call a set
	$M\subseteq \cK$ a
	\emph{minimal axiom set} or \emph{MinA} for $Q$ in $\cK$ if
	$M \models Q$ and it is minimal
	w.r.t. set inclusion.
\end{definition}
\noindent The problem of enumerating all MinAs is called \textsc{min-a-enum}.
\textsc{All-MinAs($Q,\cK$)} is the set of all MinAs for query $Q$ in knowledge base $\cK$.

A \emph{tableau} is a graph where each node represents an
individual $a$ and is labeled with the set of concepts $\cL(a)$ it belongs to. Each
edge $\langle a, b\rangle$ in the graph is labeled with the set of roles to which the couple
$(a, b)$ belongs. Then, a set of  consistency preserving tableau
expansion rules are repeatedly applied until a clash (i.e., a contradiction) is detected or a clash-free
graph is found to which no more rules are applicable. A clash is for example a
couple $(C, a)$ where $C$ and $\neg C$ are present in the label of a node, i.e. ${C, \neg C} \subseteq \cL(a)$.

Some expansion rules are non-deterministic, i.e., they generate
a finite set of tableaux. Thus the algorithm keeps a set of tableaux that is
consistent if there is any tableau in it that is consistent, i.e., that is clash-free.
Each time a clash is detected in a tableau $G$, the algorithm stops applying rules
to $G$. Once every tableau in $T$ contains a clash or no more expansion rules
can be applied to it, the algorithm terminates. If all the tableaux in the final
set $T$ contain a clash, the algorithm returns unsatisfiable as no model can be
found. Otherwise, any one clash-free completion graph in $T$ represents a possible
model for the concept and the algorithm returns satisfiable.

To compute the probability of a query, the explanations must be made mutually exclusive, so
that the probability of each individual explanation is computed and summed
with the others. To do that we assign independent Boolean random variables to the axioms contained in the explanations and defining
the Disjunctive Normal Form (DNF) Boolean formula $f_K$ which models the set of explanations. Thus
$
f_K(\mathbf{X})=\bigvee_{\kappa\in K}\bigwedge_{(E_i,1)}X_{i}\bigwedge_{(E_i,0)}\overline{X_{i}}
$
where $\mathbf{X}=\{X_{i}|(E_i,k)\in\kappa,\kappa\in K\}$ is the set of Boolean random variables.
We can now translate $f_K$ to a Binary Decision Diagram (BDD), from which we can compute the probability of the query with a dynamic programming algorithm that is linear in the size of the BDD.

In \cite{DBLP:journals/jar/BaaderP10,DBLP:journals/logcom/BaaderP10} the authors consider the problem of finding a \emph{pinpointing formula} instead of
\textsc{All-MinAs($Q,\cK$)}.
The pinpointing formula is a monotone Boolean formula in which each Boolean variable corresponds to an axiom of the KB. This formula is built using
the variables and the conjunction and disjunction connectives. It compactly encodes the set of all MinAs.
Let's assume that each axiom $E$ of a KB $\cK$ is associated with a propositional variable, indicated with $var(E)$. The set of all propositional variables is indicated with $var(\cK )$.
A valuation $\nu$ of a monotone Boolean formula is the set of propositional variables that are true. For a valuation $\nu \subseteq var(\cK)$, let $\cK_{\nu} := \{t \in \cK |var(t)\in\nu\}$.
\begin{definition}[Pinpointing formula]
	Given a query $Q$ and a KB $\cK$, a monotone Boolean formula $\phi$ over $var(\cK)$ is called a \emph{pinpointing formula} for $Q$ if for every valuation $\nu \subseteq var(\cK)$ it holds that $\cK_{\nu} \models Q$ iff
	$\nu$ satisfies $\phi$.

\end{definition}
In Lemma 2.4 of \cite{DBLP:journals/logcom/BaaderP10} the authors proved that the set of all MinAs can be obtained by transforming the pinpointing formula into a Disjunctive Normal Form Boolean formula~(DNF) and removing disjuncts implying other disjuncts.

Irrespective of which representation of the explanations we choose, a DNF or a general pinpointing formula, we can apply knowledge compilation and \textit{transform it into a Binary Decision Diagram (BDD)},
from which we can compute the probability of the query with a dynamic programming algorithm that is linear in the size of the BDD.

We refer to \cite{Zese17-SSW-BK,ZesBelRig16-AMAI-IJ} for a detailed description of the two methods.