\section{On Douglas-Rachford splittting} % (fold)
\label{sec:on_douglas_rachford_splittting}

This a short recap of a post at regularize.\footnote
\footnotetext{\url{https://regularize.wordpress.com/2017/02/24/the-origin-of-the-douglas-rachford-iteration/}}

Consider a Hilbert space $(\mathcal{H}, \langle \cdot, \cdot\rangle)$. A
point-set map $T\colon \mathcal{H} \to \mathcal{P}(\mathcal{H})$ is a monotone
operator if for any $x, y \in \mathcal{H}$ and any $u\in Tx$ and $v\in Ty$
we have
$
\langle
    x-y,
    u-v
\rangle \geq 0
$.

A problem of the form: find $x\in \mathcal{H}$ such that $0 \in Tx$, --
is called a \textbf{monotone inclusion}. It is typically solved via splitting.

We first split an operator, then find the resolvents of the components, and
finally formulate a fixed point problem, that define an iterative algorithm.

\subsection{Splitting} % (fold)
\label{sub:splitting}

% subsection splitting (end)

% section on_douglas_rachford_splittting (end)


\input{../../assignment-header}

%========================================================================
\title{Assignment 4:  Introduction to Tracking Controllers}
\date{Assigned:  Feb 6  ---  Due:  Feb 15}
\author{Tufts ME 149:  Optimal Control For Robotics}
%========================================================================
\begin{document}
\maketitle
%=================================================

\section*{Part One: Cubic Spline Math}

Suppose that you are given a single segment of a cubic spline, $x(t)$, shown below.

\begin{equation}
x(t) = \sum_{i=0}^3 c_i t^i = c_0 + c_1 t + c_2 t^2 + c_3 t ^ 3
\end{equation}

\textbf{1a:} Compute the spline coefficients $c_i$ in terms of the boundary conditions shown below. \\

\vspace{-1em}

\begin{align}
x(0) &= x_0
  \quad \quad & \quad \quad
x(h) &= x_h
  \\
\dot{x}(0) &= v_0
  \quad \quad & \quad \quad
\dot{x}(h) &= v_h
\end{align}

\textbf{1b:} Compute the acceleration $\ddot{x}(t)$ at each boundary ($t=0$ and $t=h$)
in terms of the boundary conditions.\\

\textbf{1c:} Compute the value of the
``minimum-jerk'' objective function for this segment in terms of the boundary conditions.
Note that ``jerk'' is the technical term for the time-derivative of acceleration.

\begin{equation}
  J(x(t)) = \int_0^h \! \dddot{x}^2(\tau) \, d\tau
\end{equation}

\textbf{Note:}
You may perform these calculations either by hand or using the Matlab symbolic toolbox.
If you do the calculations by hand, then please
scan\footnote{There are several phone apps that will easily scan a document and save to pdf.}
your solution and upload it as a pdf.
If you choose to use the Matlab symbolic toolbox then upload your derivation script.
The script should print the solution to the command prompt;
please copy this output directly into the comments of the script before submitting.
Your file should be named either: \\
\texttt{hw\_04\_studentName\_splineMath.pdf} \\
\texttt{hw\_04\_studentName\_splineMath.m}

\section*{Part Two: Tracking Controller Design}

For this part of the assignment you will design a reference trajectory
and implement a tracking controller for two model systems:
a simple pendulum and a double pendulum.
In both cases you will construct a reference trajectory that performs a
swing-up maneuver, moving the pendulum from the state of minimum potential energy
to maximum potential energy.

\subsection*{kinematic reference trajectory design}

For both systems you will be given boundary conditions:
the state of the system at the initial and final time.
Your job is to construct a piece-wise polynomial (PP)
reference trajectory that satisfies those boundary conditions.
The easiest way to do this is to construct a
clamped cubic spline for the position (angle) reference using the
Matlab \texttt{spline} command (see the help file for how to specify boundary slopes).
You can then use the \texttt{ME149/codeLibrary/splines/ppDer} function in the code library
to construct the velocity (anglular rate) and
acceleration (angular acceleration) reference splines
by differentiating the position spline.

\par
Use at least two segments in your spline (minimum of three knot points).
Later in the class we will learn to select the position at the intermediate knot points
using trajectory optimization. For now, you can just select them by hand.

\par
Later in the assignment you will compute an objective function to evaluate the
integral of the feed-forward torque squared along the reference trajectory.
Experiment with your reference trajectory to see if you can modify the intermediate knot
points to reduce the overall trajectory cost.
The student who achieves the lowest overall objective function value (and has a correct solution)
will receive two bonus points on this homework assignment.
There will be one winner for the simple pendulum and another for the double pendulum.
The maximum score on the assignment will be capped at 50 points.

\subsection*{feed-forward torque calculation}

Both the simple and double pendulum models include
a function to copmute the inverse dynamics for the system.
You can call the inverse dynamics inside of your tracking controller to
compute a feed-forward torque term.

\subsection*{feed-back controller design}

The form of your controller will look something like:

\begin{equation}
  u_{ref} = \text{invDyn}\big(x_{ref}(t), \, \dot{x}_{ref}(t), \, \ddot{x}_{ref}(t) \big)
\end{equation}
\begin{equation}
  u = u_{ref} + K_p \cdot \big(x - x_{ref}(t)\big) + K_d \cdot \big(\dot{x} - \dot{x}_{ref}(t)\big)
\end{equation}

This type of controller is called an \textit{open-loop} trajectory tracking controller,
because the reference trajectory does not depend on the state of the system.
Notice that there is a still a feed-back term in the controller ---
it is used to stabilize the system about the current reference.

You may use any method to select the gains $K_p$ and $K_d$.
Please include (as comments in your code) a description of how you selected the gains.
Hand-tuning is an acceptable solution.
The function
\texttt{ME149/codeLibrary/control/secondOrderSystemGains}
in the code library might be useful, although you are not required to use it.

\subsection*{objective function calculation}

Compute the torque-squared objective function for you candidate reference trajectory,
defined below, where $T$ is the duration of the trajectory,
$\bm{z}(t)$ is the state trajectory (position and velocity reference), and
$\bm{u}(t)$ is the control trajectory (feed-forward torques).
Note that $n$ is the number of actuators;
$n=1$ for the simple pendulum and
$n=2$ for the double pendulum.

\begin{equation}
  J(\bm{z}(t), \bm{u}(t))
  = \int_0^T \! \norm{\bm{u}(\tau)}^2 \, d\tau
  = \sum_{i=0}^n \int_0^T \! u_i^2(\tau) \, d\tau
\end{equation}

% For this example, the objective function depends only on control,
% although in general it can depend on time, state, and control.

This objective function needs to be computed numerically.
Unlike the solution to the dynamics, the objective function is purely dependent on
time. This means that it can be computed using simple quadrature methods,
such as the trapezoid rule or Simpson's rule.
In Matlab, it can be computed using the \texttt{integral} command.

\par
Once we start doing trajectory optimization, it is important to compute the
path integral using the exact same numerical method as was used to evaluate
the system dynamics. There is an easy way to do this: just pass the integrand
of the objective function to your simulation function!

For this assignment you may use either technique to numerically evaluate the objective function.

\subsection*{simulation of the closed-loop system}

For the final part of the assignment you will run a simulation of your closed-loop system,
using the tracking controller to follow your reference trajectory.
You may use any simulation method that you like, such as those from your previous assignments,
the solutions in the code library, or Matlab functions such as \texttt{ode45}.

\par Introduce a small tracking error at the start of the trajectory.
This will make it easier to see if your tracking controller is effective.
Choose the size of the perturbation to be $0.1$ radians in each initial angle: \\
\texttt{q0 = ppval(qRef, 0) + 0.1;}
The initial rate should be zero, both for the simulation and the reference trajectory.

\subsection*{plots}

Create one figure for each system, with a set of plots that show the angle(s),
rate(s), and torque(s).
Make each figure full screen and then save as a pdf with the file name:\\
\texttt{hw\_04\_studentName\_simplePendulum.pdf} \\
\texttt{hw\_04\_studentName\_doublePendulum.pdf} \\

\par
The top row of plots should show the reference and measured (simulated) angles,
the middle row should show the reference and measured angular rates,
and the bottom row should show the feed-forward and measured torques.
There should be one column of subplots per joint.
Include axis labels, titles, and legends for each subplot.
The title for the torque plots should include the value of the objective function
for your reference trajectory.

\subsection*{simple pendulum model}

The code for the simple pendulum model is in the code library:
(\texttt{ME149/codeLibrary/simplePendulum/}).
For this assignment you should set the parameters:
 \texttt{param.freq = 1.0} and \texttt{param.damp = 0.0}.

\par The swing-up maneuver should take $T = 3$ seconds to complete.
The pendulum should start at rest in the stable equilibrium
and finish at rest in the unstable equilibrium.

\begin{align}
  &q(0) = 0   \quad & \quad   \dot{q}(0) = 0 \\
  &q(T) = \pi \quad & \quad \dot{q}(T) = 0
\end{align}

\subsection*{double pendulum model}

The code for the double pendulum model is in the code library:
(\texttt{ME149/codeLibrary/doublePendulum/}).
For this assignment you should set all of the parameters to one.

\par The swing-up maneuver should take $T = 5$ seconds to complete.
Both pendulums should start at rest in the stable equilibrium
and finish at rest in the unstable equilibrium.

\begin{align}
&q_1(0) = 0   \quad & \quad   q_2(0) = 0
\quad\quad\quad & \quad\quad\quad
\dot{q}_1(0) = 0   \quad & \quad   \dot{q}_2(0) = 0  \quad\\
&q_1(T) = \pi   \quad & \quad   q_2(T) = \pi
\quad\quad\quad & \quad\quad\quad
\dot{q}_1(T) = 0   \quad & \quad   \dot{q}_2(T) = 0 \quad
\end{align}

\section*{Deliverables}

Upload all of the Matlab code that you wrote for this assignment to Trunk,
along with the pdf files for each of the two figures and
your solution to the spline math problems.

\par
The entry-point file for your two simulations should be called
\texttt{hw\_04\_studentName.m}.
Create a header block in this file (see template)
that includes your name,
a list of collaborators and references,
how long the assignment took to complete, and
concludes with a brief outline of your code.

%=================================================
\end{document}


\subsection{Rotational Invariance}\label{ex:rotation}
This example shows that if $\qoiA$ is defined by a rotation of $\qoiB$, then the accuracy and convergence rates of $\PP^{(a)}_{\pspace, \ndiscs, \nsamps}$ are identical to $\PP^{(b)}_{\pspace, \ndiscs, \nsamps}$.
We expect this to be true since skewness is rotationally invariant, as we summarize in the following Proposition.
\begin{prop}
The quantity $S_\qoi(\param)$ is invariant under rotations performed on $\qoi$ for any $\param$. \\
\label{prop:rot_invariance}
\end{prop}
\begin{proof}
If we apply a rotation $\qoi$, then the Jacobians $J_{\qoi, \param}$ are also subject to the same rotation at each $\param$.
Since rotations are unitary operators, the norms given in Eq.~\eqref{eq:skewness} used to define skewness are unaffected.
\end{proof}

\begin{figure}
\begin{table}[H]
\begin{tabular}{ c | c | c | c }
\nsamps & $\qoiA$ & $\qoiB$ & $\qoiC$\\ \hline \hline
$200$ & $2.18E-01$ & $1.97E-01$ & $2.19E-01$\\ \hline

$400$ & $1.60E-01$ & $1.70E-01$ & $1.51E-01$\\ \hline

$800$ & $1.09E-01$ & $1.14E-01$ & $1.09E-01$\\ \hline

$1600$ & $7.43E-02$ & $7.76E-02$ & $7.53E-02$\\ \hline

$3200$ & $5.51E-02$ & $5.53E-02$ & $5.31E-02$\\ \hline

$6400$ & $4.19E-02$ & $4.09E-02$ & $4.19E-02$\\ \hline
\end{tabular}
\end{table}

\includegraphics[width=0.45\linewidth]{./images/Plot-orth-reg_BigN_40000_reg_M_1_rand_I_100000.png}

\caption{The results of $d^2_\text{TV}(\PP_{\pspace, \ndiscs, \nsamps}, \PP_{\pspace, \bar{\nsamps}})$ for three maps generated by random rotations of orthogonal linear maps.}
\label{fig:M1orth}
\end{figure}

To demonstrate this Lemma numerically, we define the space of QoI maps $\qspace = \set{ \qoiA , \qoiB, \qoiC }$, where all three are linear maps with the same local skewness $S_\qoi (\param) = 1 \; \forall \param \in \pspace$.
The map $\qoiA$ is the identity and the other two, $\qoiB$ and $\qoiC$ are rotations of $\qoiA$ by randomly chosen angles.
Following the algorithmic outline above, we perform a convergence study to $\PP_{\pspace,\bar{\nsamps}}$ with results summarized in Figure~\ref{fig:M1orth}.
The convergence rates and expected errors in the SIPs associated with each of these maps are virtually indistinguishable.
In light of Proposition~\ref{prop:rot_invariance} and these numerical results, we conclude that the accuracy of the numerical solution to the SIP is invariant under rotations to the QoI map.

\FloatBarrier


\documentclass[12pt]{article}
\usepackage{graphicx}
\usepackage{amsmath}
\providecommand{\e}[1]{\ensuremath{\times 10^{#1}}}

\setlength{\oddsidemargin}{.1in}
\setlength{\textwidth}{6.3in}
\setlength{\textheight}{8.9in}
\setlength{\topmargin}{-.5in}

\begin{document}

\section{Gamma distribution as the conjugate prior for a Poisson likelihood}
\label{a:conjugate_derivation}
Here we show that the gamma distribution is the conjugate prior for a Poisson likelihood used in Bayes' theorem; that is, choosing a gamma-distributed prior also yields a gamma-distributed posterior if the likelihood is a Poisson distribution.

Assume a series of observations $\{n_1, n_2, \ldots, n_M\}$ are independent and identically distributed from a Poisson distribution $f_{Poisson(\lambda)}(n)$. This gives a Bayesian likelihood term

\begin{align}
P(\{n_m\}\,|\,\lambda) &= \prod_{m = 1}^{M} P(n_m\,|\,\lambda) \\[5pt]
&= \prod_{m = 1}^{M} \frac{e^{-\lambda} \lambda^{n_m}}{n_m!} \\[5pt]
&= \frac{e^{-M\lambda} \lambda^{\sum_{m = 1}^{M} n_m}}{\prod_{m = 1}^{M} n_m!}.
\end{align}

We choose the prior to be a gamma distribution over $\lambda$ with shape and rate parameters $\alpha, \beta$:

\begin{equation}
P(\lambda) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} \lambda^{\alpha - 1} e^{-\beta \lambda}.
\end{equation}

Bayes' theorem gives the posterior

\begin{align}
P(\lambda\,|\,\{n_m\}) &= \frac{P(\{n_m\}\,|\,\lambda) P(\lambda)}{\int_{\lambda' = 0}^{\infty} P(\{n_m\}\,|\,\lambda') P(\lambda') \, d\lambda'} \\[5pt]
&= \frac{\frac{\beta^{\alpha}}{\Gamma(\alpha)} \lambda^{\sum_{m = 1}^{M} n_m + \alpha - 1} e^{-(M + \beta) \lambda}}{\int_{\lambda' = 0}^{\infty} \frac{\beta^{\alpha}}{\Gamma(\alpha)} \lambda'^{\sum_{m = 1}^{M} n_m + \alpha - 1} e^{-(M + \beta) \lambda'} \, d\lambda'}.
\label{e:eq32}
\end{align}

The denominator of the above expression is an integral of the form

\begin{equation}
\label{e:eq33}
c \int x^a e^{bx} \, dx
\end{equation}

where $c = \frac{\beta^{\alpha}}{\Gamma(\alpha)}$, $a = \sum_{m = 1}^{M} n_m + \alpha - 1$, $b = -(M + \beta)$, and $x = \lambda'$. This integral can be treated by iteratively applying integration by parts:

\begin{align}
\int x^a e^{bx} \, dx &= \frac{1}{b} x^a e^{bx} - \frac{a}{b} x^{a - 1} e^{bx} \, dx \\[5pt]
&= \frac{1}{b} x^a e^{bx} - \frac{a}{b}(\frac{1}{b} x^{a - 1} e^{bx} - \frac{a - 1}{b}\int x^{a - 2} e^{bx} \, dx) \\[5pt]
&= \frac{1}{b} x^a e^{bx} - \frac{a}{b}\Big[\frac{1}{b} x^{a - 1} e^{bx} - \frac{a - 1}{b}(\frac{1}{b} x^{a - 2} e^{bx} \int x^{a - 3} e^{bx} \, dx)\Big] \\[5pt]
&\ldots.  \nonumber
\end{align}

Rewriting and combining terms, we see that the process of integration by parts proceeds indefinitely like so:

\begin{align}
\int x^a e^{bx} \, dx &= \frac{1}{b} x^a e^{bx} - \frac{a}{b^2} x^{a - 1} e^{bx} + \frac{a(a - 1)}{b^3} x^{a - 2} e^{bx} - \frac{a(a - 1)(a - 2)}{b^4} x^{a - 3} e^{bx} + \ldots \nonumber \\[5pt]
&+ \frac{a(a - 1) \ldots (a - a)}{b^{a + 2}} e^{bx} + \ldots.
\end{align}

Note that the last term written above is in fact zero, and all terms after it vanish as well. So we can rewrite our integral as the sum

\begin{equation}
\int x^a e^{bx} \, dx = \sum_{n = 0}^{a} (-1)^n \frac{a!}{b^{n + 1} (a - n)!} x^{a - n} e^{bx}.
\end{equation}

Substituting our original symbols for the placeholders a, b, c, and x into expression (\ref{e:eq33}) and evaluating the integral on the limits 0 to $\infty$ gives

\begin{align}
\label{e:eq39}
&\frac{\beta^{\alpha}}{\Gamma(\alpha)} \int_{\lambda' = 0}^{\infty} \lambda'^{\sum_{m = 1}^{M} n_m + \alpha - 1} e^{-(M + \beta) \lambda'} \, d\lambda' \nonumber \\[5pt]
= &\frac{\beta^{\alpha}}{\Gamma(\alpha)} \sum_{n = 0}^{N + \alpha - 1} (-1)^n \frac{(N + \alpha - 1)!}{\Big[-(M + \beta)\Big]^{n + 1} (N + \alpha - 1 - n)!} \lambda'^{N + \alpha - 1 - n} e^{-(M + \beta)\lambda'}\Big|_{\lambda' = 0}^{\infty},
\end{align}

where we have let $N = \sum_{m = 1}^{M} n_m$. To evaluate the right hand side of the above equation, we can first note that the lower bound of the limit vanishes at $\lambda' = 0$. We can compute the upper bound by noting that the final two terms after the fraction bar are of the form

\begin{equation}
x^a e^{-bx},
\end{equation}

where $a = N + \alpha - 1 - n$, $b = M + \beta$, and $x = \lambda'$. Its limit as $x \to \infty$ can be evaluated using the Taylor series expansion of the exponential function:

\begin{align}
\lim_{x \to \infty} x^a e^{-bx} &= \lim_{x \to \infty} \frac{x^a}{e^{bx}} \\[5pt]
&= \lim_{x \to \infty} \frac{x^a}{\sum_{n = 0}^{\infty} \frac{b^n x^n}{n!}} \\[5pt]
&= \lim_{x \to \infty} \frac{x^a}{\frac{b^x x^x}{x!}} \\[5pt]
&= \lim_{x \to \infty} \frac{x^a x!}{(bx)^x}.
\end{align}

Since the $x^x$ term in the denominator increases more rapidly than the x! term in the numerator,

\begin{equation}
\lim_{x \to \infty} x^a e^{-bx} = 0.
\end{equation}

Thus it appears at first that the entire sum $\sum_{n = 0}^{N + \alpha - 1}$ in equation (\ref{e:eq39}) collapses to zero! This is not the case, however: the single term in the sum where $N + \alpha - 1 - n = 0$ leaves $\lambda'^{N + \alpha - 1 - n} e^{-(M + \beta)\lambda'}\Big|_{\lambda' = 0}^{\infty}$ undefined. We see that at the lower bound, the $\lambda'$ term is of the form

\begin{align}
\lim_{\lambda' \to 0} \lambda'^0 &= \lim_{\lambda' \to 0} 1 \\[5pt]
&= 1,
\end{align}

so

\begin{align}
\lambda'^{N + \alpha - 1 - n} e^{-(M + \beta)\lambda'}\Big|_{\lambda' = 0}^{\infty} &= e^{-(M + \beta)\lambda'}\Big|_{\lambda' = 0}^{\infty} \\[5pt]
&= -1.
\end{align}

We now see that the sum in the right hand side of equation (\ref{e:eq39}) is nonzero for the single term where $n = N + \alpha - 1$, and that this term is equal to

\begin{align}
\frac{\beta^{\alpha}}{\Gamma(\alpha)} (-1)^{N + \alpha - 1} \frac{(N + \alpha - 1)!}{(-1)^{N + \alpha} (M + \beta)^{N + \alpha}}(-1).
\end{align}

Noting that all (-1) terms in the above expression cancel with one another, we find that both sides of equation (\ref{e:eq39}), and thus also the denominator in equation (\ref{e:eq32}), are equivalent to

\begin{equation}
\label{e:eq51}
\frac{\beta^{\alpha}}{(M + \beta)^{N + \alpha}}\frac{\Gamma(N + \alpha)}{\Gamma(\alpha)}.
\end{equation}

Substituting this back into equation (\ref{e:eq32}) gives

\begin{align}
P(\lambda\,|\,\{n_m\}) &= \frac{\frac{\beta^{\alpha}}{\Gamma(\alpha)} \lambda^{N + \alpha - 1} e^{-(M + \beta) \lambda}}{\frac{\beta^{\alpha}}{\Gamma(\alpha)}\frac{\Gamma(N + \alpha)}{(M + \beta)^{N + \alpha}}} \\[5pt]
&= \frac{(M + \beta)^{N + \alpha}}{\Gamma(N + \alpha)} \lambda^{N + \alpha - 1} e^{-(M + \beta) \lambda} \\[5pt]
&= f_{gamma(N + \alpha,M + \beta)}(\lambda).
\end{align}

Altogether, then, we see that for a series of sequential observations $\{n_1, n_2, \ldots, n_M\}$, Bayes' theorem

\begin{equation}
P(\lambda\,|\,\{n_m\}) = \frac{P(\{n_m\}\,|\,\lambda) P(\lambda)}{\int_{\lambda' = 0}^{\infty} P(\{n_m\}\,|\,\lambda') P(\lambda') \, d\lambda'}
\end{equation}

is equivalent to

\begin{equation}
f_{gamma(\sum_{m = 1}^{M} n_m + \alpha,M + \beta)}(\lambda) = \frac{\Big[\prod_{m = 1}^{M} f_{Poisson(\lambda)}(n_m)\Big] f_{gamma(\alpha,\beta)}(\lambda)}{\int_{\lambda' = 0}^{\infty} \Big[\prod_{m = 1}^{M} f_{Poisson(\lambda')}(n_m)\Big] f_{gamma(\alpha,\beta)}(\lambda') \, d\lambda'}
\end{equation}

for a Poisson likelihood and a gamma-distributed prior. Note that this gives a posterior distribution of the same type as the prior distribution. Therefore, the conjugate prior to a Poisson likelihood is a gamma distribution.

\section{Mode of a gamma distribution}
\label{a:mode_derivation}
Here we derive the expression for the mode of a gamma distribution as a function of its parameters.

A gamma distribution over $\lambda$ with shape and rate parameters $\alpha,\beta$ is given by the probability density function

\begin{equation}
f_{gamma(\alpha,\beta)}(\lambda) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} \lambda^{\alpha - 1} e^{-\beta\lambda}.
\end{equation}

To find the maximum we take the partial derivative with respect to $\lambda$ and set it equal to zero:

\begin{align}
\frac{\partial f_{gamma(\alpha,\beta)}(\lambda)}{\partial \lambda} &= \frac{\partial}{\partial \lambda} \Big(\frac{\beta^{\alpha}}{\Gamma(\alpha)} \lambda^{\alpha - 1} e^{-\beta\lambda}\Big) \\[5pt]
&= \frac{\beta^{\alpha}}{\Gamma(\alpha)} \Big[(\alpha - 1)\lambda^{\alpha - 2} e^{-\beta\lambda} - \beta\lambda^{\alpha - 1} e^{-\beta\lambda}\Big] \\[5pt]
&= 0.
\end{align}

Since $\frac{\beta^{\alpha}}{\Gamma(\alpha)}$ is assumed to be nonzero, this relation is satisfied when

\begin{equation}
(\alpha - 1)\lambda^{\alpha - 2} e^{-\beta\lambda} - \beta\lambda^{\alpha - 1} e^{-\beta\lambda} = 0;
\end{equation}

that is, when

\begin{align}
(\alpha - 1)\lambda^{\alpha - 2} e^{-\beta\lambda} &= \beta\lambda^{\alpha - 1} e^{-\beta\lambda} \\[5pt]
(\alpha - 1) &= \beta\lambda
\end{align}

or when

\begin{equation}
\frac{\alpha - 1}{\beta} = \lambda.
\end{equation}

So we take $\hat\lambda = \frac{\alpha - 1}{\beta}$ to be the mode of a gamma distribution over $\lambda$.

\section{Negative binomial distribution as a gamma-Poisson mixture}
\label{a:mixture_derivation}
Here we show that a continuous mixture of Poisson distributions where the mixing distribution is a gamma distribution yields a negative binomial distribution.

Consider the continuous mixture for positive $\lambda$ of a Poisson distribution over $n$ with mean rate $\lambda$ and a gamma distribution over $\lambda$ with shape and rate parameters $\alpha,\beta$:

\begin{align}
\int_{\lambda = 0}^{\infty} f_{Poisson(\lambda)}(n) f_{gamma(\alpha,\beta)}(\lambda) \, d\lambda &= \int_{\lambda = 0}^{\infty} \frac{\lambda^n e^{-\lambda}}{n!} \frac{\beta^{\alpha}}{\Gamma(\alpha)} \lambda^{\alpha - 1} e^{-\beta\lambda} \, d\lambda \\[5pt]
&= \frac{\beta^{\alpha}}{n!\,\Gamma(\alpha)} \int_{\lambda = 0}^{\infty} \lambda^{\alpha + n - 1} e^{-(1 + \beta)\lambda} \, d\lambda.
\label{e:eq58}
\end{align}

The right hand side of the above equation contains an integral of the form dealt with in equations (\ref{e:eq33}-\ref{e:eq51}). Using the same approach as before, we find that

\begin{equation}
\int_{\lambda = 0}^{\infty} \lambda^{\alpha + n - 1} e^{-(1 + \beta)\lambda} \, d\lambda = (n + \alpha - 1)!\,(1 + \beta)^{-(\alpha + n)}.
\end{equation}

Substituting this into the right hand side of equation (\ref{e:eq58}) and performing further manipulations gives

\begin{align}
\frac{\beta^{\alpha}}{n!\,\Gamma(\alpha)} (n + \alpha - 1)!\,(1 + \beta)^{-(\alpha + n)} &= \frac{\Gamma(\alpha + n)}{n!\,\Gamma(\alpha)} \frac{\beta^{\alpha + n}}{(1 + \beta)^{\alpha + n} \beta^n} \\[5pt]
&= \frac{\Gamma(\alpha + n)}{n!\,\Gamma(\alpha)} \Big(\frac{\beta}{1 + \beta}\Big)^{\alpha + n} \Big(\frac{1}{\beta}\Big)^n.
\label{e:eq61}
\end{align}

Isolating and performing manipulations on the two rightmost terms of the above equation gives

\begin{align}
\Big(\frac{\beta}{1 + \beta}\Big)^{\alpha + n} \Big(\frac{1}{\beta}\Big)^n &= \Big(\frac{\beta}{1 + \beta}\Big)^{\alpha} \Big(\frac{\beta}{1 + \beta}\Big)^n \Big(\frac{1}{\beta}\Big)^n \\[5pt]
&= \Big(\frac{\beta}{1 + \beta}\Big)^{\alpha} \Big(\frac{1}{1 + \beta}\Big)^n \\[5pt]
&= \Big(\frac{1}{1 + \beta}\Big)^n \Big(1 - \frac{1}{1 + \beta}\Big)^{\alpha},
\end{align}

once it is seen that $\frac{\beta}{1 + \beta} = \frac{1 + \beta - 1}{1 + \beta} = 1 - \frac{1}{1 - \beta}$.

Substituting this back into the right hand side of equation (\ref{e:eq61}) and reducing further, we have

\begin{align}
\frac{\Gamma(\alpha + n)}{n!\,\Gamma(\alpha)} \Big(\frac{1}{1 + \beta}\Big)^n \Big(1 - \frac{1}{1 + \beta}\Big)^{\alpha} &= \frac{(n + \alpha - 1)!}{n!\,(\alpha - 1)!} \Big(\frac{1}{1 + \beta}\Big)^n \Big(1 - \frac{1}{1 + \beta}\Big)^{\alpha} \\[5pt]
&= \binom{n + \alpha - 1}{n} \Big(\frac{1}{1 + \beta}\Big)^n \Big(1 - \frac{1}{1 + \beta}\Big)^{\alpha} \\[5pt]
&= f_{NB(\alpha,\frac{1}{1 + \beta})}(n).
\end{align}

The result is a negative binomial distribution with parameters $\alpha,\frac{1}{1 + \beta}$. So, altogether we can write

\begin{equation}
\int_{\lambda = 0}^{\infty} f_{Poisson(\lambda)}(n) f_{gamma(\alpha,\beta)}(\lambda) \, d\lambda = f_{NB(\alpha,\frac{1}{1 + \beta})}(n).
\end{equation}

Therefore, a continuous mixture of Poisson distributions where the mixing distribution is a gamma distribution yields a negative binomial distribution.

\end{document}


\documentclass{ximera}
\input{../preamble}
\title{Exercises: Surface Area}
%%%%%\author{Philip T. Gressman}

\begin{document}
\begin{abstract}
Various exercises related to the computation of areas of surfaces of revolution.
\end{abstract}
\maketitle

\begin{exercise}%[APEX0704ARCL29]
Find the surface area of the solid formed by revolving \(y=2x\) on \([0,1]\) about the \(x\)-axis.
\[ A = \answer{2\pi\sqrt{5}}. \]
%
%
\end{exercise}

\begin{exercise}%[APEX0704ARCL30]
Find the surface area of the solid formed by revolving \(y=x^2\) on \([0,1]\) about the \(y\)-axis.
\begin{hint}
To compute the integral, you will need to make a substitution like $u = 1 + 4x^2$ or something similar.
\end{hint}
\[ A = \answer{\frac{(5 \sqrt{5} - 1) \pi}{6}}. \]
%\(2\pi\int_0^1 x\sqrt{1+4x^2}\ dx = \pi/6(5\sqrt{5}-1)\)
%
%
\end{exercise}

\begin{exercise}%[APEX0704ARCL31]
Find the surface area of the solid formed by revolving \(y=x^3\) on \([0,1]\) about the \(x\)-axis.
\begin{hint}
To compute the integral, you will need to make a substitution like $u = 1 + 9 x^4$ or something similar.
\end{hint}
\[ L = \answer{\frac{(10 \sqrt{10} - 1) \pi}{27}}. \]
%\(2\pi\int_0^1 x^3\sqrt{1+9x^4}\ dx = \pi/27(10\sqrt{10}-1)\)
%
%
\end{exercise}

\section*{Sample Exam Questions}

\begin{question}%%%%%[2016.36]
Give an integral formula for the area of the surface generated by revolving the curve $y = \ln x$ between $x=1$ and $x=2$ about the $y$-axis. Explain your answer. You do not need to evaluate the integral.
\begin{multiplechoice}
\choice[correct]{\(\displaystyle \int_1^2 2 \pi \sqrt{x^2+1} ~ dx\)}
\choice{\(\displaystyle \int_1^2 2 \pi (\ln x) \frac{\sqrt{x^2+1}}{x} ~ dx\)}
\choice{\(\displaystyle \int_1^2 \frac{2 \pi}{x} \sqrt{1 + (\ln x)^2} ~ dx\)}
\choice{\(\displaystyle \int_1^2 \frac{1}{2 \pi \sqrt{x^2+1}} ~ dx\)}
\choice{\(\displaystyle \int_1^2 2 \pi (\ln x) \frac{x}{\sqrt{x^2+1}} ~ dx\)}
\choice{\(\displaystyle \int_1^2 \frac{2 \pi x}{\sqrt{1 + (\ln x)^2}} ~ dx\)}
\end{multiplechoice}
\end{question}

\begin{question}
 The curve $y = \frac{x^2}{8}$ between $x=0$ and $x = 3$ is revolved around the $y$-axis. Compute the surface area of the resulting surface.
\begin{multipleChoice}
\choice{$\displaystyle \frac{31 \pi}{6}$}
\choice{$\displaystyle \frac{41 \pi}{6}$}
\choice[correct]{$\displaystyle \frac{61 \pi}{6}$}
\choice{$\displaystyle \frac{71 \pi}{6}$}
\choice{$\displaystyle \frac{91 \pi}{6}$}
\choice{none of the above}
\end{multipleChoice}
\end{question}

\end{document}


\newpage\section{Binomial Identities}

\theo{https://en.wikipedia.org/wiki/Vandermonde's_identity}{Vandermonde's identity}{
    For every positive integer $m, n, k$
    \begin{center}
        \tcboxmath[arc=0mm, colback=black!5!white, boxrule=0mm]{
            \binom{m+n}{k} = \sum_{i=0}^{k} \binom{m}{i}\binom{n}{k-i}
        }
    \end{center}
    \begin{center}
        \tcboxmath[arc=0mm, colback=black!5!white, boxrule=0mm]{
            {2n \choose n} = {n \choose 0}^2+ {n \choose 1}^2 \dots + {n \choose n}^2
        }
    \end{center}
}

\prob{https://artofproblemsolving.com/community/c6h1061914p4600670}{Somewhere}{M}{
    Let $n$ and $m$ be positive integers with $n<m$. Prove that
    \begin{center}
        \tcboxmath[arc=0mm, colback=black!5!white, boxrule=0mm]{
        \sum_{k=0}^n (-1)^{n-k}\frac{1}{m-k}\binom{n}{k}=\frac{1}{(n+1)\binom{m}{n+1}}
        }
    \end{center}
}

\solu{[MellowMelon]
    The left side is the $n$th finite difference of the unique $n$th degree
    polynomial satisfying $P(x) = \frac{1}{m-x}$ for $x = 0,1, \ldots, n$.
    This means that if $a_n$ is the $x^n$ coefficient of $P$, then the left
    side is equal to $n! \cdot a_n$. Setting this equal to the right side, it
    suffices to show that
    \[ a_n = \frac{1}{(n+1)! \binom{m}{n+1}} = \frac{1}{m(m-1)\cdots (m-n)}. \]

    Let $Q(x) = (x-m)P(x) + 1$. Then $Q$ is degree $n+1$, has roots
    $0,1,\ldots,n$, and has leading coefficient $a_n$. So
    \[ Q(x) = a_nx(x-1)(x-2)\cdots (x-n). \]
    We also know $Q(m) = 1$. Plugging $x = m$ into the above equation and
    isolating $a_n$ gives us exactly what we need to show.
}

\rem{
    The idea to express the left hand side with finite differences is pretty
    simple. The crux idea is to finding a $Q$ that works. Since we have to
    show $a_n\left(m\right)\left(m-1\right) \dots \left(m-n\right) =1$, which
    looks like a polynomial with leading coefficient $a_n$ with roots $0, 1, 2,
    \dots n$ and $Q(m) = 1$. How can we get that?

    The takeaway is, if the problem requires you to find a polynomial that
    almost looks like the one given in the problem, try to modify the given
    polynomial in some way first.

    In this particular case was to modify $P$ bu using the fact $(x-m)P(x) =
    -1$.
}

\solu{[Hydroxide]
    Let $P$ be the unique polynomial of degree at most $n$ such that
    $P(0)=P(1)=\cdots=P(n)=1$. Obviously $P(x)=1$, but we can also find $P$
    from Lagrange interpolation. This gives us the equality of polynomials

    \[ \sum_{k=0}^n \frac{(x-0)\cdots (x-(k-1))(x-(k+1))\cdots
    (x-n)}{(k-0)\cdots (k-(k-1))(k-(k+1))\cdots (k-n)}=1. \]

    Plugging in $x=m$ and doing a ton of rearranging gives the desired result.
}

\rem{
    Both solutions can easily be extended to prove the more general result
    that if $0\le p<n<m$ are integers, then
    \begin{center}
        \tcboxmath[arc=0mm, colback=black!5!white, boxrule=0mm]{
    \sum_{k=0}^n (-1)^{n-k}\frac{1}{m-k} \binom{n}{k}k^p=\frac{m^p}{(n+1)\binom{m}{n+1}}
        }
    \end{center}
}

\prob{https://artofproblemsolving.com/community/c6h358774}{USA TST 2010
    P8}{M, 10/10}{
    Let $m,n$ be positive integers with $m \geq n$, and let $S$ be the
    set of all $n$-term sequences of positive integers $(a_1, a_2, \ldots
    a_n)$ such that $a_1 + a_2 + \cdots + a_n = m$. Show that
    \begin{center}
        \tcboxmath[arc=0mm, colback=black!5!white, boxrule=0mm]{
            \sum_S 1^{a_1} 2^{a_2} \cdots n^{a_n} = {n \choose n} n^m -
            {n \choose n-1} (n-1)^m + \cdots + (-1)^{n-2} {n \choose 2} 2^m +
            (-1)^{n-1} {n \choose 1}
        }
    \end{center}
}

\solu{[Combinatorial, MellowMelon]
    Look at the right side, try to translate it to
    combinatorial model. If we had used the inclusion-exclusion method, the
    right side would be the number of ways to color $ m $ balls with
    $ n $ colors, with each color appearing at least once. Now our remaining
    job is to prove the same for the left side.\\

    Trying to interpret the left hand side, we first notice that we need to
    partition the balls in $n$ parts with sizes $a_1, a_2, \dots  a_n$ where
    the first partition will have $1$ color to choose from, the second one
    will have $2$ and so on. We need to refine this idea for using it.
}

\solu{[Generating Function, MellowMelon]
    The first step of a gf solution is to decide on a gf solution. Now we can
    think of the left hand side as a function of $m$, namely $b_m$. So we can
    make a generating function with it.\\

    We let $F_n(x) = b_0 + b_1 x + b_2 x^2 + \cdots$ and so we get:
    \begin{align*}
        F_n(x) &= (x+x^2+\cdots)(2x+4x^2+\cdots)\cdots (nx+n^2x^2+\cdots), \\
               &= \frac{x}{1-x} \frac{2x}{1-2x} \cdots \frac{nx}{1-nx}, \\
               &= \frac{n! x^n}{(1-x)(1-2x) \cdots (1-nx)}. \\
    \end{align*}

    Now we need a generating function for the right hand side as well. The
    right hand side looks the $n$th finite difference of the function
    $_mg_0(x) = x^m$, that is
    \[_mg_n(0) = {n \choose n} n^m - {n \choose n-1} (n-1)^m + \cdots +
    (-1)^{n-2} {n \choose 2} 2^m + (-1)^{n-1} {n \choose1}\]

    Now we can make a generating function with this:
    \[G(y) =\ _0g_n(0)y^0 +\ _1g_n(0)y^1 \dots \]
    But this isn't very useful as we are getting the $n$th finite differences
    on the right side, where we want to work with the original polynomials.\\

    So what we do is, we unravel the $_mg_n(0)$ as original polynomials and
    get:
    \[G(x, y) = x^0y^0 + x^1y^1 + x^2+y^2\dots \]
    Where our desired generating function for $n$ is:
    \[G_n(0, y) =\ _0g_n(0)y^0 +\ _1g_n(0)y^1 +\dots \]

    And we want to show that
    \[G_n(0, y) = \frac{n! x^n}{(1-x)(1-2x) \cdots (1-nx)} \]

    Now we notice that we can actually use induction on $n$ since we definded
    \[G_{n+1}(x, y) = G_n(x+1, y) - G_n(x, y)\]

    And so we are done.
}

\lem{}{
    \[G(x)=\sum_{n=0}^{\infty} \binom{n+k}{k} x^n=\frac{1}{(1-x)^{k+1}}\]
}

\begin{prooof}
    We have the above relation because ${n+k \choose k}$ can be interpret as
    the number of ways write $n$ as the sum of $k+1$ numbers. And the
    generating function for that is:
    \[G(x) = (1+x+x^2\dots )^{k+1}\]
\end{prooof}

\prob{https://artofproblemsolving.com/community/c6h1076934p5964577}{Binom1}{}{
    Solve for positive integer $n$
    \begin{center}
        \tcboxmath[arc=0mm, colback=black!5!white, boxrule=0mm]{
            {2n \choose 0}-{2n-1 \choose 1}+{2n-2 \choose 2}- \dots  +(-1)^n
            {n \choose n}=?
        }
    \end{center}
}

\solu{[Generating funtion, Tintarn]
    Denote this sum $a_n$. Denote $F(x)=\sum_{n=0}^{\infty} a_nx^{2n}$.
    \begin{align*}
        F(x)&=\sum_{k=0}^{\infty} (-1)^k \sum_{n=0}^{\infty} \binom{2n-k}{k}
        x^{2n}\\
        &=\sum_{k=0}^{\infty} (-x^2)^k \sum_{n=0}^{\infty} \binom{2n+k}{k}
        x^{2n}
    .\end{align*}
    Now, using the well-known power series
    \[\boxed{G(x)=\sum_{n=0}^{\infty} \binom{n+k}{k}
    x^n=\frac{1}{(1-x)^{k+1}}}\]
    we find
    \[\sum_{n=0}^{\infty} \binom{2n+k}{k} x^{2n}=\frac{G(x)+G(-x)}{2}\]
    and hence
    \begin{align*}
        F(x)&=\frac{1}{2(1-x)} \sum_{k=0}^{\infty} \left( \frac{x^2}{x-1} \right)^k
        +\frac{1}{2(1+x)} \sum_{k=0}^{\infty} \left( \frac{-x^2}{x+1} \right)^k\\[.5em]
            &=\frac{1}{2(x^2-x+1)}+\frac{1}{2(x^2+x+1)}\\[.5em]
            &=\frac{x^2+1}{x^4+x^2+1}\\[.5em]
            &=\frac{1-x^4}{1-x^6}\\
            &=(1-x^4)(1+x^6+x^{12}+x^{18}+\dotsc)\\
            &=1-x^4+x^6-x^{10}+x^{12}-x^{16}+\dots
    .\end{align*}

    \vspace{1em}
    and hence $\boxed{a_{3k}=1, a_{3k+1}=0, a_{3k+2}=-1}$.
}

\solu{[Combinaorial Model, MellowMelon]
    Consider the number of ways to tile a $1 \times 2n$ rectangle with squares
    and dominoes. Let $E$ be the set of ways to do it with an even number of
    dominoes, and let $O$ be the same for odd. We want to find $|E| - |O|$.\\

    Trying some basic cases, we see that the only values we get are $-1, 0,
    1$. So we want to find a pairing between the two sets that is almost a
    bijection, but doesn't work for only one element. And that bijection is:\\

    \textit{Consider the first consecutive pair of squares or domino which
    either comes at the very start or comes right after another domino. Swap
    the pair of squares to a domino or vice versa.}
}

\prob{https://artofproblemsolving.com/community/c6h1076934p5964577}{Binom 2}{}{
    \[{n \choose 0}^2-{n \choose 1}^2+{n \choose 2}^2-\dots +(-1)^n{n \choose n}^2=?\]
}

\solu{[Generating function, TinTarn]
    Denote \[f(m,n)=\sum_{k=0}^{\infty} (-1)^k \binom{n}{k} \binom{n}{m-k}\]
    We are looking for $f(n,n)$.  We have
    \begin{align*}
        \sum_{m=0}^{\infty} f(m,n) x^m &=\sum_{k=0}^{\infty} (-1)^k \binom{n}{k}
        \sum_{m=0}^{\infty} \binom{n}{m-k} x^m\\
        &=\left[\sum_{k=0}^{\infty} (-x)^k \binom{n}{k} \right]
        \left[\sum_{m=0}^{\infty} x^m \binom{n}{m} \right]\\
        &=(1-x)^n(1+x)^n\\
        &=(1-x^2)^n\\
        &=\sum_{t=0}^{\infty} (-1)^t \binom{n}{t} x^{2t}
    .\end{align*}

    Hence we see that for odd $m$ we have $f(m,n)=0$ and hence for odd $n$ we have
    $\boxed{f(n,n)=0}$.\\

    Now, for $n=2k$ even, we find $\boxed{f(2k,2k)=(-1)^k \binom{2k}{k}}$.
}

\solu{[Combinatorial, MellowMelon]
    Consider the number of ways to paint the squares of a $2 \times n$
    rectangle red and blue such that both rows have the same number of red
    squares. Let $E$ be the set of ways with an even number of red squares in each
    row, and let $O$ be the same for odd. We want to find $|E| - |O|$.\\

    \textit{Pair the selections as follows: take the first column with both
        squares the same color, and flip both colors. This maps elements in
        $E$ to $O$ and vice versa and is invertible, so except cases where the
    pairing is undefined, $E$ and $O$ have the same number of elements.}
}


\documentclass{ximera}
\input{../preamble}
\title{Exercises: Integration by Parts}
%%%%%\author{Philip T. Gressman}

\begin{document}
\begin{abstract}
Various exercises relating to integration by parts.
\end{abstract}
\maketitle

Compute the indefinite integrals below. Since there are many possible answers (which differ by constant values), use the given instructions if needed to choose which possible answer to use.

\begin{exercise}%[APEX0602IBP05]
\[  \int x\sin x\ dx = \answer{\sin x - x\cos x} +C\]
(Add a constant to your answer if needed so that it equals $0$ at $x = 0$.)
%
%
\end{exercise}

\begin{exercise}%[APEX0602IBP06]
\[ \int xe^{-x}\ dx = \answer{-e^{-x}-xe^{-x}}+C\]
(Add a constant to your answer if needed so that it equals $-1$ at $x = 0$.)
%
%
\end{exercise}

\begin{exercise}%[APEX0602IBP10]
\[  \int x^3e^{x}\ dx\ = \answer{x^3e^x-3x^2e^x+6xe^x-6e^x}+C\]
(Write your answer so that it has no constant term.)
%
%
\end{exercise}

\begin{exercise}%[APEX0602IBP25]
\[ \int x^2\ln |x| \ dx = \answer{\frac{1}{3} x^3 \ln |x|-\frac{x^3}{9}}+C\]
Don't forget absolute values in your logarithm.
(Add a constant to your answer as necessary so that it equals $-1/9$ at $x = 1$.)
%
%
\end{exercise}

\begin{exercise}%[APEX0602IBP12]
\[ \int e^x\sin x\ dx = \answer{\frac{1}{2} e^x(\sin x-\cos x)}+C\]
(Add a constant to your answer if needed so that it equals $-1/2$ at $x = 0$.)
\begin{hint}
This is a case in which you need to treat the integration by parts as an equation and \textit{solve} for the answer.
\end{hint}
%
%
\end{exercise}

\begin{exercise}%[APEX0602IBP17]
\[ \int \arcsin x\ dx = \answer{\sqrt{1-x^2}+x \arcsin(x)}+C \]
(Add a constant to your answer if needed so that it equals $1$ at $x = 0$.)
\begin{hint}
Write $\arcsin x = 1 \cdot \arcsin x$.
\end{hint}
%
%
\end{exercise}

\section*{Sample Quiz Questions}
\begin{question}%%%%%[IBPibp:explin1413]

Compute the definite integral
\[\int_1^4 e^{3x} (x + 1) ~ dx.\]
(Hints won't be revealed until after you choose a response.)
\begin{multiplechoice}
\choice[correct]{\(\displaystyle \frac{14}{9} e^{12} - \frac{5}{9} e^{3}\)}
\choice{\(\displaystyle \frac{17}{9} e^{12} - \frac{5}{9} e^{3}\)}
\choice{\(\displaystyle \frac{17}{9} e^{12} - \frac{8}{9} e^{3}\)}
\choice{\(\displaystyle \frac{20}{9} e^{12} - \frac{8}{9} e^{3}\)}
\choice{\(\displaystyle \frac{20}{9} e^{12} - \frac{11}{9} e^{3}\)}
\choice{\(\displaystyle \frac{23}{9} e^{12} - \frac{11}{9} e^{3}\)}
\end{multiplechoice}
\begin{feedback}
Integrate by parts, integrating the exponential and differentiating polynomials.
\begin{hint}
\[\begin{aligned}
    \int_1^4 e^{3x} &  (x + 1) ~ dx \\
    & = \left. \frac{e^{3x}}{3} (x+1) \right|_1^4 - \int_1^4 \frac{e^{3x}}{3} dx \\
    & = \frac{5}{3} e^{12} - \frac{2}{3} e^{3} - \left. \frac{e^{3x}}{9} \right|_1^4 \\
    & = \frac{14}{9} e^{12} - \frac{5}{9} e^{3}
\end{aligned}\]
\end{hint}
\end{feedback}

\end{question}
\begin{question}%%%%%[IBPibp:fourier2alt4]

Compute the definite integral
\[\int_{\pi}^{2 \pi} x \sin 4 x ~ dx.\]
(Hints won't be revealed until after you choose a response.)
\begin{multiplechoice}
\choice{\(\displaystyle 0\)}
\choice{\(\displaystyle \frac{\pi}{4}\)}
\choice[correct]{\(\displaystyle -\frac{\pi}{4}\)}
\choice{\(\displaystyle \frac{3 \pi}{4}\)}
\choice{\(\displaystyle -\frac{3 \pi}{4}\)}
\choice{\(\displaystyle \frac{7 \pi}{4}\)}
\end{multiplechoice}
\begin{feedback}
Integrate by parts, integrating the trig functions and differentiating polynomials.
\begin{hint}
\[ \begin{aligned}
    \int_{\pi}^{2 \pi} x & \sin 4 x ~ dx \\
    & = \left. - \frac{\cos 4x}{4} x \right|_{\pi}^{2 \pi} + \int_{\pi}^{2 \pi} \frac{\cos 4x}{4} dx \\
    & = - \frac{2 \pi }{4} + \frac{(-1)^{4} \pi }{4} - \left. \frac{\sin 4x}{16} \right|_{\pi}^{2 \pi} = -\frac{\pi}{4}
\end{aligned}\]
\end{hint}
\end{feedback}

\end{question}
\begin{question}%%%%%[IBPibp:miscarctan5]

Compute the indefinite integral
\[\int \arctan 5x ~ dx.\]
(Hints won't be revealed until after you choose a response.)
\begin{multiplechoice}
\choice[correct]{\(\displaystyle x \arctan 5x - \frac{1}{10} \ln | 1 + 25x^2|  + C\)}
\choice{\(\displaystyle x \arctan 5x - \frac{1}{12} \ln | 1 + 25x^2|  + C\)}
\choice{\(\displaystyle x \arctan 5x - \frac{1}{14} \ln | 1 + 25x^2|  + C\)}
\choice{\(\displaystyle x \arctan 5x + \frac{1}{10} \ln | 1 + 25x^2|  + C\)}
\choice{\(\displaystyle x \arctan 5x + \frac{1}{12} \ln | 1 + 25x^2|  + C\)}
\choice{\(\displaystyle x \arctan 5x + \frac{1}{14} \ln | 1 + 25x^2|  + C\)}
\end{multiplechoice}
\begin{feedback}
Integrate by parts, integrating the coefficient 1 and differentiating arctangent.
\begin{hint}
\[\begin{aligned}
    \int & \arctan 5x ~ dx \\
    & = x \arctan 5x - \int \frac{5x}{1+25x^2} ~ dx \\
    & = x \arctan 5x - \frac{1}{10} \int \frac{50x}{1+25x^2} ~ dx \\
    & = x \arctan 5x - \frac{1}{10} \ln | 1 + 25x^2|  + C
\end{aligned}\]
\end{hint}
\end{feedback}

\end{question}
\begin{question}%%%%%[IBPibp:expcos35]

Compute the indefinite integral
\[\int e^{3x}  \cos 5x ~ dx.\]
(Hints won't be revealed until after you choose a response.)
\begin{multiplechoice}
\choice{\(\displaystyle \frac{e^{3x} (2 \cos 5x + 5 \sin 5x)}{29} + C\)}
\choice[correct]{\(\displaystyle \frac{e^{3x} (3 \cos 5x + 5 \sin 5x)}{34} + C\)}
\choice{\(\displaystyle \frac{e^{3x} ( \cos 5x + 3 \sin 5x)}{20} + C\)}
\choice{\(\displaystyle \frac{e^{3x} ( \cos 5x + 2 \sin 5x)}{15} + C\)}
\choice{\(\displaystyle \frac{e^{3x} (2 \cos 5x + 7 \sin 5x)}{53} + C\)}
\choice{\(\displaystyle \frac{e^{3x} (3 \cos 5x + 7 \sin 5x)}{58} + C\)}
\end{multiplechoice}
\begin{feedback}
Integrate by parts, integrating the exponential and differentiating cosine (or vice-versa), then solve for the antiderivative.
\begin{hint}
\[ \begin{aligned}
    \int e^{3x} & \cos 5x ~ dx \\
    & = \frac{e^{3x}}{3} \cos 5x - \int \frac{e^{3x}}{3} (-5 \sin 5x) ~ dx \\
    & = \frac{e^{3x} \cos 5x}{3} + \frac{5}{3} \int e^{3x} \sin 5x ~ dx \\
    & = \frac{e^{3x} \cos 5x}{3} + \frac{5}{3} \frac{e^{3x}}{3} \sin 5x - \frac{5}{3} \int \frac{e^{3x}}{3} (5 \cos 5x) ~ dx \\
    & = \frac{e^{3x} (3 \cos 5x + 5 \sin 5x)}{9} - \frac{25}{9} \int e^{3x} \cos 5x ~ dx  \\
    \Rightarrow   & \qquad \frac{34}{9}  \int e^{3x}  \cos 5x ~ dx  = \frac{e^{3x} (3 \cos 5x + 5 \sin 5x)}{9} \\
    \Rightarrow & \qquad  \int  e^{3x}  \cos 5x ~ dx = \frac{e^{3x} (3 \cos 5x + 5 \sin 5x)}{34} + C
\end{aligned}\]
\end{hint}
\end{feedback}

\end{question}

\section*{Sample Exam Questions}

\begin{question}%%%%%[2015C.11]

Compute the integral below.
\[ \int_{\frac{1}{2}}^\infty \frac{ \ln (2x)}{x^2} dx \]
\begin{multiplechoice}
\choice{\(1 - \ln 2\)}
\choice[correct]{\(2\)}
\choice{\(\displaystyle \ln 2 - \frac{1}{2}\)}
\choice{\(\displaystyle \frac{1}{2}\)}
\choice{\(2 - 2 \ln 2\)}
\choice{the integral diverges}
\end{multiplechoice}

\end{question}

\begin{question}%%%%%[2016C.05]

Compute the indefinite integral indicated below. [Hint: Write \(\displaystyle \frac{1}{\cos^2 \theta} = \sec^2 \theta\) and integrate by parts.]
\[ \int \left( 1 + \frac{\ln | \sin \theta|}{\cos^2 \theta} \right) d \theta \]
\begin{multiplechoice}
\choice{\(\displaystyle (\sin \theta) \ln |\sin \theta| + C\)}
\choice{\(\displaystyle (\cos \theta) \ln |\sin \theta| + C\)}
\choice[correct]{\(\displaystyle (\tan \theta) \ln |\sin \theta| + C\)}
\choice{\(\displaystyle (\csc \theta) \ln |\sin \theta| + C\)}
\choice{\(\displaystyle (\sec \theta) \ln |\sin \theta| + C\)}
\choice{\(\displaystyle (\cot \theta) \ln |\sin \theta| + C\)}
\end{multiplechoice}

\end{question}

\end{document}


\chapter{Second Order Boundary Value Problems}

\section{2nd order two-point boundary value problems}

\begin{intro}
  We have already seen, that boundary value problems have very
  different stability properties than initial value
  problems. Here, we will discuss a special class of boundary value
  problems of the form
  \begin{gather}
    \label{eq:bvp-second:1}
    -u''(x) + \beta(x) u'(x) + \gamma(x)u(x) = f(x),
    \qquad u(a) = u_a,
    \quad u(b) = u_b.
  \end{gather}

  In order to make this problem more amenable to mathematical
  investigation, we introduce the set
  \begin{gather*}
    \mathcal B =
    \Bigl\{ u\in C^2(a,b) \cap C[a,b]
    \;\Big|\; u(a) = u_a\;\wedge\; u(b) = u_b
    \Bigr\}.
  \end{gather*}

  Then, we can see the left hand side of the differential equation as
  a differential operator applied to $u$ and thus mapping $\mathcal B$
  to the set of continuous functions. Namely, we define
  \begin{gather}
    \label{eq:fd:2}
    \begin{split}
      L: \mathcal B &\to C[a,b] \\
      u & \mapsto -u'' + \beta u' + \gamma u.
    \end{split}
  \end{gather}
  In addition, we would like to simplify our life and get rid of the
  inhomogeneous boundary values $u_a$ and $u_b$. To this end, let
  \begin{gather*}
    u_B(x) = u_a \frac{b-x}{b-a} + u_b \frac{x-a}{b-a},
  \end{gather*}
  and introduce the new function $u_0 = u+u_B$. Then, $u_0$ solves
  the boudnary value problem
  \begin{multline*}
    -u_0''(x) + \beta(x) u'(x) + \gamma(x)u(x) = f(x) -
    \beta(x)\frac{u_b-u_a}{b-a} - \gamma(x)u_B(x),
    \\ u(a) =  u(b) = 0.
  \end{multline*}
  Thus, it is sufficient to consider the boundary value problem
\end{intro}

\input{definitions/bvp-second}

\begin{remark}
  This definition exhibits a major change in paradigm. Before, we
  considered a differential equation as an equation which determines
  the derivative of a function in a point. Now, we are looking at a
  linear system of equations, albeit one, which is not of finite
  dimension. This paradigm change will be essential when we consider
  partial differential equations in future semesters.

  On the other hand, the equality in equation~\eqref{eq:bvp-second:2}
  is understood point-wise, such that in fact nothing but our point of
  view has changed.
\end{remark}

\begin{intro}
  Again, we subdivide the interval $I= [a,b]$ into subintervals, but
  the subdivision does not involve IVP solvers on subintervals, but
  much more like in the original subdivision in
  Definition~\ref{Definition:partitioning}, the the solution will only
  be defined at the partitioning points $t_k$, $k=0,\dots,n$.

  Thus, like with one-step and multistep methods we will have values
  $y_0, y_1,\dots, y_n$, but the sequence has a defined end at $y_n$
  due to the right boundary of the interval.

  While one-step methods directly discretize the Volterra integral
  equation in order to compute a solution at every new step,
  \textbf{finite difference methods} discretize the differential
  equation on the whole interval at once and then solve the resulting
  discrete (finite-dimensional) system of equations.

  We have accompished the first step and decided that instead of
  function values in every point of the interval $I$, we only
  approximate $u(t_k)$ in the points of the partition. What is left is
  the definition of the discrete operator representing the equation.
\end{intro}

\input{definitions/difference-operators}

\begin{remark}
  The 3-point stencil is the product of forward and backward
  difference operators.
  \begin{gather*}
    D^2_h u(x) = D^+_hu(x)D^-_hu(x) = D^-_hu(x)D^+_hu(x).
  \end{gather*}
  For simplicity, we only present finite differences of uniform
  subdivisions. Nevertheless, the definition of the operators can be
  extended easily to $h$ changing between intervals.
\end{remark}

\input{definitions/fd-consistency}
\input{theorems/fd-consistency}

\begin{proof}
  We begin to show consistency of the first two operators by Taylor
  expansion: for some $\xi\in(x,x+h)$, there holds
  \begin{align*}
    u'(x) - D^+_h u(x) &= u'(x) - \frac{u(x+h) - u(x)}h \\
    &= u'(x) - \frac{u(x)+h u'(x) + \tfrac{h^2}{2} u''(\xi) - u(x)}h
    \\
    &= \tfrac h2 u''(\xi).
  \end{align*}
  The same computation can be applied to $D^-_h u(x)$. It is clear
  that we need an additional symmetry argument for te other two,
  otherwise their consistency order would be lower. Therefore, we
  follow the line of argument that we introduced in
  Lemma~\ref{Lemma:lmm-bramble-hilbert}, and which here reads: a
  difference operator $D^\alpha_h$ approximating a derivative of order
  $\alpha$ is consistent of order $p$, if and only if it is exact for all
  polynomials of degree $p+\alpha-1$. We realize this by computing the
  Taylor polynomial $p(h)$ of degree $p+\alpha-1$ and the remainder term
  involving $u^{(p+\alpha)}(\xi)$. Then,
  \begin{gather*}
    D^\alpha_h u(x) = \frac{p(h)+\frac{h^{\alpha+p}}{(\alpha+p)!}u^{(p+\alpha)}(\xi)}{h^\alpha}.
  \end{gather*}
  Now we employ that the formula is exact for $p(h)$ and thus
  \begin{gather*}
    u^{(\alpha)}(x) - D^\alpha_h u(x)
    = \frac{h^p}{(\alpha+p)!}u^{(p+\alpha)}(\xi).
  \end{gather*}
  We now write
  \begin{gather*}
    p(\xi) = a_0 + a_1 (\xi-x) + a_2 (\xi-x)^2 + a_3 (\xi-x)^3 + \cdots
  \end{gather*}
  The central difference $D^c_h$ is exact for linear polynomials,
  since ti evaluates to zero for a constant and $D^c_h (\xi-x) =
  1$. But additionally, we observe
  \begin{gather*}
    \frac{d}{d\xi} (\xi-x)^2 \Bigr|_{\xi=x}
    = D^c_h (\xi-x)^2 \Bigr|_{\xi=x}= 0.
  \end{gather*}
  Thus, the central difference is exact for polynomials of degree 2
  and consistent of second order.

  For the 3-point stencil, we observe that $D^2_h u(x)=0$ for any
  function $u$ such that $u(x+h) - u(x-h) = u(x)$, in particular any
  odd polynomial in $\xi-x$. Furthermore,
  \begin{gather*}
    D^2_h  (\xi-x)^2 \Bigr|_{\xi=x}= \frac{h^2-0+h^2}{h^2}
    = 2 = \frac{d^2}{d\xi^2} (\xi-x)^2
  \end{gather*}
\end{proof}

\begin{remark}
  When applied to the equation $u'=f(t,u)$ the solutions obtained by
  forward and backward differences correspond to the explicit and
  implicit Euler methods, respectively.
\end{remark}

\input{definitions/fd}

\input{definitions/fd-example}

\begin{remark}
  Like our view to the continuous boundary value problem has changed,
  the discrete one is now a fully coupled linear system which has to
  be solved by methods of linear algebra, not by time stepping
  anymore. In fact, we have $n+1$ variables $y_0,\dots, y_n$ and $n+1$
  equations, such that here existence and uniqueness of solutions are
  equivalent.
\end{remark}

\input{definitions/fd-matrix}

\begin{remark}
  The first and last row of the matrix $L_h$ are redundant, since they
  simply say $y_0 = y_n = 0$. They can be eliminated, such that we
  obtain the reduced system
  \begin{gather}
    \label{eq:fd:8}
    L_h y =
    \begin{pmatrix}
      \lambda_1 & \nu_1\\
      \mu_2 & \lambda_2 & \ddots \\
      & \ddots & \ddots & \nu_{n-2} \\
      &&\mu_{n-1}& \lambda_{n-1}
    \end{pmatrix}
    \begin{pmatrix}
      y_1\\\vdots\\y_{n-1}
    \end{pmatrix}
    =
    \begin{pmatrix}
      f_1 \\\vdots\\f_{n-1}
    \end{pmatrix} = f_h.
  \end{gather}
  In this form, the operator $L_h$ is consistent with $L$ in the sense
  that it only describes the differential operator, not the boundary
  values. Therefore, we will use this form in our further analysis.

  When it comes to implementation, both versions have their
  merits. Obviously, the new operator involves less unknowns. On the
  other hand, the discretization with boundary unknowns is more
  straight-forward.
\end{remark}

\section{Existence, stability, and convergence}

\begin{intro}
  Since the solution of the discretized boundary value problem is a
  problem in linear algebra, we have to study properties of the matrix
  $L_h$. The shortest and most elegant way to prove stability is
  through the properties of M-matrices, which we present here very
  shortly. We are not dwelling on this approach too long, since it is
  sufficient for stability, but by far not necessary and constrained
  to low order methods.

  The fact that $L_h$ is an M-matrix requires some knowledge of
  irreducible weakly diagonal dominant matrices, which the author
  considers as outdated as the whole concept of m-matrices. We will
  just quote this result without proof.
\end{intro}

\input{definitions/m-matrix}

\begin{Lemma}{m-matrix-fd1}
  The matrix $L_h$ defined above is an M-matrix provided that
  \begin{gather}
    \label{eq:fd:3}
    \gamma_k \ge 0, % > -\frac2{h^2},
    \qquad
    \abs{\beta_k} < \frac2h.
  \end{gather}
\end{Lemma}

\begin{proof}
  It is clear that these two conditions are sufficient for the first
  M-matrix property. The proof of positivity of the inverse is based
  on irreducible diagonal dominance, which is too long and too
  specialized for these notes.
\end{proof}

\begin{remark}
  The finite element method provides much more powerful to deduce
  solvability and stability of the discrete problem.
\end{remark}

\input{theorems/m-matrix-inverse}

\begin{proof}
  Let $x\in \R^n$ and $y = A^{-1} x$. Then,
  \begin{align*}
    \abs{y_i} & = \abs{\sum c_{ij} x_j} \\
    & \le \sum c_{ij} \abs{x_j} \\
    & \le \norm{x}_\infty \sum c_{ij} v_j.
  \end{align*}
  Thus,
  \begin{gather*}
    \abs{y_i} \le  \norm{x}_\infty \bigl(A^{-1} v\bigr)_i
    = \norm{x}_\infty \bigl(A^{-1} Aw\bigr)_i \le \norm{x}_\infty \abs{w_i}.
  \end{gather*}
  Taking the maximum over all $i$, we obtain
  \begin{gather*}
    \norm{A^{-1}}_\infty = \sup_{u\in
      \R^n}\frac{\norm{A^{-1}u}_\infty}{\norm{u}_\infty}
    \le \norm{w}_\infty.
  \end{gather*}
\end{proof}

\input{theorems/fd-stability}

\begin{proof}
  Take the function
  \begin{gather*}
    p(x) = (x-a)(b-x) = -x^2+(a+b)x-ab,
  \end{gather*}
  with derivatives $p'(x) = a+b-2x$ and $p''(x) = -2$, and a maximum
  of $(b-a)^2/4$ at $x=(a+b)/2$. Choose the values $p_k = p(x_k)$. By
  consistency, we have and $k=1,\dots,n-1$
  \begin{gather*}
    (L_h p)_k \ge 2 - \beta_k\abs{b-a} + \gamma_k \abs{b-a}^2
    \ge 2-\delta.
  \end{gather*}
  Thus, the vector with entries $w_k = p_k/(2-\delta)$ can be used to
  bound the inverse of $L_h$ by Lemma~\ref{Lemma:m-matrix-inverse}.
\end{proof}

\begin{remark}
  The assumptions of the previous theorem involve two sets of
  conditions on the parameters $\beta_k$ and
  $\gamma_k$. Condition~\eqref{eq:fd:1} is actually a condition on the
  continuous problem. The condition on $\gamma_k$ is indeed necessary,
  as will be seen when we study partial differential equations. The
  condition on $\beta_k$ is not necessary in this form, but a better
  estimate again requires far advanced analysis.

  The other set of conditions relates the coefficients to the mesh
  size. Again, the condition on $\beta_k$ can be avoided as seen in
  the next example. The condition on $\gamma_k$ is already implied by
  $-\gamma_k \le (b-a)^2$, which is a small restriction compared
  to~\eqref{eq:fd:1}, as soon as the partition has 3 interior
  points. Thus, it is not crucial.
\end{remark}

\input{definitions/upwind}
\input{theorems/fd-convergence}

\begin{proof}
  We apply the difference operator $L_h$ to $u$ (as the vector of
  function values in the points $t_k$) and $y$ to obtain
  \begin{gather*}
    L_h(u-y) = (L_h - L) u + Lu - L_h y = \tau + f - f = \tau,
  \end{gather*}
  where $\tau = (\tau_1,\dots,\tau_{n-1})^T$ is the vector, which
  measures the consistency error  $(L_h-L) u$ in each $t_k$. The
  entries $\tau_k$ are bounded by $c h^p$ by the consistency estimate.
  For the error, there holds
  \begin{gather*}
    u-y = L_h^{-1} L_h(u-y) =  L_h^{-1} \tau.
  \end{gather*}
  Using the stability assumption, we obtain
  \begin{gather*}
    \norm{u-y}_\infty \le \norm{L_h^{-1}}_\infty \norm{\tau}_\infty
    \le c h^p,
  \end{gather*}
  where the constant $c$ depends on $\norm{L_h^{-1}}_\infty$ and the
  constant in the consistency estimate, but not on $h$.
\end{proof}

\begin{remark}
  Finite differences can be generalized to higher order by extending
  the stencils by more than one point to the left and right of the
  current point. Whenever we add two points to the symmetric
  difference formulas, we can gain two orders of consistency.
  \begin{gather*}
    \underbrace{\bullet\quad
      \underbrace{\bullet\quad
        \underbrace{\bullet\quad\phantom{\bullet}\quad\bullet}_{u'+\mathcal O(h^2)}
        \quad\bullet}_{u'+\mathcal O(h^4)}
      \quad\bullet}_{u'+\mathcal O(h^6)}
    \qquad\qquad
    \underbrace{\bullet\quad
      \underbrace{\bullet\quad
        \underbrace{\bullet\quad\bullet\quad\bullet}_{u''+\mathcal O(h^2)}
        \quad\bullet}_{u''+\mathcal O(h^4)}
      \quad\bullet}_{u''+\mathcal O(h^6)}
  \end{gather*}
  Similarly, we can define one-sided difference formulas, which get us
  close to multistep methods. The matrices generated by these formulas
  are not M-matrices anymore, although you can show for the 4th order
  formula for the second derivative that it yields a product of two
  M-matrices. While this rescues the theory in a particular instance,
  M-matrices do not provide a theoretical framework for general high
  order finite differences anymore.

  Very much like the starting procedures for high order multistep
  methods, high order finite differences cause problems at the
  boundaries. Here, the formulas must be truncated and for instance be
  replaced by one-sided formulas of equal order.

  All these issues motivate the study of different discretization
  methods in the next course.
\end{remark}

\section{The Laplacian and harmonic functions}

\begin{intro}
  The two-point boundary value problem has a natural extension to
  higher dimensions. There, we deal with partial derivatives
  $\frac{\partial}{\partial x}$, $\frac{\partial}{\partial y}$, and
  $\frac{\partial}{\partial z}$. As an outlook towards topics
  discussed in classes on partial differential equations and their
  numerical analysis, we close these notes by a short introduction at
  hand of examples.
\end{intro}

\begin{Definition}{laplace-equation}
  the \define{Laplacian} in two (three) space dimensions is the sum of
  the second partial derivatives
  \begin{gather}
    \label{eq:fd:10}
    \Delta u = \frac{\partial^2}{\partial x^2}u
    + \frac{\partial^2}{\partial y^2}u
    \left(+ \frac{\partial^2}{\partial z^2}u\right)
    = \operatorname{div}(\nabla u)
  \end{gather}
  The \define{Laplace equation} is the partial differential equation
  \begin{gather}
    \label{eq:fd:11}
    -\Delta u = 0.
  \end{gather}
  The \define{Poisson equation} is the partial differential equation
  \begin{gather}
    \label{eq:fd:9}
    -\Delta u = f.
  \end{gather}
  Solutions to the Laplace equations are called \define{harmonic functions}.
\end{Definition}

\subsection{Properties of harmonic functions}

\begin{Theorem*}{mean-value}{Mean-value formula for harmonic functions}
  Let $u\in C^2(\Omega)$ be a solution to the Laplace equation. Then,
  $u$ has the mean value property
  \begin{gather}
    \label{eq:mean-value:1}
    u(\mathbf x) = \frac1{r^{d-1}\omega(d)}
    \int_{\partial B_r(\mathbf x)} u(\mathbf y) \,ds,
  \end{gather}
  where $\partial B_r(\mathbf x) \subset \Omega$ is the sphere of
  radius $r$ around $\mathbf x$ and $\omega(d)$ is the volume of the
  unit sphere in $\R^d$.
\end{Theorem*}

\begin{proof}
  First, we rescale the problem to
  \begin{gather*}
    \Phi(r) = \frac1{r^{d-1}\omega(d)}
    \int_{\partial B_r(\mathbf x)} u(\mathbf y) \,ds
    = \frac1{\omega(d)} \int_{\partial B_1(0)} u(\mathbf x+r\mathbf z) \,ds.
  \end{gather*}
  Then, we notice that
  \begin{align*}
    \Phi'(r)
    &= \frac1{\omega(d)} \int_{\partial B_1(0)}
      \nabla u(\mathbf x+r\mathbf z)\cdot \mathbf z \,ds_z\\
    &= \frac1{r^{d-1}\omega(d)} \int_{\partial B_r(\mathbf x)}
      \nabla u(\mathbf y)\cdot\frac{\mathbf y-\mathbf x}{r} \,ds_y\\
    &= \frac1{r^{d-1}\omega(d)} \int_{\partial B_r(\mathbf x)}
    \frac{\partial}{\partial \mathbf n} u(\mathbf y) \,ds_y\\
    &= \frac1{r^{d-1}\omega(d)} \int_{B_r(\mathbf x)}
    \Delta u(\mathbf y)\,d\mathbf y = 0.
  \end{align*}
  Between the last two lines, we used the Gauß theorem for the vector
  valued function $\nabla u$. Therefore, $\Phi(r)$ is
  constant. Because of continuity, we have
  \begin{gather*}
    \lim_{r\to 0} \Phi(r) =
    \lim_{r\to 0}\frac1{r^{d-1}\omega(d)} \int_{\partial B_r(\mathbf x)} u(\mathbf y)
    \,ds
    = u(\mathbf x),
  \end{gather*}
  which proves our theorem.
\end{proof}

\input{theorems/maximum}

\begin{proof}
  Let $\mathbf x_0$ be such a maximum and let $r>0$ such that
  $B_r(\mathbf x_0) \subset \Omega$. Assume that there is a point
  $\mathbf x$ on $\partial B_r(\mathbf x_0)$, such that $u(\mathbf x)
  < u(\mathbf x_0)$. Then, this holds for points $\mathbf y$ in a
  neighborhood of $\mathbf x$. Thus, in order that the mean value
  property holds, there must be a subset of $\partial B_r(\mathbf
  x_0)$ where $u(\mathbf y) > u(\mathbf x_0)$, contradicting that
  $\mathbf x_0$ is a maximum. Thus, $u(\mathbf x) = u(\mathbf x_0)$
  for all $\mathbf x \in B_r(\mathbf x_0)$ for all $r$ such that
  $B_r(\mathbf x_0) \subset \Omega$.

  Let now $\mathbf x\in \Omega$ be arbitrary. Then, there is a
  (compact) path from $\mathbf x_0$ to $\mathbf x$ in $\Omega$. Thus,
  the path can be covered by a finite set of overlapping balls inside
  $\Omega$, and the argument above can be used iteratively to conclude
  $u(\mathbf x) = u(\mathbf x_0)$.
\end{proof}

\begin{corollary}
  Let $u \in C^2(\Omega)$ be a solution to the Laplace equation. Then,
  its maximum and its minimum lie on the boundary, that is, there are
  points
  $\underline{\mathbf x}, \overline{\mathbf x} \in\partial\Omega$,
  such that
  \begin{gather*}
    u(\underline{\mathbf x}) \le u(\mathbf x)
    \le u(\overline{\mathbf x})
    \quad\forall \mathbf x\in\Omega.
  \end{gather*}
\end{corollary}

\begin{proof}
  If the maximum of $u$ is attained in an interior point, the maximum
  principle yields a constant solution and the theorem holds
  trivially. On the other hand, the maximum principle does not make
  any prediction on points at the boundary, which therefore can be
  maxima. The same holds for the minimum, since $-u$ is a solution to
  the Laplace equation as well.
\end{proof}

\begin{corollary}
  Solutions to the Poisson equation with homogeneous boundary
  conditions are unique.
\end{corollary}

\begin{proof}
  Assume there are two functions $u,v\in C^2(\Omega)$ with $u=v=0$ on
  $\partial\Omega$ such that
  \begin{gather*}
    -\Delta u = -\Delta v = f.
  \end{gather*}
  Then, $w=u-v$ solves the Laplace equation with $w=0$ on
  $\partial\Omega$. Due to the maximum principle, $w \equiv 0$ and
  $u=v$.
\end{proof}

\section{Finite differences}
\input{definitions/square-domain}
\input{definitions/dirichlet}
\input{definitions/cartesian-grid}
\input{definitions/lexicographic}
\input{definitions/5-point-stencil}

\begin{remark}
  From now on, we will call the discrete solution $u_h$ in order to
  avoid confusion with the coordinate direction $y$.
\end{remark}

\input{theorems/fd-matrix}

\begin{Theorem}{5-point-m-matrix}
  The matrix generated by the 5-point stencil is an M-matrix and the
  discrete problem
  \begin{gather*}
    L_h u_h = f
  \end{gather*}
  is stable in the sense that there is a constant $c$ independent of
  the grid spacing $h$ such that
  \begin{gather*}
    \norm{L_h^{-1}} \le c.
  \end{gather*}
\end{Theorem}

\begin{proof}
  The proof of M-matrix property is identical to the proof for 2-point
  boundary value problems, which was omitted there.  The same way as
  there, the function
  \begin{gather*}
    w(x,y) = (x-a)(b-x)(y-a)(b-y)
  \end{gather*}
  can be employed to show boundedness of $\norm{L_h^{-1}}$.
\end{proof}

\input{theorems/fd2-consistency}

\begin{proof}
  We apply the analysis of Lemma~\ref{Lemma:fd-consistency} in $x$-
  and $y$-directions separately, obtaining
  \begin{align*}
    \abs{\frac{\partial^2}{\partial x^2} u(x,y)
      - \frac{u(x+h,y) -2u(x,y) + u(x-h,y)}{h^2}} & \le c h^2 \\
    \abs{\frac{\partial^2}{\partial y^2} u(x,y)
      - \frac{u(x,y+h) -2u(x,y) + u(x,y-h)}{h^2}} & \le c h^2,
  \end{align*}
  an conclude consistency of the sum.
\end{proof}

\input{theorems/fd2-maximum}

\begin{proof}
  From equation~\eqref{eq:5-point-stencil:1}, it is clear that a
  discrete mean value property holds, that is, $y_k$ is the mean value
  of its four neighbors. Therefore, if $y_k \ge y_j$, for all
  neighboring indices $j$ of $k$, we have $y_j = y_k$. We conclude by
  following a path through the grid points.
\end{proof}

\section{Evolution equations}

After an excursion to second order differential equations depending on
a spatial variable, we are now returning to problems depending on
time. But this time, on time \emph{and} space. As for the
nomenclature, we have encountered ordinary differential equations as
equations or systems depending on a time variable only, then partial
differential equations (PDE) with several, typically a spatial
independent variables. While the problems considered here are covered
by the definition of PDE, time and space are fundamentally different
as long as we stay away from black holes, such that a distinction is
reasonable. Therefore, we introduce the concept of

\begin{Definition}{evolution-equation}
  An equation of the form
  \begin{gather}
    \partial_t u(t,x) + F(t,u(t,x)) = 0,
  \end{gather}
  where $u(t,.)$ is in a function space $V$ on a domain $\Omega$, and
  $F$ is a differential operator with respect to the spatial variables
  $x$ only, is an \define{evolution equation} of first order (in
  time).

  An \define{initial boundary value problem} (\define{IBVP}) for this
  evolution equation completes the differential equation by conditions
  \begin{xalignat}2
    u(0,x) &= u_0 & x&\in\Omega \\
    u(t,x) &= g & x&\in\partial\Omega,\; t>0.
  \end{xalignat}
\end{Definition}

\section{Fundamental solutions}

\begin{Definition}
  The function
  \begin{gather}
    \label{eq:fd:fundamental}
    \Phi(x) =
    \begin{cases}
      -\tfrac{1}{2\pi} \log \abs{x} & d=2\\
      \tfrac1{d(d-2)\omega_d} \frac1{\abs{x}^{d-2}} & d\ge 3,
    \end{cases}
  \end{gather}
  for $x\in \R^d$ is the \define{fundamental solution} to the Laplace
  equation. Here, $\omega_d$ is the volume of the unit ball in $\R^d$.
\end{Definition}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "notes"
%%% End:


\documentclass[10pt]{article}

\title{Alignment with dynamic programming}
\author{Heng Li}

\begin{document}

\maketitle

\section{General notations}

Suppose we have two sequences: a \emph{target} sequence and a \emph{query}
sequence. The length of the target sequence is $\ell_t$ with each residue
indexed by $i$.  The length of query is $\ell_q$ with each residue indexed by
$j$. Gaps on the target sequence are \emph{deletions} and gaps on the query are
\emph{insertions}. Function $S(i,j)$ gives the score between two residues on
the target and the query, respectively. $q>0$ is the gap open/initiation
penalty and $e>0$ the gap extension penalty. A gap of length $k$ costs
$q+k\cdot e$.

\section{Global alignment with affine-gap penalties}

\subsection{Durbin's formulation}

The original Durbin's formulation is:
\begin{eqnarray*}
M_{ij}&=&\max\{M_{i-1,j-1}, E_{i-1,j-1}, F_{i-1,j-1}\} + S(i,j)\\
E_{ij}&=&\max\{M_{i-1,j}-q, E_{i-1,j}\} - e\\
F_{ij}&=&\max\{M_{i,j-1}-q, F_{i,j-1}\} - e
\end{eqnarray*}
This formulation disallows a deletion immediately followed an insertion, or
vice versa. A more general form is:
\begin{eqnarray*}
M_{ij}&=&\max\{M_{i-1,j-1}, E_{i-1,j-1}, F_{i-1,j-1}\} + S(i,j)\\
E_{ij}&=&\max\{M_{i-1,j}-q, E_{i-1,j}, F_{i-1,j}-q\} - e\\
F_{ij}&=&\max\{M_{i,j-1}-q, E_{i,j-1}-q, F_{i,j-1}\} - e
\end{eqnarray*}

\subsection{Green's formulation}

If we define:
\[H_{ij}=\max\{M_{ij},E_{ij},F_{ij}\}\]
the Durbin's formulation can be transformed to
\begin{eqnarray*}
E_{ij} &=& \max\{H_{i-1,j}-q, E_{i-1,j}\} - e \\
F_{ij} &=& \max\{H_{i,j-1}-q, F_{i,j-1}\} - e \\
H_{ij} &=& \max\{H_{i-1,j-1}+S(i,j), E_{ij}, F_{ij}\}
\end{eqnarray*}
I first saw this formulation in Phrap developed by Phil Green, though it may
have been used earlier. If we further introduce
\begin{eqnarray*}
E'_{ij}&=&E_{i+1,j}\\
F'_{ij}&=&F_{i,j+1}
\end{eqnarray*}
we have
\begin{eqnarray*}
H_{ij} &=& \max\{H_{i-1,j-1}+S(i,j),E'_{i-1,j},F'_{i,j-1}\}\\
E'_{ij}&=& \max\{H_{ij}-q,E'_{i-1,j}\}-e\\
F'_{ij}&=& \max\{H_{ij}-q,F'_{i,j-1}\}-e
\end{eqnarray*}
In fact, we more often use this set of equations in practical implementations.
The initial conditions are
\begin{eqnarray*}
H_{-1,j}&=&
  \left\{\begin{array}{ll}
    0 & (j=-1)\\
	-q-(j+1)\cdot e & (0\le j<\ell_q)
  \end{array}\right.\\
H_{i,-1}&=&
  \left\{\begin{array}{ll}
    0 & (i=-1)\\
	-q-(i+1)\cdot e & (0\le i<\ell_t)
  \end{array}\right.\\
E'_{-1,j}&=&E_{0,j}=H_{-1,j}-q-e=-2q-(j+2)\cdot e\\
F'_{i,-1}&=&F_{i,0}=-2q-(i+2)\cdot e
\end{eqnarray*}

\subsection{Suzuki's formulation}

\subsubsection{Standard coordinate}
Now let
\begin{eqnarray*}
u'_{ij}&=&H_{ij}-H_{i-1,j}\\
v'_{ij}&=&H_{ij}-H_{i,j-1}\\
x'_{ij}&=&E'_{ij}-H_{ij}\\
y'_{ij}&=&F'_{ij}-H_{ij}
\end{eqnarray*}
We have
\begin{eqnarray}\label{eq:x}
x'_{ij}&=&\max\{-q,E'_{i-1,j}-H_{i-1,j}+H_{i-1,j}-H_{ij}\}-e\\\nonumber
&=&\max\{-q,x'_{i-1,j}-u'_{ij}\}-e
\end{eqnarray}
Similarly
\begin{equation}
y'_{ij}=\max\{-q,y'_{i,j-1}-v'_{ij}\}-e
\end{equation}
To derive the equation to compute $u'(i,j)$ and $v'(i,j)$, we note that
\begin{eqnarray*}
H_{ij}-H_{i-1,j-1}
&=&\max\{S(i,j),E'_{i-1,j}-H_{i-1,j-1},F'_{i,j-1}-H_{i-1,j-1}\}\\
&=&\max\{S(i,j),x'_{i-1,j}+v'_{i-1,j},y'_{i,j-1}+u'_{i,j-1}\}
\end{eqnarray*}
and
\[H_{ij}-H_{i-1,j-1}=u'_{ij}+v'_{i-1,j}=v'_{ij}+u'_{i,j-1}\]
We can derive the recursive equation for $u'_{ij}$ and $v'_{ij}$:
\begin{eqnarray*}
z'_{ij}&=&\max\{S(i,j),x'_{i-1,j}+v'_{i-1,j},y'_{i,j-1}+u'_{i,j-1}\}\\
u'_{ij}&=&z'_{ij}-v'_{i-1,j}\\
v'_{ij}&=&z'_{ij}-u'_{i,j-1}\\
x'_{ij}&=&\max\{0,x'_{i-1,j}+v'_{i-1,j}-z'_{ij}+q\}-q-e\\
y'_{ij}&=&\max\{0,y'_{i,j-1}+u'_{i,j-1}-z'_{ij}+q\}-q-e
\end{eqnarray*}

From eq.~(\ref{eq:x}) we can infer that $x'_{ij}\ge-q-e$ and similarly
$y'_{ij}\ge-q-e$. We further have:
\[
u'_{ij}=H_{ij}-H_{i-1,j-1}-v'_{i-1,j}\ge x'_{i-1,j}\ge-q-e
\]
Therefore, we have a lower bound $-q-e$ for $u'$, $v'$, $x'$ and $y'$.
This motivates us to redefine the four variables as:
\begin{eqnarray*}
u''_{ij}&=&H_{ij}-H_{i-1,j}+q+e\\
v''_{ij}&=&H_{ij}-H_{i,j-1}+q+e\\
x''_{ij}&=&E'_{ij}-H_{ij}+q+e\\
y''_{ij}&=&F'_{ij}-H_{ij}+q+e
\end{eqnarray*}
The recursion becomes
\begin{eqnarray*}
z''_{ij}&=&\max\{S(i,j)+2q+2e,x''_{i-1,j}+v''_{i-1,j},y''_{i,j-1}+u''_{i,j-1}\}\\
u''_{ij}&=&z''_{ij}-v''_{i-1,j}\\
v''_{ij}&=&z''_{ij}-u''_{i,j-1}\\
x''_{ij}&=&\max\{0,x''_{i-1,j}-u''_{ij}+q\}=\max\{0,x''_{i-1,j}+v''_{i-1,j}-z''_{ij}+q\}\\
y''_{ij}&=&\max\{0,y''_{i,j-1}-v''_{ij}+q\}=\max\{0,y''_{i,j-1}+u''_{i,j-1}-z''_{ij}+q\}
\end{eqnarray*}
Here $z_{ij}$ is a temporary variable. $u''$, $v''$, $x''$ and $y''$ are all
non-negtive.

\subsubsection{Rotated coordinate}

We let
\begin{eqnarray*}
r&=&i+j\\
t&=&i
\end{eqnarray*}
We have
\begin{eqnarray*}
z_{rt}&=&\max\{S(t,r-t)+2q+2e,x_{r-1,t-1}+v_{r-1,t-1},y_{r-1,t}+u_{r-1,t}\}\\
u_{rt}&=&z_{rt}-v_{r-1,t-1}\\
v_{rt}&=&z_{rt}-u_{r-1,t}\\
x_{rt}&=&\max\{0,x_{r-1,t-1}+v_{r-1,t-1}-z_{rt}+q\}\\
y_{rt}&=&\max\{0,y_{r-1,t}+u_{r-1,t}-z_{rt}+q\}
\end{eqnarray*}
Due to the definition of $r$ and $t$, the following inequation must stand:
\[0\le r-t \le\ell_q-1\]
\[0\le t \le\ell_t-1\]
where $\ell_t$ is the length of the sequence indexed by $i$ and $\ell_q$ the
length indexed by $j$. In case of banded alignment with a fixed diagonal band
of size $w$,
\[-w\le j-i\le w\]
In the $(r,t)$ coordinate, it is:
\[\frac{r-w}{2}\le t\le \frac{r+w}{2}\]
Putting these together:
\[0\le r\le \ell_q+\ell_t-2\]
\[\max\left\{0,r-\ell_q+1,\frac{r-w}{2}\right\}\le t\le\min\left\{\ell_t-1,r,\frac{r+w}{2}\right\}\]

\subsubsection{Initial conditions}
\[x_{r-1,-1}=x''_{-1,r}=E'_{-1,r}-H_{-1,r}+q+e=0\]
\[y_{r-1,r}=y''_{r,-1}=0\]
\[v_{r-1,-1}=v''_{-1,r}=H_{-1,r}-H_{-1,r-1}+q+e=\left\{\begin{array}{ll}
  q & (r>0) \\
  0 & (r=0)
\end{array}\right.\]
\[u_{r-1,r}=u''_{r,-1}=H_{r,-1}-H_{r-1,-1}+q+e=\left\{\begin{array}{ll}
  q & (r>0) \\
  0 & (r=0)
\end{array}\right.\]

\section{Alignment with dual affine-gap penalties}

\subsection{Green's formulation}

\begin{eqnarray*}
H_{ij} &=& \max\{H_{i-1,j-1}+S(i,j),E'_{i-1,j},F'_{i,j-1},\tilde{E}'_{i-1,j},\tilde{F}'_{i,j-1}\}\\
E'_{ij}&=& \max\{H_{ij}-q,E'_{i-1,j}\}-e\\
F'_{ij}&=& \max\{H_{ij}-q,F'_{i,j-1}\}-e\\
\tilde{E}'_{ij}&=& \max\{H_{ij}-\tilde{q},\tilde{E}'_{i-1,j}\}-\tilde{e}\\
\tilde{F}'_{ij}&=& \max\{H_{ij}-\tilde{q},\tilde{F}'_{i,j-1}\}-\tilde{e}
\end{eqnarray*}
The initial conditions are:
\begin{eqnarray*}
H_{-1,j}&=&
  \left\{\begin{array}{ll}
    0 & (j=-1)\\
	\max\{-q-(j+1)\cdot e,-\tilde{q}-(j+1)\cdot\tilde{e}\} & (0\le j<\ell_q)
  \end{array}\right.\\
H_{i,-1}&=&
  \left\{\begin{array}{ll}
    0 & (i=-1)\\
	\max\{-q-(i+1)\cdot e,-\tilde{q}-(i+1)\cdot\tilde{e}\} & (0\le i<\ell_t)
  \end{array}\right.\\
E'_{-1,j}&=&E_{0,j}=H_{-1,j}-q-e\\
F'_{i,-1}&=&F_{i,0}=H_{i,-1}-q-e\\
\tilde{E}'_{-1,j}&=&\tilde{E}_{0,j}=H_{-1,j}-\tilde{q}-\tilde{e}\\
\tilde{F}'_{i,-1}&=&\tilde{F}_{i,0}=H_{i,-1}-\tilde{q}-\tilde{e}
\end{eqnarray*}

\subsection{Suzuki's formulation}

\begin{eqnarray*}
z'_{ij}&=&\max\{S(i,j),x'_{i-1,j}+v'_{i-1,j},y'_{i,j-1}+u'_{i,j-1},\\
&&\tilde{x}'_{i-1,j}+v'_{i-1,j},\tilde{y}'_{i,j-1}+u'_{i,j-1}\}\\
u'_{ij}&=&z'_{ij}-v'_{i-1,j}\\
v'_{ij}&=&z'_{ij}-u'_{i,j-1}\\
x'_{ij}&=&\max\{0,x'_{i-1,j}+v'_{i-1,j}-z'_{ij}+q\}-q-e\\
y'_{ij}&=&\max\{0,y'_{i,j-1}+u'_{i,j-1}-z'_{ij}+q\}-q-e\\
\tilde{x}'_{ij}&=&\max\{0,\tilde{x}'_{i-1,j}+v'_{i-1,j}-z'_{ij}+\tilde{q}\}-\tilde{q}-\tilde{e}\\
\tilde{y}'_{ij}&=&\max\{0,\tilde{y}'_{i,j-1}+u'_{i,j-1}-z'_{ij}+\tilde{q}\}-\tilde{q}-\tilde{e}
\end{eqnarray*}
In the rotated coordinate:
\begin{eqnarray*}
z_{rt}&=&\max\{S(t,r-t),x_{r-1,t-1}+v_{r-1,t-1},y_{r-1,t}+u_{r-1i,t},\\
&&\tilde{x}_{r-1,t-1}+v_{r-1,t-1},\tilde{y}_{r-1,t}+u_{r-1,t}\}\\
u_{rt}&=&z_{rt}-v_{r-1,t-1}\\
v_{rt}&=&z_{rt}-u_{r-1,t}\\
x_{rt}&=&\max\{0,x_{r-1,t-1}+v_{r-1,t-1}-z_{rt}+q\}-q-e\\
y_{rt}&=&\max\{0,y_{r-1,t}+u_{r-1,t}-z_{rt}+q\}-q-e\\
\tilde{x}_{rt}&=&\max\{0,\tilde{x}_{r-1,t-1}+v_{r-1,t-1}-z_{rt}+\tilde{q}\}-\tilde{q}-\tilde{e}\\
\tilde{y}_{rt}&=&\max\{0,\tilde{y}_{r-1,t}+u_{r-1,t}-z_{rt}+\tilde{q}\}-\tilde{q}-\tilde{e}
\end{eqnarray*}
By definition, it is easy to see the initial conditions except $u$ and $v$:
\[x_{r-1,-1}=x'_{-1,r}=E'_{-1,r}-H_{-1,r}=-q-e\]
\[y_{r-1,r}=y'_{r,-1}=F'_{r,-1}-H_{r,-1}=-q-e\]
\[\tilde{x}_{r-1,-1}=-\tilde{q}-\tilde{e}\]
\[\tilde{y}_{r-1,-1}=-\tilde{q}-\tilde{e}\]
\[v_{r-1,-1}=H_{-1,r}-H_{-1,r-1}=\left\{\begin{array}{ll}
\max\{-q-e,-\tilde{q}-\tilde{e}\} & (r=0)\\
-e & (r<\lceil\frac{\tilde{q}-q}{e-\tilde{e}}-1\rceil)\\
r(e-\tilde{e})-(\tilde{q}-q)-\tilde{e} & (r=\lceil\frac{\tilde{q}-q}{e-\tilde{e}}-1\rceil)\\
-\tilde{e} & (r>\lceil\frac{\tilde{q}-q}{e-\tilde{e}}-1\rceil)
\end{array}\right.\]

\end{document}


\section{The Fundamental Theorem of Calculus}\label{sec:FTC}
Let's recast the first example from the previous section. Suppose that
the speed of the object is $3t$ at time $t$. How far does the object
travel between time $t=a$ and time $t=b$? We are no longer assuming
that we know where the object is at time $t=0$ or at any other
time. It is certainly true that it is {\it somewhere,} so let's
suppose that at $t=0$ the position is $k$. Then just as in the
example, we know that the position of the object at any time is
$\ds 3t^2/2+k$. This means that at time $t=a$ the position is
$\ds 3a^2/2+k$ and at time $t=b$ the position is $\ds 3b^2/2+k$. Therefore the
change in position is $\ds 3b^2/2+k-(3a^2/2+k)=3b^2/2-3a^2/2$. Notice that
the $k$ drops out; this means that it doesn't matter that we don't
know $k$, it doesn't even matter if we use the wrong $k$, we get the
correct answer.

What about the second approach to this problem, in the new form? We
now want to approximate the change in position between time $a$ and
time $b$. We take the interval of time between $a$ and $b$, divide it
into $n$ subintervals, and approximate the distance traveled during
each. The starting time of subinterval number $i$ is now
$a+(i-1)(b-a)/n$, which we abbreviate as $\ds t_{i-1}$, so that
$\ds t_0=a$, $\ds t_1=a+(b-a)/n$, and so on. The speed of the object is
$f(t)=3t$, and each subinterval is $(b-a)/n=\Delta t$ seconds long.
The distance traveled during subinterval number
$i$ is approximately $\ds f(t_{i-1})\Delta t$, and the total change in
distance is approximately
$$
  f(t_0)\Delta t+f(t_1)\Delta t+\cdots+f(t_{n-1})\Delta t.
$$
The exact change in position is the limit of this sum as $n$ goes to
infinity. We abbreviate this sum using \dfont{sigma notation}:
$$
  \sum_{i=0}^{n-1} f(t_i)\Delta t =
f(t_0)\Delta t+f(t_1)\Delta t+\cdots+f(t_{n-1})\Delta t.
$$
The notation on the left side of the equal sign uses a large capital
sigma, a Greek letter, and the left side is an abbreviation for the
right side. The answer we seek is
$$
  \lim_{n\to\infty}\sum_{i=0}^{n-1} f(t_i)\Delta t.
$$
Since this must be the same as the answer we have already obtained, we
know that
$$
  \lim_{n\to\infty}\sum_{i=0}^{n-1} f(t_i)\Delta t={3b^2\over
  2}-{3a^2\over 2}.
$$
The significance of $\ds 3t^2/2$, into which we substitute $t=b$ and
$t=a$, is of course that it is a function whose derivative is $f(t)$.
As we have discussed, by the time we know that we want to compute
$$
  \lim_{n\to\infty}\sum_{i=0}^{n-1} f(t_i)\Delta t,
$$
it no longer matters what $f(t)$ stands for---it could be a speed, or
the height of a curve, or something else entirely. We know that
the limit can be computed by finding any function with derivative
$f(t)$, substituting $a$ and $b$, and subtracting. We summarize this
in a theorem. First, we introduce some new notation and terms.

We write
$$
  \int_a^b f(t)\,dt = \lim_{n\to\infty}\sum_{i=0}^{n-1} f(t_i)\Delta t
$$
if the limit exists. That is, the left hand side means, or is an
abbreviation for, the right hand side. The symbol $\int$ is called an
\dfont{integral sign}, and the whole
expression is read as ``the integral of $f(t)$ from $a$ to $b$.'' What
we have learned is that this integral can be computed by finding a
function, say $F(t)$, with the property that $F'(t)=f(t)$, and then
computing $F(b)-F(a)$. The function $F(t)$ is called an \dfont{antiderivative} of $f(t)$.
Now the theorem:

\begin{theorem}{Fundamental Theorem of Calculus}{fundamental_theorem_I}
Suppose that $f(x)$ is
continuous on the interval $[a,b]$. If $F(x)$ is any antiderivative of
$f(x)$, then
$$
  \int_a^b f(x)\,dx = F(b)-F(a).
$$
\end{theorem}

Let's rewrite this slightly:
$$
  \int_a^x f(t)\,dt = F(x)-F(a).
$$
We've replaced the variable $x$ by $t$ and $b$ by $x$. These are just
different names for quantities, so the substitution doesn't change the
meaning. It does make it easier to think of the two sides of the
equation as functions. The expression
$$
  \int_a^x f(t)\,dt
$$
is a function: plug in a value for $x$, get out some other value. The
expression $F(x)-F(a)$ is of course also a function, and it has a nice
property:
$$
  {d\over dx} (F(x)-F(a)) = F'(x) = f(x),
$$
since $F(a)$ is a constant and has derivative zero. In other words, by
shifting our point of view slightly, we see that the odd looking
function
$$
  G(x)=\int_a^x f(t)\,dt
$$
has a derivative, and that in fact $G'(x)=f(x)$. This is really just a
restatement of the Fundamental Theorem of Calculus, and indeed is
often called the Fundamental Theorem of Calculus. To avoid confusion,
some people call the two versions of the theorem ``The Fundamental
Theorem of Calculus, part I'' and ``The Fundamental
Theorem of Calculus, part II'', although unfortunately there is no
universal agreement as to which is part I and which part II. Since it
really is the same theorem, differently stated, some people simply
call them both ``The Fundamental
Theorem of Calculus.''

\begin{theorem}{Fundamental Theorem of Calculus}{fundamental_theorem_II}
Suppose that $f(x)$ is
continuous on the interval $[a,b]$ and let
$$
  G(x)=\int_a^x f(t)\,dt.
$$
Then $G'(x)=f(x)$.
\end{theorem}

We have not really proved the Fundamental Theorem. In a nutshell, we
gave the following argument to justify it: Suppose we want to know the
value of
$$
  \int_a^b f(t)\,dt = \lim_{n\to\infty}\sum_{i=0}^{n-1} f(t_i)\Delta t.
$$
We can interpret the right hand side as the distance traveled by an
object whose speed is given by $f(t)$. We know another way to compute
the answer to such a problem: find the position of the object by
finding an antiderivative of $f(t)$, then substitute $t=a$ and $t=b$
and subtract to find the distance traveled. This must be the answer to
the original problem as well, even if $f(t)$ does not represent a
speed.

What's wrong with this? In some sense, nothing. As a practical matter
it is a very convincing argument, because our understanding of the
relationship between speed and distance seems to be quite solid. From
the point of view of mathematics, however, it is unsatisfactory to
justify a purely mathematical relationship by appealing to our
understanding of the physical universe, which could, however unlikely
it is in this case, be wrong.

A complete proof is a bit too involved to include here, but we will
indicate how it goes. First, if we can prove the second version of the
Fundamental Theorem, Theorem~\ref{thm:fundamental_theorem_II}, then
we can prove the first version from that:

\begin{proof} Proof of Theorem~\ref{thm:fundamental_theorem_I}.

We know from Theorem~\ref{thm:fundamental_theorem_II} that
$$
  G(x)=\int_a^x f(t)\,dt
$$
is an antiderivative of $f(x)$, and therefore any antiderivative
$F(x)$ of $f(x)$ is of the form $F(x)=G(x)+k$. Then
\begin{eqnarray*}
  F(b)-F(a)=G(b)+k-(G(a)+k) &=& G(b)-G(a)\cr
  &=&\int_a^b f(t)\,dt-\int_a^a f(t)\,dt.\cr
\end{eqnarray*}
It is not hard to see that $\ds \int_a^a f(t)\,dt=0$, so this means that
$$
  F(b)-F(a)=\int_a^b f(t)\,dt,
$$
which is exactly what Theorem~\ref{thm:fundamental_theorem_I} says.
\end{proof}

So the real job is to prove
Theorem~\ref{thm:fundamental_theorem_II}. We will sketch the proof,
using some facts that we do not prove. First, the following identity
is true of integrals:
$$
  \int_a^b f(t)\,dt = \int_a^c f(t)\,dt + \int_c^b f(t)\,dt.
$$
This can be proved directly from the definition of the integral, that
is, using the limits of sums. It is quite easy to see that it must be
true by thinking of either of the two applications of integrals that
we have seen. It turns out that the identity is true no matter what
$c$ is, but it is easiest to think about the meaning when
$a\le c\le b$.

First, if $f(t)$ represents a speed, then we know that the three
integrals represent the distance traveled between time $a$ and time $b$;
the distance traveled between time $a$ and time $c$; and
the distance traveled between time $c$ and time $b$. Clearly the sum of
the latter two is equal to the first of these.

Second, if $f(t)$ represents the height of a curve, the three
integrals represent the area under the curve between $a$ and $b$;
the area under the curve between $a$ and $c$;
and the area under the curve between $c$ and $b$. Again it is clear
from the geometry that the first is equal to the sum of the second and
third.

\begin{proof} Proof of Theorem~\ref{thm:fundamental_theorem_II}.

We want to compute $G'(x)$, so we start with the definition of the
derivative in terms of a limit:
\begin{eqnarray*}
  G'(x)&=&\lim_{\Delta x\to0}{G(x+\Delta x)-G(x)\over\Delta x}\cr
  &=&\lim_{\Delta x\to0}{1\over \Delta x}\left(
  \int_a^{x+\Delta x} f(t)\,dt - \int_a^x f(t)\,dt\right)\cr
  &=&\lim_{\Delta x\to0}{1\over \Delta x}\left(
  \int_a^{x} f(t)\,dt + \int_x^{x+\Delta x} f(t)\,dt -
  \int_a^x f(t)\,dt\right)\cr
  &=&\lim_{\Delta x\to0}{1\over \Delta x}\int_x^{x+\Delta x} f(t)\,dt.\cr
\end{eqnarray*}
Now we need to know something about
$$
  \int_x^{x+\Delta x} f(t)\,dt
$$
when $\Delta x$ is small; in fact, it is very close to
$\Delta x f(x)$, but we will not prove this. Once again, it is easy to
believe this is true by thinking of our two applications:
The integral
$$
  \int_x^{x+\Delta x} f(t)\,dt
$$
can be interpreted as the distance traveled by an object over a very
short interval of time. Over a sufficiently short period of time, the
speed of the object will not change very much, so the distance
traveled will be approximately the length of time multiplied by the
speed at the beginning of the interval, namely, $\Delta x
f(x)$. Alternately, the integral may be interpreted as the area under
the curve between $x$ and $x+\Delta x$. When $\Delta x$ is very small,
this will be very close to the area of the rectangle with base $\Delta
x$ and height $f(x)$; again this is $\Delta x
f(x)$. If we accept this, we may proceed:
$$
  \lim_{\Delta x\to0}{1\over \Delta x}\int_x^{x+\Delta x} f(t)\,dt
  =\lim_{\Delta x\to0}{\Delta x f(x)\over \Delta x}=f(x),
$$
which is what we wanted to show.
\end{proof}

It is still true that we are depending on an interpretation of the
integral to justify the argument, but we have isolated this part of
the argument into two facts that are not too hard to prove. Once the
last reference to interpretation has been removed from the proofs of
these facts, we will have a real proof of the Fundamental Theorem.

Now we know that to solve certain kinds of problems, those that lead
to a sum of a certain form, we ``merely'' find an antiderivative and
substitute two values and subtract. Unfortunately, finding
antiderivatives can be quite difficult. While there are a small number
of rules that allow us to compute the derivative of any common
function, there are no such rules for antiderivatives. There are some
techniques that frequently prove useful, but we will never be able to
reduce the problem to a completely mechanical process.

Due to the close relationship between an integral and an
antiderivative, the integral sign is also used to mean
``antiderivative''. You can tell which is intended by whether the
limits of integration are included:
$$
  \int_1^2 x^2\,dx
$$
is an ordinary integral, also called a
\dfont{definite integral},
because it has a definite value, namely
$$
  \int_1^2 x^2\,dx={2^3\over3}-{1^3\over3}={7\over3}.
$$
We use
$$
  \int x^2\,dx
$$
to denote the antiderivative of $\ds x^2$, also called an
\dfont{indefinite integral}.
So this is evaluated as
$$
  \int x^2\,dx = {x^3\over 3}+C.
$$
It is customary to include the constant $C$ to indicate that there are
really an infinite number of antiderivatives. We do not need this $C$
to compute definite integrals, but in other circumstances we will need
to remember that the $C$ is there, so it is best to get into the habit
of writing the $C$.
When we compute a definite integral, we first find an antiderivative
and then substitute. It is convenient to first display the
antiderivative and then do the substitution; we need a notation
indicating that the substitution is yet to be done. A typical solution
would look like this:
$$
  \int_1^2 x^2\,dx=\left.{x^3\over 3}\right|_1^2 =
  {2^3\over3}-{1^3\over3}={7\over3}.
$$
The vertical line with subscript and superscript is used to indicate
the operation ``substitute and subtract'' that is needed to finish the
evaluation.

We seem to have found a pattern. When attempting to solve a previous question, we found the antiderivative of $x^2$ to be $x^3/3+c$ (as it was when solving the indefinite integral). Likewise, when we first began, we were trying to determine a position based on velocity, and $3t$ gave rise to $3t^2/2+k$.

As will be formalized later, we see that in these cases, the power is increased to $n+1$, but we also divide through by this factor, $n+1$. So $x$ becomes $x^2/2$, $x^2$ becomes $x^3/3$, and $x^3$ will become $x^4/4$.

Now we will also try with negative and fraction values in the following example.

\begin{example}{Fundamental Theorem of Calculus}{FundamentalTheoremCalculus}
Evaluate $\ds\int_1^4 x^3+\sqrt{x}+\frac{1}{x^2}\,dx $.
\end{example}

\begin{solution}
\[ \begin{array}{lcl}
\ds\int_1^4 x^3+\sqrt{x}+\frac{1}{x^2}\,dx
	& = & \ds\left.\frac{x^4}{4}+\frac{2x^{3/2}}{3}-x^{-1}\right|_1^4\\
\\
	& = & \ds\left(\frac{(4)^4}{4}+\frac{2(4)^{3/2}}{3}-4^{-1}\right) \\
\\
	& & \ds-\left(\frac{(1)^4}{4}+\frac{2(1)^{3/2}}{3}-1^{-1}\right)\\
\\
	& = & \ds\frac{415}{6}
\end{array}\]\
\end{solution}

\begin{formulabox}[Properties of Definite Integrals]
Some properties are as follows:
$$\mbox{Order of limits matters:}\qquad\int_a^b f(x)\,dx=-\int_b^a f(x)\,dx$$
$$\mbox{If interval is empty, integral is zero:}\qquad\int_a^a f(x)\,dx=0$$
$$\mbox{Constant Multiple Rule:}\qquad\int_a^b cf(x)\,dx=c\int_a^b f(x)\,dx$$
$$\mbox{Sum/Difference Rule:}\qquad\int_a^b f(x)\pm g(x)\,dx=\int_a^b f(x)\,dx\pm\int_a^b g(x)\,dx$$
$$\mbox{Can split up interval $[a,b]=[a,c]\cup[c,b]$:}\qquad\int_a^b f(x)\,dx=\int_a^c f(x)\,dx+\int_c^b f(x)\,dx$$
$$\mbox{The variable does not matter!:}\qquad\int_a^b f(x)\,dx=\int_a^b f(t)\,dt$$
\end{formulabox}

The reason for the last property is that a definite integral is a \ifont{number}, not a function, so the variable is just a placeholder that won't appear in the final answer.

Some additional properties are \ifont{comparison} types of properties.

\begin{formulabox}[Comparison Properties of Definite Integrals]
$$\mbox{If $f(x)\geq 0$ for $x\in[a,b]$, then:}\qquad\int_a^b f(x)\,dx\geq 0.$$
$$\mbox{If $f(x)\geq g(x)$ for $x\in[a,b]$, then:}\qquad\int_a^b f(x)\,dx\geq \int_a^b g(x)\,dx.$$
$$\mbox{If $m\leq f(x)\leq M$ for $x\in[a,b]$, then:}\qquad m(b-a)\leq \int_a^b f(x)\,dx\leq M(b-a).$$
\end{formulabox}

\begin{example}{Properties of Definite Integrals}{PropertiesDefiniteIntegrals}
Suppose $\ds{\int_a^b f(x)~dx=7}$ and $\ds{\int_a^b g(x)~dx=3}$. Find:
\begin{multicols}{2}
\begin{enumerate}
	\item	$\ds\int_a^b 2f(x)-3g(x)\,dx$.
	\item	$\ds\int_{b}^{a} 2g(x)\,dx$.
	\item	$\ds\int_a^a f(x)\cdot g(x)\,dx$.
	\item	$\ds\int_a^c f(x)~dx+\int_c^b f(x)\,dx$.
\end{enumerate}
\end{multicols}
\vspace{5mm}
\end{example}
\begin{solution}
\begin{enumerate}
	\item	$\ds\int_a^b 2f(x)-3g(x)\,dx=\ds 2\int_a^b f(x)\,dx-3\int_a^b g(x)\,dx=2(7)-3(3)=5$.
	\item	$\ds\int_{b}^{a} 2g(x)\,dx=\ds -2\int_{a}^{b} g(x)\,dx=-2(3)=-6$.
	\item	$\ds\int_a^a f(x)\cdot g(x)\,dx=0$.
	\item	$\ds\int_a^c f(x)\,dx+\int_c^b f(x)\,dx=\ds\int_a^b f(x)\,dx=7$.
\end{enumerate}
\end{solution}

We next evaluate a definite integral using three different techniques.

\begin{example}{Three Different Techniques}{ThreeDifferentTechniques}
Evaluate $\ds{\int_0^2 x+1~dx}$ by
\begin{enumerate}
\item Using FTC II (the shortcut)
\item Using the definition of a definite integral (the limit sum definition)
\item Interpreting the problem in terms of areas (graphically)
\end{enumerate}
\end{example}

\begin{solution}
1. The shortcut (FTC II) is the method of choice as it is the fastest.
Integrating and using the `\ifont{top minus bottom}' rule we have:
\begin{eqnarray*}
\int_0^2 x+1~dx&=&\left.\frac{x^2}{2}+x\right|_0^2\\
&=&\left[\frac{2^2}{2}+2\right]-\left[\frac{0^2}{2}+0\right]=4.
\end{eqnarray*}

2. We now use the definition of a definite integral.
We divide the interval $[0,2]$ into $n$ subintervals of equal width $\Delta x$, and from each interval choose a point $x_i^*$.
Using the formulas
$$\Delta x = \frac{b-a}{n}\qquad\mbox{and}\qquad x_i=a+i\Delta x,$$
we have
$$\Delta x = \frac{2}{n}\qquad\mbox{and}\qquad x_i=0+i\Delta x=\frac{2i}{n}.$$
Then taking $x_i^*$'s as right endpoints for convenience (so that $x_i^*=x_i$), we have:
\begin{eqnarray*}
\int_0^2 x+1~dx & = & \ds\lim_{n\to\infty}\sum_{i=1}^n f(x_i^*)\Delta x\\
\\
& = & \ds\lim_{n\to\infty}\sum_{i=1}^n f\left(\frac{2i}{n}\right) \frac{2}{n}\\
\\
& = & \ds\lim_{n\to\infty}\sum_{i=1}^n \left(\frac{2i}{n}+1\right) \frac{2}{n}\\
\\
& = & \ds\lim_{n\to\infty}\sum_{i=1}^n \left(\frac{4i}{n^2}+\frac{2}{n}\right)\\
\\
& = & \ds\lim_{n\to\infty}\left(\sum_{i=1}^n \frac{4i}{n^2}+\sum_{i=1}^n \frac{2}{n} \right)\\
\\
& = & \ds\lim_{n\to\infty}\left(\frac{4}{n^2}\sum_{i=1}^n i+\frac{2}{n}\sum_{i=1}^n 1 \right)\\
\\
& = & \ds\lim_{n\to\infty}\left(\frac{4}{n^2}\frac{n(n+1)}{2}+\frac{2}{n}n \right)\\
\\
& = & \ds\lim_{n\to\infty}\left(2+\frac{2}{n}+2 \right)\\
\\
& = & 4.
\end{eqnarray*}

3. Finally, let's evaluate the net area under $x+1$ from $0$ to $2$.
$$\includegraphics[width=5in]{images/int-ex}$$
Thus, the area is the sum of the areas of a rectangle and a triangle.
Hence,
\begin{eqnarray*}
\int_0^2 x+1~dx&=&\mbox{Net Area}\\
&=&\mbox{Area of rectangle} + \mbox{Area of triangle}\\
&=&(2)(1)+\frac{1}{2}(2)(2)\\
&=&4.
\end{eqnarray*}
\end{solution}

We next apply FTC to differentiate a function.

\begin{example}{Using FTC}{UsingFTC}
\exfont{Differentiate} the following function:
$$g(x)=\int_{-2}^x \cos(1+5t)\sin t\,dt.$$
\end{example}

\begin{solution}
We simply apply the Fundamental Theorem of Calculus directly to get:
$$g'(x)=\cos(1+5x)\sin x.$$
\end{solution}

Using the Chain Rule we can derive a formula for some more complicated problems.
We have:
$$\frac{d}{dx}\int_a^{v(x)}f(t)\,dt=f(v(x))\cdot v'(x).$$

Now what if the upper limit is constant and the lower limit is a function of $x$?
Then we interchange the limits and add a minus sign to get:
$$\frac{d}{dx}\int_{u(x)}^af(t)\,dt=-\frac{d}{dx}\int_a^{u(x)} f(t)\,dt=-f(u(x))\cdot u'(x).$$

Combining these two we can get a formula where both limits are a function of $x$.
We break up the integral as follows:
$$\int_{u(x)}^{v(x)} f(t)\,dt=\int_{u(x)}^a f(t)\,dt+\int_a^{v(x)}f(t)\,dt.$$
We just need to make sure $f(a)$ exists after we break up the integral.
Then differentiating and using the above two formulas gives:

\begin{formulabox}[FTC I + Chain Rule:]
$$\frac{d}{dx}\int_{{u(x)}}^{{v(x)}} f(t)\,dt=f({v(x)}){v'(x)}-f({u(x)}){u'(x)}$$
\end{formulabox}

Many textbooks do not show this formula and instead to solve these types of problems will use FTC I along with the tricks we used to derive the formula above.
Either method is perfectly fine.

\begin{example}{FTC I + Chain Rule}{FTCIChainRule}
Differentiate the following integral:
$$\int_{10x}^{x^2} t^3\sin(1+t) \,dt.$$
\end{example}

\begin{solution}
We will use the formula above.
We have $f(t)=t^3\sin(1+t)$, $u(x)=10x$ and $v(x)=x^2$.
Then $u'(x)=10$ and $v'(x)=2x$.
Thus,
\begin{eqnarray*}
\frac{d}{dx}\int_{10x}^{x^2} t^3\sin(1+t) \,dt&=&(x^2)^3\sin(1+(x^2))(2x)-(10x)^3\sin(1+(10x))(10)\\
\\
&=&2x^7\sin(1+x^2)-10000x^3\sin(1+10x)
\end{eqnarray*}
\end{solution}

\begin{example}{FTC I + Chain Rule}{FTCIChainRule2}
Differentiate the following integral with respect to $x$:
$$\int_{x^3}^{2x} 1+\cos t\,dt$$
\end{example}

\begin{solution}
Using the formula we have:
$$\frac{d}{dx}\int_{x^3}^{2x} 1+\cos t\,dt=(1+\cos(2x))(2)-(1+\cos(x^3))(3x^2).$$
\end{solution}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Opensolutionfile{solutions}[ex]
\section*{Exercises for Section \ref{sec:FTC}}

\begin{enumialphparenastyle}

%%%%%%%%%%
\begin{ex}
Evaluate $\ds \int_1^4 t^2+3t\,dt$
\begin{sol}
 $87/2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Evaluate $\ds \int_0^\pi \sin t\,dt$
\begin{sol}
 $2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Evaluate $\ds \int_1^{10} {1\over x}\,dx$
\begin{sol}
 $\ln(10)$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Evaluate $\ds \int_0^5 e^x\,dx$
\begin{sol}
 $\ds e^5-1$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Evaluate $\ds \int_0^3 x^3\,dx$
\begin{sol}
 $\ds 3^4/4$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Evaluate $\ds \int_1^2 x^5\,dx$
\begin{sol}
 $\ds 2^6/6 -1/6$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 Find the derivative of $\ds G(x)=\int_1^x t^2-3t\,dt$
\begin{sol}
 $\ds x^2-3x$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 Find the derivative of $\ds G(x)=\int_1^{x^2} t^2-3t\,dt$
\begin{sol}
 $\ds 2x(x^4-3x^2)$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 Find the derivative of $\ds G(x)=\int_1^x e^{t^2}\,dt$
\begin{sol}
 $\ds e^{x^2}$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 Find the derivative of $\ds G(x)=\int_1^{x^2} e^{t^2}\,dt$
\begin{sol}
 $\ds 2xe^{x^4}$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 Find the derivative of $\ds G(x)=\int_1^x \tan(t^2)\,dt$
\begin{sol}
 $\ds \tan(x^2)$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 Find the derivative of $\ds G(x)=\int_{10x}^{x^2} \tan(t^2)\,dt$
\begin{sol}
 $\ds 2x\tan(x^4)-10\tan(100x^2)$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Suppose $\int_{1}^{4}f(x)\,dx=2$ and $\int_{1}^{4}g(x)\,dx=7$. Find $\int_{1}^{4}(5f(x)+3g(x))\,dx$ and $\int_{1}^{4}(6-2f(x))\,dx$.
\begin{sol}
	31, 14
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Suppose $\int_{-2}^{5}f(x)\,dx=3$ and $\int_{1}^{5}f(x)\,dx=-2$. Find $\int_{-2}^{1}f(x)\,dx$.
\begin{sol}
	5
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
If $f$ is continuous on $[a,b]$, we define the average of $f(x)$ on $[a,b]$ to be
\begin{equation*}
\text{avg}_{[a,b]}(f)=\frac{1}{b-a}\int_{a}^{b}f(x)\,dx.
\end{equation*}
\begin{enumerate}
	\item	What is the average of $\sqrt{x}$ on the interval $[0,1]?$
	\item	If the average of $f(x)$ on $[0,2]$ and on $[2,5]$ are 6 and 4 respectively, then what is its average on $[0,5]$?
\end{enumerate}
\begin{sol}
\begin{enumerate}
	\item	2/3
	\item	24/5
\end{enumerate}
\end{sol}
\end{ex}

\end{enumialphparenastyle}