\lab{Data Visualization}{Data Visualization}
\label{lab:DataVis}
\objective{Correctly presenting and interpreting data with visualizations is both a science and an art.
% Indeed, effectively communicating complex relationships through data visualization is crucial to gaining the understanding needed to make important and difficult decisions.
In this lab we discuss and demonstrate the principles of good data visualization, including the key characteristics of good graphs, common mistakes, and how to avoid unethical visualizations.
\\ \indent We strongly recommend completing this lab as a Jupyter Notebook.}

\section*{The Importance of Visualizations} % =================================

Visualizations of data often reveal insights that may not immediately obvious from simple statistics.
The following data set, known as \emph{Anscombe's quartet}, is a famous example of the importance of graphing data.

\begin{table}[H]
\small{
\begin{tabular}{rr|rr|rr|rr}
    \multicolumn{2}{c|}{I}    & \multicolumn{2}{|c|}{II} &
    \multicolumn{2}{|c|}{III} & \multicolumn{2}{|c}{IV} \\
    \multicolumn{1}{c}{$x$}   & \multicolumn{1}{c|}{$y$} &
    \multicolumn{1}{|c}{$x$}  & \multicolumn{1}{c|}{$y$} &
    \multicolumn{1}{|c}{$x$}  & \multicolumn{1}{c|}{$y$} &
    \multicolumn{1}{|c}{$x$}  & \multicolumn{1}{c}{$y$} \\
    \hline
    10.0 & 8.04  & 10.0 & 9.14 & 10.0 & 7.46  & 8.0  & 6.58  \\
    8.0  & 6.95  & 8.0  & 8.14 & 8.0  & 6.77  & 8.0  & 5.76  \\
    13.0 & 7.58  & 13.0 & 8.74 & 13.0 & 12.74 & 8.0  & 7.71  \\
    9.0  & 8.81  & 9.0  & 8.77 & 9.0  & 7.11  & 8.0  & 8.84  \\
    11.0 & 8.33  & 11.0 & 9.26 & 11.0 & 7.81  & 8.0  & 8.47  \\
    14.0 & 9.96  & 14.0 & 8.10 & 14.0 & 8.84  & 8.0  & 7.04  \\
    6.0  & 7.24  & 6.0  & 6.13 & 6.0  & 6.08  & 8.0  & 5.25  \\
    4.0  & 4.26  & 4.0  & 3.10 & 4.0  & 5.39  & 19.0 & 12.50 \\
    12.0 & 10.84 & 12.0 & 9.13 & 12.0 & 8.15  & 8.0  & 5.56  \\
    7.0  & 4.82  & 7.0  & 7.26 & 7.0  & 6.42  & 8.0  & 7.91  \\
    5.0  & 5.68  & 5.0  & 4.74 & 5.0  & 5.73  & 8.0  & 6.89  \\
\end{tabular}}
\end{table}

Each section of Anscombe's quartet shares the following statistical properties:
\begin{itemize}
\setlength\itemsep{0em}
    \item The means are $9$ for $x$ and $7.5$ for $y$.
    \item The sample variances are $11$ for $x$ and $3.75$ for $y$.
    \item The correlation between $x$ and $y$ is $.816$.
    \item The linear least squares regression line is $y=\frac{1}{2}x+3$.
\end{itemize}

Despite these similarities, each section is quite different from the others.

\begin{problem} % Describe Anscombe's quartet.
The file \texttt{anscombe.npy} contains Anscombe's quartet in the same format as displayed above.
Plot each section of the quartet separately as a scatter plot.
Also plot the regression line $y = \frac{1}{2}x + 3$ on the domain $x\in[0,20]$ over each scatter plot.

Write a few sentences describing what makes each section unique.
\label{prob:anscombes-quartet}
\end{problem}

\section*{Principles of Good Data Visualization} % ============================

Understanding a data set through visualizations is an iterative process.
Examine the data, start with an initial visualization, then adjust the original visualization or create new ones.
Ask the following questions while searching for insights:
%
\begin{enumerate}
\item Does the visualization represent the data accurately?
\item Would a different visualization communicate more information?
\item Would visualizing a subset of the data provide more information?
\item Would transforming the data reveal a hidden pattern?
\end{enumerate}

Effectively visualizing data involves technical expertise combined with design knowledge.
No visualization is perfect, but every good visualization must contain the following essential elements.

\begin{itemize}
    \item \textbf{Clarity}.
    Good visualizations are as self-explanatory as possible.
    Always use specific titles and axis labels, and include units of measure.
    Use a legend or other annotations where appropriate.

    \item \textbf{Simplicity}.
    A good visualization communicates everything that it needs to, but nothing more.
    Anything in a plot that fails to communicate information or that misrepresents the data is called \emph{chartjunk}, a term coined by Edward Tufte.
    Make visualizations as simple, clean, and readable as possible.

    \item \textbf{Integrity}.
    Tell the truth, the whole truth, and nothing but the truth.
    It is usually alarmingly easy to manipulate a visualization so that it supports a specific agenda.
    Resist the temptation to morph a visualization into something that misrepresents the true nature of the data, even though that misrepresentation might support your hypotheses about the data.

    Every visualization should be presented together with information on who created it, where the data was obtained, how it was collected, whether it was cleaned or transformed, and whether there are conflicts of interest or possible biases present.
    Cite your sources.
\end{itemize}

This list could be expanded, but virtually every good data visualization principle fits into these three categories in one way or another.

\newpage

\section*{Improving Specific Types of Visualizations} % =======================

Data can be visualized in many forms and styles.
However, Most data sets are more naturally described with one type of visualization than another.
In the following sections, we explore how the plots we commonly use can be improved and refined.

\subsection*{Line Plots} % ----------------------------------------------------

A line plot connects ordered $(x,y)$ points with straight lines, and is therefore best for visualizing one or two ordered arrays, such as functional outputs over an ordered domain or a sequence of values over time.

When creating a line plot, consider the following details.
%
\begin{itemize}
    \item What do the axes represent? How should they be labeled?
    \item Would a linear scale or a logarithmic scale most clearly reveal patterns?
    \item What should the window limits be?
    \item Is the line an appropriate thickness and color?
    \item Should each point be distinctly marked, or is a smooth line preferable?
    \item Should multiple lines be in the same plot, or in separate subplots?
\end{itemize}

The \emph{Chebyshev polynomials} are a family of orthogonal polynomials that are recursively defined as follows.
\[T_0(x) = 1 \qquad T_1(x) = x \qquad\qquad T_{n+1} = 2xT_n(x) - T_{n-1}(x)\]
NumPy's \li{polynomial} module has a convenient tool for constructing these and other important polynomials.%
\footnote{\li{numpy.polynomial} also has tools for computing other important polynomial families, including the Legendre, Hermite, and Laguerre polynomials.}
However, plotting several the polynomials on top of each other, especially with a legend, results in a very cluttered visual.

\begin{lstlisting}
>>> import numpy as np
>>> from matplotlib import pyplot as plt
>>> % matplotlib inline                     # Display notebook plots inline.

# Plot the first 9 Chebyshev polynomials in the same plot.
>>> T = np.polynomial.Chebyshev.basis
>>> x = np.linspace(-1, 1, 200)
>>> for n in range(9):
...     plt.plot(x, T(n)(x), label="n = "+str(n))
...
>>> plt.axis([-1.1, 1.1, -1.1, 1.1])        # Set the window limits.
>>> plt.legend()
\end{lstlisting}

\begin{figure}[H] % Chebyshev polynomials (bad example).
    \centering
    \includegraphics[width=.7\linewidth]{figures/chebyshev_bad.pdf}
\end{figure}

This line plot can be improved in several easy ways.
%
\begin{enumerate}
    \item
    Use subplots to split the visualization into smaller, comparable pieces.
    Instead of using a legend, give each subplot a title.
    This method, called \emph{small multiples}, was made famous by Edward Tufte.
    \item Increase the line thicknesses to 2 or 3 (the default is 1).
    \item Remove extra tick marks and axis labels.
\end{enumerate}

Matplotlib's \li{plt.tick_params()}, summarized below, controls which tick marks and labels are displayed.
%
\begin{table}[H]
\begin{tabular}{r|c|l}
    Argument & Options & Description
    \\ \hline
    \li{axis} & \li{'x'}, \li{'y'}, \li{"both"} & Axis on which to operate. \\
    \li{which} & \li{"major"}, \li{"minor"}, \li{"both"} & Operate on major or minor ticks.\\
    \li{color} & Any Matplotlib color & Tick color. \\
    \li{labelcolor} & Any Matplotlib color & Tick label color. \\
    \li{bottom}, \li{top}, \li{left}, \li{right} & \li{"on"}, \li{"off"} & Turn ticks on or off. \\
    \li{labelbottom}, \li{labeltop}, & \li{"on"}, \li{"off"} & Turn tick labels on or off. \\
    \li{labelleft}, \li{labelright} & &
\end{tabular}
\end{table}

\begin{lstlisting}
>>> for n in range(9):
...     plt.subplot(3, 3, n+1)
...     plt.plot(x, T(n)(x), lw=2)
...     plt.axis([-1.1, 1.1, -1.1, 1.1])
...
...     # Turn off extra tick marks and axis labels.
...     plt.tick_params(which="both", top="off", right="off")
...     if n < 6:                   # Remove x-axis label on upper plots.
...         plt.tick_params(labelbottom="off")
...     if n % 3:                   # Remove y-axis label on right plots.
...         plt.tick_params(labelleft="off")
...     plt.title("n = "+str(n))
\end{lstlisting}

\begin{figure}[H] % Chebyshev polynomials (bad example).
    \centering
    \includegraphics[width=.7\linewidth]{figures/chebyshev_good.pdf}
\end{figure}

\begin{info} % LaTex with Matplotlib text.
Matplotlib titles and annotations can be formatted with \LaTeX, a system for creating technical documents.%
\footnote{See \url{http://www.latex-project.org/} for more information.}
% (this lab manual, for example, is written in \LaTeX).
To do so, use an `r' before the string quotation mark and surround the text with dollar signs.
For example, try replacing the final line of code in the previous example with the following line.

\begin{lstlisting}
...     plt.title(r"$T_{}(x)$".<<format>>(n))
\end{lstlisting}

The string's \li{<<format>>()} method inserts the input $n$ at the curly braces.
The title of the sixth subplot, instead of being ``n = 5,'' will then be ``$T_5(x)$.''
\end{info}

% TODO: Make sure the Bernstein polynomial notation is consistent with the book (it is consistent with Wikipedia currently)

\begin{problem} % Plot the Bernstein polynomials.
The $n+1$ Bernstein basis polynomials of degree $n$ are defined as follows.
\[b_{v,n} = {{n} \choose {v}} x^v (1-x)^{n-v},\qquad v = 0,\ 1,\ \ldots,\ n\]

Plot at least the first $10$ Bernstein basis polynomials ($n = 0,\ 1,\ 2,\ 3$) as small multiples on the domain $[0,1] \times [0,1]$.
Label the subplots for clarity, adjust tick marks and labels for simplicity, and set the window limits of each plot to be the same.
Consider arranging the subplots so that the rows correspond with $n$ and the columns with $v$.
\\(Hint: The constant ${{n} \choose {v}} = \frac{n!}{v!(n-v)!}$ is called the \emph{binomial coefficient} and can be efficiently computed with \li{scipy.special.binom()} or \li{scipy.misc.comb()}.)
\end{problem}

\subsection*{Scatter Plots} % -------------------------------------------------

A scatter plot draws $(x,y)$ points without connecting them.
Connecting the points would imply an order or relation between the points, so scatter plots are best for displaying data sets without a natural order, or where each point is a distinct, individual instance.

Consider the following questions when making a scatter plot.
Note that some of them are the same questions that should be asked when creating a line plot.
%
\begin{itemize}
    \item What do the axes represent? How should they be labeled?
    \item Would a linear scale or a logarithmic scale most clearly reveal patterns?
    \item What should the window limits be?
    \item Which marker is best? Are the markers an appropriate size and color?
\end{itemize}

A scatter plot can be drawn with either \li{plt.plot()} (specify a point marker such as \li{'.'}, \li{','}, \li{'o'}, or \li{'+'}) or \li{plt.scatter()}.
While \li{plt.plot()} is the more flexible function in general, \li{plt.scatter()} provides a few extra tools.
Most useful are the keywords \li{s} and \li{c}, which correspond to marker size and marker color, respectively.
Each keyword can either be a single entry or an array.
Using an array specifies the sizes or colors of each individual marker, allowing a scatter plot to have up to four dimensions of information.

\begin{comment}
% Window size, also should go later.
Figure \ref{fig:scatter_correlation} displays two scatter plots.
The first appears to have a weak correlation and the second appears to have a strong correlation.
However, the same data is being plotted and the only difference is the scale and window size.
Manipulating these can change your interpretation and should be done with careful consideration.
\end{comment}

Consider a collection of rectangular boxes where the lengths, widths, and heights are given.
A scatter plot of length against width mostly describes the sizes of the boxes; tying the third dimension (height) to the color of the points can provide the additional information.
Setting the marker size as the volume of the boxes also adds some depth to the visualization, though modifying both the color and the size might be considered overkill.

Since adjusting the marker size may lead to overlapping points, we specify the \emph{alpha value} of the color to make the markers slightly transparent.
The keyword \li{alpha} accepts a value in the interval $[0,1]$; 0 makes the markers completely transparent, while a $1$ makes the markers completely opaque.

Finally, as with heat maps and contour plots, a color bar can be added with \li{plt.colorbar()} to indicate the values that the colors represent.
This color bar can be given a label as well.

\begin{lstlisting}
>>> length, width, height = np.random.randint(1, 20, (3,50))

>>> plt.subplot(221)                # Plot length against width.
>>> plt.scatter(length, width, s=100)
>>> plt.grid()
>>> plt.ylabel("Width (inches)")
>>> plt.tick_params(labelbottom="off")
>>> plt.axis([0, 20, 0, 20])

# Continued on the next page.
\end{lstlisting}

\newpage

\begin{lstlisting}
>>> plt.subplot(222)                # Set the marker color to the height.
>>> plt.scatter(length, width, c=height, s=100)
>>> cbar = plt.colorbar()
>>> cbar.set_label("Height (inches)")
>>> plt.grid()
>>> plt.tick_params(labelbottom="off", labelleft="off")
>>> plt.axis([0, 20, 0, 20])

>>> plt.subplot(223)                # Set the marker size to half the volume.
>>> plt.scatter(length, width, s=length*width*height/2., alpha=.7)
>>> plt.grid()
>>> plt.xlabel("Length (inches)")
>>> plt.ylabel("Width (inches)")
>>> plt.axis([0, 20, 0, 20])

>>> plt.subplot(224)                # Use color and marker size together.
>>> plt.scatter(length, width, c=height, s=length*width*height/2., alpha=.7)
>>> cbar = plt.colorbar()
>>> cbar.set_label("Height (inches)")
>>> plt.grid()
>>> plt.tick_params(labelleft="off")
>>> plt.xlabel("Length (inches)")
>>> plt.axis([0, 20, 0, 20])
\end{lstlisting}

\begin{figure}[H] % Scatter plots.
\centering
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/scatter_1.pdf}
\end{subfigure}
%
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/scatter_2.pdf}
\end{subfigure}
\\
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/scatter_3.pdf}
\end{subfigure}
%
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/scatter_4.pdf}
\end{subfigure}
\end{figure}

In scatter plots, connecting the points into a line plot usual results in extreme clutter.
A regression line, however, highlights a pattern in the data without overshadowing the actual data points.%
\footnote{See the Least Squares lab (QR 2), especially Problems 2–--4, for a refresher on regression lines.}

\begin{problem} % Scatter plots with visualizations.
The file \texttt{MLB.npy} contains measurements from over 1,000 recent Major League Baseball players, compiled by UCLA.\footnote{See \url{http://wiki.stat.ucla.edu/socr/index.php/SOCR_Data_MLB_HeightsWeights}.}
Each row in the array represents a different player; the columns are the player's height (in inches), weight (in pounds), and age (in years), in that order.

Describe the data with at least one scatter plot.
Your graph(s) should demonstrate whether height, weight, or age correlated with each other in the MLB.
Consider plotting linear regression lines to indicate trends.
\end{problem}

\subsection*{Histograms} % ----------------------------------------------------

A histogram partitions an interval into a number of bins and counts the number of values that fall into each bin.
Histograms are ideal for visualizing how unordered data in a single array is distributed over an interval.
If the data are draws from a probability distribution, a histogram approximates the distribution's probability density function (PDF).

The following are important factors to consider when constructing a histogram.
%
\begin{itemize}
    \item What does the $x$-axis represent? How should it be labeled?
    \item Is a linear or a logarithmic scale more appropriate for the frequency axis?
    \item How many bins should be used? Over what range should the bins be?
    This is perhaps the most important question, as a histogram with too few or too many bins usually fails to give a clear view of the distribution.
\end{itemize}

% Use \li{plt.hist()} to create a histogram.
% The arguments \li{bins} and \li{<<range>>} specify the number of bins to draw and over what domain.

For most histograms, the most desirable insight is the general shape of the distribution.
The lines separating the bins, axis tick marks, and even the labels on the $y$ axis are all unnecessary (and potentially distracting) details.
Removing the lines between bins is easy: set the line width to $0$.
To ensure the gaps where the lines used to be are filled, specify the \li{histtype} as \li{"stepfilled"} (using \li{histtype="step"} will draw the outline without filling it in).
Finally, to get rid of extra markings on the axes, use \li{plt.tick_params()}.

A histogram can be converted into a line plot by using \li{np.histogram()}.
This function returns the number of values in each bin and the locations of the edges of the bins (in fact, \li{plt.hist()} employs this function).
Use the edges of the bins to calculate the center of the bins, then plot the bin centers against the frequency.

\begin{lstlisting}
# Get 10,000 random samples from a Beta distribution.
>>> data = np.random.beta(a=5, b=2, size=N)

>>> plt.subplot(131)                # Draw a regular histogram.
>>> plt.hist(data, bins=30)

>>> plt.subplot(132)                # Draw a clean histogram.
>>> plt.hist(data, bins=30, lw=0, histtype="stepfilled")
>>> plt.tick_params(left="off", top="off", right="off", labelleft="off")
# Continued on the next page.
\end{lstlisting}

\newpage

\begin{lstlisting}
>>> plt.subplot(133)                # Convert the histogram to a line plot.
>>> freq, bin_edges = np.histogram(data, bins=30)
>>> bin_centers = (bin_edges[:-1] + bin_edges[1:])/2.
>>> plt.plot(bin_centers, freq, 'k-', lw=4)
>>> plt.tick_params(left="off", top="off", right="off", labelleft="off")
\end{lstlisting}

\begin{figure}[H] % Cleaning up a histogram.
\centering
\begin{subfigure}{.325\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/histogram_2.pdf}
\end{subfigure}
%
\begin{subfigure}{.325\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/histogram_3.pdf}
\end{subfigure}
%
\begin{subfigure}{.325\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/histogram_4.pdf}
\end{subfigure}
\end{figure}

Finally, if the frequency domain is better visualized on a logarithmic scale, use \li{log=True} as an argument to \li{plt.hist()}.
This is the histogram equivalent of using \li{plt.semilogy()} for line or scatter plots.

\begin{warn} % Line plots versus histograms.
Line plots should \textbf{not} be used for data that has no natural progression.
For example, consider a collection of random draws from a statistical distribution.
In this case, a plain line plot is completely useless, because consecutive random draws are completely unrelated.
A histogram, on the other hand, provides an approximation of the distribution's probability density function.

\begin{lstlisting}
# Get 1,000 random samples from the standard normal distribution.
>>> data = np.random.normal(size=1000)

>>> plt.subplot(121)                # Regular line plot of the data.
>>> plt.plot(data)

>>> plt.subplot(122)                # Histogram of the data.
>>> plt.hist(data, bins=30)
\end{lstlisting}

\begin{figure}[H] % line plot versus a histogram.
\centering
\begin{subfigure}{.496\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/histogram_1_bad.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/histogram_1_good.pdf}
\end{subfigure}
\end{figure}
\end{warn}

\begin{problem} % Earthquake data
The file \texttt{earthquakes.npy} contains data from over 17,000 earthquakes between 2000 and 2010 that were at least a 5 on the Richter scale.\footnote{See \url{http://earthquake.usgs.gov/earthquakes/search/}.}
Each row in the array represents a different earthquake;
the columns are the earthquake's date (as a fraction of the year), magnitude (on the Richter scale), longitude, and latitude, in that order.

Because each earthquake is a distinct event, a good way to start visualizing this data might be a scatter plot of the years versus the magnitudes of each earthquake.

\begin{lstlisting}
>>> year, magnitude, longitude, latitude = np.load("earthquakes.npy").T
>>> plt.plot(year, magnitude, '.')
>>> plt.xlabel("Year")
>>> plt.ylabel("Magnitude")
\end{lstlisting}

\begin{figure}[H] % Bad visualization: earthquake data, year vs magnitude.
    \centering
    \includegraphics[width=.7\textwidth]{figures/earthquake.pdf}
\end{figure}

Unfortunately, this plot communicates very little information because the data is so cluttered.
Describe the data with two or three better visualizations, including line plots, scatter plots, and histograms as appropriate.
Your plots should clearly answer the following questions:
\begin{enumerate}
    \item How many earthquakes happened every year?
    \item How often do stronger earthquakes happen compared to weaker ones?
    \item Where do earthquakes happen? Where do the strongest earthquakes happen?
    \\(Hint: Use \li{plt.axis("equal")} to fix the aspect ratio, which may improve comparisons between longitude and latitude.)
\end{enumerate}
\end{problem}

\subsection*{Heat Maps and Contour Plots} % -----------------------------------

Let $f:\mathbb{R}^2\rightarrow\mathbb{R}$ be a scalar-valued function on a 2-dimensional domain.
A heat map of $f$ assigns a color to each $(x,y)$ point in the domain based on the value of $f(x,y)$, while a contour plot is a drawing of the \emph{level curves} of $f$.
The level curve corresponding to the constant $c$ is the set $\left\{(x,y)\mid c = f(x,y)\right\}$.
A filled contour plot, which colors in the sections between the level curves, is a discretized version of a heat map.

Consider the following questions when plotting a heat map or contour plot:
%
\begin{itemize}
    \item Is the domain sufficiently refined?
    \item Which color scheme is most clear and effective?
    \item How many / which contour lines should be drawn, if any?
    \item Is a linear or a logarithmic scale more appropriate for the color?
\end{itemize}

It is often sufficient to choose a fixed number of level curves.
In this case, the values of $c$ corresponding to the level curves are automatically chosen to be evenly spaced over the range of values of $f$ on the domain.
However, it is sometimes better to strategically specify the curves by providing a list of $c$ constants.

Consider the function $f(x,y) = y^2 - x^3 + x^2$ on the domain $[-\frac{3}{2}, \frac{3}{2}] \times [-\frac{3}{2}, \frac{3}{2}]$.
A heat map of $f$ reveals that it has a large basin around the origin.
Then since $f(0,0) = 0$, choosing several level curves close to $0$ more closely describes the topography of the basin.
The fourth subplot in the following example uses the curves with $c = -1,\ -\frac{1}{4},\ 0,\ \frac{1}{4},\ 1,$ and $4$.

\begin{lstlisting}
# Construct a 2-D domain with np.meshgrid() and calculate f on the domain.
>>> x = np.linspace(-1.5, 1.5, 200)
>>> X, Y = np.meshgrid(x, x.copy())
>>> Z = Y**2 - X**3 + X**2

>>> plt.subplot(221)                # Plot a heat map of f.
>>> plt.pcolormesh(X, Y, Z, cmap="viridis")
>>> plt.colorbar()

>>> plt.subplot(222)                # Plot a contour map with 6 level curves.
>>> plt.contour(X, Y, Z, 6, cmap="viridis")
>>> plt.colorbar()

>>> plt.subplot(223)                # Plot a filled contour map with 12 levels.
>>> plt.contourf(X, Y, Z, 12, cmap="viridis")
>>> plt.colorbar()

>>> plt.subplot(224)                # Plot specific level curves and a heat map.
>>> plt.contour(X, Y, Z, [-1, -.25, 0, .25, 1, 4], colors="white")
>>> plt.pcolormesh(X, Y, Z, cmap="viridis")
>>> plt.colorbar()
\end{lstlisting}

\begin{figure}[H] % Heat maps and contour plots.
\centering
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/heatmap_1.png}
\end{subfigure}
%
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/contour_1.pdf}
\end{subfigure}
\\
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/contour_2.pdf}
\end{subfigure}
%
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/heatmap_2.png}
\end{subfigure}
\end{figure}

There are two main kinds of color maps: sequential and diverging.
Sequential color maps, like \li{"hot"} and \li{"cool"}, transition very gradually between two colors; diverging color maps, like \li{"seismic"} and \li{"coolwarm"}, transition very rapidly from one color to another at the mean value.
When in doubt, use \li{"viridis"} or \li{"plasma"}, two specialized sequential color schemes.
For the complete list of Matplotlib color maps, see \url{http://matplotlib.org/examples/color/colormaps_reference.html}.

The color map can be changed to a logarithmic scale by using the keyword argument \li{norm=matplotlib.colors.LogNorm()} (this works for \li{plt.scatter()} as well).
As an example, consider the same $f$ defined above on the larger domain $[-6,6]\times [-6,6]$.
Log scaling can only be done on arrays of all positive values, so we visualize $|f|$.

\begin{lstlisting}
>>> from matplotlib.colors import LogNorm

>>> x = np.linspace(-6, 6, 200)
>>> X, Y = np.meshgrid(x, x.copy())
>>> Z = np.<<abs>>(Y**2 - X**3 + X**2)

>>> plt.subplot(121)              # Plot a regular heat map of |f|.
>>> plt.pcolormesh(X, Y, Z, cmap="plasma")
>>> plt.colorbar()

>>> plt.subplot(122)              # Plot a filled contour plot with log scaling.
>>> plt.contourf(X, Y, Z, 6, cmap="plasma", norm=LogNorm())
>>> plt.colorbar()
\end{lstlisting}

\begin{figure}[H] % Heat maps and contour plots.
\centering
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/heatmap_3.png}
\end{subfigure}
%
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/contour_3.pdf}
\end{subfigure}
\end{figure}

\begin{problem} % Contour map problem. TODO: ROSENBROCK!!!
The \emph{Rosenbrock function} is defined as follows.
\[f(x,y) = (1-x)^2 + 100(y-x^2)^2\]
The minimum value of $f$ is $0$, which occurs at the point $(1,1)$ at the bottom of a steep, banana-shaped valley of the function.

Use heat maps and contour plots to visualize the Rosenbrock function in such a way that the minimum value, and the valley that it lies in, is apparent.
Consider plotting the minimizer $(1,1)$ on top of the plot as well.
\end{problem}

\subsection*{Bar Charts} % ----------------------------------------------------

A bar chart plots categorical data in a sequence of bars.
They are best for small, discrete, one-dimensional data sets.
Use \li{plt.bar()} (vertical) or \li{plt.barh()} (horizontal) to create a bar chart in Matplotlib.
These functions receive the locations of each bar, then the height of each bar (as lists or arrays).

Consider the following questions when constructing a bar chart.

\begin{itemize}
    \item What are the different categories in the data?
    \item How should the data be sorted?
    \item Should the value axis have a linear or a logarithmic scale?
\end{itemize}

Horizontal bar charts are usually preferable to vertical bar charts because horizontal labels are easier to read than vertical labels.
Labels can be rotated, but it is better when the reader doesn't have to turn his or her head.

Data in a bar chart should also be sorted in a logical way.
Sort the categories by bar size, alphabetize the labels, or use some other intuitive ordering.

\newpage

\begin{lstlisting}
>>> labels = ["Lobster Thermador", "Baked Beans", "Crispy Bacon",
...             "Smoked Sausage", "Hannibal Ham", "Eggs", "Spam"]
>>> values = [10, 11, 18, 19, 20, 21, 22]
>>> positions = np.arange(len(labels))

>>> plt.subplot(121)
>>> plt.bar(positions, values, align="center")
>>> plt.xticks(positions, labels)

>>> plt.subplot(122)
>>> plt.barh(positions, values, align="center")
>>> plt.yticks(positions, labels)
\end{lstlisting}

\begin{figure}[H] % Horizontal vs. vertical bar chart.
\centering
\begin{subfigure}{.47\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/bar_1.pdf}
\end{subfigure}
%
\begin{subfigure}{.52\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/bar_2.pdf}
\end{subfigure}
\end{figure}

\section*{Practices to Avoid} % ===============================================

\subsection*{Bad Types of Visualization} % ------------------------------------

Some kinds of visualizations, especially for categorical data, are popular even though they interfere with the reader's ability to interpret the displayed information.
For example, it is more difficult to see the differences in a pie chart than on a bar chart since differences in area are more difficult to detect than differences in length.
By the same logic, bar chart with several bars per category are almost always better with the bars side by side rather than stacked.

\begin{figure}[H] % Pie chart --> Bar chart
\centering
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\textwidth]{figures/bad_pie_chart.png}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\textwidth]{figures/bar_2.pdf}
\end{subfigure}
\caption{Transforming the horrendous 3-D pie chart on the left into the horizontal bar chart on the right both simplifies and clarifies the information.}
\end{figure}

\subsection*{Misrepresenting Data} % --------------------------------------

To conclude, we return to the issue of integrity.
It is very easy to misrepresent the true nature of a set of data with a visualization.
Perhaps the easiest way to do so is with poorly chosen window limits or axis scales.
Consider the following example of a scatter plot, presented four different ways.
The fourth subplot is the only one that correctly shows the position of the points in relation to the origin and that has a reasonable scales on both axes.

\begin{lstlisting}
>>> x = np.linspace(5, 10, N) + np.random.normal(size=N)/3.
>>> y = .5*x + 4 + np.random.normal(size=N)/2.

>>> plt.subplot(221)                # Plot a default scatter plot of the data.
>>> plt.plot(x, y, 'o', ms=10)

>>> plt.subplot(222)                # Change the window limits so that the
>>> plt.plot(x, y, 'o', ms=10)          # data appears steep in the y direction.
>>> plt.xlim(-5,20)

>>> plt.subplot(223)                # Change the y-axis scale so that the data
>>> plt.semilogy(x, y, 'o', ms=10)      # appears flat in the y direction.

>>> plt.subplot(224)                # Plot the data with equal axis scales and
>>> plt.plot(x, y, 'o', ms=10)          # presenting the relation to the origin.
>>> plt.xlim(xmin=0)
>>> plt.ylim(ymin=0)
\end{lstlisting}

\begin{figure}[H] % Heat maps and contour plots.
\centering
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/dishonest_1.pdf}
\end{subfigure}
%
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/dishonest_2.pdf}
\end{subfigure}
\\
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/dishonest_3.pdf}
\end{subfigure}
%
\begin{subfigure}{.495\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/honest.pdf}
\end{subfigure}
\end{figure}

Do everything in your power to avoid visualizing data unethically.

\begin{problem}
The file \texttt{countries.npy} contains information from 20 different countries.
Each row in the array represents a different country; the columns are the 2015 population (in millions of people), the 2015 GDP (in billions of US dollars), the average male height (in centimeters), and the average female height (in centimeters), in that order.%
\footnote{
See \url{https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal)}, \url{http://www.averageheight.co/}, and \url{https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population}.}

The countries corresponding are listed below in order.

\begin{lstlisting}
countries = ["Austria", "Bolivia", "Brazil", "China",
            "Finland", "Germany", "Hungary", "India",
            "Japan", "North Korea", "Montenegro", "Norway",
            "Peru", "South Korea", "Sri Lanka", "Switzerland",
            "Turkey", "United Kingdom", "United States", "Vietnam"]
\end{lstlisting}

Visualize this data set with at least four plots, using at least one scatter plot, one histogram, and one bar chart.
Write a few sentences summarizing the major insights that your visualizations reveal.
\\(Hint: consider using \li{np.argsort()} and fancy indexing to sort the data for the bar chart.)
\end{problem}

\begin{comment} % TODO: Add sections on box plots and hexbin plots.

\newpage

\subsection*{Additional Material} % ===========================================

There are many software packages that facilitate the visual exploration of data.
One Python library is Glue (see \cite{glue}).

Some packages for making nicer looking plots include \li{Seaborn} and \li{prettyplotlib}.

For more about visualization of data, we highly recommend the following books and websites:

% TODO: get full refs for the following

\begin{itemize}
    \item \emph{How to Lie with Statistics} by Darrell Huff (1954)
    \item \emph{Envisioning Information} by Edward Tufte
    \item \emph{The Visual Display of Quantitative Information} by Edward Tufte (2nd edition)
    \item \emph{Beautiful Evidence} by Edward Tufte
    \item \emph{Now you see it} by Stephen Few
    \item \url{http://www.informationisbeautiful.net/}
\end{itemize}
\end{comment}


\section{Macroscopic observables}
\label{sec:analysis}%
Spatial distributions of charge and current densities can provide a better insight in the microscopic mechanisms of charge transport.
%
If $O$ is an observable which has a value $O_\alpha$ in a state $\alpha$, its ensemble average at time $t$ is a sum over all states weighted by the probability $P_\alpha$ to be in a state $\alpha$ at time $t$
%
\begin{equation}
\label{equ:ensemble}
\left< O \right> = \sum_{\alpha} O_\alpha P_\alpha.
\end{equation}
%
If $O$ does not explicitly depend on time, the time evolution of $\left< O \right>$ can be calculated as
\begin{equation}
\begin{split}
\frac{d \left< O \right>}{dt} = \sum_{ \alpha, \beta}
      \left[ P_\beta \Omega_{\beta \alpha} -
       P_\alpha \Omega_{\alpha \beta} \right]
      O_\alpha %\\
 %
      = \sum_{ \alpha, \beta}
      P_\beta \Omega_{\beta \alpha}
      \left[ O_\alpha - O_\beta \right] .
\end{split}
\end{equation}
%
If averages are obtained from KMC trajectories, $P_\alpha = s_\alpha / s$, where $s_\alpha$ is the number of Markov chains ending in the state $\alpha$ after time $t$, and $s$ is the total number of chains.

Alternatively, one can calculate time averages by analyzing a single Markov chain. If the total occupation time of the state $\alpha$ is $\tau_\alpha$ then
\begin{align}
\label{equ:time}
\overline{ O }
= \frac{1}{\tau} \sum_{\alpha} O_\alpha \tau_\alpha \,,
\end{align}
where $\tau = \sum_{\alpha} \tau_\alpha$ is the total time used for time averaging.

For ergodic systems and sufficient sampling times, ensemble and time averages should give identical results.
In many cases, the averaging procedure reflects a specific experimental technique. For example, an ensemble average over several KMC trajectories with different starting conditions corresponds to averaging over injected charge carriers in a time-of-flight experiment.  In what follows, we focus on the single charge carrier (low concentration of charges) case.

\subsection{Charge density}
\label{sec:occupation}

For a specific type of particles, the microscopic charge density of a site $i$ is proportional to the occupation probability of the site, $p_i$
\begin{equation}
 \rho_i = e p_i / V_i\, ,
\end{equation}
where,  for an irregular lattice, the effective volume $V_i$ can be obtained from a Voronoi tessellation of space. For reasonably uniform lattices (uniform site densities) this volume is almost independent of the site and a constant volume per cite, $V_i = V/N$, can be assumed.  In the macroscopic limit, the charge density can be calculated using a sxtpthing kernel function, i.e. a distance-weighted average over multiple sites. Site occupations $p_i$ can be obtained from \equ{ensemble} or  \equ{time} by using the occupation of site $i$ in state $\alpha$ as an observable.

If the system is in thermodynamic equilibrium, that is without sources or sinks and without circular currents (and therefore no net flux) a condition, known as detailed balance, holds
%
\begin{equation}
\label{equ:detailed_balance}
  p_j \omega_{ji} = p_i \omega_{ij},
\end{equation}
%
It can be used to test whether the system is ergodic or not by correlating $\log p_i$ and the site energy $E_i$. Indeed, if $\lambda_{ij} = \lambda_{ji}$ the ratios of forward and backward rates are determined solely by the energetic disorder, $\omega_{ji} / \omega_{ij} = \exp(-\Delta E_{ij} / k_\text{B} T)$ (see \equ{marcus}).

\subsection{Current}
\label{sec:vaverage}
\index{current}
If the position of the charge, $\vec{r}$, is an observable, the time evolution of its average $\left<\vec{r}\right>$ is the total current in the system
\begin{equation}
 \vec{J} = e \left< \vec{v} \right> = e \frac{d \left< \vec{r}
   \right>} {dt} = e \sum_{i, j} p_{j} \omega_{ji} ( \vec{r}_i -
 \vec{r}_j ) .
\label{equ:current_def}
\end{equation}
Symmetrizing this expression we obtain
\begin{equation}
  \vec{J} = \frac{1}{2} e \sum_{i, j} \left( p_{j}
  \omega_{ji} - p_{i} \omega_{ij} \right) \vec{r}_{ij} ,
 \label{equ:current}
\end{equation}
where $\vec{r}_{ij} = \vec{r}_{i} - \vec{r}_{j}$. Symmetrization ensures equal flux
splitting between neighboring sites and absence of local average fluxes in equilibrium. It allows to define a local current through site $i$ as\index{current!local}
\begin{equation}
  \vec{J_i} = \frac{1}{2} e \sum_{ j} \left( p_{j}  \omega_{ji} - p_{i} \omega_{ij} \right) \vec{r}_{ij} .
 \label{equ:site_current}
\end{equation}
A large value of the local current indicates that the site contributes considerably to the total current. A collection of such sites thus represents most favorable charge pathways~\cite{van_der_holst_modeling_2009}.

\subsection{Mobility and diffusion constant}
\label{sec:mobility}
\index{mobility}
For a single particle, e.g. a charge or an exciton, a zero-field mobility can be determined by studying particle diffusion in the absence of external fields. Using the particle displacement squared, $\Delta {\bm r}_i^2$, as an observable  we obtain
 \begin{equation}
\begin{split}
2d D_{\gamma \delta} =  \frac{d \left<  \Delta{r}_{i, \gamma} \Delta{r}_{i, \delta} \right>}{dt}
= \sum_{\substack{i,j \\ i\ne j}} p_j\omega_{ji}
 \left( \Delta r_{i,\gamma}\Delta r_{i,\delta} - \Delta r_{j,\gamma}\Delta r_{j,\delta} \right)
= \sum_{\substack{i,j\\ i\ne j}} p_j \omega_{ji} \left( r_{i,\gamma} r_{i,\delta} - r_{j,\gamma} r_{j,\delta} \right) \, .
\end{split}
\label{equ:diffusion}
\end{equation}
Here $\vec{r}_i$ is the coordinate of the site $i$, $D_{\gamma \delta}$ is the diffusion tensor, $\gamma, \delta = x,y,z$, and $d=3$ is the system dimension. Using the Einstein relation,
\begin{equation}
 D_{\gamma \delta} = k_\text{B}T \mu_{\gamma \delta} \, ,
\end{equation}
one can, in principle, obtain the zero-field mobility tensor $\mu_{\gamma \delta}$. \Equ{diffusion}, however, does not take into account the use of periodic boundary conditions when simulating charge dynamics. In this case, the simulated occupation probabilities can be compared to the solution of the Smoluchowski equation with periodic boundary conditions  (see the supporting information for details).

Alternatively, one can directly analyze time-evolution of the KMC trajectory and obtain the diffusion tensor from a linear fit to the mean square displacement, $\overline{ \Delta{r}_{i, \gamma} \Delta{r}_{i, \delta}} = 2d D_{\gamma \delta} t$.

The charge carrier mobility tensor, $\hat{\mu}$, for any value of the external field can be determined either from the average charge velocity defined in
\equ{current_def}
\begin{equation}
%\begin{split}
 \langle \vec{v} \rangle =  \sum_{i,j}  p_j  \omega_{ji}  (\vec{r}_i - \vec{r}_j) = \hat{\mu} \vec{F} \, ,
%\end{split}
\end{equation}
or directly from the KMC trajectory. In the latter case the velocity is calculated from the unwrapped (if periodic boundary conditions are used) charge displacement vector divided by the total simulation time. Projecting this velocity on the direction of the field $\vec{F}$ yields the charge carrier mobility in this particular direction. In order to improve statistics, mobilities can be averaged over several KMC trajectories and MD snapshots.


% Preamble.
\documentclass[12pt]{article}
\usepackage[margin=1.25in]{geometry}
\usepackage[fleqn]{amsmath}
\usepackage{textcomp}
\usepackage{gensymb}
\usepackage{amsfonts}
\usepackage{enumitem}
%\usepackage{tikz}  % Include for figures.
%\usepackage{subfiles}  % Include for subfiles.

%% Title macros.
\newcommand{\HOMEWORKNUM}{26}
\newcommand{\NAME}{D. Choi}
\newcommand{\DATE}{2020-07-01}

\title{\vspace{-2\baselineskip}MATH 225 - Homework \#\HOMEWORKNUM}
\author{\NAME}
\date{\DATE}

%% Formatting options.
%\pagenumbering{gobble}  % Include for single-page document.

% Document.
\begin{document}
\maketitle

\section*{1.}
\textit{Verify that}
\begin{equation*}
	y = C_1 e^x + C_2 x e^x
\end{equation*}
\textit{is a solution to the differential equation}
\begin{equation*}
	y^{\prime\prime} - 2y^\prime + y = 0
	.
\end{equation*}
First, find $y^\prime$ and $y^{\prime\prime}$.
\begin{align*}
	y &= C_1 e^x + C_2 e^x x, \\
	y^\prime &= C_1 e^x + C_2 e^x x + C_2 e^x, \\
	y^{\prime\prime} &= C_1 e^x + C_2 e^x x + 2C_2 e^x.
\end{align*}
Substitute the right-hand sides into the equation.
\begin{gather*}
	0 \overset{?}{=}
	y^{\prime\prime} - 2y^\prime + y
	\\
	0 \overset{?}{=}
	(C_1 e^x + C_2 e^x x + 2C_2 e^x)
	- 2(C_1 e^x + C_2 e^x x + C_2 e^x)
	+ (C_1 e^x + C_2 e^x x)
	\\
	0 \overset{?}{=}
	(2 - 2)(C_1 e^x)
	+
	(2 - 2)(C_2 e^x x)
	+
	(2 - 2)(C_2 e^x)
	\\
	0 =
	0
	.
\end{gather*}

\section*{2.}
Verify the solution to the differential equation.
\begin{enumerate}[label=(\alph*)]
	\item
	\textit{Equation: $y^\prime = 2x$.}
	\textit{Solution: $y = x^2$.}
	\begin{equation*}
		y^\prime = \frac{d}{dx} y = \frac{d}{dx} x^2 = 2x.
	\end{equation*}
	\newpage

	\item
	\textit{Equation: $y^\prime + 3y = 6x + 11$.}
	\textit{Solution: $y = e^{-3x} + 2x + 3$.}
	\begin{equation*}
		y^\prime = -3e^{-3x} + 2.
	\end{equation*}
	Thus,
	\begin{gather*}
		6x + 11 \overset{?}{=} y^\prime + 3y \\
		6x + 11 \overset{?}{=} (-3e^{-3x} + 2) + 3(e^{-3x} + 2x + 3) \\
		6x + 11 \overset{?}{=} (3 - 3)(e^{-3x}) + 3(2x) + 3(3) + 2 \\
		6x + 11 = 6x + 11.
	\end{gather*}

	\item
	\textit{Equation: $y^{\prime\prime} - 3y^\prime + 2y = 24e^{-2x}$.}
	\textit{Solution: $y = 3e^x - 4e^{2x} + 2e^{-2x}$.}
	\begin{align*}
		y &= 3e^x - 4e^{2x} + 2e^{-2x}, \\
		y^\prime &= 3e^x - 8e^{2x} - 4e^{-2x}, \\
		y^{\prime\prime} &= 3e^x - 16e^{2x} + 8e^{-2x}.
	\end{align*}
	Thus,
	\begin{align*}
		24e^{-2x} &\overset{?}{=} y^{\prime\prime} - 3y^\prime + 2y \\
		24e^{-2x} &\overset{?}{=}
		(3e^x - 16e^{2x} + 8e^{-2x}) \\
		&\quad - 3(3e^x - 8e^{2x} - 4e^{-2x}) \\
		&\quad + 2(3e^x - 4e^{2x} + 2e^{-2x}) \\
		24e^{-2x} &\overset{?}{=}
		(3 - 9 + 6)(e^x) + (-16 + 24 - 8)(e^{2x}) + (8 + 12 + 4)(e^{-2x}) \\
		24e^{-2x} &= 24e^{2x}.
	\end{align*}
\end{enumerate}

\end{document}


\section{Additive Schwarz methods}

\begin{intro}
  In this section, we study preconditioners, which are related to
  subspace decompositions of the space $V$ or its finite dimensional
  subspaces. We will develop the theory in an abstract way, but always
  keep the model problem~\eqref{eq:itintro:1} in mind when we do so. In
  particular, the subspaces chosen will be associated with either
  coarser mesh levels or with meshes on subdomains of $\Omega$.

  This section follows in part~\cite[Chapter 7]{BrennerScott02}. A
  more detailed discussion with extension of the methods developed
  here can be found in~\cite{ToselliWidlund05}
\end{intro}

\subsection{The abstract framework}

\begin{intro}
  Let $V$ be a Hilbert space with inner product $\scal(.,.)$ and let
  $a(.,.): V\times V$ be a symmetric and $V$-elliptic not necessarily
  bounded bilinear form. Let a set of subspaces
  $\{V_j\}_{j=1,\dots,J}$ of $V$ be chosen such that
  \begin{gather*}
    V = \sum_{j=1}^J V_j.
  \end{gather*}
  The sum is not required to be direct, that is, a vector $v\in V$ may
  have several decompositions $v = \sum \alpha_j v_j$ with
  $v_j\in V_j$. Methods below based on the subspaces $V_j$ are called
  \define{subspace correction} methods.

  Alternatively, we consider methods based on abstract auxiliary
  spaces $\hat V_j$, such that
  \begin{gather*}
    V = \sum_{j=1}^J R_j^T \hat V_j,
  \end{gather*}
  where $R_j^T$ is an operator embedding the auxiliary space
  $\hat V_j$ into $V$.  Such methods are called \define{auxiliary
    space methods}. Subspace correction and auxiliary space methods
  are equivalent by considering $V_j = R_j^T \hat V_j$.
\end{intro}

\begin{assumption}
  \label{lemma:schwarz:1}
  There are bilinear forms $a_j(.,.)$ defined on $V_j$ such that the weak formulation
  \begin{gather}
    \label{eq:schwarz:1}
    a_j(u_j,v_j) = f(v_j),
    \quad\forall v_j\in V_j,
  \end{gather}
  has a unique solution $u_j\in V_j$ for all $f\in V_j^*$. In the
  context of auxiliary space methods, we assume that there are
  bilinear forms $\hat a_j(.,.)$ defined on $\hat V_j$ such
  that the weak formulation
  \begin{gather}
    \label{eq:schwarz:1a}
    \hat a_j(u_j,v_j) = f(v_j),
    \quad\forall v_j\in \hat V_j,
  \end{gather}
  has a unique solution $u_j\in \hat V_j$ for all $f\in \hat V_j^*$.
\end{assumption}

\begin{Definition}{subspace-projection}
  \label{definition:schwarz:1}
  \index{Pj@$P_j$|see {Ritz projection}} Let the \define{subspace
    projection} operator $P_j: V \to V_j$ be defined such that
  $P_j u \in V_j$ is the unique (Assumption~\ref{lemma:schwarz:1})
  solution to the problem
  \begin{gather}
    \label{eq:schwarz:2}
    a_j(P_j u,v_j) = a(u,v_j),\quad\forall v_j\in V_j.
  \end{gather}
  Note that this is a projection only if $a_j(.,.)$ is the restriction
  of the bilinear form $a(.,.)$ to the subspace $V_j$. In this
  case,\index{A-orthogonal projection@$A$-orthogonal projection|see
    {Ritz projection}} we call $P_j$ the $A$-orthogonal projection or
  \define{Ritz projection} to $V_j$. If $a_j(.,.)$ is not the
  restriction of $a(.,.)$, the operator $P_j$ still maps into $V_j$,
  but it is not idempotent.

  Since $V_j$ is a subspace of $V$, we will also understand $P_j$ as
  an endomorphism of $V$ iteself. The left hand side of this equation
  induces an operator $A_j: V_j \to V_j^*$ by $A_j u_j = a_j(u_j,.)$.
\end{Definition}

\begin{Definition}{auxiliary-space-projection}
  \label{definition:schwarz:1a}
  Let the operator $\hat P_j: V \to \hat V_j$ be defined such that $\hat P_j u \in \hat V_j$ is
  the unique (Assumption~\ref{lemma:schwarz:1}) solution to the problem
  \begin{gather}
    \label{eq:schwarz:2a}
    \hat a_j(\hat P_j u,v_j) = a(u,R_j^T v_j),\quad\forall v_j\in \hat V_j.
  \end{gather}
  The left hand side of this equation induces an operator $\hat A_j:
  \hat V_j \to \hat V_j^*$ by $\hat A_j u_j = \hat a_j(u_j,.)$.
\end{Definition}

\begin{Lemma}{schwarz-ritz}
  \label{lemma:schwarz:ritz}
  If the local bilinear forms $a_j(.,.)$ are the restrictions of
  $a(.,.)$ to $V_j$, then the projections $P_j$ as mappings from $V$
  to itself are self-adjoint with respect to the $a(.,.)$-inner
  product and positive semi-definite. Furthermore, $P_j$ acts as
  identity on $P_j$ and there holds $P_j^2 = P_j$.
\end{Lemma}
\begin{proof}
  This is a well-known fact about orthogonal projections, which we
  will prove shortly.
  First, we note that by the uniqueness in
  Lemma~\ref{lemma:schwarz:1} $P_j u_j = u_j$
  for all $u_j\in V_j$. Thus, for all $u\in V$: $P_j P_j u = P_j u$.
  Let now $u,v\in V$ arbitrary. Then, there holds
  \begin{gather*}
    a(u, P_j v) = a(P_j u, P_j v) = a(P_j v, P_j u)
    = a(v, P_j u) = a(P_j u, v).
  \end{gather*}
  Furthermore,
  \begin{gather*}
    a(P_j u,u) = a(u, P_j u) = a(P_j u, P_j u)\ge 0,
  \end{gather*}
  since $a(.,.)$ is positive definite.
\end{proof}

\begin{Definition}{l2-projection}
  \label{definition:schwarz:1a}
  For subspace and auxiliary space methods, we define the operators
  \begin{gather}
    \begin{split}
      \Pi_j: V &\to V_j\\
      R_j: V &\to\hat V_j
    \end{split}
  \end{gather}
  \index{Pij@$\Pi_j$}
  such that $\Pi_j u\in V_j$ is the solution to the problem
  \begin{gather}
    \scal(\Pi_j u_j,v_j) = \scal(u,v_j),\quad\forall v_j\in V_j.
  \end{gather}

  For auxiliary spaces $\hat V_j$, we define the projection like
  operators  through
  \begin{gather}
    \scal(R_j u_j,v_j)_{\hat V_j} = \scal(u,R_j^T v_j),\quad\forall v_j\in \hat V_j.
  \end{gather}

  % \index{Pijt@$\Pi^T_j$}
  % We define its dual $\Pi_j^T: V_j^* \to V^*$ by
  % \begin{gather}
  %   \label{eq:schwarz:3}
  %   \scal(\Pi_j^T \phi_j,v)_{V^*\times V} =  \scal( \phi_j,\Pi_j
  %   v)_{V_j^*\times V_j}
  % \end{gather}
\end{Definition}

% \begin{todo}
%   Show that $\Pi^T$ is an orthogonal projection, and onto which space.
% \end{todo}

\begin{Lemma}{schwarz-ritz-representation}
  \label{lemma:schwarz:2}
  There holds
  \begin{gather}
    \label{eq:schwarz:15}
    A_j P_j = \Pi^T_j A.
  \end{gather}
\end{Lemma}

\begin{proof}
  Let $u,v\in V$ arbitrary. Let $v_j=\Pi_j v$. We rewrite
  equation~\eqref{eq:schwarz:2} as
  \begin{gather*}
    \scal(A_j P_j u, v)_{V^*\times V}
    = \scal(A_j P_j u, v_j)_{V^*\times V}
    = \scal(A u, v_j)_{V^*\times V}
    = \scal(\Pi^T_j A u, v)_{V^*\times V}.
  \end{gather*}
\end{proof}

\begin{Definition}{additive-schwarz}
  The \define{additive Schwarz preconditioner} for the operator $A$ associated
  with the symmetric, and $V$-elliptic bilinear form $a(.,.)$ with
  respect to the subspace decomposition $V_j$ is the mapping $B:
  V \to V^*$ such that
  \begin{gather}
    \label{eq:schwarz:4}
    B^{-1} = \sum_{j=1}^J P_j A^{-1}.
  \end{gather}
\end{Definition}

\begin{example}
  \label{example:schwarz:Jacobi}
  The \putindex{Jacobi method} may serve as a guiding example for the
  definition of these methods. To this end, let $V = \R^n$ with its
  Euclidean inner product $\scal(.,.)$. let $V_j =
  \operatorname{span}\{e_j\}$ be the space spanned by the $j$th unit
  vector. Let $A$ be a symmetric, positive definite matrix and $a(u,v)
  = v^T A u$. Then, equation~\eqref{eq:schwarz:2} becomes
  \begin{gather}
    \label{eq:schwarz:27}
    e_j^T A u_j = e_j^T A u
    \quad \Leftrightarrow \quad
    P_j u = u_j = \frac1{a_{j j}}(A u)_j.
  \end{gather}
  Since for this decomposition, the sum $V=\bigoplus V_j$ is direct,
  we obtain with $D=\operatorname{diag}(a_{11},\dots,a_{n n})$ the
  matrix representation
  \begin{gather*}
    (B^{-1} v)_j = \frac1{a_{j j}}(A A^{-1} v)_j = \frac1{a_{j j}} v_j
    \quad \Leftrightarrow \quad
    B^{-1} = D^{-1}.
  \end{gather*}
  We enter this preconditioner into the Richardson method in operator
  form~\eqref{eq:richardson:11} to obtain the iteration
  \begin{gather}
    \label{eq:schwarz:28}
    \begin{split}
      u^{(k+1)} &= u^{(k)} - \omega_k \sum_{j=1}^J P_j \bigl(u^{(k)} -
      A^{-1}f\bigr)\\
      &= u^{(k)} - \omega_k D^{-1} \bigl(A u^{(k)} - f\bigr).
    \end{split}
  \end{gather}
\end{example}

\begin{Lemma}{additive-positive-definite}
  \label{lemma:schwarz:3}
  If $A$ is symmetric and positive definite, so is $B^{-1}$ as defined
  in~\eqref{eq:schwarz:4}.
\end{Lemma}

\begin{proof}
  By Lemma~\ref{lemma:schwarz:2} and the fact that $P_j$ maps into $V_j$, we have that
  \begin{gather}
    \label{eq:schwarz:16}
    B^{-1} = \sum_{j=1}^J A_j^{-1} \Pi^T_j.
  \end{gather}
  Due to equation~\eqref{eq:schwarz:1}, $A_j$ inherits its symmetry
  and positive definiteness from $A$, and thus $A_j^{-1}$ is s.p.d.
  Therefore, for each term in this sum and arbitrary elements
  $\phi,\psi\in V^*$, we have
  \begin{gather*}
    \scal(A_j^{-1} \Pi^T_j \phi, \psi)_{V\times V^*}
    = \scal(A_j^{-1} \Pi^T_j \phi, \Pi^T_j \psi)_{V\times V^*}
    = \scal(\Pi^T_j \phi, A_j^{-1}\Pi^T_j \psi)
    = \scal(\phi, A_j^{-1}\Pi^T_j \psi).
  \end{gather*}
  The result now follows by linearity.
\end{proof}

\begin{Lemma}{additive-minimum}
  \label{lemma:schwarz:5}
  For $v\in V$ holds
  \begin{gather}
    \label{eq:schwarz:5}
    b(v,v) \equiv \scal(B v,v) = \min_{v=\sum v_j} \sum_{j=1}^J a(v_j,v_j),
  \end{gather}
  where the minimum is taken over all possible decompositions of $v$
  into a sum of elements $v_j\in V_j$ with $j=1,\dots,J$.
\end{Lemma}

\begin{proof}
  Since $B^{-1}$ is s.p.d., so is $B$. Therefore, $\scal(.,A_j^{-1}.)$
  is an inner product on $V_j^*$, for which the \putindex{Bunyakovsky-Cauchy-Schwarz
  inequality} holds. Thus, for an arbitrary decomposition $v=\sum v_j$
  with $v_j\in V_j$, the computation
  \begin{align*}
    b(v,v)
    &= \sum_{j=1}^J b(v, A_j^{-1} A_j v_j)
    = \sum_{j=1}^J \scal(\Pi^T_j B v, A_j^{-1} A_j v_j) \\
    &\le \sum_{j=1}^J \sqrt{\scal(\Pi^T_j B v, A_j^{-1}\Pi^T_j B v)}
    \sqrt{\scal(A_j v_j, A_j^{-1}A_j v_j)} \\
    & \le \sqrt{\sum_{j=1}^J \scal(\Pi^T_j B v, A_j^{-1}\Pi^T_j
      B v) }
    \sqrt{\sum_{j=1}^J \scal(A_j v_j,A_j^{-1}A_j v_j)} \\
    &= \sqrt{\scal(B v, {\sum A_j^{-1}\Pi^T_j} B v)}
    \sqrt{\sum_{j=1}^J \scal(A_j v_j, v_j)} \\
    &= \sqrt{b(v,v)} \sqrt{\sum_{j=1}^J a(v_j, v_j)},
  \end{align*}
  yields for arbitrary decompositions
  \begin{gather}
    \label{eq:schwarz:17}
    b(v,v) \le \sum_{j=1}^J a(v_j, v_j),
  \end{gather}
  and thus in particular, that the left hand side is bounded by the
  minimum of the right. Now we choose a special decomposition, showing
  that it cannot be less than the minimum. To this end, let
  \begin{gather}
    \label{eq:schwarz:18}
    v_j = A_j^{-1} \Pi^T_j B v.
  \end{gather}
  By Lemma~\ref{lemma:schwarz:2}, we have
  \begin{gather*}
    \sum v_j = \sum A_j^{-1} \Pi^T_j B v = B^{-1} B v = v.
  \end{gather*}
  Furthermore,
  \begin{align*}
    \sum_{j=1}^J \scal(A_j v_j, v_j)
    &= \sum_{j=1}^J \scal(A_j A_j^{-1} \Pi^T_j B v, A_j^{-1} \Pi^T_j
    B v) \\
    &= \sum_{j=1}^J \scal(\Pi^T_j B v, A_j^{-1} \Pi^T_j v) \\
    &= \scal(B v, \sum A_j^{-1} \Pi^T_j B v) = b(v, v).
  \end{align*}
\end{proof}

\begin{Theorem}{schwarz-equivalence}
  \label{theorem:schwarz:1}
  Let $A$ be s.p.d. and $B$ defined by
  equation~\eqref{eq:schwarz:4}. Then, the \putindex{spectral
    equivalence}~\eqref{eq:richardson:12} holds with
  positive constants
  \begin{gather}
    \label{eq:schwarz:19}
%    \begin{split}
    \Lambda(B,A) = \max_{v\in V} \frac{a(v,v)}{\min\limits_{v=\sum v_j}
      \sum\limits_{j=1}^J a(v_j, v_j)}
    ,\qquad
    \lambda(B,A) = \min\limits_{v\in V} \frac{a(v,v)}{\min\limits_{v=\sum v_j}
      \sum\limits_{j=1}^J a(v_j, v_j)}.
%    \end{split}
  \end{gather}
\end{Theorem}

\begin{proof}
  Here we use the fact, that $b(.,.)$ is an inner product on
  $V$ and that by
  \begin{gather*}
    b(B^{-1}A v,v) = a(v,v) = b(v, B^{-1}A v),
  \end{gather*}
  the operator $B^{-1}A$ is symmetric with respect to this inner
  product. Thus, the \putindex{Rayleigh quotient} qualifies to
  estimate the extremal eigenvalues, for instance,
  \begin{gather*}
    \Lambda(B^{-1}A)
    = \max_{v\in V} \frac{b(B^{-1}A v,v)}{b(v,v)}
    = \max_{v\in V} \frac{a(v,v)}{\min\limits_{v=\sum v_j}
      \sum\limits_{j=1}^J a(v_j, v_j)},
  \end{gather*}
  and the same for the minimum.
\end{proof}

\begin{remark}
  \label{note:schwarz:1}
  In order to estimate the condition number $\Lambda(B,A)/\lambda(B,A)$ of a Schwarz
  preconditioner, it is now sufficient to bound the two quotients
  in~\eqref{eq:schwarz:19} from above and below. In particular, in
  order to find a bound for $\Lambda(B,A)$, we have to find an estimate of
  the form
  \begin{gather}
    \label{eq:schwarz:23}
    a(v,v) \lesssim \min_{v=\sum v_j}\sum_{j=1}^J a(v_j, v_j),
  \end{gather}
  or in other words, $a(v,v)$ has to be bounded by the sum on the
  right for any decomposition $v=\sum v_j$. On the other hand, in
  order to bound $1/\lambda(B,A)$, we need an estimate in the opposite direction,
  where it is sufficient to find one decomposition $v=\sum v_j$ such
  that it holds. We reduce these
  conditions to the following two abstract assumptions, which guarantee that
  Theorem~\ref{theorem:schwarz:1} holds true.
\end{remark}

\begin{assumption}[Stable decomposition]
  \label{assumption:schwarz:stable-decomposition}
  For each $v\in V$ there is a decomposition
  \begin{gather*}
    v = \sum_{j=1^J} v_j, \qquad v_j\in V_j,
  \end{gather*}
  such that there holds
  \begin{gather}
    \label{eq:schwarz:24}
     \min_{v=\sum v_j}\sum_{j=1}^J a(v_j, v_j) \lesssim a(v,v).
  \end{gather}
\end{assumption}

\begin{assumption}[Strengthened Cauchy-Schwarz inequalities]
  \label{assumption:schwarz:1}
  \defindex{strengthened Cauchy-Schwarz inequalities}
  \index{E@$\mathcal E$} There is a symmetric $J\times J$-matrix
  $\mathcal E$ with entries $\epsilon_{ij} \in [0,1]$ and a constant
  $C$ independent of $J$, such that for the spectral radius
  $\rho(\mathcal E)$ there holds
  \begin{gather*}
    \rho(\mathcal E) \le C
  \end{gather*}
  and for all $1 \le i,j \le J$, $v_i \in V_i$ and $v_j \in V_j$ there
  holds
  \begin{gather}
    \label{eq:schwarz:25}
    \left| a(v_i, v_j)\right|
    \le \epsilon_{ij} \sqrt{a(v_i,v_{\phantom ji})} \sqrt{a(v_j,v_j)}.
  \end{gather}
\end{assumption}

\begin{remark}
  Inequality~\eqref{eq:schwarz:25} with $\epsilon_{ij} \equiv 1$ holds
  trivially by the regular \putindex{Bunyakovsky-Cauchy-Schwarz
    inequality}.  But for such a matrix, the spectral radius is
  $J$. As the following lemma will reveal, it is necessary to obtain
  $\rho(\mathcal E)$ independent of $J$ to obtain
  estimate~\eqref{eq:schwarz:23}.
\end{remark}

\begin{Lemma}{schwarz-maximum}
  \label{lemma:schwarz:7}
  Let the estimate~\eqref{eq:schwarz:25} hold. Then,
  estimate~\eqref{eq:schwarz:23} holds with the constant
  $\rho(\mathcal E)$.
\end{Lemma}

\begin{proof}
  Let $v\in V$ and its decomposition $v=\sum v_j$ with $v_j\in V_j$ be
  chosen arbitrarily. Then,
  \begin{multline*}
    a(v,v)
    = a\left(\sum_i v_i, \sum_j v_j\right)
    = \sum_{i,j=1}^J a(v_i, v_j)
    \le \sum_{i,j=1}^J \epsilon_{ij} \sqrt{a(v_{\vphantom{j}i},v_i)} \sqrt{a(v_j,v_j)}.
  \end{multline*}
  The latter sum corresponds to a matrix-vector product of the form
  $\vec x^T \mathcal E \vec x$, where the entries of $x$ are of the
  form $\sqrt{a(v_i,v_i)}$. Since $\mathcal E$ is symmetric positive
  definite, this product can be estimated by $\rho(\mathcal E) |\vec
  x|^2$, and thus
  \begin{gather}
    \label{eq:schwarz:38}
    a(v,v) \le \rho(\mathcal E) \sum_{j=1}^J a(v_j,v_j).
  \end{gather}
\end{proof}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Two-level additive Schwarz preconditioner}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{intro}
  This preconditioner is in the class of \putindex{domain
    decomposition} methods. The attribute \putindex{two-level} refers
  to the fact that we are considering finite element discretizations
  of~\eqref{eq:itintro:1} on two finite element meshes, the
  \putindex{fine mesh} $\T_h$ on which we desire to compute the
  solution, and the auxiliary \putindex{coarse mesh} $\T_H$. Both
  meshes cover the whole domain $\Omega$ (see
  Figure~\ref{fig:schwarz:ddmeshes}), and each cell of the coarse mesh
  is the union of cells of the fine mesh ($4\times 4$ fine cells in
  the figure).

  In addition to these two meshes, we introduce subdomains
  $\Omega_1,\Omega_2,\dots,\Omega_J$ of $\Omega$ such that each
  $\Omega_j$ is the union of cells in $\T_h$. We require that those
  subdomains overlap each other like the three examples in
  Figure~\ref{fig:schwarz:ddmeshes} on the right. A more precise
  definition of the required overlap follows.
\end{intro}

\begin{figure}[tp]
  \centering
  \begin{tikzpicture}
    \draw[step=.25cm] (-2,-2) grid (2,2);
    \draw[step=1cm,thick] (-2,-2) grid (2,2);
    \draw[step=1cm,thick] (3,-2) grid (7,2);
    \draw[step=.25cm] (3.74,-1.25) grid (5.25,0.25);
    \draw[step=.25cm,red] (3,-2) grid (4.25,-0.75);
    \draw[step=.25cm,green] (4.74,-0.25) grid (7,1.25);
  \end{tikzpicture}
  \caption{Fine mesh and coarse mesh (left) for overlapping domain
    decomposition. Examples for a subdomain decomposition on the
    right.}
  \label{fig:schwarz:ddmeshes}
\end{figure}

\begin{definition}{dd-overlap}
  \label{definition:schwarz:overlap}
  A covering of $\Omega$ with subdomains $\Omega_j$ is called
  \define{overlapping} with minimal \define{overlap} $\delta$, if for
  each $\Omega_j$ and all $x\in\Omega_j$ holds:
  \begin{gather*}
    \operatorname{dist}(x,\partial\Omega_J\setminus\partial\Omega) <
    \delta\;
    \Rightarrow\; \exists k\neq j: x\in \Omega_k.
  \end{gather*}
\end{definition}

\begin{Definition}{dd-finite-covering}
  \label{definition:schwarz:finite-covering}
\defindex{NO@$N_O$}
  \defindex{finite covering}
  We say that a family of coverings is finite, if
  there is a constant $N_O$ independent of $\T_h$ and the number of
  subdomains, such that for each $j$ the intersection
  $\Omega_j\cap\Omega_k$ is nonempty for at most $N_O$ subdomains
  $\Omega_k$.
\end{Definition}

\begin{Definition}{partition-of-unity}
  A smooth \define{partition of unity} with respect to the subdomains
  $\Omega_1,\Omega_2,\dots,\Omega_J$ of $\Omega$ is a set of
  nonnegative functions $\{\phi_1,\dots,\phi_J\}\subset
  C^\infty(\overline\Omega)$ such that
  \begin{xalignat}2
    \label{eq:schwarz:6}
    \phi_j(x) &  = 0
    & \forall x & \in \Omega\setminus\Omega_j, \quad j=1,\dots,J
    \\
    \label{eq:schwarz:7}
    \sum_{j=1}^J \phi_j(x) &= 1
    & \forall x&\in\overline\Omega.
  \end{xalignat}
  Similarly, we can define partitions of unity in $H^1(\Omega)$ or
  partitions of unity which are piecewise $C^1$.

  Furthermore, we assume that there is a positive constant $\delta$,
  called \define{overlap}, such that for all $j=1,\dots,J$ there holds
  \begin{gather}
    \label{eq:schwarz:8}
    \norm{\nabla\phi j}_{L^\infty(\Omega)} \lesssim \frac 1\delta,
  \end{gather}
  where the implicit constant is independent of $h$, $\delta$ and $J$.
\end{Definition}

\begin{remark}
  The term overlap for $\delta$ is justified by the following
  consideration. Let $x \in \Omega_j$ be a point which is not in any
  other $\Omega_i$. Then, $\phi_j(x) = 1$. If~\eqref{eq:schwarz:8} is
  to hold, then it is necessary that
  $\operatorname{dist}(x,\partial\Omega_J) \ge \delta$ (up to a
  constant, but this constant is already
  in~\eqref{eq:schwarz:8}). Thus, the points of distance less than
  $\delta$ from $\partial\Omega_j$ must be elements of another
  subdomain as well, which is then said to overlap with $\Omega_j$.
\end{remark}

\begin{example}
  \label{example:schwarz:2}
  On uniform meshes, overlapping subdomains with an overlap of
  $\delta = n h$ with $n=1,2,\ldots$ can be achieved easily by the
  following procedure:
  \begin{enumerate}
  \item Begin with a non overlapping subdivision $\{\Omega_j^0\}$,
    for instance aligned with the mesh cells of $\T_H$.
  \item Add all cells that share at least a vertex with a cell in
    $\Omega_j^0$ to obtain a domain $\Omega_j^1$. After this
    procedure, two neighboring domains will overlap by two cells,
    resulting in $\delta = 2h$.
  \item Repeat this procedure to obtain larger overlaps.
  \end{enumerate}

  On the resulting partitions, a partition of unity in $H^1$ can be
  constructed with piecewise linear (bilinear on quadrilaterals)
  functions. For instance for $\Omega_j^1$ this function is
  constructed as follows:
  \begin{enumerate}
  \item Choose $\phi_j(x) = 1/2$ in all vertices on $\partial \Omega_j^0$.
  \item Choose $\phi_j(x) = 0$ in all vertices on $\partial \Omega_j^1$ and outside
    $\Omega_j^1$.
  \item Choose $\phi_j(x) = 1$ in all remaining vertices inside
    $\Omega_j^0$.
    \item Connect these values by linear (on simplicial meshes), bilinear
      (quadrilateral meshes) of trilinear (hexahedral meshes)
      polynomials inside each mesh cell $T\in \T_h$.
  \end{enumerate}
  This partition of unity achieves the estimate~\eqref{eq:schwarz:8}
  with a constant of $1/2$.
\end{example}

\begin{notation}
  \label{par:schwarz:1}
  The solution space of our problem is the space $V=V_h$ given by the
  finite element space on the mesh $\T_h$. We define finite element
  spaces on $\Omega_j$ by
  \begin{gather}
    \label{eq:schwarz:9}
    V_j = \bigl\{ v\in V_h \big| \forall x\in\Omega\setminus\Omega_j :
    v(x) =0\bigr\}.
  \end{gather}
  Additionally, we define the space $V_0 \equiv V_H$ as the finite
  element space on the coarse mesh $\T_H$.  Since the meshes are
  nested, $V_H$ is indeed a subspace of $V_h$. Thus, we obtain a
  decomposition of $V_h$ into $J+1$ subspaces
  \begin{gather*}
    V_h = V_0 + \sum_{j=1}^J V_j,
  \end{gather*}
  where the last $J$ are associated with the subdomains. In fact,
  Lemma~\ref{lemma:schwarz:4} below states that already the spaces
  $V_1$ to $V_J$ are sufficient to span $V_h$. Nevertheless, the
  coarse grid space plays a crucial role in the efficiency of the
  method due to Lemma~\ref{lemma:schwarz:stable-decomposition}.
\end{notation}

\begin{Definition}{two-level-additive}
  Let the spaces $V_j$, $j=0,\dots,J$ be defined as above. Then, the
  \define{two-level additive Schwarz preconditioner} is defined as
  \begin{gather}
    \label{eq:schwarz:10}
    B^{-1}_{\text{TLS}} = \sum_{j=0}^J P_j A^{-1} = \sum_{j=0}^J A_j^{-1} \Pi^T_j,
  \end{gather}
  where $P_j$ is defined according to~\eqref{eq:schwarz:2} and $A_j:
  V_j\to V_j^*$ by
  \begin{gather}
    \scal(A_j u_j, v_j)_V = a(u_j, v_j),\quad\forall u_j, v_j \in V_j.
  \end{gather}
\end{Definition}

\begin{Notation}{H-equals-0}
  In order to simplify notation, we have assigned index zero to
  $V_H$. Thus, sums in future terms may either start at one, summing
  over subdomains, or at zero, summing over all subspaces.
\end{Notation}

\begin{Lemma}{schwarz4}
  \label{lemma:schwarz:4}
  There holds
  \begin{gather}
    \label{eq:schwarz:11}
    V_h = \sum_{j=1}^J V_j.
  \end{gather}
\end{Lemma}

\begin{proof}
  Let $I_h: C(\overline\Omega) \to V_h$ be the interpolation operator
  of the finite element space. Then, for any given $v\in V_h$ define
  $v_j = I_h(\phi_j v)$, where $\phi_j$ is the function associated to
  $\Omega_j$ of a partition of unity for $\Omega_1,\dots,\Omega_J$.

  By definition of $\phi_j$, there holds $\phi_j v = 0$ on
  $\Omega\setminus\Omega_j$. Furthermore, we assumed that a mesh cell
  of $\T_h$ is either completely in $\Omega_j$ or completely in its
  complement. Since nodal values of a cell are located in the cell
  itself, this implies that $I_h (\phi_j v) = 0$ on
  $\Omega\setminus\Omega_j$. Therefore, $I_h (\phi_j v) \in V_j$.

  On the other hand, we use the linearity of the interpolation
  operator to obtain
  \begin{gather*}
    \sum_{j=1}^J v_j = \sum_{j=1}^J I_h(\phi_j v)
    = I_h\left(v\sum_{j=1}^J \phi_h\right)
    = I_h v = v,
  \end{gather*}
  thus, the $v_j$ are indeed a decomposition of $v$. Since $v\in V_h$
  was chosen arbitrarily, the lemma is proven.
\end{proof}

\begin{Lemma}{finite-overlap-cauchy-schwarz}
  \label{lemma:schwarz:8}
  Let the covering $\{\Omega_j\}_{j=1,\dots,J}$ for $\Omega$ be finite
  according to
  Definition~\ref{definition:schwarz:finite-covering}. Then, the
  \putindex{strengthened Cauchy-Schwarz inequalities}~\eqref{eq:schwarz:25} hold
  with a spectral radius
  \begin{gather}
    \label{eq:schwarz:26}
    \rho(\mathcal E) \le N_O.
  \end{gather}
\end{Lemma}

\begin{proof}
  The term $a(v_i, v_j)$ is nonzero only if the supports of the two
  functions have a nonempty intersection. Accordingly, for each index
  $i$ only a maximum of $N_O$ of the coefficients $\epsilon_{ij}$ are
  nonzero. We set these equal to one and use Gershgorin's theorem to
  estimate the greatest eigenvalue.
\end{proof}

\begin{Lemma}{two-level-bound1}
  \label{lemma:schwarz:6}
  Let $v_j\in V_j$ for $j=0,\dots,J$ be a composition of $v\in V_h$
  such that
  \begin{gather*}
    v=\sum_{j=0}^J v_j.
  \end{gather*}
  Then,
  \begin{gather}
    \label{eq:schwarz:12}
    a(v,v) \lesssim \sum_{j=0}^J a(v_j, v_j),
  \end{gather}
  where the implicit constant does not depend on $h$, $H$, or $J$.
\end{Lemma}

\begin{proof}
  First a note: the inequality would be obvious, if $V_h$ was a direct
  sum of the spaces $V_j$, and it would hold with a constant of one if
  they were mutually orthogonal. Thus, we have to show some kind of
  orthogonality between the spaces.

  We start out by stating that
  \begin{align*}
    a(v,v) &= a\left(v_0+\sum_{j=1}^J v_j, v_0+\sum_{j=1}^J v_j\right)
    \\
    &\le 2 \left(a(v_0, v_0) + a\!\left(\sum_{j=1}^J v_j,\sum_{j=1}^J
        v_j\right)\right)
    \\
    &= 2 \left(a(v_0, v_0) + \sum_{j,k=1}^J a(v_j,v_k)\right) \\
    & \le 2 a(v_0, v_0) + 2 N_O \sum_{j=0}^J a(v_j, v_j),
  \end{align*}
  where the last inequality is due to Lemma~\ref{lemma:schwarz:7} and
  Lemma~\ref{lemma:schwarz:8}. Since $N_O$ is assumed independent of
  $h$, $H$, and $J$, the lemma is proven.
\end{proof}

\begin{Lemma}{stable-decomposition}
  \index{stable decomposition}
  \label{lemma:schwarz:stable-decomposition}
  For each $v\in V_h$ there exists a decomposition $v=\sum_{j=0}^J
  v_j$ with $v_j\in V_j$, such that
  \begin{gather}
    \label{eq:schwarz:13}
    \sum_{j=0}^J a(v_j, v_j)
    \lesssim \left(1+\frac H\delta\right)^2 a(v,v).
  \end{gather}
\end{Lemma}

\begin{proof}
  Let $\tilde I_H: H^1_0(\Omega) \to V_H$ be an interpolation operator
  continuous on $H^1(\Omega)$, for instance the interpolation operator
  by Clement or the one by Scott and Zhang. For a given function
  $v\in V_h$, let $v_H = \tilde I_H v$. Then
  \begin{gather}
    \label{eq:schwarz:20}
    \begin{split}
      |v_H|_1 &\lesssim |v|_1 \\
      \norm{v-v_H}_0 &\lesssim H |v|_1.
    \end{split}
  \end{gather}
  From~\eqref{eq:schwarz:20} there holds
  \begin{gather}
    \label{eq:schwarz:22}
    a(v_H,v_H) = |v_H|_1^2 \lesssim |v|_1^2 = a(v,v).
  \end{gather}
  Let $w=v-v_H$. Using a partition of unity $\{\phi_j\}$ for the
  subdomains $\{\Omega_j\}$ and the nodal interpolant $I_h$ as in the
  proof of Lemma~\ref{lemma:schwarz:4}, and let
  \begin{gather*}
    v_j = I_h(\phi_j w).
  \end{gather*}
  Thus, $v = \sum v_j$. We point out, that we can use the nodal
  interpolant, since $w$ is a finite element function on $\T_h$ and
  $\phi_j$ is either smooth or piecewise polynomial. For the remainder
  of this proof, we will assume the piecewise polynomial case (see
  Example~\ref{example:schwarz:2}) and leave the arguments for a
  smooth function $\phi_j$ to the reader.

  The interpolation operator is exact for polynomials of degree $k$
  (assuming such an order for the finite element being
  used). Therefore,
  \begin{gather*}
    a(v_j,v_j) = |v_j|_1^2 \lesssim |\phi_j w|_1^2
    \lesssim \norm{\nabla \phi_j w}_0^2 + \norm{\phi_j \nabla w}_0^2.
  \end{gather*}
  Using the properties of the partition of unity, we obtain
  \begin{gather*}
    a(v_j,v_j) \lesssim \frac1{\delta^2} \norm{\chi(\Omega_j) w}_0^2
    + |\chi(\Omega_j)w|_1^2.
  \end{gather*}
  Summing up yields
  \begin{gather}
    \label{eq:schwarz:21}
    \begin{split}
      \sum_{j=1}^J a(v_j, v_j)
      &\lesssim \sum_{j=1}^J \left(
        \frac1{\delta^2} \norm{\chi(\Omega_j) w}_0^2
        + |\chi(\Omega_j)w|_1^2\right) \\
      &\le N_O \left(\frac1{\delta^2}\norm{w}_0^2 + |w|_1^2\right) \\
      &=  N_O \left(\frac1{\delta^2} \norm{v-v_H}_0^2 + |v-v_H|_1^2\right)
      \\
      & \lesssim \frac{H^2}{\delta^2} |v|_1^2 + |v|_1^2 \\
      &= \left(1+\frac{H^2}{\delta^2}\right) a(v,v).
    \end{split}
  \end{gather}
  The estimate~\eqref{eq:schwarz:13} now follows
  from~\eqref{eq:schwarz:22} and~\eqref{eq:schwarz:21}.
\end{proof}

\begin{Theorem}{two-level-additive-convergence}
  \label{theorem:schwarz:two-level-convergence}
  Under the assumptions made so far in this section, there holds
  \begin{gather}
    \label{eq:schwarz:14}
    \kappa(B^{-1}_{\text{TLS}} A_h)
    = \frac{\Lambda(B_{\text{TLS}}, A_h)}{\lambda(B_{\text{TLS}},
      A_h)}
    \lesssim \left(1+\frac H\delta\right)^2,
  \end{gather}
  where the implicit constant is independent of $h$, $\delta$, $H$, and $J$.
\end{Theorem}

\begin{proof}
  The proof follows Note~\ref{note:schwarz:1}. Indeed,
  Lemma~\ref{lemma:schwarz:6} proves inequality~\eqref{eq:schwarz:23}
  and Lemma~\ref{lemma:schwarz:stable-decomposition}
  proves~\eqref{eq:schwarz:24}.
\end{proof}

\begin{remark}
  We have constructed a preconditioner $B_{\text{TLS}}$ for the finite
  element discretization of the Poisson problem~\eqref{eq:itintro:1}
  such that the preconditioned system has a bounded condition number
  independent of the mesh size $h$. Thus, a Richardson iteration or a
  conjugate gradient method using this preconditioner will reduce the
  error by a certain amount within a fixed number of steps.

  Closer inspection of the estimate~\eqref{eq:schwarz:14} in view of
  Example~\ref{example:schwarz:2} reveals a problem though: typically,
  $\delta$ is of the order of $h$, such that the estimate becomes
  \begin{gather*}
    \kappa(B^{-1}_{\text{TLS}} A_h) \lesssim \left(1+\frac H h\right)^2.
  \end{gather*}
  If we choose $H$ constant, this is exactly as bad as the condition
  number of $A$ itself. Under mild further assumptions, the square on
  the right hand side can be avoided~\cite{DryjaWidlund94}, which
  is an improvement compared to the operator without preconditioning,
  but is not uniform with respect to $h$. Therefore, we are left with
  two options:
  \begin{enumerate}
  \item Increase the overlap such that it is $\mathcal O(H)$. This
    procedure yields a \putindex{uniform preconditioner}, but it
    introduces a problem: when refining $h$, more and more
    cells belong to several subdomains and thus computations on them
    have to be performed several times. Therefore, the effort per
    preconditioning step is increased considerably.
  \item Keep the overlap at a small multiple of $h$ and choose $H$
    such that $H/h$ is bounded by a constant. Then, the preconditioner
    remains uniform, the overlap remains small. This way, the
    difficulty has been transferred to the coarse grid problem on
    $\T_H$, since now this problem becomes more and more difficult to
    solve, when $h$ decreases.
  \end{enumerate}

  While the problems of the first option above are inherent and
  unavoidable, the second option is at least seemingly optimal and
  more creativity may be invested into the solution of the coarse grid
  problem. Therefore, the latter is usually preferred. The coarse grid
  problem then leads to the idea of multigrid methods, which will be
  dealt with in Chapter~\ref{cha:iteration:multigrid-methods}.
\end{remark}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Multiplicative Schwarz methods}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{example}
  If we write the Jacobi method in equation~\eqref{eq:schwarz:28} for
  each line, we get for each $j=1,\ldots,J$:
  \begin{gather}
    \label{eq:schwarz:29}
      u_j^{(k+1)} = u_j^{(k)} - \omega_k P_j \bigl(u^{(k)} - A^{-1}f\bigr).
  \end{gather}
  Since the updates are orthogonal, this is equivalent to the sum
  in~\eqref{eq:schwarz:28}. In the \putindex{Gauss-Seidel method}, these
  projections are done consecutively. For instance, if we introduce
  broken indices, we can write it in the form
  \begin{gather}
    \label{eq:schwarz:30}
      u_j^{(k+\frac{j}{J})} = u_j^{(k+\frac{j-1}{J})}
      - \omega_k P_j \bigl(u^{(k+\frac{j-1}{J})} - A^{-1}f\bigr).
  \end{gather}
  This means, we apply the corrections consecutively one after the
  other. In order to understand convergence of this method, we are
  bringing it into a different form and study the propagation of the
  error. Let $uA^{-1}f$ be the solution of the problem. Then, the
  error propagates like
  \begin{multline}
    \label{eq:schwarz:31}
      u^{(k+\frac{j}{J})} - u = u^{(k+\frac{j-1}{J})} - u
      - \omega_k P_j \left(u^{(k+\frac{j-1}{J})} - u\right) \\
      = (I-\omega_k P_j) \left(u^{(k+\frac{j-1}{J})} - u\right).
  \end{multline}
  The error after a whole Gauss-Seidel step is
  \begin{gather}
    \label{eq:schwarz:32}
      u^{(k+1)} - u
      = (I-\omega_k P_J)(I-\omega_k P_{J-1})\dots(I-\omega_k P_1) \left(u^{(k)} - u\right).
  \end{gather}
  The corresponding error equation for the Jacobi method is
  \begin{gather}
    \label{eq:schwarz:33}
    u^{(k+1)} - u = \left(I-\omega_k \sum_{j=1}^J P_j\right) \left(u^{(k)} - u\right).
  \end{gather}
  Thus the notions of multiplicative and additive methods.
\end{example}

\begin{intro}
  The example of the Jacobi and Gauss-Seidel methods is generic in the
  way that for any subspace decomposition of $V$ into the sum of
  $V_j$, we can define an additive and a multiplicative method. Not
  surprisingly, their analysis also rests on the same ingredients.
\end{intro}

\begin{Definition}{multiplicative-schwarz}
  The \define{multiplicative Schwarz preconditioner} for the operator
  $A$ associated with the symmetric, and $V$-elliptic bilinear form
  $a(.,.)$ with respect to the subspace decomposition $V_j$ is the
  mapping $B_m: V \to V^*$ such that
  \begin{gather}
    \label{eq:schwarz:34}
    B_m^{-1} = (I -  E_J) A^{-1},
  \end{gather}
  where $E_J$ is the multiplicative \define{error propagation operator}
  \begin{gather}
    \label{eq:schwarz:35}
    E_J = (I - P_J)(I - P_{J-1})\dots(I - P_1).
  \end{gather}
\end{Definition}

\begin{Lemma}{error-propagation}
  The error after one step of the iteration
  \begin{gather*}
    u^{(k+1)} = u^{(k)} - B_m^{-1}\left(A u^{(k)} - f\right)
  \end{gather*}
  is given by
  \begin{gather*}
    u^{(k+1)} - u = E_J \left(u^{(k)} - u\right).
  \end{gather*}
\end{Lemma}

\begin{proof}
  First, we use the definition of $B_m$ to obtain
  \begin{gather*}
    u^{(k+1)} = u^{(k)} - (I -  E_J) \left(u^{(k)} - u\right).
  \end{gather*}
  Therefore,
  \begin{gather*}
    u^{(k+1)} - u = u^{(k)} - u - (I -  E_J) \left(u^{(k)} - u\right)
    = (I-I+E_J) \left(u^{(k)} - u\right).
  \end{gather*}
\end{proof}

\begin{remark}
  We can define the error propagation operator $E_j$ through the
  recursion
  \begin{gather}
    \label{eq:schwarz:45}
    E_{0} = I, \qquad E_{k} = (I - P_k) E_{k-1}, \quad k=1,\dots,J.
  \end{gather}
\end{remark}

\begin{Lemma}{error-propagation-recursion}
  \label{lemma:schwarz:9}
  The error propagation operator $E_j$ has the following porperties:
  \begin{align}
    \label{eq:schwarz:44}
    E^*_{j-1}E_{j-1} - E^*_{j}E_{j} &= E^*_{j-1} P_j E_{j-1}
    \\
    \label{eq:schwarz:46}
    I - E_j &= \sum_{k=1}^{j} P_k E_{k-1}.
  \end{align}
  Here, $E^*$ is the $a(.,.)$-adjoint of $E$.
\end{Lemma}

\begin{proof}
  In order to prove the first identity, we use the recursion
  formula~\eqref{eq:schwarz:45} to obtain (using $P_j = P_j^*$
  and $P_j^2 = P_j$)
  \begin{align*}
    E^*_{j}E_{j} &= E^*_{j-1} (I-P_j^*) (I-P_j) E_{j-1} \\
    &= E^*_{j-1} E_{j-1} - E^*_{j-1}P_jE_{j-1}.
  \end{align*}
  The second identity is proven by induction with
  \begin{align*}
    (I-E_0) &= I - I = 0, \\
    (I-E_j) &= I-(I-P_j) E_{j-1} = P_j  E_{j-1} + I - E_{j-1} = P_j
    E_{j-1} +\sum_{k=1}^{j-1} P_k E_{k-1}.
  \end{align*}
\end{proof}

\begin{lemma}{schwarz10}
  \label{lemma:schwarz:10}
  Let Assumption~\ref{assumption:schwarz:1} (\putindex{strengthened
    Cauchy-Schwarz inequalities}) be satisfied. Then the following
  inequalities hold for $0\le j,k \le J$ and for $u,v\in V$:
  \begin{align}
    a(P_j u,v) &\le \sqrt{a(P_j u,u)} \sqrt{a(P_j v,v)}, \\
    a(P_j u,P_k v) &\le \epsilon_{jk}
    \sqrt{a(P_j u,u)} \sqrt{a(P_k v,v)}.
  \end{align}
\end{lemma}

\begin{proof}
  By the definition of $P_j$, the fact that $P_j$ is $a(.,.)$-self
  adjoint (Lemma~\ref{lemma:schwarz:ritz}) and the \putindex{Bunyakovsky-Cauchy-Schwarz
  inequality}, we have
  \begin{multline*}
    a(P_j u,v) = a(u, P_j v) = a(P_j u, P_j v)
    \\
    \le \sqrt{a(P_j u,P_j u)} \sqrt{a(P_j v,P_j v)}
    \le \sqrt{a(P_j u,u)} \sqrt{a(P_j v,v)}.
  \end{multline*}
  The second inequality follows readily by
  \begin{gather*}
    a(P_j u,P_k v)
    \le \epsilon_{jk}\sqrt{a(P_j u,P_j u)} \sqrt{a(P_j v,P_j v)}
    = \epsilon_{jk}\sqrt{a(P_j u,u)} \sqrt{a(P_j v,v)}.
  \end{gather*}
\end{proof}

\begin{Theorem}{schwarz-multiplicative-convergence}
  Let Assumptions~\ref{assumption:schwarz:1} and the
  estimate~\eqref{eq:schwarz:23} be satisfied. Then, the error
  propagation operator satisfies
  \begin{gather}
    \label{eq:schwarz:47}
    \norm{E_J}_A^2 \le 1 - \frac{1}{\rho(\mathcal E) \constref{eq:schwarz:24}} < 1,
  \end{gather}
\end{Theorem}

\begin{proof}
  We use equation~\eqref{eq:schwarz:44} of Lemma~\ref{lemma:schwarz:9} to obtain
  \begin{gather*}
    I- E^*_J E_J = \sum_{j=1}^{j} \left(E^*_{j-1}E_{j-1} -
      E^*_{j}E_{j}\right)
    = \sum_{j=1}^J E^*_{j-1} P_j E_{j-1}.
  \end{gather*}
  Since the operators $P_j$ are positive semi-definite, by rearranging
  we obtain for all $v\in V$ the estimate
  \begin{gather}
    \label{eq:schwarz:49}
    a(E_J v, E_J v) \le a(v,v) - \sum_{j=1}^J a(E_{j-1}v, P_j E_{j-1}v).
  \end{gather}
  Thus, $\norm{E_J}_A^2 \le 1$, but we have to show that the sum on the
  right is sufficiently positive to get an estimate of the convergence
  rate. Therefore, we start with equation~\eqref{eq:schwarz:46} of
  Lemma~\ref{lemma:schwarz:9}, yielding by Lemma~\ref{lemma:schwarz:10}
  \begin{align*}
    a(P_j v,v)
    &= a(P_j v, E_{j-1}v) + \sum_{k=1}^{j-1} a(P_j v, P_k E_{k-1} v)
    \\
    &\le \sqrt{a(P_j v,v)} \left(\sqrt{a(P_j E_{j-1}v,E_{j-1}v)}
    + \sum_{k=1}^{j-1} \epsilon_{jk} \sqrt{a(P_k E_{k-1}v,E_{k-1}v)}
    \right) \\
    & \le \sqrt{a(P_j v,v)} \left(\sum_{k=1}^{j} \epsilon_{jk}
      \sqrt{a(P_k E_{k-1} v,E_{k-1} v)}
    \right)
  \end{align*}
  Now let $z \in \R^J$ be the vector with entries $z_k = \sqrt{a(P_k
    E_{k-1}v,E_{k-1}v)}$. Then, we rewrite the previous estimate as
  \begin{gather*}
    a(P_j v,v) \le (\mathcal E z)_j^2.
  \end{gather*}
  Summing over $j$ yields

  \begin{gather}
    \label{eq:schwarz:48}
    \sum_{j=1}^J a(P_j v,v) \le \norm{\mathcal E z}^2
    \le \rho(\mathcal E)^2 \norm{z}^2
    = \rho(\mathcal E)^2 \sum_{k=1}^{j} a(P_k E_{k-1} v,E_{k-1} v).
  \end{gather}
  At this point, we use Lemma~\ref{lemma:schwarz:11} to estimate
  \begin{gather*}
    \frac1{\constref{eq:schwarz:24}^2} a(v,v)
    \le \sum_{j=1}^J a(P_j v,v)
    \le \rho(\mathcal E)^2 \sum_{k=1}^{j} a(P_k E_{k-1} v,E_{k-1} v)
  \end{gather*}
  Finally, entering this estimate into~\eqref{eq:schwarz:49}, we
  obtain
  \begin{gather*}
    a(E_J v, E_J v)
    \le a(v,v) \left(1-\frac1{\rho(\mathcal
        E)^2\constref{eq:schwarz:24}^2}
      \right),
  \end{gather*}
  which is the statement of the theorem.
\end{proof}

\begin{Lemma}{schwarz11}
  \label{lemma:schwarz:11}
  Let inequality~\eqref{eq:schwarz:24} hold for some \putindex{stable
    decomposition} $v=\sum v_j$ with the constant
  $\constref{eq:schwarz:24}$. Then,
  \begin{gather}
    \label{eq:schwarz:50}
    \sum_{j=1}^J a(P_j v,v) \ge \frac1{\constref{eq:schwarz:24}^2} a(v,v).
  \end{gather}
\end{Lemma}

\begin{proof}
  From inequality~\eqref{eq:schwarz:24} for the decomposition $v=\sum
  v_j$,\ and the definition of $P_j$, and the
  \putindex{Bunyakovsky-Cauchy-Schwarz inequality}, we obtain
  \begin{align*}
    a(v,v) &= \sum_{j=1}^J a(v,v_j) \\
    &= \sum_{j=1}^J a(P_j v, v_j) \\
    &\le \sqrt{\sum_{j=1}^J a(P_j v, P_j v)} \;\sqrt{\sum_{j=1}^J a(v_j,
      v_j)} \\
    & \le \sqrt{\sum_{j=1}^J a(P_j v, P_j v)}\;
    \constref{eq:schwarz:24} \sqrt{a(v,v)}.
  \end{align*}
  Thus,
  \begin{gather*}
    a(v,v) \le \constref{eq:schwarz:24}^2 \sum_{j=1}^J a(P_j v, P_j v)
    = \constref{eq:schwarz:24}^2 \sum_{j=1}^J a(v, P_j v).
  \end{gather*}
  This proves the lemma.
\end{proof}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Extensions}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{intro}
  In order to keep the presentation and the proofs simple, we have
  presented Schwarz methods in their most straight forward form. The
  framework allows for extensions, which each add minor
  complications to the proofs and yield slightly modified results. We
  are going to mention a few of them.
\end{intro}

\begin{remark}
  The definition of the additive Schwarz operator in
  equation~\eqref{eq:schwarz:4} and the multiplicative Schwarz
  operator~\eqref{eq:schwarz:34} rely on the Ritz projections $P_j$,
  which in turn, through their definition in~\eqref{eq:schwarz:2}
  require solving the local problems with the operators $A_j$ exactly.

  In some cases, it might be advantageous either to solve a different
  local problem, or to solve the local problem approximately. In both
  cases, we can rewrite the algorithm as using a different local
  projection $\widetilde P_j$ which instead of the Ritz
  projection~\eqref{eq:schwarz:2} is defined by the equation
  \begin{gather}
    \label{eq:schwarz:42}
    \widetilde a_j (\widetilde P_j u_j,v_j) = a(u,v_j),\quad\forall v_j\in V_j,
  \end{gather}
  with a corresponding operator $\widetilde A_j: V_j \to V_j^*$. We
  will continue to assume that $\widetilde a_j (.,.)$ is symmetric and
  elliptic in the same way as $a(.,.)$ is, but possibly with different
  constants.

  It is obvious, that we will need assumptions on the modified
  bilinear forms $\widetilde a_j (.,.)$and their relationship with
  $a(.,.)$. But it turns out, that if we make these additional
  assumptions, we can make the replacements of $P_j$ by $\widetilde
  P_j$ in the algorithm and the analysis carries through with just one
  additional parameter involved.

  The modifications in the analysis are as follows: first, replace the
  stable decomposition lemma~\ref{lemma:schwarz:stable-decomposition}
  by assumption~\ref{assumption:schwarz:stable-decomposition-2}. Then,
  introduce the additional assumption
  \ref{assumption:schwarz:local-stability}, which is ellipticity of
  the modified forms in $V_j$ with respect to the norm established by the
  original bilinear form. Both assumptions together establish a
  relaxed form (only for the sum over $j$) of spectral equivalence for
  $\widetilde a_j(.,.)$ and $a(.,.)$ on $V_j$. The \putindex{strengthened
  Cauchy-Schwarz inequalities} in Assumption~\ref{assumption:schwarz:1}
  remain the same.
\end{remark}

\begin{assumption}[Stable decomposition]
  \index{stable decomposition}
  \label{assumption:schwarz:stable-decomposition-2}
    For each $v\in V_h$ there exists a decomposition $v=\sum_{j=0}^J
  v_j$ with $v_j\in V_j$, such that
  \begin{gather}
    \label{eq:schwarz:36}
    \sum_{j=0}^J \widetilde a_j(v_j, v_j)
    \lesssim a(v,v),
  \end{gather}
  where the implicit constant is independent of the number of
  subdomains $J$.
\end{assumption}

\begin{assumption}[Local stability]
  \index{local stability}
  \label{assumption:schwarz:local-stability}
  There is a constant $\omega > 0$ such that
  \begin{gather}
    \label{eq:schwarz:37}
    a(v_j, v_j) \le \omega \widetilde a_j(v_j, v_j)
    \quad\forall j=1,\dots,J \; \forall v_j\in V_j.
  \end{gather}
\end{assumption}

\begin{remark}
  With these two assumptions, the convergence estimates change as
  follows: first, using the \putindex{strengthened Cauchy-Schwarz
  inequalities}~\eqref{eq:schwarz:25}, we extend ~\eqref{eq:schwarz:38}
  to
  \begin{gather}
    \label{eq:schwarz:39}
    a(v,v) \le \omega \rho(\mathcal E) \sum_{j=1}^J \widetilde a_j(v_j, v_j).
  \end{gather}
  Then, the proof of Theorem~\ref{theorem:schwarz:1} can be conducted
  in the very same way as before, using~\eqref{eq:schwarz:36}
  and~\eqref{eq:schwarz:39}.
\end{remark}

\begin{example}
  A simple example is the introduction of a relaxation parameter
  $\omega$ such that
  \begin{gather*}
    \widetilde a_j(v_j, v_j) = \frac1\omega a(v_j, v_j).
  \end{gather*}
  Then, obviously, the local stability~\eqref{eq:schwarz:37}
  holds. Furthermore, the constant in the stable decomposition
  estimate changes by a factor $1/\omega$. Thus, in this case, the
  upper and lower bounds $\Lambda(B,A)$ and $\lambda(B,A)$ change by
  the same factor and the condition number stays the same.
\end{example}

\begin{remark}
  The second extension is, that we can replace the subspaces $V_j$ by
  \putindex{auxiliary space}s $X_j$, which are not subspaces of
  $V$. In such a situation, we have to require the existence of a
  \putindex{prolongation} or \putindex{extension operator} $R_j^T: X_j
  \to V$. This situation is adapted most easily to our existing
  framework by introducing subspaces $V_j$ as the range of $R_j^T$,
  namely,
  \begin{gather}
    \label{eq:schwarz:40}
    V_j = \bigl\{ R_j^T x \in V \big| x\in X_j \bigr\}.
  \end{gather}
  Then, the local forms are defined on the auxiliary spaces,
  \begin{gather}
    \label{eq:schwarz:41}
    \widetilde a_j(.,.) : X_j\times X_j \to \R, \quad j=1,\dots,J,
  \end{gather}
  and wherever we need a vector in $V_j$, we use the interpolation
  operator. Thus, we introduce decompositions of the form $v = \sum
  \alpha_j R_j^T x_j$ and the operator $\widetilde P_j: V \to X_j$ is
  defined by
  \begin{gather}
    \label{eq:schwarz:43}
    \widetilde a_j(\widetilde P_j u, y_j) = a(u, R_j^T y_j),
    \quad \forall y_j \in X_j.
  \end{gather}
  These modifications introduce new operators, but they do not affect
  the analysis.
\end{remark}

\begin{example}
  When we implement a block-Jacobi method, we have $V = \R^n$ and
  $m$-dimensional subspaces with $J=n/m$ (we assume the quotient is
  integer). In the standard formulation with subspaces $V_j$, the
  operators $A_j$, which have to be inverted, are $n\times n$-matrices
  with only $m$ rows and columns different from zero. This is not a
  useful description of the local problems. Instead, we want an
  invertible matrix $A_j \in \R^{m\times m}$. This can be achieved by
  choosing the extension operator
  \begin{gather}
    R_j^T : \R^m \to \R_n,
    \qquad
    \begin{pmatrix}
      x_1 \\ \vdots \\ x_m
    \end{pmatrix}
    \mapsto
    \begin{pmatrix}
      0\\ \vdots \\ 0 \\
      v_{mj+1} = x_1 \\ \vdots \\ v_{m(j+1)} = x_m
      \\ 0\\ \vdots \\ 0
    \end{pmatrix}.
  \end{gather}
\end{example}

\begin{example}
  Other examples are finite element methods with nonnested spaces on
  different levels, for instance, when the meshes of a hierarchy are
  not nested.
\end{example}

\begin{remark}
  We have discussed additive and multiplicative Schwarz methods, but
  we are not forced to consider only methods organized strictly in one
  way or the other. Instead, some of the operations can be performed
  in the additive, some in the multiplicative way.

  This is advantageous for instance on multicore hardware: an
  inspection of the algorithms shows, that application of all
  operators $P_j$ can be implemented in parallel, while the
  applications of the operators $(I-P_j)$ in the multiplicative method
  is sequential.
\end{remark}

\begin{example}
  Let the indices $j=1,\dots,J$ be grouped into $M$ subsets $I_m$,
  such that
  \begin{gather*}
    P_iP_k = P_k P_i = 0,\qquad \forall i,k\in I_m.
  \end{gather*}
  Such a distribution of indices is also called
  \define{coloring}. Then,
  \begin{gather*}
    (I-P_i)(I-P_k) = I- (P_i+P_k),
  \end{gather*}
  and by application to the whole subset and all subsets, the error
  propagation operator of the multiplicative method can be rewritten
  as
  \begin{gather*}
    E_J = \left(I-\sum_{j\in I_1} P_j \right)
    \cdots
    \left(I-\sum_{j\in I_m} P_j \right).
  \end{gather*}
  While the operations inside each ``color'' are arranged in an
  additive and thus parallelizable manner, the colers themselves are
  arranged in a multiplicative way.
\end{example}

\begin{example}
  Another example is a rearrangement of the two-level Schwarz method
  in a way, that the domain decomposition subspaces are still dealt
  with in an additive fashion, while the coarse space is connected
  multiplicatively. An example for a symmetric version of this method
  is described by the error propagation operator
  \begin{gather}
    \label{eq:schwarz:51}
    E_{TL} = \left(I-\sum_{j=1}^J P_j \right) (I-P_0)
    \left(I-\sum_{j=1}^J P_j \right),
  \end{gather}
  which is the two-level operator with Schwarz smoother discussed in
  the next chapter.
\end{example}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "main"
%%% End:


%!TEX root = ../OGUSAdoc.tex

The production side of the \ogindia model is populated by a unit measure of identical perfectly competitive firms that rent capital $K_t$ and hire labor $L_t$ to produce output $Y_t$. Firms also face a flat corporate income tax $\tau^{corp}$ as well as a tax on the amount of capital they depreciate $\tau^\delta$.

\section{Production Function}\label{EqFirmsProdFunc}

  Firms produce output $Y_t$ using inputs of capital $K_t$ and labor $L_t$ according to a general constant elasticity (CES) of substitution production function,
  \begin{equation}\label{EqFirmsCESprodfun}
    Y_t = F(K_t, L_t) \equiv Z_t\biggl[(\gamma)^\frac{1}{\ve}(K_t)^\frac{\ve-1}{\ve} + (1-\gamma)^\frac{1}{\ve}(e^{g_y t}L_t)^\frac{\ve-1}{\ve}\biggr]^\frac{\ve}{\ve-1} \quad\forall t
  \end{equation}
  where $Z_t$ is an exogenous scale parameter (total factor productivity) that can be time dependent, $\gamma$ represents the capital share of income, and $\ve$ is the constant elasticity of substitution between capital and labor. We have included constant productivity growth $g_y$ as the rate of labor augmenting technological progress.

  A nice feature of the CES production function is that the Cobb-Douglas production function is a nested case for $\ve=1$.
  \begin{equation}\label{EqFirmsCDprodfun}
    Y_t = Z_t(K_t)^\gamma(e^{g_y t}L_t)^{1-\gamma} \quad\text{for}\quad \ve=1 \quad\forall t
  \end{equation}

\section{Optimality Conditions}\label{EqFirmsFOC}

  The profit function of the representative firm is the following.
  \begin{equation}\label{EqFirmsProfit}
    PR_t = (1 - \tau^{corp})\Bigl[F(K_t,L_t) - w_t L_t\Bigr] - \bigl(r_t + \delta\bigr)K_t + \tau^{corp}\delta^\tau K_t \quad\forall t
  \end{equation}
  Gross income for the firms is given by the production function $F(K,L)$ because we have normalized the price of the consumption good to 1. Labor costs to the firm are $w_t L_t$, and capital costs are $(r_t +\delta)K_t$. The per-period economic depreciation rate is given by $\delta$.

  Taxes enter the firm's profit function \eqref{EqFirmsProfit} in two places. The first is the corporate income tax rate $\tau^{corp}$, which is a flat tax on corporate income. As is the case in the U.S., corporate income is defined as gross income minus labor costs. This will cause the corporate tax to only distort the firms' capital demand decision.

  The next place where tax policy enters the profit function \eqref{EqFirmsProfit} is through a refund of a percent of depreciation costs $\delta^\tau$ refunded at the corporate income tax rate $\tau^{corp}$. When $\delta^\tau=0$, no depreciation expense is deducted from the firm's tax liability. When $\delta^\tau=\delta$, all economic depreciation is deducted from corporate income.

  Taking the derivative of the profit function \eqref{EqFirmsProfit} with respect to labor $L_t$ and setting it equal to zero and taking the derivative of the profit function with respect to capital $K_t$ and setting it equal to zero, respectively, characterizes the optimal labor and capital demands.
  \begin{align}
    w_t &= e^{g_y t}(Z_t)^\frac{\ve-1}{\ve}\left[(1-\gamma)\frac{Y_t}{e^{g_y t}L_t}\right]^\frac{1}{\ve} \quad\forall t \label{EqFirmFOC_L} \\
    r_t &= (1 - \tau^{corp})(Z_t)^\frac{\ve-1}{\ve}\left[\gamma\frac{Y_t}{K_t}\right]^\frac{1}{\ve} - \delta + \tau^{corp}\delta^\tau \quad\forall t \label{EqFirmFOC_K}
  \end{align}

  We discuss how to calibrate the values of $\tau^{corp}$ and $\tau^\delta$ from the \btax microsimulation model in Chapter \ref{Chap_BTax}.