\section{Convergence Diagnostics} % (fold)
%\label{sec:convergence_diagnostics}

Valid inferences from sequences of MCMC samples are based on the assumption that samples are derived from the true posterior distribution of interest. Theory guarantees this condition as the number of iterations approaches infinity. It is important, therefore, to determine the minimum number of samples required to ensure a reasonable approximation to the target posterior density. Unfortunately, no universal threshold exists across all problems, so convergence must be assessed independently each time MCMC estimation is performed. The procedures for verifying convergence are collectively known as convergence diagnostics.

One approach to analyzing convergence is analytical, whereby the variance of the sample at different sections of the chain are compared to that of the limiting distribution. These methods use distance metrics to analyze convergence, or place theoretical bounds on the sample variance, and though they are promising, they are generally difficult to use and are not prominent in the MCMC literature. More common is a statistical approach to assessing convergence. Statistical techniques, rather than considering the properties of the theoretical target distribution, only consider the statistical properties of the observed chain. Reliance on the sample alone restricts such convergence criteria to heuristics, and hence, convergence cannot be guaranteed. Although evidence for lack of convergence using statistical convergence diagnostics will correctly imply lack of convergence in the chain, the absence of such evidence will not \emph{guarantee} convergence in the chain. Nevertheless, negative results for one or more criteria will provide some measure of assurance to users that their sample will provide valid inferences.

For most simple models, convergence will occur quickly, sometimes within the first several hundred iterations, after which all remaining samples of the chain may be used to calculate posterior quantities. For many more complex models, convergence requires a significantly longer burn-in period; sometimes  orders of magnitude more samples are needed. Frequently, lack of convergence will be caused by poor \emph{mixing} (Figure \ref{fig:mix}). Mixing refers to the degree to which the Markov chain explores the support of the posterior distribution. Poor mixing may stem from inappropriate proposals (if one is using the Metropolis-Hastings sampler) or from attempting to estimate models with highly correlated variables.

\begin{figure}[ht]
\begin{center}
\includegraphics[height=3in]{poormixing.pdf}
\caption{An example of a poorly-mixing sample in two dimensions. Notice that the chain is trapped in a region of low probability relative to the mean (dot) and variance (oval) of the true posterior quantity.}
\label{fig:mix}
\end{center}
\end{figure}

\subsection{Informal Methods}

The most straightforward approach for assessing convergence is based on simply plotting and inspecting traces and histograms of the observed MCMC sample. If the trace of values for each of the stochastics exhibits asymptotic behaviour\footnote{Asymptotic behaviour implies that the variance and the mean value of the sample stays relatively constant over some arbitrary period.} over the last $m$ iterations, this may be satisfactory evidence for convergence. A similar approach involves plotting a histogram for every set of $k$ iterations (perhaps 50-100) beyond some burn-in threshold $n$; if the histograms are not visibly different among the sample intervals, this is some evidence for convergence. Note that such diagnostics should be carried out for each stochastic estimated by the MCMC algorithm, because convergent behaviour by one variable does not imply evidence for convergence for other variables in the model. An extension of this approach can be taken when multiple parallel chains are run, rather than just a single, long chain. In this case, the final values of $c$ chains run for $n$ iterations are plotted in a histogram; just as above, this is repeated every $k$ iterations thereafter, and the histograms of the endpoints are plotted again and compared to the previous histogram. This is repeated until consecutive histograms are indistinguishable.

\begin{figure}[h]
\begin{center}
\includegraphics[height=3in]{metastable.pdf}
\caption{An example of metastability in a two-dimensional parameter space. The chain appears to be stable in one region of the parameter space for an extended period, then unpredictably jumps to another region of the space.}
\label{fig:metas}
\end{center}
\end{figure}

Another \emph{ad hoc} method for detecting convergence is to examine the traces of several MCMC chains initialized with different starting values. Overlaying these traces on the same set of axes should (if convergence has occurred) show each chain tending toward the same equilibrium value, with approximately the same variance. Recall that the tendency for some Markov chains to converge to the true (unknown) value from diverse initial values is called \emph{ergodicity}. This property is guaranteed by the reversible chains constructed using MCMC, and should be observable using this technique. Again, however, this approach is only a heuristic method, and cannot always detect lack of convergence, even though chains may appear ergodic.

A principal reason that evidence from informal techniques cannot guarantee convergence is a phenomenon called metastability. Chains may appear to have converged to the true equilibrium value, displaying excellent qualities by any of the methods described above. However, after some period of stability around this value, the chain may suddenly move to another region of the parameter space (Figure \ref{fig:metas}). This period of metastability can sometimes be very long, and therefore escape detection by these convergence diagnostics. Unfortunately, there is no statistical technique available for detecting metastability.

\subsection{Formal Methods}

Along with the \emph{ad hoc} techniques described above, a number of more formal methods exist which are prevalent in the literature. These are considered more formal because they are based on existing statistical methods, such as time series analysis.

PyMC currently includes functions for two formal convergence diagnostic procedures. The first, proposed by \citet{Geweke:1992gm}, is a time-series approach that compares the mean and variance of segments from the beginning and end of a single chain.
\begin{equation}
z = \frac{\bar{\theta}_a - \bar{\theta}_b}{\sqrt{Var(\theta_a) + Var(\theta_b)}}
\end{equation}
where $a$ is the early interval and $b$ the late interval. If the z-scores (theoretically distributed as standard normal variates) of these two segments are similar, it can provide evidence for convergence. PyMC calculates z-scores of the difference between various initial segments along the chain, and the last 50\% of the remaining chain. If the chain has converged, the majority of points should fall within 2 standard deviations of zero.

Diagnostic z-scores can be obtained by calling the \code{geweke} function. It accepts either (1) a single trace, (2) a Node or Stochastic object, or (3) an entire Model object.

\paragraph{Method Usage}
\begin{verbatim}
geweke(pymc_object, first=0.1, last=0.5, intervals=20)
\end{verbatim}
\begin{itemize}

\item \verb=pymc_object=: The object that is or contains the output trace(s).

\item \verb=first= (optional): First portion of chain to be used in Geweke diagnostic. Defaults to 0.1 (i.e. first 10\% of chain).

\item \verb=last= (optional): Last portion of chain to be used in Geweke diagnostic. Defaults to 0.5 (i.e. last 50\% of chain).

\item \verb=intervals= (optional): Number of sub-chains to analyze. Defaults to 20.
\end{itemize}

The resulting scores are best interpreted graphically, using the \code{geweke_plot} function. This displays the scores in series, in relation to the 2 standard deviation boundaries around zero. Hence, it is easy to see departures from the standard normal assumption.

\begin{figure}[h!]
\begin{center}
\includegraphics[height=6in]{geweke.png}
\caption{Sample plots of Geweke z-scores for a variable using \textbf{geweke_plot}. The occurrence of the scores well within 2 standard deviations of zero gives does not indicate lack of convergence (top), while deviations exceeding 2 standard deviations suggests that additional samples are required to achieve convergence (bottom).}
\label{fig:geweke}
\end{center}
\end{figure}

\code{geweke_plot} takes either a single set of scores, or a dictionary of scores (output by \code{geweke} when an entire Sampler is passed) as its argument:

\paragraph{Method Usage}
\begin{verbatim}
geweke_plot(scores, name='geweke', format='png', suffix='-diagnostic', \
                path='./', fontmap = {1:10, 2:8, 3:6, 4:5, 5:4}, verbose=1)
\end{verbatim}
\begin{itemize}

\item \verb=scores=: The object that contains the Geweke scores. Can be a list (one set) or a dictionary (multiple sets).

\item \verb=name= (optional): Name used for output files. For multiple scores, the dictionary keys are used as names.

\item \verb=format= (optional): Graphic output file format (defaults to \emph{png}).

\item \verb=suffix= (optional): Suffix to filename (defaults to \emph{-diagnostic})

\item \verb=path= (optional): The path for output graphics (defaults to working directory).

\item \verb=fontmap= (optional): Dictionary containing the font map for the labels of the graphic.

\item \verb=verbose= (optional): Verbosity level for output (defaults to 1).
\end{itemize}

To illustrate, consider a model \code{my_model} that is used to instantiate a MCMC sampler. The sampler is then run for a given number of iterations:
\begin{verbatim}
	>>> S = pm.MCMC(my_model)
	>>> S.sample(10000, burn=5000)
\end{verbatim}
It is easiest simply to pass the entire sampler \code{S} to the \code{geweke} function:
\begin{verbatim}
	>>> scores = pm.geweke(S, intervals=20)
	>>> pm.Matplot.geweke_plot(scores)
\end{verbatim}
Alternatively, individual stochastics within \code{S} can be analyzed for convergence:
\begin{verbatim}
	>>> trace = S.alpha.trace()
	>>> alpha_scores = pm.geweke(trace, intervals=20)
	>>> pm.Matplot.geweke_plot(alpha_scores, 'alpha')
\end{verbatim}
An example of convergence and non-convergence of a chain using \code{geweke_plot} is given in Figure \ref{fig:geweke}.

The second diagnostic provided by PyMC is the \citet{raftery} procedure. This approach estimates the number of iterations required to reach convergence, along with the number of burn-in samples to be discarded and the appropriate thinning interval. A separate estimate of both quantities can be obtained for each variable in a given model.

As the criterion for determining convergence, the Raftery and Lewis approach uses the accuracy of estimation of a user-specified quantile. For example, we may want to estimate the quantile $q=0.975$ to within $r=0.005$ with probability $s=0.95$. In other words,

\begin{equation}
	Pr(|\hat{q}-q| \le r) = s
\end{equation}

From any sample of $\theta$, one can construct a binary chain:

\begin{equation}
	Z^{(j)} = I(\theta^{(j)} \le u_q)
\end{equation}

where $u_q$ is the quantile value and $I$ is the indicator function. While $\{\theta^{(j)}\}$ is a Markov chain, $\{Z^{(j)}\}$ is not necessarily so. In any case, the serial dependency among $Z^{(j)}$ decreases as the thinning interval $k$ increases. A value of $k$ is chosen to be the smallest value such that the first order Markov chain is preferable to the second order Markov chain.

This thinned sample is used to determine number of burn-in samples. This is done by comparing the remaining samples from burn-in intervals of increasing length to the limiting distribution of the chain. An appropriate value is one for which the truncated sample's distribution is within $\epsilon$ (arbitrarily small) of the limiting distribution. See \citet{raftery} or \citet{Gamerman:1997tb} for computational details. Estimates for sample size tend to be conservative.

This diagnostic is best used on a short pilot run of a particular model, and the results used to parameterize a subsequent sample that is to be used for inference.

\paragraph{Method Usage}
\begin{verbatim}
raftery_lewis(pymc_object, q, r, s=.95, epsilon=.001, verbose=1)
\end{verbatim}
\begin{itemize}

	\item \verb=pymc_object=: The object that contains the Geweke scores. Can be a list (one set) or a dictionary (multiple sets).

    \item \verb=q=: Desired quantile to be estimated.

    \item \verb=r=: Desired accuracy for quantile.

    \item \verb=s=(optional): Probability of attaining the requested accuracy (defaults to 0.95).

    \item \verb=epsilon= (optional) : Half width of the tolerance interval required for the q-quantile (defaults to 0.001).

    \item \verb=verbose= (optional) : Verbosity level for output (defaults to 1).
\end{itemize}

The code for \code{raftery_lewis} is based on the FORTRAN program \emph{gibbsit}, which was written by Steven Lewis.

For example, consider again a sampler \code{S} run for some model \code{my_model}:
\begin{verbatim}
	>>> S = pm.MCMC(my_model)
	>>> S.sample(10000, burn=5000)
\end{verbatim}
One can pass either the entire sampler \code{S} or any stochastic within \code{S} to the \code{raftery_lewis} function, along with suitable arguments. Here, we have chosen $q=0.025$ (the lower limit of the equal-tailed 95\% interval) and error $r=0.01$:
\begin{verbatim}
	>>> pm.raftery_lewis(S, q=0.025, r=0.01)
\end{verbatim}
This yields diagnostics as follows for each stochastic of \code{S}, as well as a dictionary containing the diagnostic quantities:

\begin{verbatim}
	========================
	Raftery-Lewis Diagnostic
	========================

	937 iterations required (assuming independence) to achieve 0.01 accuracy
	with 95 percent probability.

	Thinning factor of 1 required to produce a first-order Markov chain.

	39 iterations to be discarded at the beginning of the simulation (burn-in).

	11380 subsequent iterations required.

	Thinning factor of 11 required to produce an independence chain.
\end{verbatim}

Additional convergence diagnostics are available in the
\href{http://lib.stat.cmu.edu/R/CRAN/}{R statistical package}, via the
\href{http://www-fis.iarc.fr/coda/}{CODA module}. PyMC includes a method
\code{coda} for exporting model traces in a format that may be
directly read by CODA.

\paragraph{Method Usage}
\begin{verbatim}
pm.utils.coda(pymc_object)
\end{verbatim}
\begin{itemize}

\item \verb=pymc_object=: The PyMC sampler for which output is desired.

\end{itemize}
Calling \verb=coda= yields two files, one containing raw trace values (suffix
\verb=.out=) and another containing indices to the trace values (suffix
\verb=.ind=).

% section convergence_diagnostics (end)

\subsection{Autocorrelation Plots} % (fold)
%\label{sec:autocorrelation_plots}

Samples from MCMC algorithms are ususally autocorrelated, due partly to the
inherent Markovian dependence structure. The degree of autocorrelation can
be quantified using the autocorrelation function:
\begin{eqnarray*}
    \rho_k &=& \frac{\mbox{Cov}(X_t,
X_{t+k})}{\sqrt{\mbox{Var}(X_t)\mbox{Var}(X_{t+k})}} \\
            &=& \frac{E[(X_t - \theta)(X_{t+k} - \theta)]}{\sqrt{E[(X_t -
\theta)^2] E[(X_{t+k} - \theta)^2]}}
\end{eqnarray*}
PyMC includes a function for plotting the autocorrelation function for each
stochastic in the sampler (Figure \ref{fig:autocorr}). This allows users
to examine the relationship among successive samples within sampled chains.
Significant autocorrelation suggests that chains require thinning prior to
use of the posterior statistics for inference.

\begin{figure}[h]
        \begin{center}
        \includegraphics[scale=0.4]{autocorr.png}
    \end{center}
    \caption{Sample autocorrelation plots for two Poisson variables from
coal mining disasters example model.}
    \label{fig:autocorr}
\end{figure}

\begin{verbatim}
autocorrelation(pymc_object, name, maxlag=100, format='png', suffix='-acf',
path='./', fontmap = {1:10, 2:8, 3:6, 4:5, 5:4}, verbose=1)
\end{verbatim}
\begin{itemize}
	\item \verb=pymc_object=: The object that is or contains the output
trace(s).

	\item \verb=name=: Name used for output files.

	\item \verb=maxlag=: The highest lag interval for which
autocorrelation is calculated.

	\item \verb=format= (optional): Graphic output file format
(defaults to \emph{png}).

	\item \verb=suffix= (optional): Suffix to filename (defaults to
\emph{-diagnostic})

	\item \verb=path= (optional): The path for output graphics
(defaults to working directory).

	\item \verb=fontmap= (optional): Dictionary containing the font map
for the labels of the graphic.

	\item \verb=verbose= (optional): Verbosity level for output
(defaults to 1).
\end{itemize}

Using any given model \code{my_model} as an example, autocorrelation plots can be obtained simply by passing the sampler for that model to the \code{autocorrelation} function (within the \code{Matplot} module) directly:
\begin{verbatim}
	>>> S = pm.MCMC(my_model)
	>>> S.sample(10000, burn=5000)
	>>> pm.Matplot.autocorrelation(S)
\end{verbatim}
Alternatively, variables within a model can be plotted individually. For example, a hypothetical variable \code{beta} that was estimated using sampler \code{S} will yield a correlation plot as follows:
\begin{verbatim}
	>>> pm.Matplot.autocorrelation(S.beta)
\end{verbatim}

% section autocorrelation_plots (end)

\section{Goodness of Fit} % (fold)
%\label{sec:goodness_of_fit}

Checking for model convergence is only the first step in the evaluation of MCMC model outputs. It is possible for an entirely unsuitable model to converge, so additional steps are needed to ensure that the estimated model adequately fits the data. One intuitive way for evaluating model fit is to compare model predictions with actual observations. In other words, the fitted model can be used to simulate data, and the distribution of the simulated data should resemble the distribution of the actual data.

Fortunately, simulating data from the model is a natural component of the Bayesian modelling framework. Recall, from the discussion on imputation of missing data, the posterior predictive distribution:

\begin{equation}
	p(\tilde{y}|y) = \int p(\tilde{y}|\theta) f(\theta|y) d\theta
\end{equation}

Here, $\tilde{y}$ represents some hypothetical new data that would be expected, taking into account the posterior uncertainty in the model parameters. Sampling from the posterior predictive distribution is easy in PyMC. The code looks identical to the corresponding data stochastic, with two modifications: (1) the node should be specified as deterministic and (2) the statistical likelihoods should be replaced by random number generators. As an example, consider the Poisson data likelihood of the coal mining disasters example:
\begin{verbatim}
	@pm.stochastic(observed=True, dtype=int)
	def disasters(  value = disasters_array,
	                early_mean = early_mean,
	                late_mean = late_mean,
	                switchpoint = switchpoint):
	    """Annual occurences of coal mining disasters."""
	    return pm.poisson_like(value[:switchpoint],early_mean) +
	 pm.poisson_like(value[switchpoint:],late_mean)
\end{verbatim}
This is a mixture of Poisson processes, one with a higher early mean and another with a lower late mean. Here is the corresponding sample from the posterior predictive distribution:
\begin{verbatim}
	@pm.deterministic
	def disasters_sim(early_mean = early_mean,
	                late_mean = late_mean,
	                switchpoint = switchpoint):
	    """Coal mining disasters sampled from the posterior predictive distribution"""
	    return concatenate( (pm.rpoisson(early_mean, size=switchpoint),
	 pm.rpoisson(late_mean, size=n-switchpoint)))
\end{verbatim}
Notice that \code{@pm.stochastic} has been replaced with \code{@pm.deterministic} and \code{pm.poisson_like} with \code{pm.rpoisson}. The simulated values from each of the Poisson processes are concatenated together before returning them.

The degree to which simulated data correspond to observations can be evaluated in at least two ways. First, these quantities can simply be compared visually. This allows for a qualitative comparison of model-based replicates and observations. If there is poor fit, the true value of the data may appear in the tails of the histogram of replicated data, while a good fit will tend to show the true data in high-probability regions of the posterior predictive distribution (Figure~\ref{fig:gof}).

\begin{figure}[htdevi!]
        \begin{center}
        \includegraphics[height=3in]{gof.png}
    \end{center}
    \caption{Data sampled from the posterior predictive distribution of a model for some observation \textbf{x}. The true value of \textbf{x} is shown by the dotted red line.}
    \label{fig:gof}
\end{figure}

The Matplot module in PyMC provides an easy way of producing such plots, via the \code{gof_plot} function. To illustrate, consider a single observed data point \code{x} and an array of values \code{x_sim} sampled from the posterior predictive distribution. The histogram is generated by calling:

\begin{verbatim}
	pm.Matplot.gof_plot(x_sim, x, name='x')
\end{verbatim}

A second approach for evaluating goodness of fit using samples from the posterior predictive distribution involves the use of a statistical criterion. For example, the Bayesian p-value \citep{Gelman:1996gp} uses a discrepancy measure that quantifies the difference between data (observed or simulated) $x$ and the expected value $e$, conditional on some model. One such discrepancy measure is the Freeman-Tukey statistic \citep{Brooks:2000il}:

\begin{equation}
	D(x|\theta) = \sum_j (\sqrt{x_j}-\sqrt{e_j})^2
\end{equation}

Model fit is assessed by comparing the discrepancies from observed data to those from simulated data. On average, we expect the difference between them to be zero; hence, the Bayesian p-value is simply the proportion of simulated discrepancies that are larger than their corresponding observed discrepancies:

\begin{equation}
	p = Pr[ D(\text{sim}) > D(\text{obs}) ]
\end{equation}

If $p$ is very large (e.g. $>0.975$) or very small (e.g. $<0.025$) this implies that the model is not consistent with the data, and thus is evidence of lack of fit. Graphically, data and simulated discrepancies plotted together should be clustered along a 45 degree line passing through the origin, as shown in Figure~\ref{fig:deviate}.

\begin{figure}[ht!]
        \begin{center}
        \includegraphics[height=3in]{deviates.png}
    \end{center}
    \caption{Plot of deviates of observed and simulated data from expected values. The cluster of points symmetrically about the 45 degree line (and the reported p-value) suggests acceptable fit for the modeled parameter.}
    \label{fig:deviate}
\end{figure}

The \code{discrepancy} function in the \code{diagnostics} module can be used to generate discrepancy statistics from arrays of data, simulated values, and expected values:

\begin{verbatim}
	D = pm.diagnostics.discrepancy(observed, simulated, expected)
\end{verbatim}

A call to this function returns two arrays of discrepancy values (one for observed data and one for simulated data), which can be passed to the \code{discrepancy_plot} function in the Matplot module to generate a scatter plot, and if desired, a p-value:

\begin{verbatim}
	pm.Matplot.discrepancy_plot(D, name='D', report_p=True)
\end{verbatim}

Additional optional arguments for \code{discrepancy_plot} are identical to other PyMC plotting functions.

% section goodness_of_fit (end)


\chapter{Disjoint product}

I remind that
$\coprod X = \bigcup_{i\in\dom X} (i,X_i)$
for every indexed family~$X$ of sets.

\begin{obvious}
$\coprod X\in\mathbf{Set}(\dom X,\im X)$.
\end{obvious}

\begin{defn}
I will call disjoint join of an indexed family~$\mathcal{X}$ of filters the following reloid: $\coprod\mathcal{X} = \bigsqcup_{i\in\dom f}(\{i\}\times^{\mathsf{RLD}}\mathcal{X}_i)$.
\end{defn}

\section{Some funcoids}

\begin{prop}
$\supfun{x\mapsto(i,x)}\mathcal{X}=\{i\}\times^{\mathsf{RLD}}\mathcal{X}$ for every filter~$\mathcal{X}$.
\end{prop}

\begin{proof}
$\supfun{x\mapsto(i,x)}\mathcal{X}=\bigsqcap\setcond{\rsupfun{x\mapsto(i,x)}X}{X\in\up\mathcal{X}}=\bigsqcap\setcond{\{i\}\times X}{X\in\up\mathcal{X}}=\{i\}\times^{\mathsf{RLD}}\mathcal{X}$.
\end{proof}

\begin{prop}
$\supfun{(x\mapsto(i,x))^{-1}}\mathcal{X}=\im(\mathcal{X}|_{\{i\}})$ for a filter~$\mathcal{X}$ on the set $\mathscr{U}\cup\{{\mathscr{U}}\}$ where $\mathscr{U}$~is
a Grothendieck universe.
\end{prop}

\begin{proof}
$\supfun{(x\mapsto(i,x))^{-1}}\mathcal{X}=\bigsqcap\setcond{\rsupfun{(x\mapsto(i,x))^{-1}}X}{X\in\up\mathcal{X}}=\bigsqcap\setcond{\setcond{x}{(i,x)\in X}}{X\in\up\mathcal{X}}=\bigsqcap\setcond{x\in\im X|_{\{i\}}}{X\in\up\mathcal{X}}=\im\bigsqcap\setcond{x\in X|_{\{i\}}}{X\in\up\mathcal{X}}=\im(\mathcal{X}|_{\{i\}})$.
\end{proof}

\section{Cartesian product of funcoids}

\subsection{Definition and basic properties}

\begin{defn}
\emph{Cartesian product} of an indexed family~$f$ of funcoids is
a funcoid \[ \prod^{(J)} f=\bigsqcup_{i\in\dom f}((x\mapsto(i,x))\circ f_i\circ(x\mapsto(i,x)^{-1}). \]
\end{defn}

\begin{prop}
$\supfun{\prod^{(J)}f}\mathcal{X}=\coprod_{i\in\dom f}\supfun{f_i\circ(x\mapsto(i,x)^{-1}}\mathcal{X}$.
\end{prop}

\begin{proof}
\begin{multline*}
\coprod_{i\in\dom f}\supfun{f_i\circ(x\mapsto(i,x)^{-1}}\mathcal{X}=\\
\bigsqcup_{i\in\dom f}(\{i\}\times^{\mathsf{RLD}}\supfun{f_i\circ(x\mapsto(i,x)^{-1}}\mathcal{X})=\\
\bigsqcup_{i\in\dom f}(\supfun{x\mapsto(i,x)}\supfun{f_i\circ(x\mapsto(i,x)^{-1}}\mathcal{X})=\\
\supfun{\prod^{(J)}f}\mathcal{X}.
\end{multline*}
\end{proof}

\subsection{Projections}

\begin{thm}
$f_i$~can be restored from the value of $\prod^{(J)}f=f_i$.
\end{thm}

\begin{proof}
$f_i = (x\mapsto(i,x)^{-1})\circ\prod^{(J)}f\circ(x\mapsto(i,x)^{-1})$
(taken into account that $x\mapsto(i,x)^{-1}$ is a principal funcoid).
\end{proof}

\section{Arrow product of funcoids}

\begin{defn}
\emph{Arrow product} of an indexed family~$f$ of funcoids is
a funcoid \[ \prod^{\rightarrow} f=\bigsqcup_{i\in\dom f}((x\mapsto(i,x))\circ f_i. \]
\end{defn}

\begin{prop}
$\supfun{\prod^{\rightarrow}f}\mathcal{X}=\coprod_{i\in\dom f}\supfun{f_i}\mathcal{X}$.
\end{prop}

\begin{proof}
\begin{multline*}
\coprod_{i\in\dom f}\supfun{f_i}\mathcal{X}=\\
\bigsqcup_{i\in\dom f}(\{i\}\times^{\mathsf{RLD}}\supfun{f_i}\mathcal{X})=\\
\bigsqcup_{i\in\dom f}(\supfun{x\mapsto(i,x)}\supfun{f_i}\mathcal{X})=\\
\supfun{\prod^{\rightarrow}f}\mathcal{X}.
\end{multline*}
\end{proof}

\subsection{Projections}

\begin{defn}
\emph{Arrow projections}
$\pi^{\rightarrow}_i = (x\mapsto(i,x))^{-1}$.
\end{defn}

\begin{thm}
$\pi^{\rightarrow}_i\circ\prod^{\rightarrow}f=f_i$
\end{thm}

\begin{proof}
Because $\pi^{\rightarrow}_i$ is a principal funcoid, we have
\[ \pi^{\rightarrow}_i\circ\prod^{\rightarrow}f=
\bigsqcup_{j\in\dom f}((x\mapsto(i,x))^{-1}\circ(x\mapsto(j,x))\circ f_j). \]
But $(x\mapsto(i,x))^{-1}\circ(x\mapsto(j,x))$ is the idenitity if $i=j$ or
empty otherwise. So
$\pi^{\rightarrow}_i\circ\prod^{\rightarrow}f=f_i$.
\end{proof}

\section{Cartesian product of reloids}

\subsection{Definition and basic properties}

\begin{defn}
\emph{Cartesian product} of an indexed family~$f$ of reloids is
a reloid \[ \prod^{(J)} f=\bigsqcup_{i\in\dom f}((x\mapsto(i,x))\circ f_i\circ(x\mapsto(i,x)^{-1}). \]
\end{defn}

\begin{conjecture}
$\prod^{(J)}(g\circ f)=\prod^{(J)}g\circ\prod^{(J)}f$.
\end{conjecture}

\subsection{Projections}

\begin{thm}
$f_i$~can be restored from the value of $\prod^{(J)}f=f_i$.
\end{thm}

\begin{proof}
$f_i = (x\mapsto(i,x)^{-1})\circ\prod^{(J)}f\circ(x\mapsto(i,x)^{-1})$.
\end{proof}

\section{Arrow product of reloids}

\begin{defn}
\emph{Arrow product} of an indexed family~$f$ of reloids is
a reloid \[ \prod^{\rightarrow} f=\bigsqcup_{i\in\dom f}((x\mapsto(i,x))\circ f_i. \]
\end{defn}

\subsection{Projections}

\begin{defn}
\emph{Arrow projections}
$\pi^{\rightarrow}_i = (x\mapsto(i,x))^{-1}$.
\end{defn}

\begin{prop}
$\pi^{\rightarrow}_i\circ\prod^{\rightarrow}f=f_i$.
\end{prop}

\begin{proof}
Because $x\mapsto(i,x)$ is a pricipal funcoid, we have
$\pi^{\rightarrow}_i\circ\prod^{\rightarrow}f=
\pi^{\rightarrow}_i\circ\bigsqcup_{i\in\dom f}((x\mapsto(i,x))\circ f_i=
\bigsqcup_{j\in\dom f}((x\mapsto(i,x))^{-1}\circ(x\mapsto(j,x))\circ f_i$.

But $(x\mapsto(i,x))^{-1}\circ(x\mapsto(j,x)$ is the identity if $i=j$ or empty otherwise. So $\pi^{\rightarrow}_i\circ\prod^{\rightarrow}f=f_i$.
\end{proof}


\documentclass[10pt]{report}

\usepackage{subcaption} % for subfigures
\usepackage{amsthm} % for QED
\usepackage{mathtools} % for delimiter

\usepackage{listings} % for code
\lstset{
	language=R,
	basicstyle=\footnotesize\ttfamily,
	numbers=none,
	stepnumber=1,
	numbersep=8pt,
	showspaces=false,
	showstringspaces=false,
	showtabs=false,
	frame=single,
	tabsize=2,
	captionpos=t,
	breaklines=true,
	breakatwhitespace=false
}

\usepackage{float} % for figure [H]
\usepackage{booktabs} % for tabular
\usepackage{caption} % for \caption*
\usepackage[export]{adjustbox} % for valign=t
\usepackage{array} % for column type m
\usepackage{verbatim}
\usepackage{graphicx}
%\graphicspath{ {imgs/} }
\usepackage{fancyhdr}
\usepackage{amssymb}
\usepackage{amsmath}

%%%%%% Pagination
\setlength{\topmargin}{-.3 in}
\setlength{\oddsidemargin}{0in}
\setlength{\evensidemargin}{0in}
\setlength{\textheight}{9.in}
\setlength{\textwidth}{6.5in}

%Title page
\newcommand{\hwTitle}{Homework \#3}
\newcommand{\hwCourse}{Applied Statistics}
\newcommand{\hmClassInstructor}{Professor Lulu Kang}

\title{
	\vspace{2in}
	\textmd{\textbf{\hwCourse\\\hwTitle}}\\
	\vspace{0.3in}\large{\textit{\hmClassInstructor}}
	\vspace{3in}
}
\author{\textbf{Zhihao Ai}}
\date{}

%Header setting.
\pagestyle{fancy}
\fancyhead[L]{Zhihao Ai}
\fancyhead[C]{Math 484}
\fancyhead[R]{Homework 3}
%%%%%%

%Global setting.
%\everymath{\displaystyle}
\setlength\parindent{0pt}

%Custom general commands.
\newcommand{\ds}{\displaystyle}
\newcommand{\ts}{\textstyle}

\newcolumntype{N}{ >$ c <$}
\newcolumntype{M}[1]{>{\centering\arraybackslash $}m{#1}<{$}}

\newcommand{\abs}[1] {\left| #1 \right|}

\DeclarePairedDelimiter\autoparen{(}{)}
\newcommand{\pa}[1]{\autoparen*{#1}}

\newcommand{\var} {\text{var}}

\newcommand{\m}[1] {\mathbf{#1}}

\begin{document}

\maketitle

\section*{Problem 1}
(Ex. 6.5) \textbf{Brand preference}
\begin{enumerate}
	\item [a.]
	Obtain the scatter plot matrix and the correlation matrix. What information do these diagnostic aids provide here?

	Scatter plot matrix:
	\begin{figure}[H]
		\centering
		\vspace{-4ex}
		\includegraphics[width=.5\linewidth]{p1/5a.png}
		\vspace{-5ex}
	\end{figure}
	Correlation matrix:
	\lstinputlisting{p1/5a.txt}
	From the scatter plot matrix, we can see there is a strong correlation between $Y$ and $X_1$. From the correlation matrix, we can see that the correlation between $Y$ and $X_1$ are stronger than that between $Y$ and $X_2$ ($0.89>0.39$). Also, there is no linear correlation between $X_1$ and $X_2$ since the correlation is 0.

	\item [b.]
	Fit regression model (6.1) to the data. State the estimated regression function. How is $b_1$ interpreted here?

	After fitting regression model to the data, we obtain the following coefficients:
	\lstinputlisting{p1/5b.txt}
	So, the estimated regression function is $\hat{Y} = 37.65 + 4.425 x_1 + 4.375 x_2$. The interpretation is that when sweetness ($X_2$) is fixed, the brand liking ($Y$) will increase by $b_1 = 4.425$ units if mosture content ($X_1$) increases by 1 unit.

	\item [c.]
	Obtain the residuals and prepare a box plot of the residuals. What information does this plot provide?

	The residuals are
	\lstinputlisting{p1/5c.txt}
	Box plot of the residuals:
	\begin{figure}[H]
		\centering
		\includegraphics[width=.35\linewidth]{p1/5c.png}
	\end{figure}
	The plot shows that the residuals are symmetrically centered around 0.

	\item [d.]
	Plot the residuals against $\hat{Y}, X_1, X_2$ and $X_1 X_2$ on separate graphs. Also prepare a normal probability plot. Interpret the plots and summerize your findings.
	\begin{figure}[H]
		\centering
		\begin{subfigure}[b]{.4\linewidth}
			\includegraphics[width=\linewidth]{p1/5d1.png}
		\end{subfigure}
		\begin{subfigure}[b]{.4\linewidth}
			\includegraphics[width=\linewidth]{p1/5d2.png}
		\end{subfigure}\\
		\begin{subfigure}[b]{.4\linewidth}
			\includegraphics[width=\linewidth]{p1/5d3.png}
		\end{subfigure}
		\begin{subfigure}[b]{.4\linewidth}
			\includegraphics[width=\linewidth]{p1/5d4.png}
		\end{subfigure}
	\end{figure}
	There are no linear relationships shown in the plots above, indicating constancy of error variance and that the model fits the data well. The plot of residuals against $X_1 X_2$ indicates there is no interaction effect present.
	\begin{figure}[H]
		\centering
		\includegraphics[width=.5\linewidth]{p1/5dqq.png}
	\end{figure}
	The normal QQ-plot above shows a strong linear correlation, confirming that the error terms are fairly normally distributed.

	\item [e.]
	Conduct the Breusch-Pagan test for constancy of the error variance, assuming $\log \sigma_i^2 = \gamma_0 + \gamma_1 X_{i1} + \gamma_2 X_{i2}$; use $\alpha = .01$. State the alternatives, decision rule, and conclusion.

	The alternatives are
	\begin{align*}
		H_0 &: \gamma_1 = \gamma_2 = 0\\
		H_a &: \gamma_1 \ne 0 \text{ or } \gamma_2 \ne 0
	\end{align*}
	The BP-statistic is
	\[
	\chi^2_{BP} = \frac{SSR^*}{p} \div \pa{\frac{SSE}{n}}^2
	\]
	where $SSR^*$ is the regression sum of squares when regressing $e^2$ on $X$.
	The decision rule is
	\begin{align*}
	\text{If } \chi^2_{BP} \le \chi^2(0.99; 2) = 9.21034, \text{ conclude } H_0\\
	\text{If } \chi^2_{BP} > \chi^2(0.99; 2) = 9.21034, \text{ conclude } H_a
	\end{align*}
	According to the ANOVA tables of two models, we have
	\[
	\chi^2_{BP} = \frac{67.34+5.06}{3} \div \pa{\frac{94.30}{16}}^2 = 0.694759 < 9.21034
	\]
	Therefore, we conclude $H_0$, that the error variance is constant.

	\item [f.]
	Conduct a formal test for lack of fit of the first-order regression function; use $\alpha = .01$. State the alternatives, decision rule, and conclusion.

	The alternatives are
	\begin{align*}
		H_0 &: E(Y) = \beta_0 + \beta_1 X_1 + \beta_2 X_2 \\
		H_a &: E(Y) \ne \beta_0 + \beta_1 X_1 + \beta_2 X_2
	\end{align*}
	The test statistic is
	\[
	F^* = \frac{SSLF}{c-3} \div \frac{SSPE}{n-c}
	\]
	The decision rule is
	\begin{align*}
		\text{If } F^* \le F(1-\alpha; c-2, n-c) = 6.370681, \text{ conclude } H_0\\
		\text{If } F^* > F(1-\alpha; c-2, n-c) = 6.370681, \text{ conclude } H_a
	\end{align*}
	The ANOVA table is shown below:
	\lstinputlisting{p1/5f.txt}
	From the table, we have $F^* = (37.30/5)/(57.00/8) = 1.047018 < 6.370681$. So, we conclude $H_0$, that $E(Y) = \beta_0 + \beta_1 X_1 + \beta_2 X_2$.

\end{enumerate}

\section*{Problem 2}
(Ex. 6.18) \textbf{Commercial properties}
\begin{enumerate}
	\item [a.]
	Prepare a stem-and-leaf plot for each predictor variable. What information do these plots provide?

	Stem-and-leaf plot for $X_1$:
	\lstinputlisting{p2/18a1.txt}
	Stem-and-leaf plot for $X_2$:
	\lstinputlisting{p2/18a2.txt}
	Stem-and-leaf plot for $X_3$:
	\lstinputlisting{p2/18a3.txt}
	Stem-and-leaf plot for $X_4$:
	\lstinputlisting{p2/18a4.txt}
	From the plots, we can see that $X_1$ ranges from 0 to 20, $X_2$ ranges from 2 to 14.6, $X_3$ ranges from 0 to 0.73 and mostly takes value less than 0.1, $X_4$ ranges from 30000 to 480000.

	\item [b.]
	Obtain the scatter plot matrix and the correlation matrix. Interpret these and state your principal findings.

	Scatter plot matrix:
	\begin{figure}[H]
		\centering
		\includegraphics[width=.65\linewidth]{p2/18b.png}
	\end{figure}
	Correlation matrix:
	\lstinputlisting{p2/18b.txt}
	The scatter plot matrix shows the relationships among the variables and the correlation matrix provides the corelation coefficients. We can see that there is no strong linear correlation among the variables, but the correlation between $Y$ and $X_2$, and also the correlation between $Y$ and $X_4$, are relatively high.

	\item [c.]
	Fit regression model (6.5) for four predictor variables to the data. State the estimated regression function.

	After fitting regression model to the data, we obtain the following coefficients:
	\lstinputlisting{p2/18c.txt}
	So, the estimated regression function is $\hat{Y} = 12.2 - 0.142 x_1 + 0.282 x_2 + 0.619 x_3 + 0.00000792 x_4$.

	\item [d.]
	Obtain the residuals and prepare a box plot of the residuals. Does the distribution appear to be fairly symmetrical?

	The residuals are in the residuals list of the model. Below is the box plot of the residuals:
	\begin{figure}[H]
		\centering
		\includegraphics[width=.4\linewidth]{p2/18d.png}
	\end{figure}
	The distribution appears to be fairly symmetrical. Just a few outliners.

	\item [e.]
	Plot the residuals against $\hat{Y}$, each predictor variable, and each two-factor interaction term on separate graphs. Also prepare a normal probability plot. Analyze your plots and summerize your findings.

	\begin{figure}[H]
		\centering
		\begin{subfigure}[b]{.25\linewidth}
			\includegraphics[width=\linewidth]{p2/18e_yhat.png}
		\end{subfigure}%%
		\begin{subfigure}[b]{.25\linewidth}
			\includegraphics[width=\linewidth]{p2/18e_x1.png}
		\end{subfigure}%%
		\begin{subfigure}[b]{.25\linewidth}
			\includegraphics[width=\linewidth]{p2/18e_x2.png}
		\end{subfigure}%%
		\begin{subfigure}[b]{.25\linewidth}
			\includegraphics[width=\linewidth]{p2/18e_x3.png}
		\end{subfigure}\\
		\begin{subfigure}[b]{.25\linewidth}
			\includegraphics[width=\linewidth]{p2/18e_x4.png}
		\end{subfigure}%%
		\begin{subfigure}[b]{.25\linewidth}
			\includegraphics[width=\linewidth]{p2/18e_x1x2.png}
		\end{subfigure}%%
		\begin{subfigure}[b]{.25\linewidth}
			\includegraphics[width=\linewidth]{p2/18e_x1x3.png}
		\end{subfigure}%%
		\begin{subfigure}[b]{.25\linewidth}
			\includegraphics[width=\linewidth]{p2/18e_x1x4.png}
		\end{subfigure}\\
		\begin{subfigure}[b]{.25\linewidth}
			\includegraphics[width=\linewidth]{p2/18e_x2x3.png}
		\end{subfigure}%%
		\begin{subfigure}[b]{.25\linewidth}
			\includegraphics[width=\linewidth]{p2/18e_x2x4.png}
		\end{subfigure}%%
		\begin{subfigure}[b]{.25\linewidth}
			\includegraphics[width=\linewidth]{p2/18e_x3x4.png}
		\end{subfigure}%%
		\begin{subfigure}[b]{.25\linewidth}
			\includegraphics[width=\linewidth]{p2/18eqq.png}
		\end{subfigure}
	\end{figure}
	In the plots with $x$-axis representing $X_3, X_1 X_3, X_2 X_3$ and $X_3 X_4$, the residuals are getting more concentrated around 0 as the predictors increase. The S shape in the QQ-plot indicates the error is not normally distributed.

	\item [f.]
	Can you conduct a formal test for lack of fit here?

	No, because there are no observations with the same values of predictor variables.

	\item [g.]
	Divide the 81 cases into two groups, placing the 40 cases with the smallest fitted values $\hat{Y}_i$ into group 1 and the remaining cases into group 2. Conduct the Brown-Forsythe test for constancy of the error variance, using $\alpha = .05$. State the decision rule and conclusion.

	The alternatives are
	\begin{align*}
		H_0 &: \text{error variance is constant}\\
		H_a &: \text{error variance is not constant}
	\end{align*}
	The test statistic is
	\[
	t_{BF}^* = \frac{\bar{d}_1 - \bar{d}_2}{s\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}}
	\]
	The decision rule is
	\begin{align*}
		\text{If } t_{BF}^* \le t(1-\alpha/2; n-2) = 1.99045, \text{ conclude } H_0\\
		\text{If } t_{BF}^* > t(1-\alpha/2; n-2) = 1.99045, \text{ conclude } H_a
	\end{align*}
	We can obtain $s$ from the pooled variance,
	\[
	s = \sqrt{\frac{\sum (d_{i1} - \bar{d}_1) + \sum (d_{i2} - \bar{d}_2)}{n-2}} = \sqrt{\frac{20.3091 + 22.4447}{81-2}} = 0.7357
	\]
	Then the test statistic is given by
	\[
	t_{BF}^* = \frac{0.8696 - 0.7793}{0.7357\sqrt{\frac{1}{40} + \frac{1}{41}}} = 0.5521
	\]
	Since $t_{BF}^* = 0.5521 < 1.99045$, we conclude $H_0$, that the error variance is constant.

\end{enumerate}

\section*{Problem 3}
(Ex. 6.19) Refer to \textbf{Commercial properties} Problem 6.18. Assume that regression model (6.5) for four predictor variables with independent normal error terms is appropriate.
\begin{enumerate}
	\item [a.]
	Test whether there is a regression relation; use $\alpha = .05$. State the alternatives, decision rule, and conclusion. What does your test imply about $\beta_1, \beta_2, \beta_3,$ and $\beta_4$? What is the $P$-value of the test?

	The alternatives are
	\begin{align*}
		H_0 &: \beta_1 = \beta_2 = \dots = \beta_{p-1} = 0\\
		H_a &: \text{not all } \beta_i=0, i=1,\dots,p-1
	\end{align*}
	The test statistic is
	\[
	F^* = \frac{MSR}{MSE}
	\]
	The decision rule is
	\begin{align*}
	\text{If } F^* \le F(1-\alpha; p-1, n-p) = 2.492049, \text{ conclude } H_0\\
	\text{If } F^* > F(1-\alpha; p-1, n-p) = 2.492049, \text{ conclude } H_a
	\end{align*}
	The ANOVA table is shown below:
	\lstinputlisting{p3/19a.txt}
	So, the test statistic $F^* = \frac{(14.819+72.802+8.381+42.325)/4}{98.231/76} = 26.75543 > 2.492049$, we conclude $H_a$, that not all $\beta_i$ is zero. The $P$-value is $7.272e-14$.

	\item [b.]
	Estimate $\beta_1, \beta_2, \beta_3,$ and $\beta_4$ jointly by the Bonferroni procedure, using a 95 percent family confidence coefficient. Interpret your results.

	We can obtain the standard errors of $\hat{\beta}_i$ from the summary table below:
	\lstinputlisting{p3/19b.txt}
	The confidence limits with family confidence coefficient 0.95 are
	\[
	\hat{\beta}_i \pm B\cdot s\{\hat{\beta}_i\}
	\]
	where $B = t(1-\alpha/2g; n-p) = t(0.99375; 76) = 2.558541$. So, the Bonferroni 95\% joint CIs for $\beta_1, \beta_2, \beta_3,$ and $\beta_4$ are
	\begin{align*}
		\beta_1 &: -0.142034 \pm 2.558541 \cdot 0.021343 = (-0.196641, -0.087427)\\
		\beta_2 &: 0.282017 \pm 2.558541 \cdot 0.063172 = (0.1203888, 0.443645)\\
		\beta_3 &: 0.619344 \pm 2.558541 \cdot 1.086813 = (-2.161312, 3.4)\\
		\beta_4 &: 0.000008 \pm 2.558541 \cdot 0.000001 = (0.000004, 0.000011)
	\end{align*}
	meaning that we are 95\% confident the $\beta_i$'s will fall in the intervals simultaneously.

	\item [c.]
	Calculate $R^2$ and interpret this measure.

	From the ANOVA table, we have
	\[
	R^2 = \frac{SSR}{SSTO} = \frac{14.819+72.802+8.381+42.325}{14.819+72.802+8.381+42.325+98.231} = \frac{138.327}{236.558} = 0.584749
	\]
	It measures the proportionate reduction of total variation in $Y$ associated with the use the set of $X$ variables.
\end{enumerate}

\section*{Problem 4}
(Ex. 6.20) Refer to \textbf{Commercial properties} Problem 6.18. Assume that regression model (6.5) for four predictor variables with independent normal error terms is appropriate. The researcher wishes to obtain simultaneous interval estimates of the mean rental rates for four typical properties specified as follows:
\begin{table}[H]
	\centering
	\begin{tabular}{*{5}{N}}
		& 1 & 2 & 3 & 4 \\ \midrule
		X_1: & 5.0 & 6.0 & 14.0 & 12.0 \\
		X_2: & 8.25 & 8.50 & 11.50 & 10.25 \\
		X_3: & 0 & 0.23 & 0.11 & 0 \\
		X_4: & 250000 & 270000 & 300000 & 310000 \\
	\end{tabular}
\end{table}
Obtain the family of estimates using a 95 percent family confidence coefficient. Employ the most efficient procedure.

Applying Bonferroni procedure, we have
\[
B = t(1-\alpha/2g; n-p) = t(0.99375; 76) = 2.558541
\]
So the CIs are
\begin{align*}
	Y_{h1} &: \hat{Y}_{h1} \pm B\hat{\sigma} \sqrt{x_{h1}' (X'X)^{-1} x_{h1}^{}} = 15.79813 \pm 2.558541 \cdot 1.137102 \cdot 0.244601 = (15.08651, 16.50975)\\
	Y_{h2} &: \hat{Y}_{h2} \pm B\hat{\sigma} \sqrt{x_{h2}' (X'X)^{-1} x_{h2}^{}} = 16.02754 \pm 2.558541 \cdot 1.137102 \cdot 0.207519 = (15.4238, 16.63128)\\
	Y_{h3} &: \hat{Y}_{h3} \pm B\hat{\sigma} \sqrt{x_{h3}' (X'X)^{-1} x_{h3}^{}} = 15.90072 \pm 2.558541 \cdot 1.137102 \cdot 0.195411 = (15.33221, 16.46923)\\
	Y_{h4} &: \hat{Y}_{h4} \pm B\hat{\sigma} \sqrt{x_{h4}' (X'X)^{-1} x_{h4}^{}} = 15.84339 \pm 2.558541 \cdot 1.137102 \cdot 0.227928 = (15.18027, 16.50651)
\end{align*}

\section*{Problem 5}
Suppose that $\m{X}= [\m{x}_1, \dots, \m{x}_k]$, where $\m{x}_i$, $i=1, \dots k$ are linearly independent columns of $\m{X}$ (one of $\m{x}_i$ can be a column of $\m{1}$ if intercept is included in the model). Show that $\var(\hat{\beta}_k) \ge \sigma^2(\m{x}_k' \m{x}_k)^{-1}$ with equality if and only if $\m{x}_k' \m{x}_j = 0, j=0,1,\dots,k-1$.

Since $\var(\hat{\beta}_k) = \sigma^2 \pa{\m{X}' \m{X}}^{-1}_{k,k}$,
\begin{align*}
\var(\hat{\beta}_k) \ge \sigma^2(\m{x}_k' \m{x}_k)^{-1}
&\Leftrightarrow
\sigma^2 \pa{\m{X}' \m{X}}^{-1}_{k,k} \ge \sigma^2(\m{x}_k' \m{x}_k)^{-1}\\
&\Leftrightarrow
\pa{\m{X}' \m{X}}^{-1}_{k,k} \ge (\m{x}_k' \m{x}_k)^{-1}
\end{align*}
\iffalse
Partition $\m{X}$ into two blocks then we have
\[
\m{X} =
	\left[
	\begin{array}{c}
	\m{X}_{-k}\\ \hline
	\m{x}'_k
	\end{array}
	\right]
,
\m{X}' =
	\left[
	\begin{array}{c|c}
	\m{X}'_{-k} & \m{x}_k
	\end{array}
	\right]
\]
where $\m{x}_k$ is the $k$-th row of $\m{X}$, and $\m{X}_{-k}$ is the matrix with $\m{x}_k$ removed. By block matrix multiplication, we obtain
\[
\m{X}' \m{X}= \m{X}'_{-k} \m{X}_{-k} + \m{x}_k \m{x}'_k
\]
\fi
Dividing $\m{X}' \m{X}$ into four blocks, we have
\[
\m{X}' \m{X} =
	\left[
	\begin{array}{c|c}
	A & B \\ \hline
	B' & C
	\end{array}
	\right]
\]
where $C_{1\times1} = \m{x}_k' \m{x}_k$. Taking the inverse of $\m{X}' \m{X}$,
\[
(\m{X}' \m{X})^{-1} =
	\left[
	\begin{array}{c|c}
	D & E \\ \hline
	E' & F
	\end{array}
	\right]
\]
According to a theorem of inverse of a partitioned symmetric matrix,
\[
F = (C - B'A^{-1}B)^{-1} = C^{-1} + C^{-1}B'(A-BC^{-1}B')^{-1} BC^{-1}
\]
Substitute $C= \m{x}_k' \m{x}_k$ into the equation above
\begin{align*}
	\pa{\m{X}' \m{X}}^{-1}_{k,k}
	&= F\\
	&= C^{-1} + C^{-1}B'(A-BC^{-1}B')^{-1} BC^{-1}\\
	&= (\m{x}_k' \m{x}_k)^{-1} + (\m{x}_k' \m{x}_k)^{-1} B'(A-B(\m{x}_k' \m{x}_k)^{-1}B')^{-1} B(\m{x}_k' \m{x}_k)^{-1}
\end{align*}
Because $\m{x}_k' \m{x}_k$ represents the sum of square of the $k$-th column, $\m{x}_k' \m{x}_k\ge 0$, and $(\m{x}_k' \m{x}_k)^{-1}\ge 0$.
Since $\m{x}_i$ are linearly independent columns, and $(\m{x}_k' \m{x}_k)^{-1}\ge 0$, $(\m{x}_k' \m{x}_k)^{-1} B'(A-B(\m{x}_k' \m{x}_k)^{-1}B')^{-1} B(\m{x}_k' \m{x}_k)^{-1}\ge 0$; so $\pa{\m{X}' \m{X}}^{-1}_{k,k} \ge (\m{x}_k' \m{x}_k)^{-1}$.
Because $B = [\m{x}_1, \m{x}_2, \dots ,\m{x}_{k-1}]'\m{x}_k$ and $B' = \m{x}'_k [\m{x}_1, \m{x}_2, \dots ,\m{x}_{k-1}]$, $B=B'=\m{0}$ if and only $\m{x}_k' \m{x}_j = 0$ for $j=0,1,\dots,k-1$; with $B=B'=\m{0}$, $\pa{\m{X}' \m{X}}^{-1}_{k,k} = (\m{x}_k' \m{x}_k)^{-1}$.
Therefore, by proving $\pa{\m{X}' \m{X}}^{-1}_{k,k} \ge (\m{x}_k' \m{x}_k)^{-1}$ with equality if and only if $\m{x}_k' \m{x}_j = 0, j=0,1,\dots,k-1$, we show that $\var(\hat{\beta}_k) \ge \sigma^2(\m{x}_k' \m{x}_k)^{-1}$ with equality if and only if $\m{x}_k' \m{x}_j = 0, j=0,1,\dots,k-1$.
\end{document}


\chapter{Relativistic Free Field Theories}

\section{The Lorentz Invariance}

The metric for ($3+1$)-dimensional flat space-time is chosen to be
\begin{equation}
	g_{\mu\nu}=g^{\mu\nu}=\mathrm{diag}(+1,-1,-1,-1).
\end{equation}
The Lorentz transformation ${\Lambda^{\mu}}_{\nu}$ satisfies
\begin{equation}
{\Lambda^{\mu}}_{\alpha}{\Lambda^{\nu}}_{\beta} g^{\alpha\beta} = g^{\mu\nu}.
\end{equation}
From this we have
\begin{equation}
	g^{\gamma\alpha}{\Lambda^{\mu}}_{\alpha}{\Lambda^{\nu}}_{\beta} g_{\mu\nu}
	= g^{\gamma\alpha}g_{\alpha\beta}
	\quad \Longrightarrow \quad
	{\Lambda_{\nu}}^{\gamma}{\Lambda^{\nu}}_{\beta}
	= {\delta^{\gamma}}_{\beta},
\end{equation}
The inverse Lorentz transformation satisfies:
\begin{equation}
	{(\Lambda^{-1})^{\mu}}_{\nu} = {\Lambda_{\nu}}^{\mu}.
\end{equation}
The infinitesimal transformation is denoted as
\begin{equation}
\begin{aligned}
	{\Lambda^{\mu}}_{\nu} &= \delta^{\mu}_{\nu}+\delta{\omega^{\mu}}_{\nu}, \\
	{(\Lambda^{-1})^\mu}_\nu &= \delta^{\mu}_{\nu}-\delta{\omega^\mu}_\nu.
\end{aligned}
\end{equation}
which means $\delta {\omega^\mu}_\nu = -\delta {\omega_\nu}^\mu$.
We can further use the metric tensor $g_{\mu\nu}$ to lower the indices and get $\delta\omega_{\alpha\beta} = -\delta\omega_{\beta\alpha}$, i.e., the infinitesimal parameter $\delta \omega_{\mu\nu}$ is anti-symmetric for ($\mu \leftrightarrow \nu$).

In general, a representation of Lorentz group $U_R(\Lambda)$ can be parametrized as:
\begin{equation}
	U_R(\Lambda) = \exp\left(\frac{i}{2}\omega_{\mu\nu}M_R^{\mu\nu}\right).
\end{equation}
Another useful parametrization is
\begin{equation}
	\theta_i \equiv \frac{1}{2}\varepsilon_{ijk}\omega_{jk}, \quad
	\beta_i \equiv \omega_{i0}.
\end{equation}
A new set of generators are:
\begin{equation}
	J_i \equiv \frac{1}{2}\varepsilon_{ijk}M^{jk}, \quad
	K_i \equiv M^{i0},
\end{equation}
where $J_i$'s are the generators of the spatial rotations, and $K_i$'s are the generators of Lorentz boosts.

In the fundamental representation, the generators are represented by
\begin{equation}
\begin{aligned}
	J_1 &= \left[\begin{array}{cccc} 0 & & & \\ & 0 & & \\ & & 0 & -i \\ & & i & 0 \end{array}\right], &
	J_2 &= \left[\begin{array}{cccc} 0 & & & \\ & 0 & & i \\ & & 0 & \\ & -i & & 0 \end{array}\right], &
	J_3 &= \left[\begin{array}{cccc} 0 & & & \\ & 0 & -i & \\ & i & 0 & \\ & & & 0 \end{array}\right], \\
	K_1 &= \left[\begin{array}{cccc} 0 & -i & & \\ -i & 0 & & \\ & & 0 & \\ & & & 0 \end{array}\right], &
	K_2 &= \left[\begin{array}{cccc} 0 & & -i & \\ & 0 & & \\ -i & & 0 & \\ & & & 0 \end{array}\right], &
	K_3 &= \left[\begin{array}{cccc} 0 & & & -i \\ & 0 & & \\ & & 0 & \\ -i & & & 0 \end{array}\right].
\end{aligned}
\end{equation}
The Lie algebra of the Lorentz algebra can be explicitly done using the fundamental representation.
The result is
\begin{equation}
\begin{aligned}
	\left[J_i, J_j\right] &= i \varepsilon_{ijk} J_k, \\
	\left[J_i, K_j\right] &= i \varepsilon_{ijk} K_k, \\
	\left[K_i, K_j\right] &= -i\varepsilon_{ijk} J_k.
\end{aligned}
\end{equation}

\subsection{Left and Right Spinors}
We introduce a new set of generators:
\begin{equation}
	N_i^{L} \equiv \frac{J_i - i K_i}{2}, \quad
	N_i^{R} \equiv \frac{J_i + i K_i}{2}.
\end{equation}
They satisfies two independent $\mathfrak{su}(2)$ algebra:
\begin{equation}
\begin{aligned}
	\left[N_i^L, N_j^L \right] &= i\varepsilon_{ijk}N_k^L, \\
	\left[N_i^R, N_j^R \right] &= i\varepsilon_{ijk}N_k^R, \\
	\left[N_i^L, N_j^R \right] &= 0.
\end{aligned}
\end{equation}
That is, the Lorentz algebra is isomorphic to two $\mathfrak{su}(2)$ algebra,
\begin{equation}
	\mathfrak{so}(3,1) \approx \mathfrak{su}_L(2)\oplus\mathfrak{su}_R(2).
	\label{eq:Lorentz-alg-decomp}
\end{equation}
From Eq.~(\ref{eq:Lorentz-alg-decomp}), we know that the representation of the Lorentz algebra can be labelled by $j_L$ and $j_R$.
Note that the fundamental representation correspond to
\begin{equation*}
	\left(j_L=\frac{1}{2},j_R=\frac{1}{2}\right).
\end{equation*}
The specific form of the group is
\begin{equation}
	\Lambda(\vec\theta,\vec\beta)
	=\exp\left[i(\vec\theta+i\vec\beta)\cdot \vec N^L + i(\vec\theta-i\vec\beta)\cdot \vec N^R\right].
\end{equation}
The spinor representations are those with $j_L=1/2$ or $j_R=1/2$.
Specifically, we define the left-hand spinor $\psi_L$ and right-hand spinor $\psi_R$ that transform as:
\begin{equation}\label{eq:qft-left-right-spinor-rep}
\begin{aligned}
	\Lambda_L(\vec\theta,\vec\beta)\psi_L
	&= \exp\left(\frac{i}{2}\vec\theta\cdot\vec\sigma-\frac{1}{2}\vec\beta\cdot\vec\sigma \right) \psi_L, \\
	\Lambda_R(\vec\theta,\vec\beta)\psi_R
	&= \exp\left(\frac{i}{2}\vec\theta\cdot\vec\sigma+\frac{1}{2}\vec\beta\cdot\vec\sigma \right) \psi_R.
\end{aligned}
\end{equation}
Using the fact $\sigma^2 \cdot \vec\sigma^* \cdot\sigma^2 = -\vec\sigma$, the left-hand and the right-hand representations are related by:
\begin{equation}
\begin{aligned}
	\sigma^2 \Lambda_L^* \sigma^2 &= \Lambda_R, & \sigma^2 \Lambda_L^T \sigma^2 &= \Lambda_L^{-1}, \\
	\sigma^2 \Lambda_R^* \sigma^2 &= \Lambda_L, & \sigma^2 \Lambda_R^T \sigma^2 &= \Lambda_R^{-1}.
\end{aligned}
\end{equation}
For this reason, the left-hand and right-hand spinor can be interchanged by
\begin{equation}
\begin{aligned}
	\sigma^2 \psi_L^* &\sim \chi_R, & \psi_L^\dagger \sigma^2 &\sim \chi^\dagger_R, \\
	\sigma^2 \psi_R^* &\sim \chi_L, & \psi^\dagger_R \sigma^2 &\sim \chi^\dagger_L.
	\label{eq:left-right-spinor-rel}
\end{aligned}
\end{equation}

\subsection{The Invariant Symbols}
The invariant symbols can be thought as the Clebsch-Gordan coefficients that help to form singlets.
The first singlet comes from the decomposition
\begin{equation*}
	\frac{1}{2}\otimes \frac{1}{2} \approx 0 \oplus 1.
\end{equation*}
Correspondingly, we can check that for each-hand-side spinor, the quadratic forms
\begin{equation}
	\psi_L^T\sigma^2\chi_L \quad \text{or} \quad
	\psi_R^T\sigma^2\chi_R
	\label{eq:inner-product-inv-symbol}
\end{equation}
are singlets.
We can define the first invariant symbol as\footnote{We use the dotted symbol to denote the right-hand spinor indices.}
\begin{equation}
	\varepsilon^{ab} = \varepsilon^{\dot a \dot b} = i(\sigma^2)_{ab}, \quad
	\varepsilon_{ab} = \varepsilon_{\dot a \dot b} = -i(\sigma^2)_{ab}.
\end{equation}
The symbol $\varepsilon^{ab}$ or $\varepsilon_{ab}$ also serve as the index raising/lowering symbol, i.e.,
\begin{equation}
	\varepsilon^{ab}\psi_b = \psi^a,\
	\varepsilon_{ab}\psi^b = \psi_a.
\end{equation}
The singlet (\ref{eq:inner-product-inv-symbol}) is then defined as the inner product of two spinors:
\begin{equation}
	\psi\cdot\chi
	\equiv \varepsilon_{ab}\psi^a\chi^b
	= \psi^a\chi_{a}
	= -\varepsilon_{ba}\psi^a\chi^b
	= -\psi_b\chi^b.
\end{equation}
In addition, because of (\ref{eq:left-right-spinor-rel}), the expressions
\begin{equation*}
	\psi_L^\dagger \chi_R \quad \text{and} \quad \psi_R^\dagger \chi_L
\end{equation*}
are also singlets.

Besides, we know there should be another invariant symbol from the decomposition
\begin{equation*}
	\left(\frac{1}{2}, 0\right) \otimes \left(0,\frac{1}{2}\right)
	\approx \left(0, 0\right) \oplus \cdots.
\end{equation*}
For this reason, we are searching for the symbol $M$ that the expression
\begin{equation*}
	M^\mu_{a\dot b} \psi^a_L \chi^{\dot b}_R
\end{equation*}
transforms as the Lorentz vector.
The matrix $M^\mu$ should transform as
\begin{equation*}
	M^\mu \longrightarrow \Lambda_L^T \cdot M^\mu \cdot \Lambda_R = {\Lambda^\mu}_\nu M^\nu.
\end{equation*}
Use the fact that $\sigma^2 \cdot \Lambda_L^T \cdot \sigma^2 = \Lambda_L^{-1}$, the above equation transforms to
\begin{equation*}
	\left(\sigma^2 M^\mu\right) \longrightarrow \Lambda_L^{-1} \cdot \left(\sigma^2 M^\mu\right)\cdot \Lambda_R.
\end{equation*}
We then show the matrices $\sigma^\mu = (\sigma^0,\vec\sigma)$ satisfies the requirement.
Firstly, for the spatial rotation,
\begin{equation}
	\Lambda_L(\vec\theta,\vec 0) = \Lambda_R(\theta,\vec 0) = \exp\left(i\vec\theta\cdot \frac{\vec\sigma}{2}\right)
\end{equation}
The Pauli matrix transform as
\begin{equation*}
	\left(1-i\delta\vec\theta\cdot\frac{\vec\sigma}{2}\right)\sigma^j\left(1+i\delta\vec\theta\cdot \frac{\vec\sigma}{2}\right)
	= \sigma^j + i\delta\theta_i \left(-i \varepsilon_{ijk}\sigma^k \right)
\end{equation*}
Secondly, for the boosts,
\begin{equation}
	\Lambda_{L/R}(\vec 0, \vec\beta) =
	\exp\left(\mp\vec\beta\cdot \frac{\vec\sigma}{2}\right).
\end{equation}
The Pauli matrix transform as
\begin{equation*}
	\left(1+\delta\vec\beta\cdot\frac{\vec\sigma}{2}\right)\sigma^\mu \left(1+\delta\vec\beta\cdot \frac{\vec\sigma}{2}\right) = \begin{cases}
		 \sigma^0 + i\delta\beta_i \cdot (-i\sigma^i), & \mu = 0 \\
		 \sigma^j + i\delta\beta_j (-i\sigma^0), & \mu = j
	\end{cases}.
\end{equation*}
We thus have shown indeed that
\begin{equation}
	\psi_L^T \sigma^2 \sigma^\mu \chi_R
\end{equation}
is a Lorentz vector.
Further more, from (\ref{eq:left-right-spinor-rel}), we know that
\begin{equation}
	\eta_R^\dagger \sigma^\mu \chi_R
\end{equation}
is also a Lorentz vector.
Similarly, consider the Lorentz vector
\begin{equation*}
	N^\mu_{\dot a b} \psi^{\dot a}_R \chi^{b}_R,
\end{equation*}
which together with $\sigma^2$ should transforms as
\begin{equation*}
	\left(\sigma^2 N^\mu\right) \longrightarrow
	\Lambda_R^{-1} \cdot \left(\sigma^2 N^\mu\right)\cdot \Lambda_L.
\end{equation*}
We can check that $\bar\sigma^\mu = (\sigma^0,-\vec\sigma)$ satisfies the requirement, and thus
\begin{equation}
	\eta_L^\dagger \bar\sigma^\mu \chi_L
\end{equation}
is also a Lorentz vector.

\subsection{Lorentz-invariant Lagrangian}
In relativistic quantum field theory, the Lagrangian should be a singlet under Lorentz transformation.
Different free fields correspond to different representation of the Lorentz algebra.
The symmetry under Lorentz transformation also restrict the possible terms that can appear in the Lagrangian.

\subsubsection{Scalar Field}
The simplest case is when $j_L=j_R = 0$, corresponding to the scalar field, which we denote as $\phi(x)$.
Since the field it self is singlet, any polynomial of the field in principle can appear in the theory.
When considering the free theory, we restrict our attention to the quadratic terms.
We require the field theory to have a dynamical term, which contains derivative the the field.
The derivative operator $\partial^\mu$ transforms as the fundamental representation.
To be Lorentz invariant, the allowed free theory can only be
\begin{equation}
	\mathcal L_{\mathrm{KG}} = \frac{1}{2}\partial^\mu \phi \partial_\mu \phi -\frac{m^2}{2}\phi^2
	\simeq -\frac{1}{2}\phi (\partial^2+m^2) \phi.
\end{equation}

For general discussion, we consider the field theory on $d$-dimensional space-time.
Note that the space-time Fourier transformation is defined as
\begin{equation}
\begin{aligned}
	\tilde{\phi}(k) &= \int d^{d}x e^{ik\cdot x} \phi(x), \\
	\phi(x) &= \int \frac{d^{d}k}{(2\pi)^{d}} e^{-ik\cdot x}\tilde{\phi}(k),
\end{aligned}
\end{equation}
where the inner product of two $d$-momentum and $d$-coordinate is
\begin{equation}
	k\cdot x \equiv \omega t-\vec k\cdot \vec x.
\end{equation}
The action can be expressed as
\begin{equation}
	S_{\mathrm{KG}} = \int \frac{d^d k}{(2\pi)^d} \frac{1}{2} \tilde{\phi}^*(k)(k^2-m^2)\tilde{\phi}(k).
\end{equation}

\subsubsection{Vector Field}
If we can choose $j_L=j_R=1/2$, the field is transformed as Lorentz vector.
We denote the field as $A^\mu(x)$.
Some possible quadratic forms for the vector field that forms singlets are
\begin{equation}
	A^\mu A_\mu,\ (\partial_\mu A^\mu)^2,\ A^\nu \partial^2 A_\nu,\
	\varepsilon_{\mu\nu\rho\lambda} \partial^\mu A^\nu \partial^\rho A^\lambda.
\end{equation}
For the field theory describe the electromagnetic field, we require the theory to further have gauge symmetry, i.e., invariant under
\begin{equation}
	A^\mu(x) \rightarrow A^\mu(x) + \partial^\mu \alpha(x).
\end{equation}
The gauge invariant forbids the first term, and forces the second and third term to combine as
\begin{equation*}
	(\partial_\mu A^\mu)^2 - A^\nu \partial^2 A_\nu
	\sim \frac{1}{2}(\partial^\mu A^\nu - \partial^\nu A^\mu)(\partial_\mu A^\nu-\partial_\nu A_\mu)
	\equiv \frac{1}{2} F^{\mu\nu}F_{\mu\nu}.
\end{equation*}
where we have define a field-strength tensor
\begin{equation}
	F^{\mu\nu}\equiv (\partial^\mu A^\nu - \partial^\nu A^\mu)
	= \left[\begin{array}{cccc}
		0 & -E_1 & -E_2 & -E_3 \\
		E_1 & 0 & -B_3 & B_2 \\
		E_2 & B_3 & 0 & -B_1 \\
		E_3 & -B_2 & B_1 & 0
	\end{array} \right],
\end{equation}
where we notice that from Maxwell equations:
\begin{equation}
	E^i = \partial_t \vec A = -\vec\nabla A^0, \quad B^i = \nabla \times \vec A.
\end{equation}
Note that the fourth term is called the \textit{theta term}, which can be written as a boundary term
\begin{equation}
	\varepsilon_{\mu\nu\rho\lambda} \partial^\mu A^\nu \partial^\rho A^\lambda
	= \partial^\mu (\varepsilon_{\mu\nu\rho\lambda} A^\nu \partial^\rho A^\lambda).
\end{equation}
The Lagrangian describing the electromagnetic field is given by
\begin{equation}
	\mathcal{L}_{\mathrm{Maxwell}} = -\frac{1}{4}F_{\mu\nu}F^{\mu\nu}.
\end{equation}

\subsubsection{Spinor Field}

Based on previous discussion, the Lagrangian for spinor field can have
\begin{equation}
	\psi_L^\dagger \bar\sigma^\mu \partial_\mu \psi_L,\
	\psi_R^\dagger \sigma^\mu \partial_\mu \psi_R,\
	\psi_L^\dagger \psi_R,\ \psi_R^\dagger \psi_L,\
	\psi_L \cdot \psi_L,\ \psi_R \cdot \psi_R.
\end{equation}
The Dirac field describe the theory with both left-hand and right-hand spinors.
The Lagrangian is
\begin{equation}
	\mathcal{L}_{\mathrm{Dirac}}
	= \bar\psi \left(i\gamma^\mu \partial_\mu - m\right)\psi,
\end{equation}
where
\begin{eqnarray}
	\psi = \left(\begin{array}{c}
		\psi_L \\ \psi_R
	\end{array}\right),\
	\bar\psi = \left(\begin{array}{cc}
		\psi_R^\dagger & \psi_L^\dagger
	\end{array}\right),\
	\gamma^\mu = \left(\begin{array}{cc}
		0 & \sigma^\mu \\
		\bar\sigma^\mu & 0
	\end{array}\right).
\end{eqnarray}
In addition, we could consider using the last two terms as the mass, the result theory is the \textit{Majorana field theory}:
\begin{equation}
\begin{aligned}
	\mathcal{L}^{L}_{\mathrm{Majorana}}
	&= \psi_L^\dagger \left(i\bar\sigma^\mu \partial_\mu -m \sigma^2 \right) \psi_L, \\
	\mathcal{L}^{R}_{\mathrm{Majorana}}
	&= \psi_R^\dagger \left(i\sigma^\mu \partial_\mu -m \sigma^2 \right) \psi_R.
\end{aligned}
\end{equation}
For the spinor basis, the Dirac Algebra is generated by
\begin{equation}\label{eq:qft-diract-generator}
	S^{\mu\nu} = \frac{i}{4}[\gamma^\mu, \gamma^\nu].
\end{equation}
The Lorentz group is represented by
\begin{equation}\label{eq:qft-dirac-rep}
	\Lambda_{\frac{1}{2}} = \exp\left(\frac{i}{2}\omega_{\mu\nu} S^{\mu\nu}\right).
\end{equation}
Using the familiar parametrization,
\begin{equation}
	S^{i0} = \frac{i}{2}\left[\begin{array}{cc}
		\sigma^i & 0 \\ 0 & -\sigma^i
	\end{array}\right], \quad
	S^{ij} = \frac{1}{2}\epsilon^{ijk} \left[\begin{array}{cc}
		\sigma^k & 0 \\ 0 & -\sigma^k
	\end{array}\right],
\end{equation}
which agree with the transformation property (\ref{eq:qft-left-right-spinor-rep}).

\section{Canonical Quantization}

\subsection{Scalar Field}

For the Klein-Gordon Lagrangian
\begin{equation}
	\mathcal L = -\frac{1}{2}\phi(x)(\partial^2+m^2)\phi(x),
\end{equation}
the equation of motion is:
\begin{equation}\label{eq:rkg-eom}
\begin{aligned}
	&\ \partial_\mu \left[\frac{\partial \mathcal L}{\partial(\partial_\mu \phi)}\right] - \frac{\partial \mathcal L}{\partial \phi} = 0 \\
	\Rightarrow &\
	(\partial_t^2-\nabla^2+m^2)\phi(\bm x,t) = 0.
\end{aligned}
\end{equation}
The (classical) solution to Eq.~(\ref{eq:rkg-eom}) is proportional to the plane wave:
\begin{equation}
	\phi_{\bm k}(\bm x, t) \propto e^{-i\omega_{\bm{k}}t+i\bm{k}\cdot\bm{x}} + e^{i\omega_{\bm{k}}t-i\bm{k}\cdot\bm{x}},
\end{equation}
where the energy is $\omega_{\bm{k}}=\bm{k}^2+m^2$ and $\bm k$ is the momentum as the conserved quantity.
The general solution to Eq.~(\ref{eq:rkg-eom}) is
\begin{equation}
	\phi(\bm x,t) \propto \int \frac{d^{3} k}{(2\pi)^{3}} \left(
		a_{k}e^{-i\omega_{\bm{k}}t+i\bm{k}\cdot\bm{x}} +
		a^*_{k}e^{i\omega_{\bm{k}}t-i\bm{k}\cdot\bm{x}}
	\right),
\end{equation}
where $a_k$'s are arbitrary c-numbers.

The canonical quantization promote the coefficient $a_{k}/a_{k}^*$ to the particle annihilation/creation operator $a_{k}/a_{k}^\dagger$, with the commutation relation
\begin{equation}
	[a_{k}, a_{p}^\dagger] = (2\pi)^{3} \delta^{3}(\bm{k}-\bm{p}).
\end{equation}

\subsubsection{Single-particle States}
The single-particle state with momentum $\bm k$ is created by $a_{k}^{\dagger}$ operators acting on the vacuum:
\begin{equation}
	|\bm{k}\rangle \equiv \sqrt{2\omega_{\bm k}} a_{k}^{\dagger}|0\rangle,
	\label{eq:rel-single-particle}
\end{equation}
where $|\bm{k}\rangle$ is a state with a single particle of momentum $\bm{k}$.
The factor of $\sqrt{2 \omega_{\bm k}}$ in (\ref{eq:rel-single-particle}) is a convention to ensure Lorenz invariant.
To compute the normalization of one-particle states, we start by requiring the vacuum state to be of unit norm:
\begin{equation}
	\langle 0|0\rangle=1,
\end{equation}
which, together with the canonical commutation relation of particle annihilation and creation operators leads to
\begin{equation}
	\langle\bm{p}|\bm{k}\rangle
	= 2\sqrt{\omega_{\bm p} \omega_{\bm k}}\left\langle 0\left|a_{p} a_{k}^{\dagger}\right| 0\right\rangle
	= 2 \omega_{\bm p}(2\pi)^{3} \delta^{3}(\bm{p}-\bm{k}).
\end{equation}
The identity operator for one-particle states under such norm is
\begin{equation}
	1=\int \frac{d^{3} p}{(2\pi)^{3}} \frac{1}{2\omega_{\bm p}}|\bm{p}\rangle\langle\bm{p}|, \label{eq:rel-identity}
\end{equation}
which we can check with
\begin{equation*}
	|\bm{k}\rangle
	=\int \frac{d^{3} p}{(2\pi)^{3}} \frac{1}{2\omega_{\bm p}}|\bm{p}\rangle\langle\bm{p}|\bm{k}\rangle
	=\int \frac{d^{3} p}{(2\pi)^{3}} \frac{1}{2\omega_{\bm p}} 2\omega_{\bm p}(2\pi)^3 \delta^3(\bm{p}-\bm{k})|\bm{p}\rangle
	=|\bm{k}\rangle.
\end{equation*}
We see that the identity operator (\ref{eq:rel-identity}) under such convention is Lorentz invariant, since it can be expressed as
\begin{equation}
	1 = 2\pi \int \frac{d^{3} p d\omega}{(2\pi)^{4}} \delta(\omega^2-{\bm{p}}^2-m^2) |\bm p\rangle\langle \bm p|.
\end{equation}

The single-particle defined above can be used to fix the normalization:
\begin{equation}
	\langle \bm k|\phi(\bm x,0)|0\rangle = e^{-i \bm k\cdot \bm x},
\end{equation}
leading to the field expansion
\begin{equation}
	\phi(x)
	=\int \frac{d^{3} k}{(2\pi)^{3}} \frac{1}{\sqrt{2\omega_{\bm k}}}\left(a_k
		e^{-i k \cdot x}+a_k^{\dagger} e^{i k \cdot x}\right).
\end{equation}

\subsubsection{Hamiltonian}

We can obtain the Hamiltonian for the Klein-Gordon field using the Legendre transformation:
\begin{equation}
\begin{aligned}
	H &= \int d^4 x\ \left[\pi(x) \dot{\phi}(x) - \mathcal L(x) \right] \\
	&= \int d^4 x\ \frac{1}{2} \left[\pi^2 + (\nabla \phi)^2 + m^2 \phi^2 \right]
\end{aligned}
\end{equation}
where the canonical momentum is defined as
\begin{equation}
\begin{aligned}
	\pi(x) &= \frac{\partial \mathcal L}{\partial \dot{\phi}} = \dot{\phi}(x) \\
	&= -i\int \frac{d^{3} k}{(2\pi)^{3}} \sqrt{\frac{\omega_{k}}{2}}\left(a_k
		e^{-i k \cdot x} - a_k^{\dagger} e^{i k \cdot x}\right)
\end{aligned}
\end{equation}
The $\pi^2$ term expands as
\begin{equation}
	\pi^2(x) = \int \frac{d^{3} k_1}{(2\pi)^{3}} \frac{d^{3} k_2}{(2\pi)^{3}}
		\frac{\sqrt{\omega_{k_1} \omega_{k_2}}}{2} \left(a^\dagger_{k_1}a_{k_2}e^{i(k_1-k_2)x} - a^\dagger_{k_1} a^\dagger_{k_2} e^{i(k_1+k_2)x} + h.c.\right).
\end{equation}
We note that after integrate over $x$, the phase factor $e^{i(k_1-k_2)x}$ produce a delta function for $k_1$ and $k_2$.
The $a^\dagger_{k_1} a^\dagger_{k_2}$ terms will finally be cancelled by other terms.
We temporally ignore such term.
The contribution from the first term is then
\begin{equation}
	\int d^4 x\ \pi^2(x) = \int \frac{d^3 k}{(2\pi)^3} \frac{\omega_k}{2} a_k^\dagger a_k + h.c.
\end{equation}
The second term is
\begin{equation}
	(\nabla \phi)^2 = \int \frac{d^{3} k_1}{(2\pi)^{3}} \frac{d^{3} k_2}{(2\pi)^{3}}
		\frac{\bm k_1 \bm k_2}{2\sqrt{\omega_{k_1}\omega_{k_2}}} \left(a^\dagger_{k_1}a_{k_2}e^{i(k_1-k_2)x} - a^\dagger_{k_1} a^\dagger_{k_2} e^{i(k_1+k_2)x} + h.c.\right).
\end{equation}
The third term is
\begin{equation}
	m^2 \phi^2 = \int \frac{d^{3} k_1}{(2\pi)^{3}} \frac{d^{3} k_2}{(2\pi)^{3}}
		\frac{m^2}{2\sqrt{\omega_{k_1}\omega_{k_2}}} \left(a^\dagger_{k_1}a_{k_2}e^{i(k_1-k_2)x} + a^\dagger_{k_1} a^\dagger_{k_2} e^{i(k_1+k_2)x} + h.c.\right).
\end{equation}
All three contributions sum up as
\begin{equation}
\begin{aligned}
	H &= \int \frac{d^3 k}{(2\pi)^3} \frac{1}{2}\left(\frac{\omega_k}{2} + \frac{\bm k^2+m^2}{2 \omega_k}\right) \left(a_k^\dagger a_k + h.c. \right) \\
	&= \int \frac{d^3 k}{(2\pi)^3} \omega_k \left(a_k^\dagger a_k +\frac{1}{2} \right).
\end{aligned}
\end{equation}

We can now check that the $a^\dagger a^\dagger$ terms indeed have no contributions, as the total contribution for each momentum $k$ is
\begin{equation}
	-\frac{\omega_k}{2} + \frac{\bm k^2}{2\omega_k} + \frac{m^2}{2\omega_k} = 0.
\end{equation}

The Hamiltonian in the operator form also make it manifest that
\begin{equation}
	H |\bm k\rangle = \omega_k |\bm k\rangle.
\end{equation}

\subsubsection{Correlation Function}
Consider the two-point correlation (propagator):
\begin{equation}
\begin{aligned}
	i\Delta(x_1-x_2) &\equiv \langle 0|T \phi(x_1) \phi(x_2) |0\rangle \\
	&= \theta(t_1-t_2) \langle 0|\phi(x_1) \phi(x_2) |0\rangle
	+ \theta(t_2-t_1) \langle 0|\phi(x_2) \phi(x_1) |0\rangle.
\end{aligned}
\end{equation}
Note that
\begin{equation}
	\langle 0|\phi(x_1) \phi(x_2) |0\rangle
	= \int\frac{d^{3} k}{(2\pi)^{3}}\frac{1}{2\omega_k} e^{i\bm k\cdot (\bm x_1-\bm x_2)-i\omega_{\bm k}\tau},
\end{equation}
where $\tau =t_1-t_2$.
The propagator can be written as
\begin{equation}
\begin{aligned}
	i\Delta(x_1-x_2)
	&= \int\frac{d^{3} k}{(2\pi)^{3}}\frac{1}{2\omega_k} e^{i\bm k\cdot (\bm x_1-\bm x_2)}\left[e^{-i\omega_{\bm k}\tau}\theta(\tau)+e^{i\omega_{\bm k}\tau}\theta(-\tau)\right] \\
	&= \int\frac{d^{3} k}{(2\pi)^{3}} e^{i\bm k\cdot (\bm x_1-\bm x_2)}\int \frac{d\omega}{2\pi i}\frac{-e^{i\omega\tau}}{\omega^2-\omega_k^2+i\epsilon} \\
	&= \int\frac{d^{4} k}{(2\pi)^{4}} e^{-i k\cdot (x_1-x_2)}\frac{i}{k^2-m^2+i\epsilon}.
\end{aligned}
\end{equation}
We have used the identity
\begin{equation*}
	\frac{1}{2\omega_k} \left[e^{-i\omega_{\bm k}\tau}\theta(\tau)+e^{i\omega_{\bm k}\tau}\theta(-\tau)\right]
	= \int \frac{d\omega}{2\pi i} \frac{-e^{i\omega\tau}}{\omega^2-\omega_k^2+i\epsilon},
\end{equation*}
where an infinitesimal number $\epsilon$ is included to move the singularities away from the real axis.
Any final result shall take the ($\epsilon \rightarrow 0^+$) limit.
Sometimes the infinitesimal $\epsilon$ will be absorbed into the mass, i.e., $m^2 \rightarrow m^2-i\epsilon$.

\subsection{Vector Field}

Although forbid by gauge invariance, we consider a vector field with nonzero mass term.
Actually the vector field can obtain mass from the spontaneous symmetry breaking.
For example the W and Z boson in the week interaction have nonzero mass.
The action with mass term is:
\begin{equation}
	S = \int \frac{d^4 k}{(2\pi)^4} \tilde{A}^{\mu *}(k) \left(-k^2 g_{\mu\nu}+k_\mu k_\nu + m^2 \right) \tilde{A}^\nu(k).
\end{equation}
The equation of motion for the action is
\begin{equation}
\begin{aligned}
	\frac{\delta S}{\delta \tilde{A}^{\mu *}(k)} = 0
	\quad \Longrightarrow \quad
	\left(-k^2 g_{\mu\nu}+k_\mu k_\nu + m^2 \right) \tilde{A}^\nu(k) = 0.
\end{aligned}
\end{equation}
Such equation is sometimes called the \textit{Proca equation}.
Note that the Proca equation implies
\begin{equation}
	\partial_\mu A^\mu = k_\nu \tilde{A}^\nu = 0.
\end{equation}
So the Proca equation becomes
\begin{equation}
	\left(-k^2 g_{\mu\nu} + m^2 \right) \tilde{A}^\nu(k) = 0,
\end{equation}
which is similar to the Klein-Gordon field with multiple components.

\subsubsection{Polarization Vectors}
Since we are now dealing with a field with space-time indices, it is helpful to introduce a set of basis vectors.
The general solution to the Proca equation is also a plane wave labelled by momentum $\bm k$.
For each momentum, we introduce a \textit{longitudinal polarization vector}
\begin{equation}
	\epsilon(\bm k,3) \equiv \left(\frac{|\bm k|}{m}, \frac{\bm k}{|\bm k|}\frac{k_0}{m} \right)
\end{equation}
and two \textit{transverse polarization vectors}
\begin{equation}
	\epsilon(\bm k, 1) \equiv (0, \bm \epsilon(\bm k, 1)), \quad
	\epsilon(\bm k, 2) \equiv (0, \bm \epsilon(\bm k, 2)),
\end{equation}
satisfying the orthogonal relation
\begin{equation}
	\bm \epsilon(\bm k, 1)\cdot \bm k = \bm \epsilon(\bm k, 2)\cdot \bm k = k^\mu \epsilon_\mu(\bm k, 3) = 0.
\end{equation}
So these three polarization vector together with 4-momentum $k$ form a basis for the space-time.
For the notational convenience, we define
\begin{equation}
	\epsilon(\bm k,0) \equiv \frac{k}{m}.
\end{equation}
The vector field can be
\begin{equation}
	A_\mu(\bm k, \lambda; x) \propto e^{-i\omega_k t + \bm k \cdot \bm x} \epsilon_\mu(\bm k, \lambda)
\end{equation}

The condition $k \cdot \tilde A = 0$ is satisfied if we require no mode in the $\epsilon(\bm k, 0)$ polarization.
Then the vector field is basically three independent scalar field, leading to the  field expansion:
\begin{equation}
	A^\mu(x) = \int \frac{d^{3} k}{(2\pi)^{3}}\frac{1}{\sqrt{2\omega_k}}
	\sum_{j=1}^3 \left[\epsilon^\mu(\bm k, j) a_{k,j} e^{-ik\cdot x} +
	\epsilon^{\mu}(\bm k, j) a^\dagger_{k,j} e^{ik\cdot x}\right].
\end{equation}
A single-particle state with polarization vector $\epsilon(\bm k, j)$ is defined as
\begin{equation}
	|k,\epsilon_j\rangle = \epsilon(\bm k, j) \sqrt{2\omega_k} a^\dagger_{j}|0\rangle.
\end{equation}

\subsubsection{Massless Polarization Vectors}

For the massless vector field, the polarization $\epsilon(\bm k, 3)$ is not well-defined.
We modify the definition to
\begin{equation}
\begin{aligned}
	\epsilon(\bm k, 0) &\equiv (1,0,0,0), \\
	\epsilon(\bm k, 3) &\equiv (0,0,0,1).
\end{aligned}
\end{equation}
Note that we have choose the spatial direction so that the momentum point to the z-direction.

We can add two types of virtual particles generated by $a^\dagger_{k,0}$ and $a^\dagger_{k,3}$ respectively, which are usually called the \textit{scalar photons} and \textit{longitudinal photons}.
However, the gauge fixing condition requires
\begin{equation}
	\partial_\mu A^\mu(x)|\psi\rangle = 0 \quad \Longrightarrow \quad
	(a_{k,0}-a_{k,3})|\psi\rangle = 0
\end{equation}
for all state $|\psi\rangle$ in the gauge-fixed Hilbert space.

We can show that the scalar and longitudinal modes are just the result of gauge transformation.
The physical polarization are the transverse polarization modes, and the field expansion is
\begin{equation}
	A^\mu(x) = \int \frac{d^{3} k}{(2\pi)^{3}}\frac{1}{\sqrt{2\omega_k}}
	\sum_{j=1}^2 \left[\epsilon^\mu(\bm k, j) a_{k,j} e^{-ik\cdot x} +
	\epsilon^{\mu}(\bm k, j) a^\dagger_{k,j} e^{ik\cdot x}\right].
\end{equation}

\subsubsection{Correlation Function}
To obtain the correlation for the massless vector field, we consider a modified Lagrangian:
\begin{equation}
	\mathcal L = -\frac{1}{4}F_{\mu\nu}F^{\mu\nu}-\frac{\xi}{2}(\partial_\mu A^\mu)^2.
\end{equation}
In the momentum space:
\begin{equation}
	\tilde{\mathcal L}_k = \tilde{A}^\mu(-k)\left(-k^2 g_{\mu\nu}+(1-\xi)k_\mu k_\nu\right) \tilde{A}^\nu(k)
\end{equation}
To construct the inverse matrix we make a general symmetric ansatz
\begin{equation}
	(G_\gamma^{-1})^{\mu\nu}(k)=A\left(k^{2}\right) g^{\mu\nu}+B\left(k^{2}\right) k^{\mu} k^{\nu}.
\end{equation}
Requiring that
\begin{equation}
	(G_\gamma)_{\mu\sigma}(k)(G_\gamma^{-1})^{\sigma\nu}(k) = \delta_\mu^\nu,
\end{equation}
and comparing the coefficients, we get the conditions
\begin{equation}\label{eq:qft-propagator-equation}
\begin{aligned}
	-k^{2} A\left(k^{2}\right) &=1, \\
	\xi k^{2} B\left(k^{2}\right) &=(\xi-1) A\left(k^{2}\right).
\end{aligned}
\end{equation}
In the case $\xi=0$ these equations are not compatible. Without the gauge-fixing term the matrix $(G_\gamma)_{\mu \nu}$ cannot be inverted (since the determinant vanishes) and the Feynman propagator cannot be constructed. If $\xi \neq 0$, however, no problems arise and the system of equations (\ref{eq:qft-propagator-equation}) is solved by
\begin{equation}
	A\left(k^{2}\right)=-\frac{1}{k^{2}}, \quad B\left(k^{2}\right)=\frac{\xi-1}{\xi} \frac{1}{\left(k^{2}\right)^{2}},
\end{equation}
which leads to
\begin{equation}
	G_\gamma(k) = \frac{-g^{\mu\nu}+(1-\xi)k^\mu k^\nu}{k^2}.
\end{equation}
Different choice of $\xi$ correspond to different gauge fixing.
The Landau gauge choose $\xi=1$, and the propagator has the simplest form
\begin{equation}
	G_\gamma(k) = \frac{-g_{\mu\nu}}{k^2}.
\end{equation}

\subsection{Spinor Field}
The equation of motion for Dirac field is
\begin{equation}
\begin{aligned}
	\partial_\mu\frac{\partial\mathcal{L}}{\partial(\partial_\mu\psi)} - \frac{\partial \mathcal L}{\partial \psi} = 0
	\quad &\Longrightarrow \quad
	\bar\psi(i\overleftarrow{\cancel \partial}-m) = 0, \\
	\partial_\mu\frac{\partial\mathcal{L}}{\partial(\partial_\mu\bar\psi)} - \frac{\partial \mathcal L}{\partial \bar\psi} = 0
	\quad &\Longrightarrow \quad
	(i\overrightarrow{\cancel \partial}-m)\psi = 0.
\end{aligned}
\end{equation}
This EOM is a matrix equation.
The general solution of the Dirac equation can be written as a linear combination of plane waves (with positive and negative energy):\footnote{Note that we have chosen to put the $+$ sign into the exponential, rather than having $p^{0}<0$.}
\begin{equation}
	\psi_p(x) = \begin{cases}
		u(p) e^{-i p \cdot x} & p^{0}>0 \\
		v(p) e^{+i p \cdot x} & p^{0}<0
	\end{cases}, \quad p^{2}=m^{2}.
\end{equation}
In momentum space, $u(p)$ and $v(p)$ satisfies:
\begin{equation}
	\left[\begin{array}{cc}
		-m & p \cdot \sigma \\ p\cdot \bar\sigma & -m
	\end{array} \right] u_s(p) =
	\left[\begin{array}{cc}
		-m & -p \cdot \sigma \\ -p\cdot \bar\sigma & -m
	\end{array} \right] v_s(p) = 0
\end{equation}
For massive Dirac field, we can choose the rest frame where $p = (m,0,0,0)$, the matrix equation is\footnote{We first consider the case where there is only one spatial dimension. It correspond to the choice of coordinate such that the momentum point to the $z$ direction.}
\begin{equation}
\begin{aligned}
	\left[\begin{array}{cc}
		-m & m \\
		m & -m
	\end{array}\right] u_s = 0
	\quad &\Longrightarrow \quad
	u_s = \sqrt{m}\left[\begin{array}{c}
		\xi_s \\ \xi_s
	\end{array}\right], \\
	\left[\begin{array}{cc}
		m & m \\
		m & m
	\end{array}\right] v_s = 0
	\quad &\Longrightarrow \quad
	v_s = \sqrt{m}\left[\begin{array}{c}
		\eta_s \\ -\eta_s
	\end{array}\right],
\end{aligned}
\end{equation}
where $\xi$ and $\eta$ has two independent solutions.
For example, four linearly independent solutions are
\begin{equation}
	u_{\uparrow} = \left[\begin{array}{c} 1 \\ 0 \\ 1 \\ 0 \end{array}\right], \quad
	u_{\downarrow} = \left[\begin{array}{c} 0 \\ 1 \\ 0 \\ 1 \end{array}\right], \quad
	v_{\uparrow} = \left[\begin{array}{c} -1 \\ 0 \\ 1 \\ 0 \end{array}\right], \quad
	v_{\downarrow} = \left[\begin{array}{c} 0 \\ 1 \\ 0 \\ -1 \end{array}\right].
\end{equation}
The Dirac spinor is a complex four-component object, with eight real degrees of freedom.
The equations of motion reduce it to four degrees of freedom, which, as we will see, can be interpreted as spin up and spin down for particle and antiparticle.

\subsubsection{Solution in General Frame}
To derive a more general expression, we can solve the equations again in the boosted frame and match the normalization.
If $p=(E,0,0,p_z)$ then
\begin{equation}
	p \cdot \sigma=\left[\begin{array}{cc}
		E-p_{z} & 0 \\
		0 & E+p_{z}
	\end{array}\right], \quad
	p \cdot \bar{\sigma}=\left[\begin{array}{cc}
		E+p_{z} & 0 \\
		0 & E-p_{z}
	\end{array}\right].
\end{equation}
Let $a=\sqrt{E-p_{z}}$ and $b=\sqrt{E+p_{z}}$, then $m^{2}=\left(E-p_{z}\right)\left(E+p_{z}\right)=a^{2} b^{2}$ and Dirac equation becomes
\begin{equation}
	\left[\begin{array}{cccc}
		-a b & 0 & a^{2} & 0 \\
		0 & -a b & 0 & b^{2} \\
		b^{2} & 0 & -a b & 0 \\
		0 & a^{2} & 0 & -a b
	\end{array}\right] u_{s}(p) =
	\left[\begin{array}{cccc}
		a b & 0 & a^{2} & 0 \\
		0 & a b & 0 & b^{2} \\
		b^{2} & 0 & a b & 0 \\
		0 & a^{2} & 0 & a b
	\end{array}\right] v_{s}(p) = 0.
\end{equation}

The solutions are
\begin{equation}
	u_{s}=\left(\begin{array}{ll}
	\left[\begin{array}{ll}
		a & 0 \\
		0 & b
	\end{array}\right] \xi_{s} \\
	\left[\begin{array}{ll}
		b & 0 \\
		0 & a
	\end{array}\right] \xi_{s}
	\end{array}\right), \quad
	v_{s}=\left(\begin{array}{ll}
	\left[\begin{array}{ll}
		a & 0 \\
		0 & b
	\end{array}\right] \eta_{s} \\
	-\left[\begin{array}{ll}
		b & 0 \\
		0 & a
	\end{array}\right] \eta_{s}
	\end{array}\right).
\end{equation}
Using
\begin{equation}
	\sqrt{p \cdot \sigma}=\left[\begin{array}{cc}
		\sqrt{E-p_{z}} & 0 \\
		0 & \sqrt{E+p_{z}}
	\end{array}\right], \quad
	\sqrt{p \cdot \bar{\sigma}}=\left[\begin{array}{cc}
		\sqrt{E+p_{z}} & 0 \\
		0 & \sqrt{E-p_{z}}
	\end{array}\right],
\end{equation}
we can write more generally
\begin{equation}
	u_{s}(p) = \left(\begin{array}{c}
		\sqrt{p \cdot \sigma} \xi_{s} \\
		\sqrt{p \cdot \bar{\sigma}} \xi_{s}
	\end{array}\right), \quad
	v_{s}(p) = \left(\begin{array}{c}
		\sqrt{p \cdot \sigma} \eta_{s} \\
		-\sqrt{p \cdot \bar{\sigma}} \eta_{s}
	\end{array}\right),
\end{equation}
where the square root of a matrix can be defined by changing to the diagonal basis, taking the square root of the eigenvalues, then changing back to the original basis.
In practice, we will usually pick $p$ along the $z$ axis, so we do not need to know how to make sense of $\sqrt{p \cdot \sigma}$. Then the four solutions are
\begin{equation}\label{eq:qft-dirac-solutions}
\begin{aligned}
	u^{1}(p) &= \left(\begin{array}{c}
		\sqrt{E-p_{z}} \\ 0 \\
		\sqrt{E+p_{z}} \\ 0
	\end{array}\right), &
	u^{2}(p) &= \left(\begin{array}{c}
		0 \\ \sqrt{E-p_{z}} \\
		0 \\ \sqrt{E+p_{z}}
	\end{array}\right), \\
	v^{1}(p) &= \left(\begin{array}{c}
		\sqrt{E-p_{z}} \\ 0 \\
		-\sqrt{E+p_{z}} \\ 0
	\end{array}\right), &
	v^{2}(p) &= \left(\begin{array}{c}
		0 \\ \sqrt{E-p_{z}} \\
		0 \\ -\sqrt{E+p_{z}}
	\end{array}\right).
\end{aligned}
\end{equation}
In any frame $u^{s}$ are the positive frequency electrons, and the $v^{s}$ are negative frequency electrons, or equivalently, positive frequency positrons.

For massless spinors, $p_{z}=\pm E$ and the explicit solutions in Eq. (\ref{eq:qft-dirac-solutions}) are 4-vectors with one non-zero component describing spinors with fixed helicity.
The spinor solutions for massless electrons are sometimes called polarizations, and are useful for computing polarized electron scattering amplitudes.

For Weyl spinors, there are only four real degrees of freedom off-shell and two real degrees of freedom on-shell.
Recalling that the Dirac equation splits up into separate equations for $\psi_{L}$ and $\psi_{R}$, the Dirac spinors with zeros in the bottom two rows will be $\psi_{L}$ and those with zeros in the top two rows will be $\psi_{R}$.
Since $\psi_{L}$ and $\psi_{R}$ have two degrees of freedom each, these must be particle and antiparticle for the same helicity.
The embedding of Weyl spinors into fields this way induces irreducible unitary representations of the Poincare group for $m=0$.

\subsubsection{Normalization and Spin Sum}
The normalization chosen this way gives the orthogonal relation:
\begin{equation}\label{eq:qft-dirac-otho-1}
\begin{aligned}
	\bar{u}^{r}(p) u^{s}(p) &= +2 m \delta^{r s}, \\
	\bar{v}^{r}(p) v^{s}(p) &= -2 m \delta^{r s}.
\end{aligned}
\end{equation}
This is the (conventional) normalization for the spinor inner product for massive Dirac spinors.
It is also easy to check that
\begin{equation}
	\bar u_s(p) v_{s'}(p) = \bar v_s(p) u_{s'}(p) = 0.
\end{equation}
We can further check that an additional orthogonal relation hold
\begin{equation}
\begin{aligned}
	u^{r \dagger}(p) u^{s}(p) &= -2 \omega_{\bm p} \delta^{r s}, \\
	v^{r \dagger}(p) v^{s}(p) &= +2 \omega_{\bm p} \delta^{r s}.
\end{aligned}
\end{equation}
And if we define $\bar p \equiv (E,-\vec p)$, there is another set of orthogonal relation:
\begin{equation}
	u^{r\dagger}(p) v^{s}(\bar p) =
	v^{r\dagger}(p) u^{s}(\bar p) =0.
\end{equation}
A useful identity is the spin sum identity:
\begin{equation}
\begin{aligned}
	\sum_{s} u^{s}(p) \bar{u}^{s}(p) &= \cancel p+m, \\
	\sum_{s} v^{s}(p) \bar{v}^{s}(p) &= \cancel p-m.
\end{aligned}
\end{equation}

\subsubsection{Field Expansion and Correlation}
The Dirac field expansion is
\begin{equation}
\begin{aligned}
	\psi(x) &=\int \frac{d^{3} p}{(2 \pi)^{3}} \frac{1}{\sqrt{2 \omega_{\mathbf{p}}}}
		\sum_{s}\left(a_{\mathbf{p}}^{s} u^{s}(p) e^{-i p \cdot x}
		+b_{\mathbf{p}}^{s \dagger} v^{s}(p) e^{i p \cdot x}\right), \\
	\bar{\psi}(x) &=\int \frac{d^{3} p}{(2 \pi)^{3}} \frac{1}{\sqrt{2 \omega_{\mathbf{p}}}}
		\sum_{s}\left(b_{\mathbf{p}}^{s} \bar{v}^{s}(p) e^{-i p \cdot x}
		+a_{\mathbf{p}}^{s \dagger} \bar{u}^{s}(p) e^{i p \cdot x}\right).
\end{aligned}
\end{equation}
Now let us investigate the propagator
\begin{equation}
\begin{aligned}
	iD_{F,\alpha\beta}(x_1-x_2) &= \langle0|T\psi_\alpha(x_1)\bar\psi_\beta(x_2)|0\rangle \\
	&= \theta(\tau) \langle0|\psi_\alpha(x_1)\bar\psi_\beta(x_2)|0\rangle - \theta(-\tau) \langle0|\bar\psi_\beta(x_2)\psi_\alpha(x_1)|0\rangle.
\end{aligned}
\end{equation}
On the RHS, the first term is
\begin{equation*}
\begin{aligned}
	\langle0|\psi_\alpha(x_1)\bar\psi_\beta(x_2)|0\rangle
	&= \int \frac{d^{3} p}{(2 \pi)^{3}} \frac{1}{\sqrt{2 \omega_{\mathbf{p}}}} \left[\sum_s u_\alpha^s(p)\bar u_\beta^s(p)\right]e^{-i p\cdot (x_1-x_2)} \\
	&= (i\cancel \partial+m)_{\alpha\beta}\int \frac{d^{3} p}{(2 \pi)^{3}} \frac{1}{\sqrt{2 \omega_{\mathbf{p}}}} e^{-i p\cdot (x_1-x_2)}.
\end{aligned}
\end{equation*}
For the second term:
\begin{equation*}
\begin{aligned}
	\langle0|\bar\psi_\beta(x_2)\psi_\alpha(x_1)|0\rangle
	&= \int \frac{d^{3} p}{(2 \pi)^{3}} \frac{1}{\sqrt{2 \omega_{\mathbf{p}}}} \left[\sum_s \bar v_\beta^s(p)v_\alpha^s(p)\right]e^{i p\cdot (x_1-x_2)} \\
	&= -(i\cancel \partial + m)_{\alpha\beta}\int \frac{d^{3} p}{(2 \pi)^{3}} \frac{1}{\sqrt{2 \omega_{\mathbf{p}}}} e^{i p\cdot (x_1-x_2)}.
\end{aligned}
\end{equation*}
Together, the Dirac propagator is:
\begin{equation}
\begin{aligned}
	iD_F(x_1-x_2) &= (i\cancel \partial+m)i\Delta(x_1-x_2) \\
	&= \int\frac{d^{4} p}{(2\pi)^{4}} e^{-i p\cdot (x_1-x_2)}\frac{i(\cancel p+m)}{p^2-m^2+i\epsilon}.
\end{aligned}
\end{equation}

\section{Path-integral Quantization}

\subsection{Scalar Field}
Consider the action for free field with source
\begin{equation}
	S_0[\phi,J]
	= \int d^dx\left[\mathcal{L}_0 + J(x)\cdot\phi(x) \right].
\end{equation}
In the path integral formalism, we consider the partition function
\begin{equation}
	Z_0[J] = \int D[\phi] \exp(iS_0[\phi,J])
	\equiv Z[0] \exp(iW_0[J]).
\end{equation}
where we have introduced a new quantity
\begin{equation}
\begin{aligned}
	W_0[J] = -\frac{1}{2}\int d^dx_1 d^dx_2 J(x_1)\Delta_0(x_1-x_2)J(x_2).
\end{aligned}
\end{equation}
For free field, the free propagator $\Delta_0(x_1-x_2)$ is:
\begin{equation}
	i\Delta_0(x_1-x_2) = \langle 0| T\phi(x_1)\phi(x_2)|0\rangle
	= \frac{\delta}{i\delta J(x_1)}\frac{\delta}{i\delta J(x_2)} iW_0[J].
\end{equation}
Now we evaluate the propagator in the path-integral formalism.
In momentum space, the free action (with source) is
\begin{equation*}
	\frac{1}{V}\sum_k \left[\frac{1}{2}\tilde\phi^*(k)( k^2-m^2)\tilde\phi(k)+\tilde J^*(k)\cdot\tilde\phi(k)+\tilde\phi^*(k)\cdot\tilde J(k)\right].
\end{equation*}
For real field, $\tilde\phi^*(k) = \tilde\phi(-k)$.
For our convenience, we have expressed the momentum integral as summation.
Actually, consider the $d$-dimensional box of size $L^d$, the momentum along each axis is multiple of $2\pi/L$, so when $L\rightarrow \infty$, the summation approaches in integral,
\begin{equation*}
	\frac{1}{V}\sum_k \rightarrow \int \frac{d^d k}{(2\pi)^d}.
\end{equation*}
Let us omit the $1/V$ factor, the summation can be formally expressed as
\begin{equation}
	\frac{1}{4}\mathbf{v}^T \cdot \mathbf M\cdot \mathbf{v} + \frac{1}{2}\mathbf{j}^T \cdot \mathbf{v}
\end{equation}
where
\begin{equation*}
	\mathbf v = \bigoplus_{|\mathbf k|} \left[
	\begin{array}{c}
		\tilde{\phi}(k) \\
		\tilde{\phi}^*(k)
	\end{array}\right],\
	\mathbf M = \bigoplus_{|\mathbf k|} \left[
	\begin{array}{cc}
		0 & k^2-m^2 \\
		k^2-m^2 & 0
	\end{array}\right],\
	\mathbf j = \bigoplus_{|\mathbf k|} \left[
	\begin{array}{c}
		\tilde{J}^*(k) \\
		\tilde{J}(k)
	\end{array}\right].
\end{equation*}
Note that in the above expression, we have made an infinitesimal shift of mass ($m^2 \rightarrow m^2 - i\epsilon$) to ensure the convergence of the Gaussian integral.
The integrated variables $v_i$ is not real.
To use the real Gaussian integral formula, we make use of a unitary transformation:
\begin{equation*}
	\mathbf U = \frac{1}{\sqrt 2} \left[\begin{array}{cc}
		1 & 1 \\
		-i & i
	\end{array}\right], \quad
	\mathbf U \cdot \left[
	\begin{array}{c}
		\tilde{\phi}(k) \\
		\tilde{\phi}^*(k)
	\end{array}\right]
	= \frac{1}{\sqrt 2}\left[
	\begin{array}{c}
		\tilde\phi(k)+\tilde\phi^*(k) \\
		-i\tilde\phi(k)+i\tilde\phi^*(k)
	\end{array}\right]
	\equiv \left[
	\begin{array}{c}
		\tilde\phi_1(k) \\
		\tilde\phi_2(k)
	\end{array}\right]
\end{equation*}
The path integral then becomes a real field integral.
Recall the real Gaussian integral formula:
\begin{equation}
	\int d\mathbf x \exp\left(-\frac{1}{2}\mathbf{x}^T \cdot \mathbf A \cdot \mathbf{x} + \mathbf{B}^T \cdot \mathbf{x}\right)
	= \sqrt{\frac{(2\pi)^N}{\det{\mathbf A}}}\exp\left(\frac{1}{2}\mathbf{B}^T \cdot \mathbf{A}^{-1} \cdot \mathbf{B}\right),
	\label{eq:real-gaussian-integral}
\end{equation}
For the field integral, we absorbed the $(2\pi)^{N/2}$ term into the measure, and express the path integral for the Gaussian field as:
\begin{equation}
	W_0[J]
	= -\frac{i}{4}\int \frac{d^d k}{(2\pi)^d} \mathbf j^T_k \cdot \mathbf M^{-1}_k \cdot \mathbf j_k
	= -\frac{1}{2} \int \frac{d^d k}{(2\pi)^d}  \tilde{J}^*(k) \tilde{\Delta}_0(k) \tilde{J}(k).
\end{equation}
This gives the propagator in the momentum space:
\begin{equation}
	\tilde{\Delta}_0(k) = \frac{i}{k^2-m^2}
	\quad \Longrightarrow \quad
	\Delta_0(x_1-x_2) = i\int\frac{d^{4} k}{(2\pi)^{4}} \frac{e^{-i k\cdot (x_1-x_2)}}{k^2-m^2}.
\end{equation}

\subsubsection{From Field to Force}
Consider two separate particle described by the delta function $J_a(x) = \delta^{(3)}(\bm x - \bm x_a)$, together the source is
\begin{equation}
	J(x) = J_1(x) + J_2(x).
\end{equation}
Adding the source,
\begin{equation*}
	W_0[J] = -\frac{1}{2}\int d^4x_1 d^4 x_2 J(x_1) \Delta_0(x_1-x_2) J(x_2)
\end{equation*}
Omit the self energy terms $J_1^2(x), J_2^2(x)$, $W_0[J]$ is
\begin{equation}
\begin{aligned}
	W_0[J] &= -\int d^4 y_1 d^4 y_2\ e^{-ik^0(y_1^0-y_2^0)}\int \frac{d^4 k}{(2\pi)^4} J_1(y_1)\frac{e^{i\bm k\cdot (\bm y_1-\bm y_2)}}{k^2-m^2} J_2(y_2) \\
	&= -\int  dt \int d (y_1^0 - y_2^0) \ e^{-ik^0(y_1^0-y_2^0)}\int \frac{d^4 k}{(2\pi)^4} \frac{e^{i\bm k\cdot (\bm y_1-\bm y_2)}}{k^2-m^2} \\
	&= \left(\int dt \right)\int \frac{d^3 k}{(2\pi)^3} \frac{e^{i\bm k\cdot (\bm y_1-\bm y_2)}}{\bm k^2 + m^2}
\end{aligned}
\end{equation}
Recall that the partition function is actually infinite:
\begin{equation}
	Z_0 \sim \langle 0| e^{-i H_0 T} |0\rangle \quad \Longrightarrow \quad
	W_0 = -i E T,
\end{equation}
where $E$ is the energy.
Writing $\bm r \equiv \bm y_1 - \bm y_2$, and $u \equiv \cos\theta$ with $\theta$ the angle between $\bm k$ and $\bm r$, the volume form is $dk \cdot kd\theta \cdot  2\pi k \sin \theta = 2\pi k^2 dk du$, and the integral is
\begin{equation}
\begin{aligned}
	E &= -\int \frac{d^3 k}{(2\pi)^3} \frac{e^{i k r u}}{k^2 + m^2} \\
	&= - \frac{1}{(2\pi)^2} \int_0^\infty k^2 dk \int_{-1}^1 du \frac{e^{ikru}}{k^2 +m^2} \\
	&= -\frac{1}{2\pi^2 r} \int_0^\infty k  \frac{\sin kr}{k^2 +m^2} dk.
\end{aligned}
\end{equation}
Since the integral is even, we can extend the integral to
\begin{equation}
\begin{aligned}
	E &= -\frac{1}{4\pi^2 r} \int_{-\infty}^\infty k  \frac{\sin kr}{k^2 +m^2} dk \\
	&= \frac{i}{4\pi^2 r} \int_{-\infty}^\infty \frac{k e^{ikr}}{k^2 +m^2} dk
\end{aligned}
\end{equation}
The residue theorem gives
\begin{equation}
	\int_{-\infty}^\infty \frac{k e^{ikr}}{k^2 +m^2} dk = \pi ie^{-mr}
\end{equation}
So we get the potential of two particles:
\begin{equation}\label{eq:field-to-force}
	V(r) = -\frac{e^{-mr}}{4\pi r},
\end{equation}
and the attractive force is
\begin{equation}
	F(r) = -\frac{dV}{dr} = -(1+mr)\frac{e^{-mr}}{4\pi r^2}.
\end{equation}
We see that in the massless case, the force gives the long-range Coulomb force $F \propto 1/r^2$, while in the massful field theory, the force is short-ranged, with the decay length proportional to the mass.

\subsection{Vector Field}
We define the gauge fixing function
\begin{equation*}
	G(A) = \partial_\mu A^\mu(x) -\omega(x) = 0
\end{equation*}
The gauge transformation has the form:
\begin{equation*}
	A^\alpha_\mu(x) = A_\mu(x) + \partial_\mu \alpha(x).
\end{equation*}
We then have
\begin{equation*}
	1 \propto \int D[\alpha] \det\left(\frac{\delta G(A^\alpha)}{\delta \alpha}\right) \delta(G(A)).
\end{equation*}
Inset the identity operator into the path integral formula
\begin{equation*}
	Z[J] \propto \det\left(\partial^2 \right) \int D[\alpha]D[A] e^{iS[A,J]} \delta(\partial_\mu A^\mu -\omega(x)).
\end{equation*}
The above equation does not depend on $\omega(x)$.
We can then integrate over $\omega(x)$ with gaussian weight
\begin{equation*}
\begin{aligned}
	Z[J] &\propto \int D[\omega] e^{-i\int d^d x \frac{\omega^2}{2\xi}} \int D[\alpha]D[A] e^{iS[A,J]}
	\delta(\partial_\mu A^\mu-\omega) \\
	&= \int D[A] e^{iS[A,J]} \exp\left\{i \left[S[A,J]-\int d^d x \frac{1}{2\xi}(\partial_\mu A^\mu)^2 \right]\right\}.
\end{aligned}
\end{equation*}
In momentum space, the modified Langriangian is
\begin{equation*}
	\tilde{\mathcal{L}}_\xi(k) = \tilde{A}^\mu(k)\left[
		-k^2 g_{\mu\nu}+\left(1-\frac{1}{\xi}\right)k_\mu k_\nu
		\right] \tilde{A}^\nu(-k) +
		\tilde{J}_\mu(k) \tilde{A}^\mu(-k) +
		\tilde{A}^\mu(k) \tilde{J}_\mu(-k).
\end{equation*}
In the momentum space, the photon propagator is
\begin{equation}\label{eq:qft-photon-momentum-propagator}
\begin{aligned}
	\tilde G^{\mu\nu}(k)
	&= \left[-k^2 g_{\mu\nu}+\left(1-\frac{1}{\xi}\right)k_\mu k_\nu\right]^{-1} \\
	&= \frac{-g^{\mu\nu}+(1-\xi)k^\mu k^\nu}{k^2}.
\end{aligned}
\end{equation}
Thus, the partition function is
\begin{equation}
	\frac{Z_{\mathrm{maxwell}}[J]}{Z_{\mathrm{maxwell}}[0]}
	= \exp\left[-\frac{i}{2}\int d^dx_1 d^dx_2 J_\mu(x_1) G^{\mu\nu}(x_1-x_2) J_\nu(x_2) \right],
\end{equation}
where the real-space propagator is
\begin{equation}
	G^{\mu\nu}(x_1-x_2) = \int \frac{d^d k}{(2\pi)^d} e^{-ik\cdot(x_1-x_2)}\frac{-g^{\mu\nu}+(1-\xi)k^\mu k^\nu}{k^2}.
\end{equation}
Note that the propagator is related to the two-point correaltion:
\begin{equation}
\begin{aligned}
	\langle 0|T A^\mu(x_1) A^\nu(x_2) |0\rangle
	&= \left.\frac{1}{Z_{\mathrm{Maxwell}}[0]}\frac{\delta}{iJ_\mu(x_1)}\frac{\delta}{iJ_\nu(x_2)} Z_{\mathrm{Maxwell}}[J]\right|_{J=0} \\
	&= iG^{\mu\nu}(x_1-x_2).
\end{aligned}
\end{equation}

\subsection{Spinor Field}
Consider the partition function with source
\begin{equation}
	Z_{\mathrm{Dirac}}[J]
	= \int D[\bar\psi,\psi] \exp\left[i\int d^dx \left(\mathcal{L}_{\mathrm{Dirac}}+\bar{\eta}\psi + \bar\psi\eta \right) \right].
\end{equation}
In momentum space:
\begin{equation}
	S = \int\frac{d^d k}{(2\pi)^d} \left[
		\tilde{\bar\psi}(k)(\cancel{k}-m)\tilde{\psi}(k) +
		\tilde{\bar\eta}(k) \tilde{\psi}(k) +
		\tilde{\bar\psi}(k) \tilde{\eta}(k)
	\right].
\end{equation}
Using the Gaussian integral formula (for Grassman variables), the partition function is:
\begin{equation}
\begin{aligned}
	\frac{Z_{\mathrm{Dirac}}[J]}{Z_{\mathrm{Dirac}}[0]}
	&= \exp\left[-i\int \frac{d^d k}{(2\pi)^d} \tilde{\bar\eta}(k)\frac{1}{\cancel{k}-m}\tilde\eta(k)\right] \\
	&= \exp\left[-i\int d^dx_1 d^d x_2 \bar{\eta}(x_1)\cdot D_F(x_1-x_2)\cdot \eta(x_2) \right]
\end{aligned}
\end{equation}

where
\begin{equation}
	D_F(x_1-x_2) = \int \frac{d^d k}{(2\pi)^d} \frac{e^{-i k \cdot (x_1-x_2)}}{\cancel{k}-m}
	= \int \frac{d^d k}{(2\pi)^d} \frac{\cancel{k}+m}{k^2-m^2} e^{-i k \cdot (x_1-x_2)}.
\end{equation}
Note that the propagator is
\begin{equation}
\begin{aligned}
	\langle 0| T \psi^\alpha(x_1) \bar\psi^\beta(x_2) |0\rangle
	&= \left.\frac{1}{Z_{\mathrm{Dirac}}[0]}\frac{\delta}{i\delta \bar{\eta}_\alpha(x_1))}\frac{i\delta}{\delta\eta_\beta(x_2)} Z_{\mathrm{Dirac}}[\bar\eta,\eta]\right|_{\eta=\bar\eta=0} \\
	&= i D^{\alpha\beta}_F(x_1-x_2),
\end{aligned}
\end{equation}
where the sign in the variational derivative comes from the anti-commutation relation of the fermionic fields.