\section{Mathematical logic}\label{sec:mathematical_logic}

Mathematical logic uses mathematics to study logic and vice versa.

We start with objects that are purely logical in nature --- formulas --- which are strings of symbols that represent truth values. Formal definitions for formulas are given here using \hyperref[def:formal_grammar]{grammars}, which in turn depend on \hyperref[def:formal_language]{languages}. Formal definitions for truth values are given using \hyperref[def:heyting_algebra]{Heyting} and \hyperref[def:boolean_algebra]{Boolean algebras}

These definitions help us define the theory necessary to study following two important intertwined topics:
\begin{itemize}
  \item We are interested in establishing whether the formula \( \varphi \) logically entails the formula \( \psi \). This is done using \hyperref[def:deductive_system]{deductive systems} which specify precisely how we can manipulate strings of symbols. This aspect is called \term{syntactic} or \term{logical} and is the basis or \hyperref[def:proof_derivability]{proof theory}. Formulas allow us to express statements about mathematics and proof theory allows us to systematically study the relationships between them.

  \item Given a formula, we are interested in assigning a meaning to it. Different logical systems provide different syntax that is useful for different purposes - \hyperref[def:propositional_syntax/formula]{propositional formulas} allow us to express complex relationships between propositions via \hyperref[subsec:boolean_functions]{Boolean functions} while \hyperref[def:first_order_syntax/formula]{first-order logic formulas} allow us to go one level lower and give a precise meaning to these propositions via \hyperref[def:first_order_structure]{structures}. This aspect of logic is called \term{semantical} and is the basis of \hyperref[subsec:first_order_models]{model theory}. Model theory allows us to study logical formulas using pre-existing mathematics.
\end{itemize}

There are two aspects in which logical systems are categorized:
\begin{itemize}
  \item \hyperref[subsec:propositional_logic]{Propositional} and \hyperref[subsec:first_order_logic]{first-order logic} (among others) differ in what their syntax allows us to express. This also means that they differ in what their semantics can express, but, just as the syntax of first-order logic is a superset of the syntax of propositional logic, \hyperref[subsec:boolean_functions]{Boolean functions} can express relations between quantifierless atomic formulas in any \hyperref[def:first_order_structure]{structures}. In other words, semantics are identical in places where the syntax is the same.

  \item \hyperref[def:classical_logic]{Classical} and \hyperref[def:intuitionistic_logic]{intuitionistic logic} (among others) differ in their semantics and their logical \hyperref[def:judgment/inference_rule]{inference rules}. This has two implications
  \begin{itemize}
     \item Boolean functions describe \hyperref[def:classical_logic]{classical logic}, however they fail to describe \hyperref[def:intuitionistic_logic]{intuitionistic logic} because double negation elimination \eqref{eq:thm:minimal_propositional_negation_laws/dne} no longer holds and neither do other similar statements. So, while retaining the same syntax, we must resort to much more complicated semantical frameworks like \hyperref[def:propositional_heyting_algebra_semantics]{Heyting} or \hyperref[def:propositional_topological_semantics]{topological semantics}.

     \item The proof theory that describes classical logic no longer matches the semantics, hence we must resort to other proof systems. This turns out not to be trivial because we need a clear understanding of which logical axioms imply the others. \Fullref{subsec:deductive_systems} lists different proof systems and their corresponding semantics.
  \end{itemize}
\end{itemize}

\begin{remark}\label{rem:metalogic}
  The statements of mathematical logic can themselves be studied logically. We distinguish between the \term{object logic} which we study and the \term{metalogic} which we use to study it. It is possible, for example, to study intuitionistic propositional logic using classical first-order logic. The metalogic is usually less formal and its statements are written in prose for the sake of easier understanding.

  It is an exercise in futility to try and completely formalize the language, syntax and theory of the metalogic --- the \term{metalanguage}, \term{metasyntax} and \term{metatheory}. We must take a given metalogical framework for granted and then study a certain object logical framework. This is not to say that the principles and rules that hold in the metalogic are immaterial --- see for example the discussion of the differences between \hyperref[def:intuitionistic_logic]{intuitionistic logic} and \hyperref[def:classical_logic]{classical logic}. This is to say that it makes little sense to attempt to study the metalogic because at that point it becomes the object logic and the still more abstract conceptual framework in which we reason about the metalogic now becomes the new metalogic. We can thus form a hierarchy that is unbounded in both directions --- we can study a more concrete object logic within the object logic, and we can jump from one metalogical level to the next.

  An important connection between the logic and metalogic is given in \fullref{rem:set_definition_recursion}.
\end{remark}

\begin{definition}\label{def:classical_logic}
  Classical logic is a vague term that we use to describe a semantic framework, \fullref{def:propositional_semantics}, and a matching \hyperref[def:deductive_system]{deductive system}, \fullref{def:classical_propositional_deductive_systems}, for \hyperref[subsec:propositional_logic]{propositional logic} and also a semantic framework, \fullref{def:first_order_semantics}, and matching proof deduction deduction, \fullref{def:first_order_natural_deduction_system}, for \hyperref[subsec:first_order_logic]{first-order logic}, among others.

  It is characterized by the ability to use the law of double negation elimination \eqref{eq:thm:minimal_propositional_negation_laws/dne}. A more popular (but less accurate due to \fullref{thm:minimal_propositional_negation_laws}) characterization is that the law of the excluded middle \eqref{eq:thm:minimal_propositional_negation_laws/lem} holds. Within the metalogic, this law is called the \term{principle of bivalence} and states that either a statement holds or its negation holds.
\end{definition}

\begin{definition}\label{def:intuitionistic_logic}
  Intuitionistic logic is a generalization of \hyperref[def:classical_logic]{classical logic}. It is also called \term{constructive logic} due to the \hyperref[def:brouwer_heyting_kolmogorov_interpretation]{Brouwer-Heyting-Kolmogorov interpretation}. See \fullref{rem:brouwer_heyting_kolmogorov_interpretation_compatibility} for further discussion of the topic.

  Instead of the law of the excluded middle \eqref{eq:thm:minimal_propositional_negation_laws/lem}, we have the strictly weaker principle of explosion \eqref{eq:thm:minimal_propositional_negation_laws/efq} stating that everything can be proved given a contradiction.

  To these ideas there correspond \hyperref[def:propositional_heyting_algebra_semantics]{Heyting algebra semantics} and \hyperref[def:propositional_topological_semantics]{topological semantics} and a matching deductive system, \fullref{def:intuitionistic_propositional_deductive_systems}, for \hyperref[subsec:propositional_logic]{propositional logic}.
\end{definition}

\begin{definition}\label{def:minimal_logic}
  Minimal logic is a generalization of \hyperref[def:intuitionistic_logic]{intuitionistic logic}.

  Instead of the law of the excluded middle \eqref{eq:thm:minimal_propositional_negation_laws/lem} and the strictly weaker principle of explosion \eqref{eq:thm:minimal_propositional_negation_laws/efq}, we have the even weaker law of non-contradiction \eqref{eq:thm:minimal_propositional_negation_laws/lnc}.

  Metalogically speaking, we can only conclude that there is no statement such that both the statement and its negation are true. If the statement instead does not hold, we cannot automatically conclude that its negation holds.

  \Fullref{def:minimal_propositional_axiomatic_deductive_system} provides a deductive system for \hyperref[subsec:propositional_logic]{propositional logic}, but we avoid studying semantics of minimal logic.
\end{definition}

\begin{remark}\label{rem:mathematical_logic_conventions}
  We will only work in \hyperref[def:classical_propositional_deductive_systems]{classical metalogic}. Outside the section on logic, we will use formulas and, more generally, use object logic only in dedicated places like \fullref{def:group/theory} describing the \hyperref[def:first_order_theory]{logical theory} of groups. Most axioms like \ref{def:norm/N1}-\ref{def:norm/N3} for norms are formulated entirely within the metalanguage under the assumption that we are working within a model of set theory. To keep a clear distinction between logical formulas and non-logical axioms and, more generally, to distinguishing between logic and metalogic, we use the following conventions:

  \begin{thmenum}
    \thmitem{rem:mathematical_logic_conventions/variable_symbols} Variables in the object language are denoted by the small Greek letters, usually \( \xi, \eta, \zeta \), while variables in the metalanguage are denoted by small Latin letters, usually \( x, y, z \). If needed, we add subscripts with indices.

    \thmitem{rem:mathematical_logic_conventions/formula_term_symbols} Formulas, which we only consider in the object language, are also denoted by small Greek letters --- \( \varphi, \psi, \theta, \chi \) --- and, so are terms --- \( \tau, \sigma, \rho, \kappa, \mu, \nu \).

    \thmitem{rem:mathematical_logic_conventions/propositional_constants} The propositional constants denoting truth and falsity are denoted by \( \top \) and \( \bot \) in the object language and by \( T \) and \( F \) in the metalanguage. This is only for the sake of following an established convention, and we still use \( \top \) and \( \bot \) in general \hyperref[def:semilattice/lattice]{lattices}.

    \thmitem{rem:mathematical_logic_conventions/connective_symbols} We usually prefer prose to symbolic quantifiers and connectives in the metalanguage. The longer double arrows \( \implies \) and \( \iff \) are sometimes used within the metalogic outside this section.

    \thmitem{rem:mathematical_logic_conventions/structure_pairs} We conflate structures in the metalogic (i.e. sets with functions and/or relations defined on them) with their domain --- see \fullref{rem:first_order_model_notation} for a discussion.

    \thmitem{rem:mathematical_logic_conventions/shorthands} We additionally use syntactic shorthands like \fullref{rem:propositional_formula_parentheses} and \fullref{rem:first_order_formula_conventions} when writing formulas.

    \thmitem{rem:mathematical_logic_conventions/quantification} We avoid writing excessive universal quantification and instead rely on implicit universal quantification as described in \fullref{thm:implicit_universal_quantification}. If we need the formulas to be closed, such as in the case of \hyperref[def:first_order_theory]{first-order theories} for example, we assume all formulas are closed and if they are not, we add explicit universal quantifiers in front.
  \end{thmenum}

  Some axioms like \eqref{eq:def:magma/idempotent} are formulated within the metalogic for convenience and clarity, but are used as formulas in the object language in theorems like \fullref{thm:positive_formulas_preserved_under_homomorphism}. In places like this, it is usually straightforward to translate axioms from the metalogic into logical formulas.
\end{remark}

\begin{remark}\label{rem:higher_order_logic}
  Since we describe first-order logic, it may be helpful to clarify why is it named, so. It is merely a shorthand for \enquote{first-order predicate logic}. There are other predicate logical frameworks, namely second-order predicate logic (described in \cite[ch. VIII]{OpenLogicFull}) and higher-order predicate logic, also known as \enquote{simple type theory} (described in \cite[sec. 3]{Farmer2008}).

  Second-order logic allows us to quantify over relations between variables. In that case, we refer to the variables of first-order logic as \enquote{individuals} and to the relations as \enquote{relation variables}. This allows us, for example, to avoid axiom schemas like the \hyperref[def:zfc/specification]{axiom schema of specification} by instead replacing them with a single axiom that quantifies over unary relations. A downside of second-order logic is that it has worse properties --- it is incomplete in the sense that there exists no \hyperref[def:deductive_system]{deductive system} that is both, sound and complete\footnote{refer to \cite[thm. 39.6]{OpenLogicFull}} and it is not compact in the sense that the analogue to \fullref{thm:first_order_compactness_theorem} does not hold\footnote{refer to \cite[thm. 39.7]{OpenLogicFull}}. This is attributed to the expressive power of second-order logic because a first-order axiom schema may have only a countable number of axioms while a second-order quantifier may range over uncountably many relations.

  Clearly anything that extends second-order logic must suffer from the same consequences, however higher-order logic is still useful because it allows us to utilize some very powerful concepts. Rather than quantifying over relations over the relations over individuals that would happen in third-order logic, we instead consider the more abstract frameworks of type theory. Type theory itself comes in many flavors, but simple type theory can be viewed as a generalization of first-order logic --- see \cite[thm. 2]{Farmer2008}. The rough idea is that rather than having individual variables, relation variables, etc., we have \term{base types} and \term{type constructors}. The individual variables have a dedicated base type, for example, and the types of functions and predicates are easily constructed using the basic type constructors, hence it is also easy to construct higher-order functions and predicates. The syntax of simple type theory is inspired by \( \lambda \)-calculus, which is a huge topic in itself and one of the frameworks for studying computability theory. The semantics of simple type theory are merely an extension of first-order semantics with different universes for different types. Like second-order logic, however, type theories have worse properties than those of first-order logic.

  Another benefit of type theories is that they allow for multiple base types. For example, in the definition of a \hyperref[def:vector_space]{vector space}, we have scalars and vectors, and we introduce an axiom schema parameterized by the scalars. In contrast, we could have a type for scalars and a type for vectors. This is also easily achievable in first-order logic via the, so-called \enquote{many-sorted first-order logic}, where the types are called \enquote{sorts}. We lack type constructors, and thus we are restricted in how our functions and predicates are defined, however for simple cases many-sorted first-order logic is just as useful as simple type theory. As a matter of fact, both many-sorted first-order logic languages and simple type theory languages can be reformulated as first-order logic languages --- see \cite[ch. 8]{Farmer2008}.

  We circumvent the need for any of these higher-order logical frameworks by using set theory --- see \fullref{rem:first_order_theories_in_zfc}.
\end{remark}


\subsection{Linear probability model}

\(p=xB\). can be outside \([0,1]\).


\chapter{Linear Models: Regression}
\label{ch:regress}

Aims of this chapter\footnote{Here you work with the script file {\tt regress.R}}:

\begin{compactitem}
	\item More functions for plotting data and models.
	\item Calculating correlation coefficients.
	\item Fitting a regression model and significance testing.
	\item Using diagnostic plots to assess model suitability.
\end{compactitem}

As with the previous chapter, we'll start with creating a new blank
script for you to fill in during the practical. We'll also be using the
genome size data again, so:

\begin{compactitem}[$\quad\star$]
	\item Open R and change to the {\tt code} directory.
	\item Create a new blank script called `Regression.R' and add some
	introductory comments.
	\item Add code to your script to load the genome size data into R and
	check it.
\end{compactitem}

\section{Exploring the data}

In previous chapters we used {\tt plot} to create a scatterplot between
two variables. If you have a set of variables to explore, writing code
for each plot is tiresome, so R provides a the function {\tt pairs},
which creates a grid of scatterplots between each pair of variables.
All it needs is a dataset.

\begin{compactitem}[$\quad\star$]
	\item Add {\tt pairs(genome, col=genome\$Suborder)} into your script
	and run the code.
\end{compactitem}

The result is messy! There are far too many variables in {\tt genome}
for this to be useful. We need to cut down the data to fewer variables.
In Chapter \ref{ch:ExpDesign}, we used indices to select colours; here, we can use
indices to select columns from the data frame. This again uses square
brackets ({\tt x[ ]}), but a data frame has two dimensions, rows and
columns, so  you need to provide an index for each dimension, separated
by commas. If an index is left blank, then all of that dimension (i.e.
all rows or columns) are selected. Try the following to re-acquaint
yourself to access data frame content using indices:

\begin{lstlisting}
# create a small data frame:
> dat <- data.frame(A = c("a", "b", "c", "d", "e"), B = c(1, 2, 3, 4, 5))
> dat[1, ] # select row 1 (all columns selected)
	A B
1 a 1

> dat[, 2] # select column 2 (all rows selected)
[1] 1 2 3 4 5
> dat[2, 1] # select row 2, column 1
[1] "b"

\end{lstlisting}

Now let's get started with the actual analysis. We will look at five
key variables: genome size, body weight, total length, forewing length
and forewing area. If you look at the output of {\tt str(genome)},
you'll see that these are in columns 4, 7, 8, 12 and 14. We can record
the indices of these columns and use this to select the data in the
pairs plot.

\begin{lstlisting}
morpho <- c(4, 7, 8, 12, 14)
pairs(genome[, morpho], col = genome$Suborder)
\end{lstlisting}

\begin{compactitem}[$\quad\star$]
	\item Add the code above to your script and run it
\end{compactitem}

The {\tt pairs} plot should give you something like the plot below:

\begin{center}
	\includegraphics[width=\textwidth]{pairs.pdf}
\end{center}

Each scatterplot is shown twice, with the variables swapping between
the $x$ and $y$ axes. You can see immediately that the relationships
between the four morphological measurements and genome size are fairly
scattered but that the plots comparing morphology show much clearer
relationships.

\section{Correlations}

One way of summarising how close strong the relationship between these
variables are is to calculate a correlation coefficient. Pearson
correlations look at the difference of each point from the mean of each
variable (and since it uses means, it is a parametric statistic).

It is calculated using of the differences from the mean on each axis.
The key calculation is --- for each point -- to get the product of the
differences on each axis and add them up. If the points are mostly top
left ($-x$, $y$) or bottom right ($x$, $-y$) then these products are
mostly negative ($-xy$); if the points are mostly top right ($x$, $y$)
or bottom left ($-x$, $-y$) then the products are mostly positive
($xy$).

\begin{center}
	\includegraphics[width=\textwidth]{corr.pdf}
\end{center}

The plots above show three clear cases where all the values of $xy$ are
negative or positive or where both are present and sum to zero. The
Pearson correlation coefficient simply scales these sums of $xy$ to be
between -1 (perfectly negatively correlated) and 1 (perfectly
positively  correlated) via zero (no correlation).

We will use two functions to look at correlations. The first is {\tt
cor}, which can calculate correlations between pairs of variables, so
is a good partner for {\tt pairs} plots. The second is {\tt cor.test},
which can only compare a single pair of variables, but uses a $t$ test
to assess whether the correlation is significant.

\begin{compactitem}[$\quad\star$]
	\item Try the following (and include it in your R script file)
\end{compactitem}

\begin{lstlisting}
> cor(genome[, morpho], use = "pairwise")
\end{lstlisting}

You should see a correlation matrix. Then,

\begin{lstlisting}
> cor.test(genome$GenomeSize, genome$TotalLength, use = "pairwise")

Pearson's product-moment correlation

data: genome$GenomeSize and genome$TotalLength
t = 3.551, df = 96, p-value = 0.0005972

alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
  0.1526 0.5050
sample estimates:
    cor
 0.3407
\end{lstlisting}

The {\tt use='pairwise'} in the above tells R to omit observations with
missing data and use complete pairs of observations. The first function
confirms our impressions from the graphs: the correlations between
genome size and morphology are positive but comparatively weak and the
correlations between morphological measurements are positive and very
strong (i.e. close to 1). The correlation test tells us that genome
size and body length are positively correlated (r=0.34, $t$ = 3.5507,
df = 96, $p$ = 0.0006).

\begin{compactitem}[$\quad\star$]
	\item Again, remember this example when reporting correlations!
\end{compactitem}

\section{Transformations and allometric scaling}

There is one problem with the correlations above: {\it correlations
assume a straight line relationship}. Some of the scatterplots above
are fairly straight but there are some strongly curved relationships.
This is due to the allometric scaling mentioned in Chapter \ref{ch:ExpDesign}: two of
the variables are in linear units (total and forewing length), one is
in squared units (forewing area) and one in cubic units (body weight,
which is approximately volume).

The relationships between these variables can be described using a
power law: $y = ax^b$. Fortunately, if we log transform this equation,
we get $\log(y) = \log(a) + b \log(x)$. This is the equation of a
straight line ($y=a+bx$), so we should be able to make these plots
straighter by logging both axes. We saw in Chapter \ref{ch:ExpDesign} that we can
create a new logged variable in the data frame like this:

\begin{lstlisting}
> genome$logGS <- log(genome$GenomeSize)
\end{lstlisting}

\begin{compactitem}[$\quad\star$]
	\item Using this line as a template, create a new logged version of
	the five variables listed above.
	\item Using {\tt str}, work out which column numbers the logged
	variables are and create a new variable called {\tt logmorpho}
	containing these numbers.
	\item Copy the {\tt pairs} and {\tt cor} test from earlier in your
	script and modify them to run these functions for the columns given
	in {\tt logmorpho}.
\end{compactitem}

The correlations should give the following output:
\begin{lstlisting}
> cor(genome[, logmorpho], use = "pairwise")

         logGS   logBW  logTL  logFL   logFA
 logGS 1.00000 0.08406 0.2224 0.1150 0.06808
 logBW 0.08406 1.00000 0.8892 0.9456 0.94996
 logTL 0.22244 0.88919 1.0000 0.9158 0.86207
 logFL 0.11500 0.94565 0.9158 1.0000 0.97916
 logFA 0.06808 0.94996 0.8621 0.9792 1.00000
\end{lstlisting}

The scatterplots should look like this and show that logging the data
has very successfully removed allometric scaling effects in the data:
\begin{center}
\includegraphics[width=\textwidth]{pairsLog.pdf}
\end{center}

\section{Regression}

We'll now look at fitting the first linear model of this course, to
explore whether log genome size explain log body weight. The first
thing to do is to plot the data:

\begin{center}
	\includegraphics[width=0.5\textwidth]{gsVbw.pdf}
\end{center}

It is clear that the two suborders have very different relationships:
to begin with we will look at dragonflies (Anisoptera). We will
calculate two linear models:

\begin{compactdesc}
	\item [The null model] This is the simplest linear model: nothing is
	going on and the response variable just has variation around the
	mean: $y = \beta_1$. This is written as an R formula as {\tt y
	\textasciitilde{} 1}.
	\item [Linear regression] This models a straight line relationship
	between the response variable and a continuous explanatory variable:
	$y= \beta_1 + \beta_{2}x$.
\end{compactdesc}

The code below fits these two models.

\begin{lstlisting}
> nullModelDragon <- lm(logBW ~ 1, data = genome, subset = Suborder ==
"Anisoptera")
> genomeSizeModelDragon <- lm(logBW ~ logGS, data = genome, subset =
Suborder == "Anisoptera")
\end{lstlisting}

\begin{compactitem}[$\quad\star$]
	\item Note the long names for the models. Short names are easier to
	type but calling R objects names like {\tt mod1},  {\tt mod2},  {\tt
	xxx} swiftly get confusing!
	\item Enter these models into your script and run them.
\end{compactitem}

Now we want to look at the output of the model. Remember from the
lecture that a model has {\it coefficients} (the $\beta$ values in the
equation of the model) and {\it terms} which are the explanatory
variables in the model. We'll look at the {\it coefficients} first:

\begin{lstlisting}
> summary(genomeSizeModelDragon)
 Call:
 lm(formula = logBW ~ logGS, data = genome, subset = Suborder ==
     "Anisoptera")

 Residuals:
    Min     1Q Median     3Q    Max
 -1.324 -0.612  0.097  0.519  1.324

 Coefficients:
             Estimate Std. Error t value Pr(>|t|)
 (Intercept)  -2.3995     0.0908  -26.41  < 2e-16 ***
 logGS         1.0052     0.2398    4.19  9.5e-05 ***
 ---
 Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

 Residual standard error: 0.697 on 58 degrees of freedom
   (2 observations deleted due to missingness)
 Multiple R-squared: 0.233,	Adjusted R-squared: 0.219
 F-statistic: 17.6 on 1 and 58 DF,  p-value: 9.54e-05
\end{lstlisting}

There is a lot of information there: the model description (`{\tt
Call}'), a summary of the residuals, a table of coefficients and then
information on residual standard error, r squared and an $F$ test. All
of these will become clearer during this course --- for the moment,
concentrate on the coefficients table.

There are two rows in the coefficient table, one for each coefficient
in $y=\beta_1 + \beta_2x$ --- these are the intercept and the slope of
the line. The rest the details on each row are a $t$ test  of whether
the slope and intercept are significantly different from zero.

Now we will look at the {\it terms} of the model using the {\tt anova}
function. We will have a proper look at ANOVA (Analysis of Variance) in
chapter \ref{ch:ANOVA}. Meanwhile, for our current purposes, all you need to
know is that ANOVA tests how much variation in the response variable is
explained by each explanatory variable. We only have one variable and
so there is only one row in the output:

\begin{lstlisting}
> anova(genomeSizeModelDragon)

 Analysis of Variance Table

 Response: logBW
           Df Sum Sq Mean Sq F value  Pr(>F)
 logGS      1   8.53    8.53    17.6 9.5e-05 ***
 Residuals 58  28.14    0.49
 ---
 Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
\end{lstlisting}

This table is comparing the variation in log body weight explained by
log genome size to the total variation in log body weight. We are
interested in how much smaller the residuals are for the genome size
model than the null model. Graphically, how much shorter are the red
residuals than the blue residuals:

\begin{center}
\includegraphics[width=0.8\textwidth]{regResid.pdf}
\end{center}

We can get the sums of the squares of these residuals from the two
models using the function {\tt resid}, and then square them and add
them up:

\begin{lstlisting}
> sum(resid(nullModelDragon) ^ 2)
 [1] 36.67

> sum(resid(genomeSizeModelDragon) ^ 2)
 [1] 28.14
\end{lstlisting}

So we have five columns in the table:
\begin{compactdesc}
	\item[Df] This shows the degrees of freedom. Each fitted parameter/coefficient takes up
	a degree of freedom from the total sample size, and the left over are the residuals degree of freedom. In this
	case, genome size adds a slope (compare the null model $y=\beta_1$
	and this model $y=\beta_1 + \beta_2x$ --- there is one more $\beta$).
	\item[Sum Sq] This shows sums of squares. The bottom line is the
	residual sum of squares for the model and the one above is the
	variation explained by genome size. Using the two values from above,
	the sums of square residuals for the null model are 36.67. In the
	genome size model, the sum of square residuals are 28.14 and so
	$36.67-28.14=8.53$ units of variance have been explained by this
	model.
	\item[Mean Sq] These are just the Sum Sq (Sum of Squares) values divided by the
	degrees of freedom. The idea behind this is simple: if we explain
	lots of variation with one coefficient, that is good (the null model), and if we explain a
	small amount of variation with a loss of degree of freedom (by adding and then estimating more parameters), then that is bad.
	\item[F value] This is the ratio of the Mean Sq for the variable and
	the residual Mean Sq. This is used to test whether the explained
	variation is large or small.
	\item[Pr(>F)] This is a $p$ value --- the probability of the variable
	explaining this much variance by chance.
\end{compactdesc}

In this case, it is clear that genome size explains a significant
variation in body weight.

\begin{compactitem}[$\quad\star$]
	\item Include the {\tt summary} and {\tt anova} commands for {\tt
	genomeSizeModelDragon} in your script, run them and check you are
	happy with the output.
	\item Using this code as a template, create a new model called {\tt
	genomeSizeModelDamsel} that fits log body weight as a function of log
	genome size for damselflies.
	\item Write and run code to get the  {\tt summary} and {\tt anova}
	tables for this new model.
\end{compactitem}

\section{Plotting the model}

Now we can plot the data and add lines to show the models. For simple
regression models, we can use the function {\tt abline(modelName)} to
add a line based on the model.
\begin{compactitem}[$\quad\star$]
 \item You already know how to create and customise scatterplots from
 previous chapters. Create a plot of log body weight as a function of
 log genome size, picking your favourite colours for the points.
 \item Use {\tt abline} to add a line for each model and use the {\tt
 col} option in the function to colour each line to match the points.
 For example: {\tt abline(genomeSizeModelDragon, col='red')}.
\end{compactitem}

You should get something like Figure \ref{fig:GenoRegModels}.

\begin{figure} \centering
	\includegraphics[width=0.5\textwidth]{GenoRegModels.pdf}
	\caption{Linear regression models fitted to the body weight vs.
	genome size to the Dragonfly (red) and Damselfly (blue) subsets of
	the data.}
	\label{fig:GenoRegModels}
\end{figure}

\section{Model diagnostics}

Now that we have our models, we need to check that they are appropriate
for the data. For this, we will inspect ``diagnostic plots''. Producing
diagnostic plots is easy in R --- if you {\tt plot} a model, then R
produces a set of diagnostic plots!

\begin{compactitem}[$\quad\star$]
	\item Try the following code (and include in the R script file):
\end{compactitem}
\begin{lstlisting}
> par(mfrow = c(2, 2), mar = c(5, 5, 1.5, 1.5))
> plot(genomeSizeModelDragon)
\end{lstlisting}
This should give the plots shown in figure \ref{fig:DiagModDragon}.
\begin{figure} \centering
	\includegraphics[width=0.7\textwidth]{DiagModDragon.pdf}
	\caption{Diagnostics for the {\tt lm} fit to the Dragonfly data
	subset.}
	\label{fig:DiagModDragon}
\end{figure}
And,
\begin{lstlisting}
> par(mfrow = c(2, 2), mar = c(5, 5, 1.5, 1.5))
> plot(genomeSizeModelDamsel)
\end{lstlisting}
This should give the plots shown in figure \ref{fig:DiagModDamsel}.

\begin{figure}\centering
	\includegraphics[width=0.7\textwidth]{DiagModDamsel.pdf}
	\caption{Diagnostics for the {\tt lm} fit to the Damselfly data
	subset.}
	\label{fig:DiagModDamsel}
\end{figure}

The diagnostic plots are:
\begin{compactdesc}

		\item[Residuals vs Fitted] This plot is used to spot if the
		distribution of the residuals (the vertical distance from a point
		to the regression line) has {\it similar variance} for different
		predicted values (the y-value on the line corresponding to each
		x-value). There should be no obvious patterns (such as curves) or
		big gaps. If there was no scatter, if all the points fell exactly
		on the line, then all of the dots on this plot would lie on the
		gray horizontal dashed line. The red line is a smoothed curve to
		make it easier to see trends in the residuals. It is flat in the
		Dragonfly model fit (Figure \ref{fig:DiagModDragon}), and a bit
		more wavy than we would like in the in the Damselfly model fit
		(Figure \ref{fig:DiagModDamsel}), but there are no clear trends in
		either, which is what you hope to see.

		\item[Normal Q--Q] This plot is to check whether the residuals are
		{\it normally distributed } --- are the values of the observed
		residuals similar to those expected under a normal distribution?
		Ideally, the points should form a perfectly straight line,
		indicating that the observed residuals exactly match the expected.
		Here, note that the points lie pretty close to the dashed line in
		both Figures \ref{fig:DiagModDragon} \& \ref{fig:DiagModDamsel},
		but deviate at the ends, especially for Damselflies. However, some
		deviation is to be expected near the ends --- here these deviations
		are just about acceptable.

		\item[Scale--Location] The x-axis on this plot is identical to the
		Residuals vs Fitted plot -- these are the fitted values. The y-axis
		is the square root of the {\it standardized residuals}, which are
		residuals rescaled so that they have a mean of zero and a variance
		of one. As a result, all y-axis values are positive. Thus large
		residuals (both positive and negative) plot at the top, and small
		residuals plot at the bottom (so only their {\it scale} is
		retained). Thus, all of the numbered points (which will be the same
		in all plots) plot at the top here. The red line here shows the
		trend, just like the Residuals vs Fitted plot. The regression
		analysis has assumed homoscedasticity, that the variance in the
		residuals doesn't change as a function of the predictor. If that
		assumption is correct, the red line should be relatively flat. It
		is not quite as flat as we would like, especially for the Dragonfly
		analysis (Figure \ref{fig:DiagModDragon}).

		\item[Residuals vs Leverage] This plot shows the standardized
		residuals against leverage. ``Leverage'' is a measure of how much
		each data point influences the linear model's coefficient
		estimates. Because the regression line must pass through the
		centroid (``pivot point'') of the data (Figure \ref{fig:Leverage}),
		points that lie far from the centroid have greater leverage, and
		their leverage increases if there are fewer points nearby. There
		are two key things to note about this plot:
		\begin{figure} \centering
			\includegraphics[width=0.4\textwidth]{Leverage.pdf}
			\caption{Leverage of data points on slope of a regression. The
			points further away from the centroid in the x-axis direction
			have more leverage, and can therefore move the regression line up
			or down (dashed red lines).}
			\label{fig:Leverage}
		\end{figure}

		\begin{enumerate}
			\item The standardized residuals (y-axis) are centered around
			zero and reach 2-3 standard deviations away from zero. They
			should also lie symmetrically about zero, as would be expected
			for a normal distribution. This is the case for the Damselfly
			plot (Figure \ref{fig:DiagModDamsel}) , but not so much for the
			Dragonfly plot Figure \ref{fig:DiagModDragon}.
			\item The contours values show {\it Cook's distance} (only
			visible in the Damsefly plot), which measures how much the
			regression would change if a point was deleted. Cook's distance
			is increased by leverage and by large residuals: a point far from
			the centroid with a large residual can severely distort the
			coefficient estimates from the regression. On this plot, you want to see that the red smoothed
			line stays close to the horizontal gray dashed line and that no
			points have a large Cook's distance (i.e, >0.5). Both are true
			here.
		\end{enumerate}
		This is an important diagnostic plot in regression analyses in
		particular because it tells you whether your estimate of the slope
		coefficient in particular is strongly affected by certain data
		points.

\end{compactdesc}
Note that certain points are numbered in all the plots --- these are
points to pay special attention to because they are {\it potential}
outliers. The numbers correspnd to the row number for that dataset in
your data frame. You can easily identify these points in your data plot
(Figure \ref{fig:GenoRegModels}) because the order of the points along
the fitted values axis (y-axis) in the diagnostic plot matches the
order along the x-axis in the data plot. So, fo example here, in Figure
\ref{fig:DiagModDragon}, the two numbered points (46, 10) near the
bottom correspond in the data plot (Figure \ref{fig:GenoRegModels}) to
the two red points near the center-left that lie farthest below the red
line.

Thus, neither the Drangonfly nor the Damselfly diagnostic plots look
perfect, but this level of deviation from assumptions of linear models
is acceptable. The main worrying factors are that the QQ plot for
Damselflies indicates the observed residuals are a bit more extreme
than expected, and the Scale--Location plot for Dragonflies suggests
some pattern in the standardized residuals wrt location of the fitted
values.

\begin{compactitem}[$\quad\star$]
 \item Copy the code to create the diagnostic plots into your script to
 keep a record of the code and run it.
\end{compactitem}

\section{Reporting the model}

Now we know that the models are appropriate and we have a plot, the
last thing is to report the statistics. For the damselfly model, here
is one summary that would do: log genome size explains significant
variation in log body weight in dameselflies (F=10.5, df=1,36, p=0.0025)
and shows that body weight decreases with genome size (intercept:
-4.65,  se=0.09; slope: -1.14, se=0.35).


\begin{verbatim}
\subsection{LU Decomposition}

Given the linear system \eqref{eq:varyRHS}
\begin{equation}
  \label{eq:varyRHS}
  \begin{cases}
    3x + y + z = m \\
    x - 5y + 2z = 5 \\
    2x + y + 5z = 10
  \end{cases}
\end{equation}
where $m = 0, 1, 2, \ldots, 20$. Using LU Decomposition we can obtain the
solution to the linear system \eqref{eq:varyRHS} for corresponding $m$
(see Table \ref{tab:solution} and Figure \ref{fig:solution}).

\input{../src/solution.tex}

\begin{figure}[!hbtp]
  \centering
  \includegraphics[width=0.85\textwidth]{../src/lab_06_plot.pdf}
  \caption{Solution to the linear system vs. $m$}
  \label{fig:solution}
\end{figure}
\end{verbatim}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS240: Programming in C
% Copyright 2016 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% https://github.com/ghorbanzade/UMB-CS240-2016S/blob/master/LICENSE
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\def \topDirectory {.}
\def \resDirectory {\topDirectory/src/c/main/ls05}
\def \texDirectory {\topDirectory/src/tex}
\def \styDirectory {\texDirectory/sty}
\def \cfgDirectory {\texDirectory/cfg}
\def \imgDirectory {\texDirectory/img}

\documentclass[compress]{beamer}
%\mode<presentation>
%\usetheme{default}

\usepackage{\styDirectory/directives}
\input{\cfgDirectory/config}
\usepackage{\styDirectory/beamerthemePejman}
\doc{number}{5}
%\setbeamertemplate{footline}[text line]{}

\usepackage{booktabs}
\usepackage{graphics}

\begin{document}

\prepareCover

\section{Operators}

\begin{slide}
	\begin{block}{Definition}

	An operation is an action performed on one or more values either to modify the value or to produce a new one.

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Arithmetic Operators}

	\begin{columns}
	\column{0.4\textwidth}

	\begin{terminal}
	int a = 10;
	int b = 25;
	\end{terminal}

	\column{0.6\textwidth}
	\begin{table}
	\resizebox{\columnwidth}{!}{
	\begin{tabular}{lcc}
	\toprule
	Opeartor & Example & Output \\
	\midrule
	addition & \texttt{a + b} & 35 \\
	subtraction & \texttt{a - b} & -15 \\
	multiplication & \texttt{a * b} & 250 \\
	division & \texttt{b / a} & 2 \\
	modulus & \texttt{b \% a} & 5 \\
	\bottomrule
	\end{tabular}
	}
	\end{table}
	\end{columns}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Quiz}

	What is the final value of variables \texttt{d} and \texttt{e}?

	\begin{terminal}
	char c = 'a';
	int d = c + 3;
	char e = d + 1;
	\end{terminal}

	\end{block}

	\pause

	\begin{block}{Answer}

	\begin{terminal}
	printf("%d", c); // prints: 97
	printf("%d", d); // prints: 100
	printf("%c", e); // prints: 'e'
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Relational Operators}

	\begin{columns}
	\column{0.4\textwidth}

	\begin{terminal}
	int a = 99;
	int b = 100;
	char c = 'c';
	char d = 'd';
	\end{terminal}

	\column{0.6\textwidth}
	\begin{table}
	\resizebox{\columnwidth}{!}{
	\begin{tabular}{lcc}
	\toprule
	Operation & Example & Output \\
	\midrule
	greater than & \texttt{a > b} & 0 \\
	less than & \texttt{a < b} & 1 \\
	greater than or equal to & \texttt{a >= c} & 1 \\
	less than or equal to & \texttt{b <= d} & 1 \\
	equal to & \texttt{a == c} & 1 \\
	not equal to & \texttt{b != d} & 0 \\
	\bottomrule
	\end{tabular}
	}
	\end{table}
	\end{columns}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Logical Operators}

	\begin{columns}
	\column{0.4\textwidth}

	\begin{terminal}
	int a = 99;
	int b = 100;
	char c = 'c';
	char d = 'd';
	\end{terminal}

	\column{0.6\textwidth}
	\begin{table}
	\resizebox{\columnwidth}{!}{
	\begin{tabular}{lcc}
	\toprule
	Operation & Example & Output \\
	\midrule
	logical AND & \texttt{a < c \&\& a < d} & 0 \\
	logical OR & \texttt{a < b || a < c} & 1 \\
	logical NOT & \texttt{!(a < b)} & 0 \\
	\bottomrule
	\end{tabular}
	}
	\end{table}
	\end{columns}

	Expressions connected by logical operators are evaluated from left to right. Evaluation terminates once the final value is determined.

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{increment and decrement operators}

	\begin{columns}
	\column{0.4\textwidth}

	\begin{terminal}
	int a = 1;
	\end{terminal}

	\column{0.6\textwidth}
	\begin{table}
	\resizebox{\columnwidth}{!}{
	\begin{tabular}{lcc}
	\toprule
	Operation & Example & Output \\
	\midrule
	increment & \texttt{a++} & 2 \\
	increment & \texttt{++a} & 2 \\
	decrement & \texttt{a--} & 0 \\
	decrement & \texttt{--a} & 0 \\
	\bottomrule
	\end{tabular}
	}
	\end{table}
	\end{columns}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Quiz}

	What is the output of the following code snippet?

\begin{terminal}
int a = 1;
int b = 2;
if (++a && a == b)
    printf("Programming");
if (b++ == a)
    printf(" is fun");
\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Bitwise Operators}

	\begin{columns}
	\column{0.45\textwidth}
	\begin{terminal}
	int a = 0b11110000;
	int b = 0b10101010;
	\end{terminal}

	\column{0.55\textwidth}
	\begin{table}
	\resizebox{\columnwidth}{!}{
	\begin{tabular}{lcc}
	\toprule
	Operation & Example & Output \\
	\midrule
	bitwise AND & \texttt{a \& b} & 0b10100000 \\
	bitwise inclusive OR & \texttt{a | b} & 0b11111010 \\
	bitwise exclusive OR & \texttt{a \^{} b} & 0b01011010 \\
	left binary shift & \texttt{a << 1} & 0b11100000 \\
	right binary shift & \texttt{a >> 1} & 0b01111000 \\
	one's complement & \texttt{\~{}a} & 0b00001111 \\
	\bottomrule
	\end{tabular}
	}
	\end{table}
	\end{columns}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Assignment Operators}

	An assignment operator (\alert{\texttt{=}}) stores a new value into a variable.

	Assignment statements such as ${exp}_1 = {exp}_1 {op} {exp}_2$ can equivalently be written as ${exp}_1 {op}= {exp}_2$ where ${op}$ is one of the following operators:

	\texttt{+}\hfill \texttt{-}\hfill \texttt{*}\hfill \texttt{/}\hfill \texttt{\%}\hfill \texttt{<<}\hfill \texttt{>>}\hfill \texttt{\&}\hfill \texttt{\^{}}\hfill \texttt{|}

	An assignment statement has a value and can occur in expressions.

	\end{block}
\end{slide}

\section{Conditionals}

\begin{slide}
	\begin{block}{Statements}

	A statement is the smallest standalone element that provide a meaningful machine instruction. A statement is terminated by a semicolon.

	Function of a program is determined based on the set of statements in its \texttt{main} function executed in succession.
	A program terminates once all statements in its \texttt{main} function are executed.

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{if} statement}

	An \texttt{if} statement contains an expression followed by a block of statements.
	Statements within the block are executed if the value of expression is evaluated to non-zero.

	\begin{terminal}
	if (@*\textit{expression}*@) {
	    @*$\mathit{statement}_1$*@;
	    @*$\mathit{statement}_2$*@;
	}
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Coding Style Convention}

	\begin{terminal}
	if (number_statements > 1) {
	    printf("MUST");
	    printf("preserve braces");
	}
	if (number_statements == 1)
	    printf("remove braces");
	if (number_statements == 0);
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Coding Style Convention}

	\begin{terminal}
	int a = 2;
	if (a != 0)
	    printf("Hello");
	\end{terminal}

	It is often recommended to avoid redundancy in expressions.

	\begin{terminal}
	int a = 2;
	if (a)
	    printf("Hello");
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{if-else} statement}

	\begin{terminal}
	int a = 2;
	if (a == 2)
	    printf("Hello");
	else
	    printf("Can you hear me?");
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{if-else-if} statement}

	\begin{terminal}
	int a = 3;
	if (a == 1)
	    printf("Hello");
	else if (a == 2)
	    printf("Can you hear me?");
	else
	    printf("How are you?");
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Coding Style Convention}

	Braces should be used for all branches if at least one of them contains more than one statement.

	\begin{terminal}
	int a = 3;
	if (a == 1) {
	    printf("Hello");
	} else {
	    printf("How are you?");
	    printf("Can you hear me?);
	}
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{switch} statement}

	The \texttt{switch} statement is a multi-way decision that tests whether an expression matches one of a number of \alert{constant} integer values, and branches accordingly.

	\begin{terminal}
	switch (@*\textit{expression}*@) {
	case @*$\mathit{val}_1$*@:
	    statements;
	case @*$\mathit{val}_2$*@:
	    @*\textit{statements}*@;
	default:
	    @*\textit{statements}*@;
	}
	\end{terminal}

	\end{block}
\end{slide}

\section{Loops}

\begin{slide}
	\begin{block}{\texttt{while} statement}

	A \texttt{while} statement contains an expression followed by a block of statements.

	Statements within the block are executed \emph{as long as} the value of expression is evaluated to non-zero.

	\begin{terminal}
	while (@*\textit{expression}*@) {
	    @*\textit{statements}*@;
	}
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{do-while} statement}

	A \texttt{do-while} statement is a \texttt{while} loop whose expression is evaluated after execution of its block of statements.

	\begin{terminal}
	do {
	    @*\textit{statements}*@;
	} while (@*\textit{expression}*@);
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{for} statement}

	A \texttt{for} statement contains three expressions followed by a block of statements.

	\begin{itemize}
	\item[] $\mathit{exp}_1$ pre-loop expression
	\item[] $\mathit{exp}_2$ iteration condition
	\item[] $\mathit{exp}_3$ post-iteration expression
	\end{itemize}

	\begin{terminal}
	for (@*$\mathit{exp}_1$*@; @*$\mathit{exp}_2$*@; @*$\mathit{exp}_3$*@) {
	    @*\textit{statements}*@;
	}
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Remember}

	\begin{itemize}
	\item[] A \texttt{for} statement is often used when the number of loop iterations is already known.
	\item[] Mastering the \texttt{while} statement can boost your code quality.
	\end{itemize}

	\begin{terminal}
	int i = 0;
	while ((ch = str[i++]) != '\0');
	printf("length of str: %d\n", i);
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Infinite Loops}

	\begin{terminal}
	for (;;) {
	    @*\textit{statements}*@;
	}
	\end{terminal}

	\begin{terminal}
	while (1) {
	    @*\textit{statements}*@;
	}
	\end{terminal}

	An infinite \texttt{while} loop is often preferred for its superior readability.

	\end{block}
\end{slide}

\begin{slide}
	\begin{figure}
	\includegraphics[width=0.5\textwidth]{\imgDirectory/wonka.jpg}
	\end{figure}
\end{slide}

\section{Unconditional Branches}

\begin{slide}
	\begin{block}{Definition}

	Unconditional branches are statements that change program control flow without the need to evaluate an expression.

	Bohm-Jacopini theorem states that any computing application can be realized without the use of Unconditional branches.

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{break} statement}

	A \texttt{break} statement immediately terminates the loop regardless of its iteration condition and moves program control flow to the next statement after the loop.

	\begin{terminal}
	int i = 2;
	int num = 25;
	int prime = 1;
	while (i <= num / 2)
	    if (num % i == 0) {
	        prime = 0;
	        break;
	    }
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{continue} statement}

	A \texttt{continue} statement skips the remaining statements of the loop and moves program control flow to the next iteration of the loop.

	\begin{terminal}
	int i = 2;
	int num = 25;
	int prime = 1;
	while (i <= num / 2) {
	    if (num % i)
	        continue;
	    prime = 0;
	}
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{goto} statement}

	A \texttt{goto} statement changes program control flow to the label it points to.
	\begin{terminal}
	    a = 2;
	    if (a == 2)
	        goto ERROR;
	    printf("Hello");

	ERROR:
	    printf("Goodbye");
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Labels}

	A label is simply a name for a certain point in the program control flow that may be referred to by a \texttt{goto statement}.

	\begin{terminal}
	    a = 2;
	    if (a == 2)
	        goto ERROR;
	    printf("Hello");

	ERROR:
	    printf("Goodbye");
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{quotation} \scriptsize \normalfont

	C provides the infinitely-abusable \texttt{goto} statement and labels to branch to.
	[...]
	There are a few situations where \texttt{goto}s may find a place such as breaking out of two or more loops at once.

	Code involving a \texttt{goto} can always be written without one, though perhaps at the price of some repeated tests or an extra variable.
	[...]
	With a few exceptions code that relies on \texttt{goto} statements is generally harder to understand and to maintain than code without \texttt{goto}s.
	Although we are not dogmatic about the matter, it does seem that \texttt{goto} statements should be used rarely, if at all.

	\begin{flushright}-- The C Programming Language (K\&R)\end{flushright}

	\end{quotation}
\end{slide}

\begin{slide}
	\begin{quotation} \scriptsize \normalfont

	Albeit deprecated by some people, the equivalent of the \texttt{goto} statement is used frequently by compilers in form of the unconditional jump instruction.

	The \texttt{goto} statement comes in handy when a function exits from multiple locations and some common work such as cleanup has to be done.
	If there is no cleanup needed then just return directly.

	The rationale for using \texttt{goto}s is:
	(1) Unconditional statements are easier to understand and follow.
	(2) Nesting is reduced.
	(3) Errors by not updating individual exit points when making modifications are prevented.
	(4) Unconditional statements save the compiler work to optimize redundant code away.

	\begin{flushright}-- Linux Kernel Coding Style\end{flushright}

	\end{quotation}
\end{slide}

\end{document}


%------------------------------------------------------------
%	FILENAME: final_exam.tex
%	 PROJECT: mathbootcamp
%	  AUTHOR: Brett R. Devine
%	   EMAIL: brett.devine@wsu.edu
%------------------------------------------------------------
\documentclass[a4paper, 11pt]{article}
\usepackage{final_exam_style}
\newcounter{problem}
\newenvironment{problem}[1][]{%
	\refstepcounter{problem}\par \medskip
	\noindent \textbf{Problem~\theproblem.#1 \rmfamily}{\medskip}
}

\newenvironment{solution}{ \noindent \textbf{Solution: \medskip}}{}
% To make solutions not visible use the command \excludecomment{solution} %
%\excludecomment{solution}

\title{ Final Exam 2016 \\ \vspace{1em }
	\large WSU Economics PhD Mathematics Bootcamp
}

\begin{document}
\maketitle

\begin{problem}\\
	Consider the market for commodity $x$.
	The quantity demanded in this market depends on the market price for $x$, $p$, and the value of an external unpriced factor, $r$, such as the ``status symbol'' value of owning commodity $x$ which increases quantity demanded at any price.
	Given a known constant $b_D$ we have the demand function:
	\[
		q = b_D - 2p + r
	\]
	The supply of $x$ to the market increases with market price and we assume for some reason that higher values of $r$, the status symbol value, decrease the willingness to supply at any given price.
	Important cost factors faced by suppliers result in a known constant $b_S$ and we have the supply function:
	\[
		q = b_S + 2p - r
	\]
	Finally, the status symbol value $r$ increases the more expensive the commodity is and it decreases as more of the commodity is supplied because it becomes more common.
	Given some known constant value $b_r$, the status symbol value is determined by the function:
	\[
		r = 2b_r + 4p - 2q
	\]
	\begin{enumerate}[(a)]
		\item Using the functions provided, construct a proper system of linear equations to represent the market for commodity $x$.
		\item Derive a matrix expression for the system in part (a) that takes the form $A\mathbf{x} = \mathbf{b}$.
		\item The system of equations describing the market for commodity $x$ has three equations in three unknowns.
		The system either has no solution, infinitely many solutions or a unique solution.
		Perform a test on the matrix $A$ (show your process and results) from part(b) that can tell you whether there is a unique solution and interpret the result.
		\item Find the inverse matrix of $A$ (show it) and use it to solve the system for the equilibrium values of $q,p,r$ when we have the constants
		\[
			\begin{bmatrix}
				b_D \\
				b_S \\
				b_r
			\end{bmatrix} =
			\begin{bmatrix}
				10 \\
				8 \\
				2
			\end{bmatrix}
		\]
	\end{enumerate}
\end{problem}
\begin{solution}
	\begin{enumerate}[(a)]
		\item The system of linear equations is
		\begin{align}
			q + 2p - r &= b_D \nonumber \\
			q - 2p + r &= b_S \nonumber \\
			q - 2p + \frac{1}{2}r &= b_r \nonumber
		\end{align}
		\item
		\begin{align}
			\begin{bmatrix}
				1 & 2 & -1 \\
				1 & -2 & 1 \\
				1 & -2 & \frac{1}{2}
			\end{bmatrix}
			\begin{bmatrix}
				q \\
				p \\
				r
			\end{bmatrix} &=
			\begin{bmatrix}
				b_D \\
				b_S \\
				b_r
			\end{bmatrix} \nonumber \\
			A\mathbf{x} &= \mathbf{b} \nonumber
		\end{align}
		\item $\det A = 2 \neq 0$ so $A$ is invertible and there exists a unique solution to the system.
		\item \begin{align}
			A^{-1} &=
			\begin{bmatrix}
				\frac{1}{2} & \frac{1}{2} & 0 \\
				\frac{1}{4} & \frac{3}{4} & -1 \\
				0 & 2 & -2
			\end{bmatrix} \nonumber
		\end{align}
		and the solution is found as $\mathbf{x} = (p,q,r) = A^{-1}\mathbf{b}$ where $b=(10,8,2)$.
		The solution is $(p,q,r) = (9, \frac{13}{2}, 12)$.
	\end{enumerate}
\end{solution}

\begin{problem}\\
	Consider the market for another commodity, $y$, with the following demand and supply functions where $b_D$ and $b_S$ are constants.
	\begin{align}
		\text{Supply} \; q_S &= b_S e^{p} \nonumber \\
		\text{Demand} \; q_D &= b_D e^{-p} \nonumber
	\end{align}
	\begin{enumerate}[(a)]
		\item Find the equilibrium price function $p^*:\mathbb{R}^2_{++} \rightarrow \mathbb{R}_{++}$ such that for some values $(b_D,b_S)$ the equilibrium price for $y$ will be $p^*(b_D,b_S)$.
		\item Derive the gradient function $\nabla p^*$.
		\item We are interested in the behavior of the equilibrium price in the market for commodity $y$ as structural aspects, $b_D$ and $b_S$ change.
		Suppose through econometric analysis it is determined that the current values are $(b_D, b_S) = (5,1)$.
		Calculate the total differential of equilibrium price $p^*$ as the values of $b_D$ and $b_S$ change slightly $(db_D, db_S) = (1/9, -1/7)$.
		\item Using the values from part (c), calculate a linear approximation of the equilibrium market price as the structural parameters change as in part (c).
	\end{enumerate}
\end{problem}
\begin{solution}
	\begin{enumerate}[(a)]
		\item $p^*(b_D,b_S) = (1/2)\ln(b_D) - (1/2)\ln(b_S)$.
		\item $\nabla p^* = (1/(2b_D), -1/(2b_S))$.
		\item $dp^* = \frac{26}{315}$
		\item $p^*(5,1) + dp^* = (1/2)\ln(5) - (1/2)\ln(1) + \frac{26}{315} = \frac{26}{315} + \frac{1}{2}\ln(5)$.
	\end{enumerate}
\end{solution}

\begin{problem}\\
	Let $F:\mathbb{R}^n \rightarrow \mathbb{R}$ be a continuously differentiable function.
	Let $V = \{ \mathbf{v} \in \mathbb{R}^n : \| \mathbf{v} \| = 1\}$ be the set of all vectors in $\mathbb{R}^n$ of unit length.
	Also, let $\mathbf{x}_0 \in \mathbb{R}^n$ such that $\nabla F(\mathbf{x}_0) \neq \mathbf{0}$.
	Prove that over the set $V$, the direction $\mathbf{v}$ in which $F$ increases most rapidly at the point $\mathbf{x}_0$ is the direction of $\nabla F(\mathbf{x}_0)$.
\end{problem}
\begin{solution}\\
	Let $F:\mathbb{R}^n \rightarrow \mathbb{R}$ and let $\mathbf{x}_0 \in \mathbb{R}^n$ such that $\nabla F(\mathbf{x}_0) \neq \mathbf{0}$.
	Furthermore, let $V$ be the set of $n$-dimensional unit vectors.
	For unit vectors $\mathbf{v} \in V$, the derivative of $F$ at $\mathbf{x}_0$ in the direction $\mathbf{v}$ is
	\[
		D_{v}F(\mathbf{x}_0) = \nabla F(\mathbf{x}_0) \cdot \mathbf{v}
	\]
	The dot product can be expressed in terms of the norms of the vectors and the angle between them.
	\[
		\nabla F(\mathbf{x}_0)\cdot \mathbf{v} = \| \nabla F(\mathbf{x}_0)\| \| \mathbf{v} \| \cos \theta
	\]
	where $\theta$ is the angle between the vectors $\nabla F(\mathbf{x}_0)$ and $\mathbf{v}$.
	Since $\mathbf{v}$ is a unit vector $\| \mathbf{v} \| = 1$ so we have
	\[
		D_{v}F(\mathbf{x}_0) = \| \nabla F(\mathbf{x}_0) \| \cos \theta
	\]
	As we vary $\mathbf{v}$ over $V$, the value of $\cos \theta$ changes.
	It takes its greatest value when $\theta = 0$ meaning that $\mathbf{v}$ and $\nabla F$ will be coincident and point in the same direction.
	Thus, $D_vF(\mathbf{x}_0)$ is greatest when taken in the direction $\mathbf{v} = \nabla F(\mathbf{x}_0)$. \qedsymbol
\end{solution}

\begin{problem}
	Prove or disprove the following statement.
	\emph{There is no largest integer}.
\end{problem}
\begin{solution}\\
	Let $N \in \mathbb{Z}$ and assume to the contrary that $N$ is the largest integer.
	Now let $n = N + 1$.
	Since for any integer $z \in \mathbb{Z}$ we know that $z + 1 \in \mathbb{Z}$, we know that $n + 1$ is an integer.
	But $n > N$ which contradicts our assumption that $N$ is the largest integer.
	Therefore, there is no largest integer. \qedsymbol

\end{solution}

\begin{problem}\\
	Consider two firms engaging in Cournot competition in a market $M$.
	Both firms face an inverse demand curve $P(Q,\mu)$ where $P$ is the market price when total market quantity is $Q$ and consumer taste intensity is $\mu > 0$.
	Furthermore $P(Q,\mu)$ follows the law of demand so that $P_Q(Q,\mu) < 0$ and more intense preferences increase demand $P_{\mu}(Q,\mu) > 0$.

	Firm 1 has total production costs $C_1(q_1)$ when it produces $q_1$ units.
	Firm 2 has total production costs $C_2(q_2)$ when it produces $q_2$ units.
	For both firms costs increase in $q$, so that $C_i'(q_i) > 0$.
	Total market output $Q = q_1 + q_2$.
	The firm's profits are as follows
	\begin{align}
		\pi_1(q_1,q_2,\mu) &= P(q_1 + q_2; \mu)q_1 - C_1(q_1) \nonumber \\
		\pi_2(q_1,q_2,\mu) &= P(q_1 + q_2; \mu)q_2 - C_2(q_2) \nonumber
	\end{align}
	Each firm wants to choose its output quantity $q_i$ to maximize its profit $\pi_i$ given the output decision of the other firm.
	The solution to this maximization problem yields a \emph{best response} function $R_i(q_{-i},\mu)$ which has the interpretation that, for example, firm 1 will choose its output $q_1 = R_1(q_2,\mu)$.
	So $R$ is a function that takes firm 2's output as an argument and tells firm 1 what output, $q_1$ will maximize its own profit.
	Firm 2 has a similar function $q_2 = R_2(q_1,\mu)$.

	Using $q_1 = R_1(q_2,\mu)$ and $q_2 = R_2(q_1,\mu)$ we can create a composite function
	\[
		q' = R_1( R_2(q_1,\mu), \mu) = F(q,\mu)
	\]
	An equilibrium will be a \emph{fixed point} of this composite function, which means that if you put a $q$ in for some fixed $\mu$ and you get the same $q$ back out, then that $q$ is a Nash Equilibrium.
	Let $q_e$ represent such values.
	\begin{enumerate}[(a)]
		\item Calculate the partial derivative $\pi_{q_1\mu}$ for firm 1's profit function.
		\item Calculate the partial derivative $\pi_{q_1q_2}$ for firm 1's profit function.
		\item For the identity $q_e = F(q_e, \mu) = R_1(R_2(q_e,\mu),\mu)$, calculate the derivative $\frac{d q_e}{d \mu}$, telling us how the Nash equilibrium quantity changes as consumer taste intensity changes.
		[Hint: Consider the use of total differentials and um...the chain rule.]
	\item (Bonus) Can you sign the derivative in part (a)? i.e., does the equilibrium output increase or decrease as $\mu$ increases?
	\end{enumerate}
\end{problem}
\begin{solution}
	\begin{enumerate}[(a)]
		\item \begin{align}
			\frac{\partial \pi_1}{\partial q_1} &= P_Q(q_1 + q_2;\mu)q_1 + P(q_1+q_2;\mu) - C_1'(q_1) \nonumber \\
			\frac{\partial^2 \pi_1}{\partial q_1 \partial \mu } &=  P_{Q\mu}(q_1+q_2;\mu)q_1 + P_{\mu}(q_1+q_2;\mu) \nonumber
		\end{align}
		\item \begin{align}
			\pi_{q_1,q_2} &= P_{QQ}(q_1 + q_2;\mu)q_1 + P_Q(q_1+q_2;\mu) \nonumber
		\end{align}
		\item So because its an identity, the total change on one side has to equal the total change on the other side.
		\begin{align}
			dq_e &= \frac{\partial R_1}{\partial R_2}\frac{\partial R_2}{\partial q_e}d q_e + \frac{\partial R_1}{\partial R_2}\frac{\partial R_2}{\partial \mu}d\mu + \frac{\partial R_1}{\partial \mu}d\mu \nonumber \\
			dq_e \left[1 - \frac{\partial R_1}{\partial R_2}\frac{\partial R_2}{\partial q_e}\right] &=  \left[\frac{\partial R_1}{\partial R_2}\frac{\partial R_2}{\partial \mu} + \frac{\partial R_1}{\partial \mu} \right] d\mu \nonumber
		\end{align}
		Finally we have
		\begin{align}
			\frac{d q_e}{d \mu} &= \frac{\left[\frac{\partial R_1}{\partial R_2}\frac{\partial R_2}{\partial \mu} + \frac{\partial R_1}{\partial \mu} \right]}{\left[1 - \frac{\partial R_1}{\partial R_2}\frac{\partial R_2}{\partial q_e}\right]} \nonumber
		\end{align}
	\end{enumerate}
\end{solution}

\paragraph{True, False or Uncertain}
For the following problems, determine whether the statement, as given, is true, false or if its uncertain.
If you determine the statement is false or uncertain, provide an explanation.

\begin{problem}\\
	\textbf{True, False or Uncertain}: Suppose $f(x,y)$ is defined on a set $D$ that contains a point $(a,b)$.
	If the partial derivative functions $f_{xy}$ and $f_{yx}$ are both continuous on $D$, then $f_{xy}(a,b) = f_{yx}(a,b)$.
\end{problem}
\begin{solution}
	True
\end{solution}

\begin{problem}\\
	\textbf{True, False or Uncertain}: For a function $f(x,y)$ if the partial derivatives $f_x$ and $f_y$ exist near a point $(a,b)$ then $f$ is differentiable at $(a,b)$.
\end{problem}
\begin{solution}\\
	False: The implication isn't true because even though $f_x$ and $f_y$ exist near $(a,b)$ if they are not continuous then it is possible for discontinuities to prevent the existence of a limiting value for the rate of change of the function.
	Also, note that differentiability is sufficient for continuity.
	The implication is true if $f_x$ and $f_y$ exist and are continuous near $(a,b)$.
	This is an important result.
	For very general functions such as $F:\mathbb{R}^n \rightarrow \mathbb{R}^m$ we can know that this function is differentiable at some point $\mathbf{x}$ if all the partial derivatives of $F$ (the Jacobian Matrix) exist and are continuous around $\mathbf{x}$.
\end{solution}

\begin{problem}\\
	\textbf{True, False or Uncertain}: If $f$ is a differentiable function of $x$ and $y$, then $f$ has a directional derivative in the direction of any vector $\mathbf{v}$ and
	\[
		D_v f(x,y) = \nabla f \cdot \mathbf{v}
	\]
\end{problem}
\begin{solution}\\
	False or Uncertain: Generally, this requires $\mathbf{v}$ to be a unit vector i.e., $\| \mathbf{v} \| = 1$.
	Some people don't require this, but we lose some of the meaning as when it is a unit vector the expression represents a projection of $\nabla f$ onto vectors in the same direction as $\mathbf{v}$.
\end{solution}

\end{document}


\documentclass[11pt]{article}
\usepackage[utf8]{inputenc}

\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{amsmath}

\usepackage{graphicx}

\usepackage{geometry}
\newgeometry{tmargin=2.5cm, bmargin=2.5cm, lmargin=2.5cm, rmargin=2.5cm}
\linespread{1.1}

\begin{document}

\section*{Hyperelect - Leader election in directed hypercube network}

\subsection*{Idea behind the algorithm}
The algorithm will work over several stages. During each stage a \textit{candidate} (called a \textit{duellist}) will have a match against another duellist. One of them will win and proceed to the next stage, while the other will become \textit{defeated}. After each stage only half of the duellists will enter the next stage. At the end, only one duellist will be standing. This duellist will become \textit{leader} and notify the others.

Let us take a closer look on how to execute aforementioned idea. We will start by explaning how to pair the duellist, then how to perform a match and how to notify all nodes.

Let $H_{k}$ be a $k$-dimensional hypercube and $H_{k:i}$ be a collection of $i$-dimensional hypercubes obtained from $H_{k}$ by removing all connections between nodes in dimensions $>i$, e.g. $H_{3:1}$ will be collection of 4 segments (1-dimensional hypercubes). After stage $i$ we want to have exactly one duellist left in each hypercube $H_{k:i}$. So after stage 2 we want to have only two duellist in $H_{3}$, one in each hypercube $H_{3:2}$. Let us also observe that hypercubes from collection $H_{k:i}$ will be nicely paired in colection $H_{k:i+1}$. So we start from $H_{k:0}$ where every node is a duellist, pair them up according to collection $H_{k:1}$ and have a match between them to determine the winner. We continue this process up to $H_{k:k}$. At the end, we are left with one duellist that will become the leader.

In order to perform a match, a \textit{match} message has to get to the other duellist. We will do so in two steps. First, send a message to the other hypercube. Second, forward a message to a duellist. A duellist can perform the first step by itself as it has the connection to the other hypercube. In order to forward the message every node that was defeated will remember the shortest path (what dimensions to travel) to its opponent (that won). The message will be forwarded in that fashion until it reaches the duellist.

After electing a leader we need to notify all nodes. In $k$-dimensional hypercube we will perform that process over $k$ rounds. In a round $i\in[1, k]$ every node that is a leader or a \textit{follower} will notify their neighbor over connection in dimension $k-i+1$ to become a follower. So after round $i$ exactly one node in each hypercube $H_{k:k-i}$ will be a leader or a follower.

\subsection*{Implementation details}

Let us now discuss some aspects that might be helpful while implementing the algorithm. We will explain how to retrieve shortest path, then how to process received messages and how to close communication at the end.

As the \textit{match} message travels to its duellist, nodes can mark in which dimensions is the message forwarded. Observe, that we are only interested whether the message was transmitted odd or even number of times in a particular dimension. List of dimensions that were used odd number of times will constitute the shortest route to other duellist.

As some paths might be shorter than others, some \textit{match} messages might be sent to the other hypercube before it reaches appropriate stage. In that case every node will store some messages (at most one per stage) that will be delayed until node reaches given stage as a duellist or becomes defeated and message will be automatically forwarded.

To gently close communication over the network every node that is/becomes a leader or a follower will send only \textit{follow} messages in that round. That will not disrupt the algorithm as the leader was already selected.

\subsection*{Complexity - number of messages}

Let $N = 2^{k}$ be the number of nodes. Let $d(i)$ be the maximal length of the shortest path from node defeated in stage $i$ to the winner. Clearly, $d(i) = i$.

Sending \textit{match} message in stage $i$ can cost up to
$$l(i) = 1 + \sum_{j=1}^{i-1}d(j) = 1 + \sum_{j=1}^{i-1}j = 1 + \frac{i(i-1)}{2}.$$

In stage $i$ there will be $2\cdot 2^{k-i} = 2^{k-i+1}$ \textit{match} messages.

So in total the communication will cost us
\begin{align*}
    M[Hyperelect] &\leq N-1 + \sum_{i=1}^{k}2^{k-i+1}\cdot l(i)
    = N-1 + \sum_{i=1}^{k}2^{k-i+1} + \sum_{i=1}^{k}2^{k-i}i(i-1)\\
    &= N-1 + 6\cdot 2^{k}-k^{2}-3k-6 = 7N - (log\,N)^{2} - 3\,log\,N-7.
\end{align*}

\subsection*{Complexity - time}

The time complexity can be determined using above definitions
\begin{align*}
    T[Hyperelect] &\leq k + \sum_{i=1}^{k}l(i)
    = k + \sum_{i=1}^{k}1+\sum_{i=1}^{k}\frac{i(i-1)}{2}
    = 2k + \frac{(k-1)k(k+1)}{6} \\
    &= O(log^{3}\,N).
\end{align*}

\subsection*{Useful resources}
For more information take a look at:
\begin{itemize}
    \item N. Santoro, Design and analysis of distributed algorithms, section 3.5 (Election in cube networks)
    \item P. Flocchini, B. Mans - Optimal elections in labeled hypercubes
    \item S. Robbins, K. A. Robbins - Choosing a leader on a hypercube
\end{itemize}

\end{document}


\section{Heterogeneous Systems}
\noindent
Now that we know how to solve the ``easy case'' of homogeneous systems of linear differential equations, we can tackle the general case of heterogeneous systems.
Much like for single equations, we'll look for a homogeneous solution and particular solution, and we'll use undetermined coefficients or variation of parameters to find the particular solution.
These methods will then lead into seeing how systems of first order equations relate to single higher order equations.

% Undetermined coefficients
\input{./linearSystems/heterogeneousSystems/undeterminedCoefficients.tex}
% Variation of Parameters
\input{./linearSystems/heterogeneousSystems/variationOfParameters.tex}


\section{Special Relativity}
The Newtonian Mechanics works perfectly (maybe not) well in low-speed cases, but when the object has gotten a pretty big velocity, Newtonian physics is no longer a good approximation to the situation that arises.
Therefore, in 1905, Albert Einstein proposed the Special Theory of Relativity, in which the main differences involved are due to the treatment of the speed of light $c=299792458{\rm ms^{-1}}\approx 3\times 10^8{\rm ms^{-1}}$.\\
Special Relativity is based on two postulates:
\begin{postulate}[Principle of Relativity]
    The laws of physics are the same in all inertial frames.
\end{postulate}
\begin{postulate}[Speed of Light]
    The speed of light in vacuum is the same in all inertial frames.
\end{postulate}
The need for the second postulate arises from many experiments that failed to detect the dependence of speed of light relative to inertial frames.
But the addition of this postulate then leads to a radical revision of our understanding of space and time and the relationships of energy, momentum and mass.\\
Consider two frames $S,S'$, then if they are related by Galilean transformation, we have
$$x'=x-vt,y'=y,z'=z,t'=t$$
Write the path of light ray in $S$ as $x=ct$, then in $S'$, we have $x'=x-vt=(c-v)t'$, so it doesn't work.
Therefore we need a new form of transformation to describe inertial frames in order to accomodate our postulates.
We have to treat space and time equally.
\subsection{Lorentz Transformation}
Consider inertial frames $S,S'$.
Assume their origins coincide, i.e. the spacial origins of the frames coincide when $t=t'=0$.
Suppose $S'$ is moving along the $x$ direction relative to $S$ with speed $v$, then we can ignore the $y,z$ directions for the moment.
So we are interested in the relationship between $(x,t)$ and $(x',t')$.
By the Principle of Relativity, something moving in constant velocity in $S$ must also do so in $S'$.
In $(x,t)$ plane, the constant velocity path is a straight line, so it is also the case in $(x',t')$.
So the transformation must be linear.
The origin of $S'$ moves with speed $v$ in $S$, this implies that $x'=\gamma(x-vt)$.
where $\gamma$ depends on $|v|$.
By symmetry, $x=\gamma(x'+vt')$.
Consider a light ray going through the origins at time $t=t'=0$.
In $S$, the equation of the light ray in $S$ is $x=ct$ and also in $S'$, $x'=ct'$.
So if we plug these in, then $ct=\gamma(c+v)t'$ and $ct'=\gamma(c-v)t$.
We then have
$$\gamma^2(1-v/c)(1+v/c)=1\implies \gamma=\frac{1}{\sqrt{1-v^2/c^2}}$$
We call $\gamma$ the Lorentz factor.
Consequently we obtain the Lorentz transformation (or Lorentz Boost):
$$\begin{cases}
    x'=\gamma(x-vt)\\
    t'=\gamma(t-vx/c^2)
\end{cases},\begin{cases}
    x=\gamma(x+vt)\\
    t=\gamma(t'+vx'/c^2)
\end{cases}$$
The coordinates $y,z,y',z'$ does not change if the velocity is entirely on the $x$-direction.
Now $\gamma>1$ whenever $v\neq 0$ and $\gamma\to\infty$ if $|v|\to c$.
When $v$ is small, we can approximate $\gamma=1$, which gives us the standard Galilean transformation.\\
To check that the speed of light indeed remains constant in two frames.
Suppose a light ray travels in $x$ direction, then $x=ct$, so
$$x'=\gamma(x-vt)=\gamma(c-v)t=\gamma^2(c-v)\left( t'+\frac{vx'}{c^2} \right)\implies x'=ct'$$
For a light ray that is travelling in the $y$ direction, $y=ct,x=z=0$.
In $S'$, we have
$$x'=\gamma(-vt)=-\gamma vt,t'=\gamma t,y'=ct,z'=0$$
So the speed of the light ray will be
$$\sqrt{\left( \frac{-\gamma t}{\gamma} \right)^2+\left( \frac{c}{\gamma} \right)^2}=\sqrt{c^2}=c$$
So the speed of light is not changed, but the direction of the light ray has.\\
From a more general viewpoint, we consider the metric
$$c^2t'^2-r'^2=(ct')^2-(x'^2+y'^2+z'^2)=(ct)^2-(x^2+y^2+z^2)=c^2t^2-r^2$$
By some calculation.
So this quantity is preserved.\\
Consider the case where there is only one spacial dimension $x$ in an inertial frame $S$ with time $t$.
Conventionally we plot $x$ in the horizontal axis and $ct$ in the vertical direction.
The trajectory of a particle in space-time then is a curve in the plane.
We call this the Minkowski space-time, where each point $(x,ct)$ in the space-time represents an event.
We call the curve that represents the motion of some particle a world line.
In particular, the world line is straight iff the particle moves in uniform velocity.
Light rays through the origin then travels in vertical lines of the form $x=\pm ct$, which are the vertical lines that has a inclination of $\pi/4$ to either axis.
As a particle is not allowed to move with velocity greater than the speed of light, its motion (assuming that it goes through the origin) is restricted to the upper and lower cones that are split by the lines $x=\pm ct$.\\
How about viewing from another inertial frame $S'$?
The $t'$ axis corresponds to $x'=0$, so it corresponds to $x=vt=(v/c)ct$, and the $x'$ axis is $t'=0$, hence the axis is $ct=(v/c)x$.
Thus the axes moves by the same degree closer to the diagonal (where the light ray travels) if $v\ge 0$ and further from the diagonla otherwise.
This is consistent with the postulate that the speed of light doesn't change across the frames.
\subsection{Relativistic Physics}
Consider two events $P_1=(x_1,t_1),P_2=(x_2,t_2)$ that are points in the frame $S$ in one-dimensional Minkowski space-time.
They are called simultaneous if $t_1=t_2$.\\
So the line $P_1P_2$ is then parallel to the $x$-axis.
This is called the line of simultaneity in $S$.
But in $S'$, assuming $v\neq 0$, $P_1,P_2$ are no longer in a line of simultaneity in $S'$ (which is of the form $t-vx/c^2=d$ where $d$ is a constant).
In particular, if $x_1<x_2$, then in $S'$ the event $P_2$ occurs first (with $v>0$).
Hence in general the simultaneity is frame-dependent.\\
The question of causality then arises, as different observers in different frames see different orders of events.
So we want to see a consistent ordering of cause and effect.
Note that the lines of simultaneity in $S'$ viewed in $S$ cannot incline more than $\pi/4$ since $|v|<c$.
In higher dimensions, lines and surfaces emerging from an event $P$ with $\pi/4$ inclination to the axes forms the light cones, the past light cone and the future light cone (depending on signs).
All observes agree that the event $Q$ occurs after $P$ if $Q$ is in the future light cone, but whether or not the event $R$, not in the light cones, occurs after $P$ is frame dependent.
The fact that $R$ is outside of the light cone of $P$ then implies that $R$ cannot be influenced by $P$, and vice versa, since matters cannot travel faster than the speed of light (which is the boundary of the light cone).
In general, an event can only be influenced by events in its past light cone and influence events in its future light cone.
So causality does preserve.\\
Now consider a clock stationary in $S'$ and tips in constant intervals $\delta t'$.
We want to know what time interval is perceived by observers in $S$.
Recall the inverse Lorentz transformation gives $t=\gamma(t'+x'v/c^2)$, but $x'$ is constant since the clock is stationary in $S'$.
So $\delta t=\gamma\delta t'$, so moving clocks are slower in moving frames.
This is called time dilation.
We say the time observed in the rest frame of a particular object the proper time.\\
Consider two twins, Luke and Leia.
Luke is staying home and Leia is going to a far planet and return home with speed $v$ relative to Luke.
In Luke's frame of reference, take the origin to be home.
Suppose the planet is at $x=P$ and Leia arrives at the planet at time $cT$.
Time experienced by Leia in this part of the journey is then
$$T'=\gamma(T-\frac{v}{c^2}vT)=\frac{T}{\gamma}$$
Same for going back.
So during the entire journey, Leia aged $2T/\gamma$ while Luke aged $2T$, so Leia becomes younger than Luke.
From Leia's perspective, Luke travels away from her and returns, so if the problem is symmetric, then Luke should be younger, which is a contradiction.
So the paradox is the lack of symmetries in this problem.
Let $X$ be the intersection point between the line of simultaneity in Leia's outward frame through $P$, so at $A$, we have $x=0,t=T,t'=T/\gamma$ and $X$ has $x=0,t'=T/\gamma$, so the time experienced by Leia would be $t=T/\gamma^2$ in Luke's frame at $A$.
As for the return journey, the line of simultaneity changes sign.
So in the return journey, Luke sees Leia aging from $A$ to $R$ and Leia sees Luke aging from $Z$ to $R$ (where $Z$ is the event with $x=0$ that is simultaneous with $A$ in the frame of Leia on her return journey).
The reason for the paradox is the discontinuity of time (from $X$ to $Z$) when Leia changes direction, so Luke has aged instaneously from $X$ to $Z$.\\
Now we shall talk about length contraction.
Consider a rod of length $L'$ stationary in $S'$, we want to know about the length of the rod in $S$.
Suppose the ends of the rod are at $x'=0$ and $x'=L'$, so the world lines of the ends are simply the two vertical lines described by these equations.
So $x'=0$ mapsto $\gamma(x-vt)=0$ in $S$ and $x'=L'$ mapsto $\gamma(x-vt)=L'$, so these two lines are still parallel but the horizontal (in $S$) distance between them are now $L=L'/\gamma$, so moving objects are contracted in the direction in which they move.
We define the proper length to be the length measured in the rest frame of the rod, which is essentially the greatest length of it over all frames.\\
A practical problem is that does a train of proper length $2L$ fits in a platform of proper length $L$ if it travels at a certain speed.
So we want $\gamma=2$.
Now for the observers at the platform, this would work if the train attains the desired speed.
As for the observers on the train, the platform contracts to length $L/\gamma=L/2$ so it doesn't fit.
Suppose the platform is defined by $x=0$ and $x=L$.
The train is defined by $x'=0$ and $x'=2L$, which are mapped to some slanted lines in $S$, the frame of the platform.
Consider the event $E$ where the rear of the platform and the rear of the train coincide.
For simplicity, this happens at $t=t'=0$.
Now the front of the train is $x'=2L$ and the platform is $x=L$.
Let $F$ be the event which is simultaneous with $E$ in $S$ at the front of the train, so $x'=\gamma(x-vt),2L=\gamma(L-vt)$ which implies $t=0$, so in the platform, $E$ is simultaneous with $F$, but in the train $S'$, we have $t'<0$ by calculation.
So in the train $F$ occurs before $E$ in $S'$.\\
Now that both length and time become different in different frames, what about velocities?
Suppose we have a particle moving with constant velocity $u'$ in $S'$ which moves with constant velocity $v$ relative to $S$.
We want to know the velocity $u$ of the particle as measured in $S$.
The world line of the particle in $S'$ can be taken as $x'=u't'$, so we have $\gamma(x-vt)=u'\gamma(t-vx/c^2)$ which gives $u=(u'+v)/(1+u'v/c^2)$.
In particular, if $u',v<<c$, then $u\approx u'+v$ which is the standard Galilean transformation.
Note also that we still cannot get to the speed of light given $u',v<c$ which is a combination of successive boosts.
\subsection{The Geometry of Space-time}
\begin{definition}
    Consider two points $P,Q$ in space-time having coordinates $(x_1,ct_1),(x_2,ct_2)$, so $\delta t=t_2-t_1$ and the space seperation is $\delta x=x_2-x_1$.
    We define the invariant interval between $P,Q$ to be $\delta s^2=c^2\delta t^2-\delta x^2$.
\end{definition}
Note that as we observed before, one can show that all observers agree on the value of $\delta s^2$.
\begin{definition}
    If we have three spacial dimensions $(x,y,z)$, we define $\delta s^2=c^2\delta t^2-\delta x^2-\delta y^2-\delta z^2$.
\end{definition}
If the seperation between $P,Q$ becomes small, then $\mathrm ds^2=c^2\,\mathrm dt^2-(\mathrm dx^2+\mathrm dy^2+\mathrm dz^2)$ which looks like a distance (no it doesn't).
We can (no we can't) say that space-time is topologically equivalent to $\mathbb R^4$ endowed by the distance measure $\delta s$, but note that this is not even positive definite.
The space-time endowed with this ``measure of distance'' is called the Minkowski space-time.
\begin{definition}
    Two events having $\delta s^2<0$ are said to be time-like seperated, and two that have $\delta s^2<0$ are said to be space-like seperated.
\end{definition}
So two time-like seperated events are at the same space position in some frame of reference and space-like seperated events are at the same time position in some frame.
\begin{definition}
    if $\delta s^2=0$, we say $P,Q$ are light-like seperated, so they can be connected by a light ray.
\end{definition}
Note also that events that are light-like seperated may not be the same.
\begin{definition}
    Take event $P$ in $S$, we can write its coordinates as a $4$-vector $X^\mu=(ct,x,y,z),\mu=0,1,2,3$, so $X^0=ct$ etc..
\end{definition}
We can define a new ``inner product'' on $4$-vectors by $X\cdot Y=X^\top\eta X=X^\mu\eta_{\mu\nu}X^\nu$ where
$$\eta=\begin{pmatrix}
    1&0&0&0\\
    0&-1&0&0\\
    0&0&-1&0\\
    0&0&0&-1
\end{pmatrix}$$
So we have $X\cdot X=c^2t^2-x^2-y^2-z^2$.
We call this the Minkowski metric.
$4$-vectors with $X\cdot X>0$ are time-like, those with $X\cdot X<0$ are space-like and those with $X\cdot X=0$ are light-like (or null).
The Lorentz transformation is a lineaR transformation that takes the components of a $4$-vector in $S$ to those of a $4$-vector in $S'$.
Hence we can write it as a matrix $\Lambda$ where $X'=\Lambda X$.
The set of all $\Lambda$ that preserves the Minkowski metric then forms a group, called the Lorentz group.
I.e. we want $X\cdot X=(\Lambda X)\cdot (\Lambda X)$ for all $4$-vector $X$.
By substitution we have $\Lambda^\top\eta\Lambda=\eta$.\\
If $\Lambda$ is just a spacial transformation, i.e.
$$\Lambda=\begin{pmatrix}
    1&0&0&0\\
    0&&&\\
    0&&R&\\
    0&&&
\end{pmatrix}$$
So $R$ must be a rotation.
If it is not the case, we can also have the boost (WLOG in the $x$-direction)
$$\Lambda=\begin{pmatrix}
    \gamma&-\gamma\beta&0&0\\
    -\gamma\beta&\gamma&0&0\\
    0&0&1&0\\
    0&0&0&1
\end{pmatrix},\beta=\frac{v}{c}$$
The Lorentz group $\operatorname{O}(1,3)$ also consists of spacial reflections and time reversals.
And its subgroup $\operatorname{SO}(1,3)$ with determinant $1$ is called the proper Lorentz group.
This includes composition of time reversals and spacial reflections.
The subgroup that preserves the direction of time and spacial orientation is called the restrictive Lorentz group $\operatorname{SO}^+(1,3)$, which is generated by spacial rotations and boosts (in all directions).\\
A way to label the Lorentz transformations is by a concept of rapidity.
We now focus on the $(ct,x)$ space (i.e. the $2\times 2$ submatrix on the top left corner operating on $(ct,x)$), where we define
$$\Lambda[\beta]=\begin{pmatrix}
    \gamma&-\gamma\beta\\
    -\gamma\beta&\gamma
\end{pmatrix}$$
So if we combine two boosts in the $x$ direction, then we have $\Lambda[\beta_1]\Lambda[\beta_2]=\Lambda[(\beta_1+\beta_2)/(1+\beta_1\beta_2)]$ with appropriate values of $\gamma$'s.
Recall that for spacial rotations, we have $R(\theta_1+\theta_2)=R(\theta_1)R(\theta_2)$.
For Lorentz boosts, we define the rapidity $\phi$ by $\beta=\tanh\phi$, so $\gamma=\cosh\phi$ and $\gamma\beta=\sinh\phi$.
Hence
$$\Lambda[\phi]=\begin{pmatrix}
    \cosh\phi&-\sinh\phi\\
    -\sinh\phi&\cosh\phi
\end{pmatrix}$$
and thus $\Lambda[\phi_1]\Lambda[\phi_2]=\Lambda[\phi_1+\phi_2]$.
This suggests that Lorentz transformations are hyperbolic rotations of space-time.
\subsection{Relativistic Kinematics}
Consider a particle moving along some trajectory $\underline{x}(t)$, then $\underline{u}(t)=\mathrm d\underline{x}/\mathrm dt$, so the path of it in space-time is parameterized by $t$.
But in special relativity the dependent variable $t$ is also going to change, so the path of it in a new frame would be non-trivial.
Consider a particle at rest in $S'$, so $\underline{x}=\underline{x_0}$ in $S'$, so the invariant interval would be $\delta s^2=c^2\delta t'^2$.
Define the proper time as the time $\tau$ with $c^2\delta\tau^2=\delta s^2$, so $\delta\tau$ is the time experienced by the particle.
Due to invariance, this equation holds in all frame, and $\tau$ is real in time-like intervals.
So the world line of a particle can be parameterized by $\tau$.
In terms of an infinitesimal interval, if $\underline{u}$ is the speed of the particle, we have
$$\mathrm d\tau=\frac{\mathrm ds}{c}=\frac{1}{c}\sqrt{c^2\,\mathrm dt^2-\mathrm dx^2}=\sqrt{1-\frac{|\underline{u}|^2}{c^2}}\mathrm dt$$
Hence $\mathrm dt/\mathrm d\tau=\gamma_u$ where $\gamma_u=1/\sqrt{1-u^2/c^2}$.
The total time experienced by the particle is then
$$T=\int\mathrm d\tau=\int\frac{\mathrm dt}{\gamma_u}$$
To study this, we introduce the concept of a $4$-velocity.
The position $4$-vector of a particle is the column vector $X(\tau)=(ct(\tau),\underline{x}(\tau))^\top$ where $\underline{x}$ is a $3$-vector.
\begin{definition}
    The $4$-velocity is
    $$U=\frac{\mathrm dX}{\mathrm d\tau}=\begin{pmatrix}
        c\,\mathrm dt/\mathrm d\tau\\
        \mathrm d\underline{x}/\mathrm d\tau
    \end{pmatrix}=\gamma_u\begin{pmatrix}
        c\\
        \underline{u}
    \end{pmatrix},\underline{u}=\frac{\mathrm d\underline{x}}{\mathrm dt}$$
\end{definition}
If I have two frames $S,S'$ such that the components of $X,X'$ of the position vector are related by $X'=\Lambda X$, then $U'=\Lambda U$.
In general, everything that transforms in this way is called a $4$-vector.
And in particular, $U$ is a $4$-vector since $X$ is and $\tau$ is invariant.
Note that $\mathrm dX/\mathrm dt$ is however not a $4$-vector.
The scalar product $U\cdot U$ will hence be Lorentz invariant.
That is, $U\cdot U=U'\cdot U'$.
In the rest frame where the particle with $4$-velocity $U$, then $U=(c,\underline{0})^\top$, so $U\cdot U=c^2$, so for any $u$, we have $c^2=\gamma_u^2(c^2-|\underline{u}|^2)$.
We have seen that the rule of transformation of velocity in special relativity is not as simple as in Galilean transformations.
However, we do have a fairly simple transformation law for $4$-vectors, which we can apply to $4$-velocity, which gives $U'=\Lambda U$.
\begin{example}
    In a frame $S$ where our favourite particle is moving with a speed $u$ at an angle $\theta$ to the $x$-axis in the $x-y$ plane.
    Its $4$-velocity is then $U=\gamma_u(c,u\cos\theta,u\sin\theta,0)^\top$.
    Consider another frame $S'$ which moves with speed $v$ in the $x$ direction of $S$.
    Suppose the velocity in $S$ is $u'$ and it makes an angle $\theta'$ to the $x$-direction in $S'$.
    So $U'=\gamma_{u'}(c,u'\cos\theta',u'\sin\theta',0)$ with
    $$U'=\begin{pmatrix}
        \gamma_v&-\gamma_v\beta_v&0&0\\
        -\gamma_v\beta_v&\gamma_v&0&0\\
        0&0&1&0\\
        0&0&0&1
    \end{pmatrix}U$$
    which we can solve to get $\theta',u'$ in terms of other things.
    $$u'\cos\theta'=\frac{u\cos\theta-v}{1-uv\cos\theta/c^2},\tan\theta'=\frac{u\sin\theta}{\gamma_v(u\cos\theta-v)}$$
    This change in angle, i.e. apparent change of direction, of the motion of the particle due to the motion of the observer is called an aberation.
    This also applied with the particle is a photon, so $u=c$, so although the speed of light cannot change across inertial frames, the direction of light ray can.
\end{example}
We also want to talk about $4$-momentum.
The $4$-momentum of a particle of mass $m$ moving with $4$-velocity $u$ is given by $P=mU=m\gamma_u(c,\underline{u})^\top$ with components $\mu=0,1,2,3$ where the component $\mu=0$ is interpreted as time.
For $P$ to be a $4$-vector, $m$ must be an invariant, so we must take $m$ to be the rest mass of the particle.
The $4$-momentum of a system of particles is the sum of the individual particles which conserves in the absence of external forces.
The spacial components of $P$ corresponds to the relativistic $3$-momentum $\underline{p}=\gamma_um\underline{u}$ which is the same as the Newtonian expression except that mass is modified to $\gamma_um$, which is interpreted as the apparent mass of the moving particle.
In particular, when $|\underline{u}|\to c$, the apparent mass tends to infinity.
For the zero component,
$$P^0=\gamma_umc=\frac{1}{c}\left(mc^2+\frac{1}{2}m|\underline{u}|^2+\cdots\right)$$
We see the kinetic energy in the second term, so the natural interpretation of $P$ is $P=(E/c,\underline{p})$ where $E$ is called the relativistic energy, so $E=\gamma_umc^2=mc^2+m|\underline{u}|^2/2+\cdots$ and $P$ is sometimes called the energy-momentum $4$-vector.
Note that $E\to\infty$ as $|\underline{u}|\to c$.
So for a stationary particle, we have $E=mc^2$, and for a moving particle we have $E=mc^2+(\gamma_u-1)mc^2$ where the second term is the kinetic energy, which reduces to the Newtonian kinetic energy for small $u$.
Now $P\cdot P=E^2/c^2-|\underline{p}|^2$ is conserved under Lorentz transformation and hence equals to $m^2c^2$, so we have $E^2=|\underline{p}|^2c^2+m^2c^4$.
In Newtonian physics, mass and energy are seperated idea in the sense that they are seperately conserved.
But in relativity, mass is not conserved and is a form of energy, i.e. we can convert mass into kinetic energy and vice versa.\\
Consider a massless particles (i.e. particles with zero rest mass) like a photon.
It can have non-zero ($4$-)momentum and hence nonzero relativistic energy.
Suppose this particle has the speed of light, then $0=m^2c^2=P\cdot P=E^2/c^2-|\underline{p}|^2$.
We say this particle is light-like and it travels through a light-like trajectory.
Consequently, there is no proper time for this particle.
Note that in this case $E=c|\underline{p}|$, so
$$P=\frac{E}{c}\begin{pmatrix}
    1\\
    \underline{n}
\end{pmatrix}$$
where $\underline{n}$ is a unit vector.
In special relativity, we can write Newton's Law as
$$\frac{\mathrm dP}{\mathrm d\tau}=F$$
where $F$ is the $4$-force, i.e.
$$F=\gamma_u\begin{pmatrix}
    \underline{F}\cdot\underline{u}/c\\
    \underline{F}
\end{pmatrix}$$
which is also a $4$-vector.
Note that if we transform from proper time to time experienced, then Newton's Second Law pops up, so it is consistent.
Equivalently, for a particle with rest mass $m$, then one can write $F=mA$ where $A=\mathrm dU/\mathrm d\tau$ is the $4$-acceleration.
\subsection{Examples in Particle Physics}
We want to explore the use of the conservation of total $4$-momentum in problems in particle physics.
Consider $P=(E/c,\underline{p})^\top$ for a system of particles.
A useful way to consider the system is to introduce the notion of a center-of-momentum frame (CM frame), which is the frame where the total $4$-momentum is $0$ (possible whenever all particles have positive rest mass).
\begin{example}
    Particle decay.
    Consider a particle of mass $m_1$ with momentum $P_1$ which is deemed to decay into two particles of mass $m_2,m_3$ and momenta $P_2,P_3$ respectively.
    So we have $P_1=P_2+P_3$.
    Consider the zero component, then $E_1=E_2+E_3$.
    Consider the spacial components gives $\underline{p_1}=\underline{p_2}+\underline{p_3}$.
    In the CM frame, $P_1=0$, therefore $P_2=-P_3$.
    Also
    $$m_1c=E_1/c=E_2/c+E_3/c=\sqrt{|\underline{p_2}|^2+m_2^2c^2}+\sqrt{|\underline{p_3}|^2+m_3^2c^2}\ge (m_2+m_3)c$$
    So this decay is possible only if $m_1\ge m_2+m_3$.
    Note that is possible that we don't have the equality (unlike in Newtonian mechanics) where some mass has been converted to energy.
\end{example}
\begin{example}
    A Higgs particle $h$ is decayed into two photons $\gamma$, then $P_h=P_{\gamma_1}+P_{\gamma_2}$, then in the rest frame of $h$, $P_h=(m_hc,\underline{0})$.
    So if we look at the spacial components, then $\underline{0}=\underline{P_{\gamma_1}}+\underline{P_{\gamma_2}}$.
    And since the photons have zero rest mass,
    $$\frac{E_{\gamma_1}}{c}=|\underline{p_{\gamma_1}}|=|\underline{p_{\gamma_2}}|=\frac{E_{\gamma_2}}{c}$$
    So each of the photons has half of the Higgs particle's total energy.
    Note that in this case mass does not conserve.
\end{example}
\begin{example}
    Consider two identical particles which collide and retain their identities.
    Let $P_1,P_2$ be the $4$-momenta before and $P_3,P_4$ after respectively.
    Suppose $S$ is the laboratory frame where $\underline{p_2}=0$, let the horizontal to be the line joining the two particles and let $\theta$ be the inclination of particle $1$ after the collision and $\phi$ be that of particle $2$.
    We want to study the relationship between $\theta$ and $\phi$.\\
    Now we go to the CM frame where the particles are horizontal before the collision, then the trajectories form two lines crossing each other.
    Let $\theta'$ be the angle between those two lines.
    Let $v$ be the speeds before the collision and $w$ be that after.
    We put a $'$ to indicate we are in the CM frame.
    Then
    $$P_1'=\begin{pmatrix}
        m\gamma_vc\\
        m\gamma_vv\\
        0\\
        0
    \end{pmatrix},P_2'=\begin{pmatrix}
        m\gamma_vc\\
        -m\gamma_vv\\
        0\\
        0
    \end{pmatrix}$$
    and
    $$P_3'=\begin{pmatrix}
        m\gamma_wc\\
        m\gamma_ww\cos\theta'\\
        m\gamma_ww\sin\theta'\\
        0
    \end{pmatrix},P_4'=\begin{pmatrix}
        m\gamma_wc\\
        -m\gamma_ww\cos\theta'\\
        -m\gamma_ww\sin\theta'\\
        0
    \end{pmatrix}$$
    The first component gives $v=w$.
    Now we apply the Lorentz transformation from the CM frame $S'$ back to $S$.
    The velocity of the transformation is $v$, so
    $$\Lambda=\begin{pmatrix}
        \gamma_v&\gamma_vv/c&0&0\\
        \gamma_vv/c&\gamma_v&0&0\\
        0&0&1&0\\
        0&0&0&1
    \end{pmatrix}$$
    Now before the collision,
    $$P_1=\begin{pmatrix}
        m\gamma_v^2(c+v^2/c)\\
        m\gamma_v^2(v+v)\\
        0\\
        0
    \end{pmatrix}=\begin{pmatrix}
        m\gamma_uc\\
        m\gamma_uu
    \end{pmatrix}$$
    where $u$ is the initial velocity of particle $1$.
    Consider the situation after the collision and set $q$ to be the velocity of particle $1$ after the collision, we get
    $$P_3=\begin{pmatrix}
        m\gamma_v^2(c+(v^2/c)\cos\theta')\\
        m\gamma_v^2(v+v\cos\theta')\\
        m\gamma_vv\sin\theta'\\
        0
    \end{pmatrix}=\begin{pmatrix}
        m\gamma_qc\\
        m\gamma_qq\cos\theta\\
        m\gamma_qq\sin\theta\\
        0
    \end{pmatrix}$$
    So by comparing the $1$ and $2$ components, we get
    $$\tan\theta=\frac{m\gamma_v}{m\gamma_v^2}\frac{v\sin\theta'}{v(1+\cos\theta')}=\frac{1}{\gamma_v}\tan\frac{\theta'}{2}$$
    Similarly,
    $$\tan\phi=\frac{1}{\gamma_v}\cot\frac{\theta'}{2}$$
    So $\tan\theta\tan\phi=1/\gamma_v^2=2/(1+\gamma_u)$.
    When $\gamma_u\to 1$ (i.e. in the Newtonian limit), we get $\tan\theta\tan\phi=1$, so the angle after the collision would be $\pi/2$.
\end{example}