 \documentclass [12pt]{article}

\usepackage {amsmath}
\usepackage {amsthm}
\usepackage {amssymb}
\usepackage {graphicx}
\usepackage {float}
\usepackage {multirow}
\usepackage {xcolor}
\usepackage [ruled,vlined,commentsnumbered,titlenotnumbered]{algorithm2e} \usepackage {array}
\usepackage {booktabs}
\usepackage {url}
\usepackage {parskip}
\usepackage [margin=1in]{geometry}
\usepackage [T1]{fontenc}
\usepackage {cmbright}
\usepackage [many]{tcolorbox}
\usepackage [colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage {enumitem}
\usepackage {xparse}
\usepackage {verbatim}
\usepackage{algpseudocode}
\usepackage{listings}
\usepackage{xcolor}
\lstset { %
    language=C++,
    backgroundcolor=\color{black!5}, % set backgroundcolor
    basicstyle=\footnotesize,% basic font setting
}
\newtheorem{theorem}{Theorem}
\newtheorem{remark}{Remark}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{corollary}[theorem]{Corollary}
\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]
\newtheorem{claim}{Claim}
\newtheorem{proposition}{Proposition}

\DeclareTColorBox {Solution}{}{breakable, title={Solution}} \DeclareTColorBox {Solution*}{}{breakable, title={Solution (provided)}} \DeclareTColorBox {Instruction}{}{boxrule=0pt, boxsep=0pt, left=0.5em, right=0.5em, top=0.5em, bottom=0.5em, arc=0pt, toprule=1pt, bottomrule=1pt} \DeclareDocumentCommand {\Expecting }{+m}{\textbf {[We are expecting:} #1\textbf {]}} \DeclareDocumentCommand {\Points }{m}{\textbf {(#1 pt.)}}

\begin {document}

\vspace {1em}
\begin {Instruction}
Adapted From Virginia Williams'lecture notes.
\end {Instruction}

{\LARGE \textbf {COMP 285 (NC A\&T, Spr `22)}\hfill \textbf {Lecture 32} }

\begin{centering}
\section*{MSTs II, Max Flow}
\end{centering}

\section{Kruskal's Algorithm}

At a high level, the set A maintained by Kruskal's algorithm is a set of disjoint trees. During update step $i$, if the ith smallest edge connects different trees, merge the two trees connected by this edge. The algorithm progresses until eventually only one tree remains at which point the set A represents an MST of the graph.

Kruskal's algorithm utilizes the union-find (aka disjoint set) data structure in order to handle the merging of the disjoint trees maintained by the algorithm. The union-find data structure supports disjoint sets with the following operations:

\begin{itemize}
\item \textbf{makeset}$(u)$: creates a new set containing $u$ provided that $u$ is not in any other set
\item \textbf{find}$(u)$: returns the name of the set containing $u$
\item \textbf{union}$(u, v )$: merge the set containing $u$ and the set containing $v$ into one set
\end{itemize}

The algorithm itself can be structured as follows:

\begin{algorithm}
\caption{Kruskal($G$)}
\label{alg:kruskal_algorithm}
\begin{algorithmic}
\State $A \gets \emptyset$
\State $E' \gets$ sort edges by weight in non-decreasing order
\State \For{ $v \in V$} {
    \State makeset(v)
}
\State \For{ $(u,v) \in E'$} {
    \If{find($u) \neq $ find(v)} {
        \State $A \gets A \cup \{(u,v)\}$
        \State union(u,v)
    }
}
\State \Return $A$
\end{algorithmic}
\end{algorithm}

\textbf{Correctness} The correctness follows from Theorem \ref{thm:lemma}.

\textbf{Running time}

The runtime of Kruskal's algorithm depends on two factors: the time to sort the edges by weight and the runtime of the union-find data structure operations. While $\Omega(m \log n)$ time is required for sorting the edges if we use comparison-based sorting, in many cases, we may be able to sort the edges in linear time. (Recall, that RadixSort can be used to sort the edges in $O(m)$ time if the weights are given by integers bounded by a polynomial in $m$.) In this case, the runtime is bounded by the runtime of the union-find operations and is given by $O(nT(\text{makeset}) + mT(\text{find}) +nT(\text{union}))$. The best known data structure supporting the union-find operations runs in amortized time $O(\alpha(n))$ where $\alpha(n)$ is the inverse Ackermann function. Interestingly, the value of the inverse Ackermann is tiny for all practical purposes:
$$
\alpha(n) \leq 4, \forall n < \text{\# atoms in the universe}
$$

and thus for all practical purposes, the union-find operations run in constant time. Thus, in many settings, the runtime of Kruskal's algorithm is nearly linear in the number of edges.

The actual definition of $\alpha(n)$ is $\alpha(n) = \min\{k \mid A(k) \geq n\}$, where $A(k)$ is the Ackermann function evaluated at $k$. $A(k)$ itself is defined using the more general Ackermann function as $A(k) = A_k(2)$. $A_k(x)$ is defined recursively:

\begin{align*}
A_m(x) =
    \begin{cases}
        x + 1 & m = 0 \\
        A_{m-1}(1) & m > 0, x = 0  \\
        A_{m-1}(A_m(x - 1)) & \text{ else }
    \end{cases}
\end{align*}

For example
\begin{itemize}
    \item $A_0(x) = 1 + x$, so $A_0(2) = 3$
    \item TO compute $A_1(x)$, we see:
    $$
    A_1(x) = A_0(A_1(x-1)) = A_0(A_0(A_1(x-2))) = \cdots = A_0(A_0(\cdots (A_0(A_1(0))))) = A_0(A_0(\cdots(A_0(2))))
    $$
    that is, we have ``iterated'' $A_0 x$ times. If we work it out, we get:
    $$
    A_1(x) = 2x
    $$
    Thus, $A_1(2) = 4$
    \item $A_2(x) = 2^x x$ ($A_1$ iterated $x$ times), so $A_2(2) = 8$
    \item $A_3(x) \geq 2^{2^{2^{2\cdots}}}$, a "tower" of $x$ 2s ($A_2$ iterated $x$ times); it turns out that $A_3(2) \geq 2^11$
    \item $A_4(x)$ is larger than the total number of atoms in the known universe, and also greater that the number of nanoseconds since the Big Bang. (Thus, $\alpha(n) \leq 4$ for all practice purposes)
\end{itemize}

\textbf{Example} In this example (Figure \ref{fig:kruskal0}) we will run through the steps of Kruskal's algorithm in order to find an MST for the following graph:

\begin{figure}[h!]
\centering
\includegraphics[scale=0.8]{kruskal0.png}
\caption{}
\label{fig:kruskal0}
\end{figure}

We begin by creating a new set for each node in the graph. We then begin iterating over the edges in non-decreasing order. The first edge we examine is $(g, h)$. This edge connects nodes $g$ and $h$ which are currently not part of the same set. We thus include this edge in $A$ and union the sets containing $g$ and $h$ as shown in Figure \ref{fig:kruskal1}.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.8]{kruskal1.png}
\caption{}
\label{fig:kruskal1}
\end{figure}

The next edge in the sorted order is a tie between edges $(c, i)$ and $(f , g)$. Picking either one will yield a correct result, so let's say the algorithm picks $(c, i)$. Since $c$ and $i$ are not part of the same set we include this edge in $A$ and union the sets containing $c$ and $i$ as shown in Figure \ref{fig:kruskal2}.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.8]{kruskal2.png}
\caption{}
\label{fig:kruskal2}
\end{figure}

The next edge in the sorted order is $(f , g)$. We union the sets containing $f$ and $g$ and add edge $(f , g)$ to $ A$ as shown in Figure \ref{fig:kruskal3}.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.8]{kruskal3.png}
\caption{}
\label{fig:kruskal3}
\end{figure}

The next edge in the sorted order is a tie between edges $(a, b)$ and $(c, f )$. Let's say the
algorithm picks $(a, b)$. We union the sets containing $a$ and $b$ and add edge $(a, b)$ to $A$ as shown in Figure \ref{fig:kruskal4}.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.8]{kruskal4.png}
\caption{}
\label{fig:kruskal4}
\end{figure}

The next edge in the sorted order is $(c, f )$. We union the sets containing $c$ and $f$ and add edge $(c, f )$ to $A$ as shown in Figure \ref{fig:kruskal5}.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.8]{kruskal5.png}
\caption{}
\label{fig:kruskal5}
\end{figure}

The next edge in the sorted order is $(i, g)$. Note that $i$ and $g$ are contained in the same set which means that adding the edge $(i, g)$ would lead to a cycle in $A$. We therefore ignore $(i, g)$ and pick the next edge in sorted order which is a tie between $(c, d)$ and $(h, i)$. Let's say the algorithm picks $(c, d)$. We union the sets containing c and d and add edge $(c, d)$ to $ A$ as shown in Figure \ref{fig:kruskal6}.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.8]{kruskal6.png}
\caption{}
\label{fig:kruskal6}
\end{figure}

The next edge in the sorted order is $(h, i)$, but $h$ and $i$ are contained in the same set so we ignore it. The next edge is a tie between edges $(a, h)$ and $(b, c)$. Let's say the algorithm picks $(a, h)$. We union the sets containing a and h and add edge $(a, h)$ to $A$ as shown in Figure \ref{fig:kruskal7}.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.8]{kruskal7.png}
\caption{}
\label{fig:kruskal7}
\end{figure}

The next edge in the sorted order which has both vertices in different sets is $(d, e)$. We union the sets containing $ d$ and $e$ and add edge $(d, e)$ to $A$ as shown in Figure \ref{fig:kruskal8}. At this point all nodes are contained in the same set, so no further edges are added to A.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.8]{kruskal8.png}
\caption{}
\label{fig:kruskal8}
\end{figure}

\section{The Latest and Greatest Algorithms}

While the greedy algorithms mentioned above are reasonably efficient ways to compute a minimum spanning tree of a graph, recent research has yielded more efficient algorithms. In 1995, Karger, Klein, and Tarjan discovered a randomized linear time $(O(E + V ))$ algorithm based on Borůvka's algorithm and the reverse-delete algorithm. In 2000, Chazelle discovered the current fastest determistic algorithm which runs in time $O(E \alpha(V ))$ using soft heaps where $\alpha$ is the inverse Ackermann function.

\section{History of Flows and Cuts}

Today we will study a very interesting problem called the max flow problem. Before we get to the algorithm and math, we briefly discuss the interesting history behind max flow! During the Cold War, the US Air Force at that time was very interested in the Soviet train networks. In reports that were declassified in 1999, it was revealed that the Air Force collected enough information about the train network that they were able to determine how resources were shipped from the Soviet Union to Europe. The Air Force was very interested in determining how much resources can be transported from the Soviet Union to Europe, and what needed to be done to destroy this movement of resources. What this translates to is the min cut problem, i.e., cutting the minimum number of train tracks so that nothing goes to Europe. Here, cutting an edge meant dropping a bomb. Nowadays, however, there are much more peaceful applications of this problem, for instance, understanding the flow of information through the Internet.

\section{Formulation of the Maximum Flow Problem}

You are given an input graph $G = (V, E)$, where the edges are directed. There is a function $c : E \to R\geq 0$ that defines the capacity of each edge. We also label two nodes in $G$, $s$ and $t$, as the source and destination, respectively. The task is to output a flow of maximum value. We will shortly define what a flow is and what a flow of maximum value means.

A flow $f$ is a function $f : E \to R\geq 0$ such that
\begin{enumerate}
    \item Capacity constraints are satisfied:
    $$
        \forall (u, v ) \in E : 0 \leq f (u, v ) \leq c(u, v )
    $$.
    \item Flow conservation constraints are satisfied:
    $$
    \forall v \in V \setminus \{s, t\} : \sum_{x \in N_{\text{in}}(v)} f (x, v ) = \sum_{y\ in N_{\text{out}}(v)} f (v, y )
    $$
    Here $N_{\text{in}}(v )$ denotes the set of nodes with an edge that points to $v$ and $N_{\text{out}}(v )$ denotes the set of nodes that $v$ points to.
\end{enumerate}

\begin{figure}[h!]
    \centering
    \includegraphics[scale=0.6]{max_flow_ex.png}
    \caption{(Left) Graph $G$ with edge capacities (Right) Graph $G$ with a sample flow.}
    \label{fig:max_flow_ex}
\end{figure}

Suppose that there are no edges going into $s$ and no edges coming out of $t$. From the above, you can verify yourself that $\sum_{x \in N_{\text{out}}(s)} f (s, x) = \sum_{y \in N_{\text{in}}(t)} f (y, t)$. We define the value $x \in N_{\text{out}}(s) f (s, x)$ to be the value of the flow $f$ . We usually denote the value of a flow $f$ as $|f |$. If there are edges going into $s$ and out of $t$, then the value of $f$ is $$
|f | = \sum_{x \in N_{\text{out}}(s)} f (s, x) - \sum_{ y \in N_{\text{in}}(s)} f (y, s)
$$.

The max flow problem is to find some flow f such that $|f |$ is maximized. Remark 1. In the analysis below we consider graphs with a single source s and a single sink t. However, if we need to work with a graph with multiple sources, we can do so by adding a new source node, and then adding edges with capacity infinity from it to each of the multiple sources. Similarly, if we want to have multiple sinks, we add a new sink node and add edges from the multiple sinks to that sink with capacity infinity.

\section{Example}

In Fig. \ref{fig:max_flow_ex}, we have a graph $G$ and a sample flow $f$ . Observe that the two constraints for a flow are satisfied. There can be multiple other flows possible that can satisfy the constraints. For our given flow, $|f | = 16$. The max flow for this graph is actually $18$, as we will see shortly.

\section{Formulation of the Minimum Cut Problem}

Now, we give a formulation of the min cut problem defined for directed graphs with source and destination nodes $s$ and $t$. Note that there is also a version of the min cut problem without a source and sink node, though we won't discuss that now. An s-t cut is a partition $V = S \cup T$ where $S$ and $T$ are disjoint and $s \in S, t \in T$, and the size/cost of an s-t cut is $c(S, T) := \sum_{x\in S, y\in T} c(x, y )$.

For our graph $G$ shown above, if we set$ S = \{s, a, c\}$ and $T = \{b, t\}$, then the cost of the cut is $c(a, b) + c(c, b) + c(c, t) = 5 + 3 + 10 = 18$. If we take another cut $S
' = \{s, c\}, T' = \{a, b, t\}$, then $c(S', T') = c(s, a) + c(c, b) + c(c, t) = 10 + 3 + 10 = 23$. Note that we do
not consider the edge $\{a, c\}$ as it is in the wrong direction (we only consider edges from $S'$ to $ T'$).

\section{The Max-Flow Min-Cut Theorem}

\begin{lemma}
For any flow $f$ and any s-t cut $(S, T)$ of $G$, we have $|f | \leq c(S, T)$. In particular,the value of the max flow is at most the value of the min cut.
\end{lemma}

\begin{proof}

\begin{align*}
|f| &= \sum_{x \in N_{\text{out}}(s)} f(s,x) - \sum_{y \in N_{\text{in}}(s)} f(y,s) \\
&=\sum_{v \in S} \left(\sum_{x \in N_{\text{out}}(v)} f(v,x) - \sum_{y \in N_{in}(v)} f(y,v) \right) \tag{by flow conservation constraing for $v \neq s$} \\
&=\sum_{v \in S} \left(\sum_{x \in N_{\text{out}}(v) \cap S} f(v,x) - \sum_{y \in N_{in}(v) \cap S} f(y,v) \right) + \sum_{v \in S} \left(\sum_{x \in N_{\text{out} \cap T}(v)} f(v,x) - \sum_{y \in N_{in}(v) \cap T} f(y,v) \right) \\
&=\sum_{v \in S} \left(\sum_{x \in N_{\text{out} \cap T}(v)} f(v,x) - \sum_{y \in N_{in}(v) \cap T} f(y,v) \right) \tag{first term sums to $0$} \\
&\leq \sum_{v \in S, x \in T, x \in N_{\text{out}}(v)} f(v,x) \\
&=\leq \sum_{v \in S, x \in T, x \in N_{\text{out}}(v)} c(v,x) \\
&= c(S,T)
\end{align*}

In the proof, $\sum_{v \in S} \left(\sum_{x\in N_{\text{out}}(v)} f(v,x) - \sum_{y\in N_{\text{int}}(v)} f(y,v)\right) = 0$ since we add and subtract the flow $f (u, v )$ for every $u, v \in S$ such that $(u, v ) \in E$.
\end{proof}

We get the following consequence.

\begin{corollary}
If we can find $f$ and $(S,T)$ such that $|f| = c(S,T)$ then $f$ is a max-flow and $(S,T)$ is a min s-t cut.
\end{corollary}

It turns out we can always find such an $f$ and $(S,T)$ for any graph.

\begin{theorem}{Max-Flow Min-cut Theorem}
For any graph $G$, source $s$ and destination $t$, the value of the max flow is equal to the cost of the min cut
\end{theorem}

We will show this by coming up with an algorithm. The algorithm will take the graph $G$ and some flow $f$ that has already been constructed, and create a new graph that is called the residual graph. In this new graph, the algorithm will try to find a path from s to t. If no such path exists, we will show that the value of the flow we started with is the value of the maximum flow. If not, we show how to increase the value of our flow by pushing some flow
on that path.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.6]{max_flow_res.png}
\caption{The Residue network given the flow presented in Figure \ref{fig:max_flow_ex}}
\label{fig:max_flow_res}
\end{figure}

\end{document}


\documentclass{article}

\usepackage{amsmath}
\usepackage{numprint}

\author{Daniel Fernandes Martins (danielfmt)}
\title{Question \#9 Solution}

\begin{document}

\maketitle

\textbf{Disclaimer.} This is the reasoning I used to solve the problem; it
may be wrong though. This is intended just as food for thought.

\section{Finding The Lower Bound}

Let's start with two hypothesis sets with break point $k=2$ and $N=2$. This
means that these hypothesis sets cannot shatter any combination of 2 points,
which implies that $d_{vc}=1$ for both of them.

These are the dichotomies realized by $\mathcal{H}_1$ on these 2 points:

\begin{equation*}
\begin{split}
\{a=-1, b=+1\} \\
\{a=+1, b=-1\}
\end{split}
\end{equation*}

For $\mathcal{H}_2$:

\begin{equation*}
\begin{split}
\{a=-1, b=-1\} \\
\{a=+1, b=+1\}
\end{split}
\end{equation*}

Notice that the dichotomies realized by $\mathcal{H}_1$ are different than the
ones realized by $\mathcal{H}_2$, which means the intersection
$\mathcal{H}_1\bigcap\mathcal{H}_2$ is an empty set, so $d_{vc}=0$.

\section{Finding The Upper Bound}

Let's say you have two hypothesis sets $\mathcal{H}_1$ and $\mathcal{H}_2$. The
biggest set you can get from any intersection of these two sets is by having
two sets such that $\mathcal{H}_2\subseteq\mathcal{H}_1$. Therefore, the upper
bound can be at most as big as the smallest $\mathcal{H}_k$.

\section{Solution}

\begin{equation*}
0 \leq \\
  d_{vc}(\bigcap_{k=1}^K\mathcal{H}_k) \leq \\
  \min\{d_{vc}(\mathcal{H}_k)\}_{k=1}^K
\end{equation*}

\end{document}


\chapter{Inner model theory}
Model theory is \emph{really} meta, so you will have to pay attention here.

Roughly, a ``model of $\ZFC$'' is a set with a binary relation that satisfies the $\ZFC$ axioms,
just as a group is a set with a binary operation that satisfies the group axioms.
Unfortunately, unlike with groups, it is very hard for me to give interesting examples of models,
for the simple reason that we are literally trying to model the entire universe.

\section{Models}
\prototype{$(\omega, \in)$ obeys $\PowerSet$, $V_\kappa$ is a model for $\kappa$ inaccessible (later).}
\begin{definition}
	A \vocab{model} $\MM$ consists of a set $M$ and a
	binary relation $E \subseteq M \times M$.
	(The $E$ relation is the ``$\in$'' for the model.)
\end{definition}
\begin{remark}
	I'm only considering \emph{set-sized} models where $M$ is a set.
	Experts may be aware that I can actually play with $M$ being a class,
	but that would require too much care for now.
\end{remark}
If you have a model, you can ask certain things about it,
such as ``does it satisfy $\EmptySet$?''.
Let me give you an example of what I mean and then make it rigorous.
\begin{example}
	[A stupid model]
	Let's take $\MM = (M,E) = \left( \omega, \in \right)$.
	This is not a very good model of $\ZFC$, but let's see if we can
	make sense of some of the first few axioms.
	\begin{enumerate}[(a)]
		\ii $\MM$ satisfies $\Extensionality$, which is the sentence
		\[ \forall x \forall y \forall a :
			\left( a \in x \iff a \in y \right) \implies x = y. \]
		This just follows from the fact that $E$ is actually $\in$.

		\ii $\MM$ satisfies $\EmptySet$, which is the sentence
		\[ \exists a : \forall x \; \neg (x \in a). \]
		Namely, take $a = \varnothing \in \omega$.

		\ii $\MM$ does not satisfy $\Pairing$, since $\{1,3\}$ is not in $\omega$,
		even though $1, 3 \in \omega$.

		\ii Miraculously, $\MM$ satisfies $\Union$, since for any $n \in \omega$,
		$\cup n$ is $n-1$ (unless $n=0$).
		The Union axiom statements that
		\[ \forall a \exists U \quad \forall x \;
		\left[ (x \in U) \iff (\exists y : x \in y \in a) \right]. \]
		An important thing to notice is that the ``$\forall a$'' ranges only over
		the sets in the model of the universe, $\MM$.
	\end{enumerate}
\end{example}
\begin{example}
	[Important: this stupid model satisfies $\PowerSet$]
	Most incredibly of all: $\MM = (\omega, \in)$ satisfies $\PowerSet$.
	This is a really important example.

	You might think this is ridiculous. Look at $2 = \{0,1\}$.
	The power set of this is $\{0, 1, 2, \{1\}\}$ which is not in the model, right?

	Well, let's look more closely at $\PowerSet$. It states that:
	\[ \forall x \exists a \forall y (y \in a \iff y \subseteq x). \]
	What happens if we set $x = 2 = \{0,1\}$?
	Well, actually, we claim that $a = 3 = \{0,1,2\}$ works.
	The key point is ``for all $y$'' -- this \emph{only ranges over the objects in $\MM$}.
	In $\MM$, the only subsets of $2$ are $0 = \varnothing$,
	$1 = \{0\}$ and $2 = \{0,1\}$.
	The ``set'' $\{1\}$ in the ``real world'' (in $V$) is not a set in the model $\MM$.

	In particular, you might say that in this strange new world,
	we have $2^n = n+1$, since $n = \{0,1,\dots,n-1\}$ really does
	have only $n+1$ subsets.
\end{example}

\begin{example}
	[Sentences with parameters]
	The sentences we ask of our model are allowed to have ``parameters'' as well.
	For example, if $\MM = (\omega, \in)$ as before then $\MM$ satisfies the sentence
	\[ \forall x \in 3 (x \in 5). \]
\end{example}

\section{Sentences and satisfaction}
With this intuitive notion, we can define what it means for a model to satisfy a sentence.
\begin{definition}
Note that any sentence $\phi$ can be written in one of five forms:
\begin{itemize}
	\ii $x \in y$
	\ii $x = y$
	\ii $\neg \psi$ (``not $\psi$'') for some shorter sentence $\psi$
	\ii $\psi_1 \lor \psi_2$ (``$\psi_1$ or $\psi_2$'')
	for some shorter sentences $\psi_1$, $\psi_2$
	\ii $\exists x \psi$ (``exists $x$'') for some shorter sentence $\psi$.
\end{itemize}
\end{definition}
\begin{ques}
	What happened to $\land$ (and) and $\forall$ (for all)?
	(Hint: use $\neg$.)
\end{ques}
Often (almost always, actually) we will proceed by so-called ``induction on formula complexity'',
meaning that we define or prove something by induction using this.
Note that we require all formulas to be finite.

Now suppose we have a sentence $\phi$, like $a = b$ or $\exists a \forall x \neg (x \in a)$,
plus a model $\MM = (M,E)$.
We want to ask whether $\MM$ satisfies $\phi$.

To give meaning to this, we have to designate certain variables as \vocab{parameters}.
For example, if I asked you
\begin{quote}
	``Does $a=b$?''
\end{quote}
the first question you would ask is what $a$ and $b$ are.
So $a$, $b$ would be parameters: I have to give them values for this sentence to make sense.

On the other hand, if I asked you
\begin{quote}
	``Does $\exists a \forall x \neg (x \in a)$?''
\end{quote}
then you would just say ``yes''.
In this case, $x$ and $a$ are \emph{not} parameters.
In general, parameters are those variables whose meaning is not given by some $\forall$ or $\exists$.

In what follows, we will let $\phi(x_1, \dots, x_n)$ denote a formula $\phi$,
whose parameters are $x_1$, \dots, $x_n$.
Note that possibly $n=0$, for example all $\ZFC$ axioms have no parameters.

\begin{ques}
	Try to guess the definition of satisfaction before reading it below.
	(It's not very hard to guess!)
\end{ques}

\begin{definition}
	Let $\MM=(M,E)$ be a model.
	Let $\phi(x_1, \dots, x_n)$ be a sentence, and let $b_1, \dots, b_n \in M$.
	We will define a relation
	\[ \MM \vDash \phi[b_1, \dots, b_n] \]
	and say $\MM$ \vocab{satisfies} the sentence $\phi$ with parameters $b_1, \dots, b_n$.

	The relationship is defined by induction on formula complexity as follows:
	\begin{itemize}
		\ii If $\phi$ is ``$x_1=x_2$'' then $\MM \vDash \phi[b_1, b_2] \iff b_1 = b_2$.
		\ii If $\phi$ is ``$x_1\in x_2$'' then $\MM \vDash \phi[b_1, b_2] \iff b_1 \; E \; b_2$. \\
		(This is what we mean by ``$E$ interprets $\in$''.)
		\ii If $\phi$ is ``$\neg \psi$'' then
		$\MM \vDash \phi[b_1, \dots, b_n] \iff \MM \not\vDash \psi[b_1, \dots, b_n]$.
		\ii If $\phi$ is ``$\psi_1 \lor \psi_2$'' then $\MM \vDash \phi[b_1, \dots, b_n]$
		means $\MM \vDash \psi_i[b_1, \dots, b_n]$ for some $i=1,2$.
		\ii Most important case: suppose $\phi$ is $\exists x \psi(x,x_1, \dots, x_n)$.
		Then $\MM \vDash \phi[b_1, \dots, b_n]$ if and only if
		\[ \exists b \in M \text{ such that } \MM \vDash \psi[b, b_1, \dots, b_n]. \]
		Note that $\psi$ has one extra parameter.
	\end{itemize}
\end{definition}
Notice where the information of the model actually gets used.
We only ever use $E$ in interpreting $x_1 \in x_2$; unsurprising.
But we only ever use the set $M$ when we are running over $\exists$ (and hence $\forall$).
That's well-worth keeping in mind:
\begin{moral}
	The behavior of a model essentially comes from $\exists$ and $\forall$,
	which search through the entire model $M$.
\end{moral}

And finally,
\begin{definition}
	A \vocab{model of $\ZFC$} is a model $\MM = (M,E)$ satisfying all $\ZFC$ axioms.
\end{definition}

We are especially interested in models of the form $(M, \in)$, where $M$ is a \emph{transitive} set.
(We want our universe to be transitive,
otherwise we would have elements of sets which are not themselves
in the universe, which is very strange.)
Such a model is called a \vocab{transitive model}.
\begin{abuse}
	If $M$ is a transitive set, the model $(M, \in)$ will be abbreviated to just $M$.
\end{abuse}
\begin{definition}
	An \vocab{inner model} of $\ZFC$ is a transitive model satisfying $\ZFC$.
\end{definition}

\section{The Levy hierarchy}
\prototype{$\mathtt{isSubset}(x,y)$ is absolute. The axiom
$\EmptySet$ is $\Sigma_1$, $\mathtt{isPowerSetOf}(X,x)$ is $\Pi_1$.}
A key point to remember is that the behavior of a model is largely determined by $\exists$ and $\forall$.
It turns out we can say even more than this.

Consider a formula such as
\[ \mathtt{isEmpty}(x) : \neg \exists a (a \in x) \]
which checks whether a given set $x$ has an element in it.
Technically, this has an ``$\exists$'' in it.
But somehow this $\exists$ does not really search over the entire model,
because it is \emph{bounded} to search in $x$.
That is, we might informally rewrite this as
\[ \neg (\exists a \in x) \]
which doesn't fit into the strict form,
but points out that we are only looking over $a \in x$.
We call such a quantifier a \vocab{bounded quantifier}.
%and write it
%in the way we see it in most mathematics, such as
%\[ (\exists x \in a) \quad\text{or}\quad (\forall x \in a). \]
%To be painfully explicit,
%$\exists x \in a \psi$ is short for $\exists x (x \in a \land \psi)$,
%while $\forall x \in a \psi$ is short for $\forall x (x \in a \implies \psi)$.

We like sentences with bounded quantifiers because they designate
properties which are \vocab{absolute} over transitive models.
It doesn't matter how strange your surrounding model $M$ is.
As long as $M$ is transitive,
\[ M \vDash \mathtt{isEmpty}(\varnothing) \]
will always hold.
Similarly, the sentence
\[ \mathtt{isSubset}(x,y) : x \subseteq y \text { i.e. } \forall a \in x (a \in y) \]
is absolute.
Sentences with this property are called $\Sigma_0$ or $\Pi_0$.

The situation is different with a sentence like
\[
	\mathtt{isPowerSetOf}(y,x) :
	\forall z \left( z \subseteq x \iff z \in y  \right)
\]
which in English means ``$y$ is the power set of $x$'', or just $y = \PP(x)$.
The $\forall z$ is \emph{not} bounded here.
This weirdness is what allows things like
\[ \omega \vDash \text{``$\{0,1,2\}$ is the power set of $\{0,1\}$''} \]
and hence
\[ \omega \vDash \PowerSet \]
which was our stupid example earlier.
The sentence $\mathtt{isPowerSetOf}$ consists of an unbounded $\forall$ followed
by an absolute sentence, so we say it is $\Pi_1$.

More generally, the \vocab{Levy hierarchy} keeps track of how bounded our
quantifiers are.
Specifically,
\begin{itemize}
	\ii Formulas which have only bounded quantifiers
	are $\Delta_0 = \Sigma_0 = \Pi_0$.
	\ii Formulas of the form $\exists x_1 \dots \exists x_k \psi$
	where $\psi$ is $\Pi_n$ are considered $\Sigma_{n+1}$.
	\ii Formulas of the form $\forall x_1 \dots \forall x_k \psi$
	where $\psi$ is $\Sigma_n$ are considered $\Pi_{n+1}$.
\end{itemize}
(A formula which is both $\Sigma_n$ and $\Pi_n$ is called $\Delta_n$, but we won't
use this except for $n=0$.)

\begin{example}
	[Examples of $\Delta_0$ sentences]
	\listhack
	\begin{enumerate}[(a)]
		\ii The sentences $\mathtt{isEmpty}(x)$, $x \subseteq y$, as discussed above.
		\ii The formula ``$x$ is transitive'' can be expanded as a $\Delta_0$ sentence.
		\ii The formula ``$x$ is an ordinal'' can be expanded as a $\Delta_0$ sentence.
	\end{enumerate}
\end{example}
\begin{exercise}
	Write out the expansions for ``$x$ is transitive'' and ``$x$ is ordinal''
	in a $\Delta_0$ form.
\end{exercise}
\begin{example}
	[More complex formulas]
	\listhack
	\begin{enumerate}[(a)]
		\ii The axiom $\EmptySet$ is $\Sigma_1$; it is $\exists a (\mathtt{isEmpty}(a))$,
		and $\mathtt{isEmpty}(a)$ is $\Delta_0$.
		\ii The formula ``$y = \PP(x)$'' is $\Pi_1$, as discussed above.
		\ii The formula ``$x$ is countable'' is $\Sigma_1$.
		One way to phrase it is ``$\exists f$ an injective map $x \injto \omega$'',
		which necessarily has an unbounded ``$\exists f$''.
		\ii The axiom $\PowerSet$ is $\Pi_3$:
		\[ \forall y \exists P \forall x (x\subseteq y \iff x \in P). \]
	\end{enumerate}
\end{example}

\section{Substructures, and Tarski-Vaught}
Let $\MM_1 = (M_1, E_1)$ and $\MM_2 = (M_2, E_2)$ be models.
\begin{definition}
	We say that $\MM_1 \subseteq \MM_2$ if $M_1 \subseteq M_2$ and
	$E_1$ agrees with $E_2$; we say $\MM_1$ is a \vocab{substructure} of $\MM_2$.
\end{definition}

That's boring. The good part is:
\begin{definition}
	We say $\MM_1 \prec \MM_2$, or $\MM_1$ is an \vocab{elementary substructure} of $\MM_2$,
	if $\MM_1 \subseteq \MM_2$ and for \emph{every} sentence $\phi(x_1, \dots, x_n)$
	and parameters $b_1, \dots, b_n \in M_1$, we have
	\[
		\MM_1 \vDash \phi[b_1, \dots, b_n] \iff
		\MM_2 \vDash \phi[b_1, \dots, b_n].
	\]
\end{definition}
In other words, $\MM_1$ and $\MM_2$ agree on every sentence possible.
Note that the $b_i$ have to come from $\MM_1$; if the $b_i$ came from $\MM_2$ then
asking something of $\MM_1$ wouldn't make sense.

Let's ask now: how would $\MM_1 \prec \MM_2$ fail to be true?
If we look at the possible sentences, none of the atomic formulas,
nor the ``$\land$'' and ``$\neg$'', are going to cause issues.

The intuition you should be getting by now is that things go
wrong once we hit $\forall$ and $\exists$.
They won't go wrong for bounded quantifiers.
But unbounded quantifiers search the entire model, and that's where things go wrong.

To give a ``concrete example'':
imagine $\MM_1$ is MIT, and $\MM_2$ is the state of Massachusetts.
If $\MM_1$ thinks there exist hackers at MIT,
certainly there exist hackers in Massachusetts.
Where things go wrong is something like:
\[ \MM_2 \vDash \text{``$\exists x$ : $x$ is a course numbered $> 50$''}. \]
This is true for $\MM_2$ because we can
take the witness $x = \text{Math 55}$, say.
But it's false for $\MM_1$,
because at MIT all courses are numbered $18.701$ or something similar.
\begin{moral}
	The issue is that the \emph{witnesses}
	for statements in $\MM_2$ do not necessarily propagate
	down to witnesses for $\MM_1$.
\end{moral}

The Tarski-Vaught test says this is the only impediment:
if every witness in $\MM_2$ can be replaced by one in $\MM_1$ then $\MM_1 \prec \MM_2$.
\begin{lemma}
	[Tarski-Vaught]
	Let $\MM_1 \subseteq \MM_2$.
	Then $\MM_1 \prec \MM_2$ if and only if:
	For every sentence $\phi(x, x_1, \dots, x_n)$ and parameters
	$b_1, \dots, b_n \in M_1$:
	if there is a witness $\tilde b \in M_2$
	to $\MM_2 \vDash \phi(\tilde b, b_1 \dots, b_n)$
	then there is a witness $b \in M_1$ to $\MM_1 \vDash \phi(b, b_1, \dots, b_n)$.
\end{lemma}
\begin{proof}
	Easy after the above discussion.
	To formalize it, use induction on formula complexity.
\end{proof}

\section{Obtaining the axioms of $\ZFC$}
We now want to write down conditions for $M$ to satisfy $\ZFC$ axioms.
The idea is that almost all the $\ZFC$ axioms are just $\Sigma_1$
claims about certain desired sets,
and so verifying an axiom reduces to checking some appropriate ``closure'' condition:
that the witness to the axiom is actually in the model.

For example, the $\EmptySet$ axiom is ``$\exists a (\mathtt{isEmpty}(a))$'',
and so we're happy as long as $\varnothing \in M$, which is of course
true for any nonempty transitive set $M$.

\begin{lemma}[Transitive sets inheriting $\ZFC$]
	\label{lem:transitive_ZFC}
	Let $M$ be a nonempty transitive set. Then
	\begin{enumerate}[(i)]
		\ii $M$ satisfies $\Extensionality$, $\Foundation$, $\EmptySet$.
		\ii $M \vDash \Pairing$ if $x,y \in M \implies \{x,y\} \in M$.
		\ii $M \vDash \Union$ if $x \in M \implies \cup x \in M$.
		\ii $M \vDash \PowerSet$ if $x \in M \implies \PP(x) \cap M \in M$.
		\ii $M \vDash \Replacement$ if for every $x \in M$
		and every function $F : x \to M$
		which is $M$-definable with parameters,
		we have $F``x \in M$ as well.
		\ii $M \vDash \Infinity$ as long as $\omega \in M$.
	\end{enumerate}
\end{lemma}
Here, a set $X \subseteq M$ is \vocab{$M$-definable with parameters}
if it can be realized as
\[ X = \left\{ x \in M \mid \phi[x, b_1, \dots, b_n] \right\} \]
for some (fixed) choice of parameters $b_1,\dots,b_n \in M$.
We allow $n=0$, in which case we say $X$ is \vocab{$M$-definable without parameters}.
Note that $X$ need not itself be in $M$!
As a trivial example, $X = M$ is $M$-definable without parameters
(just take $\phi[x]$ to always be true), and certainly we do not have $X \in M$.
\begin{exercise}
	Verify (i)-(iv) above.
\end{exercise}
\begin{remark}
	Converses to the statements of \Cref{lem:transitive_ZFC}
	are true for all claims other than (vi).
\end{remark}

\section{Mostowski collapse}
Up until now I have been only talking about transitive models,
because they were easier to think about.
Here's a second, better reason we might only care about transitive models.

\begin{lemma}
	[Mostowski collapse lemma]
	Let $X = (X, \in)$ be a model,
	where $X$ is a set (possibly not transitive).
	Then there exists an isomorphism $\pi \colon X \to M$ for
	a transitive model $M = (M,\in)$.
\end{lemma}

This is also called the \emph{transitive collapse}.
In fact, both $\pi$ and $M$ are unique.

\begin{proof}
	The idea behind the proof is very simple.
	Since $\in$ is well-founded and extensional
	(satisfies $\Foundation$ and $\Extensionality$, respectively),
	we can look at the $\in$-minimal element $x_\varnothing$
	of $X$ with respect to $\in$.
	Clearly, we want to send that to $0 = \varnothing$.

	Then we take the next-smallest set under $\in$, and send it to $1 = \{\varnothing\}$.
	We ``keep doing this''; it's not hard to see this does exactly what we want.

	To formalize, define $\pi$ by transfinite recursion:
	\[ \pi(x) \defeq \left\{ \pi(y) \mid y \in x \right\}. \]
	This $\pi$, by construction, does the trick.
\end{proof}

\begin{remark}
	[Digression for experts]
	Earlier versions of Napkin claimed this was true for general models
	$\mathscr X = (X, E)$ with $\mathscr X \vDash \Foundation + \Extensionality$.
	This is false; it does not even imply $E$ is well-founded,
	because there may be infinite descending chains of subsets of $X$
	which do not live in $X$ itself.
	Another issue is that $E$ may not be set-like.
\end{remark}

The picture of this is ``collapsing'' the elements of $M$ down
to the bottom of $V$, hence the name.
\missingfigure{Picture of Mostowski collapse}

\section{Adding an inaccessible}
\prototype{$V_\kappa$}
At this point you might be asking, well, where's my model of $\ZFC$?

I unfortunately have to admit now: $\ZFC$ can never prove that there is a model of $\ZFC$
(unless $\ZFC$ is inconsistent, but that would be even worse).
This is a result called G\"odel's incompleteness theorem.

Nonetheless, with some very modest assumptions added,
we can actually show that a model \emph{does} exist:
for example, assuming that there exists a strongly inaccessible cardinal $\kappa$ would do the trick,
$V_\kappa$ will be such a model (\Cref{prob:inaccessible_model}).
Intuitively you can see why: $\kappa$ is so big that any set of rank lower than it can't escape it
even if we take their power sets, or any other method that $\ZFC$ lets us do.

More pessimistically,
this shows that it's impossible to prove in $\ZFC$ that such a $\kappa$ exists.
Nonetheless, we now proceed under $\ZFC^+$ for convenience,
which adds the existence of such a $\kappa$
as a final axiom.
So we now have a model $V_\kappa$ to play with. Joy!

Great. Now we do something \emph{really} crazy.
\begin{theorem}[Countable transitive model]
	Assume $\ZFC^+$. Then there exists a transitive model $X$ of $\ZFC$
	such that $X$ is a \emph{countable} set.
\end{theorem}
\begin{proof}
	Fasten your seat belts.

	First, since we assumed $\ZFC^+$,
	we can take $V_\kappa = (V_\kappa, \in)$ as our model of $\ZFC$.
	Start with the set $X_0 = \varnothing$.
	Then for every integer $n$, we do the following to get $X_{n+1}$.
	\begin{itemize}
		\ii Start with $X_{n+1}$ containing every element of $X_n$.
		\ii Consider a formula $\phi(x, x_1, \dots, x_n)$
		and $b_1, \dots, b_n$ in $X_n$.
		Suppose that $V_\kappa$ thinks there is a $b \in V_\kappa$ for which
		\[ V_\kappa \vDash \phi[b, b_1, \dots, b_n]. \]
		We then add in the element $b$ to $X_{n+1}$.
		\ii We do this for \emph{every possible formula in the language of set theory}.
		We also have to put in \emph{every possible set of parameters} from the previous set $X_n$.
	\end{itemize}
	At every step $X_n$ is countable.
	Reason: there are countably many possible finite sets of parameters in $X_n$,
	and countably many possible formulas, so in total we only ever add in countably many things
	at each step.
	This exhibits an infinite nested sequence of countable sets
	\[ X_0 \subseteq X_1 \subseteq X_2 \subseteq \dots \]
	None of these is a substructure of $V_\kappa$,
	because each $X_n$ relies on witnesses in $X_{n+1}$.
	So we instead \emph{take the union}:
	\[ X = \bigcup_n X_n. \]
	This satisfies the Tarski-Vaught test, and is countable.

	There is one minor caveat: $X$ might not be transitive.
	We don't care, because we just take its Mostowski collapse.
\end{proof}

Please take a moment to admire how insane this is.
It hinges irrevocably on the fact that there are countably many sentences we can write down.

\begin{remark}
	This proof relies heavily on the Axiom of Choice
	when we add in the element $b$ to $X_{n+1}$.
	Without Choice, there is no way of making these decisions all at once.

	Usually, the right way to formalize the Axiom of Choice usage is,
	for every formula $\phi(x, x_1, \dots, x_n)$, to pre-commit (at the very beginning)
	to a function $f_\phi(x_1, \dots, x_n)$, such that given any $b_1, \dots, b_n$
	$f_\phi(b_1, \dots, b_n)$ will spit out the suitable value of $b$ (if one exists).
	Personally, I think this is hiding the spirit of the proof, but it does
	make it clear how exactly Choice is being used.

	These $f_\phi$'s have a name: \vocab{Skolem functions}.
\end{remark}

The trick we used in the proof works in more general settings:
\begin{theorem}
	[Downward L\"owenheim-Skolem theorem]
	Let $\MM = (M,E)$ be a model, and $A \subseteq M$.
	Then there exists a set $B$ (called the \vocab{Skolem hull} of $A$)
	with $A \subseteq B \subseteq M$,
	such that $(B,E) \prec \MM$, and
	\[ \left\lvert B \right\rvert =
		\max \left\{ \omega, \left\lvert A \right\rvert \right\}. \]
\end{theorem}
In our case, what we did was simply take $A$ to be the empty set.
\begin{ques}
	Prove this. (Exactly the same proof as before.)
\end{ques}

\section{FAQ's on countable models}
The most common one is ``how is this possible?'',
with runner-up ``what just happened''.

Let me do my best to answer the first question.
It seems like there are two things running up against each other:
\begin{enumerate}[(1)]
	\ii $M$ is a transitive model of $\ZFC$, but its universe is uncountable.
	\ii $\ZFC$ tells us there are uncountable sets!
\end{enumerate}
(This has confused so many people it has a name, Skolem's paradox.)

The reason this works I actually pointed out earlier:
\emph{countability is not absolute, it is a $\Sigma_1$ notion}.

Recall that a set $x$ is countable if
\emph{there exists} an injective map $x \injto \omega$.
The first statement just says that \emph{in the universe $V$},
there is a injective map $F: M \injto \omega$.
In particular, for any $x \in M$ (hence $x \subseteq M$, since $M$ is transitive),
$x$ is countable \emph{in $V$}.
This is the content of the first statement.

But for $M$ to be a model of $\ZFC$, $M$ only has to think statements in $\ZFC$ are true.
More to the point, the fact that $\ZFC$ tells us there are uncountable sets means
\[ M \vDash \text{$\exists x$ uncountable}. \]
In other words,
\[ M \vDash \exists x \forall f
	\text{ If $f : x \to \omega$ then $f$ isn't injective}. \]
The key point is the $\forall f$ searches only functions in our tiny model $M$.
It is true that in the ``real world'' $V$, there are injective functions $f : x \to \omega$.
But $M$ has no idea they exist!
It is a brain in a vat: $M$ is oblivious to any information outside it.

So in fact, every ordinal which appears in $M$ is countable in the real world.
It is just not countable in $M$.
Since $M \vDash \ZFC$, $M$ is going to think there is some smallest uncountable cardinal,
say $\aleph_1^M$.
It will be the smallest (infinite) ordinal in $M$
with the property that there is no bijection \emph{in the model $M$}
between $\aleph_1^M$ and $\omega$.
However, we necessarily know that such a bijection is going to exist in the real world $V$.

Put another way, cardinalities in $M$ can look vastly different from those in the real world,
because cardinality is measured by bijections, which I guess is inevitable, but leads to chaos.

\section{Picturing inner models}
Here is a picture of a countable transitive model $M$.

\begin{center}
	\begin{asy}
		size(14cm);
		pair A = (12,30);
		pair B = -conj(A);
		pair M = midpoint(A--B);
		pair O = origin;
		MP("V", A, dir(10));
		draw(A--O--B);

		fill(A--O--B--cycle, opacity(0.3)+palecyan);
		MP("M", 0.7*M, 3*dir(0)+dir(45));

		MP("V_0 = \varnothing", origin, dir(-20));
		MP("V_1 = \{\varnothing\}", 0.05*A, dir(0));
		MP("V_2 = \{\varnothing, \{\varnothing\} \}", 0.10*A, dir(0));

		pair A1 = 0.4*A;
		pair B1 = 0.4*B;
		draw(MP("V_\omega", A1, dir(0))--B1);
		draw(MP("V_{\omega+1} = \mathcal P(V_\omega)", 0.45*A, dir(0))--0.45*B);
		Drawing("\omega", 0.45*M, dir(45));

		filldraw(O--A1--(A1+0.15*M)..(0.7*M)..(B1+0.15*M)--B1--cycle,
			opacity(0.3)+lightgreen, heavygreen+1);
		draw(O--0.7*M, heavygreen+1);

		Drawing("\aleph_1^V", 0.80*M, dir(0));
		Drawing("\aleph_2^V", 0.90*M, dir(0));
		Drawing("\aleph_1^M", 0.55*M, dir(0));
		Drawing("\aleph_2^M", 0.60*M, dir(0));

		pair F = 0.6*B+0.15*A;
		Drawing("f", F, dir(135));
		draw(F--0.55*M, dotted, EndArrow, Margins);
		draw(F--0.45*M, dotted, EndArrow, Margins);

		draw(0.7*M--M);
		MP("\mathrm{On}^V", M, dir(90));
		MP("\mathrm{On}^M", 0.7*M, dir(135));
	\end{asy}
\end{center}

Note that $M$ and $V$ must agree on finite sets,
since every finite set has a formula that can express it.
However, past $V_\omega$ the model and the true universe start to diverge.

The entire model $M$ is countable, so it only occupies a small
portion of the universe, below the first uncountable cardinal $\aleph_1^V$
(where the superscript means ``of the true universe $V$'').
The ordinals in $M$ are precisely the ordinals of $V$ which happen to live inside the model,
because the sentence ``$\alpha$ is an ordinal'' is absolute.
On the other hand, $M$ has only a portion of these ordinals, since it is only
a lowly set, and a countable set at that.
To denote the ordinals of $M$, we write $\On^M$, where the superscript means
``the ordinals as computed in $M$''.
Similarly, $\On^V$ will now denote the ``set of true ordinals''.

Nonetheless, the model $M$ has its own version of the first uncountable
cardinal $\aleph_1^M$.
In the true universe, $\aleph_1^M$ is countable (below $\aleph_1^V$),
but the necessary bijection witnessing this might not be inside $M$.
That's why $M$ can think $\aleph_1^M$ is uncountable,
even if it is a countable cardinal in the original universe.

So our model $M$ is a brain in a vat.
It happens to believe all the axioms of $\ZFC$, and so every
statement that is true in $M$ could conceivably be true in $V$ as well.
But $M$ can't see the universe around it; it has no idea that what it believes is
the uncountable $\aleph_1^M$ is really just an ordinary countable cardinal.

\section\problemhead
\begin{sproblem}
	Show that for any transitive model $M$, the set of ordinals in $M$
	is itself some ordinal.
\end{sproblem}

\begin{dproblem}
	Assume $\MM_1 \subseteq \MM_2$. Show that
	\begin{enumerate}[(a)]
		\ii If $\phi$ is $\Delta_0$,
		then $\MM_1 \vDash \phi[b_1, \dots, b_n] \iff \MM_2 \vDash \phi[b_1, \dots, b_n]$.
		\ii If $\phi$ is $\Sigma_1$,
		then $\MM_1 \vDash \phi[b_1, \dots, b_n] \implies \MM_2 \vDash \phi[b_1, \dots, b_n]$.
		\ii If $\phi$ is $\Pi_1$,
		then $\MM_2 \vDash \phi[b_1, \dots, b_n] \implies \MM_1 \vDash \phi[b_1, \dots, b_n]$.
	\end{enumerate}
	(This should be easy if you've understood the chapter.)
\end{dproblem}

\begin{dproblem}[Reflection]
	\gim
	Let $\kappa$ be an inaccessible cardinal such that $|V_\alpha| < \kappa$ for all $\alpha < \kappa$.
	Prove that for any $\delta < \kappa$ there exists $\delta < \alpha < \kappa$
	such that $V_\alpha \prec V_\kappa$; in other words,
	the set of $\alpha$ such that $V_\alpha \prec V_\kappa$ is \emph{unbounded} in $\kappa$.
	This means that properties of $V_\kappa$ reflect down to properties of $V_\alpha$.
	\begin{hint}
		This is very similar to the proof of L\"owenheim-Skolem.
		For a sentence $\phi$, let $f_\phi$
		send $\alpha$ to the least $\beta < \kappa$ such that for all $\vec b \in V_\alpha$, if there exists $a \in M$ such that $V_\kappa \vDash \phi[a, \vec b]$ then $\exists a \in V_\beta$ such that $V_\kappa \vDash \phi[a, \vec b]$.
		(To prove this $\beta$ exists, use the fact that $\kappa$ is cofinal.)
		Then, take the supremum over the countably many sentences for each $\alpha$.
	\end{hint}
	\begin{sol}
		For a sentence $\phi$ let \[ f_\phi : \kappa \to \kappa \]
		send $\alpha$ to the least $\beta < \kappa$ such that for all $\vec b \in V_\alpha$, if there exists $a \in V_\kappa$ such that $V_\kappa \vDash \phi[a, \vec b]$ then $\exists a \in V_\beta$ such that $V_\kappa \vDash \phi[a, \vec b]$.

		We claim this is well-defined.
		There are only $\left\lvert V_\alpha \right\rvert^n$ many possible choices of $\vec b$,
		and in particular there are fewer than $\kappa$ of these
		(since we know that $\left\lvert V_\alpha \right\rvert < \kappa$; compare \Cref{prob:strongly_inaccessible}).
		Otherwise, we can construct a cofinal map from $\left\lvert V_\alpha^n \right\rvert$
		into $\kappa$ by mapping each vector $\vec b$ into a $\beta$ for which the proposition fails.
		And that's impossible since $\kappa$ is regular!

		In other words, what we've done is fix $\phi$ and then use Tarski-Vaught on all the $\vec b \in V_\alpha^n$.
		Now let $g : \kappa \to \kappa$ be defined by
		\[ \alpha \mapsto \sup f_\phi(\alpha). \]
		Since $\kappa$ is regular and there are only countably many formulas, $g(\alpha)$ is well-defined.

		Check that if $\alpha$ has the property that $g$ maps $\alpha$ into itself (in other words, $\alpha$ is closed under $g$), then by the Tarski-Vaught test, we have $V_\alpha \prec V_\kappa$.

		So it suffices to show there are arbitrarily large $\alpha < \kappa$ which are closed under $g$.
		Fix $\alpha_0$. Let $\alpha_1 = g(\alpha_0$), et cetera and define
		\[ \alpha = \sup_{n < \omega} \alpha_n. \]
		This $\alpha$ is closed under $g$, and by making $\alpha_0$ arbitrarily large we can make $\alpha$ as large as we like.
	\end{sol}
\end{dproblem}

\begin{sproblem}
	[Inaccessible cardinals produce models]
	\label{prob:inaccessible_model}
	\gim
	Let $\kappa$ be an inaccessible cardinal.
	Prove that $V_\kappa$ is a model of $\ZFC$.
\end{sproblem}


\section{Measuring distance between vectors}


    \foldertitle{trec}{Time-Recursive Subscript Objects for Time-Recursive Expressions}{trec/Contents}

	Time subscript objects enable to create and evaluate time-recursive
expressions based on tseries objects. Read below carefully when IRIS
fails to evaluate time-recursive expessions correctly.

Trec methods:

\paragraph{Constructor}\label{constructor}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \href{trec/trec}{\texttt{trec}} - Create new recursive time subscript
  object.
\end{itemize}

\paragraph{Creating lags and leads}\label{creating-lags-and-leads}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \href{trec/plus}{\texttt{plus}} - Create time-recursive lead of
  tseries object.
\item
  \href{trec/minus}{\texttt{minus}} - Create time-recursive lag of
  tseries object.
\end{itemize}

\paragraph{Using Time-Recursive
Subscripts}\label{using-time-recursive-subscripts}

Time-recursive expressions are expressions that are evaluated period by
period, with each result assigned immediately to the left-hand side
tseries variable, and used in subsequent periods evaluated afterwards.

To construct and evaluate time-recursive expressions, use tseries
referenced by a trec object, or a lag or lead created from a trec
object. Every tseries object on both the left-hand side (i.e.~the
variable under construction) and the right-hand side (i.e.~the variables
in the expression that is being evaluated) must be referenced by a trec
object (or possibly a lag or lead). When referencing a tseries object by
a trec, you can use either curly braces, \texttt{\{...\}}, or round
brackets, \texttt{(...)}; there is no difference between them in
time-recursive expressions.

$\attention$ See the description of circumstances under which IRIS fails
to evaluate time-recursive expressions correctly, and how to avoid/fix
such situations.

\paragraph{Example}\label{example}

Construct an autoregressive sequence starting from 10 with a
autoregressive coefficient 0.8:

\begin{verbatim}
 T = trec(qq(2010,1):qq(2020,4));
 X = tseries(qq(2009,4),10);
 X{T} = 0.8*X{T-1};
\end{verbatim}

\paragraph{Example}\label{example-1}

Construct a first-order autoregressive process, \texttt{x}, with
normally distributed innovations, \texttt{e}:

\begin{verbatim}
T = trec(qq(2010,1):qq(2020,4));
x = tseries(qq(2009,4),10);
e = tseries(qq(2010,1):qq(2020,4),@randn);
x{T} = 10 + 0.8*x{T-1} + e{T};
\end{verbatim}

\paragraph{Example}\label{example-2}

Construct a second-order log-autoregressive process going backward.

\begin{verbatim}
T = trec(yy(2020):-1:yy(2000));
B = tseries();
B(yy(2022)) = 1.56;
B(yy(2021)) = 1.32;
B{T} = B{T+1}^1.2 * B{T+2}^0.6;
\end{verbatim}

\paragraph{Example}\label{example-3}

Construct Fibonacci sequence:

\begin{verbatim}
 T = trec(3:20);
 F = tseries(1:2,1);
 F{T} = F{T-1} + F{T-2};
\end{verbatim}

\paragraph{When IRIS Fails to Evaluate Time-Recursive Expressions
Correctly}\label{when-iris-fails-to-evaluate-time-recursive-expressions-correctly}

IRIS fails to evaluate time-recursive expressions correctly (without any
indication of an error) when the following two circumstances occur at
the same time:

\begin{itemize}
\item
  The time series used in the expression are within a database (struct),
  or a cell array;
\item
  At least one tseries object on the right-hand side has been created by
  copying the left-hand side tseries object with no further
  manipulation.
\end{itemize}

Under these circumstances, the right-hand side tseries variable will be
assigned (updated with) the results calculated in iteration as if it
were the tseries variable on the left-hand side.

\paragraph{Example}\label{example-4}

Create a database with two tseries. Create one of the tseries by simply
copying the other (i.e.~plain assignment with no further manipulation).

\begin{verbatim}
d = struct();
d.x = tseries(1:10,1);
d.y = d.x;

T = trec(2:10);
d.x{T} = 0.8*d.y{T-1}; % Fails to evaluate correctly!
\end{verbatim}

The above time-recursive expression will be incorrectly evaluated as if
it were \texttt{d.x\{T\} = 0.8*d.x\{T-1\}}.

Note that when the tseries objects are not stored within a database
(struct), the expression will evaluate correctly:

\begin{verbatim}
x = tseries(1:10,1);
y = x;

T = trec(2:10);
x{T} = 0.8*y{T-1}; % Evaluates correctly.
\end{verbatim}

\paragraph{Workaround}\label{workaround}

To evaluate the expression correctly, simply apply any kind of operator
or function to the tseries \texttt{d.y} before it enters the
time-recursive expression. Below are examples of some simple
manipulations that do the job without changing the tseries \texttt{d.y}:

\begin{verbatim}
d = struct();
d.x = tseries(1:10,1);
d.y = 1*d.x;
\end{verbatim}

or

\begin{verbatim}
d = struct();
d.x = tseries(1:10,1);
d.y = d.x{:};
\end{verbatim}

or

\begin{verbatim}
d = struct();
d.x = tseries(1:10,1);
d.y = d.x;
d.y = d.y + 0;
\end{verbatim}


\section{Partially Observable Monte Carlo Planning (POMCP)}
\frame{\tableofcontents[currentsection, hideothersubsections]}

\begin{frame}
\frametitle{POMCP: Key ideas}
POMCP extends \textbf{MCTS} to \textbf{partially observable} environments
\pause
\begin{itemize}
    \item sample start states from the belief state \\
    (break the curse of dimensionality) \pause
    \item sample histories using a black box simulator \\
    (break the curse of history) \pause
    \item evaluate each state in a search tree by the \textbf{average outcome} of simulations from that state
\end{itemize}

\end{frame}

\begin{frame}
\frametitle{POMCP: $|S|= 50$, $|A|=2$, $|Z|=2$, {\small no intermediate rewards}}
\begin{columns}
  \column{0.5\textwidth}
    \begin{figure}
        \centering
        \includegraphics[scale=0.3,trim={0 0 30cm 0},clip]{pomcp_illustration}
    \end{figure}
  \column{0.5\textwidth}
    \begin{itemize}
        \item construct a search tree from multiple simulations, and
        \item evaluate each history by its mean return
    \end{itemize}
\end{columns}
\end{frame}

\begin{frame}
\frametitle{POMCP: $|S|= 50$, $|A|=2$, $|Z|=2$, {\small no intermediate rewards}}
\begin{columns}
  \column{0.5\textwidth}
    \begin{figure}
        \centering
        \includegraphics[scale=0.3,trim={20cm 0 15cm 0},clip]{pomcp_illustration}
    \end{figure}
  \column{0.5\textwidth}
    \begin{itemize}
        \item select a real action $a$, and
        \item observe a real observation $o$
    \end{itemize}
\end{columns}
\end{frame}

\begin{frame}
\frametitle{POMCP: $|S|= 50$, $|A|=2$, $|Z|=2$, {\small no intermediate rewards}}
\begin{columns}
  \column{0.5\textwidth}
    \begin{figure}
        \centering
        \includegraphics[scale=0.3,trim={35cm 0 0 0},clip]{pomcp_illustration}
    \end{figure}
  \column{0.5\textwidth}
    \begin{itemize}
        \item prune the tree and
        \item begin a new search from the updated history $hao$
    \end{itemize}
\end{columns}
\end{frame}

\begin{frame}
\frametitle{POMCP: algorithm}
\begin{figure}
    \centering
    \includegraphics[scale=0.4]{pomcp_algo}
\end{figure}
\end{frame}