\chapter{$\lambda$-Calculus}
\thispagestyle{empty}
In this chapter we define the $\lambda$-calculus as a formal language of computation, and on this we shall define the concept of typing, ubiquitous in programming. The main references  for the general approach of this chapter are \cite{selinger2008lecture} and \cite{hindley2008lambda}. For the historical motivation for the development of the theory  we refer to \cite{cardone2006history}.  \\

In the 19th century, mathematicians developed the concept of function. In this time it was question whether two functions were the same. The most widespread view is the \emph{extensional} approach. This approach consider two functions to be the same whenever for the same input they have the same outputs. A function $f$ as a pairing of an X domain to an Y codomain. In other words, a function $f$ is seen as a pairing $f\subset X \times Y$.\\

For example, let $p$ be a prime big enough, and let $f$ and $g$ be two endomorphisms of  $\mathbb{Z}/p\mathbb{Z}$. One could argue that the endomorphism of  $f(x)\to x^2$ is different to $g(x) \to \log_a a^{x+2}$. Despite having the same output for the same input, they are clearly different in \emph{complexity}, with complexity understood as the cost of computing a function. It even involve the resolution of a discrete logarithm, that is a highly non-trivial task. \\

This view, in which not only is the result of the function important but how is that result obtained, is called the \emph{intensional} approach. In early 1930 mathematicians such as Church\cite{church1932set}, Gödel\cite{adams2011early} or Turing\cite{turing1938computable} started formalizing what is to be computable. Nowadays, after the wake of computation, the intensional approach have come to be as relevant as the extenisional approach.

% (to the point on which correctness is often partially dropped to easy time complexity, for example \cite{hofmeister2002probabilistic}). \\

% In $\lambda$-calculus we consider function as formulas, and thus, we consider an intensional approach. This intensional approach was also backed by the constructivist, to whom we will later on relate with the study of the Curry-Howard isomorphism  .\\

\section{Untyped $\lambda$-Calculus } \label{lambda-calc-untyped-section}
We will start defining the most simple version of $\lambda$-calculus: untyped $\lambda$-calculus. Let us define some concepts:

\begin{enumerate}
\item An alphabet $A$ is an arbitrary, maybe infinite, non-empty set.
\item A symbol $a$ is an element of the alphabet.
\item A word is an ordered finite sequence of symbols.
\item The collection of all possible finite words over an alphabet $A$ is denoted by $A^*$.
\item A language $L$ over $A$  is a subset of $A^*.$
\end{enumerate}

There are a lot of languages. For example, Spanish is a language, with a well-known alphabet $L$, with a proper subset of words over $L^*$. In the same fashion, we define $\lambda$-calculus as a formal language, defining its syntax, that is, what words are valid.

\subsubsection{Syntax of untyped $\lambda$-calculus}
We start with the basic building blocks, which collectively form what is
called the alphabet:

\begin{itemize}
\item We use $x, y, z,...$ to denote variables. As more variables are necessary sub-indexes will be used, up to countable many variables.
\item We consider an abstract connector $\lambda$.
\item Auxiliary, we consider the characters $".", "("$ and $")"$.
\end{itemize}
Now, we are ready to formally define the untyped $\lambda$-calculus:

\begin{definition}[Syntax of Untyped $\lambda$-calculus (section 2.1 \cite{selinger2008lecture})]\label{def:untyped-lambda-calc}
  A $\lambda$-calculus term (sometimes called a formula) is defined inductively:
  \begin{itemize}
  \item Every variable $x,y,z...$ is a valid formula.
  \item If $A,B$ is a formula, then $AB$ is a valid formula.
  \item If $A$ is a formula and $x$ is a variable, then $\lambda x.M$ is a valid formula.
  \end{itemize}
  The set of all variables is denoted by $\mathcal{V}$ and the set of all $\lambda$-formulas is denoted by $\Lambda$.
\end{definition}
\begin{remark}
  $\Lambda$ is countable.
\end{remark}

When dealing with formal languages we make use of similar inductive statements more often than not. we find it useful to introduce the Backus-Naur Form notation \cite{knuth1964backus}, BNF for short.  For this purpose, we shall find it useful BNF specification is a set of derivation rules, written as

$$\operatorname{word1}, \operatorname{word2} ... ::= \operatorname{expression1} | \operatorname{expression2} |...,$$

where each $\operatorname{word}$ is a generic valid word from the language, and each  expression consists of a derived valid formulas. Expressions are separated by the vertical bar: $|$. For example, we can revisit definition \ref{def:untyped-lambda-calc} as follow:

\begin{definition}
  The formulas of  $\lambda$-calculus are built via the BNF:
  $$A,B ::= x\ |\ (AB)\ |\ (\lambda x.A) ,$$
  where $x$ denote any variable in $\mathcal{V}$.
\end{definition}
\begin{remark}
  Note that, in this case, there is no need to specify the alphabet aside from the set $\mathcal{V}$.
\end{remark}

From this point on we know how the $\lambda$-terms are constructed. This point is better understood with some examples, using natural numbers.  We recommend to trust us in their existence, and use a naive intuition of what a natural is when  reading this examples. The notion of natural numbers is formalized in definition \ref{def:untyped-natural} and revisited in section \ref{section:natural-revisited}.\\

\subsection{Reductions in untyped $\lambda$-calculus}
Let us now explain the idea behind the formalism. Consider the expression $\lambda x.(x+1)$. This expression represents the idea of the function $f(x)=x+1$ that take a variable $x$ and return $x+1$. The formula $\lambda x.M$ is called the \emph{abstraction} of $M$.\\

From the notion of abstraction naturally arises the second one: \emph{application}. Consider terms $M  =\lambda x. x+1$ and $N = 3$. Then $MN = (\lambda x. x+1)(3)$ represents the application of $M$ to $N$. In untyped $\lambda$-calculus, $N$ can be any term. Thus for example, the term $\lambda f.\lambda g. fg$ just represent the composition of terms.

\begin{example} The term $$\lambda g.(\lambda f.(\lambda x. (gf)(x) )) (x+1)(x+2),$$
  can be understood as $(g\circ f) (x)$ where $g(x) = x+1$ and $f(x)=x+2$.
\end{example}

In a expression $M = (\lambda x. N)$ we say that the variable $x$ is \emph{bound} in $M$.

The idea of $\alpha$-equivalence, $=_{\alpha}$, is that expressions such as $\lambda x.x$ and $\lambda y.y$ are essentially the same. That is, we consider terms up to \emph{renaming} of variables. To formalise this, we have to formalize the concept of free and bound variables, and the concept of renaming.

\begin{definition}
  We have a \emph{free variable} function $FV:\Lambda \to \mathcal{P}(\mathcal{V})$ defined recursively as:
  \begin{itemize}
  \item $FV(x) = \{x\}$ for every $x\in \mathcal{V}$.
  \item $FV(MN) = FV(M)\cup FV(N)$ for every $M,N\in \Lambda$.
  \item $FV(\lambda x.M) = FV(M)\backslash \{x\}$ for every $M\in \Lambda, x\in \mathcal{V}$.
  \end{itemize}
  Given a term $M$, if $x\in FV(M)$ we say that $x$ is a free variable in $M$ or that $M$ has a free variable $x$.
\end{definition}

\begin{definition}
  We say that a term is \emph{closed} if it has no free variables.
\end{definition}
We can now define process of \emph{substitution}, and define a rename as a particular case. This process is the one behind the intuitive idea of evaluation. For example when we consider $(\lambda y. y^2+y)(4) = 4^2+4 = 20$ we are replacing the value of $x$ by the term $4$.
\begin{definition}\label{def:substition}
  The substitution of $N$ for free occurrences of $x$ in $M$, denoted by $M[N/x]$ is defined recurrently in the structure of $\lambda$-terms by:
  \begin{align*}
    x[N/x]& \equiv N,\\
    y[N/x]& \equiv y, &  \text{if } x\ne y&,\\
    (MP)[N/x]& \equiv (M[N/x])(P[N/x]),\\
    (\lambda x.M)[N/x] & \equiv \lambda x.M,\\
    (\lambda y.M)[N/x] & \equiv \lambda y.(M[N/x]), & \text{if } x\ne y \text{ and } y \not \in FV(N)&,\\
    (\lambda y.M)[N/x] & \equiv \lambda y'.((M[y'/y])[N/x]), & \text{if } x\ne y \text{ and } y\in FV(N),\\
          & & \text{ with } y' \not \in FV(N) \cup \{x\}&.
  \end{align*}
  When $N = y$ a variable we say that $[N/x] = [y/x]$ is a rename.
\end{definition}
\begin{remark}
  In equality $  (\lambda x.M)[N/x]  \equiv \lambda x.M,$ we remark that, although being both the variable $x$, no substitution is carried out as $x$ is bounded. This is called, \emph{non-capturing substitution}.
\end{remark}

\begin{definition}
  We define the $\alpha$-equivalence $=_\alpha$ as the smallest congruence relation on $\Lambda$ such that:
  $$\lambda x. M =_\alpha \lambda y. M[x/y], \qquad \forall y \in \mathcal{V} \backslash FV(M).$$
\end{definition}
In other words is the congruence that consider every renaming of a bounded variable as equals. This type of property-oriented definition is commonplace in $\lambda$-calculus, as it allows for some synthetic and goal-oriented definition.

Using this same tool we are going to define the $\beta$-reduction. This is not going to be an equivalence implication, but rather a relationship. It abstracts the notion of  $(\lambda x. 2+x)(2)$  being computed as $4$. Formally:

\begin{definition}[Section 2.5 \cite{selinger2008lecture}]
  We define the \emph{single-steps} $\beta$\emph{-reduction} as the smallest relationship $\to_\beta$ such that:%\footnote{Should i explain the notation ${A \over B}$}
  \begin{align*}
    &(\beta) \qquad\ \ \  \ {\ \over(\lambda x.M)N \to_\beta M[N/x]},\\
    &(\operatorname{cong}_1)\qquad\qquad{ M \to_\beta M' \over MN \to_\beta M'N },\\
    &(\operatorname{cong}_2)\qquad\qquad{ N \to_\beta N' \over MN \to_\beta MN' },\\
    &(\zeta) \qquad\qquad \ \  \ {M\to_\beta M' \over(\lambda x.M) \to_\beta \lambda x.M'}.\\
  \end{align*}
\end{definition}

\begin{remark}
  In this definition we can see that rule $(\beta)$ is the main objective, while the others are necessary so that it maintain the structure.
\end{remark}
\begin{definition}
  We define the \emph{multiple-step} $\beta$\emph{-reduction} $\twoheadrightarrow_\beta$ as the reflexive, transitive closure of $\to_\beta$.
\end{definition}
\begin{definition}
  We define the $\beta$-equivalence $=_\beta$ as the symmetric closure of $\twoheadrightarrow_\beta$.
\end{definition}

Up to this point the focus of the system was to define an intensional language for computation. $\beta$-reduction encapsulates the concept of computation, where we have evaluation.

% \begin{example}
%   \begin{itemize}
%   \item We can see that $(\lambda x. \lambda y. (x+ y))(6)(7)$ is reduced  to $(6+y)(7)$ and then to 13.
%   \end{itemize}
% \end{example}

Should we want to consider a way of having an extensional approach, for example, to generate normal forms for the terms, we would need more machinery. The $\eta$-equivalence provides us with the tools to consider $\lambda$-calculus in a extensional way. \\

\begin{definition}
  We define the single-step $\eta$-reduction $\to_\eta$ as the smallest relationship such that:
  \begin{align*}
    &(\eta) \qquad\qquad \ \ \ \  \ {\ \over(\lambda x.Mx) \to_\eta M} \qquad \forall x \not  \in FV(M),\\
    &(\operatorname{cong}_1)\qquad\qquad{ M \to_\eta M' \over MN \to_\eta M'N },\\
    &(\operatorname{cong}_2)\qquad\qquad{ N \to_\eta N' \over MN \to_\eta MN' },\\
    &(\zeta) \qquad\qquad \ \  \ {M\to_\eta M' \over(\lambda x.M) \to_\eta \lambda x.M'}.\\
  \end{align*}
  Similarly, we define the multiple-step $\eta$-reduction $\twoheadrightarrow_\eta$ as the transitive reflexive closure of $\to_\eta$, and the $\eta$-equivalence $=_\eta$ as the symmetric closure of $\twoheadrightarrow_\eta$.
\end{definition}
\begin{definition}
  We define the single-step $\beta\eta$-reduction $\to_{\beta\eta}$ as the union of $\to_\beta$ and $\to_\eta$.  We define the multiple-step $\beta\eta$-reduction $\twoheadrightarrow_{\beta\eta}$ as the transitive reflexive closure of $\to_{\beta\eta}$ and $=_{\beta\eta}$ ad the symmetric closure of $\twoheadrightarrow_{\beta\eta}$.
\end{definition}

% \begin{proposition}
%   In the presence of all other axioms defined in $\beta$ and $\eta$ equivalences, $\eta$ rule is equivalent to:\footnote{TODO: Revisar si es equivalencia o reducción.}
%   $$(ext) \qquad\ \ \  \ {\ (Mx=M'x)\text{ where } x\not\in FV(M)\cup FV(M') \over M \to_\eta M' \land M' \to_\eta M} $$
% \end{proposition}
% \begin{proof}
%   By extenisionality we have that $M' = $.
% \end{proof}

% \begin{remark}
%   This proof that the $\eta$-reduction maintain extensionality in some sense. Also, note that we need to consider $\eta$-equivalence and not only $\eta$-reduction.
% \end{remark}

\begin{table}[h]
  \begin{center}
    \begin{tabular}{|l|l|l|}
      \hline
      Name & Main rule & Equivalence \\
      \hline
      $\alpha$ & $ (\lambda x. M) =_\alpha \lambda y. M[x/y]$& Yes\\
      $\beta$ & $(\lambda x.M)N \to_\beta M[N/x]$& No\\
      $\eta$ & ${\displaystyle (\lambda x.Mx) \to_\eta M }$& No\\
      \hline
    \end{tabular}
  \end{center}
  \caption{\label{tab:reductions}Considered reductions in $\lambda$-calculus.}
\end{table}

\subsection{Church-Rosser Theorem}

This subsection we present an important result for $\lambda$-calculus: the \emph{Church-Rosser} theorem. The idea behind this theorem is to prove that every reduction (either $\beta$, $\eta$ or a mix) provide an unified sense of reduction. First, we present some definitions.

\begin{definition}
  Consider a relation $\to$ and let $\twoheadrightarrow$ be its reflexive transitive closure. We can define three relations:

  \begin{enumerate}
  \item The Church-Rosser Property: $$M\twoheadrightarrow N, M\twoheadrightarrow P \implies \exists Z : N\twoheadrightarrow Z, P\twoheadrightarrow Z.$$
  \item The Quasidiamond Property: $$M\to N, M\to P \implies \exists Z : N\twoheadrightarrow Z, P\twoheadrightarrow Z.$$
  \item The Diamond Property : $$M\to N, M\to P \implies \exists Z : N\to Z,P\to Z$$
  \end{enumerate}
\end{definition}

\begin{remark}
  Note that, while $3)\implies 1)$, it is not necessary that $2)\implies 1)$.
\end{remark}

With this notation, we introduce the Church-Rosser Theorem, proved as done by Martin-Löf.

\begin{theorem}[Church-Rosser]The reductions in untyped $\lambda$-calculus satisfy the following properties:\label{theo:church-rosser}
  \begin{enumerate}
  \item $\twoheadrightarrow_{\beta\eta}$ satisfy the Church-Rosser property.
  \item $\twoheadrightarrow_{\beta}$ satisfy the Church-Rosser property.
  \end{enumerate}
\end{theorem}

We prove the theorem only for the $ \twoheadrightarrow_{\beta\eta}$ case. We refer to  \cite[Theorem 1.32]{hindley2008lambda} for a full proof for $\twoheadrightarrow_\beta$. he first step in the proof is going to be an alternative definition of $\toheadrightarrow_{\beta\eta}$ as a completion of the new reduction $\triangleright$.
\begin{definition}[parallel one-step reduction]
  We define the parallel one-step reduction  $\triangleright$ as the smallest relationship such that,\\
  \begin{align*}
    (1)&& {\displaystyle \over x \triangleright x}\qquad\qquad\qquad\qquad\qquad\\
    (2)&& { M \triangleright M'\qquad N\triangleright N'  \over MN \triangleright M'N' }\qquad\qquad\qquad\ \  \ \\
    (3)&&{M \triangleright M' \over(\lambda x.M) \triangleright \lambda x.M'}\qquad\qquad\qquad\qquad\\
    (4)&& {M \triangleright M'\qquad N \triangleright N' \over (\lambda x.M)N \triangleright M'[N'/x]}\qquad\qquad\qquad\\
    (5)&& {\ M\triangleright M'\over(\lambda x.Mx) \triangleright M'} \qquad \forall x \not  \in FV(M)\qquad
  \end{align*}

  where $x$ is any variable and $M,N,M',N'$ any term.
\end{definition}

\begin{lemma}[Characterization of $\triangleright$]\label{lemma:cr1}
  Let $M,M'$ be terms, then:
  \begin{enumerate}
  \item $M\to_{\beta\eta} M'\implies M \triangleright M'$
  \item $M\triangleright M'\implies M \twoheadrightarrow_{\beta\eta} M'$
  \end{enumerate}
\end{lemma}
\begin{proof}
  \begin{enumerate}
  \item  We  apply induction on the structure of $\to_{\beta\eta}$. That is, we know that $\to_{\beta\eta}$ have to be constructed following a series of rules. We are going to use induction on the last rule used. Therefore we conclude  every rule use for $\M \to_{\beta\eta} M'$ implies $M \triangleright M'$. At every point, $x$ denote some variable, and $M,N,M',N',P,Q,Q'$ denote some term.

    \begin{itemize}
    \item[($\beta$)] In this case, then $M=(\lambda x.Q)M$ and $M' = Q[N/x]$. Then using (4) $M \triangleright M'$.
    \item[($\eta$)] In this case, then $M=(\lambda x.Qx)$ and $M' = Q$. Then using (5) $M \triangleright M'$.
    \item[($\operatorname{cong}_1$)] In this case, then $M=PQ$ and $M' = PQ'$ for some $Q\to_{\beta\eta} Q'$. Using induction we get $Q\triangleright Q'$ and using (5) $M \triangleright M'$.
    \item[($\operatorname{cong}_2$)]  Analogous.
    \item[($\zeta$)] In this case, then $M=\lambda x.Q$ and $M' = \lambda x.Q'$ for some $Q\to_{\beta\eta} Q'$. Using induction we get $Q\triangleright Q'$ and using (3) $M \triangleright M'$.
    \end{itemize}

  \item Similarly, every possible in which $M\triangleright M'$ rule derive in  $M \twoheadrightarrow_{\beta\eta} M'$:
    \begin{itemize}
    \item[(1)] By reflexivity of $ \twoheadrightarrow_{\beta\eta}$.
    \item[(2)] By $\operatorname{cong}_1,\operatorname{cong}_2$ in either definition of $\to_\beta$ and $\to_\eta$.
    \item[(3)] By $\zeta$ in either definition of $\to_\beta$ and $\to_\eta$.
    \item[(4)] Then we have $(\lambda x.M)N \triangleright M'[N'/x]$ with $M \triangleright M'\qquad N \triangleright N'$. By induction $M \twoheadrightarrow_{\beta\eta} M'$ and $N \twoheadrightarrow_{\beta\eta} N'$. By transitive closure:
      $$(\lambda x.M)N \twoheadrightarrow_{\beta\eta}(\lambda x.M')N \twoheadrightarrow_{\beta\eta} (\lambda x.M')N' \twoheadrightarrow_{\beta\eta} (M')[N'/x].$$
    \item[(5)] Then we have $(\lambda x.Mx) \triangleright M'$, for some $M\triangleright M'$ and for some $x \not  \in FV(M)$. Finish using $(\operatorname{cong}_2)$ and $\eta$.
    \end{itemize}

  \end{enumerate}

\end{proof}

\begin{lemma}[Substitution Lemma]If $M \triangleright M'$ and $U \triangleright U'$ , then $M [U/y] \triangleright M' [U' /y]$.\label{lemma:overconstrains}
\end{lemma}
\begin{proof}
  We have defined in definition \ref{def:substition} a capture-avoiding substitution, in the sense that we do not let bound variable to be substituted. Similarly to previous lemma, we proceed by induction, studying the last rule applied. The induction is proceed on the rules applied to $M\triangleright M'$.

  \begin{itemize}
  \item[(1)] In this case $M=M'=x$. If $x\ne y$ the substitution does not alter $M$ so we have finished. If $x = y$, the term is only $U\triangleright U'$, that we know by hypothesis.
  \item[(2)] In this case $M=NP, M'=P'N'$ for some terms $N,N',P,P'$ such that $N\triangleright N'$, $P\triangleright P'$. Proceed by induction on these implications and apply (2).
  \item[(3)] In this case $M=\lambda x.N$ and $M=\lambda x.N'$, for some $N\triangleright N'$. Apply induction on $N$ and end by (3).
  \item[(4)] Then we have $(\lambda x.M)N \triangleright M'[N'/x]$ with $M \triangleright M'$ and $N \triangleright N'$. By induction on both of the relationship, and applying (4).
  \item[(5)] Then we have $(\lambda x.Mx) \triangleright M'$, for some $M\triangleright M'$ and for some $x \not  \in FV(M)$. Finish using induction on $M\triangleright M'$ and by (5).
  \end{itemize}
\end{proof}

While the proof is rather straightforward, one can see that it does require induction, and thus the length of it.

\begin{definition}[Maximal parallel one-step reduction]
  Given a term $M$, the \emph{maximal parallel one-step reduction} $M^*$ is defined inductively:
  \begin{itemize}
  \item $(x)^*  =x$
  \item $(PN)^*=P^*N^*$ but for $\beta$-reductions.
  \item $((\lambda x.P)N)^* = Q^*[N/x] $
  \item $(\lambda x.N)^*=\lambda x.N^*$ but for $\eta$-reductions,
  \item $(\lambda x.Nx)^*=N^*$
  \end{itemize}
  where $x$ is any variable, and $N,P$ is any term.
\end{definition}

\begin{lemma}[Maximal Parallel one-step]\label{lemma:cr2}
  If $M\triangleright M'$ then $M'\triangleright M^*$.
\end{lemma}
\begin{proof}
  We proceed by induction on $M$ again:
  \begin{itemize}

  \item[(1)] In this case $M=M'=x$. Then $M^*=x$.
  \item[(2)] In this case $M=NP, M'=P'N'$ for some terms $N,N',P,P'$ such that $N\triangleright N'$, $P\triangleright P'$.
    \begin{itemize}
    \item If $NP$ is not a $\beta$-reduction, then $M^* = P^*N^*$ and we can use the  induction hypothesis on $N^*$ and $P^*$ and (2).
    \item If $NP$ is a $\beta$-reduction, then $N = \lambda x. Q$, thus $M^* = Q^*[N^*/x]$. $P = \lambda x.Q$ and $P \triangleright P'$ could be derived using congruence to $\lambda$-abstraction (2) or extensionality (5). In the first case use induction on $Q$ and (4). On the second case $Q=Rx$, use induction on $R$, and apply substitution lemma.
    \end{itemize}

    Proceed by induction on these implications and apply (2).
  \item[(3)] In this case $M=\lambda x.N$ and $M=\lambda x.N'$, for some $N\triangleright N'$. If $M$ is not an $\eta$-reduction, then use induction hypothesis and finish with (3). Otherwise we have that $N = Py$, and distinguish two cases on the last rule applied to $N$:
    \begin{itemize}
    \item[(2)\to (3)]\footnote{Notation $(2)\to (3)$ means that the last to step in the reduction has been $(2)$ and $(3)$.} Then apply induction hypothesis to $P$ and end using (5).
    \item[(4)\to (3)] We have that $N = P = \lambda y.Q \triangleright N' = Q'[x/y]$. Apply induction hypothesis using that $M ' = \lambda x.N' = \lambda x.Q'[x/y] = \lambda y. Q'$.
    \end{itemize}
  \item[(4)] Then we have $(\lambda x.P)N \triangleright P'[N'/x]$ with $P \triangleright P'$ and $P \triangleright P'$. The result follows by the substitution lemma \ref{lemma:overconstrains}.
  \item[(5)] Then we have $(\lambda x.Px) \triangleright P'$, for some $P\triangleright P'$ and for some $x \not  \in FV(P)$. Finish using induction on
  \end{itemize}
\end{proof}

\begin{proof}[Proof of Theorem \ref{theo:church-rosser}]   If a relation $\to$ satisfy the diamond property, its reflexive and transitive closure $\twoheadrightarrow$ satisfy the Church-Rosser property. As a consequence of lemma \ref{lemma:cr2}, we know that $\triangleright$ satisfy the diamond property.  By lemma \ref{lemma:cr1} we know that the reflexive and transitive closure of $\triangleright$ is $\twoheadrightarrow_{\beta\eta}$.
\end{proof}

Now let take a moment to comment on the usefulness of each reduction. We can say that $\beta$ reduction is the soul of computation  while $\eta$ is useful to cleanup the results.  This can be used to define \emph{normal} forms for untyped $\lambda$-terms. More infomation on normalization can be found on \cite[Section 7]{selinger2008lecture} and \cite{baader1999term}. \\

\subsection{Fixed points and Programming}
This section is an interlude. We will stop analyzing for a while the theoretical properties of lambda calculus. Instead, in this section we will ponder upon how lambda calculus actually captures the notion of programming. In particular, we explain how to define the basic rudiments for programming in untyped $\lambda$-calculus. In particular, we explain how to define, for example, a booleans, an integer type and a recursion operator, thus showing the computational potential of recursively enumerable languages. Let us start with the booleans.

\begin{definition}[Boolean in untyped $\lambda$-calculus] \label{def:untyped-natural}
  We define the two values $T$ and $F$, usually referred to as true and false, as:
  $$\operatorname{True} = \lambda xy.z,\qquad    \operatorname{False} = \lambda xy.z.$$

  We also define the operations:
  \begin{align*}
    \operatorname{Not} &= \lambda a.a\operatorname{False}\operatorname{True},\\
    \operatorname{And} &= \lambda ab.ab\operatorname{False},\\
    \operatorname{Or} &= \lambda ab.a\operatorname{True}b.
  \end{align*}
\end{definition}

We can check that this construction left us with the basis of a boolean logic, after checking the truth values of the different operations provided. This definition of truth value is really convenient, as it allow us to easily implement control flow of programs.

\begin{definition}
  We define the $\operatorname{If} = \lambda x.x$.
\end{definition}

We can see that in this case, $\operatorname{If} \operatorname{True} M N = M$ and $\operatorname{If} \operatorname{False} M N = N$. This construction is very natural and widely used in computer programs.\\

Next, we will define Naturals number.

\begin{definition}[Natural numbers in untyped $\lambda$-calculus] \label{def:untyped-natural}
  Let $f,x$ be fixed $\lambda$-terms, and write $f^nx = f(f(f(...(f(x))...)))$. Then, for each $n \in \mathbb N$, we define the \emph{nth Church numeral} as $\overline n=\lambda f x.f^nx$.
\end{definition}
\begin{remark}
  We abuse the notation with $\lambda f x.f^nx$, which mean $\lambda f.\lambda x.f^nx$
\end{remark}

Finally, we consider the notion of recursive function. This is done with a elegant artefact, based on an idea of fixed points. Let's begin:

\begin{theorem}[Fixed points]
  For every term $F$ in untyped $\lambda$-calculus, there is a fixed point.
\end{theorem}
\begin{proof}
  Let $A=\lamdba xy.y(xxy)$ and $\Theta =AA$. Then, for every $\lambda$-term $F$ we have that $N=\theta F = FN$, thus being a fixed point. In fact:
  \begin{align*}
    N = \Theta F = AAF = (\lambda xy.y(xxy))AF \twoheadrightarrow_\beta F(AAF) = F(\Theta F) = FN.
  \end{align*}
\end{proof}
The proof of this theorem led to a new definition:

\begin{definition}
  The term $\Theta$ used in the previous theorem is called \emph{Turing fixed point combinator}.
\end{definition}

This existence of fixed point theorem is really useful, as we can now define \emph{recursion}. The idea is to define a function as a term with itself as a parameters and proceed to evaluate a fixed point. Let us first present an example.

\begin{definition}
  We define the terms:
  \begin{align*}
    \operatorname{add} &= \lambda nm f x.nf (mf x),\\
    \operatorname{mult} &= \lambda nm f.n(mf),\\
    \operatorname{iszero} &= \lambda nxy.n(\lambda z.y)x,\\
    \operatorname{predecesor} &=\lambda n.\lambda f.\lambda x. n (\lambda g.\lambda h. h (g f)) (\lambda u.x) (\lambda u.u).
  \end{align*}

\end{definition}
Suppose that we want to define the factorial. We want that:
$$\operatorname{fact} n = \operatorname{If}(\operatorname{iszero} n)(1)(\operatorname{mult}(n)(\operatorname{fact} (\operatorname{pred}(n))),$$
in order to do that, look for a fixed point of:
$$\lambda f. \lambda n.\operatorname{If}(\operatorname{iszero}n)(1)(\operatorname{mult}(n)(f (\operatorname{pred}(n))),$$
and thus $\operatorname{fact}=\Theta \lambda f.\lambda n. \operatorname{If}(\operatorname{iszero}n)(1)(\operatorname{mult}(n)(f (\operatorname{pred}(n)))$. In general:

\begin{definition}
  Given an stop condition $g$, an stop value $s$ and a recursive step $f$, we define the recursive term $F$ that computes $(g,s,f)$ as
  $$F = \Theta \lambda f. \operatorname{If}(gn)(n)(f (\operatorname{pred}(n))$$
\end{definition}

\begin{remark}
Another implementation of the fixed point operator is \emph{Curry's paradoxical fixed point operator}.  We define Curry's paradoxical fixed point operator $\operatorname{Y}$ as:
  $$\operatorname{Y}=\lambda f.(\lambda x.f(x x)) (\lambda x.f(x x)).$$
This operator is used for the Curry's paradox \cite{sep-curry-paradox}.\\
\end{remark}

% \subsection{Rosser-Kleene and Curry's Paradoxes}

% In this subsection we explain the deficits in untyped $\lambda$-calculus, that was the original Church construction, that eventually leads into the definition of \emph{typed} $\lambda$-\emph{calculus}.\\

% We can start by having a quick reasoning, as shown in \cite{sep-lambda-calculus}.

% {TODO This Summer: expand this subsection properly.}

% An aproximation of this argument can be done if we consider domains of functions to be sets. In particular untyped $\lambda$-calculus allow functions to be applied to themselves. This is going to be troublesome, considering domains as sets, as this will be a clear infringement of \emph{ZF Axiom Theory}\cite{kunen2014set}. Namely, the infinite descending sequence
% $$f\ni \{f,f(f)\}\ni f \ni \{f,f(f)\}...$$
% in contradiction with the \emph{regularity axiom}. An approach that seek solving this problems is presented in the next section, starring the introduction of types.  \\

% This last idea, despite being clear and loud to the intuition, is not a proof of inconsistency as there is no need for domains to be sets. Nonetheless, there was another problem that is to be deal that was noticed before. $\lamdba$-calculus was proposed to be a deductive system. The  \emph{Kleene-Rosser paradox}, first exhibited in \cite{kleene1935inconsistency} proved that simply typed $\lambda$-calculus is inconsistent. This paradox was perfected in 1958 by Curry \cite{curry1958combinatory} with the so called \emph{Curry's paradox}. \\

% To solve these problems, a very natural idea is included: the use of types in our computation system. This idea is very natural nowadays, as nearly everyone learns to program priors to a deep academic career. However, we must not forget that the origin was not to use types because we liked them initially, but because they really allow us to avoid logical problems.

\section{Typed $\lambda$-Calculus}
\emph{Typed $\lambda$-calculus} is a refinement of the untyped $\lambda$-calculus, on which the concept of typing is introduced. We are going to present three aproximation to simply typing: \emph{minimal simply typed }$\lambda$\emph{-calculus}, \emph{basic simply typed }$\lambda$\emph{-calculus} and \emph{extended simply typed }$\lambda$\emph{-calculus}.

\subsection{Definition}

We synthesize the definitions of typing presented in \cite{lambek1988introduction} or \cite{selinger2008lecture}. We structure this definition in three steps: types, terms  and equations.

\begin{definition}
  The types of basic simply typed $\lambda$-calculus are built via the BNF:
  $$A,B ::= \iota |\ A\to B\ |\ A \times B  \ |\ 1.$$
  where $\iota$ denote a basic type.
\end{definition}
\begin{remark}
  We usually refer to the basic simply typed $\lambda$-calculus just as simply typed $\lambda$-calculus.
\end{remark}
\begin{definition}
  We say that a term is closed if it has no free variables.
\end{definition}
\begin{remark}
Let us have a word on the intuitive notion of $A\to B$. As it happens in sets, where we have that we can consider the sets of functions between two set, here we can consider the type of function between two types. This types are called \emph{function types}.\\
\end{remark}

% Lets start laying some bricks useful for the intuition. $\iota$ type is foundational type. We consider also the type $1$. Others author, such as \cite{lambek1988introduction} prefer to include a type for every natural. Nonetheless, we consider that we can repeat the construction realized in untyped lambda-calculus to consider this typing. \\

\begin{definition}
  The \emph{raw terms} of basic simply typed $\lambda$-calculus are built via the BNF:
  $$A,B ::= x\ |\ AB\ |\ \lambda x^t.A \ |\ \langle A,B \rangle\ |\ \pi_1A\ |\ \pi_2A\ |\ *.$$
  where $x$ denote any variables, and $t$ denote any type.
\end{definition}

\begin{remark}
  We avoid the meticulous redefinition of free and bounded variable and use the one that naturally translates from untyped $\lambda$-calculus as explained in Section \ref{lambda-calc-untyped-section}
\end{remark}

The main difference with untyped calculus is that every bound variable has a type. That is, for the expression $(\lambda x^t.M)N$ to be coherent we require the term $N$ to be of type $t$. This led us to formulate some sort of condition to $N$ for being of type $t$.\\

Nonetheless raw terms have some other meaningless terms, such as projections of a non-pair $\pi_1(\lambda x^t.x)$. We solve all this problems at once with the \emph {Typing rules}. This rules will restrict the semantics of terms. We start by defining \emph{typing context}.

\begin{definition}
  We state by $M:t$  that a term $M$ is of type $t$. A typing context is a set of assumptions $\Gamma = \{x_1:A_1\}$, on which we assume each variable $x_i$ to be of type $A_i$.
\end{definition}

\begin{remark}
  From this definition, we can state the typing rules, from which we will be able to provide typing. We use the notation $\Gamma \vdash M:A$ meaning to state that the typing context $\Gamma$ derives that term $M$ is of type $A$.
\end{remark}

\begin{definition}[Typing Rules]\label{def:typing-rules}
  We define the following typing rules, for each typing context $\Gamma$.
  \begin{itemize}
  \item Every variable is of the type marked by the context, namely:
    $$  (var)\qquad  {\displaystyle \over \Gamma \vdash x_i:A_i},\qquad  i=  1,..,n.$$
    In addition,  $*$ is of type 1.
    $$  (*)\qquad  {\displaystyle \over \Gamma\vdash *:1}.$$

  \item From a term of type $A\to B$ we can derive a term of type $B$ from a term of type $A$:
    $$(app)\qquad  {\displaystyle \Gamma \vdash M:A\to B\qquad \Gamma \vdash N:A      \over \Gamma \vdash MN:B}.$$
    Conversely,
    $$(abs)\qquad  {\displaystyle \Gamma, x: A\vdash M: B  \over \Gamma \vdash \lambda x^A.m:A\to B}.$$
  \item The projections takes types pairing to each typing:
    $$(\pi_i) \qquad {\displaystyle \Gamma\vdash M: A_1\times A_2 \over \Gamma \vdash \pi_i M: A_i},\qquad i=1,2,$$
    and conversely:
    $$(pair) \qquad {\displaystyle \Gamma\vdash M: A\qquad \Gamma\vdash N: B \over \Gamma \vdash \langle M, N\rangle:  A\times B}.$$
  \end{itemize}
\end{definition}

\begin{remark}
  Not every term can be typed, namely the two previous examples of bad-behaviour: $\pi_1(\lambda x^t. x)$ and $(\lambda x^t.M)N$ where $N$ is not of type $t$. Thus, in simply typed $\lambda$-calculus, we only work with typed terms.
\end{remark}

\begin{definition}
  The terms of \emph{basic simply typed $\lambda$-calculus} is a raw typed $\lambda$-term that, together with a typing context of every variable, satisfies to be typed in the sense of definition \ref{def:typing-rules}.
\end{definition}
\begin{remark}
  Typing rules are often called \emph{term-forming operation}. As the only terms that we consider here are those suitable to be typed. In this sense,  we can see these typing rules as rules to construct terms inductively.
\end{remark}

By assigning a type we mean assigning types to the free and bound variables. Further on, when talking about terms we will refer to simply type $\lambda$-terms. That is, to fix a context, we can define substitution just like we defined it on typed $\lambda$-calculus.

\subsection{Reductions in simply typed $\lambda$-calculus}

To talk about Church-Rosser properties, we have to talk about reduction. Reductions in the typed lambda calculus are basically the same but for the additional structure added about we need to respect:

\begin{definition}[Section 2.5 \cite{selinger2008lecture}]
  We define the \emph{single-step} $\beta$\emph{-reduction} as the smallest relationship $\to_\beta$ such that:
  \begin{itemize}
  \item[]$(\beta)$: $ (\lambda x^t.M)N \to_\beta M[N/x]$.\\
  \item[]$(\beta_{\times,i})$: $\pi_i\langle M_1,M_2\rangle \to_\beta M_i$ for $i=1,2$.\\
  \item[]$(\operatorname{cong}_1)$: If $ M \to_\beta M'$ then $MN \to_\beta M'N$.\\
  \item[]$(\operatorname{cong}_2)$: If $ N \to_\beta N'$ then $ MN \to_\beta MN'$.\\
  \item[]$(\zeta)$: If $M\to_\beta M'$ then $(\lambda x^t.M) \to_\beta \lambda x^t.M'$.\\
  \end{itemize}
  We define the \emph{multiple-step} $\beta$\emph{-reduction} $\twoheadrightarrow_\beta$ as the reflexive, transitive closure of $\to_\beta$ and $\beta$-equality $=_\beta$ as the symmetric closure of $\twoheadrightarrow_\beta$.
\end{definition}

\begin{definition}\label{def:eta_x}
  We define the single-step $\eta$-reduction $\to_\eta$ as the smallest relationship such that:
  \begin{itemize}
  \item[]$(\eta)$: $(\lambda x^t.Mx) \to_\eta M}$,para todo $ x \not  \in FV(M)$.\\
\item[]$(\eta_1)$: $\langle\pi_1 M, \pi_2 M\rangle\to_\eta M$.\\
\item[]$(\eta_\times)$: If $M:1$ then $M \to_\eta *$ . \\
\item[]$(\operatorname{cong}_1)$: If $ M \to_\eta M'$ then $MN \to_\eta M'N$.\\
\item[]$(\operatorname{cong}_2)$: If $ N \to_\eta N'$ then $ MN \to_\eta MN'$.\\
\item[]$(\zeta)$: If $M\to_\eta M'$ then $(\lambda x.M) \to_\eta \lambda x.M'$.\\
\end{itemize}

Similarly, we define the multiple-step $\eta$-reduction $\twoheadrightarrow_\eta$ as the transitive reflexive closure of $\to_\eta$, and the $\eta$-equivalence as the symmetric closure of $\twoheadrightarrow_\eta$.
\end{definition}

\begin{remark} Note that we basically just add new rules so that $\beta$ and $\eta$ reductions match with the new syntax provided by the product.
\end{remark}
We can now talk about about Church-Rosser.  Let us check that it does not hold. For example, if $x: A\times 1$ then we can consider $M=\langle \pi_1x, \pi_2x\rangle$. Because of the rule $\eta_1$, we have that $M \to_{\eta}\langle \pi_1 x, *\rangle$, but taking $\eta_\times$ into account we can check that $M \to_{\eta} x$. \\

Although the Church-Rosser property is not satisfied for $\eta$-reduction, it is satisfied for $\beta$-reduction. This has an important significance. Recall that $\beta$-reduction is an abstraction of the idea of computation, while $\eta$-reduction is an abstraction of the idea of equivalence. So, in a simply typed system, we can perform computations all with the same end, because of the Curh-Rosser property of $\beta$-reduction. We cannot do the same with $\eta$-reduction, thus having no universal normal forms.

\subsection{Minimal and Expanded Typing}
\subsubsection{Minimal Typing}
The already explained $\lambda$-calculus can be reduced to a slimmer version. In fact, the only type truly necessary are the function types. We present it succinctly.

\begin{definition}
  The types of minimal simply typed $\lambda$-calculus are here built via the BNF:
  $$A,B ::= \iota |\ A\to B,$$
  where $\iota$ denote a basic type.
\end{definition}

\begin{definition}
  The raw term of minimal simply typed $\lambda$-calculus are built via the BNF:
  $$A,B ::= x\ |\ AB\ |\ \lambda x^t.A,$$
  where $x$ denote any variables, and $t$ denote any type.
\end{definition}

\begin{definition}[Typing Rules]\label{def:typing-rules}
  We define the following typing rules, for each typing context $\Gamma$.
  \begin{itemize}
  \item Every variable is of the type marked by the context, namely:
    $$  (var)\qquad  {\displaystyle \over \Gamma \vdash x_i:A_i},\qquad  i=  1,..,n.$$

  \item From a term of type $A\to B$ we can derive a term of type $B$, from a term of type $A$:
    $$(app)\qquad  {\displaystyle \Gamma \vdash M:A\to B\qquad \Gamma \vdash N:A      \over \Gamma \vdash MN:B}.$$
    Conversely, we can deduce th
    $$(abs)\qquad  {\displaystyle \Gamma, x: A\vdash M: B  \over \Gamma \vdash \lambda x^A.m:A\to B}.$$
  \end{itemize}
\end{definition}

\begin{definition}
  The terms of minimal simply typed $\lambda$-calculus are the raw terms that are subject to being typed under a certain Type context.
\end{definition}
\begin{remark}
  We have simply trimmed the fundamental part of the basic definition.
\end{remark}
\subsubsection{Expanded Typing}
In a different direction than minimal typing, expanded typing seeks to enlarge the definition of typing by considering the sum type. We can add to the definition of typing by considering the sum Type. The idea is to have two types $A$ and $B$ and be able to consider a term $\lambda x^{A+B}. x+1$. That is, we can consider two types to be acceptable by a function.

\begin{definition}
  The types of expanded simply typed $\lambda$-calculus are built via the BNF:
  $$A,B ::= \iota |\ A\to B\ |\ A \times B \ |\ A + B  \ |\ 1\ |\ 0.$$
  where $\iota$ denote a basic type.
\end{definition}

We proceed to define the raw typed $\lambda$-terms.

\begin{definition}
  The raw terms of expanded simply typed $\lambda$-calculus are built via the BNF:
  \begin{align*}
    A,B, C ::= x\ &|\ AB\ |\ \lambda x^t.A \ |\ \langle A,B \rangle\ |\ \pi_1A\ |\ \pi_2A\ |\ * \\
                  &|\ \iin_1 A\ |\ \iin_2 A \ |\ \ccase A ; x^{t_1}.B\oo x^{t_2}.C \ |\ \square^t.
  \end{align*}
  where $x$ denote any variables, and $t$ denote any type.
\end{definition}

The term $\ccase A ; x^{t_1}.B\oo x^{t_2}.C$ abstracts the idea of a function on  an union term behaves differently in each type that is included. In the typing laws we will see that it is necessary that the type returned by the application of a variable in a function of type union must always converge to the same type ($case$ rule).\\

For this typing, three new rules are included for type derivation, in addition to the previously defined in \ref{def:typing-rules}:
\begin{definition}[Typing Rules]
  We define the following typing rules, for each typing context $\Gamma$.
  \begin{itemize}
  \item Every type is part of a union type including it:
    $$  (\iin_i)\qquad  {\displaystyle\Gamma \vdash M:A_i \over \Gamma \vdash \iin_i M:A_1+A_2},\qquad  i=  1,2.$$
    Conversely, a term on a union must always have the same type after application.:
    $$(\ccase) \qquad {\displaystyle \Gamma\vdash M: A+B\qquad \Gamma, x:A\vdash N: C\qquad \Gamma, x:B\vdash P: C \over \Gamma \vdash \ccase A ; x^{A}.M\oo x^{B}.P:C}.$$
  \item The $\square^\cdot$ extract a term of any type from the void type.
    $$  (\square)\qquad  {\displaystyle\Gamma \vdash M:0 \over \Gamma \vdash \square^A M:A}.$$
  \end{itemize}
\end{definition}

We have the same consideration as previously, so that only those being subject to typing will be considered terms. Unification of typing considerations also holds.
\begin{definition}
  The terms of expanded simply typed $\lambda$-calculus are the raw terms that are subject to being typed under a certain Type context.
\end{definition}
\subsubsection{Pure and impure typed $\lambda$-calculus}
\label{section:pureandimpure}
We have seen that a typing context can vary which terms are subject to be formed and which does not. This consideration will be important later in our discussion, when we consider the category of $\lambda$-calculus. In this sense, we can assume that, associated with each typing context, a different $\lambda$-computation is generated. This distinction inspires the following definition.

\begin{definition}
  A basic (resp. minimal, extended) $\lambda$-calculus is a triple $(\Xi, \Lambda, \Gamma)$ such that:
  \begin{itemize}
  \item $\Xi$ is the collection of types of the basic lambda-calculus (resp. minimal, extended),
  \item $\Lambda$ is the collection of terms of the basic lambda-calculus (resp. minimal, extended),
  \item $\Gamma$ is a typing context,
  \end{itemize}
  where all rules of basic the (resp. minimal, extended) $\lambda$-calculus are satisfied. When there
  exist no more types and terms than those derived from the definition, and $\Gamma=\{\}$, it is called a \emph{pure basic (resp. minimal, extended) $\lambda$-calculus}.
\end{definition}
\begin{remark}
  Often, in a $\lambda$-calculus, is usual to consider terms up to $\alpha\beta\eta$-equivalence.
\end{remark}

The way to generate a new $\lambda$-calculus, is to add a new type $t$, (often, based on other types), and to create a (possibly abstract) new term $M$ with typing context $\Gamma = \{M:t\}$.
\subsection{Unification of typing}

We can consider two standards in typing: \emph{Church Style} and \emph{Curry Style}.
\begin{itemize}
\item Church style,  first shown in \cite{church1940formulation}, is an explicit system, as we do only consider expression with well defined types, in the same way that a function is consider with regard to it domain and codomain. In summary, the typing of a function is the first part of the definition of it, so you can define $f:(0,1)\to (0,1)$ and just then define $f(x)=1/x$. This domain is not it only a possible domain, but it what we choose it to be.
\item Curry style on the other hand, gain some of the notions of untyped $\lambda$-calculus and consider expression as function. To continue the real function analogy, it consider that the function $f(x)=x^2$ just exists, and one can check that it is well defined on $\R$. \\
\end{itemize}
More information about the typing style can be found on Chapters 10 and 11 of \cite{hindley2008lambda}. \\

Despite this apparent differences, both typing styles can be \emph{unified}. A unifier is a pair of substitutions that makes two type styles equal as typed templates. Such a unifier is based onta an algorithm based on type inference. This is instrumental to languages  as relevant as \texttt{python}, which works with implicit typed variables. How this algorithms for unification is shown in great detail in Chapter 9  of \cite{selinger2008lecture}. Due to this unification, we can consider only explicitely typed terms without any remorse.

\section{Curry-Howard bijection}
\subsection{Natural deduction}
In this section we succinctly introduce the notation of propositional intuitionistic logic, in order to work with it further this Chapter. As we will discuss them in more depth the deduction system in next chapter, we spare our readers of yet another comprehensive introduction of the widely known intuitional propositional logic. Should it be required, more information of propositional logic can be found in, for example, \cite{marek2009introduction} and \cite{wadler2015propositions}. Natural deduction originally appeared in the works of \cite{gentzen1935untersuchungen}. In this section, we present it in its most general form. \\

Throughout this section we replicate notations and process done when we defined typed $\lambda$-calculus. This is done purposely, as our aim is to prove an equivalence between both system. We start by considering the alphabet consisting a set of countable many variables $x,y,z,...$ as done previously in $\lambda$-calculus, together with two new symbols: $\top$ and $\bot$

\begin{definition}
  The formulas of propositional intuitionistic logic are built via the BNF:
  $$A,B ::= x |\ A\to B\ |\ A \land B \ |\ A \lor B \ |\ \top \ |\ \bot .$$
  where $x$ denotes any variable.
\end{definition}

Having the syntax done, is time to provide meaning. We want $\top$ to be the truth value. A formula is to be true is encoded in the formula $\top \to A$. Conversely, a formula is \emph{not-true} or \emph{false} (sometimes denoted as $\neg A$) is encoded as $ A \to \bot$. In addition, we want all the formulas from which we can derive certain to be true. For this we will define, analogously to the typing-contex from the previous chapter, what a \emph{truth-assumption} is and  what the rules of deduction are.

\begin{definition}
  A truth-assumption is a set of variables $\Gamma = \{x_1,...,x_n\}$ that we assume to be true. An \emph{judgement} $\Gamma \vdash B$ states that from the truth assumption $\Gamma$, the formula $B$ can be deduced to be true.
\end{definition}

Sometimes we can write a truth assumption $\Gamma, A_1, ..., A_n$, that will denote that additionally $A$ is assume to be true. As most times, we will only need a formula $A$ to be true or not without any interest on what exact variable configuration made this possible, so by abuse of notation we can consider $\Gamma = \{x_1, ..., x_n, A_1, ..., A_m\}$ to have both variables and formulas.
\begin{definition}[Deduction Rules]
  We define the following deduction rules, for each truth assumption $\Gamma=\{x_1,...,x_n\}$.
  \begin{itemize}
  \item Every variable assumed to be true is true:
    $$  (ax)\qquad  {\displaystyle \over \Gamma \vdash x_i},\qquad \forall i \in 1,..,n.$$
    In addition, $\top$ is always true:
    $$  (\top)\qquad  {\displaystyle \over \Gamma \vdash \top}.$$
  \item From a true formula $A\to B$ and a true formula $A$, $B$ can be derived:
    $$(\to_1)\qquad  {\displaystyle \Gamma \vdash A \to B\qquad \Gamma \vdash A      \over \Gamma \vdash B}.$$
    Conversely, if assuming $A$ deduces $B$, then $A\to B$ is true.
    $$(\to_2)\qquad  {\displaystyle \Gamma, A \vdash B      \over \Gamma \vdash (A\to B)}.$$

  \item The conjunction being true imply each element to be true:
    $$(\land_1) \qquad {\displaystyle \Gamma\vdash A_1\land A_2 \over \Gamma \vdash A, \qquad \Gamma \vdash B },$$
    and conversely:
    $$(\land_2) \qquad {\displaystyle \Gamma\vdash A\qquad \Gamma\vdash B \over \Gamma \vdash  A \land B}.$$
  \item An element being true implies the union being true:
    $$  (\lor_1^i)\qquad  {\displaystyle\Gamma \vdash A_i \over \Gamma \vdash A_1+A_2},\qquad  i=  1,2.$$
    Every type is part of a union type including it:
    $$  (\lor_1^i)\qquad  {\displaystyle\Gamma \over \Gamma A_i  \vdash A_1+A_2},\qquad  i=  1,2.$$
    conversely, A term on a union must always have the same type after application.:
    $$(\lor_2) \qquad {\displaystyle \Gamma\vdash A\lor B\qquad \Gamma, A\vdash C\qquad \Gamma, B\vdash C \over \Gamma \vdash C}.$$
  \item the \emph{ex falsum quodlibet} (i.e. everything is derivable from falsity)  holds:
    $$  (\bot)\qquad  {\displaystyle\Gamma \vdash \bot \over \Gamma \vdash A},$$
    for all term $A$.
  \end{itemize}
\end{definition}

Note that we consider a logic without law of excluded middle. This approach, although somewhat demodé in modern mathematics, was highly popular at the beginning of the twentieth century. It sought to solve the problems of mathematical foundations, in particular the \emph{principle of explosion}. \\

To work with this logic, we usually work in the usual \emph{truth derivation}. That is to deduce a formula we use the previously introduced rules in other to see whether can they be derivated from the truth assumption. More often than not, we consider the empty assumption, therefore we are working with \emph{tautologies}.

\begin{example}
  We can consider the formula $F=((x\to y) \land (y\to z)) \to ((x \to z)$ and $\Gamma$ the empty assumption. Then we can deduce:
  \begin{align*}
    \Gamma=\{\}& \vdash    ((x\to y) \land (y\to z)) \to (x\to z),\\
    \Gamma=\{(x\to y)\land (y\to z)\} & \vdash    (x \to z),\\
    \Gamma=\{(x\to y), (y\to z)\} & \vdash    (x \to z),\\
    \Gamma=\{ (y\to z)\} & \vdash    (y \to z),\\
    \Gamma=\{\} & \vdash \top,
  \end{align*}
  therefore the formula $F$ is true.
\end{example}

As a final note, the definitions of intuitionistic logic and $\lambda$-calculus introduced in this work are made to be matching but are not unique. For example, other sources such as  consider that the types of simply typed $\lambda$-calculus and consider only the types generated by:
$$A,B ::= x |\ A\to B\ |\ A \land B \ |\ \top .$$

This is called \emph{positive intuitionistic calculus}. In the next chapter we are going to define the concept of \emph{deduction system} and from it, grow into the different approaches to logic.

% We decided not to follow this line, which is equally valid, for consistency with \cite{seely1984locally} However, it is useful to have this consideration resolved from the beginning, especially for engineering applications.

\subsection{Bijection}
The first approach of $\lambda$-calculus to be seen as a deduction system was observed by Curry in 1934 \cite{curry1934functionality}, early in the development of this area and was completed by Howard in 1980 \cite{howard1980formulae}. \\% {\color{red} Añadir un poquito de introducción histórica que no cuesta na.}\\

Having already asked ourselves when does a term have a type, it is natural to arise the new question: given a type when does it exist a term of that type, This is in fact the fundamental idea of the Curry-Howard isomorphism. For example, considering whether it exists a term for the type$(A \times B) \to B$, is analogue to considering the formula $(A\land B)\to B$ to be a tautology. Moreover, the term $\lambda x^{A\times B}. \pi_2 x$ can be seen as a proof of the tautology! we have thus arrived at the dream of a constructivist mathematician: computational algorithms are demonstrations, and demonstrations are nothing but algorithms. Let us formalize this intuition.\\

We can create a pairing between types of a simply typed lambda calculus and formulas of intuitionistic logic  by pairing variables with variables and types with formulas as described in table \ref{tabla:curry-f}:
\begin{table}[!h]\label{tabla:curry-h}
  \begin{center}
    \begin{tabular}{|l|c|c|}
      \hline
      Typing name & Types  & Formulas  \\
      \hline
      Minimal     & Function type $\to$   & Implication $\to$  \\
      \hline
      Basic      & Type 1 & $\top$ \\
                  & Product type $\times$ & Conjuction $\land$ \\
      \hline
      Expanded   & Type 0 & $\bot$ \\
                  & Sum type $+$     & Disjuction $\lor$ \\
      \hline
    \end{tabular}
    \caption*{\label{tab:table-name} Pairing of formulas and terms.}
  \end{center}
\end{table}

And we can see that this pairing matches perfectly with the typying and deduction rules since:

\begin{itemize}
\item We pair the concept of a formula being true with the concept of a type having a term. We formalize this by pairing truth assumption $\Gamma=\{A\}$ is with the typing context $\Gamma\vdash M:A$.
\item We can pair $(var)$ with $(ax)$ and $(*)$ with $(\top)$.
\item We can pair $(app)$ with $(\to_1)$ and $(abs)$ with $(\to_2)$.
\item We can pair $(\pi_i)$ with $(\land_1)$ and $(\land_2)$ with $(pair)$.
\item We can pair $(\iin_i)$ with $(\lor_1^i)$ and $(\lor_2)$ with $(\ccase)$.
\item We can pair $0$ with $\bot$.
\end{itemize}

Finally, we pair the terms  with the truth derivations. Consider a term $M$ of type $C$ with associated formula $F$. By replacing each typing rule needed to deduce $M:C$  with the associated derivation rule, we can get a truth derivation, and conversely with a truth derivation of $F\to \top$ from $\Gamma=\{\}$.\\

\section{Lambda calculus as a computation system}
% TODO: Expand this section properly.
Around 1930, people start thinking about what does it mean for a function to be computable. There were three major answer to this question (section 4, \cite{cardone2006history}):
\begin{itemize}
\item In 1930s Alonzo Church introduced the notion of $\lambda$-calculus, in which a function is computable if we can write it as a recursive-lambda term onto Church's numerals, that is, if we can deduce the result after a (hopefully finite) deduction steps.
\item Later, Alan Turing, who had previously been a doctoral student of Alonzo Church, developed the concept of the Turing machine, in which a function is computable if a tape machine with a limited set of operations is capable of reproducing it.
\item Meanwhile, Gödel defined the computable function as the minimum set of function such that some properties (such as the existence of a successor function) are includes, and is closed under some operations.
\end{itemize}

The goal of this three mathematicians was to solve the Entscheidungsproblem\cite{hilbert1999principles}.\\

The interest for this type of problems was clear for the mathematical community as the goal was to propose an algorithm that solve every theorem and can be computed. The Church-Turing thesis fromally proved that these three independently generated notions were in fact equivalent\cite{copeland1997church}.\\

This thesis is also formulated in a philosophical background, to state that every effective computable function system is equivalent any of  those three. While it can not be formally proven, it has a near-universal acceptance to date. Any computing system that can replicate, and thus is equivalent, to either Turing Machines Calculus or $\lambda$-Calculus is said to be \emph{Turing Complete}.\\

Nonetheless, after the formalization of the solution of this problem there were a last problem that was not adverted: the finiteness of time. SAT is the problem that has as input a propositional logic formula and deduces whether it is satisfiable. This problem is NP-complete, quite famously the firs of these problems\cite{cook1971complexity}. If we consider the analogous problem for first order algebra, we have an even more complex P-space-complete problem, being almost intractable in worst cases to this date. As a happy consequence for us, much work is left to be done in this area \cite{cook2006p}.


\section{Shapley Value approximation}\label{sec:shapley}

The Shapley Value is a cornerstone measure in cooperative game theory (introduced in section \ref{subsec:the_shapley_value}).
It is an axiomatic approach to allocating a divisible reward or cost between participants where there is a clearly defined notion of how much surplus or profit a coalition of participants could achieve by themselves.
It has many applications,
including analysing the power of voting blocks in weighted voting games \citep{Bachrach2009ApproximatingPI},
in cost and surplus division problems  \citep{AzizEtal2016,archie_paper1},
and as a measure of network centrality \citep{Michalak:2013}.
But primarily, is useful to us as a method of allocating financial payments in electricity network contexts (see section \ref{the_value_def}).

%Formally, a \textit{cooperative game}, $\langle N,v\rangle\in\mathbb{G}_N$, comprises a set of $n$ players, $N=\{1,2,\dots,n\}$, and a \textit{characteristic function}, $v:S\subset N\rightarrow \mathbb{R}$, which is a function specifying the reward which can be achieved if a subset of the players $S\subset N$ cooperate, where $v(\emptyset)=0$.
%In this context the Shapley value $\varphi$ is a unique mapping from cooperative games to the player rewards $\mathbb{G}_N\rightarrow\mathbb{R}^n$ which satisfies axioms:

%\begin{itemize}
%\item
%\textbf{Efficiency}: That the total reward is divided up: $\sum_i\varphi_i(\langle N,v\rangle) = v(N)$
%\item
%\textbf{Symmetry}: If two players $i$ and $j$ are totally equivalent `substitutes' then the receive the same reward: ie. if $v(S\cup i)=v(S\cup j)~~\forall S\subseteq N\setminus\{i,j\}$, then $\varphi_i(\langle N,v\rangle) = \varphi_i(\langle N,v\rangle)$
%\item
%\textbf{Null Player}: If the addition of a player $i$ to any coalition brings nothing, and is a `null player', then it receives reward of zero: i.e if $v(S\cup i)=v(S)~~\forall S\subseteq N$ then $\varphi_i(\langle N,v\rangle)=0$
%\item
%\textbf{Additivity}: That for any two games the reward afforded each player is each is the sum of the games considered together: i.e. for any $v_1$ and $v_2$, that: $\varphi(\langle N,v_1+v_2\rangle)=\varphi(\langle N,v_1 \rangle) + \varphi(\langle N,v_2\rangle)$
%\end{itemize}

%Specifically, the Shapley value is expressed as:
%\begin{equation}\label{shap1_X}\varphi_i(\langle N,v\rangle) = \sum_{S\subset N, i\notin S}\frac{(n-|S|-1)!\,|S|!}{n!}(v(S\cup\{i\})-v(S))\end{equation}
%That is, under the Shapley value each player is afforded their average marginal contribution across every possible sequence of player join orderings.
%Or, if $v_{i,k}$ is the average marginal contribution which player $i$ can make across coalitions of size $k$:
%\begin{equation}
%v_{i,k} = \frac{1}{\binom{n-1}{k}}\sum_{S\subset N\setminus \{ i\} , |S|=k} %\frac{(n-|S|-1)!\,|S|!}{(n-1)!}
%(v(S\cup\{i\})-v(S))
%\end{equation}
%then the Shapley value can be expressed as an average:
%\begin{equation}\label{shap2_X} \varphi_i(\langle N,v\rangle) = \frac{1}{n}\sum_{k=0}^{n-1}v_{i,k} \end{equation}

Though the Shapley Value is conceptually simple, its use is hampered by the fact that its total expression involves exponentially many evaluations of the characteristic function (there are $n\times 2^{n-1}$ possible marginal contributions between $n$ players).
\DIFdelbegin %DIFDELCMD <

%DIFDELCMD < %%%
\DIFdelend However, since the Shapley Value is expressible as an average over averages by Equation~\eqref{shap2},
it is possible to approximate these \DIFaddbegin \DIFadd{inner }\DIFaddend averages via sampling techniques, and \DIFaddbegin \DIFadd{then }\DIFaddend these averages are naturally stratified by coalition size\DIFdelbegin \DIFdel{.
}\DIFdelend \DIFaddbegin \DIFadd{, forming an instance of stratified sampling.
For a given inner average in the Shapley Value expression, we approximate such an average by randomly selecting marginal contributions, calculating the sample average.
Thus the question then becomes how many samples should we compute for each strata to attain the best estimate of the Shapley Value.
}

\DIFaddend In previously published literature, other techniques have been used to allocate samples in this context \DIFaddbegin \DIFadd{of Shapley Value sampling approximation}\DIFaddend , particularly simple sampling \citep{DBLP:journals/cor/CastroGT09}, Neyman allocation \citep{CASTRO2017180,DBLP:journals/tsg/OBrienGR15}, and allocation to minimise Hoeffding's inequality \citep{2013arXiv1306.4265M}.

We assess the benefits of using our bound by comparing its performance to the approaches above in the context of some example cooperative games, with results analysed in Section~\ref{sec:discussion}.
The example games are described below:

\begin{example_game}[Airport Game]
An $n=15$ player game with characteristic function:
$$v(S)=\max_{i\in S}w_i$$
where
$w=\{w_1,\dots,w_{15}\} %\scriptstyle\scriptsize
=\{ 1, 1, 2, 2, 2, 3, 4, 5, 5, 5, 7, 8, 8, 8, 10\}$.
The maximum marginal contribution is $10$, so we assign $D_i=10$ for all $i$.
\end{example_game}

\begin{example_game}[Voting Game]
An $n=15$ player game with characteristic function:
$$v(S)=\begin{cases}
       1, &\quad\text{if}\quad \sum_{i\in S}w_i>\sum_{j\in N}w_j/2\\
       0, &\quad\text{otherwise}\\
     \end{cases}$$
where
$w=\{w_1,\dots,w_{15}\} %\scriptstyle\scriptsize
=\{ 1, 3, 3, 6, 12, 16, 17, 19, 19, 19, 21, 22, 23, 24, 29\}$.
The maximum marginal contribution is $1$, so we assign $D_i=1$ for all $i$.
\end{example_game}

\begin{example_game}[Simple Reward Division]
An $n=15$ player game with characteristic function:
$$v(S)=\frac{1}{2}\left(\sum_{i\in S}\frac{w_i}{100}\right)^2$$
where
$w=\{w_1,\dots,w_{15}\} = \{ 45, 41, 27, 26, 25, 21, 13, 13, 12, 12, 11, 11, 10, 10, 10 \}$\\
The maximum marginal contribution is $1.19025$, so we assign $D_i=1.19025$ for all $i$.
\end{example_game}

\begin{example_game}[Complex Reward Division]
An $n=15$ player game with characteristic function:
$$v(S)=\left(\sum_{i\in S}\frac{w_i}{50}\right)^2 - \floor[\Bigg]{\left(\sum_{i\in S}\frac{w_i}{50}\right)^2}$$
where
$w=\{w_1,\dots,w_{15}\} = \{ 45, 41, 27, 26, 25, 21, 13, 13, 12, 12, 11, 11, 10, 10, 10 \}$\\
In this game, we assign $D_i=2$ for all $i$.
\end{example_game}

%\input{table-2.tex}

\input{figs/shapley_table.tex}

For each game, we compute the exact Shapley Value, and then the average absolute errors in the approximated Shapley Value for a given budget $m$ of marginal-contribution samples across multiple computational runs.
The results are shown in Table \ref{Table2},
where the average absolute error in the Shapley Value is computed by sampling with Maleki's method \citep{2013arXiv1306.4265M} is denoted $e^{Ma}$, $e^{sim}$ is Castro's stratified simple sampling method \citep{DBLP:journals/cor/CastroGT09}, $e^{Ca}$ is Castro's Neyman sampling method \citep{CASTRO2017180}, and $e^{SEBM}$ is the error associated with our method, SEBM.
The results in Table~\ref{Table2} show that our method performs well across the benchmarks.
A discussion of all of the results is considered in the next section.


\documentclass[11pt]{article}
\usepackage{fullpage}
\usepackage{titling}
\usepackage{indentfirst}
\usepackage{amsmath, amssymb}
\usepackage{tikz}
\usepackage[colorlinks,linkcolor=blue]{hyperref}

\setlength{\parindent}{0em}
\setlength{\droptitle}{-3em}

\begin{document}

\title{Factor Graph}
\date{}
\maketitle
\vspace{-5em}

\section{Variables and Factors}
DeepDive uses factor graphs to perform learning and inference. A factor graph is a type of probabilistic graphical model.
There are two types of nodes in a factor graph, (random) variables and factors. A random variable can be used to quantitatively describe an event. For example, we can use a random variable to denote if John smokes. If John smokes, the random variable takes a value of 1, and 0 if John does not smoke. For now, DeepDive only supports boolean variables, so we will constrain our discussion to boolean variables.

A factor is a function of variables, and is used to evaluate the relations among variable(s). For example, a function imply(A, B) means if A, then B. Now suppose we have relation that ``if John smokes then he has cancer". Here we have two variables, one indicating if John smokes, the other indicating if John has cancer. Thus, imply(smoke, cancer) expresses the rule above.

The figure shows an example of a factor graph, where $v_1$ and $v_2$ are two variables, and $f_1, f_2$ are two factors. Factor $f_1$ is connected with $v_1$ and $v_2$, while $f_2$ is connected with $v_2$. We will use this example to illustrate some basic concepts about factor graphs.

\begin{center}
\includegraphics[width=2in]{factor_graph.png}
\end{center}

\section{Possible Worlds and Probabilities}
A possible world is a particular possible assignment to every variable, denoted by $I$. We can also think of it as each variable taking a particular value.
\begin{itemize}
\item How many possible worlds in the factor graph above? Each variable can take value 0 or 1, and there are two variables. So we have four possible worlds. The possible worlds are shown in the table below, with each column representing a possible world.
    \begin{center}
    \begin{tabular}{|l|llll|}
      \hline
      $v_1$ & 0 & 0 & 1 & 1\\
      \hline
      $v_2$ & 0 & 1 & 0 & 1\\
      \hline
    \end{tabular}
    \end{center}
\end{itemize}
How do we define the probability of a possible world? We define it through factor functions. We give different weight to factor functions, to express the relative influence of each factor on the probability. Factors with larger weight have greater impacts on the probability. The probability of a possible world graph is then defined to be proportional to some measure of weighted combination of factor functions (for how to define such a measure, please refer to [Factor Graphs and the Sum-Product Algorithm] \url{http://www.comm.utoronto.ca/~frank/papers/KFL01.pdf}), i.e., for the above graph,
\[ \text{Pr}(I) \propto \text{measure}\{w_1 f_1(v_1, v_2) + w_2 f_2(v_2)\}. \]
Here, $w_1, w_2$ are weights associated with factor functions.
\begin{itemize}
\item If $f_1$ is imply function with weight $w_1 = 1$, and $f_2$ is isTrue with weight $w_2 = 0.5$ (for explaination on types of factor functions, see \url{http://deepdive.stanford.edu/inference_rule_functions.html}). What is the probability of possible world $v_1 = 1, v_2 = 0$ proportional to (in terms of measure)?

    Here, $f_1(v_1, v_2)$ = imply(1, 0) = 0, and $f_2(v_2)$ = isTrue(0) = 0. Thus, the answer is easily computed by measure$(w_1 f_1(v_1, v_2) + w_2 f_2(v_2))$ = measure$(1 \cdot 0 + 0.5 \cdot 0)$ = measure(0).
\end{itemize}

It is not convenient to express the probability to be proportional to something rather than to have an absolute value. To define absolute probabilities of possible worlds, we can simply normalize the probabilities above against all possible worlds. That is, we define the probability of possible world I as
\[ \text{Pr}(I) = \frac{\text{measure}\{w^T f(I)\}}{\sum_{J} \text{measure}\{w^T f(J)\}}, \]
where the sum is over all possible worlds.
\begin{itemize}
\item What's the probability of the possible world $v_1=1,v_2=0$?
\end{itemize}

\section{Marginal Inference and Weight Learning}

Now, we can perform marginal inference on factor graphs. A marginal inference is to infer the probability of one variable taking a particular value. For example, if we would like to infer whether John has cancer, and it is expressed using a variable $v_1$, this means we would like to infer the probability of $v_1 = 1$. It is straightforward to define the probability of it as just the sum of probabilities of possible worlds that contains the specific value for that variable. This is similar to marginal probability and joint probability. The marginal inference for the event $\{v_1 = 1\}$ is expressed as
\[ \text{Pr}\{v_1 = 1\} = \sum_{I:v_1=1} \text{Pr}(I). \]
\begin{itemize}
\item What is the result of the marginal inference $\text{Pr}\{v_1 = 1\}$?
\end{itemize}

In DeepDive, you can assign factor weights manually, or you can let DeepDive learn weights automatically. In order to learn weights automatically, you must have enough training data available. DeepDive chooses weights that agree most with the training data. Formally, the training data is just set of possible worlds, and we choose weights by maximizing the probabilities of these possible worlds.

\end{document}


\documentclass{article}
\usepackage{geometry}
 \geometry{
 a4paper,
 total={170mm,257mm},
 left=20mm,
 top=20mm,
 }

 \usepackage{mathptmx}
 \usepackage{amsmath}
 \usepackage{diagbox}
 \usepackage{booktabs}
\usepackage{colortbl}
\usepackage{amssymb}

\usepackage{hyperref}
\hypersetup{
    colorlinks=True,
    linkcolor={blue!20!black},
    filecolor=magenta,
    urlcolor=cyan,
}

 \usepackage{caption}
\usepackage[export]{adjustbox} %% for picture frame
\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage{fancyhdr}

\pagestyle{fancy}
\fancyhf{}
\rhead{{Signal Analysis}}
\lhead{{Amrit Prasad Phuyal (PUL074BEX004)}}
\rfoot{\thepage}

%%% format and command for lab ans vhdl
\input{./Matlab.tex}
%%%%>>>>>>>........
\input{./CoverPage.tex}
\DeclareUnicodeCharacter{2212}{-}

\begin{document}

%----------------------------------------------------------------------------------------
%	TITLE PAGE
%----------------------------------------------------------------------------------------
\CP{Signal Analysis}{All in One}
{Visualization of Signals, Fourier Series \& Transform, Convolution and Frequency Response}
{Department of Electronics and Computer Engineering}

%----------------------------------------------------------------------------------------
\pagenumbering{gobble}
\tableofcontents
\vspace{1in}
%\pagebreak
\listoffigures
\vspace{1in}
%\pagebreak
\lstlistoflistings
\pagebreak
\pagenumbering{arabic}

% \anscode{testing matolahb cod4e}{./CODES/sino.m}

% \begin{figure}[H]
%     \centering
%     \includegraphics[scale=1,cframe=blue 0.5pt 3pt]{./FIG/sino}
%     \caption{..sad gd }
% \end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%
\section{Signal Visualization}
Signal is a function of one or more independent variables, which contain some information.Analog signal is a continuous signal in which one time-varying quantity represents another time-based variable and denoted as $x(t), y(t)$.A digital signal is a signal that is used to represent data as a sequence of separate values at any point in time denoted as $ x[n], y[n]$.

\subsection{Sinusoidal Signal}
Sinusoidal signal is in the form of x(t) = A cos(${w}_{0}\,\pm \phi$) or A sin(${w}_{0}\,\pm \phi$).
In matlab \textbf{sin} and \textbf{cos} function are used to calculate sine and cosine values and \textbf{plot} function to plot continuous Sinusoidal signal. \textbf{hold on} \& \textbf{hold off} command is used to display both signal in single plot. Similarly \textbf{xlabel}, \textbf{ylabel} and \textbf{title} are used for Labeling purposes.

\anscode{Visualization of signal x(t) \& y(t)}{./CODES/sinu.m}
\begin{figure}[H]
    \centering
    \includegraphics[scale=1.2,cframe=blue 0.5pt 3pt]{./FIG/sinu}
    \caption{Visualization of Sinusoidal signal }
\end{figure}

% %%%%%%%%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%%%%%

\subsection{Ramp Signal}
Ramp signal is denoted by $r[n]=an $and $r(t)=at$ for discrete and continuous signal. To plot discrete signal \textbf{stem} function is used in MATLAB.

\anscode{Discrete and Continuous ramp Signal}{./CODES/ramp.m}
\begin{figure}[H]
    \centering
    \includegraphics[scale=1.4,cframe=blue 0.5pt 3pt]{./FIG/ramp}
    \caption{Discrete and Continuous ramp Signal when a=2}
\end{figure}

% %%%%%%%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%%%%%
\subsection{Exponential Signal}
Continous time exponential signal $y(t)=ce^(at)$ and discrete time exponential signal $y[n]=ce^(an)$ were plotted using MATLAB.

\anscode{Discrete and Continuous Exponential Signal}{./CODES/exp.m}

\begin{figure}[H]
    \centering
    \includegraphics[scale=1.4,cframe=blue 0.5pt 3pt]{./FIG/exp}
    \caption{Discrete and Continuous Exponential Signal when c=1.5 \& a=0.15 }
\end{figure}

% %%%%%%%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%%%%%
\subsection{Unit Step Signal}
\textbf{u(t) }  \&  \textbf{ u[n] } are continuous and discrete Unit step function which has value 1 for variable(t or n) $\geqq 0$
\anscode{Discrete and Continuous Unit Step Signal}{./CODES/unit.m}

\begin{figure}[H]
    \centering
    \includegraphics[scale=1.35,cframe=blue 0.5pt 3pt]{./FIG/unit}
    \caption{Discrete and Continuous Unit Step Signal}
\end{figure}

% %%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%

% %%%%%%%%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%%%%%
\section{Fourier Series}
A continuous time signal $x(t)$ with period $T$ is represented by Fourier series as

\begin{equation*}x(t)=\sum_{k = -\infty}^{\infty} a_ke^{jkw_ot}
\end{equation*}

$\quad \quad {here,w_o=2\pi/T} $\\
\begin{equation*}
    a_k=\int_{T}^{}x(t)e^{-jw_ot}  \,dx
\end{equation*}

The Fourier series representation of a square wave with period T and amplitude a is given by:
\begin{equation*}
    x(t)=\frac{4a}{\pi} \sum_{k = 1}^{\infty}\frac{sin((2k-1)w_ot)}{2k-1}
\end{equation*}

The sum of all the odd harmonics of the sinusoidal signal forms a square wave approximation.So, we varies the no of terms in summation.

\anscode{Fourier Series Representation of a Square Wave }{./CODES/FouSer.m}

\begin{figure}[H]
    \centering
    \includegraphics[scale=0.83,cframe=blue 0.5pt 3pt]{./FIG/FouSer.pdf}
    \caption{Fourier Series Representation of a Square Wave}
\end{figure}

% %%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%

% %%%%%%%%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%%%%%
\section{Fourier Transform}
The fourier transform of a continuous time signal $x(t)$ is mathematically given as,
$$
    X(j\omega) = \int_{-\infty}^{\infty} x(t) e^{-j\omega t} dt
$$
Likewise, the fourier transform of a discrete time signal $x[n]$ is mathematically given as,
$$
    X(e^{j\omega})=\sum_{n=-\infty}^{\infty} x[n] e^{-j\omega n}
$$
The \textbf{fft} function in MATLAB returns the real and imaginary parts of the fast fourier transform for the input argument. The real and imaginary parts are plotted separately.
\anscode{Fourier transform of x[n] = [0,1,2,3] }{./CODES/FouTra.m}

\begin{figure}[H]
    \centering
    \includegraphics[scale=0.55,cframe=blue 0.5pt 3pt]{./FIG/FouTra}
    \caption{Fourier transform of x[n] = [0,1,2,3]}
\end{figure}

% %%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%

% %%%%%%%%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%%%%%
\section{Convolution}
The convolution of two continuous time signals $x(t)$ and $h(t)$, called the convolution integral, is mathematically given as,
$$
    y(t)=x(t)*h(t)=\int_{-\infty}^{\infty}x(u)h(t-u)du
$$
Likewise, the convolution of two discrete time signals $x[n]$ and $h[n]$, called the convolution sum, is mathematically given as,

$$
    y[n]=x[n]*h[n]=\sum_{k=-\infty}^{\infty} x[n]h[n-k]
$$
The \textbf{conv} function returns the convolution for two discrete sequences in MATLAB.

\anscode{Convolution for two discrete sequences $x[n]$ = [1,3,2,1,1] and $h[n]$ = [1,2,−1,1] }{./CODES/Conv1.m}

\begin{figure}[H]
    \centering
    \includegraphics[scale=0.55,cframe=blue 0.5pt 3pt]{./FIG/conv1}

    \caption{Convolution for two discrete sequences $x[n]$ = [1,3,2,1,1] and $h[n]$ = [1,2,−1,1]}
\end{figure}

Convolution for two discrete sequences $x[n]$ = 0.5 n and $h[n] = u[n]$

\anscode{Convolution for two discrete sequences $x[n]$ = 0.5 n and $h[n] = u[n]$ }{./CODES/Conv2.m}

\begin{figure}[H]
    \centering
    \includegraphics[scale=0.55,cframe=blue 0.5pt 3pt]{./FIG/Conv2}
    \caption{Convolution for two discrete sequences $x[n]$ = 0.5 n and $h[n] = u[n]$ }
\end{figure}

% %%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%

% %%%%%%%%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%%%%%
\section{Frequency Response of a System}
For a system with $h(t)$ as the impulse response, and $x(t)$ as the input signal, the output $y(t)$ is related to the input as the convolution,
$$
    y(t)=x(t)*h(t)=\int_{-\infty}^{\infty}x(u)h(t-u)du
$$
According to the convolution property of fourier transforms, the fourier transforms of the signals are related as,
$$
    Y(j\omega)=H(j\omega)X(j\omega)
$$
where $H(j\omega)$, the fourier transform of the impulse response of the system is the frequency response of the system. Likewise, for discrete time input signal $x[n]$ to a system with the impulse response $h[n]$, the output in frequency domain is mathematically given as,
$$
    Y(e^{j\omega})=H(e^{j\omega})X(e^{j\omega})
$$
where $H(e^{j\omega})$, the fourier transform of the impulse response of the system is the frequency response of the system.
During the lab experiment, we plotted the frequency response given as,
$$
    H(z)=\frac{0.008-0.033z+0.05z^2-0.033z^3+0.008z^4}{1+2.37z+2.7z^2+1.6z^3+0.5z^4}
$$
The \textbf{freqz} function returns the frequency response of a system whose amplitude and phase were plotted separately in MATLAB.

\anscode{ Frequency response of $H(z)$ }{./CODES/FreqRes.m}

\begin{figure}[H]

    \centering
    \includegraphics[scale=0.55,cframe=blue 0.5pt 3pt]{./FIG/FreqRes}
    \caption{ Frequency response of $H(z)$ }
\end{figure}

% %%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%

\section{Conclusion}
In this Lab we got familiarize to various MATLAB function. We were able to Plot some basic function like unit, exponential, Sinusoidal, Ramp etc. We also find and plot Fourier Series and Transform and additionally performed Convolution with \textbf{conv} function and view frequency response of the system. Thus all the LAB exercises were completed.

\end{document}

Signal Analysis all in one LAB/Signal Analysis ( Amrit Prasad Phuyal) All Lab .tex


\chapter{Project Design}

This chapter outlines the design behind each component of the text classification model and the respected process each component may entail, several components will execute more than one step to achieve the desired result. The design is reflected within the final model and each component is broken down to display the functionality and theory within this project. Within section \ref{section:FunctionalRequirements}, it is detailed there won’t be a GUI for the interaction and thus the design section relates to the inner workings of the model itself, that being: system architecture, logistics and theory.

\section{Classical vs Modern}

As originally intended, this project would have seen two differing implementations of the same concept, one being of a classical nature and the other being a machine learning variation, as previously mentioned this project experienced time management issues due to unexpected issue, to which resulted in only focusing on the machine learning implementation of a novel approach. The design for the first model would have been of the following:

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{figures/chapter-5/ClassicalNLP.pdf}
    \caption[ClassicalNLP]{Classical NLP pipeline for text classification.
    \label{fig:ClassicalNLP}}
\end{figure}

Classical or "traditional" methods for NLP include: N-Gram, Hidden Markov Models using Markov Chains and Part-Of-Speech Tagging. The developer had originally intended to implement a traditional approach within a machine learning model. The combination of Part-Of-Speech with a machine learning model to calculate a word’s vector based on its TF-IDF value would have been the start, such that:

\begin{equation}
    tf(t,d) = \frac{f_t, _d}{\sum{t \in_d} f _t {_`}, _d}
\end{equation}

It would have also used the inverse TF-IDF value as the project model covers multiple datasets, such that:

\begin{equation}
    idf(t, D) = \log \frac{N}{|d \in D : t \in d|}
\end{equation}

Where the traditional aspect of the Bag-of-Words would produce a vector for each item in a corpus, it’s TF-IDF value would have been calculate through a series of CNN nodes.

Machine learning concepts can be classed as a “black-box” of functionality as the user does not necessarily see what is being executed within the hidden layers, a high-level abstraction for this project can be generalised into the following diagram:

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{figures/chapter-5/MLNLP.pdf}
    \caption[MachineLearningNLP]{Machine Learning Model for an NLP pipeline for text classification.
    \label{fig:MLNLP}}
\end{figure}

\section{Planning the ML Model Design}

Initial prototyping of the machine learning model for a new amalgamation of NLP techniques helped to indicate what the best route of development could be, the planning stage piggybacked off existing work flowchart diagrams in-order to apply the most appropriate method and technique combination. The sequence flowchart is as follows:

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{figures/chapter-5/GooglePlan.pdf}
    \caption[GooglePlan]{Text Classification Flowchart \parencite{google2021TCF}.
    \label{fig:GooglePlan}}
\end{figure}

\section{Supervision}

The development of the project model is based on a supervised approach due to the datasets located, it was most appropriate to use a supervised approach due to the datasets having no labels or lexical categories to train the model on; the model has user input to account for missing labels on data which have been manually and algorithmically added. The supervision for this project can be represented as the following diagram:

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{figures/chapter-5/SupervisedLearningChart.pdf}
    \caption[SupervisedLearning]{Sequence control for Supervised learning.
    \label{fig:SupervisedLearningChart}}
\end{figure}

\section{Pipeline Design}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{figures/chapter-5/Pipeline.pdf}
    \caption[MLTCPipeline]{Pipeline for model classification.
    \label{fig:MLTCPipeline}}
\end{figure}

There are five main steps for a text classification pipeline:

\begin{enumerate}
    \item \textbf{\textit{Pre-processing}}: prepare the raw dataset to be trained.
    \item \textbf{\textit{Splitting}}: split the processed dataset to be trained and validated.
    \item \textbf{\textit{Tuning}}: identity valuable parameters within trained data.
    \item \textbf{\textit{Training}}: train the current iteration of the model with updated hyper-parameters.
    \item \textbf{\textit{Testing}}: test and collect statistics for analysis to make further predictions.
\end{enumerate}

\section{Data Preparing and Pre---processing}

Data Preparing

\section{Model}

% Skip-gram rather than Continuous Bag-Of-Words (CBOW) as it yields better results with large datasets - such as student feedback - skip gram can also be context aware as it converts neighboring lexemes to vectors

% \subsection{Skip-Gram}

% \begin{figure}[H]
%     \centering
%     \includegraphics[width=0.49\textwidth]{figures/chapter-5/SkipGramModel.pdf}
%     \caption[SkipGramModel]{Input flow of Skip-Gram model.
%     \label{fig:SkipGramModel}}
% \end{figure}

% The skip-gram model can be seen as the inverse of the bag-of-words model as it attempts to vectorize neighboring words first to identify corpus context, whereas, bag-of-words takes each lexeme first, produces a vector sum and then categorises each word.

% https://machinelearningmastery.com/gentle-introduction-bag-words-model/
% https://towardsdatascience.com/pos-tagging-using-crfs-ea430c5fb78b#1c6a
% https://nathanrooy.github.io/posts/2018-03-22/word2vec-from-scratch-with-python-and-numpy/
% https://stackoverflow.com/questions/37394970/tensorflow-word2vec-cbow-model
% https://www.azlifa.com/morphology-%E2%80%93-the-structure-of-words/

\subsection{The Artefact}

POS TAGGING + Machine Learning

"POS tags are another way of labeling text data, in particular we are labeling the to- kens/words in a text. ... In a way POS tags are in themselves features of words; they give classification and context to words that allow us to better understand the purpose of word choice and the meaning of sentences as a whole."

% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------