\documentclass[../main.tex]{subfiles}
\begin{document}
The purpose of the
%%%%%%%%%%%%%%%%%%%%%binary search tree%%%%%%%%%%%%%%%%%%%%%%
\section{Binary Search Tree}
\label{sec_binary_search_tree}
 In computer science, a \textbf{search tree} is a tree data structure used for locating specific keys from within a set. In order for a tree to function as a search tree, the key for each node must be greater than any keys in subtrees on the left and less than any keys in subtrees on the right.

The advantage of search trees is their efficient search time ( $O(\log n)$) given the tree is reasonably balanced, which is to say the leaves at either end are of comparable depths as we introduced the \textbf{balanced binary tree}.

The search tree data structure supports many dynamic-set operations, including \textbf{Search} for a key, minimum or maximum, predecesor or successor, \textbf{insert} and \textbf{delete}. Thus, a search tree can be both used as a dictionary and a priority queue.
\begin{figure}[!ht]
    \centering
    \includegraphics[width = 0.4\columnwidth]{fig/Binary_search_tree.png}
    \caption{Example of Binary search tree of depth 3 and 8 nodes.}
    \label{fig:bst}
\end{figure}

% Search trees are often used to implement an associative array. The search tree algorithm uses the key from the key-value pair to find a location, and then the application stores the entire key–value pair at that location.

% In this section, we will introduce the most commonly used two types of searching trees: binary searching tree (BST) and Trie where the keys are usually numeric numbers and strings respectively.

% \subsection{Binary Searching Tree}
% \label{concept_binary_search_tree}
A binary search tree (BST) is a search tree with children up to two.  There are three possible ways to properly define a BST, and we use $l$ and $r$ to represent the left and right child of node $x$: 1)$l.key \leq x.key < r.key$, 2) $l.key  < x.key \leq r.key$, 3) $l.key < x.key < r.key$. In the first and second definition, our resulting BST allows us to have duplicates, while not in the case of the third definiton. One example of BST without duplicates is shown in Fig~\ref{fig:bst}.

% an organized searching tree structure in binary tree, as the name suggests. Binary search trees whose internal nodes each store a key (and optionally, an associated value), each node have two distinguished sub-trees (if only one sub-tree the other is None).

% BST keep their keys in sorted order, so that lookup and other operations can use the \textit{principle of binary search tree}:

% \indent Let $x$ be a node in a binary search tree, if $y$ is a node in the left subtree of x, them $y.key \leq x.key$. If $y$ is a node in the right subtree of $x$, then $y.key \geq x.key$.

\subsubsection{Operations}
When looking for a key in a tree (or a place to insert a new key), we traverse the tree from root to leaf, making comparisons to keys stored in the nodes of the tree and deciding, on the basis of the comparison, to continue searching in the left or right subtrees. On average, this means that each comparison allows the operations to skip about half of the tree, so that each SEARCH, INSERT or DELETE takes time proportional to the logarithm of the number of items stored in the tree. This is much better than the linear time required to find items by key in an (unsorted) array, but slower than the corresponding operations on hash tables.

% \textbf{Definition} A binary search tree is a rooted binary tree, whose internal nodes each store a key (and optionally, an associated value) and each have two distinguished sub-trees, commonly denoted left and right. The tree additionally satisfies the binary search property, which states that the key in each node must be greater than or equal to any key stored in the left sub-tree, and less than or equal to any key stored in the right sub-tree.[1]:287 The leaves (final nodes) of the tree contain no key and have no structure to distinguish them from one another.

In order to build a BST, we need to INSERT a series of elements in the tree organized by the searching tree property, and in order to INSERT, we need to SEARCH the position to INSERT this element. Thus, we introduce these operations in the order of SEARCH, INSERT and GENERATE.
\begin{figure}[h]
    \centering
    \includegraphics[width=0.6\columnwidth]{fig/bst_insertion.png}
    \caption{The lightly shaded nodes indicate the simple path from the root down to the position where the item is inserted. The dashed line indicates the link in the tree that is added to insert the item. }
    \label{fig:bst_operation}
\end{figure}
\paragraph{SEARCH}

There are two different implementations for SEARCH: recursive and iterative.
\begin{lstlisting}[language = Python]
# recursive searching
def search(root,key):
    # Base Cases: root is null or key is present at root
    if root is None or root.val == key:
        return root

    # Key is greater than root's key
    if root.val < key:
        return search(root.right,key)

    # Key is smaller than root's key
    return search(root.left,key)
\end{lstlisting}
Also, we can write it in an iterative way, which helps us save the heap space:
\begin{lstlisting}[language = Python]
# iterative searching
def iterative_search(root,key):
    while root is not None and root.val != key:
        if root.val < key:
            root = root.right
        else:
            root = root.left
    return root
\end{lstlisting}
\paragraph{INSERT}
Assuming we are inserting a node $13$ into the tree shown in Fig~\ref{fig:bst_operation}. A new key is always inserted at leaf (there are other ways to insert but here we only discuss this one way). We start searching a key from root till we hit an empty node. Then we new a TreeNode and insert this new node either as the left or the child node according to the searching property. Here we still shows both the recursive and iterative solutions.
\begin{lstlisting}[language = Python]
# Recursive insertion
def insertion(root, key):
    if root is None:
        root = TreeNode(key)
        return root
    if root.val < key:
        root.right = insertion(root.right, key)
    else:
        root.left = insertion(root.left, key)
    return root
\end{lstlisting}
The above code needs return value and reassign the value for the right and left every time, we can use the following code which might looks more complex with the if condition but works faster and only assign element at the end.
\begin{lstlisting}[language=Python]
# recursive insertion
def insertion(root, val):
    if root is None:
        root = TreeNode(val)
        return
    if val > root.val:
        if root.right is None:
            root.right = TreeNode(val)
        else:
            insertion(root.right, val)
    else:
        if root.left is None:
            root.left = TreeNode(val)
        else:
            insertion(root.left, val)
\end{lstlisting}
We can search the node iteratively and save the previous node. The while loop would stop when hit at an empty node.  There will be three cases in the case of the previous node.
\begin{lstlisting}[numbers=none]
1. The previous node is None, which means the tree is empty, so we assign a root node with the value
2. The previous node has a value larger than the key, means we need to put key as left child.
3. The previous node has a value smaller than the key, means we need to put key as right child.
\end{lstlisting}
\begin{lstlisting}[language = Python]
# iterative insertion
def iterativeInsertion(root, key):
    pre_node = None
    node = root
    while node is not None:
        pre_node = node
        if key < node.val:
            node = node.left
        else:
            node = node.right
    # we reached to the leaf node which is pre_node
    if pre_node is None:
        root = TreeNode(key)
    elif pre_node.val > key:
        pre_node.left = TreeNode(key)
    else:
        pre_node.right = TreeNode(key)
    return root
\end{lstlisting}
\paragraph{BST Generation}
First, let us declare a node as BST which is the root node. Given a list, we just need to call INSERT for each element. The time complexity can be $O(n\log_n)$.
\begin{lstlisting}[language=Python]
datas = [8, 3, 10, 1, 6, 14, 4, 7, 13]
BST = None
for key in datas:
    BST = iterativeInsertion(BST, key)
print(LevelOrder(BST))
# output
# [8, 3, 10, 1, 6, 14, 4, 7, 13]
\end{lstlisting}
\paragraph{DELETE}
Before we start to check the implementation of DELETE, I would suggest the readers to read the next subsection--the Features of BST at first, and then come back here to finish this paragraph.

When we delete a node, three possibilities arise.
\begin{lstlisting}[numbers=none]
1) Node to be deleted is leaf: Simply remove from the tree.

              50                            50
           /     \         delete(20)      /   \
          30      70       --------->    30     70
         /  \    /  \                     \    /  \
       20   40  60   80                   40  60   80

2) Node to be deleted has only one child: Copy the child to the node and delete the child

              50                            50
           /     \         delete(30)      /   \
          30      70       --------->    40     70
            \    /  \                          /  \
            40  60   80                       60   80

3) Node to be deleted has two children: Find inorder successor of the node. Copy contents of the inorder successor to the node and delete the inorder successor. Note that inorder predecessor can also be used.

              50                            60
           /     \         delete(50)      /   \
          40      70       --------->    40    70
                 /  \                            \
                60   80                           80

The important thing to note is, inorder successor is needed only when right child is not empty. In this particular case, inorder successor can be obtained by finding the minimum value in right child of the node.
\end{lstlisting}

\subsubsection{Features of BST}
\label{concept_features_bst}
\paragraph{Minimum and Maximum} The operation is similar to search, to find the minimum, we always traverse on the left subtree.  For the maximum, we just need to replace the ``left'' with ``right'' in the key word.  Here the time complexity is the same $O(lgn)$.
\begin{lstlisting}[language=Python]
# recursive
def get_minimum(root):
    if root is None:
        return None
    if root.left is None: # a leaf or node has no left subtree
        return root
    if root.left:
        return get_minimum(root.left)

# iterative
def iterative_get_minimum(root):
    while root.left is not None:
        root = root.left
    return root
\end{lstlisting}

Also, sometimes we need to search two additional items related to a given node:  successor and predecessor. The structure of a binary search tree allows us to determine the successor or the predecessor of a tree without ever comparing keys.

\paragraph{Successor of a Node}  A successor of node $x$ is the smallest item in the BST that is strictly greater than $x$. It is also called in-order successor, which is the next node in Inorder traversal of the Binary Tree. Inoreder Successor is None for the last node in inorder traversal. If our TreeNode data structure has a parent node.

Use parent node: the algorihtm has two cases on the basis of the right subtree of the input node.
\begin{lstlisting}[numbers=none]
For the right subtree of the node:
1) If it is not None, then the successor is the minimum node in the right subtree. e.g. for node 12, successor(12) = 13 = min(12.right)
2) If it is None, then the successor is one of its ancestors. We traverse up using the parent node until we find a node which is the left child of its parent. Then the parent node here is the successor. e.g.  successor(2)=5
\end{lstlisting}
 The Python code is provided:
\begin{lstlisting}[language = Python]
def Successor(root, n):
# Step 1 of the above algorithm
    if n.right is not None:
        return get_minimum(n.right)
# Step 2 of the above algorithm
p = n.parent
while p is not None:
    if n == p.left :# if current node is the left child node, then we found the successor, p
        return p
    n = p
    p = p.parent
return p
\end{lstlisting}
However, if it happens that your tree node has no parent defined, which means you can not traverse back its parents. We only have one option. Use the inorder tree traversal, and find the element right after the node. \begin{lstlisting}[numbers=none]
For the right subtree of the node:
1) If it is not None, then the successor is the minimum node in the right subtree. e.g. for node 12, successor(12) = 13 = min(12.right)
2) If it is None, then the successor is one of its ancestors. We traverse down from the root till we find current node, the node in advance of current node is the successor. e.g.  successor(2)=5
\end{lstlisting}
\begin{lstlisting}[language=Python]
def SuccessorInorder(root, n):
    # Step 1 of the above algorithm
    if n.right is not None:
        return get_minimum(n.right)
    # Step 2 of the above algorithm
    succ = None
    while root is not None:

        if n.val > root.val:
            root = root.right
        elif n.val < root.val:
            succ = root
            root = root.left
        else: # we found the node, no need to traverse
            break
    return succ
\end{lstlisting}

\paragraph{Predecessor of A Node}  A predecessor of node $x$ on the other side, is the largest item in BST that is strictly smaller than $x$. It is also called in-order predecessor, which denotes the previous node in Inorder traversal of BST. e.g. for node 14, predecessor(14)=12= max(14.left). The same searching rule applies, if node $x$'s left subtree exists, we return the maximum value of the left subtree. Otherwise we traverse back its parents, and make sure it is the right subtree, then we return the value of its parent, otherwise the reversal traverse keeps going.
\begin{lstlisting}[language = Python]
def Predecessor(root, n):
# Step 1 of the above algorithm
    if n.left is not None:
        return get_maximum(n.left)
# Step 2 of the above algorithm
p = n.parent
while p is not None:
    if n == p.right :# if current node is the right node, parent is smaller
        return p
    n = p
    p = p.parent
return p
\end{lstlisting}
 The worst case to find the successor or the predecessor of a BST is to search the height of the tree: include the one of the subtrees of the current node, and go back to all the parents and greatparents of this code, which makes it the height of the tree. The expected time complexity is $O(lgn)$. And the worst is when the tree line up and has no branch, which makes it $O(n)$.

% \item Smallest and the Biggest Value
% \begin{lstlisting}[language = Python]
% #recursive
% def getSmallest(node):
%     if not node:
%          return None
%     if node.left:
%          return getSmallest(node.left)
%     return node
% #iterative
% def getSmallest(node):
%     while node:
%           node=node.left
%     return node
% \end{lstlisting}
% \end{enumerate}

% \textbf{Insertion and  Generation of BST} Insertion and deletion is not as easy as the operations shown before, because they cause the dynamic set represented by the binary tree to change. Therefore, the data structure must be re-structured to reflect the change in order to keep holding the binary-search-tree property.

% Insertion is more straightforward, it is a key component to build a BST. To build a BST, we start from an empty root, and we go through a sequence of data that we want to store in BST structure, and insert each one in the right place with the binary search tree property. To insert a node into the BST, at first we search the tree level-by-level starts from the root node, if the value is smaller that the comparing node's key, we move to its left subtree or else to the right subtree till we reach to leaf node. Look at an example, the follwing tree,

% the code is as follows:

% Thus the code for building a BST is as follows:

% \textcolor{red}{to do: write the whole operations and the definition of BST as a python class}

% DELETE:
% \begin{lstlisting}[language = Python]
% def delete(root,key):

%     # Base Cases: root is null or key is present at root
%     if root is None or root.val == key:
%         return root

%     # Key is greater than root's key
%     if root.val < key:
%         return search(root.right,key)

%     # Key is smaller than root's key
%     return search(root.left,key)
% \end{lstlisting}

Now  we put a table here to summarize the space and time complexity for each operation.
\begin{table}[h]
\begin{small}
\centering
\noindent\captionof{table}{ Time complexity of operations for BST in big O notation }
 \noindent \begin{tabular}{|p{0.33\columnwidth}|p{0.33\columnwidth}| p{0.33\columnwidth}|}
  \hline
 Algorithm & Average & Worst Case  \\ \hline
Space  & $O(n)$& $O(n)$ \\
Search   & $O(lgn)$ & $O(n)$ \\ \hline

Insert & $O(lgn)$ & $O(n)$ \\
Delete & $O(lgn)$ & $O(n)$ \\ \hline
\end{tabular}
  \label{tab:msrc_precession}
  \end{small}
\end{table}

% \paragraph{Advanced Features}
% For a BST, the left subtree all have smaller values than the current node, and the right subtree are all bigger than the current node. This concept is useful in trimming BST, see example, $669$. Trim a Binary Search Tree.

% \section{Augmented Tree}
% According to \textit{Introduction to Algorithms}, augmenting data stuctures are defined as a textbook data structure augmented by storing additional information in it. In this Section, we introduce two types of augmented tree: Trie for pattern matching in static String and Segment Tree for Range Query.

%https://www.mimuw.edu.pl/~szczurek/TSG2/04_suffix_arrays.pdf

%%%%%%%%%%%%%%Segment Tree%%%%%%%%%%%%%%%
\section{Segment Tree}
\label{sec_segment_tree}
% In this subsection, we discuss another data structure which can efficiently answer dynamic range queries. As a starting point, we discuss a problem of finding the index of the minimum element
% in an array given a range: [i..j]. This is more commonly known as the Range Minimum Query(RMQ). For example, given an array A of size 7 below, RMQ(1, 3) = 2, as the index 2 contains the minimum element among A[1], A[2], and A[3]. To check your understanding of RMQ, verify that on array A below, RMQ(3, 4) = 4, RMQ(0, 0) = 0, RMQ(0, 1) = 1, and RMQ(0, 6) = 5.

% There are several ways to solve this RMQ. One of the trivial algorithm is to simply iterate the
% array from index i to j and report the index with the minimum value. But this is O(n) per query.
% When n is large, such algorithm maybe infeasible.

Segment Tree is a static full binary tree similar to heap that is used for storing the intervals or segments. `Static` here means once the data structure is build, it can not be modified or extended.  Segment tree is a data structure that can efficiently answer numerous \textit{dynamic range queries} problems
(in logarithmic time) like finding minimum, maximum, sum, greatest common divisor, least common denominator in array. The ``dynamic" means there are constantly modifications of the value of elements (not the tree structure).   For instance, given a problem to find the index of the minimum/maximum/sum of all elements in an given range of an array: [i:j].

\paragraph{Definition} Consider an array A of size n and a corresponding Segment Tree T (here a range [0, n-1] in A is represented as A[0:N-1]):
\begin{enumerate}
    \item The root of T represents the whole array A[0:N-1].
    \item Each internal node in the Segment Tree T represents the interval of A[i:j] where $0 < i < j < n$.
    \item Each leaf in T represents a single element A[i], where $0 \leq i<N$.
    \item If the parent node is in range [i, j], then we separate this range at the middle position $m = (i+j)\/\/2$, the left child takes range [i, m], and the right child take the interval of [m+1, j].
\end{enumerate}

Because in each step of building the segment tree, the interval is divided into two halves, so the height of the segment tree will be $\log N$. And there will be totally N leaves and N-1 number of internal nodes, which makes the total number of nodes in segment tree to be $2N-1$ and make the segment tree a \textit{full binary tree}.

Here, we use the Range Sum Query (RSQ) problem to demonstrate how segment tree works:
\begin{examples}[resume]
\item \textbf{307. Range Sum Query - Mutable (medium).} Given an integer array nums, find the sum of the elements between indices i and j ($i \leq j$), inclusive.The \textit{update(i, val)} function modifies nums by updating the element at index i to val.
\begin{lstlisting}[numbers=none]
Example:

Given nums = [1, 3, 5]

sumRange(0, 2) -> 9
update(1, 2)
sumRange(0, 2) -> 8
\end{lstlisting}
Note:
\begin{enumerate}
    \item
    The array is only modifiable by the update function.
   \item  You may assume the number of calls to update and sumRange function is distributed evenly.
\end{enumerate}
\paragraph{Solution: Brute-Force.} There are several ways to solve the RSQ. The \textbf{brute-force solution} is to simply iterate the array from index i to j to sum up the elements and return its corresponding index. And it gives $O(n)$ per query, such algorithm maybe infeasible if queries are constantly required.  Because the update and query action distributed evenly, it still gives $O(n)$ time complexity and $O(n)$ in space, which will get LET error.

\paragraph{Solution: Segment Tree.}  With Segment Tree, we can store the TreeNode's val as the sum of elements in its corresponding interval. We can define a TreeNode as follows:
\begin{lstlisting}[language=Python]
class TreeNode:
    def __init__(self, val, start, end):
        self.val = val
        self.start = start
        self.end = end
        self.left = None
        self.right = None
\end{lstlisting}
As we see in the process, it is actually not necessary if we save the size of the array, we can decide the start and end index of each node on-the-fly and saves space.
\begin{figure}[h]
    \centering
    \includegraphics[width=0.8\columnwidth]{fig/307_RSQ_SegmentTree.png}
    \caption{Illustration of Segment Tree. }
    \label{fig:segment_tree}
\end{figure}
\paragraph{Build Segment Tree.} Because the leaves of the tree is a single element, we can use divide and conquer to build the tree recursively.  For a given node, we first build and return its left and right child(including calculating its sum) in advance in the `divide` step, and in the `conquer' step, we calculate this node's sum using its left and right child's sum, and set its left and right child. Because there are totally $2n-1$ nodes, which makes the time and space complexity $O(n)$.
\begin{lstlisting}[language=Python]
def _buildSegmentTree(self, nums, s, e): #start index and end index
    if s > e:
        return None
    if s == e:
        return self.TreeNode(nums[s])

    m = (s + e)//2
    # divide
    left = self._buildSegmentTree(nums, s, m)
    right = self._buildSegmentTree(nums, m+1, e)

    # conquer
    node = self.TreeNode(left.val + right.val)
    node.left = left
    node.right = right
    return node
\end{lstlisting}
\paragraph{Update Segment Tree.} Updating the value at index i is like searching the tree for leaf node with range [i, i]. We just need to recalculate the value of the node in the path of the searching. This operation takes $O(\log n)$ time complexity.
\begin{lstlisting}[language=Python]
def _updateNode(self, i, val, root, s, e):
    if s == e:
        root.val = val
        return
    m = (s + e)//2
    if i <= m:
        self._updateNode(i, val, root.left, s, m)
    else:
        self._updateNode(i, val, root.right, m+1, e)
    root.val = root.left.val + root.right.val
    return
\end{lstlisting}
\paragraph{Range Sum Query.} Each query range [i, j], will be a combination of ranges of one or multiple ranges. For instance, as in the segment tree  shown in Fig~\ref{fig:segment_tree}, for range [2, 4], it will be combination of [2, 3] and [4, 4]. The process is similar to the updating, we starts from the root, and get its middle index m: 1) if [i, j] is the same as [s, e] that i == s and j == e, then return the value, 2) if the interval [i, j] is within range [s, m] that j <=m , then we just search it in the left branch. 3) if [i, j] in within range [m+1, e] that i>m, then we search for the right branch. 4) else, we search both branch and the left branch has target [i, m], and the right side has target [m+1, j], the return value should be the sum of both sides.  The time complexity is still $O(\log n)$.
\begin{lstlisting}[language=Python]
def _rangeQuery(self, root, i, j, s, e):
    if s > e or i > j:
        return 0
    if s == i and j == e:
        return root.val if root is not None else 0

    m = (s + e)//2

    if j <= m:
        return self._rangeQuery(root.left, i, j, s, m)
    elif i > m:
        return self._rangeQuery(root.right, i, j, m+1, e)
    else:
        return self._rangeQuery(root.left, i, m, s, m) + self._rangeQuery(root.right, m+1, j, m+1, e)
\end{lstlisting}
The complete code is given:
\begin{lstlisting}[language=Python]
class NumArray:
    class TreeNode:
        def __init__(self, val):
            self.val = val
            self.left = None
            self.right = None

    def __init__(self, nums):
        self.n = 0
        self.st = None
        if nums:
            self.n = len(nums)
            self.st = self._buildSegmentTree(nums, 0, self.n-1)

    def update(self, i, val):
        self._updateNode(i, val, self.st, 0, self.n -1)

    def sumRange(self, i, j):
        return self._rangeQuery(self.st, i, j, 0, self.n-1)
\end{lstlisting}
\end{examples}

Segment tree can be used here to lower the complexity of each query to $O(log n)$.

%%%%%%%%%%%%%%%%%%%Trie%%%%%%%%%%%%%%%%%
\section{Trie for String}
\label{concept_trie}
\paragraph{Definition} Trie comes from the word re\textbf{Trie}val. In computer science, a trie, also called digital tree, radix tree or prefix tree which like BST is also a kind of search tree for finding substring in a text. We can solve string matching in $O(|T|)$ time,  where |T| is the size of our text.  This purely algorithmic approach has been studied extensively in the algorithms:  Knuth-Morris-Pratt, Boyer-Moore, and Rabin-Karp. However, we entertain the possibility that multiple queries will be made to the same text.  This motivates the development of data structures that preprocess the text to allow for more efficient queries. Such efficient data structure is Trie, which can do each query in $O(P)$, where P is the length of the pattern string. Trie is an ordered tree structure, which is used mostly for storing strings (like words in dictionary) in a compact way.
\begin{enumerate}
    \item In a Trie, each child branch is labeled with letters in the alphabet $\sum$. Actually, it is not necessary to store the letter as the key, because if we  order the child branches of every node alphabetically from left to right, the position in the tree defines the key which it is associated to.
    \item The root node in a Trie represents an empty string.
\end{enumerate}
% An ordered tree data structure used to store a dynamic set or associative array where the keys are usually strings. Unlike a binary search tree, no node in the tree stores the key associated with that node; instead, its position in the tree defines the key with which it is associated.

Now, we define a trie Node: first it would have a bool variable to denote if it is the end of the word and a children which is a list of of 26 children TrieNodes.
\begin{lstlisting}[language= Python]
class TrieNode:
    # Trie node class
    def __init__(self):
        self.children = [None]*26
        # isEndOfWord is True if node represent the end of the word
        self.isEndOfWord = False
\end{lstlisting}
\begin{figure}[h]
    \centering
    \includegraphics[width=0.6\columnwidth]{fig/trie_compact_trie.jpg}
    \caption{Trie VS Compact Trie}
    \label{fig:trie_compact_trie}
\end{figure}

\paragraph{Compact Trie} If we assign only one letter per edge, we are not taking full advantage of the trie’s tree structure. It is more useful to consider compact or compressed tries, tries where we remove the one letter per edge constraint, and contract non-branching paths by concatenating the letters on these paths.
In this way, every node branches out, and every node traversed represents a choice between two different words.  The compressed trie that corresponds to our example trie is also shown in Figure
~\ref{fig:trie_compact_trie}.

\paragraph{Operations: INSERT, SEARCH}
% Now, let us solve an LeetCode problem together which requires us to implement a complete Trie that with the operations INSERT, SEARCH, STARTWITH. All of these operations are actually quickly similar and they all require us to simultaneously iterate each character in the input string (or word) and each level of the Trie on the location of that character. So, it would not be hard to get the worst time complexity when we searched the whole tree or finished iterating the characters in the input.
Both for INSERT and SEARCH, it takes $O(m)$, where m is the length of the word/string we wand to insert or search in the trie. Here, we use an LeetCode problem as an example showing how to implement INSERT and SEARCH. Because constructing a trie is a series of INSERT operations which will take $O(n*m)$, n is the total numbers of words/strings, and m is the average length of each item. The space complexity fof the non-compact Trie would be $O(N*|\sum|)$, where $|\sum|$ is the alphlbetical size, and N is the total number of nodes in the trie structure. The upper bound of N is $n*m$.
\begin{figure}[h]
    \centering
    \includegraphics[width=0.6\columnwidth]{fig/Trie.png}
    \caption{Trie Structure}
    \label{fig:trie}
\end{figure}
\begin{examples}
\item \textbf{208. Implement Trie (Prefix Tree) (medium).} Implement a trie with insert, search, and startsWith methods.
\begin{lstlisting}
Example:
Trie trie = new Trie();
trie.insert("apple");
trie.search("apple");   // returns true
trie.search("app");     // returns false
trie.startsWith("app"); // returns true
trie.insert("app");
trie.search("app");     // returns true
\end{lstlisting}
\textit{Note: You may assume that all inputs are consist of lowercase letters a-z. All inputs are guaranteed to be non-empty strings.}

\paragraph{INSERT} with INSERT operation, we woould be able to insert a given word in the trie, when traversing the trie from the root node which is a TrieNode, with each letter in world, if its corresponding node is None, we need to put a node, and continue. At the end, we need to set that node's endofWord variable to True. thereafter, we would have a new branch starts from that node constructured. For example, when we first insert ``app`` as shown in Fig~\ref{fig:trie_compact_trie}, we would end up building branch ``app``, and with ape, we would add nodes ``e`` as demonstrated with red arrows.
\begin{lstlisting}[language=Python]
def insert(self, word):
    """
    Inserts a word into the trie.
    :type word: str
    :rtype: void
    """
    node = self.root #start from the root node
    for c in word:
        loc = ord(c)-ord('a')
        if node.children[loc] is  None: # char does not exist, new one
            node.children[loc] = self.TrieNode()
        # move to the next node
        node = node.children[loc]
    # set the flag to true
    node.is_word = True
\end{lstlisting}

\paragraph{SEARCH} For SEARCH, like INSERT, we traverse the trie using the letters as pointers to the next branch. There are three cases: 1) for word P, if it doesnt exist, but its prefix does exist, then we return False. 2) If we found a matching for all the letters of P, at the last node, we need to check if it is a leaf node where is\_word is True.  STARTWITH is just slightly different from SEARCH, it does not need to check that and return True after all letters matched.
\begin{lstlisting}[language=Python]
def search(self, word):
    node = self.root
    for c in word:
        loc = ord(c)-ord('a')
        # case 1: not all letters matched
        if node.children[loc] is None:
            return False
        node = node.children[loc]
    # case 2
    return True if node.is_word else False
\end{lstlisting}
\begin{lstlisting}[language=Python]
def startWith(self, word):
    node = self.root
    for c in word:
        loc = ord(c)-ord('a')
        # case 1: not all letters matched
        if node.children[loc] is None:
            return False
        node = node.children[loc]
    # case 2
    return True
\end{lstlisting}
Now complete the given Trie class with TrieNode and \_\_init\_\_ function.
\begin{lstlisting}[language=Python]
class Trie:
    class TrieNode:
        def __init__(self):
            self.is_word = False
            self.children = [None] * 26 #the order of the node represents a char

    def __init__(self):
        """
        Initialize your data structure here.
        """
        self.root = self.TrieNode() # root has value None
\end{lstlisting}
\end{examples}

\begin{examples}
\item \textbf{336. Palindrome Pairs (hard).} Given a list of unique words, find all pairs of distinct indices (i, j) in the given list, so that the concatenation of the two words, i.e. words[i] + words[j] is a palindrome.
\begin{lstlisting}
Example 1:

Input: ["abcd","dcba","lls","s","sssll"]
Output: [[0,1],[1,0],[3,2],[2,4]]
Explanation: The palindromes are ["dcbaabcd","abcddcba","slls","llssssll"]

Example 2:

Input: ["bat","tab","cat"]
Output: [[0,1],[1,0]]
Explanation: The palindromes are ["battab","tabbat"]
\end{lstlisting}
\textbf{Solution: One Forward Trie and Another Backward Trie.}  We start from the naive solution, which means for each element, we check if it is palindrome with all the other strings. And from the example 1, [3,3] can be a pair, but it is not one of the outputs, which means this is a combination problem, the time complexity is ${C_n}{C_{n-1}}$, and multiply it with the average length of all the strings, we make it $m$, which makes the complexity to be $O(mn^2)$. However, we can use Trie Structure,
\begin{lstlisting}[language = Python]
from collections import defaultdict

class Trie:
    def __init__(self):
        self.links = defaultdict(self.__class__)
        self.index = None
        # holds indices which contain this prefix and whose remainder is a palindrome
        self.pali_indices = set()

    def insert(self, word, i):
        trie = self
        for j, ch in enumerate(word):
            trie = trie.links[ch]
            if word[j+1:] and is_palindrome(word[j+1:]):
                trie.pali_indices.add(i)
        trie.index = i

def is_palindrome(word):
    i, j = 0, len(word) - 1
    while i <= j:
        if word[i] != word[j]:
            return False
        i += 1
        j -= 1
    return True

class Solution:
    def palindromePairs(self, words):
        '''Find pairs of palindromes in O(n*k^2) time and O(n*k) space.'''
        root = Trie()
        res = []
        for i, word in enumerate(words):
            if not word:
                continue
            root.insert(word[::-1], i)
        for i, word in enumerate(words):
            if not word:
                continue
            trie = root
            for j, ch in enumerate(word):
                if ch not in trie.links:
                    break
                trie = trie.links[ch]
                if is_palindrome(word[j+1:]) and trie.index is not None and trie.index != i:
                    # if this word completes to a palindrome and the prefix is a word, complete it
                    res.append([i, trie.index])
            else:
                # this word is a reverse suffix of other words, combine with those that complete to a palindrome
                for pali_index in trie.pali_indices:
                    if i != pali_index:
                        res.append([i, pali_index])
        if '' in words:
            j = words.index('')
            for i, word in enumerate(words):
                if i != j and is_palindrome(word):
                    res.append([i, j])
                    res.append([j, i])
        return res
\end{lstlisting}
\textbf{Solution2: .}Moreover, there are always more clever ways to solve these problems. Let us look at a clever way:
 abcd, the prefix is ''. 'a', 'ab', 'abc', 'abcd', if the prefix is a palindrome, so the reverse[abcd], reverse[dc], to find them in the words, the words stored in the words with index is fastest to find. $O(n)$. Note that when considering suffixes, we explicitly leave out the empty string to avoid counting duplicates. That is, if a palindrome can be created by appending an entire other word to the current word, then we will already consider such a palindrome when considering the empty string as prefix for the other word.
 \begin{lstlisting}[language = Python]
 class Solution(object):
    def palindromePairs(self, words):
        # 0 means the word is not reversed, 1 means the word is reversed
        words, length, result = sorted([(w, 0, i, len(w)) for i, w in enumerate(words)] +
                                   [(w[::-1], 1, i, len(w)) for i, w in enumerate(words)]), len(words) * 2, []

        #after the sorting,the same string were nearby, one is 0 and one is 1
        for i, (word1, rev1, ind1, len1) in enumerate(words):
            for j in xrange(i + 1, length):
                word2, rev2, ind2, _ = words[j]
                #print word1, word2
                if word2.startswith(word1): # word2 might be longer
                    if ind1 != ind2 and rev1 ^ rev2: # one is reversed one is not
                        rest = word2[len1:]
                        if rest == rest[::-1]: result += ([ind1, ind2],) if rev2 else ([ind2, ind1],) # if rev2 is reversed, the from ind1 to ind2
                else:
                    break # from the point of view, break is powerful, this way, we only deal with possible reversed,
        return result
 \end{lstlisting}
 \end{examples}

 %https://fizzbuzzed.com/top-interview-questions-5/
% \paragraph{Searching}
% \paragraph{Insertion}
% \paragraph{Deletion}

% Let us see the complete code of a Trie Class:
% \begin{lstlisting}[language = Python]

% class Trie:

%     # Trie data structure class
%     def __init__(self):
%         self.root = self.getNode()

%     def getNode(self):

%         # Returns new trie node (initialized to NULLs)
%         return TrieNode()

%     def _charToIndex(self,ch):

%         # private helper function
%         # Converts key current character into index
%         # use only 'a' through 'z' and lower case

%         return ord(ch)-ord('a')

%     def insert(self,key):

%         # If not present, inserts key into trie
%         # If the key is prefix of trie node,
%         # just marks leaf node
%         pCrawl = self.root
%         length = len(key)
%         for level in range(length):
%             index = self._charToIndex(key[level])

%             # if current character is not present
%             if not pCrawl.children[index]:
%                 pCrawl.children[index] = self.getNode()
%             pCrawl = pCrawl.children[index]

%         # mark last node as leaf
%         pCrawl.isEndOfWord = True

%     def search(self, key):

%         # Search key in the trie
%         # Returns true if key presents
%         # in trie, else false
%         pCrawl = self.root
%         length = len(key)
%         for level in range(length):
%             index = self._charToIndex(key[level])
%             if not pCrawl.children[index]:
%                 return False
%             pCrawl = pCrawl.children[index]

%         return pCrawl != None and pCrawl.isEndOfWord

% # driver function
% def main():

%     # Input keys (use only 'a' through 'z' and lower case)
%     keys = ["the","a","there","anaswe","any",
%             "by","their"]
%     output = ["Not present in trie",
%               "Present in tire"]

%     # Trie object
%     t = Trie()

%     # Construct trie
%     for key in keys:
%         t.insert(key)

%     # Search for different keys
%     print("{} ---- {}".format("the",output[t.search("the")]))
%     print("{} ---- {}".format("these",output[t.search("these")]))
%     print("{} ---- {}".format("their",output[t.search("their")]))
%     print("{} ---- {}".format("thaw",output[t.search("thaw")]))

% if __name__ == '__main__':
%     main()
% \end{lstlisting}
There are several other data structures, like balanced trees and hash tables, which give us the possibility to search for a word in a dataset of strings. Then why do we need trie? Although hash table has $O(1)$ time complexity for looking for a key, it is not efficient in the following operations :
\begin{itemize}
    \item Finding all keys with a common prefix.
    \item Enumerating a dataset of strings in lexicographical order.
\end{itemize}

\paragraph{Sorting}
Lexicographic sorting of a set of keys can be accomplished by building a trie from them, and traversing it in pre-order, printing only the leaves' values. This algorithm is a form of radix sort. This is why it is also called radix tree.

% \paragraph{Dynamic Programming for Static Array}

%%%%%%%%%%%%%%%%%%%%%%%%bonus
\section{Bonus}
\paragraph{Solve Duplicate Problem in BST} When there are duplicates, things can be more complicated, and the college algorithm book did not really tell us what to do when there are duplicates.  If you use the definition "left <= root < right" and you have a tree like:
\begin{lstlisting}[numbers=none]
      3
    /   \
  2       4
\end{lstlisting}

then adding a ``3'' duplicate key to this tree will result in:
\begin{lstlisting} [numbers=none]
      3
    /   \
  2       4
    \
     3
\end{lstlisting}
Note that the duplicates are not in contiguous levels.

This is a big issue when allowing duplicates in a BST representation as the one above: duplicates may be separated by any number of levels, so checking for duplicate's existence is not that simple as just checking for immediate children of a node.

An option to avoid this issue is to not represent duplicates structurally (as separate nodes) but instead use a counter that counts the number of occurrences of the key. The previous example would then have a tree like:
\begin{lstlisting}
      3(1)
    /     \
  2(1)     4(1)
  \end{lstlisting}

and after insertion of the duplicate "3" key it will become:
\begin{lstlisting}
      3(2)
    /     \
  2(1)     4(1)
  \end{lstlisting}

This simplifies SEARCH, DELETE and INSERT operations, at the expense of some extra bytes and counter operations. In the following content, we assume using definition three so that our BST will have no duplicates.

%%%%%%%%%%%%%%%%%%%%%%%%%Exercise%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{LeetCode Problems}

\begin{enumerate}
    \item 144. Binary Tree Preorder Traversal
    \item 94. Binary Tree Inorder Traversal
    \item 145. Binary Tree Postorder Traversal
    \item 589. N-ary Tree Preorder Traversal
    \item 590. N-ary Tree Postorder Traversal
    \item 429. N-ary Tree Level Order Traversal
    \item 103. Binary Tree Zigzag Level Order Traversal(medium)
    \item 105. Construct Binary Tree from Preorder and Inorder Traversal
\end{enumerate}

938. Range Sum of BST (Medium)

Given the root node of a \textbf{binary search tree}, return the sum of values of all nodes with value between L and R (inclusive).

The binary search tree is guaranteed to have unique values.
\begin{lstlisting}
Example 1:

Input: root = [10,5,15,3,7,null,18], L = 7, R = 15
Output: 32

Example 2:

Input: root = [10,5,15,3,7,13,18,1,null,6], L = 6, R = 10
Output: 23
\end{lstlisting}
\textbf{Tree Traversal+Divide and Conquer}. We need at most $O(n)$ time complexity. For each node, there are three cases: 1) L <= val <= R, 2)val < L, 3)val > R. For the first case it needs to obtain results for both its subtrees and merge with its own val. For the others two, because of the property of BST, only the result of one subtree is needed.
\begin{lstlisting}[language=Python]
def rangeSumBST(self, root, L, R):
    if not root:
        return 0
    if L <= root.val <= R:
        return self.rangeSumBST(root.left, L, R) + self.rangeSumBST(root.right, L, R) + root.val
    elif root.val < L: #left is not needed
        return self.rangeSumBST(root.right, L, R)
    else: # right subtree is not needed
        return self.rangeSumBST(root.left, L, R)
\end{lstlisting}

\end{document}


%----------------------------------------------------------------------------------------
% Introduction
%----------------------------------------------------------------------------------------
\setcounter{page}{1} % Sets counter of page to 1

\section{Introduction} % Add a section title

In mathematical statistics, a problem that often comes up with basic tests (t-tests, Analysis of Variance, etc) is dealing with small sample sizes. When looking specifically at the two-sample t-test, the condition that has to be met and is taught in all introductory statistics courses is normality of both samples. However, in the case where this conditions cannot be met, we hope to have a sample size greater than $30$. But this magic number $30$ can be misleading. Real world data often contains more than $30$ observations. Creating models such as linear regression models is difficult because real world data often fails to meet critical model conditions such as normality of residuals and homoescedasticity. Because of this, we cannot blindly trust parameters like a $\hat{\beta_i}\text{'s}$ in linear regression models and residual deviance in logistic regression models. There are also times when real world data is too small and then we can’t even use the Central Limit Theorem to assume our statistics come from an approximation of a normal distribution. However, there is a solution to both of these problems: randomization based inference. Another common solution is bootstrapping, however, that is most useful when we want to develop robust estimates for statistics and standard errors as well as counteract sampling error. We are focusing on randomization methods, which focus on randomization of units within our sample. Forgetting all model distribution assumptions, randomization-based inference only cares whether the sample that you have is typical of the population. In this paper, we will explain what randomization based inference is in detail while walking through a short example pointing out each step in the context of our paper, explain why this process works, elaborate on some of the limitations that come with this test, and explore Monte Carlo methods for re-randomization.


\section{The \reversecopy algorithm}
\Label{sec:reversecopy}

The \reversecopy
algorithm of the \cxx Standard Library \cite[\S 28.6.10]{cxx-17-draft} inverts the order of elements
in a sequence.
\reversecopy does not change the input sequence, and
copies its result to the output sequence.
For our purposes we have modified the generic implementation
to that of a range of type \valuetype.
The signature now reads:

\begin{lstlisting}[style=acsl-block]

  void reverse_copy(const value_type* a, size_type n, value_type* b);
\end{lstlisting}

%\subsection{The predicate \Reverse}

Informally, \reversecopy copies the elements from the array \inl{a} into
array \inl{b} such that the copy is a reverse of the original array.
In order to concisely formalize these conditions we define in the following
listing the predicate \logicref{Reverse} (see also Figure~\ref{fig:Reverse}).

\input{Listings/Reverse.acsl.tex}

We also define several overloaded variants of \Reverse that
provide default values for some of the parameters.
These overloaded versions enable us to write later more concise \acsl annotations.

\begin{figure}[hbt]
\centering
\includegraphics[width=0.60\textwidth]{Figures/reverse_logic.pdf}
\caption{\Label{fig:Reverse} Sketch of predicate~\Reverse}
\end{figure}

\FloatBarrier

\subsection{Formal specification of \reversecopy}

The specification of \specref{reversecopy} is shown in the following listing
We use the second version of predicate \logicref{Reverse}
in order to formulate the postcondition of \reversecopy.

\input{Listings/reverse_copy.h.tex}

\subsection{Implementation of \reversecopy}

The implementation of \implref{reversecopy} is shown in the following listing.
%
For the postcondition to be true, we must ensure that for
every element \inl{i}, the comparison \inl{b[i] == a[n-1-i]} holds.
This is formalized by the loop invariant~\inl{reverse} where we employ
the first version of \logicref{Reverse}.

\input{Listings/reverse_copy.c.tex}

\clearpage


\section{Results \& discussion}
\label{sec:resul}

We have not mentioned the polynomial function due to the factor that is always performing worst in our tests.
In the following table you can see the result for linear and Radial Basis function kernel type and different values of C.

\begin{table}[!ht]
	\begin{center}
	\begin{tabular}{ccc}
		\hline
		Kernel Type  & C & Accuracy  \\
		\hline
		\multirow{4}{*}{Radial Basis function}       & 0.5     & 75$\%$ \\
				 & 1        & 78 $\%$         \\
				 & 3        & 84  $\%$       \\
				 & 5        & 83 $\%$    	    \\
		\hline
		\multirow{4}{*}{Linear}       & 0.5     & 78$\%$ \\
				 & 1        & 80 $\%$         \\
				 & 3        & 82  $\%$       \\
				 & 5        & 77 $\%$    	    \\ \hline
	\end{tabular}
	\end{center}
	\caption{Summary of accuracy with different kernel and different C.}
	\label{tb:table2}
\end{table}

Another important parameter to tune was the minimum word frequency. The following table contains evaluation of different word frequencies.

\begin{table}[!ht]
	\begin{center}
	\begin{tabular}{cc}
		\hline
		Word Freequency & Accuracy  \\
		\hline
		3      & 75$\%$ \\
				5  & 79 $\%$         \\
				   7 & 77  $\%$       \\
				 10    & 76 $\%$    	    \\
		\hline
	\end{tabular}
	\end{center}
	\caption{Summary of accuracy with different kernel and different C.}
	\label{tb:table3}
\end{table}

Based on this knowledge we chose the following hyper-parameters:

\begin{table}[!h]
	\begin{center}
	\begin{tabular}{c | c}
		\hline
		Word Freequency & 5 \\ \hline

		Kernel Type & Radial Basis \\  \hline

		C & 3 \\  \hline

		Training Accuracy & 95$\%$ \\  \hline

		Test Accuracy & 85$\%$ \\
		\hline
	\end{tabular}
	\end{center}
	\caption{Summary of accuracy with different kernel and different C.}
	\label{tb:table4}
\end{table}


\chapter{Lecture 4 May 09 2018}
  \label{chapter:lecture_4_may_09_2018}

\section{Groups (Continued)} % (fold)
\label{sec:groups_continued}

\subsection{Groups (Continued)} % (fold)
\label{sub:groups_continued}

\begin{propo}[Cancellation Laws]\label{propo:cancellation_laws}
  Let $G$ be a group and $g, h, f \in G$. Then
  \begin{enumerate}
    \item \begin{enumerate}
        \item (\hlnoteb{Right Cancellation}) $gh = gf \implies h = f$
        \item (\hlnoteb{Left Cancellation}) $hg = fg \implies h = f$
      \end{enumerate}
    \item The equation $ax = b$ and $ya = b$ have unique solution for $x, y \in G$.
  \end{enumerate}
\end{propo}

\begin{proof}
  \begin{enumerate}
    \item \begin{enumerate}
      \item By left multiplication and associativity,
        \begin{equation*}
          gh = gf \iff g^{-1} gh = g^{-1} gf \iff h = f
        \end{equation*}
      \item By right multiplication and associativity,
        \begin{equation*}
          hg = fg \iff hgg^{-1} = fgg^{-1} \iff h = f
        \end{equation*}
    \end{enumerate}

    \item Let $x = a^{-1} b$. Then
      \begin{equation*}
        a x = a (a^{-1} b) = (aa^{-1}) b = b.
      \end{equation*}
      If $\exists u \in G$ that is another solution, then
      \begin{equation*}
        au = b = ax \implies u = x
      \end{equation*}
      by Left Cancellation. The proof for $ya = b$ is similar by letting $y = ba^{-1}$.
  \end{enumerate}\qed
\end{proof}

% subsection groups_continued (end)

\subsection{Cayley Tables} % (fold)
\label{sub:cayley_tables}

For a finite group, defining its operation by means of a table is sometimes convenient.

\begin{defn}[Cayley Table]\label{defn:cayley_table}
\index{Cayley Table}
  Let $G$ be a group. Given $x, y \in G$, let the product $xy$ be an entry of a table in the row corresponding to $x$ and column corresponding to $y$. Such a table is called a \hlnoteb{Cayley Table}.
\end{defn}

\begin{note}
  By \autoref{propo:cancellation_laws}, the entries in each row (and respectively, column) of a Cayley Table are all distinct.
\end{note}

\begin{eg}
  Consider the group $(\mathbb{Z}_2, +)$. Its Cayley Table is
  \begin{center}
    \begin{tabular}{c|c|c}
      $\mathbb{Z}_2$ & $[0]$ & $[1]$ \\
      \hline
      $[0]$     & $[0]$ & $[1]$ \\
      $[1]$     & $[1]$ & $[0]$
    \end{tabular}
  \end{center}
  where note that we must have $[1] + [1] = [0]$; otherwise if $[1] + [1] = [1]$ then $[1]$ does not have its additive inverse, which contradicts the fact that it is in the group.
\end{eg}

\marginnote {
  If we replace $1$ by $[0]$ and $-1$ by $[1]$, the Cayley Tables of $\mathbb{Z}_2$ and $\mathbb{Z}^*$ are the same. In thie case, we say that $\mathbb{Z}_2$ and $\mathbb{Z}^*$ are \hlnotea{isomorphic}, which we denote by $\mathbb{Z}_2 \cong \mathbb{Z}^*$.
}

\begin{eg}
  Consider the group $\mathbb{Z}^* = \{1. -1\}$. Its Cayley Table (under multiplication) is
  \begin{center}
    \begin{tabular}{c|c|c}
      $\mathbb{Z}^*$ & $1$    & $-1$ \\
      \hline
      $1$              & $1$  & $-1$ \\
      $-1$             & $-1$ & $1$
    \end{tabular}
  \end{center}
\end{eg}

\begin{eg}\label{eg:cyclic_group_cayley_table}
  Given $n \in \mathbb{N}$, the \hldefn{Cyclic Group} of order $n$ is defined by
  \begin{equation*}
    C_n = \{1, a, a^2, ..., a^{n - 1}\} \quad \text{with } a^n = 1.
  \end{equation*}
  We write $C_n = \langle a : a^n = 1 \rangle$ and $a$ is called a generator of $C_n$. The Cayley Table of $C_n$ is
  \begin{center}
    \begin{tabular}{c | c c c c c c}
      $C_n$     & $1$       & $a$       & $a^2$  & $\hdots$ & $a^{n - 2}$ & $a^{n - 1}$ \\
      \hline
      $1$       & $1$       & $a$       & $a^2$  & $\hdots$ & $a^{n - 2}$ & $a^{n - 1}$ \\
      $a$       & $a$       & $a^2$     & $a^3$  & $\hdots$ & $a^{n - 1}$ & $1$ \\
      $a^2$     & $a^2$     & $a^3$     & $a^4$  & $\hdots$ & $1$         & $a$ \\
      \vdots    & \vdots    & \vdots    & \vdots &          & \vdots      & \vdots \\
      $a^{n-2}$ & $a^{n-2}$ & $a^{n-1}$ & $1$    & $\hdots$ & $a^{n-4}$   & $a^{n-3}$ \\
      $a^{n-1}$ & $a^{n-1}$ & $1$       & $a$    & $\hdots$ & $a^{n-3}$   & $a^{n-2}$
    \end{tabular}
  \end{center}
\end{eg}

\begin{propo}\label{propo:small_groups}
  Let $G$ be a group. Up to isomorphism, we have
  \begin{enumerate}
    \item if $\abs{G} = 1$, then $G \cong \{1\}$.
    \item if $\abs{G} = 2$, then $G \cong C_2$.
    \item if $\abs{G} = 3$, then $G \cong C_3$.
    \item if $\abs{G} = 4$, then either $G \cong C_4$ or $G \cong K_4 \cong C_2 \times C_2$ \marginnote{$K_n$ is known as the \hldefn{Klein n-group}}.
  \end{enumerate}
\end{propo}

\begin{proof}
  \begin{enumerate}
    \item If $\abs{G} = 1$, then it can only be $G = \{1\}$ where $1$ is the identity element.
    \item $\abs{G} = 2 \implies G = \{1, g\}$ with $g \neq 1$. The Cayley Table of $G$ is thus
      \begin{center}
        \begin{tabular}{c | c c}
        $G$ & $1$ & $g$ \\
        \hline
        $1$ & $1$ & $g$ \\
        $g$ & $g$ & $1$
        \end{tabular}
      \end{center}
      where we note that $g^2 = 1$; otherwise if $g^2 = g$, then we would have $g = 1$ by \autoref{propo:cancellation_laws}, which contradicts the fact that $g \neq 1$. Comparing the above Cayley Table with that of $C_2$, we see that $G = \langle g : g^2 = 1 \rangle \cong C_2$.
    \item $\abs{G} = 3 \implies G = \{1, g, h\}$ with $g \neq 1 \neq h$ and $g \neq h$. We can then start with the following Cayley Table:
      \begin{center}
        \begin{tabular}{c | c c c}
        $G$ & $1$ & $g$ & $h$ \\
        \hline
        $1$ & $1$ & $g$ & $h$ \\
        $g$ & $g$ &     &     \\
        $h$ & $h$ &     &
        \end{tabular}
      \end{center}
      We know that by \autoref{propo:cancellation_laws}, $gh \neq g$ and $gh \neq h$. Thus $gh = 1$. Similarly, we get that $hg = 1$.

      \underline{Claim:} Entries in a row (or column) must be distinct. Suppose not. Then say $g^2 = 1$. But since $gh = 1$, by \autoref{propo:cancellation_laws}, we have that $h = g$, which is a contradiction.

      With that, we can proceed to fill in the rest of the entries: with $g^2 = h$ and $h^2 = g$. Therefore,
      \begin{center}
        \begin{tabular}{c | c c c}
        $G$ & $1$ & $g$ & $h$ \\
        \hline
        $1$ & $1$ & $g$ & $h$ \\
        $g$ & $g$ & $h$ & $1$ \\
        $h$ & $h$ & $1$ & $g$
        \end{tabular}
      \end{center}

      Recall that the Cayley Table for $C_3$ is:
      \begin{center}
        \begin{tabular}{c | c c c}
        $C_3$ & $1$   & $a$   & $a^2$ \\
        \hline
        $1$   & $1$   & $a$   & $a^2$ \\
        $a$   & $a$   & $a^2$ & $1$ \\
        $a^2$ & $a^2$ & $1$   & $a$
        \end{tabular}
      \end{center}
      $\therefore G \cong C_3$ (by identifying $g = a$ and $h = a^2$).

    \item Suppose $G$ is a group of order $4$, i.e. $\abs{G} = 4$. Then, let $G = \{1, g h, f\}$, where $1, g, h, f$ are distinct. We can then draw the following Cayley Table, wherein the blank entries will be discussed.

          \begin{center}
            \begin{tabular}{c|c|c|c|c}
              $G$   & 1 & g & h & f \\
              \hline
              1     & 1 & g & h & f \\
              g     & g &   &   &   \\
              h     & h &   &   &   \\
              f     & f &   &   &
            \end{tabular}
          \end{center}

          We know that $gh \neq h$, otherwise by Right Cancellation, we would have $g = 1$, which is not true since $1$ and $g$ are distinct elements of $G$. Thus, $gh$ is either $f$ or $1$.

          \underline{Case 1: $g^2 = 1$} \\
          If $g^2 = 1$, then $fg = h$. Otherwise, if $fg = f$, we would have $g = 1$ which is a contradiction to the fact that $1$ and $g$ are distinct. Consequently, $hg = f$. Similarly, since $gf = f$ would contradict the fact that $g \neq 1$ through Right Cancellation, we have $gh = f$ and $gf = h$. We now have the following form of the Cayley Table:
          \begin{center}
            \begin{tabular}{c|c|c|c|c}
              $G$   & 1 & g & h & f \\
              \hline
              1     & 1 & g & h & f \\
              g     & g & 1 & f & h \\
              h     & h & f &   &   \\
              f     & f & h &   &
            \end{tabular}
          \end{center}

          Now there are 2 options, either $h^2 = 1$ or $h^2 = g$.

          \underline{Case 1-1: $h^2 = 1$} \\
          If $h^2 = 1$, then through elimination, $hf = g$, $fh = g$ and $f^2 = 1$. We then have the following Cayley Table:

          \begin{center}
            \begin{tabular}{c|c|c|c|c}
              $G$   & 1 & g & h & f \\
              \hline
              1     & 1 & g & h & f \\
              g     & g & 1 & f & h \\
              h     & h & f & 1 & g \\
              f     & f & h & g & 1
            \end{tabular}
          \end{center}

          This is clearly the Cayley Table of the Klein $4$ group.

          \underline{Case 1-2: $h^2 = g$} \\
          If $h^2 = g$, then through elimination, $hf = 1$, $fh = 1$ and $f^2 = g$. We then have the following Cayley Table:

          \begin{center}
            \begin{tabular}{c|c|c|c|c}
              $G$   & 1 & g & h & f \\
              \hline
              1     & 1 & g & h & f \\
              g     & g & 1 & f & h \\
              h     & h & f & g & 1 \\
              f     & f & h & 1 & g
            \end{tabular}
          \end{center}

          We can rearrange the elements and hence the Cayley Table to the following:

          \begin{center}
            \begin{tabular}{c|c|c|c|c}
              $G$   & 1 & f & g & h \\
              \hline
              1     & 1 & f & g & h \\
              f     & f & g & h & 1 \\
              g     & g & h & 1 & f \\
              h     & h & 1 & f & g
            \end{tabular}
          \end{center}

          which is the Cayley Table of $C_4$.

          Now note that the following case will cover for 2 cases, i.e. $g^2 = h$ and $g^2 = f$, since we can proceed with the argument without loss of generality.

          \underline{Case 2: $g^2 = f$} \\
          If $g^2 = f$, we have that $hg = 1$, since we can only have distinct elements in a column and in a row. Consequently, we have $fg = h$. Similarly, we must have that $gh = 1$ and consequently $gf = h$. Thus we have the following Cayley Table:

          \begin{center}
            \begin{tabular}{c|c|c|c|c}
              $G$   & 1 & g & h & f \\
              \hline
              1     & 1 & g & h & f \\
              g     & g & f & 1 & h \\
              h     & h & 1 &   &   \\
              f     & f & h &   &
            \end{tabular}
          \end{center}

          Note that $h^2 \neq g$, because we would then have $fh = f$, which would imply $h = 1$ through Left Cancellation, a contradiction to the fact that $h \neq 1$. Thus $h^2 = g$. Again, since we can only have distinct elements in a row (and a column), we will end up with the following Cayley Table:

          \begin{center}
            \begin{tabular}{c|c|c|c|c}
              $G$   & 1 & g & h & f \\
              \hline
              1     & 1 & g & h & f \\
              g     & g & f & 1 & h \\
              h     & h & 1 & f & g \\
              f     & f & h & g & 1
            \end{tabular}
          \end{center}

          We can rearrange the elements to get the following Cayley Table:

          \begin{center}
            \begin{tabular}{c|c|c|c|c}
              $G$   & 1 & h & f & g \\
              \hline
              1     & 1 & h & f & g \\
              h     & h & f & g & 1 \\
              f     & f & g & 1 & h \\
              g     & g & 1 & h & f
            \end{tabular}
          \end{center}

          in which we observe is the Cayley Table for $C_4$.

          Since we have explored all the possibilities, we have that the only possible groups of order $4$ is the cyclic group $C_4$ and the Klein $4$ group $K_4$.
  \end{enumerate}\qed
\end{proof}

% subsection cayley_tables (end)

% section groups_continued (end)

\section{Subgroups}
\label{sec:subgroups}

\subsection{Subgroups}
\label{sub:subgroups}

\begin{defn}[Subgroup]\label{defn:subgroup}
\index{Subgroup}
  Let $G$ be a group and $H \subseteq G$. If $H$ itself is a group, then we say that $H$ is a subgroup of $G$
\end{defn}

% subsection subgroups (end)

% section subgroups (end)

% chapter lecture_4_may_09_2018 (end)


\chapter{List of Symbols}

\emph{Will be updated.}

	\begin{longtable}{c | l | c}
	\textbf{Symbol(s)} & \textbf{Reading} & \textbf{Definition}\\\hline
	$\in$ & Member, element & 3.1.1\\
	$\emptyset$ & Empty set & 3.1.5\\
	$\{x:\Phi(x)\}$ & Class abstract & 3.1.6\\
	$\subseteq$ & Subset & 3.2.1\\
	$\subset$ & Proper subset & 3.2.3\\
	$\cup$ & Union & 3.4.1\\
	$\cap$ & Intersection & 3.4.2\\
	$\setminus$ & Difference & 3.4.5\\
	$X\times Y$ & Cartesian product & 3.5.3--4\\
	$X^n$ & Cartesian $n$-cube & 3.5.4\\
	$f,g,h, \mathellipsis$ & Functions & 3.6.4\\
	$dom(f)$ & Domain of $f$ \\
	$rg(f)$ & Range of $f$\\
	$\mathbb{N}$ & Natural numbers & 3.7.1\\
	$\neg$ & Negation & 4.1.3\\
	$\land$ & Conjunction \\
	$\lor$ & Disjunction \\
	$\to$ & Conditional \\
	$\leftrightarrow$ & Biconditional \\
	$\mathcal{P}$ & The set of sentence letters & 4.1.5\\
	$\mathcal{L}$ & The set of formulas & 4.1.6\\
	$\phi,\psi,\theta,\mathellipsis$ & Formulas\\
	$T(\phi)$ & The parsing tree of $\phi$ & 4.3.5 \\
	$c(\phi)$ & The complexity of $\phi$ & \\
	$v,u,w,\mathellipsis$ & Valuations & 5.1.2\\
	$\llbracket\phi\rrbracket_v$ & The truth-value of $\phi$ under $v$ & 5.1.9\\
	$\Gamma\vDash\phi$ & $\Gamma$ entails $\phi$ & 5.2.2\\
	\end{longtable}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "../../logic.tex"
%%% End:


\subsection{The trivial group}

The trivial group is the group with just the identity member \(I\).


\section{Examples}
\label{sec:examples}

\paragraph{Decidable set.}
\label{sec:decidable-set}

We now consider the theory of a decidable set.
Recall that in constructive mathematics a set~$S$ is said to be
\emph{decidable} if $x = y$ or $x \neq y$ for all $x, y \in S$.
The input to RZ is
%
{\small\VerbatimInput{decidable.thy}}%
%
\noindent and the output is
%
{\small\VerbatimInput{decidable.mli}}
%
\noindent
The output signature asks for \Verb|decidable| to be a function
accepting two realizers~$x$ and~$y$ and returning one of two tokens
\Verb|`or0| and \Verb|`or1|, depending on whether~$x$ and~$y$ realize
the same element. (Disjunction is written \Verb|cor| to emphasize that
it is classical or.) This is nothing but a computable decision procedure
for equality on \Verb|s| with respect to \Verb|=s=|, as
should be expected.

We remark that nothing requires the partial equivalence relation
\Verb|=s=| to be computable, so not every modest set is decidable. In
fact, there are many natural and important examples of non-computable partial
equivalence relations, such as \emph{extensional equality} of functions
from natural numbers to natural numbers. (If we could computably
decide whether two functions always give equal results on equal
arguments, we could construct a Halting Oracle.)

\paragraph{Natural Numbers.}
\label{sec:natural-numb}

Next we consider the theory of natural numbers. This example shows how
axioms can be parameterized by theories. Recall that the natural
numbers are the initial algebra with one constant and one unary
operation (such algebras are sometimes called ``iteration algebras''):
%
{\small\VerbatimInput{nat.thy}}%
%
\noindent
The theory \Verb|Iteration| is an auxiliary theory. The theory
\Verb|Nat| postulates the existence of a model \Verb|N| of theory
\Verb|Iteration| which satisfies the initiality axiom stating that
there exists exactly one algebra morphism from \Verb|N| to any other
iteration theory \Verb|I|. The output generated by RZ is shown in
Figure~\ref{fig:nat}. The initiality axiom has been translated to a
functor which expects an implementation \Verb|I| of an iteration
theory and outputs a realizer for the axiom. A closer look at the
assertion reveals that it essentially says that the realizer defines a
function from natural numbers to \Verb|I.s| by simple recursion.

\begin{figure*}
  \centering
  {\small\VerbatimInput{nat.mli}}
  \caption{Output for theories \texttt{Iteration} and \texttt{Nat}}
  \label{fig:nat}
\end{figure*}

\paragraph{Axiom of Choice.}
\label{sec:axiom-choice}

As a third example, we look at the realizability interpretation of the
Axiom of Choice. We work with the formulation of the axiom which
states that every total relation has a choice function:
%
\[
  (\forall x \in A.\,
   \exists y \in B.\, R(x, y)) \implies %\\
  \exists g \in B^A.\,
  \forall x \in A.\, R(x, g(x)) \;.
\]
%
We could write this as a theory parameterized by sets $A$, $B$ and the
relation~$R$, but to keep things simple, we use the following version:
%
{\small\VerbatimInput{choice.thy}}%
%
The output is shown in Figure~\ref{fig:choice}. The interesting bit is
the assertion for \Verb|choice|, which says that \Verb|choice| takes
as an argument a realizer $f$ for the $\forall\exists$ statement and
outputs a pair of functions, of which the first is the choice
function~$g$ and the second one provides realizers witnessing that the
choice function does its job. However, there is a problem: the
realizer $f$ is not required to respect \Verb|=a=| while the choice
function $g$ is. In general there is no way for \Verb|choice| to
transform $f$ into a \Verb|=a=|-respecting function. It follows that
in general the Axiom of Choice is \emph{not} valid in the
realizability interpretation. This is another important difference
between realizability and propositions-as-types.

\begin{figure*}
  \centering
  {\small \VerbatimInput{choice.mli}}
  \caption{Output for theory \texttt{Choice}}
  \label{fig:choice}
\end{figure*}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "case"
%%% End:


% \vspace{-.12in}
\paragraph{\textbf{Notation: }}
Unless otherwise specified, lower-case English alphabet characters indicate scalars $x \in \Real$. Bold indicates column vectors $\mb{x} \in \Real^p$,
and upper-case bold indicates matrices, $\bX \in \Real^{p \times q}$.  Parameters and constants are Greek characters.  Time is $t \in [0,T]$,
$i \in [N]$ indexes the $N$ neurons, where $[N]=\{1,2,\ldots,N\}$. Script denotes sets and pipes denote the cardinality of the set, e.g. $|\mc{T}|$.
\vspace{-.1in}

Our overall model is then:
\begin{subequations}
\begin{align}
  \mc{T}_i\ \  &\sim \text{PP}(\lambda_i) \quad i \in \NN, \quad &\text{ where } \mathsf{\Lambda}(\cdot)&=\sum_{i=1}^{\infty} \lambda_i \delta_{\theta^*_i} \sim \Gamma \text{P}(\alpha, \mathsf{H}(\cdot| {\phi})), \\ %\mathcal{NW}(\mu, \Sigma)) \\ \\
\vspace{-.8in}
  x_i(t) &= \sum_{j = 1}^{|\mc{T}_i|}  \sum_{k = 1}^{K} y^*_{ijk} \mathsf{d}_k(t - \tau_{ij}), \quad &\text{ where }\by^*_{ij}  &\sim \mathsf{N}_K(\mb{\mu}^*_i, \Sigma^*_i) \quad i,j \in \NN, \\
  x(t)   &= \sum_{i=1}^{\infty} x_i(t) + \eps_t, \quad &\text{ where at any time $t$, } \eps_t &\sim \mathsf{N}(0,\Sigma_x) \text{ independently}
\end{align}
\end{subequations}

\begin{algorithm}
\caption{Generative mechanism for the multi-electrode, non-stationary, discrete-time process}\label{alg:gen_proc}
\begin{tabular}{p{1.2cm}p{12.4cm}}
Input:&  a) the number of bins $T$, and the bin-width $\Delta$\\
  &  b) the $K$-by-$L$ dictionary $\bD$ of $K$ basis functions\\
  &  c) the DP hyperparameters $\alpha$ and $\phi$.\\
  &  d) the transition matrix $\bB$ of the neuron AR process \\
Output:& \  An $M$-by-$T$ matrix $\bX$ of multielectrode recordings. % defined by a set of state and time pairs.
\end{tabular}
\begin{algorithmic}[1]
\State Initialize the number of clusters $C_1$ to $0$.
\State Draw the overall spiking rate $\Lambda \sim \text{Gamma}(\alpha, 1)$.
%\State Set $A_{s_i}(\tau) = \sum_j A_{s_i,j} (\tau)$ and define $u_{s_i}(\tau) \ge A_{s_i}(\tau) \forall \tau$. \label{alg:loop}
%\State Let $\tau_o = (w_{i} - l_i)$. \label{alg:smjp_loop}
\For{$t$ in $[T] $}
\State Sample $\mt{z}_t \sim \text{Bernoulli}(\Lambda \Delta)$, with $\mt{z}_t = 1$ indicating a spike in bin $t$.
\If{$\mt{z}_t = 1$}   \label{enum:thin}
  \State Sample $\mt{\nu}_t$, assigning the spike to a neuron, with
%\begin{align*}
$  \mathsf{P}({\mt{\nu}_t} = i) \propto
  \begin{cases}
   |\mc{T}^t_i| \quad i \in [C] \\
   \alpha \quad\ i = C + 1 \\
  \end{cases}$
%\end{align*}
       \If{ $\nu_t = C_t + 1$}
          \State  $C_{t+1} \leftarrow C_t + 1$.
		\State Set $\theta^*_{C_{t+1}} \sim H_{\phi}(\cdot)$, and $\mc{T}_{C_{t+1}}$=\{t\}.
       \Else \State  $\mc{T}_{\nu_t} \leftarrow \mc{T}_{\nu_t} \cup \{t\}$.
    \EndIf
\State Set $\theta_t = \theta^*_{\nu_t}$, recalling that $\theta_t \equiv (\mb{\mu}_t, \Sigma_t)$.
\State Sample $\by_t = (\by^1_t; \cdots; \by^M_1) %\equiv (\by_{t1}, \cdots, \by_{t\Upsilon})
           \sim \mathsf{N}(\mb{\mu}_t, \Sigma_t)$, determining the spike shape at all electrodes.
%\State $\bx^h_{t:t+L} = A\by^h$
\State $ x^m_t = \sum_{h = 1}^L \bD_{:,h}^{\T} \by^m_{t-h-1} + \epsilon^m_t \text{\qquad where $\epsilon^m_t \sim \mathsf{N}(0,\sigma^2), m \in [M]$.} $
\State Update the cluster parameters: ${\mb{\mu}}^*_i = \mathbf{B} {\mb{\mu}}^*_i + r_i \quad i \in [C_{t+1}]$
\EndIf
\EndFor
\end{algorithmic}
\end{algorithm}


\documentclass[12pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{float}
\usepackage{amsmath}

\usepackage[hmargin=3cm,vmargin=6.0cm]{geometry}
%\topmargin=0cm
\topmargin=-2cm
\addtolength{\textheight}{6.5cm}
\addtolength{\textwidth}{2.0cm}
%\setlength{\leftmargin}{-5cm}
\setlength{\oddsidemargin}{0.0cm}
\setlength{\evensidemargin}{0.0cm}

\begin{document}

\section*{Student Information }
%Write your full name and id number between the colon and newline
%Put one empty space character after colon and before newline
Full Name : Zeynep Özalp \\
Id Number : 2237691 \\

% Write your answers below the section tags
\section*{Answer 1}
\subsection*{a)}
If $f^{-1}$, $g^{-1}$ and $(gof)^{-1}$ do not exist, there is nothing to prove. So assuming that they exist, let $g\circ f=z$. Then, $z^{-1}=(gof)^{-1}$. Let a, b, c such that
$$(a \in A_0) \wedge (b \in B_0 \wedge f(a)=b) \wedge (c \in C_0 \wedge g(b)=c)$$
$$(A_0 \subseteq A) \wedge (B_0 \subseteq B) \wedge (C_0 \subseteq C)$$
Therefore,
$$z(a)=c$$
$$z^{-1}(c)=a$$
Note that, $g^{-1}(c)=b$ and $f^{-1}(b)=a$. Thus,
$$f^{-1}(g^{-1}(c))=a$$ which is equal to equation $z^{-1}(c)=a$.
$$z^{-1}(c)=f^{-1}(g^{-1}(c))$$
$$(g\circ f)^{-1}(c)=f^{-1}(g^{-1}(c))$$
Since the last equation is true for some arbitrary element of $C_0$, one can conclude that the equation is valid for all elements of $C_0$. Hence,
$$(g\circ f)^{-1}(C_0)=f^{-1}(g^{-1}(C_0))$$

\subsection*{b)}
\subsubsection*{Injectivity of f}
Assume that f is not injective.
$$(f(a_1)=f(a_2)) \wedge (a_1 \neq a_2)$$
Therefore,
$$((g\circ f)(a_1)=(g\circ f)(a_2)) \wedge (a_1 \neq a_2)$$
$$\perp$$
Contradiction since $g\circ f$ is injective. Therefore, our assumption is false and f is injective.
\subsubsection*{Injectivity of g}
When the domain of g is the range of f, g must be injective. However, g may have other other elements in its domain such that those elements are not in the range of f and g may not be injective for those elements. Therefore, g is not necessarily injective. It can be injective or not.

\subsection*{c)}
\subsubsection*{Surjectivity of g}
Assume that there exists some z such that
$$(z \in C) \wedge (z \notin g(B_0)) \ where \ B_0 \subset B$$
Therefore,
$$(z \in C) \wedge (z \notin (g\circ f)(A_0)) \ where \ A_0 \subset A \ and\ f(A_0)=B_0$$
$$\perp$$
Contradiction since $(g\circ f)$ is surjective. Therefore, our assumption is false and f is injective.
\subsubsection*{Surjectivity of f}
\textbf{i)} Assume that f is not surjective so that its range is $B_0$.
$$f(A)=B_0 \subset B$$
$$(g \circ f)(A)=g(B_0)=C$$
This equation is valid and makes no contradiction since we know that $g\circ f$ is surjective.\\
\textbf{ii)} Assume that f is surjective so that its range is $B$.
$$f(A)=B$$
$$(g \circ f)(A)=g(B)=C$$
This equation is valid and makes no contradiction since we know that $g\circ f$ is surjective.\\
Therefore, f is not necessarily surjective. It can be surjective or not.

\section*{Answer 2}

\subsection*{a)}
\subsubsection*{Injectivity of f}
Assume that
$$f(x)=f(y)$$
$$(g\circ f)(x)=(g\circ f)(y)$$
$$x=y\ since\ (g\circ f)(x)=x\ and\ (g\circ f)(y)=y$$
Therefore, f is injective.
\subsubsection*{Surjectivity of f}
For some $b \in B$ the following is true.
$$(f\circ h)(b)=b$$
Thus, every element in of B has a pre-image in A and f is surjective.

\subsection*{b)}
Yes, a function can have more than one left and right inverses. Consider these examples:\\
$f:\{1,2\} \rightarrow \{1\}$ defined by $f(1)=f(2)=1$\\\\
$g_1:\{1\} \rightarrow \{1,2\}$ defined by $g_1(1)=1$ so that $(g_1\circ f)(1)=1$ \\
$g_2:\{1\} \rightarrow \{1,2\}$ defined by $g_2(1)=2$ so that $(g_2\circ f)(2)=2$\\\\
$h_1:\{1\} \rightarrow \{1, 2\}$ defined by $h_1(1)=1$ so that $(f\circ h_1)(1)=1$ \\
$h_2:\{1\} \rightarrow \{1, 2\}$ defined by $h_2(1)=2$ so that $(f\circ h_2)(1)=1$\\\\
Clearly, $g_1$ and $g_2$ are left inverses of f and $h_1$ and $h_2$ are right inverses of f.

\subsection*{c)}
If f has a left inverse $g$ then it is injective and if f has a right inverse $h$ then it is subjective. Since f has both right and left inverses, it is both injective and subjective, i.e., f is bijective.\\
Now, choose an arbitrary $x\in B$.
$$f(h(x))=x$$
Apply g to both sides.
$$g(f(h(x)))=g(x)$$
However, for all $x_0 \in A$, $g(f(x_0))=x_0$. Thus,
$$g(f(h(x)))=h(x)=g(x)$$
Therefore, $g=h=f^{-1}$.

\section*{Answer 3}
\subsection*{Bijectivity of f}
Choose arbitrary $x_1,\ x_2$ in the domain of f and $y_1,\ y_2$ in A such that $f(x_1,y_1)=f(x_2,y_2)$. Therefore,
$$(x_1+y_1-1, y_1)=(x_2+y_2-1, y_2)$$
Thus, $x_1+y_1-1=x_2+y_2-1$ and $y_1=y_2$. Since $x_1+y_1=x_2+y_2$ and $y_1=y_2$, clearly $x_1=x_2$. Thus, f is injective.

$$f(x_1,y_1)=(x_1+y_1-1, y_1)$$
For all positive integers $x_1$ and $y_1$, $y_1\le x_1+y_1-1$. Since for all of the pairs $(x_1+y_1-1, y_1)$ is in A, f is surjective.\\
Function f is bijective.
\subsection*{Bijectivity of g}
Let $h,i: Z^+\rightarrow Z^+$, $h(x)=\dfrac{1}{2}(x^2-x)$ and $i(y)=y$.\\
$$i(y_1)=i(y_2) \wedge (y_1\neq y_2))$$
$$y_1=y_2$$
$$\perp$$
$$h(x_1)=h(x_2) \wedge (x_1 \neq x_2)$$
$$\dfrac{1}{2}(x_1{^2}-x_1)=\dfrac{1}{2}(x_2{^2}-x_2)$$
$$x_1{^2}-x_1=x_2{^2}-x_2$$
$$x_1{^2}-x_2{^2}=x_1-x_2$$
Since $x_1 \neq x_2$, $x_1-x_2\neq 0$, we can divide the equation by $x_1-x_2$.
$$x_1+x_2=1$$
$$\perp$$
Contradiction since $x_1$ and $x_2$ are defined as positive integers. So, $h$ and $i$ is injective.\\

Note that the sum of injective functions may not injective since for some function $k(x)=x$, $l(x)=-x$ and then $(k+l)(x)=0$ is not injective. However, the domain of $g(x,y)$ is A such that $x,y\in Z^+$; therefore, the sum of injective functions is injective on that domain. Thus, $g(x,y)=h(x)+i(y)$ is injective.\\

By Theorem.2 on p.239, $\forall a,d \in Z$, $d>0$, there exists unique $q,r$ such that $a=qd+r$ where $0 \leq r <d$.
$$q=1/2(x-1)$$
$$d=x$$
$$r=y$$
$$g(x,y)=qd+r$$
The function g is surjective. Thus, it is bijective.

\section*{Answer 4}
\subsection*{a)}
WE know that the set of positive rational numbers are countable from ex.4/p.172 in our textbook. Since the mapping $Q^{+} \rightarrow Q^{-}$ is bijective, $Q^-$ is also countable. The union of countable sets $Q^+$, $Q^-$, $\{ 0 \}$ is also countable. So, the set of rational numbers is countable.\\

Let
$$P_n=\{x^n+a_{n-1}x^{n-1}+...+a_1x+a_0\ |\ a_i \in Q,\ i=0,...,n-1\}$$ and
$$Q^n=\{(a_{n-1},...,a_0)\ |\ a_i \in Q,\ i=0,...,n-1\}$$.
Clearly, the mapping $P_n \rightarrow Q^n$ is a bijective from $P_n $ to $Q^n$ and therefore, $P_n $ has the same cardinality with $Q^n$.\\

$Q^n$ is the set of n-tuples where all $a_i \in Q,\ i=0,...,n-1$. The elements of $a_0,...,a_{n-1}$ need not to be distinct, so $Q^n$ is also countable. So is $P_n$. Since the set of all polynomials with rational coefficients is the union of all $P_i$'s where $i=0,...,n$, it is countable. Since each polynomial has finite number of roots, the set of all polynomials with countable roots are also countable. Thus, the set of algebraic real numbers is countable.

\subsection*{b)}
Let $A$ be the set of algebraic real numbers and $T$ is the set of trancendental real numbers. Note that $A \subset R$, $T \subset R$, $A=R-T$ and $A$ is countable, so is $R-T$. Assume that $T$ is countable.
$$T \cup (R-T) = R$$
From our assumption, the union of two countable sets should be countable. Hovewer, the set of real numbers are not countable and our assumption is false. Thus, the set of trancendental real numbers are not countable.

\section*{Answer 5}
For positive $a_1$, $a_2$ where $a_1 < a_2$ and sufficiently large $n$
$$a_1n.ln(n) < k < a_2n.ln(n)$$
$$ln(a_1n.ln(n)) <ln(k) <ln(a_2n.ln(n))$$
Divide the first equation by the second one.
$$\dfrac{a_1n.ln(n)}{ln(a_1n.ln(n))}<\dfrac{k}{ln(k)}<\dfrac{a_1n.ln(n)}{ln(a_2n.ln(n))}$$
$$a_1n.ln(n-a_1n.ln(n)) < \dfrac{k}{ln(k)} < a_2n.ln(n-a_2n.ln(n))$$
$$n-a_1n.ln(n)>0$$ by the definiton of $ln$. Thus,
$$n>a_1n.ln(n)$$
Since $n \neq 0$, divide by $n$.
$$a_1ln(n)<1$$

Assume that $ln(n-a_1n.ln(n))>n$. We can write
$$ln(n-a_1n.ln(n))>a_1n.ln(n)$$
since $n>a_1n.ln(n)$.
Divide by $ln(n) \neq 0$.
$$ln(a_1n.ln(n))>a_1n$$
Let $b_1=a_1ln(n)$ and thus $b_1<1$. Put $b_1$.
$$ln(b_1a_1)>a_1n$$
$$\perp$$
Contradiction since $b_1<1$, $a_1>0$ and n is sufficiently large.
So, our assumption is false and $ln(n-a_1n.ln(n))<n$. The same steps is valid for $a_2$ and $b_2=a_2ln(n)$. Therefore,
$$\Theta (\dfrac{k}{ln(k)}) = n$$

\section*{Answer 6}
\subsection*{a)}
$$6=1+2+3$$
$$28=1+2+4+7+14$$
\subsection*{b)}
Suppose $k=2^p-1$ is prime and let $n=2^{p-1}(2^p-1)$.\\
Let $f(n)$ be the sum of the positive divisors of $n$. Note that:\\
1. When $n$ is perfect, $f(n)=2n$.\\
2. When $n$ is prime, $f(n)=n+1$.\\
3. $f(2^a)=1+2^1+2^2+...+2^{a-1}=2^a-1$\\
4. $f(n)=f(p_1^{a_1})f(p_2^{a_2})...f(p_n^{a_n})$ where $p_i$'s are prime divisors of $n$ and $a_i$'s are the powers of them. So, $f(mn)=f(m)f(n)$ where $m$ and $n$ are relatively prime.\\

Since $2^p-1$ is prime, $2^p-1$ and $2^{p-1}$ are relatively prime. Therefore,
$$f(n)=f(2^{p-1})f(2^p-1)=(2^p-1)2^p=2n$$
Thus, $n=2^{p-1}(2^p-1)$ is perfect when $2^p-1$ is prime.

\section*{Answer 7}
\subsection*{a)}
By the definition,
$$x=nd_2+c_2=md_1+c_1$$
$$nd_2-md_1=c_1-c_2$$
Let $gcd(m,n)=g$. Take $(mod\ g)$ of both sides.
$$0 \equiv c_2-c_1(mod\ g)$$
Therefore, $gcd(m,n)\ |\ c_2-c_1$.
\subsection*{b)}
Let $d=gcd(m,n)$. By Bezout's Theorem on p.269 in our textbook, there exist $s,t\in Z$ such that $d=sm+tn$. Let $(q_1,r)$ and $(q_2,r)$ be the quotient and remainder of $c_1$ and $c_2$ upon division by $d$. Then $x=q_1sm+q_2tn+r$ is a unique modulo $lcm(m,n)$.

Assume that there is two distinct $x_1>0,\ x_2>0$ and $m,\ n$ are relatively prime. Therefore, $lcm(m,n)=mn$.
$$x_1-x_2=0$$
$$x_1-x_2\equiv 0(mod\ m)\equiv 0(mod\ n)$$
$$x_1-x_2\equiv 0(mod\ mn)$$
Since $x_1-x_2$ has no remainder, $x_1-x_2\geq lcm(mn)$. Thus, there is no distict $x_1>0,\ x_2>0$ in the interval $[0,\ lcm(m,n))$.\\
Now, suppose that $m,\ n$ are not relatively prime. Let $m=ay$ and $n=by$ then $lcm(m,n)=aby$. Since $m,n$ are not relatively prime, $c_1=c_2$.
$$x_1 \equiv c_1 (mod\ ay)\equiv c_1(mod\ by)$$
$$x_2 \equiv c_1 (mod\ ay)\equiv c_1(mod\ by)$$
Therefore,
$$x_1-x_2= ay(d_1-d_3)$$
$$or$$
$$x_1-x_2= by(d_2-d_4)$$

Clearly, $x_1$ and $x_2$ are divisable by both $ay$ and $by$. Thus,
$$x_1-x_2\equiv 0 (mod\ aby)$$
Since $x_1-x_2$ has no remainder, $x_1-x_2\geq lcm(mn)$. Thus, there is no distict $x_1>0,\ x_2>0$ in the interval $[0,\ lcm(m,n))$.\\

\end{document}