\section*{\href{https://sicp.comp.nus.edu.sg/chapters/3}{Numbers}}

We use decimal notation for numbers, with an optional decimal dot. ``Scientific notation''
(multiplying the number with $10^x$) is indicated with the letter \texttt{e}, followed
by the exponent $x$.
Examples for numbers are \texttt{5432}, \texttt{-5432.109}, and \texttt{-43.21e-45}.


\subsection{Na\"ive set theory}\label{subsec:naive_set_theory}

Na\"ive set theory is traditionally defined informally by only specifying that a set is an unordered collection of objects without repetition. It turns out that this can easily be formalized as a \hyperref[def:first_order_theory]{first-order theory}, albeit an inconsistent one. Still, this theory is useful for introducing important concepts that can ease the introduction of more elaborate theories like \hyperref[def:zfc]{\logic{ZFC}}. The definitions we introduce and the proofs we provide will turn out to be valid in \logic{ZFC} also. In other words, we will transparently utilize \hyperref[def:naive_set_theory/unrestricted_comprehension]{unrestricted comprehension} for constructing sets and later in \fullref{thm:zfc_existence_theorems} we will prove that they exist not only in na\"ive set theory, but also in \logic{ZFC}.

\begin{remark}\label{rem:pure_set_theory}
  What we lose in this formalization are objects which are not sets, usually called \term{atoms} or \term{urelements} (because of the German prefix \enquote{ur}, meaning primordial). It is not necessary for us to add such elements since we can encode everything via sets. Theories without atoms, like our versions of nai\"ve set theory and \hyperref[def:axiom_of_universes]{\logic{ZFC+U}}, are called \term{pure set theories}.
\end{remark}

\begin{definition}\label{def:naive_set_theory}
  The \term{language of na\"ive set theory} is a \hyperref[def:first_order_syntax]{first-order language} \( \mscrL \) with only a single \hyperref[rem:first_order_formula_conventions/infix]{infix} binary relation \( \in \) called \term{set membership}. If \( \xi \in \eta \), we say that \( \xi \) is a \term{member} or \term{element} of \( \eta \) and that \( \eta \) \term{contains} \( \xi \).

  For the sake of simplicity, we will not introduce into the language any other functional or predicate symbols, but will use \hyperref[rem:predicate_formula]{predicate formulas} when needed mostly for formulating axioms. See the \( \ref{eq:def:grothendieck_universe/predicate}[\tau] \) predicate for an extreme example.

  \term{Na\"ive set theory} is a \hyperref[def:first_order_theory]{first-order theory} axiomatized by the following:
  \begin{thmenum}
    \thmitem{def:naive_set_theory/extensionality}\mcite[sec. 61.1]{OpenLogicFull} The \term{axiom of extensionality}, which states that two sets are equal if and only if they have the same members. Symbolically,
    \begin{equation}\label{eq:def:naive_set_theory/extensionality}
      \parens[\Big]{ \qforall \xi (\xi \in \tau \leftrightarrow \xi \in \sigma) } \rightarrow \parens[\Big]{ \tau \doteq \sigma }.
    \end{equation}

    As a consequence, a set is only distinguished by what it contains and thus the ordering and repetition of members of a set play no role. This axiom is also important in \logic{ZFC} --- see \fullref{def:zfc/extensionality}.

    It is very common when dealing with sets, as in \eqref{eq:def:naive_set_theory/extensionality}, to use \hyperref[rem:first_order_formula_conventions/relativization]{relativization of quantifiers} with \( \in \).

    As explained in \fullref{rem:mathematical_logic_conventions/quantification}, we avoid excessive universal quantification. We actually add as an axiom of the theory the \hyperref[thm:implicit_universal_quantification]{universal closure} of \eqref{eq:def:naive_set_theory/extensionality}:
    \begin{equation}\label{eq:def:naive_set_theory/extensionality_quantified}
      \qforall \tau \qforall \sigma \parens[\Bigg]{ \parens[\Big]{ \qforall \xi (\xi \in \tau \leftrightarrow \xi \in \sigma) } \rightarrow \parens[\Big]{ \tau \doteq \sigma } }.
    \end{equation}

    The \hyperref[def:material_implication/converse]{converse} of \eqref{eq:def:naive_set_theory/extensionality} obvious.

    \thmitem{def:naive_set_theory/unrestricted_comprehension} The \term{axiom schema of unrestricted comprehension} states that any formula defines a set. For each formula \( \varphi \) not containing \( \tau \) as a free variable, the following is an axiom:
    \begin{equation}\label{eq:def:naive_set_theory/unrestricted_comprehension}
      \qexists \tau \qforall \xi (\xi \in \tau \leftrightarrow \varphi).
    \end{equation}

    It is important to highlight that \( \varphi \) may have any number of free variables as long as \( \tau \) is not among them. Of course, this axiom is only interesting if \( \xi \in \boldop{Free}(\varphi) \). If \( \eta_1, \ldots, \eta_n \) are all the other free variables of \( \varphi \), then the \hyperref[thm:implicit_universal_quantification]{universal closure} of the corresponding axiom is
    \begin{equation}\label{eq:def:naive_set_theory/unrestricted_comprehension_quantified}
      \qforall {\eta_1} \cdots \qforall {\eta_n} \qexists \tau \qforall \xi (\xi \in \tau \leftrightarrow \varphi).
    \end{equation}

    In other words, the set \( \tau \) is not unique in general, but depends on the free variables \( \eta_1, \ldots, \eta_n \). For this reason, they are called \term{parameters} of the axiom.

    Compare this axiom schema to \hyperref[def:zfc/specification]{restricted comprehension}. In the context of na\"ive set theory they are equivalent because each is a special case of the other one.

    Because our goal is for all our constructions to be valid in \hyperref[def:zfc]{\logic{ZFC}}, we only use unrestricted comprehension where the existence of the set is justified by other axioms of \logic{ZFC}.
  \end{thmenum}
\end{definition}

\begin{remark}\label{rem:epsilon_and_set_membership}
  The symbol \( \in \) is derived from \( \varepsilon \). Some older books like \cite{Kelley1955} even use \( \varepsilon \) for set membership. \Fullref{thm:epsilon_induction} is named after set membership.
\end{remark}

\begin{definition}\label{def:set}
  Assume that we have a fixed \hyperref[rem:standard_model_of_set_theory]{standard} \hyperref[rem:transitive_model_of_set_theory]{transitive} model \( \mscrV = (V, I) \) of \hyperref[def:naive_set_theory]{na\"ive set theory} or \hyperref[def:zfc]{\logic{ZFC}}, with or without the \hyperref[def:axiom_of_universes]{axiom of universes}. We will assume \logic{ZFC+U} by default.

  We say that any member of \( V \) is a \term{set}. If \( A \) is a set and \( x \in A \), we say that \( x \) is a \term{member} or \term{element} of \( A \) or, in accordance with \fullref{def:point}, a \term{point} in \( A \).

  We usually refer to \( V \) as our \term{universe} or \term{universal set}. When working with \hyperref[def:grothendieck_universe]{Grothendieck universes}, we may wish to further distinguish between the universal set and some Grothendieck universe. Fortunately, we very rarely refer to \( V \) itself.
\end{definition}

\begin{remark}\label{rem:standard_model_of_set_theory}
  We will say that a model \( \mscrV = (V, I) \) of set theory is \term{standard} if the interpretation \( I(\in) \) of the membership predicate symbol is precisely the membership relation in the metatheory. We will only consider standard models of set theory. This is immensely important for the following reasons:

  \begin{itemize}
    \thmitem{rem:standard_model_of_set_theory/set_builder_notation} Set-builder notation relies on constructing sets in the metatheory and then using them in the object theory. If the model is not standard, then it does not hold that \( (\xi \in \eta)\Bracks{\xi \mapsto x, \eta \mapsto y} = T \) if and only if \( x \in y \).

    \thmitem{rem:standard_model_of_set_theory/skolems_paradox} It is possible that cardinality is incompatible between the object theory and metatheory --- see \fullref{ex:skolems_paradox}. We want to avoid countable sets in the metatheory to be uncountable in the object theory, for example.
  \end{itemize}

  Therefore, it is reasonable to assume that all our models of set theory are standard. We also want their domains to be transitive sets -- see \fullref{rem:transitive_model_of_set_theory}.
\end{remark}

\begin{definition}\label{def:set_builder_notation}
  As mentioned in \fullref{rem:set_definition_recursion}, set theory somewhat blurs the line between logic and metalogic. In particular some \hyperref[def:first_order_definability]{definable} subsets of the universe \( U \) of the fixed model \( \mscrU \) are themselves sets within the object logic.

  Fix a formula \( \varphi \) whose free variables are \( \xi \) and \( \eta_1, \ldots, \eta_n \). In the simplest case, \( n = 0 \) and \( \xi \) is the only free variable of \( \varphi \).

  Also fix an \( n \)-tuple \( u_1, \ldots, u_n \) of members of \( U \), which we will call \term{parameters}. Denote by \( A \) the subset of \( U \) consisting of members \( x \) of \( U \) such that \( \varphi\Bracks{x, u_1, \ldots, u_n} = T \).

  We introduce a special convenience notation for \( A \) called \term{set-builder notation}:
  \begin{equation*}
    A \coloneqq \set{ x \given \varphi\Bracks{x, u_1, \ldots, u_n} }.
  \end{equation*}

  Since set-builder notation is metalogical, we do not impose strict syntax rules and use prose where it is straightforward to translate it into a logical formula.

  For example, the \hyperref[def:basic_set_operations/intersection]{intersection} of the sets \( B \) and \( C \) is given by the formula \( \xi \in \eta \wedge \xi \in \zeta \), where \( B \) is a value for the parameter \( \eta \) and \( C \) is a parameter for \( \zeta \). The intersection can thus be written as
  \begin{equation*}
    A \coloneqq \set{ x \given x \in B \T{and} x \in C }.
  \end{equation*}

  Note that at this point \( A \) is a set within the metatheory and its members are sets within the object logic, however \( A \) may not be a set within the object logic and its members may not be sets within the metatheory.

  Nonetheless, within na\"ive set theory, as a consequence of the \hyperref[def:naive_set_theory/unrestricted_comprehension]{axiom schema of unrestricted comprehension}, \( A \) is also a set within the object logic. More precisely, given our choice of parameters \( u_1, \ldots, u_n \), the axiom schema instance \eqref{eq:def:naive_set_theory/unrestricted_comprehension_quantified} guarantees the existence of a member \( a \) of \( U \), such that
  \begin{equation*}
    \parens{ \xi \in \tau }\Bracks{ \tau \mapsto a, \xi \mapsto x } = T
    \T{if and only if}
    x \isinE A,
  \end{equation*}
  where we have denoted set membership within the object logic by \( \in \) and within the metatheory by \( \isinE \). We will not further use this symbol and the two membership relations will be used interchangeably.

  This is where the line between logic and metalogic blurs --- we can speak about roughly the same sets within the object logic and the metatheory.

  Examples such as \fullref{thm:russels_paradox} show that unrestricted comprehension can easily lead to an inconsistent object logic. In more elaborate set theories like \hyperref[def:zfc]{\logic{ZFC}}, we only allow restricted comprehension via the \hyperref[def:zfc/specification]{axiom schema of specification}. Instead of defining \( A \) as a set of all members of \( U \) satisfying a certain property, restricted comprehension allows us to define \( A \) as a subset not of the universe \( U \), but of some well-behaved subset \( B \) of \( U \). The corresponding notation is
  \begin{equation*}
    \set{ x \in B \given \varphi\Bracks{x, u_1, \ldots, u_n} }.
  \end{equation*}

  Of course, we may still use unrestricted comprehension of the result is guaranteed to be a set within the object logic.

  Within \logic{ZFC}, subsets of \( U \) which are not sets in the object logic are called \term{proper classes}. Sets and proper classes are collectively called \term{classes}. We avoid referencing proper classes because that can easily lead us to an inconsistent theory. See \fullref{def:large_and_small_sets} for a clever workaround.

  A bigger problem that may happen is described in \fullref{rem:transitive_model_of_set_theory}.

  Other liberties regarding set-builder notation include the following:
  \begin{itemize}
    \item We often place arbitrary terms on the left side rather than only sets. This is simply a convenient metalogical notation; the symbols that are used in these terms are often not part of the object language. For example, we write the odd integers as
    \begin{equation*}
      \set{ 2n + 1 \given n \in \BbbZ }.
    \end{equation*}

    \item Instead of using the delimiter \( \given \), we sometimes also use \( : \), especially when dealing with absolute values and norms:
    \begin{equation*}
      \set{ \abs{x} : \abs{x}^2 < 1 }
    \end{equation*}
    can be more readable than
    \begin{equation*}
      \set{ \abs{x} \given \abs{x}^2 < 1 }.
    \end{equation*}

    \item If a set has only a small finite amount of members, we usually prefer to enumerate them as
    \begin{equation*}
      \set{ 1, 3, 9, 27 }.
    \end{equation*}

    Because of the \hyperref[def:naive_set_theory/extensionality]{axiom of extensionality}, the order and repetition of the objects inside the curly braces are irrelevant. Nevertheless, using any unconventional order does not benefit us in any way.

    \item We can also place an ellipsis if a certain pattern is obvious:
    \begin{equation*}
      \set{ 1, 3, 9, 27, \ldots }.
    \end{equation*}

    This works specifically for defining countable sets.
  \end{itemize}

  Note that we have used certain numbers, but this was only for illustrative purposes because even the \hyperref[def:set_of_natural_numbers]{natural numbers} are not yet defined in terms of sets.
\end{definition}

\begin{remark}\label{rem:multile_set_membership_shorthand}
  Within the metatheory, we often use the notation \( x_1, \ldots, x_n \in A \) to mean that \( x_k \in A \) for \( k \in 1, \ldots, n \).
\end{remark}

\begin{remark}\label{rem:singleton_sets}
  Sets with a single elements are usually called \term{singletons}. It is sometimes convenient, especially with connection to geometry or \hyperref[def:multi_valued_function]{multi-valued functions} (e.g. when dealing with \hyperref[def:net_convergence/limit]{limits of nets} or \hyperref[def:subdifferentials]{subdifferentials}), to not distinguish between singleton sets and their corresponding element.
\end{remark}

\begin{definition}\label{def:empty_set}
  A very important set is the \term{empty set}
  \begin{equation*}
    \varnothing \coloneqq \set{ x \given \bot },
  \end{equation*}
  which contains no elements. We will also find useful the \hyperref[rem:predicate_formula]{predicate formula}
  \begin{equation*}\taglabel[\op{IsEmpty}]{eq:def:empty_set/predicate}
    \ref{eq:def:empty_set/predicate}[\tau] \coloneqq \qforall \eta \neg \eta \in \tau.
  \end{equation*}

  We will often refer to \term{nonempty sets}, which are exactly what they, sound --- sets that are not the empty set.
\end{definition}

\begin{theorem}[Russell's paradox]\label{thm:russels_paradox}
  \hyperref[def:naive_set_theory]{Na\"ive set theory} is \hyperref[def:first_order_theory_consistency]{inconsistent}. More precisely, the instance of the \hyperref[def:naive_set_theory/unrestricted_comprehension]{schema of unrestricted comprehension} with
  \begin{equation}\label{eq:thm:russels_paradox_comprehension_formula}
    \varphi = (\xi \not\in \xi)
  \end{equation}
  allows us to derive \( \bot \) in \hyperref[def:classical_logic]{classical logic}.

  Thus, the set
  \begin{equation}\label{eq:thm:russels_paradox_set}
    R \coloneqq \set{ x \given x \not\in x }
  \end{equation}
  of all sets that do not contain themselves is not well-defined. Indeed, from \( R \not\in R \) it follows that \( R \in R \) and from \( R \in R \) it follows that \( R \not\in R \).
\end{theorem}
\begin{proof}
  After substituting \eqref{eq:thm:russels_paradox_comprehension_formula} in \eqref{eq:def:naive_set_theory/unrestricted_comprehension}, we obtain the following axiom of na\"ive set theory:
  \begin{equation}\label{eq:thm:russels_paradox_comprehension_axiom}
    \psi \coloneqq \qexists \tau \qforall \xi (\xi \in \tau \leftrightarrow \neg (\xi \in \xi)).
  \end{equation}

  We will show that the negation \( \neg\psi \) of \( \psi \) is also derivable in this theory. An explicit form of the negation can be obtained by utilizing the equivalences \fullref{thm:first_order_quantifiers_are_dual} and \fullref{thm:boolean_equivalences/biconditional_negation}:
  \begin{equation*}
    \neg\psi = \qforall \tau \qexists \xi (\xi \in \tau \leftrightarrow \xi \in \xi).
  \end{equation*}

  This holds when \( \xi \) and \( \tau \) take on the same value, hence it is satisfiable in na\"ive set theory. By \fullref{thm:classical_first_order_logic_is_sound_and_complete}, \( \neg\psi \) is derivable in the theory.

  Thus, \( \psi \) and \( \neg\psi \) are both derivable in the same theory, and we can use \eqref{eq:def:minimal_propositional_natural_deduction_system/neg/elim} to also derive \( \bot \), which shows that na\"ive set theory is inconsistent.
\end{proof}

\begin{definition}\label{def:subset}
  We say that \( A \) is a \term{subset} of \( B \) and write \( A \subseteq B \) if every member of \( A \) is a member of \( B \). If \( A \) is a subset of \( B \), we say that B is a \term{superset} of \( A \).

  If \( A \subseteq B \) and \( A \neq B \), we say that \( A \) is a \term{proper subset} of \( B \) and write \( A \subsetneq B \).

  The relation \( \subseteq \) is called the inclusion relation, and it gives a partial ordering between sets. See \fullref{thm:boolean_algebra_of_subsets}. If an entire family of sets are not pairwise comparable, we say that they are \term{disjoint}.

  The following \hyperref[rem:predicate_formula]{predicate formula}
  \begin{equation*}\taglabel[\op{IsSubset}]{eq:def:subset/predicate}
    \ref{eq:def:subset/predicate}[\rho, \tau] \coloneqq \qforall \xi (\xi \in \rho \rightarrow \xi \in \tau),
  \end{equation*}
  which is valid when \( \rho \) is a subset of \( \tau \), will occasionally be useful for us. We have chosen the letter \( \rho \) because it is the result that would be on the right-hand side if we had a corresponding infix relation in the language.
\end{definition}

\begin{remark}\label{rem:subset_notation}
  Some authors, such as \cite{Kelley1955}, use the notation \( A \subset B \) to mean \enquote{all elements of \( A \) belong to \( B \)}, even in the case when \( A = B \). To avoid confusion, we use the notations \( A \subseteq B \) and \( A \subsetneq B \) (see \fullref{def:subset}).
\end{remark}

\begin{remark}\label{rem:family_of_sets}
  In a \hyperref[rem:pure_set_theory]{pure set theory}, everything is encoded as a set. However, it is often the case that we are not interested in how a set's elements are encoded as sets and only in how they behave, e.g. when working with \hyperref[def:set_of_natural_numbers]{natural numbers}, we are interested in the elements of \( \BbbN \) and not in the way every element of \( \BbbN \) is encoded as a set.

  In order to reduce repetitiveness, sets whose elements we consider to be other sets, are often called \term{families of sets}. In particular, if all (different) sets are \hyperref[def:subset]{disjoint}, we say that the family is a \term{disjoint family}. It is usually assumed that the sets are nonempty.

  We often consider \hyperref[def:cartesian_product/indexed_family]{indexed families}, i.e. sets which depend on a parameter, which further highlight our intention to distinguish between a point in a set, the set itself and some family of sets to which the latter belongs.
\end{remark}

\begin{definition}\label{def:basic_set_operations}
  We define the following operations:

  \begin{thmenum}
    \thmitem{def:basic_set_operations/union} Dually to \hyperref[def:basic_set_operations/intersection]{intersections}, the \term{union} of an arbitrary set \( \mscrA \) is defined as
    \begin{equation*}
      \bigcup A \coloneqq \set{ x \given \qexists {A \in \mscrA} x \in A }.
    \end{equation*}

    We define the \hyperref[rem:predicate_formula]{predicate formula}
    \begin{equation*}\taglabel[\op{IsUnion}]{eq:def:basic_set_operations/union/predicate}
      \ref{eq:def:basic_set_operations/union/predicate}[\rho, \tau] \coloneqq \qforall \xi \parens[\Big]{ \xi \in \rho \leftrightarrow \qexists {\eta \in \tau} \xi \in \rho }.
    \end{equation*}

    In particular, \( \bigcup \varnothing = \varnothing \).

    For two sets \( A \) and \( B \), we define the \term{binary union} as
    \begin{equation*}
      A \cup B \coloneqq \bigcup \set{ A, B } = \set{ x \given x \in A \T{or} x \in B }.
    \end{equation*}

    \thmitem{def:basic_set_operations/intersection} The \term{intersection} of a nonempty set \( \mscrA \) is
    \begin{equation*}
      \bigcap \mscrA \coloneqq \set{ x \given \qforall {A \in \mscrA} x \in A }.
    \end{equation*}

    We also introduce the \hyperref[rem:predicate_formula]{predicate formula}
    \begin{equation*}\taglabel[\op{IsIntersection}]{eq:def:basic_set_operations/intersection/predicate}
      \ref{eq:def:basic_set_operations/intersection/predicate}[\rho, \tau] \coloneqq \qforall \xi \parens[\Big]{ \xi \in \rho \leftrightarrow \qforall {\eta \in \tau} \xi \in \eta }.
    \end{equation*}

    We leave \( \bigcap \varnothing \) undefined because it should be a \hyperref[def:partially_ordered_set_extremal_points/top_and_bottom]{top element} in the \hyperref[thm:boolean_algebra_of_subsets]{Boolean algebra of all sets}, but the latter object is an ambiguous object and does not even exist in \logic{ZFC} --- see \fullref{thm:zfc_existence_theorems/universe}. It does nonetheless satisfy \( \ref{eq:def:basic_set_operations/intersection/predicate}[\rho, \tau] \).

    For two sets \( A \) and \( B \), we define the \term{binary intersection} as
    \begin{equation*}
      A \cap B \coloneqq \bigcap \set{ A, B } = \set{ x \given x \in A \T{and} x \in B }.
    \end{equation*}

    \thmitem{def:basic_set_operations/difference} The \term{difference} of the sets \( A \) and \( B \) is
    \begin{equation*}
      A \setminus B \coloneqq \set{ x \in A \given x \not\in B }.
    \end{equation*}

    We define the \hyperref[rem:predicate_formula]{predicate formula}
    \begin{equation*}\taglabel[\op{IsDifference}]{eq:def:basic_set_operations/difference/predicate}
      \ref{eq:def:basic_set_operations/difference/predicate}[\rho, \tau, \sigma] \coloneqq \qforall \xi \parens[\Big]{ \xi \in \rho \leftrightarrow \xi \in \tau \wedge \neg(\xi \in \sigma) }.
    \end{equation*}

    \thmitem{def:basic_set_operations/power_set} The \term{power set} \( \pow(A) \) of \( A \) is the family of all subsets of \( A \). Symbolically,
    \begin{equation*}
      \pow(A) \coloneqq \set{ B \given B \subseteq A }.
    \end{equation*}

    The operation \( \pow \) is not technically a function since its domain is supposed to be the set of all sets, whose existence contradicts \fullref{thm:russels_paradox}. Nevertheless, this notation makes sense and is justified by \fullref{rem:unbounded_transfinite_recursion} and \fullref{ex:unary_functors_in_set}.

    We define the \hyperref[rem:predicate_formula]{predicate formula}
    \begin{equation*}\taglabel[\op{IsPowerSet}]{eq:def:basic_set_operations/power_set/predicate}
      \ref{eq:def:basic_set_operations/power_set/predicate}[\rho, \tau] \coloneqq \qforall \xi \parens[\Big]{ \xi \in \rho \leftrightarrow \ref{eq:def:subset/predicate}[\xi, \tau] }.
    \end{equation*}

    See \fullref{thm:power_set_via_subsets} for a characterization of the power set.
  \end{thmenum}
\end{definition}

\begin{proposition}\label{thm:def:set_difference/properties}
  \hyperref[def:basic_set_operations/difference]{Set difference} has the following basic properties:
  \begin{thmenum}
    \thmitem{thm:def:set_difference/properties/intersection} If \( A \) and \( B \) are subsets of \( C \), then \( A \setminus B = A \cap (C \setminus B) \).

    \thmitem{thm:def:set_difference/properties/double_difference} If \( A \subseteq B \), then \( B \setminus (B \setminus A) = A \)
  \end{thmenum}
\end{proposition}
\begin{proof}
  \SubProofOf{thm:def:set_difference/properties/intersection} Since \( a \in A \) implies \( a \in C \), we have
  \begin{align*}
    A \setminus B
    &=
    \set{ x \in A \given x \not\in B }
    = \\ &=
    \set{ x \in A \given x \in C \T{and} x \not\in B }
    = \\ &=
    A \cap (C \setminus B).
  \end{align*}

  \SubProofOf{thm:def:set_difference/properties/double_difference} By \hyperref[thm:minimal_propositional_negation_laws/dne]{double negation elimination},
  \begin{align*}
    B \setminus (B \setminus A)
    &=
    \set[\Big]{ x \in B \given x \not\in \set{ x \in B \given x \not\in A } }
    = \\ &=
    \set{ x \in B \given x \in A }
    = \\ &=
    A.
  \end{align*}
\end{proof}

\begin{proposition}\label{thm:boolean_algebra_of_subsets}
  Let \( X \) be an arbitrary set. Then the \hyperref[def:basic_set_operations/power_set]{power set} \( \pow(X) \) endowed with the \hyperref[def:subset]{inclusion} partial order \( \subseteq \) is a \hyperref[def:semilattice/complete]{complete} \hyperref[def:boolean_algebra]{Boolean algebra}. Explicitly:

  \begin{thmenum}
    \thmitem{thm:boolean_algebra_of_subsets/join} The \hyperref[def:semilattice/join]{join} of an arbitrary family \( \mscrA \) of subsets of \( X \) is simply the \hyperref[def:basic_set_operations/union]{union} \( \bigcap \mscrA \).

    \thmitem{thm:boolean_algebra_of_subsets/top} The \hyperref[def:partially_ordered_set_extremal_points/top_and_bottom]{top element} is the set \( X \) itself.

    \thmitem{thm:boolean_algebra_of_subsets/meet} The \hyperref[def:semilattice/meet]{meet} of an arbitrary family \( \mscrA \) of sets is simply the \hyperref[def:basic_set_operations/intersection]{intersection} \( \bigcup \mscrA \). Unlike for a general family of sets, we have no problem defining the intersection of an empty set to be the top element \( X \).

    \thmitem{thm:boolean_algebra_of_subsets/bottom} The \hyperref[def:partially_ordered_set_extremal_points/top_and_bottom]{bottom element} is the empty set.

    \thmitem{thm:boolean_algebra_of_subsets/complement} The \hyperref[def:boolean_algebra]{complement} \( A^\complement \) of the subset \( A \) is the \hyperref[def:basic_set_operations/difference]{difference} \( X \setminus A \).
  \end{thmenum}

  \begin{figure}
    \hfill
    \includegraphics[page=1]{output/thm__boolean_algebra_of_subsets.pdf}
    \hfill\hfill
    \caption{The \hyperref[def:hasse_diagram]{Hasse diagram} of \( \pow(\set{ A, B }) \) with respect to \hyperref[def:subset]{set inclusion}}
    \label{fig:thm:boolean_algebra_of_subsets}
  \end{figure}
\end{proposition}
\begin{proof}
  \SubProofOf{thm:boolean_algebra_of_subsets/join} The union of \( \mscrA \) exists by \fullref{thm:zfc_existence_theorems/arbitrary_union}, and it is itself a subset of \( \mscrA \). Every set in \( \mscrA \) is contained in \( \bigcup \mscrA \), hence it is indeed a join.

  \SubProofOf{thm:boolean_algebra_of_subsets/top} Clearly \( X \) contains every subset of \( X \).

  \SubProofOf{thm:boolean_algebra_of_subsets/join} The intersection of \( \mscrA \) exists by \fullref{thm:zfc_existence_theorems/arbitrary_union}, and it is itself a subset of \( \mscrA \). Every set in \( \mscrA \) contains \( \bigcap \mscrA \), hence it is indeed a meet.

  \SubProofOf{thm:boolean_algebra_of_subsets/bottom} The empty set is contained in every set, in particular in every subset of \( A \).

  \SubProofOf{thm:boolean_algebra_of_subsets/complement} The operation \( A^\complement \) is well-defined for each subset \( A \) of \( X \) due to \fullref{thm:zfc_existence_theorems/difference}.

  By definition
  \begin{equation*}
    A \vee A^\complement
    =
    A \cup (X \setminus A)
    =
    X
  \end{equation*}
  and
  \begin{equation*}
    A \vee A^\complement
    =
    A \cup (X \setminus A)
    =
    X,
  \end{equation*}
  hence \( A^\complement \) is indeed the complement of \( A \).

  Therefore, \( \pow(X) \) is a Boolean algebra.
\end{proof}

\begin{remark}\label{rem:inductive_sets}
  Induction is an important proof technique that is discussed in detail in the proof of \fullref{thm:nonzero_natural_numbers_have_predecessors}. There are more general forms of induction than \eqref{eq:def:peano_arithmetic/PA3} like \fullref{thm:well_founded_induction} and \fullref{thm:well_founded_induction}. They do, however, require concepts which in turn depend on the existence of natural numbers within set theory. As a consequence, we cannot prove \eqref{eq:def:peano_arithmetic/PA3} via \fullref{thm:well_founded_induction}.

  We will introduce the concept of inductive sets in \fullref{def:inductive_set} and prove in \fullref{thm:omega_induction} that a special inductive set \hyperref[thm:smallest_inductive_set_existence]{\( \omega \)}, which will be the domain of our model of \( \BbbN \), allows performing inductive proofs. The technique that allows us to perform inductive proofs on \( \omega \) can be seen in the proof of \fullref{thm:omega_is_transitive}. \Fullref{thm:omega_induction} will allow us to define natural numbers without relying on metalogical induction along the way. See the proof of \fullref{thm:omega_induction} for a description of natural number induction within set theory and \fullref{rem:standard_models_of_arithmetic} for a further discussion of the use of natural numbers in the metatheory and in the object logic.

  We also introduce \term{recursion} in parallel as a technique for constructing objects. See \fullref{thm:omega_recursion}.
\end{remark}

\begin{definition}\label{def:ordinal_successor}
  The \term{successor} \( \op{succ}(A) \) of a set \( A \) is the set
  \begin{equation*}
    \op{succ}(A) \coloneqq A \cup \set{ A }.
  \end{equation*}

  It is also called the \term{ordinal successor} operation since it is an important concept in the \hyperref[subsec:ordinals]{theory of ordinals}. See \fullref{rem:def:ordinal_successor} for an example of how it naturally arises. It should be distinguished from \term{successor cardinals} defined in \fullref{def:successor_and_limit_cardinal}.

  The following \hyperref[rem:predicate_formula]{predicate formula}
  \begin{equation*}\taglabel[\op{IsSucc}]{eq:def:ordinal_successor/predicate}
    \ref{eq:def:ordinal_successor/predicate}[\rho, \tau] \coloneqq \qforall \xi \parens[\Big]{ \xi \in \rho \leftrightarrow (\xi \in \tau \vee \xi = \tau) },
  \end{equation*}
  which states that \( \rho \) is the successor of \( \tau \), will be useful for us when working with \hyperref[def:inductive_set]{inductive sets}.
\end{definition}

\begin{definition}\label{def:inductive_set}
  A set is called \term{inductive} if contains the empty set and is closed under the \hyperref[def:ordinal_successor]{successor operator}.

  We introduce the following \hyperref[rem:predicate_formula]{predicate formula}
  \begin{equation*}\taglabel[\op{IsInductive}]{eq:def:inductive_set/predicate}
    \ref{eq:def:inductive_set/predicate}[\tau] \coloneqq
      \parens[\Big]{ \qexists {\xi \in \tau} \ref{eq:def:empty_set/predicate}[\xi] }
      \wedge
      \parens[\Big]{ \qforall {\xi \in \tau} \qexists {\eta \in \tau} \ref{eq:def:ordinal_successor/predicate}[\eta, \xi] }.
  \end{equation*}
\end{definition}

\begin{proposition}\label{thm:smallest_inductive_set_existence}
  There is a smallest (with respect to set inclusion) \hyperref[def:inductive_set]{inductive set}, which we denote by \( \omega \).
\end{proposition}
\begin{proof}
  We cannot directly define \( \omega \) as the intersection of all inductive sets since we want to avoid unrestricted comprehension. Fortunately, the existence of at least one inductive set \( A \) is justified by the \hyperref[def:zfc/infinity]{axiom of infinity} in \logic{ZFC} or by taking the entire universe in na\"ive set theory.

  Hence, we use restricted comprehension:
  \begin{equation*}
    \omega \coloneqq \set{ x \in A \given x \T{belongs to every inductive set} }.
  \end{equation*}

  To see that \( \omega \) is itself inductive, note that \( \varnothing \in \omega \) and that if \( x \in \omega \), then it also belongs to all inductive sets and hence \( \op{succ}(x) \) also belongs to all inductive sets, proving \( \op{succ}(x) \in \omega \).
\end{proof}

\begin{theorem}[Induction via inductive sets]\label{thm:omega_induction}
  We can perform induction on the \hyperref[thm:smallest_inductive_set_existence]{smallest inductive set \( \omega \)}. That is, we can prove that some property holds for every element of \( \omega \) by proving the following:
  \begin{itemize}
    \item The property holds for \( \varnothing \)
    \item We can prove that is holds for \( \op{succ}(n) \) by assuming that it holds for some set \( n \in \omega \).
  \end{itemize}

  This is an analog of \eqref{eq:def:peano_arithmetic/PA3} and is actually used in \fullref{thm:omega_is_model_of_pa} to prove that \( \omega \) is a model of \hyperref[def:peano_arithmetic]{\logic{PA}}. Instead of an entire theorem schema, however, for this theorem it is sufficient to use one single formula. The more general induction principles that use theorem schemas cannot be proved without natural numbers, which are a model of \logic{PA} by virtue of this theorem.

  More formally, the following is a theorem of both na\"ive set theory and \hyperref[def:zfc]{\logic{ZF}}:
  \footnotesize
  \begin{equation}\label{eq:thm:omega_induction}
    \qexists \sigma
    \qforall \tau
    \parens[\Bigg]
      {
        \parens[\Bigg]
          {
            \parens[\Big]
            {
              \underbrace{ \qexists {\xi \in \tau} \ref{eq:def:empty_set/predicate}[\xi] }_{\mathclap{\T{base case}}}
            }
            \wedge
            \qforall \xi \parens[\Big]
              {
                \overbrace
                  {
                    \underbrace{ \xi \in \tau }_{\mathclap{\substack{\T{inductive} \\ \T{hypothesis}}}}
                    \rightarrow
                    \underbrace
                      {
                        \qexists {\eta \in \tau} \ref{eq:def:ordinal_successor/predicate}[\eta, \xi]
                      }_{\mathclap{\substack{\T{inductive step} \\ \T{conclusion}}}}
                  }^{\T{inductive step}}
              }
          }
        \rightarrow
        \underbrace{ \ref{eq:def:subset/predicate}[\sigma, \tau] }_{\T{conclusion}}
      }
  \end{equation}
  \normalsize
\end{theorem}
\begin{proof}
  The antecedent of (the inner formula in) \eqref{eq:thm:omega_induction} is a restatement of the predicate formula \( \ref{eq:def:inductive_set/predicate}[\tau] \). The situation resembles the \hyperref[eq:def:zfc/infinity]{axiom of infinity}, but, instead of existence of an inductive set \( \tau \), it states the existence of a set \( \sigma \) such that if \( \tau \) is an inductive set, then \( \sigma \) is a subset of \( \tau \) (if we restrict \( \xi \) to range only over members of \( \sigma \), then we would obtain equality of \( \tau \) and \( \sigma \) instead). In other words, we have reduced the verification of \eqref{eq:thm:omega_induction} to showing that there exists a minimal inductive set in both na\"ive set theory and \logic{ZF}.

  We have already proved in \fullref{thm:smallest_inductive_set_existence} that our fixed model \( \mscrV = (V, I) \) of set theory has a minimal inductive set \( \omega \). Thus, for any variable assignment \( v: \boldop{Var} \to V \), the modified assignment \( v_{\sigma \mapsto \omega} \) satisfies \eqref{eq:thm:omega_induction} with the outer existential quantifier removed. Hence, by \fullref{def:first_order_valuation/formula_valuation}, it follows that the entire formula \eqref{eq:thm:omega_induction} is satisfied by the assignment \( v \).

  Both the assignment \( v \) and the model \( \mscrV \) were arbitrary, therefore we can conclude that \eqref{eq:thm:omega_induction} is a theorem of both na\"ive set theory and \logic{ZF}.
\end{proof}

\begin{definition}\label{def:transitive_set}
  A set \( A \) is \term{transitive} if from \( B \in A \) it follows that \( B \subseteq A \).

  See \fullref{rem:ordinal_definition} for a discussion of the motivation and terminology of transitive sets and \fullref{rem:transitive_model_of_set_theory} for their importance.

  We introduce the following \hyperref[rem:predicate_formula]{predicate formula}:
  \begin{equation*}\taglabel[\op{IsSetTransitive}]{eq:def:transitive_set/predicate}
    \ref{eq:def:transitive_set/predicate}[\tau] \coloneqq \qforall {\xi \in \tau} \qforall {\eta \in \xi} {\eta \in \tau}
  \end{equation*}
\end{definition}

\begin{proposition}\label{thm:omega_is_transitive}
  The set \( \omega \) is transitive and every member of \( \omega \) is transitive.

  This proof demonstrates usage of \fullref{thm:omega_induction}.
\end{proposition}
\begin{proof}
  \SubProof{Proof that all members of \( \omega \) are transitive} In order to demonstrate how \fullref{thm:omega_induction} works in practice, we will use inductive sets directly. Let \( T \subseteq \omega \) be the subset of all transitive members of \( \omega \). We will show that \( T \) is inductive. Since \( T \subseteq \omega \) and \( \omega \) is the smallest inductive set, we can conclude that \( T = \omega \).

  Clearly \( \varnothing \in T \)  because every member of \( \varnothing \) vacuously is a subset of \( \varnothing \).

  Now suppose that \( n \in T \) and let \( m \in \op{succ}(n) = n \cup \set{ n } \). If \( m = n \), then \( m \in \op{succ}(n) \) by definition of the successor operation. If \( m \in n \), then \( m \subseteq n \) by the inductive hypothesis and hence also \( m \subseteq n \cup \set{ n } = \op{succ}(n) \). Thus, \( \op{succ}(n) \) is also transitive.

  We have shown that \( T \) is inductive. Therefore, \( \omega = T \) and every member of \( \omega \) is transitive.

  From now on, we will not be as explicit about the use of induction on \( \omega \).

  \SubProof{Proof that \( \omega \) is transitive} We will show that for all members \( n \) of \( \omega \) we have \( n \subseteq \omega \).

  The case \( n = \varnothing \) is again trivial.

  Now suppose that \( n \subseteq \omega \) and let \( m \in \op{succ}(n) \). If \( m = n \), clearly \( m \subseteq \omega \). If \( m \in n \), then \( m \subseteq n \) and, since \( n \subseteq \omega \), we have \( m \subseteq \omega \) by transitivity of \( \subseteq \).

  Therefore, \( \omega \) is transitive.
\end{proof}

\begin{remark}\label{rem:transitive_model_of_set_theory}
  As discussed in \fullref{def:set_builder_notation}, within the \hyperref[def:naive_set_theory/unrestricted_comprehension]{axiom schema of unrestricted comprehension} it may happen that \( U \subseteq V \) is not a set within the object logic.

  But there is a bigger problem that may happen even for \hyperref[rem:standard_model_of_set_theory]{standard models}. If \( A \in V \) and \( x \in A \) (in the metatheory), it is possible that \( x \) is not in \( V \). Therefore, if we have shown that \( A \) is a set within the object logic, it is possible that its members within the metatheory are not members in the object logic. On other words, it is possible for set membership itself to be incompatible between the metatheory and object logic.

  If \( V \) is a transitive set, however, we would not have such a problem. That is, if we construct a set \( A \) in the metatheory and show that it belongs to some set \( B \) in the object logic, then \( A \) itself would also be a set in the object logic.

  For this reason, it is very important to consider only transitive models of set theory.
\end{remark}

\begin{lemma}\label{thm:members_of_omega_do_not_contain_themselves}
  No element of \( \omega \) is a member of itself.
\end{lemma}
\begin{proof}
  We will again use \fullref{thm:omega_induction}. By definition, \( \omega \not\in \omega \). Now suppose that \( n \not\in n \) for some \( n \in \omega \).

  Aiming at a contradiction, suppose that \( \op{succ}(n) \in \op{succ}(n) \). The assumption that \( n = \op{succ}(n) = n \cup \set{ n } \) implies that \( n = \set{ n } \). The assumption that \( \op{succ}(n) \in n \) implies that \( n \in n \) since \( \op{succ}(n) \) is transitive by \fullref{thm:omega_is_transitive}. In both cases we have \( n \in n \), which contradicts our inductive hypothesis. This contradiction shows that \( \op{succ}(n) \not\in \op{succ}(n) \).

  \Fullref{thm:omega_induction} allows us to conclude that no member of \( \omega \) contains itself.
\end{proof}

\begin{theorem}\label{thm:omega_is_model_of_pa_without_operations}
  The \hyperref[thm:smallest_inductive_set_existence]{smallest inductive set \( \omega \)} satisfies the axioms \eqref{eq:def:peano_arithmetic/PA1}-\eqref{eq:def:peano_arithmetic/PA3} from \hyperref[def:peano_arithmetic]{Peano arithmetic} with the following interpretation:
  \begin{thmenum}
    \thmitem{thm:omega_is_model_of_pa_without_operations/zero} \hyperref[def:peano_arithmetic/zero]{Zero} is interpreted as \( \varnothing \).

    \thmitem{thm:omega_is_model_of_pa_without_operations/succ} The \hyperref[def:peano_arithmetic/succ]{successor} operation \( s \) is interpreted as \( \op{succ} \).
  \end{thmenum}

  We will generalize this theorem to \fullref{thm:omega_is_model_of_pa} after we are able to define the arithmetic operations in \( \omega \).
\end{theorem}
\begin{proof}
  \SubProofOf{eq:def:peano_arithmetic/PA1} Let \( n, m \in \omega \) and suppose that \( \op{succ}(n) = \op{succ}(m) \). If \( n = m \), there is nothing to prove.

  Suppose that \( n \neq m \). Thus, since
  \begin{equation*}
    n \cup \set{ n } = \op{succ}(n) = \op{succ}(m) = m \cup \set{ m },
  \end{equation*}
  we have both \( n \in m \) and \( m \in n \).

  \Fullref{thm:omega_is_transitive} implies that \( n \) is transitive and hence \( n \in n \), which contradicts \fullref{thm:members_of_omega_do_not_contain_themselves}.

  The obtained contradiction shows that \( n = m \).

  \SubProofOf{eq:def:peano_arithmetic/PA2} Suppose that \( \varnothing \) has a predecessor \( n \in \omega \). Then
  \begin{equation*}
    \varnothing = \op{succ}(n) = n \cup \set{ n },
  \end{equation*}
  which implies that \( n \in \varnothing \). But this contradicts the \hyperref[def:empty_set]{definition of \( \varnothing \)}.

  Therefore, \( \varnothing \) has no predecessor.

  \SubProofOf{eq:def:peano_arithmetic/PA3} It follows from \fullref{thm:omega_induction} that \eqref{eq:thm:omega_induction} is a theorem of \logic{ZF}. Let \( \mscrV = (V, I) \) be our ambient \hyperref[rem:standard_model_of_set_theory]{standard} \hyperref[rem:transitive_model_of_set_theory]{transitive} model of \logic{ZFC}.

  Fix any variable assignment \( v: \boldop{Var} \to V \). As in the proof of \fullref{thm:omega_induction}, we consider the modified assignment \( v_{\sigma \mapsto \omega} \) that \enquote{eliminates} the outer existential quantifier in \eqref{eq:thm:omega_induction}.

  To show that \eqref{eq:thm:omega_induction} really corresponds to \eqref{eq:def:peano_arithmetic/PA3} (and hence that \( \omega \) satisfies \eqref{eq:def:peano_arithmetic/PA3}), fix some formula \( \varphi \) of \hyperref[def:peano_arithmetic]{Peano arithmetic} (\hi{not \logic{ZFC}!}) and suppose that \( \xi, \zeta_1, \ldots, \zeta_n \) are all of its free variables. Fix also some parameter values \( u_1, \ldots, u_n \in \omega \) and, as in \fullref{def:set_builder_notation}, define the set
  \begin{equation*}
    A \coloneqq \set{ x \in \omega \given \varphi\Bracks{x, u_1, \ldots, x_n} }.
  \end{equation*}

  Since \eqref{eq:thm:omega_induction} is satisfied by \( v \), the inner formula in \eqref{eq:thm:omega_induction} (without the quantifiers over \( \sigma \) and \( \tau \)) is satisfied by \( v_{\sigma \mapsto \omega, \tau \mapsto A} \).

  Since our choice of parameters \( u_1, \ldots, u_n \) was arbitrary, we can conclude that the universal closure \eqref{eq:def:peano_arithmetic/PA3_quantified} of \eqref{eq:def:peano_arithmetic/PA3} is satisfied by \( \omega \) for every formula \( \varphi \) of \logic{PA}.
\end{proof}

\begin{remark}\label{rem:set_theory_natural_numbers_without_operations}
  Due to \fullref{thm:omega_is_model_of_pa_without_operations}, we will henceforth identify the \hyperref[thm:smallest_inductive_set_existence]{smallest inductive set \( \omega \)} with the set \( \BbbN \) of \hyperref[def:set_of_natural_numbers]{natural numbers}.

  We are not yet able to add or multiply natural numbers, nor rely on their well-foundedness, however for all other purposes we are able to utilize them.

  Since the ordering in \fullref{def:natural_number_ordering} is defined via addition, we must define some other ordering. Luckily, as we shall see in \fullref{subsec:ordinals}, \( n < m \) corresponds to \( n \in m \). In particular, the members of \( m \) are ordered.

  As a consequence, every natural number \( n \) equals the set of all smaller natural numbers by the \hyperref[def:naive_set_theory/extensionality]{axiom of extensionality}. It is conventional to write \( \set{ 0, 1, \ldots, n } \) rather than \( \set{ m \given m \in n } \). The former notation will be fully justified in \fullref{subsec:ordinals}.

  This is useful, for example, in \fullref{def:sequence}.
\end{remark}


\documentclass{article}%
\usepackage[T1]{fontenc}%
\usepackage[utf8]{inputenc}%
\usepackage{lmodern}%
\usepackage{textcomp}%
\usepackage{lastpage}%
%
\input{common_symbols_and_format.tex}%
\usepackage{tocloft}%
\renewcommand{\cfttoctitlefont}{\Large\bfseries}%
%
\begin{document}%
\normalsize%
\logo%
\rulename{Static Log Difference Regression}%
\tblofcontents
\ruledescription{Regresses the one-day price changes against the lagged  difference of research to price for the specified number of days, using coefficients estimated from the start of the data.}%

\howtotrade{Given default parameter values, if the asset drift is 0.001 and the error is 0.02 (2$\%$ daily volatility), this rule will take a $0.001 / (0.02)^2  = 2.5$ or 250$\%$ position (leveraged).}

\ruleparameters{Kelly Fraction}{0.1}{Amplitude weighting (Kelly Fraction). 1.0 is maximum growth if regression is exact. <1.0 scales down positions taken.}{$\kellyfraction$}{Regression Length}{100}{This is the number of days used to estimate the regression coefficients.}{$\lookbacklength$}%
\stoptable%

\section{Equation}
Below are the equations which govern how this trading rule determines a trading position.
\begin{equation} \label{eq1}
   \regressionprice_\currenttime =  \amplitudecoefficient  (ln(\price_{\currenttime}) - ln(\research_{\currenttime})) + \constantc,
\end{equation}

The equation ($\ref{eq1}$) predict the value of the price $\regressionprice_\currenttime$ at time $\currenttime$ using the difference from the logarithm of the price and research values. Using the logarithm function properties, we can rewrite it easily as:

\begin{equation}
       \regressionprice_\currenttime =  \amplitudecoefficient  ln\left( \frac{\price_{\currenttime}}{\research_{\currenttime}}\right)  + \constantc,
\end{equation}

Since we are using a static approach the amplitude coefficient $\amplitudecoefficient$ remains constant. In order to calculate the resultant fractional portfolio allocation $\position_{\currenttime}$ we use the Kelly fraction to obtain the maximum results for the long run.

\begin{equation}
\position_\currenttime = \kellyfraction \frac{\regressionprice_\currenttime}{ \rmserror_{\regressionprice}^{2}}
\label{eq2}
\end{equation}

Additionally, the standard error $\rmserror_{\regressionprice}$ is calculated and included in equation (\ref{eq2}) to normalize the predicted price.

\assumptions%
\keyterms%
\furtherlinks%
\end{document}


\section{L'H\^opital's Rule}\label{sec:LH}\label{sec:lhopitals_rule}

This section is concerned with a technique of evaluating certain limits that will be useful for determining asymptotes, and also in the following chapters, where integration is discussed.

Our treatment of limits exposed us to ``$ 0/0 $'', an indeterminate form. If $\ds \lim_{x\to c}f(x)=0$ and $\ds \lim_{x\to c} g(x) =0$, we do not conclude that $\ds \lim_{x\to c} f(x)/g(x)$ is $0/0$; rather, we use $0/0$ as notation to describe the fact that both the numerator and denominator approach $ 0 $. The expression $ 0/0 $ has no numeric value; other work must be done to evaluate the limit.

Other indeterminate forms exist; they are: %Limits may seeming evaluate to
 $\infty/\infty$, $0\cdot\infty$, $\infty-\infty$, $0^0$, $1^\infty$ and $\infty^0$. %, expressions which have no inherent value.
 Just as ``$ 0/0 $'' does not mean ``divide $ 0 $ by $ 0 $,'' the expression ``$\infty/\infty$'' does not mean ``divide infinity by infinity.'' Instead, it means ``a quantity is growing without bound and is being divided by another quantity that is growing without bound.'' We cannot determine from such a statement what value, if any, results in the limit. Likewise, ``$0\cdot \infty$'' does not mean ``multiply zero by infinity.'' Instead, it means ``one quantity is shrinking to zero, and is being multiplied by a quantity that is growing without bound.'' We cannot determine from such a description what the result of such a limit will be.

This section introduces l'H\^opital's Rule, a method of resolving limits that produce the indeterminate forms $ 0/0 $ and $\infty/\infty$. We'll also show how algebraic manipulation can be used to convert other indeterminate expressions into one of these two forms so that our new rule can be applied.

\begin{theorem}{L'H\^opital's Rule, Part 1}{LHR}
{Let $\ds \lim_{x\to c}f(x) = 0$ and $\ds \lim_{x\to c}g(x)=0$, where $f$ and $g$ are differentiable functions on an open interval $I$ containing $c$, and $g\primeskip'(x)\neq 0$ on $I$ except possibly at $c$. Then \index{limit!L'H\^opital's Rule}\index{L'H\^opital's Rule}
$$ \lim_{x\to c} \frac{f(x)}{g(x)} = \lim_{x\to c} \frac{\fp(x)}{g\primeskip'(x)}.$$
}
\end{theorem}

We demonstrate the use of l'H\^opital's Rule in the following examples; we will often use ``LHR'' as an abbreviation of ``l'H\^opital's Rule.''\\

\begin{example}{Using l'H\^opital's Rule}{ex_lhr1}
{
Evaluate the following limits, using l'H\^opital's Rule as needed.

\noindent%
\begin{minipage}[t]{.5\textwidth}
\begin{enumerate}
\item		$\ds \lim_{x\to0}\frac{\sin x}x$
\item		$\ds \lim_{x\to 1}\frac{\sqrt{x+3}-2}{1-x}$
\end{enumerate}
\end{minipage}
\begin{minipage}[t]{.5\textwidth}
\begin{enumerate}\addtocounter{enumi}{2}
\item		$\ds \lim_{x\to0}\frac{x^2}{1-\cos x}$
\item		$\ds \lim_{x\to 2}\frac{x^2+x-6}{x^2-3x+2}$
\end{enumerate}
\end{minipage}
}

\end{example}

\begin{solution}
{\begin{enumerate}
\item		We could solve this using the Squeeze Theorem, but l'H\^opital's Rule is much simpler to apply:
$$\lim_{x\to0}\frac{\sin x}x \;\;\;\;\left(\to \frac00\right)\stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=} \lim_{x\to0} \frac{\cos x}{1}=1.$$

\item	\hfill $\ds \lim_{x\to 1}\frac{\sqrt{x+3}-2}{1-x} \;\;\;\;\left(\to \frac00\right)	 \stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=} \lim_{x \to 1} \frac{\frac12(x+3)^{-1/2}}{-1} =-\frac 14.$\hfill\null

\item		\hfill $\ds \lim_{x\to 0}\frac{x^2}{1-\cos x} \;\;\;\;\left(\to \frac00\right) \stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=}  \lim_{x\to 0} \frac{2x}{\sin x}.$ \hfill\null

This latter limit also evaluates to the $ 0/0 $ indeterminate form. To evaluate it, we apply l'H\^opital's Rule again.

\hfill $\ds  \lim_{x\to 0} \frac{2x}{\sin x}  \stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=} \frac{2}{\cos x} = 2 .$ \hfill\null

Thus $\ds \lim_{x\to0}\frac{x^2}{1-\cos x}=2.$

\item		We already know how to evaluate this limit; first factor the numerator and denominator. We then have:
$$\lim_{x\to 2}\frac{x^2+x-6}{x^2-3x+2} = \lim_{x\to 2}\frac{(x-2)(x+3)}{(x-2)(x-1)} = \lim_{x\to 2}\frac{x+3}{x-1} = 5.$$
We now show how to solve this using l'H\^opital's Rule.

$$\lim_{x\to 2}\frac{x^2+x-6}{x^2-3x+2} \;\;\;\;\left(\to \frac00\right)\stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=}  \lim_{x\to 2}\frac{2x+1}{2x-3} = 5.$$
\end{enumerate}
\vskip-\baselineskip
}
\end{solution}

%
Note that at each step where l'H\^opital's Rule was applied, it was \emph{needed}: the initial limit returned the indeterminate form of ``$0/0$.'' If the initial limit returns, for example, $ 1/2 $, then l'H\^opital's Rule does not apply.

The following theorem extends our initial version of l'H\^opital's Rule in two ways. It allows the technique to be applied to the indeterminate form $\infty/\infty$ and to limits where $x$ approaches $\pm\infty$.

\begin{theorem}{L'H\^opital's Rule, Part 2}{thm:LHR2}
{\begin{enumerate}
\item		Let $\ds\lim_{x\to a}f(x) = \pm\infty$ and $\ds\lim_{x\to a}g(x)=\pm \infty$, where $f$ and $g$ are differentiable on an open interval $I$ containing $a$. Then \index{limit!L'H\^opital's Rule}\index{L'H\^opital's Rule}
$$\lim_{x\to a} \frac{f(x)}{g(x)} = \lim_{x\to a}\frac{\fp(x)}{g\primeskip'(x)}.$$

\item		Let $f$ and $g$ be differentiable functions on the open interval $(a,\infty)$ for some value $a$, where $g\primeskip'(x)\neq 0$ on $(a,\infty)$ and $\ds\lim_{x\to\infty} f(x)/g(x)$ returns either 0/0 or $\infty/\infty$. Then
$$\lim_{x\to \infty} \frac{f(x)}{g(x)} = \lim_{x\to \infty}\frac{\fp(x)}{g\primeskip'(x)}.$$
A similar statement can be made for limits where $x$ approaches $-\infty$.
\end{enumerate}
}
\end{theorem}

\begin{example}{Using l'H\^opital's Rule with limits involving $\infty$}{ex_LHR2}
{
Evaluate the following limits.\\

$\ds 1.\ \lim_{x\to\infty} \frac{3x^2-100x+2}{4x^2+5x-1000} \qquad\qquad 2. \ \lim_{x\to \infty}\frac{e^x}{x^3}.$
}

\end{example}

\begin{solution}
{\begin{enumerate}
\item		We can evaluate this limit already using other techniques%Theorem \ref{thm:lim_rational_fn_at_infty}
; the answer is $ 3/4 $. We apply l'H\^opital's Rule to demonstrate its applicability.
$$\lim_{x\to\infty} \frac{3x^2-100x+2}{4x^2+5x-1000} \;\;\;\;\left(\to \frac\infty\infty \right) \stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=} \lim_{x\to\infty} \frac{6x-100}{8x+5} \;\;\;\;\left(\to \frac\infty\infty \right) \stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=} \lim_{x\to\infty} \frac68 = \frac34.$$

\item		$\ds \lim_{x\to \infty}\frac{e^x}{x^3} \;\;\;\;\left(\to \frac\infty\infty \right) \stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=} \lim_{x\to\infty} \frac{e^x}{3x^2} \;\;\;\;\left(\to \frac\infty\infty \right) \stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=} \lim_{x\to\infty} \frac{e^x}{6x} \;\;\;\;\left(\to \frac\infty\infty \right) \stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=} \lim_{x\to\infty} \frac{e^x}{6} = \infty.$

Recall that this means that the limit does not exist; as $x$ approaches $\infty$, the expression $e^x/x^3$ grows without bound. We can infer from this that $e^x$ grows ``faster'' than $x^3$; as $x$ gets large, $e^x$ is far larger than $x^3$. (This has important implications in computing when considering efficiency of algorithms.)
\end{enumerate}
}
\end{solution}

\subsection*{Indeterminate Forms $0\cdot\infty$ and $\infty-\infty$}

L'H\^opital's Rule can only be applied to ratios of functions. When faced with an indeterminate form such as $0\cdot\infty$ or $\infty-\infty$, we can sometimes apply algebra to rewrite the limit so that l'H\^opital's Rule can be applied. We demonstrate the general idea in the next example.
\index{limit!indeterminate form}\index{indeterminate form}\\

\begin{example}{L'H\^opital's Rule}{LHRule3}
Compute the following limits:\\
\noindent
\begin{minipage}[t]{.5\textwidth}
\begin{enumerate}
\item		$\ds \lim_{x\to0^+} x\cdot e^{1/x}$
\item		$\ds \lim_{x\to0^-} x\cdot e^{1/x}$
\item       $\ds\lim_{x\to 0^+} x\ln x$
\end{enumerate}
\end{minipage}
\begin{minipage}[t]{.5\textwidth}
\begin{enumerate}\addtocounter{enumi}{3}
\item		$\ds \lim_{x\to\infty} \ln(x+1)-\ln x$
\item		$\ds \lim_{x\to\infty} x^2-e^x$
\end{enumerate}
\end{minipage}
\end{example}

\begin{solution}
{\begin{enumerate}
\item		As $x\rightarrow 0^+$, $x\rightarrow 0$ and $e^{1/x}\rightarrow \infty$. Thus we have the indeterminate form $0\cdot\infty$. We rewrite the expression $x\cdot e^{1/x}$ as $\ds\frac{e^{1/x}}{1/x}$; now, as $x\rightarrow 0^+$, we get the indeterminate form $\infty/\infty$ to which l'H\^opital's Rule can be applied.
$$ \lim_{x\to0^+} x\cdot e^{1/x} = \lim_{x\to 0^+} \frac{e^{1/x}}{1/x} \stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=} \lim_{x\to 0^+}\frac{(-1/x^2)e^{1/x}}{-1/x^2} =\lim_{x\to 0^+}e^{1/x} =\infty.$$

Interpretation: $e^{1/x}$ grows ``faster'' than $x$ shrinks to zero, meaning their product grows without bound.

\item		As $x\rightarrow 0^-$, $x\rightarrow 0$ and $e^{1/x}\rightarrow e^{-\infty}\rightarrow 0$. The the limit evaluates to $0\cdot 0$ which is not an indeterminate form. We conclude then that $$\lim_{x\to 0^-}x\cdot e^{1/x} = 0.$$

\item As $x\to 0^+$, $\ln x\to -\infty$, so the product is indeterminate of the form $\pm 0\cdot\infty $.
 So we can  apply L'H\^opital's Rule after re-writing it in the form $\frac{\infty }{\infty }$:
$$x\ln x = \frac{\ln x}{1/x}=\frac{\ln x}{x^{-1}}.$$
Now as $x$ approaches zero, both the numerator and denominator
approach infinity (one $-\infty$ and one $+\infty$, but only the size
is important). Using  L'H\^opital's Rule:
$$\lim_{x\to 0^+} {\ln x\over x^{-1}}\stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=}
\lim_{x\to 0^+} {1/x\over -x^{-2}} =\lim_{x\to 0^+} -x = 0.$$
Interpretation: $x$ approaches zero much faster than the $\ln x$ approaches $-\infty$.
\item		This limit initially evaluates to the indeterminate form $\infty-\infty$. By applying a logarithmic rule, we can rewrite the limit as
$$ \lim_{x\to\infty} \ln(x+1)-\ln x = \lim_{x\to \infty} \ln \left(\frac{x+1}x\right).$$

As $x\rightarrow \infty$, the argument of the $\ln$ term approaches $\infty/\infty$, to which we can apply l'H\^opital's Rule.
$$\lim_{x\to\infty} \frac{x+1}x \stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=} \frac11=1.$$

Since $x\rightarrow \infty$ implies $\ds\frac{x+1}x\rightarrow 1$, it follows that
$$x\rightarrow \infty \quad \text{ implies }\quad \ln\left(\frac{x+1}x\right)\rightarrow \ln 1=0.$$

Thus $$ \lim_{x\to\infty} \ln(x+1)-\ln x = \lim_{x\to \infty} \ln \left(\frac{x+1}x\right)=0.$$
Interpretation: since this limit evaluates to 0, it means that for large $x$, there is essentially no difference between $\ln (x+1)$ and $\ln x$; their difference is essentially 0.

\item		The limit $\ds \lim_{x\to\infty} x^2-e^x$ initially returns the indeterminate form $\infty-\infty$. We can rewrite the expression by factoring out $x^2$; $\ds x^2-e^x = x^2\left(1-\frac{e^x}{x^2}\right).$ We need to evaluate how $e^x/x^2$ behaves as $x\rightarrow \infty$:
$$\lim_{x\to\infty}\frac{e^x}{x^2} \stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=} \lim_{x\to\infty} \frac{e^x}{2x} \stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=} \lim_{x\to\infty} \frac{e^x}{2} = \infty.$$

Thus $\lim_{x\to\infty}x^2(1-e^x/x^2)$ evaluates to $\infty\cdot(-\infty)$, which is not an indeterminate form; rather, $\infty\cdot(-\infty)$ evaluates to $-\infty$. We conclude that
$\ds \lim_{x\to\infty} x^2-e^x = -\infty.$

Interpretation: as $x$ gets large, the difference between $x^2$ and $e^x$ grows very large.
\end{enumerate}
}
\end{solution}

\subsection*{Indeterminate Forms\ \ $0^0$, $1^\infty$ and $\infty^0$}

When faced with an indeterminate form that involves a power, it often helps to employ the natural logarithmic function. The following Key Idea expresses the concept, which is followed by an example that demonstrates its use.

\begin{formulabox}[Limits Involving Indeterminate Powers] %Forms $0^0$, $1^\infty$ and $\infty^0$
{\label{idea:LHRpower}
If $\ds \lim_{x\to c} \ln\big(f(x)\big) = L$, then
$\ds \lim_{x\to c} f(x) = \lim_{x\to c} e^{\ln(f(x))} = e\,^L.$
\index{limit!indeterminate form}\index{indeterminate form}
}
\end{formulabox}

%

\begin{example}{Using l'H\^opital's Rule with indeterminate forms involving exponents }{ex_LHR4}
{Evaluate the following limits.
\begin{enumerate}
\item $\ds \lim_{x\to\infty} \left(1+\frac1x\right)^x  $
\item  $ \ds \lim_{x\to0^+} x^x$
\item $\ds{\lim_{x\to 1^+}x^{1/(x-1)}}.$
\end{enumerate}
}
\end{example}

\begin{solution}
{\begin{enumerate}
\item		%This equivalent to a special limit given in Theorem \ref{thm:lim_continuous};
This limit has important applications within mathematics and finance. Note that the exponent approaches $\infty$ while the base approaches $ 1 $, leading to the indeterminate form $1^\infty$. Let $f(x) = (1+1/x)^x$; the problem asks to evaluate $\ds\lim_{x\to\infty}f(x)$. Let's first evaluate $\ds \lim_{x\to\infty}\ln\big(f(x)\big)$.
\begin{align*}
\lim_{x\to\infty}\ln\big(f(x)\big) & = \lim_{x\to\infty} \ln \left(1+\frac1x\right)^x \\
			&= \lim_{x\to\infty} x\ln\left(1+\frac1x\right)\\
			&=  \lim_{x\to\infty} \frac{\ln\left(1+\frac1x\right)}{1/x}\\
			\intertext{This produces the indeterminate form 0/0, so we apply l'H\^opital's Rule.}
			&\stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=}	\lim_{x\to\infty} \frac{\frac{1}{1+1/x}\cdot(-1/x^2)}{(-1/x^2)} \\
			&= \lim_{x\to\infty}\frac{1}{1+1/x}\\
			&= 1.
\end{align*}
Thus $\ds\lim_{x\to\infty} \ln \big(f(x)\big) = 1.$ We return to the original limit and apply Key Idea \ref{idea:LHRpower}.

$$\lim_{x\to\infty}\left(1+\frac1x\right)^x = \lim_{x\to\infty} f(x) =  \lim_{x\to\infty}e^{\ln (f(x))} = e^1 = e.$$

\item		This limit leads to the indeterminate form $0^0$. Let $f(x) = x^x$ and consider first $\ds\lim_{x\to0^+} \ln\big(f(x)\big)$.
\begin{align*}
\lim_{x\to0^+} \ln\big(f(x)\big) &= \lim_{x\to0^+} \ln\left(x^x\right) \\
			&= \lim_{x\to0^+} x\ln x \\
			&= \lim_{x\to0^+} \frac{\ln x}{1/x}.\\
			\intertext{This produces the indeterminate form $-\infty/\infty$ so we apply l'H\^opital's Rule.}
			&\stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=}	\lim_{x\to0^+} \frac{1/x}{-1/x^2} \\
			&= \lim_{x\to0^+} -x \\
			&= 0.
\end{align*}
Thus $\ds\lim_{x\to0^+} \ln\big(f(x)\big) =0$. We return to the original limit and apply Key Idea \ref{idea:LHRpower}.
$$\lim_{x\to0^+} x^x = \lim_{x\to0^+} f(x) = \lim_{x\to0^+} e^{\ln(f(x))} = e^0 = 1.$$
This result is supported by the graph of $f(x)=x^x$ given in Figure \ref{fig:LHR4}.

\mfigure{.8}{A graph of $f(x)=x^x$ supporting the fact that as $x\to 0^+$, $f(x)\to 1$.}{fig:LHR4}{\begin{tikzpicture}
\begin{axis}[width=.6\textwidth,%
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
%			xtick=\empty,%
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
			ytick={1,2,3,4},
%			yticklabels={$-0.002$,$0.002$,$0.004$},
			%minor y tick num=1,
%			extra y ticks={0.001},%
%			minor x tick num=4,
			ymin=-.4,ymax=4.5,%
			xmin=-.1,xmax=2.2,%
			scaled ticks=false
]

\addplot [{\colorone},thick,smooth] coordinates {(0.01,0.955)(0.02,0.9247)(0.03,0.9001)(0.04,0.8792)(0.05,0.8609)(0.06,0.8447)(0.07,0.8302)(0.08,0.817)(0.09,0.8052)(0.1,0.7943)(0.2,0.7248)(0.3,0.6968)(0.4,0.6931)(0.5,0.7071)(0.6,0.736)(0.7,0.7791)(0.8,0.8365)(0.9,0.9095)(1.,1.)(1.1,1.111)(1.2,1.245)(1.3,1.406)(1.4,1.602)(1.5,1.837)(1.6,2.121)(1.7,2.465)(1.8,2.881)(1.9,3.386)(2.,4.)
};

\draw (axis cs:1,1) node [below right] {\scriptsize $f(x)=x^x$};

%\draw (axis cs:2.4,-0.002) node {\scriptsize $f(x)$};
\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture}
\item
This limit is of the type $\mathquotes{1^\infty}$.
To deal with this type of limit we will again use logarithms.
Let
$$L=\lim_{x\to 1^+}x^{1/(x-1)}.$$
Now, take the natural log of both sides:
$$\ln L=\lim_{x\to 1^+}\ln\left(x^{1/(x-1)}\right).$$
Using log properties we have:
$$\ln L=\lim_{x\to 1^+}\frac{\ln x}{x-1}.$$
The right side limit is now of the type $0/0$, therefore, we can apply L'H\^opital's Rule:
$$\ln L=\lim_{x\to 1^+}\frac{\ln x}{x-1}\stackrel{\ \text{ by LHR \rule[-5pt]{0pt}{3pt}} \ }{=}\lim_{x\to 1^+}\frac{1/x}{1}=1$$
Thus, $\ln L=1$ and hence, our original limit (denoted by $L$) is: $L=e^1=e$. That is,
$$L=\lim_{x\to 1^+}x^{1/(x-1)}=e.$$
In this case, even though our limit had a type of $\mathquotes{1^\infty}$, it actually had a value of $e$.
}
\end{enumerate}
}
\end{solution}

%Our brief revisit of limits will be rewarded in the next section where we consider \textit{improper integration.} So far, we have only considered definite integrals where the bounds are finite numbers, such as $\ds \int_0^1 f(x)\ dx$. Improper integration considers integrals where one, or both, of the bounds are ``infinity.'' Such integrals have many uses and applications, in addition to generating ideas that are enlightening.

%The following application of derivatives allows us to compute certain limits.
%
%\begin{definition}{Limits of the Indeterminate Forms $\frac{0}{0}$ and $\frac{\infty}{\infty}$}{IndeterminateLimits}
%A limit of a quotient $\lim\limits_{x\rightarrow a}\frac{f\left( x\right) }{%
%	g\left( x\right) }$ is said to be an \textbf{indeterminate form of the type }%
%$\frac{0}{0}$ if both $f\left( x\right) \rightarrow 0$ and $g\left( x\right)
%\rightarrow 0$ as $x\rightarrow a.$ Likewise, it is said to be an \textbf{%
%	indeterminate form of the type} $\frac{\infty }{\infty }$ if both $f\left(
%x\right) \rightarrow \pm \infty $ and $g\left( x\right) \rightarrow \pm
%\infty $ as $x\rightarrow a$ (Here, the two $\pm $ signs are independent of
%each other).
%\end{definition}
%
%\begin{theorem}{L'H\^opital's Rule}{LHRule}
%For a limit $\lim\limits_{x\rightarrow a}\frac{f\left( x\right) }{g\left(
%	x\right) }$ of the indeterminate form $\frac{0}{0}$ or $\frac{\infty }{%
%	\infty },$ $\lim\limits_{x\rightarrow a}\frac{f\left( x\right) }{g\left(
%	x\right) }=\lim\limits_{x\rightarrow a}\frac{f^{\prime }\left( x\right) }{%
%	g^{\prime }\left( x\right) }$ if $\lim\limits_{x\rightarrow a}\frac{%
%	f^{\prime }\left( x\right) }{g^{\prime }\left( x\right) }$ exists or equals $%
%\infty $ or $-\infty$.
%\end{theorem}
%
%This theorem is somewhat difficult to prove, in part because it
%incorporates so many different possibilities, so we will not prove it
%here.
%
%We should also note that there may be instances where we would need to apply L'H\^opital's Rule multiple times, but we must confirm that $\lim_{x\to a}\frac{f'(x)}{g'(x)}$ is still indeterminate before we attempt to apply L'H\^opital's Rule again. Finally, we want to mention that L'H\^opital's rule is also valid for one-sided limits and limits at infinity.
%
%\begin{example}{L'H\^opital's Rule}{lhrule0}
%Compute $\ds\lim_{x\to \pi}\frac{x^2-\pi^2}{\sin x}$.
%\end{example}
%
%\begin{solution}
%We use L'H\^opital's Rule: Since the numerator and denominator
%both approach zero,
%$$\lim_{x\to \pi}\frac{x^2-\pi^2}{\sin x}=
%\lim_{x\to \pi}\frac{2x}{\cos x},$$
%provided the latter exists. But in fact this is an easy limit, since
%the denominator now approaches $-1$, so
%$$\lim_{x\to \pi}\frac{x^2-\pi^2}{\sin x}=\frac{2\pi}{-1} = -2\pi.$$
%\end{solution}
%
%\begin{example}{L'H\^opital's Rule}{LHRule1}
%Compute $\ds\lim_{x\to \infty}{2x^2-3x+7\over
%x^2+47x+1}$.
%\end{example}
%
%\begin{solution}
%As $x$ goes to infinity, both the numerator and denominator go to
%infinity, so we may apply L'H\^opital's Rule:
%$$\lim_{x\to \infty}\frac{2x^2-3x+7}{x^2+47x+1}=
%\lim_{x\to \infty}\frac{4x-3}{2x+47}.$$
%In the second quotient, it is still the case that the numerator and
%denominator both go to infinity, so we are allowed to use
%L'H\^opital's Rule again:
%$$\lim_{x\to \infty}\frac{4x-3}{2x+47}=\lim_{x\to \infty}\frac{4}{2}=2.$$
%So the original limit is 2 as well.
%\end{solution}
%
%\begin{example} {L'H\^opital's Rule}{LHRule2}
%Compute $\ds\lim_{x\to 0}\frac{\sec x - 1}{\sin x}$.
%\end{example}
%
%\begin{solution}
%Both the numerator and denominator approach zero, so applying
%L'H\^opital's Rule:
%$$\lim_{x\to 0}\frac{\sec x - 1}{\sin x}=
%\lim_{x\to 0}\frac{\sec x\tan x}{\cos x}=\frac{1\cdot 0}{1}=0.$$
%\end{solution}
%
%L'H\^{o}pital's rule concerns limits of a quotient that are indeterminate forms. But not all
%functions are given in the form of a quotient. But all the same, nothing
%prevents us from re-writing a given function in the form of a quotient.
%Indeed, some functions whose given form involve either a product $f\left(
%x\right) g\left( x\right) $ or a power $f\left( x\right) ^{g\left( x\right)
%} $ carry indeterminacies such as $0\cdot \pm \infty $ and $1^{\pm \infty }.$
%Something small times something numerically large (positive or negative)
%could be anything. It depends on how small and how large each piece turns
%out to be. A number close to 1 raised to a numerically large (positive or
%negative) power could be anything. It depends on how close to 1 the base is,
%whether the base is larger than or smaller than 1, and how large the
%exponent is (and its sign). We can use suitable algebraic manipulations to
%relate them to indeterminate quotients. We will illustrate with two
%examples, first a product and then a power.
%
%\begin{example}{L'H\^opital's Rule}{LHRule3}
%Compute $\ds\lim_{x\to 0^+} x\ln x$.
%\end{example}
%
%\begin{solution}
%This doesn't appear to be suitable for L'H\^opital's Rule, but it also
%is not ``obvious''. As $x$ approaches zero, $\ln x$ goes to $-\infty$,
%so the product looks like:
%$$(\hbox{something very small})\cdot(\hbox{something very large and negative}).$$
%This could be anything: it depends on {\it how small} and
%{\it how large} each piece of the function turns out to be.
%As defined earlier, this is a type of $\pm\mathquotes{0\cdot\infty}$, which is
%indeterminate. So we can in fact apply L'H\^opital's Rule after re-writing
%it in the form $\frac{\infty }{\infty }$:
%$$x\ln x = \frac{\ln x}{1/x}=\frac{\ln x}{x^{-1}}.$$
%Now as $x$ approaches zero, both the numerator and denominator
%approach infinity (one $-\infty$ and one $+\infty$, but only the size
%is important). Using  L'H\^opital's Rule:
%$$\lim_{x\to 0^+} {\ln x\over x^{-1}}=
%\lim_{x\to 0^+} {1/x\over -x^{-2}} =\lim_{x\to 0^+} {1\over x}(-x^2)=
%\lim_{x\to 0^+} -x = 0.$$
%One way to interpret this is that since $\ds\lim_{x\to
%  0^+}x\ln x = 0$, the $x$ approaches zero much faster than the $\ln x$
%approaches $-\infty$.
%\end{solution}
%
%Finally, we illustrate how a limit of the type $\mathquotes{1^\infty}$ can be indeterminate.
%
%\begin{example} {L'H\^opital's Rule}{LHRule4}
%Evaluate $\ds{\lim_{x\to 1^+}x^{1/(x-1)}}.$
%\end{example}
%
%\begin{solution}
%Plugging in $x=1$ (from the right) gives a limit of the type $\mathquotes{1^\infty}$.
%To deal with this type of limit we will use logarithms.
%Let
%$$L=\lim_{x\to 1^+}x^{1/(x-1)}.$$
%Now, take the natural log of both sides:
%$$\ln L=\lim_{x\to 1^+}\ln\left(x^{1/(x-1)}\right).$$
%Using log properties we have:
%$$\ln L=\lim_{x\to 1^+}\frac{\ln x}{x-1}.$$
%The right side limit is now of the type $0/0$, therefore, we can apply L'H\^opital's Rule:
%$$\ln L=\lim_{x\to 1^+}\frac{\ln x}{x-1}=\lim_{x\to 1^+}\frac{1/x}{1}=1$$
%Thus, $\ln L=1$ and hence, our original limit (denoted by $L$) is: $L=e^1=e$. That is,
%$$L=\lim_{x\to 1^+}x^{1/(x-1)}=e.$$
%In this case, even though our limit had a type of $\mathquotes{1^\infty}$, it actually had a value of $e$.
%\end{solution}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Opensolutionfile{solutions}[ex]
\section*{Exercises for \ref{sec:LH}}

Compute the following limits.

\begin{multicols}{2}[]

\begin{enumialphparenastyle}

%%%%%%%%%%
\begin{ex}
$\ds\lim_{x\to 0} {\cos x -1\over \sin x}$
\begin{sol}
 $0$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
$\ds\lim_{x\to \infty} {e^x\over x^3}$
\begin{sol}
 $\infty$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
$\ds\lim_{x\to \infty} {\ln x\over x}$
\begin{sol}
 $0$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
$\ds\lim_{x\to \infty} {\ln x\over \sqrt{x}}$
\begin{sol}
 $0$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
$\ds\lim_{x\to0}{\sqrt{9+x}-3\over x}$
\begin{sol}
 $1/6$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
$\ds\lim_{x\to2}{2-\sqrt{x+2}\over 4-x^2}$
\begin{sol}
 $1/16$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to1}{\sqrt{x}-1\over \sqrt[3]{x}-1}$
\begin{sol}
 $3/2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to0}{(1-x)^{1/4}-1\over x}$
\begin{sol}
 $-1/4$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{t\to 0}{\left(t+{1\over t}\right)((4-t)^{3/2}-8)}$
\begin{sol}
 $-3$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{t\to 0^+}\left({1\over t}+{1\over\sqrt{t}}\right)
(\sqrt{t+1}-1)$
\begin{sol}
 $1/2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to 0}{x^2\over\sqrt{2x+1}-1}$
\begin{sol}
 $0$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{u\to 1}{(u-1)^3\over (1/u)-u^2+3/u-3}$
\begin{sol}
 $-1$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to 0}{2+(1/x)\over 3-(2/x)}$
\begin{sol}
 $-1/2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to 0^+}{1+5/\sqrt{x}\over 2+1/\sqrt{x}}$
\begin{sol}
 $5$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to\pi/2}{\cos x\over (\pi/2)-x}$
\begin{sol}
 $1$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to0}{e^x-1\over x}$
\begin{sol}
 $1$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to0}{x^2\over e^x-x-1}$
\begin{sol}
 $2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to1}{\ln x\over x-1}$
\begin{sol}
 $1$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to0}{\ln(x^2+1)\over x}$
\begin{sol}
 $0$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to1}{x\ln x\over x^2-1}$
\begin{sol}
 $1/2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to0}{\sin(2x)\over\ln(x+1)}$
\begin{sol}
 $2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to1}{x^{1/4}-1\over x}$
\begin{sol}
 $0$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to1}{\sqrt{x}-1\over x-1}$
\begin{sol}
 $1/2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to0}{3x^2+x+2\over x-4}$
\begin{sol}
 $-1/2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to0}{\sqrt{x+1}-1\over \sqrt{x+4}-2}$
\begin{sol}
 $2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to0}{\sqrt{x+1}-1\over \sqrt{x+2}-2}$
\begin{sol}
 $0$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to0^+}{\sqrt{x+1}+1\over\sqrt{x+1}-1}$
\begin{sol}
 $\infty$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to0}{\sqrt{x^2+1}-1\over\sqrt{x+1}-1}$
\begin{sol}
 $0$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to1}{(x+5)\left({1\over 2x}+{1\over x+2}\right)}$
\begin{sol}
 $5$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 $\ds\lim_{x\to2}{x^3-6x-2\over x^3+4}$
\begin{sol}
 $-1/2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 {$\ds \lim_{x\to \pi} \frac{\sin x}{x-\pi}$}

\begin{sol}
  {$-1$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
{$\ds \lim_{x\to \pi/4} \frac{\sin x-\cos x}{\cos (2x)}$}

\begin{sol}
 {$-\sqrt{2}/2$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
 {$\ds \lim_{x\to 0} \frac{\sin (5x)}{x}$}

\begin{sol}
  {$5$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
{$\ds \lim_{x\to 0^+} \frac{e^x-1}{x^2}$}

\begin{sol}
 {$\infty$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
 {$\ds \lim_{x\to 0^+} \frac{e^x-x-1}{x^2}$}

\begin{sol}
  {$1/2$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
 {$\ds \lim_{x\to 0^+} \frac{x-\sin x}{x^3-x^2}$}

\begin{sol}
  {$0$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
 {$\ds \lim_{x\to \infty} \frac{\Big(\ln x\Big)^2}{x}$}

\begin{sol}
  {$0$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
 {$\ds \lim_{x\to \infty} x^3-x^2$}

\begin{sol}
 {$\infty$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
{$\ds \lim_{x\to \infty} \sqrt{x}-\ln x$}

\begin{sol}
 {$\infty$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
{$\ds \lim_{x\to -\infty} xe^x$}

\begin{sol}
 {$0$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
 {$\ds \lim_{x\to 0^+} (\sin x)^{x}$ \\ Hint: use the Squeeze Theorem.}

\begin{sol}
 {$1$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
{$\ds \lim_{x\to 1^+} (1-x)^{1-x}$}

\begin{sol}
 {$1$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
{$\ds \lim_{x\to \infty} (x)^{1/x}$}

\begin{sol}
 {$1$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
{$\ds \lim_{x\to 1^+} (\ln x)^{1-x}$}

\begin{sol}
 {$1$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
 {$\ds \lim_{x\to \infty} (1+x)^{1/x}$}

\begin{sol}
{$1$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
 {$\ds \lim_{x\to \infty} (1+x^2)^{1/x}$}

\begin{sol}
 {$1$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
 {$\ds \lim_{x\to \pi/2} \tan x\cos x$}

\begin{sol}
  {$1$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
{$\ds \lim_{x\to \pi/2} \tan x\sin (2x)$}

\begin{sol}
 {$2$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
{$\ds \lim_{x\to 1^+} \frac{1}{\ln x} - \frac{1}{x-1}$}

\begin{sol}
{$1/2$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
 {$\ds \lim_{x\to 3^+} \frac{5}{x^2-9} - \frac{x}{x-3}$}

\begin{sol}
{$-\infty$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
{$\ds \lim_{x\to \infty} x\tan (1/x)$}

\begin{sol}
{$1$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
{$\ds \lim_{x\to \infty} \frac{(\ln x)^3}{x}$}

\begin{sol}
{$0$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
 {$\ds \lim_{x\to 1} \frac{x^2+x-2}{\ln x}$}

\begin{sol}
  {$3$}
\end{sol}

\end{ex}

%%%%%%%%%%
\begin{ex}
{$\ds \lim_{x\to 0^+} xe^{1/x}$}

\begin{sol}
 {$\infty$}
\end{sol}

\end{ex}

\begin{ex}
Discuss what happens if we try to use L'H\^{o}pital's rule to find the limit $\lim\limits_{x\rightarrow \infty}\dfrac{x+\sin x}{x+1}$.
\end{ex}

\end{enumialphparenastyle}

\end{multicols}
\clearpage


\documentclass{article}
\usepackage{amsmath}
\DeclareMathOperator*{\argmax}{argmax}

\begin{document}
\begin{Large}
\begin{center}
Building an Optimal Roster for Rotissarie Baseball
\end{center}

\end{Large}

\section*{Introduction}
Three years ago my roommate invited me to participate in his 10-team 5x5 rotissarie baseball league. I played baseball growing up, but never had much interest in the MLB. I accepted the offer to join -- after all, \$10 is a small price to pay for a reason to watch baseball and an opportunity to glorify the part of baseball that matters the most - the stats. The thing that's intrigued me the most has always been the draft -- we use an auction draft, which adds an extra dimension to the strategy. With no set draft positions, there's no comfort of the ability to take the best player availble. Every pick is fought for, and to draft well requires a coherent strategy.

That first year, I had a relatively naive strategy: a list of a few players at each position that I wanted and a long list of players I didn't. I didn't put much thought into my budget, so my lists were exhausted midway through the draft. The second year, I didn't improve my strategy much...I think. Try as I might, I cannot remember anything about this draft. Every other draft I remember vividly, but not this one. I was about to graduate college, there was a lot going on -- no room in my head for rotisserie drafts I guess. Last year, with the league expanding to 12 teams, I improved my strategy a bit. I started by coming up with goal values for each category based on previous winning values. I exported Steeamer/Razzball projections into an excel sheet, set up to calculate my team's score for each category and tell me what percentage of my goal I'd reached so far. I knew I needed to step up my strategy for this year.

At its most basic, we can think of a draft as an optimization problem, albeit with a dizzying number of unknowns. But, if we assume that our Steamer/Razzball projections and valuations are basically correct -- and make a few other simplifications -- we can formulate our draft as a basic maximization problem that we can solve to determine the best overall roster available through the draft.

\section*{Problem Formulation}
We start by formulating a linear equation for our roster's overall score:
\begin{align}
S = &\sum\limits_{i \in Hitters} {H_i^{adj} + R_i^{adj} + HR_i^{adj} + RBI_i^{adj} + AVG_i^{adj}} + \nonumber \\ &\sum\limits_{j \in Pitchers} {W_j^{adj} + K_j^{adj} + SV_j^{adj} + {\frac{1}{ERA_j}}^{adj} + {\frac{1}{WHIP_j}}^{adj}}
\end{align}
There are a few things to note here. Each of these is an adjusted value. Since each category has a different magnitude but each is worth the same number of points, we'll need to adjust them. Because we want each category to have the same scale but maintain each player's relative value in each, we'll need to center their means but preserve their variances. We'll use the sklearn StandardScaler method to do so (specifying to maintain the standard deviation). I've used average, ERA, and WHIP directly in this equation, despite the fact that they're actually calculated overall for the entire team from the base statistics. I experimented with separating them out, but (somewhat suprisingly) found no added value in doing so. For clarity, I've split the pitchers and hitters into two different summations, but we will end up combinining them into one. This will be equivalent to equation (1) since hitters do not generate pitching stats and pitchers do not generate hitting stats.
\begin{align}
S = \sum\limits_{p \in Players} {H_p^{adj} + R_p^{adj} + HR_p^{adj} + RBI_p^{adj} + AVG_p^{adj}} + \\ \nonumber {W_p^{adj} + K_p^{adj} + SV_p^{adj} + {\frac{1}{ERA_p}}^{adj} + {\frac{1}{WHIP_p}}^{adj}}
\end{align}
So, now that we have a single equation to calculate our score, we need to find the group of players that maximize it within the constraints of the league.

\begin{align}
Team = &\argmax_{s \in S} \sum\limits_{p \in s} {H_p^{adj} + R_p^{adj} + HR_p^{adj} + RBI_p^{adj} + AVG_p^{adj}} + \\ \nonumber &{W_p^{adj} + K_p^{adj} + SV_p^{adj} + {\frac{1}{ERA_p}}^{adj} + {\frac{1}{WHIP_p}}^{adj}} \\
\nonumber s.t. &\sum\limits_{p \in s} c(p) \leq 260 \mbox{ where c(p) is the cost of player p}\\
\nonumber &\{C,1B,2B,3B,SS,1B/3B,2B/SS,1B/2B/SS/3B,6 OF, 8 SP, 3 RP\} \in s \\
\nonumber &\sum\limits_{p \in s} N(p) = 25 \mbox{ where } N(p) = \begin{cases}
															1 & \exists!n \in s(n\mbox{ is name of player }p) \\
															0 & else \\
                                                         \end{cases}\\
\nonumber &\mbox{where S is the set of sets of all possible combinations of 25 players}
\end{align}
\pagebreak
\section*{Results \& Conclusion}
Using Gurobi, we can solve equation (3) to find our optimal set of players.

\begin{table}[ht] \caption{Hitters}
\centering
\begin{tabular}{c c c c c c c}
\hline\hline
Name & Position & R & HR & RBI & SB & AVG \\
\hline
Jorge Polanco & SS & 94.6 & 19.3 & 84.7 & 7.6 & 0.281 \\
Jose Abreu	& 1B & 84.5 & 31.1 & 100.4 & 2.6 & 0.276 \\
Nicholas Castellanos & OF & 87.7 & 26.9 & 88.4 & 3.0 & 0.274 \\
Bryan Reynolds & OF & 83.1 & 17.9 & 66.0 & 6.3 & 0.288 \\
J.T. Realmuto & C & 77.8 & 24.0 & 76.4 & 5.8 & 0.270 \\
Matt Chapman & 3B & 94.2 & 34.7 & 98.2 & 2.7 & 0.258 \\
Paul DeJong & SS & 82.0 & 28.4 & 84.9 & 6.8 & 0.253 \\
Eric Hosmer & 1B & 79.7 & 23.4 & 80.2 & 3.4 & 0.263 \\
David Peralta & OF & 80.6 & 20.6 & 74.4 & 2.5 & 0.279 \\
Franmil Reyes & OF & 82.1 & 37.0 & 101.5 & 1.3 & 0.260 \\
Jason Heyward & OF & 72.0 & 15.8 & 72.1 & 6.5 & 0.261 \\
Shin-Soo Choo & OF & 83.3 & 21.0 & 61.6 & 9.0 & 0.249 \\
Jonathan Schoop & 2B & 67.4 & 26.5 & 77.0 & 2.3 & 0.262 \\
Brandon Lowe & 2B & 70.9 & 23.7 & 71.6 & 6.7 & 0.246 \\
\hline
Total &  & 1139.9 & 350.3 & 1137.4 & 66.5 & 0.267 \\

\hline
\end{tabular}
\end{table}

\begin{table}[ht] \caption{Pitchers}
\centering
\begin{tabular}{c c c c c c c}
\hline\hline
Name & Position & W & SV & K & ERA & WHIP \\
\hline
Gerritt Cole & SP & 14.9 & 0.0 & 273.3 & 3.250 & 1.040 \\
Trevor Bauer & SP & 11.6 & 0.0 & 228.4 & 3.920 & 1.240 \\
Luis Castillo & SP & 11.3 & 0.0 & 203.4 & 3.820 & 1.250 \\
Lucas Giolito & SP & 10.8 & 0.0 & 198.4 & 4.240 & 1.270 \\
Matthew Boyd & SP & 10.0 & 0.0 & 194.2 & 4.320 & 1.220 \\
Robbie Ray & SP & 9.9 & 0.0 & 197.5 & 4.120 & 1.320 \\
Caleb Smith & SP & 8.2 & 0.0 & 172.4 & 4.580 & 1.320 \\
Mike Foltynewicz & SP & 9.7 & 0.0 & 157.5 & 4.70 & 1.350 \\
Joe Jimenez & RP & 2.8 & 20.0 & 69.1 & 4.150 & 1.260  \\
Hansel Robles & RP & 3.3 & 20.0 & 70.9 & 4.180 & 1.270 \\
Jose Leclerc & RP & 3.5 & 20.0 & 91.9 & 3.390 & 1.210 \\
\hline
Total & & 96.0 & 60.0 & 1857.0 & 4.072 & 1.246 \\
\hline
\end{tabular}
\end{table}
\newpage
\noindent Then, we can use the scores and rankings from last season (with my team removed) to get a score for our team. The category and overall rankings are as follows:

\begin{table}[ht] \caption{Category Rankings}
\centering
\begin{tabular}{c c c c c c c c c c c}
\hline\hline
Rank & R & HR & RBI & SB & AVG & K & W & SV & ERA & WHIP \\
\hline
1 & 1241 & 433 & 1198 & 141 & 0.2768 & 1788 & 112 & 115 & 3.493 & 1.102 \\
2 & 1174 & 382 & 1147 & 127 & 0.274 & 1725 & 109 & 105 & 3.665 & 1.131 \\
3 & 1159 & 355 & 1088 & 123 & 0.2735 & 1643 & 99 & 79 & 3.76 & 1.21 \\
4 & 1136 & 353 & 1077 & 121 & 0.271 & 1598 & 98 & 73 & 3.898 & 1.216 \\
5 & 1110 & 352 & 1075 & 113 & 0.2705 & 1531 & 97 & 59 & 3.907 & 1.244 \\
6 & 1067 & 332 & 1030 & 110 & 0.2645 & 1480 & 93 & 59 & 4.107 & 1.247 \\
7 & 1006 & 321 & 1016 & 106 & 0.2642 & 1391 & 85 & 55 & 4.112 & 1.262 \\
8 & 997 & 302 & 1000 & 97 & 0.2641 & 1387 & 84 & 54 & 4.144 & 1.267 \\
9 & 974 & 291 & 955 & 94 & 0.262 & 1336 & 80 & 42 & 4.217 & 1.284 \\
10 & 966 & 284 & 905 & 73 & 0.2601 & 1330 & 74 & 39 & 4.444 & 1.291 \\
11 & 898 & 260 & 897 & 72 & 0.2592 & 1132 & 64 & 3 & 4.616 & 1.339 \\ [1ex]
\hline
\end{tabular}
\end{table}

\begin{table}[ht] \caption{Overall Rankings}
\centering
\begin{tabular}{c c}
\hline\hline
Rank & Points \\
\hline
1 & 95.5 \\
2 & 83 \\
3 & 74 \\
4 & 74 \\
5 & 73 \\
6 & 70 \\
7 & 69.5 \\
8 & 57 \\
9 & 51 \\
10 & 45.5 \\
11 & 24 \\
[1ex]
\hline
\end{tabular}
\end{table}

We can obtain scores for each of our values by comparing them to our table of results for last season. As a refresher, rotissarie baseball leagues are scored by combining each of the category for the whole team, comparing all teams for each category, and granting 12 points to the first place team, 11 points to the second, etc.

\begin{table}[ht] \caption{Team Score}
\centering
\begin{tabular}{c c}
\hline\hline
Value & Points \\
\hline
R & 9 \\
HR & 5 \\
RBI & 9 \\
SB & 5 \\
AVG & 7 \\
K & 12 \\
W & 7 \\
SV & 8 \\
ERA & 7 \\
WHIP & 7 \\
\hline
Total & 76 \\
[1ex]
\hline
\end{tabular}
\end{table}
\newpage
All things considered, this is not a bad draft - if we refer to table 4, this puts us in 3rd place in the league. However, if we were to accept that the draft is a simple linear optimization problem, we would expect that our team would be in first place. So, why isn't it? We could conclude that simply drafting the perfect team is not good enough to win a rotissarie baseball league -- that drafting perfectly could get you a very good team, but playing the waiver wire is required to come in first place. I think this is probably true, but I think there's something else that it's very important we do not overlook: a rotissarie baseball auction draft cannot actually be represented by a linear optimization problem. Once your team gets to first place in any particular category adding to that category further gives you no additional value. Nothing is gained by running up the score in any category, but our model doesn't represent this. To truly model the draft, we'd need to optimize a stepwise function, applying the scoring function to our model (and, of course, come up with a new way to score our model - perhaps by applying it in a series of mock drafts). This will be explored in future work. Despite this limitation, our model does very well at approximating an optimal rotissarie baseball team for our particular league. Another thing that is important to note is that this model depends completely on the accuracy of the Steamer/Razzball predictions. Despite using these predictions for a number of years, I have not analyzed their accuracy.
\newpage
\appendix
\section*{Appendix}
\end{document}


\subsection{Accuracy of Set-Based Inversion}\label{sec:ch03-set}

%%%%%%%%%%%%%% discretization discussion, software contribution in later section

In this section, we address how to measure the distance between measures that are defined on different discretizations to ensure that the computations make sense mathematically.
There is a correspondence between the objects from measure theory that are involved in the SIP, and the finite-state counterparts in the algorithmic implementation.
Therefore, we begin by summarizing the relationship between the random samples we generate and the Borel $\sa$.

The measures computed from Algorithm~\ref{alg:inv_density} are defined on a set of samples $S = \set{\param^{(j)}}_{j=1}^{\nsamps}$ which implicitly define a Voronoi-cell partition $\set{\VV^{(j)}}_{j=1}^{\nsamps}$ of the parameter space $\pspace$.
We let $\BB_{\pspace, \nsamps}$ denote the \emph{computational algebra} generated by $\set{\VV^{(j)}}_{j=1}^{\nsamps}$, i.e., using standard measure theory notation,
$$
	\BB_{\pspace,\nsamps} = \sigma\left(\set{\VV^{(j)}}_{j=1}^{\nsamps}\right).
$$
$\BB_{\pspace, \nsamps}\subset \BB_\pspace$ and the events $A\in B_{\pspace, \nsamps}$ represent the $A\in\BB_\pspace$ for which we can compute probabilities and make inferences.
While Algorithm~\ref{alg:inv_density} ultimately defines a probability measure implicitly on $(\pspace,\BB_\pspace)$, computationally this is almost never done; instead, the measures are only interrogated on the computational algebra associated with the finite set of samples.

Different sets $S_k = \set{\param^{(i)}}_{i=1}^{\nsamps_k}$, where the $\param^{(i)}$'s and $\nsamps_k$'s may differ for each $k$, will lead to different measures computed from Algorithm~\ref{alg:inv_density}.
Each $S_k$ induces a computational algebra which we index using the notation $\BB_{k}$ for simplicity, where it is understood that $\BB_k = \BB_{\pspace, \nsamps_k}$.
The fact that solutions are defined on different sub-$\sa$s poses an immediate problem with respect to a computational approach to computing the distances between measures $\PP_{\pspace, \ndiscs, \nsamps_1}$ and $\PP_{\pspace, \ndiscs, \nsamps_2}$ even if $\nsamps_1=\nsamps_2$.
See Figure~\ref{fig:voronoi_issues} for an example illustration of this scenario.

\begin{figure}[ht]
\centering
	\begin{minipage}{.4875\textwidth}
		\includegraphics[width=\linewidth]{./images/voronoi_diagrams/voronoi_diagram_N25_r0}
	\end{minipage}
	\begin{minipage}{.4875\textwidth}
		\includegraphics[width=\linewidth]{./images/voronoi_diagrams/voronoi_diagram_N25_r10}
	\end{minipage}
\caption{
Two different Voronoi partitions induced by $\nsamps_1 = \nsamps_2 = 25 $ uniform i.i.d.~random samples.
}
\label{fig:voronoi_issues}
\end{figure}

The proof of the following Lemma describes how to extend {\em any} probability measure defined on a computational algebra to the full $\sa$ $\pborel$, which we exploit in Algorithm~\ref{alg:totvar_disc}.

\begin{lem}
\label{lem:measuresets}
Let $\mu$ be a measure on $(\pspace, \BB_\pspace)$, $\set{\VV^{(j)}}_{j=1}^{\nsamps}$ be a partition of $\pspace$, and $\BB_{\pspace, \nsamps}$ the computational algebra generated by $\set{\VV^{(j)}}_{j=1}^{\nsamps}$.
Assume $\mu (\VV^{(j)}) > 0 \; \forall \; j=1,\hdots, \nsamps$.
Then, there exists a probability measure $\eta$ on $(\pspace, \BB_\pspace)$ such that $\eta(A) = \eta_\nsamps(A) \; \forall \; A\in\BB_{\pspace, \nsamps}$.
\end{lem}
In the proof below, we use $\eta_\nsamps$ and $\mu$ to construct a type of discrete Radon-Nikodym derivative of $\eta$.
This is motivated by the formal structure of solutions given by Algorithm~\ref{alg:inv_density}.
The proof of Lemma~\ref{lem:measuresets} can be found in Appendix~\ref{app:measuresets}, but the two key equations involved are reproduced here for later reference:

\begin{equation}\label{eq:finiteradon}
f_\nsamps (\param) = \sum_{j=1}^{\nsamps} \frac{\eta_\nsamps (\VV^{(j)}) }{\mu (\VV^{(j)})} \Chi_{\VV^{(j)}} (\param).
\end{equation}

Then, for any $A\in\BB_\pspace$, define

\begin{equation}\label{eq:approxmeasure}
\eta (A) = \int_A f_\nsamps (\param) \, d\mu.
\end{equation}

We note that in practice, $\Chi_{\VV^{(j)}} (\param)$ requires the use of nearest-neighbor computations, but otherwise evaluation of Eq.~\eqref{eq:finiteradon} is straightforward.

\begin{algorithm}
\DontPrintSemicolon
\caption{Total Variation Discretization}
\label{alg:totvar_disc}
Let $(\pspace, B_{\pspace, \nsamps_1}, \eta_{\nsamps_1} )$ and $(\pspace, B_{\pspace, \nsamps_2}, \eta_{\nsamps_2} )$ be given.\\

Construct $f_{\nsamps_1}$ and $f_{\nsamps_2}$ and corresponding $\eta_1, \eta_2$ using Eq.~\eqref{eq:finiteradon} and Eq.~\eqref{eq:approxmeasure}, respectively.

Use Monte Carlo sampling to approximate
$$d_\text{TV}(\eta_1, \eta_2) = \int_\pspace f_{\nsamps_1}\lam - f_{\nsamps_2}\lam \, d\mu.$$
\end{algorithm}

Since we now have a way to extend probability measures defined on $(\pspace, \BB_{\pspace, \nsamps})$ to  probability measure on $(\pspace, \BB_{\pspace})$, we can use simple Monte Carlo approximation schemes to the Total Variation distance between two probability measures defined on two separate computational algebras.
This is demonstrated in Algorithm~\ref{alg:totvar_disc}.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
\section{Numerical Results and Analysis}\label{sec:ch03-examples}
We first investigate what values of $\nsamps$ are appropriate provided the goal is to resolve $d(\PP_{\pspace, \ndiscs, \nsamps}, \PP_{\pspace, \ndiscs})$ to a desired level of accuracy so that
\begin{equation}\label{eq:newobjective}
d(\PP_{\pspace, \ndiscs, \nsamps}, \PP_{\pspace,\ndiscs}) < \tau,
\end{equation}
where $\tau$ is some designated tolerance.
%
% Recall that this discussion of error is in reference to some fixed $\qoi\in\qspace$ to which Algorithm~\ref{alg:inv_density} is applied.
% The inequality presented in Eq.~\eqref{eq:set-triangleineq} holds for probability measures induced by any map in $\qspace$, though we obscure the dependence on $\qoi$ for the time being.
% To this end, we introduce notation of the form $\PP_{\pspace, \ndiscs, \nsamps}^{\qoi}$ when we want to distinguish between measures constructed from inverting a particular map $\qoi\in\qspace$.
%
The reason for this is because the choice of $\qoi$ will influence the number of model solutions necessary to accurately solve the SIP as shown in \cite{BGE+15}.
Different choices for $\qoi$ may lead to radically different values for $\nsamps$ in order to achieve the same bound on $d(\PP_{\pspace, \ndiscs, \nsamps}, \PP_{\pspace, \ndiscs})$ as we see in the following examples.
By phrasing the analysis in terms of metrics, we are able to answer more broadly generalizable questions about error, including those regarding convergence rates and global accuracy of the estimates.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input{ch03/experimental_setup}
\input{ch03/rotation_example}
\input{ch03/skew_example}
\input{ch03/skew_example_3d}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


% Created 2018-02-09 vie 13:42
\documentclass[letterpaper]{scrartcl}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{fixltx2e}
\usepackage{graphicx}
\usepackage{longtable}
\usepackage{float}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{marvosym}
\usepackage{wasysym}
\usepackage{amssymb}
\usepackage{hyperref}
\tolerance=1000
\usepackage{khpreamble}
\author{Kjartan Halvorsen}
\date{\today}
\title{Sampling and anti-aliasing exercise}
\hypersetup{
  pdfkeywords={},
  pdfsubject={},
  pdfcreator={Emacs 24.5.1 (Org mode 8.2.10)}}
\begin{document}

\maketitle

\section*{The Bessel filter}
\label{sec-1}
  A second order Bessel filter
\[ H(s) = \frac{3}{\big(s/\omega_0\big)^2 + 3\big(s/\omega_0\big) + 3}, \]
has been designed to give attenuation of 0.1 at the Nyquist frequency, i.e. \(|H(i\omega_N)| = 0.1\).

\begin{center}
\includegraphics[width=0.8\linewidth]{../figures/ps7-bessel-bode}
\end{center}

\begin{enumerate}
\item What is the bandwidth of the filter?
\item Determine the filter parameter \(\omega_0\) in terms of \(\omega_N\).
\item What is the phase shift at the Nyquist frequency?
\item Assume that the cross-over frequency of the loop gain is \(\omega_c = 0.1\omega_N\). What is the phase shift at this frequency?
\item What delay $T$  does this correspond to? Approximate the filter as a pure delay \(H(i\omega) \approx \mexp{-i\omega T}\)?
\item How large is the delay in terms of the sampling period \(h = \frac{\pi}{\omega_N}\)?
\end{enumerate}

\section*{Choice of bandwidth of antialiasing filter}
\label{sec-2}

\begin{center}
\includegraphics[width=0.4\linewidth]{../figures/Astrom-fig73.png}
\end{center}
The table can be used to find the delay incurred by an antialiasing filter. For instance, if we want to have an attenuation at the Nyquist frequency equal to \(\beta = 0.01\), then a fourth-order Bessel low-pass filter will give a delay of about \(T_d = 3.2 h\), that is a delay of more than 3 sampling periods.
\begin{enumerate}
\item What is the advantage of using an antialiasing filter of higher order than 2?
\item If we can only accept a time delay of maximum two sampling periods, what is the attenutation we can obtain at the Nyquist frequency?
\item \(\omega_B = \omega_N\) means that the antialiasing filter is chosen so that the Nyquist frequency is the same as the bandwidth of the filter. The attenuation at \(\omega_N\) is then 0.7=-3dB. If the sampling period is 0.4 seconds. What is the time delay due to a sixth-order antialiasing filter?
\item What is the attenuation of a forth-order filter at \(5\omega_B\)
\end{enumerate}

\clearpage

\section*{Notch filter, preliminary exercise}
\label{sec-3}
Sketch the Bode diagram of the (unrealizable) filter
\[ F(s) = s^2 + \omega_1^2 = (s+i\omega_1)(s-i\omega_1)\]
\[ |F(i\omega)| = |(i\omega + i\omega_1)||(i\omega - i\omega_1)| = |\omega - (-\omega_1)||\omega - \omega_1|\]
\[ \arg F(i\omega) = \arg i(\omega + \omega_1) + \arg i(\omega -\omega_1) = 2\arg i + 0 + \arg(\omega - \omega_1) \]
\begin{center}
\includegraphics[width=0.85\linewidth]{../figures/bode-w1-empty}
\end{center}

\section*{Notch filter}
\label{sec-4}
Sketch the Bode diagram (\(Q=10\))of the second order continous-time notch filter
\[ F(s) = \frac{s^2 + \omega_1^2}{s^2 + \frac{\omega_1}{Q}s + \omega_1^2} \]

\begin{center}
\includegraphics[width=0.85\linewidth]{../figures/bode-notch-empty}
\end{center}
% Emacs 24.5.1 (Org mode 8.2.10)
\end{document}