\subsection{First-order satisfiability}\label{subsec:first_order_satisfiability}

\begin{definition}\label{def:first_order_substitution}
  As in \hyperref[subsec:propositional_logic]{propositional logic}, we sometimes want to perform substitution, however we have different types of syntactic objects (terms and formulas) which have different substitution rules. The notion of free and bound variables further complicates us --- see for example the problems outlined in \fullref{rem:first_order_substitution_renaming_justification}. In particular, this means that an analogous to \fullref{thm:propositional_substitution_equivalence} theorem cannot longer justify substitution as it is done in \fullref{alg:conjunctive_normal_form_reduction} --- we can have weaker statements as in \fullref{thm:propositional_substitution_equivalence} that implicitly rely on variable renaming in order to hold. This implies that it is of no practical use to define substitution of a first-order subformula inside another formula as it is done in \fullref{def:propositional_substitution}. Instead, we concert ourselves with substituting variables --- propositional variables with first-order formulas and first-order variables with first-order terms. Furthermore, since this does not complicate us, we allow substituting arbitrary terms rather than only first-order variables.

  While substituting a propositional variable is the syntactic analog to applying \hyperref[def:boolean_function]{Boolean functions} to different variables or propositional constants, substituting a first-order variable can express applying \hyperref[def:function]{arbitrary functions} to different first-order variables or arbitrary constants. For example, in a suitable language, we can apply \( \log(x) \) to the constant \( e \) by substituting \( x \) with \( e \) to obtain the ground term \( \log(e) \).

  As in \fullref{def:propositional_substitution}, we define different kinds of (single) \term{substitution} in more generality that in e.g. \cite[def. 15.25]{OpenLogicFull}. Where applicable, \term{simultaneous substitution} is defined via the same trick as in \fullref{def:propositional_substitution}.

  \begin{thmenum}
    \thmitem{def:first_order_substitution/propositional} Let \( \varphi \) be a \hyperref[def:propositional_syntax/formula]{propositional formula} with variables \( \boldop{Var}(\varphi) = \set{ P_1, \ldots, P_n } \). For brevity, denote \( V \coloneqq \boldop{Var}(\varphi) \). Let \( \Theta = \set{ \theta_1, \ldots, \theta_n } \) be a set of \hyperref[def:first_order_syntax/formula]{first-order formulas}.

    It does not make sense to replace a single propositional variable by a single formula. Furthermore, a first-order formula \( \theta_k \) cannot possibly contain any of the propositional variables \( P_1, \ldots, P_n \). This allows us to introduce a simplification of the simultaneous substitution based on \eqref{eq:def:propositional_substitution/single} as
    \begin{equation}\label{eq:def:first_order_substitution/propositional}
      \varphi[V \mapsto \Theta] \coloneqq \begin{cases}
        \varphi,                                                    &\varphi \in \set{ \top, \bot } \\
        \theta_k,                                                   &\varphi = \theta_k \T{for some} k = 1, \ldots, n \\
        \neg \psi[V \mapsto \Theta],                                &\varphi = \neg \psi \\
        \psi_1[V \mapsto \Theta] \bincirc \psi_2[V \mapsto \Theta], &\varphi = \psi_1 \bincirc \psi_2, \bincirc \in \Sigma.
      \end{cases}
    \end{equation}

    As in \fullref{def:propositional_substitution}, it is not strictly necessary for any of the variables to belong to \( \boldop{Var}(\varphi) \).

    \thmitem{def:first_order_substitution/term_in_term} We define the substitution of the \hyperref[def:first_order_syntax/term]{first-order term} \( \kappa \) with \( \mu \) in the term \( \tau \) as
    \begin{equation}\label{eq:def:first_order_substitution/term_in_term}
      \tau[\kappa \mapsto \mu] \coloneqq \begin{cases}
        \mu,                                                               &\tau = \kappa, \\
        \tau,                                                              &\tau \neq \kappa \T{and} \tau \in \boldop{Var}, \\
        f(\tau_1[\kappa \mapsto \mu], \ldots, \tau_n[\kappa \mapsto \mu]), &\tau \neq \kappa \T{and} \tau = f(\tau_1, \ldots, \tau_n).
      \end{cases}
    \end{equation}

    It is not strictly necessary for \( \kappa \) to be a \hyperref[def:first_order_syntax/subterm]{subterm} of \( \tau \).

    \thmitem{def:first_order_substitution/term_in_formula} This case is more complicated. We define the substitution of the term \( \kappa \) with the term \( \mu \) in the first-order formula \( \varphi \) as
    \begin{equation}\label{eq:def:first_order_substitution/term_in_formula}
      \varphi[\kappa \mapsto \mu] \coloneqq \begin{cases}
        \varphi,                                                           &\varphi \in \set{ \top, \bot }, \\
        p(\tau_1[\kappa \mapsto \mu], \ldots, \tau_n[\kappa \mapsto \mu]), &\varphi = p(\tau_1, \ldots, \tau_n), \\
        \tau_1[\kappa \mapsto \mu] \doteq \tau_2[\kappa \mapsto \mu],      &\varphi = \tau_1 \doteq \tau_2, \\
        \neg \psi[\kappa \mapsto \mu],                                     &\varphi = \neg \psi, \\
        \psi_1[\kappa \mapsto \mu] \bincirc \psi_2[\xi \mapsto \mu],       &\varphi = \psi_1 \bincirc \psi_2, \bincirc \in \Sigma, \\
        (\dagger),                                                         &\varphi = \quantifier{Q}{\xi} \psi, Q \in \set{ \forall, \exists },
      \end{cases}
    \end{equation}
    where
    \begin{empheq}[left=(\dagger) \coloneqq \empheqlbrace]{align}
      &\varphi,                                                                        && \xi \in \boldop{Var}(\kappa), \label{eq:def:first_order_substitution/term_in_formula/quantifiers/trivial} \\
      &\quantifier{Q}{\xi} \parens[\Big]{\psi[\kappa \mapsto \mu]},                    && \xi \not\in \boldop{Var}(\kappa) \cup \boldop{Var}(\mu), \label{eq:def:first_order_substitution/term_in_formula/quantifiers/direct} \\
      &\quantifier{Q}{\eta} \parens[\Big]{\psi[\xi \mapsto \eta][\kappa \mapsto \mu]}, && \xi \not\in \boldop{Var}(\kappa) \T{and} \xi \in \boldop{Var}(\mu) \T{and} &\label{eq:def:first_order_substitution/term_in_formula/quantifiers/renaming} \\
                                                                                &&& \eta \not\in \boldop{Var}(\kappa) \cup \boldop{Var}(\mu) \cup \boldop{Var}(\psi). \nonumber
    \end{empheq}

    In \eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/renaming}, we chose a new variable \( \eta \). We implicitly assume that there exist enough variables in the language, so that we can find \( \eta \) that satisfies the condition. In order to fully avoid nondeterminism in the choice of \( \eta \), we can pick a well-ordering on the set \( \boldop{Var} \) and always choose \( \eta \) to be the smallest variable not present in \( \varphi \). This rule is called \term{renaming of the bound variables} \( \xi \) to \( \eta \) and is done to mitigate capturing as described in \fullref{rem:first_order_substitution_renaming_justification/capturing}.

    We could avoid the rule for renaming (as it is done in \cite[def. 15.25]{OpenLogicFull}), however renaming both free and bound variables is natural and is often done in practice. For example, consider the \hyperref[def:peano_arithmetic]{Peano arithmetic} formula \enquote{there exists \( n \) such that \( nm \) is even}. Note that the bound variable \( n \) is renamed to \( k \) and the free variable \( m \) to \( n \) in the larger formula \enquote{for every \( n \) there exists \( k \) such that \( kn \) is even}.

    The rule \eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/trivial} may seem redundant, but when doing inductive proofs (e.g. the proof of \fullref{thm:renaming_assignment_compatibility}), we usually need to separately consider the cases where \( \xi \in \boldop{Var}(\kappa) \) and \( \xi \not\in \boldop{Var}(\kappa) \setminus \boldop{Var}(\mu) \) and the rule \eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/direct} being trivial simplifies the proofs.

    See \fullref{rem:first_order_substitution_parentheses} regarding the additional parentheses in \eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/renaming}.

    See \fullref{ex:first_order_substitution} for examples of applying the different quantifier rules.
  \end{thmenum}
\end{definition}

\begin{remark}\label{rem:first_order_substitution_renaming_justification}
  The renaming rule \eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/renaming} is designed to mitigate the following two problems (compared to \eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/direct}):

  \begin{thmenum}
    \thmitem{rem:first_order_substitution_renaming_justification/capturing} Renaming mitigates \enquote{capturing} free variables as in
    \begin{equation*}
      \parens[\Big]{ \qforall \eta p(\xi, \eta) }[\xi \mapsto \eta] = \qforall \eta p(\eta, \eta)
    \end{equation*}
    by instead producing, up to a choice of new variables, the formula
    \begin{equation*}
      \parens[\Big]{ \qforall \eta p(\xi, \eta) }[\xi \mapsto \eta] = \qforall \zeta p(\eta, \zeta).
    \end{equation*}

    \thmitem{rem:first_order_substitution_renaming_justification/colliding} Renaming mitigates \enquote{colliding} multiple bound variables as in
    \begin{equation*}
      \parens[\Big]{ \qforall \xi \qforall \eta p(\xi, \eta) }[\xi \mapsto \eta] = \qforall \xi \qforall \eta p(\eta, \eta)
    \end{equation*}
    by instead producing, up to a choice of new variables, the formula
    \begin{equation*}
      \parens[\Big]{ \qforall \xi \qforall \eta p(\xi, \eta) }[\xi \mapsto \eta] = \qforall \zeta \qforall \sigma p(\zeta, \sigma).
    \end{equation*}
  \end{thmenum}
\end{remark}

\begin{remark}\label{rem:first_order_substitution_parentheses}
  When performing \hyperref[def:propositional_substitution]{substitution}, it is sometimes convenient to add additional parentheses to avoid ambiguity. For example, while parentheses around quantifier expressions are not necessary by the syntax of first-order logic, adding such parentheses helps avoid the ambiguity in
  \begin{equation*}
    \qforall \xi p(\xi, \eta) [\eta \mapsto \zeta].
  \end{equation*}

  Instead, we either write
  \begin{equation*}
    \parens[\Big]{ \qforall \xi p(\xi, \eta) } [\eta \mapsto \zeta]
  \end{equation*}
  or
  \begin{equation*}
    \qforall \xi \parens[\Big]{ p(\xi, \eta)[\eta \mapsto \zeta] }.
  \end{equation*}

  This convention is only part of the metasyntax and the parentheses are not part of the syntax of the formulas themselves.
\end{remark}

\begin{example}\label{ex:first_order_substitution}
  The following term substitutions should justify the distinct cases in \eqref{eq:def:first_order_substitution/term_in_formula}:
  \begin{thmenum}
    \thmitem{ex:first_order_substitution/1} The trivial case without actual substitution:
    \begin{align*}
      &\phantom{{}={}}
      \parens[\Big]{\qforall \xi p(\xi, \eta)}[\xi \mapsto \eta]
      = \\ &=
      \parens[\Big]{\qforall \xi p(\xi, \eta)}[\xi \mapsto \eta]
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/trivial}} = \\ &=
      \qforall \xi p(\xi, \eta).
    \end{align*}

    \Fullref{ex:first_order_substitution/5} demonstrates that this does not work for nested substitution.

    \thmitem{ex:first_order_substitution/2} A simple substitution without renaming:
    \begin{align*}
      &\phantom{{}={}}
      \parens[\Big]{\qforall \xi p(\xi, \eta)}[\eta \mapsto \zeta]
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/direct}} = \\ &=
      \qforall \xi \parens[\Big]{p(\xi, \eta)[\eta \mapsto \zeta]}
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula}} = \\ &=
      \qforall \xi p(\xi, \zeta).
    \end{align*}

    \thmitem{ex:first_order_substitution/3} A simple renaming without actual substitution:
    \begin{align*}
      &\phantom{{}={}}
      \parens[\Big]{\qforall \xi p(\xi, \eta)}[\eta \mapsto \xi]
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/renaming}} = \\ &=
      \qforall \zeta \parens[\Big]{p(\xi, \eta)[\xi \mapsto \zeta][\eta \mapsto \xi]}
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula}} = \\ &=
      \qforall \zeta p(\zeta, \xi).
    \end{align*}

    \thmitem{ex:first_order_substitution/4} \Fullref{ex:first_order_substitution/3}, but with \( \mu \) in \eqref{eq:def:first_order_substitution/term_in_formula} containing \( \xi \) indirectly:
    \begin{align*}
      &\phantom{{}={}}
      \parens[\Big]{\qforall \xi p(\xi, \eta)}[\eta \mapsto f(\xi)]
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/renaming}} = \\ &=
      \qforall \zeta \parens[\Big]{p(\xi, \eta)[\xi \mapsto \zeta][\eta \mapsto f(\xi)]}
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula}} = \\ &=
      \qforall \zeta p(\zeta, f(\xi)).
    \end{align*}

    \thmitem{ex:first_order_substitution/5} Only renaming with multiple quantifiers which shows the limitations of \eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/trivial}:
    \begin{align*}
      &\phantom{{}={}}
      \parens[\Big]{\qforall \eta \qexists \xi p(\xi, \eta)}[\xi \mapsto \eta]
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/renaming}} = \\ &=
      \qforall \zeta \parens*{ \parens[\Big]{ \qforall \xi p(\xi, \eta) }[\eta \mapsto \zeta][\xi \mapsto \eta]}
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula}} = \\ &=
      \qforall \zeta \parens*{ \parens[\Big]{ \qforall \xi p(\xi, \zeta) }[\xi \mapsto \eta]}
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/trivial}} = \\ &=
      \qforall \zeta \qforall \xi p(\xi, \zeta).
    \end{align*}

    \thmitem{ex:first_order_substitution/6} Both renaming and substitution with multiple quantifiers:
    \begin{align*}
      &\phantom{{}={}}
      \parens[\Big]{ \qforall \eta (p(\xi, \eta) \vee \qforall \xi p(\xi, \eta)) }[\xi \mapsto \eta]
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/renaming}} = \\ &=
      \qforall \zeta \parens*{ \parens[\Big]{ p(\xi, \eta) \vee \qexists \xi p(\xi, \eta) }[\eta \mapsto \zeta][\xi \mapsto \eta] }
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula}} = \\ &=
      \qforall \zeta \parens*{ p(\eta, \zeta) \vee \parens[\Big]{ \qexists \xi p(\xi, \eta) }[\eta \mapsto \zeta][\xi \mapsto \eta] }
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/direct}} = \\ &=
      \qforall \zeta \parens*{ p(\eta, \zeta) \vee \parens[\Big]{ \qexists \xi p(\xi, \zeta) }[\xi \mapsto \eta] }
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/trivial}} = \\ &=
      \parens[\Big]{ \qforall \zeta p(\eta, \zeta) } \vee \parens[\Big]{ \qexists \xi p(\xi, \zeta) }.
    \end{align*}

    \thmitem{ex:first_order_substitution/7} Substitution of more general terms than variables with renaming of term's variables:
    \begin{align*}
      &\phantom{{}={}}
      \parens*{\qforall \xi p(\xi, \eta, f(\eta))}[f(\eta) \mapsto g(\eta, \xi)]
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/renaming}} = \\ &=
      \qforall \zeta \parens[\Big]{p(\xi, \eta, f(\eta))[\xi \mapsto \zeta][f(\eta) \mapsto g(\eta, \xi)]}
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula}} = \\ &=
      \qforall \zeta p(\zeta, \eta, g(\eta, \xi)).
    \end{align*}
  \end{thmenum}
\end{example}

\begin{proposition}\label{thm:renaming_assignment_compatibility}
  We will show how \hyperref[rem:first_order_substitution_renaming_justification]{syntactic renaming} is compatible with a certain \enquote{semantic renaming}.

  Fix a \hyperref[def:first_order_syntax]{first-order language} \( \mscrL \), a \hyperref[def:first_order_structure]{structure} \( \mscrX = (X, I) \) on \( \mscrL \) and a \hyperref[def:first_order_valuation/variable_assignment]{variable assignment} \( v \) in \( \mscrX \).

  \begin{thmenum}
    \thmitem{thm:renaming_assignment_compatibility/terms} For any term \( \tau \) and any two variables \( \xi \) and \( \eta \), we have
    \begin{equation}\label{eq:thm:renaming_assignment_compatibility/terms}
      \tau\Bracks{v_{\xi \mapsto \eta}}
      =
      \parens[\Big]{ \tau[\xi \mapsto \eta] }\Bracks{v}.
    \end{equation}

    \thmitem{thm:renaming_assignment_compatibility/formulas} For any formula \( \varphi \), any variable \( \xi \) and any other variable \( \eta \) not in \( \boldop{Var}(\varphi) \) we have
    \begin{equation}\label{eq:thm:renaming_assignment_compatibility/formulas}
      \varphi\Bracks{v_{\xi \mapsto \eta}}
      =
      \parens[\Big]{ \varphi[\xi \mapsto \eta] }\Bracks{v}.
    \end{equation}
  \end{thmenum}
\end{proposition}
\begin{proof}
  In both cases, we use structural induction on the definition of the substitution.

  \SubProofOf{thm:renaming_assignment_compatibility/terms}

  \begin{itemize}
    \item If \( \tau = \xi \), then
    \begin{equation*}
      \tau[\xi \mapsto \eta] = \xi[\xi \mapsto \eta] = \eta
    \end{equation*}
    and \eqref{eq:thm:renaming_assignment_compatibility/terms} follows directly.

    \item If \( \tau \) is a variable and \( \tau \neq \xi \), then
    \begin{equation*}
      \tau[\xi \mapsto \eta] = \tau
    \end{equation*}
    and \eqref{eq:thm:renaming_assignment_compatibility/terms} again holds trivially.

    \item If \( \tau = f(\tau_1, \ldots, \tau_n) \) and if the inductive hypothesis holds for \( \tau_1, \ldots, \tau_n \), then
    \begin{balign*}
      \parens[\Big]{ \tau[\xi \mapsto \eta] }\Bracks{v}
      &=
      \parens[\Big]{ f(\tau_1[\xi \mapsto \eta], \ldots, \tau_n[\xi \mapsto \eta]) }\Bracks{v}
      = \\ &=
      I(f) \parens[\Bigg]{ \parens[\Big]{ \tau_1[\xi \mapsto \eta] }\Bracks{v}, \ldots, \parens[\Big]{ \tau_1[\xi \mapsto \eta] }\Bracks{v} }
      \reloset {\T{ind.}} = \\ &=
      I(f) \parens[\Big]{ \tau_1\Bracks{v_{\xi \mapsto \eta}}, \ldots, \tau_n\Bracks{v_{\xi \mapsto \eta}} }
      = \\ &=
      \parens[\Big]{ f(\tau_1, \ldots, \tau_n) }\Bracks{v_{\xi \mapsto \eta}}
      = \\ &=
      \tau\Bracks{v_{\xi \mapsto \eta}}.
    \end{balign*}
  \end{itemize}

  In all cases, \eqref{eq:thm:renaming_assignment_compatibility/terms} holds.

  \SubProofOf{thm:renaming_assignment_compatibility/formulas}
  \hfill
  \begin{itemize}
    \item If \( \varphi \in \set{ \top, \bot } \), then \( \varphi \) has no subterms and thus \eqref{eq:thm:renaming_assignment_compatibility/formulas} holds vacuously.

    \item If \( \varphi = p(\tau_1, \ldots, \tau_n) \), then since \eqref{eq:thm:renaming_assignment_compatibility/terms} holds for all \( \tau_k \), we have
    \begin{equation*}
      \parens[\Big]{ \tau_k[\xi \mapsto \eta] }\Bracks{v} = \tau_k\Bracks{v_{\xi \mapsto \eta}}
    \end{equation*}
    and thus
    \begin{equation*}
      I(p)\parens[\Big]{ \parens[\Big]{ \tau_1[\xi \mapsto \eta] }\Bracks{v}, \ldots, \parens[\Big]{ \tau_1[\xi \mapsto \eta] }\Bracks{v} }
      \reloset {\T{ind.}} =
      I(p)\parens[\Big]{ \tau_1\Bracks{v_{\xi \mapsto \eta}}, \ldots, \tau_n\Bracks{v_{\xi \mapsto \eta}} }.
    \end{equation*}

    Therefore,
    \begin{balign*}
      \parens[\Big]{ \varphi[\xi \mapsto \eta] }\Bracks{v}
      &=
      \parens[\Big]{ p(\tau_1[\xi \mapsto \eta], \ldots, \tau_n[\xi \mapsto \eta]) }\Bracks{v}
      = \\ &=
      \parens[\Big]{ p(\tau_1, \ldots, \tau_n) }\Bracks{v_{\xi \mapsto \eta}}
      = \\ &=
      \varphi\Bracks{v_{\xi \mapsto \eta}}.
    \end{balign*}

    \item The case \( \varphi = \tau_1 \doteq \tau_2 \) is proved analogously.

    \item The cases \( \varphi = \neg \psi \) and \( \varphi = \psi_1 \bincirc \psi_2 \) are proved in an straightforward manner.

    \item Let \( \varphi = \qforall \zeta \psi \), where the inductive hypothesis holds for \( \psi \). We consider three cases
    \begin{itemize}
      \item Suppose that \( \zeta = \xi \). By definition, we have
      \begin{equation*}
        \varphi[\xi \mapsto \eta]
        =
        \varphi,
      \end{equation*}
      hence \eqref{eq:thm:renaming_assignment_compatibility/formulas} holds trivially.

      \item Suppose that \( \zeta \neq \xi \). It follows that
      \begin{equation*}
        \varphi[\xi \mapsto \eta]
        =
        \qforall \zeta \parens[\Big]{ \psi[\xi \mapsto \eta] }.
      \end{equation*}

      \begin{itemize}
        \item If \( \parens[\Big]{\varphi[\xi \mapsto \eta]}\Bracks{v} = T \), by definition of \hyperref[def:first_order_valuation/formula_valuation]{quantifier formula valuation}, for any \( x \in X \) we have
        \begin{equation}\label{eq:thm:renaming_assignment_compatibility/formulas/true_modified_assignment}
          \parens[\Bigg]{\underbrace{\qforall \zeta \parens[\Big]{ \psi[\xi \mapsto \eta] }}_{\varphi[\xi \mapsto \eta]} }\Bracks{v}
          =
          \parens[\Big]{\psi[\xi \mapsto \eta]}\Bracks{v_{\zeta \mapsto x}}
          =
          T.
        \end{equation}

        On the other hand, by the inductive hypothesis,
        \begin{equation*}
          \parens[\Big]{ \psi[\xi \mapsto \eta] }\Bracks{v} = \psi\Bracks{v_{\xi \mapsto \eta}}
        \end{equation*}
        and, as a special case, for any \( x \in X \),
        \begin{equation}\label{eq:thm:renaming_assignment_compatibility/formulas/ind_hyp_modified_assignment}
          \parens[\Big]{ \psi[\xi \mapsto \eta] }\Bracks{v_{\zeta \mapsto x}} = \psi\Bracks{v_{\xi \mapsto \eta, \zeta \mapsto x}}.
        \end{equation}

        Combining \eqref{eq:thm:renaming_assignment_compatibility/formulas/true_modified_assignment} and \eqref{eq:thm:renaming_assignment_compatibility/formulas/ind_hyp_modified_assignment}, we obtain
        \begin{equation*}
          \parens[\Big]{ \varphi[\xi \mapsto \eta] }\Bracks{v}
          =
          \parens[\Big]{ \psi[\xi \mapsto \eta] }\Bracks{v_{\zeta \mapsto x}}
          =
          \underbrace{\psi\Bracks{v_{\xi \mapsto \eta, \zeta \mapsto x}}}_{T \T*{for all} x \in X}
          =
          \varphi\Bracks{v_{\xi \mapsto \eta}},
        \end{equation*}
        which proves the case.

        \item If \( \parens[\Big]{\varphi[\xi \mapsto \eta]}\Bracks{v} = F \), then there exists \( x \in X \) such that
        \begin{equation*}
          \parens[\Big]{\psi[\xi \mapsto \eta]}\Bracks{v_{\zeta \mapsto x}} = F.
        \end{equation*}

        Since \eqref{eq:thm:renaming_assignment_compatibility/formulas/ind_hyp_modified_assignment} holds by the inductive hypothesis, we have
        \begin{equation*}
          \psi\Bracks{v_{\xi \mapsto \eta, \zeta \mapsto x}} = F
        \end{equation*}
        for the same \( x \).

        It follows that \( \varphi\Bracks{v_{\xi \mapsto \eta}} = F \), which proves the case.
      \end{itemize}
    \end{itemize}

    \item We can prove the case \( \varphi = \qexists \zeta \psi \) using double negation on the previous case.
  \end{itemize}

  In all cases, \eqref{eq:thm:renaming_assignment_compatibility/formulas} holds.
\end{proof}

\begin{proposition}\label{thm:first_order_substitution_equivalence}
  Analogously to \fullref{thm:propositional_substitution_equivalence}, we will show that all substitutions defined in \fullref{def:first_order_substitution} types of substitution preserve the corresponding \hyperref[def:first_order_semantics]{semantics}.

  By induction, this proposition also holds for \hyperref[def:propositional_substitution/simultaneous]{simultaneous substitution}.

  Fix a \hyperref[def:first_order_structure]{structure} \( \mscrX = (X, I) \) and a \hyperref[def:first_order_valuation/variable_assignment]{variable assignment} \( v \).

  \begin{thmenum}
    \thmitem{thm:first_order_substitution_equivalence/propositional} As in \fullref{def:first_order_substitution/propositional}, let \( \varphi \) be a \hyperref[def:propositional_syntax/formula]{propositional formula} with variables \( {V = \set{ P_1, \ldots, P_n }} \) and let \( \Theta = \set{ \theta_1, \ldots, \theta_n } \) be a set of \hyperref[def:first_order_syntax/formula]{first-order formulas}.

    Furthermore, let \( J \) be a \hyperref[def:propositional_valuation/interpretation]{propositional interpretation} such that, for all \( k = 1, \ldots, n \),
    \begin{equation}\label{eq:thm:first_order_substitution_equivalence/propositional/compatibility}
      P_k\Bracks{J} = \theta_k\Bracks{v}.
    \end{equation}

    Then
    \begin{equation}\label{eq:thm:first_order_substitution_equivalence/propositional}
      \parens[\Big]{ \varphi[V \mapsto \Theta] }\Bracks{v} = \varphi \Bracks{J}.
    \end{equation}

    In particular, \( \vDash \varphi \) in the sense of \fullref{def:propositional_semantics/tautology} implies \( \vDash \varphi[V \mapsto \Theta] \) in the sense of \fullref{def:first_order_semantics/tautology}.

    \thmitem{thm:first_order_substitution_equivalence/term_in_term} Let \( \tau \) be a \hyperref[def:first_order_syntax/term]{first-order term} and let \( \kappa \) be a \hyperref[def:first_order_syntax/subterm]{subterm} of \( \tau \). Let \( \mu \) be another term such that
    \begin{equation}\label{eq:thm:first_order_substitution_equivalence/term_in_term/compatibility}
      \mu\Bracks{v} = \kappa\Bracks{v}.
    \end{equation}

    Then
    \begin{equation}\label{eq:thm:first_order_substitution_equivalence/term_in_term}
      \tau[\kappa \mapsto \mu]\Bracks{v} = \tau\Bracks{v}.
    \end{equation}

    \thmitem{thm:first_order_substitution_equivalence/term_in_formula} Let \( \varphi \) be a \hyperref[def:first_order_syntax/formula]{first-order formula} and let \( \kappa \) be a \hyperref[def:first_order_syntax/formula_terms]{term of \( \varphi \)}. Let \( \mu \) be another term such that
    \begin{equation}\label{eq:thm:first_order_substitution_equivalence/term_in_formula/compatibility}
      \mu\Bracks{v} = \kappa\Bracks{v}.
    \end{equation}

    Then
    \begin{equation}\label{eq:thm:first_order_substitution_equivalence/term_in_formula}
      \varphi[\kappa \mapsto \mu]\Bracks{v} = \varphi\Bracks{v}.
    \end{equation}
  \end{thmenum}
\end{proposition}
\begin{proof}
  In all cases, we use structural induction by the definition of the substitution. The inductive hypothesis for a formula is that the proposition holds for arbitrary substitutions and valuations.

  \SubProofOf{thm:first_order_substitution_equivalence/propositional} Let \( \varphi \) be a propositional formula.
  \begin{itemize}
    \item If \( \varphi \in \set{ \top, \bot } \), no substitution is performed and thus \eqref{eq:thm:first_order_substitution_equivalence/propositional} holds trivially.

    \item If \( \varphi = P_k \) for some \( k = 1, \ldots, n \), then follows \eqref{eq:thm:first_order_substitution_equivalence/propositional} from \eqref{eq:thm:first_order_substitution_equivalence/propositional/compatibility}.

    \item If \( \varphi = \neg \psi \) and if the inductive hypothesis holds for \( \psi \), then
    \begin{equation*}
      \parens[\Big]{ \psi[V \mapsto \Theta] }\Bracks{v}
      =
      \overline{\parens[\Big]{ \psi[V \mapsto \Theta] }\Bracks{v}}
      \reloset {\T{ind.}} =
      \overline{\psi \Bracks{J}}
      =
      \varphi \Bracks{J}.
    \end{equation*}

    \item If \( \varphi = \psi_1 \bincirc \psi_2, \bincirc \in \Sigma \) and if the inductive hypothesis holds for both \( \psi_1 \) and \( \psi_2 \), then
    \begin{equation*}
      \parens[\Big]{ \psi[V \mapsto \Theta] }\Bracks{v}
      =
      \parens[\Big]{ \psi_1[V \mapsto \Theta] }\Bracks{v} \bincirc \parens[\Big]{ \psi_2[V \mapsto \Theta] }\Bracks{v}
      \reloset {\T{ind.}} =
      \psi_1 \Bracks{J} \bincirc \psi_2\Bracks{J}
      =
      \varphi\Bracks{J}.
    \end{equation*}
  \end{itemize}

  In all cases, \eqref{eq:thm:first_order_substitution_equivalence/propositional} holds.

  \SubProofOf{thm:first_order_substitution_equivalence/term_in_term} The proof is identical to that of \fullref{thm:renaming_assignment_compatibility/terms}.

  \SubProofOf{thm:first_order_substitution_equivalence/term_in_formula} The proof is identical to that of \fullref{thm:renaming_assignment_compatibility/formulas} except for the special cases where \hyperref[rem:first_order_substitution_renaming_justification]{renaming} occurs, i.e. \( \varphi = \qforall \xi \psi \) and \( \varphi = \qexists \xi \psi \), where
  \begin{itemize}
    \item \( \xi \in \boldop{Free}(\mu) \).
    \item \( \eta \not\in \boldop{Var}(\kappa) \cup \boldop{Var}(\mu) \cup \boldop{Var}(\psi) \).
    \item The inductive hypothesis holds for \( \psi \).
  \end{itemize}

  We will only show the case \( \varphi = \qforall \xi \psi \) since the existential case is handled similarly.

  Since \( \xi \in \boldop{Free}(\mu) \), we have
  \begin{equation*}
    \varphi[\kappa \mapsto \mu]
    =
    \qforall \eta \parens[\Big]{ \psi[\xi \mapsto \eta][\kappa \mapsto \mu] },
  \end{equation*}
  which does not allow us to use the inductive hypothesis directly.

  We proceed to prove the statement by nested induction on the number of quantifiers. We have already shown the case of \( 0 \) quantifiers. Suppose that the statement holds for all formulas with strictly less than \( n \) quantifiers and suppose that \( \varphi \) has exactly \( n \) quantifiers.

  Furthermore, for formulas with \( n \) quantifiers with \( \forall \) as the outermost one, the non-renaming cases \eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/trivial} and \eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/direct} hold. Therefore, since \( \eta \not\in \boldop{Free}(\mu) \),
  \begin{equation}\label{eq:thm:first_order_substitution_equivalence/term_in_formula/nested_induction}
    \begin{aligned}
      &\phantom{{}={}}
      \varphi[\kappa \mapsto \mu]\Bracks{v}
      = \\ &=
      \parens[\Bigg]{\qforall \eta \parens[\Big]{ \psi[\xi \mapsto \eta][\kappa \mapsto \mu] }}\Bracks{v}
      \reloset {\eqref{eq:def:first_order_substitution/term_in_formula/quantifiers/direct}} = \\ &=
      \parens[\Bigg]{ \parens*{ \qforall \eta \parens[\Big]{ \psi[\xi \mapsto \eta] } }[\kappa \mapsto \mu] }\Bracks{v}
      \reloset {\T{ind.}} = \\ &=
      \parens[\Bigg]{\qforall \eta \parens[\Big]{ \psi[\xi \mapsto \eta] } }\Bracks{v},
    \end{aligned}
  \end{equation}
  where we have implicitly used that \( \psi \) has \( n - 1 \) quantifiers.

  On the other hand, due to \fullref{thm:renaming_assignment_compatibility/formulas},
  \begin{equation*}
    \parens[\Big]{ \psi[\xi \mapsto \eta] }\Bracks{v} = \psi\Bracks{v_{\xi \mapsto \eta}}
  \end{equation*}
  and, in particular, for any \( x \in X \),
  \begin{equation*}
    \parens[\Big]{ \psi[\xi \mapsto \eta] }\Bracks{v_{\eta \mapsto x}}
    =
    \psi\Bracks{v_{\xi \mapsto \eta,\eta \mapsto x}}
    =
    \psi\Bracks{v_{\xi \mapsto x}},
  \end{equation*}
  where the last equality holds because \( \eta \not\in \boldop{Var}(\psi) \).

  Hence,
  \begin{equation*}
    \underbrace{ \parens[\Bigg]{\qforall \eta \parens[\Big]{ \psi[\xi \mapsto \eta] } }\Bracks{v} }_{\varphi[\xi \mapsto \eta]\Bracks{v}}
    =
    \underbrace{ \parens[\Big]{\qforall \xi \psi }\Bracks{v_{\xi \mapsto \eta}} }_{\varphi\Bracks{v_{\xi \mapsto \eta}}}.
  \end{equation*}

  This proves \eqref{eq:thm:first_order_substitution_equivalence/term_in_formula}.
\end{proof}

\begin{remark}\label{rem:predicate_formula}
  As explained in \fullref{rem:first_order_formula_conventions/necessary_signature}, we avoid adding to a language more predicates than necessary. For this reason, we sometimes use \term{predicate formulas}. For example, if \( \leq \) is a \hyperref[def:partially_ordered_set/theory]{partial order} symbol and we want to have a predicate for whether \( \xi \) is the \hyperref[def:partially_ordered_set_extremal_points/top_and_bottom]{bottom element}, we can define the formula
  \begin{equation}\taglabel[\op{IsBottom}]{rem:predicate_formula/bottom}
    \ref{rem:predicate_formula/bottom}[\xi] \coloneqq \qforall \eta (\xi \leq \eta).
  \end{equation}

  Note that \( [\xi] \) is only a notational convenience for highlighting which variables are free, the actual formula is named \( \op{IsBottom} \). This is consistent with \fullref{rem:first_order_formula_valuation_without_variable_assignment} which allows us to write \( \op{IsBottom}[\eta] \) rather than \( \op{IsBottom}[\xi \mapsto \eta] \) to verify if \( \eta \) is a bottom element
\end{remark}

\begin{proposition}\label{thm:first_order_quantifiers_are_dual}
  For any formula \( \varphi \) and any variable \( \xi \) over \( \mscrL \), we have the following equivalences:
  \begin{align}
    \neg \qforall \xi \varphi &\gleichstark \qexists \xi \neg \varphi \label{thm:first_order_quantifiers_are_dual/negation_of_universal} \\
    \neg \qexists \xi \varphi &\gleichstark \qforall \xi \neg \varphi \label{thm:first_order_quantifiers_are_dual/negation_of_existential}
  \end{align}
\end{proposition}
\begin{proof}
  The two equivalences are connected using \hyperref[thm:boolean_equivalences/double_negation]{double negation}. We will only prove \eqref{thm:first_order_quantifiers_are_dual/negation_of_universal}.

  Let \( \mscrX = (X, I) \) be a structure over \( \mscrL \) and let \( v \) be a variable assignment. Then
  \begin{align*}
    (\neg \qforall \xi \varphi)\Bracks{v}
    &=
    \overline{(\qforall \xi \varphi)\Bracks{v}}
    = \\ &=
    \overline{\bigwedge\set{ \varphi\Bracks{v_{\xi \mapsto x}} \given x \in X }}
    \reloset {\eqref{eq:thm:de_morgans_laws/complement_of_meet}} = \\ &=
    \bigvee\set{ \overline{\varphi\Bracks{v_{\xi \mapsto x}}} \given x \in X }
    = \\ &=
    \bigvee\set{ (\neg \varphi)\Bracks{v_{\xi \mapsto x}} \given x \in X }
    = \\ &=
    (\qexists \xi \neg \varphi)\Bracks{v}.
  \end{align*}
\end{proof}

\begin{proposition}\label{thm:implicit_universal_quantification}
  For any formula \( \varphi \) and any variable \( \xi \) over \( \mscrL \), the formulas \( \varphi \) and \( \qforall \xi \varphi \) are \hyperref[def:first_order_semantics/equivalence]{semantically equivalent}.

  This allows us to skip quantifiers when writing formulas without changing their validity. Given a formula \( \varphi \) with free variables \( \xi_1, \ldots, \xi_n \), we call
  \begin{equation*}
    \qforall {\xi_1} \cdots \qforall {\xi_n} \varphi
  \end{equation*}
  its \term{universal closure} and say that \( \varphi \) itself is \term{implicitly universally quantified}. Universal closures of quantifierless formulas are called \term{universal formulas}.

  See \fullref{ex:def:first_order_natural_deduction_system/eigenvariables/invalid_universal_closure} for how this fails for derivability.
\end{proposition}
\begin{proof}
  Let \( \mscrX = (X, I) \) be a structure that satisfies \( \varphi \). Let \( v \) be a variable assignment in \( \mscrX \). Then for any \( x \in X \), the modified variable assignment \( v_{\xi \mapsto x} \) also satisfies \( \varphi \), i.e.
  \begin{equation*}
    \varphi\Bracks{v} = \varphi\Bracks{v_{\xi \mapsto x}} = T.
  \end{equation*}

  Thus, \( \mscrX \) is also a model for \( \qforall \xi \varphi \).

  Conversely, suppose that \( \mscrX \) satisfies \( \qforall \xi \varphi \) and \( v \) is any variable assignment. Then
  \begin{equation*}
    \varphi\Bracks{v_{\xi \mapsto x}} = T
  \end{equation*}
  for any \( x \), including \( x \coloneqq v(\xi) \). Thus,
  \begin{equation*}
    \varphi\Bracks{v_{\xi \mapsto v(\xi)}} = \varphi\Bracks{v} = T.
  \end{equation*}

  Therefore, \( \mscrX \) is also a model for \( \varphi \).
\end{proof}

\begin{proposition}\label{thm:quantifier_satisfiability}
  Let \( \mscrL \) be a first-order language, \( \varphi \) be a formula, \( \xi \) be any variable and \( \tau \) be a \hyperref[def:first_order_syntax/ground_term]{ground term} in \( \mscrL \). The following hold:

  \begin{thmenum}
    \thmitem{thm:quantifier_satisfiability/universal} \( \qforall \xi \varphi \vDash \varphi[\xi \mapsto \tau] \),

    \thmitem{thm:quantifier_satisfiability/existential} \( \varphi[\xi \mapsto \tau] \vDash \qexists \xi \varphi \).
  \end{thmenum}

  See also \fullref{def:first_order_natural_deduction_system/terms} for inference rules corresponding to this proposition.
\end{proposition}
\begin{proof}
  The proof is very straightforward, but the technical details make it look a bit more complicated.

  First note that if the formulas on the left are unsatisfiable, the proof is trivial. Hence, we will assume that they are satisfiable.

  \SubProofOf{thm:quantifier_satisfiability/universal} From \fullref{thm:implicit_universal_quantification} it follows that \( \qforall \xi \varphi \vDash \varphi \). If \( \xi \) is not free in \( \varphi \), then \( \varphi[\xi \mapsto \tau] = \varphi \) and the proof is finished. Suppose that \( \xi \) is free in \( \varphi \).

  Let \( \mscrX = (X, I) \) be a model of \( \qforall \xi \varphi \). Let \( v \) be a variable assignment in \( \mscrX \) and let \( t \coloneqq \tau\Bracks{v} \). To avoid the case where \( \xi \in \boldop{Var}(\tau) \), we replace it with \( \eta \) that is not a variable in neither \( \tau \) nor \( \varphi \). Then
  \begin{align*}
    \varphi[\xi \mapsto \tau]\Bracks{v}
    &=
    \varphi[\xi \mapsto \eta][\eta \mapsto \tau]\Bracks{v}
    = \\ &=
    \varphi[\xi \mapsto \eta][\eta \mapsto \tau]\Bracks{v_{\eta \mapsto t}}
    \reloset {\ref{thm:first_order_substitution_equivalence/term_in_formula}} = \\ &=
    \varphi[\xi \mapsto \eta]\Bracks{v_{\eta \mapsto t}}
    \reloset {\ref{thm:renaming_assignment_compatibility/formulas}} = \\ &=
    \varphi\Bracks{v_{\xi \mapsto \eta, \eta \mapsto t}}
    = \\ &=
    \varphi\Bracks{v_{\xi \mapsto t}}.
  \end{align*}

  Since \( (\qforall \xi \varphi)\Bracks{v} = T \), by definition of valuation of \( \forall \) we have
  \begin{equation*}
    \varphi\Bracks{v_{\xi \mapsto t}} = T.
  \end{equation*}

  Therefore, \( \varphi[\xi \mapsto \tau]\Bracks{v} = T \) and, since \( v \) was chosen arbitrarily, we have \( \mscrX \vDash \varphi[\xi \mapsto \tau] \).

  \SubProofOf{thm:quantifier_satisfiability/existential} For any model \( \mscrX = (X, I) \) of \( \varphi \), any assignment \( v \) satisfies \( \varphi[\xi \mapsto \tau] \). Thus, \( \varphi\Bracks{v_{\xi \mapsto t}} = T \) for \( t \coloneqq \tau\Bracks{v} \) and hence \( v \) satisfies \( \qexists \xi \varphi \). Therefore, \( \mscrX \) is also a model of \( \qexists \xi \varphi \).
\end{proof}

\begin{proposition}\label{thm:existential_quantifier_removal}
  Let \( \varphi \) be a formula in the language \( \mscrL \). If \( \tau = c \) for some constant \( c \) that does not occur in \( \varphi \), the formulas \( \varphi[\xi \mapsto c] \) and \( \qexists \xi \varphi \) are equisatisfiable.

  If there is no such constant in \( \mscrL \), we can instead define the extension \( \widetilde \mscrL \) of \( \mscrL \) by adjoining a new constant \( c \).

  This general procedure of creating a new extension language in order to obtain an equisatisfiable formula without the outer quantifier is called \term{existential quantifier elimination}.
\end{proposition}
\begin{proof}
  \Fullref{thm:quantifier_satisfiability/existential} implies that \( \qexists \xi \varphi \vDash \varphi[\xi \mapsto c] \), which is even stronger than the statement that if \( \qexists \xi \varphi \) is satisfiable, so it \( \varphi[\xi \mapsto c] \).

  For the other direction, suppose that \( \mscrX = (X, I) \) satisfies \( \qexists \xi \varphi \). Fix a variable assignment \( v \). Then there exists a value \( x \) such that
  \begin{equation}\label{eq:thm:quantifier_satisfiability/existential/existence_modified_assignment}
    \varphi\Bracks{v_{\xi \mapsto x}} = T.
  \end{equation}

  Define the interpretation
  \begin{equation*}
    \widetilde I (a) \coloneqq \begin{cases}
      x, &a = c, \\
      a, &a \in \boldop{Func}, \\
      a, &a \in \boldop{Pred}
    \end{cases}
  \end{equation*}
  as \( I \) modified at \( c \), so that \( c\Bracks{v} = x \). It remains to show that the structure \( (X, \widetilde I) \) is a model of \( \varphi[\xi \mapsto \tau] \). Since the new structure has the same domain, \( v \) is a variable assignment in this new structure. Nevertheless, we will denote it by \( \widetilde v \) in order to distinguish between valuations in the two structures.

  Since \( \xi\Bracks{v} = c\Bracks{v} = x \), from \fullref{thm:first_order_substitution_equivalence/term_in_formula} it follows that
  \begin{equation*}
    \varphi[\xi \mapsto c]\Bracks{v} = \varphi\Bracks{v_{\xi \mapsto x}} \reloset{\eqref{eq:thm:quantifier_satisfiability/existential/existence_modified_assignment}} = T.
  \end{equation*}

  Since the above holds for any assignment \( v \), the structure \( (X, \widetilde I) \) is a model of \( \varphi[\xi \mapsto c] \).
\end{proof}

\begin{theorem}[First-order semantic deduction theorem]\label{thm:semantic_deduction_theorem}\mcite[thm. 16.29]{OpenLogicFull}
  Let \( \Gamma \) be a set of formulas over some first-order language, let \( \psi \) be an arbitrary formula and let \( \varphi \) be a \hyperref[def:first_order_syntax/ground_formula]{closed formula}.

  Then the entailment \( \Gamma, \varphi \vDash \psi \) holds if and only if \( \Gamma \vDash \varphi \to \psi \) holds.

  See \fullref{rem:deduction_with_free_variables} for the importance of the condition that \( \varphi \) is a closed formula.

  Due to \fullref{rem:propositional_logic_as_first_order_logic}, this theorem also holds for propositional formulas.

  Compare this result with \fullref{thm:syntactic_deduction_theorem}.
\end{theorem}
\begin{proof}
  \SufficiencySubProof Let \( \Gamma, \varphi \vDash \psi \) and let \( \mscrX = (X, I) \) be a model for \( \Gamma \).

  \begin{itemize}
    \item If \( \mscrX \vDash \varphi \), then \( \mscrX \vDash \Gamma \cup \set{ \varphi } \) and from our assumption we conclude \( \mscrX \vDash \psi \). Hence, for any variable assignment \( v \) we have
    \begin{equation*}
      (\varphi \rightarrow \psi)\Bracks{v}
      =
      (\varphi\Bracks{v} \rightarrow \psi\Bracks{v})
      =
      (T \rightarrow T)
      =
      T.
    \end{equation*}

    \item If \( \mscrX \not\vDash \varphi \), then, since \( \varphi \) is closed and the valuation \( \varphi\Bracks{v} \) does not depend on the variable assignment \( v \), for any \( v \) we have
    \begin{equation}\label{eq:thm:semantic_deduction_theorem/closedness_condition}
      (\varphi \rightarrow \psi)\Bracks{v}
      =
      (\varphi\Bracks{v} \rightarrow \psi\Bracks{v})
      =
      (F \rightarrow \psi\Bracks{v})
      =
      T.
    \end{equation}

  \end{itemize}

  In both cases we conclude that \( \mscrX \vDash \varphi \rightarrow \psi \). Therefore, \( \Gamma \vDash \varphi \rightarrow \psi \).

  \NecessitySubProof Let \( \Gamma \vDash \varphi \rightarrow \psi \) and let \( \mscrX = (X, I) \) be a model for \( \Gamma \cup \set{ \varphi } \). Let \( v \) be an arbitrary assignment in \( \mscrX \).

  Obviously \( \mscrX \vDash \Gamma \) and \( \mscrX \vDash \varphi \). We thus have \( (\varphi \rightarrow \psi)\Bracks{v} = T \) and \( \varphi\Bracks{v} = T \), which only leaves \( \psi\Bracks{v} = T \) as a possible option.

  Hence, \( \mscrX \vDash \psi \) and, consequently, \( \Gamma \cup \set{ \varphi } \vDash \psi \).
\end{proof}

\begin{remark}\label{rem:deduction_with_free_variables}
  In order to highlight the importance of closed formulas in certain theorems, we will take a close look at the proof of \fullref{thm:semantic_deduction_theorem}.

  Note that \eqref{eq:thm:semantic_deduction_theorem/closedness_condition} only holds because the formula \( \varphi \) is closed. If it were not closed, we could only conclude that there exists a variable assignment \( v_0 \) such that \( \varphi\Bracks{v_0} = F \). Clearly then \( (\varphi \rightarrow \psi)\Bracks{v_0} = T \). But there may exist another assignment \( v \) such that \( \varphi\Bracks{v} = T \) and \( \psi\Bracks{v} = F \), which would imply that \( \mscrX \not\vDash \varphi \rightarrow \psi \).
\end{remark}


\documentclass{article}

\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{color}
\newtheorem{lemma}{Lemma}

\DeclareMathOperator{\Ker}{Ker}
\let\Im\undefined
\DeclareMathOperator{\Im}{Im}
\let\span\undefined
\DeclareMathOperator{\span}{span}

\begin{document}

Let us consider an optimisation problem
\begin{displaymath}
\begin{array}{rcl}
 \bar{\gamma} & := & \arg \min\limits_{\gamma \in \Omega} L_{\Theta}(\gamma) \\
 \gamma & := & [\gamma_1, \dots, \gamma_K] \in \mathbb{R}^{KT}, \\
 \gamma_k & := & [\gamma_k(1), \dots, \gamma_k(T)] \in \mathbb{R}^{T} \\
 L^{\epsilon}_{\Theta}(\gamma) & := & \frac{1}{T} b_{\Theta}^T \gamma + \frac{\epsilon^2}{T} \gamma^T H \gamma, \\
 \Omega & := & \lbrace \gamma \in \mathbb{R}^{KT}: \gamma \geq 0 \wedge \sum\limits_{k=1}^K \gamma(t) = 1, \forall t = 1,\dots T \rbrace
\end{array}
\end{displaymath}
and $H \in \mathbb{R}^{KT,KT}$ is block-diagonal matrix, whose blocks $L_k \in \mathbb{R}^{T,T}$ are formed by Laplace matrix. \newline
In what follows, we proved that this problem has always unique solution.

\section{Properties}

\begin{itemize}
\item $H$ is SPS (since blocks are SPS) and
 \begin{equation}
  \label{eq:kerH}
  \begin{array}{rcl}
   \Ker H & = & \span \lbrace [c, {\bf 0}, \dots, {\bf 0}]^T, [{\bf 0}, c, {\bf 0} \dots, {\bf 0}]^T, \dots, [{\bf 0} \dots, {\bf 0}, c]^T \rbrace \subset \mathbb{R}^{KT}, \\
   c & := & [ 1, \dots, 1] \in \mathbb{R}^T
  \end{array}
 \end{equation}
\item $L^{\epsilon}_{\Theta}(\gamma)$ is continuous (not strictly) convex function $\mathbb{R}^{KT} \rightarrow \mathbb{R}$,
%\item gradient is given by $\nabla_{\gamma} L^{\epsilon}_{\Theta}(\gamma) = \frac{1}{T} b_{\Theta} + \frac{2\epsilon^2}{T} H \gamma$ ("well known" - can be proved using Taylor expansion),
\item $\Omega \subset \mathbb{R}^{KT}$ is bounded closed convex set {\color{red}(has to be proved?)},
\item the matrix $B = [I, \dots I] \in \mathbb{R}^{T,KT}$ ($I \in \mathbb{R}^{T,T}$ denotes identity matrix) forms the equivalent definition of $\Omega$ given by
 \begin{displaymath}
  \Omega = \lbrace \gamma \in \mathbb{R}^{KT}: \gamma \geq 0 \wedge B\gamma = c \rbrace
 \end{displaymath}
\item $\Ker H \cap \Ker B = \lbrace 0 \rbrace$, proof: let $d := [\alpha_1 c, \dots, \alpha_K c] \neq 0$ be a vector from $\Ker H$, then
\begin{displaymath}
 Bd = \left[\sum\limits_{k=1}^K \alpha_k, \dots, \sum\limits_{k=1}^K \alpha_k \right]^T
\end{displaymath}
and because $d$ is nonzero (not all $\alpha_k$ is equal zero), then $Bd \neq 0$ and therefore $d \notin \Ker B$.
\end{itemize}

\begin{lemma}
\label{th:penalized}
Let $A \in \mathbb{R}^{n \times n}$ be a SPS matrix, let $B \in \mathbb{R}^{m \times n}$, $\rho > 0$, and let $\Ker A \cap \Ker B = \lbrace 0 \rbrace$.
Then matrix
\begin{displaymath}
A_{\rho} = A + \rho B^T B
\end{displaymath}
is SPD.
\end{lemma}

\begin{proof}
Let us follow proof by Dost\'{a}l \cite{DosBOOK-2009}, Lemma 1.2. \newline
If $x \in \mathbb{R}^n \setminus \lbrace 0 \rbrace$ and $\Ker A \cap \Ker B = \lbrace 0 \rbrace$, then either $Ax \neq 0$ or $Bx \neq 0$.
% (equivalently either $\Vert Ax \Vert \neq 0$ or $\Vert Bx \Vert \neq 0$)
Since $Ax \neq 0$ is equivalent to $A^{\frac{1}{2}}x \neq 0$,
% ($\Vert Ax \Vert \neq 0$ is equivalent to $\Vert A^{\frac{1}{2}}x \Vert \neq 0$)
we get for $\rho > 0$
\begin{displaymath}
 x^T A_{\rho} x =
 x^T (A + \rho B^T B) x = x^T A x + \rho x^T B^T B x =
 \Vert A^{\frac{1}{2}}x \Vert^2 + \rho \Vert Bx \Vert^2 > 0~\mathrm{.}
\end{displaymath}
Thus $A_{\rho}$ is positive definite. \newline
\end{proof}

\section{"The" proof}

Let us define equivalent penalised optimisation problem
\begin{displaymath}
 \bar{\gamma} =  \arg \min\limits_{\gamma \in \Omega} L_{\Theta}^{\epsilon}(\gamma) + \frac{\epsilon^2 \rho}{T} \Vert B \gamma - c \Vert^2 ,
\end{displaymath}
where $\rho > 0$ is arbitrary number (\emph{penalisation} parameter).
Obviously, any solution of this problem is also the solution of original problem and vice versa (the penalisation term is equal to $0$ for all feasible $\gamma \in \Omega$).
However, the object function of the new problem can be written in form
\begin{displaymath}
  L_{\Theta}^{\epsilon}(\gamma) + \frac{\epsilon^2 \rho}{T} \Vert B \gamma - c \Vert^2 = \frac{\epsilon^2}{T} \gamma^T (H + \rho B^T B) \gamma + \frac{1}{T} \gamma^T (b_{\Theta} - 2 \epsilon^2 \rho B^T c) + \frac{\varepsilon^2 \rho}{T} c^T c
\end{displaymath}
and the Hessian matrix is given by $(H + \rho B^T B)$, which is SPD (see Lemma \ref{th:penalized}) and consequently, the object function is strictly convex (in this case, penalisation works as regularisation).
Since strictly convex QP on closed convex set has always unique solution (see additional Lemma below), the solution of original problem is also unique.

Therefore, we can conclude that in the problems of this type, {\it equality constraints regularise original problem}.

\section{QP solvability - strictly convex cost function on convex set}

For completeness, let us review the proof of uniqueness of solution of QP with strictly convex cost function (SPD Hessian matrix) on closed convex set. \newline

Let $\bar{x} \in \Omega$ be a solution of problem
\begin{displaymath}
 \min\limits_{x \in \Omega} f(x), ~~~~ f(x) := \frac{1}{2} x^T A x - b^T x,
\end{displaymath}
where $A \in \mathbb{R}^{n,n}$ is SPD matrix, $b \in \mathbb{R}^n$ and $\Omega$ is closed convex set. \newline
Let us consider arbitrary $y \in \Omega \setminus \lbrace \bar{x} \rbrace$ (in what follows, we will show that $f(y) > f(\bar{x})$)
and let us denote $d := y - x \neq 0$. Then using the definition of $f$ we can write
\begin{displaymath}
 \begin{array}{rcl}
 f(y) = f(\bar{x} + d) & = & \frac{1}{2} (\bar{x}+d)^T A (\bar{x}+d) - b^T (\bar{x} + d) \\
 & = & \frac{1}{2} \bar{x}^T A \bar{x} + \frac{1}{2} \bar{x}^T A d + \frac{1}{2} d^T A \bar{x} + \frac{1}{2} d^T A d - b^T \bar{x} - b^T d \\
 \mathit{"A = A^T"}~~~ & = & f(\bar{x}) + d^T( A \bar{x} - b) + \frac{1}{2} d^T A d \\
 & > & f(\bar{x})
 \end{array}
\end{displaymath}
Last inequality holds since
\begin{itemize}
 \item $d^T A d > 0$ because $d \neq 0$ and $A$ is SPD,
 \item $d^T( A \bar{x} - b) = d^T \nabla f(\bar{x}) \geq 0$ since $\bar{x}$ is minimiser of $f$ and $\Omega$ is convex (every vector from $\bar{x}$ directed to fesible set is ascent (i.e. not descent) vector).
\end{itemize}

For curiosity: in proof we found that $f(\bar{x} + d) = f(\bar{x}) + d^T( A \bar{x} - b) + \frac{1}{2} d^T A d$ which is in fact full Taylor expansion of $f$.

\end{document}


\chapter{Compute the diameter}
\section{Definition of diameter}
The diameter is the maximum distance between two vertices
\section{A simple approach}
In the general case the best approach is calculate APSP (All Pairs Shortest Path) then find the maximum distance, that has a time complexity of $ O(n^{2.38}) $ where the 2.38 derive from some optimization on matrix calculation(see below) for dense graph while $ O(mn) $ for sparse graph. The problem with this simple approach is that is not usable in real-world graphs because contains millions of nodes and edges.
\section{Lower bound for time complexity of diameter computation}
\subsection{SETH Hypothesis}
The Strong Exponential Time Hypothesis (SETH) says that there not exists an algorithm that solves k-SAT in less than $ O((2-\epsilon)^n) $ where $ \epsilon > 0 $
\subsection{Reduction between two problems}
Given two problems A and B, the relative sets of instances of the problem $ I_A $ and $ I_B $ and the solutions set $ A(x) $ and $ B(x) $ of a instance $ x $ we can say that A is reducible to B if exists two functions \textit{f} and \textit{g} where given $ x \in I_A $, $ x' = f(x) \in I_B $, $ y' = B(x') $ and $ g(y', x) \in A(x) $

\begin{figure}[H]
	\centering
	\includegraphics[width=0.7\linewidth]{img/problem_reduction}
	\caption{Graphical representation of problem reduction}
	\label{fig:problemreduction}
\end{figure}
\subsection{K-SAT*}
A variant of the k-sat problem is the $ K-SAT^* $ where between them change only the input: The $ K-SAT^* $ receive two sets of assignments $ X  $ and $ Y$ of size $ n = 2^{\frac{m}{2}} $ to respectively the first half and second half of variables where m is the number of variables and a set of clauses $ C $.\\
In order to find an assignment that satisfy $ C $ we combine each first-half assignment of X to Y, then try to check if applied to the set of clauses returns \textit{true}.\\
So the complexity of the algorithm is virtually $ O(n^2) $ on the input size, but if we substitute n in function of m (the number of variables) we obtain $ O((2^{\frac{m}{2}})^2)  = O(2^m)$. Also this problem cannot have the complexity $ O(n^{2-\varepsilon}) $ unless SETH is false.
Remember that SAT clauses are in CNF form that means:\\ \medskip
\noindent
$ (X_{1}\vee X_{2}\vee \cdots \vee X_{n})\wedge (Y_{1}\vee X_{2}\vee \cdots \vee X_{n})\wedge (X_{1}\vee Y_{2}\vee \cdots \vee X_{n})\wedge (Y_{1}\vee Y_{2}\vee \cdots \vee X_{n})\wedge \cdots \wedge (Y_{1}\vee Y_{2}\vee \cdots \vee Y_{n}). $\\
To resume in words, in CNF  variables in the clause are concatenated with OR operator while the clauses are concatenated with AND operator
\subsection{Disjoint set problem}
Given a set of sets C, the solution is 1 when exists two sets $ A,B \in C $ such that $ A \cap B = \emptyset $.
The complexity of this algorithm is $ O(|C|^2) $.
\subsection{Reduction from K-SAT* to disjoint set}
Given the set of all clauses $ C $ and X,Y respectively the assignment to the first and second half of variables, we define the collection $ S = S_1 \cup S_2 $ as follow:
\begin{center}
	$ S_1:= \{ \{t_1\} \cup \{ c : x \in X \nvDash c \,\, \forall c \in C \} \}$ \\ \medskip
	$ S_2:= \{ \{t_2\} \cup \{ c : y \in Y \nvDash c \,\, \forall c \in C \} \}$
\end{center}
$ t_1 $ and $ t_2 $ are only tokens to avoid the empty intersection between set from the same assignment.\\
To resume, if exists a good assignment in $ K-SAT^* $ then in disjoint sets should exists an empty intersection between two sets, one in $ S_1 $ and another in $ S_2 $.\\
To explain the reduction from $ K-SAT^* $ to disjoin set, a good assignment should satisfy all the clauses in K-SAT* and this thing is transposed in disjoint set with the intersection operator: given $ s_1 \in S_1 $ and $ s_2 \in S_2 $ if $ s_1 \cap s_2 \ne \emptyset$ then exists a clause not satisfied both from X and Y so the clause if false, then is not a good assignment while a good assignment should satisfy all the clauses so it should not have any clause in the intersection.
\subsection{From disjoint set to diameter computation}
Given in input a set of sets $ C $ and the variables $ X $ we can build a clique of the size of  $ X $, then add a node for each subset in $ C $ and add an edge between the node  $ c_i \in C $  and $ x_j $ if $ x_j $ appears in the set $ c_i $.\\
Then we can interpreter the diameter between $ c_i \text{ and } c_j $ in the following mode :
\begin{itemize}
	\item $ diameter(c_i, c_j) = 2  \rightarrow $  exists an intersection between $ c_i $ and $ c_j $
	\item $ diameter(c_i, c_j) = 3  \rightarrow $ not exists an intersection between $ c_i $ and $ c_j $
\end{itemize}
\begin{figure}[H]
	\centering
	\includegraphics[width=0.5\linewidth]{img/disjoint_set_to_diameter}
	\caption{Graphical representation of reduction from disjoint sets to diameter computation}
	\label{fig:disjointsettodiameter}
\end{figure}
\subsection{Complexity of diameter computation}
With the above reductions we have demonstrated that the complexity of diameter computation cannot be in the form $ O(n^{2 - \varepsilon}) $ unless SETH is false, so the complexity of diameter cannot be lower than $ O(n^2)  $
\section{Heuristic for computing the diameter}
\subsection{BFS and diameter}
The height of BFS tree is a lower bound for computing the diameter. We can use this value also as approximation of the diameter doing many BFS from a random vertex. Below we present two methods to approximate the diameter with BFS. This approximations works well in various types of graphs (including social networks) but in other types not, for example road networks.
\subsection{2-SWEEP}
\begin{enumerate}
	\item Pick a random vertex \textit{r}
	\item Do a BFS from r
	\item Pick \textit{x}, one of the farther vertexes from r
	\item Do a BFS from \textit{x} and return the height of the BFS tree as diameter
\end{enumerate}
Complexity: $ 2 \cdot m $

\begin{figure}[H]
	\includegraphics[width=0.4\linewidth]{img/2sweep}
	\caption{2-SWEEP graphical representation}
	\label{fig:2sweep}
\end{figure}
\subsection{4-SWEEP}
\begin{enumerate}
	\item Do a 2-SWEEP
	\item Pick one of the middle vertexes in the longest path of the 2-SWEEP
	\item Do a 2-SWEEP from that vertex
\end{enumerate}
Complexity: $ 4 \cdot m $
\section{Exact heuristic of diameter}
\subsection{Eccentricity}
The eccentricity of a vertex $ ecc(v) $ is the maximum distance between $ v $ and any other node.. In formula it is $ecc(v) = \max_{u\in V}d(u,v) $\\
We can express the diameter in terms of eccentricity: it is the maximum eccentricity among all nodes, or in formula $diameter(G) =  \max_{v \in V}ecc(v) $
\subsection{How heuristic works and complexity}
The intent of this exact heuristic is to find the maximum eccentricity among all nodes in order to return the diameter but the worst case doesn't change: it is always $ O(nm) $ so the reduction from $ K-SAT^* $ is valid but in the mean case this heuristic should be very effective in terms of time cost. For this purpose we use the concept of eccentricity, some definitions and a demonstration.
\subsection{Some terms}
\begin{itemize}
	\item $ F(u)  = \{v | d(u,v) = ecc(u)\}$ so the set of nodes at maximum distance from \textit{u}
	\item $ F_i(u)  = \{v | d(u,v) = i\}$ so the nodes at distance \textit{i} from \textit{u} (e.g. $ F_{ecc(u)(u) = F(u)} $ and $ F_{1}(u) = Neightbours(u) $)
	\item $ B_i(u) = \max_{z \in F_i(u)}ecc(z) $ so the maximum eccentricity between nodes at distance \textit{i} from \textit{u}
\end{itemize}
\subsection{To the upper bound}
Let's demonstrate the following assert:
For any $ 1 \leq i \leq ecc(u) $ and $ 1 \leq k < i $ and for any $ x \in F_{i-k)}(u) $ such that $ ecc(x) >2(i-1) $ there exists $ y \in F_j(u) $ such that $ d(x,y) = ecc(x) $ with $ j \geq i $.
Let's first demonstrate that $ j \geq i $:\\
First, we can do some observations:
\begin{itemize}
	\item for $ x \in F_i(u) $ or $ y \in F_i(u) $ we have that $ d(x,y) < B_i(u) $ because $ d(x,y) \leq \min\{ecc(x), ecc(y)\} \leq B_i(u) $. Remember that $ B_i(u) = max_{x \in F_i(u)}ecc(x) $
	\item for any $ 1 \leq i, j \leq ecc(u) $ and $ \forall x \in F_i(u) , y \in F_j(u) $ we have $ d(x,y) \leq i + j \leq 2 \max\{i,j\} $
\end{itemize}
With this observations we can demonstrate that $ j \geq i $. Suppose for absurd that $ i > j $, $ x \in F_{i-k}(u) $, $ ecc(x) > 2(i-1) $ and $ y_x \in F_j(u)  $ at distance $ ecc(x) $ from $ x $ we have that:
\begin{center}
	$  2(i-1)< ecc(x) = d(x,y_x) \leq 2\max\{i-k, j\} \leq 2\max\{i-k, i-1\} = 2(i-1) $
\end{center}
which is a contradiction so $ j \geq i $.\\
In words, it means that if a node has a eccentricity more than $ 2(i-1) $ and stay above the level \textit{i} then a node at maximum distance from \textit{x} stay below \textit{x} in the \textit{BFS tree} of \textit{u}.\\
Now let's define \textit{lb} as the maximum eccentricity below the level \textit{i} or in then we have an upper bound on $ x \in F_i(u) $ that is $  ecc(x) \leq \max\{lb, 2(i-1)\}$. We can have two cases about $ ecc(x) $ that are:
\begin{itemize}
	\item $ ecc(x) \leq 2(i-1)  \rightarrow $ is true that $ ecc(x) \leq \max\{lb, 2(i-1)\} $
	\item $ ecc(x) > 2(i-1)  \rightarrow $ in this case exists a node \textit{y} below  \textit{x} that has eccentricity greatest or equal than \textit{x}. In the latter assert we have demonstrate that if a node has eccentricity more than \textit{2(i-1)} above the level \textit{i} then exists a node \textit{y} below the level \textit{i} such that $ d(x,y) = ecc(x) $. This imply that the eccentricity of $ y $ is at minimum $ d(x,y) $ but can be also more so we can assert that $ ecc(y) \geq ecc(x) $ . In conclusion $ ecc(x)  \leq ecc(y) \leq lb $, so the dis-equation $ ecc(x) \leq \max{2(i-1), lb}  $ is respected
\end{itemize}
\subsection{The algorithm}
With the upper bound $ \max\{2(i-1), lb\} $ we can assert that if $ M > 2(i-1) $ below the level i doesn't exists a node with eccentricity more than $ lb $
so we can return $ lb $.
We can schematize the algorithm in the following steps:
\begin{enumerate}
	\item Do a BFS from a node \textit{u}
	\item Set $ i = ecc(u) \text{ and } M = B_i(u)$
	\item If $ M > 2(i-1) \text{ return } M; \text{ else set } i = i-1  \text{ and } M = \max\{M, B_i(u)\}$ then repeat this step while not return a value
\end{enumerate}


SymPy includes several submodules that allow users to solve domain specific
physics problems. For example, a comprehensive physics submodule is included
that is useful for solving problems in mechanics, optics, and quantum
mechanics along with support for manipulating physical quantities with units.

\subsection{Classical Mechanics}
One of the core domains that SymPy suports is the physics of classical
mechanics. This is in turn separated into two distinct components:
vector algebra and mechanics.

\subsubsection{Vector Algebra}
% TODO This section requires some citations.

The \verb|sympy.physics.vector| submodule provides reference frame-, time-,
and space-aware vector and dyadic objects that allow for three-dimensional
operations such as addition, subtraction, scalar multiplication, inner and
outer products, and cross products. The vector and dyadic objects both can be
written in very compact notation that make it easy to express the vectors and
dyadics in terms of multiple reference frames with arbitrarily defined
relative orientations. The vectors are used to specify the positions,
velocities, and accelerations of points; orientations, angular velocities, and
angular accelerations of reference frames; and forces and torques. The dyadics
are essentially reference frame-aware $3 \times 3$
tensors~\cite{tai1997generalized}. The vector and dyadic objects can be used
for any one-, two-, or three-dimensional vector algebra, and they provide a
strong framework for building physics and engineering tools.

The following Python code demonstrates how a vector is created using
the orthogonal unit vectors of three reference frames that are oriented with
respect to each other, and the result of expressing the vector in the $A$
frame. The $B$ frame is oriented with respect to the $A$ frame using Z-X-Z
Euler Angles of magnitude $\pi$, $\frac{\pi}{2}$, and
$\frac{\pi}{3}$, respectively, whereas the $C$ frame is oriented
with respect to the $B$ frame through a simple rotation about the $B$ frame's
$X$ unit vector through $\frac{\pi}{2}$.

\begin{verbatim}
>>> from sympy.physics.vector import ReferenceFrame
>>> A, B, C = symbols('A B C', cls=ReferenceFrame)
>>> B.orient(A, 'body', (pi, pi/3, pi/4), 'zxz')
>>> C.orient(B, 'axis', (pi/2, B.x))
>>> v = 1*A.x + 2*B.z + 3*C.y
>>> v
A.x + 2*B.z + 3*C.y
>>> v.express(A)
A.x + 5*sqrt(3)/2*A.y + 5/2*A.z
\end{verbatim}

\subsubsection{Mechanics}

The \verb|sympy.physics.mechanics| submodule utilizes the \texttt{sympy.\allowbreak{}physics.\allowbreak{}vector} submodule
to populate time-aware particle and rigid-body objects to fully describe the
kinematics and kinetics of a rigid multi-body system. These objects store all
of the information needed to derive the ordinary differential or differential
algebraic equations that govern the motion of the system, i.e., the equations
of motion. These equations of motion abide by Newton's laws of motion and can
handle arbitrary kinematic constraints or complex loads. The submodule
offers two automated methods for formulating the equations of motion based on
Lagrangian Dynamics~\cite{Lagrange1811} and Kane's Method~\cite{kane1985dynamics}.
Lastly, there are automated linearization routines for constrained dynamical
systems~\cite{Peterson2014}.

\subsection{Quantum Mechanics}
\label{sec:quantum}

The \verb|sympy.physics.quantum| submodule has extensive capabilities to
solve problems in quantum mechanics, using Python objects to represent the
different mathematical objects relevant in quantum theory~\cite{sakurai2011modern}:
states (bras and kets), operators (unitary, Hermitian, etc.), and basis sets, as
well as operations on these objects such as representations, tensor products,
inner products, outer products, commutators, and anticommutators. The base
objects are designed in the most general way possible to enable any particular
quantum system to be implemented by subclassing the base operators and defining
the relevant class methods to provide system-specific logic.

Symbolic quantum operators and states may be defined, and one can perform
a full range of operations with them.
\begin{verbatim}
>>> from sympy.physics.quantum import Commutator, Dagger, Operator
>>> from sympy.physics.quantum import Ket, qapply
>>> A, B, C, D = symbols('A B C D', cls=Operator)
>>> a = Ket('a')
>>> comm = Commutator(A, B)
>>> comm
[A,B]
>>> qapply(Dagger(comm*a)).doit()
-<a|*(Dagger(A)*Dagger(B) - Dagger(B)*Dagger(A))
\end{verbatim}
Commutators can be expanded using common commutator identities:
\begin{verbatim}
>>> Commutator(C+B, A*D).expand(commutator=True)
-[A,B]*D - [A,C]*D + A*[B,D] + A*[C,D]
\end{verbatim}

On top of this set of base objects, a number of specific quantum systems have
been implemented in a fully symbolic framework. These include:

\begin{itemize}

\item Many of the exactly solvable quantum systems, including simple harmonic
oscillator states and raising/lowering operators, infinite square well states,
and 3D position and momentum operators and states.

\item Second quantized formalism of non-relativistic many-body quantum
mechanics~\cite{fetter2003quantum}.

\item Quantum angular momentum~\cite{zare1988angular}. Spin operators and their
eigenstates can be represented in any basis and for any quantum numbers.
A rotation operator representing the Wigner D-matrix, which may be defined
symbolically or numerically, is also implemented to rotate spin eigenstates.
Functionality for coupling and uncoupling of arbitrary spin eigenstates is
provided, including symbolic representations of Clebsch-Gordon coefficients and
Wigner symbols.

\item Quantum information and computing~\cite{nielsen2010quantum}. Multidimensional
qubit states, and a full set of one- and two-qubit gates are provided and can
be represented symbolically or as matrices/vectors. With these building blocks,
it is possible to implement a number of basic quantum algorithms including the
quantum Fourier transform, quantum error correction, quantum teleportation,
Grover's algorithm, dense coding, etc. In addition, any quantum circuit may be
plotted using the \verb|circuit_plot| function (Figure~\ref{fig-circuitplot-qft}).

\end{itemize}

Here are a few short examples of the quantum information and computing capabilities
in \verb|sympy.physics.quantum|. Start with a simple four-qubit state and flip the second
qubit from the right using a Pauli-X gate:

\begin{verbatim}
>>> from sympy.physics.quantum.qubit import Qubit
>>> from sympy.physics.quantum.gate import XGate
>>> q = Qubit('0101')
>>> q
|0101>
>>> X = XGate(1)
>>> qapply(X*q)
|0111>
\end{verbatim}
Qubit states can also be used in adjoint operations, tensor products, inner/outer
products:
\begin{verbatim}
>>> Dagger(q)
<0101|
>>> ip = Dagger(q)*q
>>> ip
<0101|0101>
>>> ip.doit()
1
\end{verbatim}
Quantum gates (unitary operators) can be applied to transform these states and
then classical measurements can be performed on the results:
\begin{verbatim}
>>> from sympy.physics.quantum.qubit import measure_all
>>> from sympy.physics.quantum.gate import H, X, Y, Z
>>> c = H(0)*H(1)*Qubit('00')
>>> c
H(0)*H(1)*|00>
>>> q = qapply(c)
>>> measure_all(q)
[(|00>, 1/4), (|01>, 1/4), (|10>, 1/4), (|11>, 1/4)]
\end{verbatim}
\begin{figure}[htbp]
\begin{center}
\includegraphics[scale=0.65]{images/fig1-circuitplot-qft}
\caption{The circuit diagram for a three-qubit quantum Fourier transform
generated by SymPy.}
\label{fig-circuitplot-qft}
\end{center}
\end{figure}
Lastly, the following example demonstrates creating a three-qubit quantum Fourier
transform, decomposing it into one- and two-qubit gates, and then generating a
circuit plot for the sequence of gates (see Figure~\ref{fig-circuitplot-qft}).
% This depends on matplotlib. We want to make sure the rest of the paper
% doesn't depend on it, so skip.
% no-doctest
\begin{verbatim}
>>> from sympy.physics.quantum.qft import QFT
>>> from sympy.physics.quantum.circuitplot import circuit_plot
>>> fourier = QFT(0,3).decompose()
>>> fourier
SWAP(0,2)*H(0)*C((0),S(1))*H(1)*C((0),T(2))*C((1),S(2))*H(2)
>>> c = circuit_plot(fourier, nqubits=3)
\end{verbatim}


\documentclass[a4paper]{article}

%  math support
\usepackage{mathtools}
\usepackage{autobreak}
\usepackage{extarrows}
\usepackage[thmmarks,amsmath]{ntheorem}
\usepackage{amssymb}
\usepackage{esint}
% \usepackage{amsmath}
\usepackage{hyperref}

% theorems
\newtheorem{thm}{Theorem}[section]
\newtheorem{defi}[thm]{Definition}
\newtheorem{lemma}[thm]{Lemma}
\newtheorem{prop}[thm]{Proposition}
\newtheorem{coro}[thm]{Corollary}
{
    \theoremstyle{plain}
    \theoremheaderfont{\sffamily\bfseries}
    \theorembodyfont{\normalfont}
    % auto add \QED
    \theoremsymbol{\mbox{$\Box$}}
    \newtheorem{proof}{Proof}[section]
}
{
    \theoremstyle{nonumberplain}
    \newtheorem{myDef}{Definition}
}
% abbr
\newcommand*\Laplace{\mathop{}\!\mathbin\bigtriangleup}
\newcommand\st{\quad \text{s.t.} \quad}
\newcommand\diff{\,\mathrm{d}}
\newcommand\compact{\subset \subset}
\newcommand\xiff\xLongleftrightarrow
\newcommand\ximpliedby\xLongleftarrow
\newcommand\ximplies\xLongrightarrow
\newcommand\xeq\xlongequal
\newcommand\nto\nrightarrow

% declare \norm macro
\DeclarePairedDelimiter{\norm}\lVert\rVert
\DeclarePairedDelimiter{\set}\lbrace\rbrace
\DeclarePairedDelimiter{\abs}\lvert\rvert
% Functional Analysis Symbols
\def\L{\mathcal{L}}
\def\K{\mathcal{K}}
\def\R{\mathbb{R}}
\DeclareMathOperator{\Ima}{Im}
\DeclareMathOperator{\Ker}{Ker}
\DeclareMathOperator{\spann}{span}
\DeclareMathOperator{\tr}{tr}

\newcommand\spanset[1]{\ensuremath\spann\{#1\}}
% \DeclareMathOperator{\dim}{dim}
% domain range
\DeclareMathOperator{\domain}{D}
\DeclareMathOperator{\range}{R}

\begin{document}
\section{Solutions}
\setcounter{proof}{3}
\begin{proof}
For $y = (a, b)$, set $y_1 = (a, -b)$, $y_2 = (-a, b)$, $y_3 = (-a, -b)$, then
\begin{equation*}
G(x, y) = \Gamma(x,y) - \Gamma(x,y_1) - \Gamma(x,y_2) + \Gamma(x,y_3)
\end{equation*}
\end{proof}
\begin{proof}
Set $x = (x_1, \cdots, x_n)$, $y = (a, 0, \cdots, 0)$,
then we use Poisson integral and maximum principle for harmonic functions to get
\begin{equation*}
\begin{split}
& \int_{\partial B_1} \left[ (x_1 - a)^2 + x_2^2 + \cdots + x_n^2)\right]^{-n/2} \diff S_x \\
={} & \int_{\partial B_1} \frac{\diff S_x}{\abs{x-y}^n}
= \frac{1-\abs{y}^2}{n\omega_n}\int_{\partial B_1} \frac{n\omega_n}{1-\abs{y}^2}\frac{1}{\abs{x-y}^n} \diff S_x \\
={} & \frac{n\omega_n}{1-\abs{y}^2}
\end{split}
\end{equation*}
\end{proof}
\setcounter{proof}{6}
\begin{proof}
We use the mean value inequalities for $u(x)$, for any ball $B(x,r) \subset \subset \R^n$, we have
\begin{align*}
\abs{u} &= \abs{\frac{1}{\omega_n r^n}\int_{B(x,r)} u \diff x} \\
&\leq \frac{1}{\omega_n r^n}\int_{B(x,r)} \abs{u} \diff x \\
&\leq (\omega_n r^n)^{-1/p}\norm{u}_{L_p(B(x,r))} \\
&\leq (\omega_n r^n)^{-1/p}\norm{u}_{L_p(\R^n)} \to 0 \quad \text{as} \quad r \to \infty
\end{align*}
Hence $u(x) = 0$
\end{proof}
\setcounter{proof}{8}
\begin{proof}
Set
\begin{equation*}
f(x) = x\log{x},
\end{equation*}
we have
\begin{equation*}
f'(x) = \log{x} + 1 \leq 0 \quad \text{in} \quad (0, 1/e],
\end{equation*}
hence $f(x)$ is nonincreasing in $[0, 1/e]$. Since $f(x) \in C^1((1/e, 1])$, we consider $f(x)$ in $[0, 1/e]$. \\
Set
\begin{equation*}
g(h) = f( x + h ) - f(x) - f(h),
\end{equation*}
we have
\begin{equation*}
g'(h) = \log{\frac{x+h}{h}} \geq 0 \quad \text{in} \quad (0, 1-x],
\end{equation*}
since $g(0)=0$, we have
\begin{equation*}
f(x) - f(x + h) \leq -f(h).
\end{equation*}
Since $f(x)$ is nonincreasing in $[0, 1/e]$, we have
\begin{equation*}
\abs{\frac{f(x)-f(x+h)}{h^\alpha}} = \frac{f(x)-f(x+h)}{h^\alpha} \leq -\frac{f(h)}{h^\alpha} = -h^{1-\alpha}\log{h}.
\end{equation*}
hence $f(x) \in C^{0,\alpha}[0,1]$, where $\alpha \in (0,1)$, and $f(x) \notin C^{0,1}[0,1]$.
\end{proof}
\begin{proof}
Since $A,B$ are both positive-definite matrix, we have
\begin{equation*}
A = P\sqrt{\bar{A}}\sqrt{\bar{A}}P^T, \qquad B = Q\bar{B}Q^T.
\end{equation*}
Then
\begin{align*}
\det{A}\det{B} &= \det{(\sqrt{\bar{A}}P^TQ\bar{B}Q^TP\sqrt{\bar{A}}^T)} \\
&\leq \left( \frac{\tr{(\sqrt{\bar{A}}P^TQ\bar{B}Q^TP\sqrt{\bar{A}}^T)}}{n} \right)^n \\
&= \left( \frac{\tr{(AB)}}{n} \right)^n
\end{align*}
since $\sqrt{\bar{A}}P^TQ\bar{B}Q^TP\sqrt{\bar{A}}^T$ is positive-definite, and use AM-GM inequality.
\end{proof}
\setcounter{proof}{11}
\begin{proof}
WLOG $\forall x > 0$, $\eta \in (0,1)$, we consider $(x\eta, x)$, set
\begin{equation*}
u = \log{x}, \quad \bar{u} = \fint_{x\eta}^x \log{x} \diff x, \quad \tilde{u} = \fint_{x\eta}^x \abs{u - \bar{u}} \diff x,
\quad \bar{u}_0 = \fint_{\eta}^1 \log{t} \diff t.
\end{equation*}
Calculate
\begin{align*}
\bar{u} &= \fint_{\eta}^1 \log{xt} \diff t = \bar{u}_0 + \log{x} \\
\tilde{u} &= \fint_{\eta}^1 \abs{\log{t} + \log{x} - \bar{u}} \diff t = \fint_{\eta}^1 \abs{\log{t} - \bar{u}_0} \diff t \\
\bar{u}_0 &= \frac{\eta \log{\eta} - \eta + 1}{\eta - 1}.
\end{align*}
Since
\begin{equation*}
\fint_{\eta}^1 (\log{t} - \bar{u}_0) \diff t = 0
\end{equation*}
there exist $t_0 \in (\eta, 1)$ s.t. $\log{t_0} = \bar{u}_0$, hence
\begin{equation*}
\frac{1}{1 - \eta}\int_{t_0}^1 (\log{t} - \bar{u}_0) \diff t
= -\frac{1}{1 - \eta}\int^{t_0}_{\eta} (\log{t} - \bar{u}_0) \diff t \geq 0.
\end{equation*}
Then we have
\begin{equation*}
\tilde{u} = \frac{1}{1- \eta}\left( \int_{t_0}^1 - \int_{\eta}^{t_0} (\log{t} - \bar{u}_0) \right) \diff t
= \frac{2}{1-\eta}\int_{t_0}^1 (\log{t} - \bar{u}_0) \diff t
\end{equation*}
since
\begin{equation*}
\log{t} - \bar{u}_0 \leq \log{1} - \bar{u}_0 \leq 1
\end{equation*}
we have
\begin{equation*}
\tilde{u} \leq \frac{2(1-t_0)}{1-\eta} \leq 2.
\end{equation*}
Hence $\log{x} \in BMO(0,\infty)$.
\end{proof}

\section{G-T Chapter 2 Solutions}
\setcounter{proof}{1}
\begin{proof}
Fix $x_0 \in T$, where $T$ is the open, smooth portion of $\partial \Omega$, then we can find $r > 0$, s.t. $B_r(x_0)$ is
divided into  two parts by $T$, then we extend $u$ with zero in $B_r(x_0)-\Omega$.

Now we prove $u$ is harmonic in $B_r(x_0)$. Since $u$ is harmonic in $B_r(x_0) \cap \Omega$ and
$u \equiv 0$ in $B_r(x_0)-\Omega$, all we need to show is $u$ is harmonic in $B_r(x_0) \cap T$,
so we need prove for every ball $B_R(y) \subset \subset \Omega \cup B_r(x_0)$ it satisfies the mean value property,
where $y \in B_r(x_0) \cap T$. By the proof of the mean value inequalities, we only need to show
\begin{equation}\label{2.2.1}
\int_{\partial B_R(y)} \frac{\partial u}{\partial \nu} \diff s = 0.
\end{equation}
$u = \partial u/\partial \nu = 0$ on the smooth portion $T$ guarantees that $u \in C^1$ in $T$,
then we have the integral \eqref{2.2.1} is well-defined, since $u \equiv 0$ in $B_r(x_0) - \Omega$, we have
\begin{equation*}
\int_{\partial B_R(y)} \frac{\partial u}{\partial \nu} \diff s = \int_{B_R(x_0) \cap \Omega} \Laplace u \diff x = 0
\end{equation*}
hence $u$ is harmonic in $\Omega\cup B_r(x_0)$, and $u \equiv 0$ in $B_r(x_0) - \Omega$, by analytic, $u \equiv 0$ in $\Omega$.
\end{proof}
\begin{proof}
\begin{enumerate}
\item Fix $x,y \in \Omega$, $x \neq y$, write
\begin{equation*}
v(z) \coloneqq G(x, z), \quad w(z) \coloneqq G(y, z), \quad z \in \Omega,
\end{equation*}
then
\begin{align*}
\Laplace v(z) &= 0, \qquad z \neq x, \\
\Laplace w(z) &= 0, \qquad z \neq y
\end{align*}
and $w = v = 0$ on $\partial\Omega$.
Consider Green's identity on $\tilde{\Omega} \coloneqq \Omega\setminus[B_{\epsilon}(x)\cup B_{\epsilon}(y)]$
\begin{align*}
& \int_{\tilde{\Omega}} (v\Laplace w - w\Laplace v) \diff z
= \int_{\partial\Omega} \left(v\frac{\partial w}{\partial\nu} - w\frac{\partial v}{\partial\nu}\right)\diff s \\
+{} & \int_{\partial B_{\epsilon}(x)} \left(v\frac{\partial w}{\partial\nu} - w\frac{\partial v}{\partial\nu}\right)\diff s
+ \int_{\partial B_{\epsilon}(y)} \left(v\frac{\partial w}{\partial\nu} - w\frac{\partial v}{\partial\nu}\right)\diff s,
\end{align*}
since $v, w$ is harmonic in $\tilde{\Omega}$, and vanishes on $\partial\Omega$, we have
\begin{equation*}
\int_{\partial B_{\epsilon}(x)} \left(v\frac{\partial w}{\partial\nu} - w\frac{\partial v}{\partial\nu}\right)\diff s
+ \int_{\partial B_{\epsilon}(y)} \left(v\frac{\partial w}{\partial\nu} - w\frac{\partial v}{\partial\nu}\right)\diff s = 0,
\end{equation*}
since
\begin{equation*}
\abs*{\int_{\partial B_{\epsilon}(x)} v\frac{\partial w}{\partial\nu}\diff s}
\leq C\int_{\partial B_{\epsilon}(x)}\abs{v}\diff s \leq C\epsilon
\end{equation*}
and
\begin{equation*}
\lim_{\epsilon \to 0} \int_{\partial B_{\epsilon}(x)}\frac{\partial v}{\partial\nu}w\diff s
= \lim_{\epsilon \to 0} \int_{\partial B_{\epsilon}(x)}\frac{\partial\Gamma}{\partial\nu}(x-z)w(z)\diff s = w(x).
\end{equation*}
Similarly we deduce the other integral to get $w(x) = v(y)$, hence $G(x,y) = G(y,x)$.
\item Consider $w(x) = G(x,y) = \Gamma(x-y) + h$ is harmonic in $\Omega\setminus\set{y}$, since $h$ is harmonic in $\Omega$,
and $\abs*{\Omega} < \infty$, we have $\abs{h} < \infty$ in $\Omega$. But when $x \to y$, $\Gamma(x-y) \to -\infty$,
hence there exist $r > 0$ s.t. $w<0$ in $\partial B_r(y)$

By the strong maximum principle, since $w(x) \equiv 0$ in $\partial\Omega$, we have $w<0$ in $\Omega\setminus B_r(y)$.
\item Fix $x_0 \in \partial\Omega$, since $f$ is bounded, w.l.o.g, we assume $f(y) \equiv 1$, then
\begin{equation*}
\int_{\Omega} \abs{G(x,y)}\diff y = \int_{\Omega\cap B_{2\epsilon}(x_0)} \abs{G(x,y)}\diff y
+ \int_{\Omega\setminus B_{2\epsilon}(x_0)} \abs{G(x,y)}\diff y \eqqcolon I + J.
\end{equation*}
To estimate $I$, since $G(x_0, y) = 0$, for $\epsilon$ sufficiently small,
we have $h(x,y) > 0$ where $x$ in $B_{\epsilon}(x_0)$, hence $\abs{G(x,y)} < \abs{\Gamma(x,y)}$,
and $B_{2\epsilon}(x_0) \subset B_{3\epsilon}(x)$, so we have
\begin{align*}
I \leq \int_{\Omega\cap B_{3\epsilon}(x_0)} \abs{G(x,y)}\diff y
\leq \int_{\Omega\cap B_{3\epsilon}(x_0)} \abs{\Gamma(x,y)}\diff y \leq C\epsilon^2,
\end{align*}
then we have
\begin{align*}
\lim_{x \to x_0}I = 0.
\end{align*}
To estimate $J$, we have
\begin{align*}
\abs{G(x,y)} \leq \abs{\Gamma(\epsilon)} \quad \forall y \in \Omega\setminus B_{2\epsilon}(x_0), x \in B_{\epsilon}(x_0),
\end{align*}
and for any fixed $y$
\begin{align*}
G(x,y) \to 0 \quad \text{as} \quad x \to x_0,
\end{align*}
use Lebesgue’s dominated convergence theorem, we have
\begin{align*}
\lim_{x \to x_0} J = 0,
\end{align*}
hence we completes the proof.
\end{enumerate}
\end{proof}
\begin{proof}
It is suffices to show that $U$ is harmonic on $T$, $\forall x \in T$.

There is a ball $B = B(x)$, by the reflection, we have $U$ is continuous on $\partial B$,
thus we can find a harmonic functions $v$ in $B$,
and $v \equiv U$ on $\partial B$ by Poisson integral formula.

Since $U$ is defined as odd reflection, use Poisson integral we have $v \equiv u \equiv 0$ on $T$,
use the maximum principle on $\Omega^+\cap B$ and $\Omega^-\cap B$, we get $U \equiv v$ in $B$.
Hence $U$ is harmonic in $\Omega^+\cup T\cup\Omega^-$.
\end{proof}
\begin{proof}
WLOG we set the annular region as $B_{R}(0)\setminus B_r(0)$, where $R>r>0$.
All we need to do is let the Green's function vanishes on $\partial B_R\cup\partial B_r$
by combine the fundamental solution $\Gamma$.
\begin{enumerate}
\item Let it vanishes on $\partial B_R$, we have
\begin{align*}
h_1 = \Gamma(x,y) - \frac{\abs{y}}{R}\Gamma(x,\frac{R^2}{\abs{y}^2}y),
\end{align*}
\item let it vanishes on $\partial B_r$, we have
\begin{align*}
h_2 = h_1 - \frac{\abs{y}}{r}\Gamma(x,\frac{r^2}{\abs{y}^2}y) + \frac{R}{r}\Gamma(x,\frac{r^2}{R^2}y),
\end{align*}
\item let it vanishes on $\partial B_R$, we have
\begin{align*}
h_3 = h_2 + \frac{r}{R}\Gamma(x,\frac{R^2}{r^2}y) - \frac{r\abs{y}}{R^2}\Gamma(x,\frac{R^4}{r^2\abs{y}^2}y),
\end{align*}
\item let it vanishes on $\partial B_r$, we have
\begin{align*}
h_4 = h_3 - \frac{R\abs{y}}{r^2}\Gamma(x,\frac{r^4}{R^2\abs{y}^2}y) + \frac{R^2}{r^2}\Gamma(x,\frac{r^4}{R^4}y),
\end{align*}
\item let it vanishes on $\partial B_R$, we have
\begin{align*}
h_5 = h_4 + \frac{r^2}{R^2}\Gamma(x,\frac{R^4}{r^4}y) - \frac{r^2\abs{y}}{R^3}\Gamma(x,\frac{R^6}{r^4\abs{y}^2}y),
\end{align*}
\item $\cdots$
\end{enumerate}
Set
\begin{align*}
g_n &= \left(\frac{r}{R}\right)^{n-1}\Gamma(x,\left(\frac{r}{R}\right)^{2(1-n)}y)
- \left(\frac{r}{R}\right)^{n-1}\frac{\abs{y}}{R}\Gamma(x, \left(\left(\frac{r}{R}\right)^{n-1}\frac{\abs{y}}{R}\right)^{-2}y) \\
&- \left(\frac{R}{r}\right)^{n-1}\frac{\abs{y}}{r}\Gamma(x, \left(\left(\frac{R}{r}\right)^{n-1}\frac{\abs{y}}{r}\right)^{-2}y)
+ \left(\frac{R}{r}\right)^{n}\Gamma(x,\left(\frac{R}{r}\right)^{-2n}y),
\end{align*}
we get the Green's function
\begin{align*}
G(x,y) = \sum_{n=1}^\infty g_n
\end{align*}
where the RHS is convergence by M-test.
\end{proof}
\begin{proof}
Fix $y \in B_R$, $\forall x \in B_R$
\begin{align*}
\abs*{\frac{x-y}{R-\abs{y}}} \geq 1 = \abs*{\frac{x}{R}},
\end{align*}
hence
\begin{align*}
(R-\abs{y})^n\int_{\partial B_R} \frac{u\diff s}{\abs{x-y}^n} \leq R^n \int_{\partial B_R} \frac{u\diff s}{\abs{x}^n},
\end{align*}
it implies that
\begin{align*}
u(y) \leq \frac{R^{n-2}(R+\abs{x})}{(R-\abs{x})^{n-1}}u(0).
\end{align*}
Similarly we can get the lower bound.
\end{proof}
\begin{proof}
$\forall z \in \partial\Omega$, w.l.o.g we assume $z = 0$ and $x_n = 0$ is the tangent hyperplane of $\partial\Omega$ at $z$.
By the definition of $C^2$ boundary, we can choose $x_n = 0$ as the supporting hyperplane and define a support function $f$
in $B_{\epsilon}$ is $C^2(\R^{n-1})$ and satisfying
\begin{align*}
f(x') = x_n > 0 \quad \text{in} \quad B_{\epsilon}\setminus\set{0} \text{, and} \quad f(0)=0,
\end{align*}
where $x' = (x_1, \cdots, x_{n-1})$, and $(x',x_n) \in \partial\Omega$.
Then we consider a ball $B_r(x_0)$ where $r \leq \epsilon$
tangent with the supporting hyperplane $x_n = 0$ at $z$, we can define the support function
\begin{align*}
g(x') = r - \sqrt{r^2 - \abs{x'}^2},
\end{align*}
set $h=g-f$, and $h(0) = 0$, all we need to do is to prove there exist $r$ s.t. $D^2h(0)$ is positive-definite. Since
\begin{align*}
h_{ij} = (r^2 - \abs{x'}^2)^{-3/2}x_ix_j + \delta_{ij}(r^2 - \abs{x'}^2)^{-1/2} - \partial_{ij}f(x'),
\end{align*}
we have
\begin{align*}
\left(h_{ij}(0)\right) = r^{-1}I - (\partial_{ij}f(0)),
\end{align*}
hence we choose $r$ s.t.
\begin{align*}
\frac{1}{r} \geq \max{\abs{\lambda_i}},
\end{align*}
where $\lambda_i$ are eigenvalues of $D^2f(0)$.
\end{proof}
\setcounter{section}{3}
\section{G-T Chapter 4 Solutions}
\setcounter{proof}{6}
\begin{proof}
\begin{equation}\label{4.7.1}
\begin{split}
\Laplace_x u(x/\abs{x}^2)
&= \sum_i \frac{\partial}{\partial x_i}\left(\sum_k\frac{\partial u}{\partial y_k}\frac{\partial y_k}{\partial x_i}\right)
= \sum_i\sum_l\frac{\partial}{\partial y_l}\left(\sum_k\frac{\partial u}{\partial y_k}\frac{\partial y_k}{\partial x_i}\right)
\frac{\partial y_l}{\partial x_i} \\
&= \sum_i\sum_l\sum_k\left(\frac{\partial^2 u}{\partial y_l\partial y_k}\frac{\partial y_k}{\partial x_i}
+\frac{\partial^2 y_k}{\partial x_i\partial y_l}\frac{\partial u}{\partial y_k}\right)\frac{\partial y_l}{\partial x_i} \\
&= \sum_i\sum_l\sum_k\frac{\partial^2 u}{\partial y_l\partial y_k}\frac{\partial y_k}{\partial x_i}
\frac{\partial y_l}{\partial x_i} + \sum_i\sum_k\frac{\partial^2 y_k}{\partial x_i^2}\frac{\partial u}{\partial y_k}
\end{split}
\end{equation}
since
\begin{align*}
\frac{\partial y_k}{\partial x_i} &= \partial_i \frac{x_k}{\abs{x}^2}
= \frac{\delta_{ik}}{\abs{x}^2} - 2\frac{x_ix_k}{\abs{x}^4}, \\
\frac{\partial^2 y_k}{\partial x_i^2}
&= \frac{-2\delta_{ik}x_i}{\abs{x}^4} + 8\frac{x_i^2x_k}{\abs{x}^6}
- 2\frac{x_k}{\abs{x}^4} - 2\frac{x_i\delta_{ik}}{\abs{x}^4} \\
&= -4\frac{\delta_{ik}x_i}{\abs{x}^4} + 8\frac{x_i^2x_k}{\abs{x}^6} - 2\frac{x_k}{\abs{x}^4},
\end{align*}
we have
\begin{align}
\eqref{4.7.1} &= \sum_i\sum_l\sum_k\frac{\partial^2 u}{\partial y_l\partial y_k}
\left(\frac{\delta_{ik}}{\abs{x}^2} - 2\frac{x_ix_k}{\abs{x}^4}\right)
\left(\frac{\delta_{il}}{\abs{x}^2} - 2\frac{x_ix_l}{\abs{x}^4}\right) \label{4.7.a} \\
&+ \sum_i\sum_k\frac{\partial u}{\partial y_k}\left(-4\frac{\delta_{ik}x_i}{\abs{x}^4}
+ 8\frac{x_i^2x_k}{\abs{x}^6} - 2\frac{x_k}{\abs{x}^4}\right) \label{4.7.b}.
\end{align}
For \eqref{4.7.a} we have
\begin{align*}
\eqref{4.7.a} &= \sum_i\sum_l\sum_k\frac{\partial^2 u}{\partial y_l\partial y_k}\frac{\delta_{ik}\delta_{il}}{\abs{x}^4}
- 4\sum_l\sum_k\frac{\partial^2 u}{\partial y_l\partial y_k}\frac{x_kx_l}{\abs{x}^6}
+ 4\sum_i\sum_l\sum_k\frac{x_i^2x_kx_l}{\abs{x}^8} \\
&= \sum_i\frac{\partial^2 u}{\partial y_i^2}\frac{1}{\abs{x}^4}
- 4\sum_l\sum_k\frac{\partial^2 u}{\partial y_l\partial y_k}\frac{x_kx_l}{\abs{x}^6}
+ 4\sum_i\frac{x^2_i}{\abs{x}^2}\sum_l\sum_k\frac{x_kx_l}{\abs{x}^6} \\
&= \sum_i\frac{\partial^2u}{\partial y_i^2}\frac{1}{\abs{x}^4},
\end{align*}
and for \eqref{4.7.b} we have
\begin{align*}
\eqref{4.7.b} &= -4\sum_k\frac{\partial u}{\partial y_k}\frac{x_k}{\abs{x}^4}
+ 8\sum_k\frac{\partial u}{\partial y_k}\frac{x_k}{\abs{x}^4}
- 2n\sum_k\frac{\partial u}{\partial y_k}\frac{x_k}{\abs{x}^4} \\
&= 2(2-n)\sum_k\frac{\partial u}{\partial y_k}\frac{x_k}{\abs{x}^4}.
\end{align*}
Consider
\begin{align*}
\Laplace v(x) = \Laplace{u(x)}\abs{x}^{2-n} + u(x)\Laplace{\abs{x}^{2-n}} + 2\nabla{u}\nabla{\abs{x}^{2-n}},
\end{align*}
since
\begin{align*}
\nabla{u}\nabla{\abs{x}^{2-n}} &= \sum_i\frac{\partial u}{\partial x_i}\frac{\abs{x}^{2-n}}{x_i}
= \sum_i\left(\sum_j\frac{\partial u}{\partial y_j}\frac{\partial y_j}{\partial x_i}\right)(2-n)\abs{x}^{-n}x_i \\
&= (2-n)\abs{x}^{-n}\sum_i\sum_j\frac{\partial u}{\partial y_j}
\left(\frac{\delta_{ij}}{\abs{x}^2} - 2\frac{x_ix_j}{\abs{x}^4}\right)x_i \\
&= (2-n)\left(\abs{x}^{-2-n}\sum_j\frac{\partial u}{\partial y_j}x_j
-2\abs{x}^{-2-n}\sum_j\frac{\partial u}{\partial y_j}x_j\right) \\
&= -(2-n)\abs{x}^{-2-n}\sum_j\frac{\partial u}{\partial y_j}x_j,
\end{align*}
hence
\begin{align*}
\Laplace v(x) &= \Laplace_yu\abs{x}^{-2-n} + 2(2-n)\sum_k\frac{\partial u}{\partial y_k}\frac{x_k}{\abs{x}^{2+n}}
- 2(2-n)\sum_k\frac{\partial u}{\partial y_k}\frac{x_k}{\abs{x}^{2+n}} \\
&= \Laplace_yu\abs{x}^{-2-n}.
\end{align*}
\end{proof}
\end{document}


\section{Densely Connected Convolutional Networks - DenseNet}

A traditional convolutional network consists of L layers and have L connections - one between each layer and its subsequent layer. The DenseNet also known as Densely Connected Convolutional Networks has $L(L+1)/2$ direct connections. The DenseNet has several compelling advantages. For example it alleviates the vanishing-gradient problem, it strengthens the feature propagation and substantially reduce the number of parameters.

A DenseNet does not solely consist of dense blocks. It also consists of convolution, pooling and some classification layer at the end. This can be seen in figure \ref{fig:architecture}. The convolution and pooling layer are referred to as the transition layers.

\myFigure{dens.PNG}{A deep densely connected convolutional network where there are three dense blocks. The blocks between the three dense blocks are adjacent blocks also referred to as transition layers\citep{DENSE}}{fig:architecture}{1}

To improve the flow of information in the DenseNet, a direct connection is used from any layer to all subsequent layers. The results of this is that the $l^{th}$ layer receives the feature-map of all the past layers. The output of the $l^{th}$ layer is denoted as $X_l$.

\begin{equation}
X_l=H_l([x_0,x_1,...,X_{l-1}])
\end{equation}

$[x_0,x_1,...,X_{l-1}]$ refers to the concatenation of the feature-maps, which the layers produces. This network architecture is referred to as DenseNet. The direct connection is illustrated in figure \ref{fig:dense}. The multiple inputs of $H_e(.)$ are concatenated into a single tensor\footnote{Tensors are geometric objects that describe the linear relation between geometric vectors, scalars and other tensors}.

Convolutional networks's becomes deeper. With the increasing depth new problem emerges. Such as the information of the input or gradient are passed through many layers, it can vanish this is also known as the vanishing-gradient problem. A solution for this is to bypass the signal from one layer to the next. The solution in DenseNet is to connect all layers directly with each other. This connection preserves the feed-forward nature and each layers receives additional inputs from all preceding layers. This is seen in figure \ref{fig:dense}.

\myFigure{denselayers.PNG}{A 5 layer dens block \citep{DENSE}}{fig:dense}{0.7}

In figure \ref{fig:dense} a 5 layers dens block is illustrated. Between each of the layers in the dense block \emph{Batch Normalization}, \emph{ReLU} and convolutional layer are applied.