% This is part of the TFTB Tutorial.
% Copyright (C) 1996 CNRS (France) and Rice University (US).
% See the file tutorial.tex for copying conditions.

In contrast with the linear time-frequency representations which
decompose the signal on elementary components (the atoms), the purpose of
the energy distributions is to distribute the {\it energy} of the signal
over the two description variables\,: time and frequency.

  The starting point is that since the energy of a signal $x$ can be deduced
from the squared modulus of either the signal or its Fourier transform,
\begin{eqnarray}
\label{Ex1}
E_x = \int_{-\infty}^{+\infty} |x(t)|^2\ dt\ =\ \int_{-\infty}^{+\infty}
|X(\nu)|^2\ d\nu,
\end{eqnarray}
we can interpret $|x(t)|^2$ and $|X(\nu)|^2$ as energy densities, respectively
in time and in frequency. It is then natural to look for a j{\it oint time and
frequency} energy density $\rho_x(t,\nu)$, such that
\begin{eqnarray}
\label{Ex2}
E_x = \int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} \rho_x(t,\nu)\ dt\
d\nu,
\end{eqnarray}
which is an intermediary situation between those described by
(\ref{Ex1}). As the energy is a quadratic function of the signal, the
time-frequency energy distributions will be in general quadratic
representations.

Two other properties that an energy density should satisfy are the
following {\it marginal properties}\,:\index{marginal properties}
\begin{eqnarray}
\label{fmarg}
\int_{-\infty}^{+\infty} \rho_x(t,\nu)\ dt   &=& |X(\nu)|^2\\
\label{tmarg}
\int_{-\infty}^{+\infty} \rho_x(t,\nu)\ d\nu &=& |x(t) |^2,
\end{eqnarray}
which mean that if we integrate the time-frequency energy density along one
variable, we obtain the energy density corresponding to the other variable.

The main references for this chapter are \cite{FLA93}, \cite{COH89},
\cite{AUG91}, \cite{HLA91} and \cite{HLA92}.

\section{The Cohen's class}
%~~~~~~~~~~~~~~~~~~~~~~~~~~
\label{cohenclass}
\index{Cohen's class}
Since there is much more than one distribution satisfying properties
(\ref{Ex2}), (\ref{fmarg}) and (\ref{tmarg}), we can impose additional
constraints on $\rho_x$ so that this distribution satisfies other desirable
properties. Among these, the covariance principles are of fundamental
importance. The {\it Cohen's class}, to which is dedicated this section,
and whose definition can be found in subsection \ref{cohendef}, is the
class of time-frequency energy distributions {\it covariant by translations
in time and in frequency} \cite{COH89}.

  The spectrogram, that we considered in the previous part, is an element
of the Cohen's class since it is quadratic, time- and frequency- covariant,
and preserves energy (property (\ref{Ex2})).  However, taking the squared
modulus of an atomic decomposition is only a restrictive possibility to
define a quadratic representation, and this definition presents the
drawback that the marginal properties (\ref{fmarg}) and (\ref{tmarg}) are
not satisfied.

\subsection{The Wigner-Ville distribution}
%'''''''''''''''''''''''''''''''''''''''''
\label{WVD}
\subsubsection{Definition}
\index{Wigner-Ville distribution}A time-frequency energy distribution which
is particularly interesting is the {\it Wigner-Ville distribution} (WVD)
defined as\,:
\begin{eqnarray}
\label{wvd}
W_x(t,\nu)=\int_{-\infty}^{+\infty} x(t+\tau/2)\ x^*(t-\tau/2)\ e^{-j2\pi
\nu \tau}\ d\tau,
\end{eqnarray}
or equivalently as
\[W_x(t,\nu)=\int_{-\infty}^{+\infty} X(\nu+\xi/2)\ X^*(\nu-\xi/2)\
e^{j2\pi \xi t}\ d\xi.\] This distribution satisfies a large number of
desirable mathematical properties, as summarized in the next
sub-section. In particular, the WVD is always real-valued, it preserves
time and frequency shifts and satisfies the marginal properties.

  An interpretation of this expression can be found in terms of probability
density\,: expression (\ref{wvd}) is the Fourier transform of an acceptable
form of characteristic function for the distribution of the energy.

  Before looking at the theoretical properties of the WVD, let us see what
we obtain on two particular synthetic signals.
\begin{itemize}
\item {\it Example 1}\,: The first signal is the academic linear chirp
signal that we already considered. The WVD is available thanks to the
M-file \index{\ttfamily tfrwv}{\ttfamily tfrwv.m} of the Time-Frequency
Toolbox (see fig. \ref{En1fig1}).
\begin{verbatim}
     >> sig=fmlin(256);
     >> tfrwv(sig);
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig1.eps}}
\caption{\label{En1fig1}Wigner-Ville distribution of a linear chirp signal
: almost perfect localization in the time-frequency plane}
\end{figure}
If we choose a 3-dimension plot to represent it, we can see that the WVD
can take negative values, and that the localization obtained in the
time-frequency plane for this signal is almost perfect.

\item {\it Example 2}\,: When a car goes in front of an observer with a
constant speed, the signal heard by this person from the engine changes
with time\,: the main frequency decreases (at a first level of
approximation) from one value to another. This phenomenon, known as the
{\it doppler effect}\index{Doppler effect}, expresses the dependence of the
frequency received by an observer from a transmitter on the relative speed
between the observer and the transmitter. The corresponding signal can be
generated thanks to the M-file \index{\ttfamily doppler}{\ttfamily
doppler.m} of the Time-Frequency Toolbox. Here is an example of such a
signal (see fig. \ref{En1fig2})\,:
\begin{verbatim}
     >> [fm,am,iflaw]=doppler(256,50,13,10,200);
     >> sig=am.*fm;
     >> tfrwv(sig);
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig2.eps}}
\caption{\label{En1fig2}WVD of a doppler signal : many interference terms
are present, due to the bilinearity of the distribution}
\end{figure}
Looking at this time-frequency distribution, we notice that the energy is
not distributed as we could expect for this signal. Although the signal
term is well localized in the time-frequency plane, numerous other terms
(the interference terms, due to the bilinearity of the WVD) are present at
positions in time and frequency where the energy should be null. We will
see earlier how to get rid of these terms.
\end{itemize}

\subsubsection{Properties}
\label{propertieswvd}
  Here is a list of the main properties of the WVD \cite{FLA93}.
\begin{enumerate}
\item {\it Energy conservation}\index{energy conservation}\,: by
integrating the WVD of x all over the time-frequency plane, we obtain the
energy of $x$\,:
\[E_x = \int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} W_x(t,\nu)\ dt\
d\nu\]

\item {\it Marginal properties}\index{marginal properties}\,: the energy
spectral density and the instantaneous power can be obtained as marginal
distributions of $W_x$\,:
\begin{eqnarray*}
\int_{-\infty}^{+\infty} W_x(t,\nu)\ dt &=& |X(\nu)|^2\\
\int_{-\infty}^{+\infty} W_x(t,\nu)\ d\nu &=& |x(t)|^2
\end{eqnarray*}

\item {\it Real-valued}\,: \[W_x(t,\nu)\ \in \Rset,\ \forall\ t, \nu\]

\item {\it Translation covariance}\index{translation covariance}\,: the WVD
is time and frequency covariant\,:
\begin{eqnarray*}
y(t)=x(t-t_0) &\Rightarrow & W_y(t,\nu)=W_x(t-t_0,\nu)\\
y(t)=x(t) e^{j2\pi \nu_0 t} &\Rightarrow & W_y(t,\nu)=W_x(t,\nu-\nu_0)
\end{eqnarray*}

\item {\it Dilation covariance}\index{dilation covariance}\,: the WVD also
preserves dilations\,:
\begin{eqnarray*}
y(t)=\sqrt{k}\ x(kt)\ ;\ k>0\ \Rightarrow\  W_y(t,\nu)=W_x(kt,\frac{\nu}{k})
\end{eqnarray*}

\item {\it Compatibility with filterings}\index{compatibility with
filterings}\,: it expresses the fact that if a signal $y$ is the
convolution of $x$ and $h$ (i.e. the output of filter $h$ whose input is
$x$), the WVD of $y$ is the time-convolution between the WVD of $h$ and the
WVD of $x$\,:
\[y(t)=\int_{-\infty}^{+\infty} h(t-s)\ x(s)\ ds\  \Rightarrow\
W_y(t,\nu)=\int_{-\infty}^{+\infty} W_h(t-s,\nu)\ W_x(s,\nu)\ ds\]

\item {\it Compatibility with modulations}\index{compatibility with
modulations}\,: this is the dual property of the previous one\,: if $y$ is
the modulation of $x$ by a function $m$, the WVD of $y$ is the
frequency-convolution between the WVD of $x$ and the WVD of $m$\,:
\[y(t)=m(t)\ x(t)\ \Rightarrow\ W_y(t,\nu)=\int_{-\infty}^{+\infty}
W_m(t,\nu-\xi)\ W_x(t,\xi)\ d\xi\]

\item {\it Wide-sense support conservation}\index{support conservation}\,:
if a signal has a compact support in time (respectively in frequency), then
its WVD also has the same compact support in time (respectively in
frequency)\,:
\begin{eqnarray*}
	x(t)=0,\ |t|>T  &\Rightarrow &  W_x(t,\nu)=0,\ |t|>T\\
	X(\nu)=0,\ |\nu|>B  &\Rightarrow &  W_x(t,\nu)=0,\ |\nu|>B
\end{eqnarray*}

\item {\it Unitarity}\label{unitarity}\index{unitarity}\,: the unitarity property
expresses the conservation of the scalar product from the time-domain to
the time-frequency domain (apart from the squared modulus)\,:
\[\left|\int_{-\infty}^{+\infty} x(t)\ y^*(t)\ dt\right|^2 = \int_{-\infty}^{+\infty}
\int_{-\infty}^{+\infty} W_x(t,\nu)\ W_y^*(t,\nu)\ dt\ d\nu.\]
This formula is also known as the Moyal's formula.

\item {\it Instantaneous frequency}\index{instantaneous frequency}\,: the
instantaneous frequency of a signal $x$ can be recovered from the WVD as
its first order moment (or center of gravity) in frequency\,:
\[f_x(t)=
{\int_{-\infty}^{+\infty} \nu W_{x_a}(t,\nu)\ d\nu
\over
\int_{-\infty}^{+\infty} W_{x_a}(t,\nu)\ d\nu}\]
where $x_a$ is the analytic signal associated to $x$.

\item {\it Group delay}\index{group delay}\,: in a dual way, the group
delay of $x$ can be obtained as the first order moment in time of its WVD\,:
\[t_x(\nu)={\int_{-\infty}^{+\infty} t\ W_{x_a}(t,\nu)\
dt\over \int_{-\infty}^{+\infty} W_{x_a}(t,\nu)\ dt}\]

\item {\it Perfect localization on linear chirp signals}\index{perfect
localization}\,:
\[x(t)=e^{j2\pi \nu_x(t) t}  \mbox{ with }  \nu_x(t)=\nu_0+2\beta t\
		  \Rightarrow\ W_x(t,\nu)=\delta(\nu-(\nu_0+\beta t)).\]
\end{enumerate}

\subsubsection{Interferences}
\index{interferences} As the WVD is a bilinear function of the signal $x$,
the {\it quadratic superposition principle}\index{quadratic superposition
principle} applies\,:
\[W_{x+y}(t,\nu)\ =\ W_x(t,\nu)\ +\ W_y(t,\nu)\ +\
2\Re{\{W_{x,y}(t,\nu)\}}\]
where
\[W_{x,y}(t,\nu)\ =\ \int_{-\infty}^{+\infty} x(t+\tau/2)\ y^*(t-\tau/2)\
e^{-j2\pi \nu \tau}\ d\tau\]
is the cross-WVD of $x$ and $y$. This can be easily generalized to $N$
components, but for the sake of clarity, we will only consider the
two-component case.

  Unlike the spectrogram interference terms, the WVD interference terms
will be non-zero regardless of the time-frequency distance between the two
signal terms. These interference terms are troublesome since they may
overlap with auto-terms (signal terms) and thus make it difficult to
visually interpret the WVD image. However, it appears that these terms must
be present or the good properties of the WVD (marginal properties,
instantaneous frequency and group delay, localization, unitarity \ldots)
cannot be satisfied. Actually, there is a trade-off between the quantity of
interferences and the number of good properties.\\

  o {\it Interference geometry}\\
  The rule of interference construction of the WVD can be summarized as
follows\,: two points of the time-frequency plane interfere to create a
contribution on a third point which is located at their geometrical
midpoint. Besides, these interference terms oscillate perpendicularly to
the line joining the two points interfering, with a frequency proportional
to the distance between these two points.

  This can be seen on the following example\,: we consider two atoms in the
time-frequency plane, analyzed by the WVD, whose relative distance is
increasing from one realization to the other, and then decreasing. The WVDs
were calculated and saved on the file {\ttfamily movwv2at.mat}. We load
them and run the sequence using the function {\ttfamily movie} (see
fig. \ref{En1fig3})\,:
\begin{verbatim}
     >> load movwv2at
     >> clf; movie(M,10);
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=10cm
\centerline{\epsfbox{figure/en1fig3.eps}}
\caption{\label{En1fig3}Structure of the interferences between
2 components with different locations in time and frequency : we can notice
the change in the direction of the oscillations, as well as the change in
the period of these oscillations}
\end{figure}

We can notice, from this movie, the evolution of the interferences
when the distance between the two interfering terms changes, and in
particular the change in the direction of the oscillations.

\subsubsection{Pseudo-WVD}
\label{PWVD}\index{pseudo Wigner-Ville distribution}
  The definition (\ref{wvd}) requires the knowledge of the quantity
\[q_x(t,\tau)=x(t+\tau/2)\ x^*(t-\tau/2)\] from $\tau=-\infty$ to
$\tau=+\infty$, which can be a problem in practice. That is why we often
replace $q_x(t,\tau)$ in (\ref{wvd}) by a windowed version of it, leading
to the new distribution\,:
\[PW_x(t,\nu)=\int_{-\infty}^{+\infty} h(\tau)\ x(t+\tau/2)\ x^*(t-\tau/2)\
e^{-j2\pi \nu \tau}\ d\tau\] where $h(t)$ is a regular window. This
distribution is called the {\it pseudo Wigner-Ville distribution} (noted
pseudo-WVD or PWVD in the following). This windowing operation is
equivalent to a frequency smoothing of the WVD since
\[PW_x(t,\nu) = \int_{-\infty}^{+\infty} H(\nu-\xi)\ W_x(t,\xi)\ d\xi\]
where $H(\nu)$ is the Fourier transform of $h(t)$. Thus, because of their
oscillating nature, the interferences will be attenuated in the pseudo-WVD
compared to the WVD. However, the consequence of this improved readability
is that many properties of the WVD are lost\,: the marginal properties, the
unitarity, and also the frequency-support conservation\,; the
frequency-widths of the auto-terms are increased by this operation.\\

  * {\it Example}\,: The M-file \index{\ttfamily tfrpwv}{\ttfamily
tfrpwv.m} calculates the pseudo-WVD of a signal, with the possibility to
change the length and shape of the smoothing window. If we consider a
signal composed of four gaussian atoms (obtained thanks to \index{\ttfamily
atoms}{\ttfamily atoms.m}), each localized at a corner of a rectangle,
\begin{verbatim}
     >> sig=atoms(128,[32,.15,20,1;96,.15,20,1;...
                       32,.35,20,1;96,.35,20,1]);
\end{verbatim}
and compute its WVD (see fig. \ref{En1fig4})
\begin{verbatim}
     >> tfrwv(sig);
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig4.eps}}
\caption{\label{En1fig4}WVD of 4 gaussian atoms : many interferences are
present}
\end{figure}
we can see the four signal terms, along with six interference terms (two of
them are superimposed). If we now compute the pseudo-WVD (see
fig. \ref{En1fig5}),
\begin{verbatim}
     >> tfrpwv(sig);
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig5.eps}}
\caption{\label{En1fig5}The frequency-smoothing operated by the pseudo-WVD
attenuates the interferences oscillating perpendicularly to the frequency
axis}
\end{figure}
we can note the important attenuation of the interferences oscillating
perpendicularly to the frequency axis, and in return the spreading in
frequency of the signal terms.

\subsubsection{Sampling the WVD\,; the analytic signal}
\index{analytic signal}
  Because of the quadratic nature of the WVD, its sampling has to be done
with care. Let us write it as follows\,:
\[W_x(t,\nu)=2\int_{-\infty}^{+\infty} x(t+\tau)\ x^*(t-\tau)\ e^{-j4\pi
\nu \tau}\ d\tau\]
If we sample $x$ with a period $T_e$, write $x[n]=x(nT_e)$, and evaluate
the WVD at the sampling points $nT_e$ in time, we obtain a discrete-time
continuous-frequency expression of it\,:
\[W_x[n,\nu)=2\ T_e \sum_k x[n+k]\ x^*[n-k]\ e^{-j4\pi \nu k}.\]
As this expression is periodic in frequency with period $\frac{1}{2\ T_e}$
(contrary to period $\frac{1}{T_e}$ obtained for the Fourier transform of a
signal sampled at the Nyquist rate), the discrete version of the WVD may be
affected by a spectral aliasing, in particular if the signal $x$ is
real-valued and sampled at the Nyquist rate.  Two alternatives to this
problem can be found. The first one consists in oversampling the signal by
a factor of at least 2, and the second one in using the analytic
signal. Indeed, as its bandwidth is half the one of the real signal, the
aliasing will not take place in the useful spectral domain $[0,1/2]$ of
this signal. This second solution presents another advantage\,: since the
spectral domain is divided by two, the number of components in the
time-frequency plane is also divided by two. Consequently, the number of
interference terms decreases significantly. To illustrate this phenomenon,
we consider the WVD of the real part of a signal composed of two atoms (see
fig. \ref{En1fig6})\,:
\begin{verbatim}
     >> sig=atoms(128,[32,0.15,20,1;96,0.32,20,1]);
     >> tfrwv(real(sig));
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig6.eps}}
\caption{\label{En1fig6}WVD of a real signal composed of 2 gaussian atoms :
when the analytic signal is not considered, spectral aliasing and additional
interferences appear in the time-frequency plane}
\end{figure}
We can see that four signal terms are present instead of two, due to the
spectral aliasing. Besides, because of the components located at negative
frequencies (between -1/2 and 0), additional interference terms are
present. If we now consider the WVD of the same signal, but in its complex
analytic form (see fig. \ref{En1fig7}),
\begin{verbatim}
     >> tfrwv(sig);
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig7.eps}}
\caption{\label{En1fig7}WVD of the previous signal, but in its analytic
form}
\end{figure}
the aliasing effect has disappeared, as well as the terms corresponding to
interferences between negative- and positive- frequency components.

\subsection{The Cohen's class}
%'''''''''''''''''''''''''''''
\label{cohendef}\index{Cohen's class}
\subsubsection{Presentation}

  Among the desirable properties of an energy time-frequency distribution,
two of them are of particular importance\,: {\it time and frequency
covariance}. Indeed, these properties guaranty that, if the signal is
delayed in time and modulated, its time-frequency distribution is
translated of the same quantities in the time-frequency plane. It has been
shown that the class of energy time-frequency distributions verifying these
covariance properties possesses the following general expression\,:
\[C_x(t,\nu;f)=\int\int\int_{-\infty}^{+\infty}
e^{j2\pi \xi(s-t)}\ f(\xi,\tau)\ x(s+\tau/2)\ x^*(s-\tau/2)\ e^{-j2\pi \nu
\tau}\ d\xi\ ds\ d\tau,\] where $f(\xi,\tau)$ is a two-dimensional function
called the {\it parameterization function}\index{parameterization
function}. This class of distributions is known as the {\it Cohen's class},
which can also be written\,:
\begin{eqnarray}
\label{defcohen1}
C_x(t,\nu;\Pi)=\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty}
\Pi(s-t,\xi-\nu)\ W_x(s,\xi)\ ds\ d\xi,
\end{eqnarray}
where
\[\Pi(t,\nu)=\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty} f(\xi,\tau)\
e^{-j2\pi(\nu \tau+\xi t)}\ dt\ d\nu\]
is the two-dimensional Fourier transform of the parameterization function
$f$. This class is of significant importance since it includes a large number
of the existing time-frequency energy distributions.  Of course, the WVD is
the element of the Cohen's class for which the function $\Pi$ is a double
Dirac\,: $\Pi(t,\nu)=\delta(t)\ \delta(\nu)$, i.e. $f(\xi,\tau)=1$.

  In the case where $\Pi$ is a smoothing function, expression
(\ref{defcohen1}) allows one to interpret $C_x$ as a smoothed version of
the WVD\,; consequently, such a distribution will attenuate in a particular
way the interferences of the WVD.

  Before considering different kinds of smoothing functions $\Pi$, let us
point out the different advantages of such a unified formulation\,:
\begin{enumerate}
\item by specifying the parameterization function $f$ arbitrarily, it is
possible to obtain most of the known energy distributions\,;
\item it is easy to convert a constraint that we wish for the distribution
in an admissibility condition for the parameterization function\,;
\item it is possible, by using such admissibility arguments, to
check {\it a priori} the properties of a particular definition, or to construct a
class of solutions according to a specified schedule of conditions.
\end{enumerate}

\subsubsection{Coupled smoothing}

  If we look at the Moyal's formula (property 9. see page
\pageref{unitarity}), it is easy to express the spectrogram as a smoothing
of the WVD\,:
\begin{eqnarray}
\label{spectro}
S_x(t,\nu)=\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty}
W_h(s-t,\xi-\nu)\ W_x(s,\xi)\ ds\ d\xi.
\end{eqnarray}
Thus, the spectrogram is the element of the Cohen's class for which
$\Pi(s,\xi)$ is the WVD of the window $h$. This new formulation provides us
with another interpretation of the embarrassing trade-off between the time
and frequency- resolutions of the spectrogram\,: if we choose a short
window $h$, the smoothing function will be narrow in time and wide in
frequency, leading to a good time resolution but bad
frequency resolution\,; and vice-versa.

\subsubsection{Separable smoothing}
\label{SPWVD}\index{smoothed-pseudo Wigner-Ville distribution}
  The problem with the previous smoothing function $\Pi(s,\xi)=W_h(s,\xi)$
is that it is controlled only by the short-time window $h(t)$. If we add a
degree of freedom by considering a separable smoothing function
\[\Pi(t,\nu)=g(t)\ H(-\nu)\] (where $H(\nu)$ is the Fourier transform of a
smoothing window $h(t)$), we allow a progressive and independent control,
in both time and frequency, of the smoothing applied to the WVD. The
obtained distribution
\[SPW_x(t,\nu)=\int_{-\infty}^{+\infty} h(\tau)\ \int_{-\infty}^{+\infty}
g(s-t)\ x(s+\tau/2)\ x^*(s-\tau/2)\ ds\ e^{-j2\pi \nu \tau}\ d\tau\] is
known as the {\it smoothed-pseudo Wigner-Ville distribution} (noted
smoothed-pseudo-WVD or SPWVD). The previous compromise of the spectrogram
between time and frequency- resolutions is now replaced by a compromise
between the joint time-frequency resolution and the level of the
interference terms\,: the more you smooth in time and/or frequency, the
poorer the resolution in time and/or frequency.

Note that if we only consider a smoothing in frequency  i.e. if
$g(t)=\delta(t)$, we obtain the pseudo-WVD.\\

  * {\it Example}\,: The signal that we consider here is composed of two
components\,: the first one is a complex sinusoid (normalized frequency
0.15) and the second one is a Gaussian signal shifted in time and
frequency\,:
\begin{verbatim}
     >> sig=fmconst(128,.15) + amgauss(128).*fmconst(128,0.4);
\end{verbatim}
If we display the WVD, the pseudo-WV and the smoothed-pseudo-WVD of this signal (see
fig. \ref{En1fig8}, fig. \ref{En1fig9} and fig. \ref{En1fig10}),
\begin{verbatim}
     >> tfrwv(sig);
     >> tfrpwv(sig);
     >> tfrspwv(sig);
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig8.eps}}
\caption{\label{En1fig8}WVD of a signal composed of a gaussian atom and a
complex sinusoid. Interferences are present between the two components}
\end{figure}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig9.eps}}
\caption{\label{En1fig9}Pseudo-WVD of the same signal : the frequency
smoothing done by the pseudo-WVD degrades the frequency resolution without
really attenuating the interferences}
\end{figure}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig10.eps}}
\caption{\label{En1fig10}Smoothed-pseudo-WVD of the same signal : the
time-smoothing carried out by the smoothed-pseudo-WVD considerably reduces
these interferences}
\end{figure}
we can make the following remarks\,: from the WVD, we can see the two signal
terms located at the right positions in the time-frequency plane, as well
as the interference terms between them. As these interference terms
oscillate globally perpendicularly to the time-axis, the frequency
smoothing done by the pseudo-WVD degrades the frequency resolution without
really attenuating the interferences. On the other hand, the time-smoothing
carried out by the smoothed-pseudo-WVD considerably reduces these
interferences\,; and as the time resolution is not of fundamental importance
here, this representation is suitable for this signal.

  An interesting property of the smoothed-pseudo WVD is that it allows a
continuous passage from the spectrogram to the WVD, under the condition
that the smoothing functions $g$ and $h$ are gaussian. The time-bandwidth
product then goes from 1 (spectrogram) to 0 (WVD), with an independent
control of the time and frequency resolutions. This is clearly illustrated
by the function \index{\ttfamily movsp2wv}{\ttfamily movsp2wv.m}, which
considers different transitions, on a signal composed of four atoms. To
visualize these snapshots, load the mat-file {\ttfamily movsp2wv} (obtained
by running {\ttfamily movsp2wv.m}\,; but as it takes a long time to run, we
saved the result in a mat file) and run {\ttfamily movie} (see
fig. \ref{En1fig11})\,:
\begin{verbatim}
     >> load movsp2wv
     >> clf; movie(M,10);
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=10cm
\centerline{\epsfbox{figure/en1fig11.eps}}
\caption{\label{En1fig11}Different transitions from the spectrogram to the
WVD, using the smoothed-pseudo-WVD. The signal is composed of 4 gaussian
atoms}
\end{figure}
This movie shows the effect of a (time/frequency) smoothing on the
interferences and on the resolutions\,: the WVD gives the best resolutions
(in time and in frequency), but presents the most important interferences,
whereas the spectrogram gives the worst resolutions, but with nearly no
interferences\,; and the smoothed-pseudo WVD allows to choose the best
compromise between these two extremes.

\subsection{Link with the narrow-band ambiguity function}
%''''''''''''''''''''''''''''''''''''''''''''''''''''''''
\subsubsection{Definition and properties}
\label{NBAF}\index{narrow-band ambiguity function}
  A function of particular interest, especially in the field of radar
signal processing, is the {\it narrow-band ambiguity function} (noted AF),
defined as
\[A_x(\xi,\tau)=\int_{-\infty}^{+\infty} x(s+\tau/2)\ x^*(s-\tau/2)\
e^{-j2\pi \xi s}\ ds.\]

This function, also known as the (symmetric) {\it Sussman ambiguity
function}, is a measure of the time-frequency correlation of a signal $x$,
i.e. the degree of similarity between $x$ and its translated versions in
the time-frequency plane. Unlike the variables '$t$' and '$\nu$' which are
"absolute" time and frequency coordinates, the variables '$\tau$' and
'$\xi$' are "relative" coordinates (respectively called {\it delay} and
{\it doppler}).\index{delay}\index{doppler}

  The AF is generally complex-valued, and satisfies the Hermitian even
symmetry\,:
\[A_x(\xi,\tau) = A_x^*(-\xi,-\tau).\]

  An important relation exists between the narrow-band ambiguity function
and the WVD, which says that the ambiguity function is the two-dimensional
Fourier transform of the WVD\,:
\[A_x(\xi,\tau)=\int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty}
W_x(t,\nu)\ e^{j2\pi(\nu \tau-\xi t)}\ dt\ d\nu.\]
Thus, the AF is the dual of the WVD in the sense of the Fourier
transform. Consequently, for the AF, a dual property corresponds to nearly
all the properties of the WVD. Among these properties, we will
restrict ourselves to only three of them, which are important for the
following\,:
\begin{itemize}
\item Marginal properties

  The temporal and spectral auto-correlations are the cuts of the AF along
the $\tau$-axis and $\xi$-axis respectively\,:
\[ r_x(\tau)=A_x(0,\tau) \mbox{ and } R_x(\xi)=A_x(\xi,0).\]
The energy of $x$ is the value of the AF at the origin of the
$(\xi,\tau)$-plane, which corresponds to its maximum value\,:
\[|A_x(\xi,\tau)|\ \leq\ A_x(0,0)\ =\ E_x,\ \forall \xi, \tau.\]

\item TF-shift invariance

  Shifting a signal in the time-frequency plane leaves its AF invariant
apart from a phase factor (modulation)\,:
\[y(t) = x(t-t_0)\ e^{j2\pi \nu_0 t}\
 \Rightarrow A_y(\xi,\tau) = A_x(\xi,\tau)\ e^{j2\pi(\nu_0 \tau-t_0 \xi)}\]

\item Interference geometry

  In the case of a multi-component signal, the elements of the AF
corresponding to the signal components (denoted as the AF-signal terms) are
mainly located around the origin, whereas the elements corresponding to
interferences between the signal components (AF-interference terms) appear
at a distance from the origin which is proportional to the time-frequency
distance between the involved components. This can be noticed on a simple
example\,:\\

  * {\it Example}\,: The M-file \index{\ttfamily ambifunb}{\ttfamily
ambifunb.m} of the TF Toolbox implements the narrow-band ambiguity
function. We apply it on a signal composed of two linear FM signals with
gaussian amplitudes\,:
\begin{verbatim}
     >> N=64; sig1=fmlin(N,0.2,0.5).*amgauss(N);
     >> sig2=fmlin(N,0.3,0).*amgauss(N);
     >> sig=[sig1;sig2];
\end{verbatim}
Let us first have a look at the WVD (see fig. \ref{En1fig12})\,:
\begin{verbatim}
     >> tfrwv(sig);
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig12.eps}}
\caption{\label{En1fig12}WVD of  2 chirps with gaussian amplitudes and
different slopes}
\end{figure}
We have two distinct signal terms, and some interferences oscillating in
the middle. If we look at the ambiguity function of this signal (see
fig. \ref{En1fig13}),
\begin{verbatim}
     >> ambifunb(sig);
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig13.eps}}
\caption{\label{En1fig13}Narrow-band ambiguity function of the previous
signal : the AF-signal terms are located around the origin, whereas the
AF-interference terms are located away from the origin}
\end{figure}
we have around the origin (in the middle of the image) the AF-signal terms,
whereas the AF-interference terms are located away from the origin. Thus,
applying a 2-D low pass filtering around the origin on the ambiguity
function, and returning to the WVD by 2-D Fourier transform will attenuate
the interference terms. Actually, this 2-D filtering is operated, in the
general expression of the Cohen's class, by the parameterization function $f$,
as we discuss it now.
\end{itemize}

\subsubsection{New interpretation of the Cohen's class}

  The dual expression of the Cohen's class formulation (expression
(\ref{defcohen1})) in terms of AF writes
\begin{eqnarray}
\label{defcohen2}
C_x(t,\nu;f) = \int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty}
f(\xi,\tau)\ A_x(\xi,\tau)\ e^{-j2\pi(\nu \tau+\xi t)}\ d\xi\ d\tau
\end{eqnarray}
(recall that $f$ is the two-dimensional Fourier transform of $\Pi$).  This
expression is very instructive about the role played by the parameterization
function $f(\xi,\tau)$. Indeed, $f$ acts as a weighting function that tries to
let the signal terms unchanged, and to reject the interference
terms. Actually, the change from the time-frequency plane to the ambiguity
plane allows a precise characterization of the weighting function $f$, and
thus of the smoothing function $\Pi(t,\nu)$.

  For example, the WVD corresponds to a constant parameterization
function\,: $f(\xi,\tau)=1,\ \forall\ \xi,\ \tau$\,: no difference is made
between the different regions of the ambiguity plane. For the spectrogram,
$f(\xi,\tau)=A_h^*(\xi,\tau)$\,: the ambiguity function of the window $h$
determines the shape of the weighting function. And for the
smoothed-pseudo-WVD, we have $f(\xi,\tau)=G(\xi)\ h(\tau)$\,: the weighting
function is separable in time and frequency, which is very useful to adapt
it to the shape of the AF-signal terms.

  We will end this section by presenting other energy distributions that
are members of the Cohen's class.

\subsection{Other important energy distributions}
%''''''''''''''''''''''''''''''''''''''''''''''''
\subsubsection{The Rihaczek and Margenau-Hill distributions}\label{MHD}
\index{Rihaczek distribution}\index{Margenau-Hill distribution} Another
  possible definition of a time-frequency energy density is given by the
  Rihaczek distribution. If we consider the interaction energy between a
  signal $x$ restricted to an infinitesimal interval $\delta_T$ centered on
  $t$, and $x$ passed through an infinitesimal bandpass filter $\delta_B$
  centered on $\nu$, it can be approximated by the following expression\,:
\[\delta_T\ \delta_B\ [x(t)\ X^*(\nu)\ e^{-j2\pi \nu t}].\]
This leads us to interpret the quantity
\[R_x(t,\nu)=x(t)\ X^*(\nu)\ e^{-j2\pi \nu t},\]
called the {\it Rihaczek distribution}, as a complex energy density at
point $(t,\nu)$.  This distribution, which corresponds to the element of
the Cohen's class for which $f(\xi,\tau)=e^{j\pi \xi \tau}$, verifies many
good properties (1-2, 4-11, see section \ref{propertieswvd}). However, it
is complex valued, which can be awkward in practice. It is implemented
under the name \index{\ttfamily tfrri}{\ttfamily tfrri.m}. The real part of
the Rihaczek distribution is also a time-frequency distribution of the
Cohen's class ($f(\xi,\tau)=\cos{(\pi \xi \tau)}$), known as the {\it
Margenau-Hill distribution} (see the M-file \index{\ttfamily
tfrmh}{\ttfamily tfrmh.m}). It has also numerous interesting properties\,:
1-5, 8, 10-11. As for the WVD, we can define smoothed versions of the
Rihaczek and Margenau-Hill distributions. The file \index{\ttfamily
tfrpmh}{\ttfamily tfrpmh.m} computes the pseudo Margenau-Hill distribution.

  The interference structure of the Rihaczek and Margenau-Hill
distributions is different from the Wigner-Ville one\,: the interference
terms corresponding to two points located on $(t_1,\nu_1)$ and
$(t_2,\nu_2)$ are positioned at the coordinates $(t_1,\nu_2)$ and
$(t_2,\nu_1)$. This can be seen on the following example (see
fig. \ref{En1fig14})\,:
\begin{verbatim}
     >> sig=atoms(128,[32,0.15,20,1;96,0.32,20,1]);
     >> tfrmh(sig);
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig14.eps}}
\caption{\label{En1fig14}Margenau-Hill distribution of 2 atoms : the
position of the interferences is quite different from the one obtained with
the WVD}
\end{figure}
Thus, the use of the Rihaczek (or Margenau-Hill) distribution for signals
composed of multi-components located at the same position in time or in
frequency is not advised, since the interference terms will then be
superposed to the signal terms.

\subsubsection{The Page distribution}
\index{Page distribution}
  Motivated by the construction of a causal energy density, Page proposed
the following distribution (the {\it Page distribution})\,:
\begin{eqnarray*}
P_x(t,\nu) &=& {d\over dt}\,
\left\{{
|\int_{-\infty}^t x(u)\ e^{-j2\pi \nu u}\
du|^2]
}\right\}\\
 &=& 2\ \Re{\left\{x(t)\ \left(\int_{-\infty}^t\ x(u)\ e^{-j2\pi \nu
u}du\right)^* \ e^{-j2\pi \nu t}\right\}}
\end{eqnarray*}
It is the derivative of the energy spectral density of the signal
considered before time $t$. It corresponds to the element of the Cohen's
class with parameterization function $f(\xi,\tau)=e^{-j\pi \xi |\tau|}$, and
verifies the properties 1-5, 7-10 (see section \ref{propertieswvd}).
Actually, it is the only distribution of the Cohen's class which is
simultaneously causal, unitary, compatible with modulations, and preserves
time-support.

\index{pseudo-Page distribution} The function \index{\ttfamily
tfrpage}{\ttfamily tfrpage.m} computes this distribution. A
frequency-smoothed version of the Page distribution, called the {\it
pseudo-Page distribution}, is also available (see the file \index{\ttfamily
tfrppage}{\ttfamily tfrppage.m}).

\subsubsection{Joint-smoothings of the WVD}

  The following distributions correspond to particular cases of the Cohen's
class for which the parameterization function depends only on the product of
the variables $\tau$ and $\xi$\,:
\begin{eqnarray}
\label{paramfun}
f(\xi,\tau)=\Phi(\tau\xi)
\end{eqnarray}
where $\Phi$ is a decreasing function such that $\Phi(0)=1$ (the Rihaczek
and Margenau-Hill distributions are particular elements of this class). A
direct consequence of this definition is that the marginal properties will
be respected. Besides, since $\Phi$ is a decreasing function, $f$ is a
low-pass function, and according to (\ref{defcohen2}), this parameterization
function will reduce the interferences. That is why these distributions are
also known as the {\it Reduced Interference Distributions}.\index{Reduced
Interference Distributions}
\label{RID}
\begin{itemize}
\item The {\it Choi-Williams distribution	}

\index{Choi-Williams distribution}
  One natural choice for Phi is to consider a gaussian function\,:
\[f(\xi,\tau)=\exp{\left[-\frac{(\pi \xi \tau)^2}{2\sigma^2}\right]}.\]
The corresponding distribution,
\[CW_x(t,\nu)=\sqrt{\frac{2}{\pi}}
\int\int_{-\infty}^{+\infty}
{\sigma\over |\tau|}\
e^{-2\sigma^2(s-t)^2/\tau^2}\ x(s+\frac{\tau}{2})\ x^*(s-\frac{\tau}{2})\
e^{-j2\pi \nu \tau}\ ds\ d\tau\] is the Choi-Williams distribution. Note
that when $\sigma\ \longrightarrow\ +\infty$, we obtain the WVD. Inversely,
the smaller $\sigma$, the better the reduction of the interferences. This
distribution verifies properties 1-5, 10-11, and can be computed with the
M-file \index{\ttfamily tfrcw}{\ttfamily tfrcw.m}.  The "cross"-shape of
the parameterization function of the Choi-Williams distribution implies that
the efficiency of this distribution strongly depends on the nature of the
analyzed signal. For instance, if the signal is composed of synchronized
components in time or in frequency, the Choi-Williams distribution will
present strong interferences. This can be observed on the following
example\,: we analyze four gaussian atoms positioned at the corners of a
rectangle rotating around the center of the time-frequency plane (see
fig. \ref{En1fig15})\,:
\begin{verbatim}
     >> load movcw4at
     >> clf; movie(M,5);
\end{verbatim}
\begin{figure}[htb]
\epsfxsize=10cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig15.eps}}
\caption{\label{En1fig15}Choi-Williams distribution of 4 atoms rotating
around the middle of the time-frequency plane : when the time/frequency
supports of the atoms overlap, strong interferences appear on the overlap
support}
\end{figure}
When the time/frequency supports of the atoms overlap, some AF-interference
terms are not completely attenuated (those present around the axes of
the ambiguity plane), and the efficiency of the distribution is quite poor.

\item The {\it Born-Jordan} and {\it Zhao-Atlas-Marks distributions}
\index{Born-Jordan distribution}\index{Zhao-Atlas-Marks distribution}

  If we impose to the distributions defined by (\ref{paramfun}) the further
condition to preserve time- and frequency- supports, the simplest choice
for $f$ is then\,:
\[f(\xi,\tau)={\sin{(\pi \xi \tau)}\over\pi \xi \tau}\]
which defines the {\it Born-Jordan distribution}\,:
\[BJ_x(t,\nu)=\int_{-\infty}^{+\infty} \frac{1}{|\tau|}\
\int_{t-|\tau|/2}^{t+|\tau|/2} x(s+\tau/2)\ x^*(s-\tau/2)\ ds\ e^{-j2\pi
\nu \tau} d\tau.\]

Properties 1-5, 8, 10-11 are verified by this distribution, and the
corresponding M-file of the Time-Frequency Toolbox is \index{\ttfamily
tfrbj}{\ttfamily tfrbj.m}.

  If we smooth the Born-Jordan distribution along the frequency axis, we
obtain the {\it Zhao-Atlas-Marks distribution}, defined as
\[ZAM_x(t,\nu)=\int_{-\infty}^{+\infty} \left[\ h(\tau)\
\int_{t-|\tau|/2}^{t+|\tau|/2} x(s+\tau/2)\ x^*(s-\tau/2)\ ds\right]\
e^{-j2\pi \nu \tau}\ d\tau. \]

This distribution, also known as the {\it Cone-Shaped Kernel distribution},
validates properties 3-4 and 8 (only for time) (see the M-file
\index{\ttfamily tfrzam}{\ttfamily tfrzam.m} for its computation).
\end{itemize}

\subsubsection{Comparison of the parameterization functions}

  To illustrate the differences between some of the presented
distributions, we represent their weighting (parameterization) function in
the ambiguity plane, along with the result obtained by applying them on a
two-component signal embedded in white gaussian noise\,: the signal is the
sum of two linear FM signals, the first one with a frequency going from
0.05 to 0.15, and the second one from 0.2 to 0.5. The signal to noise ratio
is 10\,dB.

  On the left-hand side of the figures \ref{En1fig16} and \ref{En1fig17},
the parameterization functions are represented in a schematic way by the
bold contour lines (the weighting functions are mainly non-zeros inside
these lines), superimposed to the ambiguity function of the signal. The
AF-signal terms are in the middle of the ambiguity plane, whereas the
AF-interference terms are distant from the center. On the right-hand side,
the corresponding time-frequency distributions are represented.
\begin{figure}[htb]
\epsfxsize=12cm
\epsfysize=10cm
\centerline{\epsfbox{figure/en1fig16.eps}}
\caption{\label{En1fig16}Two chirps embedded in a 10 dB white gaussian
noise analyzed by different quadratic distributions. On the left-hand side,
the parameterization function is represented by a bold contour line,
superimposed to the ambiguity function of the signal. The AF-signal terms
are in the middle of the ambiguity plane, whereas the AF-interference terms
are distant from the center. On the right-hand side, the corresponding
time-frequency distribution is represented}
\end{figure}
\begin{figure}[htb]
\epsfxsize=12cm
\epsfysize=8cm
\centerline{\epsfbox{figure/en1fig17.eps}}
\caption{\label{En1fig17}Two chirps embedded in a 10 dB white gaussian
noise analyzed by different quadratic distributions (concluding)}
\end{figure}

From these plots, we can conclude that the ambiguity plane is very
enlightening with regard to interference reduction in the case of
multicomponent signals. On this example, we notice that the
smoothed-pseudo-WVD is a particularly convenient and versatile
candidate. This is due to the fact that we can adapt independently the
time-width and frequency-width of its weighting function. But in the
general case, it is interesting to have several distributions at our
disposal since each one is well adapted to a certain type of
signal. Besides, for a given signal, as a result of the different
interference geometries, these distributions offer complementary
descriptions of this signal.

\subsection{Conclusion}
%''''''''''''''''''''''
  The Cohen's class, which gather all the quadratic time-frequency
distributions covariant by shifts in time and in frequency, offers a wide
set of powerful tools to analyze non-stationary signals. The basic idea is
to devise a joint function of time and frequency that describes the energy
density or intensity of a signal simultaneously in time and in
frequency. The most important element of this class is probably the
Wigner-Ville distribution, which satisfies many desirable properties. Since
these distributions are quadratic, they introduce cross-terms in the
time-frequency plane which can disturb the readability of the
representation. One way to attenuate these interferences is to smooth the
distribution in time and in frequency, according to their structure. But
the consequence of this is a decrease of the time and frequency
resolutions, and more generally a loss of theoretical properties. The
general formulation proposed by Cohen is very useful to have a better
understanding of the existing solutions, as well as the connection with the
ambiguity function.

But there exists other time-frequency energy distributions, which are not
elements of the Cohen's class, i.e. which are not covariant by shifts in
time or in frequency. This is the case for example of the affine
distributions, which are presented in the next chapter.


\subsection{Eigenvalue Method}
\noindent
This method allows us to find fundamental solutions using the eigenvalues of the matrix $A$.
Of course, just like with roots of the auxiliary equation, we need to consider the cases of real distinct eigenvalues and eigenvectors, repeated eigenvalues, complex eigenvalues, and defective matrices.

% n distinct eigenvalues
\input{./linearSystems/homogeneousSystems/realDistinctEigenvalues.tex}
% Repeated eigenvales with enough eigenvectors
\input{./linearSystems/homogeneousSystems/repeatedEigenvalues.tex}
% Defective matrices
\input{./linearSystems/homogeneousSystems/complexEigenvalues.tex}
% Complex roots
\input{./linearSystems/homogeneousSystems/defectiveMatrices.tex} \\

\noindent
This final approach for defective matrices actually gives a general way to solve $\vec{x}' = A\vec{x}$ for any real $n \times n$ matrix $A$.
\begin{enumerate}[label = \arabic*)]
	\item
		Compute the characteristic polynomial $p(\lambda) = \det{(A - \lambda I)}$.
		Find the zeroes of the characteristic to find the distinct eigenvalues $\lambda_1, \ldots, \lambda_k$ with corresponding multiplicities $m_1, \ldots, m_k$.
	\item
		For each eigenvalue $\lambda_i$ find the $m_i$ corresponding linearly independent eigenvectors of rank $m_i$ by solving the system $(A - \lambda_i)^{m_i}\vec{v_i} = \vec{0}$.
	\item
		For each generalized generalized eigenvector $\vec{v_i}$, compute the corresponding fundamental solution
		\begin{equation*}
			\vec{x_i} = e^{At}\vec{v_i} = e^{\lambda t}\left(\vec{v_i} + t(A - \lambda I)\vec{v_i} + \frac{t^2}{2!}(A - \lambda I)^2\vec{v_i} + \ldots\right).
		\end{equation*}
		Note that this summation terminates is at most $m_i$ terms.
	\item
		Combine these fundamental solutions as columns of a a square matrix to obtain the fundamental matrix of the system $X(t)$.
		The general solution to the system is
		\begin{equation*}
			\vec{x} = X(t)\begin{bmatrix}
				C_1 \\
				\vdots \\
				C_n
			\end{bmatrix}.
		\end{equation*}
\end{enumerate}


\section{Trust Flow}
  Everything is in place to define the indirect trust from one player to another.
  \subimport{common/definitions/}{indirecttrust.tex}
  \noindent Note that $Tr_{A \rightarrow B} \geq DTr_{A \rightarrow B}$. The next theorem shows that $Tr_{A \rightarrow B}$ is
  finite.
  \subimport{thesis/theorems/}{convergencetheorem.tex}
  \subimport{common/proofsketches/}{convergenceproofsketch.tex}
  Full proofs of all theorems and lemmas can be found in the Appendix.

  In the setting of \texttt{TransitiveGame(}$\mathcal{G}$\texttt{,}$A$\texttt{,}$B$\texttt{)}, we make use of the notation
  $Loss_A = Loss_{A, j}$, where $j$ is a turn in which the game has converged. It is important to note that $Loss_A$ is not
  the same for repeated executions of this kind of game, since the order in which players are chosen may differ between
  executions and the conservative players are free to choose which incoming direct trusts they will steal and how much from
  each.

  Let $G$ be a weighted directed graph. We will investigate the maximum flow on this graph. For an introduction to the
  maximum flow problem see \cite{clrs} p. 708. Considering each edge's capacity as its weight, a flow assignment
  $X = [x_{vw}]_{\mathcal{V} \times \mathcal{V}}$ with a source $A$ and a sink $B$ is valid when:
  \begin{equation}
  \label{flow1}
    \forall (v, w) \in \mathcal{E}, x_{vw} \leq c_{vw} \mbox{ and}
  \end{equation}
  \begin{equation}
  \label{flow2}
    \forall v \in \mathcal{V} \setminus \{A,B\}, \sum\limits_{w \in N^{+}(v)}x_{wv} = \sum\limits_{w \in N^{-}(v)}x_{vw}
    \enspace.
  \end{equation}
  We do not suppose any skew symmetry in $X$. The flow value is $\sum\limits_{v \in N^{+}\left(A\right)}x_{Av}$, which is
  proven to be equal to $\sum\limits_{v \in N^{-}\left(B\right)}x_{vB}$. There exists an algorithm that returns the maximum
  possible flow from $A$ to $B$, namely $MaxFlow\left(A, B\right)$. This algorithm evidently needs full knowledge of the
  graph. The fastest version of this algorithm runs in $O\left(|\mathcal{V}||\mathcal{E}|\right)$ time \cite{maxflownm}. We
  refer to the flow value of $MaxFlow\left(A, B\right)$ as $maxFlow\left(A, B\right)$.

  We will now introduce two lemmas that will be used to prove the one of the central results of this work, the Trust Flow
  theorem.
  \subimport{thesis/lemmas/}{flowgamelemma.tex}
  \subimport{common/proofsketches/}{flowgameproofsketch.tex}
  \subimport{thesis/lemmas/}{gameflowlemma.tex}
  \subimport{common/proofsketches/}{gameflowproofsketch.tex}
  \subimport{common/theorems/}{trustflowtheorem.tex}
  \subimport{thesis/proofs/}{trustflowproof.tex}

  \noindent We note that the maxFlow is the same in the following two cases: When a player chooses the evil strategy and when
  the same player chooses a variation of the evil strategy where she does not nullify her outgoing direct trust.

  Further justification of trust transitivity through the use of $MaxFlow$ can be found in the sociological experiment
  conducted in \cite{kmrs}.

  Here we see another important theorem that gives the basis for risk-invariant transactions between different, possibly
  unknown, parties.
  \subimport{common/theorems/}{riskinvtheorem.tex}
  \subimport{common/proofs/}{riskinvproof.tex}
  \noindent It is intuitively obvious that it is possible for $A$ to reduce her outgoing direct trust in a manner that
  achieves (\ref{primetrust}), since $maxFlow\left(A, B\right)$ is continuous with respect to $A$'s outgoing direct trusts. We
  leave this calculation as part of further research.


\section{Bellman-Ford and Floyd-Warshall Algorithms}

%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Bellman-Ford algorithm}
  \begin{exampleblock}{Bellman-Ford algorithm (Problem 6.30)}
	\begin{itemize}
	  \item digraph $G = (V, E), \exists e: w(e) < 0$
	  \item $\forall s, t: |s \leadsto^{\SP} t| \le k$
	  \item Given $s,t$, find $s \leadsto^{\SP} t$.
	\end{itemize}
  \end{exampleblock}

  $\text{d}(v,k)$: shortest path distance from $s$ to $v$ using $\le k$ edges

  \[
	s \leadsto^{\SP; \le k-1} u \to^{=1} v
  \]

  \[
	\text{d}(v, k) = \min_{u \in N(v)} \text{d}(u, k-1) + w(u,v)
  \]

  \begin{center}
	\begin{minipage}{0.50\linewidth}
	  \begin{algorithmic}[c]
		\ForAll{$i = 1 \to n - 1$}
		  \ForAll{$e \in E$}
			\State update $e$
		  \EndFor
		\EndFor
	  \end{algorithmic}
	\end{minipage}
  \end{center}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Floyd-Warshall algorithm}
  \[
	\dist(i,j,k) = \min(\dist(i,j,k-1), \dist(i,k,k-1) + \dist(k,j,k-1))
  \]

  \[
	\# k's = 1 \implies \dist(i,k,k-1)
  \]
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Routing table}
  \begin{exampleblock}{Routing table (Problem 6.25)}
	Contruct routing table and extract shortest paths from it.
  \end{exampleblock}

  \begin{columns}
	\column{0.50\textwidth}
	  \begin{algorithmic}
		\State Init: $\text{Go}(i,j) \gets \text{Null}$
		\Statex
		\State $\forall (i,j) \in E: \text{Go}(i,j) \gets j$
		\Statex
		\If{$\dots$}
		  \State $\text{Go}(i,j) \gets \text{Go}(i,k)$
		\EndIf
	  \end{algorithmic}
	\column{0.50\textwidth}
	  \begin{algorithmic}
		\If{$Go(i,j) = \text{Null}$}
		  \State $\dots$
		\EndIf
		\Statex
		\While{$i \neq j$}
		  \State $i \gets \text{Go}(i,j)$
		\EndWhile
	  \end{algorithmic}
  \end{columns}

  \vspace{0.50cm}
  \[
	\text{Prev}(i,j) \gets \text{Prev}(k,j)
  \]
  \[
	\text{Intermediate}(i,j) \gets k
  \]
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Shortest cycle in digraph}
  \begin{exampleblock}{Shortest cycle in digraph}
	Find shortest cycle in digraph $G = (V, E), w(e) > 0$.
  \end{exampleblock}

  \vspace{0.50cm}
  \centerline{Initialize $\dist[v][v] \gets \infty$ in Floyd-Warshall algorithm}

  \[
	\exists v: \dist[v][v] < 0 \text{\emph { vs. }} \forall v: \dist[v][v] = \infty
  \]
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Max-min path problem}
  \begin{exampleblock}{Max-min path problem (Problem 6.26)}
	\begin{itemize}
	  \item $G = (V, E)$: network of oil pipelines
	  \item $c(u,v)$: capacity of $(u,v)$
	  \item (2) Compute all-pair $\text{cap}(u,v)$.
	\end{itemize}
  \end{exampleblock}

  \[
	\text{cap}(u,v,k) = \max(\text{cap}(u,v,k-1), \min(\text{cap}(u,k,k-1), \text{cap}(k,v,k-1)))
  \]
\end{frame}
%%%%%%%%%%%%%%%%%%%%


\documentclass[a4paper]{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{textcomp}
\usepackage[english]{babel}
\usepackage{amsmath, amssymb}
\usepackage[]{amsthm} %lets us use \begin{proof}
\usepackage[]{amssymb} %gives us the character \varnothing

% figure support
\usepackage{import}
\usepackage{xifthen}
\pdfminorversion=7
\usepackage{pdfpages}
\usepackage{transparent}
\newcommand{\incfig}[1]{%
	\def\svgwidth{\columnwidth}
	\import{./figures/}{#1.pdf_tex}
}

\pdfsuppresswarningpagegroup=1

\begin{document}

\section*{Section 2.3}

\subsection*{Exercise 2.3.1}

Prove Lemma 2.3.2. (Hint: modify the proofs of Lemmas 2.2.2, 2.2.3 and Proposition 2.2.4.)

\begin{proof}

$ $\newline

We need to show $m \times 0 = m$ and $n \times \left(  m\text{++} \right) = \left( n\times m \right) + n$ before proving the multiplication is commutative.

To show $m\times 0 = m$ we use induction. For the base case $0\times 0 = 0$ follows since we know that $0\times m = m$ by definition. Now suppose inductively that $m\times 0 = m$, we need to show $\left( m\text{++} \right) \times 0 = 0$. By definition $\left( m\text{++} \right) \times 0 = \left( m\times 0 \right) + 0 = 0$, hence the induction is closed.

To show $n \times \left(  m\text{++} \right) = \left( n\times m \right) + n$, we induct on  $n$. For the base case $0 \times \left( m\text{++} \right) = \left( 0 \times m \right) + 0$ follows by the definitions of addition and multiplication. Now suppose inductively $n \times \left(  m\text{++} \right) = \left( n\times m \right) + n$, we need to show $n\text{++} \times \left(  m\text{++} \right) = \left( n\text{++}\times m \right) + n\text{++}$. The left-hand side is $n\times \left( m\text{++} \right) + \left( m\text{++} \right) = n\times m + n + m + 1$ by definition of multiplication and the hypothesis. The right-hand side is $\left( n\times m + m \right) + n\text{++} = n\times m + n + m + 1$ by definition of multiplication. Thus both sides are equal to each other, and we have closed the induction.

To show multiplication is commutative, we induct on $n$. For the base case $0\times m = m\times 0$ follows since both sides euqal to zero by definition of multiplication and $m\times 0 = 0$. Now suppose inductively that $n \times m = m\times n$, we need to show $n\text{++}\times m = m\times n\text{++}$. The left-hand side is $n\times m + m$ by definition. The right-hand side is $m\times n + m$ by the lemma we've just proved. And by the hypothesis the right-hand then equals to $n\times m + m$. Thus both sides are equal to each other, and we have closed the induction.

\end{proof}

\subsection*{Exercise 2.3.2}

Prove Lemma 2.3.3. (Hint: prove the second statement first.)

\begin{proof}

$ $\newline

The statement is equivalent to "$n\times m$ is positive iff both $n$ and  $m$ are positive".

First we need to show $n\times m$ is positive implies both $n$ and  $m$ are positive. For the sake of contradiction that $n$ equals to zero, by definition of multiplication  $0\times m = 0$, a contradiction. Similar contradiction holds for $m$ with Lemma 2.3.2. Thus we have proved the statement.

Then we need to show both $n$ and  $m$ are positive implies $n\times m$ is positive. For the sake of contradiction that $n\times m = 0$, by Lemma 2.2.10 there exists extractly one natural nunmber $a$ such that  $a\text{++} = n$, thus by definition of multiplication we have $n\times m = \left( a\text{++} \right) \times m = \left( a\times m \right) \text{++}$. Since $a\times m$ is a natural number, by Axiom 2.3 $\left( a\times m \right) \text{++} \neq 0$, a contradiction. Thus $n\times m$ is positive.

Thus we have proved the original statement.

\end{proof}

\subsection*{Exercise 2.3.3}

Prove Proposition 2.3.5. (Hint: modify the proof of Proposition
2.2.5 and use the distributive law.)

\begin{proof}

	We use induction on $a$. For the base case $\left( 0\times b \right) \times c = 0 \times \left( b\times c \right) $ follows, since the left-hand side equals to $0\times c = 0$ by definition of multiplication, and the right-hand side equals to $0$ since $b\times c$ is a natural number and $0$ times a natural number equals to $0$. Now suppose inductively $\left( a\times b \right) \times c = a\times \left( b\times c \right) $, we need to show $\left( a\text{++}\times b \right) \times c = a\text{++}\times \left( b\times c \right) $. The left-hand side equals to $\left( a\times b + b \right) \times c$ by definition of multiplication, then equals to $\left( a\times b \right) \times c + b\times c$ by the distributive law. The right-hand side equals to $a\times \left( b\times c \right) + b\times c$ by definition of multiplication. By the hypothesis both sides are equal to each other, thus we have closed the induction.

\end{proof}

\subsection*{Exercise 2.3.4}

Prove the identity $\left( a + b \right) ^2 = a^2 + 2ab + b^2$ for all natural
numbers $a$, $b$.

\begin{proof}

	$\left( a+b \right) ^2 = \left( a+b \right) ^1 \left( a + b \right) = \left( a + b \right) \left( a + b \right) $ by definition of exponentiation. Thus equals to  $a\times \left( a + b \right) + b\times \left( a+b \right) = a\times a + a\times b + b\times a + b\times b$ by the distributive law. $a\times a = a^2$ and $b\times b = b^2$ by definition of exponentiation. $a\times b + b\times a = a\times b + a\times b = 2ab$ since multiplication is commutative and the definition of multiplication. Thus we have proved the statement.

\end{proof}

\subsection*{Exercise 2.3.5}

Prove Proposition 2.3.9. (Hint: fix $q$ and induct on $n$.)

\begin{proof}

	We use induction on $n$ (keep $q$ fixed). For the base case there exists natural numbers $m = 0$, $r = 0$ such that $0 \le r < q$ and $0 = mq + r$. Now suppose inductively there exists $m$, $r$ such that $0 \le r < q$ and $n = mq + r$, we need to show there exists $m'$,  $r'$ such that  $0 \le r' < q$ and $n = m'q + r'$. Thus $r\text{++} \le q$ by Proposition 2.2.12 (e). If $r\text{++} < q$, we can simply set $m' = m$ and  $r' = r + 1$. Otherwise if  $r\text{++} = q$, then $n\text{++} = mq + q = \left( m\text{++} \right) \times q + 0$, thus we can set $m' = m\text{++}$ and $r' = 0$. Thus we have close the induction.

\end{proof}

\end{document}


\chapter{How to program a proof tool}\label{tool}

Users of \HOL{} can create their own theorem proving tools by combining
predefined rules and tactics. The \ML{} type-discipline
ensures that only logically sound methods can be used to create values
of type \ml{thm}.
In this chapter, a simple but real\footnote{The example
is `real' in that the need for it came up last week.} example is described.

Several implementations of the tool are given to illustrate various styles
of proof programming. The first implementation is the obvious one, but
is very slow because of the `brute force' method used. The second
implementation produces a much more streamlined proof, but still has a
brute force component, namely the use of a tautology checker from the
library \ml{taut}. The third implementation replaces the general
tautology checker with a special purpose derived inference rule. The
fourth and final implementation uses an optimised implementation of
the special purpose rule; understanding it is left as an exercise in
using \DESCRIPTION.

The timings in this chapter are based on Version 1.12. Later versions
of \HOL{} have an optimised tautology checker library due to Richard
Boulton. This new tautology checker is actually faster than the
special purpose derived rule described in
Section~\ref{bogus-optimization}.  Thus with the new tautology checker
the so called ``even more efficient implementation'' is actually
slower than the program it replaces! This was only discovered (by
Juanito Camilleri) during the preparation of Version 2 of the
tutorial. Rather than completely rewriting the chapter, it was decided
to leave it essentially as it was (except for the addition of this
 paragraph). The methods
that are described are still useful, and there is an important lesson
here: optimizations can become obsolete.  The really dedicated reader
could learn a lot by studying the old and new tautology checker
({\small\verb%contrib/icl-taut%} and
{\small\verb%Library/taut%}, respectively) to find out how they work.
Besides improving the tautology library, Richard Boulton also
reimplemented rewriting using ideas from Tom Melham and Roger Fleming.
As a result, in versions later than 1.12 the various rewriting tools
are quite a bit faster and generate fewer intermediate theorems.

It is sometimes claimed that `\LCF-style' systems can never be
practical, because the efficiency needed to handle real examples can
only be obtained with decision procedures coded as primitive rules. It
is hoped that this chapter, as well as the \ml{taut} library, shows
that the truth of such claims is not obvious. Research is currently in
progress to see if a variety of practical decision algorithms can be
implemented as efficient derived rules.

The tool described here is a tactic that puts conjunctions into the
normal form obtained by right associating, sorting the conjuncts into
a canonical order and then removing repetitions. This canonical order
uses the built-in polymorphic infix \ml{<<}, which orders any pair of
\ML{} values with the same type.

\section{A simple implementation}

A first implementation uses `brute-force' rewriting with
the equations:

\begin{hol}\begin{verbatim}
   |- (t1 /\ t2) /\ t3 = t1 /\ (t2 /\ t3)     % Associativity          %

   |- t1 /\ t2 = t2 /\ t1                     % Symmetry (if t2 << t1) %
   |- t1 /\ (t2 /\ t3) = t2 /\ (t1 /\ t3)     % Symmetry (if t2 << t1) %

   |- t /\ t = t                              % Cancel repeated terms  %
   |- t1 /\ (t1 /\ t2) = t1 /\ t2             % Cancel repeated terms  %
\end{verbatim}\end{hol}

\noindent These equations are easily proved using the
library \ml{taut}. Note that \HOL{} Version 1.12 is used in
this chapter. Versions of \HOL{} later than 1.12 contain improved
rewriting tools and a new version of the library \ml{taut} (the old version
of the library is preserved in the directory
{\small\verb%contrib/icl-taut%}).

\setcounter{sessioncount}{0}
\begin{session}\begin{verbatim}
scaup% hol
          _  _    __    _      __    __
   |___   |__|   |  |   |     |__|  |__|
   |      |  |   |__|   |__   |__|  |__|

          Version 1.12 (Sun3/Franz), built on Feb 23 1991

#load_library `taut`;;
Loading library `taut` ...
........................
Library `taut` loaded.
() : void
\end{verbatim}\end{session}

\noindent The library \ml{taut} defines \ml{TAUT\_RULE}\footnote{The function \ml{TAUT\_RULE} has been replaced by a function called \ml{TAUT\_PROVE}
in the new version of the \ml{taut} library available in versions of
\HOL{} later than 1.12}
which converts a term to the corresponding theorem, if the term is a tautology.
\vfill
\newpage
\begin{session}\begin{verbatim}
#let ASSOC = TAUT_RULE "(t1 /\ t2) /\ t3 = t1 /\ t2 /\ t3";;
ASSOC = |- (t1 /\ t2) /\ t3 = t1 /\ t2 /\ t3

#let SYM1 = TAUT_RULE "t1 /\ t2 = t2 /\ t1";;
SYM1 = |- t1 /\ t2 = t2 /\ t1

#let SYM2 = TAUT_RULE "t1 /\ t2 /\ t3 = t2 /\ t1 /\ t3";;
SYM2 = |- t1 /\ t2 /\ t3 = t2 /\ t1 /\ t3

#let CANCEL1 = TAUT_RULE "t /\ t = t";;
CANCEL1 = |- t /\ t = t

#let CANCEL2 = TAUT_RULE "t1 /\ t1 /\ t2 = t1 /\ t2";;
CANCEL2 = |- t1 /\ t1 /\ t2 = t1 /\ t2
\end{verbatim}\end{session}

\noindent One cannot just use \ml{REWRITE\_TAC} with \ml{SYM1} and
\ml{SYM2}, because it would loop.  What is needed is a special
rewriting tool that will only apply symmetry when terms are out of
order. Such a tool can be implemented as a {\it conversion\/}.

Conversions are described in detail in \DESCRIPTION. The idea, which
is due to Larry Paulson \cite{lcp_rewrite}, is that a conversion is an
\ML{} function that maps a term $t_1$ to an equation:

\medskip
{\small\verb%|- %}$t_1${\small\verb% = %}$t_2$.
\medskip

\noindent The intention is that a conversion will only apply to a
subset of terms: on members of this subset it will generate an
equation, on all other terms it will fail. Because conversions are so
central to theorem-proving in \HOL, the \ML{} type
{\small\verb%term->thm%} is abbreviated to {\small\verb%conv%}.
Conversions are applied using the function:

\begin{hol}\begin{verbatim}
   REWR_CONV : thm -> conv
\end{verbatim}\end{hol}

\noindent This takes an equation {\small\verb%|- %}$t_1${\small\verb% = %}$t_2$
and generates a conversion (\ie\ \ML{} function of type {\small\verb%term->thm%})
that maps any term $u$ that matches $t_1$ to the theorem
{\small\verb%|- %}$u${\small\verb% = %}$v$, where $v$ is
obtained by applying the substitution obtained by matching $u$ with $t_1$ to $t_2$.
If $u$ doesn't match $t_1$ then the application of \ml{REWR\_CONV} fails.

\begin{session}\begin{verbatim}
#REWR_CONV ASSOC "(A /\ B) /\ C";;
|- (A /\ B) /\ C = A /\ B /\ C

#REWR_CONV ASSOC "A /\ (B /\ C)";;
evaluation failed     REWR_CONV: lhs of theorem doesn't match term

#REWR_CONV SYM1 "B /\ A";;
|- B /\ A = A /\ B

#REWR_CONV SYM1 "A \/ B";;
evaluation failed     REWR_CONV: lhs of theorem doesn't match term
\end{verbatim}\end{session}

\noindent For our application, the required conversion should map
a conjunction

\medskip
$t_1${\small\verb% /\ (%}$t_{2_1}${\small\verb% /\ %}$t_{2_2}${\small\verb%)%}
\medskip

\noindent in which
$t_{2_1}${\small\verb% << %}$t_1$ to the equational theorem:

\medskip

{\small\verb%|- %}$t_1${\small\verb% /\ (%}$t_{2_1}${\small\verb% /\ %}$t_{2_2}${\small\verb%)  =  %} $t_{2_1}${\small\verb% /\ (%}$t_1${\small\verb% /\ %}$t_{2_2}${\small\verb%)%}

\medskip

\noindent If $t_1${\small\verb% << %}$t_{2_1}$ then the conversion fails
(in the \ML{} sense) when applied to
$t_1${\small\verb% /\ (%}$t_{2_1}${\small\verb% /\ %}$t_{2_2}${\small\verb%)%}.
In addition, if the right conjunct is not itself a conjunction, then
the conversion should reorder if necessary. More precisely, if the conversion
is applied to $t_1${\small\verb% /\ %}$t_2$ where $t_2$ is not a conjunction and
$t_2${\small\verb% << %}$t_1$, then it should generate the equation:

\medskip
{\small\verb%|- %}$t_1${\small\verb% /\ %}$t_2${\small\verb%  =  %}$t_2${\small\verb% /\ %}$t_1$
\medskip

\noindent Such a conversion is easily implemented in \ML{} using
\ml{SYM1} and \ml{SYM2} proved above, together with the \ML{} syntax
processing functions \ml{is\_conj} and \ml{dest\_conj}, where:

\begin{hol}\begin{verbatim}
   is_conj   : term -> bool
   dest_conj : term -> (term # term)
\end{verbatim}\end{hol}

\noindent These are functions that test whether a term is a conjunction, and
splits a term into its two conjuncts, respectively. For example:

\begin{session}\begin{verbatim}
#is_conj "A /\ B";;
true : bool

#is_conj "A \/ B";;
false : bool

#dest_conj "A /\ B";;
("A", "B") : (term # term)

#dest_conj "A \/ B";;
evaluation failed     dest_conj
\end{verbatim}\end{session}

The implementation of the special purpose conversion,
\ml{CONJ\_ORD\_CONV}, is now straightforward.

\begin{session}\begin{verbatim}
#let CONJ_ORD_CONV t =
# let t1,t2 = dest_conj t
# in
# if is_conj t2
#   then (let t21,t22 = dest_conj t2
#         in
#         if t21 << t1 then REWR_CONV SYM2 t else fail)
#   else (if t2  << t1 then REWR_CONV SYM1 t else fail);;
CONJ_ORD_CONV = - : conv
\end{verbatim}\end{session}

\noindent This is illustrated by:

\begin{session}\begin{verbatim}
#"A:bool" << "B:bool";;
true : bool

#"B:bool" << "C:bool";;
true : bool

#CONJ_ORD_CONV "B /\ A";;
|- B /\ A = A /\ B

#CONJ_ORD_CONV "A /\ B";;
evaluation failed     fail
\end{verbatim}\end{session}

The process of normalizing a conjunction can be split into four phases:

\begin{enumerate}
\item Right associate the conjunction by repeatedly applying:
\begin{quote}
\ml{REWR\_CONV\ ASSOC}
\end{quote}
\item Put the conjuncts in canonical order by repeatedly applying:
\begin{quote}
\ml{CONJ\_ORD\_CONV}
\end{quote}
\item Remove repetitions of $t$ of the form $t${\small\verb% /\ %}$t$
by repeatedly applying:
\begin{quote}
\ml{REWR\_CONV\ CANCEL1}
\end{quote}
\item Remove repetitions of $t_1$ in
$t_1${\small\verb% /\ (%}$t_1${\small\verb% /\ %}$t_2${\small\verb%)%}
by repeatedly applying:
\begin{quote}
\ml{REWR\_CONV\ CANCEL2}
\end{quote}
\end{enumerate}

To implement this, a method of repeatedly applying a conversion to
subterms of a term is needed. This is provided by the operator

\begin{hol}\begin{verbatim}
   TOP_DEPTH_CONV : conv -> conv
\end{verbatim}\end{hol}

\noindent If $c$ is a conversion then \ml{TOP\_DEPTH\_CONV}~$c$ is a
conversion that repeatedly applies $c$ to all subterms until $c$ is no
longer applicable to any subterms. The function \ml{TOP\_DEPTH\_CONV}
is one of a family of operators that apply conversions throughout
terms. Members of this family differ in the order in which subterms
are visited and the amount of repetition that is done. For more
details, see the chapter on conversions in \DESCRIPTION.

\begin{session}\begin{verbatim}
#let ex1 = "A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D";;
ex1 = "A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D" : term

#REWR_CONV ASSOC ex1;;
evaluation failed     REWR_CONV: lhs of theorem doesn't match term

#TOP_DEPTH_CONV (REWR_CONV ASSOC) ex1;;
|- A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D =
   A /\ B /\ C /\ A /\ C /\ A /\ D /\ D
\end{verbatim}\end{session}

\noindent The right hand side of this theorem is \ml{ex1} in right-associated
form. The conclusion of a theorem can be extracted with the \ML{} function
\ml{concl} and the right hand side of an equation can be extracted with
\ml{rhs}. Thus, continuing the session:

\begin{session}\begin{verbatim}
#let ex2 = rhs(concl it);;
ex2 = "A /\ B /\ C /\ A /\ C /\ A /\ D /\ D" : term

#TOP_DEPTH_CONV CONJ_ORD_CONV ex2;;
|- A /\ B /\ C /\ A /\ C /\ A /\ D /\ D =
   A /\ A /\ A /\ B /\ C /\ C /\ D /\ D
\end{verbatim}\end{session}

\noindent The right hand side of this is the result of canonicalizing
the order of the conjuncts in the left hand side. Next, the repetitions
can be eliminated using \ml{CANCEL1} and \ml{CANCEL2}.

\begin{session}\begin{verbatim}
#let ex3 = rhs(concl it);;
ex3 = "A /\ A /\ A /\ B /\ C /\ C /\ D /\ D" : term

#TOP_DEPTH_CONV (REWR_CONV CANCEL1) ex3;;
|- A /\ A /\ A /\ B /\ C /\ C /\ D /\ D =
   A /\ A /\ A /\ B /\ C /\ C /\ D
\end{verbatim}\end{session}

\begin{session}\begin{verbatim}
#let ex4 = rhs(concl it);;
ex4 = "A /\ A /\ A /\ B /\ C /\ C /\ D" : term

#TOP_DEPTH_CONV (REWR_CONV CANCEL2) ex4;;
|- A /\ A /\ A /\ B /\ C /\ C /\ D = A /\ B /\ C /\ D
\end{verbatim}\end{session}

To make the conjunction normalizer, the four stages just described
must be performed in sequence. Conversions can be applied in sequence using
the infixed function:

\begin{hol}\begin{verbatim}
   THENC : conv -> conv -> conv
\end{verbatim}\end{hol}

\noindent If $c_1\ t_1$ evaluates to $\ml{ |- }t_1\ml{=}t_2$ and
$c_2\ t_2$ evaluates to $\ml{ |- }t_2\ml{=}t_3$, then
$\ml{(}c_1\ \ml{THENC}\ c_2\ml{)}\ t_1$ evaluates to
$\ml{\ |-\ }t_1\ml{=}t_3$. If the
evaluation of $c_1\ t_1$ or the evaluation of $c_2\ t_2$ fails,
then so does the evaluation of $c_1\ \ml{THENC}\ c_2$. \ml{THENC} is
justified by the transitivity of equality.

Using \ml{THENC}, the normalizer is defined by

\begin{session}\begin{verbatim}
#let CONJ_NORM_CONV =
# TOP_DEPTH_CONV(REWR_CONV ASSOC)   THENC
# TOP_DEPTH_CONV CONJ_ORD_CONV         THENC
# TOP_DEPTH_CONV(REWR_CONV CANCEL1) THENC
# TOP_DEPTH_CONV(REWR_CONV CANCEL2);;

CONJ_NORM_CONV = - : conv

#CONJ_NORM_CONV ex1;;
|- A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D = A /\ B /\ C /\ D
\end{verbatim}\end{session}

This conversion can now be converted to a rule or tactic using the functions
\ml{CONV\_RULE} or \ml{CONV\_TAC}, respectively.

\begin{hol}
\begin{verbatim}
   CONV_RULE : conv -> thm -> thm
   CONV_TAC  : conv -> tactic
\end{verbatim}
\end{hol}

\noindent $\ml{CONV\_RULE}\ c\ \ml{(|- }t\ml{)}$ returns $\ml{|- }t'$, where
$c\ t$ evaluates to the equation
$\ml{|-}\ t\ml{=}t'$.
$\ml{CONV\_TAC}\ c$ is a tactic that
converts the conclusion of a goal using $c$. For more details see \DESCRIPTION.

\begin{session}\begin{verbatim}
#let CONJ_NORM_TAC = CONV_TAC CONJ_NORM_CONV;;
CONJ_NORM_TAC = - : tactic
\end{verbatim}\end{session}

Here is an example. It uses {\it antiquotation\/}: if $x$ is an \ML\
indentifier bound to term, then occurrences of
{\small\verb%^%}$x$ inside a quotation
denotes the term bound to $x$.

\begin{session}\begin{verbatim}
#g "^ex1 ==> B";;
"A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D ==> B"

() : void

#e CONJ_NORM_TAC;;
OK..
"A /\ B /\ C /\ D ==> B"
\end{verbatim}\end{session}

To summarize, here is the \ML{} code implementing the normalizer:

\begin{hol}\begin{verbatim}
   load_library `taut`;;

   let ASSOC   = TAUT_RULE "(t1 /\ t2) /\ t3 = t1 /\ t2 /\ t3"
   and SYM1    = TAUT_RULE "t1 /\ t2 = t2 /\ t1"
   and SYM2    = TAUT_RULE "t1 /\ t2 /\ t3 = t2 /\ t1 /\ t3"
   and CANCEL1 = TAUT_RULE "t /\ t = t"
   and CANCEL2 = TAUT_RULE "t1 /\ t1 /\ t2 = t1 /\ t2";;

   let CONJ_ORD_CONV t =
    let t1,t2 = dest_conj t
    in
    if is_conj t2
      then (let t21,t22 = dest_conj t2
            in
            if t21 << t1 then REWR_CONV SYM2 t else fail)
      else (if t2  << t1 then REWR_CONV SYM1 t else fail);;

   let CONJ_NORM_CONV =
    TOP_DEPTH_CONV(REWR_CONV ASSOC)   THENC
    TOP_DEPTH_CONV CONJ_ORD_CONV         THENC
    TOP_DEPTH_CONV(REWR_CONV CANCEL1) THENC
    TOP_DEPTH_CONV(REWR_CONV CANCEL2);;

   let CONJ_NORM_TAC = CONV_TAC CONJ_NORM_CONV;;
\end{verbatim}\end{hol}

\section{A more efficient implementation}

The normalizer just given is rather slow. This can be shown by switching on the
system timer using the function:

\begin{hol}\begin{verbatim}
   timer : bool -> bool
\end{verbatim}\end{hol}

\noindent Evaluating \ml{timer~true} switches on timing; evaluating
\ml{timer~false} switches it off (the previous value of the timing flag
is returned). Garbage collection times are also shown, together with a
count of the number of intermediate theorems that are generated (which
gives an estimate of the number of primitive inferences done).

\begin{session}\begin{verbatim}
#timer true;;
false : bool
Run time: 0.0s

#CONJ_NORM_CONV ex1;;
|- A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D = A /\ B /\ C /\ D
Run time: 1.1s
Garbage collection time: 0.5s
Intermediate theorems generated: 73
\end{verbatim}\end{session}

\noindent Here is a bigger example:

\begin{session}\begin{verbatim}
#CONJ_NORM_CONV "^ex1 /\ (^ex1 /\ (^ex1 /\ ^ex1 /\ ^ex1) /\ ^ex1)";;
|- (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
   (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
   ((A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
    (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
    A /\
    (B /\ C /\ A) /\
    (C /\ A /\ D) /\
    D) /\
   A /\
   (B /\ C /\ A) /\
   (C /\ A /\ D) /\
   D =
   A /\ B /\ C /\ D
Run time: 38.3s
Garbage collection time: 11.5s
Intermediate theorems generated: 16761
\end{verbatim}\end{session}

The reason that \ml{CONJ\_CANON\_CONV} is slow is because of the
repeated pattern matching done during rewriting. A much more efficient
approach is to normalize the conjunction by \ML{} programming outside
the logic, and then to prove that the normalized term is equal to the
original one. An even more efficient approach, which is not explored
here, would be to avoid having to do this proof by verifing the
normalization code by some sort of meta-theoretic
reasoning about \ML. How to do this in
\HOL{} is not clear, but work on this approach has been done in the
context of {\small FOL} \cite{FOL}, the Boyer-Moore prover \cite{BoyerMoore}
and Nuprl \cite{Nuprl}. These approaches all use logically
sophisticated extra axioms, called reflection principles, that enable
metatheorems to be `reflected' into the logic as object level
theorems.

To normalize the term by \ML{} programming, the conjuncts are extracted,
repeated elements are deleted and the resulting list is sorted.

\HOL{} already has a predefined function:

\begin{hol}\begin{verbatim}
   conjuncts : term -> term list
\end{verbatim}\end{hol}

\noindent for extracting conjuncts.
\HOL{} also has a predefined \ML{} function for removing repeated elements of
a list:

\begin{hol}\begin{verbatim}
   setify : * list -> * list
\end{verbatim}\end{hol}

\noindent Both \ml{conjuncts} and \ml{setify} are illustrated below:

\begin{session}\begin{verbatim}
#timer false;;
true : bool

#conjuncts ex1;;
["A"; "B"; "C"; "A"; "C"; "A"; "D"; "D"] : term list

#setify it;;
["B"; "C"; "A"; "D"] : term list

#let ex1_list = it;;
ex1_list = ["B"; "C"; "A"; "D"] : term list
\end{verbatim}\end{session}

There is a predefined sorting function in \ML:

\begin{session}\begin{verbatim}
#sort;;
sort = - : (((* # *) -> bool) -> * list -> * list)

#sort $< [3;2;5;6;1;1;7;9;3];;
[1; 1; 2; 3; 3; 5; 6; 7; 9] : int list

#sort $<< ex1_list;;
["A"; "B"; "C"; "D"] : term list
\end{verbatim}\end{session}

Using this function, the list of conjuncts of the normalization of a
term is easily computed.  The predefined \ML{} function:

\begin{hol}\begin{verbatim}
   list_mk_conj : term list -> term
\end{verbatim}\end{hol}

\noindent can then be used to build the normalized conjunction.

\begin{session}\begin{verbatim}
#ex1;;
"A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D" : term

#let ex1_norm = list_mk_conj(sort $<< (setify(conjuncts ex1)));;
ex1_norm = "A /\ B /\ C /\ D" : term
\end{verbatim}\end{session}

\noindent The calculation of \ml{ex1\_norm} from \ml{ex1} has been
done by (unverified) \ML{} code. What is required is the theorem
asserting that they are equal. This can be proved using the tautology
checker.

\begin{session}\begin{verbatim}
#TAUT_RULE "^ex1 = ^ex1_norm";;
|- A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D = A /\ B /\ C /\ D
\end{verbatim}\end{session}

\noindent A conversion that normalizes conjunctions is thus:

\begin{session}\begin{verbatim}
#let CONJ_NORM_CONV2 t =
# if is_conj t
#  then TAUT_RULE "^t = ^(list_mk_conj(sort $<< (setify(conjuncts t))))"
#  else fail;;
CONJ_NORM_CONV2 = - : conv

#CONJ_NORM_CONV2 ex1;;
|- A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D = A /\ B /\ C /\ D
\end{verbatim}\end{session}

\noindent \ml{CONJ\_CANON\_CONV2} is more than an order of magnitude faster
than \ml{CONJ\_CANON\_CONV}:

\begin{session}\begin{verbatim}
#timer true;;
false : bool
Run time: 0.0s

#CONJ_NORM_CONV2 "^ex1 /\ (^ex1 /\ (^ex1 /\ ^ex1 /\ ^ex1) /\ ^ex1)";;
|- (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
   (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
   ((A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
    (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
    A /\
    (B /\ C /\ A) /\
    (C /\ A /\ D) /\
    D) /\
   A /\
   (B /\ C /\ A) /\
   (C /\ A /\ D) /\
   D =
   A /\ B /\ C /\ D
Run time: 1.9s
Garbage collection time: 0.5s
Intermediate theorems generated: 1273
\end{verbatim}\end{session}

\section{An even more efficient implementation}\label{bogus-optimization}

Although the implementation just given is much faster than the first
naive one, it can be improved further by replacing the call to the
general tautology checker with a special purpose conjunction-equivalence
prover.

To see how this works, the equivalence of \ml{ex1} and \ml{ex1\_norm} will first
be proved manually. The general form of this proof will then be abstracted into
a derived rule.

The goal is to prove that \ml{ex1} and \ml{ex1\_norm} are equal.

\begin{session}\begin{verbatim}
#timer false;;
true : bool

#g "^ex1 = ^ex1_norm";;
"A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D = A /\ B /\ C /\ D"

() : void
\end{verbatim}\end{session}

\noindent The predefined tactic \ml{EQ\_TAC} splits an equation into two
implications (see Section~\ref{EQTAC}).

\begin{session}\begin{verbatim}
#e EQ_TAC;;
OK..
2 subgoals
"A /\ B /\ C /\ D ==> A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D"

"A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D ==> A /\ B /\ C /\ D"

() : void
\end{verbatim}\end{session}

\noindent Each of these can be solved by:
\begin{enumerate}
\item moving the antecedent of the
implication to the assumption list (using \ml{DISCH\_TAC}, see
Section~\ref{DISCHTAC});
\item breaking up the remaining
goal (the consequent of the implication) into one subgoal per conjunct (using
\ml{CONJ\_TAC}, see Section~\ref{CONJTAC});
\item  solving each of these
conjuncts using the antecedent (which is now an assumption)
\end{enumerate}

\noindent Step 1--3 are now done interactively.

\begin{session}\begin{verbatim}
#e DISCH_TAC;;
OK..
"A /\ B /\ C /\ D"
    [ "A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D" ]

() : void
\end{verbatim}\end{session}

\noindent \ml{CONJ\_TAC} is repeated using the tactical \ml{REPEAT}
described in Section~\ref{THEN}.

\begin{session}\begin{verbatim}
#e (REPEAT CONJ_TAC);;
OK..
4 subgoals
"D"
    [ "A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D" ]

"C"
    [ "A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D" ]

"B"
    [ "A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D" ]

"A"
    [ "A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D" ]

() : void
\end{verbatim}\end{session}

\noindent The final step is to use the assumption
{\small\verb%"A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D"%} to solve each goal.
To do this, the assumption is grabbed using the tactical:

\begin{hol}\begin{verbatim}
   POP_ASSUM : (thm -> tactic) -> tactic
\end{verbatim}\end{hol}

\noindent Given a function \ml{$f$ : thm -> tactic}, the tactic
\ml{POP\_ASSUM}\ $f$ applies $f$ to the (assumed) first
assumption of a goal
and then applies the tactic created thereby to the original goal
minus its top assumption:

\begin{hol}\begin{alltt}
   POP_ASSUM \(f\) ([\(t\sb{1}\);\(\ldots\);\(t\sb{n}\)],\(t\)) = \(f\) (ASSUME \(t\sb{1}\)) ([\(t\sb{2}\);\(\ldots\);\(t\sb{n}\)],\(t\))
\end{alltt}\end{hol}

\noindent \ML{} functions such as $f$,
with type \ml{thm -> tactic} are abbreviated to \ml{thm\_tactic} (see
\DESCRIPTION\ for further details).

After grabbing the assumption, it is split into its individual conjunctions
using the predefined derived rule:

\begin{hol}\begin{verbatim}
   CONJUNCTS : thm -> thm list
\end{verbatim}\end{hol}

\noindent For example:

\begin{session}\begin{verbatim}
#CONJUNCTS(ASSUME "A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D");;
[. |- A; . |- B; . |- C; . |- A; . |- C; . |- A; . |- D; . |- D]
: thm list
\end{verbatim}\end{session}

\noindent Among the individual conjuncts is the goal, which can thus be
solved immediately using \ml{ACCEPT\_TAC} (see Section~\ref{ACCEPTTAC}).
The appropriate assumption can be chosen with the predefined tactical
\ml{MAP\_FIRST}, which
is characterized by:

\begin{hol}\begin{alltt}
   MAP_FIRST \(f\) [\(x\sb{1}\); \(\ldots\) ;\(x\sb{n}\)]  =  \(f\)(\(x\sb{1}\)) ORELSE \(\ldots\) ORELSE \(f\)(\(x\sb{n}\))
\end{alltt}\end{hol}

\noindent Returning to the proof: the final step is now performed by
popping the assumption and applying to it the function obtained by
composing \ml{CONJUNCTS} and \ml{MAP\_FIRST} using the \ML{} infixed
function composition operator \ml{o}
(where \ml{(}$f$~\ml{o}~$g$\ml{)}$x$~\ml{=}~$g$\ml{(}$f$\ml{(}$x$\ml{))}).

\begin{session}\begin{verbatim}
#e(POP_ASSUM(MAP_FIRST ACCEPT_TAC o CONJUNCTS));;
OK..
goal proved
. |- A

Previous subproof:
3 subgoals
"D"
    [ "A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D" ]

"C"
    [ "A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D" ]

"B"
    [ "A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D" ]

() : void
\end{verbatim}\end{session}

\noindent The remaining subgoals are solved identically. Stitching together
the tactics just used results in:

\begin{hol}\begin{verbatim}
   EQ_TAC          THEN
   DISCH_TAC       THEN
   REPEAT CONJ_TAC THEN
   POP_ASSUM(MAP_FIRST ACCEPT_TAC o CONJUNCTS)
\end{verbatim}\end{hol}

\noindent With this, the entire proof can be done in one step.

\begin{session}\begin{verbatim}
#g "^ex1 = ^ex1_norm";;
"A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D = A /\ B /\ C /\ D"

() : void

#e(EQ_TAC          THEN
#  DISCH_TAC       THEN
#  REPEAT CONJ_TAC THEN
#  POP_ASSUM(MAP_FIRST ACCEPT_TAC o CONJUNCTS));;
OK..
goal proved
|- A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D = A /\ B /\ C /\ D

Previous subproof:
goal proved
() : void
\end{verbatim}\end{session}

Using this tactic, a derived rule \ml{CONJ\_EQ} can be defined that proves
two conjunctions equal. This is what is needed to replace the call to
\ml{TAUT\_RULE}.
\ml{CONJ\_EQ} is defined with the predefined function:

\begin{hol}\begin{verbatim}
   PROVE : term # tactic -> theorem
\end{verbatim}\end{hol}

\noindent \ml{PROVE}\ml{(}$t$\ml{,}$T$\ml{)} applies the tactic $T$ to
the goal \ml{([],}$t$\ml{)}; if this goal is proved by $T$ then the
resulting justification is applied to \ml{[]} to obtain the theorem
\ml{|-}~$t$, which is returned. If $T$ does not solve the goal, then
the application of \ml{PROVE} fails. Using \ml{PROVE}, the definition
of \ml{CONJ\_EQ} is:

\begin{hol}\begin{verbatim}
   let CONJ_EQ t1 t2 =
    PROVE ("^t1 = ^t2",
           EQ_TAC          THEN
           DISCH_TAC       THEN
           REPEAT CONJ_TAC THEN
           POP_ASSUM(MAP_FIRST ACCEPT_TAC o CONJUNCTS))
\end{verbatim}\end{hol}

\noindent Replacing the call to \ml{TAUT\_RULE} in the definition
of \ml{CONJ\_NORM\_CONV2} results in:

\begin{hol}\begin{verbatim}
   let CONJ_NORM_CONV3 t =
    if is_conj t
     then CONJ_EQ t (list_mk_conj(sort $<< (setify(conjuncts t))))
     else fail
\end{verbatim}\end{hol}

\noindent Continuing the session:

\begin{session}\begin{verbatim}
#let CONJ_EQ t1 t2 =
# PROVE ("^t1 = ^t2",
#        EQ_TAC          THEN
#        DISCH_TAC       THEN
#        REPEAT CONJ_TAC THEN
#        POP_ASSUM(MAP_FIRST ACCEPT_TAC o CONJUNCTS));;
CONJ_EQ = - : (term -> conv)

#let CONJ_NORM_CONV3 t =
# if is_conj t
#  then CONJ_EQ t (list_mk_conj(sort $<< (setify(conjuncts t))))
#  else fail;;
CONJ_NORM_CONV3 = - : conv
\end{verbatim}\end{session}

\noindent \ml{CONJ\_NORM\_CONV3} is almost twice
as efficient as \ml{CONJ\_NORM\_CONV2}. To show this, the timer is switched
back on.

\begin{session}\begin{verbatim}
#timer true;;
false : bool
Run time: 0.0s
\end{verbatim}\end{session}

\noindent Here is the big example with \ml{CONJ\_NORM\_CONV3}:

\begin{session}\begin{verbatim}
#CONJ_NORM_CONV3 "^ex1 /\ (^ex1 /\ (^ex1 /\ ^ex1 /\ ^ex1) /\ ^ex1)";;
|- (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
   (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
   ((A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
    (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
    A /\
    (B /\ C /\ A) /\
    (C /\ A /\ D) /\
    D) /\
   A /\
   (B /\ C /\ A) /\
   (C /\ A /\ D) /\
   D =
   A /\ B /\ C /\ D
Run time: 1.0s
Garbage collection time: 0.5s
Intermediate theorems generated: 775
\end{verbatim}\end{session}

\section{Further optimizations}

Further improvements are still possible. As an exercise the reader
might want to decipher the following highly optimized definition of
\ml{CONJ\_EQ}.

The function \ml{PROVE\_CONJ}, defined below,
converts a term $t$ to the theorem \ml{|-}~$t$
if that theorem occurs in a supplied list of theorems (\ml{ths} in the
code below), or $t$ is a conjunction each of whose conjuncts occurs in
the list. The definition of \ml{PROVE\_CONJ} uses the following
predefined \ML{} functions:

\begin{itemize}
\item \ml{uncurry}~$f$~\ml{(}$x$\ml{,}$y$\ml{)}~~\ml{=}~~$f$~$x$~$y$

\item \ml{(}$f${\small\verb% # %}$g$\ml{)}\ml{(}$x$\ml{,}$y$\ml{)}~~\ml{=}~~\ml{(}$f\ x$~\ml{,}~$g\ y$\ml{)}

\item \ml{find}~$p$~\ml{[}$x_1\ml{;}\ldots\ml{;}x_n$\ml{]}~~=~~{\it the first $x_i$ for which $p\ x_i$ is true\/}

\end{itemize}

\noindent and the inference rule \ml{CONJ}:

\[ \Gamma_1\turn
t_1\qquad\qquad\qquad\Gamma_2\turn t_2\over \Gamma_1\cup\Gamma_2 \turn t_1\conj
t_2 \]

\noindent Here is the definition of \ml{PROVE\_CONJ}:

\begin{hol}\begin{verbatim}
   letrec PROVE_CONJ ths tm =
    (uncurry CONJ ((PROVE_CONJ ths # PROVE_CONJ ths) (dest_conj tm))) ?
    find (\th. concl th = tm) ths
\end{verbatim}\end{hol}

\noindent Using this, the optimized \ml{CONJ\_EQ}, called
\ml{CONJ\_EQ2}, is defined using \ml{IMP\_ANTISYM\_RULE} (a predefined rule):

\[ \Gamma_1 \turn t_1 \imp t_2 \qquad\qquad \Gamma_2\turn t_2 \imp t_1\over
\Gamma_1 \cup \Gamma_2 \turn t_1 = t_2\]

\noindent The definition is:

\begin{hol}\begin{verbatim}
   let CONJ_EQ2 t1 t2 =
    let imp1 = DISCH t1 (PROVE_CONJ (CONJUNCTS(ASSUME t1)) t2)
    and imp2 = DISCH t2 (PROVE_CONJ (CONJUNCTS(ASSUME t2)) t1)
    in IMP_ANTISYM_RULE imp1 imp2
\end{verbatim}\end{hol}

\noindent Loading these \ML{} function definitions into \HOL:

\begin{session}\begin{verbatim}
#letrec PROVE_CONJ ths tm =
# (uncurry CONJ ((PROVE_CONJ ths # PROVE_CONJ ths) (dest_conj tm))) ?
# find (\th. concl th = tm) ths;;
PROVE_CONJ = - : (thm list -> conv)
Run time: 0.0s

#let CONJ_EQ2 t1 t2 =
# let imp1 = DISCH t1 (PROVE_CONJ (CONJUNCTS(ASSUME t1)) t2)
# and imp2 = DISCH t2 (PROVE_CONJ (CONJUNCTS(ASSUME t2)) t1)
# in IMP_ANTISYM_RULE imp1 imp2;;
CONJ_EQ2 = - : (term -> conv)
Run time: 0.0s
\end{verbatim}\end{session}

\noindent A version of \ml{CONJ\_NORM\_CONV} that
uses \ml{CONJ\_EQ2} is defined by:

\begin{hol}\begin{verbatim}
   let CONJ_NORM_CONV4 t =
    if is_conj t
     then CONJ_EQ2 t (list_mk_conj(sort $<< (setify(conjuncts t))))
     else fail
\end{verbatim}\end{hol}

\noindent Loading this into \ML:

\begin{session}\begin{verbatim}
#let CONJ_NORM_CONV4 t =
# if is_conj t
#  then CONJ_EQ2 t (list_mk_conj(sort $<< (setify(conjuncts t))))
#  else fail;;
CONJ_NORM_CONV4 = - : conv
Run time: 0.0s
\end{verbatim}\end{session}

\noindent This is even faster than \ml{CONJ\_NORM\_CONV3}:

\begin{session}\begin{verbatim}
#CONJ_NORM_CONV4 "^ex1 /\ (^ex1 /\ (^ex1 /\ ^ex1 /\ ^ex1) /\ ^ex1)";;
|- (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
   (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
   ((A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
    (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\
    A /\
    (B /\ C /\ A) /\
    (C /\ A /\ D) /\
    D) /\
   A /\
   (B /\ C /\ A) /\
   (C /\ A /\ D) /\
   D =
   A /\ B /\ C /\ D
Run time: 0.4s
Intermediate theorems generated: 155
\end{verbatim}\end{session}

\section{Normalizing all subterms}

There is an important difference in the functionality of
\ml{CONJ\_NORM\_CONV} and the various optimised versions of it. The
difference is that \ml{CONJ\_NORM\_CONV} applies to any term,
normalizing all subterms that are conjunctions. However the functions
\ml{CONJ\_NORM\_CONV}{\small $n$} (where {\small $n = 2,3,4$}) all
fail on non-conjunctions.

\begin{session}\begin{verbatim}
#CONJ_NORM_CONV "^ex1 ==> ^ex1";;
|- A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D ==>
   A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D =
   A /\ B /\ C /\ D ==> A /\ B /\ C /\ D
Run time: 2.0s
Garbage collection time: 0.6s
Intermediate theorems generated: 1307

#CONJ_NORM_CONV4 "^ex1 => ^ex1";;
need 2 nd branch to conditional
skipping: ex1 " ;; parse failed

#CONJ_NORM_CONV4 "^ex1 ==> ^ex1";;
evaluation failed     fail
\end{verbatim}\end{session}

What is needed is a function that will apply a conversion to all
conjunctive subterms of a term. Such a function is \ml{TOP\_DEPTH\_CONV}, however

\begin{hol}\begin{verbatim}
   TOP_DEPTH_CONV CONJ_NORM_CONV4
\end{verbatim}\end{hol}

\noindent would loop, because \ml{CONJ\_NORM\_CONV4} never fails on
conjunctions, so \ml{TOP\_DEPTH\_CONV} would keep applying it forever!
This is easily got around using:

\begin{hol}\begin{verbatim}
   CHANGED_CONV : conv -> conv
\end{verbatim}\end{hol}

\noindent \ml{CHANGED\_CONV}~$c$ behaves like $c$, except that if $c$
has no effect, then \ml{CHANGED\_CONV}~$c$ fails.

\begin{session}\begin{verbatim}
#TOP_DEPTH_CONV(CHANGED_CONV CONJ_NORM_CONV4) "^ex1 ==> ^ex1";;
|- A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D ==>
   A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D =
   A /\ B /\ C /\ D ==> A /\ B /\ C /\ D
Run time: 0.5s
Intermediate theorems generated: 292
\end{verbatim}\end{session}

Although this works, the scanning through subterms done by
\ml{TOP\_DEPTH\_CONV} is a bit `brute force'. Further efficiency can
be obtained by writing a special scanning function that
just applies a conversion to maximal conjunctive subterms.
This is provided by the code below.
The understanding of this is a fairly hard exercise for the reader. The
section on conversions in \DESCRIPTION\ should be helpful.

First, an auxiliary derived rule for combining two equations into
a single equation by conjoining  the left hand sides and  the
right hand sides.

\begin{session}\begin{verbatim}
#let MK_CONJ th1 th2 = MK_COMB(AP_TERM "$/\" th1, th2);;
MK_CONJ = - : (thm -> thm -> thm)
Run time: 0.0s
\end{verbatim}\end{session}

\noindent Next a function that conjoins the left hand sides and right
hand sides of lists of equations.

\begin{session}\begin{verbatim}
#letrec MK_CONJL l =
# if null l     then fail
# if null(tl l) then hd l
#               else MK_CONJ (hd l) (MK_CONJL(tl l));;
MK_CONJL = - : proof
Run time: 0.0s
\end{verbatim}\end{session}

\noindent Next, a function that applies a conversion $c$ to all
conjunctive subterms of a term. This uses the \ML{} function:

\bigskip
\ml{map}~$f$~\ml{[}$x_1\ml{;}\ldots\ml{;}x_n$\ml{]}~~=~~\ml{[}$f\ x_1\ml{;}\ldots\ml{;}f\ x_n$\ml{]}

\bigskip

\noindent and the rules \ml{MK\_COMB}:

\[ \Gamma_1 \turn f = g \qquad\qquad \Gamma_2\turn x = y \over
\Gamma_1 \cup \Gamma_2 \turn f\ x = g\ y\]

\noindent and \ml{MK\_ABS}:

\[ \Gamma \turn \forall x.\ t_1 = t_2 \over
\Gamma \turn (\lambda x.\ t_1) = (\lambda x.\ t_2)\]

\noindent and \ml{GEN}:

$$\Gamma\turn t\over\Gamma\turn\uquant{x} t$$
\begin{itemize}
\item Where $x$ is not free in $\Gamma$.
\end{itemize}

\noindent and \ml{REFL}:

$$ \over\turn t = t$$

\begin{itemize}
\item\ml{REFL}~$t$~~\ml{=}~~ $\turn t = t$.
\end{itemize}

\noindent The definition of \ml{CONJ\_DEPTH\_CONV} also uses:

\begin{hol}\begin{verbatim}
   is_comb   : term -> bool
   dest_comb : term -> (term # term)

   is_abs    : term -> bool
   dest_abs  : term -> (term # term)
\end{verbatim}\end{hol}

\noindent which are the tests and destructors for combinations and
abstractions, respectively.

\begin{session}\begin{verbatim}
#letrec CONJ_DEPTH_CONV c tm =
# if is_conj tm
#  then (c THENC (MK_CONJL o map (CONJ_DEPTH_CONV c) o conjuncts)) tm
# if is_comb tm
#  then (let rator,rand = dest_comb tm in
#         MK_COMB (CONJ_DEPTH_CONV c rator, CONJ_DEPTH_CONV c rand))
# if is_abs tm
#  then (let bv,body = dest_abs tm in
#         let bodyth = CONJ_DEPTH_CONV c body in
#         MK_ABS (GEN bv bodyth))
#   else (REFL tm);;
CONJ_DEPTH_CONV = - : (conv -> conv)
Run time: 0.0s
\end{verbatim}\end{session}

\noindent The next session shows that \ml{CONJ\_DEPTH\_CONV} is an
improvement over \ml{TOP\_DEPTH\_CONV}.

\begin{session}\begin{verbatim}
#CONJ_DEPTH_CONV CONJ_NORM_CONV4 "^ex1 ==> ^ex1";;
|- A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D ==>
   A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D =
   A /\ B /\ C /\ D ==> A /\ B /\ C /\ D
Run time: 0.4s
Intermediate theorems generated: 95
\end{verbatim}\end{session}

\noindent However, the figures show that we are getting to a point of
diminishing returns.

Finally, the tactic for normalizing all conjunctions in a goal is:

\begin{session}\begin{verbatim}
#let CONJ_NORM_TAC = CONV_TAC (CONJ_DEPTH_CONV CONJ_NORM_CONV4);;
CONJ_NORM_TAC = - : tactic
Run time: 0.0s
\end{verbatim}\end{session}

\noindent This is illustrated by:

\begin{session}\begin{verbatim}
#g "^ex1 ==> ^ex1 /\ ^ex1_norm";;
"A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D ==>
 (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\ A /\ B /\ C /\ D"

() : void
Run time: 0.1s

#e CONJ_NORM_TAC;;
OK..
"A /\ B /\ C /\ D ==> A /\ B /\ C /\ D"

() : void
Run time: 0.3s
Intermediate theorems generated: 110
\end{verbatim}\end{session}

\noindent To show how much faster the optimized version is, here is
the last step repeated with the first version of the tool. The \ML\
function \ml{b} backs up to the last goal.

\begin{session}\begin{verbatim}
#b();;
"A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D ==>
 (A /\ (B /\ C /\ A) /\ (C /\ A /\ D) /\ D) /\ A /\ B /\ C /\ D"

() : void
Run time: 0.1s
\end{verbatim}\end{session}

\noindent Expanding with the slow tactic:

\begin{session}\begin{verbatim}
#e(CONV_TAC CONJ_NORM_CONV);;
OK..
"A /\ B /\ C /\ D ==> A /\ B /\ C /\ D"

() : void
Run time: 3.5s
Garbage collection time: 1.0s
Intermediate theorems generated: 1932
\end{verbatim}\end{session}

\noindent it is 10 times slower and generates almost 20 times as many
primitive inference steps!

Here is the complete \ML{} program for the optimized normalizer.

\begin{hol}\begin{verbatim}
   letrec insert ord x l =
    if null l
     then [x]
    if ord(x,hd l)
     then x.l
     else hd l.(insert ord x (tl l));;

   letrec sort ord l =
    if null l
     then []
     else insert ord (hd l) (sort ord (tl l));;
\end{verbatim}\end{hol}

\begin{hol}\begin{verbatim}
   letrec PROVE_CONJ ths tm =
    (uncurry CONJ ((PROVE_CONJ ths # PROVE_CONJ ths) (dest_conj tm))) ?
    find (\th. concl th = tm) ths;;

   let CONJ_EQ t1 t2 =
    let imp1 = DISCH t1 (PROVE_CONJ (CONJUNCTS(ASSUME t1)) t2)
    and imp2 = DISCH t2 (PROVE_CONJ (CONJUNCTS(ASSUME t2)) t1)
    in IMP_ANTISYM_RULE imp1 imp2;;
\end{verbatim}\end{hol}

\begin{hol}\begin{verbatim}
   let CONJ_NORM_CONV t =
    if is_conj t
     then CONJ_EQ t (list_mk_conj(sort $<< (setify(conjuncts t))))
     else fail;;
\end{verbatim}\end{hol}

\begin{hol}\begin{verbatim}
   let MK_CONJ th1 th2 = MK_COMB(AP_TERM "$/\" th1, th2);;

   letrec MK_CONJL l =
    if null l     then fail
    if null(tl l) then hd l
                  else MK_CONJ (hd l) (MK_CONJL(tl l));;
\end{verbatim}\end{hol}

\begin{hol}\begin{verbatim}
   letrec CONJ_DEPTH_CONV c tm =
    if is_conj tm
     then (c THENC (MK_CONJL o map (CONJ_DEPTH_CONV c) o conjuncts)) tm
    if is_comb tm
     then (let rator,rand = dest_comb tm in
            MK_COMB (CONJ_DEPTH_CONV c rator, CONJ_DEPTH_CONV c rand))
    if is_abs tm
     then (let bv,body = dest_abs tm in
            let bodyth = CONJ_DEPTH_CONV c body in
            MK_ABS (GEN bv bodyth))
     else (REFL tm);;
\end{verbatim}\end{hol}

\begin{hol}\begin{verbatim}
   let CONJ_NORM_TAC = CONV_TAC (CONJ_DEPTH_CONV CONJ_NORM_CONV);;
\end{verbatim}\end{hol}

Although the optimized implementation is much more efficient, it uses
less general methods. An advantage of the simple implementation based
on rewriting is that essentially the same algorithm can be used to
normalize terms built out of any associative, commutative and
idenpotent operation. The two exercises that follow (whose solutions are not
supplied) suggest that the reader try to extract general principles from the
conjunction normalizer and use these to implement generic tools.

\subsection{Exercise 1}

Implement a normalizer for any associative and commutative operator.

\begin{hol}\begin{verbatim}
   AC_CANON_CONV : thm # thm -> conv
\end{verbatim}\end{hol}

\noindent The two theorem arguments should be the
associative and commutative laws for the operator. For example:

\begin{hol}\begin{verbatim}
   AC_CANON_CONV(ASSOC,SYM1)
\end{verbatim}\end{hol}

\noindent would be a canonicalizer for conjunctions.
Use the `brute force' rewriting method described at the beginning of this chapter.

\subsection{Exercise 2}

Implement an optimized canonicalizer:

\begin{hol}\begin{verbatim}
   FAST_AC_CANON_CONV : thm # thm -> conv \end{verbatim}\end{hol}
\noindent This should use tuned rewriting (\eg\ a generalization of
\ml{CONJ\_DEPTH\_CONV}) and be as fast as possible. Try to think up
tricks to minimise the amount of general matching and to make every
inference count.