\chapter{Schemes}

\section{BLISS Software Implementation}

\subsection{Introduction}
The emphasis (at this stage) of the library is to create a flexible software library on which various lattice-based cryptographic schemes can be developed and implemented to a level of production quality. It is a requirement that the library is highly portable and used as a research and development tool, therefore highly optimized inline assembly and extracting every cycle of performance are not the goals, but are possible. It is a requirement that confidence in \textit{libsafecrypto} is achieved by using thorough testing of the software (unit testing, functional testing and known-answer tests) together with a build system that integrates this testing into its build process. Also, where cryptographic primitives are required they will be drawn from well-established and well-respected implementations in the public domain.

The first algorithm to be implemented with the \textit{libsafecrypto} library is BLISS, which is a lattice-based digital signature scheme based on the Ring-LWE problem. The BLISS software implementation is originally based on the hilabliss \url{(https://github.com/mjosaarinen/hilabliss)} and BLZZRD (\url{https://github.com/mjosaarinen/blzzrd}) implementations created by Markku-Juhani O. Saarinen. This in turn is based on the BLISS-B variant of BLISS \url{(https://eprint.iacr.org/2014/874}) by Léo Ducas, derived from the original paper \url{(https://eprint.iacr.org/2013/383)}.

\subsection{Key Generation}

\begin{itemize}
\item $f$, $g$ are uniform random polynomials with a specific number of $\pm 1$ and $\pm 2$ coefficients
\item $g = 2g + 1$
\item $a = g / f$, if $f$ is not invertible then \textbf{Restart}.
\end{itemize}

In summary, the private key $(f, g)$ and public key $a$ are generated. In a practical implementation the following is also performed:

\begin{itemize}
\item To save cycles the public key is stored and distributed in the NTT domain. This saves 1 NTT in the signature and verification schemes. \textbf{What if someone finds something better than the NTT?}
\item The sign of $g$ must be inverted \textbf{(WHY???)}. From the hilabliss/BLZZRD original this is done by INTT, sign inversion, followed by NTT. The same can be achieved by simply performing the sign inversion in the NTT domain - giving an approx. 25\% boost in performance by saving a single NTT and INTT. I think this should be regarded as a bug fix rather than an optimisation
\item The two private keys are sparse polynomials and are ripe for compression, whereas the public key is randomly distributed across the range of the modulus and lossless compression tends to make it bigger. Note that the $g$ polynomial contains only the coefficients $\pm 2$ and $\pm 4$ with the exception of the first coefficient - this is exploited by all of the entropy coders.
\end{itemize}

\subsection{Signatures}

\begin{itemize}
\item Generate $y_1$ and $y_2$ using a Gaussian sampler.
\item $u = y_1 * a + y_2 \bmod 2q$
\item $c = H(\lfloor u \rceil_d \bmod p, \mu)$
\item Choose a random bit $b$
\item $z_1 = y_1 + (-1)^b s_1 c$
\item $z_2 = y_2 + (-1)^b s_2 c$
\item \textbf{Continue} if a randomly chosen value between 0.0 and 1.0 is greater than a rejection threshold computed using a parameter set factor $M$  and the scalar product of the polynomials $s$, $c$, $y_1$ and $y_2$. Otherwise \textbf{Restart}.
\item $z_2 = ( \lfloor u \rceil_d - \lfloor u - z_2 \rceil_d \bmod p$
\item \textbf{Output} $(z_1, z_2, c)$
\end{itemize}

The main bottlenecks in this operation are the NTT/INTT, the hash function \textit{H()} and of course the rejection sampling which causes the signature to retry with a probability associated with the parameter set variables (e.g. 1.6 with BLISS-B-IV). To overcome these performance issues the following measures have been taken:

\subsubsection{Hash Functions}

\begin{itemize}
\item We've provided a choice of hashing function's that offer different performance and security properties (SHA-2 \cite{sha2_gladman}, SHA-3 \cite{tiny_sha3}, Whirlpool \cite{whirlpool} and BLAKE2-B \cite{blake2}). Currently it is fixed to SHA3-512, but this can be overridden with the relevant flags passed to the \textit{create()} function. The security of the hash must be relative to the security of the signature scheme, therefore there is a possibility that we can use smaller hash blocks with lower security parameter sets rather than the 512-bit message digest that is used by default with all parameter sets.
\begin{itemize}
  \item BLAKE2-B and SHA2 offer much better performance than SHA3, with Whirlpool not far behind (see Figure \ref{fig:bliss_b_hash_comparison}).

\pgfplotsset{compat=1.13,width=14cm,height=12cm}
\begin{figure}[ht!]
\centering
\begin{tikzpicture}
\begin{axis}[xbar=0pt, xmin=0, ytick=data, enlarge y limits=0.08, symbolic y coords={SHA3-512, SHA3-384, SHA3-256, SHA3-224, SHA2-512, SHA2-384, SHA2-256, SHA2-224, BLAKE2-512, BLAKE2-384, BLAKE2-256, BLAKE2-224, WHIRLPOOL-512}, y tick label style={/pgf/number format/1000 sep=}, nodes near coords, nodes near coords align={horizontal}, legend style={at={(0.5,-0.15)},anchor=north,legend columns=-1}]
\addplot
  coordinates
    {(5053,SHA3-512) (5608,SHA3-384) (5176,SHA3-256) (5309,SHA3-224)
     (7517,SHA2-512) (7299,SHA2-384) (6467,SHA2-256) (6431,SHA2-224)
     (7790,BLAKE2-512) (7743,BLAKE2-384) (7368,BLAKE2-256) (7242,BLAKE2-224)
     (6815,WHIRLPOOL-512)};

\addplot
  coordinates
    {(14418,SHA3-512) (17687,SHA3-384) (15486,SHA3-256) (16075,SHA3-224)
     (30569,SHA2-512) (30540,SHA2-384) (23177,SHA2-256) (22621,SHA2-224)
     (33619,BLAKE2-512) (34045,BLAKE2-384) (29763,BLAKE2-256) (29366,BLAKE2-224)
     (24878,WHIRLPOOL-512)};
\legend{Signatures (per sec), Verifications (per sec)}
\end{axis}
\end{tikzpicture}
\caption{BLISS-B-IV performance with various hash functions (Intel i7 6700 CPU @ 3.4GHz)}
\label{fig:bliss_b_hash_comparison}
\end{figure}

  \item BLAKE2-B lost out to Keccak in the SHA3 NIST competition, but it is an RFC (7693, draft).

\end{itemize}
\item A standalone performance analysis of the various hashing functions integrated into \textit{libsafecrypto} is provided in Figure \ref{fig:bliss_b_bytes_per_sec}:
\end{itemize}

\pgfplotsset{compat=1.13,width=14cm,height=22cm}
\begin{figure}[ht!]
\centering
\begin{tikzpicture}
\begin{axis}[xbar=0pt, xmin=0, xmax=1000000000, ytick=data, enlarge y limits=0.05, symbolic y coords={SHA3-512, SHA3-384, SHA3-256, SHA3-224, SHA2-512, SHA2-384, SHA2-256, SHA2-224, BLAKE2-512, BLAKE2-384, BLAKE2-256, BLAKE2-224, WHIRLPOOL-512}, y tick label style={/pgf/number format/1000 sep=}]
\addplot
  coordinates
    {(41733600,SHA3-512) (66453490,SHA3-384) (33580860,SHA3-256) (26117498,SHA3-224) (90001193,SHA2-512) (111962015,SHA2-384) (137634227,SHA2-256) (133880238,SHA2-224) (318378956,BLAKE2-512) (245563664,BLAKE2-384) (143380969,BLAKE2-256) (341531003,BLAKE2-224) (63398434,WHIRLPOOL-512)};
\addplot
  coordinates
    {(67311617,SHA3-512) (47430787,SHA3-384) (60777654,SHA3-256) (39262358,SHA3-224) (169106491,SHA2-512) (176718677,SHA2-384) (246871771,SHA2-256) (250791574,SHA2-224) (577727014,BLAKE2-512) (582344548,BLAKE2-384) (589684613,BLAKE2-256) (591718794,BLAKE2-224) (131113251,WHIRLPOOL-512)};
\addplot
  coordinates
    {(81599679,SHA3-512) (76776751,SHA3-384) (62408720,SHA3-256) (43766790,SHA3-224) (217818471,SHA2-512) (214763352,SHA2-384) (335508930,SHA2-256) (336483310,SHA2-224) (718552357,BLAKE2-512) (729769587,BLAKE2-384) (738456684,BLAKE2-256) (732518714,BLAKE2-224) (149474568,WHIRLPOOL-512)};
\addplot
  coordinates
    {(87154415,SHA3-512) (81770554,SHA3-384) (36837864,SHA3-256) (44490799,SHA3-224) (231389170,SHA2-512) (230322287,SHA2-384) (372942248,SHA2-256) (373587750,SHA2-224) (771918396,BLAKE2-512) (748316921,BLAKE2-384) (788611685,BLAKE2-256) (783909562,BLAKE2-224) (158717452,WHIRLPOOL-512)};
\legend{128 bits, 512 bits, 2048 bits, 8192 bits}
\end{axis}
\end{tikzpicture}
\caption{Bytes per second performance of hash functions with varying message lengths [Intel i7 6700 CPU @ 3.4GHz]}
\label{fig:bliss_b_bytes_per_sec}
\end{figure}

\begin{itemize}
\item Profiling of the functional test executable for BLISS-B in \textit{valgrind/kcachegrind} shows that the performance bottlenecks are principally the hash function and the NTT, therefore small improvements to the associated functions provide reasonable performance gains.

\begin{itemize}

  \item Markku's Keccak/SHA3 has been optimised with some measures to manually perform loop unrolling so that the compiler doesn't have to use high (and potentially unstable) optimisation levels to achieve the same level of loop unrolling. However, this will be at the cost of an increased image size.

\begin{verbatim}
#ifdef SHA3_UNROLLED
    bc[0] = st[0] ^ st[0 + 5] ^ st[0 + 10] ^ st[0 + 15] ^ st[0 + 20];
    bc[1] = st[1] ^ st[1 + 5] ^ st[1 + 10] ^ st[1 + 15] ^ st[1 + 20];
    bc[2] = st[2] ^ st[2 + 5] ^ st[2 + 10] ^ st[2 + 15] ^ st[2 + 20];
    bc[3] = st[3] ^ st[3 + 5] ^ st[3 + 10] ^ st[3 + 15] ^ st[3 + 20];
    bc[4] = st[4] ^ st[4 + 5] ^ st[4 + 10] ^ st[4 + 15] ^ st[4 + 20];
#else
    for (i = 0; i < 5; i++) {
        bc[i] = st[i] ^ st[i + 5] ^ st[i + 10] ^ st[i + 15] ^ st[i + 20];
    }
#endif
\end{verbatim}

  \item Small indexing calculations in SHA-3 that are used repeatedly in loops where providing a large latency, therefore this has been replaced with 5 small LUTs to store the precomputed values.

\begin{verbatim}
    int i4mod5[5] = {4, 0, 1, 2, 3};
    int i1mod5[5] = {1, 2, 3, 4, 0};

    ...

    for (i = 0; i < 5; i++) {
        t = bc[i4mod5[i]] ^ ROTL64(bc[i1mod5[i]], 1);
        for (j = 0; j < 25; j += 5) {
            st[j + i] ^= t;
        }
    }
\end{verbatim}

As opposed to the original:

\begin{verbatim}
    for (i = 0; i < 5; i++) {
        t = bc[(i + 4) % 5] ^ ROTL64(bc[(i + 1) % 5], 1);
        for (j = 0; j < 25; j += 5) {
            st[j + i] ^= t;
        }
    }
\end{verbatim}
\end{itemize}
\end{itemize}

\clearpage
\subsubsection{Signature Restart}
\begin{itemize}
\item To improve the performance of the \textbf{Restart} if the rejection threshold is not met we have implemented a cheap but effective trick - every other time the signature algorithm is restarted the two random polynomials (sampled from a Gaussian distribution in the previous iteration) are simply swapped rather than re-generated, gaining approx. 10\%-25\% more signatures per second as shown in Figure \ref{fig:bliss_b_gaussian_swapping}.

\pgfplotsset{compat=1.13,width=15cm,height=7cm}
\begin{figure}[ht!]
\centering
\begin{tikzpicture}
\begin{axis}[xbar, xmin=0, xmax=12500, xtick={0,2000,4000,6000,8000,10000}, ytick=data, enlarge y limits=0.5, symbolic y coords={Without Swapping, With Swapping}, y tick label style={/pgf/number format/1000 sep=}, nodes near coords, nodes near coords align={horizontal}, legend style={at={(0.5,-0.15)},anchor=north,legend columns=-1},]
\addplot coordinates {(9917,Without Swapping) (10877,With Swapping)};
\addplot coordinates {(5654,Without Swapping) (7061,With Swapping)};
\addplot coordinates {(8143,Without Swapping) (9746,With Swapping)};
\addplot coordinates {(5871,Without Swapping) (6444,With Swapping)};
\legend{BLISS-B-I, BLISS-B-II, BLISS-B-III, BLISS-B-IV}
\end{axis}
\end{tikzpicture}
\caption{Signatures per second performance of BLISS-B with no entropy coding and Restarts with/without Gaussian swapping [Intel i7 6700 CPU @ 3.4GHz]}
\label{fig:bliss_b_gaussian_swapping}
\end{figure}

\begin{itemize}
  \item In other schemes with a retrial a similar technique could be used. As BLISS must generate two Gaussian distributions it can quickly swap memory pointers rather than swapping the actual memory contents. Other schemes with only a single Gaussian distribution could employ methods to filter, scramble or partially update the existing distribution rather than create an entirely new polynomial using a Gaussian sampler.
\end{itemize}
\end{itemize}

\subsubsection{Entropy Coding}
\begin{itemize}
\item The signature is composed of three polynomials, $z_1$, $z_2$ and $c$. $z_1$ is a 12-bit code with a Gaussian distribution which can be exploited with an entropy coder to reduce the signature size. $z_2$ is a 2, 3 or 4 bit sparse polynomial (depending on the parameter set selected) that can also be readily compressed. The third polynomial $c$ contains random indices for an $n$ element polynomial and is short in length (12 to 39 elements) and therefore compression gains are minimal. The performance of BLISS-B signature compression is seen in Figure \ref{fig:bliss_b_signature_compression}, whilst the performance of the various entropy coders is shown in Figure \ref{fig:bliss_b_signature_compression_performance}.

\pgfplotsset{compat=1.13,width=14cm,height=8cm}
\begin{figure}[ht!]
\centering
\begin{tikzpicture}
\begin{axis}[xbar=0pt, xlabel=Compression (\%), xmin=0, xmax=120, xtick={0,20,40,60,80,100}, ytick=data, enlarge y limits=0.15, symbolic y coords={None, BAC, BAC with RLE, strongSwan Huffman, Huffman}, y tick label style={/pgf/number format/1000 sep=}, nodes near coords, nodes near coords align={horizontal}]
\addplot coordinates {(100,None) (78.709,BAC) (80.606,BAC with RLE) (78.487,strongSwan Huffman) (80.721,Huffman)};
\addplot coordinates {(100,None) (78.918,BAC) (85.116,BAC with RLE) (77.555,strongSwan Huffman) (79.869,Huffman)};
\addplot coordinates {(100,None) (84.829,BAC) (89.264,BAC with RLE) (78.737,strongSwan Huffman) (80.36,Huffman)};
\legend{BLISS-B-I, BLISS-B-III, BLISS-B-IV}
\end{axis}
\end{tikzpicture}
\caption{BLISS-B Signature Compression with various entropy coders}
\label{fig:bliss_b_signature_compression}
\end{figure}

\pgfplotsset{compat=1.13,width=14cm,height=8cm}
\begin{figure}[ht!]
\centering
\begin{tikzpicture}
\begin{axis}[xbar=0pt, xlabel=Signatures (per second), xmin=0, xmax=16000, xtick={0,2000,4000,6000,8000,10000,12000}, ytick=data, enlarge y limits=0.15, symbolic y coords={None, BAC, BAC with RLE, strongSwan Huffman, Huffman}, y tick label style={/pgf/number format/1000 sep=}, nodes near coords, nodes near coords align={horizontal}]
\addplot coordinates {(11067,None) (6506,BAC) (6658,BAC with RLE) (10714,strongSwan Huffman) (10369,Huffman)};
\addplot coordinates {(9537,None) (5824,BAC) (6054,BAC with RLE) (9211,strongSwan Huffman) (8913,Huffman)};
\addplot coordinates {(7010,None) (4599,BAC) (4888,BAC with RLE) (6778,strongSwan Huffman) (6395,Huffman)};
\legend{BLISS-B-I, BLISS-B-III, BLISS-B-IV}
\end{axis}
\end{tikzpicture}
\caption{Performance of BLISS-B Signature Compression with various entropy coders [Intel i7 6700 CPU @ 3.4GHz]}
\label{fig:bliss_b_signature_compression_performance}
\end{figure}

\item The norms of the signature are currently checked by the signature algorithm, however this is not required and can be removed for a small performance gain.
\end{itemize}

\clearpage
\subsubsection{NTT/INTT}
\begin{itemize}

  \item The NTT loops have been modified to reduce some loop overhead in generating control variables, where possible they've been modified to encourage the compiler to use loop vectorisation and SIMD instructions.
  \item The NTT being used requires bit shuffling. This requires a function to swap each element of the polynomial with the element indexed by the bit reverse of it's index (\textit{see inverse\_shuffle\_32() in ntt.c}). For example, ARM instruction sets have a bit reverse instruction that can readily obtain the correct index for the swap operation.

\begin{verbatim}

UINT32 sc_bit_reverse_32(UINT32 x)
{
#ifdef __arm__
    UINT32 y;
    __asm__("rbit %0, %1\n" : "=r"(y) : "r"(x));
    return y;
#else
    x = (((x & 0xaaaaaaaa) >> 1) | ((x & 0x55555555) << 1)); // Swap odd and even
    x = (((x & 0xcccccccc) >> 2) | ((x & 0x33333333) << 2)); // Swap pairs
    x = (((x & 0xf0f0f0f0) >> 4) | ((x & 0x0f0f0f0f) << 4)); // Swap nibbles
    x = (((x & 0xff00ff00) >> 8) | ((x & 0x00ff00ff) << 8)); // Swap bytes
    return (x >> 16) | (x << 16);                            // Swap pairs of bytes
#endif
}

...

// Make use of a bit reversal instruction
UINT32 bits = 32 - sc_ctz_32(n);
for (i = 1; i < n-1; i++) {       // 00..0 and 11..1 remain same
    UINT32 r = sc_bit_reverse_32(i);
    r >>= bits;
    if (i < r) {
        SINT32 x = v[i];
        v[i] = v[r];
        v[r] = x;
    }
}
\end{verbatim}

  \item On x86 we can use the gcc \textit{builtin\_ctz} function (if available) to provide a more optimal bit reversal function by incrementing the MSB's to obtain bit reversal.

\begin{verbatim}
// If a BUILTIN funtion exists for CTZ then inverse incremental
// counting can be achieved by incrementing the MSB's
UINT32 bits = sc_ctz_32(n) - 1;
j = n >> 1;
for (i = 1; i < n - 1;) {       // 00..0 and 11..1 remain same
    if (i < j) {
        SINT32 x = v[i];
        v[i] = v[j];
        v[j] = x;
    }
    UINT32 mask = i++;
    mask ^= i;
    UINT32 len = sc_ctz_32(i);
    mask <<= bits - len;
    j ^= mask;
}
\end{verbatim}

  \item A fallback method is provided that increments the MSB's without the help of the \textit{builtin\_ctz} function and provides a generic solution. However, this method requires a \textit{while} loop.

\begin{verbatim}
// This is the fallback method that also increments the MSB's
j = n >> 1;
for (i = 1; i < n - 1; i++) {       // 00..0 and 11..1 remain same
    if (i < j) {
        SINT32 x = v[i];
        v[i] = v[j];
        v[j] = x;
    }
    k = n;
    do {
        k >>= 1;
        j ^= k;
    } while ((j & k) == 0);
}
\end{verbatim}

  \item As the performance of the loops in the NTT are critical it was seen that reduction scheme specific implementations of the main NTT function were beneficial to performance. So rather than set a function pointer to use the desired multiplication and modular reduction routine, specific NTT functions were added for each reduction scheme that remove the overhead of function calls and in addition permit the compiler to perform loop unrolling and loop vectorisation, leading to some reasonable performance gains. A summary describing the performance gains achieved through simple optimisation of the NTT is shown in Figure \ref{fig:bliss_b_optimised_ntt}.

\pgfplotsset{compat=1.13,width=11cm,height=9cm}
\begin{figure}[ht!]
\centering
\begin{tikzpicture}
\begin{axis}[xbar, xmin=0, xmax=25000, xtick={0,5000,10000,15000,20000,25000}, ytick=data, symbolic y coords={Optimized Loops with Barrett and builtin\_ctz, Barrett and builtin\_ctz, Barrett Reduction, Original}, enlarge y limits=0.2, y tick label style={/pgf/number format/1000 sep=}, nodes near coords, nodes near coords align={horizontal}, legend style={at={(0.5,-0.15)},anchor=north,legend columns=-1},]
\addplot coordinates {(21846,Optimized Loops with Barrett and builtin\_ctz) (20124,Barrett and builtin\_ctz) (19273,Barrett Reduction) (12397,Original)};
\addplot coordinates {(6967,Optimized Loops with Barrett and builtin\_ctz) (6711,Barrett and builtin\_ctz) (6561,Barrett Reduction) (4837,Original)};
\addplot coordinates {(17897,Optimized Loops with Barrett and builtin\_ctz) (16713,Barrett and builtin\_ctz) (16964,Barrett Reduction) (5682,Original)};
\legend{Verifications (per sec), Signatures (per sec), Key Generation (per sec)}
\end{axis}
\end{tikzpicture}
\caption{Performance gains from optimised NTT loops [Intel i7 6700 CPU @ 3.4GHz]}
\label{fig:bliss_b_optimised_ntt}
\end{figure}

  \item The NTT roots of unity tables are stored using the minimum bit width type required. When these coefficients are used in multiplication routines they are typically cast to an int64\_t or a double, and within the loop operations of the NTT/INTT they are read infrequently relative to other variables. Therefore using the minimal storage type reduces the memory use within the image whilst having no impact on performance.
\end{itemize}

\subsubsection{CSPRNG}
\begin{itemize}

  \item A range of PRNG's are provided by the SAFEcrypto library: ISAAC \cite{issac}, KISS \cite{kiss}, AES-PRNG (utilizes Brian Gladman's AES \cite{aes_gladman}) and CHACHA20-CSPRNG \cite{chacha20_csprng}.
  \item A functional test of the various CSPRNG options was used to determine their relative performance - the results are shown in Figure \ref{fig:hash_mbyte_per_sec}.

\pgfplotsset{compat=1.13,width=10cm,height=9cm}
\begin{figure}[ht!]
\centering
\begin{tikzpicture}
\begin{axis}[ybar=8pt, xtick=data, enlarge x limits=0.5, symbolic x coords={32-bit, 64-bit}, x tick label style={/pgf/number format/1000 sep=}, nodes near coords, nodes near coords align={vertical}, legend style={at={(.5,-0.15)},anchor=north,legend columns=-1}]
\addplot coordinates {(32-bit, 66.933) (64-bit, 89.083)};
\addplot coordinates {(32-bit, 148.129) (64-bit, 217.928)};
\addplot coordinates {(32-bit, 88.041) (64-bit, 87.73)};
\addplot coordinates {(32-bit, 45.615) (64-bit, 45.882)};
\addplot coordinates {(64-bit, 239.004)};
\legend{POSIX random(), ISAAC, AES-PRNG, CHACHA20-CSPRNG, KISS}
\end{axis}
\end{tikzpicture}
\caption{Performance of various CSPRNG schemes in MByte/sec [Intel i7 6700 CPU @ 3.4GHz]}
\label{fig:hash_mbyte_per_sec}
\end{figure}

\noindent \textbf{NOTE: KISS is 64-bit only, we probably need to implement the 32-bit version for comparison purposes.}

\item The performance of BLISS-B with no entropy coding whilst using a range of CSPRNG's is shown in Figure \ref{fig:bliss_b_csprng_comparison}. As expected there is no real difference in improvement for the verification operation as it requires no random number generation. The higher performance of KISS and ISAAC translates to higher performance within BLISS-B, with both CSPRNG's offering very similar performance.

\pgfplotsset{compat=1.13,width=15cm,height=10cm}
\begin{figure}[ht!]
\centering
\begin{tikzpicture}
\begin{axis}[xbar=0pt, ytick=data, enlarge y limits=0.15, symbolic y coords={POSIX random, ISAAC, AES-PRNG, CHACHA20-CSPRNG, KISS}, y tick label style={/pgf/number format/1000 sep=}, nodes near coords, nodes near coords align={horizontal}, legend style={at={(.5,-0.15)},anchor=north,legend columns=-1}]
\addplot coordinates {(22200,POSIX random) (22124,ISAAC) (22233,AES-PRNG) (22355,CHACHA20-CSPRNG) (22123,KISS)};
\addplot coordinates {(6981,POSIX random) (7803,ISAAC) (6993,AES-PRNG) (5932,CHACHA20-CSPRNG) (7713,KISS)};
\addplot coordinates {(14948,POSIX random) (16175,ISAAC) (15179,AES-PRNG) (13314,CHACHA20-CSPRNG) (16266,KISS)};
\legend{Verifications (per sec), Signatures (per sec), Key Generation (per sec)}
\end{axis}
\end{tikzpicture}
\caption{Performance of various CSPRNG schemes in BLISS-B-IV [Intel i7 6700 CPU @ 3.4GHz]}
\label{fig:bliss_b_csprng_comparison}
\end{figure}

\end{itemize}

\clearpage
\subsection{Verification}

\begin{itemize}
\item Reject if $||(z_1 | 2^d . z_2)||_2 > B_2$
\item Reject if $||(z_1 | 2^d . z_2)||_\infty > B_2$
\item Accept if $c = H(\lfloor a_1 . z_1 + q . c \rceil_d + z_2 mod p, \mu)$
\end{itemize}

Verification utilises some of the same functions as signing (NTT, Hashing, checking the signature norms) so achieves the same gains when these are improved.

\begin{itemize}
\item There are some operations that require a polynomial to be reduced using \textit{2q} or \textit{p} rather than \textit{q}. These have also been accelerated using Barrett and Floating Point reduction techniques, taking care to structure the code such that automatic vectorisation is enabled by the compiler.
\end{itemize}

\subsection{General Performance Improvements}

\begin{itemize}
\item Intermediate storage is achieved using heap memory allocated when the schemes call the \textit{create()} functions. This memory is released when the scheme is destroyed. This means that when any cryptographic functions are called there is no dynamic memory allocation or static memory used to store intermediate variables.
\item Integer and floating point types of 32-bit or larger are preferred on x86/AMD64 as the compiler can exploit SIMD/AVX auto-vectorization. This compiler optimization is to be investigated further for other microprocessor architectures.
\end{itemize}

\subsection{Summary}

The original hilabliss/BLZZRD implementations of BLISS are reference designs used to show the algorithms and functional operation of the signature scheme; as such they are intended to be non-optimal.

We have shown that the use of a Binary Arithmetic Coder within BLZZRD is sub-optimal in terms of performance, both as a Gaussian Sampler and an entropy coder. Huffman is significantly faster, whilct as a Gaussian Sampler it is statistically poor.

Also, hilabliss/BLZZRD and many other lattice-based cryptographic schemes are using SHA-3 as a hash function when implementing a random oracle. It is shown that this is a poor choice in terms of performance if not security. Therefore it is strongly suggested that a more optimal choice of hash would be SHA-2 or BLAKE2-B, both are standardized and strong hash functions in line with SHA-3.

The performance gains that have been achieved are summarised in Figures \ref{fig:bliss_b_no_entropy_comparison} and \ref{fig:bliss_b_entropy_comparison}. In this diagram the original algorithm is shown in comparison to (a) a version in which the NTT's and modular reduction have been optimised and (b) a version in which the the BLISS-B algorithm has been fully optimised with the addition of the ISAAC as a CSPRNG and BLAKE2 as a hash function. In all of these implementations entropy coding is disabled.

\noindent \textbf{I hope to add a multithreaded version in which the CSPRNG and Gaussian Sampler have been placed into worker threads which should be significantly faster again, even if it is cheating!}

\pgfplotsset{compat=1.13,width=12cm,height=8cm}
\begin{figure}[ht!]
\centering
\begin{tikzpicture}
\begin{axis}[xbar=2pt, xmax=50000, ytick=data, enlarge y limits=0.4, symbolic y coords={Original, Optimized, Optimized with ISAAC and BLAKE2}, y tick label style={/pgf/number format/1000 sep=}, nodes near coords, nodes near coords align={horizontal}, legend style={at={(.5,-0.15)},anchor=north,legend columns=-1}]
\addplot coordinates {(40682,Optimized with ISAAC and BLAKE2) (21696,Optimized) (12386,Original)};
\addplot coordinates {(11254,Optimized with ISAAC and BLAKE2) (6824,Optimized) (4824,Original)};
\addplot coordinates {(19677,Optimized with ISAAC and BLAKE2) (17384,Optimized) (5655,Original)};
\legend{Verifications (per sec), Signatures (per sec), Key Generation (per sec)}
\end{axis}
\end{tikzpicture}
\caption{Performance of SAFEcrypto BLISS-B-IV with no entropy coding [Intel i7 6700 CPU @ 3.4GHz]}
\label{fig:bliss_b_no_entropy_comparison}
\end{figure}

\pgfplotsset{compat=1.13,width=12cm,height=8cm}
\begin{figure}[ht!]
\centering
\begin{tikzpicture}
\begin{axis}[xbar=2pt, xmax=25000, ytick=data, enlarge y limits=0.4, symbolic y coords={Original, Optimized, Optimized with ISAAC and BLAKE2}, y tick label style={/pgf/number format/1000 sep=}, nodes near coords, nodes near coords align={horizontal}, legend style={at={(.5,-0.15)},anchor=north,legend columns=-1}]
\addplot coordinates {(20262,Optimized with ISAAC and BLAKE2) (13648,Optimized) (8848,Original)};
\addplot coordinates {(10235,Optimized with ISAAC and BLAKE2) (6401,Optimized) (4526,Original)};
\addplot coordinates {(19647,Optimized with ISAAC and BLAKE2) (17701,Optimized) (5677,Original)};
\legend{Verifications (per sec), Signatures (per sec), Key Generation (per sec)}
\end{axis}
\end{tikzpicture}

\textbf{[Put comparisons to other implementations of BLISS-B in here as it is compatible.]}
\caption{Performance of SAFEcrypto BLISS-B-IV with Huffman compression [Intel i7 6700 CPU @ 3.4GHz]}
\label{fig:bliss_b_entropy_comparison}
\end{figure}

\clearpage
In Figure \ref{fig:bliss_b_comparison} the original implementation of BLZZRD \cite{markku_blzzrd} is compared to a compatible implementation created using the SAFEcrypto library (i.e. SHA-3, Binary Arithmetic Coding and Blinding countermeasures). Verification is shown to be approximately 14\% slower using the SAFEcrypto library which can be attributed to the poor performance of the BAC decoder (\textit{some more effort could be exercised in trying to improve this}). However, the signature performance is approximately 25\% faster whilst key generation is significantly faster by 330\%.

A higher performance version of BLZZRD, here called \textit{BLZZRD+}, has been implemented to address the poor performance associated with the BAC and hash. BLZZRD+ utilises the more efficient BLAKE2 hash function, Ziggurat Gaussian Sampling and Huffman coding, whilst maintaining blinding countermeasures. As such it gains much greater performance for signing and verifying, being approximately 250\% and 170\% faster respectively than the original BLZZRD implementation.

\textit{The paper in \cite{markku_blzzrd} quotes performance on a 2.5GHz i7 in milliseconds per operation. To scale these figures for use here they have been inverted to convert to operations per second and muliplied by $1.36$ to scale to a 3.4GHz i7.}

\pgfplotsset{compat=1.13,width=12cm,height=8cm}
\begin{figure}[ht!]
\centering
\begin{tikzpicture}
\begin{axis}[xbar=2pt, xmax=30000, ytick=data, enlarge y limits=0.4, symbolic y coords={SAFEcrypto BLZZRD+, SAFEcrypto BLZZRD, Original BLZZRD}, y tick label style={/pgf/number format/1000 sep=}, nodes near coords, nodes near coords align={horizontal}, legend style={at={(.5,-0.15)},anchor=north,legend columns=-1}]
\addplot coordinates {(28391,SAFEcrypto BLZZRD+) (9595,SAFEcrypto BLZZRD) (11146,Original BLZZRD)};
\addplot coordinates {(5164,SAFEcrypto BLZZRD+) (3704,SAFEcrypto BLZZRD) (2968,Original BLZZRD)};
\addplot coordinates {(16097,SAFEcrypto BLZZRD+) (15971,SAFEcrypto BLZZRD) (4770,Original BLZZRD)};
\legend{Verifications (per sec), Signatures (per sec), Key Generation (per sec)}
\end{axis}
\end{tikzpicture}
\caption{Performance comparison of SAFEcrypto BLZZRD-IV [Intel i7 6700 CPU @ 3.4GHz]}
\label{fig:bliss_b_comparison}
\end{figure}


\documentclass[11pt]{article}
\usepackage[margin=.75in]{geometry}
\usepackage{nopageno}
\usepackage{hyperref}
\usepackage{multicol}

\usepackage{amsmath,amssymb,amsthm}
\newtheorem*{proposition}{Proposition}
\newtheorem*{theorem}{Theorem}

\newcommand{\Z}{\mathbb{Z}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\N}{\mathbb{N}}

\usepackage{xcolor}
\newcommand{\blue}{\textcolor{blue}}

\usepackage{versions}
\includeversion{solution}

\newcommand{\bs}{\begin{solution}}
\newcommand{\es}{\end{solution}}

\begin{document}
\vspace{-1.2in}
\begin{center} \textbf{\Large{Skill Mastery Quizzes - Question Bank}} \\
Communicating in Mathematics (MTH 210-02)\\
Fall 2017
\end{center}

\section*{Skills}

\subsection*{Logic}

\begin{itemize}
\item[L1] Identify the hypothesis and conclusion of a conditional statement, determine its truth value, and apply it.
	\begin{enumerate}

			\item[L1-1] Consider the following conditional statement:
	\begin{center}
	If $n$ is a prime number then $n^2$ has three positive factors.
	\end{center}
	Identify the hypothesis and conclusion.

	\bs \blue{ The hypothesis is ``$n$ is a prime number" and the conclusion is ``$n^2$ has three positive factors". Notice we leave off the if and the then when writing the hypothesis and conclusion.} \end{solution}

	Assume the above conditional statement is true. Assuming \emph{only} the conditional statement and that  $6$ is not prime what can you conclude (if anything)? Explain your answer.

	\bs \blue{We are given that the conditional statement is true and that $6$ is not prime. The fact that $6$ is not prime tells us that our hypothesis is false. Given a false hypothesis, we know nothing about the conclusion. So we can not conclude anything. We can also look at this from a truth table perspective. Consider the truth table for the conditional statement $P\rightarrow Q$:
 }
 \begin{center}
 \begin{tabular}{c|c|c}
$P$ &$Q$ &$P\rightarrow Q$\\
\hline
T &T &T\\
T &F &F\\
F &T &T\\
F &F &T
\end{tabular}
\end{center}
\textcolor{blue}{In the third and fourth rows the hypothesis of the conditional statement is false (that is, $n$ is not a prime number), and the conditional statement is true (that is, if $n$ is a prime number then $n^2$ has $3$ positive factors). Since $Q$ (that is, $n^2$ has $3$ positive factors), is true in the third row and false in the fourth row we can't conclude anything about the truth value of $Q$.} \end{solution}

\item[L1-2] Consider the following (true) conditional statement:
	\begin{center}
	If the function $f$ is continuous at $a$, then $\displaystyle{\lim_{x\to a} f(x)}$ exists.
\end{center}
	Identify the hypothesis and conclusion of this conditional statement.

	\textcolor{blue}{The hypothesis is ``the function $f$ is continuous at $a$" and the conclusion is ``$\displaystyle{\lim_{x\to a} f(x)}$ exists".}

Assume the above conditional statement is true. Assuming \emph{only} the conditional statement and that a function $f$ is not continuous at $7$, what can you conclude (if anything)?  Explain your answer.

	\textcolor{blue}{Given that the function $f$ is not continuous at $7$ we know the hypothesis of the conditional statement is false. Thus we cannot conclude anything, since the statement makes no promises about what happens if a function is not continuous at a given $a$. See Quiz 1 solutions for an explanation with a truth table.}

\item[L1-2] Consider the following conditional statement:
	\begin{center}
	If $p\neq 2$ and $p$ is an even number, then $p$ is not prime.
\end{center}
	Identify the hypothesis and conclusion of this conditional statement.

Assume the above conditional statement is true. Assuming \emph{only} the conditional statement and that $7$ is not equal to $2$ and is not even what can you conclude (if anything)?  Explain your answer.

	\item[L1-3] If $f: \mathbb{R}\to\mathbb{R}$ is a differentiable function at a real number $a$ then $f$ is a continuous function at $a$.  State the hypothesis and conclusion of this conditional statement.  Suppose you know a function $g: \mathbb{R}\to\mathbb{R}$ is continuous at $5$.  What can you conclude (if anything)?  Explain your answer.

	\item[L1-token]  Consider the following conditional statement:
	\begin{center}
	If $p$ is a prime number then $p=2$ or $p$ is an odd number.
	\end{center}
	Identify the hypothesis and conclusion.
	Assume the above conditional statement is true. Assuming \emph{only} the conditional statement and that $7$ is odd what can you conclude (if anything)?  Explain your answer.

	\end{enumerate}

	\newpage

\item[L2] State precisely the definition of an even and odd integer and outline the proof of a statement using these terms.
	\begin{enumerate}
	\item[L2-1] State the definition of even integer precisely.
		An integer $n$ is even provided that...

	\bs\textcolor{blue}{ there exists an integer $q$ such that $n=2q$.}\end{solution}

	Then outline a proof that if $x$ is even and $y$ is odd then $x+y$ is odd. (Make sure to include key details - like what things are integers.)

	\bs\textcolor{blue}{Suppose $x$ is even and $y$ is odd. Then there exist integers $a,b$ such that $x=2a$ and $y=2b+1$. Then $x+y=(2a)+(2b+1)$ by substitution. By the distributive property $x+y=2(a+b)+1$. Note that $a+b$ is an integer by closure of the integers under addition. Let $q=a+b$. Then $x+y=2q+1$ for the integer $q$ and so $x+y$ is an odd integer.}\end{solution}

	\item[L2-2] State the definition of odd integer precisely. Then outline a proof that if $x,y\in \Z$ are odd integers then $x+y$ is even.
	\item[L2-3] State the definition of even integer precisely. Then outline a proof that if $x,y\in \Z$ are even integers then $xy$ is even.
	\item[L2-token] State the definition of odd integer precisely. Then outline a proof that if $x,y\in \Z$ are odd integers then $xy$ is odd.
	\end{enumerate}

\newpage

\item[L3] Construct truth tables for statements that use the logical operators and, or, not, and implies.
	\begin{enumerate}
	\item  Construct a truth table for $(\neg P\vee Q) \rightarrow R$.
	\item Construct a truth table for $P \implies (Q\wedge R)$
	\item Construct a truth table for $P \implies (Q\vee R)$.
	\end{enumerate}

\newpage

\item[L4] Write sets using set builder notation and interpret sets written in this notation.
	\begin{enumerate}
		\item[L4-1] Write the set $\left\{ \sqrt{2}, \left(\sqrt{2}\right)^3, \left(\sqrt{2}\right)^5,\dots\right\}$ in set builder notation.

	\item[L4-2] Describe what the set $\{x\in\mathbb{R} \mid 3\leq x\leq 5\}$ is in words. Then write what the set is in roster notation or explain why you can not.

		\item[L4-3] Write the set $\{\dots, -5,-3,1,1,3,5,\dots\}$ using set builder notation.
	\end{enumerate}

\newpage

\item[L5]  Negate a statement with and, or, not, implies, exists, and/or for all.
	\begin{enumerate}
		\item[L5-1] Write a useful negation of the following statement:
		\begin{center}
		There exists $x\in\mathbb{Z}$ such that if $y\in \mathbb{Z}$ then $\frac{y}{x}\in\mathbb{Z}$.
		\end{center}
		Useful negations don't start with ``It is not true that..." and avoid the word not.

		\item[L5-2] Write a useful negation for the following statement:
		\begin{center}
		For every positive real number $\epsilon$ there exists a natural number $n$ with $\frac{1}{n}<\epsilon$.
		\end{center}
		Useful negations don't start with ``It is not true that..." and avoid the word not.

	\item[L5-3] For all integers $n$ and $m$, if $nm$ is even then $n$ is even or $m$ is even.

	\item[L5-token?]  Write a useful negation of the following statement:
		\begin{center}
		For all $m\in\mathbb{Z}$ there exists $n\in\mathbb{Z}$ such that $m>n$.
		\end{center}
		Useful negations don't start with ``It is not true that..." and avoid the word not.
	\end{enumerate}
\end{itemize}

\newpage

\subsection*{Proofs}
\begin{itemize}
\item[P1]  Given a theorem, correctly state what will be assumed in a direct proof, proof by contradiction, and proof by contrapositive.
	\begin{enumerate}
	\item[P1-1] Consider the following statement:
		\begin{center}
		Every even integer greater than $2$ can be expressed as the sum of two (not necessarily distinct) prime numbers.
		\end{center}
	State what you would assume in a direct proof. State what you would assume in a proof by contradiction.
	\item[P1-2]  Consider the following statement:
		\begin{center}
		For all natural numbers $p$ and $q$, if $p$ and $q$ are twin primes other than $3$ and $5$ , then $pq+1$ is a perfect square and $36$ divides $pq+1$.
		\end{center}
		State what you would assume in a direct proof. State what you would assume in a proof by contradiction.
	\item[P1-3]  Consider the following statement
		\begin{center}
		Let $a$ and $b$ be integers with $a\neq 0$. If $a$ does not divide $b$ then the equation $ax^3+bx+(b+a) = 0$ does not have a solution that is a natural number.
		\end{center}
		State what you would assume in a direct proof. State what you would assume in a proof by contradiction.
	\end{enumerate}

	\newpage

\item[P2]  Identify situations in which it is appropriate to use induction and state the procedure for proving a statement by induction.

\begin{enumerate}
	\item[P2-1] For which of the following situations is it more appropriate to use induction. Explain.
		\begin{enumerate}
		\item For all $a\in\Z$ the equation $ax^3+ax + a = 0$ does not have a solution that is a natural number.
		\item Let $a$ and $b$ be integers and $n\in \mathbb{N}$. For all $m\in\mathbb{N}$ if $a\cong b \pmod{n}$ then $a^m \cong b^m \pmod{n}$.
		\end{enumerate}
	For the statement you chose, state what your steps would be in a proof by induction.

	\item[P2-2] For which of the following situations is it more appropriate to use induction. Explain.
		\begin{enumerate}
		\item For all integers $a$ and $b$, $(a+b)^2 \equiv (a^2 +b^2) \pmod{2}$
		\item For each natural number $n$, $3$ divides $4^n-1$.
		\end{enumerate}
	For the statement you chose, state what your steps would be in a proof by induction.

	\item[P2-3] For which of the following situations is it more appropriate to use induction. Explain.
		\begin{enumerate}
		\item For all $a\in\Z$ the equation $ax^3+ax + a = 0$ does not have a solution that is a natural number.
		\item For all $n\in\N$, $1+2+3+\cdots + n = \dfrac{n(n+1)}{2}$.
		\end{enumerate}
	For the statement you chose, state what your steps would be in a proof by induction.

\end{enumerate}

\newpage

\item[P3] Clearly and correctly disprove a statement using a counterexample.

	\begin{enumerate}
	\item[P3-1] The following statement is incorrect:
		\begin{center}
		If $n$ is an integer then $n^2\equiv 1\pmod{3}$.
		\end{center}
			Show the statement is false using a counterexample. You should clearly explain why the counterexample you found shows the statement is false.

	\item[P3-2] The following statement is incorrect:
		\begin{center}
		The set of natural numbers is closed under subtraction.
		\end{center}
		Show the statement is false using a counterexample. You should clearly explain why the counterexample you found shows the statement is false.

		\item[P3-3] The following statement is incorrect:
		\begin{center}
		For each integer $n$, $(n^2+1)$ is a prime number.
		\end{center}
	Show the statement is false using a counterexample. You should clearly explain why the counterexample you found shows the statement is false.

	\end{enumerate}

\newpage

\item[P4] Evaluate if a given proof is valid and adheres to our writing guidelines.
	\begin{enumerate}
	\item[P4-1] Consider the following proposition and proof. Is the proof correct? If not, explain why not. If so, does the proof meet our writing guidelines?

		\begin{proposition}
		If $m$ is an odd integer then $m+6$ is an odd integer.
		\end{proposition}
		\begin{proof}
		For $m+6$ to be an odd integer there must exist an integer $n$ such that
			\[m+6 = 2n+1.\]
		By subtracting $6$ from both sides of this equation we obtain
			\begin{align*}
			m &= 2n-6+1\\
			&=2(n-3)+1.
			\end{align*}
		By the closure properties of integers, $n-3$ is an integer, and hence, the last equation implies that $m$ is an odd integer. This proves that if $m$ is an odd integer then $m+6$ is an odd integer.
		\end{proof}

\item[P4-2] Considering the following proposition and proof. Is the proof correct? If not, explain why not. If so, does the proof meet our writing guidelines?

	\begin{theorem}
	For each integer $n$, $3\mid n^2 + 2$.
	\end{theorem}

	\begin{proof}
	We will consider two cases, $n\equiv 1\pmod{3}$ and $n\equiv 2\pmod{3}$. When $n\equiv 1 \pmod{3}$, there exists an integer $k$ such that $3k = n-1$. Then
		$$n^2+2 = (3k+1)^2 + 2 = 9k^2 + 6k + 1 + 2 = 3(3k^2+2k+1).$$
	Since integers are closed under addition and multiplication  $3k^2+2k+1$ is an integer. Thus $3\mid n^2+2$ in this case.

	When $n\equiv 2 \pmod{3}$ there exists an integer $k$ such that $3k=n-2$. Then
		$$n^2+2 = (3k+2)^2 +2 =9k^2+12k+4+2 = 3(3k^2+4k+2).$$
	Since integers are closed under addition and multiplication, $3k^2+4k+2$ is an integer. Thus $3\mid n^2+2$ in this case as well.

	Since we've proven that  $3\mid n^2+2$ in all possible cases we have completed the proof.
	\end{proof}
	\end{enumerate}

\item[P4-3]  Considering the following proposition and proof. Is the proof correct? If not, explain why not. If so, does the proof meet our writing guidelines?

	\begin{theorem}
	For all integers $m$ and $n$, if $mn$ is an even integer, then $m$ is even or $n$ is even.
	\end{theorem}
	\begin{proof}
	For either $m$ or $n$ to be even there exists an integer $k$ such that $m=2k$ or $n=2k$. So if we multiply $m$ and $n$ the product will contain a factor of $2$ and, hence, $mn$ will be even.
	\end{proof}

\end{itemize}

\newpage

\subsection*{Sets, Functions, and Equivalence Relations}
\begin{itemize}
\item[S1] Use the symbols $\in, \notin, =,\neq,\subseteq,\not\subseteq,\subset,\not\subset$ correctly.
	\begin{enumerate}
	\item[S1-1] Let $A = \{ 1,\{2\}, \{3,4\}, 5\}$.  Fill in a correct symbol for each of the following:
		\begin{itemize}
		\item $\{1\} \underline{\hspace{.25in}} A$
		\item $\{2\} \underline{\hspace{.25in}} A$
		\item $\{1,2,3,4,5\} \underline{\hspace{.25in}}A$
		\end{itemize}
	\item[S1-2] Let $A = \{1,2,4\}$ and $B = \{1,2,3,5\}$.  Fill in a correct symbol for each of the following:
		\begin{itemize}
		\item $A\underline{\hspace{.25in}} B$
		\item $\emptyset \underline{\hspace{.25in}} A$
		\item $\{4,2,1\} \underline{\hspace{.25in}} B$
		\end{itemize}
	\item[S1-3]  Let $A = \{0,1,2,3,\{4\}\}$.  Fill in a correct symbol   (from $\in$, $\subset$, $\subseteq$, $=$, $\neq$) for each of the following.
		\begin{itemize}
		\item $\{4\} \underline{\hspace{.25in}} A$
		\item $\{2\} \underline{\hspace{.25in}} A$
		\item $\{1,2,3\} \underline{\hspace{.25in}}A$
		\end{itemize}
	\end{enumerate}

\newpage

\item[S2] Given two sets and a universal set identify the union, intersection, complement, and set difference and find the power set of a given set.
	\begin{enumerate}
	\item[S2-1] Let $U = \{1,2,3,4,5,6,7,8,9,10\}$ be the universal set. Let $A = \{3,4,5,6,7\}$ and $B=\{1,5,7,9\}$.
		\begin{enumerate}
		\item Find $A\cap B$
		\item Find $A \cup B$
		\item Find $A^C$
		\item Find $A\setminus B$ (or $A-B$).
		\end{enumerate}
	\item[S2-2]  Let $U = \Z$. Let $A = \{x\in \Z: x\ge 7\}$ and $B = \{x\in\Z: x is odd\}$.  (Roster method is okay for your answers.)
		\begin{enumerate}
		\item Find $A\cap B$
		\item Find $A \cup B$
		\item Find $A^C$
		\item Find $A\setminus B$ (or $A-B$).
		\end{enumerate}
	\item[S2-3]  Let $U = \{1,2,3,4,5,6,7,8,9,10\}$ be the universal set.  Let $A= \{2,4,6,8,10\}$ and $B = \{1,3\}$.
		\begin{enumerate}
		\item Find $A\cap B$
		\item Find $A^C$
		\item Find  $A-B$
		\item Find $A\cup B$.
		\end{enumerate}
	\end{enumerate}

\newpage

\item[S3] Correctly use function terminology such as domain, codomain, range, dependent variable, independent variable, image, and preimage.
	\begin{enumerate}
	\item[S3-1] Let $f: \mathbb{R} \to \mathbb{R}$ be defined by $f(x) = x^2-2x$.
		\begin{itemize}
		\item State the domain, codomain, and range of $f$. (Clearly state which one is which.)
		\item Find the image(s) of $3$ under $f$.
		\item Find the preimage(s) of $0$.
		\end{itemize}
	\item[S3-2] Let $R^* = \{x\in \R: x\ge 0\}$.  Let $s: \R^* \to \R^*$ be defined by $f(x) = x^2$.
		\begin{itemize}
		\item State the domain, codomain of $f$. (Clearly state which one is which.)
		\item Find the image(s) of $3$ under $f$.
		\item Find the preimage(s) of $4$.
		\end{itemize}
	\item[S3-3]  Let $f: \Z\to \Z$ be defined by $f(m) = 3-m$.
		\begin{itemize}
		\item State the domain, codomain, and range of $f$. (Clearly state which one is which.)
		\item Find the image(s) of $3$ under $f$.
		\item Find the preimage(s) of $0$.
		\end{itemize}
	\end{enumerate}

	\newpage

\item[S4] State the definition of injection, surjection, and bijection.
	\begin{enumerate}
	\item[S4-1] Let $A$ and $B$ be sets. Carefully complete the definitions of the following terms:
		\begin{itemize}
		\item  A function $f: A \to B$ is injective provided that...
		\item A function $f: A\to B$ is surjective provided that...
		\item A function $f: A\to B$ is bijective provided that...
		\end{itemize}
	\end{enumerate}

\newpage

\item[S5] Prove or disprove that a given relation is reflexive, symmetric, and/or transitive.
	\begin{enumerate}
	\item For all $a,b\in \Z$ say $a\sim b$ if and only if $a\mid b$.  Is $\sim$ an equivalence relation? Explain.
	\item For all $a,b \in \Z$ say $a\sim b$ if and only if $a\leq b$.  Is $\sim$ an equivalence relation? Explain.
	\item For all $a,b\in \R$ say $a\sim b$ if and only if $|a-b|<5$.  Is $\sim$ an equivalence relation? Explain.
	\end{enumerate}

\newpage

\item[S6] State the definition of ``$a$ divides $b$" and ``$a$ is congruent to $b$ modulo $n$" and correctly apply these definitions in examples.
	\begin{enumerate}
	\item  Carefully state the definition of $a\mid b$ (for nonzero $a$) and $a \equiv b\pmod{n}$ (for nonzero $n$).  Then give an example of integers $a$ and $b$ such that $a\cong b \pmod{15}$ and $b<0$.
	\item Carefully state the definition of $a\mid b$ (for nonzero $a$) and $a \equiv b\pmod{n}$ (for nonzero $n$).  Then give an example of integers $a$ and $b$ such that $a \nmid b$.
	\item Carefully state the definition of $a\mid b$ (for nonzero $a$) and $a \equiv b\pmod{n}$ (for nonzero $n$).  Then give an example of integers $a$ and $b$ such that $a\not\equiv b \pmod{3}$.
	\end{enumerate}
\end{itemize}

\end{document}


\documentclass[12pt, letterpaper]{article}
\usepackage[english]{babel}
\usepackage{amsmath}
\usepackage{unicode-math}
\usepackage[margin=0.7in]{geometry}
\usepackage{graphicx}
%\usepackage{mathrsfs}
\usepackage{fontspec}
\setmainfont{Calibri}
% must be compiled with XeLaTeX, not pdfLaTeX

  %%%%%%%%%%%%
  % PREAMBLE %
  %%%%%%%%%%%%
\begin{document}
\selectlanguage{english}

\section*{Postrefinement: The model {\tt rs}. }
  \par This is intended to be the simplest possible model for the reciprocal lattice point (RLP),
  describing the RLP as a sphere of radius ${r_s}$, which is globally constant over the whole dataset
  consisting of an ensemble of crystal lattices (or frames).
\subsection*{1 The size of the RLP model}

  \par The constant value of ${r_s}$ is computed as follows.
  From model refinement and integration, each crystal has associated with it a list of Miller indices
  $\mathbf{h_i}$ and a reciprocal space orientation matrix $\mathbf{A}$
  defined by Rossmann $\textit{et\ al.}$ (1979),
    \begin{equation}
    \mathbf{A} =
    \left(
    \begin{array}{c c c}
      a_{x}^{*}   &  b_{x}^{*}   & c_{x}^{*}  \\
      a_{y}^{*}   &  b_{y}^{*}   & c_{y}^{*}  \\
      a_{z}^{*}   &  b_{z}^{*}   & c_{z}^{*}  \\
    \end{array}
    \right)
    \text{.}
    \label{eqn:RossmannA}
  \end{equation}
  The reciprocal space coordinates of RLP $i$ are computed with
    \begin{equation}
    \mathbf{q} = \mathbf{A}\mathbf{h}
    \text{,}
    \label{eqn:Ah}
  \end{equation}
  leading to a reciprocal position Q, with a small distance offset
   ${r_h}$ away from the Ewald sphere that represents the perfect diffracting condition.
  The fact that $|{r_h}|$ is non-zreo is indicative that
  Bragg observations from still shots represent partial reflections.  Note that array index $i$ denoting
  a specific Miller index is dropped on occasion for clarity.  The geometry is explained in Fig. 1.

  \begin{figure}[htb]
  \begin{center}
  \includegraphics[scale=1.5]{Figure_1.pdf}
  \label{fig:1}
  \end{center}
  \begin{center}
  {Fig. 1. Ewald sphere construction.}
  \end{center}
  \end{figure}

  The quantity $|{r_h}|$ is given by
    \begin{equation}
    {r_h} = ||\mathbf{q}+\mathbf{s_0}||-||\mathbf{s_1}|| = ||\mathbf{q}+\mathbf{s_0}||-\dfrac{1}{ \lambda}
    \text{,}
    \label{eqn:Rh}
  \end{equation}

  where ${s_0}$ and ${s_1}$ are respectively the beam vector and the diffracted ray vector,
  each of length $1/\lambda$.  For the model {\tt rs}, the constant value of ${r_s}$ is taken
  as the root mean-squared value of
  ${r_h}$ over all Bragg spots integrated from a given crystal.  It therefore depends on whatever
  algorithm has been used to predict spots, regardless of whether there is measurable signal
  in the spots.
  \subsection*{2 The geometry of the RLP model}

  \par The intention is to create a model of the RLP similar to a hard sphere, so that if any portion
  of the sphere touches the Ewald sphere there is signal expected, otherwise none.  However, this
  is a discontinuous model (in terms of the spot partiality expressed as a function of ${r_h}$ and
  therefore not easily amenable to parameter fitting.  Therefore we relax the requirement for a hard
  sphere and adopt a radial profile somewhat smoother.  For the Uervirojnangkoorn (2015) paper
  we used a profile based on a Lorentzian function.  The derivation is as follows.

\par A suggestion from James Holton defines the Bragg spot partiality as

  \begin{equation}
    p = \frac{\text{Area of intersection between the Ewald sphere and } F_{hkl}}
             {\text{Area of intersection between the Ewald sphere and } F_{000}}
    \text{.}
    \label{eqn:part}
  \end{equation}

The "areas of intersection" in question are really spherical caps that represent the Ewald
sphere's intersection with the reciprocal space ball of radius $r_s$.  However, we're not going to
insist on such detail; instead we will simply take a circular area of radius $r_p$ such that we
have the right triangle

  \begin{equation}
    {r}_p^2 = {r}_s^2 - {r}_h^2
    \text{,}
    \label{eqn:pyth}
  \end{equation}

and then the approximate expression for partiality becomes (model A),

  \begin{equation}
    p_A = \frac{\pi r_p^2}{\pi r_s^2} = 1 - \frac{r_h^2}{r_s^2} \text{ for }|r_h|<r_s, 0 \text{ otherwise}
    \text{.}
    \label{eqn:fexprr}
  \end{equation}

Partiality as a function of $r_h$ is a simple inverted parabola with $p_A=1$ at $r_h=0$ and
roots at $\pm r_s$ (Fig. 2).  Outside of this domain the partiality is 0.

  \begin{figure}[htb!]
  \begin{center}
  \includegraphics[scale=0.65]{Figure_2.pdf}
  %\includesvg{Figure_2}
  \label{fig:2}
  \end{center}
  \begin{center}
  {Fig. 2. Three partiality models:
  a simple hard-sphere model ($p_A$, blue), a soft-sphere Lorentzian
  function ($p_B$, red), and an intermediate model based on a Gaussian function
  ($p_G$, green).  }
  \end{center}
  \end{figure}

However, having a
mathematically discontinuous
expression ($p_A$) will leave us at a disadvantage for postrefinement.  The postrefinement strategy
will be to express the lack of closure $r_h$ in terms of model parameters such as the unit cell
dimensions and crystal orientation.  Then optimize a target function $f$ expressed in
terms of the partiality $p$, attempting to find parameter values to minimize $f$.  It is
crucial in this procedure to have an expression for partiality that is smooth and differentiable.

We will therefore have to modify our simple model of the Bragg spot as a reciprocal space ball.
One functional form that might have the desired properties is the Lorentzian function
  \begin{equation}
    L = \frac{1}{\pi}\frac{\frac{1}{2}\Gamma}{x^2 + (\frac{1}{2}\Gamma)^2}
    \text{,}
    \label{eqn:loren}
  \end{equation}
where $\Gamma$ is the full-width at half maximum (FWHM).

Let's tinker with this expression so it conforms to expectation...first multiply by
a scaling constant to get $L'(0)=1$:
  \begin{equation}
    L^{\prime} = \frac{\pi \Gamma}{2}L
    \text{,}
    \label{eqn:lorenA}
  \end{equation}

and finally setting the FWHM to the FWHM value obtained from eqn \eqref{eqn:fexprr},
  \begin{equation}
    \Gamma = \frac{2r_s}{\sqrt{2}}
    \text{,}
    \label{eqn:fwhm}
  \end{equation}
so we get a new partiality expression (model B):

  \begin{equation}
    p_B = \frac{r_s^2}{2r_h^2 + r_s^2}
    \text{.}
    \label{eqn:pb}
  \end{equation}

Finally, for postrefinement we'll need the partial derivative of $p_B$ with respect to $r_h$ (use the
quotient rule):
  \begin{equation}
    \frac{\partial{p_B}}{\partial{r_h}} = \frac{-4r_s^2r_h}{(2r_h^2 + r_s^2)^2}
    \text{.}
    \label{eqn:deriv}
  \end{equation}

  \subsection*{3 Model parameters and target function}
  \par The goal of this work is to refine the parameters of the partiality model so that the
  observed intensities, corrected to their full spot equivalents, offer the best agreement over
  repeated measurements of the same asymmetric-unit Miller index.  In practice,
  the parameters representing each crystal lattice are refined
  against a set of reference intensities $I_{\mathrm{ref}}$.
  Program {\it prime} uses simple scaling to create an initial reference, after which repeated
  cycles of postrefinement are performed, with the reference being created from the corrected,
  merged intensities from the previous cycle.  In {\it cxi.merge} the reference is an isomorphous
  atomic structure, from which intensities $I_{\mathrm{ref}}$ are calculated, and only one
  cycle is performed.  The polarization-corrected measurements are denoted $I_{\mathrm{obs}}$.
  The parameters refined for each crystal lattice are:

  \par $G$: the scale factor.
  \par $B$: the Wilson $B$-factor.
  \par $\theta_{x}$: incremental crystal rotation angle on $x$-axis ($\perp$ to beam).
  \par $\theta_{y}$: incremental crystal rotation angle on $y$-axis ($\perp$ to beam).
  \par The least-squares target function used to achieve best agreement between model and
  observation is

  \begin{equation}
 \mathscr{F} =  \sum_{i} \limits
    ( G  \exp(\dfrac{-8B\sin^2\theta}{\lambda^2}) p_B I_{\mathrm{ref}} - I_{\mathrm{obs}})^{2}
\end{equation}
where $\theta$ is the Bragg diffraction angle defined in Fig. 1, and $\lambda$ the wavelength, both
treated as constants, and the sum is over all measurements integrated from a single crystal lattice.

  \subsection*{4 Necessary derivatives for parameter refinement}

  \par Given the least-squares form, derivatives of the target functional with respect to
  parameter $\mathscr{P}$ are in general

    \begin{equation}
    \frac{\partial\mathscr{F}}{\partial\mathscr{P}} = 2\sum_{i} \limits
    \mathscr{R}_i\dfrac{\partial\mathscr{R}_i}{\partial\mathscr{P}}
    \text{,}
    \label{eqn:genFP}
  \end{equation}
where the residual comparison on each observation is

  \begin{equation}
 \mathscr{R}_i =
    G  \exp(\dfrac{-8B\sin^2\theta}{\lambda^2}) p_B I_{\mathrm{ref}} - I_{\mathrm{obs}}
    \text{.}
    \end{equation}
The derivatives in the Jacobian matrix, required for parameter optimization, are more-or-less
straightforward:

    \begin{equation}
    \frac{\partial\mathscr{R}_i}{\partial G} =
    \exp(\dfrac{-8B\sin^2\theta}{\lambda^2}) p_B I_{\mathrm{ref}}
    \text{,}
    \label{eqn:dG}
  \end{equation}

     \begin{equation}
    \frac{\partial\mathscr{R}_i}{\partial B} =
    G  \exp(\dfrac{-8B\sin^2\theta}{\lambda^2}) p_B I_{\mathrm{ref}}
    \left( \dfrac{-8\sin^2\theta}{\lambda^2}\right)
    \text{.}
    \label{eqn:dB}
  \end{equation}

  \par The derivatives with respect to $\theta_{x}$ and $\theta_{y}$ require more work.  All of the
  dependence on crystal orientation comes through the expression for partiality:

      \begin{equation}
    \frac{\partial\mathscr{R}_i}{\partial \theta_{x\mathrm{|}y}} =
  \frac{\partial\mathscr{R}_i}{\partial p_B}
   \frac{\partial p_B} {\partial \theta_{x\mathrm{|}y}}
    \text{,}
    \label{eqn:dthxy}
  \end{equation}

with

      \begin{equation}
  \frac{\partial\mathscr{R}_i}{\partial p_B} =
  G  \exp(\dfrac{-8B\sin^2\theta}{\lambda^2}) I_{\mathrm{ref}}
    \text{.}
    \label{eqn:P_terms}
  \end{equation}

As for the variation of the partiality model $p_B$ defined in \eqref{eqn:pb}, the {\tt rs}
model assumes that the sphere radius $r_s$ is fixed, thus the only remaining variable is the
distance  $r_h$ between RLP and Ewald sphere:

   \begin{equation}
   \frac{\partial p_B} {\partial \theta_{x\mathrm{|}y}} =
  \frac{\partial p_B}{\partial r_h}
   \frac{\partial r_h} {\partial \theta_{x\mathrm{|}y}}
    \text{.}
    \label{eqn:dpb2}
  \end{equation}

  \par An expression for $\dfrac{\partial p_B}{\partial r_h}$ has already been
  given in \eqref{eqn:deriv}, so it now
  remains to investigate the derivative $\dfrac{\partial r_h} {\partial \theta_{x\mathrm{|}y}}$,
  based on the definition of $r_h$ given in \eqref{eqn:Rh}.

Introduce the vector $\mathbf{S}$:
   \begin{equation}
   \mathbf{S} = \mathbf{q}+\mathbf{s_0}
    \text{,}
    \label{eqn:SS}
  \end{equation}

   \begin{equation}
 {r_h} = ||\mathbf{S}||-\dfrac{1}{ \lambda}
     \text{,}
    \label{eqn:sss}
  \end{equation}

    \begin{equation}
   \dfrac{\partial r_h} {\partial \theta_{x\mathrm{|}y}} =
   \dfrac
   {\mathbf{S}\cdot{
   \dfrac{\partial \mathbf{S}} {\partial \theta_{x\mathrm{|}y}}
   }}
   {||\mathbf{S}||}
    \text{,}
    \label{eqn:rhth}
  \end{equation}

    \begin{equation}
   \dfrac{\partial \mathbf{S}} {\partial \theta_{x\mathrm{|}y}} =
  {
   \dfrac{\partial \mathbf{q}} {\partial \theta_{x\mathrm{|}y}}
   }
    \text{.}
    \label{eqn:derivsq}
  \end{equation}

Finally we investigate the derivative of the RLP position $\mathbf{q}$ with respect to the crystal
rotations.  The effective orientation matrix $\mathbf{A}$ may be expressed as the
reference orientation matrix $\mathbf{A}_\mathrm{ref}$ determined during crystal refinement and
integration, composed with additional rotational operators $\mathbb{R}_x$ and
$\mathbb{R}_y$ determined by the postrefined angles $\theta_{x}$ and
$\theta_{y}$:

    \begin{equation}
   \mathbf{A} =
\mathbb{R}_y( \mathbb{R}_x ( \mathbf{A}_\mathrm{ref} )  )
    \text{.}
    \label{eqn:compoase}
  \end{equation}

The derivatives of $\mathbf{q}$ work out as follows:

  \begin{equation}
   \dfrac{\partial \mathbf{q}} {\partial \theta_{x}} =
  \mathbb{R}_y \dfrac{\partial\mathbb{R}_x}{\partial \theta_{x}}  \mathbf{A}_\mathrm{ref} \mathbf{h}
    \text{,}
    \label{eqn:qthx}
  \end{equation}
  \begin{equation}
   \dfrac{\partial \mathbf{q}} {\partial \theta_{y}} =
  \dfrac{\partial\mathbb{R}_y}{\partial \theta_{y}} \mathbb{R}_x \mathbf{A}_\mathrm{ref} \mathbf{h}
    \text{.}
    \label{eqn:qthy}
  \end{equation}

The derivatives of the rotation operator are already encoded in the cctbx library ({\tt scitbx/matrix/\_\_init\_\_.py}).  Formulae for the rotation operator and its derivative with
respect to angle $\theta$ are given in the LaTeX documentation included in that directory.

  \subsection*{5 The model { \tt rs\_hybrid}: Additional refinement of the parameter $r_s$ }
After refining the parameters $G$, $B$, $\theta_{x}$, and $\theta_{y}$
we now decide to add a second minimization round to refine an additional parameter for each crystal lattice:
$r_s$, the RLP radius, as shown in Fig. 1.

\par We thus need the derivative of the residual $\mathscr{R}_i$ with respect to this new parameter:

   \begin{equation}
    \frac{\partial\mathscr{R}_i}{\partial r_s} =
  \frac{\partial\mathscr{R}_i}{\partial p_B}
   \frac{\partial p_B} {\partial r_s}
    \text{,}
    \label{eqn:dRrs}
  \end{equation}

where  $\dfrac{\partial\mathscr{R}_i}{\partial p_B}$ has already been given by Eqn. \eqref{eqn:P_terms}.

The remaining factor is derived from the partiality expression associated with the Lorentzian RLP
profile, Eqn. \eqref{eqn:pb}.  Using the quotient rule, with $N$=numerator and $D$=denominator,

   \begin{equation}
    \frac{\partial p_B} {\partial r_s} =
  \frac{
      \dfrac{\partial N}{\partial{r_s}} D-
      \dfrac{\partial D}{\partial{r_s}} N
    }
    {D^2}
    =
   \frac{4{r_s}{r_h}^2}{(2r_h^2 + r_s^2)^2}
    \text{.}
    \label{eqn:dpb_drs}
  \end{equation}

  \subsection*{6 A Gaussian-shaped radial profile for the RLP }
One criticism of the RLP model B and its Lorentzian-shaped radial profile $p_B (r_h)$ is that the
profile tails off very gradually.  There is still a significant partiality fraction many radii away from the
RLP center.  Perhaps this is unphysical; after all, we are trying to test the hypothesis that negative
measurements are in fact false predictions that are in fact too far from the Ewald sphere to contribute
any diffracted signal.  Therefore, let's choose a model that would provide more sharp delineation
between RLP and not-RLP.  We'll then test if this model fits the data better, presumably by looking
at the target functional or the correlation coefficient.
A candidate function giving a sharper cutoff, while still being smoothly
differentiable, is the Gaussian,

   \begin{equation}
    G =
  \frac{1}{\sqrt{2\pi\sigma^2}}
   \exp (- \dfrac {(x - \mu)^2}{2\sigma^2})
    \text{,}
    \label{eqn:gaussian}
  \end{equation}

with mean $\mu$ and standard deviation $\sigma$.

As before, we develop a modified function that has an amplitude of 1 at a reciprocal-distance offset $r_h$ of 0:

   \begin{equation}
    G^{\prime} =
   \exp (- \dfrac {r_h^2}{2\sigma^2})
    \text{.}
    \label{eqn:gaussprime}
  \end{equation}

To eliminate the variable $\sigma$ we set the FWHM of function \eqref{eqn:gaussprime} ,
   \begin{equation}
   2 \sqrt{(2 \ln{2})\sigma^2}
    \text{,}
    \label{eqn:fwhmgaussprime}
  \end{equation}

to be equal to the FWHM expression worked out for models A and B in eqn \eqref{eqn:fwhm}. From this
condition we can work out the value of the variance

   \begin{equation}
   \sigma^2 =
    \dfrac {r_s^2} {4\ln{2}}
    \text{,}
    \label{eqn:gsigma}
  \end{equation}

and now eliminate $\sigma$ to arrive at a new expression for the partiality (model G):

   \begin{equation}
  p_G =
    \exp( \dfrac {-(2 \ln{2}) r_h^2} {r_s^2}
    )
    \text{,}
    \label{eqn:pg}
  \end{equation}

This function is plotted in Fig. 2 (green dots), illustrating that $p_G$ is a better
approximation to the
hard-sphere RLP model (blue) than is $p_B$ (red).  Finally, for parameter refinement we need the
partial derivatives of $p_G$ with respect to its constituent variables,

  \begin{equation}
    \frac{\partial{p_G}}{\partial{r_h}} =
    - p_G \frac {(4 \ln{2}) r_h}{r_s^2}
    \text{, }
    \label{eqn:dpG_drh}
  \end{equation}
and
  \begin{equation}
    \frac{\partial{p_G}}{\partial{r_s}} = p_G\frac{(4 \ln{2})r_h^2}{r_s^3}
    \text{.}
    \label{eqn:dpG_drs}
  \end{equation}

%%%%%%%%%%%%%%%%%%%%%
\end{document}


\documentclass[a4paper]{article}

\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{xcolor}
\usepackage{amsthm}
\usepackage[mathcal]{euscript}

\usepackage{url}

\newcommand{\Hcal}{\mathcal{H}}
\newcommand{\real}{\mathbb{R}}

\title{Some notes on Sokal (2010)}
\author{Nazarov Ivan}

\date{\today}

\begin{document}
\maketitle

\section*{Uniform Boundedness Principle} % (fold)
\label{sec:uniform_boundedness_principle}

In this section we treat the Hilbert space $\Hcal$ with the natural inner product
norm as a Banach space $(\Hcal, \|\cdot\|)$. Consider a collection $(l_n)_{n\geq1}$
of bounded linear functionals $\Hcal\to\real$ that is bounded at every point:
$\sup_{n\geq1} \lvert l_n(x)\rvert < +\infty$ for any $x \in \Hcal$. The following
section follows the proof given in Sokal (2010).\footnotemark
\footnotetext{arXiv:1005.1585 -- \url{https://arxiv.org/abs/1005.1585v2}}

\paragraph{Lemma} % (fold)
\label{par:lemma}

For any $x\in \Hcal$ and $\delta > 0$ we have $\delta \|l\| \leq \sup_{z\in B(x, \delta)}
\lvert l(z) \rvert$, where $B(x, \delta) = \{z\in \Hcal\colon \|z-x\| \leq \delta\}$.
Indeed, for any $\|h\| \leq 1$ we have
\begin{equation*}
  \lvert l(h) \rvert
    = \tfrac1\delta \lvert l(\delta h) \rvert
    \leq \tfrac1{2 \delta} \bigl(
      \lvert l(x + \delta h) \rvert + \lvert l(x - \delta h) \rvert
    \bigr)
    \leq \tfrac1{2 \delta}
      2 \sup\{\lvert l(z) \rvert \colon z\in B(x, \delta)\}
      \,,
\end{equation*}
implying that the upper bound on rhs is not less than the supremum of the lhs.

% paragraph lemma (end)

\paragraph{Proof} % (fold)
\label{par:proof}

We proceed by contradiction and suppose that $\sup_{n\geq1} \|l_n\| = \infty$.

Take a sequence $L_k \uparrow +\infty$, and a pair of values $\eta, q \in (0, 1)$,
which will be specified and discussed along the proof. There exists $(n_k)_{k\geq1}
\uparrow$ such that $\|l_{n_k}\| > L_k$. Let $x_0 = 0 \in \Hcal$ and $\delta_k = q^k$.
For any $k\geq 1$ the norm $\|l_{n_k} \|$ is finite, therefore the lemma implies
\begin{equation*}
  \delta_k \eta \|l_{n_k}\|
    < \delta_k \|l_{n_k}\|
    \leq \sup\{\lvert l(z) \rvert \colon z\in B(x_{k-1}, \delta_k)\}
      \,,
\end{equation*}
which further implies the existence of $x_k \in B(x_{k-1}, \delta_k)$ with $\delta_k
\eta \|l_{n_k}\| < \lvert l_{n_k}(x_k) \rvert$. Thus, we define a sequence $(x_k)_{k\geq1}
\in \Hcal$ with $\|x_k - x_{k-1}\| \leq \delta_k$ and the just mentioned bound on
$\|l_{n_k}\|$. The constructed sequence is Cauchy: for any $p \geq k$
\begin{equation*}
  \|x_p - x_k \|
    \leq \sum_{i=k+1}^p \|x_i - x_{i-1} \|
    % \leq \sum_{i=1}^{p-k} \delta_{i+k}
    \leq \sum_{i=1}^{p-k} q^{i+k}
    % = q^{k+1} \tfrac{1 - q^{p-k}}{1-q}
    < \frac{q \delta_k}{1-q}
    \,.
    % 1 - q^n = 1 + q^1 + .. + q^{n-1} - (q^1 + q^2 + .. + q^n)
    %  = (1 - q)(1 + q^1 + .. + q^{n-1})
\end{equation*}
Since $\Hcal$ is complete, $\exists x_*\in \Hcal$ such that $\|x_k - x_*\| \to 0$.
For any $p \geq k$ we get
\begin{equation*}
  \|x_k - x_* \|
    \leq \| x_p - x_*\| + \sum_{i=k+1}^p \|x_i - x_{i-1} \|
    \leq \| x_p - x_*\| + \frac{q \delta_k}{1-q}
    \,,
\end{equation*}
and conclude that $\|x_k - x_* \| \leq \frac{q \delta_k}{1-q}$ for any $k\geq 1$.
Observe that for all $k\geq 1$
\begin{align*}
  \delta_k \eta \|l_{n_k} \|
    &< \lvert l_{n_k}(x_k) \rvert
      \leq \lvert l_{n_k}(x_*) \rvert
        + \lvert l_{n_k}(x_k - x_*) \rvert
    \\
    &\leq \lvert l_{n_k}(x_*) \rvert
      + \|x_k - x_*\| \|l_{n_k}\|
    \\
    &\leq \lvert l_{n_k}(x_*) \rvert
      + \tfrac{q}{1-q} \delta_k \|l_{n_k}\|
      \,,
\end{align*}
whence $\bigl(\eta - \tfrac{q}{1-q} \bigr) \delta_k \|l_{n_k}\| \leq \lvert l_{n_k}(x_*) \rvert$
for all $k\geq 1$.

Here we shall make the necessary specifications, that, despite affecting the particular
values, do not alter the key properties of the constructed sequence. We assume that
$1 > \eta > \tfrac{q}{1-q}$ (this implies $q < \tfrac\eta{1 + \eta}$). Since $0 \leq
L_k < \|l_{n_k}\|$ we finally get the following lower bound:
\begin{equation*}
  \tilde{L}_k
    = \bigl(\eta - \tfrac{q}{1-q} \bigr) q^k L_k
    \leq \lvert l_{n_k}(x_*) \rvert
    \,.
\end{equation*}
We are also free to require that the original $L_k \uparrow \infty$ be such that
$\tilde{L}_k \uparrow \infty$. For such choice we get the following statement: there
is $(n_k)_{k\geq1} \uparrow$ and $x_* \in \Hcal$ such that $\sup_{k\geq 1} \lvert
l_{n_k}(x_*) \rvert = +\infty$. This contradicts the original assumption that
$\sup_{n\geq 1}\lvert l_n(x) \rvert$ is bounded for any given $x\in \Hcal$.

So is it possible to meet all the requirements, outlined in the specifications above?
It is, if we pick any $\alpha, \beta > 1$ and set
\begin{itemize}
  \item $q = \tfrac1{2 (1 + \alpha)}$ and $\eta = \tfrac{\alpha q}{1-q}$
  \item $L_k = \bigl(\eta - \tfrac{q}{1-q}\bigr)^{-1} q^{-k} \beta^k$
\end{itemize}
then $\eta < 1$, $\eta - \tfrac{q}{1-q} = \tfrac{\alpha - 1}{1 + 2\alpha} > 0$,
$L_k = \tfrac{1 + 2\alpha}{\alpha - 1} (2 (1 + \alpha) \beta)^k \to \infty$,
$\delta_k = q^k \to 0$, and $\tilde{L}_k = \beta^k \to \infty$.

% paragraph proof (end)

\paragraph{Other proofs} % (fold)
\label{par:other_proofs}

There is another proof of this principle, using explicitly the continuity of the
linear functionals and the Baire Category theorem, to get a point that violates
the orignal assumptions.

% paragraph other_proofs (end)

\paragraph{Notes} % (fold)
\label{par:notes}

The uniform boundedness principle holds for arbitrarily large collection of linear
functionals and can be extended to boundned linear operators.

% paragraph notes (end)

% section* uniform_boundedness_principle (end)

\end{document}


\documentclass[12pt]{cdblatex}
\usepackage{fancyhdr}
\usepackage{footer}

\begin{document}

% =================================================================================================
% create checkpoint file

\bgroup
\CdbSetup{action=hide}
\begin{cadabra}
   import cdblib
   checkpoint_file = 'tests/semantic/output/rnc2gen.json'
   cdblib.create (checkpoint_file)
   checkpoint = []
\end{cadabra}
\egroup

% =================================================================================================
\section*{Convert from rnc to generic coordinates}

The following code is based on the {\tt gen2rnc.tex} code.

It is common to do some computations in a local RNC. Doing so makes various parts of the
computations much easier to manage than in the original non-RNC coordinates. One simple example
is the proof of the second Bianchi identities.

This code develops the inverse transformation, that is from the local RNC coordinates back to
generic coordinates. The key equation (drawn form {\tt gen2rnc.tex}) is
\begin{align}
   x^a_j = x^a_i + y^a - \sum_{k=2}^\infty\>\frac{1}{k!}\>\Gamma^{a}_{\ubk}y^{.\ubk}
\end{align}
In {\tt gen2rnc.tex} this equation was solved for the RNC coordinates $y$ given the generic
coordinates $x_j$ and $x_i$. Here we will instead take $x_i$ and $y$ as given and use this
equation to compute $x_j$. The first change we will make is to replace $x_j$ with $x$ (as the
subscript $j$ serves no useful purpose).

Thus our job will be to compute
\begin{align}
   \label{eq:rnc2xyz}
   x^a = x^a_i + y^a - \sum_{k=2}^\infty\>\frac{1}{k!}\>\Gamma^{a}_{\ubk}y^{.\ubk}
\end{align}
given $x_i$ and $y$. The generalised connections will be computed recursively by
\begin{align}
   \label{eq:GenGamma}
   \Gamma^{a}{}_{b\uc d} = \Gamma^{a}{}_{(b\uc,d)}
                   - (n+1) \Gamma^{a}{}_{p(\uc}
                           \Gamma^{p}{}_{bd)}
\end{align}

As noted in {\tt gen2rnc.tex}, the generalised connections will scale with the expensions
parameter $\eps$ according to
\begin{gather*}
   \Gamma^{a}{}_{bc} = \BigO{\eps}\>,\qquad
   \Gamma^{a}{}_{bcd} = \BigO{\eps^2}\>,\qquad
   \Gamma^{a}{}_{bcde} = \BigO{\eps^3}\>,\qquad
   \text{etc.}
\end{gather*}

\clearpage

% ===================================================================

\begin{cadabra}
   {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w#}::Indices(position=independent).

   D{#}::Derivative.
   \nabla{#}::Derivative.
   \partial{#}::PartialDerivative.

   g_{a b}::Metric.
   g^{a b}::InverseMetric.
   g_{a}^{b}::KroneckerDelta.
   g^{a}_{b}::KroneckerDelta.
   \delta^{a}_{b}::KroneckerDelta.
   \delta_{a}^{b}::KroneckerDelta.

   R_{a b c d}::RiemannTensor.
   R^{a}_{b c d}::RiemannTensor.
   R_{a b c}^{d}::RiemannTensor.

   A^{a}::Depends(\partial{#}).

   g_{a b}::Depends(\partial{#}).
   R_{a b c d}::Depends(\partial{#}).
   R^{a}_{b c d}::Depends(\partial{#}).

   Q^{a}_{b c}::Depends(\partial{#}).

   Q^{a}_{b c}::TableauSymmetry(shape={2}, indices={1,2}).
   Q^{a}_{b c d}::TableauSymmetry(shape={3}, indices={1,2,3}).
   Q^{a}_{b c d e}::TableauSymmetry(shape={4}, indices={1,2,3,4}).
   Q^{a}_{b c d e f}::TableauSymmetry(shape={5}, indices={1,2,3,4,5}).
   Q^{a}_{b c d e f g}::TableauSymmetry(shape={6}, indices={1,2,3,4,5,6}).

   Q^{p}_{a b}::Weight(label=numQ,value=1).
   Q^{p}_{a b c}::Weight(label=numQ,value=2).
   Q^{p}_{a b c d}::Weight(label=numQ,value=3).
   Q^{p}_{a b c d e}::Weight(label=numQ,value=4).
   Q^{p}_{a b c d e f}::Weight(label=numQ,value=5).

   def product_sort (obj):
       substitute (obj,$ A^{a}                     -> A001^{a}               $)
       substitute (obj,$ x^{a}                     -> A002^{a}               $)
       substitute (obj,$ g^{a b}                   -> A003^{a b}             $)
       substitute (obj,$ Q^{p}_{a b}               -> A004^{p}_{a b}         $)
       substitute (obj,$ Q^{p}_{a b c}             -> A005^{p}_{a b c}       $)
       substitute (obj,$ Q^{p}_{a b c d}           -> A006^{p}_{a b c d}     $)
       substitute (obj,$ Q^{p}_{a b c d e}         -> A007^{p}_{a b c d e}   $)
       substitute (obj,$ Q^{p}_{a b c d e f}       -> A008^{p}_{a b c d e f} $)
       sort_product   (obj)
       rename_dummies (obj)
       substitute (obj,$ A001^{a}                  -> A^{a}                  $)
       substitute (obj,$ A002^{a}                  -> x^{a}                  $)
       substitute (obj,$ A003^{a b}                -> g^{a b}                $)
       substitute (obj,$ A004^{p}_{a b}            -> Q^{p}_{a b}            $)
       substitute (obj,$ A005^{p}_{a b c}          -> Q^{p}_{a b c}          $)
       substitute (obj,$ A006^{p}_{a b c d}        -> Q^{p}_{a b c d}        $)
       substitute (obj,$ A007^{p}_{a b c d e}      -> Q^{p}_{a b c d e}      $)
       substitute (obj,$ A008^{p}_{a b c d e f}    -> Q^{p}_{a b c d e f}    $)

       return obj

   def truncateQ (obj,n):

       ans = Ex(0)

       for i in range (0,n+1):
          foo := @(obj).
          bah  = Ex("numQ = " + str(i))
          keep_weight (foo, bah)
          ans = ans + foo

       return ans

   # A^{a} = dx^a/ds

   Gamma := Q^{d}_{a b} A^{a} A^{b}.

   dAds  := A^{c} \partial_{c}{A^{d}}-> - @(Gamma).

   # =============================================================
   eq0 := @(Gamma).                        # cdb (eq0.000,eq0)

   # =============================================================
   eq1 := A^{c} \partial_{c}{@(eq0)}.      # cdb (eq1.000,eq1)

   distribute      (eq1)                   # cdb (eq1.001,eq1)
   unwrap          (eq1)                   # cdb (eq1.002,eq1)
   product_rule    (eq1)                   # cdb (eq1.003,eq1)
   distribute      (eq1)                   # cdb (eq1.004,eq1)
   substitute      (eq1,dAds)              # cdb (eq1.005,eq1)
   distribute      (eq1)                   # cdb (eq1.006,eq1)
   eq1 = truncateQ (eq1,5)                 # cdb (eq1.007,eq1)
   sort_product    (eq1)                   # cdb (eq1.008,eq1)
   rename_dummies  (eq1)                   # cdb (eq1.009,eq1)
   canonicalise    (eq1)                   # cdb (eq1.010,eq1)

   # =============================================================
   eq2 := A^{c} \partial_{c}{@(eq1)}.      # cdb (eq2.000,eq2)

   distribute      (eq2)                   # cdb (eq2.001,eq2)
   unwrap          (eq2)                   # cdb (eq2.002,eq2)
   product_rule    (eq2)                   # cdb (eq2.003,eq2)
   distribute      (eq2)                   # cdb (eq2.004,eq2)
   substitute      (eq2,dAds)              # cdb (eq2.005,eq2)
   distribute      (eq2)                   # cdb (eq2.006,eq2)
   eq2 = truncateQ (eq2,5)                 # cdb (eq2.007,eq2)
   sort_product    (eq2)                   # cdb (eq2.008,eq2)
   rename_dummies  (eq2)                   # cdb (eq2.009,eq2)
   canonicalise    (eq2)                   # cdb (eq2.010,eq2)

   # =============================================================
   eq3 := A^{c} \partial_{c}{@(eq2)}.      # cdb (eq3.000,eq3)

   distribute      (eq3)                   # cdb (eq3.001,eq3)
   unwrap          (eq3)                   # cdb (eq3.002,eq3)
   product_rule    (eq3)                   # cdb (eq3.003,eq3)
   distribute      (eq3)                   # cdb (eq3.004,eq3)
   substitute      (eq3,dAds)              # cdb (eq3.005,eq3)
   distribute      (eq3)                   # cdb (eq3.006,eq3)
   eq3 = truncateQ (eq3,5)                 # cdb (eq3.007,eq3)
   sort_product    (eq3)                   # cdb (eq3.008,eq3)
   rename_dummies  (eq3)                   # cdb (eq3.009,eq3)
   canonicalise    (eq3)                   # cdb (eq3.010,eq3)

   # =============================================================
   eq4 := A^{c} \partial_{c}{@(eq3)}.      # cdb (eq4.000,eq4)

   distribute      (eq4)                   # cdb (eq4.001,eq4)
   unwrap          (eq4)                   # cdb (eq4.002,eq4)
   product_rule    (eq4)                   # cdb (eq4.003,eq4)
   distribute      (eq4)                   # cdb (eq4.004,eq4)
   substitute      (eq4,dAds)              # cdb (eq4.005,eq4)
   distribute      (eq4)                   # cdb (eq4.006,eq4)
   eq4 = truncateQ (eq4,5)                 # cdb (eq4.007,eq4)
   sort_product    (eq4)                   # cdb (eq4.008,eq4)
   rename_dummies  (eq4)                   # cdb (eq4.009,eq4)
   canonicalise    (eq4)                   # cdb (eq4.010,eq4)

\end{cadabra}

\clearpage
\begin{dgroup*}
   \begin{dmath*} \cdb*{eq0.000} \end{dmath*}
\end{dgroup*}

\clearpage
\begin{dgroup*}
   \begin{dmath*} \cdb*{eq1.000} \end{dmath*}
   \begin{dmath*} \cdb*{eq1.001} \end{dmath*}
   \begin{dmath*} \cdb*{eq1.002} \end{dmath*}
   \begin{dmath*} \cdb*{eq1.003} \end{dmath*}
   \begin{dmath*} \cdb*{eq1.004} \end{dmath*}
   \begin{dmath*} \cdb*{eq1.005} \end{dmath*}
   \begin{dmath*} \cdb*{eq1.006} \end{dmath*}
   \begin{dmath*} \cdb*{eq1.007} \end{dmath*}
   \begin{dmath*} \cdb*{eq1.008} \end{dmath*}
   \begin{dmath*} \cdb*{eq1.009} \end{dmath*}
   \begin{dmath*} \cdb*{eq1.010} \end{dmath*}
\end{dgroup*}

\clearpage
\begin{dgroup*}
   \begin{dmath*} \cdb*{eq2.000} \end{dmath*}
   \begin{dmath*} \cdb*{eq2.001} \end{dmath*}
   \begin{dmath*} \cdb*{eq2.002} \end{dmath*}
   \begin{dmath*} \cdb*{eq2.003} \end{dmath*}
   \begin{dmath*} \cdb*{eq2.004} \end{dmath*}
   \begin{dmath*} \cdb*{eq2.005} \end{dmath*}
   \begin{dmath*} \cdb*{eq2.006} \end{dmath*}
   \begin{dmath*} \cdb*{eq2.007} \end{dmath*}
   \begin{dmath*} \cdb*{eq2.008} \end{dmath*}
   \begin{dmath*} \cdb*{eq2.009} \end{dmath*}
   \begin{dmath*} \cdb*{eq2.010} \end{dmath*}
\end{dgroup*}

\clearpage
\begin{dgroup*}
   \begin{dmath*} \cdb*{eq3.010} \end{dmath*}
\end{dgroup*}

\clearpage
\begin{dgroup*}
   \begin{dmath*} \cdb*{eq4.010} \end{dmath*}
\end{dgroup*}

% -------------------------------------------------------------------
% reformat

\clearpage

\begin{cadabra}
   def reformat (obj):
      bah := @(obj).
      distribute     (bah)
      bah = product_sort (bah)
      rename_dummies (bah)
      canonicalise   (bah)
      factor_out     (bah,$A^{a?}$)
      substitute     (bah,$A^{a}->y^{a}$)
      substitute     (bah,$Q^{a}_{b c}->\Gamma^{a}_{b c}$)
      ans := @(bah).
      return ans

   eq0 = reformat(eq0)  # cdb (eq0.100,eq0)
   eq1 = reformat(eq1)  # cdb (eq1.100,eq1)
   eq2 = reformat(eq2)  # cdb (eq2.100,eq2)
   eq3 = reformat(eq3)  # cdb (eq3.100,eq3)
   eq4 = reformat(eq4)  # cdb (eq4.100,eq4)

   checkpoint.append (eq0)
   checkpoint.append (eq1)
   checkpoint.append (eq2)
   checkpoint.append (eq3)
   checkpoint.append (eq4)

\end{cadabra}

\clearpage

% =================================================================================================
\section*{Convert from local RNC coords (y) to generic (x)}

\begin{align*}
   x^a = x^a_i + \nx{0}^{a} - \nx{1}^{a} - \nx{2}^{a} - \nx{3}^{a} - \nx{4}^{a} - \nx{5}^{a}
\end{align*}

\begin{dgroup*}
   \begin{dmath*}    \nx{0}^{a} = y^{a} \end{dmath*}
   \begin{dmath*} 2! \nx{1}^{a} = \cdb{eq0.100} \end{dmath*}
   \begin{dmath*} 3! \nx{2}^{a} = \cdb{eq1.100} \end{dmath*}
   \begin{dmath*} 4! \nx{3}^{a} = \cdb{eq2.100} \end{dmath*}
   \begin{dmath*} 5! \nx{4}^{a} = \cdb{eq3.100} \end{dmath*}
   \begin{dmath*} 6! \nx{5}^{a} = \cdb{eq4.100} \end{dmath*}
\end{dgroup*}

% =================================================================================================
% export checkpoints in json format

\bgroup
\CdbSetup{action=hide}
\begin{cadabra}
   for i in range( len(checkpoint) ):
      cdblib.put ('check{:03d}'.format(i),checkpoint[i],checkpoint_file)
\end{cadabra}
\egroup

\end{document}


\documentclass[]{article}
\usepackage{amsmath}

\hbadness=99999
\title{Linear Algebra: Week 1 Notes and Exercises}
\author{Christopher Aytona}

\begin{document}

\begin{center}
\Huge{Week 1 Notes and Exercises}
\end{center}

\section{Notes}
\textbf{Network Flow Diagrams}\\
$400 + x_2 = x_1$\\
$400 = x_1 - x_2$\\
$x_1 + x_3 - x_4 = 600$\\
$x_4 + x_5 = 100$\\
$x_2 + x_3 + x_5 = 300$\\

\textbf{Solutions to Systems of Linear Equations}\\
\textsf{A unique solution(Consistent)}\\
$x - 2y = -1$\\
$x = 2y-1$\\
$-2y+1+3y = 3$\\
$y = 2$\\
$x - 4 = -1$\\
$x = 3$\\
$(x, y) = (3, 2)$\\
\textsf{No solutions(Inconsistent)}\\
$x - 2y = -1$\\
$-x+2y=3$\\
$x = 2y-1$\\
$-2y+1+2y=3$\\
$0y=2$\\
\textsf{Infinitely many solutions(Consistent)}\\
$x-2y=-1$\\
$-x+2y=1$\\
$x=2y-1$\\
$-2y+1+2y=1$\\
$0y=0$\\

\textbf{Linear Equations}\\
\textsf{Method 1}\\
1) $x_1 - 2x_2 = -1$\\
2) $-x_1 + 3x_2 = 3$\\
Rewrite (1) as:\\
3) $x_1 = -1 + 2x_2$\\
Sub (3) into (2):\\
$-(-1+2x_2) + 3x_2 = 3$\\
$1 - 2x_2 + 3x_2 +2 = 3$\\
$1 + x_2 = 3$\\
$x_2 = 3 - 1$\\
4) $x_2 = 2$\\
Sub (4) into (1):\\
$x_1 - 2(2) = -1$\\
$x_1 - 4 = -1$\\
$x_1 = -1 + 4$\\
$x_1 = 3$\\
\textsf{Method 2}\\
1) $x_1 - 2x_2 + x_3 = 0$\\
2) $2x_2 - 8x_3 = 8$\\
3) $-4x_2 + 5x_2 + x_3 = -9$\\
Multiply (1) by 4\\
$4x_1 - 8x_2 + 4x_3 = 0$\\
$-4x_1 + 5x_2 + x_3 = -9$\\
Add both\\
$-3x_2 + 5x_3 = -9$\\
$2x_2 - 8x_3 = 8$\\
Make the coefficient the same(By multiplication)\\
$-6x_2 + 10x_3 = -18$\\
$6x_2 - 24x_3 = 24$\\
Addition\\
$-14x_3 = 6$\\
$x_3 = \frac{-6}{14}$\\

\textbf{Exercise}\\
\textsf{Problem 1}\\
1) $x_1 - 3x_3 = 8$\\
2) $2x_1 + 2x_2 + 9x_3 = 7$\\
3) $x_2 + 5x_3 = -2$\\
$x_1 = 8 + 3x_3$\\
$x_2 = -2 - 5x_3$\\
Sub\\
$2(8 + 3x_3) + 2(-2 - 5x_3) + 9x_3 = 7$\\
$16 + 6x_3 -4 -10x_3 + 9x_3 = 7$\\
$6x_3 - 10x_3 + 9x_3 = 7 - 16 + 4$\\
$5x_3 = -5$\\
$x_3 = -1$\\
$x_1 - 3(-1) = 8$\\
$x_1 = 8 - 3$\\
$x_1 = 5$\\
$2x_2 + 18(-1) = -9$\\
$2x_2 = 6$\\
$x_2 = 3$\\
$(x_1, x_2, x_3) = (5, 3, -1)$\\
\textsf{Problem 2}\\
1) $x_2 + 4x_3 = -5$\\
2) $x_1 + 3x_2 + 5x_3 = -2$\\
3) $3x_1 + 7x_2 + 7x_3 = 4$\\
$x_2 = -5 - 4x_3$\\
$x_1 = -3(-5 - 4x_3) - 5x_3$\\
$x_1 = 15 + 4x_3 - 5x_3$\\
$x_1 = -x_3 + 15$\\
$3(-x_3 + 15) + 7(-5 - 4x_3) + 7x_3 = 4$\\
$-3x_3 + 45 - 35 - 28x_3 + 7x_3 = 4$\\
$-24x_3 = -6$\\
$x_3 = \frac{1}{4}$\\
$x_2 + 4(\frac{1}{4}) = -5$\\
$x_2 = -6$\\
$x_1 + 3(-6) + 5(\frac{1}{4}) = -2$\\
$x_1 = -2 + 18 - \frac{5}{4}$\\
$x_1 = 15\frac{1}{4}$\\
$(x_1, x_2, x_3) = (15\frac{1}{4}, -6, \frac{1}{4})$\\

\textbf{Row Operations}\\
There are 3 types of row operations that can be performed on an augmented matrix without changing the solutions.\\
$\cdot$ Interchanging two rows.\\
$\cdot$ Scaling a row.\\
$\cdot$ Adding/Subtracting a row to another.\\

\textbf{Reduced Row Echelon Form}\\
$\cdot$ All non-zero rows are above any rows of all zeros.\\
$\cdot$ The leading non-zero coefficient of a non-zero row is a 1 and is strictly to the right of the leading 1 in the row above it.\\
$\cdot$ Each leading 1 is the only non-zero entry in that column.\\

\textbf{Matrices Example}\\
\[
\begin{bmatrix}
	1 & -2 & 1 & | & 0 \\
	0 & 2 & -8 & | & 8 \\
	-4 & 5 & 9 & | & -9
\end{bmatrix}
\]\\
$4R_1 + R_3$\\
\[
\begin{bmatrix}
	1 & -2 & 1 & | & 0 \\
	0 & 2 & -8 & | & 8 \\
	0 & -3 & 13 & | & -9
\end{bmatrix}
\]\\
$\frac{1}{2}R_2$\\
\[
\begin{bmatrix}
	1 & -2 & 1 & | & 0 \\
	0 & 1 & -4 & | & 4 \\
	0 & -3 & 13 & | & -9
\end{bmatrix}
\]\\
$3R_2 + R_3$\\
\[
\begin{bmatrix}
	1 & -2 & 1 & | & 0 \\
	0 & 1 & -4 & | & 4 \\
	0 & 0 & 1 & | & 3 \\
\end{bmatrix}
\]
$R_2 + 4R_3$\\
\[
\begin{bmatrix}
	1 & -2 & 1 & | & 0 \\
	0 & 1 & 0 & | & 16 \\
	0 & 0 & 1 & | & 3
\end{bmatrix}
\]\\
$R_1 - R_3$\\
\[
\begin{bmatrix}
	1 & -2 & 0 & | & -3 \\
	0 & 1 & 0 & | & 16 \\
	0 & 0 & 1 & | & 3
\end{bmatrix}
\]\\
\[
\begin{bmatrix}
	1 & 0 & 0 & | & 29 \\
	0 & 1 & 0 & | & 16 \\
	0 & 0 & 1 & | & 3
\end{bmatrix}
\]\\
$x_1 = 29 , x_2 = 16 , x_3 = 3$\\
\textsl{More examples}\\
\[
\begin{bmatrix}
	1 & 0 & 1 & | & 32 \\
	0 & 1 & 2 & | & 16 \\
	0 & 0 & 0 & | & 0
\end{bmatrix}
\]\\
It is a reduced row echelon.\\
Let $x_3 = S$\\
$x_1 = 32 - S$\\
$x_2 = 16 - 2S$\\
$x_3 = 0 + S$\\

\section{Exercise 1.1}
Solve each system in Exercise 1-4 by using elementary row operations on the equations or on the augmented matrix. Follow the systematic elimination procedure described in this section.\\

1) $x_1 + 5x_2 = 7$, $-2x_1 - 7x_2 = -5$\\
$x_1 = -5x_2 + 7$\\
$-2(-5x_2 + 7) - 7x_2 = -5$\\
$10x_2 - 14 - 7x_2 = -5$\\
$3x_2 = 9$\\
$x_2 = 3$\\
$x_1 = -5(3) + 7$\\
$x_1 = -15 + 7$\\
$x_1 = -8$\\
\[
\begin{bmatrix}
	1 & 5 & | & 7 \\
	-2 & -7& | & -5
\end{bmatrix}
\]\\
$R_2 + 2R_1$\\
\[
\begin{bmatrix}
	1 & 5 & | & 7 \\
	-2+2(1) & -7+2(5) & | & -5+2(7)
\end{bmatrix}=
\begin{bmatrix}
	1 & 5 & | & 7 \\
	0 & 3 & | & 9
\end{bmatrix}
\]\\
$\frac{1}{3}R_2$\\
\[
\begin{bmatrix}
	1 & 5 & | & 7 \\
	\frac{1}{3}(0) & \frac{1}{3(3)} & | & \frac{1}{3}(9)
\end{bmatrix}=
\begin{bmatrix}
	1 & 5 & | & 7 \\
	0 & 1 & | & 3
\end{bmatrix}
\]\\
$R_1 + -5R_2$\\
\[
\begin{bmatrix}
	1 + -5(0) & 5 + -5(1) & | & 7 + -5(3) \\
	0 & 1 & | & 3
\end{bmatrix}=
\begin{bmatrix}
	1 & 0 & | & -8 \\
	0 & 1 & | & 3
\end{bmatrix}
\]\\
$$(x_1,x_2) = (-8,3)$$\\

2) $3x_1 + 6x_2 = -3$, $5x_1 + 7x_2 = 10$\\
$3x_1 = -6x_2 - 3$\\
$x_1 = -2x_2 - 1$\\
$5(-2x_2 - 1) + 7x_2 = 10$\\
$-10x_2 - 5 + 7x_2 = 10$\\
$-3x_2 = 15$\\
$x_2 = -5$\\
$x_1 = -2(-5) - 1$\\
$x_1 = 9$\\
\[
\begin{bmatrix}
	3 & 6 & | & -3 \\
	5 & 7 & | & 10
\end{bmatrix}
\]\\
$\frac{1}{3}R_1$
\[
\begin{bmatrix}
	\frac{1}{3}(1) & \frac{1}{3}(2) &|& \frac{1}{3}(-1)\\
	5&7&|&10
\end{bmatrix}=
\begin{bmatrix}
	1&2&|&-1\\
	5&7&|&10
\end{bmatrix}
\]\\
$R_2-5R_1$
\[
\begin{bmatrix}
	1&2&|&-1\\
	5-5(1)&7-5(2)&|&10-5(-1)
\end{bmatrix}=
\begin{bmatrix}
	1&2&|&-1\\
	0&-3&|&15
\end{bmatrix}
\]\\
$-\frac{1}{3}R_2$
\[
\begin{bmatrix}
	1&2&|&-1\\
	-\frac{1}{3}(0)&-\frac{1}{3}(-3)&|&-\frac{1}{3}(15)
\end{bmatrix}=
\begin{bmatrix}
	1&2&|&-1\\
	0&1&|&-5
\end{bmatrix}
\]\\
$R_1-2R_2$
\[
\begin{bmatrix}
	1-2(0)&2-2(1)&|&-1-2(-5)\\
	0&1&|&-5
\end{bmatrix}=
\begin{bmatrix}
	1&0&|&9\\
	0&1&|&-5
\end{bmatrix}
\]\\
$$(x_1, x_2) = (9,-5)$$\\

3) Find the point $(x_1, x_2)$ that lies on the line $x_1+2x_2=4$ and on the line $x_1-x_2=1$.\\
$x_1 + 2x_2 = 4$\\
$x_1-x_2=1$\\
\[
\begin{bmatrix}
	1 & 2 & | & 4 \\
	1 & -1 & | & 1
\end{bmatrix}
\]\\
$R_2 - R_1$\\
\[
\begin{bmatrix}
	1 & 2 & | & 4 \\
	1-1 & -1-2 & | & 1 - 4
\end{bmatrix}=
\begin{bmatrix}
	1 & 2 & | & 4\\
	0 & -3 & | & -3
\end{bmatrix}
\]\\
$-\frac{1}{3}R_2$\\
\[
\begin{bmatrix}
	1 & 2 & | & 4\\
	-\frac{1}{3}(0) & -\frac{1}{3}(-3) & | & -\frac{1}{3}(-3)
\end{bmatrix}=
\begin{bmatrix}
	1 & 2 & | & 4 \\
	0 & 1 & | & 1
\end{bmatrix}
\]\\
$R_1 - 2R_2$\\
\[
\begin{bmatrix}
	1 - 2(0) & 2 - 2(1) & | & 4 - 2(1)\\
	0 & 1 & | & 1
\end{bmatrix}=
\begin{bmatrix}
	1 & 0 & | & 2\\
	0 & 1 & | & 1
\end{bmatrix}
\]\\
$$(x_1, x_2) = (2, 1)$$\\

4) Find the point of intersection of the lines $x_1+2x_2=-13$ and $3x_1-2x_2=1$.\\
$x_1+2x_2=-13$\\
$3x_1-2x_2 = 1$\\
\[
\begin{bmatrix}
	1&2&|&-13\\
	3&-2&|&1
\end{bmatrix}
\]\\
$R_2-3R_1$
\[
\begin{bmatrix}
	1&2&|&-13\\
	3-3(1)&-2-3(2)&|&1-3(-13)
\end{bmatrix}=
\begin{bmatrix}
	1&2&|&-13\\
	0&-8&|&40
\end{bmatrix}
\]\\
$-\frac{1}{8}R_2$
\[
\begin{bmatrix}
	1&2&|&-13\\
	-\frac{1}{8}(0)&-\frac{1}{8}(-8)&|&-\frac{1}{8}(40)
\end{bmatrix}=
\begin{bmatrix}
	1&2&|&-13\\
	0&1&|&-5
\end{bmatrix}
\]\\
$R_1-2R_2$
\[
\begin{bmatrix}
	1-2(0)&2-2(1)&|&-13-2(-5)\\
	0&1&|&-5
\end{bmatrix}=
\begin{bmatrix}
	1&0&|&-3\\
	0&1&|&-5
\end{bmatrix}
\]\\
$$(x_1, x_2) = (-3, -5)$$\\

17) Do the three lines $2x_1+3x_2=-1$, $6x_1+5x_2=0$, and $2x_1-5x_2=7$ have a common point of intersection? Explain.\\
\[
\begin{bmatrix}
	2&3&|&-1\\
	6&5&|&0\\
	2&-5&|&7
\end{bmatrix}
\]
$R_2-3R_1$ and $R_3-R_1$
\[
\begin{bmatrix}
	2&3&|&-1\\
	6-3(2)&5-3(3)&|&0-3(-1)\\
	2-2&-5-3&|&7-(-1)
\end{bmatrix}=
\begin{bmatrix}
	2&3&|&-3\\
	0&-4&|&3\\
	0&-8&|&8
\end{bmatrix}
\]\\
$R_3-2R_2$
\[
\begin{bmatrix}
	2&3&|&-3\\
	0&-4&|&3\\
	0-2(0)&-8-2(-4)&|&8-2(3)
\end{bmatrix}=
\begin{bmatrix}
	2&3&|&-3\\
	0&-4&|&3\\
	0&0&|&2
\end{bmatrix}
\]\\
Therefore, the three lines doesn't have a common point of intersection because the third equation shows the system is inconsistent with $0=2$\\

18) Do the three planes $2x_1+4x_2+4x_3=4$, $x_2-2x_3=-2$, and $2x_1+3x_2=0$ have at least one common point of intersection? Explain.\\
\[
\begin{bmatrix}
	2&4&4&|&4\\
	0&1&-2&|&-2\\
	2&3&0&|&0
\end{bmatrix}
\]\\
$R_3-R_1$
\[
\begin{bmatrix}
	2&4&4&|&4\\
	0&1&-2&|&-2\\
	2-2&3-4&0-4&|&0-4
\end{bmatrix}=
\begin{bmatrix}
	2&4&4&|&4\\
	0&1&-2&|&-2\\
	0&-1&-4&|&-4
\end{bmatrix}
\]\\
$R_3+R_2$
\[
\begin{bmatrix}
	2&4&4&|&4\\
	0&1&-2&|&-2\\
	0+0&-1+1&-4+(-2)&|&-4+(-2)
\end{bmatrix}=
\begin{bmatrix}
	2&4&4&|&4\\
	0&1&-2&|&-2\\
	0&0&-6&|&-6
\end{bmatrix}
\]\\
$\frac{1}{2}R_1$ and $-\frac{1}{6}R_3$
\[
\begin{bmatrix}
	\frac{1}{2}(2)&\frac{1}{2}(4)&\frac{1}{2}(4)&|&\frac{1}{2}(4)\\
	0&1&-2&|&-2\\
	-\frac{1}{6}(0)&-\frac{1}{6}(0)&-\frac{1}{6}(-6)&|&-\frac{1}{6}(-6)
\end{bmatrix}=
\begin{bmatrix}
	1&2&2&|&2\\
	0&1&-2&|&-2\\
	0&0&1&|&1
\end{bmatrix}
\]\\
Therefore, since the Reduced Row Echelon form of a matrix has a leading 1 in each row, then the corresponding system is consistent and has at least one solution.\\

In Exercise 19-22, determine the value(s) of $h$ such that the matrix is the augmented matrix of a consistent linear system.\\

19) \[
\begin{bmatrix}
	1 & h &|& 4\\
	3 & 6 &|& 8
\end{bmatrix}
\]
$R_2-3R_1$
\[
\begin{bmatrix}
	1&h&|&4\\
	3-3(1)&6-3(h)&|&8-3(4)
\end{bmatrix}=
\begin{bmatrix}
	1&h&|&4\\
	0&3h-6&|&-4
\end{bmatrix}
\]\\
If $h=2$, then the system has no solution, because $3(2)-6=0$ cannot equal $-4$. Otherwise, if $h\neq2$, the system has a solution.\\

20) \[
\begin{bmatrix}
	1&h&|&-5\\
	2&-8&|&6
\end{bmatrix}
\]
$R_2-2R_1$
\[
\begin{bmatrix}
	1&h&|&-5\\
	2-2(1)&-8-2(h)&|&6-2(-5)
\end{bmatrix}=
\begin{bmatrix}
	1&h&|&-5\\
	0&-2h-8&|&16
\end{bmatrix}
\]\\
If $h=-4$, then the system has no solution, because $-2(-4)-8=0$ cannot equal $16$. Otherwise, if $h\neq-4$, the system has a solution.\\

21) \[
\begin{bmatrix}
	1&4&|&-2\\
	3&h&|&-6
\end{bmatrix}
\]
$R_2-3R_1$
\[
\begin{bmatrix}
	1&4&|&-2\\
	3-3(1)&h-3(4)&|&-6-3(-2)
\end{bmatrix}=
\begin{bmatrix}
	1&4&|&-2\\
	0&h-12&|&0
\end{bmatrix}
\]\\
The system has infinite solutions because $R_2 = 0$.\\

22) \[
\begin{bmatrix}
	-4&12&|&h\\
	2&-6&|&-3
\end{bmatrix}
\]
$R_2+\frac{1}{2}R_1$
\[
\begin{bmatrix}
	-4&12&|&h\\
	2+\frac{1}{2}(-4)&-6+\frac{1}{2}(12)&|&-3+\frac{h}{2}
\end{bmatrix}=
\begin{bmatrix}
	-4&12&|&h\\
	0&0&|&-3+\frac{h}{2}
\end{bmatrix}
\]\\
The system is only consist if and only if $h=6$.\\

23) \\

a) Every elementary row operation is reversible.\\
True, scaling and adding or subtracting rows are reversible.\\

b) A $5\times6$ matrix has six rows.\\
False, a $5\times6$ matrix has five rows and six columns.\\

c) The solution set of a linear system involving variables $x_1, \cdots, x_n$ is a list of numbers $(s_1,\cdots,s_n)$ that makes each equation in the system a true statement when the values $s_1,\cdots,s_n$ are substituted for $x_1,\cdots, x_n$ respectively.\\
False, the solution set consists of all possible solutions as a statement is only true if it is always true. \\

d) Two fundamental questions about a linear system involve existence and uniqueness.\\
True, uniqueness implies existence and existence implies uniqueness. Then $A$ is invertible.\\

24) \\

a) Two matrices are row equivalent if they have the same number of rows.\\
False, row equivalent requires a sequence of row operations that transforms one matrix into another.\\

b) Elementary row operations on an augmented matrix never change the solution set of the associated linear system.\\
True.\\

c) Two equivalent linear systems can have different solutions sets.\\
False.\\

d) A consistent system of linear equations has one or more solutions.\\
True, a consistent system has at least one solution.\\

25) Find an equation involving $g, h,$ and $k$ that makes this augmented matrix correspond to a consistent system:
\[
\begin{bmatrix}
	1&-4&7&|&g\\
	0&3&-5&|&h\\
	-2&5&-9&|&k
\end{bmatrix}
\]\\
$R_3+2R_1$
\[
\begin{bmatrix}
	1&-4&7&|&g\\
	0&3&-5&|&h\\
	-2+2(1)&5+2(-4)&-9+2(7)&|&k+2g
\end{bmatrix}=
\begin{bmatrix}
	1&-4&7&|&g\\
	0&3&-5&|&h\\
	0&-3&5&|&k+2g
\end{bmatrix}
\]\\
$R_3+R_2$
\[
\begin{bmatrix}
	1&-4&7&|&g\\
	0&3&-5&|&h\\
	0&-3+3&5+(-5)&|&k+2g+h
\end{bmatrix}=
\begin{bmatrix}
	1&-4&7&|&g\\
	0&3&-5&|&h\\
	0&0&0&|&k+2g+h
\end{bmatrix}
\]\\
Therefore, this system is consistent if and only if $k+2g+h = 0$.\\

26) Suppose the system below is consistent for all possible values of $f$ and $g$. What can you say about the coefficients $c$ and $d$? Justify your answer.\\
$2x_1+4x_2=f$\\
$cx_1+dx_2=g$\\
\[
\begin{bmatrix}
	2&4&|&f\\
	c&d&|&g
\end{bmatrix}
\]\\
$\frac{1}{2}R_1$
\[
\begin{bmatrix}
	1&2&|&\frac{f}{2}\\
	c&d&|&g
\end{bmatrix}
\]\\
$R_2 = R_2-cR_1$
\[
\begin{bmatrix}
	1&2&|&\frac{f}{2}\\
	c-c(1)&d-c(2)&|&g-c\frac{f}{2}
\end{bmatrix}=
\begin{bmatrix}
	1&2&|&\frac{f}{2}\\
	0&d-2c&|&g-c\frac{f}{2}
\end{bmatrix}
\]\\

In Exercise 29-32, find the elementary row operation that transforms the first matrix into the second, and then find the reverse row operation that transforms the second matrix into the first.\\

29) $R_1$ swap with $R_3$.\\
\[
\begin{bmatrix}
0&-2&|&5\\
1&3&|&-5\\
3&-1&|&6
\end{bmatrix}
\begin{bmatrix}
3&-1&|&6\\
1&3&|&-5\\
0&-2&|&5
\end{bmatrix}
\]\\

30)$-\frac{1}{5}R_3$.\\
\[
\begin{bmatrix}
1&3&|&-4\\
0&-2&|&6\\
0&-5&|&10
\end{bmatrix}
\begin{bmatrix}
1&3&|&-4\\
0&-2&|&6\\
0&1&|&-2
\end{bmatrix}
\]\\

\section{Exercise 1.2}

In Exercise 1 and 2, determine which matrices are in reduced echelon form and which others are only in echelon form.\\

1) a) \[
\begin{bmatrix}
1&0&0&0\\
0&1&0&0\\
0&0&1&1
\end{bmatrix}
\]
Reduced echelon form because each leading 1 is the only non-zero entry in that column.\\

b) \[
\begin{bmatrix}
1&0&1&0\\
0&1&1&0\\
0&0&0&1
\end{bmatrix}
\]
Reduced echelon form because the leading non-zero coefficient of a non-zero row is a 1 and is strictly to the right of the leading 1 in the row above it.\\

c) \[
\begin{bmatrix}
1&0&0&0\\
0&1&1&0\\
0&0&0&0\\
0&0&0&1
\end{bmatrix}
\]
Not echelon form because not all non-zero rows are above any rows of all zeros.\\

d) \[
\begin{bmatrix}
1&1&0&1&1\\
0&2&0&2&2\\
0&0&0&3&3\\
0&0&0&0&4
\end{bmatrix}
\]
Echelon form but not reduced yet.\\
\end{document}