%!TEX root = fastZKP.tex

\section{Zero Knowledge Argument Protocols}\label{sec:zkp}

In this section, we present the construction of our new zero-knowledge argument system. In~\cite{zhang2017vsql}, Zhang et al. proposed to combine the GKR protocol with a verifiable polynomial delegation protocol, resulting in an argument system. Later, in~\cite{zkvpd,hyrax}, the construction was extended to zero-knowledge, by sending all the messages in the GKR protocol in homomorphic commitments and performing all the checks by zero-knowledge equality and product testing. This incurs a high overhead for the verifier compared to the plain version without zero-knowledge, as each multiplication becomes an exponentiation and each equality check becomes a $\Sigma$-protocol, which is around $100\times$ slower in practice.

In this paper, we follow the same blueprint of combining GKR and VPD to obtain an argument system, but instead show how to extend it to be zero-knowledge efficiently. In particular, the prover masks the GKR protocol with special random polynomials so that the verifier runs a "randomized" GKR that leaks no extra information and her overhead is small. A similar approach was used by Chiesa et.al in~\cite{zksumcheck}. In the following, we present the zero-knowledge version of each building block, followed by the whole zero-knowledge argument.

\begin{figure}[b!]
	\small{
		\centering{\centering
			\framebox{\parbox{.99\linewidth}{
					\begin{construction}
						\label{construction::zksumcheck}
						We assume the existence of a zkVPD protocol defined in Section~\ref{subsec::zkvpd}. For simplicity, we omit the randomness $r_f$ and public parameters $\pp,\vp$ without any ambiguity. To prove the claim $H = \sum\limits_{x_1, x_2, \ldots, x_\ell \in \binary} f(x_1, x_2, \ldots, x_\ell)$:
						\begin{enumerate}

							\item $\P$ selects a polynomial $g(x_1,\ldots, x_\ell) = a_{0} + g_1(x_1) + g_2(x_2) + \ldots + g_l(x_\ell)$, where $g_{i}(x_i) = a_{i,1}x_i + a_{i,2}x_i^2 + \ldots + a_{i,d}x_i^d$ and all $a_{i,j}$s are uniformly random. $\P$ sends $H = \sum\limits_{x_1, x_2, \ldots, x_\ell \in \binary} f(x_1, x_2, \ldots, x_\ell)$, $G = \sum\limits_{x_1, x_2, \ldots, x_\ell \in \binary} g(x_1, x_2, \ldots, x_\ell)$ and $\comm_g = \Commit(g)$ to $\V$.
							\item $\V$ uniformly selects $\rho \in \mathbb{F}^*$, computes $H+\rho G$ and sends $\rho$ to $\P$.
							\item $\P$ and $\V$ run the sumcheck protocol on
							$$H + \rho G = \sum\limits_{x_1, x_2, \ldots, x_\ell \in \binary}(f(x_1, x_2, \ldots, x_\ell) + \rho g(x_1, x_2, \ldots, x_\ell))$$
							\item At the last round of the sumcheck protocol, $\V$ obtains a claim $h_\ell(r_\ell) = f(r_1, r_2, \ldots, r_\ell)+\rho g(r_1, r_2, \ldots, r_\ell)$. $\P$ and $\V$ opens the commitment of $g$ at $r = (r_1,\ldots, r_\ell)$ by $(g(r), \pi)\leftarrow\Open(g,r), \Verify(\comm_g,g(r),r,\pi)$. If $\Verify$ outputs $\reject$, $\V$ aborts.
							\item $\V$ computes $h_\ell(r_\ell)-\rho g(r_1,\ldots,r_\ell)$ and compares it with the oracle access of $f(r_1,\ldots, r_\ell)$.

						\end{enumerate}
	\end{construction}}}}}
	%\vspace{-.2in}
\end{figure}

\subsection{Zero Knowledge Sumcheck}\label{subsec:zksumcheck}
As a core step of the GKR protocol, $\P$ and $\V$ execute a sumcheck protocol on Equation~\ref{eq:GKR}, during which $\P$ sends $\V$ evaluations of the polynomial at several random points chosen by $\V$. These evaluations leak information about the values in the circuit, as they can be viewed as weighted sums of these values.

To make the sumcheck protocol zero-knowledge, we take the approach proposed by Chiesa et al. in \cite{zksumcheck}, which is masking the polynomial in the sumcheck protocol by a random polynomial. In this approach, to prove
$$H = \sum\nolimits_{x_1, x_2, \ldots, x_\ell \in \binary}f(x_1, x_2, \ldots, x_\ell)$$
the prover generates a random polynomial $g$ with the same variables and individual degrees of $f$. She commits to the polynomial $g$, and sends the verifier a claim $G = \sum\limits_{x_1, x_2, \ldots, x_\ell \in \binary}g(x_1, x_2, \ldots, x_\ell)$. The verifier picks a random number $\rho$, and execute a sumcheck protocol with the prover on $$H + \rho G = \sum\nolimits_{x_1, x_2, \ldots, x_\ell \in \{0, 1\}}(f(x_1, x_2, \ldots, x_\ell) + \rho g(x_1, x_2, \ldots, x_\ell)).$$ At the last round of this sumcheck, the prover opens the commitment of $g$ at $g(r_1, \ldots, r_\ell)$, and the verifier computes $f(r_1, \ldots, r_l)$ by subtracting $\rho g(r_1, \ldots, r_\ell)$ from the last message, and compares it with the oracle access of $f$. It is shown that as long as the commitment and opening of $g$ are zero-knowledge, the protocol is zero-knowledge. Intuitively, this is because all the coefficients of $f$ are masked by those of $g$. The soundness still holds because of the random linear combination of $f$ and $g$.

Unfortunately, the masking polynomial $g$ is as big as $f$, and opening it to a random point later is expensive. In~\cite{zksumcheck}, the prover sends a PCP oracle of $g$, and executes a zero-knowledge sumcheck to open it to a random point, which incurs an exponential complexity for the prover. Even replacing it with the zkVPD protocol in~\cite{zkvpd}, the prover time is slow in practice.

In this paper, we show that it suffices to mask $f$ with a small polynomial to achieve zero-knowledge. In particular, we set $g(x_1, \ldots, x_\ell) = a_{0} + g_1(x_1) + g_2(x_2) + \ldots + g_\ell(x_\ell)$, where $g_{i}(x_i) = a_{i,1}x_i + a_{i,2}x_i^2 + \ldots + a_{i,d}x_i^d$ is a random univariate polynomial of degree $d$ ($d$ is the variable degree of $f$). Note here that the size of $g$ is only $O(d\ell)$, while the size of $f$ is $O(2^\ell)$ \babis{don't you need to include $d$ in this expression?}.

The intuition of our improvement is that the prover sends $O(d\ell)$ messages in total to the verifier during the sumcheck protocol, thus a polynomial $g$ with $O(d\ell)$ random coefficients is sufficient to mask all the messages and achieve zero-knowledge. We present the full protocol in Construction~\ref{construction::zksumcheck}.

The completeness of the protocol holds obviously. The soundness follows the soundness of the sumcheck protocol and the random linear combination in step 2 and 3, as proven in~\cite{zksumcheck}. We give a proof of zero knowledge here.

\begin{theorem}[Zero knowledge]\label{thm:zksc}
	For every verifier $\V^*$ and every $\ell$-variate polynomial $f:\F^\ell\rightarrow\F$ with variable degree $d$, there exists a simulator $\S$ such that given access to $H = \sum\nolimits_{x_1, x_2, \ldots, x_\ell \in \binary} f(x_1, x_2, \ldots, x_\ell)$, $\S$ is able to simulate the partial view of $\V^*$ in step 1-4 of Construction~\ref{construction::zksumcheck}.
\end{theorem}

\begin{proof}

	We build the simulator $\S$ as following.

	\begin{enumerate}

		\item $\S$ selects a random polynomial $g^*(x_1,\ldots,x_\ell) = a^*_{0} + g^*_1(x_1) + g^*_2(x_2) + \cdots + g^*_\ell(x_\ell)$, where $g^*_i(x_i) = a^*_{i,1}x_i + a^*_{i,2}x_i^2 + \cdots + a^*_{i,d}x_i^d$. $\S$ sends $H$, $G^* = \sum\limits_{x_1, x_2, \cdots, x_\ell \in \binary} g^*(x_1, x_2, \cdots, x_\ell)$ and $\comm_{g^*} = \Commit(g^*)$ to $\V$.

		\item $\S$ receives $\rho \neq 0$ from $\V^*$.
		\item $\S$ selects a polynomial $f^*:\F^\ell\rightarrow\F$ with variable degree $d$ uniformly at random conditioning on $\sum\limits_{x_1, x_2, \cdots, x_\ell \in \{0, 1\}}f^*(x_1, x_2, \cdots, x_\ell) = H$. $\S$ then engages in a sumcheck protocol with $\V$ on  $H+\rho G^* = \sum\limits_{x_1, x_2, \cdots, x_l \in \{0, 1\}}(f^*(x_1, x_2, \cdots, x_\ell)+\rho g^*(x_1, x_2, \cdots, x_\ell))$

		\item Let $r \in \F^\ell$ be the point chosen by $\V^*$ in the sumcheck protocol. $\S$ runs $(g^*(r), \pi)\leftarrow\Open(g^*,r)$ and sends them to $\V$.

	\end{enumerate}

	As both $g$ and $g^*$ are randomly selected, and the zkVPD protocol is zero-knowledge, it is obvious that step 1 and 4 in $\S$ are indistinguishable from those in the real world of Construction~\ref{construction::zksumcheck}. It remains to show that the sumchecks in step 3 of both worlds are indistinguishable.

	To see that, recall that in round $i$ of the sumcheck protocol, $\V$ receives a univariate polynomial $h_i(x_i) = \sum\limits_{b_{i+1}, \ldots,b_\ell\in\binary}h(r_1, \ldots, r_{i-1},x_i, b_{i+1}, \ldots, b_\ell)$ where $h = f+\rho g$. (The view of $\V^*$ is defined in the same way with $h^*,f^*,g^*$ and we omit the repetition in the following.) As the variable degree of $f$ and $g$ is $d$, $\P$ sends $\V$ $h_i(0), h_i(1), \ldots, h_i(d)$ which uniquely defines $h_i(x_i)$. These evaluations reveal $d+1$ independent linear constraints on the coefficients of $h$. In addition, note that when these evaluations are computed honestly by $\P$, $h_i(0)+h_i(1) = h_{i-1}(r_{i-1})$, as required in the sumcheck protocol. Therefore, in all $\ell$ rounds of the sumcheck, $\V$ and $\V^*$ receives $\ell(d+1) - (\ell-1) = \ell d+1$ independent linear constraints on the coefficients of $h$ and $h^*$.

	As $h$ and $h^*$ are masked by $g$ and $g^*$, each with exactly $\ell d+1$ coefficients selected randomly, the two linear systems are identically distributed. Therefore, step 3 of the ideal world is indistinguishable from that of the real world.

\end{proof}

\subsection{Zero knowledge GKR}\label{subsec::zkgkr}

To achieve zero-knowledge, we replace the sumcheck protocol in GKR with the zero-knowledge version described in the previous section. However, the protocol still leaks additional information. In particular, at the end of the zero-knowledge sumcheck, $\V$ queries the oracle to evaluate the polynomial on a random point. When executed on Equation~\ref{eq:GKR}, this reveals two evaluations of the polynomial $\tV_i$ defined by the values in the $i$-th layer of the circuit: $\tV_i(u)$ and $\tV_i(v)$.

To prevent this leakage, Chiesa et al.\cite{zksumcheck} proposed to replace the multi-linear extension $\tV_i$ with a low degree extension, such that learning $\tV_i(u)$ and $\tV_i(v)$ does not leak any information about $V_i$. Define a low degree extension of $V_i$ as

\begin{equation}\label{eq:lde}
\dot{V}_{i}(z) \overset{def}{=} \tV_i(z)+Z_i(z)\sum\nolimits_{w \in \{0, 1\}^\lambda}R_i(z, w),
\end{equation}
where $Z(z) = \prod_{i=1}^{s_i} z_i(1-z_i)$, i.e., $Z(z)=0$ for all $z\in\{0, 1\}^{s_i}$. $R_i(z,w)$ is a random low-degree polynomial and $\lambda$ is the security parameter. With this low degree extension, Equation~\ref{eq:GKR} becomes

\begin{align}\label{eq:zkGKR}
\dot{V}_{i}(g)&=\sum\nolimits_{x, y\in \binary^{s_{i+1}}}\tilde{mult}_{i+1}(g, x, y)(\dot{V}_{i+1}(x)\dot{V}_{i+1}(y))\\
&+\tilde{add}_{i+1}(g,x,y)(\dot{V}_{i+1}(x)+\dot{V}_{i+1}(y))\nonumber + Z_i(g)\sum\nolimits_{w \in \binary^\lambda}R_i(g, w)\nonumber\\
&=\sum\nolimits_{x, y\in \binary^{s_{i+1}},w \in \binary^\lambda}(I(\vec{0},w) \cdot \tilde{mult}_{i+1}(g, x, y)(\dot{V}_{i+1}(x)\dot{V}_{i+1}(y))\\
&+\tilde{add}_{i+1}(g,x,y)(\dot{V}_{i+1}(x)+\dot{V}_{i+1}(y))\nonumber + I((x, y), \vec{0})Z_i(g)R_i(g, w))
\end{align}
where $I(\vec{a},\vec{b})$ is an identity polynomial $I(\vec{a},\vec{b}) = 0$ iff $\vec{a}=\vec{b}$. The first equation holds because $\dot{V}_i$ agrees with $\tV_i$ on the Boolean hyper-cube $\binary^{s_i}$, as $Z_i(z) = 0$ for binary inputs. The second equation holds because the mask in $\dot{V}_i$ is in the form of a ``sum" and can be moved into the sumcheck equation.

When executing the zero-knowledge sumcheck protocol on Equation~\ref{eq:zkGKR}, at the end of the protocol, $\V$ receives $\dot{V}_{i+1}(u)$ and $\dot{V}_{i+1}(v)$ for random points $u,v\in\F^{s_{i+1}}$ chosen by $\V$. They no longer leak information about $V_{i+1}$, as they are masked by $Z_{i+1}(z)\sum\nolimits_{w \in \{0, 1\}^\lambda}R_{i+1}(z, w)$ for $z=u$ and $z=v$. $\V$ computes $\tilde{mult}_{i+1}(g,u,v)$ and $\tilde{add}_{i+1}(g,u,v)$ as before, computes $Z_i(g), I(\vec{0},c), I((u,v),\vec{0})$ where $c\in\F^\lambda$ is a random point chosen by $\V$ for variable $w$, opens $R_i(g,w)$ at $c$  with $\P$ through a polynomial commitment, and checks that together with $\dot{V}_{i+1}(u), \dot{V}_{i+1}(v)$ received from $\P$ they are consistent with the last message of the sumcheck.$\V$ then uses $\dot{V}_{i+1}(u), \dot{V}_{i+1}(v)$ to proceed to the next round.

Unfortunately, similar to the zero-knowledge sumcheck, the masking polynomial $R_i$ is very large in~\cite{zksumcheck}. Opening $R_i$ at a random point takes exponential time for $\P$ either using a PCP oracle as in~\cite{zksumcheck} or potentially using a zkVPD, as $R$ has $s_i+2s_{i+1}+\lambda$ variables.

In this section, we show that we can set $R_i$ to be a small polynomial to achieve zero-knowledge. In particular, $R_i$ has only two variables with variable degree 2. This is because in the $(i-1)$-th round, $\V$ receives two evaluations of $V_i$, $\dot{V}_i(u)$ and $\dot{V}_i(v)$,  which are masked by $\sum_{w}R_i(u,w)$ and $\sum_{w}R_i(v,w)$; in the $i$-th sumcheck, $\V$ opens $R_i$ at $R_i(u,c)$ and $R_i(v,c)$. It suffices to make these four evaluations linearly independent, assuming the commitment and opening of $R_i$ are using a zkVPD. Therefore, we set the low-degree term in Equation~\ref{eq:lde} as $Z_i(z)\sum\nolimits_{w \in\binary} R_i(z_1, w)$, i.e. $R_i$ only takes two variables, the first variable $z_1$ of $z$ and an extra variable $w\in\binary$ instead of $\binary^\lambda$, with variable degree 2.

The full protocol is presented in Construction~\ref{construction:zkgkr}. Here we use superscriptions (e.g., $u^{(i)}$) to denote random numbers or vectors for the $i$-th layer of the circuit.

%\begin{figure}[H]

%{\small
%\centering{\centering
%\framebox{\parbox{.99\linewidth}{
\begin{mdframed}[nobreak=false]
	\begin{construction}\label{construction:zkgkr}
		\begin{enumerate}
			\item On a layered arithmetic circuit $C$ with $d$ layers and input $\mathsf{in}$, the prover $\P$ sends the output of the circuit $\mathsf{out}$ to the verifier $\V$.

			\item $\P$ randomly selects polynomials $R_1(z_1, w), \ldots, R_d(z_1, w): \F^2\rightarrow \F$ with variable degree 2. $\P$ commits to these polynomials by sending $\comm_i\leftarrow\Commit(R_i)$ to $\V$ for $i\in[1,d]$.

			\item $\V$ defines $\dot{V}_0(z)= \tV_0(z)$, where $\tV_0(z)$ is the multilinear extension of $\mathsf{out}$. $\dot{V}_0(z)$ can be viewed as a special case with $R_0(z_1,w)$ being the 0 polynomial. $\V$ evaluates it at a random point $\dot{V}_0(g^{(0)})$ and sends $g^{(0)}$ to $\P$.

			\item $\P$ and $\V$ execute the zero knowledge sumcheck protocol presented in Construction~\ref{construction::zksumcheck} on
			\[
			\begin{aligned}
			\dot{V}_{0}(g^{(0)})=\sum_{x, y\in \binary^{s_{1}}}&\tilde{mult}_{1}(g^{(0)}, x, y)(\dot{V}_{1}(x)\dot{V}_{1}(y))\\
			&+\tilde{add}_{1}(g^{(0)},x,y)(\dot{V}_{1}(x)+\dot{V}_{1}(y))
		    \end{aligned}
			\]
			If $u_1^{(1)} = v_1^{(1)}$, $\P$ aborts. At the end of the protocol, $\V$ receives $\dot{V}_1(u^{(1)})$ and $\dot{V}_1(v^{(1)})$. $\V$ computes $\tmult_1(g^{(0)},u^{(1)},v^{(1)})$, $\tadd_1(g^{(0)},u^{(1)},v^{(1)})$ and checks that $$\tmult_1(g^{(0)},u^{(1)},v^{(1)})\dot{V}_1(u^{(1)})\dot{V}_1(v^{(1)})+\tadd_1(g^{(0)},u^{(1)},v^{(1)})(\dot{V}_1(u^{(1)})+\dot{V}_1(v^{(1)}))$$ equals to the last message of the sumcheck (evaluation oracle).

			\item For layer $i=1,\ldots, d-1$:
			\begin{enumerate}
				\item $\V$ randomly selects $\alpha^{(i)}, \beta^{(i)}\in\F$ and sends them to $\mathcal{P}$.
				\item  Let $Mult_{i + 1}(x, y) = \alpha^{(i)}\tilde{mult}_{i+1}(u^{(i)}, x, y)+\beta^{(i)}\tilde{mult}_{i+1}(v^{(i)}, x, y)$ and\\ $Add_{i + 1}(x, y) = \alpha^{(i)}\tilde{add}_{i+1}(u^{(i)}, x, y)+\beta^{(i)}\tilde{add}_{i+1}(v^{(i)}, x, y)$. $\P$ and $\V$ run the zero knowledge sumcheck on the equation\\

				$\alpha^{(i)}\dot{V}_i(u^{(i)})+\beta^{(i)}\dot{V}_i(v^{(i)})=$
				\begin{align*}
				&\sum_{\substack{x, y\in \binary^{s_{i+1}}\\w \in \binary}}(I(\vec{0},w) \cdot Mult_{i+1}(x, y)(\dot{V}_{i+1}(x)\dot{V}_{i+1}(y))\\
				&+Add_{i+1}(x, y)(\dot{V}_{i+1}(x)+\dot{V}_{i+1}(y))\nonumber\\
				&+ I((x, y), \vec{0})(\alpha^{(i)}Z_i(u^{(i)})R_i(u_1^{(i)}, w)+\beta^{(i)}Z_i(v^{(i)})R_i(v_1^{(i)}, w)))
				\end{align*}
				If $u_1^{(i+1)} = v_1^{(i+1)}$, $\P$ aborts.

				\item At the end of the zero-knowledge sumcheck protocol, $\P$ sends $\V$ $\dot{V}_{i+1}(u^{(i+1)})$ and $\dot{V}_{i+1}(v^{(i+1)})$.

				\item $\V$ computes $$a_{i+1} = \alpha^{(i)}\tilde{mult}_{i+1}(u^{(i)}, u^{(i+1)}, v^{(i+1)})+\beta^{(i)}\tilde{mult}_{i+1}(v^{(i)}, u^{(i+1)}, v^{(i+1)})$$ and $$b_{i+1} = \alpha^{(i)}\tilde{add}_{i+1}(u^{(i)}, u^{(i+1)}, v^{(i+1)})+\beta^{(i)}\tilde{add}_{i+1}(v^{(i)}, u^{(i+1)}, v^{(i+1)})$$ locally. $\V$ computes $Z_i(u^{(i)}),Z_i(v^{(i)}),I(\vec{0},c^{(i)}), I((u^{(i+1)},v^{(i+1)}),\vec{0})$ locally.
				\item $\P$ and $\V$ open $R_i$ at two points $R_i(u_1^{(i)},c^{(i)})$ and $R_i(v_1^{(i)},c^{(i)})$ using $\Open$ and $\Verify$.
				\item $\V$ computes the following as the evaluation oracle and uses it to complete the last step of the zero-knowledge sumcheck.
				%{\footnotesize
				\begin{align*}
				&I(\vec{0},c^{(i)})(a_{i+1}(\dot{V}_{i+1}(u^{(i+1)})\dot{V}_{i+1}(v^{(i+1)}))+\\
				&b_{i+1}(\dot{V}_{i+1}(u^{(i+1)})+\dot{V}_{i+1}(v^{(i+1)})))+\\
				&I((u^{(i+1)},v^{(i+1)}),\vec{0})(\alpha^{(i)}Z_i(u^{(i)})R_i(u_1^{(i)}, c^{(i)})+\beta^{(i)}Z_i(v^{(i)})R_i(v_1^{(i)}, c^{(i)}))
				\end{align*}
				%}
				If all checks in the zero knowledge sumcheck and $\Verify$ passes, $\V$ uses $\dot{V}_{i+1}(u^{(i+1)})$ and $\dot{V}_{i+1}(v^{(i+1)})$ to proceed to the $(i+1)$-th layer. Otherwise, $\V$ outputs $\reject$ and aborts.

			\end{enumerate}

			\item At the input layer $d$, $\V$ has two claims $\dot{V}_{d}(u^{(d)})$ and $\dot{V}_{d}(v^{(d)})$. $\V$ opens $R_d$ at 4 points $R_d(u_1^{(d)},0)$, $R_d(u_1^{(d)},1)$, $R_d(v_1^{(d)},0)$, $R_d(v_1^{(d)},1)$ and checks that $\dot{V}_{d}(u^{(d)}) = \tV_d(u^{(d)})+Z_d(u^{(d)})\sum\limits_{w \in \binary}R_d(u_1^{(d)},w)$ and $\dot{V}_{d}(v^{(d)}) = \tV_d(v^{(d)})+Z_d(v^{(d)})\sum\limits_{w \in \binary}R_d(v_1^{(d)},w)$, given oracle access to two evaluates of $\tV_d$ at $u^{(d)}$ and $v^{(d)}$. If the check passes, output $\accept$; otherwise, output $\reject$.

		\end{enumerate}
	\end{construction}
\end{mdframed}

%}}}}
%\end{figure}

\begin{theorem}\label{thm:zkgkr}
	Construction~\ref{construction:zkgkr} is an interactive proof protocol per Definition~\ref{def:ip}, for a function $f$ defined by a layered arithmetic circuit $C$ such that $f(\mathsf{in},\mathsf{out}) = 1$ iff $C(\mathsf{in}) = \mathsf{out}$. In addition, for every verifier $\V^*$ and every layered circuit $C$, there exists a simulator $\S$ such that given oracle access to $\mathsf{out}$, $\S$ is able to simulate the partial view of $\V^*$ in step 1-5 of Construction~\ref{construction:zkgkr}.
\end{theorem}

The completeness follows from the construction explained above and the completeness of the zero knowledge sumcheck. The soundness follows the soundness of the GKR protocol with low degree extensions, as proven in~\cite{GKR} and~\cite{zksumcheck}. We give the proof of zero knowledge here.

\begin{proof} With oracle access to $\mathsf{out}$, and the simulator $\S_{sc}$ of the zero-knowledge sumcheck protocol in Section~\ref{subsec:zksumcheck} as a subroutine, we construct the simulator $\mathcal{S}$ as following:

	\begin{enumerate}

		\item $\S$ sends the $\mathsf{out}$ to $\V^*$.

		\item $\S$ randomly selects polynomials $R^*_1(z_1, w), \ldots, R^*_d(z_1, w): \F^2\rightarrow \F$ with variable degree 2. $\S$ commits to these polynomials by sending $\comm_i\leftarrow\Commit(R^*_i)$ to $\V^*$ for $i\in[1,d]$.

		\item $\S$ receives $g^{(0)}$ from $\V^*$.

		\item $\S$ calls $\S_{sc}$ to simulate the partial view of the zero knowledge sumcheck protocol on
		{\footnotesize
			\[
			\dot{V}_{0}(g^{(0)})=\sum_{x, y\in \binary^{s_{1}}}\tilde{mult}_{1}(g^{(0)}, x, y)(\dot{V}_{1}(x)\dot{V}_{1}(y))+\tilde{add}_{1}(g^{(0)},x,y)(\dot{V}_{1}(x)+\dot{V}_{1}(y))
			\]
		}
		If $u_1^{(1)} = v_1^{(1)}$, $\S$ aborts. At the end of the sumcheck, $\S$ samples $\dot{V}^*_1(u^{(1)})$ and $\dot{V}^*_1(v^{(1)})$ such that $\tmult_1(g^{(0)},u^{(1)},v^{(1)})\dot{V}^*_1(u^{(1)})\dot{V}^*_1(v^{(1)})+\tadd_1(g^{(0)},u^{(1)},v^{(1)})$ $(\dot{V}^*_1(u^{(1)})+\dot{V}^*_1(v^{(1)}))$ equals to the last message of the sumcheck.

		\item For layer $i=1,\ldots,d-1$:
		\begin{enumerate}
			\item $\S$ receives $\alpha^{(i)}, \beta^{(i)}$ from $\V^*$.
			\item Let $Mult_{i + 1}(x, y) = \alpha^{(i)}\tilde{mult}_{i+1}(u^{(i)}, x, y)+\beta^{(i)}\tilde{mult}_{i+1}(v^{(i)}, x, y)$ and \\ $Add_{i + 1}(x, y) = \alpha^{(i)}\tilde{add}_{i+1}(u^{(i)}, x, y)+\beta^{(i)}\tilde{add}_{i+1}(v^{(i)}, x, y)$. $\S$ calls $\S_{sc}$ to simulate the partial view of the zero knowledge sumcheck protocol on \\

			$\alpha^{(i)}\dot{V}_i(u^{(i)})+\beta^{(i)}\dot{V}_i(v^{(i)})=$
			\begin{align*}
			&\sum_{\substack{x, y\in \binary^{s_{i+1}}\\w \in \binary}}(I(\vec{0},w) \cdot Mult_{i+1}(x, y)(\dot{V}_{i+1}(x)\dot{V}_{i+1}(y))\\
			&+Add_{i+1}(x, y)(\dot{V}_{i+1}(x)+\dot{V}_{i+1}(y))\nonumber\\
			&+ I((x, y), \vec{0})(\alpha^{(i)}Z_i(u^{(i)})R_i(u_1^{(i)}, w)+\beta^{(i)}Z_i(v^{(i)})R_i(v_1^{(i)}, w)))
			\end{align*}
			If $u_1^{(i+1)} = v_1^{(i+1)}$, $\S$ aborts.

			\item At the end of the zero-knowledge sumcheck protocol, if $u_1^{(i+1)} = v_1^{(i+1)}$, $\S$ aborts. Otherwise, $\S$ samples $\dot{V}^*_{i+1}(u^{(i+1)})$ and $\dot{V}^*_{i+1}(v^{(i+1)})$ randomly such that the following equals to the last message of the sumcheck protocol.
			{\footnotesize
				\begin{align*}
				I(\vec{0},c^{(i)})(a_{i+1}(\dot{V}^*_{i+1}(u^{(i+1)})\dot{V}^*_{i+1}(v^{(i+1)}))+b_{i+1}(\dot{V}^*_{i+1}(u^{(i+1)})+\dot{V}^*_{i+1}(v^{(i+1)})))\\
				+I((u^{(i+1)},v^{(i+1)}),\vec{0})(\alpha^{(i)}Z_i(u^{(i)})R^*_i(u_1^{(i)}, c^{(i)})+\beta^{(i)}Z_i(v^{(i)})R^*_i(v_1^{(i)}, c^{(i)}))
				\end{align*}
			}
			$a_{i+1} = \alpha^{(i)}\tilde{mult}_{i+1}(u^{(i)}, u^{(i+1)}, v^{(i+1)})+\beta^{(i)}\tilde{mult}_{i+1}(v^{(i)}, u^{(i+1)}, v^{(i+1)})$ and \\ $b_{i+1} = \alpha^{(i)}\tilde{add}_{i+1}(u^{(i)}, u^{(i+1)}, v^{(i+1)})+\beta^{(i)}\tilde{add}_{i+1}(v^{(i)}, u^{(i+1)}, v^{(i+1)})$. $\S$ sends $\dot{V}_{i+1}(u^{(i+1)})$ and $\dot{V}_{i+1}(v^{(i+1)})$ to $\V^*$.

			\item $\V^*$ computes the corresponding values locally as in step 5(d) of Construction~\ref{construction:zkgkr}.
			\item $\S$ opens $R^*_i$ at two points $R^*_i(u_1^{(i)},c^{(i)})$ and $R^*_i(v_1^{(i)},c^{(i)})$ using $\Open$.
			\item $\V^*$ performs the checks as in step 5(f) of Construction~\ref{construction:zkgkr}.
		\end{enumerate}
	\end{enumerate}

	Note here that $\V^*$ can actually behave arbitrarily in step 5(d) and 5(f) above. We include these steps to be consistent with the real world in  Construction~\ref{construction:zkgkr} for the ease of interpretation.

	To prove zero-knowledge, step 1,3, 5(a), 5(d) and 5(f) are obviously indistinguishable as $\S$ only receives messages from $\V^*$. Step 2 and 5(e) of both worlds are indistinguishable because of the zero knowledge property of the zkVPD, and the fact that $R^*$ and $R$ are sampled randomly in both worlds. Step 4 and 5(b) are indistinguishable as proven in Theorem~\ref{thm:zksc} for $\S_{sc}$.

	It remains to consider the messages received at the end of step 4 and in step 5(c), namely $\dot{V}_{i}(u^{(i)}), \dot{V}_{i}(v^{(i)})$ and $\dot{V}^*_{i}(u^{(i)}), \dot{V}^*_{i}(v^{(i)})$ for $i = 1,\ldots,d$. In the real world, $\dot{V}_{i}(z)$ is masked by $\sum\limits_{w \in\binary}R_i(z_1,w)$ ($Z(z)$ is publicly known), thus $\dot{V}_{i}(u^{(i)})$ and $\dot{V}_{i}(v^{(i)})$ are masked by $\sum\limits_{w \in\binary}R_i(u_1^{(i)},w)$ and $\sum\limits_{w \in\binary}R_i(v_1^{(i)},w)$ correspondingly. In addition, in step 5(e), $\V^*$ opens $R_i$ at $R_i(u_1^{(i)},c^{(i)})$ and $R_i(v_1^{(i)},c^{(i)})$. To simplify the notation here, we consider only a particular layer and omit the subscription and superscription of $i$. Let $R(z_1,w) = a_0+a_1z_1+a_2w+a_3z_1w+a_4z_1^2+a_5w^2+a_6z_1^2w^2$, where $a_0,\ldots,a_6$ are randomly chosen. We can write the four evaluations above as
	\[
	\begin{bmatrix}
	2 & 2u_1 & 1 & u_1 & 2u_1^2 & 1 & u_1^2\\
	2 & 2v_1 & 1 & v_1& 2v_1^2 & 1 & v_1^2\\
	1 & u_1 & c & cu_1 & u_1^2 & c^2 & c^2u_1^2\\
	1 & v_1 & c & cv_1 &  v_1^2 & c^2 & c^2v_1^2\\
	\end{bmatrix}
	\times
	\begin{bmatrix}
	a_0 & a_1 &a_2 &a_3&a_4&a_5&a_6
	\end{bmatrix}^T
	\]
	After row reduction, the left matrix is
	\[
	\begin{bmatrix}
	2 & 2u_1 & 1 & u_1 & 2u_1^2 & 1 & u_1^2\\
	0 & 2(v_1-u_1) & 0 & v_1-u_1 & 2(u_1^2-v_1^2) & 0 & u_1^2-v_1^2\\
	0 & 0 & 2c-1 & (2c-1)u_1 & 0 & 2c^2-1 & (2c^2-1)u_1^2\\
	0 & 0 & 0 & (2c-1)(v_1-u_1)&0 & 0 & (2c^2-1)(v_1^2-u_1^2)\\
	\end{bmatrix}
	\]
	As $u_1\neq v_1$, the matrix has full rank if $2c^2 - 1\neq 0 \mod p$, where $p$ is the prime that defines $\F$. This holds if $2^{-1}$ is not in the quadratic residue of $p$, or equivalently $p \not\equiv 1, 7 \mod 8$.\footnote{From the reduced matrix, we can see that setting $a_2 = a_3 = a_4 = 0$ does not affect the rank of the matrix, which simplifies the masking polynomial $R$ in practice.} In case $p\equiv 1, 7 \mod 8$, we can add a check to both the protocol and the simulator to abort if $2c^2 -1 =0$. This does not affect the proof of zero knowledge, and only reduces the soundness error by a small amount. \footnote{If one is willing to perform a check like this, we can simplify the masking polynomial $R$ to be multilinear. The reduced matrix will be the first 4 columns of the matrix showed above, and it has full rank if $c\neq 2^{-1}$.}

	Because of the full rank of the matrix, the four evaluations are linearly independent and uniformly distributed, as $a_0,\ldots a_6$ are chosen randomly. In the ideal world, $R^*(u_1,c)$ and $R^*(v_1,c)$ are independent and uniformly distributed, and $\dot{V}^*(u), \dot{V}^*(v)$ are randomly selected subject to a linear constraint (step 5(c)), which is the same as the real world. Therefore, they are indistinguishable in the two worlds, which completes the proof.

\end{proof}

\subsection{Zero knowledge VPD}\label{subsec::our_zkvpd}
In this section, we present the instantiations of the zkVPD protocol, as defined in Definition~\ref{def::zkvpd}. For every intermediate layer $i$, we use the same zkVPD protocol as proposed by Zhang et al. in~\cite{zkvpd} to commit and open the masking polynomials $g_i(x), R_i(z_1, w)$. In fact, as we show in the previous sections, these polynomials are very small ($g_i$ is the sum of univariate polynomials and $R_i$ has 2 variables with variable degree 2), the zkVPD protocols become very simple. The complexity of $\KeyGen, \Commit, \Open, \Verify$ and proof size are all $O(s_i)$ for $g_i$ and are all $O(1)$ for $R_i$. We omit the full protocols due to space limit.

For the zkVPD used for the input layer, we design a customized protocol based on the zkVPD protocol in~\cite{zkvpd}. Recall that at the end of the GKR protocol, $\V$ receives two evaluations of the polynomial $\dot{V}_d(z)=\tV_d(z)+Z_d(z)\sum\nolimits_{w \in\binary} R_d(z_1,w)$ at $z=u^{(d)}$ and $z=v^{(d)}$. In our zero knowledge proof protocol, which will be presented in Section~\ref{subsec::zkall}, $\P$ commits to $\dot{V}_d(z)$ using the zkVPD at the beginning, and opens it to the two points selected by $\V$.

The protocol in~\cite{zkvpd} works for any polynomial with $\ell$ variables and any variable degree, and is particularly efficient for multilinear polynomials. We modify the protocol for our zero-knowledge proof scheme and preserves the efficiency. Note that though $\dot{V}_d(z)$ is a low degree extension of the input, it can be decomposed to the sum of $\tV_d(z)$, a multilinear polynomial, and $Z_d(z)\sum\nolimits_{w \in\binary} R_d(z_1,w)$. Moreover, $Z_d(u^{(d)})$ and $Z_d(v^{(d)})$ can be computed directly by $\V$. Therefore, in our construction, $\P$ commits to $\tV_d(z)$ and $\sum\nolimits_{w \in\binary} R_d(z_1,w)$ separately, and later opens the sum together given $Z_d(u^{(d)})$ and $Z_d(v^{(d)})$, which is naturally supported because of the homomorphic property of the commitment. Another optimization is that unlike other layers of the circuit, $R_d(z_1,w)$ itself is not opened at two points ($\V$ does not receive $R_d(u^{(d)},c^{(d)})$ and $R_d(v^{(d)},c^{(d)})$ in Construction~\ref{construction:zkgkr}). Therefore, it suffices to set $\dot{V}_d(z)=\tV_d(z)+Z_d(z)R_d(z_1)$, where $R_d$ is a univariate linear polynomial. The full protocol is presented in Construction~\ref{construction::zkVPD}.

\begin{figure}[t!]
	\small{
		\centering{\centering
			\framebox{\parbox{.99\linewidth}{
					\begin{construction}
						\label{construction::zkVPD}
						Let $\mathbb{F}$ be a prime-order finite field. Let $\dot{V}(x): \F^\ell\rightarrow \F$ be an $\ell$-variate polynomial such that $\dot{V}(x) = \tV(x)+Z(x)R(x_1)$, where $\tV(x)$ is a multilinear polynomial, $Z(x) = \prod_{i=1}^\ell x_i(1-x_i)$ and $R(x_1) = a_0+a_1x_1$.

						\begin{itemize}
							\item $(\pp, \vp) \leftarrow \KeyGen(1^\lambda, \ell)$: Select $\alpha, t_1, t_2, \cdots, t_l, t_{\ell+1} \in \F$ uniformaly at random, run $\mathsf{bp} \leftarrow \mathsf{BilGen}(1^{\lambda})$ and compute $\pp = (\mathsf{bp},g^\alpha, g^{t_{\ell+1}}, g^{\alpha t_{\ell+1}}, \{g^{\prod_{i\in W}t_i}, g^{\alpha\prod_{i\in W}t_i}\}_{W\in \mathcal{W}_\ell})$, where $\mathcal{W}_\ell$ is the set of all subsets of $\{1,\ldots, \ell\}$. Set $\vp = (\mathsf{bp},g^{t_1}, \ldots, g^{t_{\ell+1}}, g^\alpha)$.

							\item $\comm \leftarrow \Commit(\dot{V}, r_V, r_R, \pp)$: Compute $c_1 = g^{\tV(t_1, t_2, \cdots, t_\ell) + r_Vt_{\ell+1}}$, $c_2 = g^{\alpha(\tV(t_1, t_2, \cdots, t_\ell) + r_Vt_{\ell+1})}$, $c_3 = g^{R(t_1)+r_Rt_{\ell+1}}$ and $c_4 = g^{\alpha(R(t_1)+r_Rt_{\ell+1})}$  output the commitment $\comm = (c_1, c_2,c_3,c_4)$.

							\item $\{\accept,\reject\}\leftarrow\CheckComm(\comm, \vp)$: Output $\accept$ if $e(c_1, g^\alpha) = e(c_2,g)$ and $e(c_3, g^\alpha) = e(c_4,g)$. Otherwise, output $\reject$.

							\item $(y,\pi)\leftarrow\Open(\dot{V}, r_V, r_R, u,\pp)$: Choose $r_1,\ldots, r_\ell\in\F$ at random, and compute polynomials $q_i$ such that
							\[
							\tV(x)+r_Vx_{\ell+1}+Z(u)(R(x_1)+r_Rx_{\ell+1}) -(\tV(u)+Z(u)R(u_1)) =
							\]
							\[ \sum_{i=1}^\ell(x_i-u_i)(q_i(x_i,\ldots,x_\ell)+r_ix_{\ell+1})+x_{\ell+1}(r_V+r_RZ(u)-\sum_{i=1}^\ell r_i(x_i-u_i)).
							\]
							Set $\pi = (\{g^{q_i(t_i\ldots, t_\ell)+r_it_{\ell+1}}, g^{\alpha(q_i(t_i\ldots, t_\ell)+r_it_{\ell+1})}\}_{i\in[1,\ell]},$ $ g^{r_V+r_RZ(u)-\sum_{i=1}^\ell r_i(t_i-u_i)}$, $ g^{\alpha(r_V+r_RZ(u)-\sum_{i=1}^\ell r_i(t_i-u_i))})$ and $y = \tV(u)+Z(u)R(u_1)$.

							\item $\{\accept,\reject\}\leftarrow\Verify(\comm,u,y,\pi,\vp)$: Parse $\pi$ as $(\pi_i,\pi_{\alpha i})$ for $i\in[1,\ell+1]$. Check $e(\pi_i,g^\alpha) = e(\pi_{\alpha i},g)$ for $i\in[1,\ell+1]$. Check $e(c_1c_3^{Z(u)}/g^y, g) = \prod_{i=1}^\ell e(\pi_i, g^{t_i-u_i}) \cdot e(g^{\pi_{\ell+1}},g^{t_{\ell+1}})$. Output $\accept$ if all the checks pass, otherwise, output $\reject$.

						\end{itemize}

	\end{construction}}}}}
\end{figure}

\begin{theorem}\label{thm:zkvpd}
	Construction~\ref{construction::zkVPD} is a zero-knowledge verifiable polynomial delegation scheme as defined by Definition~\ref{def::zkvpd}, under Assumption~\ref{asp::qSDH} and~\ref{asp::dlEPKE}.
\end{theorem}

The proof of completeness, soundness and zero knowledge is similar to that of the zkVPD protocol in~\cite{zkvpd}. We only add an extra univariate linear polynomial $R(x_1)$, which does not affect the proof. We omit the proof due to space limit. Using the same algorithms proposed in in~\cite{vram,zkvpd}, the running time of $\KeyGen$, $\Commit$ and $\Open$ is $O(2^\ell)$, $\Verify$ takes $O(\ell)$ time and the proof size is $O(\ell)$.

\subsection{Putting Everything Together}\label{subsec::zkall}

In this section, we present our zero knowledge argument scheme. At a high level, similar to~\cite{zhang2017vsql,hyrax,zkvpd}, $\V$ can use the GKR protocol to verify the correct evaluation of a circuit $C$ on input $x$ and a witness $w$, given an oracle access to the evaluation of a polynomial defined by $x,w$ on a random point. We instantiate the oracle using the zkVPD protocol. Formally, we present the construction in Construction~\ref{construction::all}, which combines our zero knowledge GKR and zkVPD protocols. Similar to the protocols in~\cite{zkvpd,hyrax}, Step 6 and 7 are to check that $\P$ indeed uses $x$ as the input to the circuit.
\begin{figure}[t!]
\small{
\centering{\centering
\framebox{\parbox{.99\linewidth}{
\begin{construction}
%\begin{protocol}
\label{construction::all}
	Let $\lambda$ be the security parameter, $\F$ be a prime field, $n$ be an upper bound on input size, and $S$ be an upper bound on circuit size. We use $\VPD_1, \VPD_2, \VPD_3$ to denote the zkVPD protocols for input layer, masking polynomials $g_i$ and $R_i$ described in Construction~\ref{construction:zkgkr}.
	\begin{itemize}
		\item $\mathcal{G}(1^\lambda, n, S)$: run $(\pp_1,\vp_1)\leftarrow\VPD_1.\KeyGen(1^\lambda, \log n)$, $(\pp_2,\vp_2)\leftarrow\VPD_2.\KeyGen(1^\lambda, \log S)$, $(\pp_3,\vp_3)\leftarrow\VPD_3.\KeyGen(1^\lambda)$. Output $\pk = (\pp_1, \pp_2, \pp_3)$ and $\vk = (\vp_1, \vp_2, \vp_3)$.

		\item $\langle\P(\pk,w), \V(\vk)\rangle(x)$: Let $C$ be a layered arithmetic circuit over $\F$ with $d$ layers, input $x$ and witness $w$ such that $|x|+|w|\le n$, $|C|\le S$ and $C(x;w) =1$. Without loss of generality, assume $|w|/|x| = 2^m -1$ for some $m\in\N$.
		\begin{enumerate}
			\item $\P$ selects a random bivariate polynomial $R_d$ with variable degree 2 and commits to the input of $C$ by sending $\comm_d\leftarrow\VPD_1.\Commit(\dot{V}_d, r_V, r_R, \pp_1)$ to $\V$, where $\tV_d$ is the multilinear extension of array $(x;w)$ and $\dot{V}_d=\tV_d+R_d$
			\item $\V$ runs $\VPD_1.\CheckComm(\comm_d,\vp_1)$. If it outputs $\reject$, $\V$ aborts and outputs $\reject$.
			\item $\P$ and $\V$ execute Step 1-5 of the zero knowledge GKR protocol in Construction~\ref{construction:zkgkr}, with the zkVPDs instantiated with $\VPD_2$ and $\VPD_3$. If Construction~\ref{construction:zkgkr} rejects, $\V$ outputs $\reject$ and aborts. Otherwise, by the end of this step, $\V$ receives two claims of $\dot{V}_d$ at $u^{(d)}$ and $v^{(d)}$.
			\item $\P$ runs $(y_1,\pi_1)\leftarrow\VPD_1.\Open(\dot{V}, r_V, r_R,u^{(d)},\pp_1)$,  $(y_2,\pi_2)\leftarrow\VPD_1.\Open(\dot{V}, r_V, r_R, v^{(d)},\pp_1)$ and sends $y_1,\pi_1,y_2,\pi_2$ to $\V$.
			\item $\V$ runs $\Verify(\comm_d,u^{(d)},y_1,\pi_1,\vp_1)$ and $\Verify(\comm_d,v^{(d)},y_2,\pi_2,\vp_1)$ and output $\reject$ if either check fails. Otherwise, $\V$ checks $\dot{V}_d(u^{(d)}) = y_1$ and $\dot{V}_d(v^{(d)}) = y_2$, and rejects if either fails.
			\item $\V$ computes the multilinear extension of input $x$ at a random point $r_x\in\F^{\log |x|}$ and sends $r_x$ to $\P$.
			\item $\P$ pads $r_x$ to $r'_x\in\F^{\log |x|}\times 0^{\log |w|}$ with $\log |w|$ 0s and sends $\V$ $(y_x,\pi_x)\leftarrow\VPD_1.\Open(\tV_d, r_V, r_R, r'_x,\pp_1)$. $\V$ checks $\Verify(\comm_d,r'_x,y_x,\pi_x,\vp_1)$ and $y_x$ equals the evaluation of the multilinear extension on $x$. $\V$ outputs $\reject$ if the checks fail. Otherwise, $\V$ outputs $\accept$.
		\end{enumerate}

	\end{itemize}
%\end{protocol}
\end{construction}}}}}
%\vspace{-0.2in}
\end{figure}
\begin{theorem}\label{theorem:main}
	For an input size $n$ and a finite field $\F$, Construction~\ref{construction::all} is a zero knowledge argument for the relation
	\[
	\mathcal{R} = \{(C,x;w): C\in\mathcal{C}_\F\wedge|x|+|w|\le n\wedge C(x;w) = 1\},
	\]
	as defined in Definition~\ref{def::zkp}, under Assumption~\ref{asp::qSDH} and~\ref{asp::dlEPKE}. Moreover, for every $(C,x;w)\in\mathcal{R}$, the running time of $\P$ is $O(|C|)$ field operations and $O(n)$ multiplications in the base group of the bilinear map. The running time of $\V$ is $O(|x|+d\cdot\log |C|)$ if $C$ is log-space uniform with $d$ layers. $\P$ and $\V$ interact $O(d\log |C|)$ rounds and the total communication (proof size) is $O(d\log |C|)$. In case $d$ is $\mathsf{polylog}(|C|)$, the protocol is a succinct argument.
\end{theorem}

\paragraph{Proof Sketch.} The correctness and the soundness follow from those of the two building blocks, zero knowledge GKR and zkVPD, by Theorem~\ref{thm:zkgkr} and~\ref{thm:zkvpd}.

To prove zero knowledge, consider a simulator $\S$ that calls the simulator $\S_{GKR}$ of zero knowledge GKR given in Section~\ref{subsec::zkgkr} as a subroutine, which simulates the partial view up to the input layer. At the input layer, the major challenge is that $\S$ committed to (a randomly chosen) $\dot{V}^*_d$ at the beginning of the protocol, before knowing the points $u^{(d)}, v^{(d)}$ to evaluate on. If $\S$ opens the commitment honestly, with high probability the evaluations are not consistent with the last message of the GKR (sumcheck in layer $d-1$) and a malicious $\V^*$ can distinguish the ideal world from the real world. In our proof, we resolve this issue by using the simulator $\S_{VPD}$ of our zkVPD protocol. Given the trapdoor $\mathsf{trap}$ used in $\KeyGen$, $\S_{VPD}$ is able to open the commitment to any value in zero knowledge, and in particular it opens to those messages that are consistent with the GKR protocol in our scheme, which completes the construction of $\S$. %We defer the formal proof to the full version of the paper.

The complexity of our zero knowledge argument scheme follows from our new GKR protocol with linear prover time, and the complexity of the zkVPD protocol for the input layer analyzed in Section~\ref{subsec::our_zkvpd}. The masking polynomials $g_i, R_i$ and their commitments and openings introduce no asymptotic overhead and are efficient in practice.


\subsection{Binary operators}
This section decribes all the functions in the file ``ptwXY\_binaryOperators.c''.

\subsubsection{ptwXY\_slopeOffset}
This function applies the math operation ( $y_i$ = slope $\times$ $y_i$ + offset ) to the y-values of \highlight{ptwXY}.
\setargumentNameLengths{offset}
\CallingC{fnu\_status ptwXY\_slopeOffset(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY,}
    \addArgument{double slope,}
    \addArgument{double offset );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY}{A pointer to the \highlight{ptwXYPoints} object.}
    \argumentBox{slope}{The slope.}
    \argumentBox{offset}{The offset.}
    \vskip 0.05 in \noindent

\subsubsection{ptwXY\_add\_double}
This function applies the math operation ( $y_i$ = $y_i$ + offset ) to the y-values of \highlight{ptwXY}.
\CallingC{fnu\_status ptwXY\_add\_double(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY,}
    \addArgument{double offset );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY}{A pointer to the \highlight{ptwXYPoints} object.}
    \argumentBox{offset}{The offset.}

\subsubsection{ptwXY\_sub\_doubleFrom}
This function applies the math operation ( $y_i$ = $y_i$ - offset ) to the y-values of \highlight{ptwXY}.
\CallingC{fnu\_status ptwXY\_sub\_double(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY,}
    \addArgument{double offset );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY}{A pointer to the \highlight{ptwXYPoints} object.}
    \argumentBox{offset}{The offset.}

\subsubsection{ptwXY\_sub\_fromDouble}
This function applies the math operation ( $y_i$ = offset - $y_i$ ) to the y-values of \highlight{ptwXY}.
\CallingC{fnu\_status ptwXY\_sub\_fromDouble(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY,}
    \addArgument{double offset );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY}{A pointer to the \highlight{ptwXYPoints} object.}
    \argumentBox{offset}{The offset.}

\subsubsection{ptwXY\_mul\_double}
This function applies the math operation ( $y_i$ = slope $\times$ $y_i$ ) to the y-values of \highlight{ptwXY}.
\CallingC{fnu\_status ptwXY\_mul\_double(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY,}
    \addArgument{double slope );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY}{A pointer to the \highlight{ptwXYPoints} object.}
    \argumentBox{slope}{The slope.}

\subsubsection{ptwXY\_div\_doubleFrom}
This function applies the math operation ( $y_i$ = $y_i$ / divisor ) to the y-values of \highlight{ptwXY}.
\CallingC{fnu\_status ptwXY\_div\_doubleFrom(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY,}
    \addArgument{double divisor );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY}{A pointer to the \highlight{ptwXYPoints} object.}
    \argumentBox{divisor}{The divisor.}
If \highlight{divisor} is zero, the error \highlight{nfu\_divByZero} is returned.

\subsubsection{ptwXY\_div\_fromDouble}
This function applies the math operation ( $y_i$ = dividend / $y_i$ ) to the y-values of \highlight{ptwXY}.
\setargumentNameLengths{dividend}
\CallingC{fnu\_status ptwXY\_div\_fromDouble(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY,}
    \addArgument{double dividend );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY}{A pointer to the \highlight{ptwXYPoints} object.}
    \argumentBox{dividend}{The dividend.}
    \vskip 0.05 in \noindent
This function does not handle safe division (see Section~\ref{ptwXYdivptwXY}). One way to do safe division is
to use the function \highlight{ptwXY\_valueTo\_ptwXY} to convert the \highlight{dividend} value to a \highlight{ptwXYPoints} object and
then use \highlight{ptwXY\_div\_ptwXY}.

\subsubsection{ptwXY\_mod}
\setargumentNameLengths{ptwXY}
This function gives the remainer of $y_i$ divide by $m$. That is, it set \highlight{ptwXY}'s y-values to
\begin{equation}
    y_i = {\rm mod}( y_i, m ) \ \ \ \ .
\end{equation}
\setargumentNameLengths{pythonMod}
\CallingC{fnu\_status ptwXY\_mod(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY,}
    \addArgument{double m,}
    \addArgument{int pythonMod );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY}{A pointer to the \highlight{ptwXYPoints} object.}
    \argumentBox{m}{The modulus.}
    \argumentBox{pythonMod}{Controls whether the Python or C form of mod is implemented.}
    \vskip 0.05 in \noindent
Python's and C's mod functions act differently for negative values. If \highlight{pythonMod} then the Python form is executed;
otherwise, the C form is executed.

\subsubsection{ptwXY\_binary\_ptwXY}
This function creates a new \highlight{ptwXYPoints} object from the union of \highlight{ptwXY1} and \highlight{ptwXY2} and then
applies the math operation
\begin{equation}
    y_i(x_i) = s_1 \times y_1(x_i) + s_2 \times y_2(x_i) + s_{12} \times y_1(x_i) \times y_2(x_i)
\end{equation}
to the new object. Here ($x_i,y_i$) is a point in the new object, $y_1(x_i)$ is \highlight{ptwXY1}'s y-value at $x_i$ and
$y_2(x_i)$ is \highlight{ptwXY2}'s y-value at $x_i$.
This function is used internally to add, subtract and multiply two \highlight{ptwXYPoints} objects. For example, addition is performed
by setting $s_1$ and $s_2$ to 1. and $s_{12}$ to 0.
\setargumentNameLengths{ptwXY1}
\CallingC{ptwXYPoints *ptwXY\_binary\_ptwXY(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY1,}
    \addArgument{ptwXYPoints *ptwXY2,}
    \addArgument{double s1,}
    \addArgument{double s2,}
    \addArgument{double s12 );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY1}{A pointer to a \highlight{ptwXYPoints} object.}
    \argumentBox{ptwXY2}{A pointer to a \highlight{ptwXYPoints} object.}
    \argumentBox{s1}{The value $s_1$.}
    \argumentBox{s2}{The value $s_2$.}
    \argumentBox{s12}{The value $s_{12}$.}

\subsubsection{ptwXY\_add\_ptwXY}
This function adds two \highlight{ptwXYPoints} objects and returns the result as a new \highlight{ptwXYPoints} object
(i.e., it calls ptwXY\_binary\_ptwXY with $s_1 = s_2 = 1.$ and $s_{12} = 0.$).
\CallingC{ptwXYPoints *ptwXY\_add\_ptwXY(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY1,}
    \addArgument{ptwXYPoints *ptwXY2 );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY1}{A pointer to a \highlight{ptwXYPoints} object.}
    \argumentBox{ptwXY2}{A pointer to a \highlight{ptwXYPoints} object.}

\subsubsection{ptwXY\_sub\_ptwXY}
This function subtracts one \highlight{ptwXYPoints} objects from another, and returns the result as a new \highlight{ptwXY} object
(i.e., it calls ptwXY\_binary\_ptwXY with $s_1 = 1.$, $s_2 = -1.$ and $s_{12} = 0.$).
\CallingC{ptwXYPoints *ptwXY\_sub\_ptwXY(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY1,}
    \addArgument{ptwXYPoints *ptwXY2 );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY1}{A pointer to a \highlight{ptwXYPoints} object which is the minuend.}
    \argumentBox{ptwXY2}{A pointer to a \highlight{ptwXYPoints} object which is the subtrahend.}

\subsubsection{ptwXY\_mul\_ptwXY}
This function multiplies two \highlight{ptwXYPoints} objects and returns the result as a new \highlight{ptwXY} object
(i.e., it calls ptwXY\_binary\_ptwXY with $s_1 = s_2 = 0.$ and $s_{12} = 1.$).  This function does not infill.
\CallingC{ptwXYPoints *ptwXY\_mul\_ptwXY(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY1,}
    \addArgument{ptwXYPoints *ptwXY2 );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY1}{A pointer to a \highlight{ptwXYPoints} object.}
    \argumentBox{ptwXY2}{A pointer to a \highlight{ptwXYPoints} object.}

\subsubsection{ptwXY\_mul2\_ptwXY}
This function multiplies two \highlight{ptwXYPoints} objects and returns the result as a new \highlight{ptwXY} object.
Unlike \highlight{ptwXY\_mul\_ptwXY}, this function will infill to obtain the desired accuracy.
\CallingC{ptwXYPoints *ptwXY\_mul2\_ptwXY(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY1,}
    \addArgument{ptwXYPoints *ptwXY2 );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY1}{A pointer to a \highlight{ptwXYPoints} object.}
    \argumentBox{ptwXY2}{A pointer to a \highlight{ptwXYPoints} object.}

\subsubsection{ptwXY\_div\_ptwXY} \label{ptwXYdivptwXY}
This function divides two \highlight{ptwXYPoints} objects and returns the result as a new \highlight{ptwXY} object.
\setargumentNameLengths{safeDivide}
\CallingC{ptwXYPoints *ptwXY\_div\_ptwXY(}{statusMessageReporting *smr,
    \addArgument{ptwXYPoints *ptwXY1,}
    \addArgument{ptwXYPoints *ptwXY2,}
    \addArgument{int safeDivide );}}
    \argumentBox{smr}{The \highlight{statusMessageReporting} instance to record errors.}
    \argumentBox{ptwXY1}{A pointer to a \highlight{ptwXYPoints} object.}
    \argumentBox{ptwXY2}{A pointer to a \highlight{ptwXYPoints} object.}
    \argumentBox{safeDivide}{If true safe division is performed.}


\section{Quantum statistical mechanics}

\newcommand{\bra}[1]{\langle #1 |}
\newcommand{\ket}[1]{| #1 \rangle}
\newcommand{\braket}[2]{\langle #1 | #2\rangle}

In this section we want to extend the classical statistical mechanics concepts that we have seen so far to quantum mechanical systems. When we do so, we will find that two particular properties of quantum systems --- the (anti-)symmetry of the wave function for fermions (resp. bosons), and the indistinguishability of particles --- have consequences for the statistics of the ensembles that they are part of.

In quantum mechanics we already have two layers of uncertainty about the state of a system: the Heisenberg uncertain principle $\Delta{p}\Delta {r}>{\hbar}/{2}$ means that rather than having a discrete point in phase space to describe the state of a system, the most precision we can hope for is some small region in phase space, centered on that point. Quantum mechanics also deals with probability distributions itself: the wave function $\psi_n({\bf r})$ (or rather $|\psi_n|^2$) describes the probability distribution of a particle being in a particular quantum state.

Statistical mechanics adds an extra layer of probability to this situation in order to deal with ensembles that have probabilities $p_n$ of being in a variety of quantum states $\psi_n$. These quantum ensembles are called \emph{mixed states}. They are distinct from the superposition of quantum states. The superposition of two wave functions implies that the system is in a state comprised of both wave functions simultaneously, while the mixed state says that the system could be in one state or the other, or perhaps both. For example: if $\ket{H}$
is the wave function for a horizontally polarized photon and $\ket{V}$ is the wave function for a vertically polarised photon, then the superposition $\frac{1}{\sqrt{2}}(\ket{H}+\ket{V})$ is a diagonally polarised photon, while the mixed state is an unpolarised photon that is half $\ket{H}$ and half $\ket{V}$ that could be in either state:  i.e. $\frac12\left(\ket{H}\bra{H}+\ket{V}\bra{V}\right)$.

Given a quantum system in a particular state, represented by the wave function $\psi_n$, the quantum expectation of an operator $\hat{A}$ is given by
$$
	\langle\hat{A}\rangle_\text{QM} = \int\psi^*_n\hat{A}\psi_n.
$$
(I'll try to always use hats to denote quantum operators, but I'll probably forget them from time-to-time.)

This is sometimes also denoted as $\langle\hat{A}\rangle_\text{pure}$ as it applies to a purely quantum state, where no possibilities of different statistical mechanical configurations exist.

The statistical mechanical expectation operator for an ensemble is given by
$$
		\langle\hat{A}\rangle = \sum_n p_n\int\psi_n^*\hat{A}\psi_n.
$$
Quantum statistical mechanics is concerned with figuring out how the properties of $\psi_n$ affect the expected value of $\hat{A}$.

\subsection{A refresher: bra-ket notation}
The bra-ket notation, popularised by Dirac, gives us an extremely convenient and compact way of dealing with elements of a vector space (in QM, a Hilbert space) in various combinations. It doesn't introduce any new physics but it does let us, for example, use the same notation for an inner product of two quantities, whether they are finite-dimensional vectors or infinite dimensional functions.

$\ket{v}$ by itself (pronounced ket $v$) is just the column vector (or function) $v$.

$\bra{u}$ by itself (pronounced bra $u$) is interpreted as the conjugate transpose of $\ket{u}$, i.e. $\bra{u}=\ket{u}^*$.

The notation becomes particularly convenient when we want to calculate inner products
$$
	\braket{u}{v} = u^*v
$$
for finite dimension vectors or
$$
	\braket{u}{v} =	\int d{\bf r}u^*({\bf r})v({\bf r})
$$
for functions.

Similarly, the outer product can be denoted

$$
\ket{u}\bra{v}=uv^* =
\begin{bmatrix}
    u_1v^*_1 & u_1v^*_2  & \dots \\
    u_2v^*_1 & u_2v^*_2  & \dots   \\
    \vdots & \vdots & \ddots
\end{bmatrix}.
$$

For an operator $\hat{A}$ acting on a ket $\ket{v}$ we have $\hat{A}\ket{v} = \ket{\hat{A}v}$ and hence
$$
	\bra{u}\hat{A}\ket{v}=\braket{u}{\hat{A}v} = u^*\hat{A}v
$$
Or, equivalently,
$$
	\braket{u}{\hat{A}v} = \int d{\bf r} u^*({\bf r}) \hat{A} v({\bf r}).
$$

Bra-ket notation has a number of convenient properties; one that we will make use of shortly is that closed bra-ket sets commute with each other. This is easy to see by considering the vector case where a closed bra-ket pair is just a scalar.

\subsection*{The density matrix/density operator}
Similar to the way that the single particle density is a convenient way of introducing a probability density into classical statistical mechanics, the density matrix $\rho = \sum_n p_n\ket{\psi_n}\bra{\psi_n}$ (or the density operator $\hat{rho}$ in the case of functions) allows us to consider a probability distribution over an ensemble of quantum states.

To see how the density matrix arises, we will begin with the expression we had earlier for the ensemble expectation of an observable corresponding to some operator $\hat{A}$, which we can now write in bra-ket notation.
\begin{equation}
	\langle\hat{A}\rangle = \sum_n p_n\bra{\psi_n}\hat{A}\ket{\psi_n}.
	\label{eq:7.2}
\end{equation}
(We are going to assume throughout this section that all the vectors/wave functions that we deal with have already been normalised.) Now, if $\phi_a$ is any orthonormal basis then the identity operator is
$$
	Id = \sum_a\ket{\phi_a}\bra{\phi_a}.
$$
If we insert this into equation (\ref{eq:7.2}) for the expected value of $\hat{A}$ we get
\begin{eqnarray*}
	\langle\hat{A}\rangle &=& \sum_n p_n\bra{\psi_n} \left(\sum_a\ket{\phi_a}\bra{\phi_a}\right) |\hat{A}\ket{\psi_n}\\
	&=& \sum_n p_n\sum_a\bra{\phi_a}\hat{A}\ket{\psi_n}\braket{\psi_n}{\phi_a}\\
	&=& \sum_a \bra{\phi_a}\hat{A} \left(\sum_n p_n \ket{\psi_n}\bra{\psi_n}\right)\ket{\phi_a}\\
	&=& \sum_a \bra{\phi_a}\hat{A}\rho\ket{\phi_a}\\
	&=& \text{Tr}(\hat{A}\rho)
\end{eqnarray*}
where $\text{Tr}(M)$ denotes the trace of $M$ (in the case of a matrix, this is the sum over the diagonal elements).
Normalisation means that we have $\text{Tr}(\rho)=1$ and $\sum_n p_n=1$.

We can now use the density matrix to give some expressions for the quantum [micro-$|$grand-]canonical ensembles.

\subsection{Quantum micro-canonical ensemble}
We will consider a system with fixed $N$ and $V$. Because we are dealing with a quantum system, rather than fixing the total energy of the system we fix the interval $(E,E+\epsilon)$ where $\epsilon$ is some (small) uncertainty. We will denote by $E_j$ the (discrete) energy eigenstates that correspond to the Hamiltonian operator $\hat{H}$.

The system has distinct accessible micro-states that correspond to the macro-state $(N,V,E;\epsilon)$. The number of distinct accessible states is given by counting over the phase space to get $\Gamma(N,V,E;\epsilon)$. We are going to assume that the probability of the system being in any of the energy states in the interval is equal --- this is the so-called \emph{equal a priori postulate}.

The probability of the system being in a particular micro-state $E_j$ is therefore
$$
	p(E_j) =
	\begin{cases}
		\frac{1}{\Gamma} \text{ if } E<E_j<E+\epsilon\\
		0 \text{ otherwise}
	\end{cases}
$$
and, if we choose a basis such that the Hamiltonian operator $\hat{H}$ is diagonal, then the density matrix is the diagonal matrix
$$
	\rho = \sum_j p(E_j) \ket{E_j}\bra{E_j}.
$$

The entropy is given by
$$
	S(E) = k_B\ln\Gamma(E).
$$
At this point, it's worth noting that when we count the states that comprise $\Gamma$ we must do so in a quantum fashion, i.e. taking account of the indistinguishability of the quantum particles.

We can also write this in a form similar to that for the Gibbs entropy that we saw in an earlier section:
$$
	S(E) = -k_b\sum_jp(E_j)\ln(p(E_j))=-k_B\text{Tr}(\rho\ln\rho).
$$

\subsection{Quantum canonical ensemble}
Here we take:
$$
	p(E_j) = \frac{1}{Z}\exp\left(\frac{-1}{k_BT}E_j\right)
$$
and
$$
	Z = \sum_j \exp\left(\frac{-1}{k_BT}E_j\right).
$$

Agin, if we assume, that we have picked a basis where $\hat{A}$ is diagonal and has energy eigenstates $E_j$ (i.e. $\hat{H}\ket{E_j} = E_j\ket{E_j}$) then, we can write
$$
	\rho = \frac{1}{Z}\exp\left(\frac{-1}{k_BT}\hat{H}\right)
$$
where
$$
	Z = \text{Tr}\left[\exp\left(\frac{-1}{k_BT}\hat{H}\right)\right].
$$

\subsection{Quantum grand-canonical ensemble}
The density matrix is given by
$$
	\rho = \frac{1}{Z}\exp\left(\frac{-1}{k_BT}(\mu\hat{N}-\hat{H})\right)
$$
with the corresponding partition function
$$
	Z = \text{Tr}\left[\exp\left(\frac{-1}{k_BT}(\mu\hat{N}-\hat{H})\right)\right].
$$

\subsection{Systems of indistinguishable particles}
This is where we will see one of the more interesting consequences of quantum mechanics on statistical mechanics - namely how the symmetry of the wave function for the particles in our system affect the statistics of the system.

We will consider a system of $N$ identical non-interacting particles --- an ideal quantum gas. Since the particles are non-interacting, the overall Hamiltonian $\hat{H}$ for the system is just the sum of the Hamiltonians for each of the individual particles:
$$
	\hat{H}({\bf q},{\bf p}) = \sum_i^N\hat{H_i},
$$
where $\hat{H}_i$ is the Hamiltonian of the $i$-th particle.

The time-independent Schr\"odinger equation for the system is
$$
	\hat{H}\ket{\psi_E} = E\ket{\psi_E}
$$
where $E$ is an energy eigenvalue of $\hat{H}$ and $\psi_E$ is the corresponding eigenfunction.

We can use the fact that $\hat{H}$ is a sum of non-interacting terms to write the solution $\psi_E$ as a product of the solutions to each of the individual particles:
$$
	\psi_E = \prod_{i=1}^Nu_{\epsilon_i}
$$
where $\sum_{i=1}^N\epsilon_i = E$ and where $u_{\epsilon_i}$ is the eigenfunction of $\hat{H_i}$, i.e. $\hat{H_i}u_{\epsilon_i}={\epsilon_i}u_{\epsilon_i}$.

Since the stationary state of the overall system can be described in terms of the constituent particles, we just need to specify how many particles are in each energy state. We write $n_i$ for the number of particles in the state with energy $\epsilon_i$ and $\{n_i\}$ for the set of all such $n_i$. We therefore have
$$
	\sum_i n_i =N \qquad \text{and} \qquad \sum_in_i\epsilon_i = E.
$$

The wave function for the overall state can now be written in terms of the particles in each of the distinct energy eigenstates
\begin{equation}
	\psi_E = \prod_{m=1}^{n_1}u_1(m)\prod_{m=n_1+1}^{n_1+n_2}u_2(m)\cdots
	\label{eq:compfn}
\end{equation}
where $u_i(m)$ is the single particle wave function $u_{\epsilon_1}({\bf q}_m)$.
That is, $\psi_E$ is the product of individual particle eigenfunction, each with degeneracy that corresponds to the number of particles in each energy state.
We will refer to this as the Boltzmann wave function, which we will denote $\psi_\text{Boltz}$.

Since the particles are identical, it is possible to permute their coordinates in (\ref{eq:compfn}) while keeping the system in the same state. I.e. the sequence of indices $(1,2,3,\ldots,N)$ are permuted to $(P1,P2,P3,\ldots,PN)$ where $P$ denotes mapping an index $m$ to the permuted index $Pm$. The resulting wave function will then be
\begin{equation}
	P\psi_E = \prod_{m=1}^{n_1}u_1(Pm)\prod_{m=n_1+1}^{n_1+n_2}u_2(Pm)\cdots.
	\label{eq:permfn}
\end{equation}

In classical statistical mechanics, though the particles may be identical, we treat them as being distinguishable. That is permutation of two identical particles corresponds to two different, though identical, micro-states of the system with the same energy. (This is what leads to the necessity of introducing the Gibbs factor in classical statistical mechanics when evaluating the free energy of the canonical ensemble.  The Gibbs factor $\frac{1}{n_1!n_2!\cdots}$ allows us to correct for the fact that the particles being permuted are \emph{effectively} indistinguishable and the permutation doesn't really correspond to a different micro-state of the system.

In quantum mechanics, the particles are, \emph{in reality} indistinguishable. I.e. all that matters is the number of particles in each state --- the micro-states that occur from permutation are the same micro-state.
Interchanging particle coordinates (i.e. the arguments of $u_i$) in the wave function (\ref{eq:compfn}) should, therefore, not lead to a new micro-state.

How can we make sure that the wave function $\psi_E$ doesn't change under any of the above permutations? One way is to make $\psi_E$ be a linear combination of all the $N!$ possible wave functions that can be formed from such permutations such that the probability density for the quantum system stays the same: $|P\psi|^2=|\psi|^2$.

There are two ways to satisfy this condition. The first is when $\psi$ is symmetric in all its arguments such that $P\psi = \psi$ for all possible permutations. The second way is when $\psi$ is anti-symmetric in its arguments such that
$$
	\psi =
	\begin{cases}
		+\psi \text{ if $P$ is an even permutation, or}\\
		-\psi \text{ if $P$ is an odd permutation.}
	\end{cases}
$$

We will denote these solutions as $\psi_S$ and $\psi_A$ respectively. They can be written in terms of the Boltzmann wave function (\ref{eq:compfn}) as
$$
	\psi_S=c\sum_P P\psi_\text{Boltz}
$$
and
$$
	\psi_A=c\sum_P \delta_PP\psi_\text{Boltz}
$$
where $c$ is a normalisation constant and $\delta_PP$ is $+1$ or $-1$ depending on whether $P$ is an even or odd permutation, respectively.

The wave function $\psi_A$ can be written as the \emph{Slater determinant}
$$
	\Psi_A = c\det\begin{bmatrix}
		u_i(1) & u_i(2) & u_i(3) & \dots & u_i(N)\\
		u_j(1) & u_j(2) & u_j(3) & \dots & u_j(N)\\
		u_k(1) & u_k(2) & u_k(3) & \dots & u_k(N)\\
		\vdots & \vdots & \vdots & \ddots & \vdots \\
		u_z(1) & u_z(2) & u_z(3) & \dots & u_z(N)
	\end{bmatrix}
$$

When calculating the terms in the expansion of the determinant, the leading diagonal is the Boltzmann wave function (\ref{eq:compfn}) while the other terms are permutations of it. The signs for the permuted terms appear automatically in the right order as the determinant is calculated.

Exchanging a pair of arguments in the wave function is the same as swapping a pair of columns of the matrix and causes the sign of the contribution to the determinate to change.

If two or more particles are put into the same energy state, then the matrix has two or more identical rows and the determinant becomes zero --- the wave function vanishes. This implies that an anti-symmetric wave function corresponds to a quantum system where no two particles can be in the same state, i.e. the Pauli exclusion principle that applies to fermions. The statistics governing such systems are called \emph{Fermi-Dirac statistics}.

For Fermi-Dirac statistics the weight factor $W_{F.D.}\{n_i\}$ for the state of the system is given by
$$
	W_{F.D.}\{n_i\} =
	\begin{cases}
		1 \text{ if } \sum_i n_i^2 = N\\
		0 \text{ if } \sum_i n_i^2 > N
	\end{cases}
$$

For symmetric wave functions, there is no such restriction on the number of particles per state $n_i$ and the resulting statistics are called \emph{Bose-Einstein statistics}.

Here the weight factor is
$$
	W_{B.E.}\{n_i\} = 1,\qquad i=0,1,2,\ldots.
$$

It is worth noting that although the properties we derived here all assumed systems of non-interacting particles, the same basic properties hold for systems of interacting particles where, even though the wave function for the system $\psi({\bf r})$ can not be expressed in terms of single-particles wave functions $u_i(r_m)$, it will nevertheless be either symmetric or anti-symmetric, resulting in either Bose or Fermi statistics.

\subsection{Fermionic and bosonic partition functions}
With what we have now covered on non-interacting quantum particles, in combination with what we learnt in the previous section on classical ensembles we are now in a position to introduce the quantum version of the partition function. We wil jump straight to the most commonly used case --- the fermionic and bosonic grand canonical partition functions.

\subsubsection{Bose-Einstein Statistics}
For bosons, all fillings $n_k$ of the energy levels are allowed. Each particle in eigenstate $\psi_k$ contributes energy $\epsilon_k$ and chemical potential $\mu$, so for each particle the partition function is
$$
	Z_k^\text{boson} = \sum_{n_k=0}^\infty\exp(-\beta(\epsilon_k-\mu)n_k) = \sum_{n_k=0}^\infty\left(\exp(-\beta(\epsilon_k-\mu))\right)^n_k = \frac{1}{1-\exp(-\beta(\epsilon_k-mu))}.
$$

The partition function for the ensemble is therefore
$$
	Z^\text{boson} = \prod_{k=0} \frac{1}{1-\exp(-\beta(\epsilon_k-mu))}.
$$

The expected number of particles in each state can be calculated as
$$
	\langle n_k \rangle^\text{boson} = -\frac{\partial}{\partial \mu}(-\beta\ln(Z)) = \frac{1}{\exp(\beta(\epsilon_k-\mu))-1}.
$$

\subsubsection{Fermi-Dirac statistics}
For fermions where $n_k=0$ or $n_k=1$ the single particle partition function is
$$
	Z_k^\text{fermion} = \sum_{n_k=0}^1 \exp(-\beta(\epsilon_k-\mu)n_k) = 1 + \exp(-\beta(\epsilon_k-\mu)),
$$
and the partition particle for an ensemble is
$$
	Z^\text{fermion} = \prod_k(1+ \exp(-\beta(\epsilon_k-\mu))).
$$
The expected number of fermions per state is therefore
$$
		\langle n_k \rangle^\text{fermion} = \frac{1}{\exp(\beta(\epsilon_k-\mu))+1}
$$

\subsubsection{Maxwell-Boltzman statistics}
Maxwell-Boltzman statistics describe the distribution of classical particles, however they are sometimes still useful in the context of quantum systems. (Partly because the maths for them is typically slightly simpler). The Maxwell-Boltzman statistics arise from initially treating particles as being distinguishable, but then dividing the number of micro-states by $N\!$ to account for the double-counting of states that results from doing this. This transformation to the statistics of indistinguishable particles amounts to the so-called dilute limit of M-B statistics in which F-D and B-E statistics are equivalent. The way to think about this is that if you have a large number of states for a small number of particles, even in a classical distribution there is a vanishingly small probability of having more than one particle per state, thus the F-D and B-E statistics are equivalent in this limit.

The partition function for M-B statistics is the familiar
$$
	Z^{MB}_{ensemb} = \prod_k \exp(-\beta(\epsilon_k-\mu))
$$
and the expected number of particles in the $k$th energy state is
$$
\langle n\rangle_{MB} = \exp(-\beta(\epsilon_k-\mu)).
$$

\subsection{An example: the quantum harmonic oscillator}
We are going to consider a quantum harmonic oscillator (QHO) with frequency $\omega$ and energy levels $E_n = (n\frac12)\hbar\omega$.

The partition function for the QHO is $Z^\text{QHO} = \sum_n\exp(-\beta E_n)$, where for convenience, I've put $\frac{1}{k_BT} = \beta$. Writing this out in full and exploiting the fact that we have a geometric series $\sum_n x^n = \frac{1}{1-x}$ we get
\begin{eqnarray*}
	Z^\text{QHO} &=& \sum_{n=0}^\infty \exp(-\beta\hbar\omega(n+\frac12))\\
	&=& \exp(-\beta\hbar\omega/2)\sum_{n=0}^\infty(\exp(-\beta\hbar\omega))^n \\
	&=& \exp(-\beta\hbar\omega/2)\frac{1}{1-\exp(-\beta\hbar\omega)}.
\end{eqnarray*}

To find the average internal energy, think back to when we first introduced the partition function to normalize the probability of a particle being in a state with a particular energy $E_n$.

We have
$$
	\langle E\rangle = \sum_n E_n P_n = \frac{\sum_n E_n \exp(-\beta E_n)}{Z} = \frac{-\partial Z}{\partial \beta}\frac{1}{Z} = -\frac{\partial}{\partial \beta}\ln{Z},
$$

So,
\begin{eqnarray*}
	\langle E\rangle^\text{QHO} &=& -\frac{\partial}{\partial}\ln(Z^\text{QHO})\\
	&=&  -\frac{\partial}{\partial\beta} \left[\ln\left( \exp(-\beta\hbar\omega/2)\frac{1}{1-\exp(-\beta\hbar\omega)}\right) \right] \\
	&=& \frac{\partial}{\partial \beta} \left[ \beta\hbar\omega/2 - \ln\left(\frac{1}{1-\exp(-\beta\hbar\omega)}\right)\right]\\
	&=& \frac{\partial}{\partial \beta} \left[ \beta\hbar\omega/2 + \ln\left({1-\exp(-\beta\hbar\omega)}\right)\right]\\
	&=& \hbar\omega/2 + \frac{\hbar\omega\exp(-\beta\hbar\omega)}{1-\exp(-\beta\hbar\omega)}\\
	&=& \hbar\omega\left(\frac12 +\frac{1}{\exp(\beta\hbar\omega)-1}\right)
\end{eqnarray*}

and expected value for the excitation level is $\langle n\rangle^\text{QHO} = 1/(\exp(\beta\hbar\omega)-1)$.
\subsection{Recommended reading}
Although I've made lots of use of \emph{Statistical Mechanics in a Nutshell} in other sections of the course, I feel like it does particularly poorly with quantum systems. \emph{Statistical Mechanics made Simple} does better, but it doesn't give much of an overview of the fundamentals. So, for this section I've made use of \emph{Statistical Mechnaics} by R.K. Pathria, particularly the fundamentals in chapter five. Chapters six through eight give more details on ideal quantum systems and Fermi and Bose systems --- have a look at them too. The other useful text is \emph{Entropy, Order Parameters and Complexity} by J.P. Sethna, specifically chapter seven. You should read sections 7.6 and 7.7 for some simple examples on bosonic and fermionic systems. Finally, chapters five and six of SMmS covers bosonic and fermionic systems well once you've got the fundamentals sorted.


\section{Cost Modelling}

It turns out Tensorflow does not have any cost model. Atleast there is no such
thing in the public release. User has to manually place computation on different
devices. There has been some work on cost modelling of relational operators for
GPU operators \cite{gpudb}. This was relatively easy due to the small number of
operators. For arbitrary operators, it can possibly be done but would require
much more work.

\subsection{Selection}

Symbols:

\begin{itemize}[label={},noitemsep]
\item $B_r$ - read bandwidth of global memory
\item $B_w$ - write bandwidth of global memory
\item $C_r$ - read segment size of global memory
\item $C_w$ - write segment size of global memory
\item $W$ - number of threads in the thread group
\item $\norm{R}$ - cardinality of table R
\item $n$ - number of projected columns
\item $K_i$ - attribute size of the ith projected column
\item $m$ - number of predicate columns
\item $P_i$ - the attribute size of the ith predicate columns
\item $r$ - selectivity of the predicates
\end{itemize}

\noindent \textbf{Old Model}: Scan and write out filter per column with predicate
\begin{align*}
T_1 &= \sum_{i=1}^{m} ( \ceil{\frac{P_i W}{C_r}} + \ceil{\frac{4 W}{C_r}} ) \times \frac{\norm{R}}{W} \times \frac{C_r}{B_r}  +  \sum_{i=1}^{m} \ceil{\frac{4 W}{C_w}} \times \frac{R}{W} \times \frac{C_w}{B_w}
\end{align*}
Read the filter and write results to global memory.
\begin{align*}
T_2 &= \sum_{i=1}^{n} (\ceil{\frac{4 W}{C_r}} + \ceil{\frac{K_i}{4}}) \times \frac{\norm{R}}{W} \times \frac{C_r}{B_r} + \norm{R} \times r \times \sum_{i=1}^{n} \ceil{\frac{K_i}{4}} \times \frac{C_w}{B_w}
\end{align*}

\noindent \textbf{New Model}: Scan all the columns and write out the filter
\begin{align*}
T_1 &= \sum_{i=1}^{m} \ceil{\frac{P_i W}{C_r}} \times \frac{\norm{R}}{W} \times \frac{C_r}{B_r}  +  \ceil{\frac{4 W}{C_w}} \times \frac{R}{W} \times \frac{C_w}{B_w}
\end{align*}
Read the filter and write results to global memory.
\begin{align*}
T_2 &= (\ceil{\frac{4 W}{C_r}} + \sum_{i=1}^{n} \ceil{\frac{K_i}{4}}) \times \frac{\norm{R}}{W} \times \frac{C_r}{B_r} + \sum_{i=1}^{n} \ceil{\frac{r R K_i}{C_w}} \times \frac{C_w}{B_w}
\end{align*}

\noindent \textbf{CPU Model}:

%\begin{center}
%\begin{tabu} to \textwidth { |X| X| X| }
 %\hline
 %Old Model & New Model & CPU Model \\
 %\hline

 %& cell5

 %& cell6 \\
 %\hline
 %cell7 & cell8 & cell9 \\
 %\hline
%\end{tabu}
%\end{center}


\documentclass{article}
\usepackage{amsmath}
\usepackage{enumitem}
\usepackage{graphicx}
\usepackage{listings}
\usepackage{color} %red, green, blue, yellow, cyan, magenta, black, white

\title{Assignment1}
\author{Jakub Slowinski}
\begin{document}
\begin{itemize}
\item Q2.31

\definecolor{mygreen}{RGB}{28,172,0} % color values Red, Green, Blue
\definecolor{mylilas}{RGB}{170,55,241}
\lstset{language=Matlab,%
    %basicstyle=\color{red},
    breaklines=true,%
    morekeywords={matlab2tikz},
    keywordstyle=\color{blue},%
    morekeywords=[2]{1}, keywordstyle=[2]{\color{black}},
    identifierstyle=\color{black},%
    stringstyle=\color{mylilas},
    commentstyle=\color{mygreen},%
    showstringspaces=false,%without this there will be a symbol in the places where there is a space
    numbers=left,%
    numberstyle={\tiny \color{black}},% size of the numbers
    numbersep=9pt, % this defines how far the numbers are from the text
    emph=[1]{for,end,break},emphstyle=[1]\color{red}, %some words to emphasise
    %emph=[2]{word1,word2}, emphstyle=[2]{style},
}

\section*{Matlab Code}

\lstinputlisting{Determinant.m}
\includegraphics{deta1.png}
\\
\includegraphics{detbi.png}
\\

\item Q3.2
  \\ root of f(x) = x - $2e^{-x}$
  \begin{enumerate}[label=(\alph*)]
  \item Bisection method
  \[ a=0, b=1\]
  \[f(0)=0-2e^0=-2\]
  \[f(1)=1-2e^{-1}=0.2642 \]
  \\First iteration:
  \[ x_1 = \frac{a+b}{2} = \frac{0+1}{2} = 0.5 \]
  \[f(x_1)=0.5 - 2e^{-0.5}=-0.7 \]
  \\ as this is a negative number and you need a negative value and positive value between a and b; a now equals $c_1$
  \\Second iteration:
  \[ a=0.5, b=1\]
  \[ x_2 = \frac{0.5+1}{2}=0.75 \]
  \[f(x_2)=0.75 - 2e^{-0.75}=-0.1947 \]
  \\ as this is a negative number, it replaces the previous negative number, a
  \\Third iteration:
  \[ a=0.75, b=1\]
  \[ x_3 = \frac{0.75+1}{2}=0.875 \]
  \[f(x_3)=0.875 - 2e^{-0.875}=0.04 \]
  \\ as this is a positive number, this is the new b
  \\Final iteration:
  \[ a=0.75, b=0.875\]
  \[ x_4 = \frac{0.75+0.875}{2}=0.8125 \]

  \item Secant method
  \[x_1 = 0, x_2=1 \]
  \[f(x_1)= 0-2e^0=-2 \]
  \[f(x_2)= 1-2e^{-1}=0.2642\]
  \\First iteration
  \[x_3 = x_2 - f(x_2) * \frac{x_2-x_1}{f(x_2)-f(x_1)}\]
  \[ x_3= 1-0.2642*\frac{1-0}{0.2642+2} =0.8516 \]
  \[f(x_3)= 0.8833-2e^{-0.8833}=0.0565 \]
  \\Second iteration
  \[x_4 = x_3 - f(x_3) * \frac{x_3-x_2}{f(x_3)-f(x_2)}\]
  \[x_4=0.8833-0.0565*\frac{0.8833-1}{0.0565-0.2642} =0.8516\]
  \[f(x_4)= 0.8516-2e^{-0.8516}=-0.0019 \]

  Third iteration
  \[x_5 = x_4 - f(x_4) * \frac{x_4-x_3}{f(x_4)-f(x_3)}\]
  \[x_5=0.8516+0.0019*\frac{0.8516-0.8833}{-0.019-0.0565} =0.8529\]
  \[f(x_5)= 0.8529-2e^{-0.8529}=0.00054 \]

  Final iteration
  \[x_6 = x_5 - f(x_5) * \frac{x_5-x_4}{f(x_5)-f(x_4)}\]
  \[x_6=0.8529-0.00054*\frac{0.8529-0.8516}{0.00054+0.0019} =0.85261\]

  \item Newton's method
  \[ f(x) = x-2e^{-x}, x_1=1 \]
  \[ f'(x)=2e^{-x}+1\]
  \[ f(x_1)= 1-2e^{-1}=0.2642\]
  \[ f'(x_1)=2e^{-1}+1=1.7358\]

  First iteration
  \[ x_2=x_1 - \frac{f(x_1)}{f'(x_1)}\]
  \[ x_2=1 - \frac{0.2642}{1.7358}=0.8478\]
  \[ f(x_2)= 0.8478-2e^{-0.8478}=-0.0089\]
  \[ f'(x_2)=2e^{-0.8478}+1=1.8567\]

  Second iteration
  \[ x_3=x_2 - \frac{f(x_2)}{f'(x_2)}\]
  \[ x_3=0.8478 - \frac{-0.0089}{1.8467}=0.8526\]
  \[ f(x_2)= 0.8526-2e^{-0.8526}=-0.0001\]
  \[ f'(x_2)=2e^{-0.8526}+1=1.8526\]

  Third iteration
  \[ x_4=x_3 - \frac{f(x_3)}{f'(x_3)}\]
  \[ x_4=0.8526 - \frac{-0.00001}{1.8526}=0.8526\]
  \[ f(x_4)= 0.8526-2e^{-0.8526}=-0.0001\]
  \[ f'(x_4)=2e^{0.8526}+1=1.8567\]

  Final iteration
  \[ x_5=0.8526 - \frac{-0.00001}{1.8526}=0.8256\]
  \end{enumerate}
\item Q4.24
\\
\section*{Matlab Code}
\lstinputlisting{Inverse.m}
\begin{figure}[htbp]
\hspace*{-2cm}
\includegraphics[scale=0.65]{3i.png}
\\
\hspace*{-2cm}
\includegraphics[scale=0.65]{3ii.png}
\end{figure}

\end{itemize}
\end{document}


\subsection{Plane strain bracket}
\paragraph{}
In this example, a plane strain bracket with a downward uniform distributed load on the top is considered (see Fig.~\ref{qdt_fig:ex_bracket_geo_bc}).
The material properties are: Young’s modulus $E = \SI{2e5}{\newton \per \square \meter}$ and Poisson’s ratio $\nu = 0.3$.
    \begin{figure}[H]
        \centering
        \begin{subfigure}[b]{1\linewidth}
            \centering
            \scalebox{1.2}{
                \includegraphics{quadtree/ex_images/ex_bracket_geo.eps}
            }
        \end{subfigure}
        \begin{subfigure}[b]{1\linewidth}
            \centering
            \scalebox{1.3}{
                \includegraphics{quadtree/ex_images/ex_bracket_load.eps}
            }
        \end{subfigure}
        \caption{ Plane strain bracket: geometry and boundary conditions}
        \label{qdt_fig:ex_bracket_geo_bc}
    \end{figure}

\paragraph{}
A total strain energy of $\SI{282.927}{\joule}$ is determined by ANSYS with the mesh shown in Fig.~\ref{qdt_fig:ex_bracket_ansys_mesh}
    \begin{figure}[H]
        \centering
        \scalebox{0.35}{
            \includegraphics{quadtree/ex_images/ex_bracket_ansys_mesh_31002nodes.png}
        }
        \caption{ANSYS mesh for plane strain bracket (62004 DOF)}
        \label{qdt_fig:ex_bracket_ansys_mesh}
    \end{figure}
%
Drawing in AutoCAD will be divided into two parts: base and the holes as shown in Fig.~\ref{qdt_fig:ex_bracket_cad}
    \begin{figure}[H]
        \centering
        \begin{subfigure}[b]{1\linewidth}
            \centering
            \scalebox{0.4}{
                \includegraphics{quadtree/ex_images/ex_bracket_cad_full.png}
            }
            \caption{All}
        \end{subfigure}
        \begin{subfigure}[b]{0.4\linewidth}
            \scalebox{0.2}{
                \includegraphics{quadtree/ex_images/ex_bracket_cad_base.png}
            }
            \caption{Base}
        \end{subfigure}
        \begin{subfigure}[b]{0.4\linewidth}
            \scalebox{0.2}{
                \includegraphics{quadtree/ex_images/ex_bracket_cad_holes.png}
            }
            \caption{Holes}
        \end{subfigure}
        \caption{CAD drawing of plane strain bracket}
        \label{qdt_fig:ex_bracket_cad}
    \end{figure}
% fig- 64,32/8,3
Generated background mesh, coloring and the final result with $res=32$, $s_{max}=4$ and $s_{min}=1$ are shown in Fig.~\ref{qdt_fig:ex_chole_background_mesh}, Fig.~\ref{qdt_fig:ex_chole_mesh_coloring} and Fig.~\ref{qdt_fig:ex_chole_mesh_final}.
\begin{figure}
    \centering
    \scalebox{0.4}{
        \includegraphics{quadtree/ex_images/ex_bracket_background.eps}
    }
    \caption[Background mesh of the bracket]{Background mesh of the bracket : Bold lines represents the input geometry}
    \label{qdt_fig:ex_chole_background_mesh}
\end{figure}
%
\begin{figure}
    \centering
    \scalebox{0.5}{
        \includegraphics{quadtree/ex_images/ex_bracket_colored.eps}
    }
    \caption[Mesh coloring of the bracket]{Mesh coloring of the bracket : Grey area represents the bracket}
    \label{qdt_fig:ex_chole_mesh_coloring}
\end{figure}
%
\begin{figure}
    \centering
    \scalebox{0.3}{
        \includegraphics{quadtree/ex_images/ex_bracket_final.eps}
    }
    \caption[Final mesh of the bracket]{Final mesh of the bracket}
    \label{qdt_fig:ex_chole_mesh_final}
\end{figure}
% -1 - 282.927
% 382*2  - 274.5638 (32-4/8-3)
% 950*2  - 280.1255 (64-4/8-3)
% 2110*2 - 281.0799 (128-4/15-4)
% 4414*2 - 282.0363 (256-4/15-4)
%
% mlp
% 530 - 279.8076
% 1554 - 282.0745
% 4208 - 282.6581
% err = abs([279.8076 ,282.0745,282.6581]-282.927)/282.927; dof = [530,1554,4208]; polyfit(log(dof),log(err),1)
%
% uniform
% 530 - 279.8076
% 1810 - 282.3186
% 6464 - 282.674
% err_un = abs([279.8076 ,282.3186 ,282.674]-282.927)/282.927; dof_un = [530,1810,6464]; polyfit(log(dof_un),log(err_un),1)
%
% ansys-2nd
% 406*2 -  282.412
% 1144*2 - 282.895
% 4270*2 - 282.923
%
% 16462*2 -  282.926
% err_a2 = abs([282.412,282.895,282.923]-282.927)/282.927; dof_a2 = [406,1144,4270]*2; polyfit(log(dof_a2),log(err_a2),1)
%
% ansys-1st
% 185*2 - 275.635
% 650*2 -  280.006
% 1227*2 -  281.502
% 2405*2 - 282.136
% err_a1 = abs([275.635,280.006  ,281.502 ,282.136]-282.927)/282.927; dof_a1 = [185,650,1227,2405]*2; polyfit(log(dof_a1),log(err_a1),1)
Mesh with different parameters are plotted in Fig.~\ref{qdt_fig:ex_bracket_mesh_all} and the convergence study is plotted in Fig.~\ref{qdt_fig:ex_bracket_mesh_conv}
%
\begin{figure}[H]
    \begin{subfigure}[b]{1\linewidth}
        \centering
        \scalebox{0.4}{
            \includegraphics{quadtree/ex_images/ex_bracket_mesh_64_4.eps}
        }
        \caption{Mesh with $res=64$, $s_{max}=4$, 1656 DOFs}
    \end{subfigure}
    \\
    % \begin{subfigure}[b]{1\linewidth}
    %     \centering
    %     \scalebox{0.4}{
    %         \includegraphics{quadtree/ex_images/ex_bracket_mesh_128_4.eps}
    %     }
    %     \caption{Mesh with $res=128$, $s_{max}=4$, 2548 DOFs}
    % \end{subfigure}
\end{figure}

\begin{figure}[H]\ContinuedFloat
    \begin{subfigure}[b]{1\linewidth}
        \centering
        \scalebox{0.4}{
            \includegraphics{quadtree/ex_images/ex_bracket_mesh_128_4.eps}
        }
        \caption{Mesh with $res=128$, $s_{max}=4$, 2548 DOFs}
    \end{subfigure}
    \begin{subfigure}[b]{1\linewidth}
        \centering
        \scalebox{0.4}{
            \includegraphics{quadtree/ex_images/ex_bracket_mesh_256_4.eps}
        }
        \caption{Mesh with $res=256$, $s_{max}=4$, 5464 DOFs}
    \end{subfigure}
    \caption[Mesh of the plane strain bracket]{Mesh of the plane strain bracket}
    \label{qdt_fig:ex_bracket_mesh_all}
\end{figure}

\begin{figure}[H]
    \centering
    \scalebox{0.75}{
        \includegraphics{quadtree/ex_images/ex_bracket_conv.eps}
    }
    \caption[Convergence of the plane strain bracket]{Convergence of the plane strain bracket}
    \label{qdt_fig:ex_bracket_mesh_conv}
\end{figure}
%
Fig.~\ref{qdt_fig:bracket_stress_contour} shows the von Mises equivalent stress for the plane strain bracket.
From Fig.~\ref{qdt_fig:bracket_stress_contour}, it can be observed that the results from the present approach qualitatively match with the FE solution.
\begin{figure}[h!]
    \begin{subfigure}[b]{1\linewidth}
        \centering
        \scalebox{0.4}{
            \includegraphics{quadtree/ex_images/ex_bracket_ansys_vmstr.png}
        }
        \caption{FEM}
    \end{subfigure}
    \begin{subfigure}[b]{1\linewidth}
        \centering
        \scalebox{0.4}{
            \includegraphics{quadtree/ex_images/ex_bracket_strcontour.eps}
        }
        \caption{Quadtree SBFEM}
    \end{subfigure}

    \caption{Von Mises equivalent stress contours for plane strain bracket}
    \label{qdt_fig:bracket_stress_contour}
\end{figure}


\chapter{Problem Formulation} \label{sec:Problem Formulation}
Our models strongly draw from the previous studies \cite{Xue2016Avi2, Xue2016Avi1}, with modifications directed at reducing computation time, as well as getting better results. Since GPUs enable faster computation on tensors (see \Cref{sec:Gpu's Strengths}), both the Identification and the Pricing Problem are formulated as tensor-based 3-layered and 2-layered neural networks respectively using the PyTorch library \cite{PTDocs}. Recognizing that NVIDIA GPUs easily pair with PyTorch \cite{PTDocs}, accelerating tensor operations using NVIDIA's APIs --- CUDA and cuDNN \cite{CUDADocs, cuDNNPaper}, we aim to maximize parallel operations and minimize our models' computation time.

\section{Identification Problem} \label{sec:Identification Problem}
As discussed in \Cref{sec:Introduction}, the model should learn parameters that caused the change in agents' behavior when a certain set of rewards was applied to locations in the experiment region. Learning those parameters will help us understand how agents behaved with a fixed reward distribution, and will enable organizers to redistribute rewards based on that behavior.

Specifically, given datasets $\vect{x_t}$ and $\vect{y_t}$ of agents' visit densities at time $t$, before and after the rewards $\vect{r_t}$ were placed respectively, we want to find the weights that caused the change from $\vect{x_t}$ to $\vect{y_t}$, considering possible influence from environmental factors $\matr{f}$ and distances between locations $\matr{D}$. Although the previous study proposed to learn a single set of weights $\matr{w}$ \cite{Xue2016Avi2}, our proposed model considers two sets of weights $\matr{w_1}$ and $\matr{w_2}$ as it may theoretically result into higher accuracy and lower loss value. Mathematically, the model can be formulated as:
\begin{equation} \label{eqn:iden_problem}
    \begin{aligned}
        & \underset{\matr{w_1}, \matr{w_2}}{\text{minimize}}
        & & Z_I(\matr{w_1}, \matr{w_2}) = \sum_{t} (\omega_t(\vect{y_t} - \matr{P}(\matr{f}, \vect{r_t}; \matr{w_1}, \matr{w_2})\vect{x_t}))^{2}
    \end{aligned}
\end{equation}
where $\omega_t$ is a set of weights (not a learnable parameter) at time $t$ capturing penalties relative to the priority of homogenizing different locations at time $t$. In other words, it highlights if the organizer wishes higher homogeneity at one time unit over another. Elements $p_{u, v}$ of $\matr{P}$ are given as ($\vect{\Theta_i}$ substituted for $\vect{w_i}_v^T$):
\begin{equation} \label{eqn:puv_equation}
p_{u, v} = \frac{\exp(\vect{\Theta_2} \cdot \text{reLU} (\matr{\Theta_1} \cdot [d_{u, v}, \vect{f_{u}}, r_{u}]))}{\sum_{u'} \exp(\vect{\Theta_2} \cdot \text{reLU} (\matr{\Theta_1} \cdot [d_{u', v}, \vect{f_{u'}}, r_{u'}]))} = \frac{\exp(\Gamma_{u, v})}{\sum_{u'}\exp(\Gamma_{u', v})} = \text{softmax}(\Gamma_{u, v})
\end{equation}

To optimize the loss value $Z_I(\matr{w_1}, \matr{w_2})$, the neural network learns the set of weights through multiple epochs of backpropagating the loss using gradient descent. Furthermore, the program preprocesses the dataset to avoid unnecessary sub-epoch iterations, and to promote batch operations on tensors.

\subsection{Structure of Input Dataset for Identifying Weights} \label{sec:Structure of Input Dataset for Identifying Weights}
\begin{figure}[!htbp]
    \begin{subfigure}{.64\textwidth}
        \centering
        \includegraphics[width=\linewidth]{weights_input_dataset}
        \caption{A Tensor representing the Input Dataset $\tens{F}$}
        \label{fig:A Tensor representing the complete Input Dataset}
    \end{subfigure}
    \begin{subfigure}{.35\textwidth}
        \centering
        \includegraphics[width=\linewidth]{zoomup_Fuv}
        \caption{Contents of $\tens{F}[v][u]$: This vector (length $n_F$) contains quantified causes for the change in agents' behavior from $\matr{x_t}$ to $\matr{y_t}$}
        \label{fig:Zoomed-in contents of Fvu}
    \end{subfigure}
    \caption{Visual representation of the Identification Problem's Input Dataset}
    \label{fig:Visual representation of the Identification Problem's Input Dataset}
\end{figure}
Since preprocessing the dataset reduces data operations during model execution, the input dataset, comprising of distance between locations $\matr{D}$, environmental features $\vect{f}$, and given rewards $\vect{r_t}$ (all normalized), is built into a tensor (\Cref{fig:A Tensor representing the complete Input Dataset}) such that operations can be performed on batches of slices $\tens{F}[v]$.

Another advantage of building the dataset as a tensor comes with the PyTorch library, which provides convenient handling and transfer of tensors residing on the Main Memory and GPUs' internal global memory \cite{PTDocs}. \Cref{alg:Constructing the Input Dataset} describes the steps to construct this dataset. During implementation, we preprocess the $\tens{F}$ dataset to reduce computation time during model runs (\Cref{app:Building the Dataset F}).
\begin{algorithm}[!htbp]
    \caption{Constructing the Input Dataset} \label{alg:Constructing the Input Dataset}
    \begin{algorithmic}[1]
        \Function{Build-Dataset}{$\matr{D}, \matr{f}, \matr{r_t}$}
        \State $\matr{D} \gets \Call{Normalize}{\matr{D}}$\Comment{$\matr{D}[u][v]$ is the distance between locations $u$ and $v$}
        \State $\vect{f} \gets \Call{Normalize}{\mathbf{f}, axis = 0}$\Comment{$\mathbf{f}[u]$ is a vector of env. features at location $u$}
        \State $\vect{r_t} \gets \Call{Normalize}{\vect{r_t}, axis = 0}$\Comment{$\vect{r_t}[u]$ is the reward at location $u$}
        \For{$v = 1, 2, \dots, J$}
            \For{$u = 1, 2, \dots, J$}
                \State $\tens{F}[v][u] \gets [\matr{D}[v][u], \vect{f}[u], \vect{r_t}[u]]$ \Comment{As depicted in \Cref{fig:Zoomed-in contents of Fvu}}
             \EndFor
        \EndFor
        \State \Return $\tens{F}$
        \EndFunction
    \end{algorithmic}
\end{algorithm}

\subsection{Minimizing Loss for the Identification Problem} \label{sec:Minimizing Loss for the Identification Problem}
As shown in \Cref{fig:Neural network designed for the Identification Problem}, the neural network is made of 3 fully connected layers --- the input layer, the hidden layer with rectified Linear Units (reLU), and the output layer with softmax$(\cdot)$ function units. The network can also be visualized as a stack of 1-dimensional layers (\Cref{fig:Side view of the network}), with the softmax$(\cdot)$ calculated on the stack's output.
\begin{figure}[!htbp]
    \centering
    \begin{subfigure}{\textwidth}
        \centering
        \includegraphics[width=\textwidth]{weights_net}
        \caption{3-dimensional View of the Network Slice, Taking in $\tens{F}[v]$}
        \label{fig:3-dimensional view of the network slice taking in Fv}
    \end{subfigure}
    \begin{subfigure}{.75\textwidth}
        \centering
        \includegraphics[width=\textwidth]{weights_net_side}
        \caption{Side View of the Network: Output of one such cross-section is $p_{u_i, v}$}
        \label{fig:Side view of the network}
    \end{subfigure}
    \caption{Neural Network Designed for the Identification Problem}
    \label{fig:Neural network designed for the Identification Problem}
\end{figure}

It is important to clarify that the network, which takes in $\tens{F}[v]$ as shown in \Cref{fig:3-dimensional view of the network slice taking in Fv}, is a slice of the original network, which takes in the complete tensor $\tens{F}$, and computes the complete result $\matr{P}^{T}$  per iteration of $t$. In other words, the input and the hidden layers are 3-dimensional, and the output layer is 2-dimensional. Since it is difficult to visualize the complete network on paper, slices of the network are depicted in \Cref{fig:3-dimensional view of the network slice taking in Fv}. \Cref{alg:Algorithm for the Identification Problem} details the steps for learning the parameters $\matr{w_1}$ and $\matr{w_2}$ based on \Cref{eqn:iden_problem,eqn:puv_equation}.

\begin{algorithm}[!htbp]
    \caption{Algorithm for the Identification Problem} \label{alg:Algorithm for the Identification Problem}
    \begin{algorithmic}[1]
        \State $\matr{w_1} \gets \Call{Random}{\;(J,n_F,n_F)\;}$\Comment{$\matr{w_1}$ has dimensions $J \times n_F \times n_F$}
        \State $\matr{w_2} \gets \Call{Random}{\;(J,n_F,1)\;}$\Comment{$\matr{w_2}$ has dimensions $J \times n_F \times 1$}
        \For{$e = 1, 2, \dots, \text{Epochs}$}
            \State $loss \gets 0$
            \For{$t = 1, 2, \dots, T$}
                \State $\tens{F} \gets \Call{Build-Dataset}{\matr{D}, \matr{f}, \matr{r}[t]}$\Comment{Defined in \Cref{alg:Constructing the Input Dataset}}
                \State $\matr{H} \gets  \text{reLU}(\Call{Batch-Multiply}{\tens{F}, \matr{w_1}})$
                \State $\matr{O} \gets \text{softmax}(\Call{Batch-Multiply}{\matr{H}, \matr{w_2}})$
                \State $\matr{P} \gets \matr{O}^T$
                \State $loss \gets loss + (\omega[t](\matr{y}[t] - \matr{P} \cdot \matr{x}[t]))^2$
            \EndFor
            \State $\Call{Gradient-Descent}{loss, \matr{w_1}, \matr{w_2}}$
            \State $\matr{w_1}, \matr{w_2} \gets \Call{Update-Using-Gradients}{\matr{w_1}, \matr{w_2}}$
            \State $\Call{Log-Info}{e, loss}$
        \EndFor
    \end{algorithmic}
\end{algorithm}

\section{Pricing Problem} \label{sec:Pricing Problem}
After learning the set of weights $\matr{w_1}$ and $\matr{w_2}$, highlighting the change in agents' behavior to collect observations, the Pricing Problem aims to redistribute rewards to the all locations such that the predicted agents' behavior, influenced by the new set of rewards, is homogeneous. Thus, given a budget of rewards $\mathcal{R}$, this optimization problem can be expressed as:
\begin{equation} \label{eqn:pricing_problem}
    \begin{aligned}
        & \underset{\vect{r}}{\text{minimize}}
        & & Z_P(\vect{r}) = \frac{1}{n}\lVert \vect{y} - \mean{\vect{y}} \rVert\\
        & \text{subject to}
        & & \vect{y} = \matr{P}(\matr{f}, \vect{r}; \matr{w_1}, \matr{w_2}) \, \vect{x}\\
        &&& \sum_{i} r_i \leq \mathcal{R}\\
        &&& r_i \geq 0
    \end{aligned}
\end{equation}
where elements of $\matr{P}$ are defined as in \Cref{eqn:puv_equation}.

To allocate the rewards $\vect{r}$, the calculations for the pricing problem are akin to that for the Identification Problem (see \Cref{sec:Identification Problem}). However, since only 1 set of rewards need to be optimized, we use an altered 2-layer (input and output layers) network instead of the 3-layered network used for the Identification Problem. While \Cref{eqn:pricing_problem} looks like a typical Linear Programming problem, only a part of the formulation uses LP to constrain the rewards. We calculate $\matr{P}$ using a 2-layered network that minimizes the loss function $Z_P(\vect{r})$ using gradient descent, and constrain the rewards using linear programming. Fundamental processes are described below, whereas specific implementation details are described in \Cref{app:Implementation}.

\subsection{Input Dataset for Finding Rewards}
Since it is the set of rewards $\vect{r}$ that need to be optimized, they must serve as the ``weights'' of the network (``weights'' here refer to the weighted edges of this network and not to the set of calculated weights $\matr{w_1}$ and $\matr{w_2}$). Therefore, the rewards $\vect{r}$ are no longer fed into the network but are its characteristic. Instead, the calculated weights $\matr{w_1}$ are fed into the network, and are ``weighted'' by the rewards.

The observation density datasets ($\matr{x}$ and $\matr{y}$) are also aggregated for all agents such that they give information in terms of locations $u$ only. This is also why rewards vector $\vect{r}$ does not depend on $t$ --- we want a generalized set of rewards for all time $t$ per location $u$. Therefore, the algorithm for constructing $\tens{F}$ (see \Cref{sec:Structure of Input Dataset for Identifying Weights}) is same as \Cref{alg:Constructing the Input Dataset} but with a change --- $\vect{r_t}$ replaced by $\vect{r}$.

\subsection{Calculating Rewards} \label{sec:Calculating Rewards}
\Cref{alg:Solving the Pricing Problem} for finding $\matr{P}$ is very similar to \Cref{alg:Algorithm for the Identification Problem}'s first few steps but without any epochs of $t$. Also, since the model would predict $\vect{y}$, it does not need labels $\vect{y}$ as a dataset. \Cref{alg:Solving the Pricing Problem}'s logic flow is depicted in \Cref{fig:Logic Flow of Algorithm Pricing Problem} to supplement understanding.
\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{logic_alg_pricing}
    \caption{Logic Flow of \Cref{alg:Solving the Pricing Problem}}
    \label{fig:Logic Flow of Algorithm Pricing Problem}
\end{figure}
\begin{algorithm}[!htbp]
    \caption{Solving the Pricing Problem} \label{alg:Solving the Pricing Problem}
    \begin{algorithmic}[1]
        \Function{Forward}{$\matr{D}, \matr{f}, \vect{r}, \matr{w_1}, \matr{w_2}, \vect{x}$}
            \State $\tens{F} \gets \Call{Build-Dataset}{\matr{D}, \matr{f}, \vect{r}}$\Comment{Defined in  \Cref{alg:Constructing the Input Dataset}}
            \State $\matr{O}_1 \gets \text{reLU}(\Call{Batch-Multiply}{\tens{F}, \matr{w_1}})$
            \State $\matr{O}_2 \gets \text{softmax}(\Call{Batch-Multiply}{\matr{O}_1, \matr{w_2}})$
            \State $\matr{P} \gets \matr{O}_2^T$
            \State $\vect{y} \gets \matr{P} \cdot \vect{x}$
            \State \Return $\lVert \vect{y} - \mean{\vect{y}} \rVert / J$
        \EndFunction
        \vspace*{-.7\baselineskip}\Statex\hspace*{\dimexpr-\algorithmicindent-2pt\relax}\rule{\textwidth}{0.1pt}%
        \Statex\hspace*{-\algorithmicindent}\textbf{Main Script}%
        \vspace*{-.6\baselineskip}\Statex\hspace*{\dimexpr-\algorithmicindent-2pt\relax}\rule{\textwidth}{0.1pt}% horizontal rule
        \State $\vect{r} \gets \Call{Random}{\;(J)\;}$\Comment{$\vect{r}$ has dimensions $J$}
        \State $loss \gets \Call{Forward}{\matr{D}, \matr{f}, \vect{r}, \matr{w_1}, \matr{w_2}, \vect{x}}$
        \For{$e = 1, 2, \dots, \text{Epochs}$}

        \State $\Call{Gradient-Descent}{loss, \vect{r}}$
        \State $\vect{r} \gets \Call{Update-Using-Gradients}{\vect{r}}$
        \State $\vect{r} \gets \Call{LP}{\vect{r}, \mathcal{R}}$\Comment{LP($\cdot$) explained in \Cref{sec:Constraining Rewards}}
        \State $loss \gets \Call{Forward}{\matr{D}, \matr{f}, \vect{r}, \matr{w_1}, \matr{w_2}, \vect{x}}$
        \State $\Call{Log-Best-Rewards}{loss, \vect{r}}$\Comment{Record $\vect{r}$ with the lowest $loss$ yet}
        \EndFor
    \end{algorithmic}
\end{algorithm}

\subsection{Constraining Rewards} \label{sec:Constraining Rewards}
After updating the rewards, the program constrains them using LP($\cdot$) such that $\sum_{i}r_i \leq \mathcal{R}$ and $r_i \geq 0$. The LP($\cdot$) finds another set of rewards $\vect{r'}$ such that the absolute difference between new and old rewards ($\sum_{i}|r'_i - r_i|$) is minimum. The mathematical formulation is given in \Cref{eqn:lp_math_constrain_rewards}, which was implemented using SciPy's Optimize Module \cite{SCPOptimizeDocs}. Since the module supports a standard format for doing linear programming, \Cref{eqn:lp_code_constrain_rewards} is used (see \Cref{app:Modeling the Linear Programming Problem in the Standard Format}), which is mathematically equivalent to \Cref{eqn:lp_math_constrain_rewards}.
\begin{multicols}{2}
    \begin{equation} \label{eqn:lp_math_constrain_rewards}
        \begin{aligned}
            & \underset{\vect{r'}}{\text{minimize}}
            & & \sum_{i}|r'_i - r_i|\\ \\
            & \text{subject to}
            & & \sum_{i}r'_i \leq \mathcal{R}\\
            &&& r_i \geq 0
        \end{aligned}
    \end{equation}
    \begin{equation} \label{eqn:lp_code_constrain_rewards}
        \begin{aligned}
            & \underset{[\vect{r'}, \vect{u}]}{\text{minimize}}
            & & \sum_{i} u_i\\
            & \text{subject to}
            & & r'_i - r_i \leq u_i\\
            &&& r_i - r'_i \leq u_i\\
            &&& \sum_{i} r'_i \leq \mathcal{R}\\
            &&& r'_i, u_i \geq 0
        \end{aligned}
    \end{equation}
\end{multicols}

We make a tradeoff by constraining rewards using LP instead of Mixed Integer Programming, which was used by the previous study \cite{Xue2016Avi2}. The tradeoff exists between decreasing the computation time and loosening integer constraints on each reward value. While model-learned integer rewards have worked in real-life testing \cite{Xue2016Avi2}, we cannot say if allowing non-integer values in rewards would produce better or comparable results in real-life, corresponding to the model's prediction. In other words, we can't ensure the predicted rewards' effectiveness in ground testing before \textit{actually} deploying them in the game. Nevertheless, we can estimate how good the predictions can be compared to several baseline indicators. These baseline sets are elaborated in \Cref{sec:Pricing Problem-Optimizing the Original Dataset}.