% --- [ Evaluation Process ] ---------------------------------------------------

\subsection{Evaluation Process}
\label{sec:evaluation_process}

% TODO: rephrase

A preliminary metric for the effectiveness of control flow recovery is how many of the original high-level control flow primitives that are recovered and how many are omitted. The metric for effectiveness should also includes a notion of false positives for high-level primitives that are recovered but are not part of the original source code.

For each test program, the evaluation process consists of the following steps.

\begin{enumerate}
	\item \label{itm:step_c_to_json} Parse the original C source code of the test program to determine the occurrence of each high-level control flow primitive, and store this truth table in JSON format (which serves as a \textit{ground truth} during evaluation).
	\item \label{itm:step_c_to_ll} Convert C source code to LLVM IR using Clang. For test programs consisting of several source files, the LLVM IR is produced using the WLLVM\footnote{Whole Program LLVM: \url{https://github.com/travitch/whole-program-llvm}} project which provides tools for building whole-program LLVM bitcode files from unmodified C or C++ source code.
	\item \label{itm:step_ll_to_dot} Generate a control flow graph for each function defined in the LLVM IR, and store the CFGs in Graphviz DOT format.
	\item \label{itm:step_dot_to_json} Apply each control flow recovery method on the generated CFGs, and compare the recovered high-level primitives against the truth table produced in step 1.
\end{enumerate}

Special consideration has been taken to ensure that the C source code is parsed identically in step 1 and 2, since pre-processor directives and compiler specific macros may otherwise introduce uncertainty into the results. The Python script which generates truth tables in step 1 parses C source files using the official bindings for the Clang library, and the C source files are converted to LLVM IR in step 2 using Clang (also when invoked through WLLVM).
