<reponame>chrisstanford/persephone-darkside<filename>doc/data_specification/latex/document.tex
\documentclass[11pt, letterpaper]{article}

\usepackage{xltxtra}

\usepackage{fontspec}
\defaultfontfeatures{Numbers=OldStyle}
\setmainfont{Minion Pro}

\usepackage{amsmath}    % need for subequations
\usepackage{graphicx}   % need for figures
\usepackage{verbatim}   % useful for program listings
\usepackage{color}      % use if color is used in text
\usepackage{subfigure}  % use for side-by-side figures
\usepackage{lineno}
\usepackage{bigstrut}

\begin{document}
 
\title{The Material Assay Data Format \\ v 2.01}
\author{<NAME> \\ Shanghai Jiaotong University }
\date{Revised 6 February 2013}

\input{commands}

\maketitle 
 
% ----------------------------------------------------------------------------------------------------------- 
\section{Introduction}
% ----------------------------------------------------------------------------------------------------------- 

This document specifies a data format for encoding measurements (assays) of material radiopurities. The data format consists of a JSON schema, restrictions on the content of certain fields and rules that allow the schema to be extended.

% ----------------------------------------------------------------------------------------------------------- 
\section{The data format}
% ----------------------------------------------------------------------------------------------------------- 

\subsection{Overview} 

The data structure is divided into three top level sub-structures known as \texttt{sample}, \texttt{measurement} and \texttt{data source}. The first captures information on physical characteristics and origin the thing being counted. The second deals with the actual measurement and its results. The third concerns the origin and input of the data contained in the document. This is the structure in outline:

\begin{small}
\begin{verbatim}
   {

      "type":          "measurement",

      "grouping":      "Experiment name or similar",
      
      "sample":        {...},  
      "measurement":   {...},   
      "data_source":   {...},   
 
      "specification": "X.XX"

   }
\end{verbatim}
\end{small}
where the \verb|{...}| indicate the three JSON sub-structures that will be described in detail below. Other fields are:

\begin{description}

  \item[\texttt{grouping}] The group to which the document belongs. Usually this will be the name of the experiment that made the measurement.

  Conventions: less than 100 characters; no period; required.

  Example: \textit{Majorana Demonstrator}

  \item[\texttt{specification}] The version number of the Material Assay Data Format specification used to encode the document.

  Conventions: \texttt{[MAJOR VERSION].}\texttt{[MINOR VERSION]}; required.

  Example: \textit{2.01}

\end{description}  
 
\subsection{Sample} % ............................................ 

The sample sub-structure has this form:

\begin{small}
\begin{verbatim}
   "sample": {

      "name":            "Short description",
      "description":     "Detailed description",     
      "id":              "Identification number",
      "source":          "Where it came from",

      "owner":
         {
            "name":         "Who owns it", 
            "contact":      "Institution or email/postal address"
         }
       
   }
\end{verbatim}
\end{small}

\newpage

\begin{description}

  \item[\texttt{name}] A concise description of the sample, typically indicating the kind of material and its origin. It can be considered the heading of the document. 
  
  Conventions: less than 100 characters; single line; no period; required.
  
  Example: \textit{Corning 7980 ArF fused silica}
  
  Example: \textit{Homestake rhyolite, 4850 ft level, sample A3}

  \item[\texttt{description}] A detailed description of the sample, typically including information on its form, appearance and processing. 
  
  Conventions: written as a sentence or comma-separated list of items; single line; no limit on length and level of detail; optional period; can be left blank.
  
  Example: \textit{Polished wafer of Corning 7980 ArF high purity fused silica, disc shape, 1 mm thick, 20 mm diameter, lot number 12345.}

  \item[\texttt{id}] An identification number associated with the sample, typically used by a counting facility, or other organization, to track the sample.
  
  Conventions: an alphanumeric identification string; can be left blank.
  
  Example: \textit{3128a}

  Example: \textit{2010-AF-0000004567-b}

  \item[\texttt{source}] Information on where the sample came from, typically consisting of a company name and purchase date.

  Conventions: can be left blank.
  
  Example: \textit{Mark Optics Incorporated, purchased April 30th 2011.} 
  
  \item[\texttt{owner}] Information on who owns the sample. It may refer to ownership at the time of the measurement or at some later date.

  Conventions: can be left blank.
  
  Example: \textit{<NAME> // <EMAIL>}   

  Example: \textit{<NAME> // Government National Laboratory}   
  
\end{description}

\newpage

\subsection{Measurement} % ............................................ 

The measurement sub-structure has this form:

\begin{small}
\begin{verbatim}
   "measurement": {
    
      "institution":     "Where the count was done",
      "technique":       "The technique that was used",     
      
      "date":            [],
      
      "requestor":
         {
            "name":         "Who managed the measurement", 
            "contact":      "Institution or email/postal address"
         },                  
         
      "practitioner":
         {
            "name":         "Who did the measurement", 
            "contact":      "Institution or email/postal address"
         },
         
      "description":     "Detailed multi-line description of
                          the procedure and results",
                            
      "results":
         [
            {
               "isotope":     "II-AAA or II or description",
               "type":        "measurement or limit or range",
               "value":       [],
               "unit":        "Unit"
            },
            ...
         ]
               
   }
\end{verbatim}
\end{small}

\newpage
\begin{description}

  \item[\texttt{institution}] The name of the institution where the measurement was made.

  Conventions: can be left blank.
  
  Example: \textit{Oak Ridge National Laboratory}
  
  Example: \textit{LBNL LBF}

  \item[\texttt{technique}] The technique used to make the measurement.

  Conventions: can be left blank.
  
  Example: \textit{ICP-MS}
  
  Example: \textit{Gamma}

  \item[\texttt{date}] The date on which the measurement took place. This array can contain either a single value indicating the date on which the experiment took place (or began), or a two values indicating the period over which the measurement was made.

  Conventions: entered in the format \texttt{YYYY-MM-DD} or \texttt{YYYY-MM} or \texttt{YYYY}; can be left empty.

  Example: \textit{1980-04}

  Example: \textit{1980-04-26}

  \item[\texttt{requestor}] Information on who commissioned, funded, managed or was otherwise responsible for the measurement taking place.

  Conventions: can be left blank.
  
  Example: \textit{<NAME> // <EMAIL>}     

  \item[\texttt{practitioner}] Who was responsible for actually making the measurement.

  Conventions: can be left blank.
  
  Example: \textit{<NAME> // <EMAIL>}  
  
  \newpage
  
  \item[\texttt{description}] A detailed description of the measurement. Typically this is a substantial piece of text duplicated from the practitioner's report. It may repeat data broken out into other fields.

  Conventions: multi-lined block of text; can be left blank.
  
  Example: 
  
\textit{Of the 50 packages of HYSOL Ag-epoxy you brought to the LBNL
Low Background Facility, 25 packages have been analysed at the
Berkeley Facility as a single sample of unopened packages,
consisting of all items in Lot 1 and Lot 2 (your designation).}

\textit{The total weight of all 25 unopened packages is 143 grams,
while the weight of the unmixed Ag-epoxy (at 2.65 grams per
package) is 66 grams.  Results are listed below, where all
values are based on the 143 gram total weight of the sample.}


     \textit{Detector:        MERLIN(BKY)             U:  <100    ppb}
     
     \textit{Sample:          S6MB annulus           Th:  <230    ppb}

     \textit{Weight:          43 grams               K:  84(8) ppm}

     \textit{Data file:       5695S}

     \textit{Count time:      48100 sec}
 
 \newpage
 
  \item[\texttt{results}] A list of results for each isotope is included. The three result types are \texttt{measurement}, \texttt{range} and \texttt{limit}. The meaning of the contents of the \texttt{value} array varies according to the result type and the size of the array:
    
\begin{tabular}{lll}

\hline

\texttt{type} & \texttt{value} & Description \bstb\\

\hline

  \texttt{measurement} &  \texttt{[0] }         & measurement with no error \bst\\
             & \texttt{[0,0] }     & measurement with symmetric error \\
           & \texttt{[0,0,0]}  & measurement with asymmetric error {(+, -)} \bsb\\


 
   \texttt{range} &  \texttt{[0,0] }     & range {(lower, upper)} \bst\\
            & \texttt{[0,0,0]}  & range with confidence level  \bsb\\

 
   \texttt{limit} &  \texttt{[0] }     & limit \bst\\
             & \texttt{[0,0]}  & limit with confidence level  \bsb\\

\hline

\end{tabular}\newline
    
   General conventions: the isotope name should be entered as the two letter chemical symbol with or without the mass number (in \textit{exceptional circumstances} a more detailed description, such as 'U early' may be given); permissible units are \texttt{pct, ppm, ppb, ppt, ppq, mBq/kg, uBq/kg, nBq/kg, n/a}; the confidence level (c.l.) on a limit or range is a number 0 < c.l. < 100; if the confidence level on a limit or range is omitted then it will be interpreted as a 95\% limit; confidence levels on measurements are assumed to be 68\%; documents with zero measurements \textit{are} permitted.
   
  Example: \textit{U-238 // "measurement" // [120,10] // "ppb"}  

  Example: \textit{Th-232 // "range" // [40,50,95] // "uBq/kg"}

  Example: \textit{Th-232 // "limit" // [40] // "uBq/kg"} 
  
\end{description} 

\newpage
\subsection{Data source} % ............................................ 

The data source sub-structure has this form:

\begin{small}
\begin{verbatim}
   "data_source": {

      "reference":       "Where the data came from",
      "input":
         {
            "name":         "Who created this document", 
            "contact":      "Institution or email/postal address",
            "date":         []
         },
      
      "notes":           "Comments on/issues with data entry"

   }
\end{verbatim}
\end{small}

\begin{description}

  \item[\texttt{reference}] Where the information in the document was taken from.

  Conventions: required.
  
  Example: \textit{D. Leonard et al., Nucl. Instr. Meth. A 591, 490 (2008).}

  Example: \textit{CERNEXPERIMENT internal report TECH-DOC-0004}

  \item[\texttt{input}] Information on who entered the data or who was responsible for the data entry.

  Conventions: entered in the format YYYY-MM-DD; required.
  
  Example: \textit{Alice Undergraduate // <EMAIL>}     

  \item[\texttt{notes}] Comments on issues that arose during the input of the data, typically focussing on the interpretation of the source data.
  
  Conventions: multiple line block of text; can be left blank.
  
  Example: \textit{In the results table the paper refers to `U conc.' and `Th conc.' as the things being measured. But from the text it's clear that what was actually measured were concentrations of `U-238' and 'Th-232' and so this is what I entered in the results section.}
  
\end{description}

\newpage

\subsection{Extendability} % ............................................
\label{sec:opt}

\subsubsection{The \texttt{user} array}

The sub-structures \texttt{measurement}, \texttt{sample} and \texttt{data\_source} may each contain additional structured information in form of an optional \texttt{user} array. This ability to extend the core specification should only be used in \textit{exceptional circumstances}. The \texttt{user} array takes this form:

\begin{small}
\begin{verbatim}
   "user": 
      [
         {
            "name":                "Short description",
            "description":         "Detailed description",         
            "type":                "measurement or limit 
                                      or range or string",           
            "value":               [] or "",             
            "unit":                "Associated unit"
         },     
         ...
      ]
\end{verbatim}
\end{small}



\begin{description}

  \item[\texttt{name}] Display name.

  Conventions: required.
  
  Example: \textit{Data file}


  \item[\texttt{description}] Description.

  Conventions: optional.
  
  Example: \textit{Size of the Marinelli beaker} 
  
\newpage
  \item[\texttt{type}] The type of data stored in \texttt{value}.

  Conventions: Following the same convention as the \texttt{measurement} sub-structure, this field describes the type of data stored in the \texttt{value} field; \textit{string} is added option; required. 
  
  Example: \textit{measurement} 

  Example: \textit{range}
  
  Example: \textit{limit}
  
  Example: \textit{string}  
  
  \item[\texttt{value}] An array of numbers or a single string.

  Conventions: required.

  Example: \textit{[304.1]}
  
  Example: \textit{[0.1, 0.2]}   

  Example: \textit{[21.1, 95]}   

  Example: \textit{AB444-D.spe}      

  \item[\texttt{unit}] Unit.

  Conventions: optional.

  Example: \textit{cm}

\end{description}

\newpage

\subsubsection{Example of appropriate use}

\noindent Below is an example of appropriate use. Here a user - a hypothetical low background counting facility - has decided to include a field to store the name of the measurement data file (containing the spectrum taken with the HPGe detector). The facility requires this number to have special prominence, and to be a separate field by which documents can be searched and sorted. At the same time it is a number of no interest to anyone outside of the facility.

\begin{small}
\begin{verbatim}
   "measurement": {
    
      "institution":     "...",
      "technique":       "...",
      "date":            [""],
      
      "requestor":     { "name": "...", "contact": "..." },
      "practitioner":  { "name": "...", "contact": "..." },
               
      "description":     "...",

      "results": [
         {
            "isotope":   "...",
            "type":      "measurement",
            "value":     [0],
            "unit":      "Unit"
         }
      ],

      "user": [
         { 
            "name":      "<NAME>", 
            "type":      "string",
            "value":     "2010-A-0092" 
         }
      ]
      
   }
\end{verbatim}
\end{small}

\newpage

\subsection{Changes} % ............................................

\begin{description}
  \item[2013-01-14] (to v1.01) Correct insignificant typos.
  \item[2013-01-14] (to v1.01) Add comment that uncertainties on measurements are expected to be 68\% c.l. and correct default confidence level on limits from 90\% to 95\%.
    \item[2013-01-30] (to v2.00) Change the way optional fields are handled, remove \texttt{m\_} and \texttt{u\_} prefixes.
    \item[2013-01-30] (to v2.00) Convert data to a one- or two-valued array.
    \item[2013-01-30] (to v2.00) Revise the way the results are handled. Now use a unified structure with a \texttt{type}. Combine values, error and limits into a variable-length \texttt{value} array.
    \item[2013-02-06] (to v2.01) Results type \texttt{meas} becomes \texttt{measurement}.
    \item[2013-02-06] (to v2.01) Rename \texttt{opt} array to user and revise it's structure.
\end{description}



\end{document}
