Drinking Water: DOD Has Acted on Some Emerging Contaminants but Should Improve Internal Reporting on Regulatory Compliance

According to DOD, about 3 million people in the United States receive drinking water from DOD public water systems, which are to comply with EPA and state health-based regulations. EPA and DOD have detected elevated levels of two unregulated, DOD-identified emerging contaminants found in firefighting foam—PFOS and PFOA—in drinking water at or near installations. Perchlorate, an unregulated chemical used by DOD in rocket fuel, can also be found in drinking water.

The Senate Report accompanying a bill for national defense authorization for fiscal year 2017 included a provision for GAO to review DOD management of drinking water contaminants. This report examines the extent to which DOD has (1) internally reported data on compliance with health-based drinking water regulations at military installations and used those data to assess compliance at its two types of public water systems, and (2) taken actions to address concerns with its firefighting foam and elevated levels of PFOS, PFOA, and perchlorate in drinking water at or near military installations. GAO reviewed DOD guidance and EPA drinking water regulations, advisories, and orders; analyzed DOD and EPA drinking water data; and visited seven installations from among those addressing emerging contaminants in drinking water.

The Department of Defense (DOD) has not internally reported all data on compliance with health-based drinking water regulations or used available data to assess compliance. DOD data for fiscal years 2013-2015 indicate that DOD public water systems complied with Environmental Protection Agency (EPA) and state health-based drinking water regulations at levels comparable with other systems in the United States. However, the military departments did not report all violations to DOD, i.e., while 77 installations reported violations to DOD, GAO found that at least 16 additional installations did not. Until DOD takes steps to increase the clarity and understanding of its internal reporting requirements, it may not have the data it needs to fully oversee compliance. DOD also has not used its data to determine why its two types of systems—one that provides DOD-treated water and another that provides non-DOD-treated water—have different compliance rates. Specifically, DOD's data indicate that about 99 percent of the people who received non-DOD-treated drinking water were served by systems with no violations, while about 89 percent of the people who received DOD-treated drinking water were served by systems with no violations. Absent further analysis of its data, DOD may not be able to improve overall compliance.

DOD has initiated actions to address concerns with both its firefighting foam and also with elevated levels in drinking water of perfluorooctane sulfonate (PFOS), perfluorooctanoic acid (PFOA), and perchlorate, which are DOD-identified emerging contaminants. PFOS and PFOA can be found in DOD's firefighting foam. DOD has restricted its use of this foam and is funding efforts to develop a new foam that meets DOD performance requirements. Additionally, at 11 military installations (see fig.), DOD has shut down wells, provided alternate water sources, or installed water treatment systems to respond to elevated levels of PFOS and PFOA, at times in response to EPA and state orders.

GAO is making five recommendations to improve DOD's reporting and use of data on compliance with health-based drinking water regulations. DOD concurred with the recommendations.

Every year about 3 million people who live and work on military  installations in the United States receive drinking water from a  Department of Defense (DOD) public water system, according to the  department. These people are served by two different types of public  water systems. Specifically, about two-thirds of these people are provided  DOD-treated drinking water, and about one-third are provided non-DOD- treated drinking water—that is, drinking water from DOD systems that  have been privatized or that obtain treated drinking water from a local  utility. DOD’s public water systems, like all public water systems, are  required to comply with legally enforceable drinking water regulations that  are issued by the Environmental Protection Agency (EPA) under the Safe  Drinking Water Act, as well as with any additional regulations issued by  state environmental agencies. DOD policy also requires the military  departments to report annually on compliance with health-based  regulations at their installations’ public water systems to the Assistant  Secretary of Defense for Energy, Installations, and Environment (ASD  (EI&E)) within the Office of the Secretary of Defense.

In May 2016, EPA issued a drinking water health advisory— nonenforceable technical guidance—for perfluorooctane sulfonate  (PFOS) and perfluorooctanoic acid (PFOA), which are part of a larger  group of manufactured compounds called perfluorinated chemicals  (PFCs) and can be found in firefighting foam used by DOD since the  1970s. PFCs are widely used to make everyday products more resistant  to stains, grease, and water, such as by keeping food from sticking to  cookware and making clothes and mattresses more waterproof.  According to the Agency for Toxic Substances and Disease Registry,  exposure to elevated levels of PFOS and PFOA could cause increased  cancer risk and other health issues in humans. EPA and DOD have  detected elevated levels of PFOS and PFOA in drinking water at or near  DOD installations, and EPA has also found these chemicals in drinking  water at non-DOD public water systems across the United States.

DOD has included PFOS and PFOA on its list of emerging contaminants.  DOD defines emerging contaminants as chemicals or materials that the  department currently uses or plans to use that present a potentially  unacceptable human health or environmental risk; have a reasonably  possible pathway to enter the environment; and either do not have  regulatory standards based on peer-reviewed science, or their regulatory  standards are evolving due to new science, detection capabilities, or  pathways. DOD has also included perchlorate, which is a chemical that  DOD uses in rocket fuel, on its list of emerging contaminants. Like PFOS  and PFOA, perchlorate has not been regulated by EPA under the Safe  Drinking Water Act but does have an EPA-issued interim drinking water  health advisory. Overall, DOD’s list of emerging contaminants includes 21 contaminants that can be found in drinking water: 10 that have been  regulated by EPA under the Safe Drinking Water Act and 11 that are  currently unregulated but have an EPA-issued drinking water health  advisory.

Senate Report 114-255 accompanying a bill for national defense  authorization for fiscal year 2017 included a provision for us to review  DOD’s efforts to manage contaminants in drinking water. This report  examines the extent to which DOD has (1) internally reported data on  compliance with health-based drinking water regulations at military  installations and used those data to assess compliance at its two types of  public water systems and (2) taken actions to address concerns with its  firefighting foam containing PFCs and elevated levels of PFOS, PFOA,  and perchlorate in drinking water at or near military installations.

For objective one, we analyzed data reported by the military departments  to ASD (EI&E) on compliance with and violations of health-based drinking  water regulations at DOD public water systems for fiscal years 2013  through 2015 (the most recent data available at the time of our review).  We compared data from EPA’s Safe Drinking Water Information System  for the same time frame to determine the extent to which violations  recorded in the EPA system were also reported by the military  departments to ASD (EI&E). We also analyzed the data to identify any  differences in violations between DOD- and non-DOD-treated drinking  water. We assessed the reliability of the DOD and EPA data on violations  of health-based drinking water regulations by reviewing relevant  documentation, testing the data for obvious errors, and interviewing  knowledgeable officials. As we have previously found, EPA’s data system  may not contain all public water system violations because states have  generally under-reported violations. During this review, we also found  that some public water system identification numbers for DOD  installations could not be matched with EPA’s data system and, therefore,  these identification numbers were excluded from our analysis. As a  result, some DOD installation violations may be missing from the data,  and we may not have comprehensive violations data for health-based  drinking water regulations at DOD installations. Nonetheless, we  determined that DOD and EPA data were sufficiently reliable for the  purposes of identifying whether any drinking water violations were  recorded in EPA’s system but not internally reported within DOD, as well  as to indicate possible differences in reported violations for DOD’s two  types of public water systems. We evaluated the military departments’  reported data and DOD’s use of these data to determine compliance with  DOD’s reporting requirements as defined by DOD’s environmental  compliance instruction and with Standards for Internal Control in the  Federal Government. According to these standards, quality information is  needed to achieve an organization’s objectives, management is to  monitor performance over time and promptly resolve any findings, and  actions such as improved communication to and additional training for  personnel are helpful for an organization to meet its objectives. We also  discussed our analysis with ASD (EI&E) and military department officials.

For objective two, we reviewed DOD policies on and requirements for  firefighting foam, as well as documents related to the research and  development of a PFC-free firefighting foam. We also interviewed military  department and installation officials to discuss DOD actions regarding  current and future use of firefighting foam. Additionally, we reviewed  administrative orders issued by EPA and one state regulator (Ohio)  directing DOD to address elevated levels of PFOS and PFOA, and we  interviewed officials from the EPA regions (1 and 3) and the state (Ohio)  that issued those orders, as well as DOD officials responsible for  responding to those orders. We also reviewed drinking water guidance  documents from ASD (EI&E) and the military departments on PFOS,  PFOA, and perchlorate and obtained DOD data as of December 2016  (the most recent data available at the time of our review) on testing and  response activities for those contaminants. We assessed the reliability of  the data by examining the data for obvious errors and inconsistencies;  comparing the data, where applicable, with other information collected;  and interviewing knowledgeable officials. We found the data to be  sufficiently reliable for our purposes of describing DOD-reported actions  and costs for addressing PFOS and PFOA. To obtain additional  information on DOD actions to address emerging contaminants, we  visited at least two installations per military department—seven  installations total—that we selected because DOD was investigating or  responding to unregulated DOD-identified emerging contaminants in  drinking water at those installations. We provide further details on our  scope and methodology in appendix I.

DOD has two types of public water systems that provide drinking water to  people that live and work on military installations. The first type provides  drinking water that has been treated by DOD. The second type provides  water treated by a private company or a local utility, which we refer to as  “non-DOD-treated” drinking water. Drinking water systems vary by size  and other factors, but they most typically include a supply source,  treatment facility, and distribution system. A water system’s supply source  may be a reservoir, aquifer, well, or a combination of these sources. The  treatment process for surface water generally uses sedimentation,  filtration, and other processes to remove impurities and harmful agents,  and disinfection processes such as chlorination to eliminate biological  contaminants. Distribution systems are comprised of water towers, piping  grids, pumps, and other components to deliver treated water from  treatment systems to consumers.

Drinking Water Regulations and Administrative Orders

EPA regulates drinking water contaminants under the Safe Drinking  Water Act by issuing legally enforceable standards, known as National  Primary Drinking Water Regulations, which generally limit the levels of  these contaminants in public water systems. EPA has issued such  regulations for approximately 90 drinking water contaminants. In  accordance with the Safe Drinking Water Act, EPA may authorize a state  to have primary enforcement responsibility for drinking water regulations,  as long as the state has, among other things, drinking water regulations  that are no less stringent than the National Primary Drinking Water  Regulations.

The Safe Drinking Water Act also authorizes EPA to take emergency  actions necessary to protect public health when informed that a  contaminant is present in or is likely to enter a public water system or an  underground source of drinking water that may present an imminent and  substantial endangerment. For example, EPA may issue administrative  orders, which generally include actions to be taken, such as remediating  contaminated sources of drinking water or requiring the provision of  alternative water supplies. State regulators may also issue orders to  public water systems to address contaminated drinking water.

Public water systems, including the DOD public water systems that  provide drinking water to about 3 million people living and working on  military installations, are required to comply with EPA and state drinking  water regulations. EPA divides violations of drinking water regulations  into two types: (1) health-based violations and (2) other types of  violations that include violations of monitoring, reporting, and public  notification requirements. Under the Safe Drinking Water Act, EPA also  is required to identify unregulated contaminants that present the greatest  health concern, establish a program to monitor drinking water for  unregulated contaminants, and decide whether or not to regulate at least  five such contaminants every 5 years. EPA has not regulated any new  contaminants using this process since 1996.

DOD’s environmental compliance policy states that ASD (EI&E) is  responsible for providing guidance, oversight, advocacy, and  representation for environmental compliance programs—to include  overseeing the military departments’ compliance with health-based  drinking water regulations at DOD public water systems. The policy  directs the military departments to annually report to ASD (EI&E) the total  population receiving water from both “regulated” and “other” DOD public  water systems—referred to in this report as DOD public water systems  that provide DOD- and non-DOD-treated drinking water, respectively— that did and did not attain all Safe Drinking Water Act health-based  drinking water standards. The policy also requires the military  departments to report information regarding each instance health-based  drinking water standards were not attained during the reporting period, to  include the name and location of the military installation; the nature of the  issue (e.g., the contaminant type); the DOD population affected; the  duration of the issue; the corrective actions taken or planned (e.g.,  flushing the system, resampling the water, or implementing system  upgrades); and the estimated date for achieving the standard.

In addition to issuing drinking water regulations, EPA may also publish  drinking water health advisories. In contrast to drinking water regulations,  health advisories are nonenforceable. Drinking water health advisories  provide technical guidance on health effects, analytical methodologies,  and treatment technologies. These advisories recommend the amount of  these contaminants that can be present in drinking water—”health  advisory levels”—at which adverse health effects are not anticipated to  occur over specific exposure durations, to include 1 day, 10 days, several  years, or over a lifetime. EPA issues provisional health advisories to  provide information in response to an urgent or rapidly developing  situation. DOD’s list of emerging contaminants includes 11  contaminants, including PFOS, PFOA, and perchlorate, for which EPA  has issued a drinking water health advisory. Specifically,

PFOS. PFOS is part of a larger group of fluorinated organic chemicals  that have been incorporated into an array of consumer products (i.e.,  to make some more resistant to stains, grease, and water) and also in  firefighting foam used by DOD and civilian airports. According to EPA,  the major manufacturer of PFOS in the United States voluntarily  agreed to phase out production of the chemical in 2002. According to  EPA’s health advisory, exposure to PFOS may remain possible due to  legacy uses, existing and legacy use in imported goods, and the  chemical’s “extremely high persistence” in the environment. According  to the EPA, exposure to PFOS may result in adverse health effects,  such as fetal developmental effects during pregnancy or to breastfed  infants, cancer, liver damage, immune effects, thyroid effects, and  other effects. See table 1 for details of the EPA provisional health  advisory that was issued in 2009 and the lifetime health advisory that  was issued in 2016, which superseded the provisional health  advisory.

PFOA. PFOA is a fluorinated organic chemical that has been used in  generally the same products as PFOS, including firefighting foam  used by DOD and civilian airports. According to EPA, PFOA was  voluntarily phased out by eight major companies in the manufacturing  of their products at the end of 2015. According to the EPA, adverse  health effects from exposure to PFOA are similar to those for PFOS.  See table 1 for details of the EPA provisional health advisory that was  issued in 2009 and the lifetime health advisory that was issued in  2016, which superseded the provisional health advisory.

Perchlorate. Perchlorate is commonly used in solid propellants,  fireworks, matches, signal flares, and some fertilizers, and has been  used by DOD for rocket fuel and ammunition. EPA published an  interim health advisory for perchlorate in 2008; the interim health  advisory level was set at 15 parts per billion. According to the health  advisory, perchlorate can disrupt the functions of the thyroid gland.

In 2009, DOD issued a policy on the identification, assessment, and risk  management of emerging contaminants that have the potential to impact  DOD. According to that policy, chemicals and materials used or planned  for use by DOD that meet the definition of an emerging contaminant  should be identified as early as possible. The policy further states that  DOD is to assess and, when appropriate, take action to reduce risks  posed by its emerging contaminants to people; the environment; and  DOD missions, programs, and resources. Where necessary, DOD is to  perform sampling, conduct site-specific risk assessments, and take  response actions for emerging contaminants released from DOD facilities,  in accordance with relevant statutes.

According to the DOD policy on emerging contaminants, ASD (EI&E) is to  develop and maintain a list of emerging contaminants with potential or  probable high risk to the department’s personnel and functions. As of  April 2017, DOD’s list of emerging contaminants comprised 49 chemicals  or substances. According to our analysis of EPA documents, DOD’s list  includes 21 contaminants that can be found in drinking water. Of these 21  contaminants, 10 contaminants have been regulated by EPA under the  Safe Drinking Water Act, and 11 contaminants are currently unregulated  but have an EPA-issued drinking water health advisory. The other 28  DOD-identified emerging contaminants do not have EPA drinking water  regulations or health advisories. Appendix II provides more information on  the drinking water regulatory status of DOD-identified emerging  contaminants.

DOD Has Not Internally Reported All Data on Compliance with Drinking Water Regulations or Used Available Data to Evaluate Differences between Its Drinking Water Systems

For the years we reviewed—fiscal years 2013 through 2015—the military  departments annually reported information internally to ASD (EI&E) on  compliance with EPA and state health-based drinking water regulations,  which indicate that drinking water quality at DOD public water systems  was similar to other systems in the United States. However, not all  violations of health-based regulations were reported to ASD (EI&E) during  this time frame, as is required by DOD policy. The military departments  reported that a total of 77 military installations had at least one violation at  some point from fiscal year 2013 through fiscal year 2015, but we found  that at least 16 additional installations had violations that were reported to  EPA but were not internally reported to ASD (EI&E). DOD also has not  used available compliance data to identify why DOD public water systems  that provide DOD-treated drinking water appear to have more violations  of health-based regulations than DOD systems that provide non-DOD- treated drinking water.

Military Departments Have Internally Reported Data on Compliance with Health-Based Drinking Water Regulations, but Have Not Reported All Violations

For the years we reviewed—fiscal years 2013 through 2015—the military  departments annually reported information to ASD (EI&E) on compliance  with and violations of EPA and state health-based drinking water  regulations at the DOD public water systems that provide drinking water  to military installations. The military departments’ data for fiscal years  2013 through 2015 indicate that about 92 percent of people who received  drinking water from DOD public water systems were served by a system  that complied with EPA and state health-based regulations. This is similar  to the percentage of people in the United States—also about 92 percent,  according to EPA—who received drinking water during that time frame  from a community public water system with no health-based violations.  The data for that time period also indicate that about 8 percent of people  were provided drinking water from a DOD public water system that had at  least one violation of a health-based regulation. Health-based violations  can be for any length of time during a fiscal year—for example, a violation  lasting 1 day is counted the same as a violation lasting for 1 month.  Across the 3 fiscal years, the military departments reported that a total of  77 military installations had at least one violation at some point during that  time period: 35 in fiscal year 2013, 25 in fiscal year 2014, and 17 in fiscal  year 2015. The most common types of contaminants for which the military  departments reported violations were coliform and two disinfection  byproducts—trihalomethanes and haloacetic acids—which, according to EPA, are among the most common types of contaminants for which  health-based drinking water violations occur across the United States.

However, we found that the military departments have not always  reported all violations to ASD (EI&E), as required by DOD policy. Based  on our review of data in EPA’s Safe Drinking Water Information System  for fiscal years 2013 through 2015, we found that the military departments  did not report violations to ASD (EI&E) for at least 16 installations—9 Air  Force installations, 5 Navy installations, and 2 Army installations.  According to EPA’s database, the total population served by DOD public  water systems at these installations is approximately 180,000 people, and  most of the violations that went unreported involved coliform and  disinfection byproduct contaminants. However, the actual population  number affected by these violations and the contaminants involved— along with other information such as the duration of the contamination  and the corrective actions planned or taken—were not included in the  military departments’ annual reports to ASD (EI&E). These violations  were recorded in EPA’s system, which indicates that the installations  reported the violations to the appropriate state regulatory agencies, who  then reported them to EPA’s database. However, the violations were not  reported to ASD (EI&E), as required by DOD policy.

According to military department officials, violations of health-based  drinking water regulations went unreported to ASD (EI&E) due to a lack of  clarity in DOD’s reporting requirements and misunderstandings of the  requirements on the part of installations and the military departments. We  found that violations were either not reported by the military installations  where the violations occurred or that they were not reported by the  installations’ chains of command. Navy officials cited turnover of  installation personnel as the reason some violations went unreported, as  well as misinterpretations by installation personnel of DOD’s reporting  requirements. Air Force officials also told us that most of their unreported  violations were not reported to ASD (EI&E) because the Air Force did not  interpret them as health-based violations, although DOD policy requires  these types of violations to be reported. Army officials told us that,  based on their interpretation of DOD’s policy, the policy did not require  them to report violations at installations where formal, written notification  was not received from the state regulatory agency. However, ASD (EI&E)  officials stated that all violations of health-based regulations should be  reported, whether or not the state provides formal, written notification of  the violation. Navy officials also told us that they have not reported  violations at some of the Navy’s smaller systems that purchase drinking  water from non-DOD public water systems, due in part to  misinterpretation of DOD’s internal reporting requirements. However,  Navy officials told us that ASD (EI&E) had instructed them to begin  reporting these types of violations in fiscal year 2016, and the Navy is  working with ASD (EI&E) and the other military departments to determine  whether these types of systems should regularly report health-based  violations.

Currently, ASD (EI&E) does not have complete data in accordance with  DOD’s policy, limiting its ability to conduct oversight and analyze how  many people at military installations receive drinking water with health- based violations, what contaminants were involved, the duration of the  contamination, or what corrective actions the military departments have  planned or taken to address the violation. Standards for Internal Control  in the Federal Government states that quality information is needed to  achieve an organization’s objectives. Those standards also indicate that  actions such as improved communication to and additional training for  personnel are helpful for an organization to meet its objectives.  According to DOD officials, a committee comprised of ASD (EI&E) and  military department officials began a review in 2016 of DOD’s internal  reporting requirements for drinking water compliance data. While such a  committee could be in a position to make recommendations on clarifying  the annual reporting requirements, no documentation on the committee’s  efforts was yet available at the time of our review as the committee’s work  was still in progress. In addition, at present, there are no firm dates for  when its work will be completed or when any potential changes would be  implemented. Absent actions by ASD (EI&E) to identify and implement  any necessary changes to clarify annual reporting requirements in its  environmental compliance policy, and absent actions by the military  departments to increase understanding at their installations and  commands about the requirements, adherence to DOD’s environmental  compliance policy will remain limited and DOD will lack complete data to  conduct oversight of regulatory compliance at its public water systems.

DOD Has Not Used Available Data to Assess Why DOD-Treated Water Appears to Have More Health-Based Violations Than Non-DOD-Treated Drinking Water

DOD has not used available data to assess why DOD public water  systems providing DOD-treated drinking water appear to have more  violations of health-based drinking water regulations than systems  providing non-DOD-treated drinking water. Although we found that not all  violations were reported by the military departments to ASD (EI&E), the  data that were reported during fiscal years 2013 through 2015 indicated  that about 99 percent of the people who received non-DOD-treated  drinking water were served by systems with no violations, while about 89  percent of the people who received DOD-treated drinking water were  served by systems with no violations.

When we asked ASD (EI&E) and military department officials why these  differences may exist, they were unable to provide an explanation  because they had not used the reported water quality data to identify the  reasons why DOD public water systems providing DOD-treated water  appear to have more violations than systems providing non-DOD-treated  water. Although some officials offered ideas on the reasons for  differences in compliance—including the relative expertise of utilities and  private companies, versus DOD, in providing drinking water—DOD  officials acknowledged that the agency has not evaluated the data to  identify specific reasons for why the differences may exist. All public  water systems, including DOD public water systems, are required to  comply with applicable EPA and state drinking water regulations.  According to Standards for Internal Control in the Federal Government,  management should establish and operate activities to monitor the  internal control system and evaluate the results. Such monitoring should  assess the quality of performance over time and promptly resolve any  findings. Without reviewing the data reported by the military  departments to identify why there appear to be differences in violations  between DOD’s two types of public water systems and without identifying  and implementing any actions to address any differences, ASD (EI&E)  and the military departments may not be able to improve overall  compliance with health-based drinking water regulations.

DOD Has Initiated Actions to Address Concerns with Its Firefighting Foam as Well as Elevated Levels of PFOS, PFOA, and Perchlorate in Drinking Water

DOD is taking steps to address health and environmental concerns with  its use of firefighting foam that contains PFCs—including PFOS and  PFOA—to include restricting the use of foam at its installations and  funding research into the development of a PFC-free foam that can meet  DOD performance requirements. DOD also has responded to EPA and  state orders and initiated additional actions to address elevated levels of  PFOS, PFOA, and perchlorate.

DOD Is Taking Steps to Address Health and Environmental Concerns with Firefighting Foam That Contains PFCs

DOD is taking steps to address PFOS- and PFOA-related health and  environmental concerns with its use of firefighting foam that contains  PFCs. Firefighting foam is used by DOD to put fires out quickly while  also ensuring that they do not reignite. This is critical if, for example, there  is a fire from a fighter jet on the deck of an aircraft carrier. DOD has  outlined performance requirements in its military specification for  firefighting foam, which was authored by the Navy’s Naval Sea Systems  Command but is approved for use in all of DOD. For example, the  military specification states how long it should take for firefighting foam to  extinguish a fire—based on the size of the fire and the amount of foam  used—and how long the foam should prevent the extinguished fire from  reigniting. DOD’s military specification also requires that firefighting  foam purchased and used by the department must contain PFCs.

DOD’s steps to address concerns with the use of firefighting foam include  restricting the use of existing foams that contain PFCs; testing its current  foams to identify the amount of PFCs they contain; and funding research  into the future development of PFC-free foam that can meet DOD’s  performance and compatibility requirements (see table 2). Some of these  steps, such as limiting the use of firefighting foam containing PFCs, are in  place. Others, such as determining the specific amount of PFCs in  existing firefighting foams or researching potential PFC-free firefighting  foams, are in progress with targets, in some cases, but no firm completion  dates.

Navy officials stated that they are planning to revise the military  specification after they have completed their testing—to be completed in  late 2017 or 2018—on the amounts of PFOS, PFOA, and other PFCs  found in the firefighting foam currently used by DOD. That revision,  according to Navy officials, is intended to set limits for the amount of  PFCs that are allowed in firefighting foam. According to DOD, at present  there is no PFC-free firefighting foam that meets DOD’s performance and  compatibility requirements. As a result, the Navy has no plans to remove  the requirement for firefighting foam to contain PFCs at this time.  However, if a PFC-free foam is developed in the future that can meet  DOD performance and compatibility requirements, Navy officials said that  any necessary revisions to the military specification would be made at  that time—a process that could take months to complete.

DOD Has Responded to Orders from EPA and a State Regulator and Has Initiated Additional Actions to Address Elevated Levels of PFOS and PFOA in Drinking Water at or near Military Installations

DOD has taken steps to respond to four administrative orders directing  the department to address PFOS and PFOA levels that exceeded EPA’s  health advisory levels for drinking water. One order was issued by the  Ohio Environmental Protection Agency at Wright-Patterson Air Force  Base in Ohio, and three orders were issued by the EPA directed at: the  former Pease Air Force Base in New Hampshire; Horsham Air Guard  Station in Pennsylvania; and the former Naval Air Warfare Center  Warminster in Pennsylvania. Under Section 1431 of the Safe Drinking  Water Act, EPA may issue orders necessary to protect human health  where a contaminant in a public water system presents an imminent and  substantial endangerment. EPA may do so if appropriate state and local  authorities have not acted to protect human health. These orders may  require, among other things, carrying out cleanup studies, providing  alternate water supplies, notifying the public of the emergency, and  halting disposal of the contaminants threatening human health. The Ohio  Environmental Protection Agency has similar authority.

According to information provided by officials from the Ohio  Environmental Protection Agency, EPA, and DOD, DOD has taken steps  to respond to the administrative orders. Table 3 provides further details  on each order and examples of actions by DOD to address the orders.

In addition to actions specific to these four installations, DOD has initiated  other actions to test for, investigate, and mitigate elevated levels of PFOS  and PFOA at or near installations across the military departments.  Following the release of EPA’s lifetime health advisory for PFOS and  PFOA in May 2016, each of the military departments issued guidance  directing installations to, among other things, test for PFOS and PFOA in  their drinking water and take steps to address drinking water that  contained amounts of PFOS and PFOA above the EPA’s lifetime health  advisory level. The military departments also directed their installations to  identify locations with a known or suspected prior release of PFOS and  PFOA and to address any releases that pose a risk to human health— which can include people living outside DOD installations.

As a result of these efforts, DOD has initiated actions to address PFOS  and PFOA in drinking water both on military installations and outside  military installations. As of March 2017, DOD data indicated that the  department was taking steps to address levels of PFOS and PFOA above  the EPA’s lifetime health advisory level in drinking water on 11 military  installations in the United States, 2 of which we visited during the course  of this review (see fig. 1).

According to DOD data, these installations took various corrective actions  to mitigate the presence of PFOS and PFOA in the drinking water,  including shutting down drinking water wells, providing alternative drinking  water, and installing treatment systems. For example, at Eielson Air Force  Base in Alaska, the Air Force reported shutting down three of the  installation’s six drinking water wells and installing a treatment system to  remove PFOS and PFOA from the drinking water. At Marine Corps Base  Camp Pendleton in California, the Navy reported that a well contaminated  with PFOS and PFOA was taken out of service and that the affected  reservoir was drained and replaced with water from another source;  follow-on testing showed that the presence of PFOS and PFOA were  returned to below the EPA’s lifetime health advisory level. At Fort  Leavenworth in Kansas, the Army reported that the private company that  operates the installation’s drinking water system had shut down two wells  contaminated with PFOS and PFOA and plans to install a treatment  system before returning those wells to service.

Additionally, according to DOD data as of December 2016 the military  departments had identified 391 active and closed installations with  known or suspected releases of PFOS and PFOA, and had reported  spending almost $200 million on environmental investigations and  mitigation actions at or near 263 (or about 67 percent) of those  installations. In particular, DOD had initiated mitigation actions, which  include installing treatment systems or supplying bottled water, to address  PFOS and PFOA in drinking water for people living outside 19  installations—5 of which we visited during the course of this review (see  fig. 2).

The following cost data provided by DOD were current as of December  2016, and are supplemented by additional information we obtained during  our installation visits.

The Air Force identified 203 installations with known or suspected  releases of PFOS and PFOA, spent about $120 million on  environmental investigations at those installations, and spent about  $33 million on mitigation actions at or near 14 of the 203 installations.  For example, the Air Force reported spending over $5 million on  environmental investigations and mitigation actions at Peterson Air  Force Base in Colorado. During our visit to that installation, officials  showed us the sites they are investigating—to include the current (see  fig. 3 below) and former fire training areas—to determine the extent to  which their prior use of firefighting foam may have contributed to the  discovery of PFOS and PFOA in the drinking water of three nearby  communities. Additionally, the Air Force has awarded a contract for,  among other things, installing treatment systems in those  communities. In another example, the Air Force reported spending  about $800,000 on environmental investigations at Joint Base  Langley-Eustis in Virginia, but nothing yet on mitigation actions.  During our visit to this installation, officials told us that they had not  taken any mitigation actions because they do not use the installation’s  groundwater as a drinking water source; the utility that serves the  installation, as well as the nearby city of Newport News, obtains its  drinking water primarily from a surface water source, which officials  said was approximately 20 miles from the installation.

The Navy identified 127 installations with known or suspected  releases of PFOS and PFOA, spent about $20.5 million on  environmental investigations at 47 of those installations, and spent  about $24 million on mitigation actions at or near 5 of those  installations. For example, the Navy reported spending about $15  million on environmental investigations and mitigation actions at the  former Naval Air Station Joint Reserve Base Willow Grove in  Pennsylvania. During our visit to this installation, officials told us that  the Navy is investigating the extent to which PFOS and PFOA on the  installation may have contaminated a nearby town’s drinking water.  The Navy has agreed to fund installation of treatment systems and  connections of private well owners to the town’s drinking water  system, among other things. In another example, the Navy reported  spending nearly $3 million on environmental investigations and  mitigation actions at Naval Auxiliary Landing Field Fentress in  Virginia. During our visit to this installation, officials told us that the  Navy is providing bottled water to the approximately 20 to 30  personnel who work there and plans to install a treatment system to  treat for PFOS and PFOA.

The Army identified 61 installations with known or suspected releases  of PFOS and PFOA, spent about $1.6 million on environmental  investigations at 13 of those installations, and has not yet begun any  mitigation actions at or near the identified installations. For example,  the Army reported spending about $26,000 on environmental  investigations at Fort Carson in Colorado, but nothing yet on  mitigation actions. During our visit to this installation, officials told us  that they had found PFOS and PFOA in groundwater near their  previous fire training area but that the installation does not use that  groundwater as a drinking water source, and state officials told us that  it is unlikely that PFOS and PFOA from Fort Carson had affected any  nearby drinking water sources.

According to DOD, it may take several years for the department to  determine how much it will cost to cleanup PFOS and PFOA  contamination at or near its military installations. In January 2017, we  reported that DOD had not notified Congress that the costs for  environmental cleanup at closed installations will significantly increase  due to the high cost of remediating emerging contaminants—including  PFOS and PFOA. We also reported that DOD officials had not  determined the total costs for cleaning up emerging contaminants at  closed installations. We recommended that DOD include in future annual  reports to Congress best estimates of the environmental cleanup costs for  emerging contaminants as additional information becomes available, and  DOD concurred with the recommendation and stated its commitment to  do so.

DOD Previously Directed Installations to Test for Perchlorate in Drinking Water

DOD previously directed installations to test for perchlorate in drinking  water. Following the EPA’s issuance of an interim drinking water health  advisory for perchlorate in 2008, DOD issued policy in April 2009—which  superseded similar policy that was issued in January 2006—directing  DOD-owned drinking water systems that were testing for inorganic  substances to also test for perchlorate. Installations that found  perchlorate in their drinking water were to consult with their leadership on  appropriate actions to take and to continue testing on a quarterly basis  until they determined that perchlorate levels were likely to remain below  EPA’s health advisory level, or any applicable federal or state regulation.  Citing congressional and regulatory agency concerns related to  perchlorate, DOD developed a database for storing the results of  perchlorate testing. According to ASD (EI&E), the database was last  updated in 2009 and is no longer being used by the department.

ASD (EI&E) officials stated that they are no longer regularly testing  drinking water for perchlorate unless there is a state requirement to do so;  previous testing indicated that DOD was not a primary source of  perchlorate in drinking water and that known releases of perchlorate did  not currently pose a threat to drinking water. According to EPA, the  agency expects to issue a final drinking water regulation for perchlorate  by the end of 2019. ASD (EI&E) officials told us that, once EPA has  issued a final regulation, DOD is committed to complying with it.

During the period we reviewed, DOD data indicate that DOD public water  systems complied with EPA and state health-based drinking water  regulations at a level comparable with other systems in the United States.  However, we found that the military departments did not report all  violations of these regulations to ASD (EI&E) during that period, which  illustrates that DOD’s internal reporting requirements for drinking water  data are either not clear in DOD regulations or are not clearly understood  by those implementing them. Unless ASD (EI&E) and the military  departments act to make any necessary clarifications to and increase  understanding of DOD’s annual reporting requirements, ASD (EI&E) may  not have complete data to effectively oversee the military departments’  compliance with drinking water regulations. Further, the data indicated  that systems providing DOD-treated drinking water had more reported  health-based violations than DOD systems providing non-DOD-treated  drinking water. However, DOD has not used these data to identify the  reasons that these differences may exist. Without using available data to  identify why differences in violations appear to exist between DOD’s two  types of public water systems, DOD will likely be hampered in its ability to  identify what actions, if any, could be taken to address any differences  and improve overall compliance with health-based drinking water  regulations.

We are making a total of five recommendations to DOD.

The Assistant Secretary of Defense for Energy, Installations, and  Environment, in consultation with the Secretaries of the military  departments, should identify and implement any necessary changes to  DOD’s environmental compliance policy to clarify DOD’s reporting  requirements for violations of health-based drinking water regulations.  (Recommendation 1)

The Secretary of the Army should identify and implement actions to  increase understanding at Army installations and commands about  DOD’s reporting requirements for violations of health-based drinking  water regulations. These actions may include improved communication to  or additional training for personnel. (Recommendation 2)

The Secretary of the Navy should identify and implement actions to  increase understanding at Navy installations and commands about DOD’s  reporting requirements for violations of health-based drinking water  regulations. These actions may include improved communication to or  additional training for personnel. (Recommendation 3)

The Secretary of the Air Force should identify and implement actions to  increase understanding at Air Force installations and commands about  DOD’s reporting requirements for violations of health-based drinking  water regulations. These actions may include improved communication to  or additional training for personnel. (Recommendation 4)

The Assistant Secretary of Defense for Energy, Installations, and  Environment, in consultation with the Secretaries of the military  departments, should (a) review reported compliance data to identify the  reasons for any differences in the number of violations of health-based  drinking water regulations between DOD’s two types of public water  systems and (b) identify and implement any actions needed to address  the causes of any differences in the number of violations between DOD’s  two types of public water systems. (Recommendation 5)

We provided a draft of this report to DOD and EPA for review and  comment. In its written comments, reproduced in appendix III, DOD  concurred with our recommendations. DOD and EPA also provided  technical comments, which we incorporated as appropriate. Based on  technical comments from DOD, we revised the title of the report to more  clearly specify the actions DOD should take to address the findings in our  report.

We are sending copies of this report to the appropriate congressional  committees; the Secretary of Defense; the Assistant Secretary of Defense  for Energy, Installations, and Environment; the Secretaries of the Army,  the Navy, and the Air Force; and the Administrator of EPA. In addition,  the report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  us at J. Alfredo Gómez, (202) 512-3841 or gomezj@gao.gov, or Brian J.  Lepore, (202) 512-4523 or leporeb@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

Senate Report 114-255 accompanying a bill for the national defense  authorization for fiscal year 2017 included a provision for us to review the  Department of Defense’s (DOD) efforts to manage contaminants in  drinking water. This report examines the extent to which DOD has (1)  internally reported data on compliance with health-based drinking water  regulations at military installations and used those data to assess  compliance at its two types of public water systems and (2) taken actions  to address concerns with its firefighting foam containing perfluorinated  chemicals (PFCs) and to address elevated levels of  perfluorooctanesulfonic acid (PFOS), perfluorooctanoic acid (PFOA), and  perchlorate in drinking water at or near military installations.

For objective one, we reviewed DOD’s policy on environmental  compliance in the United States, which directs the military departments to  annually report data to the Assistant Secretary of Defense for Energy,  Installations, and Environment (ASD (EI&E)) on compliance with and  violations of Environmental Protection Agency (EPA) and state health- based drinking water regulations at military installations. We analyzed  data reported by the military departments to ASD (EI&E) on compliance  with and violations of health-based drinking water regulations at DOD  public water systems located at military installations in the United States  for fiscal years 2013 through 2015, the most recent data available at the  time of our review. We analyzed the data to identify (1) the number of  people served by DOD public water systems that complied with  applicable EPA and state health-based drinking water regulations during  the fiscal year and (2) the number of people served by DOD public water  systems that violated at least one of these regulations sometime during  the fiscal year. We performed this analysis for both types of DOD public  water systems—those that provide DOD-treated drinking water, and  those that provide non-DOD-treated drinking water. We also used the  data to identify the military installations where the reported violations  occurred; the nature of the violation (including the contaminant involved);  and the number of people affected. Next, we collected data from EPA’s  Safe Drinking Water Information System for all public water systems in  the United States. We used DOD-provided public water system  identification numbers to identify in the EPA system any violations for  health-based drinking water regulations at those DOD systems for fiscal  years 2013 through 2015. We then compared the violations found in  EPA’s data to the data reported by the military departments to ASD  (EI&E) to determine the extent to which the military departments were  reporting all violations of health-based drinking water regulations to ASD  (EI&E).

We also analyzed DOD’s data to identify any differences in violations  between DOD- and non-DOD-treated drinking water. We evaluated the  military departments’ reported data and DOD’s use of these data to  determine compliance with DOD’s reporting requirements in the  department’s environmental compliance instruction and Standards for  Internal Control in the Federal Government. According to these  standards, quality information is needed to achieve an organization’s  objectives, management is to monitor performance over time and  promptly resolve any findings, and actions such as improved  communication to and additional training for personnel are helpful for an  organization to meet its objectives. We also discussed our analysis with  ASD (EI&E) and military department officials, and discussed possible  reasons for why any violations went unreported to ASD (EI&E) and why  there may be differences in violations between DOD- and non-DOD- treated drinking water. We assessed the reliability of the DOD and EPA  data on violations of health-based drinking water regulations by reviewing  relevant documentation, testing the data for obvious errors, and  interviewing knowledgeable officials. As we have previously found, EPA’s  data system may not contain all public water violations as states have  under-reported the violations. During this review, we found that some  public water system identification numbers for DOD installations could not  be matched with EPA’s system and, therefore, were excluded from our  analysis. As a result, some DOD installation violations may be missing  from the data, and we may not have comprehensive violations data for  health-based drinking water regulations at DOD installations.  Nonetheless, we determined that DOD and EPA data were sufficiently  reliable for the purpose of identifying whether any drinking water  violations were recorded in EPA’s system but not internally reported  within DOD, and to indicate possible differences in drinking water  violations, as reported by the military departments, between DOD’s two  types of public water systems.

For objective two, we reviewed policies issued by the military  departments on the use of firefighting foam that contains PFCs. We also  reviewed DOD documents related to research into PFC-free firefighting  foams that can meet the department’s performance and compatibility  requirements, as well as DOD’s military specification document that  outlines those requirements. We met with officials from ASD (EI&E) and  the military departments to discuss their policies on the use of firefighting  foam and actions taken to address concerns with the use of firefighting  foam containing PFCs, including the future use of firefighting foam.  Additionally, we met with Navy officials responsible for testing existing  firefighting foam products and setting the military specifications for  firefighting foam use in DOD.

Additionally, we obtained and reviewed four regulatory administrative  orders—three from EPA and one from the Ohio Environmental Protection  Agency—directing DOD to address elevated levels of PFOS and PFOA  contamination in drinking water at or near four active and closed military  installations, and reviewed documentation related to DOD’s efforts to  address these administrative orders. We also met with officials from Ohio  and the EPA regions that issued the orders—EPA Regions 1 and 3—as  well as DOD officials who responded to the orders, to discuss DOD’s  response to the orders. We reviewed drinking water guidance issued by  ASD (EI&E) and the military departments on testing installation drinking  water for PFOS and PFOA and responding to known or suspected  releases of PFOS and PFOA. We analyzed DOD-provided data on the  installations where DOD-conducted testing showed the presence of  PFOS and PFOA in drinking water above the EPA’s health advisory level  for those contaminants (as of March 2017) and on the costs and actions  taken to investigate and mitigate PFOS and PFOA at or near military  installations (as of December 2016). We assessed the reliability of the  data by examining the data for obvious errors and inconsistencies,  comparing the data, where applicable, with other information collected,  and by interviewing knowledgeable officials; we found the data to be  sufficiently reliable for our purposes of describing what DOD has reported  on its actions and costs for responding to PFOS and PFOA.

Additionally, we reviewed DOD policy and our prior work on testing for  and responding to perchlorate at military installations. We met with ASD  (EI&E) and military department officials to discuss DOD actions to  address PFOS, PFOA, and perchlorate. To obtain additional information  on DOD actions to address emerging contaminants in drinking water, we  conducted site visits to a nongeneralizable sample of seven current and  former military installations—at least two installations per military  department—that were selected because they were investigating or  responding to unregulated DOD-identified emerging contaminants in  drinking water; these installations are listed below. We also met with  EPA and state regulatory officials to better understand how DOD was  responding to administrative orders and addressing PFOS, PFOA, and  perchlorate at or near DOD installations. Specifically, we met with officials  from selected EPA regions and state regulatory offices that had issued an  administrative order for PFOS and PFOA or whose region or state  included the installations we visited; those EPA regions and states are  listed below. We also compared DOD’s list of emerging contaminants with  EPA documentation to determine how many DOD-identified emerging  contaminants (1) have been regulated by EPA under the Safe Drinking  Water Act or (2) are currently unregulated but have an EPA-issued  drinking water health advisory.

We visited or contacted the following offices and locations during our  review. Unless otherwise specified, these organizations are located in or  near Washington, D.C.

Office of the Assistant Secretary of Defense for Energy, Installations,

Office of the Deputy Assistant Secretary of Defense for Environment,  Safety, and Occupational Health

Office of the Assistant Chief of Staff of the Army for Installation

U.S. Army Installations Management Command, Fort Sam Houston,

U.S. Army Environmental Command, Fort Sam Houston, Texas

Office of the Assistant Secretary of the Navy for Energy, Installations,

Office of the Chief of Naval Operations, Energy and Environmental

Former Naval Air Station Joint Reserve Base Willow Grove,

Naval Auxiliary Landing Field Fentress, Virginia  Department of the Air Force

Office of the Assistant Secretary of the Air Force for Installations,

Air Force Civil Engineer Center, Joint Base San Antonio, Texas

Former Pease Air Force Base, New Hampshire  Joint Base Langley-Eustis, Virginia

Office of Enforcement and Compliance Assurance

EPA Region 3, Philadelphia, Pennsylvania

Colorado Department of Public Health and Environment

Pennsylvania Department of Environmental Protection

South Carolina Department of Health and Environmental Control  We conducted this performance audit from June 2016 to October 2017 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix II: Drinking Water Regulatory Status for Department of Defense-Identified Emerging Contaminants

The Department of Defense’s (DOD) list of emerging contaminants  includes 21 contaminants that can be found in drinking water: 10 that  have been regulated by the Environmental Protection Agency (EPA)  under the Safe Drinking Water Act and 11 that are currently unregulated  but have an EPA-issued drinking water health advisory. Table 4 shows  the regulatory status for each of the 21 contaminants.

Appendix IV: GAO Contacts and Staff Acknowledgments

In addition to the contacts named above, Maria Storts (Assistant  Director), Diane B. Raynes (Assistant Director), Kazue Chinen, Michele  Fejfar, Jennifer Gould, Karen Howard, Richard P. Johnson, Mae Jones,  Daniel Kuhn, Summer Lingard-Smith, Daniel Longo, Felicia Lopez,  Geoffrey Peck, Ophelia Robinson, Jerry Sandau, and Sara Sullivan made  key contributions to this report.

DOD and EPA have found chemicals in drinking water at or near military installations that may cause cancer and other health issues. Some of these chemicals can be found in firefighting foam and rocket propellants.

DOD has taken steps to limit individuals' exposure to some chemicals, including providing alternative drinking water supplies and installing water treatment systems. Still, DOD's incomplete internal reporting limits its ability to monitor installation compliance with safe drinking water regulations.

This report makes 5 recommendations to improve DOD's data, reporting, and oversight of drinking water regulations.


Assessment Methodology for Economic Analysis

We prepared this report to answer the question: What are key methodological elements of an economic analysis that is intended to inform decision-makers and stakeholders?

GAO identifies five key methodological elements to the baseline structure of an economic analysis: Objective and scope, Methodology, Analysis of effects, Transparency and Documentation. GAO's assessment methodology evaluates each key element and provides an overall assessment based on the assessment of the individual key elements.

In its engagements, GAO may review economic analyses that are  intended to inform decision-makers and stakeholders about the economic  effects of a public action. The economic analyses that GAO reviews may  have been performed to meet requirements described in Executive Order  or related statutes, to propose or support an action, or to examine an  action that has already been taken. The purpose of this report is to inform  the public about the methodology that GAO may use to assess these  economic analyses.

We prepared this report under the authority of the Comptroller General to  answer the question: What are key methodological elements of an  economic analysis that is intended to inform decision-makers and  stakeholders? We answered this question by reviewing economic  elements and concepts that serve as the basis for sound economic  analysis. We considered a wide range of economics documents prepared  by various government agencies and institutions on how economic  analysis should be performed. These documents included, among others,  Circulars released by the Office of Management and Budget (OMB) and  handbooks for economic analysis from federal and international agencies,  such as the U.S. Environmental Protection Agency, the U.S. Department  of Defense, the U.S. Department of Transportation, the Organization for  Economic Co-operation and Development (OECD), and HM Treasury  Government of the United Kingdom. In addition, we consulted with  experts on economic analysis of public actions. We synthesized  economic concepts embodied in these documents into the present report  for use by GAO in assessing economic analyses.

GAO’s commitment to informative and transparent economic analysis is  in line with the Office of Management and Budget’s guidelines on  achieving a high degree of transparency in data and methods in scientific  information.  An important benefit of transparency is that the public can  assess the structure of an analysis and, in particular, how much of an  analytic result hinges on the specific choices made by the authors. This  transparency indicates the solidity of the analytic choices. For example, it  allows the implication of these choices, related risks, and uncertainties to  be readily assessed. This type of sensitivity analysis is regarded as an  essential feature of high-quality analysis, and it cannot be undertaken  without a high degree of transparency.

The key methodological elements we identify in this report are not  intended to be exhaustive and to supplant or alter relevant federal and  agency requirements for economic analysis. They simply serve to  establish a sound baseline framework for the assessment of an economic  analysis. GAO’s assessment methodology examines the extent to which  an economic analysis properly dealt with these key elements and,  accordingly, informed decision-makers and stakeholders about the  economic effects analyzed.

We conducted our work from June 2017 to April 2018 in accordance with  all sections of GAO’s Quality Assurance Framework that are relevant to  our objectives. The framework requires that we plan and perform the  engagement to obtain sufficient and appropriate evidence to meet our  stated objectives and to discuss any limitations in our work. We believe  that the information and data obtained, and the analysis conducted,  provide a reasonable basis for any findings and conclusions in this  product.

For the purpose of this report, an economic analysis is defined as an  analysis that is intended to inform decision-makers and stakeholders  about the economic effects of an action. Economic effects (hereafter also  called “effects”) commonly include costs, benefits, and/or economic  transfers (for example, transfer payments). Action is defined to include a  government law, rule, regulation, project, policy, or program. An action  may be examined in the context of legislation, regulation, advocacy,  agency operations, or in response to certain events (such as a natural  disaster, for example). An economic analysis may be prospective,  examining an action that could be taken, or retrospective, examining the  outcome of an action that has already been taken. Examples of economic  analyses include:   1.  An economic analysis of the costs of a government program, project,  2.  An economic analysis of the benefits and costs of a government rule  3.  An economic analysis of the impact of a proposed or existing  regulation on regulated entities and consumers.  4.  An economic analysis of an action in response to an event (for  example, an analysis of a federal response to a natural disaster);  5.  A benefit-cost analysis or cost-effectiveness analysis.

Section 2: Key Elements of an Economic Analysis

GAO reviewed handbooks and guidance on economic analysis that have  been issued by various government agencies and institutions and  consulted with experts.  (Appendix 1 details GAO’s objective, scope, and  methodology.) In this Section, GAO synthetizes economic elements and  concepts embodied in this literature for use by GAO in its assessment  methodology for the review of an economic analysis.

GAO identifies five key methodological elements to the baseline structure  of an economic analysis. These key elements are:   1.  Objective and scope— the objective and scope of the analysis.    2.  Methodology— the methodology used to examine the economic  effects.   3.  Analysis of effects — the analysis of economic effects.   4.  Transparency— the transparency of the analysis of economic effects.   5.  Documentation— the documentation included in the analysis.

These key elements are standard to the structure of analyses, generally  speaking. That is, an analysis is performed to address an objective; the  analysis is scoped to address that objective; the analysis adopts a  methodology, which is used to analyze the economic effects of interest;  and the analysis is transparent and properly documented. The emphasis  on transparency is consistent with the final implementation guidelines of  the Office of Management and Budget (OMB). “The primary benefit of public transparency is not necessarily that errors in analytic results  will be detected, although error correction is clearly valuable. The more important benefit  of transparency is that the public will be able to assess how much of an agency’s analytic  result hinges on the specific analytic choices made by the agency.”

Having identified key elements to the structure of an economic analysis  we synthetized for each key element, economic concepts embodied in the  literature that we reviewed. For example, what might we be assessing  under the key element: Objective and Scope? We considered economic  concepts commonly identified across the documents we reviewed, and  incorporated feedback from the experts and agencies with whom  we  consulted. The documents that we reviewed included, among others,  Circulars A-94 and A-4 released by OMB, handbooks for economic  analysis from federal and international agencies, such as the U.S.  Environmental Protection Agency, the U.S. Department of Defense, the  U.S. Department of Transportation, the Organization for Economic Co- operation and Development (OECD), and the United Kingdom’s HM  Treasury.

These documents generally outline a methodical structure to an economic  analysis. This methodical structure takes the form of a set of issues or  sequence of steps to address while conducting the analysis. These  issues or steps, in turn, embody economic concepts. For the purpose of  developing our assessment methodology, we synthetized and  categorized these economic concepts with the key elements that we  identified—as listed below. The concepts for each listed key element are  not intended to be exhaustive and do not supplant or alter existing  requirements for economic analysis. Depending on the context in which  an action is examined, GAO’s assessment of a key element could  exclude some concepts, or extend beyond the concepts listed for that  element. In such cases, GAO’s written assessment of the relevant key  element will specify the concepts that were actually considered in the  review process.

The five key elements and economic concepts in GAO’s assessment  methodology for an economic analysis are:  1.  Key Element: Objective and Scope—the objective and scope of the  analysis.

The economic analysis explains the action examined and includes a  rationale and justification for the action. The analysis states its  objective. The scope of the analysis is designed to address this  objective. Unless otherwise justified, the analysis focuses on  economic effects that accrue to citizens and residents of the United  States, and its time horizon is long enough to encompass the  important economic effects of the action.  2.  Key Element: Methodology—the methodology used to examine the  economic effects.

The economic analysis examines the effects of the action by  comparing alternatives, using one of them as the baseline. Unless  otherwise justified, it considers alternatives that represent all relevant  alternatives, including that of no action. The analysis defines an  appropriate baseline. The analysis justifies that the world specified  under each alternative considered (including the baseline) represents  the best assessment of what the world would be like under that  alternative. The analysis identifies the important economic effects for  each alternative considered, their timing, and whether they are direct  or ancillary effects.  3.  Key Element: Analysis of Effects—the analysis of economic effects.

Where feasible, the economic analysis quantifies the important  economic effects and monetizes them using the concept of  opportunity cost. The analysis applies the criterion of net present  value, or related outcome measures, to compare these effects across  alternatives. It controls for inflation and uses economically justified  discount rates. Where important economic effects cannot be  quantified, the analysis explains how they affect the comparison of  alternatives. Where the equity and distributional impacts are  important, the full range of these impacts is separately detailed and  quantified, where feasible.  4.  Key Element: Transparency—the transparency of the analysis of  economic effects.

The economic analysis describes and justifies the analytical choices,  assumptions, and data used. The analysis assesses how plausible  adjustments to each important analytical choice and assumption affect  the estimates of the economic effects and the results of the  comparison of alternatives. The analysis explains the implications of  the key limitations in the data used. Where feasible, the analysis  adequately quantifies how the statistical variability of the key data  elements underlying the estimates of the economic analysis impacts  these estimates, and the results of the comparison of alternatives.  5.  Key Element: Documentation—the documentation included in the  analysis.

The economic analysis is clearly written, with a plain language  summary, clearly labeled tables that describe the data used and  results, and a conclusion that is consistent with these results. The  analysis cites all sources used and documents that it is based on the  best available economic information. The analysis documents that it  complies with a robust quality assurance process and, where  applicable, the Information Quality Act.  The analysis discloses the  use and contributions of contractors and outside consultants.

In summary, GAO identifies five key elements, with associated economic  concepts, to the structure of an economic analysis. GAO’s assessment  methodology then examines the extent to which an economic analysis  properly dealt with these key elements.

GAO’s assessment methodology has two steps: (1) an assessment of  each individual key element and (2) an overall assessment based on the  assessment of the individual key elements. Below, these two types of  assessment are discussed. While GAO’s assessment methodology  typically considers all five key elements, there may be cases, for example  depending on the scope of an engagement, where it may consider only  certain key elements. In those cases, GAO may not be able to make an  overall assessment.

Step 1: Assessing Each Individual Element

The first step in the review process is an assessment of the extent to  which the economic analysis has considered and properly dealt with each  key element. For each element, the outcome of the review is a written  assessment and an assessment score. The written assessment details  the extent to which the analysis considered and properly dealt with the  element. To the extent that important limitations are identified in the  review, the written assessment describes these limitations. This written  assessment informs the assessment score, which is one of three mutually  exclusive scores:  1.  fully met—that is, the economic analysis has considered and properly  dealt with the element;  2.  partly met—that is, the economic analysis has only partly considered  and properly dealt with the element;   3.  not met—that is, the economic analysis has not considered or not  properly dealt with the element.

If the outcome is “partly met” or “not met,” the written assessment should  describe the limitations of the analysis.

Assessments are made from expertise in economics and with  professional judgment. The guiding principles of a review are objectivity,  integrity, and compliance with generally accepted government auditing  standards (GAGAS). An assessment is contextual—that is, it is  conditional on the evidence underlying the action examined, and the  context in which it takes place. The assessment is also conditional on the  reasonably obtainable information available at the time of the economic  analysis that is being reviewed. A caveat may be added to the  assessment if new information that would affect it, has become available  and is reasonably obtainable since the analysis was made. Should the  review of a key element exclude certain concepts, or extend beyond the  concepts listed for that key element, GAO’s written assessment of the key  element will specify the concepts that were actually considered in the  assessment process.

Once each element has been individually reviewed, an overall  assessment is made of the extent to which the economic analysis  accordingly informs decision-makers and stakeholders about the  economic effects of the action examined. Four outcomes are possible:   1.  The analysis informs decision-makers and stakeholders about the  economic effects of the action examined.  2.  The analysis informs, with caveats, decision-makers and stakeholders  about the economic effects of the action examined. (If this is the  outcome, the review should describe the caveats in writing.)  3.  The analysis needs additional work to inform decision-makers and  stakeholders about the economic effects of the action examined. (If  this is the outcome, the review should describe in writing the important  limitations of the economic analysis.)  4.  The analysis does not inform decision-makers and stakeholders about  the economic effects of the action examined. (If this is the outcome,  the review should describe in writing the deficiencies of the analysis.)

An economic analysis that has fully met all the key elements should be  identified as informing decision-makers and stakeholders about the  economic effects of the action examined (this is outcome 1). This  determination is neither an endorsement of the specific findings and  conclusions of the analysis, nor is it a determination that these are  correct. For example, a prospective analysis is predictive of a potential  result, but cannot definitively determine the result. It is a determination  that the analysis is adequately and properly designed, and accordingly, it  can inform the public discourse about the economic effects of the action  examined.

The written statements added to outcomes 2–4 should refer to the review  of the individual elements. The difference between outcomes 2 and 3 is a  matter of degree and professional judgment. Generally speaking, the  caveats under outcome 2 will be relatively minor or few, whereas the  important limitations under outcome 3 are likely to be more consequential.  Should the economic analysis suffer from major deficiencies in meeting  the key elements, then outcome 4 may be appropriate.

Appendix I: Objective, Scope, and Methodology

Our objective was to identify, for the purpose of developing GAO’s  assessment methodology for the review of economic analysis, key  methodological elements to the structure of an economic analysis that is  intended to inform decision-makers and stakeholders about the economic  effects of a public action.

To address this objective, we reviewed existing Circulars issued by the  Office of Management and Budget (OMB), handbooks for economic  analysis issued by federal agencies, international government agencies  and institutions, and established textbooks on economic theory and  benefit-cost analysis. We also solicited feedback from economics experts  (in academia and public policy) and international audit agencies.

We identified in our document review, five key elements to the structure  of an economic analysis. These key elements are: (1) objective and  scope; (2) methodology; (3) analysis of effects; (4) transparency; (5)  documentation. These key elements are standard to the structure of  analyses, generally speaking. That is, an analysis is performed to  address an objective; the analysis is scoped to address that objective; the  analysis adopts a methodology, which is used to analyze the economic  effects of interest; and the analysis is transparent and properly  documented.

Having identified key elements to the structure of an economic analysis,  we synthetized for each key element economic concepts embodied in the  literature we reviewed. For example, what might we be assessing under  the key element objective and scope? To do so, we looked for economic  concepts commonly identified across the documents we reviewed and  incorporated feedback from the experts and agencies that we consulted  with. We then categorized these economic concepts across the key  elements that we identified.

Among the documents we reviewed in our process were the following:

OMB Circular A-94, Guidelines and Discount Rates for Benefit-Cost  Analysis of Federal Programs, Revised (Oct. 29, 1992). Circular A-94  provides a checklist of whether an agency has considered and  properly dealt with all the elements for sound benefit-cost and cost- effectiveness analyses.

OMB Circular A-4, Regulatory Analysis, (Sept. 17, 2003). Circular A-4,  released in collaboration with the Council of Economic Advisors,  identifies key elements to the structure of economic analyses in  regulatory proceedings.

Office of Management and Budget, Office of Information and  Regulatory Affairs, Regulatory Impact Analysis: A Primer  (Washington, D.C.: The White House). The purpose of the primer is to  offer a summary of OMB Circular A-4.

Agency-issued handbooks for economic analysis, such as, for  example, those issued by the U.S. Environmental Protection Agency,   the U.S. Department of Defense, the U.S. Department of  Transportation, and the Organization for Economic Co-operation and  Development (OECD). We also reviewed The Green Book: appraisal  and evaluation in central government, issued by HM Treasury,  Government of the United Kingdom, (London; July, 2011). HM  Treasury describes The Green Book as a best practice guide for all  central departments and executive agencies, and covers projects of  all types and size. The guide applies to appraisals—defined as any  analysis used to support a government decision to adopt a new policy,  or to initiate, renew, expand or re-orientate programs or projects that  would result in measurable benefits and/or costs to the public—and  evaluations—defined as retrospective analysis of a policy, program or  project at its completion, conclusion or revision.

The National Academies of Sciences, Engineering, and Medicine,  Guidelines for the review of Reports of the National Academies of  Sciences, Engineering, and Medicine. While these guidelines are  specific to the review of reports issued by the National Academies and  outline review criteria that apply across a broad range of disciplines,  not just economics, they provide review criteria for scientific analysis.

We provided a draft of this product for third-party outside review to  experts at various U.S. and international government agencies and  specialists. Reviewers provided technical comments which we  incorporated as appropriate.

Appendix II: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Carol Bray, Timothy Carr, Tim  Guinane, Kathleen Scholl, Paola Tena and Elaine Vaurio made key  contributions to this report.