Nuclear Nonproliferation: The Administration's 2018 Plan for Verification and Monitoring Met Most Reporting Requirements but Did Not Include Future Costs and Funding Needs

Countering the proliferation of nuclear weapons is a national security priority that is challenged by weapons advances from existing nuclear states and other actors possessing or attempting to possess nuclear weapons. To help address these issues, Congress directed the Administration in 2015 and 2017 to develop a plan for verification and monitoring relating to the potential proliferation of nuclear weapons, components of such weapons, and fissile material. GAO reviewed the first plan submitted to Congress in 2015, and an update submitted in 2017. GAO reported in March 2018 that this plan and update generally did not address the congressionally mandated reporting requirements.

In the fiscal year 2018 NDAA, Congress directed the Administration to develop another plan and included a provision for GAO to review the plan. This report assesses whether the Administration's new plan provided details on the reporting requirements included in the NDAA.

To determine whether the plan provided details on the reporting requirements, GAO reviewed the fiscal year 2018 plan and assessed whether the plan included details for each of the elements as required by the NDAA.

GAO found that the 2018 plan provided details on most of the reporting requirements in the National Defense Authorization Act (NDAA) for Fiscal Year 2018, but did not include information on future costs and funding needs (see table below). In the NDAA, Congress directed the President to produce a plan that would address four reporting requirements: (1) a plan and roadmap on verification, detection and monitoring efforts, including details on costs and funding needs over 10 years, (2) an international engagement plan, (3) a research and development plan, and (4) a description of interagency engagement. The National Nuclear Security Administration (NNSA), a separately organized agency within the Department of Energy, developed the plan and submitted it to Congress in April 2018.

According to NNSA officials, NNSA did not include long-term costs and funding needs in the plan because identifying these needs is unrealistic for several reasons, including because agencies have little influence over the spending priorities of other departments outside of the President's budget process. However, NNSA and other agencies already develop plans with long-term funding priorities and cost estimates for other programs. Because the plan does not include any estimates on future costs and funding needs, it limits congressional understanding of the long-term affordability of the nation's verification and monitoring efforts and its ability to make necessary funding and policy decisions. GAO has previously reported that providing estimates of future costs and funding needs can help congressional decisionmakers prioritize projects and identify long-term funding needs. By including in its plan estimates of future costs and funding needed to support the activities in the plan, NNSA could help provide assurance that agencies are allocating appropriate resources to the verification and monitoring effort and that these resources are aligned with future activities and processes.

GAO recommends that the Administrator of NNSA should include in its plan estimates of future costs and funding needed to support the activities in the plan. NNSA neither agreed nor disagreed with the recommendation, but said it planned no further action. GAO maintains that the recommendation is valid.

In the National Defense Authorization Act (NDAA) for Fiscal Year 2018,  Congress included a provision directing the President to develop and  submit to Congress a plan for verification and monitoring relating to the  potential proliferation of nuclear weapons, components of such weapons,  and fissile material. This provision came out of issues that were  highlighted in a 2014 report from the Department of Defense’s Defense  Science Board (DSB) on nonproliferation verification and monitoring  technologies. As a result of the DSB report, in the fiscal year 2015 and  fiscal year 2017 NDAAs, Congress directed the Administration to submit  interagency plans for verification and monitoring. The Administrations  submitted a classified plan in October 2015 and a classified update to the  plan in May 2017 in response to these mandates. We assessed the 2015  plan and 2017 update in March 2018 and found that they generally did  not address the reporting requirements.

In the fiscal year 2018 NDAA, Congress indicated that the 2015 plan and  2017 update failed to answer congressional requirements and did not  reflect the high priority of this issue. Further, the NDAA reiterated the  DSB report’s finding that “The nuclear future will not be a linear  extrapolation of the past…and the technologies and processes designed  for current treaty verification and inspections are inadequate to future  monitoring realities.” To address these issues, in the fiscal year 2018  NDAA Congress directed the President to produce a new plan that would  address four reporting requirements: (1) a plan and roadmap on  verification, detection, and monitoring efforts, including details on future  costs and funding needs over 10 years; (2) an international engagement  plan; (3) a research and development (R&D) plan to describe current and  planned efforts; and (4) a description of interagency engagement. On  behalf of the Administration, the National Nuclear Security Administration  (NNSA) prepared, and the Department of Energy submitted, a classified  plan to congressional committees in April 2018.

The conference report accompanying the fiscal year 2018 NDAA includes  a provision for GAO to review the plan and provide its assessment to  congressional committees 90 days after submission of the plan. The  objective of this report is to assess whether the Administration’s plan  provided detail on the reporting requirements in the fiscal year 2018  NDAA.

To address this objective, we reviewed the 2018 plan. We assessed the  content of the plan against the reporting requirements and rated the  extent to which it provided detail on those requirements in one of three  ways: (1) addressed the reporting requirement with detail, (2) addressed  the reporting requirement without detail, or (3) did not address the  reporting requirement. In assessing the level of detail, we considered key  factors identified in our previous work on national strategies, federal  planning, and interagency collaboration. For example, we considered  the extent to which the plan provided information regarding specific goals  or objectives; strategies to achieve those goals or objectives and how  they would be implemented; which agency or agencies would implement  the strategies; and other measurable information, such as timelines and  the resources required. We also interviewed officials at NNSA who led the  development of the plan and coordinated with other contributing  interagency partners.

We conducted this performance audit from May 2018 to August 2018 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe the  evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objective.

Countering the proliferation of nuclear weapons and other weapons of  mass destruction (WMD) remains a U.S. national security priority.  According to the 2017 National Security Strategy, terrorist groups  continue to pursue WMD-related materials, which pose a grave danger to  the United States. As also stated in the 2017 National Security Strategy,  Russia’s nuclear arsenal remains the most existential threat to the United  States, China’s nuclear arsenal is growing and diversifying, Iran has the  potential of renewing its nuclear program and North Korea has pursued  nuclear weapons despite international commitments. As the DSB report  noted, U.S. monitoring abilities are increasingly challenged by evolving  risks in 1) the capability of existing nuclear states and 2) the number of  state and nonstate actors possessing or attempting to possess nuclear  weapons. U.S. nonproliferation activities are conducted and coordinated  across multiple government agencies and organizations, as well as the  intelligence community. In addition, these efforts are coordinated with  international entities, national laboratories, industry, and academia.

U.S. nuclear nonproliferation verification and monitoring efforts are guided  by, among other things, U.S. obligations under the Treaty on the Non- Proliferation of Nuclear Weapons (NPT) and U.S. support for the  Preparatory Commission for the Comprehensive Nuclear Test-Ban Treaty  Organization (CTBTO). The NPT lays out the respective responsibilities  of nuclear-weapon and nonnuclear-weapon states with regard to the  transfer, acquisition, possession, control, and manufacture of nuclear  weapons. All nonnuclear-weapon states are required to have a  comprehensive safeguards agreement with the International Atomic  Energy Agency (IAEA) to facilitate IAEA’s safeguards activities. IAEA  safeguards are a set of technical measures and activities by which IAEA  seeks to verify that nuclear material subject to safeguards is not diverted  to nuclear weapons or other proscribed purposes. Under the  Comprehensive Nuclear Test-Ban Treaty (CTBT), which has yet to enter  into force, parties agree not to carry out any nuclear explosions. The  United States supports the work of the CTBTO to build up a verification  regime in preparation for the treaty’s entry into force.

The Administration’s fiscal year 2018 plan for verification and monitoring  described ongoing interagency efforts to support nuclear proliferation  verification and monitoring and includes information about relevant  national priorities, capability gaps, R&D initiatives, and roles and  responsibilities. The 2018 plan (40 pages) is longer and more detailed  than the 2015 plan (2 pages) or the 2017 update (4 pages). The bulk of  the 2018 plan is contained in two chapters—one chapter broadly  describes U.S. and international efforts and roles and responsibilities, and  the other chapter describes ongoing U.S. R&D efforts.

The Administration’s 2018 Plan Generally Addressed the Reporting Requirements but Did Not Identify Costs and Funding Needs

We found the Administration’s 2018 plan provided details on each of the  four major reporting requirements called for in the fiscal year 2018 NDAA  with the exception of future costs and funding needs (see table 1).

The first reporting requirement called for a plan and roadmap for  verification, detection, and monitoring with respect to policy, operations,  and research, development, testing, and evaluation, including—  Identifying requirements for verification, detection, and monitoring;  Identifying and integrating roles, responsibilities, and planning for  verification, detection, and monitoring activities; and

The costs and funding requirements over 10 years for these activities.

We found that the 2018 plan provided detail on verification, detection, and  monitoring requirements and roles and responsibilities, but did not  provide details on future costs and funding needed to support the  activities in the plan.

We found that the plan identified requirements for verification, detection,  and monitoring as required. To identify these requirements, the plan  notes that interagency partners first identified a set of verification and  monitoring priorities. From these priorities they identified a number of  technical gaps. The plan then described dozens of examples of R&D  efforts and non-technical activities to address those technical gaps. For  example, for one gap the plan identifies eight current efforts to address  this gap, including continued Department of Energy and NNSA  investment in sensor capabilities that are small, light, and can operate in  low power.

We found that the plan provided details on the requirement to identify and  integrate roles and responsibilities and planning. The plan includes details  of the roles and responsibilities of interagency partners and international  bodies that cooperate in the nonproliferation realm. For example, the plan  describes how the Department of Defense is to support U.S. verification  activities under the CTBT, including the installation, operation, and  maintenance of U.S. International Monitoring Systems.

We found that the plan did not identify costs and funding needs over a 10- year period. NNSA officials stated that they believed providing funding  information over a 10-year period is unrealistic for several reasons. First,  according to NNSA officials, it is not feasible to achieve agreement on  actual or implied budgets outside of the existing President’s budget  process. Second, according to NNSA officials, agencies have little  influence over the funding priorities of other departments outside of  existing budget efforts. Third, according to NNSA officials, long-term  funding estimates are infeasible because the President’s budget only  identifies funding levels five years into the future. However, the 2018  NDAA did not ask for budget information. Instead, the NDAA reporting  requirement called for long-term costs and funding information necessary  to support the verification and monitoring activities in the plan. Finally,  NNSA officials told us that they and officials from other agencies briefed  the appropriate congressional committees prior to the release of the 2018  plan, and discussed the challenges with providing cost and funding data.

According to NNSA officials, they verified with the congressional  committees that providing such information in the plan would be  impractical.

We have previously reported that providing estimates of future costs and  funding needs can help congressional decisionmakers prioritize projects  and identify long-term funding needs. NNSA as well as other agencies  within the federal government already develop plans with long-term  funding priorities and cost estimates. For example, in June 2014, we  reported on 10-year estimates for sustaining and modernizing U.S.  nuclear weapons capabilities. As we found in this and other reports,  even when budgets are preliminary or not yet known, plans that include a  range of potential estimates help Congress prioritize projects and  funding. Because the plan does not include any information on  interagency costs and funding needs, it limits 1) congressional  understanding of the long-term affordability of the nation’s verification and  monitoring efforts and 2) Congress’s ability to make necessary funding  and policy decisions. By including in its plan estimates of future costs and  funding needed to support the activities in the plan, NNSA could help  provide assurance that agencies are allocating appropriate resources to  the verification and monitoring effort.

In addition, including estimates of future costs and funding needs in the  plan can help ensure that interagency partners understand the amount of  resources necessary to support verification and monitoring efforts, and  determine if these resources align with agency activities. We have  previously reported on the importance of identifying resources among  collaborating agencies; we noted that without information on resource  contributions from partners in a collaborative effort, there is less  assurance that agency contributions are appropriate to successfully  sustain the effort. Similarly, providing information on future costs and  funding needs is important to help interagency partners coordinate and  develop long-term strategic plans that align with future interagency  efforts. We have found that for strategic planning to be done well, plans  should demonstrate alignment between activities, core processes, and  resources that support mission outcome. By including in its plan  estimates of future costs and funding needed to support the activities in  the plan, NNSA could help provide assurance that agencies are allocating  appropriate resources for interagency efforts and that these resources are  aligned with future activities and processes.

The second reporting requirement called for an international engagement  plan for building cooperation and transparency—including bilateral and  multilateral efforts—to improve inspections, detection, and monitoring  activities. We found that the 2018 plan provided detail on this  requirement. The 2018 plan reiterates the nation’s commitment to the  NPT and includes information on IAEA’s safeguards programs and U.S  support for those programs. For example, under the plan, interagency  partners are to continue to encourage countries through diplomatic  outreach to conclude Additional Protocol agreements with IAEA.

The third reporting requirement called for the plan to describe current and  planned R&D efforts toward improving monitoring, detection, and in-field  inspection and analysis capabilities, including persistent surveillance, remote monitoring, and rapid analysis of large data sets; and measures to  coordinate technical and operational requirements early in the process.  We found that the 2018 plan provided detail on this requirement. The plan  includes detail on a wide range of R&D efforts and non-technical efforts  that agencies are pursuing. For example, the plan reports that the  Defense Advanced Research Projects Agency is starting a program that  models millions of nodes and billions of connections to support the  detection of WMD proliferation activities. In addition, the plan describes  interagency groups involved in coordinating R&D requirements, such as  the National Science and Technology Council Subcommittee on Nuclear  Defense Research and Development.

The fourth reporting requirement called for the plan to describe the  engagement of relevant federal departments and agencies; the military  departments; national laboratories; industry; and academia. We found  that the 2018 plan provided detail on this requirement. The plan includes  detail on the roles and responsibilities for interagency partners, as well as  information on interagency organizations and working groups to  coordinate efforts and reduce duplication. For example, the plan  discusses the Department of State’s efforts to lead the interagency policy  process on nonproliferation and manage global U.S. security policy, and  the Department of Defense’s support of U.S. diplomatic efforts, including  agreements with other defense departments, R&D cooperation, and  multinational exercises.

This 2018 plan represents the third effort by Administrations to address  the nation’s nuclear proliferation verification and monitoring efforts. The  2018 plan provides more detail on these efforts than the 2015 plan and  2017 update. However, the plan does not include estimates of future  costs and funding needs as required by the fiscal year 2018 NDAA. Costs  and funding information can help congressional decisionmakers prioritize  projects and identify potential long-term funding needs. Similarly, costs  and funding information helps interagency partners understand what  resources they are expected to contribute in the future and helps to  ensure long-term strategic plans reflect an alignment between resources  and interagency activities. By including in its plan estimates of future  costs and funding needed to support the activities in the plan, NNSA  could help provide assurance that agencies are allocating appropriate  resources to the verification and monitoring effort and interagency  activities, and that these resources are aligned with future activities and  processes.

We are making the following recommendation to NNSA:  The Administrator of NNSA should include in its plan for verification and  monitoring estimates of future costs and funding needed to support the  activities in the plan. (Recommendation 1)

We provided NNSA with a draft of this report for review and comment.  NNSA provided written comments, which are summarized below and  reproduced in appendix I; the agency neither agreed nor disagreed with  our recommendation to include estimates of future costs and funding  needed to support the activities in its plan for nuclear proliferation  verification and monitoring. However, NNSA stated that it planned no  further action with regard to costs and funding data. NNSA also provided  technical comments, which we incorporated as appropriate.

NNSA stated that it appreciated our recognition of improvements in the  2018 plan for verification and monitoring over the 2015 plan and the 2017  update. In its written comments, NNSA acknowledged that it did not  include interagency cost and funding requirements in the 2018 plan over  10 years as required in the NDAA. The agency stated that it briefed the  appropriate congressional committees before the release of the plan  about the challenges and feasibility of providing the cost and funding data  and received no objections on the omission of the data from the plan.  NNSA also stated that it informed us of the briefings. We have added  clarification in our report that NNSA officials believed they received  agreement from congressional staff to exclude funding and cost estimates  from its plan.

NNSA stated that the NDAA did not prioritize the relative importance of  the reporting requirements, and that we disproportionately weighted the  one omission in our assessment, effectively overstating the importance of  providing cost and funding information. In addition, NNSA identified  challenges to the feasibility of providing interagency out-year cost and  funding estimates, including the difficulty to quantify the level of R&D and  associated funding required to achieve specific outcomes and that  departments and agencies are unable to commit to aligning 10 year  funding estimates with individual agencies’ timelines and internal  processes for planning, programming, budgeting, and execution.

NNSA’s statement suggests that it views nuclear proliferation verification  and monitoring programs as being unique and different from other federal  programs and that they should therefore be exempt from estimating their  potential long-term resource burden on the federal budget. We disagree.  Developing future cost and funding estimates for programs is central to  effective interagency planning efforts. The efforts described in NNSA’s  2018 nuclear verification and monitoring plan span a diverse range of  activities that are implemented across multiple agencies. The absence of  cost and funding estimates for these efforts in NNSA’s plan raises  questions as to whether there is an effective interagency process to  coordinate these efforts and if the process is taking adequate account of  resource constraints and making realistic assessments of program  resource needs. In addition, information on future cost and funding  estimates of federal programs provides Congress with a better  understanding of the potential long-term funding needs and costs of the  diverse efforts supporting the proliferation verification and monitoring  mission. We believe this big picture view is important given the multiple  congressional committees of jurisdiction—including appropriations,  authorization, and oversight committees—for the efforts identified in  NNSA’s plan.

Regarding the feasibility of providing 10-year cost and funding estimates,  we recognize the difficulty and uncertainty agencies face in estimating  future funding needs. However, we do not believe developing such  estimates is impossible. As we reported, the Department of Defense  (DOD) and the Department of Energy (DOE) prepare an annual plan with  10 year cost and funding estimates for their ongoing nuclear sustainment  and modernization efforts, including R&D efforts.

NNSA also provided general technical comments addressing our findings  on the cost and funding estimates that were not included in the plan,  including comments on NNSA’s authority to obtain 10-year estimates  from other agencies, and on the examples we cited of other interagency  plans that include similar estimates.

NNSA stated that it did not have authority to require other agencies to  submit 10-year budget estimates for their efforts that are included in the  plan. We noted in our report that Congress directed the President to  include this element in the nuclear proliferation verification and monitoring  plan. However, responsibility to prepare and submit the plan was  delegated by the President to DOE.

NNSA commented that the joint DOD-DOE annual nuclear sustainment  and modernization plan is not comparable to the NNSA plan because the  former primarily addresses capital projects and other material products,  while the latter primarily addresses R&D activities. The reporting  requirements for NNSA’s nuclear proliferation verification and monitoring  plan were not limited to R&D efforts, but included cost and funding  estimates for related activities and capabilities, including policy,  operations, testing, and evaluation. NNSA’s comment focuses only on the  difficulty of addressing cost and funding estimates for only one aspect  (R&D) of nuclear proliferation verification and monitoring and ignores the  possibility that estimates for non-R&D efforts may be more feasible and  less difficult to report. Moreover, we have reported that the joint DOD- DOE plan on nuclear modernization includes 10-year DOD and DOE  estimates for R&D, as well as estimates for related modernization efforts,  including infrastructure, nuclear weapon life extension programs, delivery  systems, nuclear command, control, and communications systems, and  other related activities.

We are sending copies of this report to the appropriate congressional  committees, the Administrator of NNSA, and other interested parties. In  addition, this report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-3841 or trimbled@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made significant contributions to this  report are listed in appendix II.

Appendix I: Comments from the Department of Energy

Appendix II: GAO Contact and Staff Acknowledgments:

In addition to the contact named above, William Hoehn (Assistant  Director), Dave Messman (Analyst-in-Charge), Alisa Carrigan, Antoinette  Capaccio, Ben Licht, Steven Putansu, and Gwen Kirby.

In 2015 and 2017, federal law directed the President to submit a plan to Congress for monitoring the potential spread of nuclear weapons and materials. In earlier reports, we found those plans did not address required areas.

A 2018 law again directed the President to produce a plan, which the National Nuclear Security Administration prepared. Most required subjects were included, such as plans for research and development and international engagement. However, the plan did not include information on its cost over the next 10 years.

We recommended the NNSA provide estimates of the funding needed to support the activities in the plan.


Commercial Space Launch Insurance: FAA Needs to Fully Address Mandated Requirements

The federal government shares liability risks with the commercial space launch industry for accidents that result in damages to third parties or federal property. This arrangement requires space launch companies to have a specific amount of insurance to cover these damages. The government is potentially liable for damages above that amount, up to a cap GAO estimated to be $3.1 billion in 2017, subject to appropriations in advance.

CSLCA, enacted in 2015, directed the Department of Transportation, of which FAA is a part, to evaluate its MPL methodology and, if necessary, develop a plan to update that methodology. The act also included a provision requiring GAO to assess FAA's evaluation and any actions needed to update the methodology.

This report discusses the extent to which (1) FAA's evaluation report addresses the requirements in CSLCA and (2) FAA has addressed previously identified weaknesses in the MPL methodology. GAO reviewed documents and interviewed FAA on its loss methodology evaluation and actions to address weaknesses.

The Federal Aviation Administration's (FAA) report evaluating its maximum probable loss (MPL) methodology did not fully address the evaluation and consultation requirements specified by the U.S. Commercial Space Launch Competitiveness Act (CSLCA).

Balance of Risk. CSLCA required FAA to include ensuring that the federal government is not exposed to greater indemnification costs and that launch companies are not required to purchase more insurance coverage than necessary as a result of FAA's MPL methodology. FAA said that it ensured this balance by improving its methodology, but it did not reevaluate its probability thresholds after revising its methodology. These thresholds are used to divide the risk of loss between launch companies and the government.

Impact on Costs. The act required FAA to consider the costs to both the industry and the federal government of implementing an updated methodology. FAA's report discussed the impact on indirect costs, such as data collection, but did not discuss direct costs: insurance premiums for launch companies and indemnification liability for the federal government.

Consultation. The act also required FAA to consult with the commercial space sector and insurance providers in evaluating its MPL methodology in accordance with the preceding requirements. While the agency consulted with some stakeholders, these consultations were limited in scope.

FAA officials said they have not been able to take the actions needed to fully satisfy the mandated elements because of issues such as resource limitations and the lack of available data. However, by not resolving these issues, FAA lacks assurance that launch companies are not purchasing more insurance than needed or that the federal government is not being exposed to greater indemnification costs than expected.

FAA has addressed two of three previously identified weaknesses in its MPL methodology but has not yet dealt with the remaining weakness. Specifically, the agency has revised its methodology for estimating the number of potential casualties for a launch and changed the factor it uses to derive estimated property damage from estimated casualties. However, FAA has not updated the amount used for the cost of an individual casualty. GAO recommended in a March 2017 report (GAO-17-366) that FAA update this amount. Not doing so could understate the amount of insurance launch companies are required to purchase, exposing the federal government to excess risk.

GAO also determined that while FAA has two tools and methods it can use in making its MPL estimates, it does not have guidance on determining which are most appropriate for a given launch scenario. For example, one tool is more comprehensive but also labor intensive to use, while the other is inappropriate for certain launch scenarios and could result in misleading MPL amounts. Officials said they have begun to create such guidance but do not have an estimated completion date. Without such guidance, FAA cannot ensure that the most appropriate MPL methodology is used for each launch.

FAA should fully address mandated requirements in evaluating its MPL—probability thresholds, direct costs, and stakeholder consultations— and establish an estimated completion date for developing guidance on tools and methods to use for specific launch scenarios. The Department of Transportation concurred with the recommendations, and provided technical comments.

Since 1988, the federal government has sought to assist in the  development of the commercial space launch industry by sharing liability  risks for accidents leading to damages to third parties or federal property  and personnel. This risk-sharing arrangement requires that space launch  companies—firms that launch satellites or other payloads into space or  reenter vehicles from space—purchase insurance against claims by third  parties and for loss or damage to federal property and personnel up to a  maximum probable loss (MPL) amount. This amount is to be determined  by the Federal Aviation Administration’s (FAA) Office of Commercial  Space Transportation. For claims that exceed the MPL threshold, the  federal government is potentially liable for damages, subject to  appropriation, up to $3.1 billion in 2017 (the equivalent to $1.5 billion in  1988).

In November 2015, Congress passed the U.S. Commercial Space  Launch Competitiveness Act, (CSLCA) which required FAA to evaluate its  MPL methodology and report the results of that evaluation to Congress.  In addition, it required the agency, as part of its evaluation, to consult with  the commercial space sector and insurance providers and, if necessary,  develop a plan to update that methodology so the federal government is  not exposed to greater costs than intended and launch companies are not  required to purchase more insurance than necessary. Although FAA was  required to submit its report on updating its methodology to Congress by  May 2016, it did not do so until April 2017, citing an internal agency  review process for the delay.

CSLCA also includes a provision for us to assess, among other things,  the analyses conducted and conclusions made in FAA’s MPL evaluation  report to Congress. This report examines (1) the extent to which FAA’s  mandated report addresses the requirements in CSLCA and (2) the  extent to which the agency has addressed previously identified  weaknesses in the MPL methodology.

To address the first objective, we analyzed the CSLCA requirements for  FAA’s report and evaluated the information included in FAA’s report, as  well as additional information provided by officials from FAA’s Office of  Commercial Space Transportation. Specifically, we analyzed supporting  documentation, including contractor reports and meeting minutes;  interviewed FAA officials; and compared this evidence to federal internal  control standards and GAO guidance on conducting an economic  assessment.

To address the second objective, we reviewed prior GAO  recommendations, and weaknesses in the MPL methodology identified by  us and other entities. We assessed FAA’s efforts to address these  recommendations and weaknesses using documentation from and  interviews with Office of Commercial Space Transportation officials about  the steps that FAA has taken or plans to take. We also interviewed these  officials, and reviewed documentation on guidance on determining the  most appropriate analytical tool to use calculate MPL values for a given  launch scenario. We also assessed the reasonableness of FAA’s actions  and plans using federal internal control standards where appropriate.

For both objectives, we reviewed information collected for our March  2017 report on the agency’s revisions to its MPL methodology, including  the actions the agency has taken to address the weaknesses we and  others had previously identified. For the purposes of this current report,  we are updating the information collected for the March 2017 report as  appropriate.

We conducted this performance audit from May 2017 to January 2018 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

The Commercial Space Launch Act Amendments of 1988 established the  foundation for the current U.S. policy to potentially provide federal  payment for a portion of claims by third parties for injury, damage, or loss  that results from a commercial launch or reentry accident. A stated goal  of the act was to provide a competitive environment for the U.S.  commercial space launch industry. The act also provided for, among  other things, government protection against some losses—referred to as  indemnification—while still minimizing the cost to taxpayers. All FAA- licensed commercial launches and reentries by U.S. companies, whether  unmanned or manned and from the United States or overseas, are  covered by federal indemnification for third-party damages that result  from the launch or reentry. According to agency officials, in 2016 FAA  issued five active licenses, which had an average third-party MPL of  about $51 million and ranged from $10 million to $99 million.

The amount of insurance coverage that FAA requires launch companies  to purchase—the MPL value—is intended to reflect the greatest dollar  amount of loss to third parties and the federal government for bodily injury  and property damage that can be reasonably expected to result from a  launch or reentry accident. FAA calculates separate MPL values for  potential damages to third parties and the federal government. For each  launch license that it issues, FAA determines MPL values for third parties  with the intent of estimating the greatest dollar amount of losses that  reasonably could be expected from a launch or reentry accident, which  have no less than a 1 in 10 million chance of occurring. For damages to  the federal government, FAA determines MPL values with the intent of  estimating the greatest dollar amount of losses that reasonably could be  expected from a launch or reentry accident, which have no less than a 1  in 100,000 chance of occurring. According to FAA, the agency defines  these probability thresholds to estimate the federal government’s  exposure to losses above the MPL. Agency officials said that the current  probability thresholds are set such that losses are very unlikely to exceed  launch companies’ private insurance and become potential costs for the  government under CSLCA.

FAA’s process for determining the MPL value for a launch or reentry  license generally includes three elements:  1.  Number of casualties. Estimating the number of third-party  casualties involves adding the number of direct and secondary  casualties that could result from a launch accident. Direct casualty  estimates include serious injuries and deaths. Secondary casualties  include those resulting from fires and collapsing buildings.  2.  Cost of casualties. FAA uses $3 million as an estimate of the  average loss per casualty, which is multiplied by the number of  estimated casualties.  3.  Property damage. FAA applies a predetermined factor—which it  recently changed from 50 percent to 25 percent—to the estimated  cost of casualties to derive estimated losses from property damage.

The total MPL is equal to the estimated cost of casualties plus property  damage.

FAA has revised two components of its MPL methodology since our 2012  report. For example, in April 2016, the agency adopted a new method for  estimating the number of casualties, known as the risk profile method.  This method uses different tools to simulate a range of possible scenarios  to create a distribution of potential casualty numbers and the simulated  probability of different levels of casualty numbers. The risk profile method  replaced FAA’s “overlay method,” which was a method it had used since  the early 1990s which the agency said did not work well for launches of  small launch vehicles in remote areas, or for reentries. In addition, FAA  reduced the factor it uses to estimate losses due to property damage,  based on tests of a new process for estimating such losses that showed  the previous factor was too high.

GAO Previously Reviewed FAA’s MPL Methodology

We have previously reviewed FAA’s MPL methodology in 2012 and 2017.  In 2012 we examined the U.S. government’s indemnification policy, the  federal government’s potential costs for indemnification, and the effects of  ending indemnification on the competitiveness of U.S. launch companies,  among other aspects of FAA’s MPL methodology. In 2017 we examined  the extent to which FAA had revised its MPL methodology since our 2012  report to address previously cited weaknesses and the potential effect of  any changes to that methodology on financial liabilities for the federal  government. The findings and recommendations of those reports,  including any unaddressed weaknesses, are discussed later in this report.

FAA Did Not Fully Address the CSLCA’s Three Mandated Requirements

CSLCA required FAA to evaluate its MPL methodology incorporating  three requirements, but the agency’s report did not fully address these  requirements.

First, the act required FAA to ensure a balance of risk between launch  companies and the federal government. However, agency officials  told us that they did not re-evaluate the probability thresholds—which  are used to divide the risk of loss between launch companies and the  federal government—as part of evaluating its MPL methodology when  implementing the risk profile method due to resource constraints.

Second, the act required FAA to consider the cost impact of  implementing an updated MPL methodology, but the agency did not  evaluate the impact of implementing its revised methodology on the  direct costs to launch companies (insurance premiums) and to the  federal government (indemnification liability).

Third, the act required FAA to consult with the commercial space  sector and insurance providers in evaluating its MPL, but they did not  consult such parties in response to the act.

Without fully addressing CSLCA’s mandated requirements, FAA cannot  ensure that the federal government is not exposed to greater liability  costs than intended or that launch companies are not required to  purchase more insurance coverage than necessary.

FAA Has Not Fully Evaluated the Balance between Government Liability Exposure and Industry Insurance Costs

In its report, FAA states that implementing an updated MPL methodology  in April 2016—the risk profile method—helps ensure that the federal  government is not exposed to greater liability costs than intended and that  launch companies are not required to purchase more insurance coverage  than necessary, as required under CSLCA. Further, agency officials told  us that their updated methodology is technically more valid and improves  their ability to avoid overestimating MPL values (which can cause launch  companies to purchase more insurance coverage than necessary) or  significantly underestimating MPL values (which can expose the federal  government to greater costs than intended).

While an improved model may provide a more realistic calculation of the  MPL, by changing the resulting estimates it can also change the balance  between the federal government’s exposure to liability costs and the  amount of insurance launch companies are required to purchase. For  example, if the more realistic results produced by the revised  methodology increased the MPL estimates, this would increase insurance  costs for the launch companies and reduce the federal government’s  exposure, thereby shifting the balance of costs between the two and  suggesting a reevaluation of the thresholds.

In addition, FAA officials told us that they had not reevaluated the  probability thresholds upon implementing the revised MPL methodology,  although defining these thresholds is their primary mechanism for  adjusting the balance of risk between launch companies and the federal  government. Agency officials acknowledged that an examination of the  thresholds’ continued appropriateness would be warranted in the future.

However, they told us that changing the probability thresholds would  require significant effort because it would require them to change federal  regulations and that resources are currently allocated to other rulemaking  priorities. Nevertheless, without evaluating the appropriateness of the  probability threshold as part of the mandated evaluation of the MPL  methodology, FAA cannot ensure that the federal government is not  exposed to greater liability costs than intended or that launch companies  are not required to purchase more insurance coverage than necessary.

FAA Evaluated Only Indirect Costs to Industry and Government of Implementing a New Methodology

CSLCA also required FAA to consider the cost impact on both the  commercial space launch industry and the federal government of  implementing an updated MPL methodology. In its report to Congress,  the agency discussed indirect costs to launch applicants and the federal  government. For example, FAA discussed indirect data burden costs on  launch company applicants and FAA analysts associated with the  agency’s risk profile method implementation. The report states that the  risk profile method requires more data from a launch applicant than the  previous method, but that the added burden is minimal because the  information is similar to the type of information required by FAA for a risk  analysis. Agency officials also said that the risk profile method requires  more of an FAA analyst’s time than the overlay method, but that the  added burden is minimal because the work done by FAA on risk analysis  provides much of the foundation for an MPL analysis.

However, FAA’s report did not include an evaluation of the direct costs to  launch companies and the federal government of implementing an  updated MPL methodology. The report identifies the direct cost to the  launch industry as insurance premiums, and the direct cost to the federal  government include potential indemnification payments. Agency officials  also told us that the agency does not track commercial space launch  insurance costs, and that they do not have meaningful insights on  insurance premiums paid by commercial launch companies. FAA officials  told us that they only have a general notion of insurance premiums  because the industry is reluctant to share such information. FAA officials  also told us that, outside of the work done for the report, they have not  evaluated the economic implications for launch companies of  implementing an updated MPL methodology. Without evaluating direct  costs to both the launch companies and the federal government, FAA will  be limited in its ability to consider the impact of the cost to both the  industry and the federal government of implementing an updated  methodology.

FAA Obtained Limited Input from the Commercial Space Sector and Insurance Providers

Although CSLCA required FAA to consult with the commercial space  sector and insurance providers in evaluating its MPL methodology for the  mandated report, it obtained limited input. For example, FAA officials told  us that they obtained input from their Commercial Space Transportation  Advisory Committee (COMSTAC) in April 2016 about what to include in  their report to Congress, but did not consult with the commercial space  sector and insurance providers to evaluate their MPL methodology in  response to CSLCA.

FAA officials also said that, to respond to CSLCA’s consultation  requirement, they did not think they needed to repeat the consultations  they took in 2013. In January 2013, the agency solicited input from  COMSTAC’s Business/Legal Working Group about how to best conduct a  review of FAA’s methodology for calculating MPL, in response to our July  2012 report. FAA also briefed the Business/Legal Working Group in  May 2013 to solicit input on MPL methodologies, including the risk profile  method. In the January 2013 meeting, a COMSTAC member suggested  several contractors for a study by outside experts of the complete MPL  methodology, and FAA subsequently hired one of these contractors to  develop the risk profile method that it implemented in April 2016.  However, the agency did not solicit input from COMSTAC about its risk  profile methodology prior to its April 2016 implementation or following  CSLCA’s November 2015 mandated evaluation. As a result, FAA lacks  input on the effect of its revised MPL methodology on launch companies  and the federal government, making it difficult to evaluate the balance of  risk between the two.

FAA’s Revised MPL Methodology Does Not Fully Address Certain Previously Identified Weaknesses

Our 2012 report identified concerns with all three components of FAA’s  MPL methodology: estimating the number of casualties, estimating the  cost of casualties, and deriving estimated property damage costs from  estimated casualty costs. In that report we recommended that the agency  reassess its methodology, including the reasonableness of several key  elements. As noted in our 2017 report, FAA has since made  improvements to its methodology. However, it still has not yet updated the  cost of a casualty. In addition, in our 2017 report we also noted that there  are instances where deriving estimated property damage from estimated  casualty costs is inappropriate. As of November 2017, FAA does not have  guidance to identify such instances or to guide decisions on which tools to  use in developing the MPL estimate.

FAA Has Made Improvements to Its MPL Methodology but Has Not Updated the Cost-of- Casualty Amount

FAA has taken steps designed to improve two of three elements of its  MPL methodology, including revising its methodology for estimating the  number of potential casualties for a given launch and changing the factor  it uses to derive estimated property damage from estimated casualties.  However, the agency has not updated the third element, the amount it  uses for the cost of an individual casualty, leaving a previously identified  weakness unaddressed.

Our 2012 report raised concerns with each of the three components of  FAA’s MPL calculation methodology.

First, we found that FAA’s method for estimating the number of  casualties involved use of a single loss scenario instead of applying  the insurance industry’s standard practice of catastrophe modeling,  and that the agency’s method might significantly understate the  number of potential casualties. Catastrophe modeling, unlike the  single-loss approach, generally estimates losses by using various  tools to simulate tens of thousands of scenarios to create a  distribution of potential losses and the simulated probability of  different levels of loss.

Second, we reported that FAA had been using an outdated and likely  understated figure of $3 million to estimate the cost of a single  casualty—including injury or death—which Office of Commercial  Space Transportation officials said has not been updated since they  began using it in 1988.

Third, we reported that the agency’s approach of estimating potential  property damage by adding a flat 50 percent to the estimated casualty  damage could lead to estimates that were too high in some cases.

Given these weaknesses, we recommended that FAA reassess its MPL  methodology, including assessing the reasonableness of the cost-of- casualty amount and other assumptions used. Because the agency took  actions to assess its MPL methodology, we closed the recommendation  as implemented.

In March 2017 we reported that FAA had taken steps to address  weaknesses in two of these three areas. Specifically, we reported that  FAA’s adoption of the risk profile method in April 2016 had improved its  estimates of the number of potential casualties associated with a  particular license launch. In addition, we reported that the agency had  revised the factor it uses to estimate losses from property damage in the  MPL calculation from 50 percent to 25 percent. This change has resulted  in property damage estimates that FAA officials believe are still  conservative but more realistic than previous estimates.

However, in our March 2017 report we also determined that FAA had not  yet addressed weaknesses in the cost-of-casualty amount we had  previously identified; despite the conclusion by a contractor it had hired to  study the cost-of-casualty that it was too low. Agency officials told us that  they had not addressed this weakness because of other priorities. Given  the significance of the cost-of-casualty amount to the MPL calculations,  we recommended that FAA prioritize the development of a plan to  address the identified weakness in the cost-of-casualty amount, including  setting time frames for action, and update the amount based on current  information. In October 2017, FAA officials told us that they had not yet  updated the cost-of-casualty because they have continued to prioritize  completing other work with their limited resources, such as reviewing  launch applications and fulfilling other safety responsibilities. As a result,  our recommendation remains open.

FAA officials told us that they have identified potential steps to update the  cost-of-casualty amount, including seeking public input on whether and  how to revise the amount, but that they do not expect to make a decision  on whether to make any changes to the cost-of-casualty amount until  June 30, 2018, at the earliest. FAA officials told us that in order to  prioritize the development of a plan to address the identified weakness in  the cost-of-casualty amount they will need to consult with both the  commercial space and insurance industries about the necessity and  implications of any potential increase in the cost-of-casualty amount.  Agency officials said that they plan to do such consultations through  COMSTAC. However, because COMSTAC was just reestablished in  June 2017 after not having been active since November 2016 and new  members had not been approved as of October 2017, the anticipated  decision date of June 2018 could be further delayed.

As we reported in March 2017, an understated cost-of-casualty amount  can lead to an inaccurate loss calculation, which in turn understates the  amount of insurance a launch company must obtain. This could increase  the potential exposure to the federal government, as the insurance  amount would be less than the potential losses associated with the  launch activity and the property would be inadequately protected.  Because of this potential exposure, we maintain that addressing this  weakness is a priority.

FAA Does Not Have Guidance for When to Estimate Property Damage Separately from the Number of Casualties and Which Analytical Tool to Use

As noted above, in our 2012 report we raised concerns about the first  element of FAA’s MPL methodology, which is estimating the number of  potential casualties. FAA officials said that they have implemented two  tools for estimating the number of potential casualties, and that each tool  requires a different level of resources and is more appropriate for different  launch scenarios. The Range Risk Analysis Tool creates physics-based  simulations of possible accidents using launch vehicle data, such as  launch trajectory and types of failures, and assigns each simulated  accident a probability of occurrence based on the failure rates of the  different elements of the launch vehicle. According to agency officials, the  Range Risk Analysis Tool is a comprehensive, high-fidelity tool and is the  most appropriate tool for coastal launch sites, which are often located in  heavily populated areas, and is labor intensive. The Risk Estimator Sub- orbital and Orbital Launch Vehicle and Entry tool, which in contrast to the  Range Risk Analysis Tool, is a medium-fidelity tool that can be used for  low-risk launches, such as launch sites located in very sparsely populated  areas and reentry operations that do not need the use of a high-fidelity  tool. According to FAA, this tool significantly reduces the time required to  estimate the risk from launch and reentry vehicle operations.

In our 2017 report, we also reiterated that there are cases where the third  element of FAA’s methodology, deriving estimated property damage from  estimated casualties, could lead to misleading MPL calculations.  Specifically, in March 2017 we reported that estimating losses from  property damage as a percentage of losses from casualties could lead to  overestimates. For example, FAA’s contractor found that, if a launch  accident affected a residential area, the agency’s practice of estimating  property damage based on casualties would likely overstate property  damages because residential structures have relatively low values  compared to losses from casualties. We also reported in March 2017 that  in some accidents the number of casualties may be low but property  losses could still be very large, in which case FAA’s estimating property  losses based on casualties would likely understate potential property  damage. For example, a launch vehicle could strike an unoccupied  structure that is very expensive, such as a neighboring launch complex.  Agency officials said that while deriving property losses from casualty  losses is a simpler method that may be an effective use of limited FAA  resources, it could be inappropriate in scenarios where the number of  casualties might be low but property losses could still be very large.

In October 2017, agency officials said that FAA had not developed  guidance for determining, for a given launch license, which of the  available tools would be most appropriate to estimate the number of  potential casualties, and whether it would be more appropriate to estimate  property losses separately rather than derive them from estimated  casualties. While FAA officials said they believe their current decision  process is adequate and that they do not need more formal guidance at  this time, they also told us that they were in the process of developing  internal guidance on the most appropriate tool to use for future launches.  The officials said that they did not have a projected completion date for  the guidance, primarily because the agency has other priorities and  resource limitations. As noted earlier, these priorities include reviewing  commercial space launch license applications and managing program  safety.

Federal internal control standards state that, as part of an entity’s risk  assessment component, management should identify, analyze, and  respond to risks to achieving objectives. For example, the standards  state that management should design control activities in response to the  entity’s objectives and risks to achieve an effective internal control  system.

Without such guidance, FAA could face challenges in ensuring that it is  using the most appropriate method to calculate an MPL for a given launch  and is making the most efficient use of its resources. Such guidance  could become more important as the number of commercial space  launches increases, potentially creating greater demands on its  resources. We have previously reported that the commercial space  launch industry has experienced significant growth in the number and  complexity of launches in the past half-decade. FAA has also reported  that its licensed launches have increased 60 percent and industry  revenue has increased 471 percent since 2012.

FAA’s MPL methodology is critical in balancing the encouragement of the  U.S. commercial space industry with the need to manage the federal  government’s risk exposure because it determines how much risk each  party will bear for third-party damages resulting from potential space  launch accidents. However, despite changes to the methodology, the  probability threshold that the agency uses to achieve this balance of risk  has been the same since the 1990s, and has not been reviewed for  appropriateness. In addition, while FAA evaluated the effect of its MPL  methodology on the indirect costs of launch companies and the federal  government, it did not similarly evaluate direct costs. Further, although  FAA has obtained input from some stakeholders on certain aspects of its  MPL methodology, it has not consulted with launch providers and  insurance companies to evaluate effects on key potential costs to launch  companies and the federal government, as required under CSLCA. FAA  officials told us that resource issues and pursuing other priorities have  prevented them from taking these actions. However, the longstanding  nature of these issues, as well as their importance in determining the  federal government’s financial exposure, makes their completion a  priority.

FAA has also begun improving other aspects of its MPL process, but  important actions remain incomplete. For example, the cost of a casualty,  a key component of the methodology, has not been updated since 1988.  While FAA has identified potential steps to update this amount, it has not  implemented these steps and our March 2017 recommendation to  prioritize the updating of this amount remains open. Further, agency  officials said they have begun to develop internal guidance on how to  determine which methodological tools should be used for a given launch,  but are not sure when this process will be completed. These are  important steps to help ensure the validity of the MPL methodology and  the results obtained for each launch, which in turn determine the balance  between the amount of insurance launch companies are required to  purchase and the potential financial exposure for the federal government.

We are making the following four recommendations to FAA:  The FAA Administrator should fulfill the CSLCA mandate to include  ensuring a balance of risk between the federal government and launch  companies as part of FAA’s MPL methodology evaluation by reexamining  the current probability thresholds. (Recommendation 1)

The FAA Administrator should fulfill the CSLCA mandate to analyze the  cost impact of implementing its revised MPL methodology by evaluating  the impact on the direct costs of launch companies and the federal  government. (Recommendation 2)

The FAA Administrator should fulfill the CSLCA mandate to evaluate its  MPL methodology in consultation with the commercial space sector and  insurance providers by consulting with those entities on the cost impact of  its revised MPL methodology, including an updated cost-of-casualty  amount, on the launch industry and the federal government.  (Recommendation 3)

The FAA Administrator should establish an estimated completion date for  developing and implementing a plan to establish guidance on the most  appropriate MPL methodologies and tools to use for each launch.  (Recommendation 4)

We provided a draft of this report to the Department of Transportation for  their review and comment. In its comments, reproduced in appendix I, the  Department of Transportation concurred with our recommendations. The  Department of Transportation also provided technical comments, which  we incorporated as appropriate.

We are sending copies of this report to interested congressional  committees and the Secretary of the Department of Transportation. In  addition, this report will be available at no charge on GAO’s website at  http://www.gao.gov.

If you or your staff have any questions or would like to discuss this work,  please contact Alicia Puente Cackley at (202) 512-8678 or  cackleya@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  Individuals making key contributions to this report are listed in appendix II.

Appendix I: Comments from the Department of Transportation

Appendix II: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Patrick Ward (Assistant Director),  Jessica Artis, Isidro Gomez (Analyst in Charge), Courtney La Fountain,  Maureen Luna-Long, Jessica Sandler, Jennifer Schwartz, Joseph  Silvestri, and Shana Wallace made key contributions to this report.