B61-12 Nuclear Bomb: Cost Estimate for Life Extension Incorporated Best Practices, and Steps Being Taken to Manage Remaining Program Risks

Weapons in the U.S. nuclear stockpile are aging. To refurbish or replace nuclear weapons' aging components, NNSA and DOD undertake LEPs. The B61-12 LEP is the most complex and expensive LEP to date. In October 2016, NNSA formalized a program cost estimate of about $7.6 billion, which is lower than an independent cost estimate of about $10 billion.

Senate Report 113-44 included a provision for GAO to periodically assess the status of the B61-12 LEP. This report assesses (1) the extent to which NNSA followed best practices for cost estimation in producing the program cost estimate for the B61-12 LEP; (2) the reasons for differences between the program cost estimate and the independent cost estimate and how the differences were reconciled; and (3) the extent to which NNSA and DOD have identified and managed program risks. GAO assessed the program cost estimate against best practices, reviewed NNSA and DOD documents, conducted site visits to four NNSA and Air Force sites responsible for design, production, and management activities, and interviewed NNSA and DOD officials.

The National Nuclear Security Administration (NNSA) incorporated most cost estimating best practices to develop the program cost estimate for the B61-12 Life Extension Program (LEP), which seeks to consolidate four versions of a nuclear weapon—the B61 bomb—into a bomb called the B61-12. As shown in the figure below, the program substantially met best practices for ensuring the estimate was comprehensive, well-documented, accurate, and credible.

The B61-12 LEP's program cost estimate differs from an estimate prepared by another NNSA office independent of the program primarily because the program used different methods and assumptions than the independent office. The program developed its estimate by compiling cost and schedule estimates for activities at each of the NNSA contractor sites participating in the LEP. In contrast, the independent office evaluated program activities completed to date and applied a historical model to estimate costs and durations for remaining activities. NNSA management met with officials from both offices to reconcile the estimates but did not document the rationale for adopting the program estimate unchanged. GAO recommended in a January 2018 report that NNSA document and justify such decisions, in part because GAO's prior work has shown that independent cost estimates historically are higher than programs' cost estimates because the team conducting the independent estimate is more objective and less prone to accept optimistic assumptions. In response to the January 2018 report, NNSA agreed to establish a protocol to document management decisions on significant variances between program and independent cost estimates, but it has not yet provided evidence that it has done so.

NNSA and the Department of Defense (DOD) have identified and are managing risks that could complicate efforts to meet the LEP's fiscal year 2025 completion date. Risks within the program's areas of responsibility include an aggressive flight test schedule for bomb delivery aircraft. The program is managing these and other risks with a formal risk management process. The program has also taken steps to address risks outside its direct control, such as risks related to the readiness and certification of the weapon's F-35 delivery aircraft, by providing information to the responsible DOD organizations.

GAO is making no new recommendations but discusses a prior recommendation that NNSA document and justify decisions regarding independent cost estimates. NNSA provided technical comments, which GAO incorporated as appropriate. DOD did not have any comments.

The Department of Defense (DOD) has underscored the significant  contribution of B61 nuclear bombs—the oldest nuclear weapons in the  United States’ active stockpile—to assuring U.S. allies in the North  Atlantic Treaty Organization (NATO) and elsewhere of the U.S.  commitment to global security and nuclear nonproliferation. Critical  components of these bombs are approaching the end of their operational  lives. To maintain the safety, security, and effectiveness of B61 bombs,  the Department of Energy’s National Nuclear Security Administration  (NNSA) and DOD are undertaking a life extension program (LEP) that will  result in a bomb known as the B61-12. The B61-12 LEP—one of four  ongoing LEPs—is the most complex and expensive LEP undertaken  since the Department of Energy began stockpile life extension activities in  January 1996.

Our past reports on the B61-12 LEP present a mixed record of  improvements and continuing challenges in program management. Most  recently, in February 2016, we reported positive steps, including that the  program had become the first LEP to develop and use an earned value  management system to monitor cost and schedule performance and that  it had developed a database to help monitor and manage program risks,  such as risks related to component design and procurement of parts.  Moreover, we found that the B61-12 LEP is the first NNSA defense  program to issue a cost estimate that integrates all participating NNSA  sites’ costs into a single program cost estimate. In past LEPs, according  to an NNSA official, NNSA had not integrated its individual  site  contractors’ cost estimates, which contributed to program costs being  underestimated. These improvements notwithstanding, we also reported  in February 2016 that the estimated cost and schedule of the B61-12 LEP  had changed significantly since the LEP’s inception. Specifically, we  found that the cost estimate for the program had increased from an initial  rough estimate of about $4 billion at the time of our May 2011 report on  the LEP to about $8.9 billion at the time of our February 2016 report; the  latter estimate included about $7.3 billion for NNSA’s portion of the work  and $1.6 billion for DOD’s portion, which is separately funded. In October  2016, NNSA formalized its cost estimate in a classified baseline cost  report for the LEP at a higher figure of about $7.6 billion. The baseline  cost report also established an estimated program completion date of  fiscal year 2025. Concurrently with the development of the baseline cost  report, NNSA’s Office of Cost Estimating and Program Evaluation  completed an independent cost estimate for the program. The  independent cost estimate projected a higher cost of about $10 billion and  a later completion date of fiscal year 2027 for NNSA’s work on the B61-12  LEP.

Senate Report 113-44, accompanying S. 1197, a bill for the National  Defense Authorization Act for Fiscal Year 2014, included a provision for  us to examine key elements of the B61-12 LEP and periodically review  the program as it passes through the phases of the process under which  NNSA and DOD jointly manage LEPs. This report examines (1) the extent  to which NNSA followed best practices for cost estimation in producing  the program cost estimate for the B61-12 LEP; (2) the reasons for the  differences between the program cost estimate for the LEP and the  independent cost estimate and how the differences were reconciled, if at  all; and (3) the extent to which NNSA and DOD have identified and  managed risks to the program.

To examine the extent to which NNSA followed best practices for cost  estimation in producing the program cost estimate for the B61-12 LEP,  we reviewed documentation and data on the program cost estimate and  interviewed program officials responsible for producing the cost estimate  to understand the methods, assumptions, information, and data NNSA  used to produce the estimate. Specifically, we reviewed NNSA’s October  2016 baseline cost report, which is the report through which NNSA  formally established its cost estimate for the B61-12 LEP. We also  reviewed documentation that NNSA site contractors participating in the  LEP provided to the B61-12 program office to assist it in compiling the  baseline cost report. This documentation included documents that  established the bases and assumptions for site contractors’ contributions  to the cost estimate, documents that established the contractors’ work  breakdown structures, and presentations on contractors’ cost estimating  models. In addition, we visited and interviewed federal officials and  contractors at the Pantex Plant in Amarillo, Texas; the Kansas City  National Security Campus in Kansas City, Missouri; and the B61-12  program office in Albuquerque,  New Mexico, where we also interviewed  contractors from Sandia and Los Alamos National Laboratories. We  selected these sites because they are responsible for a range of design  and production activities being undertaken in the program. During these  site visits, we interviewed officials and contractors to discuss their cost  estimating methods, and we viewed classified and unclassified systems  and documents used in compiling the baseline cost report. GAO cost  estimation specialists then assessed this information against the best  practices for cost estimating found in GAO’s cost estimating guide, which  GAO developed to establish a consistent methodology that can be used  across the federal government to develop, manage, and evaluate capital  program cost estimates. We describe these best practices in further  detail in the body of this report.

To examine the reasons for the differences between the B61-12 program  cost estimate for the LEP and the independent cost estimate and how the  differences were reconciled, if at all, we reviewed information on the  program cost estimate, as discussed above. We also reviewed  information on the independent cost estimate and interviewed officials  from NNSA’s Office of Cost Estimating and Program Evaluation who were  responsible for producing the independent cost estimate, to understand  the methods and assumptions they used. In addition, we interviewed  officials from both the program office for the B61-12 LEP and the Office of  Cost Estimating and Program Evaluation to discuss how NNSA evaluated  and used information from the independent cost estimate and any steps  they took to reconcile the two estimates.

To examine the extent to which NNSA and DOD have identified and  managed risks to the program, we reviewed documents that establish  procedures these agencies use to manage program risks, including  NNSA’s Risk and Opportunity Management Plan. During the interviews  and site visits described above, we discussed with officials program risks  and the steps NNSA and DOD are taking to manage them, viewed  information in the program’s classified Active Risk Manager database, and viewed weapon components and facilities to better understand the  items and functions described in the documentation under review. In  addition, we visited the Air Force Nuclear Weapons Center in  Albuquerque to discuss program risks and risk management steps and to  discuss and view information in the Active Risk Manager database. We  also interviewed officials from DOD’s Office of Nuclear Matters and the  NNSA Program Execution Officer for LEPs about their views on the  management of the LEP and any risks involved with the program.

We conducted this performance audit from February 2017 to May 2018 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

This section describes (1) the purpose of LEPs and the process that  NNSA and DOD use to manage them, known as the phase 6.X process;  (2) the management of the ongoing LEP for the W76 warhead—an  important historical reference for the B61-12 LEP—and the status of the  two other ongoing LEPs; (3) future nuclear modernization plans and our  past conclusions and recommendations on the affordability of these  plans; and (4) the objectives of the B61-12 LEP and the roles and  responsibilities of NNSA and the Air Force in conducting the program.

LEPs’ Purpose and the Phase 6.X Process for Managing Them

NNSA and DOD undertake LEPs to refurbish or replace nuclear weapons’  components to extend their lives, enhance their safety and security  characteristics, and consolidate the stockpile into fewer weapon types to  minimize maintenance and testing costs while preserving needed military  capabilities.

NNSA and DOD jointly manage LEPs under a multi-step process known  as the phase 6.X process. The B61-12 LEP is currently in phase 6.4  (production engineering) of this process. Figure 1 illustrates the phase  6.X process.

The phase 6.X process and the roles and functions of DOD, DOE, and  NNSA in nuclear weapon refurbishment activities are described in a  guidance document known as the Procedural Guideline for the Phase 6.X  Process. The document also calls for NNSA to formally update its  program cost estimate and reissue it as the baseline cost report prior to  entering phase 6.4. In January 2017, NNSA issued a supplemental  directive that also directs the Office of Cost Estimating and Program  Evaluation to prepare an independent cost estimate for each nuclear  weapon system undergoing life extension before an LEP enters phase  6.4.

The Procedural Guideline for the Phase 6.X Process also describes the  roles and functions of two joint bodies that provide oversight and approval  functions to LEPs and other nuclear weapons-related activities: the  Nuclear Weapons Council and its Standing and Safety Committee. The  Nuclear Weapons Council is the joint DOD and DOE activity that serves  as the focal point for interagency activities to maintain the nuclear  weapons stockpile. Its membership includes the Under Secretary of  Defense for Acquisition, Technology and Logistics (generally the Chair);  the Under Secretary of Defense for Policy; the Vice Chairman of the Joint  Chiefs of Staff; the Commander of U.S. Strategic Command; and the  Department of Energy’s Under Secretary for Nuclear Security, who also  serves as the Administrator of the National Nuclear Security  Administration. In addition, the Nuclear Weapons Council charters a  Project Officers Group for each weapon system to provide a technical  forum for weapon development and management activities. Each Project  Officers Group is led by a project officer from either the Navy or Air Force,  the two military services that maintain and operate nuclear weapons.

Management of the W76-1 LEP and Other Ongoing LEPs

According to B61-12 program officials, the W76-1 LEP—which NNSA  expects to complete in fiscal year 2019—has served as an important  historical reference as NNSA prepared its plans and cost estimates for  the B61-12 LEP. In August 2017, NNSA issued a study documenting  lessons learned from difficulties it encountered in managing the W76-1  LEP. According to the study, prior to the W76-1 LEP, NNSA had not  undertaken full-scale weapon system design activities since the 1982  design of the W88 warhead. Among other findings, the lessons learned  study stressed the importance of using modern tools to validate and  manage an LEP’s system and technical requirements to maintain cost,  schedule, and performance during all phases of the program. This finding  is consistent with our March 2009 findings that NNSA and DOD  established an unrealistic schedule for the W76-1 LEP, did not establish a  consistent cost baseline, and did not effectively manage technical risks in  the program. These problems resulted in delays, additional  expenditures, and difficulties tracking the cost of the program. Notably,  the program had to delay first production of the W76-1 from September  2007 to September 2008 when it encountered problems with the final test  batch of a key material, known as Fogbank. We recommended that  NNSA develop realistic schedules for the W76-1 and future LEPs that  build in additional time for unexpected technical challenges that may  delay the programs. NNSA agreed with our recommendation and has  taken steps toward improvement in this area, which we continue to  monitor.

In addition to the B61-12 and W76-1 LEPs, NNSA and DOD are  managing two other LEPs: the W88 Alteration 370 program and the W80- 4 LEP. Table 1 provides basic information on all four ongoing LEPs.

Future Nuclear Modernization Plans and Their Affordability

In addition to the four ongoing LEPs, NNSA and DOD have outlined plans  for several future nuclear weapon modernization programs:

Under NNSA’s current program of record, which does not yet reflect  new requirements that will be generated based on the 2018 Nuclear  Posture Review, NNSA has plans for life extension efforts to transition  the nuclear stockpile to three interoperable ballistic missile warheads  and two air-delivered weapons. NNSA has described this plan as  the 3+2 strategy. To undertake this strategy, NNSA has proposed  initiating a series of interoperable warhead programs between about  2020 and 2060. NNSA’s plans for the first ballistic missile warhead in  the 3+2 strategy—the Interoperable Warhead 1—indicate that, if  authorized by Congress, the warhead would cost an estimated $12.4  billion from 2020 to 2041. As we reported in August 2015, NNSA  paused the Interoperable Warhead 1 program in fiscal year 2014 to  provide more time to study the concept of interoperability  and to  reduce uncertainty about the agency’s ability to achieve necessary  plutonium and uranium capabilities to support the LEP. Under its  current program of record, NNSA plans to resume the Interoperable  Warhead 1 program in fiscal year 2019.

Under its current program of record, NNSA has also begun  preliminary planning for Interoperable Warhead 2, Interoperable  Warhead 3, and B61-12 follow-on programs that, if authorized, would  start in the 2020s and 2030s.

In the 2018 Nuclear Posture Review, DOD stated a near-term  intention to modify a small number of existing submarine-launched  ballistic missile warheads to provide a low-yield option, and a long- term intention to pursue a modern nuclear-armed sea-launched cruise  missile. The NNSA Administrator stated in March 2018 that NNSA  would continue to work with DOD to determine the resources, time,  and funding required to address these and other policies specified in  the Nuclear Posture Review.

As we concluded in an April 2017 report, these plans come during a  particularly challenging decade for NNSA’s nuclear modernization efforts,  as the agency plans to simultaneously execute at least four LEPs along  with major construction projects, such as efforts to modernize NNSA’s  uranium and plutonium capabilities. We further concluded that NNSA’s modernization budget estimates for fiscal years 2022 through 2026 may  exceed the funding levels programmed for modernization in future  budgets, raising affordability concerns. Moreover, we concluded that  NNSA had not addressed a projected “bow wave” of future funding  needs—that is, an impending and significant increase in requirements for  additional funds—or the mismatch between potential funding needs and  potential funding available. We recommended that NNSA include an  assessment of the affordability of NNSA’s portfolio of modernization  programs in future versions of the Stockpile Stewardship and  Management Plan—for example, by presenting options NNSA could  consider to bring its estimates of modernization funding needs into  alignment with potential future budgets. NNSA did not explicitly agree or  disagree with our recommendation, but we will continue to monitor any  actions NNSA takes in response to the recommendation.

Objectives of the B61-12 LEP and the Roles and Responsibilities of NNSA and the Air Force

The B61-12 LEP has several objectives: consolidating the nuclear bomb  stockpile, improving the accuracy of the resulting weapon through a new  guidance assembly, and addressing other age-related issues.

Consolidating the stockpile. Under the B61-12 LEP, NNSA and the  Air Force plan to consolidate and replace four of the five variants of  the B61 that were in the active stockpile at the time the B61 LEP  began.

Improving accuracy. The B61-12 is to be equipped with a new tail kit  guidance assembly that enables it to be delivered with greater  accuracy than the B61 bombs it replaces, which are equipped with  parachutes. More specifically, according to Air Force officials and  documents, the assembly will provide the B61-12 with a guided  freefall capability while retaining a ballistic (unguided) delivery  capability. The greater accuracy of the B61-12 is to enable the B61-12  to meet all the military requirements for which past versions of the  B61 were designed.

Addressing other age-related issues. The B61-12 LEP is to extend  the service life of the B61 by at least 20 years, make field  maintenance of the weapon easier for Air Force technicians, and  provide modern security features.

NNSA manages its B61-12 LEP activities through a federal program  office on Kirtland Air Force Base in Albuquerque, New Mexico, under the  direction of the federal program manager. It manages the work of six  government-owned, contractor-operated NNSA laboratories and sites that  serve as design and production agencies for the LEP. Sandia National  Laboratories, also located on Kirtland Air Force Base, serves as the  systems-level integrator for the overall weapon design. Figure 2 shows  the six sites participating in the B61-12 LEP and their respective roles.

The Air Force’s responsibilities, in addition to managing the acquisition of  the tail kit guidance assembly, include integrating the B61-12 with its  delivery aircraft and the operational flight program software of these  aircraft. The Air Force Nuclear Weapons Center, also at Kirtland Air Force  Base and under the direction of the Air Force lead project officer,  manages technical integration and other LEP-related tasks required to  qualify, certify, and field the weapon.

The delivery aircraft that carry the B61-12 are being designed to deliver  the weapon in two different modes with two different systems, the second  of which provides the enhanced capabilities offered by the new tail kit  guidance assembly. System 1 aircraft will have an analog interface with  the B61-12 that is designed to deliver the weapon in a ballistic mode, with  the tail kit in a fixed position. System 2 aircraft will have a digital interface  with the B61-12, enabling the guided delivery capability afforded by the  tail kit assembly. Figure 3 illustrates the delivery aircraft for the B61-12.

NNSA Substantially Incorporated Best Practices in Developing the B61- 12 Program Cost Estimate

NNSA substantially incorporated most of the cost estimating best  practices identified by our past work when it developed the $7.6 billion  program cost estimate for the B61-12 LEP. Our cost estimating guide  identifies best practices for developing a high-quality, reliable cost  estimate and identifies four characteristics of such an estimate: it is  comprehensive, well-documented, accurate, and credible. These four  characteristics and some of the best practices that underlie them are  illustrated in figure 4.

We assessed the B61-12 program cost estimate by comparing it with the  best practices identified in our cost estimating guide and found that it  substantially met the criteria for all four characteristics of a high-quality,  reliable cost estimate (see fig. 5).

A summary of our assessment is presented below, including reasons that  the program cost estimate substantially met the criteria under each of the  four characteristics as well as some examples of the best practices that  the cost estimate could have more fully incorporated. Appendix  I provides  additional information on our assessment.

Comprehensive: Substantially Met. The program established a  consistent and clearly defined work breakdown structure—a  hierarchical structure that subdivides the work necessary to  accomplish the program’s objectives into smaller elements—to ensure  that costs were not double-counted or omitted. The clearly defined  work breakdown structure also helped the B61-12 program office  manage the process of integrating each site contractor’s estimate for  the cost of its activities into the overall program estimate. To more  fully incorporate the criteria for a comprehensive cost estimate, the  program would have had to take additional steps, such as including  the full life-cycle costs associated with the B61-12 weapon in the  estimate. Specifically, the estimate would have had to include costs  such as program costs incurred prior to phase 6.3, the cost of NNSA  federal program office personnel, components that are being shared  by different nuclear weapon programs (such as the weapon’s radar),  and costs associated with maintenance of the B61-12 after the LEP  ends and the weapon enters the stockpile. In addition, the estimate  would have had to include an assessment of how the program would  be affected if key assumptions, such as the timing of the delivery of  the tail kit guidance assembly, did not hold true. Nevertheless, the  program incorporated practices that substantially met the criteria for a  comprehensive cost estimate, which we believe contributed to the  program’s estimate being reliable.

Well-documented: Substantially met. In our visits to NNSA sites  and our associated review of site contractors’ documents, we found  that site contractors provided detailed documentation of their  contributions to the cost estimate to the B61-12 program office. At all  of the sites we visited, experienced cost estimating teams captured  specific information on the data and data sources used to inform their  estimates. To more fully incorporate the criteria for a well-documented  cost estimate, the documentation that the site contractors provided to  the NNSA program office would have had to capture the reliability of  the underlying data and discuss how the data were normalized.  Nevertheless, the program incorporated practices that substantially  met the criteria for a well-documented cost estimate, which we believe  contributed to the program’s estimate being reliable.

Accurate: Substantially met. Technical personnel at both the NNSA  sites and the Albuquerque  federal program office discussed program  risks to ensure that the program estimate represented a most likely,  unbiased cost. Furthermore, all of the site-level cost estimates we  examined—which the federal program office integrates into the overall  program cost estimate—drew on historical data from primary sources,  including internal financial systems from either past B61 costs or  previous LEPs. Use of such sources is consistent with the best  practice of grounding the estimates in a historical record of cost  estimating and actual experiences on other comparable programs. In  addition, the federal program office routinely reviews contract  performance reports from each of the B61-12 sites to track variances  between estimated and actual costs on a monthly basis. To more fully  incorporate the criteria for an accurate cost estimate, the program  would have had to use site estimates that were calculated in base- year dollars and then uniformly adjusted for inflation at the program  level, and clearly defined the method it used to determine inflation  indexes. Instead, all of the site contractors developed their cost  estimates in then-year dollars and applied varied inflation indexes.  Nevertheless, the program incorporated practices that substantially  met the criteria for an accurate cost estimate, which we believe  contributed to the program’s estimate being reliable.

Credible: Substantially met. The B61-12 LEP became the first LEP  to undergo a statutorily required independent cost estimate,  conducted by the Office of Cost Estimating and Program Evaluation.  Additionally,  a different NNSA office developed a third cost estimate  for the program to aid in the preparation of NNSA’s budget materials.  Each of these three estimates used a different methodology. NNSA  used this third estimate to cross-check overall program costs.  Moreover, to assess risk and uncertainty in the program, most of the  site estimates we reviewed included a detailed, quantifiable risk  assessment for their portion of the overall program estimate. To more  fully incorporate the criteria for a credible cost estimate, the program’s  sensitivity analysis would have had to more fully examine and  document cost impacts for the overall estimate and the individual site  estimates. Instead, according to NNSA officials, it focused primarily  on schedule and critical path analysis. Moreover, to more fully  incorporate the criteria for a credible cost estimate, the program would  have had to address risk correlation and the calculation of confidence  levels differently. In the program’s analysis of risks and uncertainties  in the program, we found the program inconsistently examined  correlation among program risks. Specifically, according to NNSA  officials, to arrive at the 70 percent confidence level for the overall  program cost estimate, the program office added site-level cost  estimates together at the 50 percent and 70 percent confidence  levels. As noted in our cost guide, adding risk results for the  underlying estimates in this way results in an incorrect confidence  level for the overall estimate. Nevertheless, the program  incorporated practices that substantially met the criteria for a credible  cost estimate, which we believe contributed to the program’s estimate  being reliable.

We consider a cost estimate to be reliable if the overall assessment  ratings for each of the four characteristics are substantially or fully met—as was the case with the B61-12 program cost estimate, which  substantially met these criteria. For that reason, we are not making  recommendations related to the program’s use of cost estimating best  practices. However, by fully incorporating all of the best practices for the  four characteristics, NNSA can better ensure that its future cost estimates  are of high quality and reliable.

Varying Methods and Assumptions about Future Program Performance Led to Differences between the Estimates, but NNSA Did Not Document the Rationale for Adopting the Program’s Estimate Unchanged

The $7.6 billion program cost estimate for the B61-12 LEP differs from the  $10 billion independent cost estimate primarily because the program  office used different methods and assumptions than the Office of Cost  Estimating and Program Evaluation, which prepared the independent cost  estimate. The B61-12 program developed its estimate by compiling site- specific cost and schedule estimates for activities at each of the NNSA  sites participating in the LEP; in contrast, the independent cost estimate  projected a cost and completion date by evaluating program activities  completed to date and applying a historical model to estimate costs and  durations for remaining activities. As noted in our cost guide, both of  these methods are commonly applied. To reconcile the differences  between the two estimates, high-ranking NNSA officials met with officials  from the B61-12 program office and the Office of Cost Estimating and  Program Evaluation to discuss the estimates in 2016. However, NNSA  did not document the rationale for its decision to use the program office’s  lower estimate unchanged or a plan for how it would take the independent  cost estimate into consideration. We previously recommended that NNSA  should establish a requirement for its management to document and  justify key decisions based on a reconciliation of LEP cost estimates with  the Office of Cost Estimating and Program Evaluation’s independent cost  estimates. NNSA agreed with this recommendation.

The B61-12 Program Office and the Office That Developed the Independent Cost Estimate Used Different Estimating Methods and Assumptions

Cost estimating best practices specify that programs should develop a  point estimate—the best guess at the program’s cost estimate, given the  underlying data—by collecting, analyzing, and validating program data  and then using one of several commonly used methods for estimating the  program’s cost. Once a program has developed a point estimate, the  program should compare it to an independent cost estimate, which gives  an objective measure of whether the program’s point estimate is  reasonable. In January 2017, NNSA issued two directives implementing  statutory requirements for the Office of Cost Estimating and Program  Evaluation to develop independent cost estimates for NNSA programs,  including LEPs.

The differences between the respective cost estimating methods, both of  which are valid, used by the B61-12 program office and the Office of Cost  Estimating and Program Evaluation are the primary reason for the  differences between the program estimate and the independent cost  estimate. According to B61-12 program officials, the program generally  developed its point estimate by using a “bottom-up” method formally  known as the “engineering build-up” cost estimating method. In using this  method, a program subdivides the work necessary to accomplish its  objectives into a work breakdown structure. The program then develops  estimates of costs at the lowest level of the work breakdown structure,  one piece at a time, and uses the sum of the pieces to form the overall  estimate. To develop its cost estimate, the B61-12 program office  required all participating NNSA site contractors to prepare and submit  their own cost estimates for the work to be performed on the LEP and  provided instructions on what data to provide to the program office. For  example, these instructions specified that all sites must apply a bottom-up  estimating approach that includes detailed quantities and integrated  resource-loaded schedules for all work breakdown structure elements  under their management. The program office then compiled the site- provided information in a database to arrive at a total program cost. The  program office also aggregated schedule information from the sites,  which maintain detailed resource-loaded integrated site schedules, to  develop an NNSA Integrated Master Schedule. As we previously noted,  the program office estimated in October 2016, based on this process, that  NNSA’s portion of work on the B61-12 LEP would cost $7.6 billion and  that the LEP would be completed in fiscal year 2025, with a first  production unit date of March 2020.

In contrast, to develop the independent cost estimate, the Office of Cost  Estimating and Program Evaluation used an estimating method that  employed data on the B61-12 LEP’s actual performance, coupled with  historical information from the W76-1 LEP for stages of the phase 6.X  process that the B61-12 LEP had not yet reached. Specifically, the office  gathered data on 1,600 activities in the NNSA Integrated Master  Schedule for the LEP. The office tracked these 1,600 activities from  August 2014 through March 2016 by evaluating data from successive  versions of the NNSA Integrated  Master Schedule, which the B61-12  program office updates monthly based on actual program performance to  date. In a memo summarizing the office’s independent cost estimate, the  office stated that the program’s task completion rate lagged the baseline plan. The office concluded that the LEP’s first production unit date would  occur 2 years after the March 2020 target date unless the program took  measures to reduce the LEP’s scope by removing tasks, delaying  activities until after the first production unit date, or relaxing requirements  to accommodate less mature components than originally planned. The  office also concluded, based on the program’s spending rate of  approximately $45 million per month, that pre-first production unit costs  would increase by about $1 billion over the program’s estimate. To  estimate the cost and schedule of the program after the first production  unit date, the office used both B61-12 LEP actuals and historical  information from the W76-1 LEP, comparing W76-1 funding levels to B61- 12 spending levels. On the basis of its analysis, the office concluded that  full-scale production of the B61-12 would cost approximately $1 billion  more than the program office estimated. All told, the independent cost  estimate projected that the B61-12 LEP would cost approximately $10.0  billion and take about 2 years longer—with a projected completion date in  fiscal year 2027—barring changes to the program’s scope.

The B61-12 program office and the Office of Cost Estimating and  Program Evaluation also have differences of opinion regarding the  continued validity of the August 2014 schedule performance data and its  relevance to the independent cost estimate. According to B61-12 program  officials, the information in the NNSA Integrated Master Schedule  improved and changed after the Office of Cost Estimating and Program  Evaluation gathered initial schedule performance data in August 2014  and used this information as a starting point to evaluate the program’s  performance. These issues include the following:

The officials described the data available to the Office of Cost  Estimating and Program Evaluation in August 2014 as tentative,  saying that the program can now use the NNSA Integrated Master  Schedule to track performance at a more detailed level.

According to B61-12 program officials, the program made important  decisions that affected components on the program schedule’s critical  path at the time the August 2014 schedule performance data were  gathered. Subsequent to establishing the baseline, for example, the  program office restructured the path to first production unit for high- explosives components, correcting errors that had been captured in  the August 2014 data and changing to a more streamlined approach  to qualify high-explosives components from legacy material. This  decision affected the program’s critical path to first production unit,  moving the completion date earlier.

The program undertook schedule recovery efforts that eased  schedule constraints affecting other program elements that were on  the critical path at the time of the August 2014 data.

As a result of these factors, B61-12 program officials said that the entire  baseline schedule that the Office of Cost Estimating and Program  Evaluation analyzed appeared more problematic than the updated  schedule and that the entirety of the independent cost estimate was  thrown off by the obsolete August 2014 data.

Officials from the Office of Cost Estimating and Program Evaluation told  us they disagree with the B61-12 program office’s assessment of the  independent cost estimate schedule analysis. These officials said that  they understand that the schedule baseline is continuously changing but  that the independent cost estimate schedule analysis is not dependent on  a particular baseline. Rather, they said that the analysis is based on  actual schedule performance for the 1,600 activities that represent the  scope required to achieve the design maturity and that the program office  specified in August 2014 as needed to reach the first production unit  (phase 6.5) milestone.

Officials from the Office of Cost Estimating and Program Evaluation said  that although they will not formally assess the B61-12 LEP’s schedule  again until the end of phase 6.4 of the program, their informal analysis of  NNSA Integrated Master Schedule data as of February 2017 still showed  the same rate of activity completion that underpinned the office’s  independent cost estimate. At some point, according to these officials, the  program will have to double or even triple its rate of activity completion to  finish the LEP on schedule, which will increase cost. In contrast, B61-12  program officials stated that, given the improved quality of the program’s  integrated master schedule data, they expect that the independent cost  estimate that the Office of Cost Estimating and Program Evaluation  prepares at the end of phase 6.4 of the LEP will be closer to the  program’s estimate than to the October 2016 independent cost estimate.

Program officials also said that the program’s performance to date  supports their position that the program cost estimate is accurate.

The positions of the two offices also differ regarding the B61-12  program’s ability to accelerate work in the production stages of the LEP to  ensure that the LEP meets its completion date. B61-12 program officials  stated that they have options other than to complete tasks sequentially  and at a steady rate, so they do not expect the “straight-line” level of  productivity assumed in the independent cost estimate analysis to occur.  For example, some parts do not have to be built in a particular sequence.  Instead, program officials said, the production agencies can build different  lots of components when they are ready, so technologies that are ready  earlier than others can be moved to production in the war reserve lot  while other components remain in earlier stages. They also said that the  program would not maintain an even spending rate of $45 million per  month, as suggested in the independent cost estimate. Rather, they  noted, the program’s spending rate is currently $55 million a month, and  the program plans for it to rise to $65 million per month as the current  production engineering phase of the LEP draws to a close and the  production agencies accelerate their activities. These factors  notwithstanding, one Office of Cost Estimating and Program Evaluation  official observed that activities in the later stages of an LEP remain  complex and carry risks. The official cited the history of the challenges  that led to the delay of first production of the W76-1, cited earlier in this  report, and said that the B61-12 program faces the added challenges of  having to integrate with several delivery aircraft and of having more  electronic components than the W76-1. As noted in our cost estimating guide, studies have shown limited opportunity for getting a delayed  program back on track after it is more than 15 percent to 20 percent  complete.

NNSA Officials Met to Reconcile the Program’s Cost Estimate and the Independent Cost Estimate but Did Not Document the Rationale for Adopting the Program’s Estimate Unchanged

Cost estimating best practices specify that a program cost estimate and  an independent cost estimate should be reconciled and that differences  between them should be examined and discussed to achieve  understanding of overall program risk. Officials from NNSA, including  from the Office of Cost Estimating and Program Evaluation and the B61- 12 program office, told us that they held several discussions in 2016  regarding the differences between the program estimate and the  independent cost estimate. These included a meeting with the second- highest ranking official in NNSA—the principal deputy administrator— during which the respective offices presented their estimates and  explained the methods used to produce them. After these meetings, the  principal deputy administrator and the NNSA Administrator agreed to  approve the program estimate unchanged.

According to B61-12 program officials, the program adapted some of its  practices as a result of their interactions with the Office of Cost Estimating  and Program Evaluation. For example, officials said that they changed  the program’s procedure for baseline changes to ensure consistency  across the participating sites. The program also began to conduct  baseline execution index analyses, as the Office of Cost Estimating and  Program Evaluation recommended. Baseline execution index analyses  track a program’s execution of tasks to date by monitoring the percentage  of activities that a program has completed early or on time and that have  a baseline for completion within the month the analysis is conducted.  According to program officials, similar to a schedule performance index in  an earned value management system, the baseline execution index gives  an alternate cumulative measure that gives a program an opportunity to  improve as it proceeds.

However, B61-12 program officials said that they did not document the  rationale for adopting the program cost estimate without making changes  informed by the independent cost estimate. They told us that any attempt  to combine the results of the two estimates would have been difficult,  considering the significant differences between the program’s cost  estimating model and the Office of Cost Estimating and Program  Evaluation’s model. We recommended in a January 2018 report that  NNSA should establish a requirement for its management to document  and justify key decisions based on a reconciliation of LEP cost estimates  with the Office of Cost Estimating and Program Evaluation’s independent  cost estimates. We concluded in the report that without a requirement  for its management to document and justify key decisions based on a  reconciliation of program cost estimates with the Office of Cost Estimating  and Program Evaluation’s independent cost estimates, NNSA may not  have assurance that the independent cost estimates are being  appropriately incorporated into the LEP decision-making process,  potentially decreasing the reliability  of program cost estimates. Our prior  work has shown that, in general, because the independent cost estimate  team is outside the acquisition chain, is not associated with the program,  and has nothing at stake with regard to program outcome or funding  decisions, its estimate is usually considered more accurate than the  program’s internal estimate. In addition, our prior work has shown that  independent cost estimates are historically higher than program office  cost estimates because the team conducting the independent cost  estimate is more objective and less prone to accept optimistic  assumptions. However, we have also found that because independent  cost estimates are typically higher than program office cost estimates, in  some cases management may choose to ignore them because the  estimates are too high. NNSA agreed with our January 2018  recommendation, stating that by March 2018, it would establish a protocol  to document management decisions regarding significant variances  between LEP cost estimates and the independent cost estimates  produced by the Office of Cost Estimating and Program Evaluation.  However, NNSA has not provided evidence that it has done so. We  continue to believe that documenting key decisions regarding cost  estimates is particularly important in the context of LEPs, where decisions  could increase a program’s costs by billions of dollars.

NNSA and DOD Have Identified Risks and Are Taking Steps to Manage Them

NNSA and DOD have identified and are managing various risks that  could complicate efforts to meet the fiscal year 2025 completion date for  the B61-12 LEP. Some of these risks that the agencies are managing are  within the program’s areas of responsibility, such as an aggressive flight  test schedule, and additional risks could be identified within these areas.  To manage risks, the program uses a formal risk management process  and has taken steps such as consolidating flight tests and holding more  regular meetings between NNSA’s design and production agencies. The  program also faces risks that program officials told us lie outside the  program’s direct control—such as risks related to the F-35 delivery  aircraft, nuclear certification, and NATO coordination issues—and officials  said they have provided information to the responsible DOD organizations  to help address these risks.

NNSA and DOD Have Taken Steps to Identify and Help Manage Risks within the Program’s Responsibility, and They May Identify Additional Risks

NNSA and DOD have taken steps to identify and help manage risks  within the B61-12 LEP’s responsibility, and program officials said that  they may identify additional risks as the program progresses. More  specifically, the program has a formal risk management process through  which it has identified risks and could identify additional risks as the  program proceeds, according to agency officials. Risks already identified  and being managed include risks related to the program’s aggressive  schedule of flight tests and to finalizing design and coordinating  procurement and delivery of components.

The B61-12 LEP has a formal risk management process that has  identified joint NNSA and Air Force risks within the program’s areas of  responsibility that could significantly impact the overall program’s  schedule, its cost, or the technical performance of the weapon. According  to program officials and the Program Joint Risk Management Plan, this  process calls for each program element in NNSA or the Air Force to be  responsible for identifying and managing risks at the lowest level  possible. After the program element reviews and documents a risk, it then  reviews the risk to determine its applicability to be considered a joint  risk—that is, a risk that has the potential to affect any of the top-level  program milestones or the program’s ability to successfully meet system  performance requirements. Program officials told us that the Air Force  lead project officer decides whether to accept the risk into the joint risk  list. Senior management oversees those risks through a formal  management plan.

The process includes continual reviews to identify new risks that may  emerge. The Joint Risk Review Board meets as new potential risks are  identified to review their likelihood and consequence. Officials from both  the Office of Cost Estimating and Program Evaluation and DOD’s Office  of Nuclear Matters told us that during phase 6.4 and thereafter, the  program may still discover new risks—”unknown unknowns”—during  technical tests to qualify components and the development of production  processes.

The process also has steps to manage risks and remove them from the  joint high-risk list, if the Joint Risk Management Board judges them to  have been resolved to closure or a low-risk status, according to program  officials. NNSA and DOD program officials said that the program’s risk  management process has resulted in the resolution of about three- quarters of the identified high risks on the joint risk list. They also  observed that the program’s Selected Acquisition Reports, through which  NNSA and the Air Force report to the congressional defense committees  on the program’s cost and schedule, have been unchanged since 2013  regarding major program milestones. Program officials said that to  provide a 90-day schedule buffer and add flexibility to the program’s  schedule in the event of unexpected difficulties, the program has planned  to reach phase 6.5 in December 2019, ahead of the phase 6.5 date of  March 2020 that is reported in the Selected Acquisition Reports.  However, other officials told us that it is too soon to say whether the  program can manage the identified risks, or other unidentified risks, to  prevent delays in a program that has relatively little schedule margin.  Problems can emerge even during the first production stage of an LEP,  as happened in the W76-1 LEP due to the Fogbank production  challenges we discuss earlier in this report and in our March 2009 review  of W76 and B61 modernization efforts.

Managing Risks Related to the Program’s Aggressive Schedule of Flight Tests

DOD and NNSA officials we interviewed generally agreed that the  program faces risks in completing an aggressive flight testing schedule to  support the first production unit deadline. According to the officials, the  B61-12 program needs to complete more than 60 flight tests over a 3- year period to meet this deadline. Completing the tests entails actively  coordinating with the Air Force organizations that manage the various  aircraft that will carry the B61-12 weapon: the B-2 bomber and the F-15,  F-16, F-35, and PA-200 fighters. According to B61-12 program officials,  aircraft may not be available when needed for the planned flight testing.  This risk is of particular concern for B-2 bombers, they said, because only  one B-2 test unit is available and it is in heavy demand for other Air Force  purposes. Program officials characterized the flight test schedule as  aggressive and ambitious, but feasible, and told us the program has  managed the risks caused by the tight testing timeframes by coordinating  with the responsible organizations and consolidating tests to minimize the  amount of time required on each type of aircraft. Further, when aircraft  are not available as planned, program officials said they can revise the  sequence of tests. For instance, they accommodated the unavailability  of  a B-2 test asset on a planned test date by moving up a test date on the F- 16. This schedule adjustment avoided a ripple effect of delays on the  overall testing schedule, according to Air Force officials. The video in  figure 6 shows an F-16 dropping an inert B61-12 bomb during a flight test  on March 14, 2017.

NNSA and DOD have identified and taken steps to manage risks related  to finalizing the weapon’s design and coordinating the procurement and  delivery of components. These risks include:

Technical risks associated with the design and production of  various components. Officials told us some components of the  bomb and tail kit assembly are on the program’s list of joint risks.  They said that their use of the joint risk management process calls  management attention to potentially  serious risks and helps the  program to manage these risks as early and as continually as  possible. For example, NNSA officials said that when technical risks  arose in designing one classified component on the program’s critical  path—potentially affecting the design schedule—they augmented the  design team with additional scientists in an effort to ensure that the  component would be completed in time to support the production  schedule. Similarly, to manage design risks related to the exacting  specifications for certain components, Kansas City National Security  Campus is working to develop sufficiently precise gages to measure  the required specifications during production.

Late design changes from design agencies provided to the  production agencies. NNSA’s Fiscal Year 2018 Stockpile  Stewardship and Management Plan identified late changes to  component design as a risk facing the B61-12 program and other  LEPs. Contractor officials we interviewed from the Kansas City  National Security Campus and the Pantex Plant said that late  changes to weapon design requirements from the Sandia and Los  Alamos design agencies could create schedule problems for  establishing production processes at the production sites. Kansas City  National Security Campus officials expressed concerns that some  component requirements continue to change—some arising from  testing results—which creates a tension between improving the  design and stabilizing production requirements and processes. Pantex  officials also told us about a potentially significant production delay if  late design changes require Pantex to get new production tools or  testers. Late design changes could occur as scientists at the design  agencies analyze test results. Flight tests, for example, produce a  volume of information. Officials at the federal program office in  Albuquerque said that 4 test flights on the B-2, conducted in July  2017, produced 4 to 6 hours’ worth of data per flight. Officials at both  the Pantex and Kansas City sites said they have developed  management strategies to provide some flexibility in their production  schedules, such as speeding production by having staff work longer  shifts. Moreover,  because of lessons learned from prior LEPs, officials  at both sites told us that coordination between production sites and  design agencies has significantly improved over past practices— specifically, by having ongoing engagement that started earlier in the  weapon development process. For instance, Pantex officials told us  that they hold monthly meetings with design agencies to discuss  design changes. A Pantex official told us that, as a result of  addressing production concerns early, design requirements  developed at the design agencies are less likely to result in  unanticipated production problems.

Vendor risks associated with procuring various bomb  components. According to NNSA officials we interviewed, some  bomb components are procured through single commercial vendors,  in small lots, or are unusual. Kansas City National Security Campus  officials told us that they had to replace one vendor that could no  longer provide a certain material and that they generally risk losing  potential or existing vendors because vendors prefer contracts for  larger volumes of components than NNSA needs for the B61-12  bomb. In addition, unique materials for the bomb include certain  components with specific compositions of rubber and plastics.  Officials at the Kansas City National Security Campus said that they  have encountered difficulties with getting rubber and plastic  components from vendors that consistently meet composition  specifications or with sustaining vendors’ interest in producing small  batches of precision-manufactured components. In one such instance,  they said they improved incentives and communication with a vendor  to avoid losing a source for a key component. The officials said they  also contract with smaller vendors when larger vendors may not be  interested in the size of the contract NNSA offers.

Delays in delivery of components from other production sites to  Pantex for full bomb assembly. NNSA and Pantex officials told us  that Pantex, which will assemble the full B61-12 bomb, depends on  the other production sites delivering the components in a timely  manner. NNSA production sites are scheduled to provide components  to Pantex 120 days before the first production unit date. Pantex and  NNSA officials have identified some schedule flexibility  for assembling  the first production unit at Pantex, depending on which components  have delayed deliveries. Specifically, if the delayed components are  those needed later in the assembly process, such as the bomb’s nose  assembly, Pantex could stay on schedule by assembling other  delivered components until the delayed components are needed.  Delivery of other components, such as detonators, is more time- sensitive, and it is essential that these be delivered on time for  assembly to proceed as planned, according to Pantex officials.

NNSA and DOD Have Taken Steps to Identify and Help Manage Risks outside the Program’s Direct Control

According to program officials, certain risks that may have a bearing on  the B61-12 LEP or that may affect the fielding of the weapon lie in areas  outside the program’s direct control. Nevertheless, program managers  have taken steps to coordinate with other responsible parties to help  address these risks. For example, two of the three delivery aircraft  designated as system 2 aircraft—the F-35 and the B-2—have not yet  completed development and procurement of operational flight program  software that will enable the aircraft to deliver the B61-12 with the  enhanced accuracy offered by the tail kit assembly, a key feature of the  LEP. B61-12 program officials told us that the program offices  responsible for each of these aircraft must manage the development and  procurement of the operational flight program software. To help inform the  software development process, the B61-12 program provided the F-35  and B-2 program offices with information about the weapon’s interface  with the airplane, including information from flight tests performed on an  earlier version of the F-35, according to program officials. NNSA and  DOD officials characterized B-2 development related to the B61-12 as  significantly more advanced than F-35 development. Specifically, Air  Force officials said that a developmental version of the B-2 operational  flight program software was fielded and certified in 2017 and would  undergo final weapon system demonstration flight tests in October 2019  and nuclear design certification in June 2020. By contrast, they said that  the F-35 software will not be ready for nuclear design certification until  January 2023, after the B61-12 program’s first production unit date. The  F-35 program office will be responsible for funding tests and aircraft- weapon integration activities, according to the Air Force officials. Because  of the need to defer some flight tests until the software for the B-2 and F- 35 aircraft is ready, only one of the three system 2 delivery aircraft has  undergone testing of the B61-12 bomb’s capabilities in its system 2  setting: the F-15E, on which NNSA and DOD conducted the first system 2  tests of the B61-12 in August 2017.

Other risks outside the program’s direct control concern nuclear  certification and the NATO mission. Nuclear certification—ensuring that  people and objects that come into contact with the weapon will not  adversely affect its performance characteristics—is a prerequisite to  fielding the B61-12 and other nuclear weapons, but it is the responsibility  of the Air Force organizations that manage the delivery aircraft. In a  classified report issued in January 2018, we discuss risks related to  nuclear certification of dual capable aircraft, which are able to deliver  conventional munitions or nuclear bombs. B61-12 program officials told  us that they are working to address these risks by providing information  on the weapon to all of the organizations that manage the delivery  aircraft. Similarly, in another classified report issued in February 2018, we  discuss a risk related to the NATO mission that may affect the B61-12  LEP; program officials told us that they are working to address this risk,  as well. We made recommendations in the two classified reports related  to these risks; the responsible agencies agreed with our  recommendations and stated their intention to take action in response to  them.

We provided a draft of this product to NNSA and DOD for comment.  NNSA provided technical comments, which we incorporated as  appropriate. DOD indicated that it did not have any comments.

We are sending copies of this report to the appropriate congressional  committees, the Secretaries of Defense and Energy, the Administrator of  NNSA, and other interested parties. In addition, this report is available at  no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please  contact me at (202) 512-3841 or bawdena@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. GAO staff who made significant  contributions to the report are listed in appendix II.

Appendix I: Results of GAO’s Assessment of the B61-12 Life Extension Program Cost Estimate Compared with Best Practices

Best practice  The cost estimate includes all life cycle costs.  The cost estimate completely defines the program,  reflects the current schedule, and is technically  reasonable.  The cost estimate work breakdown structure—a  hierarchical structure that subdivides the work necessary  to accomplish the program’s objectives into smaller  elements—is product-oriented, traceable to the  statement of work/objective, and at an appropriate level  of detail to ensure that cost elements are neither omitted  nor double-counted.  The estimate documents all cost-influencing ground rules  and assumptions.   The documentation should capture the source data used,  the reliability of the data, and how the data were  normalized.  The documentation describes in sufficient detail the  calculations performed and the estimating methodology  used to derive each element’s cost.  The documentation describes step by step how the  estimate was developed so that a cost analyst unfamiliar  with the program could understand what was done and  replicate it.  The documentation discusses the technical baseline  description, and the data in the baseline is consistent  with the estimate.  The documentation provides evidence that the cost  estimate was reviewed and accepted by management.  The cost estimate results are unbiased, not overly  conservative or optimistic, and based on an assessment  of most likely costs.  The estimate has been adjusted properly for inflation.

The estimate contains few, if any,  minor mistakes.  The cost estimate is regularly updated to reflect  significant changes in the program so that it always  reflects current status.  Variances between planned and actual costs are  documented, explained, and reviewed.   The estimate is based on a historical record of cost  estimating and actual experiences from other  comparable programs.

Best practice  The cost estimate includes a sensitivity analysis that  identifies a range of possible costs based on varying  major assumptions, parameters, and data inputs.  A risk and uncertainty analysis was conducted that  quantified the imperfectly understood risks and identified  the effects of changing key cost driver assumptions and  factors.  Major cost elements were cross checked to see whether  results were similar.  An independent cost estimate was conducted by a group  outside the acquiring organization to determine whether  other estimating methods produce similar results.

Appendix II: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Jonathan Gill (Assistant Director),  Rob Grace (Analyst in Charge), Terry Hanford (Senior Analyst), and  Jennifer Leotta (Senior Operations Research Analyst) made key  contributions to this report. Also contributing to this report were Antoinette  C. Capaccio, Scott Fletcher, Penney Harwell Caramia, Cynthia Norris, Karen Richey, and Sara Sullivan.

The National Nuclear Security Administration undertakes life extension programs to refurbish/replace aging components of nuclear weapons. NNSA recently estimated that the program for the B61-12 nuclear bomb would cost about $7.6 billion—but an independent estimate put the cost at $10 billion.

We found that NNSA incorporated best practices for cost estimates for the program, but also found room for improvement—such as documenting management decisions on the differences between these two cost estimates.