Nuclear Nonproliferation: NNSA Needs to Improve Its Program Management Policy and Practices

The threat posed by the proliferation of nuclear and radiological weapons remains a pressing national security challenge. DNN implements nuclear nonproliferation programs worldwide. To carry out its mission, for fiscal year 2018 DNN requested an appropriation of about $1.5 billion for its 4 major programs and their 13 subprograms.

A House Armed Services Committee report, accompanying a bill for the National Defense Authorization Act for Fiscal Year 2017, included a provision for GAO to review and assess DNN's project and program management processes and systems. GAO's report examines the extent to which (1) selected DNN subprograms use program management leading practices to manage schedule and cost (2) DNN has incorporated leading practices in its revised program management policy.

GAO selected 4 DNN subprograms to review that had defined end dates and/or work scope and that GAO had not recently examined. GAO reviewed documentation on DNN and NNSA's program management policies and practices; reviewed selected leading practices published by PMI and GAO; and interviewed agency officials.

The 4 selected subprograms from the National Nuclear Security Administration's (NNSA) Office of Defense Nuclear Nonproliferation (DNN) GAO reviewed generally do not use selected program management leading practices to manage schedule and cost. According to generally recognized leading practices from the Project Management Institute (PMI) and GAO, programs should (1) establish schedules necessary to achieve the program's goal, (2) establish life-cycle cost estimates, and (3) measure performance against schedule and cost baselines. However, none of the DNN subprograms have schedule and cost estimates covering their planned life cycles and none measure performance against schedule and cost baselines. The following figure illustrates the extent to which the selected subprograms have established schedule and cost estimates compared to their planned life cycles.

NNSA officials said that the subprograms do not have schedules and cost estimates that cover their life cycles and do not measure performance against baselines, in part, because DNN management does not require such estimates or baseline measurements.

The lack of a requirement is consistent with the limitations in DNN's revised program management policy, which does not address leading practices on establishing schedule estimates, estimating life-cycle costs, and measuring against such baselines. According to leading practices, in developing schedule and cost estimates a program should define assumptions tailored to the program such as its life-cycle phases. Updating the DNN policy to include requirements and guidance on cost estimating and tracking performance against schedule and cost baselines could help ensure that NNSA managers and Congress have better information on how much DNN programs and subprograms may cost, the time they may need to achieve their goals, and how effectively they are being executed compared to plans.

GAO recommends that DNN revise its program management policy to require DNN programs and subprograms to follow life-cycle program management, such as requiring life-cycle estimates and measuring against baselines. NNSA neither agreed nor disagreed with the recommendation but plans to take action to revise its policy.

The threat of nuclear and radiological proliferation, including the concern  that non-state actors or additional countries could obtain nuclear or  radiological weapons, poses one of the greatest challenges to U.S. and  international security. To address this threat, the National Nuclear  Security Administration (NNSA)—a separately organized agency within  the Department of Energy (DOE)—implements nuclear nonproliferation  programs worldwide under its Office of Defense Nuclear Nonproliferation  (DNN). DNN’s mission is to develop and implement policy and technical  solutions to eliminate nuclear and radiological materials, enhance the  security of nuclear material in place and during transport, and limit or  prevent the spread of these materials, technology, and expertise related  to nuclear and radiological weapons and programs around the world. To  support this mission, for fiscal year 2018, DNN requested an  appropriation of approximately $1.5 billion for its 4 major programs, which  organize work under 13 subprograms, as shown in figure 1.

In recent years, we have reported on program management challenges in  DOE and NNSA, including those within DNN. For example, in November  2014, we found that DOE and NNSA programs were not required to meet  any cost-estimating best practices, such as developing a life-cycle cost  estimate. We recommended, among other things, that DOE revise its  departmental directives that apply to programs to require that DOE,  NNSA, and its contractors develop cost estimates in accordance with best  practices. We also found in November 2016 that DOE had not  established a department-wide policy addressing internal control  standards or leading practices related to program management, and we  recommended that DOE develop such a policy. DOE is in the process of  taking actions to address these recommendations or has plans to do so.  At the time of our November 2016 review, we also learned that DNN did  not have a program management policy in effect because its 2005 policy  was outdated and fell out of use around 2010.

In February 2017, DNN approved a revised program management  policy. The revised policy outlines program management processes,  roles, and responsibilities for DNN programs and subprograms. The  policy requires that program management functions be conducted over  the next 5 fiscal years, referred to as the Future Years Nuclear Security  Program (FYNSP). In addition, the policy’s objectives include  establishing a DNN-wide policy that incorporates leading practices for  program management and that facilitates the implementation of methods  for programs and subprograms to monitor, measure, analyze, and  improve management processes.

Program management involves aligning multiple components to achieve  the program’s goals and allows for optimized or integrated cost, schedule,  and effort. The Project Management Institute (PMI) and GAO have  established standards and guides that are generally recognized as  leading practices for program management. When organizations apply  leading program management practices they may be able to enhance  their chances of achieving success across a range of programs.

A House Armed Services Committee report, accompanying a bill for the  National Defense Authorization Act for Fiscal Year 2017, included a  provision for us to review and assess DNN’s project and program  management processes and systems. Our report examines the extent  to which (1) selected DNN subprograms use program management  leading practices to manage schedule and cost, and (2) DNN has incorporated program management leading practices in its revised  program management policy.

To conduct this work, we reviewed 4 selected DNN subprograms – the  Nuclear Material Removal and Highly Enriched Uranium (HEU) Reactor  Conversion subprograms, which DNN manages under its Material  Management and Minimization program and the Radiological Security  and International Nuclear Security subprograms, which DNN manages  under its Global Material Security program. We selected these  subprograms for review because they had defined start dates, end dates,  and/or work scope and were not the subject of other ongoing or recently  completed GAO reviews. The information we obtained from these  subprograms is not generalizable, but we obtained important insights into  DNN’s schedule and cost management of these subprograms.

To examine the extent to which the selected DNN subprograms use  program management leading practices to manage schedule and cost,  we identified selected leading practices from PMI and GAO. The  selected leading practices we identified were the use of a master  schedule necessary to achieve a program’s goals, cost estimates that  cover the full life cycle of a program, and schedule and cost baselines to  measure performance. We reviewed documentation on the use of these  practices by the 4 selected DNN subprograms including documentation  on their schedule and cost estimates and their use of baselines, and we  compared the documentation to the leading practices. We also  interviewed NNSA officials who manage the selected DNN subprograms  about the use of these practices.

To examine the extent to which DNN has incorporated leading practices  into its revised program management policy, we reviewed the revised  DNN policy and interviewed NNSA officials about the development of the  new policy. We also reviewed program management plans for the  selected subprograms. We reviewed program management leading  practices identified by PMI in The Standard for Program Management and  by GAO in its schedule and cost guides and federal internal control  standards. We compared these practices with DNN guidance and  requirements contained in the revised DNN policy. During interviews with  NNSA officials, we obtained their views on why specific practices were  included in the revised policy and others were not.

We conducted this performance audit from June 2016 to September 2017  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. Appendix I provides more  detail on our scope and methodology.

Within DNN, the work of the four selected subprograms—Nuclear  Material Removal, HEU Reactor Conversion, Radiological Security, and  International Nuclear Security—focuses on efforts to remove and dispose  of excess nuclear material from civilian sites worldwide, convert civilian  research reactors to the use of non-weapons-useable nuclear fuel, secure  radiological materials at their source in the United States and abroad, and  improve the security of weapons-useable nuclear material in key  countries. The selected subprograms organize their work in programmatic  areas which we refer to as components, and under each component the  subprograms manage projects. Table 1 below describes the work of each  subprogram and the components in which the subprogram organizes its  work scope.

Program Management Leading Practices Related to Schedule and Cost Management

PMI’s The Standard for Program Management and GAO’s schedule and  cost guides identify program management leading practices related to  schedule and cost estimating and measuring performance against  baselines, as follows:

PMI guidelines. According to PMI’s guidelines, programs practice  life-cycle management, which involves schedule and financial  management throughout the course of the program’s life-cycle  phases—program definition, benefits delivery, and closure. In  particular, PMI states that in conducting program schedule  management, programs use a master schedule that integrates the  schedules of program components necessary to achieve the  program’s goal. In program financial management, program cost  estimates should be clearly defined and should consider the full life- cycle costs of the program. According to PMI, programs should also  establish and measure performance against baselines for both  schedule and cost.

GAO schedule and cost guides. GAO’s schedule and cost guides,  which draw from federal organizations and industry, define best  practices about the processes needed for the development and  management of high-quality and reliable schedule and cost  estimates. Similar to PMI’s guidelines, according to the GAO guides,  programs should establish and use an integrated master schedule,  establish cost estimates that cover the full life cycle of the program,  document and define assumptions tailored to the program,  incorporate analysis of program risk and uncertainty in schedule and  cost estimates, and manage a program’s schedule and cost by  measuring against a baseline.

Selected DNN Subprograms Generally Do Not Use Selected Leading Practices to Manage Schedule and Cost

The four DNN subprograms we chose for review generally do not use  selected leading program management practices to manage schedule  and cost. Specifically, at the time of our review, none of the subprograms  had schedule and cost estimates that encompassed its entire life cycle,  although one subprogram planned to develop such estimates for its  recently-extended life cycle. In addition, none of the selected  subprograms measure their overall schedule and cost performance  against baseline estimates. NNSA officials said that the subprograms had  not developed schedule and cost estimates that cover their life cycles and  did not measure the subprograms against baselines due, in part, to  uncertainty in planning scope and schedules that rely on the cooperation  of other countries. DNN also does not require subprograms to have such  estimates or to measure performance against schedule and cost  baselines. Following these practices, however, would provide NNSA  managers and other stakeholders more complete information to evaluate  how much the subprograms may cost to achieve their goals, the amount  of time they may need to achieve these goals, and their actual versus  planned performance.

According to leading practices, programs should (1) establish a master  schedule that integrates the schedules of program components  necessary to achieve the program’s goal, such as specified performance  to be achieved over a defined life cycle, (2) determine costs that consider  the full life-cycle costs of the program, and (3) measure performance  against baselines for both schedule and cost. Figure 2 illustrates the  extent to which the selected subprograms have established schedule and  cost estimates compared to their planned life-cycle completion dates, if  any.

The Nuclear Material Removal subprogram had schedule and cost  estimates that encompassed all three of its subprogram components  through the subprogram’s previously planned completion date of fiscal  year 2022. However, the subprogram had yet to update its schedule and  cost estimate through its new planned completion date of fiscal year  2027, which was established in May 2017. The subprogram did not have  readily available information on performance against its former schedule  and cost estimates. Specifically:

Schedule. As of April 2017, the subprogram’s schedule, which  encompassed all three subprogram components, included 52 ongoing  and planned projects with estimated completion dates by the end of  fiscal year 2022 for most of these projects to reach a goal to remove  or disposition a total of 8,466 kilograms of nuclear material. In May  2017, the subprogram extended its life cycle from fiscal year 2022 to  fiscal year 2027 but at the time of our review had yet to update its  schedule of planned projects to be completed during fiscal years 2023  through 2027. According to NNSA officials, they extended the  subprogram’s life cycle in part because certain projects planned to be  completed by fiscal year 2022 were delayed and the subprogram’s  work was expanded.

Cost. The subprogram had a cost estimate for its planned work  through fiscal year 2022 but at the time of our review had yet to  update its cost estimate for the overall subprogram through its new  planned completion date of fiscal year 2027. Specifically, as of June  2017, the subprogram had a cost estimate of about $595 million,  according to our analysis of information provided by the subprogram.  This estimate covered the planned work scope of all three  subprogram components to be completed during fiscal year 2017  through 2022. The subprogram, however, did not have estimated  costs for completing work scope planned during fiscal years 2023  through 2027. According to NNSA officials, as of June 2017, they  were developing a cost estimate for the remaining years, although the  officials did not specify when the cost estimate would be completed.

Measuring performance against baselines. The subprogram did  not measure its overall performance against schedule and cost  baselines. NNSA reported to Congress in July 2014 that the  subprogram planned to remove or disposition approximately 3,000  kilograms of nuclear material by fiscal year 2022 at an estimated cost  of about $600 million. However, the subprogram did not track  information on its performance against the cost estimate. According to  NNSA officials, removal projects have too many uncertain costs.  Instead, NNSA officials said that they update the subprogram’s life- cycle cost each year as part of the annual planning for the next fiscal  year’s budget request. Until the subprogram develops schedule and  cost estimates to support the recently revised life-cycle completion  date of fiscal year 2027, it does not have the baselines it needs to  measure its overall schedule and cost performance. Although the  subprogram did not measure its overall performance against  established schedule and cost baselines, according to monthly  performance reports, the subprogram baselined and measured the  schedule performance of individual removal projects by tracking the  difference in number of days between forecasted project completion  dates and baseline completion dates. However, the subprogram did  not have information that integrated project performance information  to provide an overall picture of schedule performance for the entire  subprogram.

The HEU Reactor Conversion subprogram had schedule and cost  estimates that covered the remaining work scope to complete two of three  subprogram components by fiscal year 2033 but not for a third  component estimated to be completed in fiscal year 2035. The  subprogram also did not measure its overall performance against  schedule and cost baselines. Specifically:

Schedule. The HEU Reactor Conversion subprogram did not have a  schedule for the overall subprogram through completion of its life  cycle. Instead, the subprogram had a schedule for all work scope  planned for the 5-year FYNSP, which included the schedule for the  remaining work to complete one of the three subprogram  components–Molybdenum 99 (Mo99) efforts. Beyond the FYNSP  planning period, the subprogram has an estimated completion date of  fiscal year 2033 for a second component—U.S reactor conversions— and has developed a schedule for completion of the component. For  the third subprogram component—international reactor conversions— the subprogram estimates a fiscal year 2035 completion date for its  remaining work scope to convert or verify the shutdown of 44  international reactors, but it had not developed a complete schedule  to meet that date. Specifically, the subprogram’s schedule was not up- to-date for 22 of the 44 international reactors in the subprogram’s  planned work scope to support the estimated fiscal year 2035  completion date for these reactors. Instead, in the subprogram’s  schedule, these reactors had estimated completion dates by fiscal  year 2030. NNSA officials explained that the schedule was not up-to- date for these reactors because the reactors are in countries where  the subprogram cannot currently plan or implement the conversions  due to limitations in cooperation with these countries. For example,  DNN cannot plan the schedule for conversion of reactors in Russia  that are in the subprogram’s scope until the United States and Russia  resume joint nuclear security activities that the United States  discontinued following Russia’s invasion of Ukraine in 2014. NNSA  officials said that the 2035 date is their best judgment of the earliest  date when the subprogram could complete the conversions or verify  certain reactors’ shutdowns based on the assumption that the United  States and Russia may resume nuclear security cooperation in the  2020s. Because of the high degree of uncertainty with this date, the  subprogram did not update the schedule to reflect the 2035 date,  according to the officials. Appendix II provides tables that list the  planned reactor and facility projects in the HEU Reactor Conversion  subprogram, their locations, and estimated conversion or shutdown  completion dates.

Cost. The HEU Reactor Conversion subprogram did not have a life- cycle cost estimate for the overall subprogram, but had overall life- cycle cost estimates for two of the three subprogram components.  The subprogram had cost estimates that totaled approximately $1.1  billion through fiscal year 2033 and that included the remaining  estimated life-cycle costs for the subprogram’s U.S. reactor  conversions component and its Mo99 efforts. For the third  component—international reactor conversions—the subprogram only  estimated costs for the 5-year FYNSP, not through the estimated  completion date for the component of fiscal year 2035. According to  NNSA officials, developing a cost estimate that includes all remaining  international reactor conversions through 2035 would be challenging  because the costs for these projects are highly uncertain and vary  depending on the willingness of each country to cooperate as well as  the unique technical, regulatory, and other factors that vary for each  reactor in each country. The subprogram, however, had established  estimated life-cycle budgets for completing the conversion or verifying  the shutdown of each reactor in its work scope, which could be used,  along with other information, to develop a cost estimate for the  subprogram component.

Measuring performance against baselines. The subprogram did  not measure overall subprogram performance against schedule and  cost baselines. Specifically, as mentioned above, the subprogram did  not have schedule and cost estimates for the overall subprogram that  it could use to establish baselines to measure the performance of the  overall subprogram. Although the subprogram had life-cycle estimates  for its U.S. reactors and Mo99 components, the subprogram did not  use these estimates as baselines to measure the overall subprogram  components’ performance. The subprogram measured schedule  performance of individual projects under its three components against  baselines by tracking the difference in number of days and months  between forecasted project completion dates and baseline completion  dates. However, it did not integrate and roll up the project information  to provide an assessment of its overall schedule performance. In  addition, the subprogram baselined and measured cost performance  of the U.S. High Performance Research Reactor project—which  constitutes six of the seven reactors under its U.S. reactor  conversions component—by tracking changes in the project’s  estimated life-cycle cost. However, the subprogram did not have  similar information that tracked changes in cost estimates of other  projects under its three components.

The Radiological Security subprogram did not have schedule and cost  estimates for three components through the subprogram’s planned  completion date in fiscal year 2033. The subprogram also did not  measure overall subprogram performance against schedule and cost  baselines. Specifically:

Schedule. The subprogram has an estimated completion date of  fiscal year 2033 but did not have an overall schedule that covered its  three components for meeting the 2033 date. Instead, the subprogram  had a schedule that covered work to be completed under its three  components during the 5-year FYNSP (fiscal years 2017 through  2021). Specifically, for two of the three subprogram components— radiological source removal and nonradioisotopic technologies—the  subprogram has not established specific work scope and schedules  beyond fiscal year 2021 because of uncertainty about the future. For  example, according to the subprogram’s director, planning the  adoption of nonradioisotopic technologies is uncertain because the  timing of when such technologies can be adopted depends, in part, on  regulations and international laws, making it challenging for the  subprogram to define the scope of work. For the third subprogram  component—radiological source protection—the subprogram has an  estimated completion date of fiscal year 2033 to reach a total target to  secure 4,394 buildings in its inventory of sites worldwide with high- priority radiological sources. However, the subprogram had not  developed a schedule of specific projects to be completed beyond the  5-year FYNSP to meet that date and target. NNSA officials said that  they are often uncertain when a project will be able to start because it  depends greatly on circumstances in each country. Appendix III  provides the Radiological Security subprogram’s planned work scope  for the radiological source protection component from fiscal years  2017 through 2033.

Cost. The Radiological Security subprogram did not have a life-cycle  cost estimate for the overall subprogram through its estimated  completion date of fiscal year 2033. Specifically, the subprogram  had a cost estimate of about $849 million for all three components  covering the 5-year FYNSP. However, for two of the three  subprogram components—radiological source removal and  nonradioisotopic technologies—the subprogram had not developed  cost estimates beyond the 5-year FYNSP because, as mentioned  above, it had not developed work scope for these components in the  out-years. For example, according to the subprogram’s director, the  subprogram’s radiological source removal component depends on the  voluntary participation of users of radiological sources that register  their sources with the subprogram. Therefore, the subprogram cannot  estimate the number of sources to be removed in out-years. For the  third subprogram component—radiological protection—the  subprogram had assumed a stable budget to complete its target to  secure 4,394 buildings by fiscal year 2033. However, according to the  director of the subprogram, this budget assumption was not intended  to be a reliable life-cycle cost estimate.

Measuring performance against baselines. As mentioned above,  the subprogram did not have schedule and cost estimates for the  overall subprogram needed to establish baselines to measure their  overall performance. The subprogram, however, baselined and  measured the schedule performance of individual projects under its  three components by tracking the difference in number of days  between forecasted project completion dates and baseline completion  dates. The subprogram, however, did not integrate and roll up the  project schedule performance information to provide performance  information for the overall subprogram.

The International Nuclear Security subprogram maintained schedule and  cost estimates for the 5-year FYNSP (fiscal years 2017 through 2021) but  did not have schedule and cost estimates for work scope in the years  beyond the FYNSP. In addition, the subprogram did not measure overall  performance against baselines. Specifically:

Schedule. The International Nuclear Security subprogram had not  established a life-cycle schedule for the overall subprogram or its two  component efforts, as it had not identified specific work scope or end- point targets beyond fiscal year 2021 and considers its mission to be  enduring (i.e. without an end-date). Instead, the subprogram had only  estimated a schedule for work scope in individual countries during the  5-year FYNSP. According to the subprogram director, the subprogram  is expected to operate indefinitely and continue as long as nuclear  materials exist to improve security in countries possessing such  materials. However, the subprogram had not planned project-specific  work scope in years beyond the FYNSP because, according to the  subprogram director, it is difficult to estimate the subprogram’s likely  level of foreign counterpart engagement in individual countries beyond  5 years.

Cost. Because it has not identified out-year work scope, the  International Nuclear Security subprogram did not have an overall life- cycle cost estimate and only had an estimate of about $530 million for  the work to be completed during the 5-year FYNSP period.  According to NNSA officials, they have not developed a cost estimate  for work scope in the years beyond the FYNSP because assumptions  about future work will likely change due to the uncertainty in  relationships with partner countries.

Measuring performance against baselines. The International  Nuclear Security subprogram did not measure performance of the  subprogram against schedule and cost baselines. Specifically, as  mentioned above, the subprogram did not have the schedule and cost  estimates for the subprogram’s life cycle beyond fiscal year 2021  needed to establish baselines to measure its overall performance. In  addition, the subprogram did not use its 5-year FYNSP estimates as  baselines to measure performance. Instead, the subprogram updates  the FYNSP estimates each year in planning the next fiscal year’s  budget request. Moreover, unlike the other three subprograms, the  International Nuclear Security subprogram did not have project  schedule baseline information that could be integrated and rolled up  to provide information on the performance of the overall subprogram.

In general, NNSA officials explained that uncertainty in planning the  selected subprograms’ work scope or schedules, particularly for  components with projects that rely on the cooperation of foreign  countries, was among the reasons they did not have schedule and cost  estimates that covered the subprograms’ life cycles or that went beyond  the 5-year required planning period. In addition, according to these  officials, DNN senior management does not require subprograms to  establish schedule and cost estimates that cover the entire subprogram  life cycle and to use these estimates as baselines to measure  subprogram performance.

However, uncertainty should not prevent these subprograms from  establishing more complete or longer-term estimates to account for the  time and resources they need to achieve their goals. As mentioned  above, without such estimates, the subprograms do not have the baseline  information they need to track their performance. According to leading  practices, developing reliable schedule and cost estimates can be  achieved by following steps that address data limitations and risks and  uncertainties for a program. For example, according to the GAO schedule  guide, a reliable schedule should reflect all of a program’s activities and  recognize that uncertainties and unknown factors in schedule estimates  can stem from, among other things, data limitations. In addition,  according to the GAO cost guide, the cost-estimating process involves  defining and documenting assumptions that are tailored to the specific  program, such as about the program’s life-cycle phases, political issues,  or technology development. Assumptions should be based on historical  data to minimize uncertainty and risk. These same assumptions should  also be used to develop the program schedule. For management to make  good decisions, the program estimate must reflect the degree of  uncertainty so that a level of confidence can be given about the  estimate. Accordingly, because assumptions defined for a particular  program’s schedule and cost estimate can vary, they should always be  inputs to the program’s risk analyses of cost and schedule.

Programs use different methods to quantify uncertainty and risk in  developing a schedule or cost estimate. DOE’s cost estimating guide  describes approaches for programs to incorporate risk and uncertainty in  cost estimates such as the use of lower- and upper-bound cost ranges  that are developed based on risk analysis. Other NNSA programs use  these approaches in developing schedule and cost estimates for highly  uncertain, long-term program plans. In particular, NNSA’s Office of  Defense Programs develops and reports high- and low-range cost  estimates for elements of NNSA’s nuclear weapons modernization  programs in part to account for the uncertainty in these long-term  program estimates. As mentioned above, such estimates would provide  NNSA managers and other stakeholders information to help evaluate  resources and compare the costs and benefits of different programs and  priorities. Because the selected subprograms do not measure their overall  schedule and cost performance against baselines, NNSA managers,  stakeholders, and Congress have incomplete information about these  subprograms’ actual-versus-planned schedule and cost performance over  their duration and are, therefore, at risk of being unable to assess when a  subprogram is likely to be completed or whether it will cost more or less  than planned.

DNN’s Program Management Policy Includes Some Leading Practices, but Does Not Address Life-Cycle Schedule and Cost Management

DNN’s Revised Policy Includes Leading Practices on Risk and Quality Management

DNN’s 2017 revised policy includes new sections that address leading  practices on risk and quality management that all DNN programs and  subprograms should follow. NNSA officials said they added these  sections based on their review of leading practices in PMI’s The Standard  for Program Management and GAO’s Standards for Internal Control in the  Federal Government to ensure these leading practices were incorporated  and required for DNN programs.

Risk management. According to leading practices on risk  management, programs should have processes to manage risks,  including processes to identify, assess, and respond to risks. In the  revised DNN policy, under a new section on risk management, all  DNN programs and subprograms are required to prepare risk  management plans to help identify, analyze, handle, and monitor risk.  For example, a DNN subprogram may identify the risk of schedule  slippage due to political constraints in working with foreign countries  and could incorporate and monitor that risk in planning.

Quality management. According to program management leading  practices on quality management, program quality should be  continuously monitored. A new DNN policy section on continual  improvement requires DNN programs and subprograms to plan and  implement methods, such as program evaluations and management  assessments, in order to monitor and improve processes. For  example, a DNN subprogram may use an independent review by the  NNSA Office of Management and Budget to help improve its program  management processes, such as how it tracks cost, scope, and  schedule. The revised policy also outlines steps for corrective actions  to be taken when noncompliance is detected. These steps range from  determining the cause of noncompliance to reviewing the  effectiveness of corrective actions taken.

These new sections added requirements for DNN program management  that were not previously documented. For example, in the prior policy, risk  management was not a requirement for DNN programs and  subprograms. In addition, NNSA officials said that they added the  continual improvement section to the revised policy after reviewing PMI’s  practices on quality assurance, which they believed would clarify  responsibilities regarding management assessments and independent  reviews.

The Revised DNN Policy Does Not Include Leading Practices on Life-Cycle Schedule and Cost Management

The revised DNN policy does not address or require leading practices on  life-cycle schedule and cost management for DNN programs or  subprograms. Specifically, the revised policy does not outline  requirements for programs or subprograms to establish life-cycle cost  estimates or measure performance against schedule or cost baselines.  Instead, the revised policy provides requirements on schedule and cost  management limited to the NNSA budgeting process covering the 5-year FYNSP. For example, according to the revised DNN policy, programs and  subprograms must conduct program management activities, such as  budget formulation, in alignment with anticipated resources in the FYNSP.  Additionally, the policy requires programs and subprograms to establish  performance measurement data and track cost or schedule performance,  but only within the FYNSP.

According to leading practices, life-cycle management is important to  program management and includes schedule and cost management  activities that span the duration of the program. According to PMI, all  programs, regardless of length, have life cycles; furthermore, leading  practices indicate that activities related to managing the schedule, cost,  and scope of a program should be conducted for the life of the program.  For example, leading practices call for calculating cost estimates as close  to the beginning of a work effort as possible that consider the full program  life cycle, and then documenting this baseline to measure performance.

According to NNSA officials, the revised DNN policy does not include  requirements to practice life-cycle management, including life-cycle  schedule and cost management, because officials determined that life- cycle management did not apply to some DNN programs that NNSA  officials believe are enduring or continuous. For example, as mentioned  above, the director of the International Nuclear Security subprogram said  that the subprogram will phase out of certain areas or reduce  engagement with certain countries in the future but that it is expected to  continue as long as nuclear materials exist and will work to improve  security in countries possessing such materials.

We disagree that life-cycle program management does not apply to  programs or subprograms that may have an enduring mission. Managers  need to make informed decisions about whether a program is affordable  within the agency’s portfolio. NNSA and DNN should be able to compare  DNN’s various programs’ requirements several years beyond its 5-year  planning period. According to the GAO cost guide, in developing  estimates, programs should define assumptions tailored to the program,  such as assumptions about the program’s life-cycle phases. For example,  the International Nuclear Security subprogram could take steps to define  end-point targets for when it may phase out work in certain areas or  countries in the future. In addition, according to the GAO schedule guide,  a comprehensive schedule should reflect all of a program’s activities and  recognize that uncertainties and unknown factors in schedule estimates  can stem from, among other things, data limitations. Moreover, because  assumptions themselves can vary, they should always be inputs to  program risk analyses of cost and schedule.

According to NNSA officials, although the revised policy does not include  requirements for life-cycle cost estimating, DNN programs could address  this in their individual program management plans. NNSA officials stated  that these program management plans for programs and subprograms  should be detailed enough to also provide information on how the  program will track progress, including by identifying changes to the  planned schedule.

However, the revised DNN policy does not clearly require DNN programs  or subprograms to have program management plans, nor does it specify  elements of such plans. Specifically, the revised DNN policy requires  each program to develop “program management documentation” that  identifies program scope, schedule, and cost during the fiscal year and  operating procedures for the fiscal year, but it does not outline similar  requirements for the program’s life cycle. In addition, the revised policy  does not specify requirements or guidance, such as on cost estimation,  for what programs or subprograms are to include in the program  management documentation. In contrast, PMI indicates that programs  should develop a program management plan that includes plans for  program financial management, schedule management, and scope  management for all phases of the program’s life cycle. According to  NNSA officials, the revised DNN policy is the only directive or  documentation that spells out what is needed or required to be included in  a program management plan.

Although the revised DNN policy does not clearly require DNN programs  or subprograms to have program management plans, some DNN  programs have developed or are developing such plans. For example, the  Global Material Security program, which oversees the Radiological Security and International Nuclear Security subprograms, issued a new  program management plan in April 2017. The Global Material Security  program management plan requires that each subprogram maintain a 5- year budget for the FYNSP with cost estimates, but it does not require or  provide guidance on developing life-cycle schedule or cost estimates.  NNSA officials said that DNN underwent a major reorganization of its  programs in January 2015, and some of the new program offices are still  preparing their program management plans. For example, the Material  Management and Minimization program that oversees the Nuclear  Material Removal and HEU Reactor Conversion subprograms is still  developing its program management plan, according to NNSA officials.

In addition, the four selected subprograms had various documented  plans, but none fully addressed life-cycle schedule and cost  management.

Nuclear Material Removal. The subprogram did not have a current  program management plan that had been updated since the 2015  reorganization of DNN but instead relied on an older plan that covered  a different scope than the scope of the current subprogram.

HEU Reactor Conversion. The subprogram did not have a program  management plan for the overall subprogram. Instead, the  subprogram had project execution plans for its U.S. reactor  conversion projects and its Mo99 projects and relied on an outdated  document for its international reactor conversion projects.

Radiological Security. The subprogram had a program management  plan that included requirements for the use of project life-cycle  baselines and for conducting cost estimation for the 5-year FYNSP.  However, the plan had no requirement for developing a cost estimate  for the life cycle of the subprogram and for using such an estimate to  measure performance of the overall subprogram.

International Nuclear Security. The subprogram had a program  management plan that required cost estimating for 1 fiscal year.  However, the plan did not include requirements for life-cycle estimates  and for using initial or updated baselines to measure performance.

NNSA subprogram officials said that they do not have readily available  life-cycle cost estimates and baseline measurement data in part because  they are not asked to provide it. For example, NNSA officials from the  HEU Reactor Conversion subprogram said that they did not have  sufficient staff to track performance against initial baselines because it  was not a priority for management, although it would be possible to do so  if required. One of the stated goals of the revised DNN policy is to  facilitate DNN-wide implementation of methods for programs and  subprograms to monitor, measure, and improve management processes.  However, because the policy does not require more complete information  from DNN programs and subprograms on their cost, schedule, and  performance against baselines—consistent with leading practices—it is  not clear that this policy goal can be achieved.

When organizations apply leading program management practices—such  as establishing schedules and cost estimates covering their planned life  cycles and measuring performance against such baselines—they may be  able to enhance their chances of achieving success across a range of  programs. However, the four selected DNN subprograms are generally  not applying these selected leading practices for life-cycle program  schedule and cost management, due in part to the uncertainty and risks  in working with international partners. However, methods and approaches  exist that allow programs to account for uncertainty and risk in developing  schedule and cost estimates for their planned scope of work.  Furthermore, while the revised DNN program management policy has  incorporated some leading practices, it does not include requirements  and guidance for DNN programs and subprograms to practice life-cycle  schedule and cost estimating and does not require program management  plans that could be the vehicle for DNN programs and subprograms to  specify the use of such estimates. Updating the DNN program  management policy to include requirements for DNN programs and  subprograms to follow leading practices for life-cycle program  management would help NNSA ensure that managers, stakeholders, and  Congress have better information on how much DNN programs and  subprograms may cost to achieve their goals, the amount of time they  may need to achieve these goals, and how efficiently and effectively they  are actually being executed compared to plans.

The NNSA Deputy Administrator for DNN should revise the DNN program  management policy to require DNN programs and subprograms to follow  life-cycle program management. These requirements should include  development of schedule and cost estimates that cover the life cycle of  DNN programs and subprograms, use of methods to account for  uncertainty and risk in such estimates, use of cost and schedule  baselines to measure performance over program and subprogram life  cycles, and development of program management plans.  (Recommendation 1)

We provided NNSA with a draft of this report for its review and comment.  In written comments, which are summarized below and reproduced in  appendix IV, NNSA neither agreed nor disagreed with our  recommendation to revise the DNN program management policy to  require DNN programs and subprograms to follow life-cycle program  management. However, NNSA stated that it plans to take action in  response to the recommendation.

In general, NNSA stated that DNN will update its program management  policy to formally document current practice and clarify expectations for  addressing uncertainty. Specifically, NNSA said it will update the policy  to: (1) reflect that life-cycle cost and schedule management should be  applied at the project or subprogram level where appropriate, considering  the extent of uncertainty impacting scope, potential timelines, and  executability; (2) define the methodologies to (a) account for uncertainties  where applying these techniques would result in a reasonable range of  estimates that would be useful for planning and scheduling purposes or  (b) document risk and track actions to reduce uncertainty where  applicable; (3) address expectations for assessing cost and schedule  performance, commensurate with the level of certainty present at  baselining; and (4) address requirements for documenting program  management plans.

Although we acknowledge NNSA’s plan to update its policy, we have  concerns regarding whether its proposed actions will ensure that DNN  programs and subprograms effectively follow leading practices for life- cycle schedule and cost management in the future. First, we do not  believe that updating the DNN program management policy to formally  document current program management practice addresses our  recommendation. NNSA’s response suggests that its update to the policy  is intended to reflect current DNN program management practices rather  than signal a need for corrective action to address the DNN program  management limitations we identified. Specifically, as we stated in our  report, none of the four subprograms we reviewed had schedule and cost  estimates that encompassed the entire life cycle, although one  subprogram planned to develop such estimates for its recently-extended  life cycle. In addition, NNSA’s proposed update to the DNN program  management policy to reflect life-cycle schedule and cost management  “where appropriate” is vague, and may give programs and subprograms  too much discretion to avoid the requirement. To have an effective  requirement on life-cycle program management and to be responsive to  our recommendation, NNSA will need to clearly define the criteria for  when a program should be exempt from a requirement to follow life-cycle  program management.

Finally, the meaning of NNSA’s proposed update to the policy to address  expectations for assessing cost and schedule performance,  commensurate with the level of certainty present at baselining is unclear.  Specifically, it is unclear whether NNSA plans to require that DNN  subprograms use cost and schedule baselines to measure performance,  or whether it plans to exempt programs or subprograms from such  practices based on unstated expectations. As we stated in our report,  none of the subprograms we reviewed measured their overall schedule  and cost performance against baseline estimates. To ensure that DNN  subprograms take steps to measure schedule and cost performance  against baselines and to be responsive to our recommendation, NNSA  will need to define clear expectations for DNN programs and  subprograms to follow.

NNSA also provided general comments in its written comments regarding  DNN program management.

First, NNSA commented that DNN currently implements elements of life- cycle program management where appropriate and reasonable. However,  according to NNSA, the majority of its international activities operate with  an unusually high level of uncertainty regarding potential international  cooperation and with limited information on international operations to  understand the scope of work required to support useful planning and  estimating. In NNSA’s view, the high uncertainty would result in range  estimates so broad as to serve no useful purpose, and there is no  appreciable cost-benefit to expending resources on such calculations.

We recognize that organizations need flexibility to determine when it is  appropriate and useful to apply leading practices on life-cycle program  management. However, as noted in our report, managers need to make  informed decisions about whether a program is affordable within the  agency’s portfolio. Without more complete schedule and cost information  on DNN subprograms, NNSA managers and other stakeholders have  degraded information on the elements of DNN’s portfolio, which may limit  their ability to assess and justify the affordability of long-term plans. If  NNSA believes that some of DNN’s planned international work scope is  too uncertain for subprograms to develop estimates of schedule and cost  that cover their life cycles, then NNSA should evaluate whether it is  appropriate to identify such work scope in DNN’s long-term plans at all.

Second, NNSA commented that no specific requirement exists for DNN  programs and subprograms to implement life-cycle cost estimates, and  that DNN complies with current requirements. NNSA also commented  that the proper application of leading practices recognizes that cost- benefits, as well as the potential usefulness and reliability of estimates,  are important considerations. In instances in which uncertainty is  extremely high, NNSA stated that focus shifts to disclosure of risks, and  the establishment and tracking of actions to reduce the level of  uncertainty. According to NNSA’s comments, DNN discloses risks and  tracks actions to reduce the level of uncertainty extensively, and this was  reflected in the most recent update to the DNN program management  policy with the addition of a new section on risk management. NNSA also  stated that as uncertainty is reduced, then other principles can be applied  where appropriate.

We stated in our report that no specific requirement exists for DNN  programs and subprograms to implement life-cycle cost estimates.  Specifically, we noted that the DNN policy required that program  management functions be conducted over the 5-year FYNSP. Therefore,  we agree that the DNN subprograms we chose to review complied with  current requirements. However, our review was not focused on  compliance with requirements but rather on the use of leading or good  program management practices. We also noted that NNSA’s stated  objectives for the DNN policy include establishing a DNN-wide policy that  incorporates leading practices for program management and that  facilitates the implementation of methods for programs and subprograms  to monitor, measure, analyze, and improve management processes.  Leading practices on life-cycle program management are important for an  organization to successfully plan the resources it needs to achieve its  goals and assess its performance in doing so. DNN’s revised policy did  not acknowledge management of the program life-cycle as an essential  program management function and did not include any requirements on  leading practices on life-cycle schedule and cost management.

We agree that risk management processes should be used to monitor  risks and track actions to reduce uncertainty. As we stated in our report,  the revised DNN policy included a new section on risk management  under which all DNN programs and subprograms will be required to  prepare risk management plans to help identify, analyze, handle, and  monitor risk. However, the new section did not include criteria for DNN  subprograms to follow when uncertainty related to risks being monitored  is low enough to allow a subprogram to develop life-cycle schedule and  cost estimates.

We are sending copies of this report to the appropriate congressional  committees, the NNSA Administrator, the NNSA Deputy Administrator for  Defense Nuclear Nonproliferation, and other interested parties. In  addition, the report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-3841 or oakleys@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix V.

This report examines the extent to which (1) selected subprograms within  the National Nuclear Security Administration’s (NNSA) Office of Defense  Nuclear Nonproliferation (DNN) use program management leading  practices to manage schedule and cost, and (2) DNN has incorporated  program management leading practices in its revised program  management policy.

To conduct this work, we reviewed 4 selected DNN subprograms. DNN  has 4 major programs that manage a total of 13 subprograms (a  subprogram is a program managed as part of another program).  Specifically, we selected the Nuclear Material Removal and Highly  Enriched Uranium (HEU) Reactor Conversion subprograms, which DNN  manages under its Material Management and Minimization program. In  addition, we selected the Radiological Security and International Nuclear  Security subprograms, which DNN manages under its Global Material  Security program. We selected these subprograms for review because  they had defined start dates, end dates, and/or work scope indicating that  they had project-like aspects. These subprograms organize their work in  programmatic areas which we refer to as components and under each  component the subprograms manage various types of projects, such as  projects to remove nuclear material from civilian sites worldwide. We also  selected the 4 subprograms because they were not the subject of other  ongoing or recently completed GAO reviews. The information we  obtained from these subprograms is not generalizable, but we believe  that we obtained important insights into DNN’s cost and schedule  management of these subprograms.

To examine the extent to which the selected DNN subprograms use  program management leading practices to manage cost and schedule,  we identified selected leading practices by the Project Management  Institute (PMI) in The Standard for Program Management and by GAO in  its schedule and cost guides. The selected leading practices we  identified were the use of a master schedule necessary to achieve a  program’s goals, cost estimates that cover the full life-cycle of a program,  and schedule and cost baselines to measure performance. We collected  and reviewed subprogram planning documents, monthly performance  reports, and spreadsheet data on work scope, historical costs, schedules  and cost estimates established by the subprograms, and their use of  project baselines to measure performance. We also reviewed information  the subprograms reported in NNSA’s fiscal year 2017 and 2018  congressional budget justifications. We also interviewed NNSA officials  and their contractors who manage the program management information  system used by 3 of the 4 subprograms to manage schedule and cost  information to understand its capabilities. We interviewed NNSA officials  who manage the selected DNN subprograms about the use of these  practices and their views on challenges or limitations in using them. We  also interviewed representatives at Argonne National Laboratory and  Pacific Northwest National Laboratory, which operate projects for the  subprograms, to identify how projects develop schedule and cost  estimates and pass information on to the subprograms.

To assess the reliability of the schedule and cost estimates on the  selected subprograms, we interviewed NNSA officials and national  laboratory contractors who were knowledgeable about the process  followed to develop and update the estimates and the program  management information systems used to manage the schedule and cost  information and generate reports. We determined that the data were  sufficiently reliable for our purposes, which were to report the  subprograms’ estimated schedule completion dates and cost estimates,  as well as report the fiscal years and subprogram components and  projects covered by the subprogram schedule and cost estimates.

To examine the extent to which DNN has incorporated leading practices  into its revised program management policy, we reviewed DNN’s revised  program management policy approved in February 2017. We compared  the revised policy to the 2005 version to identify the changes included in  the revised policy. We reviewed program management leading practices  by PMI in The Standard for Program Management and by GAO in its  schedule and cost guides and federal internal control standards. For  example, we considered the applicable leading practices on schedule and  cost management identified above as well as other practices such as  those on risk management, quality management, and development of  program management plans. We compared these practices to DNN’s  requirements and guidance contained in the revised DNN policy. We  interviewed NNSA officials about the development of the new policy and  their views on the reasons specific leading practices were included in the  revised policy and others were not, as well as challenges DNN’s  programs and subprograms face in managing program schedule and  cost. We also reviewed program management plans for the 4 selected  subprograms and the major programs under which these subprograms  operate. We then interviewed NNSA officials from the selected  subprograms to determine their involvement in developing the revised  DNN program management policy and the status of individual program  management plans that were under development at the time of our  review.

Appendix II: Scope and Completion Dates for the Highly Enriched Uranium Reactor Conversion Subprogram

The Office of Defense Nuclear Nonproliferation’s Highly Enriched  Uranium (HEU) Reactor Conversion subprogram consists of three  components: (1) U.S. research reactor conversions, (2) international  research reactor conversions, and (3) Molybdenum 99 (Mo99) efforts,  which include international Mo99 isotope production reactor conversions  and projects to establish new U.S. non-HEU Mo99 production facilities.  The subprogram’s current goal is to convert or verify shutdown of 156  HEU reactors and isotope production facilities and to support the  establishment of a domestic, non-HEU-based Mo99 production capability.  Tables 2 through 4 below list the U.S. reactor conversions, international  reactor conversions or shutdowns, and Mo99 projects in the HEU Reactor  Conversion subprogram’s planned scope of work, for each of the  subprogram’s three components, as of July 2017.

Appendix III: Scope and Completion Dates for the Radiological Security Subprogram’s Source Protection Component

The Office of Defense Nuclear Nonproliferation’s Radiological Security  subprogram’s current goal for the radiological source protection  component is to upgrade security in 4,394 buildings worldwide by fiscal  year 2033. Table 5 shows the estimated number of buildings to be  completed each year as of June 2017.

Appendix IV: Comments from the National Nuclear Security Administration

Appendix V: GAO Contact and Staff Acknowledgments

Shelby S. Oakley, (202) 512-3841 or oakleys@gao.gov.

In addition to the individual named above, William E. Hoehn (Assistant  Director), Natalie M. Block, R. Scott Fletcher, Brian M. Friedman, Cindy  Gilbert, Jason T. Lee, TyAnn Lee, Duc Ngo, Jeanette Soares, Sheryl  Stein, and Sara Sullivan made key contributions to this report.

What is the U.S. government doing to keep nuclear weapons out of the wrong hands?

The National Nuclear Security Administration implements nonproliferation programs to reduce the threat of more nations or non-state actors acquiring nuclear weapons or radiological materials for a “dirty bomb.” The agency requested about $1.5 billion for 2018 to fund these programs, but we found that they do not have complete schedule and cost information, limiting the ability to assess performance toward achieving their goals.

We are making a recommendation to bring the agency's nonproliferation program management in line with leading practices.