Department of Energy: Performance Evaluations Could Better Assess Management and Operating Contractor Costs

In fiscal years 2006 through 2016, the federal government spent almost $193 billion on DOE's M&O contracts—a form of contract that traces its origins to the Manhattan Project. Six DOE offices use M&O contracts to manage and operate federally owned sites that perform work to fulfill DOE's diverse missions, such as conducting scientific research and maintaining nuclear weapons.

GAO was asked to review DOE's performance management of its M&O contracts. This report examines, among other things, (1) how DOE offices evaluated M&O contractor performance in fiscal years 2006 through 2016; (2) the extent to which DOE's fiscal year 2016 M&O contractor PERs provide information on contractors' technical, administrative, and cost performance; and (3) the results of DOE's M&O contractor performance evaluations for fiscal years 2006 through 2016.

GAO reviewed performance evaluation documents for 21 of the 22 DOE M&O contracts; analyzed DOE policies, procedures, and guidelines, and federal regulations; analyzed technical, administrative, and cost aspects of M&O contracts' 2016 PERs; and interviewed DOE officials.

In fiscal years 2006 through 2016, six offices within the Department of Energy (DOE) generally used one of three different approaches to evaluate management and operating (M&O) contractor performance. Although these approaches varied in the performance criteria and methodologies used for determining contractor ratings and incentives, all the offices annually set expectations for contractors and assessed performance.

In analyzing DOE's fiscal year 2016 Performance Evaluation Reports (PER), GAO found that these reports provided less information on M&O contractors' cost performance than on contractors' technical and administrative performance. The cost information provided in the PERs often was not detailed, did not indicate the significance of the performance being described, and applied only to specific activities. Further, the information is of limited use for acquisition decision-making, such as deciding whether to extend the length of a contract, because it does not permit an overall assessment of cost performance. A key reason PERs did not include more cost performance information is that the DOE offices' policies do not require specific assessments of cost performance or discuss how to ensure cost information is useful for future acquisition decision-making. By updating policies to require inclusion of quality cost performance information in PERs, DOE offices could better assess M&O contractors' costs, improve acquisition decision-making, and ensure performance evaluations fully address required elements.

Based on GAO's review of DOE M&O contractor performance evaluations from fiscal years 2006 through 2016, DOE generally provided high performance ratings and more than 90 percent of available performance incentives (see figure). Ratings for some areas of contractor performance, as well as ratings for contractor performance at specific DOE sites, varied from this trend. For example, three times during this period contractors received 50 percent or less of available award and incentive fees due to a major accident and safety and security issues.

GAO is making seven recommendations to DOE, including to each of the six DOE offices to update their policies requiring that PERs include quality information to enable an overall assessment of M&O contractor cost performance. In commenting on a draft of this report, DOE generally agreed with these recommendations.

In fiscal years 2006 through 2016, the federal government spent almost  $193 billion on Department of Energy (DOE) management and operating  (M&O) contracts—a form of contract that traces its origins to the  Manhattan Project during World War II. DOE relies extensively on M&O  contracts to manage and operate many of its government-owned,  contractor-operated sites. DOE’s diverse missions are carried out at  these sites, including developing, maintaining, and securing the nation’s  nuclear weapons capability and conducting basic energy and science  research and development. According to DOE’s Fiscal Year 2017 Agency  Financial Report, DOE spends approximately 90 percent of its annual  budget on contracts. As we and the DOE Inspector General have  previously found, DOE and its contractors face several management  challenges involving large future costs, including nuclear weapon  modernization, environmental liabilities, and aging and degraded  infrastructure. Because of DOE’s reliance on M&O contracts, contract  management is critical to the agency’s ability to successfully and cost- effectively meet these challenges.

DOE manages and oversees its M&O contractors through a variety of  means, including a performance evaluation process that evaluates  contractor performance, reports on those evaluations, and rewards or  penalizes contractors. Contractor performance evaluation reports cover a  wide range of activities due to the significant size and scope of M&O  contracts. These reports help form the basis of a contractor’s  performance record, which DOE and other agencies consider in awarding  future contracts. Further, when an M&O contract has reached the end of  its contract term, the Federal Acquisition Regulation (FAR) and DOE  policy require DOE to consider the contractor’s technical, administrative,  and cost performance before deciding whether to extend the contract or  open it up for competitive bids.

DOE’s history of inadequate management and oversight of its contractors  has led us, since 1990, to designate aspects of the department’s contract  management as a high-risk area vulnerable to fraud, waste, abuse, and  mismanagement. In January 2009, to recognize progress made at  DOE’s Office of Science, we narrowed the focus of the department’s high- risk designation to two DOE offices: the National Nuclear Security  Administration (NNSA) and the Office of Environmental Management  (EM). In February 2013, we further narrowed the focus of DOE’s high- risk designation to major projects and contracts (i.e., those with values of  at least $750 million) within these two DOE offices, to acknowledge  progress made in managing smaller-value efforts.

Our recent reports and those from the DOE Inspector General, as well as  from commissions, task forces, and other outside groups, have  highlighted ongoing challenges with DOE’s contracting and contract  management. For example, the Congressional Advisory Panel on  Governance of the Nuclear Security Enterprise (also known as the  Augustine-Mies Panel) and the Commission to Review the Effectiveness  of the National Energy Laboratories (CRENEL) recommended various  measures for reforming DOE’s M&O contractor management and  oversight, including DOE’s use of performance incentives. DOE also  recognizes that it has faced long-standing challenges in managing its  contracts, as outlined in a recent memo from the Deputy Secretary calling  for comprehensive reforms to improve acquisition management.

You asked us to review DOE’s performance management of its M&O  contracts. This report examines (1) how DOE offices evaluated M&O  contractor performance in fiscal years 2006 through 2016 and the extent  to which these offices have documented their evaluation approaches; (2)  the extent to which DOE’s fiscal year 2016 M&O contractor performance  evaluation reports provide information on contractors’ technical,  administrative, and cost performance; and (3) the results of DOE’s M&O  contractor performance evaluations for fiscal years 2006 through 2016.

For all three objectives, we reviewed performance evaluation  documentation—such as performance evaluation plans, performance  evaluation reports, and fee determination—for 21 of the 22 DOE M&O  contracts in place as of fiscal year 2016, the most recently completed  contract year at the time we initiated our review. We also reviewed  documentation for the remaining contract—the Bettis and Knolls Atomic  Power Laboratories’ M&O contract—but excluded it from our analysis  because the contract did not have award fee and annual performance  evaluation reports comparable to the other DOE M&O contracts. We did  not examine performance evaluations or incentives provided outside the  main M&O contract performance evaluation reports for contracts that had  separate contract line items for certain projects with separate  performance evaluations. We also interviewed DOE officials to gain a further understanding of the department’s performance evaluation  processes and results.

To examine how DOE offices have evaluated M&O contractor  performance and the extent to which these offices have policies and  procedures to guide these processes, we reviewed DOE’s policies and  procedures for performance evaluations, as well as policies and  procedures from each of the six DOE offices with M&O contracts. We  also reviewed annual performance evaluation plans and performance  evaluation reports from fiscal years 2006 through 2016. We selected  this period to provide 10 years of data on M&O contractors, and fiscal  year 2016 was the latest year for which data were available when we  began our engagement.

To evaluate the extent to which performance evaluation reports provided  information on each of the performance areas outlined in the FAR— technical, administrative, and cost—we performed content analysis of 22  DOE fiscal year 2016 performance evaluation reports for M&O  contractors. We examined the quality of the cost performance  information contained in the DOE fiscal year 2016 performance  evaluation reports by comparing the information contained in those  reports to the definition DOE provided for “quality” in its Information  Quality Guidelines. These guidelines apply to information DOE makes  available publicly, which includes many performance evaluation reports.

To examine the results of DOE’s M&O contractor performance  evaluations for fiscal years 2006 through 2016, we analyzed performance  ratings and incentives awarded in performance evaluation reports, fee  determination letters, and other performance evaluation documents. To  compare performance ratings and incentives provided to contractors at  M&O sites, we analyzed and provided information by “contract rating  sites” rather than individual contractors or physical sites, because the  individual contractors and how certain sites align with the contracts  changed over time. Accordingly, we analyzed 24 distinct contract rating  sites covered by 21 M&O contracts in place as of fiscal year 2016. There  are three more contract rating sites than the number of contracts in 2016;  two were because two individual contracts were consolidated into one  contract during the period we covered—we analyzed the two individual  contracts from prior to 2014 separately from the current consolidated  contract—and the third was because two DOE offices separately  evaluated the performance of a single contractor that performed activities  for each of those offices.

To analyze and summarize performance ratings and incentives awarded,  we used rating, award fee, and incentive fee information from 239  performance evaluations at the 24 contract rating sites. We did not  include fee ratings from years in which award or incentive fees were not  available. We also did not include information from the EM portion of the  Savannah River Site contract for fiscal years 2006 through 2009, since  award fees covered multiple years that did not align with individual fiscal  years. Appendix I provides additional information on our scope and  methodology.

We conducted this performance audit from October 2016 to February  2019 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

This section describes DOE’s M&O contracts, incentives in those  contracts, general requirements for DOE’s M&O contractor performance  evaluation processes, and contracting and performance challenges  involving DOE’s M&O contracts that have been identified by previous  reporting.

Since the Manhattan Project produced the first atomic bomb during World  War II, DOE and its predecessor agencies have depended on the  expertise of private firms, universities, and others with the scientific,  manufacturing, and engineering expertise needed to carry out research  and development work and manage and operate the government-owned,  contractor-operated facilities where the bulk of the department’s mission  activities are carried out. DOE relies on contracts in general, and M&O  contracts in particular, to do this work. According to DOE’s Fiscal Year  2017 Agency Financial Report, the department spends approximately 90  percent of its annual budget on contracts, and in fiscal year 2016 DOE  managed contracts valued at more than $24 billion. Of that amount, DOE  spent approximately 80 percent on its M&O contracts.

The work is closely related to the  agency’s mission and is of a long-term or  continuing nature, and there is a need to  ensure its continuity and for protection  covering the orderly transition of  personnel and work in the event of a  change in contractors. and sites that are contaminated from decades of nuclear weapons  production and nuclear energy research.

The Office of Fossil Energy (FE) manages the nation’s Strategic  Petroleum Reserve, which is an emergency stockpile of oil stored in  underground salt caverns in Texas and Louisiana.

NNSA, a separately organized agency within DOE, is responsible for  maintaining and enhancing the safety, reliability, and performance of  the nation’s nuclear weapons stockpile, promoting international  nuclear safety and nonproliferation, and supporting U.S. leadership in  science and technology, among other things.

The Office of Nuclear Energy’s (NE) primary mission is to advance  nuclear power as a resource capable of making major contributions in  meeting the nation’s clean energy supply and energy security needs.

The Office of Science (SC) supports scientific research for energy and  the physical sciences both by supporting (1) such research, and (2)  the development, construction, and operation of scientific user  facilities.

These DOE offices use M&O contracts to carry out their research and  development, nuclear weapons production, and other missions. For  example, for research and development, DOE is the nation’s single  largest funding source for basic physical sciences research, supporting  research in energy sciences, advanced scientific computing, physics, and  other fields. For weapons production, NNSA uses production sites to  maintain, evaluate, repair, and dismantle both the nuclear and non- nuclear components for nuclear weapons; manufacture weapons  components; and process tritium, a key isotope used to enhance the  power of nuclear weapons. DOE also uses M&O contracts for sites  dedicated to other types of missions, including nuclear waste disposal  and an emergency stockpile of oil. Figure 1 and appendix II provide  additional information on DOE’s M&O contracts.

In August 2016, we identified three key attributes associated with DOE’s  M&O contracts. First, M&O contracts have a limited competitive  environment—we found that about half of DOE’s fiscal year 2015 M&O  contract spending was on contracts awarded noncompetitively or that  received a single offer at the time they were competed. In addition, M&O  contracts include longer terms than other federal contracts, so they are  competed less frequently. Second, DOE M&O contracts have broad  scopes of work that cover nearly all aspects of work at a site. In particular,  though mission activities of M&O contractors can be highly technical,  mission support activities generally accounted for about 25 to 50 percent  of contractors’ total costs in fiscal year 2015, and encompassed such  things as managing infrastructure, facilities, and grounds; security; and  the internal audit function. Third, M&O contracts and DOE management  practices contribute to a closer relationship between contractors and the  government. For example, M&O contractors are generally more  integrated with DOE in how they are paid and in their accounting systems  than other types of contractors. With regard to payment, rather than  traditional bill payment methods including invoices, payment approval and  authorization, and disbursement of funds, M&O contractors can draw  funds directly from federal accounts through “letter of credit financing.”

With regard to accounting systems, as we reported in August 2016, DOE  requires M&O contractors to follow DOE’s Accounting Handbook and  integrate their costs and liabilities in DOE’s accounts each month. DOE  officials said that this provides visibility into contractor accounts and  allows DOE to monitor the appropriateness of the contractors’ withdrawal  of funds in near real time. According to DOE officials, this integration  carries over into how the value of contracts are determined—rather than  establishing the cost of the contract at the time of contract award, the  value of the M&O contract is determined by the amount annually  obligated on the contract by DOE, consistent with DOE’s annual  congressional appropriations.

Cost-reimbursement type contracts allow the agency to contract for work  when circumstances do not allow the agency to sufficiently define its  requirements or estimate its costs to allow for a fixed-price contract.  Under a fixed-price contract, a contractor accepts responsibility for  completing a specified amount of work for a fixed price. In contrast, under  cost-reimbursement contracts, the government reimburses a contractor  for allowable costs incurred, to the extent prescribed by the contract. The  government may also pay a fee that is either fixed at the outset of the  contract or adjustable based on performance criteria set out in the  contract.

In September 2009, we reported that cost-reimbursement contracts are  considered high risk for the government because of the potential for cost  escalation and because the government pays a contractor’s costs of  performance regardless of whether the work is completed. As such,  cost-reimbursement contracts are suitable only when (1) circumstances  do not allow the agency to define its requirements sufficiently to allow for  a fixed-price type contract; or (2) uncertainties involved in contract  performance do not permit costs to be estimated with sufficient accuracy  to use any type of fixed-price contract. One major reason for the inability  to accurately estimate costs is the lack of knowledge of the work needed  to meet the requirements of the contract, such as with research contracts,  which necessarily involve substantial uncertainties. The DOE Acquisition  Regulation (DEAR) states that cost-plus-award-fee (cost reimbursement)  contracts are generally the appropriate contract type for M&O contracts  and that the agency can choose among a number of different contract  types for its M&O contracts.

Under the FAR, cost-reimbursement contracts may include specific  incentives, such as arrangements intended to improve contractor efforts  and discourage inefficiency and waste. Table 1 provides definitions of  incentives commonly included in DOE’s M&O contracts.

Generally, according to DOE officials, award fees and incentive fees are  intended to motivate M&O contractor performance on an annual basis, as  outlined in annual performance evaluation plans. All DOE M&O contracts  GAO analyzed also include “conditional payment of fee” clauses that  permit the agency to reduce an otherwise earned fee if it determines that  the contractor’s performance did not meet minimum requirements, such  as those related to safety, health, or the environment. Under the award  term incentive, contractors can earn one additional year of performance  under the contract for each year they exceed certain thresholds in their  annual performance evaluations. (See apps. III through VIII for additional  information on the incentives included in each M&O contract, by DOE  office.)

In addition, other elements of contract administration or oversight, while  not formally incentives, can influence contractor performance. For  example, option periods—which are established in the contract—enable  the government to unilaterally extend the performance period and  performance of services. According to DOE officials, other potentially  important influences on contractor behavior include public reputation and  the ability to compete for follow-on DOE or other government contracts.

Performance Evaluation and Award Fee Requirements

The FAR, DEAR, DOE’s Acquisition Guide, and DOE policies provide  requirements and guidance for DOE’s annual performance evaluations of  contractor performance. Under the FAR, all contracts providing for award  fees must be supported by an award fee plan that establishes procedures  for evaluating award fees and an Award Fee Board to conduct award fee  evaluations. A Fee Determining Official makes the final determination  regarding the amount of award fee the contractor earns during the  evaluation period. Additionally, the FAR generally calls for entities that  administer contracts providing award fees to use a set of ratings from  Excellent to Unsatisfactory, which include performance descriptions and  associated available award fee percentages (see Table 2 below). Award  fee ratings are associated with a range of percentages of the total  available award fee that DOE offices may award to a contractor based on  the contractor’s assessed performance.

DOE offices develop two primary documents to guide and report  assessments of contractors’ performance for each fiscal year: a  Performance Evaluation and Measurement Plan (PEMP) and a  Performance Evaluation Report (PER). The PEMP is to be developed at  the beginning of each fiscal year—which is the beginning of the  evaluation period—and is to establish expectations for contractor  performance and describe how the responsible DOE office will evaluate  and measure performance against those expectations. The PEMP  provides the blueprint for what performance is expected of contractors,  how contractors’ performance will be evaluated, and how the evaluations  will be used to determine award fees, award terms, and any other  incentives. The PER is to be developed at the end of each evaluation  period—which typically is the end of the fiscal year—and is the  responsible DOE office’s evaluation of contractor performance, in which  DOE documents the performance rating and, in some cases, the fees and  other incentives that will be awarded to the contractor. Figure 2 shows the  general steps of DOE’s performance evaluation of contractors.

Further, under the FAR and DOE policy, the department is to consider  technical, administrative, and cost performance during acquisition  planning. The FAR provides that, for M&O contracts, replacement of an  incumbent contractor is largely based on an expectation of meaningful  improvement in performance or cost; thus, an agency or department  should consider three categories of performance—technical,  administrative, and cost—when deciding whether to extend or compete a  contract at the end of the contract’s term. According to DOE officials,  the annual performance evaluation process and the related PER are  important sources of information for making these decisions. Thus, the  PER should include relevant information on an M&O contractor’s  technical, administrative, and cost performance. For DOE, the M&O  contract PER is also important because DOE uses information from the  PER to update a contractor’s past performance information in the  Contractor Performance Assessment Reporting System (CPARS), which  DOE and other agencies use to understand a contractor’s performance  history and to inform their evaluations of future contract proposals.

Contracting and Performance Challenges Involving DOE’s M&O Contracts

A number of commissions, task forces, and other outside groups have  identified challenges involving DOE’s M&O contracts. For example, two  independent commissions—the Augustine-Mies Panel and CRENEL— have reported on related contract management challenges. The 2014  Augustine-Mies report focused on NNSA and made numerous  recommendations for comprehensive reforms, including addressing  dysfunctional government-M&O contractor relationships, improving  oversight of M&O contractors, and reforming award fee and performance  incentive structures. CRENEL, taking a broader view of all 17 national  laboratories across DOE, in 2015 found a similar erosion of trust between  DOE and some of its M&O contractors while noting that some  laboratories, in particular those under SC, had better, more effective  relationships. The CRENEL report recommended reforms to the  management and oversight of M&O contractors and performance  incentive structures. In addition to challenges, CRENEL also noted that  SC’s annual performance evaluation and planning processes were robust  and suggested that they be adapted by other DOE offices.

NNSA’s and EM’s contract management remains on our High-Risk List  for government operations vulnerable to fraud, waste, abuse, and  mismanagement. In addition, since 2005 we have identified a variety of  project and program outcomes associated with deficiencies in DOE’s  management and oversight of its M&O contractors. We have also  identified improvements needed in core processes and functions DOE  relies on to oversee its M&O contractors and assess their performance.  These reports include the following examples:

Since 2005, during various reviews, we found that cost accounting  practices used by NNSA’s M&O contractors have varied, making it  difficult for NNSA to compare costs across its sites or accurately  identify the total costs across its nuclear security enterprise and to  obtain reliable cost data. In January 2017, we reported on the  importance of reliable enterprise-wide cost information to effective  management and oversight and found that the plan NNSA submitted  to Congress in 2016 to improve and integrate its financial  management, as required by Congress in 2013, did not provide a  useful road map for guiding NNSA’s efforts. We recommended that  NNSA develop a plan for producing cost information that fully  incorporates leading planning practices. NNSA agreed, and we are  monitoring implementation of the recommendation.

In October 2014, we reported on actions taken to address challenges  with the Uranium Processing Facility under construction at the NNSA  Production Office Sites (specifically at the Y-12 National Security  Complex), which is managed by the M&O contractor at that site. A  challenge with this facility was that in July 2012 the M&O contractor  concluded that required equipment would not fit into the facility as  designed and that addressing this issue would cost an additional $540  million. NNSA’s analysis of the factors that contributed to this issue  identified several causes, including project oversight deficiencies— specifically, failure to ensure that requests and directives from NNSA  to the contractor were implemented.

In May 2015, we reported on NNSA’s use of contractor assurance  systems to conduct oversight and evaluate the performance of M&O  contractors. Contractor assurance systems are designed and used  by M&O contractors to oversee their own performance and to self- identify and correct potential problems. We found that NNSA had not  fully established policies or guidance for using information from these  systems to conduct oversight of M&O contractors and that NNSA  therefore did not have standards for ensuring that contractors are  overseen consistently. We recommended that NNSA establish  policies and guidance for using information from contractor assurance  systems for the oversight of M&O contractors; NNSA concurred with  the recommendations and has taken some steps to establish policies  and guidance, though it has not yet fully addressed our  recommendations.

In March 2017, we reported that DOE needed quality data to manage  its risk of fraud and recommended that DOE require contractors to  maintain sufficiently detailed transaction-level cost data that are  reconcilable with amounts charged to the government. DOE did not  concur with the recommendation and has not taken steps to  implement it. Because DOE does not require its contractors to  maintain sufficiently detailed transaction-level cost data that are  reconcilable with amounts charged to DOE, it is not well positioned to  employ data analytics as a fraud detection tool. As a result, DOE is  missing an opportunity to develop, refine, and improve its data  analytics and better meet requirements of the Fraud Reduction and  Data Analytics Act.

DOE Offices Use Different Approaches to Evaluate Contractor Performance, and all but NNSA Have Documented Their Approaches

In fiscal years 2006 through 2016, the six DOE offices generally used one  of three different approaches to evaluate M&O contractor performance.  All but one of these offices have documented their approaches in policies  and procedures; NNSA has a broad policy but does not have procedures  for implementing it, in particular for collecting and using performance  information. In the absence of documented procedures, NNSA may not  consistently collect and use performance information in evaluating  contractor performance.

DOE Offices Use Three Approaches That Differ in Their Criteria and Methodologies for Ratings and Incentives

According to DOE officials, DOE does not have a department-wide  performance evaluation process and offices developed their approaches  to performance evaluation based on their varying missions and  performance evaluation priorities.

We identified the following three general approaches:

The Science and Energy Lab approach (used by SC, EERE, and  NE) uses broad, office-wide performance criteria and a detailed  process and web-based tool to collect performance information and  determine ratings and incentives.

The NNSA approach also uses broad, office-wide performance  criteria, but ratings and incentives are determined through a series of  management meetings.

The Site Specific approach (used by FE and EM) uses more  detailed performance criteria specific to each contract and makes  rating and incentive determinations in ways that vary based on the  individual criteria.

These approaches generally differ in their (1) performance criteria, (2)  methodologies used to determine contractor ratings, and (3)  methodologies used to determine incentives. Appendixes III through VIII  provide additional information on each office’s performance evaluation  approach.

Based on our review of DOE documents, the three approaches all use a  combination of what PEMPs describe as subjective and objective  performance criteria. The Science and Energy Lab and NNSA  approaches use primarily subjective criteria, and the Site Specific  approach uses primarily objective criteria. Subjective criteria are generally  qualitative statements that describe desired contractor performance,  according to DOE officials. For example, a subjective criterion that SC  used during fiscal year 2016 was for contractors to “provide effective and  efficient strategic planning and stewardship of scientific capabilities and  program vision.” In contrast, DOE officials explained that objective criteria  generally describe performance that may be measured on a “pass/fail” or quantitative basis. For example, FE used objective criteria such as  developing a strategic plan by a specific date or ensuring that all phases  of construction were mechanically complete regarding the conversion of a  tank.

Performance criteria under the Science and Energy Lab and NNSA  approaches share a similar structure of three tiers of criteria: goals,  objectives, and notable outcomes (called key outcomes under NNSA’s  approach). The criteria are also mostly subjective and broad enough to be  consistent across all the contracts of the responsible DOE office. Based  on our review of DOE documents and information, SC and EERE have  used the Science and Energy Lab approach since fiscal year 2006 and  NE since fiscal year 2007. NNSA used the NNSA approach in fiscal  years 2013 through 2016.

Under the Science and Energy Lab and NNSA approaches, goals are  general overarching statements of the desired outcomes for each major  performance area under the contract and constitute the highest  performance criteria used to evaluate contractor performance. Based on  documentation describing these approaches, goals are to be composed  of at least two objectives, which are statements of desired results for an  organization or activity and that discuss specific actions the contractor will  undertake to accomplish a goal. Each office uses its respective goals and  objectives consistently for each of its M&O contracts (EERE and NE each  have only one site) and generally cover the same functional areas across  the offices, though some NNSA goals focus specifically on NNSA’s  nuclear weapons and national security missions. For complete lists of  goals and objectives used by the offices using the Science and Energy  Lab and NNSA approaches, see appendixes III, VI, VII, and VIII.

The third tier performance criteria used to evaluate contractor  performance is the notable outcome, which, according to agency  documents, is intended to focus the contractor on specific items that  officials identified as the most important initiatives or highest risk issues  the contractor must address. According to DOE documents, notable  outcomes differ from goals and objectives in that they (1) are usually  objective, (2) are specific to each contractor, and (3) change from year to  year. However, not all goals and objectives have associated notable  outcomes. Figure 3 provides an example of the relationship between a  goal and its related objectives and notable outcomes for SC’s  Brookhaven National Laboratory contractor for fiscal year 2016.

Our review of agency documents found that the Site Specific approach  consists primarily of objective performance criteria that are specific to  each contract, as well as a few broader, objective criteria. This is in  contrast to the other two DOE approaches to performance evaluation,  which primarily rely on broad, subjective criteria and a few objective  criteria. Based on our discussions with agency officials, both EM and FE  have generally used this Site Specific approach since fiscal year 2006.  For both offices, objective performance criteria are defined based on  quantifiable metrics (e.g., a contractor’s demonstrated waste processing  rate) and milestones (e.g., whether a contractor completed a task on or  before a scheduled date). For example, one of FE’s fiscal year 2016  objective performance criteria for the Strategic Petroleum Reserve M&O  contract is whether facilities and systems functioned at a level adequate  to meet program requirements based on average scores from its  Maintenance Performance Appraisal Rating tool. Further, our review of  agency documents showed that the Site Specific approach uses  subjective performance criteria for aspects of performance that may be  difficult to capture objectively—such as determining how effectively  measures a contractor has taken have prevented harm to workers, the  general public, and the environment. (See apps. IV and V for examples of  the objective and subjective criteria EM and FE use.)

Prior to fiscal year 2013, NNSA also used the Site Specific approach, and  it had specific, objective performance criteria that varied by contract.  Based on our review of agency documents, NNSA’s performance criteria  were generally divided into four performance areas: (1) mission, (2)  operations, (3) business, and (4) multi-site. According to NNSA officials,  as a result of “lessons learned” efforts, NNSA updated this approach to its  current one to provide more succinct, structured, and consistent reporting  by ensuring that all NNSA M&O contractors have identical goals and  objectives.

Methodologies Used to Determine Contractor Ratings

Based on our review of DOE documents, rating methodologies vary  across the three approaches—the Science and Energy Lab approach  uses a detailed, formulaic methodology; the NNSA approach determines  ratings at a series of management meetings; and in the Site Specific  approach, ratings depend primarily on whether the contractor  accomplishes specific tasks.

Based on our review of agency documents, under the Science and  Energy Lab approach, stakeholders—including officials from  headquarters, field offices, and internal and external customers— generally evaluate contractor performance against the criteria for each  objective and notable outcome (“lab customers” evaluate objectives under  science and technology goals only). Their evaluations, in the form of  narratives and numerical scores, are entered into a web-based  information collection tool that aggregates the scores using a series of  calculations and weights to generate ratings that are then approved by  the Fee Determining Official for the responsible DOE office. For  example, for SC, once individual stakeholders enter objectives’ scores  into the Laboratory Rating Tool, those scores are then weighted and  added together through a predetermined formula to provide an overall  rating of contractor performance for each goal. Under this approach, the  Laboratory Rating Tool aggregates the objective scores into numerical  goal ratings and corresponding letter grades from 4.3 (A+) to 0 (F) for the  contractor. Notable outcomes are rated on a “pass/fail” basis, meaning  that the contractor either met or did not meet them. Receiving a passing  rating for the notable outcome is required for the contractor to earn a B+  or better for the notable outcome’s associated objective. Thus, although  notable outcomes are not given their own numerical score or letter grade,  they can have a significant effect on a contractor’s objective ratings and,  ultimately, goal ratings. (See apps. III, VII, and VIII for examples of the  weighting and calculations involved in aggregating ratings for EERE, NE,  and SC M&O contractors.)

Based on our review of agency information, the methodology for the  NNSA approach to determine contractor ratings entails officials holding a  series of meetings to review various internally developed periodic reports  and other inputs (e.g., contractor self-assessments and inspection  reports). The participants in these meetings include field office managers,  program managers, and NNSA executive leadership who collaboratively  review contractor performance and determine ratings. According to NNSA  officials, at these meetings NNSA collaboratively reviews all M&O  contracts across the NNSA complex, thereby allowing officials to weigh  and compare performance. The Fee Determining Official determines the  final performance ratings for each M&O contractor using rating categories  from the FAR: Excellent, Very Good, Good, Satisfactory, and  Unsatisfactory. NNSA does not use numerical calculations to score and  weigh individual objectives or goals. Instead, NNSA officials use  professional judgment to determine overall goal ratings.

Based on our review of agency information, under the Site Specific  approach, field office officials rate contractor performance against  objective performance criteria quantitatively or pass/fail and rate  subjective performance criteria using FAR award fee categories. That is,  they evaluate performance against objective performance criteria as  completed or not completed—for example, whether the contractor  packaged 10 waste drums during the fiscal year. For the subjective  performance criteria, officials assign ratings using the FAR rating  categories in a similar manner to the NNSA approach.

Methodologies Used to Determine Incentives

Based on our review of DOE documents, the three performance  evaluation approaches also use different methodologies for determining  award and incentive fees, and two offices use similar methods to  determine whether the contractor receives award term. Based on our  review of agency documents, under the Science and Energy Lab  approach, once ratings are determined, several additional detailed  calculations determine how much of the available award fee is provided to  the contractor. Precisely how ratings are weighted to determine fee differs  by DOE office, but generally performance in technical areas is more  important in determining the amount of fee the contractor earns. For  example, SC determines award fees based on the contractor’s final  science and technology area rating and adjusts that fee if the final  management and operations area rating is 3.0 (grade B) or below. (See  app. VIII for additional information on SC’s fee determination, app. III for  EERE, and app. VII for NE.)

Based on our review of agency information, under the NNSA approach,  officials assign goals specific portions of the available award fee for each  contract at the beginning of the fiscal year. At the end of the fiscal year,  officials determine ratings and fees at the same time in the collaborative  meeting with NNSA leadership. For example, for the Los Alamos National  Laboratory contractor in fiscal year 2016, the nuclear weapons goal was  30 percent of fee, and the operations and infrastructure goal was 35  percent. As discussed earlier, the Fee Determining Official makes the  final determination on the ratings and also determines how much fee to  provide the contractor within the range defined by the FAR rating  (Excellent, Very Good, Good, Satisfactory, Unsatisfactory). In fiscal year  2016, NNSA awarded the Los Alamos National Laboratory M&O  contractor an “Excellent” rating for the nuclear weapons goal, which is  associated with the contractor earning from 91 to 100 percent of the  available fee for that goal. To determine the overall award fee for the  contract, NNSA adds up the award fees for all of its goals. (See app. VI  for an example of a NNSA fee determination letter.)

Our review of DOE documents showed that the Site Specific approach  has a different process for determining incentive and award fees,  depending on whether the fee is tied to objective or subjective  performance criteria. According to agency officials and documents, the  Site Specific approach generally provides more money toward incentive  fees tied to objective criteria than to award fees tied to subjective  criteria—about 60 to 75 percent of available fee money goes to incentive  fees. Incentive fees tied to objective performance criteria are awarded  based on completion of the specific tasks or quantitative targets defined  by the performance criteria. For example, one of the objective  performance criteria for EM’s Waste Isolation Pilot Plant (WIPP) M&O  contractor in fiscal year 2016 was to develop a maintenance and  engineering program, called the Material Condition and Aging  Management Program, and complete certain program activities. EM set  a maximum incentive fee of $500,000 in the PEMP to be awarded upon  completion of the activities.

In regard to award fees that are tied to subjective performance criteria  under the Site Specific approach, offices using this approach take a  similar method to the NNSA approach, in that they determine ratings and  fees simultaneously. Specific portions of an available award fee are  assigned to subjective performance criteria at the beginning of the fiscal  year and documented in the PEMP, and officials then determine the  percentage of fee to award and corresponding ratings from the FAR  award fee categories for each subjective performance criterion. The final  decision on the percentage of the available fee awarded for subjective  performance criteria is made by the Fee Determining Official, who is  generally an on-site official. The overall fee awarded is the sum of the  individual objective incentive fees and subjective award fees. (See apps.  IV and V for examples of how fee is assigned to specific criteria under the  Site Specific Approach.)

With regard to award term, for the SC and NNSA contracts that had  award term as an incentive, the contracts defined the conditions for  receiving it, and those conditions generally included meeting certain  rating thresholds, based on our review of documents from those offices.  For SC, the contractor (1) was to earn at least a 3.5 (A-) science &  technology area rating and a 3.1(B+) management & operations area  rating, and (2) have no individual goal ratings below 3.1(B+) for science &  technology area goals and 2.5 (B-) for management & operations area  goals. The contracting officer is to prepare and submit a standardized  document along with an annual contractor performance evaluation  presentation for review through program officials, and the Director of the  Office of Science is to make the final award term determination. For  NNSA contracts, the contractor generally must (1) earn a rating of “Very  Good” or better in four of the six goals and receive no rating of  “Satisfactory” or lower in any goal, and (2) meet any additional  requirements as specified in the contract.

All DOE Offices but NNSA Have Clearly Documented Approaches, Which May Lead NNSA to Inconsistently Collect and Use Performance Information in Contractor Evaluations

All of DOE’s offices have documented policies outlining their performance  evaluation approaches, and all but NNSA have documented how  information is to be collected and used to make rating determinations.  SC, EERE, NE, FE, and EM have included in their documented policies  and performance evaluation plans detailed procedures for collecting  information on contractors’ performance that outline, among other things,  how officials are to gather input from internal and external stakeholders  and how the officials are to use that information in making rating  determinations. For example, under SC’s Laboratory Performance  Appraisal Process and PEMP Preparation Guidance (SC’s Appraisal  Guidance), stakeholders are to provide evaluations using SC’s web- based information collection tool, the Laboratory Rating Tool, to provide  scores and narratives on contractor performance. As a result, SC’s  contractor performance evaluation approach clearly traces where  performance information comes from and how the information is used in  determining contractors’ final ratings.

Similarly, EM and FE document how officials are to collect information  and use it in PEMPs or other performance evaluation plans. For example,  EM’s PEMP for the WIPP M&O contract provides step-by-step  procedures for how field office officials are to assess contractor  performance against each performance criterion. These procedures guide  the flow of information from contractor to field office officials, who are to  check and validate the information and provide rating and fee  recommendations to the on-site Fee Determining Official. Similarly, field  office officials at EM’s Savannah River Site and FE’s Strategic Petroleum  Reserve also have detailed procedures for assessing and distributing  information regarding performance. Such detailed written procedures can  provide better assurance to agencies that officials are consistently  gathering and using performance evaluation information and that one can  trace the ultimate performance rating in the PER to the underlying  performance information.

In contrast to the detailed documented policies of other DOE offices,  during the period of our review NNSA’s documented policy did not always  match its performance evaluation approach, and the policy did not contain  procedures for how officials should collect and use information so that  one can trace the performance rating to the underlying performance  information. As noted above, NNSA changed from using the Site Specific  performance evaluation approach that focused on objective performance  criteria to the agency’s current approach in fiscal year 2013. However,  NNSA did not update its policy to reflect this change until December  2016. Thus, in fiscal years 2013 through 2016, NNSA was using a  policy intended to evaluate site-specific objective performance criteria and  incentive fees rather than the broad, office-wide subjective performance  criteria that NNSA was using during those 4 fiscal years.

NNSA brought its policy into alignment with its performance evaluation  approach in December 2016 by issuing its Corporate Performance  Evaluation Process for Management and Operating Contractors policy  (NAP-4C). NAP-4C provides a general framework under which NNSA  officials provide input into the contractor performance evaluation process;  the policy also provides a general schedule for implementing the  performance evaluation approach, as well as general references to  information collection.

However, NAP-4C does not include detailed procedures for how  performance information should be collected and used, and according to  NNSA officials, individual NNSA offices and officials determine how they  collect and distribute information. This means information may be  collected inconsistently across the agency, depending on individual  offices’ preferences. For example, NAP-4C states that officials should  “leverag information from contractor assurance systems . . . to monitor  performance” but does not discuss how and when officials should use this  information to ensure performance information is traceable to rating  determinations.

In May 2015, we reported on the importance of tracing performance  information from contractor assurance systems to performance  evaluations. We reported that a senior NNSA official told us NNSA  could not track the extent to which information from contractor assurance  systems was used in evaluating contractor performance because it could  be difficult to identify the sources of information used in performance  evaluations. We recommended that NNSA revise policy, guidance, and  procedures on performance evaluation to fully address how and under  what circumstances those responsible for evaluating M&O contractors’  performance should use information from contractor assurance systems  for this purpose. NNSA concurred with our recommendation and issued  revised policy for contractor oversight but has not yet developed guidance  or procedures for how to use information from contractor assurance  systems in its performance evaluation process. We continue to follow up  on this recommendation.

In addition to NAP-4C, NNSA’s Fee Determining Official issued  implementation guidance for the fiscal year 2016 performance evaluation  cycle. This implementation guidance directs relevant NNSA officials to  follow a series of templates for interim reports to the contractor and  provides the format of the final PER and specific dates for those reports.

The guidance does not include procedures as to how officials throughout  NNSA are to collect or use information to create the content for those  templates. For example, the guidance’s Interim Feedback Report  schedule states that the “program/functional offices provide input to field  offices.” There is no discussion of how the program/functional office is to  provide such input, what types of input are important, or how the input is  to be used. Similarly, NNSA’s PEMPs also do not discuss how officials  should collect or use performance information.

In the absence of documented, detailed procedures, NNSA may not  consistently collect and use performance information from program  managers and field office officials for contracts in a given fiscal year and  may therefore inconsistently apply NNSA’s evaluation process. For  example, we identified two instances in which the NNSA Fee Determining  Official made handwritten changes to proposed award fee amounts  during fiscal year 2012 without documenting in the PER the basis for the  changes, such as by identifying the performance information that would  support the handwritten changes to create traceability between the award  fee amounts and its supporting performance documentation. These  changes awarded (1) Los Alamos National Laboratory’s contractor a year  of award term, even though the contractor had not met the established  rating threshold for award term, and (2) Lawrence Livermore National  Laboratory’s contractor a higher award fee that also qualified the  contractor for award term it otherwise would not have received. With  these changes, these contractors received award terms and fees in a  manner inconsistent with how award terms and fees were assessed for  other M&O contractors. According to NNSA officials, this type of action  would not happen currently because the agency’s approach is rooted in a  policy (NAP-4C) and implementation guide that is supported by a more  collaborative decision-making process. However, even under the new  policy, because NNSA does not have clearly documented procedures  specifying how officials are to collect or use performance information,  NNSA leadership cannot have assurance that there is clear traceability  between the contractor evaluation and its underlying support.

Federal standards for internal control state that management should  design control activities to achieve objectives and respond to risks, such  as by clearly documenting internal control in management directives,  administrative policies, or operating manuals. NNSA has a documented  policy, but this policy does not clearly specify how to collect and use  contractor performance information to evaluate contractor performance.  NNSA officials stated that in their opinion their policy was still effective  and robust without detailed procedures for its implementation. However,  without developing and documenting clear procedures for implementing  NAP-4C that specify the process for collecting contractor performance  information and how officials are to ensure this information can be traced  to rating determinations, NNSA leadership does not have reasonable  assurance that the agency is consistently evaluating contractor  performance and that it is using relevant performance information as  intended.

Evaluation Reports Could Better Assess M&O Contractors’ Cost Performance

DOE’s Fiscal Year 2016 M&O Contractor Performance Evaluation Reports Provided Less Information on Contractors’ Cost Performance than on Other Types of Performance

We found that DOE offices’ fiscal year 2016 PERs provided less  information on M&O contractors’ cost performance—evaluations of the  contractor’s spending, budgeting, strategic sourcing, and costs, including  the contractor’s cost-effectiveness—and provided more information  regarding technical and administrative areas of performance. Specifically,  the PERs were 67 pages long on average and contained about 1 page of  cost performance-related information overall. In contrast, information on  contractors’ technical and administrative performance included in-depth  descriptions of contractors’ scientific discoveries and production progress  that spanned numerous pages. Figure 4 provides typical examples of the  type of technical, administrative, and cost performance descriptions that  we found in our review of fiscal year 2016 M&O contract PERs.

In addition, in our review of the number of performance descriptions in  DOE’s 2016 PERs, we found about 24 percent (179 of 737) of the performance descriptions in the  PERs provided information on cost performance; about 71 percent (524 of 737) provided information on administrative  performance (evaluations of contractor’s performance on mission  support activities, such as information technology, human resources,  legal activities, environmental safety and health, property  management, risk management, and leadership activities); and about 53 percent (390 of 737) provided information on technical  performance (evaluation of contractor’s performance on mission- related activities such as research and development, production,  storage, clean-up, and construction).

Cost Performance Information Included in DOE’s Performance Evaluation Reports is of Limited Use for Acquisition Decision-Making

In addition to providing less information on M&O contractors’ cost  performance than on other areas of performance, the cost information  contained in DOE offices’ PERs is of limited use for acquisition decision- making. DOE’s Information Quality Guidelines define quality, in part, as  information that is useful to DOE and the public. We examined whether  the PERs included such useful information that would permit an overall  assessment of contractor cost performance. FAR and DOE policy call for  such an overall assessment, which therefore is useful to DOE for  acquisition decision-making and to the public generally.

Our analysis showed that the information on contractors’ cost  performance in the PERs did not permit such an assessment of  contractor cost performance for two primary reasons. First, the  information consisted of statements that lacked detail, such as “within  budget,” and did not address the significance of the performance  described. For example, cost performance-related statements such as  “over/under budget” and “cost savings/cost overrun” did not commonly  provide information on the amount saved or lost, making it difficult to  identify the significance of what was reported. Information on cost  effectiveness was also rare—cost-effectiveness information was included  in about 11 percent of the instances in which cost performance was  discussed (48 of 441 instances). Second, cost performance information  commonly applied to specific activities under the contract, such as  construction activities, rather than to achievement of overall operating  efficiencies. When cost performance information is limited to specific  activities, it is not possible to assess a contractor’s overall cost  performance based on information in the PER.

We identified one reason and DOE officials identified three additional  reasons why more cost performance information was not provided in  DOE’s fiscal year 2016 PERs. We believe all of these contribute to why  the cost performance information that was included was often not useful  for acquisition decision-making:

DOE offices’ policies and PEMPs did not specifically require  PERs to include cost performance information and did not  discuss how to ensure that cost information is useful for  acquisition decision-making. Based on our review, DOE offices’  policies did not specifically require that PERs include cost  performance information, nor did they discuss information quality. In  addition, DOE offices’ PEMPs—which serve as a general blueprint for  the type of performance information that offices should include in the  corresponding PER—generally did not include specific cost  performance criteria or explicitly call for evaluations of contractors’  cost performance. In contrast, DOE offices’ fiscal year 2016 PEMPs  commonly included explicit technical and administrative performance  criteria such as: “provide S&T  results with  meaningful impact on the field” (technical) and “provide an efficient  and effective worker health and safety program” (administrative).  There were three exceptions in which PEMPs included specific cost  performance criteria: EM’s WIPP M&O contract, NE’s Idaho National  Laboratory, and FE’s Strategic Petroleum Reserve M&O contract.

Although SC does not have explicit cost performance goals or  objectives, according to SC officials, cost performance is listed as a  factor to consider in SC’s PEMPs’ descriptions of how to evaluate  certain performance criteria. However, SC officials told us that PER  performance descriptions may not include cost information for these  criteria unless there were notable cost overruns or the contractor was  doing an exceptionally good job in these areas. SC officials stated this  is, in part, to keep PERs shorter and streamlined. However, when  PERs are silent on cost performance, there is no formal documented  record of M&O contractor cost performance.

M&O contract missions made it difficult for DOE to assess  contractor cost performance, resulting in less cost performance  information in PERs. According to DOE officials, it is difficult to  assess the costs of the scientific and research missions covered by  many M&O contracts. For example, according to DOE officials, it is  difficult to develop cost estimates for research activities because it is  not always certain when scientific breakthroughs will occur or how  long they will take. DOE uses cost-reimbursement contracts for its  M&O contracts, in part because it is not possible to know with  certainty and in advance how much research and development efforts  will cost or what level of effort will be required.

While we agree that assessing cost performance for scientific and  research activities may be difficult, M&O contractors also carry out a  variety of other activities for which costs may be more readily  assessed. For example, a sizeable portion of the costs under M&O  contracts are for administrative or mission support and other business  operations activities, such as personnel, business processes, human  resources, procurement, and security. In our previous work, we found  that such administrative and support activities accounted for about 25  to 50 percent of M&O contractor costs in fiscal year 2015.Similarly,  SC’s fiscal year 2016 annual laboratory plans identify areas, such as  infrastructure and information systems, as the major cost drivers for  that year. We have found that other agencies assess cost  performance for contractors performing such administrative  activities. DOE officials we interviewed agreed that measuring cost  performance in these areas would be more feasible than measuring it  for its scientific and research missions.

The M&O contract type made it difficult to some degree for DOE  to assess contractor cost performance. According to DOE officials,  certain aspects of how DOE implements cost-reimbursement M&O  contracts create challenges to evaluating cost performance. Some  officials described these challenges as the result of “the budget-based  nature” of M&O contracts. Specifically, according to DOE officials,  M&O contract budgets (the amount contractors are allowed to spend)  are not set up front in the original contract. Rather, according to DOE  officials, M&O contract budgets are commonly determined by the  amount DOE obligates to the contract on an annual basis, based  mostly on annual congressional appropriations to the relevant DOE  programs. Further, these officials noted, because much of DOE’s  appropriated funds are available until expended rather than expiring at  the end of the fiscal year for which they were appropriated, M&O  contractors may be able to carry over those funds to spend in future  fiscal years. According to DOE officials, DOE reviews M&O  contractor estimates when developing its budget request, including  determining how much work is required by its contractors to execute  the program scope outlined in the budget request. Agency officials  also noted that, with regard to cost reimbursement contracts, the  federal government is legally required to reimburse contractors for all  allowable costs up to the approved budget amount.

We have previously reported that cost-reimbursement contracts carry  a high risk for the federal government, resulting in the potential for  cost escalation, as some expenditures may be allowable under the  contract but may not be cost effective. We recognize that M&O  contracts are unique in many ways. Nevertheless, the manner in  which DOE allocates funds to the contract, and the requirement to  reimburse contractors for allowable costs do not, by themselves,  affect DOE’s ability to assess contractor cost performance.

Some cost performance evaluation conducted outside of the  annual performance evaluation process is not included in PERs.  DOE officials told us they perform some activities related to contractor  cost performance outside the performance evaluation process for  M&O contracts, though information on these activities is not always  included in PERs. For example, according to DOE officials, some  M&O contractors participate in group purchasing efforts, where  contractors coordinate purchases to drive up competition and drive  down costs. Also, DOE offices generally monitor M&O contractor  indirect costs to ensure they do not escalate without reason. In  particular, SC’s M&O contractors include a “Cost of Doing Business”  section in their annual laboratory plans, in which SC contractors report  on indirect costs. According to SC officials, SC also uses its reviews  of the Cost of Doing Business sections as opportunities to discuss  options to reduce operational costs. SC officials stated that an internal  process in which SC’s laboratories compete and are awarded work, in  part, also serves to control costs. According to DOE officials, efforts  such as group purchasing and indirect cost monitoring and reporting  are not commonly included in PERs because the agency considers its  existing performance criteria to be sufficiently broad to assess  contractor performance.

Though these efforts may be important to address contractor costs  and information from the efforts could inform assessments of cost  performance, they do not, on their own, represent DOE office’s  evaluation of contractor’s cost performance. In addition, PERs are  important records of DOE offices’ evaluations of contractor  performance because, according to agency officials, DOE uses the  PERs to inform acquisition decisions and help form the basis for a  contractor’s performance record.

We and the DOE Inspector General have identified how important it is for  DOE to obtain quality cost information and use it to evaluate cost  performance. For example, for more than a decade, we have reported  that some DOE offices have experienced challenges obtaining quality  information that could enable the offices to make better-informed  decisions about programs’, and therefore DOE’s, budgetary needs.  Furthermore, we reported in July 2012 that NNSA based much of its  congressional budget request on contractor-generated budget proposals,  which the agency often did not thoroughly evaluate. More recently,  according to a 2017 DOE Inspector General report, challenges in  evaluating cost performance have contributed to NNSA’s and its M&O  contractors’ difficulty in demonstrating the anticipated cost savings for the  NNSA Production Office Sites contract. DOE created this contract by  consolidating the contracts for the Y-12 National Security Complex and  the Pantex Plant into a single contract for the explicit purpose of saving  costs.

While collecting quality information on, measuring, and reporting on cost  performance for M&O contracts may be challenging, this information is  important for two reasons. First, the FAR, DOE policy, and CPARS  highlight the importance of information on contractor’s cost performance  for acquisition decision-making. As we previously noted, the FAR and  DOE policy provide that decisions to extend or compete an M&O contract  be based on an expectation of meaningful improvement in performance  or cost, including consideration of a contractor’s technical, administrative,  and cost performance. In addition, according to DOE officials, they largely  copy information from PERs into the federal government database on  contractors’ past performance, CPARS, which agencies use to inform  their awarding of contracts. CPARS has several performance criteria that  agencies are required to complete, one of which is “cost control.” This is  challenging to address, according to DOE officials, because PERs do not  typically include an explicit evaluation of cost performance even though,  also according to DOE officials, PERs are the primary source of  information entered into CPARs.

Second, as we reported in 2009, there are inherent risks to the  government from cost-reimbursement contracts such as DOE’s M&O  contracts, particularly with cost escalation because the government is  required to pay the contractor’s allowable costs regardless of whether the  contractor completes the work. Because of these risks, we found that  these types of contracts involve significantly more government oversight  than do fixed-price contracts. This is, in part, because the agency needs  to monitor contractor costs to provide a reasonable assurance that  efficient methods and effective cost controls are used.

As we previously noted, FAR, DOE guidance and policy, and CPARS  highlight the importance of quality information on contractor’s cost  performance. In addition, federal standards for internal control state that  management should design control activities to achieve objectives and  respond to risks, such as by clearly documenting internal control in  management directives, administrative policies, or operating manuals.

DOE offices have policies on contractor performance evaluation, but  these do not specifically require that PERs include quality cost  performance information that can be used to make an overall assessment  of cost performance. By updating policies to require inclusion of quality  cost performance information in PERs to enable an overall assessment of  a contractor’s cost performance, DOE offices could strengthen their  oversight of M&O contractor costs. For example, DOE offices could better  inform acquisition decisions such as whether to extend or compete a  contract, complete CPARS with greater ease, inform incentives for  contractor performance, and uncover opportunities for federal cost  savings. This is particularly important given that these cost- reimbursement type contracts carry risks of cost escalation.

DOE Generally Awarded M&O Contractors High Ratings and Most Available Performance Incentives, Except in Cases of Significant Safety or Security Incidents

In reviewing DOE’s M&O contractor performance evaluations for fiscal  years 2006 through 2016, we found the results of the evaluations to  generally include high performance ratings and most available  performance incentives, including a median of 94 percent of available  award and incentive fees. During this time frame, administrative  performance sometimes had lower ratings—though these were balanced  out in overall ratings by strong performance elsewhere—and some safety  issues and accidents resulted in additional fee reductions outside the  performance evaluation process. In fiscal years 2006 through 2016, three  contractors received 50 percent or less of available award fee due to two  significant incidents—a safety and security issue and a major accident.

M&O Contractors Generally Received High Ratings in Fiscal Years 2006 through 2016 and more than 90 Percent of Available Performance Incentives

For the 239 annual M&O contractor evaluations from the 24 DOE contract  rating sites we reviewed, in fiscal years 2006 through 2016, DOE offices  provided award and incentive fees equivalent to the FAR rating  categories of Excellent or Very Good 94 percent of the time. Contractors  at more than half of the 24 contract rating sites (17 of 24) received  award and incentive fee percentages consistent with only Excellent or  Very Good ratings for all fiscal years from 2006 through 2016. As  discussed above, while the precise approaches for determining ratings  and fees vary by DOE office, ratings and fees are directly linked in all  three approaches: Fee is either determined through a formula based on  ratings, or DOE offices determine ratings and fees at the same time.  Differences between rating methodologies across offices and changes in  performance evaluation approaches over time mean directly comparing  ratings requires some caution; however, even acknowledging those  differences, there is a clear trend of a high percentage of award and  incentive fees awarded and high equivalent performance ratings across  sites and years.

From fiscal years 2006 through 2016, DOE also provided its M&O  contractors with a median of 94 percent of their available award and  incentive fees. See Table 3 for the results by FAR award fee rating  category for each contract rating site for this period, and Table 4 for an  analysis of average and median percentages of fees awarded by site.  The amount of fee available, fee as a share of total contract spending,  and the use of other incentives have varied across sites, yet performance  results have been generally similar. Appendixes III through VIII provide  additional details by DOE office.

Contract rating site by DOE office  Office of Energy Efficiency and Renewable Energy  Savannah River Site–Environmental Management  Waste Isolation Pilot Plant  Office of Fossil Energy   Strategic Petroleum Reserve Office  Office of Nuclear Energy  National Nuclear Security Administration  Kansas City National Security Campus  Lawrence Livermore National Laboratory  Los Alamos National Laboratory  Nevada National Security Site   NNSA Production Office Sites  Pantex Plant  Y-12 National Security Complex  Sandia National Laboratories  Savannah River Site–National Nuclear Security  Administration  Thomas Jefferson National Accelerator Facility  94  in annual performance evaluation plans. Fixed fees are set at the inception of the contract and do not  vary for performance.

Of further note from our analysis of the extent to which contractors earned  fees in fiscal years 2006 through 2016:

Contractors for the 24 M&O contract rating sites that included award  fees earned approximately $4.3 billion in total fees over this time.  About three-quarters ($3.4 billion) of the $4.3 billion in fees were  award fees and incentive fees, and the remaining amount was fixed  fees.

NNSA’s M&O contracts represent 68 percent of the fees paid and 55  percent of the total M&O contract spending over this period.

As discussed above, DOE offices provided a median of 90 to 95  percent of available annual award fee to 18 of 24 M&O contract rating  sites. However, six rating sites, all conducting work for NNSA, had  median award fee percentages below 90 percent. Several NNSA sites  had fixed fees in addition to award fees. When including those fixed  fees, the percentage of total fee awarded rises, with median fee  percentages rising above 90 percent for three of the sites.

Contract rating sites rarely received less than 75 percent of available  award fee.

In addition to awarding contractors high percentages of available fees,  DOE offices generally awarded M&O contractors most of the available  award term incentives. Several DOE and contractor officials we  interviewed noted that award term is perhaps the most valuable incentive  from a contractor perspective because an extra year of work on the  contract represents much more revenue for them than fees. SC and  NNSA—the two offices with contracts that had award term—awarded 92  percent of award term years available, or 76 out of 83 possible award  term years. Specifically, SC included award term in seven contracts and  awarded M&O contractors with 95 percent of potential award term years,  and NNSA included award term in four contracts and awarded contractors  with 83 percent of potential award term years (see Table 5 below). Three  of the unearned award term years are attributable to the contractor at Los  Alamos National Laboratory, which also had a fourth award term year that  NNSA revoked retroactively. According to NNSA officials, upon not  earning an award term for the fourth time, Los Alamos’s contractor—in  accordance with the terms of the contract—had all of its award terms  revoked, and NNSA decided to recompete the contract.

Administrative Performance Sometimes Had Lower Ratings, with Some Issues Resulting in Fee Reductions Outside of the Performance Evaluation Process

Within the pattern of high overall performance ratings, ratings for  administrative performance have generally been lower than ratings for  technical performance, and some administrative performance issues— particularly safety issues and accidents—resulted in fee reductions  outside the performance evaluation process, as noted in table 6 below.  For example, since fiscal year 2013, when NNSA adopted common  performance goals across its contract rating sites, about 83 percent of  possible goal ratings (134 of 162) had been rated Very Good or better. Of  the 28 goal ratings below Very Good, 22 (79 percent) were in  administrative goals. In many cases, incidents that led to lower ratings  involved site operations issues, such as in safety and security. Similarly,  the contractors at the 10 SC contract rating sites and one NE contract  rating site also showed generally higher technical performance ratings  with 9 of 11 contract rating sites having higher average technical area  scores than administrative area scores (the two other contract rating sites  had average technical area scores that were about equal to the average  administrative scores).

From our review of DOE documents and discussions with officials, one  factor that may be an important influence in the difference between  technical and administrative scores at SC and NE rating sites is that the  Science and Energy Lab performance evaluation approach does not  incentivize administrative performance above a B+. As discussed above,  contractors generally receive additional award fee for higher ratings, but  under the Science and Energy Lab approach, in the administrative area,  all scores of B+ and above lead to the same amount of award fee.  Therefore, a contractor whose only difference was an administrative  score of B+ versus A+ would receive the same amount of award fee.  According to DOE officials, this structure is meant to encourage  contractors to reinvest cost savings into technical performance rather  than improving administrative systems that already meet expectations.

Relatively low performance in certain areas can be balanced out in overall  ratings by strong performance ratings elsewhere. Of nine occasions since  fiscal year 2013 that an NNSA contractor received at least one  Satisfactory goal rating (below 50 percent), the overall rating for the  contractor remained Good or Very Good, and contractors were provided  the majority of their fees in all but one case (the contractor for Los Alamos  National Laboratory in fiscal year 2014, which we discuss further below).  For example, following the break-in of trespassers and related security  lapses at Y-12 in 2012, NNSA provided the M&O contractor with  Satisfactory ratings in operations in fiscal years 2012 and 2013. However,  Very Good and Excellent ratings in other areas meant NNSA provided an  overall rating of Good to the contractor in those years, and the contractor  received more than 50 percent of available award fees. For SC, in the five  occasions since fiscal year 2006 in which a contractor received at least  one goal rating of C (2.0) or below, overall area scores remained As and  Bs and fees above 75 percent, except for one instance. On that occasion,  Princeton Plasma Physics Laboratory in 2016 received multiple goal  ratings of C, which led to a technical score of C+ and a fee of 68 percent.  This 2016 rating for Princeton Plasma Physics Laboratory is also the only  case from fiscal years 2006 through 2016 of a Satisfactory-level goal  rating in a technical area goal, as the others were all in the administrative  areas of site operations or leadership.

The extent to which a single area of performance affects overall ratings is  influenced by the broad scope of activities under an M&O contract, the  broad types of performance required under the contract, and the weights  used to determine overall ratings and incentives. According to DOE  officials, one way the Science and Energy Lab approach addresses these  factors is to include all the ratings provided by each stakeholder and for  each objective in the PER. In this way, while a C from one stakeholder or  objective may be weighted out overall, the grade and the feedback  associated with it are still provided to the M&O contractor and clearly  visible to readers of the reports.

Another way that DOE offices have addressed individual performance  deficiencies that may get balanced out in overall ratings is through  additional fee reductions. Most offices have reduced fees outside the  performance evaluation process to address specific performance  deficiencies—generally administrative concerns, such as safety issues.  In particular, all offices except EERE have reduced fees that would have  been provided from performance evaluation results, relying on contract  clauses that allow for fee reductions. Such clauses allow DOE offices to  unilaterally reduce fees for the evaluation period if, for example, the  contractor fails to meet performance requirements of the contract relating  to environment, safety, and health. For example, NE used such clauses  in 7 of the 11 years we reviewed to reduce the fee provided to the Idaho  National Laboratory M&O contractor. FE has also frequently used fee  reductions to address issues outside its predominantly objective  performance criteria. SC, NNSA, and EM have also occasionally used  additional fee reductions outside the performance evaluation process. For  all offices, fee reductions generally resulted from administrative  performance issues—safety issues and accidents—rather than technical  performance. These fee reductions ranged from $10,000 to $35 million,  and while the fee received by the contractor was lowered, the original  ratings were not revised. In most cases, these reductions were for 10  percent or less of award and incentive fees provided and less than $1  million dollars; however, they represented large portions of contractors’  fees in a few cases. See Table 6 below for a list of fee reductions.

From Fiscal Years 2006 through 2016, Three Contractors Received 50 Percent or Less of Available Award and Incentive Fees Due to Significant Safety and Security Incidents

Three times in fiscal years 2006 through 2016, M&O contractors received  50 percent or less of available award and incentive fees due to a safety  and security issue at the Lawrence Livermore National Laboratory (LLNL)  in fiscal year 2008 and a major accident involving the WIPP in Carlsbad,  New Mexico, and the Los Alamos National Laboratory (LANL) in fiscal  year 2014.

LLNL, 2008. LLNL’s M&O contractor received 50 percent of the available  award and incentive fees—$15,795,584 out of $31,879,519—due to  weaknesses in environmental management, security, and  management/performance improvement that resulted in Satisfactory  ratings in those respective areas and an overall Satisfactory rating in  operations. In particular, an April 2008 inspection and force-on-force  exercise conducted by DOE’s Office of Health, Safety, and Security found  significant weaknesses in protective force and classified matter protection  and control programs that led to an Unsatisfactory rating in security. The  performance evaluation also reported issues with contractor assurance  system progress, staffing, and “unacceptable” losses of key personnel.  LLNL’s contractor received overall ratings of Outstanding in mission and  Good in institutional management. In addition, the contractor received  $21,862,651 in fixed fees, for a total fee award of $37,658,235.

WIPP, 2014. WIPP’s contractor received 6.9 percent—$561,266 out of  $8,192,895—of the fees available under its contract in fiscal year 2014  due to two unrelated accidents, a truck fire and a waste drum explosion,  that resulted in the suspension of waste disposal at the site—the nation’s  only facility for disposal of transuranic waste. The 6.9 percent of fees  awarded represented an additional reduction of fees from the amounts  the contractor earned for meeting a portion of its objective criteria targets  and receiving Satisfactory ratings in all four subjective criteria. WIPP did  not resume waste disposal operations until 2017.

LANL, 2014. LANL’s contractor received none of the available award fee,  and no DOE fixed fee, in fiscal year 2014 due to its improper oversight  and packaging of the waste drum that exploded at WIPP. Of $63,406,380  in available fee, LANL’s contractor received about $6.3 million in fixed fee  associated with work completed under contract with other federal  agencies that, according to NNSA officials, could not be revoked. Similar  to WIPP’s contractor, this represented an additional reduction of fees  from the amounts that would have resulted from an overall Satisfactory  rating (including an Unsatisfactory for operations and infrastructure;  Satisfactory for science, technology and engineering; Satisfactory for  leadership; and Very Good for the two mission goals). In addition to losing  fee and award term, the waste portion of the LANL contract was  withdrawn from the M&O contract and contracted out separately by EM.

In all three cases, in the year following the 50 percent or less in award  and incentive fees, performance ratings returned to at least Good levels  and contractors received at least three-quarters of available award and  incentive fees. With regard to the WIPP accident involved in two of the  three cases, efforts to recover from the waste drum incident and return to  full operations have cost hundreds of millions to date and are estimated to  cost more than $600 million in total, all of which will be costs to the  taxpayer. The combined unearned and reduced fee for both contractors  amounted to $64,788,464, or about 10 percent of total estimated costs to  the government. In addition to fee reductions, NNSA officials stated that  the WIPP accident played a significant role in NNSA’s decision to not  exercise the last 7 years of possible award term on the LANL contract  and thus recompete the contract in 2018. According to NNSA, those 7  years translate into approximately $17 billion in work and up to $500  million in fee the LANL contractor could have earned. Also, with regard to  additional actions EM took after the accidents at WIPP, according to DOE  officials, EM modified the contract terms from having a single 5-year  option period to five 1-year option periods.

While there are differences in how DOE’s offices approach performance  evaluation of M&O contractors, all of the offices use the annual  performance evaluations of the contractors and the associated rating and  fee determinations to evaluate the extent to which contractors are  operating sites as intended and accomplishing mission work, and to  justify incentives such as fee and additional contract term. These annual  performance evaluations also provide valuable information for contract  management and acquisition decisions, such as whether to renew or  compete expiring M&O contracts. DOE also recognizes the importance of  improving performance evaluation and oversight of contractors.

All of DOE’s offices except NNSA have clearly documented procedures  on how to collect and use information to make rating determinations.

NNSA provides a general framework for its performance evaluations in its  NAP-4C policy but leaves how to collect, distribute, and document  information to the discretion of individual offices and officials. In the past,  NNSA officials have made changes to incentives awarded without  underlying performance documentation to support the change. Without  developing and documenting clear procedures for implementing NAP-4C  that specify the process for collecting contractor performance information  and how officials are to ensure this information can be traced to rating  determinations, NNSA leadership does not have reasonable assurance  that it is consistently evaluating contractor performance and that it is  using relevant performance information as intended.

The cost performance information included in DOE offices’ fiscal year  2016 PERs is of limited use for acquisition decision-making in that this  information does not permit making an overall assessment of M&O  contractors’ cost performance. DOE offices have not required specific  assessment of cost performance in their performance evaluation policies,  nor discussed how to ensure that cost information is useful for acquisition  decision-making. However, the PERs are important sources of  information for contract management—particularly for acquisition  decisions and oversight of spending on cost-reimbursement contracts.  DOE officials identified challenges in evaluating M&O contractors’ cost  performance and ways this evaluation may occur outside of the annual  performance evaluation process. These challenges contribute to why  there is less cost performance-related information in PERs than for other  types of performance. While collecting, measuring, and reporting quality  cost performance information may be challenging, such information is  important for fully assessing contractor performance and managing the  inherent risks of cost-reimbursement contracts. By updating their policies  to require quality cost performance information in PERs to enable an  overall assessment of M&O contractor cost performance, the six DOE  offices with M&O contracts could strengthen their oversight of costs for  contracts worth about $20 billion a year and use this information to  improve acquisition decision-making.

We are making seven recommendations to DOE offices:  The Administrator for the National Nuclear Security Administration should  develop and document clear procedures for implementing NAP-4C,  specifying the process for collecting contractor performance information  and describing how officials are to ensure this information can be traced  to rating determinations. (Recommendation 1)

The Assistant Secretary for the Office of Energy Efficiency and  Renewable Energy should update its policy to require that Performance  Evaluation Reports include quality information on cost performance to  enable an overall assessment of Management and Operating contractor  cost performance. (Recommendation 2)

The Assistant Secretary for the Office of Environmental Management  should update its policy to require that Performance Evaluation Reports  include quality information on cost performance to enable an overall  assessment of Management and Operating contractor cost performance.  (Recommendation 3)

The Assistant Secretary for the Office of Fossil Energy should update its  policy to require that Performance Evaluation Reports include quality  information on cost performance to enable an overall assessment of  Management and Operating contractor cost performance.  (Recommendation 4)

The Administrator for the National Nuclear Security Administration should  update its policy to require that Performance Evaluation Reports include  quality information on cost performance to enable an overall assessment  of Management and Operating contractor cost performance.  (Recommendation 5)

The Assistant Secretary for the Office of Nuclear Energy should update  its policy to require that Performance Evaluation Reports include quality  information on cost performance to enable an overall assessment of  Management and Operating contractor cost performance.  (Recommendation 6)

The Director of the Office of Science should update its policy to require  that Performance Evaluation Reports include quality information on cost  performance to enable an overall assessment of Management and  Operating contractor cost performance. (Recommendation 7)

We provided a draft of this report to DOE for comment. DOE provided us  with written comments, as well as technical comments, which we  incorporated as appropriate. In its written comments, reproduced in  appendix IX, DOE agreed with four of our seven recommendations and  partially agreed with the others.

DOE partially agreed with our recommendations that three DOE offices— EERE, NE, and SC—update their policies to require that PERs include  quality information on cost performance to enable an overall assessment  of M&O contractor cost performance. In its written comments, DOE said  that the three offices have concerns that (1) our report gives the  impression that DOE does not review cost performance of their respective  national laboratories in an adequate manner, and (2) by focusing on the  annual PERs, our report does not capture the cost performance reviews  conducted in day-to-day contract oversight, the annual laboratory  planning process, and contract extend/compete decisions. In its  comments, DOE stated that since EERE, NE, and SC conduct cost  performance reviews in normal operations and at the year-end annual  evaluation process, adequate information is available to assess whether  the contractor cost performance is acceptable to the department.

In the report, we note that DOE conducts some cost performance  evaluation activities outside of the annual performance evaluation  process, although we did not assess these efforts. While there may be  adequate information available, DOE does not commonly document this  information or assessments from such activities in the PERs. We continue  to believe that the PERs are important sources of information for contract  management—particularly for acquisition decisions and oversight of  spending on cost-reimbursement contracts—and that action is needed to  improve these formal records of contractor performance. By not including  quality information on overall cost performance and assessments in  PERs, DOE offices are missing a valuable opportunity to better document  contractors’ cost performance, improve acquisition decision-making, and  strengthen oversight of billions of dollars in contracting. We continue to  believe that it is important for EERE, NE, and SC to implement the  recommendations and that by doing so, these offices would have better  assurance that M&O performance evaluations fully address required  elements.

We are sending copies of this report to the appropriate congressional  committees, the Secretary of Energy, and other interested parties. In  addition, the report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-3841 or bawdena@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix X.

Appendix I: Objectives, Scope, and Methodology

This report reviews the Department of Energy’s (DOE) performance  management of its management and operating (M&O) contracts.  Specifically, it examines (1) how DOE offices evaluated M&O contractor  performance in fiscal years 2006 through 2016 and the extent to which  these offices have documented their evaluation approaches; (2) the  extent to which DOE’s fiscal year 2016 M&O contractor performance  evaluation reports provide information on contractors’ technical,  administrative, and cost performance; and (3) the results of DOE’s M&O  contractor performance evaluations from fiscal years 2006 through 2016.

For all three objectives, we reviewed performance evaluation  documentation—performance evaluation plans, performance evaluation  reports (PERs), fee determinations, award term determinations, and  option term determinations—for 21 of the 22 DOE M&O contracts in place  as of fiscal year 2016, the most recently completed contract year at the  time we initiated our review. We also reviewed documentation for Bettis  and Knolls Atomic Power Laboratories’ M&O contract but excluded it from  our analysis because the contract does not have annual reviews and  ratings comparable to the other DOE M&O contracts. The Bettis and  Knolls contract does not have an award fee and thus NNSA’s Office of  Naval Reactors—the office responsible for overseeing the M&O  contract—does not produce annual PERs similar to those of the other  offices. In addition, we did not include in our scope the DOE contract for  the cleanup of the West Valley Demonstration Project in upstate New  York because it was not an M&O contract in fiscal year 2016; according  to DOE officials, it switched from being an M&O to a non-M&O contract in  fiscal year 2007.

In addition, we also interviewed DOE officials to gain a further  understanding of the department’s performance evaluation processes and  results, including officials at DOE headquarters and at several field offices  that are responsible for providing day-to-day oversight of the activities of  M&O contractors. To provide additional perspective, we interviewed  officials at the Department of Defense, the National Aeronautics and  Space Administration, and the Department of Homeland Security, which  we selected because they also manage government-owned, contractor- operated laboratories and sponsor work at DOE laboratories, sometimes  contributing views incorporated into DOE performance evaluations.

To examine how DOE offices have evaluated M&O contractor  performance, we reviewed DOE’s and DOE offices’ policies and  procedures for performance evaluations, as well as annual performance  evaluation and measurement plans and PERs from fiscal years 2006  through 2016. We also compared each office’s policies and procedures  for conducting performance evaluations against federal standards for  internal control, as well as the Federal Acquisition Regulation (FAR),  DOE’s Acquisition Guide, and the Department of Energy Acquisition  Regulations. In addition, to examine the extent to which these offices  have documented their evaluation approaches, we discussed the  evaluation approaches and processes with DOE officials and compared  those approaches with documented policies and procedures.

To evaluate the extent to which PERs provided information on each of the  performance areas outlined in the FAR—technical, administrative, and  cost—we performed a content analysis of 22 DOE fiscal year 2016 PERs  for M&O contractors. We developed operationalized definitions of each  of the three areas with input from DOE’s offices. Broadly, the  operationalized definition of technical performance included mission- related activities, the operationalized definition of administrative  performance included mission support activities, and the operationalized  definition of cost performance included spending-related activities.  Mission-related activities included, for example, research and  development, production, storage, clean-up, and construction. Mission  support activities included, for example, information technology, human  resources, legal activities, environmental safety and health, property  management, risk assessment, and leadership activities. Cost-related  activities included, for example, spending, budgeting, strategic sourcing,  and costs, including the contractor’s cost-effectiveness. In identifying  information related to cost performance, we considered all evaluative  statements related to cost, including broad terms such as saving, cost,  spending, and budget. Then we categorized performance descriptions  under these three performance areas and counted the number of  performance descriptions that included information in the M&O contracts’  PERs related to each of the areas. A performance description could be  categorized as related to one, two, or all three areas. Two analysts  independently reviewed each PER and then met to agree on the  categorizations. When differences arose, we included a third analyst to  arrive at a consensus.

For the vast majority of M&O contracts, we analyzed the performance  descriptions at the level of objectives—where most performance  descriptions were found—and included notable outcomes described  under those objectives. In a few instances, we used other comparable  units of analysis, such as goals, for some National Nuclear Security  Administration (NNSA) M&O contracts (in which performance information  was provided by goals, not objectives) and criteria for Office of  Environmental Management (EM) and Office of Fossil Energy (FE) (in  which performance information was provided under numerous subjective  and objective criteria). Based on our analysis, we reported the total  number of performance descriptions for each area, as well as the  percentage of performance descriptions that contained information  related to each area. Because performance criteria descriptions could  contain information related to more than one area, the percentages total  more than 100 percent.

To determine the extent of cost performance-related information in DOE’s  fiscal year 2016 PERs for its M&O contracts, we performed a content  analysis. From our analysis, we reported the total number of pages the  cost performance-related information represented, compared with the  average number of total report pages. To determine the number of pages,  we counted the number of pages of each PER.

In addition, to evaluate the quality of cost performance-related  information, we reviewed DOE Information Quality Guidelines, which  apply to information DOE offices make available to the public. We then  performed a content analysis of DOE fiscal year 2016 PERs based on the  definition of quality in the guidelines, which includes that information  generated for DOE and the public be useful. We further analyzed and  categorized the types of cost performance-related information. Types of  cost information included, for example, within budget, over budget, cost  savings, cost overrun, and cost effectiveness. We defined cost  effectiveness as good value for money spent.

To examine the results of DOE’s M&O contractor performance  evaluations from fiscal years 2006 through 2016, we analyzed  performance ratings and incentives awarded in PERs, fee determination  letters, and other performance evaluation documents. Throughout the  report, we analyzed and provided information by “contract rating sites”  rather than individual contractors or physical sites, because the individual  contractors and how certain sites align with the contracts have changed  over time. We analyzed 24 distinct contract rating sites covered by 21  M&O contracts in place as of fiscal year 2016. There are three more  contract rating sites than the number of contracts in 2016: two additional  contract rating sites because two individual contracts were consolidated  into one contract during the period we covered—we analyzed the two  individual contracts from prior to 2014 separately from the current  consolidated contract—and one additional contract rating site because  two DOE offices separately evaluated the performance of a single  contractor that performed activities for each of those offices.

To summarize the results of DOE’s annual contract performance  evaluations, we analyzed overall annual percentages of available award  and incentive fees provided at each contract rating site and presented the  corresponding FAR rating categories. We reviewed performance  evaluation ratings from 239 performance evaluations at the 24 contract  ratings sites. We also did not include ratings from the EM portion of the  Savannah River Site contract for fiscal years 2006 through 2008 because,  according to EM officials and award fee documents, it had multi-year  award fee targets that did not align with individual fiscal years.

We conducted this performance audit from October 2016 through  February 2019 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.

Appendix II: Additional Information on the Department of Energy’s Management and Operating Contracts

Table 7 provides additional information on the Department of Energy’s 22  management and operating contracts, contractors, contract award and  end year, and total spending through these contracts. Table 8 presents  the spending data adjusted for inflation.

Appendix III: Additional Information on the Office of Energy Efficiency and Renewable Energy’s Performance Evaluations

The Office of Energy Efficiency and Renewable Energy (EERE) focuses  on aiding the development and implementation of renewable energy  technologies and improving energy efficiency across various sectors.  EERE administers its management and operating (M&O) contract at the  National Renewable Energy Laboratory (NREL), in Golden, Colorado. As  we describe in our report, EERE follows a Science and Energy Lab  approach to evaluate its M&O contractor’s performance that uses broad,  office-wide performance criteria, which are mostly subjective. Table 9  provides the full list of the goals and objectives EERE used to evaluate its  M&O contractor performance in fiscal year 2016. For the most part, these  performance criteria remained unchanged from fiscal year 2006 through  fiscal year 2016.

As we describe in our report, EERE uses detailed methodologies to  determine ratings and incentives. To illustrate the detailed formulas and  calculations involved, Figure 5 provides an example of how ratings and  fees are calculated.

Table 11 shows the rating scores the contractors earned for Mission and  Operations goals.

Figure 6 shows the annual total fee (both award fee and fixed fee) EERE  provided to its M&O contractors for fiscal years 2006 through 2016.

Table 12 provides the percentage of available award and incentive fees  provided to the M&O contractors for fiscal years 2006 through 2016 by  contract rating site.

Appendix IV: Additional Information on the Office of Environmental Management’s Performance Evaluations

The Department of Energy’s Office of Environmental Management (EM)  is responsible for decontaminating and decommissioning facilities and  sites that are contaminated from decades of nuclear weapons production  and nuclear energy research. EM has two management & operating  (M&O) contract sites:  the Savannah River Site (SRS) in Aiken, South Carolina; and  the Water Isolation Pilot Plant (WIPP) in Carlsbad, New Mexico.

As we describe in our report, EM follows a Site Specific approach to  evaluate its M&O contractors that uses detailed performance criteria  specific to each contract. Under this approach, most performance criteria  we reviewed are objective criteria, and a few are broader, subjective  criteria. Tables 13 through 16 provide examples of some of the specific  criteria EM used at each site. We provide examples rather than a full list  because each site has numerous individual metrics, which are often quite  technical. Specifically, Tables 13 and 14 provide examples of EM’s  objective performance criteria, which are defined based on quantifiable  metrics (e.g., a contractor’s demonstrated waste processing rate) and  milestones (e.g., whether a contractor completed a task on or before a  scheduled date). Table 13 includes 3 of the 6 objective performance  criteria EM used to evaluate the SRS contractor’s performance during  fiscal year 2016. Table 14 contains examples of 3 of the 9 objective  criteria EM used to evaluate the WIPP contractor in fiscal year 2016.

Tables 15 and 16 provide examples of EM’s subjective criteria, which are  used for aspects of performance that may be difficult to capture  objectively. Table 15 provides examples of 3 of the 12 subjective criteria  for evaluating the SRS M&O contractor’s performance during fiscal year  2016, while Table 16 contains the fiscal year 2016 subjective criteria for  evaluating the WIPP M&O contractor’s performance.

The following tables and figure provide details on the incentives available  to and earned by EM’s M&O contractors from fiscal year 2006 through  fiscal year 2016. Table 17 shows the performance incentives that EM  included in its M&O contracts. We use the term “contract rating sites”  rather than individual contractors or physical sites, because the individual  contractors and how certain sites align with the contracts may have  changed over time.

Figure 7 shows the annual total fee (both award fee and fixed fee)  provided to EM M&O contractors for fiscal years 2006 through 2016.  Because EM and National Nuclear Security Administration activities at the  Savannah River Site are rated separately, only the EM portion of fees is  represented below.

Table 18 provides the percentage of available award and incentive fees  EM’s M&O contractors earned for fiscal years 2006 through 2016.

Appendix V: Additional Information on the Office of Fossil Energy’s Performance Evaluations

The Department of Energy’s Office of Fossil Energy (FE) manages the  nation’s Strategic Petroleum Reserve (SPR), which consists of salt  caverns storing crude oil in Texas and Louisiana. As we describe in our  report, FE follows a Site Specific approach to evaluate its M&O  contractors that uses detailed performance criteria specific to each  contract. Under this approach, most performance criteria we reviewed are  objective criteria, and a few are broader, subjective criteria. Table 19  provides examples of FE’s objective performance criteria, which are  defined based on quantifiable metrics (e.g., the contractor’s demonstrated  oil drawdown rate) and performance targets (e.g., whether a contractor  completed a task on or before a scheduled date). Table 19 includes 4 of  the 11 objective performance criteria FE used to evaluate the Strategic  Petroleum Reserve Office (SPRO) contractor’s performance during fiscal  year 2016. We provide examples rather than a full list because there were  numerous individual metrics, which are often quite technical.

Table 20 contains the full list of FE’s subjective performance criteria— which FE uses for aspects of performance that may be difficult to capture  objectively—for evaluating the SPRO M&O contractor’s performance  during fiscal year 2016.

Table 21 shows the performance incentives that FE included in its M&O  contract.

Figure 8 shows the annual total fee (both award fee and fixed fee) FE  provided to its M&O contractors for fiscal years 2006 through 2016.

Table 22 provides the percentage of available award and incentive fees  provided to M&O contractors for fiscal years 2006 through 2016.

Appendix VI: Additional Information on the National Nuclear Security Administration’s Performance Evaluations

The National Nuclear Security Administration (NNSA), a separately  organized agency within DOE, is responsible for maintaining and  enhancing the safety, reliability, and performance of the nation’s nuclear  weapons stockpile, promoting international nuclear safety and  nonproliferation, and supporting U.S. leadership in science and  technology. NNSA administers management and operating (M&O)  contracts at eight national laboratories, plants, and sites:

Bettis and Knolls Atomic Power Laboratory in West Mifflin,  Pennsylvania, and Niskayuna and West Milton, New York

Kansas City National Security Campus in Kansas City, Missouri

Lawrence Livermore National Laboratory in Livermore, California

Los Alamos National Laboratory in Los Alamos, New Mexico

Nevada National Security Site near Las Vegas, Nevada

Y-12 National Security Complex in Oak Ridge, Tennessee

Sandia National Laboratories in Albuquerque, New Mexico

Savannah River Site in Aiken, South Carolina  As we describe in our report, NNSA follows an approach to evaluate its  M&O contractors that uses broad, office-wide performance criteria that  are mostly subjective. Table 23 provides the full list of the goals and  objectives NNSA used to evaluate its M&O contractors’ performance in  fiscal year 2016. While there have been some language amendments,  overall, goals and objectives have remained the same from fiscal year  2013 through fiscal year 2016.

As we describe in our report, under the NNSA approach, goals are  assigned specific portions of the available award fee for each contract at  the beginning of the fiscal year—and at the end of the fiscal year, officials  determine ratings and fees at the same time in a collaborative meeting  with NNSA leadership. Figure 9 provides an example of award fee  amounts assigned to individual goals.

Table 24 shows the performance incentives that NNSA included in its  M&O contracts. We use the term “contract rating sites” rather than  individual contractors or physical sites, because the individual contractors  and how certain sites align with the contracts have changed over time.

Specifically, NNSA consolidated its Y-12 National Security Complex and  Pantex Plant contracts into the National Production Office Sites contract  in fiscal year 2014, and NNSA and the Office of Environmental  Management separately evaluated their respective activities carried out  by the Savannah River Site contractor.

Table 25 provides annual performance ratings by goal for fiscal years  2013 through 2016 for each NNSA contract rating site.

Figure 10 shows the annual total fee (both award fee and fixed fee)  provided to NNSA M&O contractors for fiscal years 2006 through 2016 by  contract rating site.

Table 26 provides the percentage of available award and incentive fees  provided to M&O contractors for fiscal years 2006 through 2016 by  contract rating site.

Appendix VII: Additional Information on the Office of Nuclear Energy’s Performance Evaluations

The Office of Nuclear Energy’s (NE) primary mission is to advance  nuclear power as a resource capable of making major contributions in  meeting U.S. energy supply, environmental, and energy security needs.  NE administers its management and operating (M&O) contract at the  Idaho National Laboratory (INL), in Idaho Falls, Idaho. As we describe in  our report, NE follows a Science and Energy Lab approach to evaluate its  M&O contractor that uses broad, office-wide performance criteria that are  mostly subjective. Table 27 provides the full list of the goals and  objectives NE used to evaluate its M&O contractor performance in fiscal  year 2016. For the most part, these performance criteria have remained  unchanged from fiscal year 2007 through fiscal year 2016.

As discussed above, NE uses detailed methodologies to determine  ratings and incentives. To illustrate the detailed formulas and calculations  involved, Figure 11 provides an excerpt from a fee determination letter as  an example of how ratings and fees are calculated.

Table 28 shows the performance incentives that NE included in its M&O  contract.

Table 29 shows the rating scores the contractor earned for Mission and  Operations goals.

Figure 12 shows the annual total fee (both award fee and fixed fee)  provided to NE’s M&O contractor for fiscal years 2006 through 2016.

Table 30 provides the percentage of available award and incentive fees  provided to M&O contractor for fiscal years 2006 through 2016.

Appendix VIII: Additional Information on the Office of Science’s Performance Evaluations

The Office of Science (SC) supports scientific research for energy and the  physical sciences both by directly supporting such research, for example,  through grants to and cooperative agreements with universities, and by  supporting the development, construction, and operation of scientific user  facilities. SC administers management and operating (M&O) contracts at  10 national laboratory sites:

Argonne National Laboratory in Argonne, Illinois

Brookhaven National Laboratory in Upton, New York

Fermi National Accelerator Laboratory in Batavia, Illinois

Lawrence Berkeley National Laboratory in Berkeley, California

Oak Ridge National Laboratory, in Oak Ridge, Tennessee

Pacific Northwest National Laboratory in Richland, Washington

Princeton Plasma Physics Laboratory in Princeton, New Jersey

SLAC National Accelerator Laboratory in Stanford, California

Thomas Jefferson National Accelerator Facility in Newport News,  As we describe in our report, SC follows a Science and Energy Lab  approach to evaluate its M&O contractors that uses broad, office-wide  performance criteria that are mostly subjective. Table 31 provides the full  list of the goals and objectives SC used to evaluate its M&O contractors’  performance in fiscal year 2016. Generally, these performance criteria  remained mostly unchanged from fiscal year 2006 through fiscal year  2016.

As discussed above, SC uses detailed methodologies to determine  ratings and incentives. To illustrate the detailed formulas and calculations  involved, Figure 13 provides excerpts from a performance evaluation  report as an example of how ratings and fees are calculated.

The following tables and figure provide details on the incentives available  to and earned by SC’s M&O contractors from fiscal year 2006 through  2016. Table 32 shows the performance incentives that SC included in its  M&O contracts. We use the term “contract rating sites” rather than  individual contractors or physical sites, because the individual contractors  and how certain sites align with the contracts may have changed over  time.

Table 33 shows the rating scores the contractor earned for the Science  and Technology goals and Maintenance and Operations goals, by  contract rating site.

Figure 14 shows the annual total fee (both award fee and fixed fee) SC  M&O contractors earned for fiscal years 2006 through 2016 by contract  rating site.

Table 34 provides the percentage of available award and incentive fees  SC’s M&O contractors earned for fiscal years 2006 through 2016 by  contract rating site.

Under the award term incentive, some SC M&O contractors are able to  earn one additional year of performance under the contract for each year  they exceed certain thresholds in their annual performance evaluations.  Table 36 shows award term results for fiscal years 2006 through 2016 by  contract rating site.

Appendix IX: Comments from the Department of Energy

Appendix X: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Quindi Franco (Assistant  Director), Ryan Gottschall (Analyst in Charge), Danny Baez, and Diantha  Garms made key contributions to this report. Also contributing to this  report were John Delicath, Brenna Derritt, Cindy Gilbert, Timothy  Guinane, Rich Johnson, Danny Royer, Kiki Theodoropoulos, and Tatiana  Winger.

The Department of Energy depends on the expertise of firms, universities, and others to do much of its work, including managing and operating nuclear weapons labs. In 10 years, it spent about $193 billion on management and operating contracts.

DOE generally gave these contractors high ratings and awarded over 90% of available performance incentives. However, we found DOE could do a better job reporting on how well contractors spend these federal funds.

We made 7 recommendations, including that DOE report more useful information on cost performance.

DOE remains on our High Risk List for inadequate contractor management and oversight.