Coast Guard: Actions Needed to Improve Data Quality and Transparency for Reporting on Mission Performance and Capital Planning

The Coast Guard, a component of DHS, serves as the principal federal agency responsible for maritime safety, security, and environmental stewardship in U.S. ports and waterways. To ensure that the Coast Guard is effectively fulfilling its missions, agency managers must have accurate information and base decisions on sound analyses for effective program management.

This statement discusses Coast Guard actions needed to (1) improve the quality of data used for program management and (2) improve the transparency of its data for reporting on mission performance and planning.

This statement is based on relevant products GAO issued from June 2014 through December 2017 on Coast Guard strategic planning and management issues, as well as related recommendation follow-up conducted through February 2018. GAO reviewed applicable laws, regulations, policies and guidance. GAO also interviewed Coast Guard officials responsible for administering these programs and obtained information on how they used data to inform decisionmaking. GAO interviewed a range of stakeholders, including federal and industry officials.

GAO's prior work recommended multiple actions to improve the Coast Guard's program management by improving the quality of data it uses to manage and report on its mission performance. Specifically, GAO recommended actions such as collecting more complete data and clarifying the data limitations to facilitate more effective program management. For example, in December 2017, GAO found that more information is needed to calculate vessel safety statistics that could enhance the Coast Guard's knowledge about accident, injury, and fatality trends involving commercial fishing vessels. Having more complete information could be useful to carrying out its marine safety mission, and GAO recommended, among other things, that the Coast Guard ensure that data collected during commercial fishing vessel incident investigations is accurately captured. In 2018, the Coast Guard reported taking initial steps to capture more accurate data.

GAO's prior work also identified areas where the Coast Guard could improve the transparency of the data it uses for reporting on its mission performance as well its capital planning purposes. For example, in an October 2017 report on performance goals, GAO found the Coast Guard's Annual Performance Report (APR) has not been released publicly since 2011. Consequently, there has not been full visibility over performance across all of the Coast Guard's missions. Coast Guard officials stated that a decision by Department of Homeland Security (DHS) leadership to limit the number of performance goals shared publicly had deterred the Coast Guard from public release of its APR. GAO recommended that APRs be available on the Coast Guard's website; the Coast Guard plans to publicly release future APRs. In addition, previous GAO reports found that the Coast Guard's annual 5-year capital investment plan, which projects acquisition funding needs for the upcoming 5 years, did not consistently reflect current total cost estimates or the effects of tradeoffs made as part of the annual budget cycle. GAO made recommendations to help the Coast Guard plan for future acquisitions and the difficult trade off decisions it will face given funding constraints. The Coast Guard agreed, but it is unclear when it will complete the 20-year plan.

GAO is not making new recommendations in this statement but has made them to the Coast Guard and DHS in the past on improving its program management through, among other things, better quality and more transparent data. DHS and the Coast Guard agreed with these recommendations and reported actions or plans to address them.

I am pleased to be here today to discuss U.S. Coast Guard (Coast  Guard) actions that could improve its program management by improving  the quality and transparency of the data it uses to manage its mission  performance and capital planning. The Coast Guard, within the  Department of Homeland Security (DHS), serves as the principal federal  agency responsible for maritime safety, security, and environmental  stewardship in U.S. ports and waterways. The Coast Guard’s 11 primary  statutory missions identified in the Homeland Security Act of 2002, as  amended, encompass the agency’s important roles and responsibilities.  For example, as part of its marine safety mission, the Coast Guard  performs mandatory safety inspections, conducts accident investigations,  and promotes accident prevention involving vessels at sea, including for  commercial fishing vessels.

To ensure that the Coast Guard is effectively fulfilling its missions, agency  managers must have accurate and reliable mission performance  information to monitor and track the progress the agency is making  toward achieving its goals. We have previously reported that evaluation  can play a key role in strategic planning and program management, and  that findings supported by sound analyses strengthen decision making.  Our findings are consistent with Office of Management and Budget (OMB)  guidance on assessing an agency’s performance, which emphasizes the  need for frequent, data-driven reviews, including data analyses, and  identifying steps necessary to increase agency performance. However,  we have previously raised questions about whether annual performance  goals and reported performance information accurately reflect the extent  to which the Coast Guard is accomplishing its mission goals. We have  also raised concerns about the Coast Guard’s management of its capital  asset acquisitions given the impact these assets can have on the Coast  Guard’s ability to conduct some of its missions.

In 2014, we reported that better information on performance and  funding was needed to address Coast Guard acquisition shortfalls.

In 2016, we reported that the Coast Guard did not provide field units  with realistic strategic goals to allocate their limited resources, and  testified on Coast Guard actions to improve its asset allocation  process.

In 2017, we issued five reports and testimonies that collectively  underscore the importance of collecting good data and addressing its  limitations and being transparent about it for the benefit of  congressional decisionmakers and the public.

My testimony today describes some of the findings and recommendations  from these reports. Specifically, I will discuss Coast Guard actions  needed to (1) improve the quality of data used for program management  and (2) improve the transparency of data used for reporting on its mission  performance and capital planning.

This statement is based on eight products we issued from June 2014  through December 2017 on Coast Guard strategic planning and  performance management, and recommendation follow-up activities  conducted through February 2018. To perform the work for our previous  reports, among other things, we reviewed applicable laws, regulations,  policies and guidance for selected performance goals. We interviewed  Coast Guard officials responsible for administering these programs and  obtained information on their processes for ensuring data reliability,  including performing electronic testing for obvious errors in accuracy and  completeness. We determined that data were sufficiently reliable for  reporting some information, but in other instances, data reliability  limitations precluded us from reporting information, which we describe  later in this statement. We also conducted site visits and interviewed a  range of other stakeholders, including federal officials and industry  representatives. Further details on the scope and methodology for the  previously issued reports are available within each of the published  products. In addition, after the issuance of our reports and through  February 2018 we contacted the Coast Guard to obtain updated  information and documentation, as appropriate, on the status of  recommendations we made.

We conducted the work on which this statement is based in accordance  with generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe the evidence obtained provides  a reasonable basis for our findings and conclusions based on our audit  objectives.

With a staff of over 47,000 members, the Coast Guard operates a  multimission fleet of 201 fixed and rotary-wing aircraft and over 1,400  boats and ships. Operational control of surface and air assets is divided  into two geographic Areas (Pacific and Atlantic), within which are nine  Districts consisting of 37 sectors and the stations within them. The Coast  Guard’s program oversight, policy development, and personnel  administration are carried out at the Coast Guard’s headquarters. As  shown in table 1, the Coast Guard is responsible for 11 statutory missions  identified in the Homeland Security Act of 2002, as amended. The  Coast Guard manages these missions through six mission programs, also  listed in table 1. As part of its marine safety mission, for example, the  Coast Guard conducts, among other activities, safety inspections and  vessel accident investigations, including those involving commercial  fishing vessels, which are part of an industry with one of the highest death  rates of any industry in the United States.

For each of its 11 missions, the Coast Guard has developed goals and  targets to assess and communicate agency performance. The Coast  Guard’s performance assessment process also includes identifying  performance gaps and implementing corrective actions to address unmet  performance goals. As part of its process, the Coast Guard is to establish  targets for the current and subsequent 2 fiscal years, according to Coast  Guard officials. Each target is set by the Coast Guard, but according to  the Coast Guard’s Annual Performance Report (APR), some are derived  from external factors. For example, DHS requires that Coast Guard set a  100 percent target for the percent of people in imminent danger saved in  the maritime environment. Further, several of the Coast Guard’s assets  used to conduct these missions are approaching the end of their intended  service lives. As part of its efforts to modernize its assets used to carry  out various missions, the Coast Guard has begun acquiring new vessels,  such as the National Security Cutter, the Fast Response Cutter as well as  other assets. However, concerns surrounding the affordability of this effort  remain as the Coast Guard continues to pursue multiple new acquisitions  without long-term planning to guide the affordability of its acquisition  portfolio. Figure 1 shows the Coast Guard’s Fast Response Cutter and  National Security Cutter.

Coast Guard Actions that Could Improve Data Quality

We previously reported on actions the Coast Guard could take to ensure  that, among other things, it addresses limitations posed by incomplete  data, the use of unrealistic asset performance data, and limitations with  its performance goal data, for more effective program management.  Examples of data limitations that we have recommended that the Coast  Guard take action on are below.

Improve completeness of mission data. In December 2017, we found  that several different federal agencies play a role in overseeing and  promoting commercial fishing vessel safety, including the Coast Guard.  As part of its marine safety activities, the Coast Guard conducts, among  other activities, safety inspections and vessel accident investigations.  Commercial fishing has one of the highest death rates of any industry in  the United States and vessel disasters are the leading cause of fatalities  among fishers, according to the National Institute for Occupational Safety  and Health. However, our December 2017 review found that more  information is needed to calculate vessel safety statistics that could  enhance the Coast Guard’s knowledge about accident, injury, and fatality  trends involving commercial fishing vessels.

The Coast Guard collects some data on commercial fishing vessels that  operate in federal waters—including a vessel’s length and construction  date—but data on the population of the active U.S. commercial fishing  vessel fleet are not complete. Between 2006 and 2015, the Coast Guard  investigated 2,101 commercial fishing vessel accidents that were  identified as occurring in federal waters. While the number of accidents  generally increased over this time period, the number of injuries and  fatalities declined over the same 10-year period. However, we could not  assess the number of accidents, injuries, and fatalities by fishery— meaning the area in which a certain type of fish (e.g., shrimp, salmon,  crab) is caught—because the Coast Guard’s data is not complete.  Further, we were unable to calculate the rates of commercial fishing  vessel accidents, injuries, and fatalities, because reliable data on certain  information needed to do so—including the total number of vessels that  are actively fishing and the fishery or region in which the vessel  operates—are either not maintained or are not collected by the Coast  Guard or other federal agencies. Having this information could be useful  to carrying out the Coast Guard’s marine safety mission, which includes  enforcing laws to prevent death, injury and property loss in the marine  environment.

We recommended in our December 2017 report that the Coast Guard  ensure that data it collects during commercial fishing vessel incident  investigations is accurately captured. We also recommended that the  Coast Guard work with stakeholders to form a working group to determine  an efficient means to establish a reliable estimate of the population of  active commercial fishing vessels. The Coast Guard agreed with both  recommendations, and in February 2018 informed us that it is in the  process of developing additional data fields to capture more information,  such as the fishery in which the commercial fishing is involved, and is  engaging stakeholders to establish an appropriate working group. We will  continue to monitor these actions.

Use more realistic asset performance data. In our May 2016 report on  Coast Guard strategic planning, we found that the Coast Guard did not  provide field units with realistic goals for allocating assets, by mission.  We reported that the Coast Guard’s strategic allocations of assets were  based on unrealistic assumptions about the performance capacity of its  assets and did not reflect asset condition and unscheduled maintenance.  This was due, in part, to the Coast Guard not including information from  its field units on the actual performance of its assets. For example,  agency officials noted that one of its classes of cutters was 50 years old  and these cutters were hampered by mechanical failures requiring  emergency dry dock repairs, which resulted in reduced availability to  carry out their missions during the year. In another example, a field unit  stated that based on historical use, it planned for 575 hours per vessel for  one type of cutter instead of the 825 hours performance capacity.

Because actual asset use has consistently fallen below asset  performance capacities, there is not a direct alignment between the Coast  Guard’s strategic operational goals and its prospects for achieving those  goals. As a result, the headquarters’ strategic intent, which is based on  asset capacity rather than actual performance, did not provide the field  with strategic, realistic goals for allocating assets by mission. Agencies  should use quality information that is appropriate, current, complete,  accurate, accessible, and timely to achieve objectives and address  related risks. We recommended that the Coast Guard incorporate field  unit input, such as information on assets’ actual performance, to inform  more realistic asset allocation decisions. The Coast Guard concurred with  this recommendation, and in February 2018 informed us that it plans to  address this recommendation through changes to two process  documents that are under revision, with an expected completion date in  March 2018.

Improve performance goal data. In our October 2017 review of Coast  Guard performance goals, we reported that the Coast Guard and DHS  identified limitations with two of the seven selected performance goals we  reviewed, including the five year average number of recreational boating  deaths and injuries. In particular, officials believe that many recreational  boating injuries that do not require hospitalization are not reported to the  Coast Guard. These officials believe that the amount of underreporting  may vary over time due to changes in industry trends, making it difficult to  accurately determine actual injury rates and program performance. We  determined that the data for this performance goal was not sufficiently  reliable for the purposes of our reporting objectives due to these likely  limitations. We found that the Coast Guard did not report the possible  extent of these limitations with this performance goal in its fiscal year  2016 APR.

For the other performance goal, the Coast Guard and DHS identified  limitations with the number of detected incursions of foreign fishing  vessels violating U.S. waters, which is publicly reported in DHS’s APR.  DHS’s review of this performance goal, reported in August 2015, raised  questions about the validity of this goal—that is, whether it provides a  useful measure of the Coast Guard’s performance. Specifically, the  review noted that this performance goal is intended to measure a  deterrence effect, but doing so is inherently difficult and may lead to  contradictory interpretations of performance. In October 2017, we found  that the data for this performance goal was sufficiently reliable for our  reporting objective purposes, but questions remain about its validity.  Reliable data is not a useful indication of performance unless it is also a  valid representation of the goal being addressed. DHS officials reported  that they did not include a discussion of the limitations for this  performance goal in DHS’s fiscal year 2015 APR because the  performance goal met the minimum threshold for data reliability despite  the goal’s limitations. Coast Guard officials reported they were aware of  these limitations and were working with DHS and OMB to improve the  performance goal and implement corrective actions within 1 to 2 years.  We recommended that the Coast Guard assess the extent to which  documentation on performance data reliability contains appropriate  information on known data reliability limitations and update these  documents, as needed, based on the results of the assessment. The  Coast Guard concurred and in February 2018, informed us that it had  taken initial actions to address our recommendation. However, our  preliminary review of these actions indicates that further action will be  needed to fully address our recommendation, such as documenting and  reporting the limitations of performance data.

Coast Guard Actions that Could Improve Transparency of Data for Reporting on Its Mission Performance and Capital Planning

Our previous reports have identified areas in which the Coast Guard  could improve the transparency of its data used for reporting on its  mission performance and planning.

Improve transparency of data on mission performance. In our  October 2017 report on performance goals, we found that the Coast  Guard’s APR has not been released publicly since 2011 due to a previous  DHS leadership decision. Consequently, there has not been full visibility  over performance across all of the Coast Guard’s missions. For  example, one of the Coast Guard’s missions—defense readiness—has  no goals that are publicly reported or shared with Congress, even though  measures related to defense readiness are included in the Coast Guard’s  APR.

Coast Guard officials stated that they could see the benefit of publicly  releasing their APR; however, DHS’s decision to limit the number of  performance goals shared publicly has so far deterred the Coast Guard  from pursuing the public release of its APR. DHS officials told us that the  department is concerned about conflicting information that a component’s  APR might present because it is vetted and produced separately from the  DHS APR. However, the lack of transparency regarding performance  data shared publicly and with Congress can result in an incomplete  picture of mission performance and can limit effective oversight of Coast  Guard operations. As a result, the public and Congress may be unable to  determine the extent to which the Coast Guard is meeting its missions.  We recommended that future Coast Guard APRs be available on the  Coast Guard’s public website. The Coast Guard concurred with this  recommendation and in February 2018, the Coast Guard informed us that  it had completed its 2017 APR and are determining an appropriate  approach for making it publicly available.

Improve capital planning transparency. In our previously issued work  on the Coast Guard’s annual 5-year capital investment plan (CIP), we  found that the CIP does not consistently reflect current total cost  estimates or the effects of tradeoffs that are made as part of the annual  budget cycle. We made several recommendations in recent years  intended to help the Coast Guard plan for future acquisitions and the  difficult tradeoff decisions it will likely face. The Coast Guard generally  concurred with these recommendations and is in various stages of  implementation. For example, in 2017 we reported that we have made  recommendations that DHS and the Coast Guard take several actions to  gain an understanding of what the Coast Guard needs to meet its mission  within its likely acquisition funding levels. These recommended actions  included the Coast Guard: (1) conducting a comprehensive portfolio  review across all its acquisitions to develop revised baselines that meet  mission needs and reflect realistic funding scenarios and (2) developing a  20-year plan that identifies all necessary recapitalization efforts and any  fiscal resources likely necessary to complete these efforts. For example,  in 2014 we recommended the Coast Guard develop a 20-year fleet  modernization plan that identifies all acquisitions needed to maintain the  current level of service and the fiscal resources needed to acquire them.  Without these efforts, the Coast Guard will continue, as it has in recent  years, to plan its future acquisitions through the annual budgeting  process, an approach that has led to delayed and reduced capabilities.

In 2016, the Coast Guard revised its 2005 Mission Needs Statement,  which provides a basic foundation for long-term investment planning that  is to serve as the basis for evaluating the effectiveness of various fleet  mixes, and inform the Coast Guard’s CIP. However, the 2016 Mission  Needs Statement did not identify specific assets the Coast Guard needs  to achieve its missions, nor did it update the annual hours it needs from  each asset class to satisfactorily complete its missions. The Coast  Guard has stated it is developing a 20-year Long-term Major Acquisition  Plan, but it has not stated when the plan will be completed or what will be  included in this plan, such as potential trade-offs that could be made  across the Coast Guard’s portfolio of acquisitions to better meet mission  needs within realistic funding levels. A long-term plan with a tradeoff  analysis would facilitate a full understanding of the affordability challenges  facing the Coast Guard while it builds the Offshore Patrol Cutter. DHS  concurred with our 2014 recommendation, but it is unclear when the  Coast Guard plans to complete the 20-year plan.

Chairman Hunter, Ranking Member Garamendi, and members of the sub- committee, this completes my prepared statement. I would be happy to  respond to any questions you may have at this time.

If you or your staff members have any questions about this testimony,  please contact Nathan Anderson at (202) 512-3841 or  andersonn@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this  statement. Other individuals making key contributions to this work, and  the underlying reports on which it is based, include Dawn Hoff (Assistant  Director); Andrew Curry (Analyst-in-Charge); Chuck Bausell; David Bieler;  Richard Cederholm; John Crawford; Timothy J. DiNapoli; Michele Fejfar;  Laurier R. Fish; Peter Haderlein; Eric Hauswirth; Laura Jezewski; Tracey  King; Benjamin Licht; Marie A. Mak; Gary Malavenda; Diana Moldafsky;  Heidi Nielson; Meg Ullengren; and Kayli Westling.

This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.

In the past, we've reported on ways to improve Coast Guard programs, performance, and capital planning by increasing data quality and transparency.

For example, in December 2017, we found that the Coast Guard needs more information to calculate commercial fishing vessel safety statistics. Knowing more about related accidents, injuries, and fatalities could help the Coast Guard accomplish its maritime safety mission.

We testified before Congress on this and other data issues affecting the Coast Guard.


Nuclear Regulatory Commission: Additional Action Needed to Improve Process for Billing Licensees

NRC is responsible for regulating the commercial nuclear industry, including nuclear power plants. NRC provides services, such as inspections, for regulated entities that hold licenses—that is, licensees. NRC recovers the costs for these services by assessing fees and billing licensees quarterly. In fiscal year 2016, NRC collected about $321 million in service fees. From 2006 to 2016, audits of NRC's fees identified problems with NRC's billing process. For example, a 2012 audit identified about $24 million in unbilled fees from fiscal years 2011 and 2012.

GAO was asked to review NRC's billing process for service fees. This report examines (1) the actions NRC is taking to address problems with its billing process identified by internal reviews and (2) the challenges selected licensees identified with NRC's billing process and the extent to which NRC's actions are addressing them.

GAO reviewed audits of NRC's billing process and other documents related to this process. GAO also interviewed NRC staff and a nongeneralizable sample of 13 licensees, selected based on the amount of service fees charged from October 2015 through July 2017, and compared NRC's actions against criteria on internal controls and project planning.

The Office of the Inspector General for the Nuclear Regulatory Commission (NRC) and internal reviews conducted by NRC identified several problems with the agency's billing process, and NRC has implemented or plans to implement several changes to address the recommendations. For example, the codes that NRC staff use to record their work hours on time cards—referred to as activity codes—did not describe the work and did not have a consistent naming convention, which increased the risk of staff charging their time to the wrong activity codes. This could lead, in some cases, to billing errors. To address these problems, NRC created a standard naming convention for activity codes that provides more information about the activity. See the figure below for the steps in NRC's billing process for work that NRC or contractor staff performed.

Some of the 13 licensees that GAO interviewed identified challenges with NRC's billing process, including its method for delivering paper invoices by mail. For example, two of these licensees stated that with invoices taking up to 10 days to arrive in the mail, they sometimes do not have sufficient time to properly review charges and remit payment to NRC within the 30-day deadline for paying the invoice. One licensee said that delays in receiving an invoice resulted in late fees. NRC is undertaking an initiative to transition to electronic billing, which may address the challenges the licensees identified and, according to NRC staff, improve the agency's billing process. However, NRC has not developed planning documents for this initiative and, according to staff, the planning phase is already past its original deadline of October 2017. The Project Management Institute has identified standards related to project management processes, including project planning. By developing a project management plan that is consistent with best practices and includes steps for involving licensees in system development and assessing results of the project, NRC would have reasonable assurance that it can better manage its electronic billing initiative.

GAO is making five recommendations, including that NRC develop a project management plan for its electronic billing initiative that follows project management standards and includes steps for involving licensees and assessing results. NRC agreed with these recommendations.

The Nuclear Regulatory Commission (NRC) is the federal agency  responsible for regulating the commercial nuclear industry—including  nuclear power plants and other civilian uses of radioactive material— through licensing, inspection, and enforcement of its regulations. NRC  provides licensing services, including conducting inspections, reviewing  applications for new licenses, and renewing licenses, to regulated entities  that hold a license or are applying for a license, referred to as licensees  and applicants, respectively. NRC recovers the full costs for providing  these services by assessing fees to licensees and applicants—called  service fees—and billing licensees and applicants each quarter using a  billing process that includes aggregating all the charges to be billed;  reviewing the charges and correcting errors, if any; mailing quarterly  invoices; and then addressing disputes from licensees and applicants, if  any. In fiscal year 2016, NRC collected about $321 million in service fees  from licensees and applicants. Since 2006, internal audits of NRC’s  revenue from fees have identified problems with the agency’s billing  process, some of which resulted in significant billing errors to licensees.  In fiscal year 2012, for example, an internal NRC audit identified  approximately $24 million in unbilled fees from fiscal years 2011 and  2012 that went undetected by NRC staff involved in the agency’s billing  process.

The Omnibus Budget Reconciliation Act of 1990, as amended, requires  NRC to annually assess and collect fees totaling about 90 percent of its  annual budget authority. To reach this amount, NRC assesses two types  of fees—service fees and annual fees—that are set through an annual  rulemaking process. Service fees encompass reimbursable contract  costs and reimbursable NRC staff time costs, which include time spent  performing work for a specific licensee and certain overhead costs. In  cases where NRC hires a contractor to perform work related to a specific  licensee, NRC pays the contractor for the work and then bills the licensee  for reimbursement. NRC assesses service fees for agency staff time by  establishing an hourly rate to recover the full cost of specific regulatory  services. In cases where the cost of an activity will change very little from  licensee to licensee, NRC has established flat service fees based on the  hourly rate. For service fees, charges on the invoice are to recover the  costs for work completed in the previous quarter up to the date of the  invoice and invoices are mailed to licensees quarterly via the U.S. Postal  Service to collect service fees. NRC assesses annual fees to recover  regulatory costs that are not otherwise recovered through service fees in  order to meet the 90 percent statutory requirement. Licensees’ payments  for service and annual fees are due to NRC within 30 days of the invoice  date so licensees can avoid paying interest on the charges. This report  focuses on the billing process for service fees assessed to licensees.

You asked us to review NRC’s process for billing licensees for service  fees. This report examines (1) the actions NRC is taking to address  problems with its billing process identified by internal reviews and (2) the  challenges selected licensees identified with NRC’s billing process and  the extent to which NRC’s actions address these challenges.

To determine what actions NRC is taking to address problems with its  billing process for service fees, we reviewed agency documents and  interviewed NRC staff. Specifically, we reviewed assessments of NRC’s  billing process from 2006 to 2016, including two audits by NRC’s Office of  the Inspector General (OIG) and NRC’s Business Process Improvement  Project, which was an internal review completed in March 2014 that  focused on eliminating or greatly reducing billing errors for service fees  and improving overall efficiency and effectiveness of the billing process.  We also reviewed NRC’s planning documents related to the actions it is  taking to address the problems, and NRC policy and guidance related to  its billing process. We interviewed NRC staff in the Office of the Chief  Financial Officer who are working on projects and initiatives that are  designed to address problems with NRC’s billing process. We also  interviewed NRC staff in the agency’s four regional offices (Region 1 near  Philadelphia, Pennsylvania; Region 2 in Atlanta, Georgia; Region 3 near  Chicago, Illinois; and Region 4 in Arlington, Texas) and eight offices  (Advisory Committee on Reactor Safeguards; Atomic Safety and  Licensing Board Panel; and the Offices of General Counsel, New  Reactors, Nuclear Material Safety and Safeguards, Nuclear Reactor  Regulation, Nuclear Security and Incident Response, and Nuclear  Regulatory Research) that are responsible for reviewing charges as part  of the billing process, including staff who review contractor charges.

To determine the challenges licensees have with NRC’s billing process  and the extent to which NRC is addressing them, we reviewed agency  documents related to its billing process; interviewed a stratified,  nongeneralizable sample of 13 non-federal licensees; and interviewed  NRC staff. To select licensees for our sample, we used service fee- collection data produced by NRC’s financial management system, called  the Financial Accounting and Integrated Management Information  System. From seven of NRC’s nine fee classes, we selected to interview  the 2 licensees that held the licenses that were assessed the highest  amount of service fees from October 2015 through July 2017. However,  one of the seven fee classes had only 1 licensee that was assessed  service fees in that class during this established period. Also, two fee  classes were excluded from our sample. Our selection process resulted  in 13 licensees of varying size and scope, including 1 holding multiple  licenses that was assessed service fees of almost $28 million during the  established period, and another holding a single license that was  assessed service fees of almost $420,000 during the same period. If a  licensee meeting our selection criteria was unwilling or unable to  participate, we elected to interview the licensee that held the license that  was assessed the next highest amount of service fees during this period.  We interviewed licensees using a standard set of questions to collect  consistent information. We analyzed licensee responses to our standard  set of questions to identify key themes and grouped their responses into  common categories. We interviewed NRC staff and reviewed related  documentation to determine whether the agency’s recent or planned  actions to address problems with its billing process could address  licensees’ challenges. We compared NRC’s current billing practices  against the agency’s Organizational Values and Standards for Internal  Control in the Federal Government, as applicable. We also identified  leading practices for project planning in the Project Management  Institute’s A Guide to the Project Management Body of Knowledge.

We conducted this performance audit from March 2017 to March 2018 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

NRC is an independent agency established by the Energy Reorganization  Act of 1974 to license and regulate civilian uses of nuclear materials in  the United States for commercial, industrial, medical, and academic  purposes. Under the Atomic Energy Act of 1954, as amended, NRC is  responsible for issuing licenses for civilian uses of radioactive material  and conducting oversight activities under such licenses to protect the  health and safety of the public, among other things. NRC regulates  commercial nuclear power plants; research, test, and training reactors;  nuclear fuel cycle facilities; the transport, storage, and disposal of  radioactive materials and waste; and the use of radioactive materials in  medical, academic, and industrial settings. NRC is authorized to  conduct inspections and investigations; enforce regulatory requirements  by, among other things, issuing orders and imposing civil (monetary)  penalties; and revoke licenses. NRC is headed by a five-member  Commission, with members appointed by the President and confirmed by  the Senate; one commissioner is designated by the President to serve as  the Chair and official spokesperson of the Commission. NRC staff from  headquarters and the four regional offices implement the agency’s  programs for developing regulations, licensing, inspection, enforcement,  and emergency response, among other responsibilities. NRC’s Office of  the Chief Financial Officer establishes, maintains, and oversees the  implementation and interpretation of the agency’s regulatory user fee  policies and regulations, among other responsibilities.

The Office of the Chief Financial Officer is responsible for assessing  service fees to licensees for each license they hold and sending licensees  invoices quarterly. The quarterly invoices for service fees may include  costs in the following three categories:

NRC staff work. NRC staff record their time related to services, such  as licensing, inspections, special projects, and license reviews, which  is then billed to licensees to recover the full cost of these services. To  calculate the cost of work performed by NRC staff, NRC applies an  hourly rate—as established during the agency’s annual rulemaking  process—to the number of staff hours spent on work that is directly  attributable to a specific licensee.

Overhead costs for project managers and resident inspectors.

Some licensees work with an NRC project manager or resident  inspector, and NRC allocates the overhead costs for these NRC  staff to the licensees. Overhead costs cover the costs of these staff  doing tasks that are not assigned to a specific licensee, but that  benefit licensees, such as training, according to NRC staff. Project  manager and resident inspector overhead costs are calculated for  each relevant licensee as 6 percent of the licensee’s total NRC staff  time charges for the quarter.

Contractor charges. NRC sometimes hires a commercial contractor  or other federal agency, such as the Department of Energy (referred  to collectively as contractors), to perform services that are directly  attributable to a licensee, such as reviewing license applications. In  these cases, NRC pays the contractor for the work and then bills the  licensee for reimbursement of the contractor’s charges.

NRC’s billing process for service fees begins by identifying work that can  be billed to a specific licensee and ends when the licensee pays the  quarterly invoice. Once NRC determines that billable work needs to be  done, the agency follows the steps in the billing process shown in  figure 1.

The steps in NRC’s billing process are described in more detail below.

NRC assigns activity code: After billable work is identified, NRC  assigns an activity code, which is a project code to which NRC  employees charge time for billable work performed.

NRC performs work: NRC staff perform work that is billable to a  licensee and record their time biweekly in electronic time cards in  NRC’s time and labor management system. If NRC staff discover that  they have recorded time incorrectly in a previous pay period, such as  by charging time to an incorrect activity code, they can correct the  error by making a prior-period adjustment. Adjustments that are made  within 6 weeks of the date of the error can be made directly in the time  and labor management system; adjustments made 6 weeks or more  after the date of the error require a memo with justification from the  employee’s office director to NRC’s Controller.

Supervisors review hours: At the end of each 2-week pay period,  NRC supervisors review and approve the time cards for the staff they  supervise, including the hours charged to activity codes.

Contractor performs work: If work is done by a contractor, the  contractor submits a status report and invoice to NRC each month.  Each monthly status report includes a description of the work done,  the planned completion date, the total charges for the current invoice,  the cumulative charges to date, and an estimate of future charges.

NRC reviews charges: NRC staff responsible for managing the  agency’s contracts review the monthly status reports and invoices and  must approve invoices before paying the contractor. After paying a  contractor’s invoice, NRC bills the licensee for reimbursement of the  amount NRC paid to the contractor. Contractor charges are included  on a licensee’s quarterly invoice, and NRC may bill a licensee for  contractor charges after the quarter in which the work was performed.

NRC aggregates charges: NRC’s financial management system  aggregates all NRC staff hours and charges from contractors biweekly  for each licensee. The financial management system obtains data on  staff hours from NRC’s time and labor management system.  Contractor charges are entered manually into the financial  management system.

NRC validates charges: NRC regional and program offices review  and certify all charges to licensees after the end of each quarter. To  accomplish this, the Office of the Chief Financial Officer produces  quarterly validation reports—one for staff charges and one for  contractor charges—from NRC’s financial management system.

NRC invoices licensees: NRC creates invoices each quarter and  sends them to licensees via the U.S. Postal Service. Licensees’  payments are due to NRC within 30 days of the invoice date to avoid  paying interest on the charges.

Licensee reviews and pays invoice: Licensees review the invoice  and may pay the invoice, request that NRC review the fees assessed,  or dispute the fees. These billing disputes generally start informally  with the licensee contacting NRC. According to NRC staff, most  disputes are handled informally and generally entail explanations of  the agency’s billing or licensing policies. If NRC staff are unable to  resolve a licensee’s concern informally, the licensee can write a letter  to the Chief Financial Officer, which begins a formal dispute process.  According to NRC staff, to address a licensee’s concerns with the  charges, the Office of the Chief Financial Officer reviews the charges  on the invoice and may involve the relevant regional or program  offices to determine whether the charges are valid for the work  performed. Additionally, NRC’s Office of the General Counsel may be  included in disputes regarding NRC’s fee policy. After the dispute is  resolved, the licensee pays the invoice.

NRC’s OIG and internal reviews conducted by NRC in the last 5 years  identified problems with the agency’s billing process. In 2012 and 2015,  for example, OIG audits identified problems with NRC’s management and  review of billable charges and recommended changes to the agency’s  internal processes and procedures—called internal controls—to improve  the accuracy of invoices. In 2013, NRC launched the Business Process  Improvement Project to determine the root causes of billing errors, many  of which were discovered during the quarterly validation step of the billing  process. The project was completed in 2014 and made recommendations  focused on strengthening internal controls and improving efficiency and  effectiveness of the billing process. Additionally, in 2016 NRC requested  feedback from the public, including licensees and other stakeholders, on  the general communications the agency provides about its fees, intending  to use the feedback to improve the transparency of its fees development  and invoicing practices. Following this effort, NRC launched its Fees  Transformation initiative to improve transparency of its fee-setting and  billing processes.

NRC Has Taken Action in Four Main Areas to Improve Its Billing Process

NRC has recently implemented or plans to implement changes in four  main areas of its billing process to address problems identified by NRC’s  OIG and NRC internal reviews: controls over activity codes, guidance and  training for NRC staff, quarterly validation of charges, and charging  licensees for billable overhead costs.

NRC’s OIG and internal reviews found problems with NRC’s internal  controls over activity codes, which affected the quality of data used for  billing and other agency processes. The management of activity codes  was decentralized, meaning that staff in NRC’s offices generated the  codes and each office followed its own policies and procedures regarding  the setup and use of these codes. NRC also did not have a standardized  set of activity codes to be used across the agency. Activity codes were  instead linked to specific licensees, meaning that identical work activities  for two licensees would require two different activity codes. These  conditions resulted in an excessive number of activity codes in the  agency’s time and labor management system. According to an internal  NRC review, the decentralized management and absence of  standardized activity codes weakened internal controls and put NRC at  risk for incomplete or inaccurate billing. Further, there was no consistent  naming convention, and activity code titles often lacked the specificity  necessary for NRC staff to readily identify the correct code for the work  activity performed, according to the OIG. NRC staff could also search  and access the entire inventory of activity codes, including those  unrelated to their work. According to the OIG, these conditions increased  risk for staff to inadvertently select the wrong activity codes when  recording their time; in such cases, the wrong licensee could be billed for  the work.

Starting in fiscal year 2016, the Office of the Chief Financial Officer began  taking responsibility for overseeing and managing activity codes,  including establishing, maintaining, and closing activity codes available in  the agency’s time and labor management system. Further, NRC  developed a set of standardized activity codes with titles related to the  specific work activities completed. The transition to centralized activity  code management and standardized activity codes was completed in  October 2017, according to NRC staff. Also in October 2017, the agency  implemented controls that prevent a staff member from charging time to  an activity code unless a project manager has granted that staff member  access to the code.

NRC’s OIG and internal reviews found problems with staff’s  understanding of their roles and responsibilities for accurate time and  labor reporting and management of billable contracts, which, according to  NRC documents, contributed to avoidable time card errors and billing  errors. To address these problems, NRC provided training and updated  guidance for staff covering the following two areas:

Time and labor reporting. According to an internal NRC review, staff  were making avoidable data entry errors in time cards that  supervisors who approved the timecards were not identifying,  meaning incorrect time cards were sent to the Office of the Chief  Financial Officer for billing. In late fiscal year 2015, NRC provided  training to all agency staff to emphasize the importance of accurate  time reporting, the process for selecting correct activity codes, and the  relationship of time card entry to billing. According to NRC officials,  the agency also provided specialized training to staff in offices where  errors were common. Additionally, the agency updated its time and  labor reporting guidance and provided supplemental guidance to staff  related to time and labor reporting. Furthermore, in preparation for  changes to activity codes that were implemented in October 2017,  NRC provided additional training to staff on the new activity code  structure and making corrections to their time cards.

Management of contracts. According to an internal NRC review,  approximately one-third of the billing errors identified during the  quarterly validation step of the billing process resulted from  administrative errors in managing contracts. NRC’s OIG also found  that agency guidance related to the invoice review process was  outdated and did not provide staff with sufficient criteria for verifying  information contained in contractor invoices. Without such criteria,  NRC could not ensure that it was evaluating contractor charges  consistently and appropriately before billing those charges to  licensees. In 2015, NRC provided training to staff who manage  contracts, which, according to NRC officials, resulted in an immediate  decrease in associated billing errors. NRC also revised its guidance to  clarify responsibilities, procedures, and instructions for reviewing and  approving contractor invoices.

NRC’s OIG and internal reviews identified conditions that made the  quarterly validation step in the billing process challenging for staff to  perform and that led to inconsistent validation procedures among  program and regional offices. NRC has taken or plans to take the  following two actions to address these problems:  Improving validation reports. According to NRC documents, the  quarterly validation reports contained billing data for all program and  regional offices—sometimes amounting to more than 4,000 pages of  data—and the reports did not have the sorting functionality or  querying capability that would allow NRC staff to extract relevant  information. Staff in program and regional offices instead relied on  manually generated reports to compile information they needed.  Additionally, according to the OIG, the quarterly validation reports did  not include sufficient detail on contractor charges for NRC’s staff to  properly review them. To address these problems, in 2014 NRC  started providing the quarterly validation report in electronic  spreadsheet format, which gave staff the sorting and filtering  capabilities needed to extract data relevant to their respective reviews  and eliminated the need for manually generated reports, according to  NRC staff. Further, NRC began providing validation information for  contractor charges in a separate report. The new validation report for  contractor charges has more detailed information and specific  instructions for NRC staff for verifying the accuracy of the charges.

Standardizing the quarterly validation process. According to NRC,  the current quarterly validation process is not standardized across the  regional and program offices and there is no agency guidance to  ensure that staff in different offices conduct the process consistently.  Further, there is currently no way to ensure that an adequately trained  person in each program or regional office is conducting the validation,  according to NRC staff. To address these problems, NRC is planning  to standardize the quarterly validation process and to establish clear  roles and responsibilities for staff participating in the process. One key  change NRC is planning is to have the individual leading the work  validate the accuracy of the charges. According to NRC’s planning  documents—dated August 2017—NRC expects to pilot the new  validation process in June 2018 and to implement it agency-wide by  October 2018.

At the end of fiscal year 2012, an internal NRC audit identified  approximately $24 million in unbilled overhead hours. NRC staff  explained that the hours went unbilled because project managers and  resident inspectors charged billable overhead time to nonbillable activity  codes, rather than to the billable activity codes associated with licensees.

According to an internal NRC review, these errors accounted for  approximately two-thirds of the billing errors identified during the quarterly  validation process. At the beginning of fiscal year 2016, NRC started  billing this overhead time as a separate fee on invoices that is calculated  as 6 percent of all NRC billable hours on an invoice, which eliminated the  billing errors related to overhead. However, NRC analyzed this billing  method again in fiscal year 2017 and determined that eliminating the  percentage charge and having staff charge their billable overhead time to  billable activity codes would be more equitable. NRC intends to  implement a new process for charging billable overhead time at the start  of fiscal year 2019. According to NRC staff, the agency has made  administrative changes to address the factors that contributed to project  managers and resident inspectors incorrectly charging overhead time in  the past.

Licensees We Interviewed Identified Several Challenges with NRC’s Billing Process, and NRC’s Recent and Planned Changes May Not Fully Address Them

Licensees we interviewed identified challenges with the amount of  information available about NRC’s billable work, and NRC’s recent  changes have made more information available, but some licensees are  not aware of the information. Licensees also identified challenges with  NRC’s method of delivering paper invoices by mail, and although NRC’s  recent and planned changes may help address these challenges, NRC’s  plans are incomplete.

Licensees Identified Challenges with the Information Available about NRC’s Billable Work, and NRC Has Made More Information Available, but Some Licensees Are Not Aware of It

Licensees we interviewed identified challenges with the amount of  information available about NRC’s billable work, including challenges  related to planning and budgeting for NRC work and verifying charges on  invoices. NRC has recently implemented changes that may address  some of the challenges.

Licensees we interviewed identified challenges with planning for future  work and budgeting to pay future costs because NRC does not provide  certain information about the agency’s billable work. Specifically, NRC  does not formally provide information on timeframes for completing  billable work, customized cost estimates for projects, or the status of  ongoing work. Eleven of the 13 licensees we interviewed indicated that  having timeframes, cost estimates, status reports, or a combination of  these would be useful. One licensee explained that when it receives an  invoice for work that NRC staff have performed, the licensee does not  know how much work remains and cannot budget for future expenses.

This challenge may be addressed, in part, by NRC’s Fees Transformation  initiative. Under this initiative, NRC began reporting on its public website  in September 2017 resource estimates for various licensing actions, such  as site permitting, design certifications, inspections, license amendments,  and license renewals, among others. These resource estimates include  the low, high, and average number of NRC staff hours billed for each  action, as well as some estimates for contractor charges for certain tasks.  These resource estimates are based on historical expenses and were  calculated using a sample of licensing and oversight actions, though they  may still be useful to licensees to help plan and budget for future NRC  costs. According to NRC’s website, the agency will update most of the  resource estimates every 2 years.

Licensees we interviewed said that they have challenges verifying  charges on their invoices because NRC’s invoices do not provide enough  information on work that NRC staff or contractors perform. For NRC staff  work, invoices include the total hours charged by NRC staff for each  activity code. However, activity codes often cover broad topics rather than  specific work activities. Also, activity codes have a 120-character limit,  according to NRC staff, and NRC uses some of those characters to list  each licensee’s name and other identifying information, which means that  there is limited remaining space to identify the specific work activity. Nine  of the 13 licensees we interviewed explained that more descriptive activity  codes on invoices would be helpful. One licensee said that it is difficult to  know what project it is being billed for because the activity code  descriptions are cryptic and sometimes nondescript. In addition, all 13  licensees we interviewed indicated that having NRC staff names or their  positions would be helpful in verifying the accuracy of charges. For  example, 2 of these licensees explained that they are familiar with the  NRC staff who consistently work on their projects, so they could consider  questioning charges if the invoice showed a new person working on a  project.

Additionally, licensees said that it is difficult to verify charges on their  invoices because NRC’s invoices also do not contain detailed information  on contractor charges. Invoices indicate that work was done by a  contractor and provide the total cost of the work, but they do not include  the contractor’s name or describe the work performed. Five of the  licensees we interviewed said that invoices do not have enough  information about the contractor and the work performed. Additionally, 4  of these licensees stated that they cannot determine whether the  amounts charged were accurate or reasonable without more information.

Challenges related to verifying charges may be addressed by some of  NRC’s recent changes to its billing process, which include updating  invoices. NRC is updating its invoices to include (1) standardized activity  codes that have titles describing the specific work activity completed, (2)  the names of the NRC staff charging time to the licensee, and (3) the  name of the contractor that performed the work for which the licensee is  being billed. NRC staff expected to issue the updated invoices to  licensees in January 2018, after we completed our audit work. Therefore,  we could not assess licensees’ satisfaction with the updated invoices.  According to a planning document for some of NRC’s recent changes, the  agency intends to solicit feedback from licensees in fiscal year 2018 on  whether the updated invoices have addressed licensees’ challenges.  However, NRC staff told us that they are not certain when the agency will  solicit feedback.

In addition to updating invoices, NRC can provide supplemental  information to licensees to help them verify the accuracy of the following  charges:

NRC staff charges: NRC created biweekly reports on staff charges  that it sends to licensees, when requested. These biweekly reports  provide more frequent cost data and include a level of detail that is not  provided on the quarterly invoices. For example, the biweekly reports  include NRC staff names and the charges, by employee, for that 2- week period. Three of the 13 licensees we interviewed that receive  the biweekly reports said that they use the reports to check the  quarterly invoice for accuracy by adding up the costs from the  biweekly reports and comparing them to the quarterly invoice. For  example, a licensee told us that if the biweekly reports and quarterly  invoice have similar totals, it does not raise any questions about the  charges.

Contractor charges: NRC has supplemental contractor information  that it can provide to licensees. NRC receives monthly status reports  from contractors on charges that are ultimately billed to licensees on  their quarterly invoices. These monthly status reports include current  work performed and associated charges, as well as remaining work to  be performed and an estimate of future charges. In 2015, NRC  developed a process to review and provide to licensees certain  information from the monthly status reports when licensees request it.

Although this supplemental information on staff charges and contractor  charges is available, not all licensees know it is available. Specifically, 6  of the 13 licensees we interviewed told us that the biweekly reports would  be useful, but did not know the reports are available and can be  requested. Also, 10 of the 13 licensees we interviewed told us that  detailed information on contract work would be useful, but 9 of them did  not know this information is available and can be requested.

Not all licensees know the supplemental information is available because,  according to NRC staff in the Office of the Chief Financial Officer, the  agency has not instituted a formal process to inform all licensees of its  availability. These staff added that the agency has announced the  availability of this supplemental information at industry conferences or has  told individual licensees about it. NRC staff explained that the agency is  meeting statutory requirements for issuing invoices and provides the  supplemental information as a courtesy to licensees, but is not required to  do so. According to NRC staff in the Office of the Chief Financial Officer,  the agency has not formally notified all licensees about the availability of  this supplemental information because it is time-consuming to provide it  to licensees. These staff also said that they have found that not all  licensees may need this information. This is consistent with information  from the licensees we interviewed. For example, 2 licensees told us that  they do not need biweekly reports; one said that it operates on a fixed  annual budget, so additional information on biweekly costs would not be  useful. In contrast, NRC staff noted that licensees with more complex  invoices—such as multiple sites and multiple inspections and licensing  actions—may find the supplemental information useful. Standards for  Internal Control in the Federal Government explains that management  should communicate quality information externally so that external parties  can help the entity achieve its objectives and address related risks.  Furthermore, being open and transparent in communications is part of  NRC’s Organizational Values, which guide every action it takes, how it  performs administrative tasks, and how it interacts with stakeholders.  Communicating to licensees about what information is available could  help improve the transparency of NRC’s invoices, in accordance with the  agency’s values.

Additionally, 2 licensees told us that they requested information on work  being done by a contractor but NRC staff told them that the information  could not be provided. NRC staff in the Office of the Chief Financial  Officer acknowledged that some NRC project managers may not be  aware that licensees can request contract information because there is no  policy or guidance to instruct NRC staff on what information they can  provide or how to do so. Standards for Internal Control in the Federal  Government states that agency management should clearly document— in management directives, administrative policies, or operating manuals— the processes it uses to ensure that it is achieving its objectives. By  developing a policy and guidance for NRC staff, the agency could help  ensure that staff are aware of the agency’s processes and provide quality  information consistently.

Licensees Identified Challenges with NRC’s Method for Delivering Invoices, and NRC’s Recent and Planned Changes May Address These Challenges, but Its Plans Are Incomplete

Licensees we interviewed said that NRC’s method for delivering paper  invoices by mail created challenges related to the format and timeliness  of the invoice, though NRC’s recent and planned changes may help  address these challenges. For example, one licensee told us that, without  the sorting and filtering capabilities of an electronic spreadsheet, this  licensee is not able to verify the accuracy of charges for specific  components of the work that NRC is doing. Another licensee told us that it  is difficult to track costs of projects to completion without an electronic  spreadsheet of charges. NRC now provides biweekly reports in an  electronic spreadsheet format, which may help address the challenges  these licensees cited. However, as discussed above, NRC does not  provide these biweekly reports unless they are requested, and some  licensees do not know that they are available.

Licensees also cited challenges with the timeliness of invoices they  receive via mail. For example, 2 licensees stated that, with invoices taking  up to 10 days to arrive in the mail, licensees sometimes do not have  sufficient time to conduct a proper review of charges and remit payment  to NRC within the 30-day deadline. According to one licensee we  interviewed, delays in receiving the invoices have resulted in late fees. To  address the challenge of timeliness, NRC will, upon request from a  licensee, e-mail a copy of the invoice to the licensee after the hardcopy  invoice has been mailed. This practice allows the licensee to begin  reviewing its charges while waiting for the mailed copy to arrive. However,  of the 11 licensees that told us an e-mailed copy of the invoice would be  useful, 4 of them did not know this option was available.

NRC staff said the agency intends to transition to electronic billing—that  is, sending invoices in electronic format via e-mail or providing licensees  with web access to review and pay invoices. According to NRC staff, the  agency’s transition to electronic billing is being done to improve efficiency  and internal controls in NRC’s billing process. However, doing so may  also help address challenges that some licensees experience with the  format and timeliness of invoices. For example, 11 of the 13 licensees we  interviewed affirmed that receiving electronic invoices or periodic  statements of charges electronically would be beneficial.

In October 2016, NRC’s Commission directed NRC staff to examine  opportunities to accelerate the transition to an electronic billing system.  The agency has indicated its intent to complete the planning phase by  October 2017 and fully implement a new system by October 2019.  However, according to NRC staff, the planning phase was not completed  because the agency needed to fully implement the recent changes to its  billing process before planning for the transition to electronic billing. As a  result, the agency has not yet developed any planning documents to help  ensure that it meets its deadlines, achieves its goal of increasing  efficiency, or addresses licensees’ challenges. NRC staff in the Office of  the Chief Financial Officer said that they recognize there could be delays  in planning, but still expect to implement electronic billing by October  2019.

We have previously found that federal information technology projects too  frequently incur cost overruns and schedule slippages, but that proper  planning—including incorporating best practices for project planning and  scheduling—may help mitigate these effects. The Project Management  Institute’s A Guide to the Project Management Body of Knowledge  identifies standards related to project management processes, including  project planning. In particular, the guide explains that the project plan is  a comprehensive document that defines the basis of all project work and  describes how the project will be executed, monitored, and controlled.  The project plan integrates and consolidates plans for project  components, such as plans for managing the project’s scope, schedule,  cost, quality, and risk, among others. Among other things, the project  management plan may also include requirements and techniques for  communication among stakeholders and key reviews by management. By  developing a project management plan that is consistent with best  practices, NRC would have more reasonable assurance that it is better  managing its transition to electronic billing.

Furthermore, Standards for Internal Control in the Federal Government  also states that, in deciding what information is required to achieve  objectives, management should consider the needs of both internal and  external users. Additionally, we have previously identified common factors  critical to successful information technology acquisitions. Among these  factors are (1) involving end users and stakeholders in developing  requirements and (2) including end users in testing of system functionality  prior to formal end user acceptance testing. As NRC develops a project  management plan, by involving licensees in developing system  capabilities for electronic billing, which includes soliciting and considering  licensees’ information needs, the agency would have better assurance of  a successful transition to electronic billing.

Additionally, Standards for Internal Control in the Federal Government  states that management should design control activities to achieve  objectives. Control activities may include establishing performance measures and indicators and management reviews that compare actual  performance to planned or expected results and analyze significant  differences. As NRC develops a project management plan, including  steps to assess the results of implementing electronic billing, which  includes comparing actual performance of the new electronic billing  system to planned results, would provide the agency more reasonable  assurance that the project meets desired outcomes.

NRC has recently implemented or plans to implement a number of  changes to its billing process that—if implemented as intended—could  address challenges that licensees identified in our interviews. However,  additional steps could enhance NRC’s efforts to improve its billing  process. Licensees told us that they could use more detailed information,  more timely information, and information in an electronic format. NRC has  made more detailed information on staff charges available in biweekly  reports and has developed a process to provide detailed information on  contractor work, upon request from a licensee. NRC is also providing  invoices in electronic format to some licensees, when requested.  However, some licensees that would find the information on staff and  contractor charges useful do not know that it is available, and some NRC  staff are not aware that they can provide it or how to do so. Until NRC  communicates to all licensees about what information is available and  develops a policy and guidance for agency staff, the agency cannot  ensure that it is providing quality information consistently.

Further, NRC intends to take additional action toward improving its billing  process and invoices by transitioning to electronic billing. As NRC moves  forward with this project, developing a project management plan that is  consistent with best practices, to include establishing plans for the  project’s schedule and cost, as well as involving licensees in developing  the plan and assessing the results of implementation, will give the agency  more reasonable assurance that it is better managing its transition to  electronic billing and could help ensure that the project meets desired  outcomes.

We are making the following five recommendations to NRC:  The Chief Financial Officer of NRC should formally communicate to all  licensees that supplemental billing information—including biweekly  reports and monthly status reports on contractor charges—is available  and how to request it. Formal communication that would reach all  licensees could include adding information to their quarterly invoices.  (Recommendation 1)

The Chief Financial Officer of NRC should develop agency policy and  guidance for staff on what billing information related to contractor charges  NRC staff can provide to licensees and how it should be provided.  (Recommendation 2)

As NRC plans its transition to electronic billing, the Chief Financial Officer  of NRC should develop a project plan that incorporates standards for  project management, which includes establishing plans for schedule and  cost. (Recommendation 3)

In developing the project plan for electronic billing, the Chief Financial  Officer of NRC should include steps to involve licensees in developing  system capabilities, which includes soliciting and considering licensees’  information needs. (Recommendation 4)

In developing the project plan for electronic billing, the Chief Financial  Officer of NRC should include steps to assess the results of implementing  electronic billing, which includes comparing the actual performance to  intended outcomes. (Recommendation 5)

We provided a draft of this report to NRC for review and comment. NRC  provided written comments, which are reproduced in appendix I. In its  written comments, NRC agreed with our findings and recommendations.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the appropriate  congressional committees, the Chairman of the Nuclear Regulatory  Commission, and other interested parties. In addition, the report will be  available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-3841 or ruscof@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix II.

Appendix I: Comments from the Nuclear Regulatory Commission

Appendix II: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Hilary Benedict (Assistant  Director), Wyatt R. Hundrup (Analyst in Charge), and Breanna Trexler  made key contributions to this report. Also contributing to this report were  Ellen Fried, Cindy Gilbert, Heather Keister, Benjamin Licht, Laurel Plume,  Dan C. Royer, and Barbara Timmerman.

The Nuclear Regulatory Commission inspects and licenses nuclear power plants and other civilian uses of radioactive material. It charges licensees to recover the cost of these efforts. In fiscal 2016, for example, it collected about $321 million in related fees.

We reviewed NRC's billing process and found it is taking steps to address problems that earlier audits and some licensees identified, such as a paper billing process that did not allow sufficient review time.

We recommended, among other things, that NRC follow best practices to establish electronic billing and more fully communicate to licensees what billing information is available.