Improper Payments: Selected Agencies Need Improvements in Their Assessments to Better Determine and Document Risk Susceptibility

Improper payments are a long-standing problem in the federal government, estimated at almost $141 billion for fiscal year 2017. Agencies are required to perform risk assessments to identify programs that may be susceptible to significant improper payments.

GAO was asked to review federal agencies' improper payment risk assessments. This report examines the extent to which certain agencies' improper payment risk assessments for selected programs provided a reasonable basis for determining their susceptibility to significant improper payments. GAO analyzed the most recent risk assessments, from 2015 through 2017, for the following five programs: USDA's Agriculture Risk Coverage and Price Loss Coverage programs; HHS's Head Start; DOJ's Law Enforcement; and Treasury's Interest on the Public Debt and Home Affordable Modification Program. GAO selected these programs, focusing on programs that recently underwent a risk assessment and size of programs' gross outlays—which totaled about $330 billion in fiscal year 2017 for the five programs GAO selected.

The Improper Payments Information Act of 2002, as amended (IPIA), defines “significant” improper payments as improper payments in the preceding fiscal year that may have exceeded either (1) 1.5 percent of program outlays and $10 million or (2) $100 million (regardless of the improper payment rate). GAO found that the Departments of Health and Human Services (HHS), the Treasury (Treasury), Justice (DOJ), and Agriculture (USDA) assessed the five programs GAO selected for review as at low risk for susceptibility to significant improper payments; however, HHS, Treasury, and DOJ lacked sufficient documentation to assess the extent to which their risk assessments provided a reasonable basis for their risk determinations. On the other hand, USDA's quantitative risk assessment of its program's susceptibility to significant improper payments provided a reasonable basis for its low-risk determination.

Although HHS, Treasury, and DOJ considered, among other factors, the nine risk factors from IPIA and Office of Management and Budget guidance, they did not document or effectively demonstrate how these factors affected their programs' susceptibility to significant improper payments. These programs' risk assessments did not contain sufficient documentation to determine how the agencies arrived at their risk determinations for each risk factor, or how the total scores for all risk factors led to low-risk determinations. For example, HHS determined that its Head Start program was at high risk for several risk factors—including complexity per transaction and volume of payments—but did not document how these high-risk ratings informed its overall determination that Head Start was not susceptible to significant improper payments.

Further, the agencies did not have documentation to demonstrate how they determined the weighting of each risk factor or the risk level ranges from the risk assessment templates as they relate to the programs' susceptibility to significant improper payments. For example, based on GAO's analysis of Treasury's risk assessment template, the agency could identify areas of risk related to each of the nine risk factors. But because of the assigned weights given to each risk factor, Treasury's final risk calculation would still not determine the program to be at high risk of susceptibility to significant improper payments. Without documenting the basis for the assigned weights, Treasury cannot demonstrate, and GAO cannot determine, that its process for determining its programs' susceptibility to significant improper payments was reasonable. Until HHS, Treasury, and DOJ revise their risk assessment processes to help ensure that they result in reliable assessments, they cannot be certain whether their programs are susceptible to significant improper payments and therefore whether they are required to estimate the amount of improper payments.

GAO also found that HHS did not assess many of its programs and activities at least once during the 3-year period from fiscal years 2015 through 2017, as required by IPIA. Based on the analysis of HHS information, GAO identified at least 140 programs or activities that were not assessed during the 3-year period. When not all eligible programs are reviewed as required, there is an increased risk that the agency may not identify all risk-susceptible programs and activities, resulting in incomplete improper payment estimates.

GAO recommends that Treasury, DOJ, and HHS revise their improper payment risk assessment processes, and that HHS revise its procedures to help ensure that all programs are assessed at least once every 3 years. In their responses, Treasury and HHS agreed with the recommendations, and DOJ disagreed with GAO's recommendation. GAO continues to believe that the recommendation is valid, as discussed in the report.

Improper payments—payments that should not have been made or were  made in incorrect amounts under statutory, contractual, administrative, or  other legally applicable requirements—are a long-standing, significant  problem in the federal government. For fiscal year 2017, estimated  federal improper payments totaled about $141 billion. Although agencies  report improper payment estimates annually, in our report on the Fiscal  Years 2017 and 2016 Consolidated Financial Statements of the U.S.  Government, we continued to report that the federal government is  unable to determine the full extent to which improper payments occur and  reasonably assure that actions are taken to reduce them. In addition, we  have previously noted that some inspectors general (IG) have reported  issues related to agencies’ improper payment estimates, including  processes that may not produce reliable estimates.

The Improper Payments Information Act of 2002, as amended (IPIA),  requires, among other things, that at least once every 3 years, each  federal agency review all of its programs and activities to identify those  that may be susceptible to significant improper payments—a process  commonly referred to as an improper payment risk assessment. Under  IPIA, improper payments are considered “significant” if in the preceding  fiscal year they may have exceeded either (1) 1.5 percent of program  outlays and $10 million or (2) $100 million (regardless of the improper  payment rate). Properly executed improper payment risk assessments  are the cornerstone of a government-wide effort to identify and reduce  improper payments. Agencies are required to develop improper payment  estimates and corrective action plans for any programs that agencies,  OMB, or statute identifies as susceptible to significant improper  payments.

You requested that we review federal agencies’ improper payment risk  assessments. This report examines the extent to which certain federal  agencies’ improper payment risk assessments for selected programs  provided a reasonable basis for determining their susceptibility to  significant improper payments.

To address our objective, we selected a nongeneralizable sample of 4  agencies and five programs for review. We considered improper payment  information for the 24 agencies subject to the Chief Financial Officers Act  of 1990. In considering which agencies to review, we selected a mix of  agencies that did and did not report improper payment estimates for fiscal  year 2017, and included at least one agency that administered eligibility- based programs. The 4 agencies that we selected for our review were the  Departments of Agriculture (USDA), Health and Human Services (HHS),  Justice (DOJ), and the Treasury (Treasury). We then selected five  programs across these agencies to review, focusing on programs that  underwent a risk assessment during fiscal years 2015 through 2017 and  the size of programs’ gross outlays for fiscal year 2017. The five  programs that we selected for review were USDA’s Commodity Credit  Corporation Agriculture Risk Coverage and Price Loss Coverage  programs; HHS’s Head Start program; DOJ’s Law Enforcement program;  and Treasury’s Interest on the Public Debt and Home Affordable  Modification Program. Fiscal year 2017 outlays for these programs  totaled approximately $330 billion.

We reviewed improper payment risk assessment requirements in IPIA  and the related guidance in Office of Management and Budget (OMB)  Circular A-123, Appendix C, Requirements for Effective Estimation and  Remediation of Improper Payments (OMB M-15-02). We also reviewed  relevant internal control standards to determine the activities needed to  help ensure that agencies conduct effective improper payment risk  assessments—assessments that provide a reasonable basis for  determining the susceptibility to significant improper payments. We  obtained the most recent improper payment risk assessment that USDA,  HHS, DOJ, and the Treasury conducted on the selected programs during  the 3-year period covered by our review (fiscal years 2015 through 2017).  We then analyzed those risk assessments against relevant IPIA  requirements, OMB guidance, and internal control standards to determine  whether the agencies had evaluated the appropriate risk factors for  improper payments, appropriately considered those factors in their risk  assessments, and provided a reasonable basis for their risk  determinations. Additionally, we interviewed officials at the selected  agencies on their processes for conducting improper payment risk  assessments and reviewed documented policies and procedures. For  agencies that lacked supporting documentation or did not provide a  reasonable basis for their risk determinations, we interviewed appropriate  agency officials to determine the reasons they did not. Appendix I  provides further details on our scope and methodology.

We conducted this performance audit from December 2017 to January  2019 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

IPIA defines an improper payment as any payment that should not have  been made or that was made in an incorrect amount (including  overpayments and underpayments) under statutory, contractual,  administrative, or other legally applicable requirements. It includes  duplicate payments, any payment made to an ineligible recipient, any  payment for an ineligible good or service, any payment for a good or  service not received (except for such payments where authorized by law),  and any payment that does not account for credit for applicable discounts.  OMB M-15-02 also provides that when an agency’s review is unable to  determine whether a payment was proper as a result of insufficient or lack  of documentation, this payment must also be considered an improper  payment.

IPIA also defines the scope of payments subject to improper payment  requirements. Specifically, a payment is any transfer or commitment for  future transfer of federal funds—such as cash, securities, loans, loan  guarantees, and insurance subsidies—to any nonfederal person or entity  that is made by a federal agency, a federal contractor, a federal grantee,  or a governmental or other organization administering a federal program  or activity.

Executive branch agencies are required to take various steps regarding  improper payments under IPIA and as directed by OMB M-15-02. The  steps include the following:  1.  reviewing all programs and activities and identifying those that may be  susceptible to significant improper payments (commonly referred to as  a risk assessment),  2.  developing improper payment estimates for those programs and  activities that agency risk assessments, OMB, or statute identifies as  being susceptible to significant improper payments,  3.  analyzing the root causes of improper payments and developing  corrective actions to reduce them, and  4.  reporting on the results of addressing the foregoing requirements.

Figure 1 illustrates these steps, as well as the major components of  conducting an improper payment risk assessment.

IPIA requires that agencies conduct improper payment risk assessments  for all federal programs and activities at least once every 3 years and  identify any program or activity that may be susceptible to significant  improper payments. OMB guidance provides that programs that have  been determined to be susceptible to significant improper payments and  that are already reporting an estimate—or in the process of establishing  an estimate—do not have to conduct additional improper payment risk  assessments. IPIA defines “significant” improper payments as improper  payments in the preceding fiscal year that may have exceeded either   (1) 1.5 percent of program outlays and $10 million or (2) $100 million  (regardless of the improper payment rate). OMB M-15-02 provides  guidance for implementing the IPIA requirements and covers agencies’  responsibilities for improper payment risk assessments, estimation, and  reporting.

OMB M-15-02 also lists steps that agencies should take when conducting  improper payment risk assessments. Agencies must institute a systematic  method of reviewing all programs and activities to identify those that may  be susceptible to significant improper payments, as defined by IPIA.  According to OMB M-15-02, this systematic method could be a  quantitative evaluation based on a statistical sample or a qualitative  method (e.g., a risk-assessment questionnaire). Prior to fiscal year 2018,  at a minimum, agencies were required to take into account nine risk  factors—seven specified in IPIA and two in OMB guidance—that are  likely to contribute to improper payments, regardless of which method  was used by the agency (see table 1).

In June 2018, OMB revised its guidance for improper payments in OMB  Circular A-123, Appendix C, Requirements for Payment Integrity  Improvement (OMB M-18-20). In the revised guidance, OMB no longer  directs agencies to consider the two additional risk factors that were  included in OMB M-15-02 in their risk assessments. Rather, OMB directs  agencies to take into account those risk factors that are likely to  contribute to a susceptibility of significant improper payments. The  revised guidance also states that beginning in fiscal year 2020, agencies  should use quantitative evaluations for programs or activities with outlays  exceeding $5 billion. As specified in OMB M-18-20, the end goal of the  systematic method of reviewing all programs, whether qualitative or  quantitative, is to determine whether a program is susceptible to  significant improper payments. Accordingly, OMB M-18-20 states that if a  qualitative method is used, it must be designed to accurately determine  whether the program is susceptible to significant improper payments.

When conducting improper payment risk assessments, each federal  agency, unless otherwise specified by OMB Circular A-11, after  consultation with OMB, is generally authorized to determine the grouping  of programs that most clearly identifies and reports improper payments  for the agency. The five programs we reviewed serve a variety of  purposes and are administered by various agencies across the federal  government.

HHS’s Head Start program was established in 1965 to deliver  comprehensive educational, social, health, nutritional, and psychological  services to low-income families and their children. These services include  preschool education, family support, health screenings, and dental care.  Head Start was originally aimed at 3- to 5-year-olds. The Head Start  program makes grants directly to approximately 1,600 local organizations,  including community action agencies, school systems, tribal governments  and associations, and for-profit and nonprofit organizations.

The Head Start program has several primary eligibility criteria to enroll in  the program—including that the child’s family earns income below the  federal poverty level; the child’s family is eligible for or, in the absence of  child care, would potentially be eligible for public assistance; the child is in  foster care; or the child is homeless. Head Start services are to be  provided free of charge to eligible families.

Prior to fiscal year 2013, HHS reported improper payment estimates for  the Head Start program. However, as of fiscal year 2013, HHS, in  consultation with its Office of Inspector General (OIG) and with approval  from OMB, no longer reports annual improper payment estimates related  to the program. According to HHS, Head Start’s fiscal year 2017 outlays  were approximately $9.4 billion.

Public debt is defined as Treasury-issued securities, primarily consisting  of marketable Treasury securities (i.e., bills, notes, and bonds), and a  smaller amount of nonmarketable securities, such as savings bonds and  special securities issued to state and local governments. A portion is debt  held by the public and a portion is debt held by federal government  accounts.

Debt held by the public represents federal debt held by investors outside  of the federal government, including individuals, corporations, state or  local governments, the Federal Reserve, and foreign governments. Types  of securities held by the public include Treasury bills, notes, and bonds  and State and Local Government Series securities. Debt held by the  public primarily represents the amount the U.S. government has  borrowed from the public to finance cumulative cash deficits. As of  September 30, 2017, total debt held by the public was $14.7 trillion.

Debt held by federal government accounts (intragovernmental holdings)  represents balances of federal government accounts of certain federal  agencies that are either authorized or required to invest excess receipts  in Treasury securities. As of September 30, 2017, total debt held by  federal government accounts was $5.6 trillion.

Interest calculations on the public debt differ depending on the types of  securities, their associated terms, and average interest rates. According  to Treasury, total interest paid on public debt for fiscal year 2017 was  approximately $294.8 billion.

In February 2009, as part of a broader plan to stabilize the housing  market and economy, Treasury established the Making Home Affordable  Program to help struggling families avoid possible foreclosure. As part of  this plan, Treasury announced a national modification program for first- lien mortgages, the Home Affordable Modification Program (HAMP). The  program offered eligible homeowners who are at risk of foreclosure  reduced monthly mortgage payments that are more affordable and  sustainable over the long term. Homeowners who chose to participate in  the program had to show (1) documented financial hardship and (2) an  ability to make their monthly mortgage payments after a modification.

HAMP works by encouraging participating mortgage servicers to modify  mortgages so struggling homeowners can have lower monthly payments  and avoid foreclosure. It has specific eligibility requirements for  homeowners and includes strict guidelines for servicers.

In December 2016, entrance into the Making Home Affordable program  expired. However, payments for previously approved participants in  HAMP will continue until approximately September 2023. According to  Treasury, HAMP’s fiscal year 2017 outlays were approximately   $4.1 billion.

For improper payment risk assessment purposes, DOJ has five mission- aligned program groups. The Law Enforcement group is the largest in  terms of annual outlays and consists of the following five components:  1.  the Bureau of Alcohol, Tobacco, Firearms, and Explosives;  2.  the Drug Enforcement Administration;  3.  the Federal Bureau of Investigation;  4.  Offices, Boards, and Divisions; and  5.  the United States Marshals Service.

According to DOJ, Law Enforcement’s fiscal year 2017 outlays were  approximately $11.8 billion.

Agriculture Risk Coverage and Price Loss Coverage

The Agriculture Risk Coverage (ARC) and Price Loss Coverage (PLC)  programs were authorized by the 2014 Farm Bill to provide farmers with  protection against adverse changes in market conditions. Although ARC  and PLC are considered two separate programs, they are grouped as one  program for the purposes of conducting improper payment risk  assessments. The programs are managed by the Commodity Credit  Corporation, whose activities are primarily administered by USDA’s Farm  Service Agency.

Within the ARC program, farmers have the choice of an individual-based  option, known as ARC-Individual, or a county-based option, known as  ARC-County. Both options provide revenue loss coverage to farmers  when the legislative guarantee for a crop exceeds the actual year  revenue.

PLC program payments are issued to farmers when a crop’s “reference  price,” as specified in the 2014 Farm Bill, is in excess of an average price,  which is determined at the national level each year for the covered  commodities.

ARC/PLC statutes and regulations establish a series of eligibility criteria  that farmers must meet in order to enroll in the programs. Among other  things, to be eligible farmers must produce a certain quantity of at least 1  of the 21 covered commodities, actively engage in the farming process,  meet income eligibility limits, and meet certain land conservation  requirements. According to USDA, ARC/PLC’s fiscal year 2017 outlays  were approximately $9.6 billion.

Four of the Five Risk Assessments Lacked Documentation to Support Their Risk Determinations, and Many of HHS’s Programs Were Not Assessed

HHS’s Improper Payment Risk Assessment for Head Start Lacked Documentation to Support Its Low Risk Determination, and Many Other Programs Were Not Assessed HHS’s Improper Payment Risk Assessment for Head Start Lacked Documentation to Support Its Low Risk Determination

In its fiscal year 2016 qualitative risk assessment, HHS assessed its  Head Start program as at low risk of susceptibility to significant improper  payments. However, HHS did not have sufficient documentation on how  it developed its risk assessments, so we could not determine if the risk  assessment process was designed to provide a reasonable basis for  making risk determinations.

Although HHS did take into account the nine risk factors, among other  factors, HHS did not document or effectively demonstrate how each  specific risk factor affected Head Start’s susceptibility to significant  improper payments. HHS’s improper payment risk assessment template  included the nine risk factors, among other factors, and described how  the divisions should consider each risk factor. However, HHS did not  document how the descriptors or individual risk factors relate to the  program’s susceptibility to significant improper payments. Further,  although HHS used a risk assessment template to assess each of the risk  factors, which included space for the divisions to provide additional  information regarding the risk determinations, the division responsible for  the Head Start program did not always provide sufficient documentation  or support for us to determine how it arrived at its risk determinations for  each risk factor. For example, see the following:

Eligibility determination: HHS considered the eligibility of initial  Head Start payments that HHS made to the initial grantees—local  organizations—as low risk. However, HHS did not consider the Head  Start eligibility decisions that these organizations made at the  subrecipient level—calling into question the reliability of HHS’s risk  assessment. In the Head Start program, local organizations, not HHS,  make the eligibility determinations for individuals to be enrolled in the  program. In addition, local organizations, not HHS, are responsible for  maintaining the documentation to substantiate the eligibility of  enrollees. HHS did not consider the impact of these determinations in  its improper payment risk assessment. Our analysis of improper  payment estimates from paymentaccuracy.gov for fiscal years 2016  and 2017 indicates that the inability to authenticate eligibility is one of  the largest root causes of improper payments.

Audit findings: HHS assigned a low-risk rating for findings from  oversight agencies. However, in the risk assessment, it identified nine  audit reports that the OIG issued pertaining to Head Start agencies  with findings on unallowable costs, enrollment, and misuse of grant  funds. According to agency officials, these OIG reports contained  findings related to costs and misuse of grant funds that are specific to  particular grantees and may not be indicative of widespread  programmatic issues. However, HHS did not document the rationale  for this assessment.

Program management report: HHS assigned a low-risk rating for  findings related to program management reports. According to HHS’s  Report to Congress on Head Start Monitoring for Fiscal Year 2015,  “allowable and allocable costs” was the most commonly cited  noncompliance issue in its fiscal reviews of grantees. Specifically,  8.8 percent of grantees included in a fiscal review were found to be  noncompliant with regard to allowable and allocable costs. However,  HHS did not document whether it considered the impact of  noncompliance by grantees in its Head Start risk assessment.

According to HHS officials, divisions were required to maintain supporting  documentation for their risk assessments, although submission of the  related documents along with the risk assessment was not mandatory.

HHS officials stated that this policy was orally communicated to the  divisions; however, it was not formally documented. Lack of a written  policy for the divisions to maintain such information may have contributed  to HHS’s inability to provide sufficient supporting documentation for its  low risk determinations.

HHS’s qualitative risk assessment for Head Start also did not document  or effectively demonstrate how the total score for all risk factors led to a  determination that the program was not susceptible to significant  improper payments. Our analysis of HHS’s risk assessment showed that  for several of its risk factors, HHS did not score those factors as low risk.  For example, HHS assigned a high-risk rating for three of the nine risk  factors: (1) permanency of the program, (2) volume of payments made  through the program, and (3) complexity per transaction. HHS’s risk  assessment did not document or support how it determined Head Start to  overall be at low risk for susceptibility to significant improper payments  given the high-risk ratings for certain risk factors. Without supporting  documentation, HHS cannot demonstrate, and we cannot determine, if  HHS’s low risk determination for Head Start was reasonable.

Additionally, based on HHS’s risk assessment scoring template, a  program could be considered “high risk” for all nine risk factors, but  because of the assigned weight given to each of the nine risk factors,  HHS’s final risk calculation would still not determine the program to be at  high risk of susceptibility to significant improper payments. According to  HHS officials, the agency has procedures to review the improper payment  risk assessments that the individual divisions perform; however, these  review procedures are not formally documented. HHS officials stated that  while no risk assessment has identified all nine risk factors as high risk, if  all nine risk factors were identified as high risk by a division, the agency  would require supporting documentation from the division for review and  could overrule the outcome calculated based on the risk assessment  scoring template if necessary. Without documented procedures for this  review process, HHS lacks assurance that this process, if applicable,  would consistently take place.

According to HHS, the fiscal year 2016 improper payment qualitative risk  assessment template used for Head Start was designed to calculate an  overall risk rating of low, medium, or high based on program management  responses to each individual risk factor. However, HHS did not have  documentation to demonstrate how it determined the weighting of the risk  factors or how the numerical risk level ranges from the risk assessment  template related to a program’s susceptibility to significant improper  payments. Additionally, HHS did not have documentation demonstrating  the basis for its determination that specific risk factors do or do not lead to  susceptibility to significant improper payments. HHS officials stated that  OMB does not have specific guidance on establishing weights for each  risk factor or assigning numerical risk level ranges to determine overall  susceptibility to significant improper payments. HHS officials also stated  that HHS developed its own numerical risk level ranges based on  experience and data from previous risk assessments. When asked for  documentation to support its weighting of the various risk factors, HHS  officials stated that they did not document this analysis. Without  documenting the basis for the assigned weights, HHS cannot  demonstrate, and we cannot determine, that its process for determining  Head Start’s susceptibility to significant improper payments was  reasonable.

Federal internal control standards state that management should develop  control activities to achieve objectives and respond to risks and  implement control activities through policies. As part of these standards,  management should clearly document internal controls and other  significant events in a manner that allows the documentation to be readily  available for examination. Additionally, management should periodically  review policies, procedures, and related control activities for continued  relevance and effectiveness. Further, federal internal control standards  state that management should use quality information to achieve the  entity’s objectives. As such, to reasonably determine if a program is  susceptible to significant improper payments, agencies’ risk assessments  would have a logical connection with, or bearing upon, the statutory  definition of significant improper payments. Until HHS revises its risk  assessment process to help ensure that it results in a reliable  assessment, it will be uncertain whether Head Start may be susceptible to  significant improper payments and therefore require an estimate of its  improper payments.

HHS Did Not Conduct Risk Assessments for Many of Its Programs and Activities

During our agency and program selection process, we found that HHS did  not assess many of its programs and activities at least once during the 3- year period from fiscal years 2015 through 2017, as required by IPIA.  Although HHS conducted improper payment risk assessments for a total  of 71 programs and activities during the 3-year period, based on our  analysis of HHS-provided outlay data, HHS did not conduct the required  risk assessment for at least 140 programs. For example, HHS did not  assess its Block Grants for Prevention and Treatment of Substance  Abuse program that had outlays of approximately $1.8 billion in fiscal year  2016. According to HHS officials, HHS has limited resources, so it took a  risk-based approach when selecting programs to include in its improper  payment risk assessment process. Further, HHS officials stated that HHS  was transitioning in fiscal year 2015 to a new risk assessment process.  As such, HHS’s procedures directed its divisions to select one program  per division for fiscal year 2015 and two programs per division for fiscal  years 2016 and 2017.

Federal internal control standards state that management should design  control activities to achieve objectives and respond to risks and  implement control activities through policies. Without properly designed  control activities to help ensure that all programs and activities are  assessed for susceptibility to significant improper payments at least once  every 3 years, as required by IPIA, there is an increased risk that HHS  may not identify all risk-susceptible programs and activities, resulting in  incomplete improper payment estimates.

Treasury’s Improper Payment Risk Assessments for Interest on the Public Debt and HAMP Lacked Documentation to Support Its Low Risk Determinations

In its fiscal year 2017 qualitative risk assessments, based on fiscal year  2016 outlay data, Treasury assessed its Interest on the Public Debt and  HAMP as at low risk of susceptibility to significant improper payments.  However, Treasury did not have sufficient documentation for how it  developed its risk assessments, so we could not determine if the risk  assessment process was designed to provide a reasonable basis for  making risk determinations.

Although Treasury did take into account the nine risk factors, among  other factors, it did not document or effectively demonstrate how each  specific risk factor affected the programs’ susceptibility to significant  improper payments. Treasury’s risk assessment templates for these  programs had 62 questions which required “Yes,” “No,” or “Not  applicable” responses. Treasury did not document how each of the 62  questions related to each program’s susceptibility to significant improper  payments. Further, the template did not require the bureaus responsible  for the Interest on the Public Debt and HAMP risk assessments to provide  documentation or support other than a check mark in response to these  questions. Without descriptions of how to answer the questions or  documentation to support the responses, we could not verify the  reasonableness of the Interest on the Public Debt or HAMP improper  payment risk assessments.

For example, the Interest on the Public Debt program’s risk assessment  questionnaire was completed for 11 different payment types under the  program. For the TreasuryDirect payment type, Treasury answered “No”  to the question, “Are there risks due to a high volume of payments for  TreasuryDirect?” Treasury did not provide documentation or other support  for how the agency determined that there was no risk for this question.  Further, since the template lacked descriptors, it is unclear if responses  related to the number of transactions or the dollar amount of transactions.  In fiscal year 2016, TreasuryDirect payments totaled almost $300 billion,  representing about 7.8 percent of all the federal government outlays. In  contrast, Treasury answered “Yes” to this same question for the HAMP  program, for which payments were about 1 percent (about $4 billion) of  the total payments made by TreasuryDirect.

Similarly, in the HAMP risk assessment questionnaire, Treasury  answered “No” to the question, “Are payment or payment eligibility  decisions made outside the agency?” However, under HAMP, financial  institutions, not Treasury, determine whether borrowers are eligible for  loan modification through the program. Treasury did not document why a  “No” response was appropriate.

Treasury’s risk assessments for Interest on the Public Debt and HAMP  also did not document or effectively demonstrate how the total scores for  all risk factors led to the determinations that the programs were not  susceptible to significant improper payments. For example, in its risk  assessment, Treasury’s responses indicated several improper payment  risks for Interest on the Public Debt, including (1) complexity of  administering the payment type, (2) unmitigated risks relying on  contractors to perform critical agency operations, and (3) payments being  made to incorrect payees or ineligible recipients.

Further, based on total payments for the Interest on the Public Debt,  Treasury would have to be over 99.97 percent accurate in its payments in  order for the activity to not reach the $100 million threshold for significant  improper payments. Treasury’s risk assessment did not document or  support how it determined Interest on the Public Debt to be at low risk for  susceptibility to significant improper payments considering these risks for  improper payments.

Similarly, Treasury’s responses in its risk assessment questionnaire  indicated several improper payment risks for HAMP, including (1) an  emphasis on expediting payments, (2) risks resulting from recent changes  in agency operations and personnel, (3) complicated criteria for manually  computing payments, and (4) a high volume of payments. Treasury’s risk  assessment did not document or support how it determined HAMP to be  at overall low risk for significant improper payments considering these  risks for improper payments. Without supporting documentation, Treasury  cannot demonstrate, and we cannot determine, if Treasury’s low risk  determinations for Interest on the Public Debt and HAMP were  reasonable.

Additionally, based on our analysis of Treasury’s risk assessment  template, a bureau could identify areas of risk related to each of the nine  risk factors for a program, but because of the assigned weights given to  each of the nine risk factors, Treasury’s final risk calculation would still not  determine the program to be at high risk of susceptibility to significant  improper payments.

According to Treasury officials, Treasury provides general instructions on  how to complete the risk assessment templates, but the bureaus are  responsible for assessing the risks. In addition, according to Treasury, the  fiscal year 2017 improper payment risk assessment template used for  Interest on the Public Debt and HAMP was designed to calculate an  overall risk rating of low, medium, or high based on bureau responses to  each individual question. However, Treasury did not have documentation  to demonstrate how it determined the weighting of the risk factors or the  numerical risk level ranges from the template related to the programs’  susceptibility to significant improper payments. Additionally, Treasury did  not have documentation demonstrating the basis for its determination that  specific risk factors do or do not lead to susceptibility to significant  improper payments. According to Treasury officials, Treasury considered  the severity of the impact on the program’s improper payments when  developing its weights for each question. However, Treasury officials  stated that they did not have documentary support for this analysis.  Without documenting the basis for the assigned weights, Treasury cannot  demonstrate, and we cannot determine, that its process for determining  its programs’ susceptibility to significant improper payments was  reasonable.

Federal internal control standards state that management should develop  control activities to achieve objectives and respond to risks and  implement control activities through policies. As part of these standards,  management should clearly document internal controls and other  significant events in a manner that allows the documentation to be readily  available for examination. Additionally, management should periodically  review policies, procedures, and related control activities for continued  relevance and effectiveness. Further, federal internal control standards  state that management should use quality information to achieve the  entity’s objectives. As such, to reasonably determine if a program is  susceptible to significant improper payments, agencies’ risk assessments  would have a logical connection with, or bearing upon, the statutory  definition of significant improper payments. Until Treasury revises its risk  assessment process to help ensure that it results in reliable assessments,  it will not be certain whether Interest on the Public Debt or HAMP may be  susceptible to significant improper payments and therefore require an  estimate of improper payments.

DOJ’s Improper Payment Risk Assessment for Law Enforcement Lacked Documentation to Support Its Low Risk Determination

In its fiscal year 2017 risk assessment, DOJ assessed its Law  Enforcement program as at low risk of susceptibility to significant  improper payments. However, DOJ did not have sufficient  documentation for how it developed its risk assessments, so we could not  determine if the risk assessment process was designed to provide a  reasonable basis for making risk determinations.

Although DOJ conducted a quantitative evaluation as part of its improper  payment risk assessment for its Law Enforcement program, the  evaluation did not reliably indicate the program’s susceptibility to  significant improper payments. Specifically, our analysis of Law  Enforcement’s improper payment risk assessment found that the  quantitative evaluation’s baseline was largely based on the prior fiscal  year’s improper payment amount identified through recovery activities,  which may not reliably represent the estimated improper payment amount  that the agency incurred. For example, improper payment recovery  activities do not include underpayments.

DOJ’s qualitative analysis on improper payments also did not document  or effectively demonstrate whether the program may be susceptible to  significant improper payments. Although DOJ’s risk assessment template  did take into account the nine risk factors, among other factors, and  descriptors of how the components should consider each risk factor, DOJ  did not document or effectively demonstrate how each specific risk factor  affected the program’s susceptibility to significant improper payments.  Further, although DOJ used a risk assessment template to assess each  of the risk factors, which included a voluntary comments section for each  risk factor so that components can explain answers or justify the risk  ratings, the components frequently left the comment sections blank. As  such, DOJ did not always provide sufficient documentation or support for  us to determine how the components arrived at their risk determinations  for each risk factor.

DOJ’s risk assessment for Law Enforcement also did not document or  effectively demonstrate how the total score for all risk factors led to the  determination that the program was not susceptible to significant  improper payments. For example, in its risk assessment, DOJ’s Offices,  Boards, and Divisions’ responses indicated risks for contract payments  related to (1) changes in funding, authorities, practices, or procedures;   (2) results of monitoring activities; (3) results of recapture audit activities;  (4) volume and dollar amount of payments; (5) inherent risks; and (6)  capability of personnel. DOJ’s risk assessment did not document or  support how it determined Law Enforcement to be at low risk for  susceptibility to significant improper payments given the identified risks  for certain risk factors. Without supporting documentation, DOJ cannot  demonstrate, and we cannot determine, if DOJ’s low risk determination  for Law Enforcement was reasonable. Additionally, based on our analysis  of DOJ’s risk assessment template, a component could identify areas of  risk related to each of the nine risk factors, but because of the assigned  weight given to each of the nine risk factors, DOJ’s final risk calculation  would still not determine the program to be at high risk of susceptibility to  significant improper payments.

According to DOJ, the fiscal year 2017 improper payment qualitative risk  assessment template used for Law Enforcement was designed to  calculate an overall risk rating of low, medium, or high based on  component responses to each individual risk factor. However, DOJ did  not have documentation to demonstrate how it determined the weighting  of the risk factors or the numerical risk level ranges from the template  related to the program’s susceptibility to significant improper payments.  Additionally, DOJ did not have documentation demonstrating the basis for  its determination that specific risk factors do or do not lead to  susceptibility to significant improper payments. Further, DOJ’s qualitative  risk assessment template indicated that the overall risk determination  does not relate to the program’s susceptibility to significant improper  payments. For example, the template stated that “a risk rating of high risk  for the purposes of this assessment does not mean that the payment type  is susceptible to significant improper payments but may indicate that  additional focus and testing should be placed on that payment type to  better estimate the improper payment rate for the payment type.” DOJ  officials stated that DOJ held internal discussions and considered the  severity of the impact on the program’s improper payments when  developing its weights for each risk factor. When asked for supporting  documentation, DOJ officials stated that OMB guidance does not direct  agencies to demonstrate how the weights for each risk factor or overall  risk ratings relate to the definition of significant improper payments.

However, without documenting the basis for the assigned weights, DOJ  cannot demonstrate, and we cannot determine, that its process for  determining Law Enforcement’s susceptibility to significant improper  payments was reasonable.

Federal internal control standards state that management should develop  control activities to achieve objectives and respond to risks and  implement control activities through policies. As part of these standards,  management should clearly document internal controls and other  significant events in a manner that allows the documentation to be readily  available for examination. Additionally, management should periodically  review policies, procedures, and related control activities for continued  relevance and effectiveness. Further, although OMB does not direct  agencies to demonstrate how the weights for each risk factor or overall  ratings relate to the definition of significant improper payments, federal  internal control standards state that management should use quality  information to achieve the entity’s objectives. As such, to reasonably  determine if a program is susceptible to significant improper payments,  agencies’ risk assessments would have a logical connection with, or  bearing upon, the statutory definition of significant improper payments.  Until DOJ revises its risk assessment process to help ensure that it  results in a reliable assessment, it will be uncertain whether Law  Enforcement may be susceptible to significant improper payments and  therefore require an estimate of improper payments.

USDA’s Improper Payment Risk Assessment for ARC/PLC Provided a Reasonable Basis for Its Risk Determination

USDA’s fiscal year 2017 improper payment risk assessment for ARC/PLC  consisted of a qualitative analysis and a quantitative evaluation. Both  assessments determined that the program was not susceptible to  significant improper payments. We found that the quantitative evaluation,  based on statistical sampling, provided a reasonable basis for USDA’s  determination that the program was at low risk for susceptibility to  significant improper payments. Specifically, based on its statistical  sample, USDA estimated that ARC/PLC’s improper payment rate was  0.73 percent of program outlays with an estimated improper payment  amount of $38.6 million. As such, the analysis clearly demonstrated that  ARC/PLC did not meet the statutory definition of significant improper  payments under IPIA—estimated improper payments that may have  exceeded either (1) 1.5 percent of program outlays and $10 million or   (2) $100 million (regardless of the improper payment rate).

Properly executed improper payment risk assessments are a cornerstone  of government-wide efforts to estimate and reduce such payments.  Although the qualitative risk assessments we reviewed for HHS,  Treasury, and DOJ considered the nine risk factors required by IPIA or  directed by OMB, none of them demonstrated how the factors affected a  program’s susceptibility to significant improper payments. Additionally,  despite the agencies identifying multiple factors as areas of risk in  individual program risk assessments, each of the agencies’ overall  determinations for the risk assessments we reviewed was “low risk,” and  none of the agencies had documentation with which to explain the basis  for their assessments.

Revising their processes for conducting improper payment risk  assessments, including preparing sufficient documentation to support the  assessments, would better position HHS, Treasury, and DOJ to  demonstrate the reliability of the assessments. Without properly designed  risk assessments, the departments will continue to be uncertain whether  improper payment estimates should be prepared for most programs we  reviewed, potentially affecting the completeness of their improper  payment estimates and hampering efforts to reduce improper payments.

We are making the following four recommendations—two to HHS, one to  Treasury, and one to DOJ:  The Secretary of Health and Human Services should revise HHS’s  process for conducting improper payment risk assessments for Head  Start to help ensure that it results in a reliable assessment of whether the  program is susceptible to significant improper payments. This should  include preparing sufficient documentation to support its risk  assessments. (Recommendation 1)

The Secretary of Health and Human Services should revise HHS’s  procedures for conducting improper payment risk assessments to help  ensure that all programs and activities are assessed for susceptibility to  significant improper payments at least once every 3 years, as required by  IPIA. (Recommendation 2)

The Secretary of the Treasury should revise Treasury’s processes for  conducting improper payment risk assessments for Interest on the Public  Debt and HAMP to help ensure that the processes result in reliable  assessments of whether the programs are susceptible to significant  improper payments. This should include preparing sufficient  documentation to support its risk assessments. (Recommendation 3)

The Attorney General should revise DOJ’s process for conducting  improper payment risk assessments for Law Enforcement to help ensure  that it results in a reliable assessment of whether the program is  susceptible to significant improper payments. This should include  preparing sufficient documentation to support DOJ’s risk assessments.  (Recommendation 4)

We provided a draft of this report for comment to OMB, HHS, DOJ,  Treasury, and USDA. DOJ and HHS provided written comments, which  are reproduced in appendixes II and III, respectively. OMB, HHS, and  Treasury provided technical comments, which we incorporated as  appropriate. Treasury’s Acting Director of its Risk and Control Group  notified us by email that Treasury concurred with the report and  recommendation. A USDA management analyst notified us by email that  USDA had no comments on the report.

In its written comments, HHS stated that it concurs with both  recommendations and is committed to reducing improper payments in all  of its programs. HHS also described actions it plans to take to address  these recommendations, including (1) issuing a written policy directing  divisions to maintain supporting documentation for risk assessments,   (2) documenting the agency review procedures for risk assessments that  the divisions perform and the rationale for assigning weights to the risk  factors, and (3) developing an automated program identification process  for monitoring and inclusion in risk assessments to help ensure that all  programs and activities are reviewed. The actions described by HHS, if  implemented effectively, would address our recommendations.

In its written comments, DOJ stated that it disagreed with our conclusions  and recommendation. DOJ explained that its risk assessment  methodology includes a qualitative evaluation and a quantitative analysis,  and that it considers the nine risk factors likely to contribute to improper  payments. Additionally, DOJ provided an overview of its risk assessment  tool and guidance and stated that its methodology includes all steps  required by OMB. We acknowledged in the draft report that DOJ did take  into account the nine risk factors, among others, as directed by OMB and  provided an overview of DOJ’s risk assessment template and process.

DOJ stated that it believes that some of our interpretations exceed the  risk assessment requirements, and believes that its methodology  complies with requirements and adequately demonstrates whether a  program may be susceptible to significant improper payments. DOJ  stated that the risk factor ratings summarized in its risk assessment  provided a clear link of how the individual risk factor ratings support the  overall assessed risk of significant improper payments. Further, DOJ  stated that the risk assessment tool provides sufficient documentation for  the formulas and logic for the risk rating conversions and weight-based  summarization of risk factor scoring.

We disagree that our interpretations exceed the risk assessment  requirements, and we continue to believe that DOJ’s risk assessment did  not adequately demonstrate whether a program is or is not susceptible to  significant improper payments. We believe that while agencies are not  specifically directed to demonstrate how the weights for each risk factor  or overall ratings relate to the definition of significant improper payments,  management should use quality information to achieve the entity’s  objectives as stated in federal internal control standards. As such, to  reasonably determine if a program is susceptible to significant improper  payments, agencies should have documentation to support how their risk  assessments provided a logical connection with, or bearing upon, the  statutory definition of significant improper payments. DOJ did not provide  sufficient support for how it determined the weighting of the risk factors or  the numerical risk level ranges. Because DOJ did not have sufficient  documentation for how it developed its risk assessment template, we  could not determine if the risk assessment was designed to provide a  reasonable basis for the risk determinations.

DOJ stated that the report does not accurately portray DOJ’s risk  assessment process. Specifically, DOJ stated that we incorrectly reported  that DOJ’s quantitative evaluation did not include improper payments  related to lack of documentation. Based on the information DOJ provided,  we removed the lack of documentation example from our report.

DOJ also stated that it was misleading to report that although DOJ’s risk  assessment template included a voluntary comments section for each risk  factor for components to explain answers or justify risk ratings, the  comment sections were frequently left blank. DOJ stated that its  components only need to provide a comment when they believe it is  necessary to qualify their responses and that obvious answers do not  need to be explained. However, as previously noted, DOJ did not provide  sufficient documentation or support for us to determine how the  components arrived at their risk determinations for each risk factor.  Without such documentation, DOJ cannot demonstrate, and we cannot  determine, whether DOJ’s assessment for each risk factor was  reasonable.

Further, DOJ stated that the Offices, Boards, and Divisions example was  inaccurate and misleading. DOJ stated that the summary table in its risk  assessment questionnaire documented that the risks identified were  determined to be low risk and therefore supported the conclusions  reached. DOJ also stated that its approach acknowledges that risks exist  in every disbursement process and allows process owners to assess the  level of risk that exists and determine whether a program may be  susceptible to significant improper payments. We disagree that the  Offices, Boards, and Divisions example is inaccurate or misleading.  Although we recognize that DOJ’s summary table, or scoring template as  referred to in the report, documented that the risks identified were  determined to be low risk, we do not believe that it provided support for  that determination. Specifically, the summary table was populated based  on component responses and predetermined weights to calculate an  overall risk rating of low, medium, or high; however, DOJ did not provide  documentation to demonstrate how it determined the weights of the risk  factors or the numerical risk level ranges involved in that calculation.  Without documenting the basis for the assigned weights, DOJ cannot  demonstrate, and we cannot determine, that its process for determining  Law Enforcement’s susceptibility to significant improper payments was  reasonable.

We continue to believe that our recommendation to DOJ is valid to help  ensure that DOJ’s risk assessment reliably results in determining whether  Law Enforcement may be susceptible to significant improper payments.

We are sending copies of this report to the appropriate congressional  committees, the Director of the Office of Management and Budget, the  Secretary of Agriculture, the Secretary of Health and Human Services,  the Secretary of the Treasury, the Acting Attorney General, and other  interested parties. In addition, this report is available at no charge on the  GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-2623 or davisbh@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix IV.

Appendix I: Objective, Scope, and Methodology

This report examines the extent to which certain federal agencies’  improper payment risk assessments for selected programs provided a  reasonable basis for determining susceptibility to significant improper  payments.

To address our objective, we reviewed improper payment risk  assessment requirements in the Improper Payments Information Act of  2002, as amended (IPIA), and the related guidance in Office of  Management and Budget (OMB) Circular A-123, Appendix C,  Requirements for Effective Estimation and Remediation of Improper  Payments (OMB M-15-02). We analyzed this statute and guidance to  identify key criteria that agencies must meet when conducting improper  payment risk assessments. IPIA identifies seven risk factors and OMB  guidance includes two additional risk factors that agencies must consider,  at a minimum, in their improper payment risk assessments to determine  susceptibility to significant improper payments. IPIA also directs agencies  to conduct risk assessments for all programs and activities at least once  every 3 years. We also reviewed relevant internal control standards to  determine the relevant processes and procedures needed to help ensure  that agencies conduct effective improper payment risk assessments to  determine the susceptibility to significant improper payments.

For this objective, we selected a nongeneralizable sample of 4 agencies  and five programs to review. To select the agencies, we considered data  for the 24 agencies subject to the Chief Financial Officers Act of 1990  (CFO Act). Specifically, we considered the timing of the agencies’  improper payment risk assessments, findings reported by the agencies’  inspectors general (IG), the number of programs and activities for which  the agencies reported improper payment estimates for fiscal year 2017,  the types of programs and activities that the agencies administered, and  agency gross outlays in fiscal year 2017. To ensure we were including  agencies that had most recently conducted improper payment risk  assessments, we limited our selection to agencies that conducted  improper payment risk assessments for any programs or activities in  fiscal year 2017. In order to avoid duplicate efforts, we also eliminated  agencies that reported IG findings related to risk assessments. We then  selected a mix of agencies with and without improper payment estimates  for fiscal year 2017, and ultimately selected 4 agencies based primarily  on their fiscal year 2017 outlays for programs determined to be not  susceptible to improper payments. Specifically, we selected one agency  that did not report any improper payment estimates, one agency that  reported a few improper payment estimates (for three or fewer programs  or activities), and one agency that reported several improper payment  estimates (for five or more programs or activities). We also selected one  agency that administered eligibility-based programs in fiscal year 2017  because of the unique application and approval processes generally  associated with eligibility determinations and their increased risk of  improper payments.

We then selected up to two programs or activities at each agency, for a  total of five programs. To facilitate our program selection, we requested a  listing of all programs and activities at the selected agencies that  underwent a risk assessment in fiscal years 2015 through 2017 (the most  recent 3-year period at the time of our review) along with the gross outlay  amounts associated with these programs and activities. Through our  selection process, we noted that the Department of Health and Human  Services (HHS) did not assess at least 140 of its programs and activities  in the 3-year period from fiscal years 2015 through 2017, and therefore  our program selection for HHS was limited to approximately 70 programs  or activities.

To select programs, we considered outlay data, the timing of the most  recent improper payment risk assessment conducted for each program or  activity, and whether eligibility determinations were required for payments  under each program or activity. Our selection was primarily based on the  size of program and activity gross outlays reported for fiscal year 2017.  We focused on outlays because the overall impact of any issues identified  with an agency’s risk assessment process may be greater for programs  and activities with higher gross outlays, as a higher volume of payments  or higher payment amounts could potentially involve higher improper  payments. Based on these data, we selected five programs for review.  Our findings are limited to the five selected programs and cannot be  generalized to all programs and activities at the 24 CFO Act agencies.  The agencies and relevant programs selected for review are shown in  table 3.

We interviewed officials at the selected agencies on their processes for  conducting improper payment risk assessments and reviewed  documented policies and procedures. We obtained the most recent  improper payment risk assessments that the agencies conducted on the  selected programs during the latest 3-year period at the time of our  review (fiscal years 2015 through 2017). We then analyzed those risk  assessments against relevant IPIA requirements, OMB guidance, and  internal control standards to determine whether the agencies had  evaluated the appropriate risk factors for improper payments,  appropriately considered those risk factors in their risk assessments, and  provided a reasonable basis for the risk determination. For any agencies  that did not adhere to the improper payment risk assessment  requirements, lacked supporting documentation for their risk  assessments, or did not provide a reasonable basis for the risk  determinations, we interviewed appropriate agency officials to determine  the reasons they did not. We also interviewed OMB staff regarding their  roles in developing risk assessment guidance.

Appendix II: Comments from the Department of Justice

Appendix III: Comments from the Department of Health and Human Services

Appendix IV: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Matthew Valenta (Assistant  Director), Stephanie Adams (Auditor in Charge), Marcia Carlsen, Pat  Frey, Gina Hoover, Diana Lee, Zhen Li, and Charles Varga made key  contributions to this report.