Troubled Asset Relief Program: Monitoring of the Hardest Hit Fund Program Could Be Strengthened

Treasury established the HHF program in 2010 to help stabilize the housing market and assist homeowners facing foreclosure in the states hardest hit by the housing crisis. Through HHF, Treasury has obligated a total of $9.6 billion in Trouble Asset Relief Program funds to 19 state HFAs. HFAs use funds to implement programs that address foreclosure and help stabilize local housing markets—for example, by demolishing blighted properties. Congress extended HHF in 2015, and HFAs must disburse all HHF funds by December 31, 2021, or return them to Treasury.

The Emergency Economic Stabilization Act of 2008 included a provision for GAO to report on Troubled Asset Relief Program activities. This report focuses on the HHF program and examines, among other objectives, (1) the extent to which Treasury's monitoring addresses leading practices for program oversight and (2) HFAs' progress toward program targets.

GAO reviewed documentation of Treasury's HHF monitoring practices, interviewed HFAs (selected based on differences in program types implemented) and Treasury officials, and reviewed information on how HFAs developed program targets.

For its Housing Finance Agency Innovation Fund for Hardest Hit Markets (HHF), the Department of the Treasury (Treasury) has addressed or partially addressed all 14 leading monitoring practices that GAO identified. For example, Treasury periodically collects performance data from housing finance agencies (HFA) and analyzes and validates these data. However, while Treasury requires HFAs to regularly assess the risks of their programs, it does not systematically collect or analyze these assessments. As a result, Treasury is missing an opportunity to ensure that HFAs are appropriately assessing their risk. Also, Treasury does not require HFAs to consistently document which of their staff are responsible for internal control execution. This documentation could help HFAs wind down their programs, particularly as staff turn over.

Most HFAs met Treasury's goals for drawing down HHF funds, with $9.1 billion disbursed to HFAs as of September 2018. HHF programs have assisted hundreds of thousands of distressed homeowners since 2010. However, the data Treasury has collected are of limited use for determining how well HFAs met their goals for assisting households and demolishing blighted properties, or for evaluating the HHF program overall. For example, Treasury did not develop a consistent methodology for HFAs to use when setting performance targets, which limits Treasury's ability to compare across programs or assess the HHF program as a whole. Further, GAO's guide to designing evaluations states that where federal programs operate through multiple local public or private agencies, it is important that the data these agencies collect are sufficiently consistent to permit aggregation nationwide. Although HFAs have until the end of 2021 to disburse their HHF funds, many programs are beginning to close, making it too late for meaningful changes to Treasury's approach to performance measurement. However, should Congress authorize Treasury to extend the program beyond December 2021 or establish a similar program in the future, it would be useful at that time for Treasury to develop a program evaluation design that would allow the agency to assess overall program performance, as well as performance across HFAs and program types.

GAO recommends that Treasury collect and evaluate HFAs' risk assessments and routinely update staffing documentation. Treasury agreed with these recommendations and stated that it has already taken steps toward addressing them.

The housing crisis that began in 2007 led to unprecedented home price  declines, foreclosures, and high unemployment in certain parts of the  country. In response, under the authority granted to it by the Emergency  Economic Stabilization Act of 2008, the Department of the Treasury  (Treasury) established the Troubled Asset Relief Program (TARP).  Through TARP, Treasury funded housing programs intended to help  prevent avoidable foreclosures and preserve homeownership, including  the TARP Housing Finance Agency Innovation Fund for Hardest Hit  Markets, also known as the Hardest Hit Fund (HHF). HHF, established in  2010, provides funding to selected states to develop innovative solutions  to housing market difficulties in their states. As of October 2018, Treasury  had obligated $9.6 billion to 19 state housing finance agencies (HFA)— state-chartered authorities established to help meet affordable housing  needs—to design and implement HHF programs. Participating HFAs  have until December 31, 2021, to disburse these funds.

The Emergency Economic Stabilization Act of 2008 provided GAO with  broad oversight authorities for actions taken under TARP activities and  included a provision that we report on TARP activities and performance.  We have continued to provide updates on the HHF program. This report  provides an update on the HHF program as of December 31, 2017. This  report (1) determines the extent to which Treasury’s monitoring of HHF  addresses leading practices for program oversight, (2) provides  information on HFAs’ active programs and the status of HFAs’ progress  toward program targets, and (3) describes challenges in implementing  HHF programs that HFAs and others identified.

To assess the extent to which Treasury’s monitoring of HHF addresses  leading practices for program oversight, we obtained and reviewed  documentation of Treasury’s policies and procedures for monitoring the  HHF program. We identified 14 monitoring and oversight practices based  on a review of federal internal control standards, Office of Management  and Budget (OMB) guidance, and GAO reports on leading monitoring  activities. Although Treasury is not required to follow all of the guidance  that we identified, we determined that the guidance describes practices  that are helpful for creating an effective monitoring framework. We  selected practices that focused on the structure of Treasury’s oversight  framework, the performance measures Treasury requires HFAs to track,  goal setting, and communication with external parties. We assessed the  extent to which Treasury’s current monitoring policies and procedures  addressed the leading practices. More specifically, we determined  whether Treasury’s monitoring policies and procedures addressed,  partially addressed, or did not address leading practices. Additionally, we  compared evidence of Treasury’s 2016 and 2017 monitoring activities for  all participating HFAs to the agency’s monitoring policies and procedures.

To describe active HHF programs and the status of HFAs’ progress  toward program targets, we reviewed contracts and quarterly  performance reports to identify HHF programs that were active as of  December 2017. We defined programs as active if they had a total  allocation approved by Treasury, were accepting applications, and were  disbursing HHF funds as of December 2017. For each of these programs,  we reviewed the HFA’s contract with Treasury to identify the most current  targets for either assisting homeowners or demolishing blighted  properties. We also analyzed quarterly performance reports on program  outputs (defined as the products or services delivered) and outcomes  (defined as the consequences of carrying out an activity). Through a  review of program documentation and information from knowledgeable  officials, we found that Treasury’s output data on homeowners assisted  and blighted properties demolished were sufficiently reliable for the  purpose of describing program outputs. However, as we discuss later in  this report, we found that outcomes data, such as the number of  homeowners who are no longer participating in HHF programs, were not  sufficiently reliable for our purposes due to differences in HFAs’  interpretations of outcomes data definitions, among other things.

To identify the factors that HFAs and other stakeholders viewed as  challenges for the HHF program, we reviewed related audit reports and  interviewed officials from four HFAs, selected based on their mix of HHF  programs, proportion of HHF funds disbursed, and geographic diversity.  For two of these HFAs, which had blight-elimination programs, we  conducted site visits to observe activities related to blight elimination  efforts. For this and the other objectives, we also interviewed mortgage  servicers and organizations that work with HFAs and housing counseling  agencies. To identify the factors that Treasury identified as challenges for  the HHF program, we reviewed documentation of Treasury’s monitoring  reports for 2016 and 2017 and interviewed Treasury officials. See  appendix I for a full description of our scope and methodology.

We conducted this performance audit from November 2017 to December  2018 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

Treasury established HHF in February 2010 to help stabilize the housing  market and assist homeowners facing foreclosure in the states hardest hit  by the housing crisis. The HHF program is implemented by Treasury’s  Office of Financial Stability. Treasury obligated funds to 18 states and the  District of Columbia. Treasury allocated funds to each state’s HFA to help  unemployed homeowners and others affected by house price declines.  HFAs, in turn, design their own programs under HHF specific to local  economic needs and circumstances pursuant to their contracts with  Treasury.

Treasury allocated $9.6 billion in HHF funding to 19 HFAs in five rounds.  As described below, Treasury allocated $7.6 billion to participating HFAs  during the first four rounds of funding, all of which occurred in 2010. HFAs  were required to disburse these funds by December 2017.

Round one: In February 2010, Treasury allocated $1.5 billion to the  HFAs in the five states that had experienced the greatest housing  price declines—Arizona, California, Florida, Michigan, and Nevada.

Round two: In March 2010, Treasury allocated $600 million to the  HFAs in five states with a large proportion of their populations living in  counties with unemployment rates above 12 percent in 2009—North  Carolina, Ohio, Oregon, Rhode Island, and South Carolina.

Round three: In August 2010, Treasury allocated $2 billion to the  HFAs in nine of the states funded in the previous rounds, along with  the HFAs for eight additional states and the District of Columbia, all of  which had unemployment rates higher than the national average in  2009. The additional HFAs that received funding were Alabama, the  District of Columbia, Georgia, Illinois, Indiana, Kentucky, Mississippi,  New Jersey, and Tennessee.

Round four: In September 2010, Treasury allocated an additional  $3.5 billion to the same 19 HFAs that received HHF funding through  the previous rounds.

In December 2015, the Consolidated Appropriations Act, 2016 authorized  Treasury to make an additional $2 billion in unused TARP funds available  to existing HHF participants. In early 2016, Treasury announced a fifth  round of HHF funding. According to Treasury and HFA officials and other  stakeholders, by that time some of the participating HFAs had begun to  wind down their programs by letting go of program staff or making other  changes after they had disbursed most of their funding from the first four  rounds. Treasury allocated this additional $2 billion in two phases.

Round five, phase one: In February 2016, Treasury allocated $1  billion to 18 of the HFAs that had previously been awarded HHF funds  based on each state’s population and utilization of previous HHF  funds. In order to qualify for phase one funding, states had to have  drawn at least 50 percent of their previously received funding.

Round five, phase two: In April 2016, Treasury allocated an  additional $1 billion to 13 HFAs that applied and sufficiently  demonstrated to Treasury their states’ ongoing housing market needs  and the ability to effectively utilize additional funds. The HFAs that  received funding were California, District of Columbia, Illinois, Indiana,  Kentucky, Michigan, Mississippi, New Jersey, North Carolina, Ohio,  Oregon, Rhode Island, and Tennessee.

In conjunction with the fifth round of funding, Treasury extended the  deadline for disbursement to December 31, 2021. Treasury also  determined that HFAs must finish reviewing and underwriting all  applications for final approval to participate in the program no later than  December 31, 2020. HFAs that do not disburse HHF funds by the  December 31, 2021, deadline will have to return the remainder of the  funds to Treasury. See figure 1 for an overview of the allocation amounts  and disbursement deadlines.

Under HHF, HFAs designed locally tailored programs that address HHF’s  goals of preventing foreclosures and stabilizing housing markets. These  programs had to meet the requirements of the Emergency Economic  Stabilization Act of 2008 and be approved by Treasury. Treasury  categorizes programs into six types, which are discussed in detail later in  this report, including programs that provide monthly mortgage payment  assistance and programs that reduce the principal of a mortgage.  Programs vary by state in terms of eligibility criteria and other details.

HFAs contract with various stakeholders to implement HHF programs,  including mortgage servicers and, in some cases, housing counseling  agencies and land banks. The types of stakeholders involved vary  depending on program design. For example, HFAs with blight elimination  programs may choose to provide HHF funding to a local land bank to  demolish and green blighted properties in distressed housing markets.

Also, HFAs may contract with housing counseling agencies approved by  the Department of Housing and Urban Development (HUD) to identify  eligible applicants at risk of foreclosure.

HFAs are required to report performance information on each of their  HHF programs to Treasury on a quarterly basis. This information includes  outputs, such as the number of homeowners assisted or properties  demolished, as well as outcomes, such as the number of homeowners  who are no longer participating in HHF programs. The specific types of  performance information that Treasury requires HFAs to report vary  depending on the program type and include both intended and  unintended consequences of the program. For example, HFAs with  mortgage payment assistance programs must report on the number of  homeowners who have transitioned out of the program due to specific  changes in their circumstances, such as regaining employment. HFAs do  not have to report on the number of borrowers who transitioned out of the  program into foreclosure sales, short sales, or deeds-in-lieu of foreclosure  for their down payment assistance programs because the assistance is  provided on behalf of a buyer who is purchasing, not selling or otherwise  exiting, the home. Treasury provides HFAs with spreadsheet templates,  which HFAs are to fill out and submit back to Treasury. The templates  include data-reporting guidance in the form of a data dictionary, which  describes the data elements HFAs are to report.

Participating HFAs’ HHF programs are governed by a participation  agreement, or contract, with Treasury that outlines the terms and  conditions in providing services that the HFA must meet as a recipient of  HHF funds. Each agreement includes reporting requirements, program  deadlines, and descriptions of permitted administrative expenses.  Additionally, agreements include detailed descriptions of the HHF  programs that Treasury has approved. Program descriptions include  details such as eligibility criteria, structure of assistance, and the  estimated number of participating homeowners.

Participation agreements may be amended with Treasury approval to  reflect changes to HHF programs, such as new requirements from  Treasury or changes in the amounts HFAs allocate to each program. As  an example, in 2015 Treasury added new conditions, called utilization  thresholds, to each HFA’s participation agreement. The thresholds  establish the percentage of allocated funds each HFA was required to  draw from its Treasury account by the end of each year from 2016  through 2018. If an HFA did not meet a threshold, Treasury reallocated a  portion of the additional funds received during the fifth round to HFAs that  did meet the threshold. If an HFA would like to make a change to an HHF  program, the HFA must submit a request to Treasury that outlines the  proposed change. Treasury reviews the proposal through an  interdisciplinary committee and, if the proposal is approved, amends the  participation agreement. As of December 2017, the 19 participating HFAs  had each received approval from Treasury and executed between 9 and  21 amendments to their individual participation agreements.

Treasury’s Monitoring of HHF Addresses or Partially Addresses Leading Practices for Program Oversight

Treasury’s policies and procedures to monitor HFAs’ implementation of  the HHF program address 10 leading monitoring practices, including  practices related to the collection of periodic performance reports and  validation of performance through site visits. However, Treasury’s  assessment of HFAs’ internal control programs, development of  performance indicators, documentation of goals and measures, and  documentation of HFAs’ monitoring could better address leading  practices (see fig. 2).

Treasury Addressed 10 Leading Practices for Monitoring

Regular Monitoring of Policies and Procedures

Treasury created policies and procedures to guide regular oversight of  HFAs’ implementation of HHF. According to internal control standards for  the federal government, management should design control activities to  achieve objectives and implement control activities through policies— such as by periodically reviewing policies, procedures, and related control  activities. In addition, management should establish and operate  activities to monitor the internal control system and evaluate the results— for example, through ongoing monitoring procedures and separate  evaluations. Treasury documented procedures for key areas of its  monitoring framework, including providing funds to HFAs, evaluating  HFAs’ requests to change their programs, collecting financial and  performance information from HFAs, conducting site visits, and  addressing fraud detection and mitigation for Treasury’s staff. Treasury  regularly updates the policies and procedures it created and reviews its  compliance oversight procedures annually. In addition, Treasury regularly  conducts site visits to HFAs, as discussed below.

Treasury uses a risk-based approach to selecting HFAs for its regular site  visits. This approach is consistent with leading practices we have  developed for managing fraud risk, which state that agencies should  employ a risk-based approach to fraud monitoring by taking into account  internal and external factors that can influence the control environment.  In 2018, Treasury began using a point-based, 29-factor approach to  selecting HFAs for site visits for compliance reviews, taking into account  factors such as whether prior fraud was detected or reported,  observations from HFAs’ compliance reviews, administrative dollars spent  compared to program assistance provided, and whether HFAs have  documented blight-specific policies and procedures. According to  Treasury staff, during site visits Treasury determines its test and sample  sizes for a risk-based review of an HFA’s programs.

Treasury also uses a risk-based approach to responding to potentially  impermissible payments, and according to Treasury staff, its responses  depend on the circumstances. If an HFA notifies Treasury of issues  related to inappropriate payments involving fraud, waste, or abuse,  Treasury staff notify and work with the Office of the Special Inspector  General for the Troubled Asset Relief Program (SIGTARP) to provide  technical assistance as needed. In 2017, Treasury implemented  additional procedures with regard to HFAs’ administrative expenses. If  Treasury identifies an administrative expense issue during a site visit,  Treasury requires the visited HFA to undertake a multistep review of its  administrative expenses, including reviewing additional administrative  expenses if similar problems are identified during the initial review. The  HFA is required to reimburse HHF for any administrative expenses that  were not made in accordance with federal cost principles. Additionally,  Treasury may require the HFA to create a plan for corrective action.

Periodic Collection of Performance Reports and Data from Implementing Partners

Treasury collects performance information from participating HFAs on a  regular basis, which a compliance team receives and reviews. These  efforts are consistent with internal control standards, which state that  management should use quality information to achieve the entity’s  objectives, such as by obtaining relevant data from reliable sources.  Treasury tracks its receipt of agencies’ quarterly performance reports and  financial statements, as well as HFAs’ annual internal control  certifications. Quarterly performance reports include information about  homeowners, such as the number of homeowners who receive or are  denied assistance. These reports also include program-specific  performance data, such as the median assistance amount, and  outcomes, such as the number of program participants who still own their  home. According to HFAs’ participation agreements, HFAs are required to report performance information through the end of their programs. In  addition, Treasury collects informal monthly updates from HFAs on their  program performance and is in frequent contact with HFAs by phone to  obtain information on HFAs’ performance, including any challenges states  are facing, according to Treasury staff and HFAs with whom we met.  Treasury also collects reports on the impact of blight elimination  programs, which HFAs with these programs are required to submit to  Treasury.

Treasury regularly analyzes the performance and financial data that it  collects through quarterly performance reports, quarterly unaudited  financial statements, and annual audited financial statements that HFAs  are required to submit. Periodic analysis of these materials is consistent  with standards for internal control, which state that management should  design control activities to achieve objectives and respond to risks—for  example, by establishing activities to monitor performance measures and  indicators. Treasury uses information from quarterly performance  reports to produce quarterly reports for the public on the number of  homeowners who received or were denied assistance, among other  things. Treasury also includes data on the extent to which states have  spent their HHF funding in monthly reports to Congress. Additionally,  Treasury analyzes quarterly unaudited and annual audited financial  statements to monitor HFAs’ spending of program funds and identify any  areas of concern. According to Treasury staff, the agency also uses  performance information HFAs report quarterly, such as the number of  homeowners who receive or are denied assistance, to assess whether  HFAs are making sufficient progress in effectively utilizing program funds  to reach the targets for assisting homeowners.

Procedures for Ensuring Quality of Performance Data

Treasury has procedures to assess the quality of HFAs’ performance data  when reviewing quarterly performance reports and conducting site  visits. These procedures are consistent with internal control standards,  which state that management should use quality information to achieve  the entity’s objectives, such as by evaluating data sources for reliability.  According to Treasury staff, beginning in the first quarter of 2018,  Treasury required all participating HFAs to upload their performance data  into a system that does basic data reliability testing, such as ensuring the  numbers submitted by HFAs are consistent with data submitted for  previous quarters. This system flags outliers or large changes for further  review. Prior to this requirement, HFAs could use the system optionally.  HFAs are able to upload their data as frequently as they want to check for  errors or inconsistencies.

After performance information is uploaded into the system, two Treasury  staff review any issues flagged by the system and follow up with HFAs to  resolve them. According to Treasury staff, as an additional validation  step, Treasury staff conducts a reconciliation by checking whether the  funds reported in HFAs’ performance reports match the data in the HFAs’  quarterly financial reports. After Treasury reviews each HFA’s  performance data, it combines that information to create quarterly reports.  In addition, Treasury staff told us that they do a detailed review of HFAs’  financial statements during site visits, including but not limited to the  timeliness of financial reporting, corrections to reports after the reporting  cycle, and supporting documentation for all categories of expenditures  sampled during the review.

Roles and Responsibilities of Personnel Responsible for Monitoring

Treasury documents the offices that are responsible for receiving and  reviewing monitoring materials, the deadlines for receiving this  information, and the responsibilities of staff who execute internal control.  This documentation is consistent with internal control standards, which  state that management should implement control activities through  policies, such as by documenting each unit’s internal control  responsibilities. The standards also state that management should  remediate identified internal control deficiencies on a timely basis, such  as by having personnel report internal control issues through established  reporting lines. Treasury’s policies and procedures document which  offices are in charge of executing its monitoring procedures, such as  collecting required documentation, conducting site visits, and evaluating  HHF performance. Treasury informs HFAs of reporting lines to Treasury  through phone calls and emails. Treasury and HFA staff also noted that  they are in frequent contact with each other regarding administration of  the program.

Validation of Implementing Partners’ Performance through Site Visits or Other Means of Verification

Treasury uses regular (at least biennial) site visits, biweekly calls with  HFAs, and monthly informal performance updates as means of validating  HFAs’ performance. These practices are consistent with OMB guidance,  which states that a federal awarding agency may make site visits as  warranted by program needs. Treasury uses its site visits to assess  HFAs’ program implementation, conduct its own analyses of program  results, review HFAs’ use of program funds, and review HFAs’  implementation of internal controls. According to Treasury staff, Treasury  also uses site visits to corroborate the information HFAs report on their  program performance and use of HHF funds.

According to HFAs with whom we met, site visits typically last multiple  days and include entrance and exit conferences between Treasury and  HFA staff. During site visits, Treasury staff review documentation related  to homeowners and properties associated with the programs, quality  assurance processes, antifraud procedures, information technology and  data security, finances, and legal matters. After the site visit, Treasury  issues a report documenting its observations. Within 30 days of receiving  Treasury’s written report, HFAs are required to provide Treasury with a  written response describing how they will address any issues of concern.

Treasury included some procedures for project closeout in HFAs’  participation agreements. Creating procedures for project closeout is  consistent with OMB guidance, which states that agencies should close  out federal awards when they determine that applicable administrative  actions and all required work have been completed by the nonfederal  entity. Participation agreements describe various procedures for closing  out HHF programs, including requirements for the return of unexpended  funds to Treasury and final reporting and provisions for reimbursement of  expenses.

In addition, according to Treasury staff, Treasury is in the process of  developing and issuing wind-down guidance for HFAs in stages to  address specific areas of program activity. Agency officials also  discussed winding down the HHF program during Treasury’s 2018 Annual  Hardest Hit Fund Summit. The annual summit is a meeting that HFAs,  servicers, and other stakeholders are invited to attend to facilitate  information sharing among stakeholders involved in HHF. At the 2018  summit, the agency discussed topics that included final compliance and  financial reviews, program change requests, operational timelines, and  budgeting and staffing as they relate to the wind-down of HHF programs  and operations. In addition, as states have begun to close some of their  programs, Treasury has issued clarifying guidance to HFAs in order to  effectively wind down the HHF program—including on streamlining the  process for requesting changes to programs. Treasury staff also  performed outreach to each HFA in April 2018 about their wind-down  plans and, according to Treasury staff, the agency expects to prepare  written guidelines for HFAs on certain other topics related to winding  down the program, including reporting requirements, as appropriate.

Consideration of Performance Information in Making Management Decisions

Treasury uses performance information to assess whether HFAs are  performing at a satisfactory level. This practice is consistent with internal  control standards, which state that management should establish and  operate monitoring activities to monitor the internal control system and  evaluate results, which can include evaluating and documenting the  results of ongoing monitoring and separate evaluations to identify internal  control issues. In addition, management should remediate identified  internal control deficiencies on a timely basis. This can entail  management completing and documenting corrective actions to  remediate internal control deficiencies on a timely basis. Treasury staff  described the agency’s process of assessing HFAs’ performance as  “holistic.” As a part of this process, Treasury staff review the targets HFAs  set for assisting households or demolishing blighted properties and  monitor HFAs’ utilization rates. According to Treasury staff, if  performance and financial data suggest that an HFA is not making  sufficient progress toward its performance targets or is drawing funds too  slowly, Treasury collaborates with the HFA and the HFA must create a  plan to improve its performance. If an HFA is not responsive to Treasury’s  efforts, Treasury issues a performance memorandum requiring the HFA  to create a plan to address its deficiencies. As of October 2018, Treasury  had issued performance memorandums to seven HFAs—five in 2012 and  two in 2015.

Additionally, as mentioned previously, Treasury issues a report to each  HFA following each site visit describing any issues of concern Treasury  identified. Treasury requires HFAs to provide the agency with a written  response to the report within 30 days of the report date describing the  HFA’s plan for addressing any deficiencies.

Communication with External Parties to Address Risks and Achieve Objectives

Treasury regularly communicates with HFAs, servicers, and other  stakeholders interested in HHF, which is consistent with internal control  standards that state management should externally communicate the  necessary quality information to achieve the entity’s objectives. This can  include communicating with, and obtaining quality information from,  external parties using established reporting lines. According to Treasury  staff, Treasury holds biweekly calls with HFAs and servicers, facilitates  issue-specific working groups between HFAs and stakeholders, and holds  an annual summit related to HHF. HFA staff said Treasury staff are very  responsive to program-related questions. Treasury’s annual summit  allows interested parties, such as HFAs, servicers, and other  stakeholders, to discuss important issues related to HHF.

Treasury Partially Addressed Four Leading Practices

Identification, Evaluation, and Monitoring of Risks

To assist HFAs in designing their internal control activities, including  defining program objectives, Treasury created an optional risk  assessment matrix to help HFAs and their auditors identify and assess  HFAs’ risks. The matrix includes control objectives and example control  activities, and it allows HFAs to determine their risk tolerances for each  control objective. For example, for the risk of improper use of  administrative funds, the matrix includes “ensuring that appropriate  documentation exists to support HHF administrative expenses” as a  control objective, and it lists routine review of administrative payments by  internal auditors as an example control activity. HFAs can identify their  risk tolerances as low, medium, or high in the matrix. This matrix is  consistent with federal internal control standards, which state that  management should define objectives clearly to enable the identification  of risks and define risk tolerances.

However, Treasury does not systematically collect or evaluate HFAs’ risk  assessments. HFAs’ participation agreements require them to submit an  annual certification of their internal control programs by an independent  auditor to Treasury. According to Treasury staff, independent auditors  sometimes choose to include HFAs’ risk assessments with the annual  certification, and during site visits Treasury obtains documentation of  HFAs’ internal control programs, which sometimes includes their risk  assessments. Outside of these instances, Treasury does not routinely  collect HFAs’ risk assessments. Further, in those instances when  Treasury does collect them, it does not analyze the assessments to  evaluate whether the risk levels are appropriate. While Treasury does a  more in-depth evaluation of HFAs’ internal controls during site visits, this  review does not include evaluating the appropriateness of the risk levels  HFAs identified. For example, one of the risk assessment matrixes we  reviewed listed the HFAs’ administrative expenses as low-risk despite this  HFA having a history of alleged improper-payment related issues with its  HHF program, which Treasury’s review would not have evaluated.  Treasury officials told us that during site visits they may discuss the risk  levels that HFAs determine, but Treasury has not asked or required any  HFAs to change a risk level.

Failure to collect and evaluate HFAs’ risk assessments is inconsistent  with an important practice for preventing fraud we have previously  identified—monitoring and evaluating the effectiveness of preventive  activities, including fraud risk assessments and the antifraud strategy, as  well as controls to detect fraud and response efforts. Further, according  to internal control standards, management should identify, analyze, and  respond to risks related to achieving the defined objectives, and an  oversight body may oversee management’s estimates of significance so  that risk tolerances have been properly defined.

According to Treasury staff, the risk assessment matrixes are intended for  use by HFAs and their independent auditors in preparing for the annual  certification. They said that risk tolerances, or levels, are to be assigned  by HFAs and their independent auditors, not by Treasury, and that it  would be inappropriate for Treasury to interfere with their determination.  However, agreed-upon procedures performed by HFAs’ independent  auditors do not provide assurance or conclusion as to whether HFAs’ risk  levels are appropriate. For example, in two agreed-upon procedures  reports we reviewed, the auditors stated that the procedures performed  were based on the HFAs’ risk matrixes, but they did not mention  assessing whether the risk levels assigned to different controls were  appropriate. Treasury staff also said that Treasury expands its sample  size and criteria for specific programs or categories of expenses during a  compliance review where repeated or significant observations have been  previously found. However, by not collecting and evaluating HFAs’ risk  assessments, Treasury limits its ability to monitor the effectiveness of  HFAs’ preventive activities, controls to detect fraud, and response efforts.  In addition, Treasury is missing an opportunity to help ensure that risk  levels are appropriate.

Documentation That Monitoring Plans Were Executed

Treasury’s documentation of its efforts to monitor HFAs is consistent with  internal control standards, which state that management should establish  and operate activities to monitor the internal control system and evaluate  results and remediate deficiencies on a timely basis. More specifically,  the standards cite as characteristics of these principles that management  evaluate and document the results of ongoing monitoring and separate  evaluations to identify internal control issues, and determine appropriate  corrective actions for internal control deficiencies on a timely basis.  Treasury addresses these criteria by documenting its monitoring findings  through site visit reports, as previously discussed. Treasury requires  HFAs to provide the agency with a plan to address any issue described in  the site visit report within 30 days. In addition, Treasury addresses these  criteria by documenting HFAs’ responses and assessing whether the  issue has been addressed at the next site visit. Furthermore, Treasury  sets deadlines for and documents receipt of HFAs’ annual internal control certifications, quarterly financial and performance reports, and annual  audited financial statements. When underperforming HFAs are not  responsive to Treasury’s attempts to work with them to improve their  performance, Treasury documents the issues it has found and requires  the HFAs to create and submit a corrective plan.

Treasury also directs HFAs to establish and execute their own internal  control system, but it does not require HFAs to consistently document  which of their staff are responsible for internal control execution. HFAs  were required to submit staffing information within 90 days of joining HHF.  However, HFAs are not required to regularly update this information.  Further, Treasury’s written procedures for reviewing HFAs’ internal  control programs during site visits do not include reviewing  documentation of which HFA staff are responsible for responding to or  reporting internal control issues. These practices are inconsistent with  standards for internal control, which state that management should  establish an organizational structure, assign responsibility, and delegate  authority to achieve the entity’s objectives. The standards also note that  effective documentation can assist management’s design of internal  control by establishing the “who, what, when, where, and why” of internal  control execution.

We asked Treasury if it encouraged HFAs to document which personnel  are in charge of executing internal control procedures. Treasury staff  referred us to the initial requirement that HFAs submit staffing information  within 90 days of joining HHF and stated that there is no requirement that  HFAs update this information. Further, Treasury staff said that during site  visits they interview key HFA staff who execute internal controls and  document these interviews. However, this practice does not help ensure  that HFAs consistently provided updated information to their staff about  which of their staff are responsible for internal control execution. Without  requiring HFAs to routinely update their documentation, particularly as  HFAs are winding down their HHF programs and staff begin to turn over,  Treasury cannot be assured that HFAs are keeping their staff updated  about who is responsible for monitoring issues and internal control  execution.

Development of Relevant Output and Outcome Performance Indicators

Treasury and HFAs created quantitative output and outcome measures to  assess HFAs’ performance. For example, Treasury created utilization  thresholds to help ensure HFAs spend their HHF funds in a timely  manner. Also, HFAs created performance targets to estimate the number  of homeowners they could assist (or blighted properties they could  demolish) through HHF. These activities are consistent with an attribute  of successful performance measures—specifically, that measures should  have a numerical goal.

However, some of Treasury’s performance measures are not clearly  stated, and Treasury did not create consistent methodologies for HFAs to  use to assess the performance of their HHF programs. In our previous  work on attributes of successful measures, we identified that measures  should be clearly stated and that the name and definition should be  consistent with the methodology used to calculate them. While Treasury  provided HFAs with a data dictionary to describe the information HFAs  are required to report, Treasury defined the term “unique applicants” in a  manner that allows HFAs to count applicants differently, leading to  inconsistencies in HFAs’ methodologies for calculating some performance  measures. As discussed later in this report, Treasury also allowed and  sometimes required HFAs to self-define some data elements.

Additionally, performance measures should indicate how well different  organizational levels are achieving goals. However, Treasury did not  design a consistent methodology for HFAs to use to develop targets for  the number of homeowners and properties their HHF programs may  assist, and as discussed later in this report, HFAs we interviewed used  different methodologies. Because some of Treasury’s performance  measures are not clearly stated and because Treasury did not design  consistent methodologies for HFAs to use in setting targets, as HFAs  close down their HHF programs, Treasury has a limited ability to compare  performance across HFAs or aggregate these data to evaluate how well  the HHF program as a whole is achieving its goals.

Documentation of Goals and Measures and Their Relationship to Program Outputs

Treasury created goals and measures to assess HHF performance,  consistent with a practice we previously identified of creating performance  goals and measures that address important dimensions of program  performance and balance competing priorities. Treasury addressed this  practice by creating utilization thresholds for HFAs and inserting them in  HFAs’ participation agreements. Treasury also addressed this practice by  documenting its performance measures, using standardized  spreadsheets through which HFAs regularly report on outputs and  outcomes related to the services provided to distressed homeowners.

However, Treasury has not explicitly documented the relationship  between program outputs and the overall goals of the HHF program, and  it does not generally require HFAs to establish intermediate goals unless  the HFA has not met Treasury’s performance expectations. This is  inconsistent with practices we previously identified relating to results- oriented performance goals and measures. Among these practices are  including explanatory information on goals and measures in performance  plans and using intermediate goals to show progress or contributions  toward intended results. The main goals of HHF are to prevent  foreclosures and stabilize housing markets. However, Treasury has not  documented the relationship between many of the program outputs it  tracks and the main goals of the HHF program. According to Treasury,  the relationship between its outputs and the goals of HHF can be inferred  through various memorandums and materials it issued when HHF was  created. However, these documents do not explicitly explain the rationale  for the use of these output measures to assess HHF’s ability to stabilize  neighborhoods and prevent foreclosures. By not documenting the  relationship between HHF’s program outputs and services and the overall  goals of the HHF program or requiring all HFAs to set intermediate goals,  Treasury missed the opportunity to more proactively articulate a results- oriented focus for the HHF program.

Most HFAs Have Met Thresholds for Withdrawing Funds, but Inconsistent Targets and Outcome Measures Limit the Assessment of Program Performance Most Homeowners Participating in HHF Were Assisted through Mortgage Payment Assistance Programs

As of December 2017, the 19 participating HFAs had 71 active HHF  programs. Active HHF programs fall under one of six Treasury-defined  program types: mortgage assistance, reinstatement, transition assistance,  principal reduction, down payment assistance, and blight elimination.  Participating HFAs may have implemented additional HHF programs, but  these programs had either stopped disbursing funds or had not received  a total allocation from Treasury at the time of our review. Individual HFAs  may implement multiple programs—for example, the Mississippi HFA had  two active programs, and the South Carolina HFA had five.

The most common type of HHF program as of December 2017 was  mortgage assistance, as shown in table 1. All 19 HFAs had active  mortgage payment assistance programs as of December 2017. In  contrast, 3 HFAs had active transition assistance programs.

As of December 2017, we found that the 71 active HHF programs had  assisted approximately 400,000 homeowners and demolished almost  24,000 blighted properties. According to Treasury data, the majority of  homeowners who received HHF assistance participated in a mortgage  payment assistance program. Treasury data also indicate that transition  assistance programs assisted the smallest number of homeowners  relative to other HHF program types (see table 2).

HHF programs of the same program type can vary in a number of ways,  including eligibility criteria, length of time implemented, and number of  homeowners assisted. Within each program type, HFAs designed  programs that sometimes varied based on specific housing needs. For  example, while both the Nevada and Florida HFAs had active  reinstatement programs as of December 2017, these programs had  different eligibility criteria. The Nevada HFA’s reinstatement program  targeted low-to-moderate income homeowners who had fallen behind on  their mortgages. The Florida HFA offered a similar reinstatement program  for delinquent mortgages but also offered a program for senior  homeowners who had fallen behind on property taxes and other fees.

HHF programs also varied by duration and the amounts of assistance  provided as of December 2017. For instance, since all HFAs initially  launched mortgage payment assistance programs at the beginning of  HHF, these programs have been active for an average of 7 years. In  contrast, HFAs began implementing down payment assistance programs  in 2015. Additionally, the median amount of assistance provided varied by  program type. According to analysis of Treasury data from 2010 through  2017, assistance ranged from a median amount of $4,000 per household  for transition assistance programs to over $42,000 per household for  principal reduction programs.

The HHF program is beginning to wind down. As of September 2018,  Treasury had disbursed $9.1 billion of the $9.6 billion obligated under  HHF. According to Treasury officials, although HFAs may continue  issuing new approvals through December 31, 2020, most states have  already begun to close down HHF programs or will do so by the end of  2018 as they exhaust their available funds. These include California and  Florida, the two largest states in the program.

Most HFAs Have Met Thresholds for Withdrawing Funds from Treasury

According to Treasury officials, during the fifth round of funding Treasury  established new conditions for HFAs, called utilization thresholds, to help  maximize the use of the $2 billion in newly available funds. According to  documentation from Treasury, if an HFA does not meet its utilization  threshold, Treasury will reallocate a portion of the unused funds to HFAs  that did. The amount reallocated to each HFA is determined by state  population, the percentage of funds drawn by HFAs, and other factors.

The utilization thresholds for 2016 and 2017 were structured as follows:

2016. If an HFA did not draw at least 70 percent of its funding from  rounds one through four by December 31, 2016, 50 percent of its  round five funding would have been reallocated.

2017. If an HFA did not draw at least 95 percent of its funding from  rounds one through four by December 31, 2017, 75 percent of its  round five funding would have been reallocated.

Most HFAs have met Treasury’s 2016 and 2017 utilization thresholds.  More specifically, the 18 HFAs eligible for round five funding met the 2016  utilization threshold. As a result, Treasury did not reallocate any HHF  funds for that year. As of December 2017, 17 of the 18 HFAs eligible for  round five funding met the 2017 utilization threshold. The Nevada HFA  drew 70 percent of its funding for rounds one through four as of  December 31, 2017, and therefore did not meet the 2017 utilization  threshold. As a result, Treasury reallocated approximately $6.7 million of  the Nevada HFA’s unused fifth round HHF funds to the 17 other HFAs.

As of September 2018, all HFAs had met the 2018 utilization threshold,  and Treasury had disbursed most of the funds obligated under HHF. If an  HFA did not draw at least 80 percent of its participation cap by December  31, 2018, an amount equal to the portion of round five funding that had  not been drawn from Treasury would have been reallocated.

Data on the Extent to Which HHF Programs Met Targets Are of Limited Use Because Treasury Did Not Develop a Consistent Methodology for Calculating Targets

The targets that HFAs set are of limited use for evaluating the  performance of individual programs, program types, HFAs, or the HHF  program overall. In their participation agreements, HFAs were required to  estimate the number of homeowners they intended to assist and, if they  had a blight elimination program, the number of blighted properties they  intended to demolish for each of their HHF programs. Treasury refers to  these estimates as targets.

HFAs that we spoke with used different methodologies to calculate these  targets. For instance, one of the HFAs we spoke to calculated targets for  the number of homeowners they could assist by dividing the program’s  total allocation by the average amount of assistance it anticipated  awarding to each homeowner. In contrast, another HFA calculated its  target for assisting homeowners by dividing that program’s total allocation  by the maximum amount of assistance homeowners could be awarded  through the program. According to Treasury staff, they did not develop a  consistent methodology for HFAs to use in setting these targets because,  in their view, HFAs are most familiar with local conditions and should  have flexibility in adjusting the program criteria or creating new programs  based on these conditions.

Internal control standards state that management should define  objectives clearly to enable the identification of risks and define risk  tolerances. In particular, the standards note the importance of stating  measurable objectives in a form that permits reasonably consistent  measurement. Further, our guide to designing evaluations states that  where federal programs operate through multiple local public or private  agencies, it is important that the data agencies collect are sufficiently  consistent to permit aggregation nationwide, which allows evaluation of  progress toward national goals. Because Treasury did not develop a  consistent methodology for HFAs to use when setting performance  targets, the targets HFAs developed do not permit consistent  measurement of program performance or an evaluation of how well the  HHF program as a whole met its goals. However, with the program  beginning to wind down, any changes going forward would not improve  the consistency of previously collected data or Treasury’s ability to  evaluate the program as a whole.

Treasury Collects Information on Outcomes for Some HHF Programs, but This Information Is of Limited Use

Treasury Requires HFAs to Report Some Outcome Information for Four Program Types

Treasury collects quarterly data on outcomes from HFAs that implement  four of the six HHF program types: mortgage payment assistance,  principal reduction, reinstatement programs, and transition assistance  programs. HFAs must track outcomes, both intended and unintended,  until a household is no longer involved with an HHF program. Intended  outcomes include, for example, the number of homeowners who  completed or transitioned out of an HHF program as a result of regaining  employment. Unintended outcomes include the number of homeowners  who transitioned out of an HHF program into a foreclosure sale. The type  of outcomes Treasury requires HFAs to track depends on the program  type.

Treasury did not design outcome measures in a way that would permit it  to use these data to evaluate whether HFAs or the overall program are  achieving the stated goals. More specifically, Treasury officials told us  that the data they collect on outcomes cannot be used to compare the  outcomes achieved by different HFAs or through different HHF program  types. According to Treasury officials, HFAs have historically had different  interpretations of Treasury’s outcome measures. Treasury revised its  template for HHF reporting in 2015 and 2017 to clarify certain  performance-related terms. However, Treasury officials told us that  conclusions drawn from HHF data on some outcomes are of limited use  because HFAs interpret Treasury’s guidance on these data differently.  Additionally, after it made revisions to guidance on performance reporting  in 2015, Treasury allowed—and in some cases required—HFAs to self- define certain data elements. For example, Treasury required HFAs to  define how they calculate the median principal forgiveness awarded by an  HHF program.

As previously discussed, a key attribute of effective performance  measurement is clearly stated performance measures with names and  definitions that are consistent with the methodology used to calculate the  measure. Additionally, we have noted in our guide to designing  evaluations that a program’s outcomes signal the ultimate benefits  achieved by a program and should be considered when evaluating a  program. Further, OMB has set the expectation that agencies should  conduct evaluations of federal programs. However, because Treasury  did not clarify certain outcome measures until 5 years into the program, or  take steps to ensure that HFAs calculated alternative outcomes  consistently, even after Treasury clarified its reporting guidance, the  alternative outcomes data that Treasury collects are of limited use for  evaluating the performance of HFAs, HHF programs by program type, or  the HHF program overall. As many programs are closing, further  clarification or changes would not capture the full scope of the program  and would not improve such evaluations.

Treasury Requires HFAs with Blight Elimination and Down Payment Assistance Programs to Conduct Impact Studies

Treasury requires HFAs with blight elimination and down-payment  assistance programs to identify indicators that are intended to track and  quantify the HHF program’s impact on targeted areas, although HFAs are  not required to report outcomes data to Treasury in their quarterly  performance reports for these program types. According to Treasury,  blight elimination and down payment assistance programs are focused on  stabilizing housing markets in targeted distressed areas to prevent  foreclosures, and therefore they are not required to report individual-level  outcomes for HFAs to report in quarterly performance reports. Treasury  officials told us that the impact of these program types upon  neighborhoods, such as increases in the values of properties in  neighborhoods where down-payment assistance or blight elimination  programs were used, may not be observable immediately but may appear  over time. As of August 2018, four of eight HFAs with blight elimination  programs had submitted impact studies to Treasury. Also, all HFAs with  down payment assistance programs have submitted studies to Treasury.

Three blight elimination program impact studies suggest that the  programs had positive impacts on targeted areas, although two of the  studies have important limitations. Studies on the programs in Michigan  and Ohio found that home prices increased in communities where  blighted properties were demolished. For example, the Ohio study found  there was about a 4-dollar increase in home values for every dollar spent  on the HHF-funded blight elimination program. However, this study  examined only 1 of the 18 counties that were served by the Ohio HFA’s  blight elimination program. A study on the Illinois program found that  certain key economic indicators had improved over a 6-year period in  areas targeted by the program. For example, the percentage of negative  equity mortgages in 9 of the 10 areas studied declined by an average of 7  percent between 2010 and 2016. However, the findings of this study do  not isolate the independent effect of the Illinois HFA’s blight elimination  program because other factors, such as local economic conditions, could  also affect the performance of key economic indicators.

Stakeholders Identified a Variety of Challenges in Implementing HHF Programs

Treasury, HFAs, and Mortgage Servicers Described Challenges Related to Implementing Programs

HHF stakeholders with whom we spoke described challenges in  implementing HHF programs related to staffing and multiple funding  rounds, program implementation, outreach to borrowers, program  accessibility, the variety of programs and their status, and external  factors. Both Treasury staff with responsibilities for monitoring HFAs’  implementation of HHF and stakeholders told us that these were the  types of topics discussed during regular phone calls and annual  meetings. Stakeholders included staff from four HFAs that are  implementing HHF programs, mortgage servicers and housing counseling  agencies that are involved with HHF, and other interested organizations,  including those that work with HFAs.

Staffing and multiple funding rounds. All four HFAs and various  stakeholders with whom we spoke told us that staff turnover at HFAs  presents challenges. In some cases, turnover has been related to the way  the HHF program has been funded. For example, staff from two HFAs  mentioned that either they let staff go or their temporary staff found more  permanent positions as the agencies spent down their initial HHF funds.  When Congress authorized Treasury to make additional TARP funds  available to HHF beginning in 2016, these HFAs had to hire and train new  staff. Treasury officials told us that many HFAs encountered staffing  challenges as a result of the program’s fifth funding round. Additionally,  staff from two servicers and an organization that advocates for HFAs told  us that HFA turnover presents challenges because it takes time for new  staff to become familiar with the program and for programs to ramp back  up.

Program implementation. Staff from most of the HFAs and servicers  with whom we spoke, as well as Treasury staff and other stakeholders,  told us that implementation of the HHF program was challenging. Specific  implementation challenges mentioned by HFAs included creating an in- house information system to manage HHF data; managing refinancing  requests from homeowners who have been awarded HHF funds (to help  ensure the HFA’s place as a lien-holder); and sharing information with  servicers. While Treasury helped to develop a system to facilitate the  sharing of loan-level information for the HHF program, one HFA and  some servicers noted that the system has not always worked smoothly.  Additionally, Treasury staff told us that a challenge HFAs are currently  facing is the wind-down of the HHF program. They stated that HFAs must  determine how they should advertise to the public, internal staff, and  external partners that programs are closing; when they should stop  accepting applications; and what resources are available for activities  related to program closeout.

Outreach to homeowners. All four HFAs and an advocacy organization  told us that it can be challenging to effectively reach eligible homeowners.  As an example, staff from one HFA told us that housing counseling  agencies have been an effective tool for making homeowners aware of  HHF programs but that there are fewer foreclosure counselors available  to homeowners now compared to when the HHF program started in 2010.  Staff from an HFA that closed its HHF programs to new applicants after  the initial funding rounds told us that it was challenging to communicate to  the public, and therefore to potential clients, that its HHF programs were  reopening after they received additional funding. Additionally, a  representative of a nonprofit organization that works to address  challenges in the mortgage market told us that many people did not know  about the HHF program and that program information was hard for  consumers to find on many states’ websites.

Program accessibility. According to academic research and two  stakeholders (an advocacy group and a housing counseling agency), the  accessibility of an HFA’s program can affect program participation. A  2014 study of Ohio’s HHF program found that the design of the program  hampered accessibility and therefore program participation. The  program was designed to require registrants (those who started the application process) to continue the application process by working with a  housing counseling agency. The study found that registrants who lived  within 5 miles of their assigned housing counseling agency submitted a  complete application almost 32 percent of the time, while those who lived  over 50 miles away submitted a complete application about 18 percent of  the time. Similarly, a representative for an organization that advocates on  behalf of low-income homeowners noted that the design of one state HHF  program requires applicants to meet with specific housing counseling  agencies to complete the application process. However, the housing  counseling agencies to which applicants are assigned may not be nearby.  The representative stated that in some cases, homeowners are assigned  to a housing counseling agency that is located 3 or 4 hours away from  where the homeowners live. According to the advocacy group  representative, this design is particularly challenging for elderly  homeowners who may have trouble applying online and need personal  help.

Additionally, representatives for a housing counseling agency told us that  their state HFA stopped involving community organizations to guide  applicants throughout the application process once the HFA received  additional HHF funding in 2016 and instead chose to work with applicants  directly. They said this design may hurt homeowners who do not live near  the HFA and would benefit from in-person assistance that could be  provided close to their homes. A representative from the state’s HFA  confirmed that the HFA decided to work directly with applicants once it  received additional HHF funds in 2016. The representative stated that  while homeowners could also apply for HHF assistance online (after the  HFA changed the program design in 2016), the HFA’s system did not  accept electronic signatures. Thus, homeowners without the ability to  print and scan documents would need to come to the HFA’s office to  complete the application process.

Variety of programs and their status. Treasury officials noted that the  wide variety of programs that HFAs are implementing can create  operational challenges for HFAs. As an example, the officials explained  that HFAs may encounter challenges when their programs require  coordination with local partners. For example, land banks can encounter  delays in acquiring properties for demolition, and contractors may not do  demolition work properly or may attempt to increase the amounts that  they charge for their work after winning a contract.

Five mortgage servicers with whom we spoke described similar  challenges. For example, representatives from one servicer told us that it  was challenging to work with the 19 different HFAs because they all  implemented different HHF programs. The representative added that it  was particularly challenging if an HFA had a change in either leadership  or points of contact for the HHF program. Another servicer explained that  servicers have to review each HFA’s participation agreement and  subsequent updates. This servicer noted that updates to agreements can  create challenges, as the servicer needs to determine whether it can  provide what the HFA is requesting. Representatives from this and a third  servicer told us that it would have been helpful for servicers to have an  up-to-date list of active HHF programs. Further, one servicer told us that it  is challenging to help homeowners understand that each HFA and  program has different requirements and guidelines. As previously  discussed, Treasury communicates information to stakeholders, such as  servicers, through regular conference calls. However, Treasury expects  HFAs to keep their servicers abreast of the status of HHF programs  because HFAs contract directly with servicers.

Representatives from one HFA noted that it was challenging to keep  servicers updated on changes to their HHF programs. For example, they  reported that when the HFA made changes to its unemployment program,  servicers confused the program with another of the agency’s HHF  programs. The representatives also stated that they have had to make  many phone calls to try to keep servicers up to date.

External factors. Treasury officials and other stakeholders noted that  external factors such as changing market needs and natural disasters  have created challenges for some HFAs. Treasury officials noted that  some HFAs have had to change their HHF programs over time to  respond to changes in local housing conditions. An organization that  advocates for HFAs as well as an HFA similarly noted that changing  housing markets present challenges for HFAs, which have to adjust their  program offerings in an effort to continue to serve homeowners. As  previously discussed, HFAs must obtain Treasury approval to add or  revise their HHF programs, and they must document the changes by  amending participation agreements. Treasury officials also noted that  natural disasters can affect HHF programs because HFAs have to turn  their attention to post-disaster housing needs. Additionally, Treasury  officials stated that after a natural disaster it can become difficult to verify  the eligibility of applicants, particularly if key documents have been lost or  communication channels with homeowners or servicers are affected.

Treasury and SIGTARP Also Identified Challenges through Their Monitoring and Oversight Activities

Through its on-site monitoring efforts, Treasury has identified issues that  participating HFAs must address for their HHF programs. During on-site  reviews in 2016 and 2017, Treasury staff assessed selected HFAs’ efforts  in one or more Treasury-identified areas. As previously noted,  Treasury’s policy at the time of our review was to conduct on-site reviews  of each participating HFA at least once every 2 years. In 2016 Treasury  conducted on-site monitoring visits for 14 HFAs and identified issues that  the HFAs needed to address to improve their HHF programs. Issues  Treasury identified primarily fell into two areas. The first of these was  monitoring processes and internal controls—for example, Treasury found  that one HFA had not developed documentation of its compliance  procedures for a down payment assistance program. The other primary  area was homeowner eligibility—for example, Treasury found that an  HFA had misclassified the reasons that some homeowners were not  admitted into the state’s HHF program.

In 2017 Treasury conducted site visits to 15 HFAs. For this period,  Treasury’s most common issues related to homeowner eligibility and  administrative expenses. According to Treasury officials, the increase in  issues related to administrative expenses between 2016 and 2017 was a  result of greater agency focus on this topic. Treasury observed, for  example, that one HFA lacked sufficient documentation to support some  administrative expenses and that another HFA had misclassified some  administrative expenses. As previously discussed, HFAs are required to  provide Treasury with a written plan describing how they will address  issues Treasury identifies and reimburse HHF for any impermissible  expenses.

Through its oversight activities, SIGTARP reported that some  participating HFAs have encountered challenges related to appropriate  use of administrative expenses, management of their programs, and  blight removal. In August 2017, SIGTARP reported that participating  HFAs used $3 million in HHF funds for unnecessary expenses. The  report maintained that some HFAs were using their administrative funds  for expenses that were unnecessary. In a May 2018 hearing, SIGTARP  testified that some HFAs were not following federal cost principles related  to administrative expenses. Additionally, SIGTARP has issued reports  describing mismanagement of the HHF program by specific HFAs, as well  as challenges related to blight removal. While Treasury has disagreed  with the dollar amount of administrative expenses used inappropriately by  HFAs, it has also worked with HFAs and SIGTARP to address  SIGTARP’s findings.

As HHF programs begin to close and participating HFAs take steps to  ensure they spend all of their HHF funds before the program deadline,  opportunities exist in two areas for Treasury to manage risk and improve  program operation and closeout:

By not consistently and routinely collecting HFAs’ risk assessments,  Treasury limits its ability to monitor and evaluate the effectiveness of  HFAs’ preventive activities, controls to detect fraud, and response  efforts. Further, by not evaluating these risk assessments, Treasury is  missing an opportunity to help ensure that risk levels are appropriate.

As HFAs wind down their HHF programs and HFA staff are relieved of  their HHF-related positions, maintaining updated and accurate staffing  information can help ensure that HFA staff are informed of who in  their own offices is responsible for internal control execution.

Because Treasury did not implement the HHF program in a manner that  is consistent with standards for program evaluation design we previously  identified, the performance data that Treasury collects do not provide  significant insights into the program’s effectiveness. More specifically,  Treasury did not clearly state some of its performance measures; lacks documentation of the relationship between program outputs and  overall goals; did not design consistent methodologies for HFAs to use in setting did not require participating HFAs to use consistent methodologies to  calculate outcomes.

As a result, Treasury cannot aggregate key performance data or compare  performance data across HFAs or HHF program types to demonstrate the  results of the HHF program. As we have previously reported, OMB has  set the expectation that agencies should conduct evaluations of federal  programs. Moreover, our guide to designing evaluations states that where  federal programs operate through multiple local public or private  agencies, it important to ensure the data these agencies collect are  sufficiently consistent to permit aggregation nationwide in order to  evaluate progress toward national goals. Although HHF programs must  stop disbursing funds by December 31, 2021, many of the programs have  already ended or are in the process of winding down, making it too late  for changes to Treasury’s approach to performance measurement to have  a meaningful impact. However, we note that if Treasury were to extend  the current program, as it did after Congress provided additional funding  in 2015, or if Congress were to establish a similar program due to a future  housing crisis, it would be useful at that time for Treasury to develop a  program evaluation design that would allow the agency to assess overall  program performance, as well as assess performance across HFAs and  program types.

We are making the following two recommendations to Treasury:  The Assistant Secretary for Financial Institutions should annually collect  and evaluate HFAs’ risk assessments, which include HFAs’ risk levels.  (Recommendation 1)

The Assistant Secretary for Financial Institutions should ensure that the  documentation listing the HFA staff responsible for internal control  execution is updated routinely. (Recommendation 2)

We provided a draft of this report to Treasury for review and comment. In  its comments, reproduced in appendix IV, Treasury agreed with our  recommendations and stated that it has already taken steps toward  addressing them by enhancing the existing review procedures for HFA’s  risk assessments and staffing updates. Treasury also provided a  technical comment, which we incorporated.

We are sending copies of this report to the appropriate congressional  committees, the Secretary of the Treasury, and other interested parties.  In addition, the report is available at no charge on the GAO website at  http://www.gao.gov. We will make copies available to others upon  request. The report will also be available at no charge on our website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-8678 or ortiza@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs are listed on the last page of  this report. GAO staff who made major contributions to this report are  listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

The objectives of this report were to (1) determine the extent to which the  Department of the Treasury’s (Treasury) monitoring of the Hardest Hit  Fund (HHF) addresses leading practices for program oversight, (2)  provide information on housing finance agencies’ (HFA) active programs  and the status of HFAs’ progress toward program targets, and (3)  describe challenges in implementing HHF programs that HFAs and others  identified.

To determine the extent to which Treasury’s monitoring of HHF  addresses leading practices for program oversight, we used a scorecard  methodology to compare Treasury’s monitoring policies and procedures,  as implemented by 2016, against leading practices for an effective  monitoring framework. To create the framework, we reviewed key reports  and guidance related to monitoring, oversight, and performance  management. In particular we reviewed relevant leading practices from  internal control standards; previous GAO work on results-oriented  performance goals and measures,  key attributes for successful  performance measures, characteristics for successful hierarches of  performance measures, and managing fraud risk; and Office of  Management and Budget guidance on oversight. Although Treasury is  not required to follow all of the guidance that we identified, we determined  that the guidance describes practices that are helpful for creating an  effective monitoring framework.

To select the practices for the scorecard, we focused on practices  relevant to the structure of an oversight framework (including fraud risk);  performance measures; goal setting; and communication with external  parties. We reviewed key reports and guidance and then vetted our  selected practices with stakeholders knowledgeable about performance  measurement, design methodology, fraud risk, and the law. Based on this  review and input, we consolidated identified practices into 14 leading  practices to apply to Treasury’s monitoring framework.

We then assessed Treasury’s policies and procedures against the  framework. Specifically, we reviewed the agencies’ documented policies  and procedures, reviewed documentation of how Treasury followed its  policies and procedures, conducted interviews with Treasury staff  responsible for overseeing HHF, and interviewed stakeholders, such as  mortgage servicers, about Treasury’s monitoring of HHF. We also  interviewed staff from four HFAs about Treasury’s monitoring of their  programs; we selected the HFAs based on their mix of HHF programs,  proportion of HHF funds disbursed, and geographic diversity. We also  took into account whether stakeholders indicated that an HFA’s  implementation of the program was particularly successful or challenging.  With regard to the documentation Treasury collects as part of its  monitoring, we limited our review to its 2016 and 2017 monitoring  activities, and we limited our review of Treasury’s written policies and  procedures to those implemented from January 2016 to September 2018.  Two analysts independently reviewed agency policies and procedures to  determine whether the policies were consistent with the 14 identified  leading practices. Any disagreements in the determinations were resolved  through discussion or with a third party, including the General Counsel’s  office. We categorized each practice as follows:

Addressed: Treasury’s policies and procedures reflect each  component of the leading practice.

Partially addressed: Treasury’s policies and procedures reflect some  but not all components of the leading practice.

Not addressed: Treasury’s policies and procedures do not reflect any  of the components of the leading practice  To describe active HHF programs and the status of HFAs’ progress  toward program goals, we reviewed program documents, administered a  data collection instrument, and spoke with officials at four HFAs (selected  as previously described) and Treasury. We defined active programs as  those that had a total allocation approved by Treasury and were  accepting applications and still disbursing funds to households or blight  elimination projects as of December 2017. In order to identify which  programs were active, we developed, collected, and reviewed a  questionnaire in which HFAs provided information on when each of their  HHF programs started and stopped disbursing funds. For each of the 71  active programs we identified, we reviewed quarterly performance reports  as of December 2017 to compile descriptive information such as program  outputs and outcomes.

Through the review of program documentation and interviews with  knowledgeable officials, we found that Treasury’s output data were  sufficiently reliable for our description of homeowners assisted and  properties demolished. We also found that the data Treasury collected  from HFAs on program outcomes were not reliable for the purpose of  summarizing alternative outcomes by HFA or by program type. Treasury  officials noted that the conclusions that can be drawn from alternative  outcome data are inherently limited, particularly for the purpose of making  comparisons between HFAs or program types, due to HFAs interpreting  certain outcome measures differently, among other factors. Additionally,  by comparing Treasury’s outcome measures to leading practices, we  found that their definitions were not clearly stated.

We also identified four studies on the impact of HHF blight elimination  programs and reviewed them for reliable methodology. We determined  that one of the four studies was not reliable for the purpose of assessing  the impact of blight programs on targeted areas. Two of the three studies  that we determined to be reliable had important limitations. One study  examined 1 of the 18 counties that were served by that HFA’s blight  elimination program. The other study did not isolate the independent  effect of the HFA’s blight elimination program because other factors, such as local economic conditions, could also affect the performance of key  economic indicators. We reviewed each HFA’s contract with Treasury  as of December 2017 to identify each program’s target for assisting  homeowners or demolishing blighted properties. Through comparison  with internal control standards, we found that these targets were not  reliable for the purpose of describing HFAs’ progress toward program  goals because they were not stated in a form that permitted reasonably  consistent measurement.

To describe the factors Treasury identified as challenges for the HHF  program, we analyzed Treasury’s on-site compliance monitoring reports  for 2016 and 2017. As a part of our analysis, we identified the HFAs that  Treasury visited in 2016 and 2017 and the extent to which Treasury had  observations related to five Treasury-identified areas: monitoring  processes and internal controls, eligibility, program expenses and  income, administrative expenses, and reporting.

We also interviewed key stakeholders regarding their views of challenges  related to implementation of the HHF program, particularly since 2012.  We discussed challenges with Treasury staff with responsibilities for  monitoring HFAs’ implementation of the program; staff from four HFAs  that are implementing HHF programs; six mortgage servicers that are  involved with the HHF program; and two housing counseling agencies  that are involved with the HHF program. For two of the HFAs with blight  elimination programs, we conducted site visits to observe activities  related to blight elimination. Additionally, we discussed challenges with  other interested organizations, including an association for HFAs and an  organization that brings together housing counselors, mortgage  companies, investors, and other mortgage market participants to help  address challenges in the mortgage market. Further, we reviewed reports  issued by the Special Inspector General for the Troubled Asset Relief  Program. We summarized the challenges that stakeholders described.

We conducted this performance audit from November 2017 through  December 2018 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.

To determine the extent to which the Department of the Treasury’s  (Treasury) policies and procedures for monitoring and oversight address  leading monitoring practices, we identified factors for an effective  monitoring framework based on a review of key reports and guidance and  input from stakeholders knowledgeable about performance measurement,  design methodology, fraud risk, and the law. To select the practices for  the scorecard, we focused on factors relevant to the structure of an  oversight framework (including fraud risk); performance measures; goal  setting; and communication with external parties. We consolidated  identified factors into 14 leading practices to apply to Treasury’s oversight  and monitoring framework. See Table 3 for the 14 leading practices and  their underlying factors.

Appendix III: Homeowners Assisted through the Hardest Hit Fund

As shown in table 4, housing finance agencies (HFA) were implementing  from one to seven Hardest Hit Fund (HHF) programs (excluding blight  programs) as of the fourth quarter of 2017. We included programs for  which HFAs were disbursing funds to homeowners. As of December  2017, individual HFAs had assisted from 807 to 86,220 homeowners.

Eight HFAs were implementing active blight elimination programs as of  December 2017, as shown in table 5. The number of blighted properties  demolished by individual HFAs ranged from 0 to 13,925.

The Department of the Treasury’s 2017 utilization threshold requires that  HFAs draw at least 95 percent of their HHF funding from rounds one  through four by December 31, 2017 (see table 6). As of December 2017,  17 of 18 HFAs had drawn 95 percent or more of their funding from rounds  one through four. The Nevada HFA had drawn 70 percent of its funding  from rounds one through four.

Appendix IV: Comments from the Department of the Treasury

Appendix V: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Jill Naamane, Assistant Director;  Lisa Moore, Analyst in Charge; Vida Awumey; Farrah Graham; John  Karikari; Moira Lenox; Benjamin Licht; Dan Luo; John McGrail; Marc  Molino; Jennifer Schwartz; Shannon Smith; Estelle Tsay-Huang; and Erin  Villas made key contributions to this report.

Treasury's Hardest Hit Fund provided nearly $10 billion to states that experienced high unemployment and large drops in home prices during the housing crisis. States use the funds to help homeowners avoid foreclosure or to demolish blighted properties.

Treasury monitors states' use of these funds by collecting reports on performance and spending and conducting site visits, which are important monitoring practices. However, because states don’t report the data in the same way, Treasury can’t effectively evaluate state programs or the fund overall. We recommended that Treasury improve certain monitoring practices.