Unemployment Insurance: Actions Needed to Ensure Consistent Reporting of Overpayments and Claimants' Compliance with Work Search Requirements

The UI program, which is overseen by DOL and administered by states, paid $30 billion to about 5.7 million individuals in 2017. Under federal law, to be eligible for benefits, individuals are generally required to actively search for work, but the specific work search requirements vary by state. Yet, states found that some benefits were overpaid to UI claimants who were ineligible because they were not meeting work search requirements. GAO was asked to review improper payments due to UI claimants' failure to actively search for work. Building on GAO's prior work ( GAO-18-133R ), this report examines (1) state administrative practices associated with work search overpayments; (2) selected states' approaches to address work search overpayments; and (3) DOL's oversight and support of states' efforts.

GAO analyzed DOL data, including the results of state reviews of a representative random sample of UI payments made from fiscal years 2013 through 2017. GAO also reviewed UI information from six states selected for variation in work search requirements and overpayment rates, interviewed DOL and state officials, and reviewed relevant federal laws, regulations, and guidance.

GAO's analysis of Department of Labor (DOL) data found that certain state administrative practices, such as reviewing a higher percentage of claimant-reported work search activities and frequent use of formal warnings, were associated with lower estimated work search overpayment rates for the Unemployment Insurance (UI) program. According to DOL data, 22 states were warning claimants after the first discovered occurrence of their failure to meet work search requirements (i.e., issuing formal warnings) rather than reporting that an overpayment was made, while the other states were reporting such cases as overpayments. In 2017, DOL determined that federal law does not permit states to use such policies. GAO's analysis of DOL data shows that in fiscal year 2017, estimated work search overpayments were nearly $1.4 billion (see fig.), but would have been an estimated $1.8 billion (+/-$0.2 billion) greater if states had not issued formal warnings and established overpayments. DOL officials told GAO in July 2017 that the agency would issue a letter to states informing them that federal law does not permit them to warn claimants instead of establishing an overpayment. To date, DOL has not issued the letter. Until DOL provides states with such notification, states may continue to report inconsistent information on overpayments.

State officials GAO interviewed reported using multiple approaches to address work search overpayments, including using their online systems that automate collecting information on claimants' work search activities; conducting audits of claimants work search activities beyond those required; and sending automated messages to claimants regarding their work search requirements. Officials said that their approaches encouraged claimants to conduct a more active work search and prevented work search overpayments in some cases.

DOL monitors states' work search overpayment rate estimates and has helped states address such overpayments, but lacks clear procedures for how states should verify claimants' work search activities. DOL directs states to verify a “sufficient” number of work search activities during their audits but has not provided information on what is considered sufficient. DOL data show that some states did not review claimants' work search activities for a majority of audited cases. DOL officials said that the agency plans to clarify its procedures after issuing a letter about formal warnings. By clarifying these procedures, DOL will have greater assurance that states are complying with verification requirements.

GAO is making four recommendations to DOL, including that it provide states information about its determination that the use of state formal warning policies is no longer permissible and clarify its work search verification requirements. DOL agreed with GAO's recommendations and stated that it would take action to address them.

The federal-state Unemployment Insurance (UI) program provides  temporary income support to eligible workers who become unemployed  through no fault of their own. Individuals who claim unemployment are  generally required to actively search for work as a condition of receiving  benefits. However, the specific work search requirements—such as the  number of weekly contacts a claimant must have with potential  employers—vary by state, according to the Department of Labor (DOL).

Overseen by DOL, and administered by states, the UI program paid  approximately $30 billion to about 5.7 million individuals in 2017. The UI  program had the seventh-highest reported improper payment estimate  among all federal programs in fiscal year 2017 (about $4 billion or about  12.5 percent of benefits paid). Currently, the leading reported cause of  UI improper payments is overpayments to claimants who failed to meet  work search requirements, according to DOL data.

You asked us to review states’ and DOL’s approaches to address  improper payments that result from UI claimants’ failure to actively search  for work. In November 2017, we reported that some states use formal  warnings for claimants who fail to actively seek work instead of counting  them as overpayments, which affects DOL’s improper payment estimates,  and that additional instruction from DOL could help. In this report, we  build on the findings from that prior report and provide additional  information on how selected states and DOL are addressing  overpayments due to claimants’ failure to meet work search  requirements, i.e., work search overpayments.

Specifically, we examine (1) the extent to which state administrative  practices are associated with reported work search overpayments, (2) the  approaches that selected states have used to address work search  overpayments, and (3) the extent to which DOL oversees and supports  states’ progress in reducing work search overpayments.

To examine the extent to which state administrative practices are  associated with reported work search overpayments, we analyzed DOL’s  Benefit Accuracy Measurement (BAM) Program data for fiscal years 2013  through 2017, the most recent data available. We estimated work search  overpayment rates and amounts overpaid, produced descriptive statistics,  and conducted a regression analysis. (See app. I for more information on  our regression analysis.) We assessed the reliability of the data by (1)  performing electronic testing of relevant data elements, (2) reviewing  existing information about the data and the system that produced them,  including results from prior audits, and (3) collecting information from DOL  officials knowledgeable about the data. Based on these reviews, we  determined that the data were sufficiently reliable for our purposes.  Related to these administrative practices, we also reviewed relevant  federal laws, regulations, and DOL guidance. We assessed DOL actions  against its internal procedures and federal internal controls standards  related to monitoring and communication.

To identify approaches states have used to address work search  overpayments, we reviewed documentation, such as work search  verification procedures, and interviewed state UI officials from six states– Indiana, Nevada, New Jersey, Mississippi, Pennsylvania, and Utah– selected for variation in work search requirements, overpayment rates,  and DOL region. The information we obtained from those states is not  generalizable to all states.

To determine the extent to which DOL oversees and supports states’  efforts to address work search overpayments, we interviewed DOL  officials from the national office and collected information from all six DOL  regional offices on how they oversee state BAM systems, such as  monitoring procedures and state plans submitted by our selected states.  We also reviewed relevant federal laws, regulations, and DOL guidance.  We assessed DOL actions against its internal procedures and relevant  federal internal controls standards.

We conducted this performance audit from January 2017 to August 2018  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

The federal-state UI program provides temporary cash benefits to eligible  workers who lose their jobs through no fault of their own. Under this  arrangement, states administer their own programs according to certain  federal requirements and under the oversight of DOL’s Office of  Unemployment Insurance. States have considerable flexibility to set  benefit amounts and their duration, or the maximum period of time that  the state pays benefits, and establish eligibility requirements.

UI benefits are funded primarily through state payroll taxes on employers,  and administrative costs are primarily funded through a federal payroll tax  on employers. The states collect taxes that will be used to pay UI  benefits, and the U.S. Department of the Treasury holds these funds in  trust on behalf of the states in the Unemployment Trust Fund. DOL  certifies for payment to the states administrative grants to operate their UI  programs, which amounted to about $2.7 billion in fiscal year 2017. DOL  is responsible for ensuring that state UI laws include certain provisions,  which is a condition of the state receiving its UI administrative grant.

Individuals typically claim their UI benefits by filing claims with their state  UI agency online or by phone on a weekly or bi-weekly basis. In fiscal  year 2017, the average weekly UI benefit was about $350, and claimants  remained on the program for an average of 15 weeks, according to DOL  data.

Work Search Requirements for UI Claimants

Federal law establishes a work search requirement for UI eligibility, but  the specific work search activities UI claimants are expected to conduct  vary by state, according to a DOL report. To be eligible for  unemployment benefits, individuals are generally required to actively  search for work under federal law. The Middle Class Tax Relief and Job  Creation Act of 2012 amended the Social Security Act to, among other  things, require states to have work search requirements for UI claimants  specified in their laws as a condition of eligibility for the states’ UI  administrative grants. Specifically, states must have laws that require UI  claimants to be “actively seeking work” as a condition of eligibility for  unemployment compensation for any week. Because federal law does not  specifically define actively seeking work, states have some discretion to  establish a reasonable definition, according to DOL’s 2013 guidance to  states. For example, a state can specify a minimum number of weekly  contacts a claimant must have with potential employers. Acceptable work  search activities might also include searching for jobs online, submitting  job applications, visiting a job center, attending a networking event, or  establishing a LinkedIn account, according to a DOL report.

Depending on the state, UI claimants may be directed to register for work  with their state’s Employment Service, which provides job search  assistance, job placement assistance, and referrals to employers. In  addition, in some cases UI claimants may be directed to participate in  reemployment services at an American Job Center. In 2017, DOL  provided $115 million in grants to states to provide Reemployment  Services and Eligibility Assessments (RESEA). RESEA services include  in-person reemployment services and eligibility assessments in American  Job Centers for ex-service members and UI claimants determined to have  a high likelihood of exhausting their UI benefits. RESEA-funded activities  include developing an individual reemployment plan, providing labor  market information, identifying job skills and prospects, and reviewing the  claimant’s continued eligibility for UI benefits.

Process for Identifying Work Search Overpayments in State Benefit Accuracy Measurement Audits

DOL uses its Benefit Accuracy Measurement (BAM) system to determine  the accuracy of UI benefit payments and estimate the amount and rate of  improper payments. Under the BAM system, each state reviews a  number of randomly selected cases on a weekly basis and reconstructs  the UI claims process to assess the accuracy of the payments that were  made. The state determines what the benefit payment should have  been according to its laws and policies. States report the results of their  BAM case reviews to DOL—including overpayments and  underpayments—through an online data system. DOL uses the data to  estimate improper payment rates by state, as well as to calculate a  nationwide rate.

State BAM audits involve reviews of existing records in the state’s UI  claims information system as well as original fact-finding by the state  BAM investigator. DOL requires states to use a standard claimant  questionnaire when conducting BAM audits. The questionnaire includes  numerous questions about the claimant’s circumstances—including their  work search efforts—during the week under review. The questionnaire  includes questions that could indicate that a claimant qualifies for an  exemption from work search requirements, or made specific job contacts  and the results of the job contacts, such as whether the claimant  submitted an application and received a job offer.

State BAM investigators are also expected to take steps to verify the  information reported by the claimant by collecting documentation from  claimants and contacting employers or other third parties. According to  DOL’s 2016 BAM annual report and BAM procedures, state BAM  investigators are to review a sufficient number of work search activities to  determine whether the claimant has complied with the state’s minimum  requirement for the number of weekly work search activities. The BAM  program assigns one of three classifications to each of the work search  activities reviewed:

Acceptable – Documentation exists that the work search activity  reported by the claimant, such as an employer contact, employment  application, or other state approved work search activity, was made  by the claimant and was acceptable according to the state’s law or  policy.

Unverifiable – The investigator was unable to establish sufficient  information to make a judgment of whether the work search activity  was either acceptable or unacceptable according to the state’s law or  policy.

Unacceptable – Written documentation exists that the work search  activity reported by the claimant was not made at all by claimant, or  was made but was unacceptable according to the state’s law or  policy.

According to DOL’s BAM annual report, work search activities classified  as acceptable or unverifiable may be considered in calculating whether  the claimant has satisfied the state’s required number of work search  activities for purposes of BAM. If the state investigator finds that the  claimant’s work search is unacceptable and does not meet the state’s  requirements, he or she may determine the claimant was ineligible for  benefits and establish an overpayment, depending on state law (see fig.  1). Currently, several states have formal warning policies and provide  claimants warnings for the first instance of noncompliance with work  search requirements, whereas states without these policies count these  cases as overpayments, according to DOL.

Since 2002, federal agencies have been required to identify and report  improper payments. The leading reported cause of UI improper  payments in fiscal year 2017 was overpayments to claimants who failed  to meet work search requirements. DOL data show that states made an  estimated $1.36 billion in overpayments to such claimants in fiscal year  2017. Other major reported causes of UI improper payments in fiscal year  2017 included payments made to individuals who continue to make  claims even after returning to work (benefit year earnings) and payments  made to claimants who were determined ineligible due to disqualifying job  separations, such as quitting a job without good cause or being  discharged for misconduct (separation issues). (See fig. 2. Table 8 in app.  II provides greater detail.)

According to DOL officials, many UI improper payments cannot be  prevented given certain legal requirements that states pay claims in a  timely manner and provide claimants with due process when the state  finds an eligibility issue. Specifically, according to DOL, federal law  requires that when an eligibility issue is detected, the claimant has a right  to receive notice and provide the state information before being denied  benefits. In addition, if an eligibility issue associated with work search,  or any other matter, is detected but not resolved, the state is still required  to pay for a claimed week no later than the week after an eligibility issue  is detected, according to DOL. The time it takes to work through the  necessary due process steps can prevent states from stopping the  payment before it must be paid.

Trend in Reported Work Search Overpayments

Nationally, the estimated work search overpayment rate and the  estimated amount of work search overpayments have risen in recent  years. Specifically, in fiscal year 2013, approximately $1 billion in  estimated work search overpayments were made to claimants who were  not actively searching for work and, in fiscal year 2017, the amount  increased to close to an estimated $1.36 billion (see fig. 3). The national  work search overpayment rate for such claimants also increased during  this time. (See table 9 in app. II for additional details.) According to DOL  officials, some states implemented more stringent work search  requirements, which may account for the recent trend. As work search  requirements become more stringent, the opportunities for non- compliance and errors increase and thus higher improper payment rates,  according to DOL officials. While the national work search overpayment  rate was 4.5 percent in fiscal year 2017, state work search overpayment  rates varied widely from an estimated 0 to 41 percent of the UI benefits  that states paid in fiscal year 2017. (See table 10 in app. II for state-by- state estimates of work search overpayment rates and amounts.)

States use various methods to recover overpayments to UI claimants,  including setting up payment plans, off-setting UI benefits, or deducting  refunds from federal or state income tax returns. Like all recovered UI  overpayments, recoveries of work search overpayments must be  deposited in the unemployment trust fund of the state that recovered the  money and can be used only for the payment of UI benefits, according to  DOL officials. National data are not available on the amount of work  search overpayments that states have recovered because, although DOL  collects recovery data from states, it does not require states to separate  out work search overpayment recoveries from other types of recoveries in  their reporting.

Conducting More Work Search Investigations Is Associated With Lower Estimated Work Search Overpayment Rates, As Is the Use of Formal Warnings

Based on our analysis of DOL data, we found that certain state  administrative practices, including investigating a higher percentage of  claimant-reported work search activities and frequent use of formal  warnings, were associated with lower reported state work search  overpayment rates. However, DOL recently determined that federal law  does not permit states to warn claimants the first time they failed to meet  work search requirements (i.e., issue formal warnings) instead of  establishing that an overpayment was made. Additionally, a higher  percentage of claimants required to search for work is associated with  higher reported state work search overpayment rates.

Investigating Claimant- Reported Job Contacts Is Associated with Lower Work Search Overpayment Estimates, but the Extent to Which States Verified Contacts Varied

One of the administrative practices significantly associated with lower  work search overpayment estimates was investigations of claimants’  reported job contacts. Specifically, a higher percentage of cases with  claimants whose contacts were investigated by the state UI agency as  part of the state’s BAM audit was associated with a lower work search  overpayment rate estimate. According to our analysis, for every 1  percentage point increase in the percentage of cases with claimants’  whose job contacts were investigated, there was a 0.072 percentage  point decrease in the work search overpayment rate estimate.

The extent to which states attempted to verify claimants’ reported job  contacts through these investigations varied, according to our analysis of  DOL data. Nationally, in fiscal year 2017, states investigated job contacts  in about 80 percent of BAM cases where claimants were required to  search for work. However, among the states, the proportion of cases in  which job contacts were investigated was less than 50 percent in 5  states. (Table 11 in app. II shows the percentage of contacts that were  investigated for each state.)

Furthermore, states often were not able to verify the information claimants  reported. Of the job contacts that were investigated, states reported that  about 48 percent of the job contacts were acceptable, about 8 percent  were unacceptable, and about 45 percent could not be verified (see fig.  5).

Our analysis of BAM data for fiscal year 2017 also shows that for the  overpayments that states were able to detect, that a large portion were  found through investigating and verifying claimants’ work search contacts.  Specifically, 47 percent of reported work search overpayments were  found through this practice. Interviewing claimants about their work  search was the next most common way states detected work search  overpayments. States reported identifying 32 percent of work search  overpayments in fiscal year 2017 using this practice.

Although overpayments can be the result of actions taken by the claimant  or the agency administering the program, states reported that most work  search overpayments are associated with claimants. For example,  claimants may provide inadequate or incorrect information needed by the  UI agency to determine if the claimant met work search requirements. In  fiscal year 2017, states attributed about 99 percent of overpayments at  least partially to claimant action, while they attributed about 2 percent at  least partially to administrative errors at the state agency.

Frequent Use of Formal Warnings Is Associated with Reporting Lower Work Search Overpayments, but DOL Recently Determined Their Use Is Legally Impermissible

According to DOL data, 22 states issued formal warnings to one or more  claimants at some point between fiscal year 2013 and fiscal year 2017 for  failure to meet work search requirements instead of finding that the  claimants were overpaid. Although the states that made use of these  warnings varied over this period, the number of states issuing formal  warnings has generally increased over time from 13 states issuing formal  warnings in fiscal year 2013 to 19 states in fiscal year 2017. Overall,  states that most frequently issued formal warnings had lower reported  work search overpayment rates than states that did not issue formal  warnings. However, their work search overpayment rates are lower  because, under their state policies, they did not count an overpayment  when they issued a formal warning.

Our analysis indicates that states which issued formal warnings  frequently—in 75 percent or more of cases involving work search errors— had estimated work search overpayment rates that were 3.5 percentage  points lower, on average, than states that did not issue formal warnings.  However, states that use formal warnings less frequently—in fewer than  75 percent of cases involving work search errors—reported work search  overpayment rates that were between 3 and 4 percentage points higher  than states that did not issue formal warnings. See appendix I for a  detailed discussion of our econometric analysis.

Table 1 shows the average work search overpayment rate estimates for  each of these groups of states when formal warnings are not counted as  overpayments, and the potential average work search overpayment rate  when formal warnings are counted as overpayments. Excluding formal  warnings, two of the groups—frequent and low users of formal  warnings—had average work search overpayment rate estimates lower  than the average for states that did not use formal warnings. However,  when formal warnings are included in the overpayment rate, only low  users of formal warnings have a work search overpayment rate estimate  lower than the average for states that did not use formal warnings.

GAO’s analysis of DOL data shows that in fiscal year 2017, estimated  work search overpayments were nearly $1.4 billion, but potentially would  have been an estimated $1.8 billion greater if states had not issued  formal warnings and established overpayments. Our analysis further  shows that if formal warning cases had been included in DOL’s  calculation of the UI overpayment rates for fiscal year 2017, the  nationwide UI overpayment rate would have increased by about 6  percentage points, from an estimated 12 percent to an estimated 18  percent. Moreover, these figures represent an increase from the fiscal  year 2016 figures we presented in our previous report on states’ use of  formal warnings related to work search requirements. At that time, we  found the nationwide UI overpayment rate would have increased by about  5 percentage points from an estimated 11 percent to an estimated 16  percent with the inclusion of formal warning cases. The amount of UI  payments made to claimants for weeks in which they received formal  warnings in fiscal year 2016 was about $1.6 billion. Table 12 in appendix  II shows how the estimated UI overpayment rate in each state issuing  formal warnings in fiscal year 2017 may have increased if formal  warnings were not used.

State use of formal warnings has resulted in inconsistent reporting of  work search overpayments, which affects DOL’s reported improper  payment rate for the UI program. Specifically, states that issue formal  warnings have not counted as overpayments cases in which claimants  did not actively search for work and received a UI benefit payment. On  the other hand, states that did not have formal warning policies counted  such cases as overpayments, which are factored into DOL’s reported  improper payments rate. The variation among states related to formal  warning policies makes it difficult for DOL and others to understand the  reasons behind states’ reported work search overpayments.

DOL has determined and documented in its FY 2017 Agency Financial  Report that the use of formal warnings is no longer allowed under the  2012 federal law, which generally requires UI claimants to actively seek  work. DOL officials told us in July 2017 they would soon issue a letter to  states to inform them that they are no longer permitted to use formal  warnings when they determine that claimants failed to meet work search  requirements. To date, DOL has not issued such a letter. In May 2018,  DOL officials told us that they expect to issue the letter by the end of  calendar year 2018. Federal internal control standards direct agency  management to remediate identified internal control deficiencies on a  timely basis. Federal internal controls standards also state that  management should externally communicate the necessary quality  information to achieve the entity’s objectives. Additionally, these  standards state that agency management should establish and operate  monitoring activities to monitor the internal control system and evaluate  the results.

According to DOL officials, they began discussing the need for a potential  discontinuation of formal warnings at conferences with states in the first  half of 2017. However, we found that states continued to implement their  formal warning policies, potentially resulting in an increase in the  estimated amounts of overpayment dollars associated with formal  warnings between fiscal year 2016 and 2017. Until DOL informs states of  the need to discontinue the use of formal warnings through a letter or  another mechanism, states will continue to be inconsistent in whether  they count as overpayments cases in which claimants who failed to  search for work in any week were provided benefits. Additionally, once  DOL provides additional information to states on formal warnings, it  should monitor states’ responses to help ensure that DOL achieves its  desired results. Furthermore, having more consistent information on  overpayments related to work search issues could help DOL assess how  the program is working nationwide and whether further federal and state  actions would be needed to address this leading source of reported  improper payments in the UI program.

DOL officials stated that the national work search overpayment rate is  likely to increase in the future as states begin to eliminate their formal  warning practices. Officials also stated that this may take some time as  some states may need to amend laws or regulations in order to do so.

Higher Percentages of Claimants Required to Search for Work Are Associated with Higher Work Search Overpayment Rate Estimates

State work search overpayment rate estimates were higher for states  where a higher proportion of claimants were required to search for work.  Based on our analysis, for every one percentage point increase in the  fraction of cases with claimants required to search for work, there was a  0.084 percentage point increase in the work search overpayment rate  estimate on average, all else being equal.

Nationwide in fiscal year 2017, states reported that work searches have  been required in 80 percent of cases, with requirements in individual  states ranging from 38 to 100 percent of cases. Three states reported  requiring fewer than 50 percent of claimants to perform work searches,  while five states reported requiring more than 95 percent of their  claimants to perform work searches. The most common reasons states  exempted claimants from work searches were because claimants were  “job-attached” (e.g. temporarily laid-off, recalled), or they had union  deferrals because they were seeking employment through their union,  according to DOL data. The map in figure 6 shows the range among all  the states in the percentage of UI claimants required to search for work,  according to our analysis of DOL data for fiscal year 2017.

Selected States Used Multiple Approaches to Address Work Search Overpayments, but Cited Challenges Verifying Claimants’ Work Search Activities

Selected states used multiple approaches to address work search  overpayments, including online systems to facilitate the work search  reporting and verification process, work search audits beyond the BAM  audits, and messaging to inform claimants of their work search  responsibilities. For example, three of six states in our review had online  systems where claimants could report specific work search activities as  part of filing their weekly claims, according to state officials (see table  2). Some of the approaches states used were specifically designed for  UI claimants participating in state reemployment programs.

State officials cited several benefits of the approaches they use to  address work search overpayments. The online systems, work search  audits, and messaging helped prompt work search activities and prevent  work search overpayments in some cases, according to state officials.  According to officials from the selected states that used them and a study  on work search improper payments, online systems can facilitate the work  search reporting and verification in several ways:

Automatically documenting the claimants’ work search activities.

Online reporting of work search activities can help prevent  overpayments because their work search is documented in the online  claims system, which means the claimant does not need to keep a  work search log. In addition, online job search/training systems can  be used to track the work search activities completed by claimants,  making it easier for the state to verify that the work search was  completed. For example, officials in Mississippi and Indiana told us  they piloted an online system called NextJob and required RESEA  program participants to conduct work search activities through the  system. Mississippi officials reported that NextJob motivated  claimants to conduct their job search and increased the speed of  reemployment among these individuals. Similarly, New Jersey officials  told us that UI claimants selected to participate in New Jersey’s  RESEA program are required to use an online job search and training  system called OnRamp, which, for example, allows job seekers to  create or upload their resume on the website, search for jobs, access  online training, and receive email alerts on potential job matches. In  addition, officials in Nevada said that the online reporting system is  beneficial because the work search activities are documented in the  system and are more reliably retrieved if the claimant is selected for a  BAM audit because few claimants maintain and retain their work  search effort logs.

Performing automated checks on data the claimants submit. The  online claims systems can identify potentially duplicate job contacts  and check whether the claimant reported the required number of job  contacts. For example, Utah officials reported that if a claimant enters  job contacts from another week, officials would follow up with the  claimant by phone after it is flagged by their online system. If  claimants report self-disqualifying information, such as an insufficient  number of job contacts, the system can automatically put a hold on a  claim until the issue is resolved.

Facilitating communication with the claimant. Some states added  messages to their online claims system that pop up if the claimant  enters incorrect or insufficient information. For example, Mississippi  officials stated that they used messaging to better inform claimants of  their responsibilities and to encourage them to report accurate  information. According to the officials, if claimants do not report the  required number of work search activities in the online system, a  questionnaire will pop up requesting that the claimants explain why  they did not do so. The system requires the claimants to enter more  information in order to submit their claims and to receive their benefit  payments. Mississippi officials also stated that adding targeted  messaging resulted in fewer denials of benefits due to claimant failure  to meet work search requirements and also reduced the number of  appeals related to this type of denial. Some state officials said that  messaging encouraged accurate reporting. For example, Indiana  officials told us that at the end of the online claim filing process,  claimants receive a message notifying them of the state’s work search  requirements and informing them that they are required to search for  work to continue receiving benefits. In Utah, officials developed a  video that covers claimants’ responsibilities, including work search  requirements, which claimants must view before receiving their initial  benefit payment, according to state officials.

Officials in three of the states we reviewed reported using additional work  search audits beyond BAM to help reinforce their state policies.  Pennsylvania officials reported conducting 7,182 work search audits for  RESEA program participants in 2016. As a result, officials reported that  the state issued 1,300 warnings to claimants. The two other states— Mississippi and Utah—report that their random work search audits,  coupled with their online systems, helped prevent work search  overpayments as they are able to disqualify claims before the payment  goes out. Mississippi and Utah officials also reported that they were also  able to identify and recover some work search overpayments (see table  3).

Despite implementing these approaches, state officials in five of the six  states we contacted told us they face challenges with verifying work  search activities. These officials stated that they have difficulty verifying  work search activities as some claimants do not understand the work  search requirements or do not keep accurate records of their work search  activities, which makes it difficult for the state to confirm compliance with  the state requirements. In addition, state officials also said that many  employers do not keep records of job seekers’ inquiries or do not respond  to state requests for information when they are trying to verify claimants’  work search activities. For example, Nevada officials said that work  search contacts are often virtually unverifiable as many companies  outsource their hiring processes to contractors who refer the job  candidate for a job posting and keep the job application. Officials said  these contractors also rarely respond to state inquiries about claimants’  job applications. As discussed later, DOL has provided states tools to  help address this issue, such as a messaging toolkit to help states  improve communications with claimants and employers.

DOL Monitors States’ Work Search Overpayment Rate Estimates and Provides Assistance, but Lacks Clear Procedures on Work Search Verification

DOL Monitors States’ Work Search Overpayment Rate Estimates and Has Identified Strategies and Provided Tools to Help States Reduce Their Rates

DOL uses UI performance data to monitor state progress in reducing the  estimated improper payment rate, including data on overpayments to  claimants who failed to meet work search requirements. To do so, DOL  requires states to submit State Quality Service Plans, which serve as the  performance reporting and grant application documents through which  states receive administrative funding. The plans include a summary of  state performance on various measures related to operating the UI  program, including the improper payment rate, according to DOL  documentation we reviewed. States with estimated improper payment  rates of 10 percent or more are required to submit corrective action plans  to DOL. For example, data from two of the six states we reviewed– Nevada and New Jersey–had estimated improper payment rates above  10 percent during DOL’s most recent planning cycle and developed  corrective action plans. Nevada’s corrective action plan noted that the  state expects their rate to decline due to their June 2017 implementation  of online work search reporting as part of their UI claims system. New  Jersey’s corrective action plan noted that the state plans to implement  new online tools that will help them verify wage and employment  information. DOL separately monitors each state’s estimated work search  overpayment rate. In addition, all states, including those who estimated  improper payment rates of less than 10 percent, are required to prepare a  state-specific action plan that describes the root causes of improper  payments and the state’s strategies to address them. According to  agency officials, DOL reviews plans to monitor state performance and  help states identify strategies to improve performance.

Although DOL requires states with estimated improper payment rates of  10 percent or more to develop corrective action plans, according to DOL,  the agency has limited options to require state UI agencies to take actions  to respond to high improper payment rates. DOL officials told us that,  beyond routine monitoring and providing states with technical assistance  to help reduce their improper payments rate, their enforcement options  are limited to withholding the state’s administrative funding or removing  federal tax credit reductions, which is, in effect, a tax increase for the  state’s employers. According to DOL officials, both are considered  extraordinary sanctions that require significant due process. The agency  has not withheld state administrative funding to address improper  payments, according to DOL officials. DOL officials also told us that they  are concerned about the effects on UI claimants if they were to withhold  administrative funding. The administration’s fiscal year 2019  congressional budget justification includes a legislative proposal that  would authorize the Secretary of Labor to require states to implement  corrective action measures for poor state performance in the UI program,  helping to reduce improper payments in states with the highest estimated  rates.

DOL has identified strategies to address the leading causes of UI  improper payments—including work search issues—and provided states  tools and funding to help implement them. For example:

Pathway to Reemployment Framework. In 2016, DOL and the  National Association of State Workforce Agencies published a  framework that contained a broad menu of work search options that  states could adopt to better reflect how individuals search for work,  such as allowing use of online job search tools to count as an  approved work search activity. The framework also includes  suggestions for how states could document or verify claimants’ work  search activities for eligibility purposes.

Messaging toolkit. In 2012, DOL published a messaging toolkit  designed in part to improve claimants’ understanding of work search  requirements as a condition of eligibility for benefits. According to  DOL, claimants who fully understand their responsibilities and the  consequences of not fulfilling them may be more likely to complete the  required work search activities, thereby reducing instances of  claimants’ failing to search for work. DOL provided states  supplemental funding to support improved messaging and tracked  state implementation of the strategies.

Online tool to record work search activities. In 2011, DOL  provided supplemental funding to New York to develop an online work  search record that could be replicated by other states. This tool is  designed to reduce improper payments that result from inadequate  documentation of work search activities. Claimants can use the tool to  record their work search online when they file their UI claims. The  work search record is automatically shared with state job centers,  which in turn act to enhance claimants’ work search and connect  claimants with jobs. New York used open source technologies when  designing the tool in an effort to help more states replicate the product  at a lower cost.

UI Integrity Center. The DOL-funded UI Integrity Center of  Excellence (UI Integrity Center) was established as a way to help  states develop and share innovative strategies to prevent and detect  UI improper payments, reduce fraud, and improve program integrity.  For example, the UI Integrity Center funded a pilot project in 2016 to  support a state in implementing an online tool that trains claimants on  how to effectively search for jobs and allows claimants to use the tool  to complete their work search activities. The UI Integrity Center also  hosted national conferences in 2016 and 2018 that included  presentations on state practices to address work search improper  payments.

DOL Directs States to Investigate a Sample of Claimants’ Work Searches, but Has Not Clarified Procedures on Verifying Work Search

Although DOL monitors states’ work search overpayment rate estimates  and has provided assistance to help states address such overpayments,  it has provided limited direction to states on the level of effort states must  make to verify whether claimants are actively searching for work.  Specifically, DOL’s BAM procedures direct states to investigate a  sufficient number of work search contacts in its BAM sample of UI cases  to determine if the claimants met the state’s work search requirement.  However, DOL has not provided states any additional direction on what is  considered “sufficient.” As previously mentioned, the extent to which  states attempted to verify job contacts from claimants’ work searches  varied across states. Some states reported to DOL that they did not  investigate work search activities for a majority of their BAM cases even  though work search was required and the claimant reported job contacts.

Federal internal control standards state that agency management should  externally communicate the necessary quality information to achieve its  objectives, including addressing related risks. According to DOL  officials, the agency plans to clarify its BAM procedures, to include  providing more definitive instructions to states on work search verification  requirements, after the agency issues its planned letter to states about  discontinuing the use of formal warning policies. As of May 2018, the  agency said it plans to issue the letter on formal warning policies by the  end of calendar year 2018. However, as previously stated, it has been  nearly a year since DOL initially told us they were planning to issue the  letter to states.

Effective monitoring of state compliance with the clarified work search  verification requirements will also be important once DOL revises its BAM  procedures. Federal internal control standards state that agency  management should establish and operate monitoring activities to monitor  the internal control system and evaluate the results. Monitoring states’  responses could help ensure that DOL achieves its desired results. By  providing clear direction to states about work search verification  requirements and monitoring states’ implementation of these  requirements, DOL would have greater assurance that states are  complying with its requirements.

The health of the UI program depends, in part, on the ability of states to  control its benefit payments by accurately determining individuals’  eligibility. Improper payments, including overpayments, in the UI program  have led to billions of federal and state funds being used inappropriately.

Actively seeking work has generally been an eligibility requirement for  individuals receiving unemployment benefits under federal law since  2012, but states have not consistently implemented the requirement for  claimants in similar circumstances. States with formal warning policies  reported lower work search overpayments not necessarily because they  are better at ensuring claimants’ compliance with requirements, but  because they are not counting cases where claimants receive formal  warnings as overpayments. Furthermore, although DOL determined in  2017 that states are not permitted to issue formal warnings rather than  reporting an overpayment, it has not officially told states to stop using  warnings. Without providing states with this information and monitoring  their response, states will continue to report inconsistent information on  the extent of work search overpayments.

State efforts to check whether claimants are meeting work search  requirements also vary. Our evidence suggests that states making a  greater effort to investigate work search activities tend to have lower  overpayment rate estimates associated with this issue. However, some  states are not investigating claimant-reported work search activities as  part of their BAM audits despite DOL’s procedures directing them to do  so. Until DOL provides clear direction to states about verifying work  search and monitors state compliance, DOL has little assurance that  states are complying with its requirements.

We are making the following four recommendations to the Department of  Labor:

The Assistant Secretary of DOL’s Employment and Training  Administration should provide states with information about its  determination that the use of state formal warning policies is no longer  permissible under federal law. (Recommendation 1)

The Assistant Secretary of DOL’s Employment and Training  Administration should monitor states’ efforts to discontinue the use of  formal warning policies. (Recommendation 2)

The Assistant Secretary of DOL’s Employment and Training  Administration should clarify information on work search verification  requirements in its revised Benefit Accuracy Measurement  procedures. The revised procedures should include an explanation of  what DOL considers to be sufficient verification of claimants’ work  search activities. (Recommendation 3)

The Assistant Secretary of DOL’s Employment and Training  Administration should monitor states’ compliance with the clarified  work search verification requirements. (Recommendation 4)

We provided a draft of this report to the Department of Labor (DOL) for  review and comment. In its written comments, reproduced in appendix IV,  DOL agreed with all four of our recommendations and stated that it would  take action to address them. DOL also provided technical comments,  which we incorporated as appropriate. Additionally, we provided excerpts  of the draft report to state UI officials in the selected six states we  included in our review. We incorporated their technical comments as  appropriate.

As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the appropriate  congressional committees, the Secretary of the Department of Labor, and  other interested parties. In addition, the report is available at no charge on  the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at 202-512-7215 or brownbarnesc@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix V.

Appendix I: Econometric Analysis of Estimated State Work Search Overpayment Rates

We used the Department of Labor’s (DOL) Unemployment Insurance  Benefit Accuracy Measurement (BAM) data. The BAM program is  designed to determine the accuracy of paid and denied claims for  unemployment insurance (UI) in three major UI programs—state UI,  Unemployment Compensation for Federal Employees (UCFE), and  Unemployment Compensation for Ex-Servicemembers (UCX). The BAM  data covers claimants in the 50 U.S. states, the District of Columbia, and  Puerto Rico. Each week, state workforce agencies select random  samples of paid and denied unemployment insurance claims (i.e., cases).  State BAM investigators then audit these cases to determine whether the  claimant was properly paid or was properly denied benefits in the week  for which the claim was made (i.e., key week). The bases for determining  whether paid and denied claims were accurate are federal and state law,  regulations, and policy.

We used BAM cases for paid claims for all UI programs and all states for  fiscal years 2013 through 2017. The number of cases for each state is  determined by DOL for each year. Cases are chosen randomly each  week from the population of claims for that week. The normal weekly  number of paid claims sampled in most states is 9, with a minimum of 6,  for an annual sample of around 480 cases. For the 10 states with the  smaller UI workloads, the normal weekly number of paid claims sampled  is 7, with a minimum of 5, for an annual sample of around 360 cases.

For each paid claim case, the BAM data include variables describing the  claimant, as well as information on their UI benefit year, separation from  their last job, monetary eligibility, benefit payment history, employment  services registration and work search, and the outcome of the BAM  investigation. For cases for which BAM auditors identify errors, the BAM  data also include information on the errors. In 2016, work search issues,  benefit year earnings, and separation issues were the most common  causes of reported overpayments:

Work search issues. These occur when the state finds that claimants  did not actively search for work during the key week. Federal law  generally requires people receiving unemployment compensation to  be actively searching for work. States have discretion to establish  requirements for what constitutes active work search, and these  requirements vary by state.

Benefit year earnings issues. These occur when claimants have  earnings that exceed the threshold for UI eligibility in their state or  when these earnings are not properly reported.

Separation issues. These occur when claimants are ultimately  determined to be ineligible for UI due to disqualifying job separations,  such as quitting a job without good cause or being discharged for  misconduct under the state UI law.

Other causes of overpayments include incorrect reporting of wages used  to calculate benefits, able and available to work issues, employment  service registration issues, and other issues. For some cases, BAM  investigators identify multiple errors with different causes. When this  occurs, BAM investigators determine the overpayment amount associated  with each cause.

BAM sample data is weighted to make inferences about the population. In  accordance with DOL’s method, we calculated the weight on each BAM  sample case as (1) the number of unemployment compensation  payments to claimants for the state and the week from which the BAM  sample case was selected divided by, and (2) the number of completed  BAM sample cases for that week, as long as there were two or more  completed cases for the week.

Because each state followed a probability procedure based on random  selections to pick cases, the BAM sample is only one of a large number of  samples that they might have drawn. Since each sample could have  provided different estimates, we express our confidence in the precision  of the BAM sample’s results using confidence intervals. For example, a  95 percent confidence interval is the interval that would contain the actual  population value for 95 percent of the samples that could have been  drawn. In some tables we provide the margin of error instead of the  confidence interval, where the margin of error is the half-width of the  confidence interval.

Our analysis sample consists of cases with no missing or invalid values of  variables used in our analysis that also have positive sample weights. Our  analysis sample includes 98.1percent of all cases.

Estimated Overpayment Rates Due to Failure to Meet Work Search Requirement

In accordance with DOL’s method, the estimated overpayment rate is  equal to the amount of UI benefits overpaid as a percentage of the total  amount of UI benefits paid. The amount of UI benefits overpaid includes  fraud, nonfraud recoverable, and nonfraud nonrecoverable  overpayments; and overpayments from all causes and responsible  parties. The amount overpaid excludes overpayments that DOL considers  as technically proper. An overpayment may be considered technically  proper by DOL under a finality rule, which generally means that too much  time has passed between the decision to pay the claimant and the  detection of the eligibility issue, or for some other reason. Our estimated  work search overpayment rate is calculated using the same method as  the official overpayment rate and is generally comparable to work search  overpayment rates reported by DOL.

More specifically, we used the BAM data to identify cases with  overpayments and with overpayments due to work search, excluding  formal warnings and other payments DOL considers to be technically  proper. Next, we applied DOL’s proration algorithm to allocate  overpayment amounts for each error associated with a case so that the  total amount of overpayments from all errors does not exceed the key  week amount paid. Then, we calculated the total overpayment amount  and work search overpayment amount for cases with overpayments and  work search overpayments, respectively. Finally, we tracked the key  week amount paid for each case. To estimate the work search  overpayment rate for each state and fiscal year, we calculated the  weighted sum of work search overpayment amounts as a percentage of  the weighted sum of amounts paid.

Table 4 shows the estimated national work search overpayment rate, the  average estimated state work search overpayment rate, and the median  estimated state work search overpayment rate for each fiscal year. The  national work search overpayment rate estimate has increased from 2.4  percent in 2013 to 4.5 percent in 2017. Over the same time period, the  average state work search overpayment rate estimate has increased from  2.6 to 4.6 percent, and the median state work search overpayment rate  estimate has increased from 1.4 percent to 2.7 percent. Most states’  work search overpayment rate estimates are less than 5 percent, but  some states’ rates are more than 10 percent.

To identify state practices and other factors that may be associated with  state work search overpayment rates, we reviewed DOL documents  describing the BAM program and summarizing key features of states’ UI  programs, as well as research on factors associated with time claimants  spend searching for employment. Based on our review, we identified  factors that may be associated with work search overpayments that can  be measured using variables in the BAM data:

Exemptions from work search requirements. According to DOL, while  federal law generally requires claimants to be actively seeking work, it  does allow states to exempt claimants in some circumstances. For  example, according to DOL, because states may not deny benefits to  an individual in approved training, all states provide an exemption  from the requirements to be able and available for work and  conducting an active work search for any week the individual is in  approved training. In addition, according to DOL, some states allow  work search exemptions if the worker is union-attached and finds  work through the union hall, or if a separation is classified as a  temporary lay-off and there is a reasonable expectation that the  worker will return to work soon. According to DOL, other state work  search exemptions include that the worker has a specified start date  for new employment, has jury duty, has a compelling personal reason,  is in a labor dispute with the employer, is the victim of domestic  violence, or labor market or other information indicates no suitable  employment.

It is possible for a case with a claimant who indicated he or she was  not required to search for work to have a work search error if the  claimant provided an invalid reason for being exempt from his or her  state’s work search requirement. For example, the claimant may have  said that he or she was a member of a union with a hiring hall and  obtained employment through union referrals or that he or she had a  definite recall date, and therefore, the work search requirement was  waived. However, the BAM investigator’s verification with the union  found that the claimant was not in good standing, or the investigator’s  verification with the employer found that the claimant had no definite  recall date. In such a situation, the claimant might be held ineligible for  a failure to conduct an active work search because the exemption was  invalid.

Claimant response. According to DOL, the claimant interview anchors  the BAM audit and is a major error detection point, and the claimant  questionnaire is a required standard form. Claimants may provide  information for the BAM audit either in person, over the phone, or via  mail or some other method. Some claimants may not respond to the  audit at all. For the period from 2013 to 2017, of the work search  overpayments that BAM auditors detected in their BAM sample of  cases, about 37 percent of cases were uncovered during the claimant  interview.

State investigation of job contacts. Claimants are asked to provide  information about job contacts as evidence that they were actively  searching for work as part of the required claimant questionnaire. In  addition, even if the claimant does not respond to the BAM audit, job  contacts may be available for BAM auditors to investigate if the state’s  continued claim process captured the claimant’s work search  information. BAM staff must investigate a sufficient number of  contacts to establish whether the claimant has met the state’s work  search requirement. For the period from 2013 to 2017, of the work  search overpayments that BAM auditors detected in the BAM sample  of cases, about 45 percent were uncovered by investigating claimant- provided job contacts.

State use of formal warnings. If the BAM audit of a case determines  that the claimant’s work search during the key week was not  acceptable, the state might issue a formal warning to the claimant  instead of finding that the claimant received a work search  overpayment, and these cases are not included in the calculation of  the work search overpayment rate for that state. Over the period from  2013 to 2017, of the BAM sample of cases for which work search  errors were identified, state workforce agencies issued formal  warnings for about 46 percent.

Claimant demographic and other characteristics. Characteristics of  claimants that are associated with the amount of time claimants spend  searching for work include age, education, gender, length of time  between initial claim and key week, and weekly benefit amount as a  percentage of the normal weekly wage in the claimant’s industry. To  the extent that they affect time spent searching for work, these  characteristics may also be associated with the likelihood of claimants  receiving a work search overpayment.

Some factors that may be associated with work search overpayments  cannot be measured using variables in the BAM data and, thus, are  excluded from our analysis. Examples include:

Minimum number of work search activities. According to DOL, the  minimum number of work search activities required per week varies  across states and can vary based on labor market conditions, which  can, for example, produce different requirements in a rural area  versus an urban area. However, some states do not specify a  required number of work search activities and instead require that the  number of work search activities be “reasonable,” according to a DOL  report.

Type of required employer contacts. According to DOL, some states  allow claimants to search for part-time work as well as full-time work,  while others do not. In addition, according to DOL, some states  specify that participation in reemployment services counts as a  contact, and some states require claimants to make at least one  contact through the state online system.

Frequency and method of reporting. According to DOL, some states  require weekly claimant reporting of work search activities, while  others require bi-weekly reporting. In addition, according to DOL,  some states require claimants to report online as part of their  continued claim, while others require claimants to keep a log of their  work search activities.

Our econometric model is the following:  work search overpayment rates, = α + β∗work search requireds, +  γ*n-person responses, + γ*telephone responses, + γ*mail, email,  fax, or other responses, + δ*contacts investigateds, + φ + X’Θ + year indicators + ε,  where s and y denote state and year, respectively, and the explanatory  variables in the model are the following:

Work search required is the estimated fraction of cases with claimants  for whom work search is required, according to the BAM data.

In-person response, telephone response, and mail, email, fax, or  other response are the estimated fractions of cases with claimants  who responded to the BAM audit in person, by telephone, or by mail,  email, fax, or other method, respectively.

Contacts investigated is the estimated fraction of cases for which  BAM auditors investigated one or more job contacts.

Low, moderate, and frequent formal warning use are binary indicator  variables equal to 1 if the state issued a low, moderate, or high  number of formal warnings in the fiscal year as a percentage of BAM  cases with work search errors, respectively, and 0 otherwise. We  defined low formal warning states as those that issued formal  warnings for some but less than 25 percent of the cases with work  search errors, moderate formal warning states as those that issued  formal warnings for at least 25 percent but less than 75 percent of  cases with work search errors, and high formal warning states as  those that issued formal warnings for anywhere from 75 to 100  percent of cases with work search errors. The omitted group is states  that issued no formal warnings during the fiscal year.

X is a list of other characteristics of claimants that may be associated  with work search, including the estimated distributions of cases across  claimants’ age groups, education levels, gender, length of time  between initial claim and key week, and weekly benefit amount as a  percentage of the normal weekly wage for their occupation.  ε is an error term.

The parameters of interest in our econometric model are β, γ, γ, γ, δ,  φ is an estimate of the change in the work search  overpayment rate associated with a 1 percentage point increase in the  fraction of cases with claimants who responded to the BAM audit in  person, all else being equal. The parameters γ and γ are estimates  of the change in the work search overpayment rate associated with a  1 percentage point increase in the fraction of cases with claimants  who responded to the BAM audit by telephone and by mail, email, fax,  or other method, respectively, all else being equal.

The parameter δ is an estimate of the change in the work search  overpayment rate associated with a 1 percentage point increase in the  fraction of cases for which BAM auditors investigated one or more job  contacts, all else being equal.

The parameters φTable 5 shows descriptive statistics for the explanatory variables, other  than formal warnings.

Table 6 shows descriptive statistics for states’ use of formal warnings.

Our analysis is subject to several limitations, and the results we discuss  below should be interpreted with caution.

No causality. Our econometric approach can establish correlations  between state work search overpayment rate estimates and the  factors we analyzed, but it cannot establish causal relationships. This  limitation is especially important when it comes to interpreting the  relationship between work search overpayment rate estimates and the  fraction of cases for which BAM auditors investigated job contacts.  BAM investigations can only detect work search overpayments, not  prevent them, because BAM investigators are reviewing cases after  the payment has already been made. Preventing work search  overpayments is the only way to reduce the work search overpayment  rates. Thus, the relationship between investigation of claimant- provided contacts and work search overpayment rates likely reflects  not a causal relationship but an equilibrium relationship in which  claimants who know that their job contacts will be investigated are  more likely to search for work, all else being equal.

Nongeneralizability. Our results do not generalize to other time  periods with different labor market conditions, different rules and  regulations about unemployment insurance, or other differences.

Omitted variables. As discussed above, we have excluded from our  models some state practices that may be associated with work search  overpayments because those variables were not included in the BAM  data. For example, we do not account for the fact that, according to  DOL, some states require claimants to engage in more work search  activities than other states, which could affect the likelihood that a  claimant receives a work search overpayment. In addition, according  to DOL, some states require claimants to report their work search  activities in order to continue receiving benefits, which might prevent  some work search overpayments from occurring. We have also likely  excluded some relevant claimant characteristics as well. For example,  total income for a claimant’s household and a claimant’s number of  dependents may affect their work search efforts. However, variables  describing a claimant’s household income and composition are not  included in the BAM data and thus are omitted from our analysis. To  the extent that these factors are correlated with the factors we  included, our estimates of the relationships between work search  overpayment rates and the included factors could be biased.

Measurement error. Our variables may have been measured with  error, which could bias our coefficient estimates. If the measurement  error is random, our coefficient estimates would be biased down, but if  the measurement error is systematic, then we cannot say whether our  coefficient estimates would be biased down or up.

Our baseline specification suggests that some state practices are  associated with state work search overpayment rates estimates (see  column 1 of table 7). We used the 10 percent level of significance as our  threshold for statistical significance because we have a relatively small  number of observations (259).

State work search overpayment rate estimates were higher in states  with more cases with claimants who were required to search for work.  A 1 percentage point increase in the fraction of cases with claimants  required to search for work was associated with a 0.084 percentage  point increase in the work search overpayment rate estimates on  average, all else being equal.

Work search overpayment rate estimates were lower in states with  more cases for which the BAM auditors investigated one or more job  contacts. A one percentage point increase in the fraction of cases for  which the BAM auditors investigated contacts was associated with a  0.072 percentage point reduction in the work search overpayment rate  on average, all else being equal.

Compared to states that did not issue any formal warnings, work  search overpayment rate estimates were higher in states that made  low or moderate use of formal warnings but lower in states that issued  formal warnings frequently. Work search overpayment rate estimates  in states that were low and moderate users of formal warnings were  3.9 and 3.1 percentage points higher on average than those in states  that issued no formal warnings, all else being equal. Work search  overpayment rate estimates in states that were frequent users of  formal warnings were 3.5 percentage points lower on average than  those in states that issued no formal warnings.

Our state-level analysis also suggests that some characteristics of cases  are not associated with state work search overpayment rate estimates,  but others are. State work search overpayment rate estimates were not  significantly associated with the distribution of cases by claimant age,  gender, duration of unemployment benefits receipt, or key week amount  as a percentage of the normal weekly wage in their occupation, or with  the number of cases. However, state work search overpayment rate  estimates were higher in states with more cases with claimants with some  college or an associate’s degree relative to cases with claimants without a  high school degree or GED. A one percentage point increase in the  estimated fraction of cases with claimants with some college or an  associates degree was associated with a 0.192 percentage point increase  in estimated work search overpayment rates on average, all else being  equal.

To assess the sensitivity of our results, we estimated fractional response  models with robust standard errors instead of clustered standard errors  (see column (2) of table 7). We also estimated fractional response models  that replaced the fractions of cases by claimant response method with the  fraction of cases with claimants who responded to the BAM audit by all  methods combined, with both types of standard errors. Finally, we  estimated linear models, also with both types of standard errors. While  the direction and magnitude of the relationship between estimated work  search overpayment rates and the fraction of cases with claimants  required to search for work was generally similar across specifications, it  was not always statistically significant. The direction, magnitude, and  statistical significance of the relationship between estimated work search  overpayment rates and the fraction of cases for which BAM auditors  investigated job contacts was generally similar across specifications, with  the exception of the linear model with clustered standard errors, where it  was not statistically significant. The directions, magnitudes, and statistical  significance of the relationship between estimated work search  overpayment rates and estimated state formal warning use were  generally similar across specifications.

Appendix II: Additional Benefit Accuracy Measurement Data Tables

This appendix contains several tables that show the underlying data used  throughout this report, using the Department of Labor’s Benefit Accuracy  Measurement (BAM) data for fiscal years 2013 through 2017.

According to DOL, because the BAM data are based on a statistical  survey, estimates produced from our analysis of the BAM data are  subject to sampling and non-sampling errors. Sampling errors are errors  that arise in a data collection process as a result of taking a sample from  a population rather than using the whole population. Non-sampling errors  are errors or biases that arise in a data collection process as a result of  factors other than taking a sample, such as the timeliness of data  collection, data entry errors, biased questions in fact-finding, biased  decision-making, and inappropriate analysis and conclusions completed  by state investigators or false or inaccurate information provided by  survey respondents. We express our confidence in the precision of our  results by reporting the margins of error associated with 95 percent  confidence intervals. This is the interval that would contain the actual  population value for 95 percent of the samples the respective agency  could have drawn.

In addition, it may be misleading to compare one state's work search  overpayment rates with another state's rates. According to DOL, no two  states' written laws, regulations, and policies specifying eligibility  conditions are identical, and differences in these conditions influence the  potential for error.

The following tables and information are included in this appendix:

Table 8: The estimated overpayment amounts for the Unemployment  Insurance (UI) program by cause in fiscal year 2017 (also represented  in fig. 2 in the letter portion).

Table 9: The national work search overpayment rate estimates and  dollar amounts for fiscal years 2013 through 2017 (also represented in  fig. 3 in the letter portion).

Table 10: The work search overpayment rates estimates and dollar  amount of work search overpayments in each state in fiscal year  2017, excluding cases where formal warnings were given.

Table 11: The percentage of cases in which claimants were required  to search for work and the percentage of those cases in which job  contacts were investigated in each state in fiscal year 2017.

Table 12: For states that issued formal warnings in 2017, the UI  overpayment rate estimates excluding and including cases in which  formal warnings were issued, and the difference in the dollar amount  of overpayments if formal warnings were not used.

Appendix III: Summary of Six States’ Work Search Requirements for Unemployment Insurance Claimants

Work search requirements for Unemployment Insurance claimants vary  by state. According to our review of program documents in our six  selected states, the minimum number of work search activities required  per week ranged from not specified to four. Three of the states limited the  work search activities to applying for jobs or contacting employers in other  ways and the three other states had broader definitions of what would  qualify as a work search activity. See table 13 below for a summary of the  work search requirements, confirmed by state officials, for the six states  included in our review.

Appendix IV: Comments from the Department of Labor

Appendix V: GAO Contact and Staff Acknowledgments

Cindy Brown Barnes 202-512-7215 or brownbarnesc@gao.gov.

In addition to the contact names above, Danielle Giese (Assistant  Director), Cathy Roark (Analyst in Charge), Carl Barden, Rachel Beers,  Deborah Bland, Beryl Davis, Holly Dye, Alex Galuten, Dana Hopings, Joel  Marus, Phillip McIntyre, Sheila R. McCoy, Jean McSween, Courtney  LaFountain, Stacy Spence, Almeta Spencer, Matt Valenta made  significant contributions to this report.