Human Trafficking: State and USAID Should Improve Their Monitoring of International Counter-trafficking Projects

Human trafficking is a pervasive problem throughout the world. Victims are often held against their will in slave-like conditions.

The National Defense Authorization Act for Fiscal Year 2017 includes a provision for GAO to report on the programs conducted by specific agencies, including State, DOL, and USAID, that address trafficking in persons. Among other objectives, this report (1) identifies the recent projects in international counter-trafficking in persons that key U.S. agencies have awarded to implementing partners; and, for selected projects, assesses the extent to which key agencies have (2) documented their monitoring activities and (3) ensured the reliability of project performance information.

GAO reviewed State, DOL, and USAID project documents and interviewed agency officials. GAO reviewed monitoring documents for 54 of the 57 projects that were active from the beginning of fiscal year 2016 through the end of fiscal year 2017. Of these 54 projects, GAO selected a nongeneralizable sample of 5 projects, based primarily on largest total award amounts, for review of the reliability of project performance information.

The Departments of State (State), Labor (DOL), and the U.S. Agency for International Development (USAID)—through agreements with implementing partners—managed 120 international counter-trafficking in person projects during fiscal year 2017.

GAO reviewed a selection of 54 counter-trafficking projects (37 State, 3 DOL, and 14 USAID), and found that DOL and USAID had fully documented their monitoring activities, while State had not. All three agencies used similar tools to monitor the performance of their projects, such as monitoring plans, performance indicators and targets, progress reports, and site visits. GAO found, however, that State did not fully document its monitoring activities for 16 of its 37 projects (43 percent). GAO found that State did not have the monitoring plans or complete progress reports for one-third of its projects and often lacked targets for performance indicators in its final progress reports. State officials said they had not required targets for each performance indicator for the projects GAO reviewed, or had not set targets due to limited resources in prior years. State has taken steps to improve its monitoring efforts, including issuing a November 2017 policy that requires targets to be set for each performance indicator and developing an automated data system that would require targets to be recorded. However, because the pilot data system allows targets to be recorded as “to be determined” and does not have controls to ensure entry of actual targets, it is uncertain whether performance targets will be regularly recorded. Without full documentation of monitoring activities and established performance targets, State has limited ability to assess project performance, including project efficiency or effectiveness.

GAO reviewed the reliability of project performance information for 5 of the 54 counter-trafficking projects (2 State, 1 DOL, and 2 USAID) and found that State and USAID used inconsistent and incomplete performance information, while DOL used consistent and complete information. For example, some quarterly indicator results in State and USAID progress reports were inconsistent with annual total results, and narrative explanations for significant deviations from performance targets were sometimes not present in quarterly reports. According to agency officials, performance information from these projects is regularly used not only for direct project oversight but also for internal and external reporting, program decisions, and lessons learned. GAO found that State's and USAID's processes lack sufficient controls to ensure the reliability of project performance information, but did not find inadequate controls in DOL's process. For example, neither State nor USAID consistently used automated checks on indicator results to ensure consistency and completeness of performance indicator result calculations. In contrast, DOL used automated checks as part of its process. Without implementing controls to ensure that performance information is consistent and complete, State and USAID officials cannot fully or accurately understand what projects are, or are not, achieving, and how their efforts might be improved.

GAO is making four recommendations to State and one recommendation to USAID, including that both agencies establish additional controls to improve the consistency and completeness of project performance information, and that State maintain monitoring activity documentation and establish targets for each performance indicator. State and USAID concur with GAO's recommendations.

Trafficking in persons, or human trafficking, is a longstanding and  pervasive problem throughout the world, as traffickers buy, sell, and  transport victims across national boundaries. Victims are often held  against their will in slave-like conditions or forced to work in the  commercial sex trade, garment factories, fishing boats, agriculture,  domestic service, and other types of servitude. In addition to inflicting  grave damage upon its victims, trafficking in persons is a multi-billion  dollar industry that undermines government authority, distorts markets,  fuels organized criminal groups and gangs, enriches transnational  criminals and terrorists, and imposes social and public health costs. The  Department of State (State) noted in its June 2018 Trafficking in Persons  Report that terrorist organizations also use trafficking in persons to recruit  adherents and finance their operations. Congress enacted the Trafficking  Victims Protection Act of 2000 to combat trafficking in persons, and has  reauthorized this act four times. The act, as amended, defines severe  forms of trafficking in persons as (1) sex trafficking in which a commercial  sex act is induced by force, fraud, or coercion, or in which the person  induced to perform such act is under age 18; or (2) the recruitment,  harboring, transportation, provision, or obtaining of a person for labor or  services, through the use of force, fraud, or coercion, for the purpose of  subjection to involuntary servitude, peonage, debt bondage, or slavery.  According to the International Labor Organization, in 2016, there were an  estimated 24.9 million people in forced labor, including 4.8 million people  in forced sexual exploitation. However, as we previously reported,  estimates of the number of trafficking victims are often questionable due  to data and methodological weaknesses.

The National Defense Authorization Act for Fiscal Year 2017 includes a  provision for GAO to report on the programs conducted by the  Department of State (State), the Department of Labor (DOL), the United  States Agency for International Development (USAID), the Department of  Defense (DOD), and the Department of the Treasury (Treasury) that  address human trafficking and modern slavery, including a detailed  analysis of the effectiveness of such programs in limiting human  trafficking and modern slavery. Three of these agencies—State, DOL,  and USAID—have programs that design and award counter-trafficking  projects to implementing partners, through contracts, grants, or  cooperative agreements. These agencies then oversee and monitor  these projects. Since DOD and Treasury officials did not identify these  types of projects as part of their counter-trafficking in persons efforts, we  provide background information on their efforts but do not cover these  agencies in our reporting objectives. This report (1) identifies the recent  projects in international counter-trafficking in persons that key U.S.  agencies have awarded to implementing partners, and for selected  projects, assesses the extent to which key agencies have (2) documented  their monitoring activities, (3) ensured the reliability of the performance  information they use in monitoring projects, and (4) reviewed the  usefulness of the performance indicators they use in monitoring projects.

To address these objectives, we reviewed relevant agency documents  and interviewed agency officials. To report on agencies’ programs, we  asked knowledgeable officials at State, DOL, USAID, DOD, and Treasury  to identify their projects that (1) had an international focus; (2) were  delivered by implementing partners to external recipients, such as  trafficking victims or host governments, as project beneficiaries; and (3)  addressed trafficking in persons, modern slavery, or forced labor.  Because State, DOL, and USAID managed such projects, we focus on  them as the three key agencies for the purposes of our reporting  objectives. According to officials from these three agencies, the projects  they identified range from those with counter-trafficking in persons as a  primary goal, to those in which this goal was integrated as part of the  agency’s activities. We used the lists of projects that these agencies  provided to report the relevant counter-trafficking projects that agencies  awarded to implementing partners to carry out the projects. For our first  objective, we determined the projects that were active during fiscal year  2017, including those which began, were ongoing, or ended during fiscal  year 2017, and interviewed agency officials to confirm project information.

To analyze the effectiveness of agencies’ programs in limiting human  trafficking and modern slavery, we assessed the key agencies’ monitoring  efforts for a subset of the projects. We identified State’s, DOL’s, and  USAID’s projects that started before or during October 2015, which  corresponded to the first quarter of fiscal year 2016, and were active  through September 30, 2017, which corresponded to the fourth and last  quarter of fiscal year 2017. This resulted in a selection of 54 projects— 37 from State, 3 from DOL, and 14 from USAID—for our review. To  assess the extent to which State, DOL, and USAID documented their  monitoring activities for selected projects, we reviewed documentation of  key monitoring activities as specified in agency policy or the project award  agreements, including (1) the monitoring plan for each project, (2)  evidence of performance indicators and targets for each project, (3)  fiscal year 2017 progress reports for each project, (4) final progress  reports for the projects that ended by December 2017, and (5) evidence  of the agency’s site visits for each project. Additionally, we interviewed  knowledgeable monitoring officials from each agency to understand  agencies’ monitoring process and application of monitoring requirements  for counter-trafficking in persons projects.

To assess the extent to which key agencies ensured the reliability of the  performance information, as well as the extent to which they reviewed the  usefulness of the performance indicators they use to monitor selected  international counter-trafficking in persons projects, we selected for  review a nongeneralizable sample of 5 projects—2 State projects, 1 DOL  project, and 2 USAID projects—out of the 54 counter-trafficking in  persons projects identified by agencies that started before or during  October 2015 and were active through fiscal year 2017. We based our  selection of these projects primarily on the largest total award amounts.  For these five selected projects, we collected 2 years of progress reports  and other documents to assess the quantitative and qualitative  performance information. We assessed whether quarterly or semi-annual  indicator totals were consistent with annual and cumulative totals where  these were reported. Using this quantitative information, we judgmentally  selected indicators for inclusion in agency interviews where it appeared  likely that numerical errors had occurred or there appeared to be  significant project events, such as large over- or under-performance or  the elimination of the indicator. We interviewed agency officials, including  managers of each of the 5 projects, about the consistency and  completeness of monitoring information in these projects for about 60  indicators identified through our analysis, as well as to determine whether  our findings for these selected projects reflected general agency policies  and procedures, and to understand the processes and systems agencies  use to review indicators. We assessed the completeness and  consistency of project performance data that State, DOL, and USAID use  to monitor projects as part of our data reliability assessment. We found  State and USAID data to be unreliable in the projects we reviewed. We  discuss the implications of these unreliable data for State and USAID’s  project management and reporting in our findings and recommendations.  We found the performance data that DOL used were consistent and  complete for the project we reviewed. While we examined indicator and  narrative information for consistency and completeness, we did not verify  the accuracy of performance information. See appendix I for more details  on our scope and methodology.

We conducted this performance audit from October 2017 to December  2018 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

Human trafficking exploits individuals and often involves transnational  criminal organizations, violations of labor and immigration codes, and  government corruption. Many forms of trafficking—including sex  trafficking and labor trafficking—can take place anywhere in the world and  occur without crossing country boundaries. As discussed in State’s  annual Trafficking in Persons Report, trafficking victims include, for  example, Asian and African women and men who migrate to the Persian  Gulf region for domestic labor but then suffer both labor trafficking and  sexual abuse in the homes of their employers. Some victims are children.  For example, Pakistani children as young as 5 years are sold or  kidnapped into forced labor to work in brick kilns, some of which are  owned by government officials. Other victims are subjected to sexual  exploitation. In some cases, women and girls have been bought and sold  as sex slaves by members of the Islamic State. In other cases, adult men  and women have been forced to engage in commercial sex, and children  induced to do the same. Individuals, including men, are exploited in  forced labor in a variety of industries. Burmese men, for example, have  been forced to labor 20 hours a day, 7 days a week on fishing boats in  Thailand. See figure 1 for examples of victims of trafficking in persons.

Among other U.S. agencies involved in counter-trafficking in persons,  State, DOL, USAID, DOD, and Treasury have various roles and  responsibilities related to international counter-trafficking in persons,  including some internationally-focused programs and activities that do not  involve awards made to implementing partners, as follows:

State. State leads the global engagement of the United States, and  supports the coordination of efforts across the U.S government in  counter-trafficking in persons. State’s Office to Monitor and Combat  Trafficking in Persons (TIP Office), established pursuant to the  Trafficking Victims Protection Act of 2000, is responsible for bilateral  and multilateral diplomacy, targeted foreign assistance, and public  engagement on trafficking in persons. The office also prepares and  issues an annual Trafficking in Persons Report that assesses the  counter-trafficking efforts of governments and assigns them tier  rankings. Furthermore, the TIP Office develops annual regional  programming strategies, awards projects to implementing partners  and oversees the project award process, and provides technical  assistance to implementing partners. Other parts of State, including  regional bureaus that cover geographic regions and functional  bureaus that cover global issues such as human rights, are also  responsible for work related to combating trafficking in persons.

DOL. Within DOL, the Bureau of International Labor Affairs’ (ILAB)

Office of Child Labor, Forced Labor, and Human Trafficking (OCFT)  conducts research, publishes reports, and administers projects  awarded to implementing partners on international child labor, forced  labor, and trafficking in persons. ILAB’s reports include the annual  Findings on the Worst Forms of Child Labor report, which assesses  the efforts of approximately 140 countries and territories to eliminate  the worst forms of child labor in the areas of laws and regulations,  institutional mechanisms for coordinating and enforcement, and  government policies and programs. ILAB also reports on the List of  Goods Produced by Child Labor or Forced Labor showing goods and  their source countries which ILAB has reason to believe are produced  by child labor or forced labor in violation of international standards.

USAID. USAID administers projects awarded to implementing  partners that address counter-trafficking in persons, including  increased investments in conflict and crisis areas, and integrating  such projects into broader development projects. USAID field  missions manage the majority of these counter-trafficking activities  through projects that address trafficking challenges specific to the field  mission’s region or country. USAID’s Center of Excellence on  Democracy, Human Rights and Governance (DRG Center) in  Washington, D.C. is responsible for oversight of USAID’s counter- trafficking policy. The DRG Center is responsible for coordinating and  reporting on USAID-wide counter-trafficking in persons efforts;  oversees the implementation of USAID’s counter-trafficking in persons  policy in collaboration with regional bureaus and country missions;  works with regional bureaus and country missions to gather counter- trafficking best practices and lessons learned; provides technical  assistance and training to field and Washington-based staff on  designing, managing, and monitoring and evaluating trafficking in  persons projects; and conducts and manages research and learning  activities related to combating trafficking in persons to collect data to  inform the design of field projects.

DOD. DOD’s Combating Trafficking in Persons Program Management  Office, under the Under Secretary of Defense for Personnel and  Readiness in the Defense Human Resources Activity, develops  trafficking awareness and training material for all DOD components.  On December 16, 2002, the President signed National Security  Presidential Directive 22, which declared the United States had a zero  tolerance policy for trafficking in persons. The Combating Trafficking  in Persons Program Management Office is responsible for  overseeing, developing, and providing the tools necessary for  implementing National Security Directive 22 within DOD. The office  has developed several different training programs, designed to  provide an overview of trafficking in persons (including signs of  trafficking, key policies and procedures, and reporting procedures), as  well as awareness materials for distribution to DOD components and  defense contractors overseas.

Treasury. Treasury has activities, but not specific programs, that may  support wider U.S. efforts to address counter-trafficking in persons,  according to Treasury officials. Pursuant to its mandate, components  of Treasury’s Office of Terrorism and Financial Intelligence (TFI),  including Financial Crimes Enforcement Network (FinCEN), Office of  Terrorist Financing and Financial Crimes (TFFC), and Office of  Foreign Assets Control (OFAC) work on addressing illicit finance  activities that support the wider goal of combating global trafficking in  persons.

Pursuant to the Trafficking Victims Protection Act of 2000, the President  established the President’s Interagency Task Force to Monitor and  Combat Trafficking in Persons (PITF), which is a cabinet-level entity that  consists of agencies across the federal government responsible for  coordinating implementation of the Trafficking Victims Protection Act of  2000, among other activities. It is chaired by the Secretary of State;  State, DOL, USAID, DOD, and Treasury are all PITF agencies. In  addition, the Trafficking Victims Protection Act, as amended in 2003,  established the Senior Policy Operating Group, which consists of senior  officials designated as representatives of the PITF agencies.

During Fiscal Year 2017, State, DOL, and USAID Managed 120 Counter- Trafficking in Persons Projects

State, DOL, and USAID managed 120 projects in counter-trafficking in  persons carried out by implementing partners during fiscal year 2017,  according to information provided by officials with these agencies. These  projects, as identified by agency officials, ranged from those focused on  counter-trafficking in persons, to those in which counter-trafficking in  persons was integrated into but was not the primary goal of the project.  At these agencies, project officers work with the implementing partner on  the administration and technical guidance of the project, such as  reviewing progress reports. Table 1 shows a summary of these  agencies’ project information; appendix II provides more detailed  information on all 120 projects.

During fiscal year 2017, State managed 79 counter-trafficking projects,  from those focused on individual countries, to regional and global ones  that covered several countries, with a total award amount of  approximately $62 million, according to information provided by State  officials. State TIP Office managed 75 projects with total awarded amount  of around $57 million. Award amounts per project ranged from  approximately $150,000 to $2.55 million. For example,

State TIP Office had 11 global projects totaling about $10 million and  6 regional projects in Africa amounting to about $4 million.

State TIP Office had two projects in Ghana that received the highest  amount of awards, approximately $2.5 million for each project.

State TIP Office had four projects in India amounting to around $3  million, and four in Thailand totaling around $2.35 million.

In addition to State TIP Office’s projects, State’s Bureau of Democracy,  Human Rights, and Labor (DRL) managed four counter-trafficking  projects with a reported total award amount of about $5 million, with two  projects in Mauritania making up around 70 percent of DRL’s total  awarded amount.

DOL’s ILAB/OCFT managed six projects in fiscal year 2017 with a total  award amount of approximately $31 million, according to DOL officials.  These projects ranged from one scheduled to last for 5 years with an  awarded amount of about $1 million, to one scheduled to last for about 4  years with an awarded amount of about $14 million. Three of DOL’s  projects were global projects, while two others focused on two countries  each and one project focused on one country.

USAID’s projects during fiscal year 2017 consisted of 2 regional projects  in Asia, and 33 individual projects in 22 different countries. Some of these  USAID-identified projects were integrated projects with a broader  development focus that includes USAID programmatic objectives other  than counter-trafficking in persons. According to information provided by  USAID officials, the award amount for all counter-trafficking in persons  projects active in fiscal year 2017, including all integrated projects and  standalone projects with a sole focus on combatting trafficking in persons,  totaled around $296 million; and USAID’s committed funding to these  projects’ activities related to counter-trafficking in persons was about $79  million as of September 2018. During fiscal year 2017, USAID focused  on a few countries where the agency awarded multiple counter-trafficking  projects, such as four projects in Nepal and four projects in Burma.

According to officials, State, DOL, and USAID generally design projects to  align with the “3Ps approach”—prevention, protection, and prosecution— and to consider trends and recommendations identified in agency reports  on foreign governments’ counter-trafficking efforts. According to State’s  publicly available information, the “3Ps” approach serves as the  fundamental counter-trafficking in persons framework used around the  world, and the U.S. government follows this approach to  1.  prevent trafficking in persons through public awareness, outreach,  education, and advocacy campaigns;  2.  protect and assist victims by providing shelters as well as health,  psychological, legal, and vocational services; and  3.  investigate and prosecute trafficking in persons crimes by providing  training and technical assistance for law enforcement officials, such  as police, prosecutors, and judges.

State’s publicly available information on the 3Ps noted that prevention,  protection, and prosecution efforts are closely intertwined. Prosecution,  for example, can function as a deterrent, potentially preventing the  occurrence of human trafficking. Likewise, protection can empower those  who have been exploited so that they are not victimized again once they  re-enter society. A victim-centered prosecution that enables a survivor to  participate in the prosecution is integral to protection efforts.

In addition to the “3Ps,” a “4th P”—for partnership—serves as a  complementary means to achieve progress across the “3Ps” and enlist all  segments of society in the fight against human trafficking, according to  State’s publicly available information. Addressing the partnerships  element, USAID’s counter-trafficking policy seeks to increase  coordination across a broad range of national, regional, and global  stakeholders from civil society, government, the private sector, labor  unions, media, and faith-based organizations.

DOL and USAID Fully Documented Their Monitoring Activities for All Selected Projects, but State Did Not Fully Document Its Activities for 16 of 37 Selected Projects

State, DOL, and USAID Use Similar Tools to Monitor Performance of Their Counter-Trafficking in Persons Projects

Monitoring is the collecting of data to determine whether a project is being  implemented as intended and the tracking of progress through  preselected performance indicators during the life of a project. State,  DOL, and USAID use a number of similar tools—according to their  current policies, guidance, and agency officials—to monitor the  performance of their counter-trafficking in persons projects, including  monitoring plans, indicators and targets, periodic progress reports, and  final progress reports. The agencies also conduct site visits, but their  policies vary on whether site visits are required for every project during  implementation.

Monitoring plan. The monitoring plan—according to monitoring  policies of the three agencies—documents, among other things, all of  the indicators and targets for the project as well as data collection  frequency for each indicator. In addition, according to State TIP Office  officials, the monitoring plan’s indicators and targets for TIP Office- managed counter-trafficking in persons projects are to be organized in  a logic model, which is a visual representation that shows the linkages  among the project’s goals, objectives, activities, outputs, and  outcomes (see table 2). The logic model is intended to show  relationships between what the project will do and what changes it  expects to achieve.

Indicators and Targets. Performance indicators—according to  monitoring policies of the three agencies—are used to monitor  progress and measure actual results compared to expected results.  Targets are to be set for each performance indicator to indicate the  expected results over the course of each period of performance.  According to agency officials, the monitoring plan documents  indicators and targets to be tracked and reported on through periodic  progress reports to assess whether the project is likely to achieve the  desired results. GAO has also found that a key attribute of effective  performance measures is having a measurable target.

Periodic progress reports. The reporting templates for the three  agencies show that periodic progress reports—which are submitted at  established intervals during the project’s implementation—compare  actual to planned performance and indicate the progress made in  accomplishing the goals and objectives of the project, including  reporting on progress toward the monitoring plan’s indicator targets.

Final progress report. The final progress report—according to  monitoring policies of the agencies or agency officials—is a stand- alone report that provides a summary of the progress and  achievements made during the life of the project.

Site Visits. The three agencies policies vary on whether site visits are  required for every project during implementation. For example,  State’s policy notes that site visits may be conducted to review and  evaluate recipient records, accomplishments, organizational  procedures, and financial control systems, as well as to conduct  interviews and provide technical assistance as necessary. In 2015,  the State TIP Office established a goal to conduct at least one site  visit during the life time of every project. While site visits during a  project’s implementation are not required under DOL’s policy, DOL  officials explained that they use site visits when deemed necessary to  supplement information from other forms of oversight. USAID’s policy  requires that a site visit be conducted for every project during  implementation to provide activity oversight, inspect implementation  progress and deliverables, verify monitoring data, and learn from  activity implementation.

In addition to these monitoring tools, State, USAID, and DOL officials told  us that they rely on frequent communication with implementing partners  as part of their monitoring process. Overall, monitoring is intended to help agencies determine whether the project is meeting its goals, update and  adjust interventions and activities as needed, and ensure that funds are  used responsibly.

DOL and USAID Fully Documented Their Monitoring Activities for Selected Projects, while State Did Not

We found, based on our review of 54 selected counter-trafficking in  persons projects (37 State, 3 DOL, and 14 USAID), that DOL and USAID  had fully documented their performance monitoring activities, while State  did not fully document its activities for 16 of 37 (43 percent) of the projects  we reviewed with project start dates between fiscal years 2011 to 2016.

DOL’s documented monitoring activities included the monitoring plan for  each project as well as fiscal year 2017 semi-annual progress reports,  including indicators and targets. USAID’s documented monitoring  activities included the monitoring plan for each project; fiscal year 2017  progress reports at the reporting frequency specified in the agreements  for each project; the final progress report, including indicators and targets,  for the three projects that ended as of December 2017; and evidence that  at least one site visit was conducted during each project’s  implementation. Overall, the three agencies reported having conducted at  least one site visit during the life time of the project for 47 of 54 (87  percent) of the selected projects.

As shown in table 3, State did not fully document its monitoring activities  (monitoring plan; fiscal year 2017 quarterly progress reports; and final  progress report, including indicators and targets, for projects that ended  as of December 2017) for 16 of the 37 selected projects we reviewed.  Specifically, State did not have nine monitoring plans, five complete  progress reports, or targets for each indicator in six of seven final  progress reports for projects that ended as of December 2017. (See  appendix III for detailed information on each of the 37 projects.)

For the nine projects for which the monitoring plan was not  documented, the State TIP Office indicated that it was unable to  locate these documents or they were not completed because the  projects were finalized when the TIP Office was beginning to institute  the monitoring plan requirement. Although TIP Office officials told us  that the TIP Office piloted and began to phase in the monitoring plan  requirement over the course of 2014 and early 2015, eight of the nine  projects without monitoring plans started in September or October  2015. We found that each of the nine projects had a logic model  used to report progress in the fiscal year 2017 quarterly progress  reports we reviewed, which would have provided TIP Office officials a  basis for monitoring project performance at that point. However,  federal standards for internal control call for agency management to  design monitoring activities so that all transactions are completely and  accurately recorded and so that management can evaluate project  results. Specifically, internal controls specify that monitoring should  be ongoing throughout the life of the project, which is consistent with  State’s current policy that generally requires completion of the  monitoring plan prior to award. Without timely documentation of the  monitoring plans at the start of the project, TIP Office officials may not  be able to ensure that projects are achieving their goals, as intended,  from the beginning of project operations.

For the three projects for which the quarterly progress report for the  first quarter of fiscal year 2017 had been partially completed, the State  TIP Office indicated that the implementing partners began to use the  TIP Office’s quarterly reporting template for subsequent reports after  TIP Office officials instructed the implementing partner to do so. For  the one project where the quarterly progress report was not  completed for the third quarter of fiscal year 2017, or partially  completed for the fourth quarter of fiscal year 2017, the project officer  provided possible reasons why the documents were not in the  project’s file, including that the implementing partner lacked the  capacity to design a logic model. The project ended December 31,  2017. Federal standards for internal control call for agency  management to design monitoring activities, such as performance  reporting, so that all transactions are completely and accurately  recorded, and project results can be continuously evaluated. As  previously discussed, performance progress reports should compare  actual to planned performance and indicate the progress made in  accomplishing the goals and objectives of the project. Therefore, the  TIP Office may lack information needed to assess project  performance if it does not have access to complete monitoring  documentation.

For the six projects for which targets were not fully documented in the  final progress reports, we found that targets were lacking for 110 of  253 (43 percent) of indicators across the six final progress reports.  Our prior work on performance measurement identified 10 key  attributes of performance measures—such as having a measurable  target—that GAO has found are key to successfully measuring a  project’s performance. For example, our prior work has shown that  numerical targets or other measurable values facilitate future  assessments of whether overall goals and objectives are achieved  because comparisons can be easily made between projected  performance and actual results. State TIP Office officials explained  that the final progress reports we reviewed lacked targets because the  TIP Office had not required targets for each indicator for the projects  we reviewed that started in fiscal years 2011 to 2016. State TIP Office  officials also said that project officers may not have set targets due to  limited resources in previous years. A lack of actual targets limits the  TIP Office’s ability to assess project performance, including  effectiveness, and determine if implementation is on track or if any  timely corrections or adjustments may be needed to improve project  efficiency or effectiveness.

According to State TIP Office officials, the TIP Office has taken steps to  improve its documentation of monitoring activities, such as instituting a  monitoring plan requirement; increasing staff, including hiring a  monitoring and evaluation specialist; and developing standard templates  for implementing partners to use for reporting. Moreover, in November  2017, State established a new policy asserting that, building on the logic  model or project charter, bureaus and independent offices must set  targets for each performance indicator to indicate the expected change  over the course of each period of performance. It further notes that  bureaus and independent offices should maintain documentation of  project design, including the logic model. Additionally, State TIP Office  officials said that State is developing a department-wide automated  information management system (State Assistance Management System  - Domestic, or SAMS-D) that officials expect to standardize entry of  performance information and, under the new system targets, must be  recorded for each indicator. State TIP Office officials have worked to pilot- test SAMS-D to provide feedback on the system, including suggestions to  improve the completeness of data collection, according to TIP Office  officials.

Despite these efforts, the TIP Office’s documentation of all monitoring  activities, and implementation of its November 2017 requirement to set  targets for all performance indicators, is uncertain. For example, even  though the TIP Office informed us that it began to institute a monitoring  plan requirement over the course of 2014 and early 2015, as previously  noted, eight projects we reviewed that started in September or October  2015 did not have monitoring plans. In addition, according to State  officials, in SAMS-D, targets could be recorded as “to be determined” and  there are no controls in place to ensure that “to be determined” entries  are replaced with actual targets. State officials said that SAMS-D has the  capability to implement controls to alert users to update “to be  determined” targets, but pilot users of SAMS-D, which include the TIP  Office, have not provided feedback for this capability so far. Furthermore,  State TIP Office officials informed us that the TIP Office cannot require all  implementing partners to set targets, but that the TIP Office aspires to  update relevant targets regularly in the future and would encourage  implementing partners to update target values when appropriate.  Without controls to ensure full documentation of monitoring activities and  established performance targets, State is limited in its ability to assess  project performance, including project efficiency or effectiveness.

State and USAID Do Not Have Sufficient Controls to Ensure the Reliability of Project Information, while DOL Had Consistent and Complete Performance Information in the Project We Reviewed

In our review of selected indicators in two State TIP Office and two  USAID projects, we found that State and USAID used inconsistent and  incomplete performance information to monitor these projects. We found  that State TIP Office and USAID do not have sufficient controls in place to  ensure that the performance information they use is reliable. In contrast,  we found that DOL had consistent and complete performance information  in a project we reviewed, and we identified no controls in DOL’s process  that were insufficient for assuring the reliability of this information.

State and USAID Projects We Reviewed Showed Inconsistent and Incomplete Performance Information

For selected indicators in two State TIP Office and two USAID projects,  we found numerous errors or omissions in progress reports we reviewed,  which resulted in inconsistent and incomplete performance information  agencies used to monitor these projects. Specifically, we found  examples of inconsistent information, which included many instances in  which quarterly indicator totals differed from annual or cumulative totals  reported separately on the same projects, and numbers reported in  narrative information that differed from numbers reported as indicator  values. In addition, we found examples of incomplete information,  including narrative elements that were missing in whole or in part.

Inconsistent Performance Information. We found numerous instances  in which quarterly totals differed from annual or cumulative totals reported  separately on the same projects. When these errors occurred, it was not  possible to independently determine project performance based on report  information. For example,

For one State TIP Office project, reported cumulative progress  overstated quarterly progress for at least 11 indicators (3 of which by  25 percent or more) and understated quarterly progress for at least 5  indicators (once by 25 percent or more). For example, for the indicator  “number of standardized reintegration protocols/guidelines/tools  developed (case forms, family assessment, etc.,)” State’s cumulative  performance report as of the 4th quarter of fiscal year 2017 indicated  that two tools had been developed, whereas quarterly reports showed  that only one had been developed.

For one USAID project, the indicator “number of assisted communes  allocating and accessing funds for trafficking in persons prevention  activities” showed that annual results were 60, while quarterly report  data combined showed that the number was 6, which USAID officials  confirmed was the correct figure.

For another USAID project, the indicator, “number of food security  private enterprises (for profit), producers organizations, water users  associations, women’s groups, trade and business associations, and  community-based organizations receiving U.S. government  assistance” showed an annual result of one, while quarterly totals  combined showed a total of three, which USAID officials confirmed  was the correct figure.

For the projects we reviewed, implementing partners produced narrative  descriptions of progress made to accompany indicator results. We found  cases in which numbers reported in narrative information were not  consistent with numbers reported as indicator values. For example, for  the State TIP Office indicator “number of criminal justice practitioners  trained” for one project, indicator results for two quarters differed from  results presented in the corresponding narrative during fiscal years 2016  to 2017. State officials found that the narrative information was correct for  one of these inconsistencies and the indicator result was correct for the  other. In addition, for one USAID indicator—number of public awareness  tools on trafficking in persons developed and disseminated—the narrative  report for one quarter described distributions that added up to 21,765  products, while the reported quantitative indicator total was 21,482.  USAID officials confirmed that 21,765 was the correct figure.

Incomplete Performance Information. Additionally, some quarterly  reports had narrative elements that were incomplete in whole or in part,  which made independent interpretation of project performance difficult or  impossible.

The implementing partner in one State TIP Office project copied and  pasted significant portions of narrative information in quarterly reports  for 2 years and, according to State TIP Office officials, did not fulfill a  request by State TIP Office to include only current quarterly  information in formal quarterly reports because it was focused on  other activities. For nearly the entire period, the implementing partner  indicated that it was “following up” with government entities in three  countries to set up counter-trafficking in persons training for  government officials, but no indication was made in formal quarterly  reports about the results of any of these follow-up activities.

For one State TIP Office project, the indicator “number of children  receiving care, whose cases are reported to the police” had no  narrative information or incomplete narrative information provided for  three of the four quarters in which activity occurred during our period  of review (comprising almost 90 percent of reported performance  under this indicator).

For a USAID project, the implementing partner reported a combined  performance number of approximately 200 from the first through third  quarters of fiscal year 2017 for the indicator “number of members of  producer organizations and community based organizations receiving  U.S. government assistance.” However, annual performance for fiscal  year 2017 was reported as nearly 1,700 organizations. USAID officials  explained that this difference was the result of the implementing  partner’s misinterpretation of the indicator’s definition when producing  the quarterly reports, but the annual report narrative did not explain  this correction.

Additionally, for USAID’s indicator on the “number of public  awareness tools on trafficking in persons developed and  disseminated,” no narrative information in the quarterly or annual  reports explained how the last quarter of fiscal year 2016 performance  approximately doubled from that of the previous quarter. Narrative  information in the annual report described performance for the year  only in general terms and did not clarify this significant change.

In addition to direct project oversight, State TIP Office and USAID officials  stated that performance information from progress reports that the  agencies use to monitor counter-trafficking in persons projects is regularly  used for internal and external reporting, program decisions, and lessons  learned. For example, according to officials, this information is used by  senior agency officials to inform their decision-making, in reports such as  the Attorney General’s Annual Report to Congress and Assessment of  U.S. Government Activities to Combat Trafficking in Persons, and to fulfil  other requests from Congress.

Neither State TIP Office nor USAID Has Sufficient Controls to Ensure the Reliability of Performance Information

Neither State TIP Office nor USAID has sufficient controls to ensure  consistent and complete performance information, and both face  challenges to data reliability stemming from information reported in non- standard formats, implementing partners with limited capacities to report  performance information, and the time-consuming nature of reviewing  reported information. Federal internal control standards state that  management should obtain data from reliable internal and external  sources. According to these standards, reliable internal and external  sources should provide data that are reasonably free from error and bias  and faithfully represent what they purport to represent; and management  should evaluate both internal and external sources of data for reliability.  Without implementing additional controls to ensure that performance  information are consistent and complete, State and USAID officials may  not fully or accurately understand what projects are, or are not, achieving  and, therefore, how their efforts could be altered as needed. Further,  reports that are prepared or program decisions that are made using the  TIP Office monitoring reports could be based on inconsistent or  incomplete information that does not accurately present project results.

State Lacks Adequate Controls to Ensure the Reliability of Performance Information

State TIP Office currently receives performance information using  documents submitted by implementing partners, although this information  is not compiled into a single data system and is not in a standardized  format. While State provides suggested templates for reporting  information, officials said that they cannot require implementing  organizations to use these templates and we found that implementing  partners provided information in varying formats.

According to State TIP Office officials, project officers perform manual  reviews of quantitative information in monitoring reports but have  insufficient time to carry out detailed reviews of data reliability for all  indicators. State TIP Office project officers also stated that the process of  comparing narrative information to indicator information was time  consuming and difficult. According to these officials, the quality of the  information in progress reports also depends on the priorities and  resources—which can be limited—of the implementing partner. In  addition to reviewing progress reports, State project officers we spoke to  said that they rely on site visits and frequent, less formal communication  as part of their oversight process.

Project officers for the State TIP Office projects we reviewed stated that  they did not always examine performance trends over time or review  consistency in reported cumulative totals—which should be the sums of  the previous and current quarters’ reported results—with quarterly totals,  for reasons including the difficulty in assembling quarterly information in  this manner and resource limitations. State TIP Office officials noted that  they are aware of data quality problems in counter-trafficking in persons  monitoring reports.

State is developing SAMS-D, a system that officials expect to standardize  entry of information from common performance indicators and logic  models, according to State officials. These officials stated that if SAMS-D  is deployed, State TIP Office could find it easier to analyze and revise  logic models that implementing partners submit, as well as examine  performance indicator results over time, since standardized data would be  available in a centralized location. According to State officials, SAMS-D  could be programmed with automatic checks or alerts under conditions  defined by the TIP Office and the database programmer. For example,  the system could require that fields be filled out in particular formats or  provide an alert if performance under a certain indicator has significantly  deviated from prior quarters or the indicator’s target.

State TIP Office officials said they were uncertain whether SAMS-D would  become operational in 2019, as currently planned. According to officials,  State TIP Office has participated in planning and pilot activities for SAMS- D, including testing monitoring tools with implementing partners.  According to these officials, additional work is needed to develop rules  and controls necessary to operationalize SAMS-D to meet the TIP  Office’s particular needs and ensure improved data. Another challenge to  implementation of SAMS-D, according to these officials, is that some  implementing partners are unable to maintain consistent internet  connections necessary to upload information, impeding full roll-out of the  system, and an alternative upload mechanism does not yet exist.

USAID Lacks Adequate Controls to Ensure the Reliability of Performance Information

According to USAID officials, overseas missions currently set many of  their own policies and procedures for data quality oversight. For the two  projects we reviewed, USAID relied on implementing partners to manage  information, while it reviewed this information in addition to conducting  site visits and communicating with implementing partners on a regular  basis to monitor the projects. USAID officials attributed errors in the  project reports we reviewed to factors including implementing partners’  errors in manual computation and misunderstandings of indicator  definitions. According to USAID officials, data quality errors due to factors  such as transcription errors can also occur in the performance information  USAID uses to monitor counter-trafficking in persons projects.

USAID project officers for the projects we reviewed said that they  regularly conducted manual analysis of information received from  implementing partners, but USAID and implementing partners are often  pressed for time during the quarterly reporting cycle. According to these  project officers, some of the errors GAO found had already been  identified by USAID implementing partners during their annual review  process and corrected in the annual reports we reviewed. For example,  for the USAID indicator “value of new private sector investments in select  value chains,” quarterly totals overstated corrected annual results by  more than $120,000—approximately $170,000 instead of approximately  $50,000. USAID officials said that they and the implementing partner had  identified that the implementing partner was incorrectly including  additional, unrelated data when producing its quarterly totals and while  the annual total had been corrected to approximately $50,000, the annual  report did not indicate that this error had occurred in the quarterly reports.  USAID officials noted that the quality of the information in the progress  reports also depends on the experience and capacity—which can be  limited—of the implementing partner.

According to USAID officials, USAID is currently building the  Development Information Solution (DIS), an agency-wide information  system that would provide USAID’s operating units (such as headquarters  bureaus or field missions) with a tool to better collect, track, and analyze  information to improve how they manage their projects and overall  strategies. Implementing partners would be able to access the DIS via a  portal where they would directly enter project information and upload  reports and supporting information, according to this official. In addition,  this information would better inform USAID’s decision-making at the  operating unit level and agency level. A USAID official explained that  USAID developed DIS partly as a result of USAID senior management’s  concern about the lack of one corporate system to collect data in a timely  fashion and improve efficiency.

A USAID official responsible for managing DIS informed us that the  business case for DIS was approved in fiscal year 2016. Developers have  regularly solicited input from across the agency, according to this official,  and a pilot with six missions is expected to begin in November 2018. This  official explained that USAID plans to have DIS operational by the end of  2019, but DIS’s timeframe has been accelerated by a year, to 2019 from  2020, which may create programming and budget challenges, and  unexpected challenges may also arise during the pilot process as mission  needs for DIS are more fully assessed. USAID is currently developing  training, deployment, and communications plans to prepare the agency  for implementing DIS, according to officials.

DOL Had Consistent and Complete Performance Information for the Selected Project and We Identified No Controls Insufficient to Ensure the Reliability of Performance Information

We reviewed selected indicators and targets information in one DOL  project and identified no significant consistency or completeness issues  beyond early project stages. For example, for the indicator “number of  countries that ratify the International Labor Organization Protocol on  Forced Labor,” the October 2016 report contained no reported value for  this indicator, while the subsequent report (April 2017) updated this figure  to indicate a value of “4” for October 2016. DOL officials explained that a  data reporting form had not yet been developed as of October 2016, but  indicator performance was discussed in the October 2016 narrative and  added to the data reporting form when it was developed. While DOL does  not require that a project progress report discuss every indicator  associated with an activity in the performance report narrative, according  to officials, we found that explanations were present for every significant  performance-related event that we identified for the fiscal year 2016 and  fiscal year 2017 period.

We did not identify any controls in DOL’s process that were insufficient to  ensure the reliability of performance monitoring information. DOL officials  said that they use a system of spreadsheets with automated calculations  and validation checks that are intended to standardize information  submission and assure consistency and completeness of submitted  information. These officials said that the project’s Comprehensive  Monitoring and Evaluation Plan defines rules for how information for  indicators is to be collected and how indicators are to be computed from  this information. According to these officials, DOL develops a customized  indicator reporting form for each project in conjunction with implementing  partners, which implementing partners complete as part of their regular  reporting requirements. According to these officials, these spreadsheets  contain formula checks to mitigate the risk of implementing partners  making undisclosed changes to indicator results and array information in  a standardized manner across reporting periods. Officials also  commented that for internal reporting purposes, such as the Government  Performance and Results Act, project officers can extract information  from indicator templates in a manner that is not overly burdensome.  According to officials, DOL is developing an enhancement to existing  tools, expected in late 2019, which will provide a traceable way to send  and receive reports from grant recipients; timestamps when reports are  sent, received, and accepted; and tracking of performance monitoring  communications between DOL and implementing partners. They plan to  continue to use a spreadsheet-based system for tracking indicator  information.

State Does Not Have a Process to Ensure that All Performance Indicators are Useful, while USAID and DOL Have Established Processes to Regularly Review the Usefulness of Indicators State TIP Office Does Not Have a Process to Review All Indicators to Ensure Their Usefulness

State TIP Office does not have a process to regularly review the number  and content of indicators for counter-trafficking in persons projects to  ensure that these indicators are useful and that collecting and reviewing  information for them is not overly burdensome. State TIP Office officials  acknowledged there are too many indicators for many counter-trafficking  in persons projects. Project officers have the discretion to revise  indicators if the scope of the project is not altered, according to State  officials. In addition, according to these officials, changes that alter the  project scope are possible with the consent of the implementing partner.  However, State TIP Office project officers do not formally indicate which  indicators they have determined are most useful and informed us that  they have insufficient time and resources to do so as projects progress.  One official who focuses on monitoring issues stated that, ideally, there  should be three to five indicators per activity, and efforts have been made  to reduce the number of indicators in some projects. For example, in one  of the State TIP Office projects we reviewed—which was designed prior  to the hiring of this official—had more than 230 indicators across 20  activities as of the first quarter of fiscal year 2017, which had been  reduced to about 150 by the fourth quarter of fiscal year 2017.

Our review of two State TIP Office projects showed that indicators did not  change in some situations even when the project officer considered the  indicator to have become less relevant. State project officers explained  that, instead of only relying on indicator information, they regularly spoke  with implementing partners for an understanding of what performance  level to expect. While acknowledging errors in the numerical information  for some indicators, project officers for the two projects we reviewed said  that they sometimes overlooked reviews of all reported indicators in the  quarterly progress reports because they consider some indicators to be  less useful or unimportant and not needed for monitoring purposes, and  burdensome to review in depth. These officials said that project officers  focus on the indicators that they consider to be most important for project  oversight or congressional requests.

State TIP Office officials said that logic models, which include indicators,  have improved significantly in recent years (including improvements to  the suggested logic model template and the glossary of definitions), partly  due to hiring additional monitoring staff, but that State has found the  analysis of logic models to be difficult because of the absence of  centralized and standardized information and a lack of staff capacity. In  addition, project officers stated that they often rely on implementing  partners for suggestions with regard to changing indicators. However,  according to State officials, these implementing partners may be reluctant  to bring up challenges they encounter out of concern that doing so may  damage their relationship with State.

State’s Program Design and Performance Management Toolkit, rolled-out  in 2017, states that indicators can be costly to collect and manage and  should therefore be “useful,” which includes having a clear utility for  learning, tracking, informing decisions, or addressing ongoing program  needs. This policy further states that indicators should also be  “adequate,” which includes having only as many indicators in overall  monitoring plan as are necessary and feasible to track key progress and  results, inform decisions, conduct internal learning, and meet any external  communication or reporting requirements. Further, federal internal control  standards state that management should establish and operate  monitoring activities, and, after doing so, may determine how often it is  necessary to change the design of the internal control system as  conditions change to effectively address objectives. Without a process  to ensure that the number and content of counter-trafficking in persons  project indicators are reviewed and modified as needed, project  monitoring may be less efficient and effective as implementing partners  and State TIP Office staff spend time collecting and reviewing indicator  information that is not useful for project monitoring and management.

DOL and USAID Have Established Processes to Regularly Review the Usefulness of Indicators

DOL and USAID had processes in place to regularly review indicators for  the projects we selected. DOL officials told us that project officers work  with subject-matter experts to review the relevance of indicators in each  semi-annual reporting period. These officials also stated that grantees are  required to review their monitoring and evaluation plan annually, which  includes the project’s indicators, and to provide the most recent work plan  with each semi-annual report. According to DOL officials, while not a DOL  requirement, the project we reviewed incorporated a work plan for each  component of the project defining when important activities were planned  under each output indicator. We found that DOL and the implementing  partner made regular changes to these project plans in response to  changing conditions. These plans were consistently included in the  monitoring documents and most elements were discussed in the  associated narrative text.

USAID conducts its project oversight primarily out of its overseas  missions, according to USAID officials. According to USAID officials  associated with the projects we reviewed, these officials should review  the project’s indicators annually, as well as when they determine a review  is needed, such as when projects have changes in planned activities.  USAID officials stated that this annual review process may be explicitly  required in some agreements. According to these officials, missions or  other operating units are required to manage and update reference  sheets for indicators, which officials said are intended to define each  indicator and the information to be collected to measure each indicator.  Changes to these reference sheets are tracked, according to these  officials. Projects we reviewed showed evidence of regular changes to  indicators and associated targets. We spoke to project officers about  several specific changes that we had identified. For many of these  changes, the project officers provided information about their work with  implementing partners to appropriately adjust program goals and  expectations, such as adapting the project indicators and targets to  unexpected or changing conditions.

Given the grave suffering of victims and damaging effects on society that  trafficking in persons imposes, and the U.S. government’s reliance on  implementing partners to carry out its counter-trafficking projects,  performance monitoring is important to ensure that the United States  funds projects that are effective, efficient, and achieve their intended  counter-trafficking goals. In fiscal year 2017, State, DOL, and USAID  managed 120 counter-trafficking projects and monitored the performance  of the projects. However, weaknesses in State’s and USAID’s monitoring  processes limit their ability to collect reliable performance information and  assess project performance. First, we found that the State TIP Office did  not fully document its monitoring activities for many of the projects we  reviewed that started in between fiscal years 2011 to 2016. Monitoring the  implementation of projects and fully documenting the results of such  monitoring are key management controls to help ensure that project  recipients use federal funds appropriately and effectively. The State TIP  Office was also not setting targets for some project indicators, which may  have limited the TIP Office’s ability to determine if implementation was on  track or if corrections needed to be made. Furthermore, we found that the  State TIP Office and USAID used project performance information  reported by the implementing partners—used for internal and external  reporting purposes—that was not always consistent or complete, and did  not have sufficient controls to ensure the reliability of performance  information. Finally, to ensure effective and efficient monitoring, projects  need to establish a reasonable number of indicators and update them as  needed. However, we found that the State TIP Office does not regularly  evaluate and revise all of its indicators for counter-trafficking in persons  projects, which can have large numbers of indicators. As a result, the  State TIP Office may be using information to monitor project performance  that that is less useful and relevant for understanding project progress,  and requires more resources and time for the implementing partners to  produce and agency officials to review.

State TIP Office officials noted that the TIP Office has taken steps to  improve its monitoring process, and State and USAID officials explained  that State and USAID are developing information management systems  that may increase the quality and usefulness of the monitoring information  they use. However, these systems are not fully designed or operational  and their capabilities are not yet known. Thus, the potential of these  systems to strengthen the ability of State and USAID to collect reliable  performance information and assess their efforts to combat the serious  problem of global trafficking in persons is unclear. State and USAID could  benefit from making additional improvements to ensure their projects are  being implemented as intended and achieving project goals to prevent  trafficking in persons, protect victims, and prosecute trafficking crimes.

We are making a total of five recommendations, including four to State  and one to USAID. Specifically:  The Secretary of State should ensure that the Director of the TIP Office  establishes targets for each performance indicator. (Recommendation 1)

The Secretary of State should ensure that the Director of the TIP Office  maintains documentation of all required monitoring activities, including  monitoring plans, progress reports, and performance targets.  (Recommendation 2)

The Secretary of State should ensure that the Director of the TIP Office  establishes additional controls to improve the consistency and  completeness of performance information that the TIP Office uses to  monitor counter-trafficking in persons projects. (Recommendation 3)

The Secretary of State should ensure that the Director of the TIP Office  establishes a process to review and update performance indicators, with  the participation of implementing partners, to ensure that project  monitoring remains efficient and effective. (Recommendation 4)

The Administrator of USAID should establish additional controls to  improve the consistency and completeness of performance information  that USAID uses to monitor counter-trafficking in persons projects.  (Recommendation 5)

We provided a draft of this report to State, DOL, USAID, DOD, and the  Treasury for review and comments. In State’s and USAID’s letters,  reproduced in appendixes IV and V, respectively, both agencies  concurred with our recommendations and described their planned actions  to address the recommendations. In addition, State’s letter indicated that  our draft report did not fully recognize the investment State has made,  and the changes underway, to improve the TIP Office’s performance  measurement and ensure complete and consistent documentation. State  cited additional dedicated financial and personnel resources for  monitoring and evaluation added over the past two years. We  acknowledge and report on these positive steps, including the hiring of a  monitoring and evaluation specialist and other TIP Office staff, in our  report. USAID’s letter included other comments that we have responded  to in appendix V. Furthermore, State, DOL, USAID, and the Treasury  provided technical comments, which we incorporated as appropriate.  DOD had no comments.

We are sending copies of this report to the appropriate congressional  committees; the Secretaries of State, Labor, Defense, and Treasury; and  the Administrator of USAID. In addition, the report is available at no  charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please  contact me at (202) 512-7141, or groverj@gao.gov. Contact points for our  Office of Congressional Relations and Public Affairs may be found on the  last page of this report. GAO staff who made major contributions to this  report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

The National Defense Authorization Act for Fiscal Year 2017 includes a  provision for GAO to report on the programs conducted by the  Department of State (State), the Department of Labor (DOL), the United  States Agency for International Development (USAID), the Department of  Defense (DOD), and the Department of the Treasury (Treasury) that  address human trafficking and modern slavery, including a detailed  analysis of the effectiveness of such programs in limiting human  trafficking and modern slavery. Three of these agencies—State, DOL,  and USAID—have programs that design and award counter-trafficking  projects to implementing partners, through contracts, grants, or  cooperative agreements.  These agencies then oversee and monitor  these projects. Since DOD and Treasury officials did not identify these  types of projects as part of their counter-trafficking in persons efforts, we  provided background information on their efforts but did not cover these  agencies in our reporting objectives. This report (1) identifies the recent  projects in international counter-trafficking in persons that key U.S.  agencies have awarded to implementing partners, and for selected  projects, assesses the extent to which key agencies have (2) documented  their monitoring activities, (3) ensured the reliability of the performance  information they use in monitoring projects, and (4) reviewed the  usefulness of the performance indicators they use in monitoring projects.

To address these objectives, we reviewed relevant agency documents  and interviewed agency officials. To report on agencies’ programs, we  asked knowledgeable officials at State, DOL, USAID, DOD, and Treasury  to identify their projects that (1) had an international focus; (2) were  delivered by implementing partners to external recipients, such as  trafficking victims or host governments, as project beneficiaries; and (3)  addressed trafficking in persons,   modern slavery, or forced labor. Because State, DOL, and USAID  managed such projects, we focus on them as the three key agencies for  the purposes of our reporting objectives. According to officials from these  three agencies, the projects they identified range from those with counter- trafficking in persons as a primary goal, to those in which this goal was  integrated as part of each agency’s activities. We used the lists of  projects that these agencies provided to report the relevant counter- trafficking projects that agencies awarded to implementing partners to  carry out the projects. For our first objective, we determined the projects  that were active during fiscal year 2017, including those which began,  were ongoing, or ended during fiscal year 2017, and interviewed agency  officials to confirm project information.

To analyze the effectiveness of agencies’ programs in limiting human  trafficking and modern slavery, we assessed the key agencies’ monitoring  efforts for selected projects by examining the extent to which agencies  have documented their monitoring activities, ensured the reliability of the  performance information, and reviewed the usefulness of the  performance indicators they use in monitoring projects.

To assess the extent to which State, DOL, and USAID documented their  monitoring activities for selected counter-trafficking in persons projects,  we reviewed these agencies’ monitoring policies and related guidance as  well as the full agreements for the projects to identify specific required  monitoring activities. The policies and related guidance included State’s  Grants Policy Directive Number 42 (GPD-42) related to monitoring  assistance awards; Federal Assistance Policy Directive (FAPD), which  according to a State official superseded State’s grants policy directives,  including GPD-42; Federal Assistance Directive, which superseded the  FAPD; Program Design and Performance Management Toolkit; and  Program and Project Design, Monitoring, and Evaluation Policy. We  also reviewed State’s Office to Monitor and Combat Trafficking in Persons  standard operating procedures. For DOL, we reviewed its Management  Procedures and Guidelines (MPG) as well as the Comprehensive  Monitoring and Evaluation Plan Guidance Document referenced in the  fiscal year 2017 MPG. For USAID, we reviewed—from its Automated  Directives System or ADS—Chapter 203 on Assessing and Learning  and Chapter 201 on Program Cycle Operational Policy, which according  to USAID officials superseded Chapter 203. Once we determined what  tools the agencies use to monitor their counter-trafficking in persons  projects, we sought documentation of those tools to determine whether  agencies were implementing those tools.

To assess the agencies’ monitoring efforts, we identified all of State’s,  DOL’s, and USAID’s projects that started before or during October 2015,  which corresponds to the first quarter of fiscal year 2016, and were active  through September 30, 2017, which corresponds to the fourth and last  quarter of fiscal year 2017. This produced a list of a total of 57 State,  DOL, and USAID projects. Out of these 57 projects, we excluded 3  projects from our selection for various reasons. We excluded one DOL  project because DOL identified the project as being a research project for  which certain agency performance monitoring requirements (e.g.,  indicators, targets) are not applicable. We also excluded two USAID  projects because USAID identified each project as including several  projects with various start and end dates, thus making it difficult to  determine their time frames for inclusion in our report. This resulted in a  selection of 54 projects—37 from State, 3 from DOL, and 14 from USAID.  We reviewed documentation of key monitoring activities as specified in  agency policy or the project award agreements to determine the extent to  which the agencies had full documentation of key monitoring activities.  We also applied federal standards for internal control, which call for  agency management to design monitoring activities so that all  transactions are completely and accurately recorded, and GAO’s key  attributes of effective performance measures, specifically the attribute of  having a numerical target. We made our determinations of the extent to  which agencies had full documentation of key monitoring activities, as  follows:

State (37 projects). To determine whether State had fully  documented its monitoring activities, we reviewed the monitoring plan  for each project; fiscal year 2017 quarterly progress reports for each  project; and the final progress report, including indicators and targets,  for the seven projects that ended as of December 2017.

We determined that State had “fully documented” the monitoring  plan, if State provided a monitoring plan worksheet for the project.  If State did not provide a monitoring plan worksheet for the  project, we determined the monitoring plan was “not documented.”

For each quarterly progress report for fiscal year 2017 as well as  the final progress report for projects that ended as of December  2017, we determined that State had “fully documented” the report,  if the report included both a qualitative and quantitative summary  of progress. For the State TIP Office projects we reviewed, the  qualitative summary of progress is captured in a narrative and the  quantitative summary of progress is captured in the logic model.  For the State DRL project we reviewed, the qualitative summary of  progress is captured in a narrative and the quantitative summary  of progress is captured in the monitoring plan. If either  component—narrative or quantitative summary—was not  documented, we determined that the report was “partially  documented.” If both components were not documented, we  determined that the report was “not documented.”

We determined that State had “fully documented” indicators and  targets for projects that ended as of December 2017, if the final  progress report for the project included indicators as well as  targets for each indicator. If the final progress report included  indicators but did not specify targets for each indicator, we  determined that indicators and targets were “partially  documented.” If the final progress report did not include  indicators and targets, we determined that indicators and targets  were “not documented.” (We did not find any instances of “not  documented.”)

DOL (3 projects). To determine whether DOL had full documentation  of its monitoring activities, we reviewed the monitoring plan as well as  fiscal year 2017 semi-annual progress reports for each project.  Because DOL’s three projects were ongoing as of December 2017,  we reviewed the second semi-annual progress report for fiscal year  2017 to determine whether DOL had “fully documented” indicators  and targets for each project. Overall, we determined that DOL had  “fully documented” (1) the monitoring plan for each project, if the  monitoring plan documented the performance metrics and data  collection frequency for the project; (2) each fiscal year 2017 semi- annual progress report for the project, if the report included a  qualitative and quantitative summary of progress for the period of  performance; and (3) indicators and targets for the project, if the  second semi-annual progress report included indicators as well as  targets for each applicable indicator.

USAID (14 projects). To determine whether USAID had full  documentation of its monitoring activities, we reviewed the monitoring  plan for each project; fiscal year 2017 progress reports at the  reporting frequency specified in the agreements for each project; and  the final progress report, including indicators and targets, for the three  projects that ended as of December 2017. We also reviewed evidence  of site visits conducted during the life time of the projects. Overall, we  determined that USAID had “fully documented” (1) the monitoring plan  for each project, if the monitoring plan documented performance  metrics for the project; (2) the periodic progress reports for fiscal year  2017 as well as the final progress report for projects that ended as of  December 2017, if the report included a qualitative and quantitative  summary of progress for the period of performance; and (3) indicators  and targets for the three projects that ended as of December 2017, if  the final progress report included indicators as well as targets for each  applicable indicator. We determined that USAID “fully documented” a  project’s site visit, if USAID provided evidence of having conducted at  least one site visit during the life time of the project.

Additionally, we interviewed knowledgeable monitoring officials from each  agency to understand agencies’ monitoring process and application of  monitoring requirements for counter-trafficking in persons projects.  Because State and DOL officials also identified site visits as a key tool  they use to monitor their counter-trafficking in persons projects, we  reviewed evidence of site visits conducted during the life time of the  projects to report on these efforts. We also interviewed State TIP Office  officials to discuss instances in which the agency did not have full  documentation of key monitoring activities.

To assess the extent to which key agencies have ensured the reliability of  the performance information they use to monitor selected projects, we  selected for review a nongeneralizable sample of 5 projects—2 State  projects, 1 DOL project, and 2 USAID projects—out of the 54 counter- trafficking in persons projects identified by agencies that started before or  during October 2015 and were active through fiscal year 2017. We based  our selection of these projects primarily on largest total award amounts.  For these selected projects, we obtained 2 years of progress reports and  other documents to assess the quantitative and qualitative performance  information. We developed a standardized template to capture all  quarterly or semi-annual indicator performance information reported for  each of these projects and assessed whether quarterly or semi-annual  totals were consistent with annual and cumulative totals where these  were reported. Using this quantitative information, we judgmentally  selected indicators for inclusion in agency interviews where it appeared  likely that numerical errors had occurred or there appeared to be  significant project events, such as large over- or under-performance or  the elimination of the indicator. We interviewed agency officials, including  managers of these five projects, about the consistency and completeness  of monitoring information in these projects for about 60 indicators  identified through our analysis. Additionally, we questioned these  officials about performance report narrative information describing project  activities that, in our judgement, appeared to be incomplete or  inconsistent with respect to indicator results. We also used these  interviews to determine whether our findings for these selected projects  reflected general agency policies and procedures. We assessed the  completeness and consistency of project performance data that State,  DOL, and USAID use to monitor projects as part of our data reliability  assessment. We found State and USAID data to be unreliable in the  projects we reviewed. We discuss the implications of these unreliable  data for State and USAID’s project management and reporting in our  findings and recommendations. We found the performance data that DOL  used were consistent and complete for the project we reviewed. While we  examined indicator data and narrative information for consistency and  completeness, we did not verify the accuracy of performance information.

To assess the extent to which key agencies have reviewed the  usefulness of the performance indicators they use to monitor selected  projects, we used the same nongeneralizable sample of five projects— two State projects, one DOL project, and two USAID projects. We  interviewed agency officials, including managers of these five projects,  about processes and systems they use to review the usefulness of  indicators on an ongoing basis, such as when conditions in the project  activity region change or if the agency and implementing partner learn  that certain project activities are less effective than expected. We  identified examples of indicators that had apparently been discontinued,  as well as continued indicators that showed minimal progress, and we  asked these officials to explain what had or had not been discontinued.  We also used these interviews to determine whether our findings for  these selected projects reflected general agency policies and procedures.

We conducted this performance audit from October 2017 to December  2018 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

Appendix II: Three Key U.S. Agencies’ Counter-trafficking in Persons Projects, Active in Fiscal Year 2017

The Departments of State (State) and Labor (DOL), and U.S. Agency for  International Development (USAID) managed 120 projects in counter- trafficking in persons carried out by implementing partners during fiscal  year 2017, according to information provided by officials with these  agencies. The three agencies used different approaches to identify  relevant projects. For example, State reported projects with a primary  goal of counter-trafficking in persons, while DOL and USAID included  projects that may not have counter-trafficking in persons as a primary  goal. Table 4 lists these agencies’ reported project information for  projects that were active during fiscal year 2017.

Appendix III: State Documentation for Its Performance Monitoring Activities for 37 Counter-Trafficking in Persons Projects

The Department of State (State) did not fully document its monitoring  activities (monitoring plan; fiscal year 2017 quarterly progress reports;  and final progress report, including indicators and targets, for projects that  ended as of December 2017) for 16 of the 37 selected projects we  reviewed with start dates between fiscal years 2011 to 2016. (See table  5.) For example, State’s Office to Monitor and Combat Trafficking in  Persons did not have monitoring plans for nine projects or targets for  each indicator in six of seven final progress reports for projects that  ended as of December 2017.

Appendix IV: Comments from the Department of State

Appendix V: Comments from the U.S. Agency for International Development

1.  USAID commented that it does not believe that our draft report  reflected the existing controls the USAID mission in Ghana shared  with us, and that the mission had furnished us with a file that,  according to USAID, contained correct information for all indicators  and their results from the time the activity began until our audit. While  the mission provided us with a spreadsheet, this document included  only annual performance totals for several years without  accompanying quarterly totals, or quarterly or annual narrative  information. We focused our analysis on the quarterly and annual  performance reports to understand the extent to which USAID was  ensuring the consistency and completeness of performance  information, including associated narratives, underlying its aggregate  and higher-level performance reports. We reported on inconsistent or  incomplete performance information only after discussing and  substantiating the specific errors we identified with USAID officials.  Further, we recognize USAID’s efforts to address errors that the  agency identified prior to our review and we provide an example of  such efforts in the report.  2.  We have incorporated USAID’s comment. Our report no longer  characterizes USAID’s regular activity monitoring and conversations  with implementing partners as “informal.”  3.  USAID noted that our report does not discuss how the USAID mission  in Ghana uses its third-party monitoring project—Monitoring,  Evaluation and Technical Support Services (METSS)—to work with  local organizations to improve their collection and analysis of data.  We have added a reference to USAID’s third-party monitoring project  to the report where we discussed limited capacity of local partners as  a cause of data reliability issues.  4.  USAID commented that one of the Ghana counter-trafficking in  persons indicators we examined in the integrated project (“value of  new private sector investments in selected value-chains”), was not  related to trafficking in persons and, therefore, was not directly related  to the focus of our audit. As discussed in the Objectives, Scope, and  Methodology section of our report (see app. I), we selected projects,  including the integrated project in Ghana, based on a list of counter- trafficking in persons projects provided by USAID. Because the same  operational policy that sets the monitoring and evaluation standards  for the agency applied to all indicators within a given project, we  examined available quarterly or semi-annual indicator data for all  reported indicators in selected projects to determine the completeness  and consistency of the data. We then conducted interviews with  agency officials to discuss instances in which we identified potentially  incomplete and inconsistent performance information, as well as  whether our findings about the management of performance  information for these selected projects reflected general agency  policies and procedures.

Appendix VI: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Leslie Holen (Assistant Director),  Victoria Lin (Analyst-in-Charge), Esther Toledo, and Andrew Kurtzman  made key contributions to this report. The team benefited from the expert  advice and assistance of Neil Doherty, Justin Fisher, Benjamin Licht,  Grace Lui, and Aldo Salerno.

Human Trafficking: State Has Made Improvements in Its Annual Report  but Does Not Explicitly Explain Certain Tier Rankings or Changes,  GAO-17-56 (Washington, D.C.: December 5, 2016).

Human Trafficking: Oversight of Contractors’ Use of Foreign Workers in  High-Risk Environments Needs to Be Strengthened. GAO-15-102  (Washington, D.C.: November 18, 2014).

Human Trafficking: Monitoring and Evaluation of International Projects  Are Limited, but Experts Suggest Improvements. GAO-07-1034  (Washington, D.C.: July 26, 2007).

Human Trafficking: Better Data, Strategy, and Reporting Needed to  Enhance U.S. Antitrafficking Efforts Abroad. GAO-06-825 (Washington,  D.C.: July 18, 2006).

Human trafficking victims are often held in slave-like conditions and forced to work in areas such as the commercial sex trade, factories, and agriculture.

The Departments of State and Labor, and the U.S. Agency for International Development managed 120 counter-trafficking projects in FY 2017. These projects were carried out by grantees and others, and overseen by the agencies.

We found some weaknesses in State’s and USAID’s monitoring of projects we reviewed, including the use of inconsistent and incomplete information to assess project performance.

We made 5 recommendations to improve monitoring.