Aviation Security: TSA Uses a Variety of Methods to Secure U.S.-bound Air Cargo, but Could Do More to Assess Their Effectiveness

According to TSA, the federal agency responsible for securing the nation's civil aviation system, the introduction of explosive devices in air cargo shipments is a significant threat. To mitigate this threat, TSA is to review the security procedures carried out by all air carriers with U.S.-bound flights and at foreign airports servicing those air carriers. In addition, TSA assesses the commensurability of foreign countries' air cargo security programs.

GAO was asked to evaluate TSA's progress in assessing and mitigating air cargo security risks. This report addresses (1) steps TSA takes to help ensure that U.S-bound air cargo is secure, (2) the status of TSA's efforts to recognize and monitor foreign governments' air cargo security programs, and (3) the extent to which TSA measures the effectiveness of its efforts to secure U.S.-bound air cargo. GAO reviewed TSA policies and procedures, analyzed TSA program data, observed a nongeneralizable sample of 17 air carrier inspections at two foreign airports (selected based on high air cargo volume and other factors), and interviewed TSA, foreign government, and air carrier representatives.

The Transportation Security Administration (TSA) inspects air carriers and assesses foreign airports to help ensure the security of U.S.-bound air cargo.

Air carrier inspections . GAO observed 17 air carrier inspections and found that TSA inspectors consistently followed TSA procedures. Further, GAO's analysis of TSA data found air carriers were in full compliance with cargo security requirements in 84 percent of the nearly 5,000 cargo inspections conducted during fiscal years 2012 through 2017. TSA officials were able to resolve a majority of the violations identified during the inspection process.

Foreign airport assessments . GAO analysis of TSA data found that about 75 percent of the foreign airport assessments that TSA conducted during fiscal years 2012 through 2017 fully complied with international air cargo security standards. As of the end of 2017, foreign officials had addressed about 40 percent of the non-compliance issues. TSA continues to work with foreign officials to address the remaining non-compliance issues.

As of June 2018, TSA had recognized the national cargo security programs (NCSP) of the European Union and 12 other countries as commensurate with TSA's, and TSA uses a variety of mechanisms to monitor NCSP implementation. TSA's process for NCSP recognition, which is voluntary, involves comparing air cargo security requirements to TSA's and conducting visits to the countries to validate their use. Once TSA determines a program is commensurate with TSA's, it monitors NCSP implementation through regular air carrier inspections, foreign airport assessments, and dialog with government officials. TSA may decide not to recognize a country's NCSP but, instead, make recommendations for improving air cargo security. In countries where TSA has not recognized their NCSP, all U.S.-bound cargo is subject to TSA security requirements.

TSA's performance measures do not allow it to specifically determine the effectiveness of its efforts to secure U.S.-bound air cargo. For example, TSA measures whether foreign airports take actions to address all noncompliance issues identified during airport assessments, but such a broad measure could obscure progress made in resolving cargo-specific vulnerabilities. Similarly, TSA officials stated that they are developing a measure to gauge the effectiveness of air carrier inspections, but they do not plan to differentiate efforts to secure air cargo from those for securing passengers. Developing and monitoring outcome-based performance measures that separately account for cargo noncompliance issues and violations could help TSA better determine the extent to which its foreign airport assessments and air carrier inspections improve the security of U.S.-bound air cargo. In addition, TSA measures the number of countries it has recognized in the NCSP Recognition Program, but this metric does not address the effectiveness of the program. Developing and monitoring outcome-based performance measures for the NCSP Recognition Program would help TSA better determine whether the resources invested are yielding the intended results. This is a public version of a sensitive report issued in October 2018. Information that TSA deemed to be sensitive is omitted from this report.

GAO is recommending that TSA develop and monitor outcome-based performance measures to assess the effectiveness of (1) the cargo portion of foreign airport assessments, (2) air carrier cargo inspections, and (3) the NCSP Recognition Program. TSA concurred with the recommendations.

In 2010, al-Qaeda attempted to conceal explosives in printer cartridges  on a U.S.-bound flight from Yemen to Chicago. In July 2017, a terrorist  group shipped partially-assembled components of a bomb from Turkey to  Australia with plans to detonate the assembled device on a passenger  flight. According to the Transportation Security Administration (TSA), the  security threat posed by terrorists introducing explosive devices in air  cargo shipments remains significant.

TSA, a component within the Department of Homeland Security (DHS), is  the federal agency responsible for securing the nation’s civil aviation  system and it has programs in place to help ensure the security of  passengers and property, including cargo, transported on U.S.-bound  flights. TSA’s responsibilities with respect to cargo transported on U.S.- bound flights include establishing security requirements governing U.S.  and foreign-flagged air carrier operations and overseeing implementation  of such requirements, which it does through conducting air carrier  inspections and security assessments of foreign airports, among other  things. Foreign governments also establish national cargo security  programs (NCSP) and may impose their own security requirements on air  cargo operations within their jurisdictions—including screening  requirements that may differ from TSA-established requirements—that  apply to cargo bound for the United States from their airports. Through its  NCSP recognition process, TSA analyzes the air cargo security programs  of its foreign counterparts and determines if a country’s security program  is commensurate with the level of security required under U.S. air cargo  security programs.

You requested that we evaluate TSA’s progress in assessing and  mitigating air cargo security risks. This report (1) describes steps TSA  takes to help ensure that U.S-bound air cargo is secure, (2) describes the  status of TSA’s efforts to recognize and monitor foreign governments’ air  cargo security programs, and (3) analyzes the extent to which TSA  measures the effectiveness of its efforts to secure U.S.-bound air cargo.

This report is a public version of a sensitive report that we issued in  October 2018.TSA deemed some of the information in our October  report to be Sensitive Security Information, which must be protected from  public disclosure. Therefore, this report omits sensitive information about  TSA’s risk methodology, the standards that TSA uses to assess foreign  airports, the specific results of TSA’s air carrier inspections and foreign  airport assessments, and information on the types of NCSP recognition  TSA has granted to other countries. Although the information provided in  this report is more limited, the report addresses the same objectives as  the sensitive report and uses the same methodology.

To describe the steps TSA takes to help ensure that U.S-bound air cargo  is secure, we reviewed relevant laws and regulations, TSA security  policies and procedures, and screening program requirements. We also  reviewed annual air carrier inspection and airport assessment Master  Work Plans for fiscal years 2012 through 2018—which TSA uses to track  its overseas air carrier inspection and foreign airport assessment  schedule—to better understand how TSA schedules inspections and  assessments and the types of inspections it conducts. In addition, we  conducted site visits to two foreign airports—one in South America and  one in Asia—to observe TSA inspectors conducting air carrier cargo  inspections. At one airport, we also observed the cargo portion of an  airport assessment. We selected these locations based on their  designation by TSA as airports of relatively high risk level, as well as high  volume of U.S.-bound air cargo, TSA’s inspection schedule, and  geographic dispersion. We also chose airports in these countries because  it would allow us to observe an inspection in one country where TSA has  recognized the NCSP and one country where TSA has not recognized the  NCSP. Further, we obtained data on and analyzed the results of all air  carrier cargo inspections (close to 5,000) and assessments at airports  that are last points of departure for cargo bound for the United States  (about 570) conducted by TSA inspectors and then entered by them into  TSA’s databases. We analyzed TSA data from fiscal years 2012 through  2017, to cover the period since our previous air cargo security review and  to include the 5 most recent years for which data were available at the  time of our review. We also analyzed fiscal years 2012 through 2017  TSA data on the status of noncompliance issues TSA inspectors  identified during foreign airport assessments. To assess the reliability of  TSA’s air carrier and airport assessment data, we reviewed program  documentation on system controls, interviewed knowledgeable TSA  officials, and analyzed TSA’s data for any potential gaps and errors. We  concluded that TSA’s data on air carrier inspections and foreign airport  assessments were sufficiently reliable to provide a general indication of  the level of compliance for TSA’s air carrier inspections and foreign  airport assessments over the period of our analysis. In addition, we  interviewed TSA headquarters and field officials to discuss TSA’s efforts  to ensure the security of U.S.-bound air cargo prior to being transported  to the United States. We also interviewed other stakeholders, such as  representatives from 11 air carriers—selected based on the relatively high  volume of U.S.-bound cargo they transport; their operation of flights at the  foreign airports we visited; and to obtain a range of coverage regarding  geographical regions of operation, passenger and all-cargo air carriers,  and U.S. and foreign-flagged air carriers—and officials at the European  Commission (EC) and from the civil aviation authority from the country in  Asia that we visited to discuss their experiences in coordinating with TSA  on air cargo security issues. Results from these meetings are not  generalizable, but provided us with information on stakeholders’  experiences and perspectives regarding air cargo security issues.

To describe the status of TSA’s efforts to recognize and monitor foreign  governments’ air cargo security programs, we reviewed TSA’s policies  and procedures for its NCSP Recognition Program. For example, we  reviewed TSA memos from 2012, 2013, and 2016 that documented the  recognition standards and any subsequent revisions to the NCSP  Recognition Program, as well as TSA’s process for monitoring NCSP  recognition requirements. Further, we analyzed letters that TSA provided  since 2012 to governments it determined had commensurate air cargo  security programs and NCSP information TSA officials compiled  specifically for our review to better understand TSA’s terms of recognition  with each government and the timeframes for revalidating NCSP  recognition. We also analyzed data from TSA’s Security Policy and  Industry Engagement Policy Inventory to determine how the level of  NCSP participation has changed over time. Specifically, we reviewed the  number of air carriers participating in the NCSP Recognition Program  from fiscal year 2012—when the NCSP Recognition Program began— through fiscal year 2017, which is the most recent complete fiscal year  available at the time of our review. Finally, we conducted interviews with  TSA and foreign government officials from two countries, and with  representatives of the 11 air carriers described previously to better  understand TSA’s ongoing efforts to recognize and monitor foreign  governments’ air cargo security programs. We also confirmed the status  of countries’ NCSP recognition, as of June 2018, with TSA officials.

To analyze the extent to which TSA measures the effectiveness of its  various efforts to secure U.S.-bound air cargo, we reviewed documents  that contain information on TSA’s air cargo security objectives, goals, and  performance measures, including (1) information in annual budget  documents from fiscal years 2014 through 2019, and (2) TSA’s Global  Strategies directorates’ Operational Implementation Plans from fiscal  years 2014 through 2018—the most recent years available at the time of  our review. These plans include annual objectives and milestones for  U.S.-bound air cargo security programs. We also reviewed the measures  in the annual budget documents and Operational Implementation Plans  and compared them with requirements in TSA’s Global Strategies’ Fiscal  Year 2016 Strategy and Fiscal Year 2018 Strategy Program and  applicable laws governing performance reporting in the federal  government, including the Government Performance and Results Act of  1993 (GPRA), as updated and expanded by the GPRA Modernization Act  of 2010 (GPRAMA). For example, we assessed whether the  performance measures provide information on the effectiveness of TSA’s  various air cargo security efforts. Further, we assessed TSA’s  performance measures against DHS and TSA risk management  principles. We obtained additional information on how TSA measures the  performance of its air cargo security efforts during our interviews with  TSA headquarters officials. See appendix I for more information on our  objectives, scope, and methodology.

The performance audit upon which this report is based was conducted  from July 2017 to October 2018 in accordance with generally accepted  government auditing standards. Those standards require that we plan  and perform the audit to obtain sufficient, appropriate evidence to provide  a reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives. We  subsequently worked with TSA from September 2018 to November 2018  to prepare this public version of the original sensitive report for public  release. This public version was also prepared in accordance with these  standards.

U.S.-bound Air Cargo and the Air Cargo Supply Chain

In fiscal year 2017, about 13 billion pounds of cargo was transported on  aircraft to the United States—over 5 billion pounds was transported on  passenger aircraft (e.g., Delta and United Airlines), and about 8 billion  pounds was transported on all-cargo aircraft (e.g., FedEx and United  Parcel Service)—from over 300 foreign airports, according to our analysis  of Bureau of Transportation Statistics data. U.S.-bound air cargo can  vary widely in size and include such disparate items as electronic  equipment, automobile parts, clothing, medical supplies, fresh produce,  and cut flowers.

The international air cargo shipping process involves a complex network  of business entities that include individual shippers, manufacturers,  transportation companies, freight forwarders, warehouses and air  carriers. Entities within the supply chain may provide all services  (warehousing, consolidation, and loading of air cargo, for example) or  only certain services. The standards set by the International Civil Aviation  Organization (ICAO) focus on four primary types of entities: known and  unknown consignors (i.e., individual shippers, manufacturers, other  shipping entities), regulated agents (i.e., freight forwarders, handling  agents), and commercial air carriers. Various other air cargo supply  chain entities also have responsibilities for applying specific types of  security controls in accordance with the international standards. Figure 1  shows an illustrative example of the flow of U.S.-bound air cargo and  where in the supply chain the cargo can be secured.

TSA and Air Carrier Responsibilities for Ensuring the Security of U.S.-Bound Air Cargo

The Aviation and Transportation Security Act (ATSA), enacted into law  shortly after the September 11, 2001 terrorist attacks, established TSA  and gave it responsibility for securing all modes of transportation,  including the nation’s civil aviation system, which includes U.S. and  foreign-flagged air carrier operations to, from, within, or overflying the  United States, as well as the foreign point-to-point operations of U.S.- flagged carriers. Among other things, ATSA requires, in general, that  TSA provide for the screening of all passengers and property, including  cargo transported by air carriers. ATSA further requires that a system  be in operation to screen, inspect, or otherwise ensure the security of the  cargo transported by all-cargo aircraft to, from, and within the United  States, but did not establish a firm deadline for the implementation of  such a system. Further, to help enhance civil aviation security, the  Implementing Recommendations of the 9/11 Commission Act of 2007  (9/11 Commission Act), mandated that DHS establish a system within 3  years of enactment (enacted August 3, 2007) to screen 100 percent of air  cargo transported on all passenger aircraft operated by an air carrier  traveling to, from, within, or overflying the United States. TSA reported  that it met the mandate to screen 100 percent of domestic air cargo  transported on passenger aircraft in August 2010 and U.S.-bound air  cargo transported on passenger aircraft from foreign airports in August  2013.

There is no comparable 100 percent screening requirement in statute for  cargo transported to the United States on all-cargo air carriers. However,  TSA requires that all cargo transported on U.S.-bound flights be screened  or subjected to security controls that prevent the introduction of  explosives, incendiaries, or other destructive devices. If the cargo comes  from known consignors or regulated agents, TSA’s all-cargo security  program does not require any additional screening unless the cargo piece  exceeds a certain weight. On the other hand, all-cargo air carriers must  screen all cargo that they accept from unknown consignors or  nonregulated agents.

Air carriers are responsible for implementing TSA security requirements  predominantly through TSA-approved security programs that describe the  security policies, procedures, and systems the air carriers are to  implement and maintain in order to comply with TSA security  requirements. These requirements include measures related to the  acceptance, handling, and screening of cargo; training of employees in  security and cargo screening procedures; testing employee proficiency in  cargo screening; and access to cargo areas and aircraft. If threat  information or events indicate that additional security measures are  needed to better secure the aviation sector, TSA may issue revised or  new security requirements in the form of security directives or emergency  amendments when more immediate action on behalf of air carriers is  necessary. Air carriers must implement the requirements set forth in  applicable security directives or emergency amendments (unless  otherwise approved by TSA to implement alternative security measures)  in addition to requirements already imposed and enforced by TSA in  order to remain compliant with their respective security programs.

Under TSA regulations, air carriers are responsible for ensuring the  security of the air cargo they transport, and TSA requirements specify  methods and technologies that may be used to secure U.S-bound air  cargo through screening procedures. Specific screening methods outlined  in the 9/11 Commission Act, for example, include X-ray systems,  explosives detection systems (EDS), explosives trace detection  (ETD), explosives detection canine teams certified by TSA, and physical  search together with manifest verification. The 9/11 Commission Act,  however, requires that screening involve a physical examination or non- intrusive method of assessing whether cargo poses a threat to  transportation security and not solely performing a review of information  about cargo contents or verifying the identity of the cargo’s shipper, when  not performed in conjunction with the screening methods outlined above.

Air Carrier Inspections and Foreign Airport Assessments

To assess whether air carriers properly implement security regulations,  TSA conducts regulatory compliance inspections of U.S. and foreign- flagged air carriers at all foreign airports with U.S.-bound flights. During  these inspections, a TSA inspection team is to examine air carriers’  implementation of applicable security requirements, including their TSA- approved security programs, any amendments or alternative procedures  to these security programs, and applicable security directives or  emergency amendments. In general, following a risk-informed approach,  TSA attempts to inspect all air carriers with TSA-approved security  programs at each foreign airport where they operate flights to the United  States either annually or semiannually depending on the risk level of the  airport. Compliance inspections can include reviews of documentation,  such as screening logs; interviews of air carrier personnel; and direct  observations of air cargo operations.

Consistent with the ATSA, and in accordance with existing statutory  requirements, TSA also assesses the effectiveness of security measures  at foreign airports using select ICAO security standards and  recommended practices. These standards and recommended practices  include ensuring that passengers and cargo are properly screened and  that unauthorized individuals do not have access to restricted areas of the  airport. TSA uses a risk-informed approach to schedule foreign airport  assessments, generally every 1 to 3 years, with high risk airports  assessed more frequently than medium and low risk airports. Although  TSA is authorized under U.S. law to conduct foreign airport assessments  at intervals it considers necessary, it may not perform an assessment of  security measures at a foreign airport without permission from the host  government. TSA also does not have authority to impose or otherwise  enforce security requirements at foreign airports. Instead TSA must work  with host government civil aviation officials to schedule airport visits to  conduct airport assessments (as well as air carrier inspections) and  improve upon existing conditions when deficiencies are identified. Table  1 highlights the roles and responsibilities of certain TSA positions within  Global Strategies that are responsible for implementing the air carrier  inspection and foreign airport assessment programs.

In addition to conducting air carrier inspections and foreign airport  assessments, TSA has also developed the NCSP Recognition Program,  for which TSA compares and assesses foreign air cargo security  programs and standards to determine if those programs provide a level of  security that is commensurate with TSA’s air cargo security standards.  The NCSP recognition process involves comparing foreign countries’ air  cargo security program requirements to TSA air cargo security  requirements and conducting visits to the foreign countries to observe the  security programs in operation and determine if they can be validated as  commensurate with TSA’s. The recognition decision is based on whether  the other country’s NCSP is commensurate in six pillars of cargo supply  chain security that TSA has identified, which are:

Facility Security. Procedures and mechanisms to prevent  unauthorized entry to facilities where cargo is screened, prepared,  and stored.

Chain of Custody/Transit Procedures. Methods or procedures to  prevent and deter unauthorized access to cargo while stored or in  transit between facilities prior to loading onboard aircraft.

Screening. Screening of cargo through the application of technical or  other means that are intended to identify weapons or explosives.

Personnel Security. Processes to vet individuals with unescorted  access to air cargo at any point in the air cargo supply chain.

Training. Training of personnel who screen, handle screened cargo,  or perform other duties related to air cargo screening, preparation, or  storage.

Compliance and Oversight Activities. Clearly established  requirements that regulated entities must satisfy in order to participate  in the security program, and routine audits of such entities for  compliance by appropriate authorities.

TSA first approved the NCSP recognition process for passenger aircraft  operations in fiscal year 2011 and made subsequent changes to the  process in fiscal year 2013. According to TSA, the NCSP Recognition  Program increases its visibility into recognized governments’ air cargo  security requirements and air cargo supply chains, facilitates the  identification of air cargo industry vulnerabilities, and is a key component  of TSA’s efforts to achieve 100 percent screening of U.S.-bound air cargo  and enhance global supply chain security. Within Global Strategies, the  Mitigation Plans and Programs Directorate is responsible for the NCSP  Recognition Program.

Air Cargo Advance Screening (ACAS)  The Department of Homeland Security’s  (DHS) U.S. Customs and Border Protection  (CBP) and the Transportation Security  Administration (TSA) initiated the ACAS pilot  in December 2010 to more readily identify  high risk cargo for additional screening prior to  all-cargo and passenger aircraft departing  from foreign airports to the United States.  Unlike TSA, which focuses on aviation  security, to include the security of air cargo  prior to loading on aircraft at last point of  departure airports, CBP focuses on identifying  persons and cargo that  may violate U.S. law  and are, therefore, prohibited from entry into  the United States. The aim of the pilot was to  determine whether it was feasible for air  carriers to submit air cargo manifest data to  CBP prior to departure from all foreign last  point of departure airports. This would allow  CBP to analyze, target, and, if needed, for  DHS to issue instructions to air carriers to  provide additional cargo information or take  additional security measures before such  cargo is loaded onto U.S.-bound aircraft. DHS  determined that the pilot was successful.

In 2012, we reported on the actions TSA took to enhance the security of  U.S.-bound air cargo after the October 2010 discovery of explosive  devices in packages on all-cargo aircraft bound for the United States from  Yemen. We recommended, among other things, that DHS assess the  costs and benefits of requiring all-cargo carriers to report U.S.-bound air  cargo screening data. DHS agreed with our recommendation and TSA  reported that, although all-cargo air carriers submit data to TSA as part of  the Air Cargo Advance Screening (ACAS) pilot, the all-cargo air carriers  do not need to report on the number of shipments screened for  explosives. Nevertheless, TSA reported that it will be able to utilize ACAS  data to determine the percentage of shipments transported to the United  States on all-cargo aircraft that carriers must screen for explosives.

TSA Conducts Overseas Inspections and Assessments to Help Ensure Screening of U.S.- bound Air Cargo and Compliance with Security Requirements

To help ensure compliance with cargo security requirements and  international standards, TSA inspects air carriers and assesses certain  known consignors and regulated agents. TSA also inspects cargo  security procedures during foreign airport assessments. Further, DHS has  also implemented requirements to obtain advance information on air  cargo shipments through ACAS that it uses to perform targeted risk  assessments.

TSA Inspects Air Carriers and Assesses Other Supply Chain Entities to Help Ensure Compliance with Cargo Security Requirements

TSA inspects air carriers and assesses certain known consignors and  regulated agents to help ensure compliance with cargo security  requirements. However, certain factors can limit TSA’s ability to conduct  inspections or observe various security measures, including cargo  screening.

TSA uses a multistep process to plan, conduct, and record air carrier  cargo inspections. To plan inspections, TSA develops an annual Master  Work Plan that regional operations centers use to schedule air carrier  inspections each fiscal year. Based on our review of TSA work plans for  fiscal years 2012 through 2018 and discussions with TSA officials at all  six regional operations centers, TSA separately plans for passenger  inspections and cargo inspections of both all-cargo air carriers as well as  passenger air carriers that transport cargo bound for the United States  from foreign airports.

To conduct air cargo inspections, TSA inspectors are to use  standardized, cargo-specific job aids that assess air carriers against  security program requirements in all six pillars of supply chain security.  According to TSA officials, they update the cargo inspection job aids, as  needed, to ensure they reflect changes to TSA requirements and the  current threat environment. For example, the cargo inspection job aids  prompt TSA inspectors to inquire about the transportation of cargo from  certain high risk countries. TSA inspectors we spoke with at all six  regional operations centers stated that they use the cargo inspection job  aids, and inspectors we spoke with at five regional operations centers  stated that they are helpful.

We observed 17 air carrier cargo inspections at airports in two different  countries and found that TSA inspectors consistently used the cargo  inspection job aids to assess the air carriers against TSA requirements.  These inspectors observed air carriers’ implementation of security  measures (such as cargo screening), interviewed security officials, and  reviewed air carrier records (including cargo screening and training logs).  Officials at all six regional operations centers and the air carriers we met  with confirmed these methods are routine practices. Further, officials  representing 10 of the 11 air carriers we met with confirmed that TSA  regularly inspects their cargo operations at foreign airports to ensure  compliance with screening and other security requirements.

After completing an air carrier inspection, TSA inspectors are to enter air  carrier cargo inspection results into PARIS. TSA supervisors and  managers are to review the inspection reports for quality and track their  completion. TSA officials we interviewed at TSA headquarters and all six  regional operations centers confirmed the quality review process is in  place and that they use it. In addition, TSA headquarters cargo experts  are to review a sample of air carrier cargo inspections.

Based on our analysis of PARIS data, TSA conducted close to 5,000 air  carrier cargo inspections (including both passenger air carriers and all- cargo air carriers) from fiscal year 2012 through fiscal year 2017 and  found air carriers in full compliance with applicable security requirements  in 84 percent of these inspections. TSA reported at least one instance of  noncompliance, or violation, for the remaining 16 percent of cargo  inspections. Based on the TSA data, the percentage of inspections with  violations has generally trended downward during this time period. TSA  officials attributed this downward trend to a number of factors including:  (1) TSA’s emphasis on assisting air carriers (through its international  industry representative) in implementing new air cargo security  requirements after the 2010 printer ink cartridge plot; (2) increases in the  number of TSA inspectors to ensure compliance; (3) TSA’s outreach to  foreign governments for improved cargo security under the NCSP  Recognition Program; and (4) TSA efforts to engage with air carriers,  including regional industry summits that included a cargo security focus.

According to TSA officials, if a TSA inspector finds that an air carrier is  not in compliance with any applicable security requirements, additional  steps are to be taken to correct and record those specific violations,  which can include providing on-the-spot counseling for minor violations or  opening an investigation if the violation is potentially more serious. Upon  conclusion of the investigation, TSA is to make a determination whether  to issue a warning notice, letter of correction, or notice of proposed civil  penalty. For example, based on TSA data, we determined that TSA  inspectors provided counseling (specific guidance) in certain instances  when they found that an air carrier had failed to obtain multiple views of  cargo screened using an X-ray machine. According to the TSA data, the  air carrier took immediate corrective actions and implemented the correct  procedures on-the-spot. From the data provided by TSA, we also  identified potentially more serious violations. Examples of such violations  included instances in which TSA inspectors initiated an investigation  when they found that an air carrier was not screening 100 percent of the  cargo as required under its approved security program.

According to TSA officials, TSA relies on a system of progressive  enforcement and carefully considers whether a civil penalty is warranted  based, in part, on the history of an air carrier’s inspections. TSA officials  added that they may consider options other than civil penalties, since  their objective is to encourage compliance through capacity-building  efforts with air carriers, not to generate revenue. For example, TSA will  sometimes settle a civil penalty by allowing the air carrier responsible for  the violation to invest the agreed upon penalty into improved security  measures or screening processes.

According to TSA data, TSA inspectors identified 1,128 air carrier cargo  security violations during fiscal years 2012 through 2017 for the 16  percent (781) of air carrier inspections where they found at least one  violation. For these violations, TSA took the following actions:

TSA inspectors resolved 580 of the violations (approximately half)  through counseling and referred the remaining 548 violations for  investigation since they were each potentially serious enough to  warrant an enforcement action.

TSA conducted investigations covering the 548 potentially more  serious violations, which resulted in about 220 administrative actions,  nearly 50 civil penalties, and over 30 instances where no action was  taken. According to TSA, TSA inspectors recommended total civil  penalties of approximately $23.5 million, $22.2 million of which  consisted of penalties proposed for one air carrier.

TSA Assesses Known Consignors and Regulated Agents in Recognized Countries

During air carrier inspection visits, the TSA inspection team may also  conduct assessments of known consignors and regulated agents in  countries with recognized NCSPs. According to TSA data, TSA  conducted assessments of 38 known consignors and regulated agents in  fiscal year 2017. While conducting a site visit to a foreign airport in an  NCSP country in March 2018, we observed TSA inspectors conduct  assessments of two regulated agents and the inspectors covered all of  the required questions. The assessments were primarily interviews along  with some observations that included warehouse security and limited  cargo screening. Record reviews were not part of the assessment  because that is the purview of the foreign government’s civil aviation  authority, according to the TSA inspectors. Foreign government civil  aviation authority officials attended the assessments of the two regulated  agents to observe and take notes of the visit and discussions.

According to the TSA inspectors who conducted the assessments in the  NCSP country we visited, meeting with regulated agents is invaluable  because regulated agents, not air carriers or their authorized  representatives, conduct almost all air cargo screening in that country.  The inspectors added that having the opportunity to meet with regulated  agents during foreign site visits provides them with insights regarding the  extent to which screening of U.S.-bound cargo is being conducted at  foreign last point of departure airports. In countries without a recognized  NCSP, air carriers are required under their TSA-approved security  programs to screen all cargo at the airport.

Certain Factors Can Limit TSA’s Ability to Conduct Inspections or Observe Cargo Screening

TSA inspectors are not always able to observe certain security measures  during air carrier cargo inspections or airport assessments because of  foreign government sovereignty and air cargo logistics. For example,  regional operations center officials told us that they are not always able to  observe cargo screening because of restrictions placed on them by  foreign governments, such as the number of days they are given to  complete an inspection or assessment, the hours they are allowed to  work, or the size of the TSA inspection team. TSA officials also stated  that the transportation of air cargo occurs at all hours of the day and  night, and TSA inspectors must sometimes choose which security  measures to observe. For example, the TSA officials stated that  screening may occur many hours prior to the loading of that cargo on an  aircraft. At both foreign airports we visited, we observed TSA inspectors  working late night or early morning hours to observe air carriers’ cargo  operations.

Out of the 17 air carrier cargo inspections we observed at the two foreign  airports we visited, TSA inspectors were not able to observe cargo  acceptance procedures for 11 air carriers and cargo screening for 9 air  carriers because these carriers did not receive or screen cargo during the  time of the inspections or the inspectors were busy conducting other  inspections. Because regulated agents screen the vast majority of the  cargo before transporting it to the airport in the NCSP country we visited,  TSA did not observe cargo screening in eight of the nine air carrier cargo  inspections they conducted at that airport. For inspections where TSA  inspectors cannot observe security measures, we observed (and TSA  inspectors confirmed) that they rely on interviews with officials  responsible for cargo security and screening and document reviews (such  a reviewing cargo screening logs) to determine whether air carriers are  complying with TSA air cargo security requirements. At the request of  TSA, air carriers must provide evidence of compliance with applicable  security requirements and its security programs, including copies of  records.

TSA inspectors also do not inspect air carriers at all foreign airports from  which air carriers transport U.S.-bound cargo. As we reported in May  2018, challenges prevent TSA from completing 100 percent of required  air carrier inspections in Cuba at the frequency established in its standard  operating procedures, including external factors, such as foreign  government requests to reschedule TSA inspections, and limitations in  the data TSA uses to schedule inspections. Further, TSA officials stated  that most all-cargo carriers do not have scheduled flights. Instead, they  wait until they have sufficient cargo to ship and then complete their  routes, which can make it difficult for TSA to schedule inspections— planned 3 months in advance—during times that the carrier will be flying  cargo to the United States. According to the vice president of security at  one all-cargo carrier, TSA does not always inspect all last point of  departure routes used by the airline.

TSA is taking steps to better understand air carriers’ schedules. For  example, in response to our 2018 review addressing TSA’s efforts to  ensure the security of air carrier operations between the United States  and Cuba, TSA reported that it began developing a tool in August 2017  that is designed to analyze aggregate flight data and validate or identify  last point of departure service to the United States from international  locations.

TSA Inspectors Assess Foreign Airports from which U.S-bound Cargo is Shipped to Help Ensure Proper Cargo Security Procedures Are in Place

In addition to conducting air carrier cargo inspections, TSA inspection  teams conduct assessments of foreign airports that provide passenger  and/or cargo service to the United States to determine if these airports  are maintaining and carrying out effective security measures. TSA  inspectors generally use the same process to plan, conduct, and record  airport assessments as air carrier inspections, according to TSA  headquarters and regional operations centers officials. Specifically, TSA  inspection teams assess the foreign airports using 44 ICAO standards  and recommended practices, including nine standards or practices that  are specific to the transport of cargo and mail. These standards include  measures for the acceptance, screening, and protection of air cargo. At  the end of each foreign airport assessment, TSA inspectors are to  prepare a report detailing findings on the airport’s overall security posture  and security measures that may also contain recommendations for  corrective actions.

We observed TSA inspectors conducting the cargo portion of an airport  assessment at one airport we visited and confirmed their use of this  process. Inspectors used the results of the air carrier cargo inspections  conducted earlier in the site visit to inform the cargo portion of the airport  assessment and complete the associated job aid. The TSA inspectors  obtained additional information specific to the assessment during an  interview with airport officials and an international mail facility in the  country we visited. The inspectors stated that they corroborated the  information obtained during interviews with documentation provided by  airport officials and the foreign government in advance of the visit.

TSA conducted about 570 assessments of foreign airports with U.S- bound cargo shipments from fiscal year 2012 through fiscal year 2017, and TSA inspectors determined that the airports were fully compliant with  the cargo-related ICAO standards and recommended practices in about  430 of these assessments (75 percent), according to our analysis of TSA  data. However, TSA inspectors found at least one instance of cargo  noncompliance in about 140 airport assessments (25 percent). Based  on TSA data, the percentage of airport assessments in which TSA  inspectors identified cargo noncompliance issues has generally trended  upward during fiscal years 2012 through 2017. TSA officials attributed this  upward trend to the introduction of a new ICAO standard in 2014 for  ensuring that all cargo shipments designated as higher-risk undergo  enhanced screening.

TSA assigns a vulnerability score to each ICAO standard and  recommended practice assessed using a rating system, ranging from a  category “1,” which represents full compliance with ICAO standards and  recommended practices, to a “5,” which involves the most serious or  egregious issues. For example, in a fiscal year 2017 foreign airport  assessment, TSA inspectors recorded an instance of noncompliance of  ICAO standard 4.6.3 (that requires protection of cargo from the point of  screening until departure of the aircraft) as a “3” when they identified  holes in a facility perimeter barrier allowing direct access to secured  cargo. Further, during a 2014 airport assessment, TSA inspectors  assessed an instance of noncompliance of the same standard as a “5”  when they observed two unescorted individuals in a security restricted  area without airport identification. Based on the results of TSA’s foreign  airport assessments conducted during fiscal years 2012 through 2017,  TSA inspectors assessed most noncompliance issues identified as a “2”  or “3.”

As of December 2017, TSA officials reported that certain foreign airports  took corrective actions to address noncompliance issues. As a result,  TSA closed out approximately 40 percent of the fiscal year 2012 through  2017 deficiencies identified in its assessments. According to our analysis  of TSA data, for the remaining 60 percent of noncompliance issues, the  airports have not yet taken sufficient action to fully address TSA’s  concerns, or TSA inspectors have not yet verified whether the actions  foreign airports reported that they have taken are sufficient for addressing  the noncompliance issues. The majority of unaddressed noncompliance  issues pertain to issues identified in fiscal year 2016 or 2017  assessments.

In our 2017 review of TSA’s foreign airport assessments, we reported that  TSA assists foreign airports in addressing identified noncompliance  issues (security deficiencies) in various ways, but noted that TSA could  enhance data management. As part of assisting foreign airports, TSA  inspectors educate foreign airport officials on how to mitigate identified  airport security deficiencies. Specifically, TSA provides on-the-spot  counseling, training, technical assistance, security consultations, and  security equipment. In addition, TSA representatives—the primary  liaisons between the U.S. government and foreign governments on  transportation security issues—are responsible for monitoring the  progress made by foreign officials in addressing security deficiencies  identified during TSA airport assessments. Our 2017 review found,  however, that TSA representatives did not always update key information  in TSA’s database for tracking the resolution status of security  deficiencies, including the security deficiencies’ root causes and  corrective actions. To help strengthen TSA’s analysis and decision  making, we recommended that TSA fully capture and specifically  categorize data on the root causes of security deficiencies and the status  of corrective actions to be taken. TSA concurred with our  recommendations and is taking steps to address them, as discussed  below. In addition to working with foreign airports to address deficiencies,  TSA sometimes requires air carriers to adopt security procedures through  security directives or emergency amendments to compensate for serious  vulnerabilities that TSA identified during the foreign airport assessment.  For example, at one airport in Africa, passenger air carriers must hold all  cargo for 24 hours prior to transport.

In response to our 2017 recommendations, TSA officials told us that they  are in the process of developing a vulnerability resolution tool to capture  the vulnerabilities associated with a specific location, such as a foreign  country or airport. According to TSA officials, the tool will be used to  identify and categorize root causes of vulnerabilities identified during air  carrier inspections and foreign airport assessments, as well as  incorporate other country specific information. TSA officials added that,  once completed, TSA hopes to be able to use the tool to develop  vulnerability mitigation options to, among other things, address security  vulnerabilities identified during air carrier inspections and foreign airport  assessments. For example, if TSA inspectors identify a cargo screening  vulnerability during an air carrier inspection or airport assessment, they  may determine that the root cause is a lack of national-level training  courses. In an example such as this, although TSA does not have the  authority to require a foreign government to take corrective actions, TSA  officials may develop a training curriculum that foreign governments could  deploy, if they choose, to address the identified vulnerability. According to  TSA officials, TSA inspectors and TSA representatives would  subsequently determine whether the training resolved the vulnerability  and, if necessary, consider what additional measures may be appropriate.  TSA expects to have the tool in place and staff trained to use it by the  beginning of fiscal year 2019.

DHS Has Taken Steps to Obtain Advance Air Cargo Information to Perform Targeted Risk Assessments of U.S.- Bound Flights

DHS has taken steps to require advance information on air cargo  shipments in order to conduct targeted risk assessments and help ensure  the cargo is secure before air carriers transport it to the United States. As  previously discussed, in December 2010, U.S. Customs and Border  Protection (CBP) began collecting cargo data from certain air carriers  before they loaded U.S.-bound cargo as part of the voluntary ACAS pilot  program. In response to a terrorist plot in July 2017, TSA issued  security directives and emergency amendments in September 2017  requiring air carriers transporting cargo to the United States from last  point of departure airports in Turkey to submit advance cargo data to  CBP. Further, in January 2018, TSA imposed similar requirements for  foreign air carriers operating out of certain high risk countries in the  Middle East. DHS subsequently published the ACAS interim final rule,  which requires all air carriers to submit advance air cargo information as  of June 12, 2018.

TSA and CBP identify high risk cargo based on, among other things, the  advance information air carriers submit and may require them to take  additional actions before loading the cargo onto U.S.-bound flights.  Before implementation of the ACAS interim final rule, air carriers not  participating in the ACAS pilot were required to submit manifest data to  CBP no later than 4 hours before the flight’s arrival in the United States,  or no later than the time of departure from locations in North America, the  Caribbean, Central America, and parts of South America north of the  Equator. However, under ACAS, a subset of the manifest data must be  provided prior to loading the cargo onto U.S.-bound aircraft. After  reviewing the data, DHS can mandate that an air carrier (1) provide  additional information on a particular cargo shipment, (2) perform  enhanced screening before loading the cargo, or (3) not transport the  cargo to the United States.

TSA officials are beginning to track whether air carriers have conducted  the required ACAS screening as a part of their international compliance  activities. TSA officials stated that inspectors review air carrier screening  and manifest logs during air carrier cargo inspections at foreign airports to  verify compliance with ACAS. In addition, TSA plans to fully develop the  process of assessing air carrier compliance with ACAS requirements,  according to TSA officials.

TSA Has Recognized the Air Cargo Security Programs of the European Union and 12 Other Countries and Monitors Their Implementation

TSA Has Increased the Number of Countries Recognized, as well as the Scope of Its Recognition Program

As of June 2018, TSA has recognized the passenger air cargo security  programs of the European Union, which covers the 28 European Union  member states, and 12 other countries. NCSP recognition is a voluntary  agreement between TSA and a foreign government. TSA’s NCSP  recognition process involves three phases: (1) a technical review and  analysis of a foreign country’s air cargo security program’s requirements  with TSA requirements to determine if the programs align on basic  principles; (2) validation visits to the foreign country to determine if the air  cargo security program aligns with TSA practices; and (3) a decision on  whether to recognize the foreign government’s air cargo security program  as commensurate with TSA’s air cargo security requirements. The  recognition decision is based on whether the foreign government’s NCSP  is commensurate with TSA requirements across TSA’s six pillars of cargo  supply chain security, and the potential outcomes are as follows:

Recognition with no caveats. TSA may determine that the foreign  government’s NCSP is fully commensurate with all of TSA’s air cargo  security requirements across all six supply chain security pillars or  TSA may find there are slight variations in air cargo security  requirements that nonetheless provide a commensurate level of  security and give the country’s NCSP recognition with no caveats. As  of June 2018, TSA had recognized the NCSPs of Canada, Israel, and  Norway without any caveats.

Recognition with caveats. TSA may decide to recognize a  government’s NCSP, but with certain caveats based on specific  variations within a country’s national requirements. According to TSA  officials, in this instance, TSA requires air carriers in that country to  continue to implement specific TSA requirements on U.S.-bound air  cargo to account for the variation. As of June 2018, TSA had issued  at least one caveat with nine NCSP recognized countries and the  European Union. For example, in these nine recognized countries  and the European Union, TSA requires air carriers to rescreen cargo  originating from specific third party countries according to TSA  standards before transporting it to the United States.

No recognition, but provides recommendations. TSA may  determine that a foreign government’s NCSP is not commensurate  with TSA requirements in many areas and make recommendations to  that government on how to improve its air cargo security program to  better align with TSA and global air cargo security requirements. For  example, after reviewing one country’s air cargo security program  requirements, TSA determined that its NCSP was not commensurate  and provided written recommendations on ways to improve its NCSP,  as discussed below. According to TSA officials, under such  circumstances they will continue to engage with the foreign  government. If the foreign government implements the  recommendations, TSA may reconsider the foreign government for  NCSP recognition. Notably, TSA recognized another country’s air  cargo security program only after its civil aviation authority  implemented TSA’s recommendations to improve certain procedures,  including screening of staff with access to air cargo. Where NCSP  recognition is not applicable, air carriers transporting air cargo into the  United States from last point of departure airports must continue to  apply their TSA-approved security program requirements pertaining to  cargo.

TSA originally developed the NCSP Recognition Program for passenger  air cargo security programs in fiscal year 2011, and TSA expanded the  scope of the program in fiscal year 2013 to include all-cargo operations.  As a result of this expansion, foreign governments may choose to engage  with TSA on NCSP recognition for passenger operations, all-cargo  operations, or both. According to TSA’s NCSP memo authorizing the  change, by including all-cargo operations in its evaluation of other  countries’ NCSPs, TSA can gain a greater understanding of the  international air cargo supply chain. As of June 2018, TSA had  recognized the all-cargo operations of the European Union and six other  countries. Figure 2 provides information about the foreign government  NCSPs that TSA had recognized as of June 2018.

According to TSA data, air carrier participation in the NCSP Recognition  Program has increased in recent years. Specifically, as of June 2018, 130  air carriers participate in the NCSP Recognition Program—an increase  from about 50 in fiscal year 2015 when TSA last recognized a foreign  government’s NCSP. After TSA has recognized a foreign government’s  NCSP, air carriers can request amendments to their TSA-approved  security programs to allow them to follow a recognized country’s air cargo  security program instead of having to follow both the recognized country’s  security program and separate requirements in their TSA-approved  security programs. Representatives from all 11 air carriers we met with  stated that they have submitted requests to TSA to amend their security  programs in order to implement the foreign government’s NCSP instead  of TSA requirements when operating in those countries that have NCSP  recognition. According to representatives from all 11 air carriers and TSA  officials we met with, air carriers benefit from NCSP recognition.  Specifically, they and the stakeholders in their supply chains can learn  and use the host country’s set of air cargo security requirements (and  without a need to know and implement TSA requirements for cargo  transported on U.S.-bound flights from that country).

TSA officials stated that, as of June 2018, apart from the European Union  and the 12 other countries that have NCSP programs, no additional  foreign governments are close to achieving NCSP recognition. However,  TSA NCSP Recognition Program officials continue to coordinate with  foreign governments on air cargo security issues when requested and as  TSA resources allow. According to information provided by TSA, as of  June 2018, TSA had coordinated with 21 additional foreign governments  interested in NCSP recognition that are not yet recognized. In non- recognized countries, air carriers transporting U.S.-bound air cargo must  follow the measures required by the foreign governments in addition to  their TSA-approved security programs.

TSA Uses a Variety of Mechanisms to Monitor and Revalidate Recognized Governments’ NCSP Implementation

Once TSA determines a foreign government’s NCSP is commensurate  with TSA requirements, it monitors NCSP implementation through air  carrier cargo inspections, foreign airport assessments, ongoing  engagements with foreign government officials, and revalidation of NCSP  recognition (see fig. 3). Each of these monitoring mechanisms is  discussed in greater detail below.

According to TSA officials, results from air carrier inspections and foreign  airport assessments provide TSA valuable information in determining  whether to revalidate a foreign government’s NCSP recognition because  TSA inspectors are able to verify a recognized government’s NCSP  implementation in person. We analyzed TSA data from fiscal years  2015 through 2017 and confirmed that TSA conducted air carrier cargo  inspections and assessments of foreign airports with U.S-bound cargo  shipments that covered all recognized NCSPs. Representatives from 10  of the 11 air carriers we met with and the two foreign governments we  met with confirmed that TSA conducts air carrier inspections in  recognized countries.

According to our analysis of TSA data for fiscal years 2015 through 2017,  TSA inspectors identified more air carrier violations and lower rates of  compliance with cargo-related standards and recommended practices at  foreign airports located in non-NCSP countries than in NCSP countries.  In addition to identifying lower rates of compliance in non-NCSP  countries, TSA officials also determined that the noncompliance issues in  non-NCSP countries were more serious than noncompliance issues in  NCSP countries, according to our data analysis.

According to TSA officials, TSA inspectors identified fewer violations  during air carrier cargo inspections in NCSP countries because air  carriers only need to implement one air cargo security program (the host  government’s) and, therefore, were less likely to make errors.  Additionally, TSA inspectors identified fewer noncompliance issues in  NCSP countries because TSA officials meet with foreign officials in  recognized countries on a regular basis, and this helps to improve  compliance. Representatives from 10 air carriers we met with confirmed  that they are less likely to violate air cargo security requirements in NCSP  countries because (1) the foreign government conducts regular  compliance inspections (a component of the oversight and compliance  security pillar TSA requires foreign governments implement to obtain  NCSP recognition), or (2) screeners are less likely to make errors  screening cargo because they only need to implement the foreign  government’s NCSP, which reduces confusion. For example, one air  carrier representative told us that cargo screeners do not need to  determine which security measures (TSA’s or the host government’s) to  implement for a particular flight.

Annual Meetings and TSA Representative Engagement with Foreign Government Officials

TSA and foreign government officials also discuss changes in a foreign  government’s NCSP on a regular basis, according to our review of TSA’s  documents and interviews with TSA and foreign government officials. For  example, TSA’s memos authorizing the NCSP Recognition Program and  11 of 12 letters of recognition provided to foreign governments express an  intent for TSA to hold in-person, annual meetings with officials in  countries with a recognized NCSP program to discuss issues related to  NCSP recognition. TSA officials generally held or planned to hold such  meetings in fiscal years 2017 and 2018, according to our review of TSA’s  NCSP Recognition Program fiscal year 2018 work plan. In addition, TSA  officials stationed at U.S. embassies are to meet with their foreign  government counterparts on a regular basis, according to TSA officials  and the two recognized governments with whom we met. For example,  the TSA representative who coordinates with the European Commission  in Brussels, Belgium, told us that he meets with European Commission  officials multiple times each month. He stated that these conversations  can cover regulatory and legislative changes pertaining to air cargo  security with European Commission officials and he informs TSA  headquarters and the Frankfurt Regional Operations Center of changes  that could affect NCSP recognition in Europe. TSA headquarters and  European Commission officials confirmed that these meetings occur.

TSA revalidates recognized NCSPs using the results of its air carrier  inspections, airport assessments, ongoing engagement with foreign  government officials, and additional site visits to the foreign country, if  needed. According to our analysis of TSA NCSP recognition letters and  NCSP information compiled by TSA officials, TSA has revalidated all  recognized NCSP countries at least once since fiscal year 2012. Further,  this analysis shows that TSA has generally revalidated the NCSPs of  recognized countries every 3 years, as required by the TSA memos that  established and revised the NCSP recognition process. However, in  2016, TSA authorized a change to the revalidation process that allows for  continuous NCSP recognition because, according to TSA officials and  NCSP memos, the monitoring mechanisms TSA has in place (e.g., air  carrier inspections, foreign airport assessments, and ongoing dialogue  with foreign government officials) provide sufficient information to validate  that foreign governments’ recognized NCSPs and continue to provide a  commensurate level of security to TSA’s. TSA’s 2016 NCSP memo states  that TSA can revoke continuous recognition at any time, and TSA may  not grant continuous recognition to a country if TSA determines that  additional oversight is warranted. For example, TSA officials stated that  they may only recognize a country’s NCSP on a time-limited basis if they  experience communication or access issues or have concerns about  implementation of the NCSP. As of June 2018, TSA had granted  continuous recognition to the European Union and 10 other countries and  had not revoked any government’s continuous recognition, according to  summary NCSP information provided by TSA officials.

TSA’s Existing Performance Measures Do Not Allow It to Specifically Determine the Effectiveness of Its Efforts to Secure U.S.-Bound Air Cargo

TSA has taken steps to broadly measure the effectiveness of its air  carrier inspections and foreign airport assessments, but these efforts do  not allow TSA to specifically determine the effectiveness of the cargo  portions of such inspections or assessments. In addition, TSA has not  developed measures for determining the effectiveness of its NCSP  Recognition Program.

TSA Has Not Evaluated the Effectiveness of its Air Carrier Cargo Inspections or the Cargo Portions of Foreign Airport Assessments

TSA tracks data on the results of air carrier inspections and foreign airport  assessments, and it broadly measures the effectiveness of its foreign  airport assessment program and is developing a similar measure for its  air carrier inspection program. However, TSA’s performance measures do  not allow it to specifically determine the effectiveness of its air carrier  cargo inspections or the cargo portions of foreign airport assessments.  For example, in fiscal year 2017, TSA developed a new performance  measure to track the extent to which foreign airports take actions to  address noncompliance issues identified by TSA inspectors during  foreign airport assessments. The target for this performance measure is  for 70 percent of foreign airports to implement corrective actions or other  mitigation strategies. However, that performance measure does not allow  TSA to determine the effectiveness of the cargo portions of airport  assessments because it does not separately account for cargo and  noncargo noncompliance issues. Specifically, the current measure does  not capture noncompliance issues by category, to allow TSA to determine  which noncompliance issues specifically pertain to cargo. Such a broad  measure of the effectiveness of foreign airport assessments could  obscure progress made (or lack thereof) in resolving cargo-specific  vulnerabilities. According to our analysis of TSA fiscal year 2017 foreign  airport assessment data, TSA could meet its 70 percent target if foreign  airports take actions to address noncompliance issues unrelated to  cargo—including passenger and carry-on baggage screening and access  controls—without taking any actions to address identified noncompliance  issues for cargo.

TSA officials stated that they are coordinating with the Office of  Management and Budget to develop a performance measure to gauge  the effectiveness of air carrier inspections. However, TSA officials also  stated that they have no plans to differentiate the extent to which air  carriers correct violations TSA inspectors identify related to cargo from  those identified related to passengers as they develop this measure.  Notably, TSA has regularly included a goal to secure air cargo and the  supply chain in annual operational implementation plans, but TSA has no  associated performance measures that show the effectiveness of efforts  taken to meet this goal.

TSA’s Office of Global Strategies Fiscal Year 2016 Strategy states that all  strategic goals and objectives will have corresponding, relevant  performance indicators that measure organization effectiveness in those  areas. Further, DHS and TSA guidance state that it is important to  measure the effectiveness of risk management priorities. For example,  the DHS National Infrastructure Protection Plan and Transportation  Systems Sector-Specific Plan state that setting goals and measuring the  effectiveness of risk management efforts against these goals are key  elements of a risk management framework. We have also previously  reported on the importance of developing outcome-based performance  measures—measures that address the results (effectiveness) of products  and services.

According to TSA officials, they have not developed outcome-based  performance measures that are specific to cargo security because they  believe that measuring the results of air carrier inspections and foreign  airport assessments holistically is sufficient to provide them with  information on air cargo vulnerabilities. However, as previously  discussed, TSA inspectors are identifying some potentially serious cargo  vulnerabilities during air carrier cargo inspections and the cargo portions  of airport assessments, including cargo that was not properly screened.  Given TSA’s assessment that the security threat in air cargo is significant,  developing and monitoring an outcome-based performance measure  specific to the cargo portions of foreign airport assessments—along with  differentiating the extent to which air carriers correct violations related to  cargo from those related to passengers as it develops and monitors  outcome-based performance measures for its air carrier inspection  program–could help TSA better determine the effectiveness of these  efforts and whether they are improving the security of U.S.-bound air  cargo. Such cargo-specific outcome-based performance measures could  include differentiating the percentage of cargo-related violations that TSA  has verified air carriers have addressed (as opposed to passenger- related violations) and measuring the progress that foreign airport  authorities, foreign governments, or TSA have made to address  vulnerabilities specific to ICAO’s cargo-related standards.

TSA Has Not Evaluated the Effectiveness of its NCSP Recognition Program

TSA does not measure the effectiveness of its NCSP Recognition  Program. Specifically, TSA budget documents and annual performance  reports do not include measures for gauging the success of its NCSP  Recognition Program. TSA operational implementation plans for fiscal  years 2014 through 2017 addressed program recognition—including  working toward recognition efforts with countries based on a list of  priorities and holding annual in-person meetings with each recognized  government—but TSA has not evaluated the impact of these actions. In  addition, while TSA’s operational implementation plans include  milestones to measure outputs of the NCSP Recognition Program, TSA  has not measured outcomes of its NCSP recognition efforts. For example,  TSA has not measured the extent to which non-recognized countries  implement recommendations that TSA has made to them during the  NCSP recognition process. TSA officials stated that such a measure  would help them determine the effect of the NCSP Recognition Program  on air cargo security. According to TSA officials, in the absence of formal  performance measures, the primary metric used to measure the  performance of the NCSP Recognition Program is the number of  countries TSA has recognized. However, this metric does not address the  effectiveness of the NCSP Recognition Program because it does not  measure how the program improves air cargo security.

We have previously reported on the importance of measuring program  performance. Our prior reports and guidance have stated that  performance measures should evaluate both processes (outputs) and  outcomes related to program activities. Specifically, we have noted that  output measures address the type or level of program activities  conducted, such as the number of countries recognized, while outcome- based measures address the results of products and services, such as  how recognition programs facilitate the identification of air cargo industry  vulnerabilities or contribute to improved air cargo security. Further, as  discussed earlier, TSA strategy documents and leading practices  encourage the development of relevant performance indicators that  measure program effectiveness.

TSA officials stated that TSA has not developed performance measures  associated with the NCSP Recognition Program because TSA has  reorganized and different directorates within TSA have had responsibility  for NCSP program recognition over time. TSA officials also stated that  developing NCSP Recognition Program performance measures has been  secondary to other tasks, such as developing the ACAS program.  Developing and monitoring output and outcome-based performance  measures for its NCSP Recognition Program will help TSA better assess  the effectiveness of the program and whether the resources it has  invested are yielding their intended results.

Air carriers transport billions of pounds of cargo into the United States  from foreign airports each year, and the threat posed by terrorists  attempting to conceal explosive devices in air cargo shipments remains  significant, according to TSA. TSA has taken steps to ensure that U.S- bound air cargo is secure by, for example, conducting air carrier cargo  inspections overseas, performing assessments of foreign airports that  transport cargo to the United States using ICAO cargo-related standards  and recommended practices, and evaluating and recognizing the NCSPs  of foreign countries. Although TSA tracks cargo compliance data  collected during its air carrier inspections and foreign airport assessments  and is developing a vulnerability resolution tool, TSA has not developed  outcome-based performance measures for determining the effectiveness  of its air cargo security compliance efforts. Developing and monitoring an  outcome-based performance measure for the cargo portions of airport  assessments and differentiating the extent to which air carriers correct  violations related to cargo from those related to passengers as it develops  and monitors outcome-based performance measures for its air carrier  inspection program could help TSA better assess the effectiveness of  these efforts and whether they are improving air cargo security. For  example, TSA could measure the percentage of cargo-related violations  that TSA has verified air carriers have addressed. Further, developing  and monitoring output and outcome-based performance measures for its  recognition programs will help TSA better determine the effectiveness of  the NCSP Recognition Program and whether the resources TSA has  invested are yielding their intended results. For example, TSA could  measure the extent to which non-recognized countries implement  recommendations that TSA has made to them during the NCSP  recognition process.

We are making the following three recommendations to TSA:

The Administrator of TSA should instruct Global Strategies to develop  and monitor outcome-based performance measures for determining  the effectiveness of the cargo portion of its foreign airport  assessments. (Recommendation 1)

The Administrator of TSA should instruct Global Strategies to  differentiate the extent to which air carriers correct violations related to  cargo from those related to passengers as it develops outcome-based  performance measures for its air carrier inspection program, and  monitor any measure it develops. (Recommendation 2)

The Administrator of TSA should instruct Global Strategies to develop  and monitor output and outcome-based performance measures for  determining the effectiveness of its NCSP Recognition Program.  (Recommendation 3)

In August 2018, we provided a draft of the sensitive version of this report  to the Department of Homeland Security for its review and comment. In  written comments, which are included in appendix IV, DHS stated that it  concurred with the recommendations and plans to develop cargo-specific  performance measures to help determine the effectiveness of its air  carrier inspections, foreign airport assessments, and the NCSP  Recognition Program. DHS also provided technical comments, which we  have incorporated into the report, as appropriate.

We are sending copies of this report to interested congressional  committees and the Secretary of Homeland Security. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff members have any questions about this report, please  contact Nathan Anderson at (202) 512-3841 or andersonn@gao.gov.   Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  key contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

This report: (1) describes steps the Transportation Security Administration  (TSA) takes to help ensure that U.S-bound air cargo is secure, (2)  describes the status of TSA’s efforts to recognize and monitor foreign  governments’ air cargo security programs, and (3) analyzes the extent to  which TSA measures the effectiveness of its efforts to secure U.S.-bound  air cargo.

To describe the steps TSA takes to help ensure that U.S-bound air cargo  is secure, we reviewed relevant laws and regulations, TSA security  policies and procedures, screening program requirements, and security  directives and emergency amendments relevant to air cargo. For  example, we reviewed relevant air carrier security programs and  associated cargo inspection job aids that TSA transportation security  specialists (inspectors) are to use during each air carrier cargo inspection  to ensure that requirements for air carrier security programs are fully  evaluated during each inspection. We also reviewed fiscal years 2012  through 2018 air carrier inspection and airport assessment Master Work  Plans—which TSA uses to track its overseas air carrier inspection and  foreign airport assessment schedule—to better understand how TSA  schedules inspections and assessments and the types of inspections it  conducts. We chose these fiscal years because they cover the time  period since our previous air cargo security review. In addition, we  conducted site visits to two foreign airports that operate flights that  transport air cargo directly to the United States—one in South America  and one in Asia—to observe a nongeneralizable sample of TSA  inspectors conducting a total of 17 air carrier cargo inspections. At one  airport, we also observed the cargo portion of an airport assessment. We  selected these locations based on their designation by TSA as airports of  relatively high risk level, as well as high volume of U.S.-bound air cargo;  TSA’s air carrier inspection schedule; and geographic dispersion. We also  chose these countries to allow us to observe an inspection in one country  where TSA has recognized the NCSP and one country where TSA has  not recognized the NCSP. In addition, we reviewed the final reports TSA  inspectors completed for the air carrier cargo inspections and airport  assessment we observed.

Further, we obtained and analyzed the results of all air carrier cargo  inspections (close to 5,000) and assessments at foreign airports that are  last points of departure for cargo bound for the United States (about 570)  conducted by TSA inspectors and then entered by them into TSA’s  databases. The Performance and Results Information System (PARIS)  database contains security compliance information on TSA-regulated  entities, including air carriers, and the Global Risk Analysis and Decision  Support (GRADS) system vulnerability tracking sheet contains the results  of foreign airport assessments. We analyzed PARIS and GRADS data  from fiscal years 2012 through 2017, to cover the period since our  previous air cargo security review and to include the 5 most recent years  for which data were available at the time of our review. Specifically, we  analyzed the frequency with which air carriers and foreign airports  complied with TSA air cargo security requirements and select cargo- related International Civil Aviation Organization (ICAO) aviation security  standards and recommended practices, including the seriousness of  ICAO noncompliance issues TSA inspectors identified. TSA also uses GRADS to populate the Open Standards and Recommended Practices  Finding Tool (OSFT), which tracks efforts taken by TSA and host  governments to address noncompliance issues identified during foreign  airport assessments. We analyzed fiscal years 2012 through 2017 OSFT  data to determine the status of noncompliance issues TSA inspectors  identified. We also reviewed 2017 PARIS data on the number of known  consignor and regulated agent assessments TSA inspectors conducted.

To assess the reliability of TSA’s air carrier and airport assessment data  captured in PARIS and GRADS tracking sheet and OSFT, we reviewed  program documentation on system controls, interviewed knowledgeable  TSA officials, and analyzed TSA’s data for any potential gaps and errors.  During our assessment, we found some inconsistencies in the tool TSA  uses to follow up on airport noncompliance issues. We rounded airport  compliance information to the nearest 10 for reporting purposes. We also  aggregated ICAO standards and recommended practices within the  Measures Related to Cargo, Mail, and Other Goods category for reporting  purposes because their numbering has changed over time. We  concluded that TSA’s data on air carrier inspections and foreign airport  assessments were sufficiently reliable to provide a general indication of  the level of compliance for TSA’s air carrier inspections and foreign  airport assessments over the period of our analysis.

In addition, we conducted interviews with TSA officials, foreign  government representatives, and air cargo industry stakeholders, as  follows:

We interviewed senior TSA officials, inspectors, TSA representatives  stationed overseas, and international industry representatives located  at TSA headquarters and in the field. For example, we met with the  Director of Global Compliance as well as managers and inspectors  from all six TSA regional operations centers who are responsible for  planning and conducting air carrier inspections and assessments of  foreign airports. During our interviews with TSA staff, we discussed  TSA’s efforts to ensure the security of U.S.-bound air cargo prior to  being transported to the United States and air carriers are in  compliance with the applicable TSA cargo security requirements.

We also interviewed officials at the European Commission (EC) and  from the civil aviation authority in the country in Asia that we visited to  discuss air cargo security standards and their experiences in  coordinating with TSA. We judgmentally selected these foreign  government entities because they (1) aligned with TSA’s inspection  site visit in the country in Asia that we observed and (2) represent  different models of recognition (i.e., TSA recognizes both the  passenger and all-cargo portions of the European Union national  cargo security program (NCSP) but only passenger operations in the  NCSP for the country in Asia that we visited).

Further, we met with representatives from 2 aviation associations and  11 air carriers that include U.S. and foreign-flagged air carriers, as  well as passenger and all-cargo carriers. One of the international  aviation associations includes air carriers that comprise over 80  percent of the world’s air traffic and the other aviation association  includes the 5 air carriers that transported the largest individual  amounts of U.S.-bound air cargo, by tonnage, in fiscal year 2017. We  based our selection of the 11 air carriers on the relatively high volume  of U.S.-bound cargo they transport; their operation of flights at the  foreign airports we visited; and to obtain a range of coverage  regarding their geographical regions of operation, passenger and all- cargo air carriers, and U.S. and foreign-flagged air carriers.

Results from these meetings with foreign governments and aviation  industry officials are not generalizable, but provided us with information  on stakeholders’ experiences and perspectives regarding air cargo  security issues.

To describe the status of TSA’s efforts to recognize and monitor foreign  governments’ air cargo security programs, we reviewed TSA’s policies  and procedures for its NCSP Recognition Program. For example, we  reviewed TSA memos from 2012, 2013, and 2016 that documented the  recognition standards and any subsequent revisions to the NCSP  Recognition Program; as well as TSA’s process for monitoring NCSP  recognition requirements. Additionally, we analyzed letters that TSA  provided since 2012 to the 13 governments it determined had  commensurate air cargo security programs and NCSP information TSA  officials compiled specifically for our review to better understand TSA’s  terms of recognition with each government and the timeframes for  revalidating NCSP recognition. We also reviewed letters TSA provided to  governments it had determined did not have commensurate air cargo  security programs, which provided us with insights into the recognition  process and the criteria applied to TSA’s reviews. Further, we reviewed  the NCSP Recognition Program’s fiscal years 2017 and 2018 work plans,  as well as summaries of TSA’s annual meetings with foreign governments  to better understand TSA’s efforts to engage with recognized  governments.

We also analyzed the air carrier cargo inspection and airport assessment  data discussed above to determine the number of cargo inspections and  assessments TSA completed in recognized countries from fiscal years  2015 through 2017. We chose this time period because it represents the  3 most recent complete fiscal years, and TSA last recognized a country’s  NCSP in 2015. We also analyzed data from TSA’s Security Policy and  Industry Engagement Policy Inventory on the number of air carriers  participating in the NCSP Recognition Program from fiscal year 2012— when the NCSP Recognition Program began—through fiscal year 2017— the most recent complete fiscal year available at the time of our review— to determine how the level of participation has changed over time. In  addition, we analyzed fiscal year 2017 Department of Transportation  Bureau of Transportation Statistics T-100 data bank, which contains data  on U.S.-bound departures from foreign airports, among other things, to  determine the percentage of overall U.S.-bound air cargo shipped from  NCSP countries. To assess the reliability of the T-100 data, we reviewed  documentation on system controls, interviewed knowledgeable officials  from the Bureau of Transportation Statistics, and analyzed the data for  any potential gaps and errors. We determined that the T-100 data were  sufficiently reliable for our intended purposes. Finally, we conducted  interviews with TSA and foreign government officials from two countries,  and with representatives of the 11 air carriers described previously to  better understand TSA’s ongoing efforts to recognize and monitor foreign  governments’ air cargo security programs. We also confirmed the status  of countries’ NCSP recognition, as of June 2018, with TSA officials.

To analyze the extent to which TSA measures the effectiveness of its  various efforts to secure U.S.-bound air cargo, we reviewed documents  that contain information on TSA’s air cargo security objectives, goals, and  performance measures, including (1) information reported to the Office of  Management and Budget in annual budget documents from fiscal years  2014 through 2019, and (2) TSA’s Global Strategies directorates  Operational Implementation Plans from fiscal years 2014 through 2018— the most recent years available at the time of our review. These plans  include annual objectives and milestones for U.S.-bound air cargo  security programs. We also reviewed the measures in the annual budget  documents and Operational Implementation Plans and compared them  with requirements in TSA’s Global Strategies’ Fiscal Year 2016 Strategy  and Fiscal Year 2018 Strategy Program and applicable laws governing  performance reporting in the federal government, including the  Government Performance and Results Act of 1993 (GPRA), as updated  and expanded by the GPRA Modernization Act of 2010 (GPRAMA). For  example, we assessed whether the performance measures provide  information on the effectiveness of TSA’s various air cargo security  efforts. Although GPRA and GPRAMA requirements apply to those goals  reported by departments (e.g., DHS), we have previously reported that  they can serve as leading practices at other organizational levels, such as  component agencies (e.g., the TSA) for performance management.  Further, we assessed TSA’s performance measures against risk  management principles in the DHS National Infrastructure Protection Plan  and the Transportation Systems Sector-Specific Plan. In addition, we  obtained additional information on how TSA measures the performance of  its air cargo security efforts during our interviews with TSA headquarters  officials.

Appendix II: Transportation Security Administration (TSA) Processes for Conducting Air Carrier Cargo Inspections

Air carrier cargo inspections are conducted by a team of Transportation  Security Administration (TSA) security specialists (inspectors) at foreign  airports who review passenger and all-cargo air carriers’ implementation  of requirements in their TSA-approved security programs, any  amendments or alternative procedures to these security programs, and  applicable security directives or emergency amendments. The frequency  of air carrier cargo inspections at each airport depends on a risk-informed  approach and is influenced, in part, by the airport’s vulnerability to  security breaches, since the security posture of each airport varies,  according to TSA. In general, TSA procedures require TSA to inspect air  carriers with TSA-approved security programs at each airport annually or  semiannually depending on the vulnerability level of the airport, with some  exceptions. The inspection teams—based out of TSA regional operations  centers—generally include one team leader and one team member and  typically take 1 or 2 days, but can involve more inspectors and take  longer to complete depending on the extent of service by the air carrier.  TSA inspectors may spend several days at a foreign airport inspecting air  carriers if there are multiple air carriers serving the United States from  that location. During air carrier cargo inspections, TSA inspectors are to  review applicable security manuals, procedures, and records; interview  air carrier personnel; and are to observe security measures, such as  cargo acceptance and screening, among other activities. Air carriers are  subject to inspection in six key areas of cargo supply chain security, as  described in table 2.

After completion of an air carrier inspection, TSA inspectors are to record  the results into TSA’s Performance and Results Information System  (PARIS), a database containing security compliance information on TSA- regulated entities. If an inspector finds that an air carrier is in violation of  any applicable security requirements, the inspector is to take additional  steps to record the specific violation(s) and, in some cases, pursue them  with further investigation. For example, TSA inspectors may choose to  resolve violations that are minor or technical in nature, such as an  employee not displaying their identification, through on-the-spot feedback  and instruction, referred to as “counseling.” For more serious violations,  such as inadequate screener training, TSA inspectors may pursue  administrative actions, including issuing a warning notice, or initiating an  investigation and requiring air carriers to inform TSA of the specific steps  they will take to address the issue. For more egregious violations, such  as failure to screen cargo, TSA inspectors may recommend a civil  penalty. In extreme cases, TSA may withdraw its approval of an air  carriers’ security program and suspend the air carriers’ operations.  According to TSA officials, they rely on a system of progressive  enforcement and carefully consider whether a civil penalty is warranted  based on the compliance history of an air carrier, among other factors.

Appendix III: Transportation Security Administration (TSA) Processes for Conducting Foreign Airport Assessments

Through its foreign airport assessment program, TSA determines whether  foreign airports that provide passenger or all-cargo air carrier service to  the United States are maintaining and carrying out effective security  measures. To determine the frequency of foreign airport assessments,  TSA uses a risk-informed approach to categorize airports into three risk  tiers, with high risk airports assessed more frequently than medium and  low risk airports. TSA’s assessments of foreign airports are generally  scheduled during the same site visit as air carrier inspections for a certain  location, and the same team of inspectors generally conducts both the  airport assessment and air carrier inspections. According to TSA, it  generally takes 3 to 7 days to complete a foreign airport assessment.  However, the amount of time and number of team members required to  conduct an assessment varies based on several factors, including the  size of the airport and the threat level to civil aviation in the host country.

TSA uses a multistep process to plan and conduct assessments of  foreign airports. Specifically, TSA must obtain approval from the host  government to conduct an airport assessment, and schedule the date for  the on-site assessment. After conducting an entry briefing with host  country and airport officials, the TSA team conducts an on-site visit to the  airport. During the assessment, the team of inspectors uses several  methods to determine a foreign airport’s level of compliance with 39  International Civil Aviation Organization (ICAO) standards and five ICAO  recommended practices, to include conducting interviews with airport  officials, examining documents pertaining to the airport’s security  measures, and conducting a physical inspection of the airport. ICAO  standards and recommended practices address operational issues at an  airport, such as ensuring that passengers and cargo are properly  screened and that unauthorized individuals do not have access to  restricted areas of an airport. ICAO standards and recommended  practices also address non-operational issues, such as whether a foreign  government has implemented a national civil aviation security program for  regulating security procedures at its airports and whether airport officials  that are responsible for implementing security controls are subject to  background investigations, are appropriately trained, and are certified  according to the foreign government’s national civil aviation security  program.

At the close of an airport assessment, TSA inspectors are to brief foreign  airport and government officials on the results. TSA inspectors also  prepare a report in TSA’s Global Risk Analysis and Decision Support  System (GRADS) detailing their findings on the airport’s overall security  posture and security measures, which may contain recommendations for  corrective actions and must be reviewed by TSA field and headquarters  management. As part of the report, TSA assigns a vulnerability score to  each ICAO standard and recommended practice assessed, as well as an  overall vulnerability score for the airport, which corresponds to the level of  compliance for each ICAO standard and recommended practice TSA  assesses. Further, according to TSA officials, cargo experts in TSA  headquarters review the cargo portion of each airport assessment before  the assessment report is finalized. Afterward, TSA shares a summary of  the results with the foreign airport and host government officials. In some  cases, TSA requires air carriers to implement security procedures, such  as requiring air carrier employees to guard the aircraft while on the  tarmac, to address any deficiency that TSA identified during a foreign  airport assessment through the issuance of security directives and  emergency amendments. If the Secretary of Homeland Security  determines that an airport does not maintain and carry out effective  security measures, he or she shall, after advising the Secretary of State,  take action, which generally includes notification to the appropriate  authorities of the country of security deficiencies identified, notification to  the general public that the airport does not maintain effective security  measures, and modification of air carrier operations at that airport.

Appendix IV: Comments from the Department of Homeland Security

Appendix V: GAO Contact and Staff Acknowledgments

Nathan Anderson, (206) 287-4804 or andersonn@gao.gov.

In addition to the contact above, Christopher Conrad (Assistant Director),  Paul Hobart (Analyst-in-Charge), Hiwotte Amare, Charles Bausell,  Dominick Dale, Pamela Davidson, Wendy Dye, Mike Harmond, Eric  Hauswirth, Ryan Lester, Benjamin Licht, and Tom Lombardi made key  contributions.

How does the TSA ensure that explosives aren't hidden in the cargo holds of commercial aircraft? One part of the effort is inspecting passenger and cargo airlines, for example, to see how they screen cargo. Another is assessing security at foreign airports.

Our analysis of 2012-2017 data shows TSA found violations related to air cargo security in

TSA has taken actions to resolve most of the violations and is working on the remainder. We recommended that TSA establish measures to better assess the effectiveness of its efforts.