Aviation Security: TSA Strengthened Foreign Airport Assessments and Air Carrier Inspections, but Could Improve Analysis to Better Address Deficiencies

Approximately 300 foreign airports offer last point of departure flights to the United States. TSA is the federal agency with primary responsibility for securing the nation's civil aviation system and assesses foreign airports and inspects air carriers to ensure they have in place effective security measures. While TSA is authorized under U.S. law to conduct foreign airport assessments, it does not have authority to impose or otherwise enforce security requirements at foreign airports. TSA is authorized to impose and enforce requirements on air carriers. The Aviation Security Act of 2016 includes a provision for GAO to review TSA's effort to enhance security at foreign airports.

This report addresses (1) steps TSA has taken to enhance foreign airport assessments and air carrier inspections since 2011, (2) the results of TSA's foreign airport assessments and air carrier inspections, and (3) steps TSA takes to address any deficiencies identified during foreign airport assessments and air carrier inspections. GAO reviewed TSA program data, interviewed TSA officials, and conducted site visits to TSA field locations that manage assessments and inspections.

The Transportation Security Administration (TSA) has taken steps to enhance its foreign airport assessments and air carrier inspections since 2011, including aligning resources based on risk, resolving airport access issues, making evaluations more comprehensive, and creating operational efficiencies. For example, TSA has implemented targeted foreign airport assessments in locations where risk is high and developed the Global Risk Analysis and Decision Support System to strengthen data analysis. In addition, TSA has increased the number of joint airport assessments with the European Commission. Specifically, TSA officials GAO met with indicated that TSA's strong relationship with the European Commission has afforded the agency excellent access to foreign airports in Europe and a better understanding of vulnerabilities at these locations, which has resulted in more comprehensive assessments.

In its analysis of TSA foreign airport assessment results, GAO found that during fiscal years 2012 through 2016 there was considerable regional variation among last point of departure airports in the level of compliance with select International Civil Aviation Organization security standards and recommended practices. TSA attributed this regional variation to lack of airport resources or technical knowledge, among other factors. TSA officials also stated that while these challenges are not easy to overcome, agency efforts, such as training host country staff, can help foreign airports reduce their vulnerability scores over time. GAO's analysis of TSA's foreign airport assessment data confirmed that point by demonstrating that most foreign airports categorized with poor vulnerability ratings in fiscal year 2012 improved their vulnerability score in at least one follow-up assessment during fiscal years 2012 through 2016.

Meanwhile, U.S. and foreign-flagged air carriers providing last point of departure service to the United States from foreign airports complied with all TSA security requirements in most inspections, and TSA was able to resolve the majority of security deficiencies it identified with on-the-spot counseling. In some cases, TSA inspectors submitted violations for investigation because the violations were considered serious enough to potentially warrant an enforcement action.

TSA addresses identified deficiencies at foreign airports through capacity development, such as training and on-the-spot counseling. However, GAO found that TSA's database for tracking the resolution status of security deficiencies did not have comprehensive data on security deficiencies' root causes and corrective actions. In addition, the database lacked adequate categorization mechanisms. For example, while it captures three broad categories of root causes (e.g., lack of knowledge) it does not capture subcategories (e.g., supervision) that would better explain the root causes of security deficiencies. Fully collecting these data and improving the specificity of categorization would help TSA strengthen analysis and decision making. For example, TSA would be better positioned to determine the extent to which airports that received particular types of capacity development assistance were able to close security vulnerabilities. This is a public version of a sensitive report issued in October 2017. Information that TSA deemed to be sensitive is omitted from this report.

To help strengthen TSA's analysis and decision making, GAO recommends that TSA fully capture and more specifically categorize data on the root causes of security deficiencies that it identifies and corrective actions. TSA concurred with the recommendations.

The April 2012 plot to detonate a bomb aboard a flight bound for the  United States, which U.S. intelligence officials foiled in the Arabian  Peninsula, and the October 2015 downing of a flight over the Sinai  Peninsula illustrate that civil aviation, including U.S.-bound flights,  remains a target of coordinated terrorist activity. Moreover, the threat has  become more diverse and terrorists are continually developing new  tactics to attack the aviation system, including advanced concealment of  improvised explosive devices in baggage and personal electronic  devices. Since approximately 300 airports in foreign countries offer last  point of departure flights to the United States, efforts to evaluate the  security of foreign airports and air carriers that service the United  States—and mitigating any identified security risks—are of vital  importance in ensuring the security of the aviation system.

The Transportation Security Administration (TSA), within the U.S.  Department of Homeland Security (DHS), is the federal agency with  primary responsibility for securing the nation’s civil aviation system and  has programs in place to help ensure the security of U.S.-bound flights.  Through its foreign airport assessment program, TSA determines whether  foreign airports that provide service to the United States are maintaining  and carrying out effective security measures. Although TSA is authorized  under U.S. law to conduct foreign airport assessments at intervals it  considers necessary, it may not perform an assessment of security  measures at a foreign airport without permission from the host  government. TSA also does not have authority to impose or otherwise  enforce security requirements at foreign airports and, therefore, seeks to  address security deficiencies it identifies through capacity building, such  as training of foreign airport staff and onsite consultation, and working  with U.S. and foreign-flagged air carriers (i.e., air carriers) to implement  security measures, among other things. In contrast, TSA is authorized  under U.S. law to place security requirements on air carriers that service  the United States and to take enforcement actions through its air carrier  inspection program if carriers fail to comply with the requirements. For  example, in June 2017, DHS announced new security requirements,  including heightened screening of personal electronic devices, for air  carriers operating last point of departure flights to the United States from  foreign airports.

In 2007, we recommended that TSA take steps to improve oversight of its  foreign airport assessment and air carrier inspection programs. In 2011,  we reported on TSA’s efforts to assess the security at foreign airports and  made several recommendations to enhance program efficiency and  effectiveness, among other things. DHS concurred with the  recommendations and has since taken a number of steps to address  them and improve the program. We discuss these and other actions TSA  has taken since 2011 later in this report.

The Aviation Security Act of 2016 includes a provision for GAO to review  the efforts, capabilities, and effectiveness of TSA to enhance security  capabilities at foreign airports and determine if the implementation of such  efforts and capabilities effectively secures international-inbound aviation.  This report (1) describes steps TSA has taken to enhance foreign airport  assessments and air carrier inspections since 2011, (2) describes the  results of TSA’s foreign airport assessments and air carrier inspections,  and (3) examines steps TSA takes to address any deficiencies identified  during foreign airport assessments and air carrier inspections.

This report is a public version of a sensitive report that we issued in  October 2017. TSA deemed some of the information in our October  report to be Sensitive Security Information, which must be protected from  public disclosure. Therefore, this report omits sensitive information about  TSA’s risk methodology, the standards that TSA uses to assess foreign  airports, and the specific results of TSA’s foreign airport assessments and  air carrier inspections. Although the information provided in this report is  more limited, the report addresses the same objectives as the sensitive  report and uses the same methodology.

To address these objectives, we reviewed the relevant laws and  regulations pursuant to which TSA conducts foreign airport assessments  and air carrier inspections. We reviewed various TSA documents on  program management and strategic planning. Specifically, we reviewed  TSA’s 2016 Foreign Airport Assessment Program Standard Operating  Procedures (SOP), which prescribes program and operational guidance  for assessing security measures at foreign airports and informs TSA  personnel at all levels of what is expected of them in the implementation  of the program. We also reviewed the job aids that TSA inspectors use  during each assessment and inspection, which are intended to ensure  that the TSA-specified International Civil Aviation Organization (ICAO)  aviation security standards and recommended practices (referred to  collectively in this report as ICAO standards unless otherwise noted) and  air carrier security program requirements are fully evaluated during each  assessment. In addition, we interviewed senior TSA officials, inspectors,  and country and industry liaisons located at TSA headquarters and in the  field. Specifically, we conducted site visits to a non-generalizable  selection of three of the six TSA regional operations centers (ROC)  located in Reston, Miami, and Frankfurt. We based our site visit  selections on the number and type of staff available at each location and  geographic dispersion. During our interviews with TSA staff, we  discussed TSA’s efforts to enhance foreign airport assessments and air  carrier inspections, the results of these evaluations, and the extent to  which TSA uses information at its disposal to inform capacity  development efforts for airports and air carriers. We also interviewed  other stakeholders, such as the Department of State (State) and the  European Commission (EC) to discuss efforts these organizations have in  place to enhance international aviation security and their experiences  coordinating with TSA.

To obtain a greater understanding of TSA’s foreign airport assessment  and air carrier inspection processes, including how TSA works with host  nation officials and air carrier representatives, we accompanied a team of  TSA inspectors during an air carrier inspection at an airport in Europe,  and we spoke with airport officials and representatives from two air  carriers at a separate European airport. We based our site selection on  several factors, including the air carrier locations TSA had plans to  inspect during the course of our audit work and host government  willingness to allow us to accompany TSA.

To describe the results of TSA’s foreign airport assessments and air  carrier inspections, we obtained and analyzed the results of TSA’s foreign  airport assessments and air carrier inspections from fiscal years 2012  through 2016, the five-year period since our previous review. Specifically,  we analyzed the frequency with which foreign airports and air carriers  complied with select ICAO standards that TSA uses and TSA  requirements, such as passenger screening, baggage screening, and  access controls, among others. To assess the reliability of TSA’s  assessment and inspection data, we reviewed program documentation on  system controls, interviewed knowledgeable officials from the Office of  Global Strategies (OGS), and checked TSA’s data for any potential gaps  and errors. We concluded that TSA’s data on foreign airport assessments  and air carrier inspections were sufficiently reliable for the purposes of our  review. Finally, we compared TSA’s efforts to leverage information for  capacity development to the Foreign Airport Assessment Program SOP  and criteria for obtaining and processing information in federal internal  control standards.

The performance audit upon which this report is based was conducted  from August 2016 to October 2017 in accordance with generally accepted  government auditing standards. Those standards require that we plan  and perform the audit to obtain sufficient, appropriate, evidence to provide  a reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives. We  subsequently worked with TSA from September 2017 to December 2017  to prepare this nonsensitive version of the original sensitive report for  public release. This public version was also prepared in accordance with  these standards. More details about the scope and methodology of our  work are contained in appendix I.

DHS Responsibilities for Ensuring the Security of U.S.-Bound Flights from Foreign Countries

Shortly after the September 11, 2001, terrorist attacks, Congress passed  and the President signed into law the Aviation and Transportation  Security Act (ATSA), which established TSA and gave the agency  responsibility for securing all modes of transportation, including the  nation’s civil aviation system, which includes the operations of U.S. and  foreign-flagged air carriers to, from, and within the United States, as well  as the foreign point-to-point operations of U.S.-flagged carriers.  Consistent with ATSA and in accordance with existing statutory  requirements, TSA is to assess the effectiveness of security measures at  foreign airports (1) served by a U.S. air carrier, (2) from which a foreign  air carrier serves the United States, (3) that pose a high risk of introducing  danger to international air travel, and (4) that are otherwise deemed  appropriate by the Secretary of Homeland Security. In carrying out this  function, the statute identifies measures that the Secretary must take in  the event that he or she determines that an airport is not maintaining and  carrying out effective security measures based on TSA assessments. In  addition, consistent with ATSA and in accordance with existing statutory  requirements, TSA is to conduct inspections of U.S. air carriers and  foreign air carriers servicing the United States from foreign airports to  ensure that they meet applicable security requirements, including those  set forth in an air carrier’s TSA-approved security program.

The Secretary of DHS delegated to the TSA Administrator the  responsibility for conducting foreign airport assessments but retained  responsibility for making the determination that a foreign airport does not  maintain and carry out effective security measures. Currently, the  Global Compliance Directorate, within OGS, is responsible for conducting  foreign airport assessments and air carrier inspections. Table 1 highlights  the roles and responsibilities of certain TSA positions within OGS that are  responsible for implementing the foreign airport assessment and air  carrier inspection programs.

TSA’s Process for Conducting Foreign Airport Assessments and Air Carrier Inspections

TSA assesses the effectiveness of security measures at foreign airports  using select aviation security standards and recommended practices  adopted by ICAO, a United Nations organization representing 191  countries. ICAO standards and recommended practices (referred to  collectively in this report as ICAO standards unless otherwise noted)  address operational issues at an airport, such as ensuring that  passengers and baggage are properly screened and that unauthorized  individuals do not have access to restricted areas of an airport. ICAO  standards also address non-operational issues, such as whether a foreign  government has implemented a national civil aviation security program for  regulating security procedures at its airports and whether airport officials  implementing security controls are subject to background investigations,  are appropriately trained, and are certified according to a foreign  government’s national civil aviation security program. TSA utilizes the 44  ICAO standards it sees as most critical in conducting its foreign airport  assessments, which cover the following areas: airport operations; quality  control; access control; aircraft security; passenger and cabin baggage  screening; hold baggage screening; security measures relating to cargo,  mail and other goods; security measures relating to special categories of  passengers; prevention; and security measures relating to the landside.

TSA uses a risk-informed approach to schedule foreign airport  assessments by categorizing airports into three risk tiers, with high risk  airports assessed more frequently than medium and low risk airports.  TSA’s assessments of foreign airports are conducted by a team of  inspectors, which generally includes one team leader and one team  member. According to TSA, it generally takes 3 to 7 days to complete a  foreign airport assessment. However, the amount of time and number of  team members required to conduct an assessment varies based on  several factors, including the size of the airport, the number of air carrier  inspections to be conducted at the airport, and the threat level to civil  aviation in the host country.

TSA uses a multistep process to plan, conduct, and record assessments  of foreign airports. Specifically, the TSAR must obtain approval from the  host government to allow TSA to conduct an airport assessment, and  schedule the date for the on-site assessment. After conducting an entry  briefing with State, host country officials, and airport officials, the team  conducts an on-site visit to the airport. During the assessment, the team  of inspectors uses several methods to determine a foreign airport’s level  of compliance with ICAO standards, including conducting interviews with  airport officials, examining documents pertaining to the airport’s security  measures, and conducting a physical inspection of the airport. For  example, inspectors are to examine the integrity of fences, lighting, and  locks by walking the grounds of the airport. Inspectors also make  observations on access control procedures, such as examining employee  and vehicle identification methods in secure areas, as well as monitoring  passenger and baggage screening procedures in the airport. At the close  of an airport assessment, inspectors brief foreign airport and government  officials on the results. TSA inspectors also prepare a report detailing  their findings on the airport’s overall security posture and security  measures, which may contain recommendations for corrective action and  must be reviewed by the TSAR, the ROC manager, and TSA  headquarters officials. Afterward, a summary of the results is shared with  the foreign airport and host government officials. In some cases, TSA  requires air carriers to adopt security procedures, such as additional  passenger screening, to compensate for deficiencies that TSA identified  during a foreign airport assessment.

Along with conducting airport assessments, the same TSA inspection  team also conducts air carrier inspections when visiting a foreign airport  to ensure that air carriers are in compliance with TSA security  requirements. The frequency of air carrier inspections at each airport  depends on a risk-informed approach and is influenced, in part, by the  airport’s vulnerability to security breaches, since the security posture of  each airport varies. In general, TSA procedures require TSA to inspect all  air carriers at each airport annually or semi-annually depending on the  vulnerability level of the airport, with some exceptions. For example, TSA  may elect to inspect all air carriers at a particular airport on an 18-month  cycle if the airport has no documented vulnerabilities for the three  previous visits and all air carriers at that location have demonstrated full  compliance over the past five years. When conducting inspections, TSA  inspectors examine compliance with applicable security requirements,  including TSA-approved security programs, security directives, and  emergency amendments to the security programs.

As in the case of airport assessments, air carrier inspections are  conducted by a team of inspectors, which generally includes one team  leader and one team member. An inspection of an air carrier typically  takes 1 or 2 days, but can take longer depending on the extent of service  by the air carrier. Inspection teams may spend several days at a foreign  airport inspecting air carriers if there are multiple carriers serving the  United States from that location. During an air carrier inspection,  inspectors are to review applicable security manuals, procedures, and  records; interview air carrier station personnel; and observe air carrier  employees processing passengers from at least one flight from  passenger check-in until the flight departs the gate to ensure that the air  carrier is in compliance with applicable requirements. Inspectors evaluate  a variety of security measures, such as passenger processing (e.g., use  of No Fly and Selectee lists), checked baggage acceptance and control,  aircraft security, passenger screening, cargo and mail screening, and  catering security. Inspectors record inspection results into TSA’s  Performance and Results Information System (PARIS), a database  containing security compliance information on TSA-regulated entities. If  an inspector finds that an air carrier is violating any applicable security  requirements, additional steps are to be taken to record those specific  violations and, in some cases, pursue them with further investigation.

GAO’s 2011 Review of TSA Foreign Airport Assessment Program

In 2011, we reported on TSA’s foreign airport assessment program,  including TSA’s steps taken to enhance its program, the results of TSA’s  foreign airport assessments, and opportunities for TSA to make program  improvements in several key areas, such as developing criteria and  guidance for determining foreign airport vulnerability ratings. We  reported that TSA had not taken steps to evaluate its assessment results  to identify regional and other trends over time. In addition, we found that  TSA had not developed criteria or guidance for determining foreign airport  vulnerability ratings. We also reported that there were opportunities for  TSA to increase program efficiency and effectiveness by, for example,  conducting more targeted foreign airport assessments and systematically  compiling and analyzing security best practices. As a result, we  recommended that TSA (1) develop a mechanism for trend analysis, (2)  establish criteria and guidance to help decision makers with vulnerability  ratings, and (3) consider the feasibility of conducting more targeted  foreign airport assessments and compiling best practices. DHS concurred  with the three recommendations and has since taken several actions to  address them all, including developing a mechanism to compile and  analyze best practices.

Since 2011, TSA Has Taken Various Steps to Strengthen its Foreign Airport Assessment and Air Carrier Inspection Programs

TSA Has Taken Steps to Better Target Program Resources Based on Risk

TSA established the Northern Virginia ROC. In 2012, TSA created a  dedicated ROC in Northern Virginia to oversee North Africa and the  Middle East given the high risk associated with many airports in the  region. The creation of the Northern Virginia ROC alleviated resource  burdens on the Frankfurt ROC, which previously had oversight for both  the Europe and Africa-Middle East regions. In addition, the Northern  Virginia ROC Manager stated that the small size of the ROC has  facilitated strong working relationships because foreign airport officials in  the region tend to meet with the same inspectors more frequently.

TSA created the Analysis and Risk Mitigation (ARM) Directorate. In  2013, TSA established a working group to evaluate ways to better  integrate risk management in the foreign airport assessment and air  carrier inspection programs. This working group developed a risk  framework, which, according to TSA documentation, provides a  systematic approach for analyzing risk at international airports, supports  OGS decision making, and informs efforts to mitigate security  deficiencies. In 2015, OGS created the ARM Directorate, which  formalized the risk mitigation responsibilities of the working group and  serves as the data analysis and evaluation arm of OGS. OGS officials  stated that ARM helps the program focus its resources based on risk. For  example, ARM analyzes and prioritizes activities, such as training, that  are designed to mitigate security vulnerabilities at foreign airports.

TSA conducts more targeted foreign airport assessments. Based on  a recommendation in our 2011 report, TSA has taken actions to conduct  more targeted foreign airport assessments. For example, TSA developed  the Pre-Visit Questionnaire, which host foreign airport officials fill out prior  to TSA’s visit. This information enables each TSA foreign airport  assessment team to tailor the on-site assessment at each airport and  focus TSA’s assessment efforts on specific areas of concern. Additionally,  TSA implemented more focused airport assessments, known as targeted  risk assessments, in locations where risk is high or there are other factors  that require a more focused evaluation of the site’s security posture. For  the focused assessments, inspection teams place emphasis on  observations, interviews, document reviews, and thorough analysis of  specific ICAO standards.

TSA implemented cross-directorate reviews. In 2015, TSA  implemented cross-directorate reviews, which bring together experts  across the OGS components, such as inspectors and TSARs, to identify  critical vulnerabilities at foreign airports and outline an initial plan to  mitigate those vulnerabilities. Overall, TSA completed 28 cross- directorate reviews in 2015 and 2016.

TSA Has Taken Steps to Strengthen Foreign Airport Access and the Comprehensiveness of Its Evaluations

TSA took steps to resolve foreign airport access issues. Since our  2011 review, TSA has faced delays in scheduling some foreign airport  assessments and obstacles in obtaining full access to airport operations  at certain locations. According to TSA officials, TSA has used several  tactics to resolve access issues, including deploying the same inspectors  over multiple assessments to build rapport with foreign airport officials.  For example, in one country in the Western Hemisphere region, TSA’s  access to airport operations was initially limited by the host government.  However, over time, TSA used a small pool of inspectors who officials  said were able to build trust with the host government and gain better  access, including the ability to conduct interviews of airport officials and  take photographs of the security environment. Additionally, in 2011, we  reported on TSA’s challenges in obtaining access to airports in  Venezuela. Specifically, we reported that TSA had not been able to  assess airports in Venezuela or conduct TSA compliance inspections for  air carriers, including U.S. carriers, flying from Venezuela to the United  States since 2006. According to TSA officials, in 2014, TSA regained  access in Venezuela after establishing dialogue with the new government  in place and emphasizing the benefits of the evaluation process.

TSA increased the number of joint airport assessments in Europe. In  2011, we reported that TSA took a number of actions to assess foreign  airports in Europe, including conducting joint assessments with the EC,  performing bi-lateral assessments, and executing table-top reviews in  place of on-site airports visits. According to EC officials, the main goal  under this arrangement was to better leverage resources and reduce the  number of TSA visits per year to European airports because of concerns  from EU member states about the frequency of visits from EC and U.S.  audit teams. However, since our previous review, TSA has limited the use  of table-top reviews and now primarily assesses foreign airports in  Europe through joint assessments with the EC. Frankfurt ROC officials  we met with indicated that TSA’s strong relationship with the EC has  afforded the agency excellent access to foreign airports in Europe and a  better understanding of vulnerabilities at these locations, which has  resulted in more comprehensive assessments. For example, according to  TSA, through the joint assessments, inspectors have better access to  airport training documents, the ability to observe tests conducted by EC  inspectors, and more time at checkpoints to observe screening  operations.

TSA developed airport assessment and air carrier inspection job  aids. In 2012, TSA developed job aids that provide inspectors with a set  of detailed areas to assess for each ICAO standard. For example, a job  aid for passenger and cabin baggage screening includes several prompts  related to screening roles and responsibilities, the resolution process if a  suspicious item is detected, and alternative procedures if screening  equipment is not working as intended. TSA also developed job aids for  the air carrier inspection process to better ensure that inspectors cover all  requirements associated with air carrier security programs. According to  OGS officials, these actions have led to more comprehensive evaluations  and a better understanding of foreign airport and air carrier vulnerabilities.

TSA Has Worked to Create Operational Efficiencies

TSA established the Honolulu ROC. In 2012, TSA eliminated the Los  Angeles ROC and established the Honolulu ROC given its proximity to  the Pacific Islands, which allowed the agency to reduce costs and travel  time to airports in these locations. Specifically according to TSA  documentation, inspectors in the Los Angeles ROC often spent more than  20 hours traveling to and from sites in the Asia-Pacific region because of  in-flight transit time and connection requirements. With the creation of the  Honolulu ROC, TSA officials told us that inspectors have been better able  to meet deadlines for completing foreign airport assessment reports and  conduct follow-up visits to resolve noted issues.

TSA developed the Global Risk Analysis and Decision Support  System. In 2012, TSA developed the Global Risk Analysis and Decision  Support System (GRADS) to streamline the assessment report writing  process and strengthen OGS’s data analysis capabilities of its foreign  airport assessment results. According to TSA officials, GRADS has  provided OGS personnel with a number of benefits, including the ability to  run standardized reports, extract and analyze key data, and manage  airport operational information, such as data on security screening  equipment. According to TSA documentation, prior to 2012, the agency  captured the results of its foreign airport assessments in narrative form  that often amounted to more than 80 pages, hampering the ability to  perform data analysis.

TSA standardized processes. Between 2012 and 2016, TSA deployed  standardization teams, called Standardization Effort Teams, to help  ensure more consistency among inspectors when conducting air carrier  inspections and airport assessments, and to identify and develop best  practices in areas such as training, among others. For example, in 2016,  a team developed a tool to facilitate performance evaluations of  inspectors.

TSA Foreign Airport Assessment Data Showed Variations in Compliance by Region and Across ICAO Standards, while Air Carrier Inspection Data Showed That Most Inspections Were Fully Compliant

Foreign Airports Differed in Level of Compliance by Region and Across ICAO Standards

TSA assesses the overall vulnerability level at each foreign airport using a  rating system, ranging from a category “1,” which represents full  compliance with ICAO standards, to a “4” or “5,” which involve more  serious or egregious issues. Based on our analysis of TSA’s foreign  airport assessment data, we found that compliance with ICAO standards  varied by region. For example, our analysis showed that some regions of  the world had a higher percentage of airports in vulnerability categories 4  and 5. Our analysis also showed that there are differences in compliance  across the ICAO standards. Specific information related to TSA’s airport  assessment results is deemed Sensitive Security Information.

According to TSA officials, it is difficult to draw conclusions about the  cumulative foreign airport assessment results—such as whether the  results are generally positive or negative—because the primary concern  is not whether security deficiencies are identified, but whether foreign countries are capable and willing to address security deficiencies.  Specifically, there is considerable regional variation in the level of  compliance because some foreign countries face challenges due to lack  of resources or technical knowledge, among other factors. TSA officials  stated that while these challenges are not easy to overcome, agency  efforts, such as training host country staff, can help foreign airports  reduce their vulnerability scores over time. Our analysis of TSA’s foreign  airport assessment data confirms that point. Specifically, we found that of  the foreign airports categorized with a vulnerability rating of 4 of 5 in fiscal  year 2012, the majority of these airports improved their vulnerability score  in at least one follow-up assessment during fiscal years 2012 through  2016. According to TSA documentation, in some cases, foreign airports  are able to take immediate measures to resolve security deficiencies. On  the other hand, there are situations in which foreign airports may struggle  to take corrective actions or sustain the improvements over time.  Accordingly, TSA’s regulatory authority over air carriers is an important  tool. TSA officials indicated that the agency commonly requires air  carriers to adopt security procedures, such as passenger screening, to  compensate for foreign airport security deficiencies. Moreover, if  appropriate, DHS can take secretarial action, which includes the option to  prohibit air carriers operating at a foreign airport from providing last point  of departure flights to the United States.

Most Air Carrier Inspections Were Fully Compliant and TSA Used On-the-Spot Counseling to Resolve the Majority of Deficiencies

According to air carrier inspection data maintained by TSA, between fiscal  years 2012 and 2016, air carriers providing last point of departure service  to the United States from foreign airports complied with all TSA security  requirements in most inspections. For those inspections that identified  noncompliance, data from TSA showed that the majority of violations  were corrected or addressed immediately through on-the-spot counseling.  Inspectors submitted a certain number of violations for investigation  because the violations were considered serious enough to potentially  warrant an enforcement action. TSA can impose two general types of  enforcement actions on air carriers that violate security requirements—an  administrative action, such as a warning notice, or a monetary civil  penalty. Based on information included in TSA’s investigation module  within PARIS, TSA took administrative action in the majority of cases and  levied 44 fines during fiscal years 2012 through 2016, which totaled about  $575,000 and ranged from $1,000 to $40,500. According to TSA officials,  they rely on a system of progressive enforcement and carefully consider  whether a civil penalty is warranted based on the compliance history of an  air carrier, among other factors.

TSA Addresses Security Deficiencies through Various Capacity Development Efforts, but Enhanced Data Management Could Strengthen Analysis and Decision Making

TSA Assists Foreign Airports and Air Carriers in Addressing Identified Security Deficiencies in Various Ways

As part of assisting foreign airports, inspectors work to transfer  knowledge on how to mitigate identified airport security deficiencies to  foreign airport officials and provide TSA program officials with  suggestions for capacity development that could be effective in  addressing these deficiencies. Specifically, TSA capacity development  assistance to foreign airports includes on-the-spot counseling, training,  technical assistance and consultation, and provision of security  equipment.

Inspectors counsel foreign airport staff on-the-spot. According to  TSA officials, inspectors typically offer counseling during airport  assessments when they discover deficiencies, usually of an infrequent,  less serious, or technical nature, that can be addressed immediately. For  example, during a 2013 assessment of an airport in the Europe region,  inspectors observed a total of 53 employees within the restricted area, of  which one was not displaying his badge. Airport officials immediately  requested that the individual display his badge and informed the TSA  inspection team that they will remind all staff to properly display their  airport media while in the restricted area. For the remainder of the airport  visit, no badge display issues were noted. In another example, during an  assessment in the Western Hemisphere region, inspectors observed  persons entering a restricted area without undergoing screening. The  inspectors counseled the airport’s security officials on the importance of  adhering to the airport’s security program, and observed the airport  officials take immediate action by implementing escort and screening  procedures.

TSA provides security training. TSA may provide training to foreign  airport staff to address deeper problems with staff security knowledge or  to strengthen staff knowledge in an evolving threat environment. Training  may take several forms, including traditional classroom courses or  interactive workshops, and can range in length from one or two days to  more than one week. Course topics include risk management, screening  operations, and airport security, with a broad variety of sub-topics, such  as insider risk, cargo security, and inspection techniques. According to  TSA, new courses are in development to meet the changing security  landscape. New course topics include landside security, behavioral  awareness, and the effective use of canines.

TSA arranges for technical assistance and consultation. TSA assists  foreign governments in securing technical assistance and consultation  provided by TSA and other U.S. and foreign government agencies to help  improve security at foreign airports, particularly after security incidents or  at airports in developing countries. For example, after the 2016 terrorist  attack on Brussels Airport, TSA was invited by airport officials to provide  on-site consultation during the reconstitution of the airport facilities. In  another example, TSA provided a country in the Africa-Middle East region  with on-site technical assistance for configuring and testing explosives  detection equipment at baggage screening checkpoints. In addition,  State’s Anti-Terrorism Assistance Program augments TSA’s resources in  building the aviation security capacity of foreign governments. For  instance, State provides recipient nations with courses focused on airport  security management, quality control, and fraudulent document  recognition as well as multi-day passenger and cargo security  consultations. In addition, with regard to capacity development TSA  collaborates with other countries. Partners may promote common aviation  security goals to other countries when political considerations preclude  TSA from doing so, or combine resources with TSA for joint efforts. For  example, in one collaboration, a country in the Asia-Pacific region  provided resources and facilities, while TSA provided staff so that  neighboring countries could attend aviation security training.

TSA loans and donates security equipment. TSA may loan or donate  security equipment such as explosives detection devices and metal  detection hand wands to lower-income countries. Since fiscal year 2012,  TSA has loaned X-ray screening equipment and explosives detection  devices to five countries. Enacted in July 2016, the Aviation Security Act  expressly authorizes TSA to donate security screening equipment to a  foreign last point of departure airport if such equipment can be reasonably  expected to mitigate a specific vulnerability to the security of the United  States or U.S. citizens. TSA may also provide staff at foreign airports  with demonstrations for using equipment that has been loaned or donated  by TSA, as well as equipment otherwise acquired by host governments.  For instance, in 2016 TSA provided operator training and maintenance  assistance to a country in the Africa-Middle East region that had procured  passenger body scanners.

TSA also takes steps to help air carriers address security deficiencies  identified during air carrier inspections. TSA primarily offers capacity  development support to air carriers through on-the-spot counseling and  consultation with IIRs.

Inspectors counsel air carrier representatives on-the-spot. TSA  assists air carrier representatives in addressing security deficiencies  identified during air carrier inspections. According to TSA, since carriers  have TSA-approved security programs, additional training may not be  necessary to correct small issues. Rather, officials said that counseling air  carrier staff on the proper procedures and follow up observations of them  practicing the procedures may suffice. TSA data show that of the  instances in which inspectors identified noncompliance with TSA security  requirements during fiscal years 2012 through 2016, the majority of  instances were resolved through counseling—that is, the security  deficiencies were resolved with on-site assistance or consultation  provided by TSA. For example, during an air carrier inspection in the  Europe region, inspectors observed that a passenger wearing sandals  was not screened properly. TSA counseled the screening staff that  footwear screening requirements apply to all shoes, including sandals.  The inspectors then observed proper rescreening of the passenger. TSA  also discussed the matter with airline security representatives, who  concurred with TSA.

IIRs assist air carriers with compliance. In addition to counseling  provided by inspectors when deficiencies are identified, TSA assigns  each air carrier to a representative who assists the carriers in complying  with TSA security requirements. Although these representatives, called  IIRs, do not participate in air carrier inspections, they do receive  inspection results for the carriers with whom they work. IIRs counsel the  air carriers and provide clarification regarding TSA security requirements  when necessary. For example, they provide air carriers with clarification  on the requirements contained in security directives and emergency  amendments issued by TSA. In other instances, when an air carrier  cannot comply with a TSA security requirement—such as when  complying with a TSA security requirement would cause the air carrier to  violate a host government security requirement—the air carrier works with  its IIR to develop alternative security procedures in a manner consistent  with TSA regulations. With alternative procedures, air carriers can deviate  from their TSA-approved security program while still meeting the intent of  TSA requirements. According to some IIRs with whom we spoke, these  alternative procedures are intended to provide a level of security that is  equivalent to the level of security provided by TSA’s standard  requirements while also affording air carriers with some flexibility in how  they achieve the intended security benefit of the TSA requirement.  Alternative security procedures are reviewed by the IIR, who submits  them to TSA headquarters and field officials for final review and approval.

TSA Has Taken Steps to Leverage Information for Capacity Development, but Could Enhance Data Management

Leveraging Information for Capacity Development

TSA has taken a number of steps to strengthen its analytical processes  and better understand the impact of the foreign airport assessment and  air carrier inspection programs. According to OGS officials, the  establishment and evolution of the ARM Directorate has facilitated better  data analysis and enhanced decision making pertaining to capacity  development. Specifically, TSA now conducts regional strategy meetings,  produces regional risk reports, and approves requests for assistance  based on risk.

OGS conducts regional strategy meetings. Since fiscal year 2012,  OGS has held strategy meetings to address aviation security threats and  vulnerabilities within each region. During these meetings, OGS officials  examine trend data for both airport assessments and air carrier  inspections, including vulnerability ratings over a multi-year period,  identify common areas of non-compliance, and develop capacity building  approaches customized to each region. According to agency  documentation, these meetings led OGS to recognize that each  geographic region faces its own particular challenges and risks and  requires unique mitigation approaches, such as at the country or airport  level.

ARM develops regional risk reports. In 2016, the ARM Directorate  began producing regional risk reports for use by other teams within OGS.  The purpose of these reports is to provide OGS personnel operating  within each of the four regions with an understanding of known  vulnerabilities in the region and their associated risk in order to inform  mitigation planning efforts. These reports include such information as key  risks at each location and region-wide trends on vulnerabilities. For  example, the reports show patterns in noncompliance related to critical  ICAO standards. In addition, the reports compare airports by risk level  and examine how individual airports compare to a regional average.  According to ARM staff, one of the top priorities this year is to centralize  analysis results within a web portal that allows users across OGS to sort  and filter data. ARM expects the portal to include comprehensive airport  profiles that capture the primary details for each location, such as the  largest carriers and main risks.

OGS approves requests for assistance based on risk. Requests for  capacity development assistance are submitted by OGS personnel,  including TSARs and inspectors. TSA’s Capacity Development Branch  (CDB) in ARM assesses these requests according to a standardized  criterion that includes an airport’s past and present vulnerabilities, the root  causes of these vulnerabilities, the timing of the assistance delivery, and  the suitability of the intended recipient. For instance, TSA assesses the  capabilities of the government or airport that would receive the  assistance, and considers such factors as whether the intended recipient  has the commitment necessary to institutionalize TSA-sponsored training  and the technical expertise to use any equipment that may be loaned or  donated by TSA. In addition, according to TSA officials, TSA considers  the extent to which the intended recipient has been a cooperative partner  in the past and implemented TSA’s previous security recommendations.  After CDB’s risk-based assessment of assistance requests, OGS  management makes a final determination regarding the provision of  assistance.

While TSA has taken steps to leverage the results of foreign airport  assessments and air carrier inspections to monitor system-wide  vulnerabilities and inform capacity development, TSA lacks key  information for decision making. For instance, we found that the Open  Standards and Recommended Practices Findings Tool (OSFT) — a  database for tracking the resolution status of identified foreign airport  deficiencies — has gaps and its system for categorization does not result  in sufficient specificity of information related to security deficiencies’ root  causes and corrective actions.

Root causes represent the underlying reason why an airport is not  meeting an ICAO standard and, according to TSA documentation, fall into  three general categories: lack of knowledge, lack of infrastructure, and  lack of will. For example, a foreign airport might fail to meet an ICAO  standard because of lack of knowledge stemming from insufficient  training programs or a high rate of staff turnover. According to OGS  officials, an understanding of root causes is important because the  challenges to addressing security deficiencies at foreign airports vary  extensively from country to country and corrective actions need to be  tailored to addressing the unique root causes of deficiencies that TSA  identifies. Corrective actions are efforts to mitigate security deficiencies  and might include training and other capacity building efforts. Corrective  actions can be designed to help a foreign airport add a new security  capability, enhance an existing capability, or increase the deployment of  security measures.

Although root causes and corrective actions are important variables for  decision making, we found that the OSFT has gaps in this information.  TSARs—the primary liaisons between the U.S. government and foreign  governments on transportation security issues—are responsible for  following up on progress made by foreign officials in addressing security  deficiencies identified during TSA assessments. Specifically, the Foreign  Airport Assessment Program SOP states that, for each foreign airport  assessed, the assigned TSAR is responsible for entering and updating  key information in the OSFT, including root cause and corrective action  information. According to the SOP, a thorough understanding of the  underlying reasons for each deficiency is critical to selecting the  appropriate mitigation activities. However, we found that around two  thirds of fiscal year 2016 records in the OSFT exhibited empty fields  pertaining to root cause or recommended corrective action. More  specifically, root cause data and recommended corrective action data  were each not recorded for 70 percent of findings.

During our interviews with TSARs, half (4 out of 8) indicated that they  believed the OSFT to be a cumbersome tool that has limitations for  recording status updates, among other issues, or that they preferred to  use other mechanisms, such as spreadsheets stored locally, in order to  avoid using the OSFT for certain functions. TSA headquarters officials  indicated that OGS began requiring staff to record root cause and  corrective action information in 2015 and that institutionalizing this  requirement to facilitate consistent data entry will take time. However,  complete data on root causes and corrective actions would help TSA  systematically monitor airport performance in addressing deficiencies and  leverage information for decision making regarding capacity development.  For example, with complete information TSA would be in a better position  to determine the extent to which airports were able to effectively close  security vulnerabilities based on TSA’s capacity building efforts, as well as conduct trend analysis within and across its four regions, including  identifying potential linkages between root causes and corrective actions.  Specifically, TSA could determine the extent to which corrective actions  seem to align best with certain root causes. For example, while training  might be an appropriate remedy if foreign airport personnel lack  knowledge, it might not be an appropriate solution for lack of will.

We also found that the OSFT has limitations related to the categorization  of root causes and corrective actions. The Foreign Airport Assessment  Program SOP indicates that root causes may relate to three broad  categories, as explained earlier, and twelve subcategories: aviation  security infrastructure, communication, cultural factors, human factors,  management systems, physical infrastructure, procedures, quality control,  resources, supervision, technology, and training. However, the OSFT  does not include a field to categorize root causes according to these  subcategories or other more specific areas. As a result, it does not  capture more granular information that would better explain the specific  root cause of an identified security issue.

Moreover, information on recommended corrective actions is stored  entirely in OSFT narrative fields without a drop-down list or other type of  categorization mechanism. For example, according to OSFT data, in one  Western Hemisphere region country, inspectors observed insufficient  employee screening and access control. The recommended corrective  action—”Fencing around the terminal area will be enhanced and airport  personnel counseled about employee screening”—would be difficult to  include in quantitative analysis without manual intervention. The OSFT  also includes a field for the final corrective action—how an airport  ultimately resolved a security issue. However, the categories in the OSFT  for final corrective action do not account for many key types of TSA’s  mitigation efforts (e.g., training, loaning or donating equipment, and  directing an air carrier to mitigate an airport vulnerability). Specifically, for  fiscal year 2016, we found that the OSFT only included data for three  high-level categories of final corrective actions: “airport authorities  resolved,” “national authorities resolved,” and “other.”

ARM staff stated that they recognize that the classification of data  currently contained in the OSFT could be improved, but that they have  not had an opportunity to address the issues because they have been  focused on developing the newest release of GRADS. TSA staff also  indicated that they are exploring opportunities to better classify data in  future releases of GRADS. However, according to the Foreign Airport  Assessment Program SOP, a thorough understanding of the underlying  reasons for each deficiency is critical to properly selecting the appropriate  mitigation activities. Moreover, federal internal control standards suggest  that agencies should design information systems to obtain and process  information to meet each operational process’s data requirements and to  respond to the entity’s objectives and risks. By classifying information  on root causes and corrective actions with additional specificity, and  through a standard system of categorization that would allow for system- wide analysis, TSA would be better positioned to assure that corrective  actions accurately address the specific, underlying reasons for security  vulnerabilities.

TSA’s foreign airport assessment and air carrier inspection programs play  a vital role in ensuring the security of the aviation system. TSA has taken  a number of steps to enhance foreign airport assessments and air carrier  inspections since 2011, including targeting resources based on risk,  strengthening access to foreign airports and the comprehensiveness of its  assessments and inspections, and creating operational efficiencies. While  TSA does not have authority to impose or otherwise enforce security  requirements at foreign airports, the agency makes a concerted effort to  help foreign airports improve their security posture and address security  deficiencies identified during assessments. Moreover, TSA is commonly  able to resolve air carrier security deficiencies with on-the-spot  counseling.

While TSA uses various mechanisms for capacity building, better data  management would help strengthen analysis and decision making.  Specifically, fully capturing and more specifically categorizing data on the  root causes of security deficiencies that TSA identifies and the associated  corrective actions would provide the agency with a more comprehensive  understanding of the security environment at foreign airports. For  example, TSA could leverage this information for trend analysis, including  evaluating potential linkages between root causes and corrective actions,  and determining the extent to which airports that received specific types  of capacity development services were able to close security  vulnerabilities. Accordingly, TSA would have better visibility over the  different types of capacity development that the agency offers and the  overall return on investment for these efforts.

We are making the following two recommendations to TSA:

The Assistant Administrator for the Office of Global Strategies should  ensure that data regarding the root causes of security deficiencies  and corrective actions are consistently captured in accordance with  TSA guidance. (Recommendation 1)

The Assistant Administrator for the Office of Global Strategies should  update TSA’s data systems to include more specific categories for  TSA’s data on the root causes and corrective actions related to  security deficiencies. (Recommendation 2)

We provided a draft of our report to DHS for its review and comment.  DHS provided written comments, which are noted below and reproduced  in full in appendix II. DHS concurred with both recommendations in the  report and described actions underway or planned to address them. With  regard to the first recommendation that TSA ensure that data regarding  the root causes of security deficiencies and corrective actions are  consistently captured in accordance with TSA guidance, DHS concurred  and stated that TSA will use a new tool, the Vulnerability Resolution Tool  (VRT), to capture and categorize root causes and corrective actions.  During the next fiscal year, TSA plans to train its staff in the use and  importance of the VRT, and estimates that it will complete this process by  October 31, 2018. If TSA consistently captures root causes and corrective  actions in the new tool, TSA’s planned actions would address the intent of  the recommendation. With regard to the second recommendation that  TSA update TSA’s data systems to include more specific categories for  TSA’s data on the root causes and corrective actions related to security  deficiencies, DHS concurred and stated that TSA plans to include more  specific categories for root causes and corrective actions in a future  iteration of GRADS, and expects to complete the updates by October 31,  2018. If fully implemented, these actions should address the intent of the  recommendation.

We are sending copies of this report to interested congressional  committees and the Secretary of Homeland Security, the Secretary of  State, the Administrator of the Transportation Security Administration, and  the TSA Assistant Administrator for the Office of Global Strategies. In  addition, the report is available at no charge on the GAO website at  http://gao.gov.

If you or your staff members have any questions about this report, please  contact Jennifer Grover at (202) 512-7141 or groverj@gao.gov, or  Jessica Farb at (202) 512-6991 or farbj@gao.gov. Key contributors to this  report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

To collectively address all three objectives, we reviewed the relevant laws  and regulations pursuant to which TSA conducts foreign airport  assessments and air carrier inspections. We reviewed various TSA  documents on program management and strategic planning and  interviewed TSA officials located at TSA headquarters and in the field.  We interviewed other federal and nonfederal stakeholders, such as the  Department of State (State), the European Commission (EC), and airport  and air carrier representatives. We outline the specific steps taken to  answer each objective below.

To obtain a greater understanding of the foreign airport assessment and  air carrier inspection processes, including how TSA works with host  nation officials and air carrier representatives, we accompanied a team of  TSA inspectors during an air carrier inspection at an airport in Europe.  We based our site selection on several factors, including the air carrier  locations TSA had plans to inspect during the course of our audit work  and host government willingness to allow us to accompany TSA. In  addition, we spoke with officials at a separate European airport, including  the airport operator and representatives from two air carriers.

To understand how TSA assesses and manages its foreign airport and air  carrier risk information, we obtained and reviewed documents on TSA’s  methodology for assigning individual risk rankings (called tier rankings) to  each foreign airport it assesses. TSA’s rankings are based on the  likelihood of a location being targeted, the protective measures in place at  that location, and the potential impact of an attack on the international  transportation system. Airports are then categorized as high, medium, or  low risk. We also reviewed TSA’s methodology for grouping air carriers  based on risk, which is influenced by the foreign airport risk tiers.

To describe the steps that TSA has taken to enhance foreign airport  assessments and air carrier inspections since 2011, we reviewed various  TSA documents on program management and strategic planning.  Specifically, we reviewed TSA’s 2016 Foreign Airport Assessment  Program Standard Operating Procedures (SOP), which prescribes  program and operational guidance for assessing security measures at  foreign airports, and informs TSA personnel at all levels of what is  expected of them in the implementation of the program. We also reviewed  the job aids that TSA inspectors use during each assessment and  inspection, which ensure that the TSA-specified International Civil  Aviation Organization (ICAO) aviation security standards and  recommended practices (referred to collectively in this report as ICAO  standards unless otherwise noted) and air carrier security program  requirements are fully evaluated during each assessment. In addition, we  reviewed TSA’s Office of Global Strategies (OGS) Strategic Plan for fiscal  years 2014 through 2018, and documents describing changes to the OGS  organizational structure since 2011.

To obtain stakeholder views and perspectives on steps TSA has taken to  enhance its foreign airport assessment program since 2011, we  interviewed and obtained information from various federal stakeholders.  Specifically, we interviewed OGS officials located in the Global  Compliance (GC), Global Affairs, and Analysis and Risk Mitigation (ARM)  directorates. In addition, we also conducted site visits to three of the six  TSA regional operations centers (ROC), located in Reston, Miami, and  Frankfurt, where we met with ROC managers, transportation security  specialists (henceforth referred to as inspectors) who conduct TSA’s  foreign airport assessments and air carrier inspections, TSARs who follow  up on host governments’ progress in addressing identified security  deficiencies, international industry representatives (IIR) who liaise with air  carriers, and regional directors (RD). We based our site visit selections on  the number and type of staff available at each location and geographic  dispersion. We also conducted telephone interviews with personnel from  the Honolulu ROC and other OGS staff stationed worldwide. In total, we  interviewed 4 of the 6 ROC managers, 19 of the 94 inspectors, 8 of the 29  TSARs, 8 of the 16 IIRs, and all 4 RDs. During these interviews, we  discussed these officials’ responsibilities related to the assessment and  inspection programs.

To describe the results of TSA’s foreign airport assessments and air  carrier inspections, we interviewed TSA officials on the results of its  evaluations, obtained and reviewed relevant program documents, and  conducted our own independent analysis of TSA’s assessment and  inspection results. Specifically, we obtained and reviewed TSA’s foreign  airport assessment program vulnerability results tracking sheet used by  GC to compile and track current and prior-year assessment results. This  tracking sheet included records of TSA’s compliance assessments for  each airport that TSA assessed from fiscal years 2012 through 2016.  Specifically, the tracking sheet recorded assessment results for each of  the ICAO standards used in the airport assessments, as well as an  overall vulnerability score of 1 through 5 assigned after each assessment.  This overall vulnerability score is a representation of compliance or  noncompliance with all the ICAO standards against which TSA assesses  foreign airports. We interviewed OGS officials on the steps taken to  develop the tracking sheet, including how TSA manages and updates  data. In addition, we conducted our own independent analysis of TSA’s  assessment results from fiscal years 2012 through 2016, the five-year  period since our previous review. Specifically, we analyzed data from  TSA’s foreign airport assessment program vulnerability results tracking  sheet to identify the number of airports in each vulnerability category by  region. We also analyzed TSA assessment results data to determine the  frequency with which foreign airports complied with particular ICAO  standards, such as access control, quality control, passenger screening,  and baggage screening, among others.

For air carrier inspection results, we analyzed data from PARIS on each  air carrier that TSA inspected from fiscal years 2012 through 2016. Our  analysis included the overall level of compliance, as well as the frequency  with which each air carrier complied with particular security program  requirements, such as aircraft search and passenger screening. We also  interviewed TSA managers, inspectors, and TSARs about their roles and  responsibilities in determining and documenting assessment and  inspection results. To assess the reliability of TSA’s assessment and  inspection data, we reviewed program documentation on system controls,  interviewed knowledgeable officials from OGS and checked TSA’s data  for any potential gaps and errors. Based on our overall analysis of the  data, we determined that the data were sufficiently reliable to provide a  general indication, by type or category, of the standards TSA assesses  against and the level of compliance, and frequency of compliance, for  TSA’s foreign airport assessments and air carrier inspections over the  period of our analysis.

To examine the steps TSA takes to address deficiencies identified during  foreign airport assessments and air carrier inspections, we interviewed  ARM and other TSA staff. Specifically, we discussed the full range of  options that are available to TSA for addressing airport and air carrier  security deficiencies, including a variety of capacity development tools  and collaboration with domestic agencies, such as State, and foreign  partners, such as Australia, Canada, Chile, New Zealand, Singapore,  South Africa, and the United Kingdom. During these interviews, we  discussed the circumstances in which each option is typically used and  the factors determining when an option is used. We also reviewed  program management tools TSA uses to track and manage the status of  foreign airport security deficiencies and records pertaining to capacity  development assistance deliveries from fiscal years 2012 through 2016,  including equipment loaned or donated, training courses provided, and  technical assistance delivered.

To obtain information on the extent to which TSA provided oversight of its  assessment and inspection efforts, we obtained and reviewed various  TSA program management documents and tools that TSA uses to track  and manage information for the programs. Specifically, we reviewed the  fiscal year 2017 Global Compliance Master Work Plan, which TSA uses  to track its foreign airport assessment schedule, including when various  airports are due to be assessed. We also reviewed the Open Standards  and Recommended Practices Findings Tool, which the TSA  Representatives (TSAR) use to monitor and track a foreign airport’s  progress in resolving security deficiencies identified by TSA inspectors  during previous assessments. In addition, we reviewed the tracking sheet  TSA uses to compile and track airport assessment results, including  individual airport vulnerability scores and information on which specific  ICAO standards were in noncompliance. Finally, we reviewed the results  of air carrier inspections that are contained in the inspections and  investigations modules of TSA’s Performance and Results Information  System (PARIS).

To identify challenges affecting TSA’s foreign airport assessment  program, we interviewed TSA officials, such as TSA’s Director of Global  Compliance, and field officials located at the TSA ROCs about the  challenges they experience obtaining access to foreign airports to  conduct assessments, the performance of data management systems,  and the provision of aviation security capacity development assistance to  foreign governments. We also obtained their perspectives on foreign  governments that have been reluctant to allow TSA inspectors to visit  their airports. We also interviewed TSA’s Director of Global Compliance  and headquarters and field staff on the agency’s use of databases and  other tracking mechanisms to manage assessment and inspection  results. In addition, we obtained the perspective of TSARs on challenges  to ensuring that foreign airports address security deficiencies. We also  interviewed officials within TSA’s Capacity Development Branch to better  understand the scope and types of requests for assistance that they  receive from foreign countries, the challenges that they experience in  attempting to provide assistance, and their experience collaborating with  State.

We met with State officials to better understand how they coordinate with  TSA through their Office of Anti-Terrorism Assistance and other related  efforts aimed at assisting foreign partners’ capacity to secure their  airports. In addition, we met with officials from the EC and the  International Air Transport Association to discuss efforts and programs  these organizations have in place to enhance international aviation  security.

In addition, during our interviews with ARM staff, we discussed the extent  to which TSA uses information at its disposal to inform capacity  development efforts. We also compared these efforts to criteria for  obtaining and processing information in federal internal control  standards. To identify opportunities for TSA to better leverage  information to inform capacity development, we reviewed relevant  program management documentation and tools that TSA uses to track  and analyze assessment results. Specifically, we reviewed the 2016  Foreign Airport Assessment Program SOP and program management  tools TSA uses to track and manage the status of foreign airport security  deficiencies. We also reviewed our prior work concerning how risk- informed and priority driven decisions can help inform agency decision  makers in allocating finite resources to the areas of greatest need.

Information from our interviews with government officials and members of  the aviation industry provide insight into their perspectives on TSA’s  foreign airport assessment and air carrier inspection programs. However,  this information cannot be generalized beyond those with whom we spoke  because we did not use statistical sampling techniques in selecting  individuals to interview.

Appendix II: Comments from the Department of Homeland Security

Appendix III: GAO Contacts and Staff Acknowledgments

In addition to the contacts above, Jason Bair and Chris Ferencik  (Assistant Directors); Anthony C. Fernandez (Analyst-in-Charge); Bryan  Bourgault; Elizabeth Dretsch; Jesse Elrod; Eric Hauswirth; Christopher  Lee; Tom Lombardi; Amanda Miller; and Adam Vogt made key  contributions to this report.