District of Columbia: Improved Reporting Could Enhance Management of the Tuition Assistance Grant Program

Congress funds DCTAG through an annual appropriation, which was $40 million in fiscal year 2018. DCTAG provides D.C. residents up to $10,000 per year to attend college.

The Consolidated Appropriations Act, 2017, included a provision for GAO to review DCTAG. This report examines, among other things, the characteristics of DCTAG recipients and steps taken by the program to support recipients, as well as the extent to which OSSE reports DCTAG's performance to internal and external stakeholders.

GAO assessed the most recent data available on DCTAG, covering academic years 2007–2016, as well as data on college graduation, tuition, and fees from the Department of Education's Integrated Postsecondary Education Data System for academic years 2007–2016, and data on enrollment in high schools and median household income in D.C. from the U.S. Census Bureau's American Community Survey for 2007–2016; interviewed representatives of DCTAG and the entities it partners with to support recipients; and reviewed relevant laws, the applicability of standards for internal control, and guidance on performance management.

The federally funded District of Columbia Tuition Assistance Grant (DCTAG) program was created in 1999 to give college-bound District of Columbia (D.C.) residents greater choices among institutions of higher education. Since its creation, the DCTAG program has awarded over $440 million to more than 26,000 residents to defray costs charged to out-of-state residents at some of the nation's public colleges and universities. While the program serves students from families with a wide range of household incomes, about half the students receiving a DCTAG award in academic years 2007–2016 came from the three D.C. wards with the lowest household incomes, as the figure below illustrates. DCTAG coordinates with public and private partners in the community to help students prepare for college, complete financial aid applications, and stay on track to graduate college.

Although the Office of the State Superintendent of Education (OSSE), which manages DCTAG on behalf of the Mayor of the District of Columbia, issues various annual reports, these reports do not relate program performance to the program's four goals. One of these goals is to help D.C. students make smarter college choices. OSSE officials stated that they regularly communicate information about DCTAG data and activities internally and externally. However, these efforts do not provide the context necessary for program managers, Congress, or the public to understand the program's goals, nor determine whether DCTAG is making progress toward meeting them. Standards for internal control state that program managers should communicate information that internal and external stakeholders need to help the program achieve its objectives. Absent an annual report relating performance to goals, DCTAG's stakeholders will be limited in their ability to assess the program's performance or identify opportunities to improve it.

GAO recommends OSSE issue annual reports relating DCTAG's performance to program goals. In response to the recommendation, the Mayor stated that OSSE will expand annual reporting to include direct linkages and combine data points to better illustrate the program's performance.

Federal law created the federally funded District of Columbia Tuition  Assistance Grant (DCTAG) program in 1999 to give college-bound  District of Columbia (D.C.) residents more choices among colleges and  universities. Since its creation, DCTAG has awarded over $440 million to  more than 26,000 college-bound D.C. residents. The grants can help  defray some costs associated with recipients’ postsecondary education.  For example, D.C. residents who attend public colleges and universities  in other states may pay higher “out-of-state” tuition.  States generally  provide financial support to their state university systems and can require  public institutions to offer residents lower, “in-state” tuition. In contrast to  these state university systems, which can include dozens of institutions  serving hundreds of thousands of students, D.C. has a single public  university. In fiscal year 2018, Congress appropriated $40 million to fund  DCTAG. While Congress funds DCTAG through annual appropriations,  the Mayor of the District of Columbia manages the program through the  Office of the State Superintendent of Education (OSSE).

The Consolidated Appropriations Act, 2017, includes a provision for GAO  to provide information about the performance of the DCTAG program.  Specifically, the provision sought information on trends in eligibility,  enrollment, performance, and outcomes, and asked for information on  steps taken to provide support to current participants. In addition, the  provision sought information on scholarship programs offered by other  municipalities in the United States, including a comparison of participant  requirements, outcomes, and funding sources.

In this report, we reviewed DCTAG and other selected state and local  scholarship programs to:  1. Describe trends in the eligible population, enrollment, and college graduation rates for DCTAG.

Identify steps taken under the program to support recipients and address any challenges they may face staying on track to graduate. 3. Assess the extent to which DCTAG’s performance is reported to program managers, Congress, and the public. 4. Describe other state and local scholarship programs with regard to participant eligibility, funding sources, steps to support recipients, and program outcomes.

To describe trends in DCTAG, we reviewed the most recent program data  available, including data on enrollment in DCTAG for academic years  2007–2016 and 6-year graduation rates for recipients in academic years  2012−2015. To assess the reliability of program data, we interviewed  OSSE officials about the processes used to produce these data and  reviewed the programming codes used to extract them from OSSE’s data  system. We did not independently verify recipients’ eligibility for DCTAG.  We also used American Community Survey (ACS) data to produce  estimates of D.C. household income and enrollment in D.C. high schools  for calendar years 2007−2016. To assess the reliability of these  estimates, we reviewed technical documentation for ACS. Because ACS  is based on samples and therefore subject to sampling error, we present  these estimates with their associated 95 percent confidence intervals.  Lastly, to provide context for trends in DCTAG, we reviewed data on  average tuition and required fees at 4-year public institutions for  academic years 2007–2016 and average 6-year graduation rates for  certain nationwide groups of students in academic years 2012–2016 from  the Integrated Postsecondary Education Data System (IPEDS). To  assess their reliability, we reviewed technical documentation on IPEDS.  We determined that data from these three sources were sufficiently  reliable for our purposes.

To identify steps taken to support recipients, we reviewed documents  from OSSE describing support services offered to students either directly  by DCTAG or by other entities within OSSE. For example, we reviewed  documents describing services provided by OSSE’s Office of College and  Career Readiness, which promotes college access and graduation for  D.C. students. Additionally, we interviewed officials from OSSE and the  stakeholders they partner with, including public and private school  officials and representatives of college access providers, which generally  are organizations that seek to expand access to higher education. These  organizations included the D.C. College Access Program, a privately  funded scholarship program, and the federally funded D.C. Educational  Opportunity Center, which provides D.C. residents information, guidance,  and counseling on opportunities to pursue education beyond high school.

We also reviewed program documents for these partners to determine  how DCTAG works through its partners to offer support services that can  help recipients and potential applicants prepare for college, apply for  financial aid, and stay on track to graduate college.

To assess the extent to which DCTAG’s performance is reported, we  interviewed OSSE officials and reviewed program documents and  relevant federal laws. We used standards for internal control and  information on performance management as criteria against which to  assess these efforts. We also coordinated with the D.C. Office of the  Inspector General to understand key management practices and  requirements for D.C. government agencies. We did not assess the  program’s compliance with legal reporting requirements.

To describe other state and local scholarship programs, we reviewed a  non-generalizable sample of three programs that target students in two  municipalities and one state: the Boston Tuition-Free Community College  Plan, Boston, Massachusetts; the Kalamazoo Promise, Kalamazoo,  Michigan; and the Washington State Opportunity Scholarship,  Washington. We selected programs to reflect diversity in program design  and geography. We reviewed key program documents and interviewed  managers of each program. We did not conduct an independent review of  relevant state and local laws, regulations, and municipal orders; however,  we coordinated with state and local audit offices as appropriate.

We conducted this performance audit from October 2017 to September  2018 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

When DCTAG was created, there was no income eligibility requirement.  However, in 2007, federal law limited eligibility to students from families  with annual taxable incomes less than $1,000,000. In 2015, federal law  further limited eligibility to students from families with annual taxable  incomes less than $750,000; the law provided that this limit was to be  subsequently adjusted for inflation as measured by the percentage  increase, if any, in the Consumer Price Index for All Urban Consumers.  For example, in academic year 2018, eligibility was limited to students  from families with annual taxable incomes less than $762,000 (see  textbox for selected eligibility requirements).

Complete the Free Application for Federal

Generally begin a course of study within 3  years of graduating high school or  obtaining a General Equivalency Diploma. institution’s requirements for Satisfactory  Academic Progress.

Universities (HBCU) nationwide and other participating private nonprofit  institutions in the D.C. metropolitan area.

Populations Eligible for and Enrolled in DCTAG Remained Relatively Stable as Amounts Awarded Increased and Recipients Graduated at Higher Rates than Selected National Comparison Groups

We identified the following trends in eligibility for and enrollment in  DCTAG and graduation rates for recipients:

DCTAG’s potentially eligible population. ACS data for calendar  years 2007−2016 indicate that the population of high school students  with incomes within DCTAG’s eligibility requirements has remained  relatively stable. Over this time frame, about 25,000 students in D.C.  were enrolled in high school each year, and about 90 percent of D.C.  households had annual incomes less than $200,000. Additional  households with annual incomes of $200,000 and above were also  likely eligible for DCTAG based on income.

Enrollment in DCTAG. DCTAG program data indicate that the  number of DCTAG recipients remained relatively stable over the last  decade. DCTAG provided awards to an average of about 4,750  recipients annually over academic years 2007−2016 (see fig. 1).  While enrollment in DCTAG peaked in academic year 2012, the  number of DCTAG recipients in academic year 2016, the last year in  our period of review, was similar to the number of recipients in  academic year 2007, the first year in our period of review.

Enrollment in DCTAG by type of high school attended. DCTAG  program data indicate the majority of recipients over academic years  2007–2016 graduated from D.C.’s public high school system—both  traditional public schools and public charter schools. D.C.’s traditional  public schools include six selective schools, or magnet schools, that  limit admission to students that meet certain criteria or eligibility  requirements. For example, in academic year 2016, more than 70  percent of DCTAG recipients graduated from D.C.’s public high school  system (see fig. 2). Many DCTAG recipients have also graduated  from private schools or schools outside D.C., were home schooled, or  attained their General Equivalency Diploma. For academic years  2007−2016, between about 30 and 40 percent of DCTAG recipients  came from high schools or programs outside the D.C. public school  system.

Enrollment in DCTAG by taxable household income. Although in  2007 federal law limited eligibility for DCTAG to students from families  with annual taxable incomes less than $1,000,000, DCTAG  enrollment data show the program made awards to students from  families with a wide range of household taxable incomes in academic  years 2009−2016. At the same time, enrollment data indicate the  program’s particular support for students from middle and lower  income families. Nearly 60 percent of recipients over this time frame  came from families with annual household taxable incomes of  $50,000 or less (see fig. 3).

Enrollment in DCTAG by Ward. DCTAG program data indicate that  for academic years 2007−2016, about 50 percent of recipients came  from the three D.C. wards with the lowest median household incomes,  according to American Community Survey estimates (see fig. 4).

Enrollment in DCTAG by attendance at 4-year and 2-year  institutions. DCTAG program data show that for academic years  2007−2016, about 90 percent of DCTAG recipients attended 4-year  institutions (see fig. 5). To counter the downward trend in enrollment  at 2-year institutions that began in academic year 2013, OSSE  officials told us they made programmatic changes to DCTAG for  academic year 2018. Specifically, OSSE officials told us they  determined out-of-state-tuition at 2-year public institutions attended by  DCTAG recipients exceeded in-state tuition by an average of $4,500  per year. However, the maximum annual award for recipients  attending these institutions was only $2,500. For academic year 2018,  OSSE officials said they increased the maximum annual award to  attend 2-year public institutions to $10,000 to close this gap.

Enrollment in DCTAG by amount awarded. For academic years  2007−2016, the percentage of recipients receiving DCTAG’s  maximum annual awards increased from 40 percent to 62 percent  (see fig. 6). OSSE officials linked an increase in the percentage of  recipients receiving maximum awards to rising tuition at colleges and  universities over this period. We analyzed data from IPEDS on  average tuition and required fees at 4-year public institutions and our  analysis confirmed that the average gap between out-of-state and in- state tuition exceeded DCTAG’s $10,000 maximum annual award  starting in academic year 2015.

DCTAG graduation rates. College graduation rates are an important  measure of performance for DCTAG. OSSE officials told us they  maintain a program goal of helping recipients choose schools from  which they are likely to graduate. For academic years 2012−2015, 6- year college graduation rates for DCTAG recipients were lower than  those for students nationwide. However, OSSE officials reported  that rates for recipients compare favorably to rates for national and  regional groups of students with characteristics similar to those of  DCTAG recipients. Our analysis confirmed that in academic year  2015, about 72 percent of DCTAG recipients were African-American  and the DCTAG graduation rate was about 10 percentage points  higher than for African-Americans nationwide. Similarly, in academic  year 2015, nearly 40 percent of DCTAG recipients attended  Historically Black Colleges and Universities (HBCU) and the DCTAG  graduation rate was about 15 percentage points higher than for the  nationwide population of students at these schools (see fig. 7).  Additionally, OSSE officials estimated that more than 65 percent of  DCTAG recipients were eligible for Pell Grants in academic year  2016. The National Center for Education Statistics recently started  reporting graduation rates for Pell Grant recipients, beginning with the  cohort of recipients that should have graduated by academic year  2016. Although not directly aligned, the academic year 2016  graduation rate for Pell Grant recipients nationwide was 48 percent— similar to the academic year 2015 graduation rate for DCTAG  recipients.

DCTAG and Its Partners Help Recipients Prepare for College, Complete Applications for Financial Aid, and Stay on Track to Graduate

DCTAG partners with other entities to offer support services intended to  help D.C. students prepare for college, apply for financial aid, and stay on  track to graduate college. These partners include other entities within  OSSE, as well as partners in the broader community such as public and  private high school officials and college access providers. DCTAG  provides some support services directly to students, such as individual  counseling on how to complete a DCTAG application (see fig. 8). An  OSSE official told us that DCTAG counselors instruct applicants and  renewing recipients on tasks such as how to obtain required supporting  documents to verify their residency in D.C. Additionally, to keep recipients  on track to graduate, DCTAG emails recipients a quarterly newsletter with  reminders to reapply for DCTAG and federal student aid so that they do  not disrupt their studies by losing financial assistance. OSSE officials also  said that DCTAG expands the reach of its support services by partnering  with other entities within OSSE and in the community. For example,  DCTAG works with OSSE’s Office of College and Career Readiness,  whose mission is to increase D.C. public school students’ access to  college. Through this collaboration, DCTAG helps eligible students  prepare for higher education, such as through assistance to public  schools to offer college entrance exams at no cost to students. Similarly,  by partnering with college access providers, DCTAG supplements the  support services it offers to help students stay on track to graduate. For  example, DCTAG partners with the D.C. College Access Program, a  privately funded scholarship program that offers support services for D.C.  students in college. One of their services includes using scholarship  recipients to mentor incoming D.C. students.

OSSE’s Reporting to Key Stakeholders on DCTAG Does Not Include Program Performance Information

We found that although OSSE communicates DCTAG’s program data  and activities to internal stakeholders, Congress, and the public in various  formats, these reporting methods do not include the program’s four goals  (see textbox), relate performance information to these program goals, or  describe progress toward achieving them (see table 1). For example,  OSSE’s 2017 annual report to Congress on DCTAG did not include  DCTAG’s four program goals, nor did OSSE relate information about the  performance of the program to those goals. Instead, the 2017 annual  report was comprised of descriptive statistics that were presented without  explanation or sufficient context to allow readers to understand the  significance of what was being reported. Specifically, this information was  unrelated—quantitatively or qualitatively—to DCTAG’s program goals of  ensuring D.C. residents are aware of and apply to DCTAG, or of helping  DCTAG students make smarter college choices, which OSSE officials  told us includes helping students select schools where they are more  likely to graduate. As a result, it is unclear how to interpret the information  presented in these reports and whether reported results indicate positive  or negative program performance.

Federal standards for internal control state that program managers should  communicate necessary quality information so both internal and external  parties can help the program achieve its objectives. We have previously  reported that annual reports are essential for managers of federal  programs to communicate to decision makers the progress an agency  has made toward achieving its goals during a given year and, in cases  where goals are not met, identify opportunities for improvement or  whether goals need to be adjusted. In addition, our prior work found that  managers of these programs can increase the value of their reports to  congressional decision makers and the public by relating annual  performance information to the agency’s strategic goals and mission.  Furthermore, we reported that performance measurement does not  require establishing a causal link between program activities and program  outcomes, but rather emphasizes that the nature of performance  measurement is the ongoing monitoring and reporting of program  accomplishments, particularly toward pre-established goals.

OSSE officials agreed on the importance of developing an annual report  relating performance to program goals for the DCTAG program and  concurred with our finding that they had not communicated DCTAG’s  performance information, such as progress toward program goals, in a  single annual report. They explained that developing performance  measures is challenging. For example, they said DCTAG recipients have  access to multiple support programs, which creates difficulties in  establishing causal links between a program and the desired outcome.  OSSE officials also stated that many DCTAG initiatives are new and, as a  result, complete data on those initiatives are not yet available. Although  we recognize that developing an annual report could be challenging, our  prior work has found performance measurement guidelines would not  require program managers to establish causal links as part of ongoing  performance monitoring and reporting of progress toward program goals.  Unless DCTAG’s stakeholders have access to an annual report that  relates performance information to the program’s goals, they may be  limited in their ability to judge the significance of what is being reported,  determine whether the agency is making progress toward achieving its  goals, or make informed program management and funding decisions.

The Design of Selected Scholarship Programs Reflects Unique State and Local Needs

Each of the three other selected scholarship programs we reviewed was  created to meet unique state or local needs.

Boston Tuition-Free Community College Plan. Created to make  college more affordable for the city’s low-income students.

Kalamazoo Promise. Created to promote the economic and social  well-being of the community by expanding college access with full- tuition scholarships.

Washington State Opportunity Scholarship. Created to address  shortfalls in the state’s Science, Technology, Engineering, and  Mathematics (STEM) and health care workforce and increase  educational opportunities for low-income and middle-income students.

Because each program was designed to address a unique state or local  need, they differ with regard to eligibility, funding, recipient supports, and  outcome measures. (For additional information on these three programs  see appendix I.)

Eligibility. Each of the three selected scholarship programs  established eligibility criteria, such as income requirements, residency  requirements, and grade point average (GPA) requirements among  others, that reflect program objectives. For example, to ensure that  the Boston program serves the intended low-income population, the  program requires students to be eligible for Pell grants to receive  funding.

Funding. While the selected scholarship programs have dedicated  funding streams, their funding sources reflect the origins of each  program. For example, Boston’s program was initiated by the city’s  mayor and is funded through a public charitable trust from fees for  large-scale commercial building projects while the Kalamazoo  promise was initiated by a group of anonymous donors who have  funded the program in perpetuity, according to program officials.  Alternately, the Washington program was initiated through  cooperation between the state government and private sector  companies and is funded by private donations that are matched by  state funds up to an annual maximum of $50 million.

Recipient supports. Each of the selected scholarship programs have  developed supports such as coaching and peer mentoring to help  recipients transition to college and stay on track to graduate. For  example, the Kalamazoo Promise partners with and provides funding  to two local colleges to create counseling, coaching, or peer  mentoring services for scholarship recipients, according to program  managers.

Outcome measures. The selected scholarship programs have  developed outcome measures to better understand the programs’  impact, such as whether students stay on track to graduate or find  employment post-graduation. For example, program managers with  the Washington program said they initiated a post-graduation survey  in 2015 to better understand the employment status of graduates in  STEM and health care fields, their job location, and annual salary.

Steady enrollment in DCTAG provides an encouraging signal that the  program may be meeting the purpose set forth in federal law to expand  access to higher education opportunities for D.C. students. However,  without annual reports that relate DCTAG’s performance information to  the program’s goals, it is difficult to assess the impact of the program and  its support services. The information OSSE currently makes available  about DCTAG does not provide the context needed for the program’s  internal stakeholders, Congress, or the public to determine whether the  program is meeting its goals or if any changes may be necessary.

OSSE should issue an annual report on DCTAG that relates information  about the program’s performance to the program’s goals.  (Recommendation 1)

We provided a draft of this report to the Mayor of the District of Columbia  for review and comment. Comments from the Mayor are reproduced in  appendix II. In response to our recommendation, the Mayor stated that  OSSE plans to expand DCTAG’s current annual reports to Congress with  direct links to DCTAG’s annual strategic performance goals and the  reports will combine data points to illustrate the program’s performance.  The Mayor also raised a concern about the title of the draft report, stating  that it implied OSSE is not meeting legislative requirements. We have  modified the title and text of the report to avoid this implication.

We are sending copies of this report to the appropriate congressional  committees, the District of Columbia Office of the State Superintendent of  Education, and other interested parties. In addition, this report is available  at no charge on the GAO website at http://www.gao.gov.

Appendix I: Other State and Local Scholarship Programs

Based on interviews with officials of three selected state and local  scholarship programs and a review of program documents, we present a  selection of information to provide additional context on these programs.  They include the Boston Tuition-Free Community College Plan, the  Kalamazoo Promise, and the Washington State Opportunity Scholarship.  The following tables include information on these scholarship programs’  eligibility requirements, funding sources, recipient supports, and annual  reports and performance measures. Table 2 presents a selection of  eligibility requirements for the Boston Tuition-Free Community College  Plan, the Kalamazoo Promise, and the Washington State Opportunity  Scholarship.

Table 3 presents a summary of the three selected scholarship programs’  funding sources, as well as how students may use those funds.

Table 4 presents a summary of the supports developed by the three  selected scholarship programs to support students, keep them on track to  graduate from college, and help them begin their careers.

Table 5 presents a summary of the annual reports and selected  performance measures developed by the three selected scholarship  programs.

Appendix II: Comments from the Mayor of the District of Columbia

Appendix III: GAO Contact and Staff Acknowledgments

Melissa Emrey-Arras, (617) 788-0534 or emreyarrasm@gao.gov.

In addition to the contact named above, Bill J. Keller (Assistant Director),  Tom Moscovitch (Analyst-in-Charge), and Michael C. Duane made  significant contributions. Also contributing to this report were James  Bennett, Deborah K. Bland, Sheila R. McCoy, Benjamin A. Sinoff, Rachel  R. Stoiko, and Kate van Gelder.


Federal Facility Security: Selected Agencies Should Improve Methods for Assessing and Monitoring Risk

Protecting federal employees and facilities from security threats is of critical importance. Most federal agencies are generally responsible for their facilities and have physical security programs to do so.

GAO was asked to examine how federal agencies assess facilities' security risks. This report examines: (1) how selected agencies' assessment methodologies align with the ISC's risk management standard for identifying necessary countermeasures and (2) what management challenges, if any, selected agencies reported facing in conducting physical security assessments and monitoring the results.

GAO selected four agencies—CBP, FAA, ARS, and the Forest Service—based on their large number of facilities and compared each agency's assessment methodology to the ISC Standard; analyzed facility assessment schedules and results from 2010 through 2016; and interviewed security officials. GAO also visited 13 facilities from these four agencies, selected based on geographical dispersion and their high risk level.

None of the four agencies GAO reviewed—U.S. Customs and Border Protection (CBP), the Federal Aviation Administration (FAA), the Agricultural Research Service (ARS), and the Forest Service—used security assessment methodologies that fully aligned with the Interagency Security Committee's Risk Management Process for Federal Facilities standard (the ISC Standard). This standard requires that methodologies used to identify necessary facility countermeasures—such as fences and closed-circuit televisions—must:

1. Consider all of the undesirable events (i.e., arson and vandalism) identified by the ISC Standard as possible risks to facilities.

2. Assess three factors—threats, vulnerabilities, and consequences—for each of these events and use these three factors to measure risk.

All four agencies used methodologies that included some ISC requirements when conducting assessments. CBP and FAA assessed vulnerabilities but not threats and consequences. ARS and the Forest Service assessed threats, vulnerabilities, and consequences, but did not use these factors to measure risk. In addition, the agencies considered many, but not all 33 undesirable events related to physical security as possible risks to their facilities. Agencies are taking steps to improve their methodologies. For example, ARS and the Forest Service now use a methodology that measures risk and plan to incorporate the methodology into policy. Although CBP and FAA have updated their methodologies, their policies do not require methodologies that fully align with the ISC standard. As a result, these agencies miss the opportunity for a more informed assessment of the risk to their facilities.

All four agencies reported facing management challenges in conducting physical security assessments or monitoring assessment results. Specifically, CBP, ARS, and the Forest Service have not met the ISC's required time frame of every 3 years for conducting assessments. For example, security specialists have not conducted required reassessments of two ARS and one Forest Service higher-level facilities. While these three agencies have plans to address backlogs, CBP's plan does not balance conducting risk assessments with other competing security priorities, such as updating its policy manual, and ARS and the Forest Service lack a means to monitor completion of future assessments. Furthermore, CBP, ARS, and the Forest Service did not have the data or information systems to monitor assessment schedules or the status of countermeasures at facilities, and their policies did not specify such data requirements. For example, ARS and the Forest Service do not collect and analyze security-related data, such as countermeasures' implementation. FAA does not routinely monitor the performance of its physical security program. Without improved monitoring, agencies are not well equipped to prioritize their highest security needs, may leave facilities' vulnerabilities unaddressed, and may not take corrective actions to meet physical security program objectives. This is a public version of a sensitive report that GAO issued in August 2017. Information that the agencies under review deemed sensitive has been omitted.

GAO recommends: (1) that CBP and FAA update policies to require the use of methodologies fully aligned with the ISC Standard; (2) that CBP revise its plan to eliminate the assessments backlog; and (3) that all four agencies improve monitoring of their physical security programs. All four agencies agreed with the respective recommendations.

Several incidents—such as armed citizens taking over a federal wildlife  refuge in Oregon for about 40 days in 2016; the active shooter incident at  the Washington Navy Yard in Washington, D.C., in 2013 that resulted in  several deaths; and the fatal shooting at the Anderson Federal Building in  Long Beach, California, in 2012—demonstrate that government facilities  and their employees continue to be targets of potential harm. In light of  these incidents and other emergent threats, it is important that agencies  use risk-based methodologies to assess the physical security needs of  the approximately 113,000 executive-branch, non-military federal  buildings. Security assessments of facilities can uncover vulnerabilities  and threats and recommend protective measures—called  countermeasures—such as fences, access control systems, and closed- circuit television systems to mitigate those threats.

At least 30 federal agencies are responsible for protecting about 45  percent of civilian federal facilities and their occupants from potential  threats. To help federal agencies protect and assess risks to their  facilities, the Interagency Security Committee (ISC), an organization  chaired by the Department of Homeland Security (DHS), developed  physical security standards for non-military federal facilities in the United  States.

One particular standard, called The Risk Management Process for  Federal Facilities, defines the criteria and process executive agencies  and departments must follow when assessing risks to their facilities.  However, our past work found that some federal agencies used this  standard to varying degrees leaving agencies’ facilities, workforce, and  visitors exposed to risk. You asked us to examine how federal agencies  with protective responsibilities use risk management to protect their  facilities with countermeasures that meet their security needs. This report  examines (1) how selected agencies’ assessment methodologies align  with the ISC risk management standard to identify necessary  countermeasures, and (2) what management challenges, if any, selected  agencies reported facing in conducting physical security assessments  and monitoring the results.

This product is a public version of a sensitive report that we issued in  August 2017. DHS deemed some of the information in our August report  to be sensitive, including information about facility locations, risk  assessment results, and undesirable events not assessed for federal  facilities physical security. Therefore, this report omits that information  which must be protected from public disclosure. Although the information  provided in this report is more limited, the report addresses the same  objectives as the sensitive report and uses the same methodology.

To address the objectives, we selected four agencies with responsibility  for assessing their own facilities—DHS’s U.S. Customs and Border  Protection (CBP); the Department of Transportation’s Federal Aviation  Administration (FAA); and the United States Department of Agriculture’s  (USDA) Agricultural Research Service (ARS) and Forest Service. To  determine how these agencies’ assessment methodologies align with The  Risk Management Process for Federal Facilities (the ISC Standard), we  compared facility security policies and procedures from the departments  and agencies to the criteria and process in the ISC Standard. In addition,  we selected 13 facilities within these 4 agencies for site visits based on  geographical dispersion and high levels of risk. For each of the selected  facilities, we reviewed assessment reports, toured the facilities, and  identified the status of recommended countermeasures. We interviewed  officials from the ISC, 3 departments, 4 agencies, and the 13 facilities to  understand security standards, policies, and procedures; agency-specific  assessment processes; management challenges; and guidance for  prioritizing physical security needs. We did not independently determine  what constitutes a management challenge, but relied on these facility  managers and agency security staff to identify their concerns as defined  in their own policies and procedures. We reviewed Standards for Internal  Control in the Federal Government (Standards for Internal Control)  regarding risk management criteria and the use of quality information for  our evaluation of the agencies’ abilities to monitor their physical security  program. The results of our review of the selected agencies are not  generalizable to all the ISC member agencies but provide illustrative  examples of risk assessments and how the facilities addressed needed  countermeasures. See appendix I for more details on our scope and  methodology and appendix II for a list of the 13 selected facilities visited.

The performance audit upon which this report is based was conducted  from June 2016 to August 2017 in accordance with generally accepted  government auditing standards. Those standards require that we plan  and perform the audit to obtain sufficient, appropriate, evidence to provide  a reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives. We  subsequently worked with DHS, DOT, and USDA from August 2017 to  October 2017 to prepare this version of the sensitive report for public  release. This public version was also prepared in accordance with these  standards.

Risk management, as applied to security of federal facilities, entails a  continuous process of applying a series of mitigating actions—assessing  risk through the evaluation of threats, vulnerabilities, and consequences;  responding to risks with appropriate countermeasures; and monitoring  risks using quality information (see fig. 1).

In 1995, Executive Order 12977 established the ISC after the bombing of  the Oklahoma City Alfred P. Murrah Federal Building in April 1995. The  ISC’s mandate is to enhance the quality and effectiveness of security in  and protection of federal facilities in the United States occupied by federal  employees for nonmilitary activities. The order directs the ISC to  develop and evaluate security standards for federal facilities, develop a  strategy to ensure executive agencies and departments comply with such  standards, and oversee the implementation of appropriate security  measures in federal facilities. The ISC has released a body of standards,  including the ISC Standard, designed to apply to the physical security  efforts of all federal, non-military agencies. The ISC Standard prescribes  a process for agencies to follow in developing their risk assessment  methodologies (see fig. 2).

Most federal departments and agencies are generally responsible for  protecting their own facilities and have physical security programs in  place to do so. The ISC Standard requires executive departments and  agencies to follow the risk-management process when conducting risk  assessments for each of their facilities. That process begins with  determining the facility security level, ranging from level I (lowest risk) for  facilities generally having 100 or fewer employees to level V (highest risk)  for the most critical facilities and generally having greater than 750  employees. The security level designation determines the facility’s baseline countermeasures. For each facility, departments and agencies  are required to (a) consider all of the “undesirable events” that could pose  a risk to their facilities— such as active shooters, vandalism, and  explosions—and (b) assess three factors of risk (threats, vulnerabilities,  and consequences) to specific undesirable events. Subsequently,  agencies are to combine all three factors to yield a measurable level of  risk for each undesirable event (see app. III). Based on the results of  these assessments, agencies should customize (either increase or  decrease) the countermeasures to adequately reflect the assessed level  of risk.

In addition, as part of planning for physical security resources within an  agency’s budget process, the ISC has identified the need to balance  allocations for countermeasures with other operational needs and with  competing priorities. The ISC Best Practices have some similarities with  leading practices in capital decision-making. For example, both state that  the allocation of resources should be integrated into the agency’s  mission, objectives, goals, and budget process. However, beyond the ISC  Best Practices, the Office of Management and Budget and we have  developed more comprehensive leading practices in capital decision- making that provide agencies with guidance for prioritizing budget  decisions such as for countermeasure projects. The Office of  Management and Budget and our guidance also emphasize evaluating a  full range of alternatives, informed by agency asset inventories that  contain condition information, to bridge any identified performance gap.  Furthermore, the guidance calls for a comprehensive decision-making  framework to review, rank, and select from among competing project  proposals. Such a framework should include the appropriate levels of  management review, and selections should be based on the use of  established criteria.

The following describes the mission and physical security program  characteristics for the agencies in our review:

CBP, the nation’s largest law enforcement agency, has responsibility  for securing the country’s borders. It also has responsibility for  conducting security assessments at about 1,200 facilities, including  approximately 215 federally owned and agency-controlled higher-level  facilities (facility security levels III and IV). These facilities include  border patrol stations with holding cells for people detained at the  border, office buildings, and canine-training centers. CBP conducts  these assessments.

FAA’s mission is to provide a safe and efficient aerospace system for  the country. According to agency data, FAA has 55 federally owned  and agency-controlled higher-level facilities—including critical air  traffic control towers. According to FAA officials, FAA specialists  conduct security assessments.

ARS conducts research related to agriculture and disseminates  information to ensure high-quality safe food and to sustain a  competitive agricultural economy. According to agency data, ARS has  security responsibility for four domestic federally owned and agency- controlled higher-level facilities—including laboratories for research to  improve food and crop quality, office buildings, and warehouses. ARS  security personnel have responsibility for conducting security  assessments.

The Forest Service sustains the health, diversity, and productivity of  the nation’s forests and grasslands. According to agency officials, the  Forest Service has one federally owned and agency-controlled higher- level facility—a regional headquarters office building. The Forest  Service’s security officials have responsibility for conducting security  assessments, but at the time of our review, USDA security officials  conducted the assessment at Forest Service’s one higher-level  facility.

Selected Agencies’ Assessment Methodologies Do Not Fully Align with the ISC’s Risk Management Standard

None of the four selected agencies’ security assessment methodologies  fully aligned with the ISC Standard. The ISC gives agencies some  flexibility to design their own security-assessment methodologies for  identifying necessary countermeasures as long as the chosen  methodology adheres to fundamental principles of a sound risk- management methodology. Specifically, methodologies must:  consider all of the undesirable events identified in the ISC Standard  as possible risks to federal facilities, and assess three factors of risk (threats, vulnerabilities, and  consequences) for each of the events.

Furthermore, the ISC Standard requires executive departments and  agencies to document decisions that deviate from the ISC Standard.  Agencies’ policies and methodologies reference the ISC Standard.  However, none of the agencies’ methodologies considered all of the  undesirable events during assessments although they used some type of  risk assessment methodology. In addition, the agencies did not always  adhere to these principles of risk management (see table 1).

At the time of our review, CBP’s methodology did not fully align with the  ISC Standard because it did not consider all of the 33 undesirable events  nor assess threat and consequence. CBP security specialists assessed  vulnerabilities at building entrances and exits, in interior rooms, and  around the perimeter using a yes/no checklist during the assessment  process. However, assessment reports showed that specialists did not  assess the threats and consequences of undesirable events at each  facility. According to security officials, the gap occurred because they  designed the checklist to meet requirements in the 2009 CBP Security  Policy and Procedures Handbook, which predates the first edition of the  ISC Standard issued in 2010. CBP officials told us that as of January  2017, they began using an improved methodology to assess the threats,  vulnerabilities, and consequences for 30 of 33 undesirable events— omitting three now identified in the November 2016 revision to the ISC  Standard. However, CBP has not yet updated its handbook to align with  the ISC Standard, even though it started this effort over 3 years ago in  December 2013. CBP officials did not provide a draft of its updated  handbook, but they provided a plan with milestone dates for issuing the  handbook by September 2018. CBP officials also told us that updates to  the handbook may have to wait due to competing priorities, including  efforts to address the backlog of assessments (which we discuss later in  this report). Delays in updating the handbook mean that CBP’s policy will  continue to not align with the ISC Standard. Furthermore, although CBP  security officials told us that all of the agency’s security specialists have  been trained to use the improved assessment methodology, without  documentation of the methodology in agency policy, there may be greater  risk of its inconsistent application. Standards for Internal Control  emphasize the importance of agencies developing and documenting  policies to ensure agency-wide objectives are met. Documentation serves  to retain institutional knowledge over time when questions about previous  decisions arise. Without an updated policy handbook that requires a  methodology that assesses all undesirable events consistent with the ISC  Standard, CBP cannot reasonably ensure that its facilities will have levels  of protection commensurate to their risk.

FAA’s methodology does not fully align with the ISC Standard because it  does not consider all of the 33 undesirable events nor does it assess all  three factors of risk. FAA security specialists assess vulnerabilities to the  site perimeter, entryways, and interior rooms using a yes/no checklist, but  the checklist does not assess the consequences from each of the  undesirable events at each facility. With respect to threat, FAA applies the  ISC’s baseline threat—a general federal facilities threat level that relates  directly to a set of baseline countermeasures—across all its higher-level  facilities because FAA policy states that there is no agency-specific threat  that exceeds the current baseline threat. According to FAA officials, the  baseline threat standardizes the security needs across their facilities  rather than addressing the security needs of individual facilities from  specific threats. When necessary, FAA policy allows specialists to modify  countermeasures based on an evaluation of conditions at the facility.

FAA realized that this approach was no longer appropriate given the  agency-wide goal to make risk-based decisions, a review of the  assessment process after a 2014 Chicago fire incident that destroyed  critical FAA equipment, and an awareness of ISC initiatives to assess  compliance. To address the resulting methodological gaps, FAA hired a  contractor to design, develop, test, and validate an improved risk- assessment methodology. Subsequently, FAA improved its methodology  in January 2017 to assess the threats, vulnerabilities, and consequences  for 30 of the 33 undesirable events identified in the November 2016  revision to the ISC Standard —and tested the methodology at lower-  and higher-level facilities. This revised methodology addresses the need  to assess individual facility needs rather than using a standardized  baseline approach. In April 2017, FAA officials told us of their plan for  implementing this methodology and provided tentative milestone dates to  conduct further testing, training, and analysis before deciding to use the  improved methodology, which they expect to complete by January 2018.  However, their plan lacks the necessary information to ensure successful  implementation, such as detail on how many facilities they will test and  how they will use the results of testing, training, and analysis to  implement the improved methodology within the identified 9-month time  frame. Furthermore, the improved methodology does not address  undesirable events for which ISC issued countermeasures in May 2017.  Without a detailed implementation plan to assess the methodology’s  impact on its security program, FAA cannot reasonably ensure that its  facilities have the proper countermeasures. With ongoing changes to its  security program, FAA has an opportunity to fully align its improved  methodology with the ISC Standard by including all 33 undesirable events  and to update its policy requiring the use of such a methodology.

Unlike CBP and FAA—which developed their own methodologies  separate from their parent departments (Department of Homeland  Security (DHS) and Department of Transportation (DOT), respectively)— ARS and the Forest Service follow an assessment methodology  developed by USDA. USDA’s methodology does not fully align with the  ISC Standard because it does not consider all of the 33 undesirable  events for which ISC issued countermeasures in May 2017. Security  specialists from USDA headquarters typically assess ARS’s and the  Forest Service’s higher-level facilities using a risk-based methodology  that considers the 31 undesirable events listed in the previous version of  the ISC Standard dated August 2013. However, until recently, USDA did  not assign ratings to each of the three risk factors—threat, vulnerability,  and consequence—and then combine these ratings to yield a measurable  level of risk for each undesirable event. USDA security officials said that  they have revised the assessment-reporting format to include this risk  calculation and trained their specialists to measure risk in this way. USDA  officials provided us with a new assessment template that addresses all  33 undesirable events and includes measuring risk. Additionally, USDA  officials said that they are revising their outdated physical security manual  and expect to complete it by April 2018. With a revised manual and  application of the new assessment template, USDA should be better  positioned to assess risk at its facilities.

When agencies do not use methodologies that fully align with the ISC  Standard, they could face deleterious effects, ranging from facilities  having inappropriate levels of protection to agencies having an inability to  make informed resource allocation decisions for their physical security  needs. Specifically, the ISC Standard states that facilities may face the  effect of either having (1) less protection than needed resulting in  inadequate security or (2) more protection than needed resulting in an  unnecessary use of resources. The ISC Standard also states that these  effects can be negated by determining the proper protection according to  a risk assessment. Identified excess resources in one risk area then can  be reallocated to underserved areas, thus ensuring the most cost- effective security program is implemented. As an illustration of such  potential effects, we found that two agencies assessing two higher-level  facilities came to two different conclusions in terms of their need for X-ray  machines to screen for guns, knives, and other prohibitive items in federal  facilities. Specifically, one agency based its decision on a policy that does  not deviate from the ISC’s baseline set of countermeasures, and the other  agency based its decision on professional judgement that deviated from  the ISC’s baseline set of countermeasures. Neither agency based its  decision on a risk assessment nor documented its decision—both ISC  requirements, specifically:

Without conducting a risk assessment, FAA recently expanded a  policy requirement calling for all higher-level facilities to have X-ray  machines and magnetometers. This new requirement poses a  potentially sizeable investment for the agency with an estimated cost  of X-ray machines of about $24,000 and magnetometers of about  $4,000 each. FAA may need such equipment at all its higher-level  facilities. However, the ISC Standard requires that agencies conduct  risk assessments first to justify their needs. Without conducting risk  assessments, FAA managers could unnecessarily use resources by  installing such equipment in all higher-level air traffic facilities when  there may be higher priority needs

A USDA security specialist decided, despite an ISC baseline  requirement that higher–level facilities have X-ray machines, not to  recommend an X-ray machine at a higher-level Forest Service facility.  The specialist reasoned that unlike other federal buildings with  numerous unknown visitors, this facility receives mostly known  individuals and a limited number of visitors. The ISC Standard allows  for professional judgement; however, the ISC requires that agencies  document deviations from the baseline set of countermeasures.  Reducing the facility’s level of protection without documenting an  assessment of risk could result in no record of the basis of the  decision for current and future facility managers and security officials  to review or use as justification in the case of a question of  compliance.

In another case, we found that one higher-level facility did not have  access control for employees or visitors nor did it have armed guard  patrols. The facility manager told us that intelligence and a history  without incidents gave leadership reason to believe that these measures  were not needed and that therefore the agency did not require and would  not fund such protective measures for this facility—in effect, accepting the  risks to the facility. Security officials said they also had the same  understanding and did not document the matter in the assessment report  even though agency policy and the ISC Standard require written  documentation when officials deviate from the baseline requirement.

Without security assessments that fully align with the ISC Standard and  provide measureable levels of risk, agencies do not have the information  they need to determine priorities and make informed resource allocation  decisions. For example, they may not be able to assess whether to  acquire or forego costly physical-security countermeasures—such as, X- ray machines, access control systems, and closed-circuit television  systems—for facilities. Additionally, after determining the need to acquire  a countermeasure, agencies must fund the countermeasure. As  previously discussed, leading practices in capital decision-making include  a comprehensive framework to review, rank, and select from competing  project proposals for funding. In conducting risk assessments that do  not fully align with the ISC Standard (i.e., not assessing threats,  vulnerabilities, and consequences and measuring risks), agencies miss  the opportunity for more informed funding decisions. Three of the four  agencies (CBP, ARS, and the Forest Service) currently prioritize funding  for operational needs over physical security needs (see table 2) when  agencies’ priorities might be different if they based their decisions on an  aligned risk assessment.

Selected Agencies Reported Facing Challenges in Conducting Security Assessments and Monitoring Results

Agencies Have Not Conducted Timely Security Assessments

Standards for Internal Control state that agencies should use quality  information on an ongoing basis as a means to monitor program activities  and take corrective action, as necessary. The ISC requires that agencies  assess higher-level facilities at least once every 3 years—an interval  requirement to identify and address evolving risks. We found that three of  the four agencies (CBP, ARS, and the Forest Service) did not meet this  requirement. Officials reported various challenges including (1)  assessments competing with other security activities, (2) an insufficient  number of qualified staff to conduct assessments when compared to the  number of facilities, or (3) not knowing of the required assessment  schedule.

An “information system” is the people, processes, data, and technology that  management organizes to obtain, communicate, or dispose of information. that had not been reassessed since 2010. CBP security officials attributed  the backlog to (1) having too few security specialists assigned to assess  about 1,200 facilities and (2) the specialists working on competing  priorities, such as revising the security handbook, conducting technical  inspections, and reviewing new construction designs and renovation  projects. According to CBP security officials, they have developed a plan  to eliminate the backlog by the end of fiscal year 2018 by prioritizing the  completion of assessments. While we found the plan comprehensive, the  schedule did not seem feasible. For example, the plan assumes that one  specialist can complete six assessments in 3 consecutive days and that  another specialist can complete three assessments in 1 day. In contrast,  security officials told us specialists take about 20 work hours (or 2½ days)  to conduct an on-site assessment of one facility. CBP officials said that  they believe they can meet the time frames of the plan because they have  set aside other priorities and have a thorough understanding of the scope  of work involved at the facilities. They added that it will not be easy to  meet the timeline, but they can accomplish it with a motivated and  committed workforce, adequate financial resources, and absent activities  that would otherwise require shifting of resources. We question the  feasibility of setting aside important priorities, such as updating the policy  manual and reviewing physical security elements in new construction  designs, as well as the workload assumptions for completing the  assessments. Further, these other priorities are also key to securing  facilities. Without balancing assessments with competing priorities, CBP’s  time frames for completing the assessments by the end of fiscal year  2018 may not be feasible and may also result in the agency’s not  addressing other important physical security responsibilities.

Since the ISC issued its standard in 2010, ARS and the Forest Service  have assessed their higher-level facilities at least once. However, these  agencies have not reassessed all of their higher-level facilities within the  3-year interval requirement. Specifically, security specialists have not  conducted required reassessments of two ARS and one Forest Service  higher-level facilities. The ARS headquarters official explained that the  agency had not reassessed the two facilities due to competing priorities  and insufficient internal resources. During the course of our review, ARS  headquarters officials said they began assessing one of the two ARS  facilities in May 2017 and will begin assessing the second facility in  October 2017. The Forest Service official explained that the agency  missed its security reassessment of the regional office because the  facility staff had not requested one. During our visit, facility staff  responsible for security told us that they were not aware of the ISC’s 3- year interval requirement. Facility staff requested a reassessment, and  security officials told us that they expected to complete it by mid-June  2017. Completing this one-time assessment may address the facility’s  security needs temporarily. However, ARS and the Forest Service have  not implemented a long-term schedule with key milestones and lack a  means to monitor completion of assessments of higher-level facilities at  least once every 3 years. Consequently, these agencies cannot  reasonably ensure that they have full knowledge of the risks to their  facilities.

FAA data from 2010 through 2016 show that FAA has assessed its 55  higher-level facilities at least once every 3 years. FAA policy requires  that specialists schedule assessments of higher-level facilities every 12– 18 months depending on whether the facility has met FAA physical  security standards.

Data Limitations Affect Agencies’ Ability to Fully Monitor Security Activities

The ISC Standard states that to make appropriate resource decisions,  agencies need information, such as what is being accomplished, what  needs management attention, and what is performing at expected levels.  We found that agencies’ methods of collecting and storing security  information had limitations that affected agency and facility officials’  oversight of the physical security of their facilities (see table 3).

Without having long-term, agency-wide information to monitor whether  assessments are conducted on schedule, ARS and the Forest Service  may not meet the ISC Standard, resulting in not adequately protecting  their facilities and employees.

The ISC Standard also states that agencies should measure their security  program’s capabilities and effectiveness to demonstrate the need to fund  facility security and to make appropriate decisions for allocating  resources. However, the agencies in our review were unable to  demonstrate appropriate oversight of their physical security programs  because:

CBP’s handbook does not include requirements for data collection  and analysis for monitoring physical-security program activities.  Facility managers and security officials do not enter assessment  results, such as the countermeasures recommended for facilities, in  the real property database. Consequently, they do not have  comprehensive data to manage their security program, assess overall  performance, and take any necessary corrective actions. A CBP  official told us that a comprehensive database would allow CBP to set  priorities for addressing countermeasures. Without including data  collection and analysis requirements in its updated handbook, CBP  may be unable to monitor the performance of its physical security  program.

FAA’s policy does not require ongoing monitoring of physical security  information, such as the status of recommended countermeasures or  assessment schedules. As a result, FAA officials do not proactively  use physical security information to assess the overall performance of  its physical security program and take corrective actions before an  incident occurs. Without a policy requiring ongoing monitoring of  information—an internal control activity, FAA may be unable to assess  the overall performance of its security program and take necessary  corrective actions.

USDA has a decentralized security program and places the  responsibility on agencies to create their physical security programs.  Security officials from ARS and the Forest Service told us that USDA  does not have a policy for collecting and managing agency-wide  information; however, they said that USDA is drafting a new  departmental regulation and manual that will specify (1) the roles and  responsibilities of agency and facility managers and (2) electronic- data-reporting requirements for monitoring the performance of the  physical security program. USDA officials provided a draft of USDA’s  regulation and manual for our review. The draft regulation did not  mention data reporting and monitoring, while the draft manual only  contained a table of contents that included a section entitled “Facility  Tracking Database.” USDA officials expect to issue new policies  sometime between October 2017 and April 2018. In the absence of  new departmental regulation and manual, USDA and Forest Service  officials told us that they have begun to develop a Forest Service  system for storing electronic copies of agency-wide assessments and  that they plan to expand the use of this system to track site specific  assessment dates and status of recommended countermeasures.  Forest Service officials provided milestone dates and described the  capabilities for a future information system, which they expect to  complete in September 2017. However, we could not determine  whether the manual will have information system requirements to  monitor agencies’ physical security program, an internal control  activity. Without USDA’s including data collection and analysis  requirements in its manual, its agencies may not be able to monitor  the performance of their physical security programs.

Selected Agencies Vary in Addressing Recommended Corrective Actions

Without agencies having information to monitor security activities, they  were unable to provide us information on the status of countermeasures  across their entire portfolio. In order to better understand the status of  countermeasures implemented and facilities’ experiences when  implementing countermeasures, we determined the status of  countermeasures at 13 facilities we visited.

As previously noted, risk management, as it pertains to physical security,  involves agency officials monitoring their physical security programs.  During our visits to 13 selected facilities, we found the four agencies  differed in the number of countermeasures that they had not  implemented. Facility officials provided us with some information on why  countermeasures had not been implemented, specifically:

CBP had a significant number of recommended countermeasures  from 2010 through 2016 that remained open at the eight selected  CBP facilities. CBP facility officials gave reasons why recommended  countermeasures had not been implemented. At one facility, officials  did not know about the recommended countermeasures from its last  2010 assessment because the individuals previously knowledgeable  about the assessments left the organization without communicating  the results. By taking action to improve facility security, they  implemented some needed countermeasures. However, at the time of  our review, a large number of the recommendations remained open.  At another facility, officials told us that they too had not known (for the  same reason mentioned above) of their 2010 assessment, which  contained recommended countermeasures. However, these officials  told us that they submitted a funding request a few weeks before our  visit to address all except one of the open countermeasures.  In  other cases, facilities have not implemented needed countermeasures  due to resource constraints or physical site limitations.

FAA had a large number of recommended countermeasures from  2010 through 2016 that remained open at the time of our review for  the two FAA facilities visited. In this case, the most recent security  assessment, completed in late 2016, resulted in one facility’s having  little time to implement countermeasures by the time we conducted  our analysis.

While ARS had closed almost all recommended countermeasures at  two facilities at the time of our review, one Forest Service facility had  not yet implemented a recommendation (to secure its entrance doors)  that was identified in a 2013 security assessment (see bottom center  photo, fig. 3). This countermeasure remained open because facility  officials said they continued to explore alternatives to address the  recommendation.

Figure 3 shows examples of countermeasures not fully implemented at  selected facilities we visited.

During our site visits and discussions with facility staff, we found that  physical site limitations or other priorities can make it difficult for facility  managers to implement countermeasures. For example, a  countermeasure might involve correcting a clear zone violation—that is,  moving an object (such as a brick wall) a certain distance away from the  facility’s perimeter fence to prevent a potential intruder from using the  object to climb over the fence. However, when the object near the fence  is a building and the property outside of the fence is not federally owned  (see bottom right photo, fig. 3), it may not be cost effective to correct the  clear zone violation. In this situation, the agency bears the responsibility  for exploring ways to address the vulnerability. In following the ISC  Standard, as previously noted, managers are required to justify and  document why they could not implement recommended  countermeasures—what the ISC calls risk acceptance.

Selected agencies carry a great responsibility for protecting facilities that  support border protection activities, provide safe and efficient air traffic  around the country, and protect the quality of the nation’s food supply.  With this responsibility comes the need to appropriately assess risk to  ensure the security of these agencies’ facilities. However, 7 years after  the ISC issued its initial risk-management process standard, each of four  selected agencies continued to use assessment methodologies that did  not fully align with this standard. During our review, agencies improved  their methodologies to better align with the ISC Standard, but the  agencies had not yet incorporated the methodologies into their policies  and procedures. Without updated policies and procedures requiring a  methodology that adheres to the ISC Standard (including all 33  undesirable events now identified in the November 2016 revision to the  ISC Standard), agencies may not collect the information needed to  assess risk and determine priorities for improved security. This situation  could hamper the agencies’ ability to make informed resource allocation  decisions or to recommend countermeasures commensurate to the needs  at specific facilities. To address challenges in conducting timely  assessments, agencies that had a backlog developed plans to address  them, but the assumptions used in CBP’s plans and time frames did not  appear to fully reflect the agency’s competing priorities and actual  experience. Additionally, ARS and Forest Service have not implemented  a long-term assessment schedule with key milestones to ensure that  higher-level facilities are reassessed at least once every 3 years. Further,  in cases where the agencies may have had risk assessment information,  CBP, ARS, and the Forest Service lack the means to collect, store, and  analyze this information in order to monitor the status of a facility’s  security. Without these key aspects of a comprehensive security  program—a methodology that meets the standard, policies, and  procedures that incorporate that methodology; the ability to complete  assessments on time; and information to perform monitoring—agencies  remain vulnerable to substantial security risks.

To improve agencies’ physical security programs’ alignment with the ISC  Risk Management Process for Federal Facilities and Standards for  Internal Control in the Federal Government for information and  monitoring, we recommend that the   Commissioner of U.S. Customs and Border Protection take the following  three actions: with regard to the updated Security Policy and Procedures Handbook,  the ISC’s Risk Management Process for Federal Facilities  requirement to assess all undesirable events, consider all three  factors of risk, and document deviations from the standard, and data collection and analysis requirements for monitoring the  performance of CBP’s physical security program.  revise the assumptions used in the plan to address the backlog to  balance assessments with competing priorities, such as updating the  policy manual and reviewing new construction design, to develop a  feasible time frame for completing the assessment backlog.

Secretary of Transportation direct the FAA Administrator to take the  following three actions: develop a plan that provides sufficient details on the activities needed  and time frames within the date when FAA will implement an  improved methodology; update FAA’s policy to require the use of a methodology that fully  aligns with the ISC’s Risk Management Process for Federal Facilities  for assessing all undesirable events, considering all three factors of  risk, and documenting all deviations from the standard  countermeasures; and update FAA’s policy to include ongoing monitoring of physical security  information.

Secretary of Agriculture take the following two actions:  include data collection and analysis requirements for monitoring the  performance of agencies’ physical security programs, in the  department’s revised physical-security manual, and direct the Administrator of the Agricultural Research Service and the  Chief of the Forest Service to  implement and monitor a long-term assessment schedule with key  milestones to ensure that higher-level facilities are reassessed at  least once every 3 years.

We provided a draft of this report to the Departments of Homeland  Security, Transportation, and Agriculture for review and comment. All  three departments agreed with the findings and recommendations for  their respective agencies. DHS agreed with our recommendations and  provided actions and timeframes for completion. With regard to our  recommendation to update the Security Policy and Procedures  Handbook, DHS stated that CBP is updating the handbook to include: (1)  a discussion and diagram of the ISC risk management process and its  application within CBP’s assessment processes; (2) specific guidance for  conducting risk assessments in accordance with the ISC’s Risk  Management Process for Federal Facilities; and (3) a requirement and  guidance for data collection and analysis in support of a robust physical  security program. With regard to our recommendation to revise the  assumptions used in the plan to address the assessment backlog, DHS  stated that CBP has reevaluated current priorities and believes the  current plan to eliminate the risk assessment backlog by the end of fiscal  year 2018 is achievable. DHS also provided technical comments, which  we incorporated as appropriate. DHS’s official written response is  reprinted in appendix IV.

DOT also agreed with our recommendations and by e-mail requested that  we publish the response to the sensitive version of this report. DOT  stated that FAA continues to refine its policy and develop processes that  address the ISC threats, vulnerabilities, and consequences. Further, DOT  stated that FAA would either validate that current mitigation strategies  address those risks or apply additional appropriate countermeasures.  DOT stated that it will provide a detailed response to each  recommendation within 60 days from the date of this report. DOT’s official  written response is reprinted in appendix V.

USDA agreed with our recommendations and provided the agency-wide  actions for completion. USDA provided a plan to ensure compliance with  the ISC’s Risk Management Process for Federal Facilities by  development of a standard physical-security assessment process and by  initiation of a compliance program to track assessments and monitor the  installation of countermeasures. In an e-mail, USDA provided milestone  dates and planned completion by January 2019. USDA’s official written  response is reprinted in appendix VI.

If you or your staff has any questions about this report, please contact me  at (202) 512-2834 or RectanusL@gao.gov. GAO staff who made key  contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

This report examines: (1) how selected agencies’ assessment  methodologies align with the Interagency Security Committee’s (ISC) risk  management standard for identifying necessary countermeasures and (2)  what management challenges, if any, selected agencies reported facing  in conducting physical security assessments and monitoring the results.

To determine how selected agencies’ assessment methodologies align  with ISC standards for identifying the necessary countermeasures, we  identified federal executive branch departments and agencies reported by  the Department of Homeland Security (DHS) to have received  delegations of authority to protect their own buildings. We reviewed the  Federal Real Property Council’s data on the Federal Real Property Profile  to identify federally owned and agency-controlled buildings. We  determined that these data were sufficiently reliable for the purpose of our  reporting objectives based upon our recent report that reviewed these  data fields. We selected four agencies based upon their large quantity of  reported federally owned and agency-controlled buildings: DHS, U.S.  Customs and Border Protection (CBP); Department of Transportation  (DOT), Federal Aviation Administration (FAA); United States Department  of Agriculture (USDA), Agricultural Research Service (ARS) and USDA’s  United States Forest Service (Forest Service). This methodology  purposely does not include federal buildings protected by FPS and under  the control of the General Services Administration as well as other  agencies that we reported on in our previous work. We obtained and  reviewed one particular ISC standard, The Risk Management Process for  Federal Facilities (the ISC Standard) and its related appendices for  assessing physical security and providing recommended  countermeasures at federal facilities. We obtained and analyzed the  selected departments’ and agencies’ facility-security policies and  procedures for a risk assessment methodology. According to the ISC  Standard, agencies’ risk assessment methodologies must:  consider all of the undesirable events identified in the ISC Standard  as possible risks to federal facilities as listed in appendix III; assess the threat, consequences, and vulnerability to specific produce similar or identical results when applied by various security provide sufficient justification for deviations from the ISC-defined  security baseline.

We limited the scope of this review to the first two standards above  because agencies’ adherence to these standards could be objectively  verified by reviewing and analyzing agency documentation and  interviewing agency officials, and their adherence to the two additional  standards could not be verified in this manner. We did not conduct risk  assessments with independent security professionals to evaluate: 1) the  results from prior agency evaluations and 2) the sufficiency of  justifications for deviations from the ISC-defined security baseline, as  both evaluations were outside of the scope of the engagement.  Therefore, for the purposes of this report, risk assessment policies,  procedures and resulting methodology that align with ISC standards are  those that consider all of the undesirable events and assess the threats,  consequences, and vulnerabilities to specific undesirable events. We  reviewed and analyzed information to answer the following five questions:  1.   Do the policies and procedures mention the ISC standards?  2.  Do the policies and procedures consider all of the undesirable  events?  3.  Do the policies and procedures assess the threat of specific  undesirable events?  4.  Do the policies and procedures assess the consequences of specific  undesirable events?  5.  Do the policies and procedures assess the vulnerability to specific  undesirable events?

We answered each of these questions as either a “Yes” or “No” for our  selected agencies. The “No” answer to questions 3, 4, and 5 includes the  following two possibilities: (a) the agency’s threat, consequence, or  vulnerability ratings are not tied to specific undesirable events, or (b) the  agency does not have a framework or formalized steps within which it  collects and analyzes threat-, consequence-, or vulnerability-related  information. If the answer to each of the five questions was “Yes,” then  the agency’s overall risk assessment methodology aligns with ISC risk  assessment standards for the purposes of this report. If the answer to one  or more of the five questions was “No”, then the agency’s methodology  does not to align with ISC standards for the purposes of this report.

We interviewed security officials at ISC; three departments (DHS, DOT,  and USDA); and four agencies (CBP, FAA, ARS, and the Forest Service).  We obtained and analyzed agency guidance on prioritizing physical  security needs and interviewed agencies’ facility maintenance and budget  officials. We reviewed the ISC’s best practices for planning for physical  security resources within an agency budget process. Additionally, we  reviewed the Office of Management and Budget’s and our leading  practices in capital decision-making that provide agencies with guidance  for prioritizing budget decisions such as “countermeasure projects.” We  also reviewed Standards for Internal Control in the Federal Government  because internal controls play a significant role in helping agencies  achieve their mission-related responsibilities. Our findings from our  review of the selected agencies are not generalizable to all ISC member  agencies, but provide insight into and illustrative examples about selected  agencies’ facility risk-assessment methodologies.

To determine what management challenges selected agencies reported  facing in conducting physical security assessments and monitoring  results, we interviewed agencies’ security, maintenance, and budget  officials. We requested agency security officials to provide portfolio- wide  data on facility security assessments for our review in order to select sites  to visit and analyze data for dates of assessments and the status of  findings. We assessed the reliability of this data through interviews with  knowledgeable agency staff and a review for completeness and any  unexpected values. We compiled information from physical security  assessments when no portfolio-wide agency data were available. We  determined that these data were sufficient for the purpose of our reporting  objectives and selected geographically dispersed sites with buildings with  higher reported security levels per the ISC Standard, as these higher  security levels have greater requirements and therefore the potential for  greater resource needs. See appendix II for the 13 sites we selected. For  these selected sites, we interviewed agency staff concerning the  assessment process, site-specific findings, recommendations, justification  for deviations from ISC’s baseline standards, and management  challenges faced in addressing physical security needs. We observed  and photographed the status of the findings from the site physical security  assessments. We did not independently determine what constitutes a  management challenge or a physical security finding. Rather, we relied  on these stakeholders to determine these physical security concerns as  defined in their own standards and guidance. The information from our  selected sites is illustrative and cannot be generalized to sites agency- wide.

The performance audit upon which this report is based was conducted  from June 2016 to August 2017 in accordance with generally accepted  government auditing standards. Those standards require that we plan  and perform the audit to obtain sufficient, appropriate, evidence to provide  a reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives. We  subsequently worked with DHS, DOT and USDA from August 2017 to  October 2017 to prepare this version of the original report for public  release. This public version was also prepared in accordance with these  standards.

Appendix II: Selected Facilities GAO Visited

Appendix III: The Interagency Security Committee’s Undesirable Events

Appendix IV: Comments from the Department of Homeland Security

Appendix V: Comments from the Department of Transportation

Error! No text of specified style in document.

Appendix VI: Comments from the Department of Agriculture

Appendix VII: GAO Contact and Staff Acknowledgments

Appendix VII: GAO Contact and Staff  Acknowledgments Error! No text of specified style in document.

In addition to the contact named above, Amelia Shachoy (Assistant  Director), Steve Martinez (Analyst-in-Charge), Jennifer Clayborne,  George Depaoli, Geoffrey Hamilton, Joshua Ormond, Alison Snyder,  Amelia Michelle Weathers, and Elizabeth Wood made key contributions  to this report.