Homeland Security: Research & Development Coordination Has Improved, but Additional Actions Needed to Track and Evaluate Projects

Conducting R&D on technologies is vital to enhancing the security of the nation. The Homeland Security Act of 2002, as amended, designates S&T as responsible for coordinating all R&D activities of DHS. Questions have been raised about S&T's ability to demonstrate the impact of its R&D investments. Since DHS began operations in 2003, GAO has made recommendations to help improve DHS's efforts to coordinate and oversee R&D.

GAO was asked to review DHS's R&D efforts. This report examines (1) how much DHS has obligated for R&D and what types of R&D DHS conducts, (2) to what extent S&T coordinates R&D across DHS, and (3) how, if at all, DHS identifies and tracks R&D efforts.

GAO reviewed documentation from DHS related to the conduct, coordination, tracking, and evaluation of R&D projects. GAO interviewed DHS officials with responsibilities related to, among other things, R&D financial reporting, performance evaluation, and the IPT process, including officials from the 10 DHS components that participate in the IPTs. GAO also reviewed DHS R&D budget and obligation data from fiscal years 2010 through 2017.

The Department of Homeland Security (DHS) obligated more than $10 billion for research and development (R&D) from fiscal years 2010 through 2017. Seven DHS components have budget authority to conduct R&D, and the Science and Technology Directorate (S&T) obligated nearly 80 percent of all DHS R&D funds during this time period. These components conduct a wide range of R&D, from cybersecurity to border security projects. S&T generally leads or funds R&D projects by providing technology and knowledge products to support four homeland security mission areas:

Disaster resilience . Improving community resilience to natural disasters through technology and tools;

Critical incidents . Improving response technological capabilities;

Border security . Improving the nation's ability to detect, interdict and prosecute illegal activity across air, land and sea.

Cybersecurity . Developing technologies and tools to secure systems and critical infrastructures against cyberattacks.

S&T strengthened its R&D coordination efforts across DHS, but some challenges remain. In 2015, DHS established an R&D coordination mechanism, to be led by S&T, and in 2017 issued R&D coordination-related guidance. Specifically, to improve coordination, DHS established an Integrated Product Team (IPT) process to serve as the key R&D coordination mechanism within DHS. All ten DHS components that GAO interviewed stated that the IPT process improved visibility into DHS R&D efforts. However, the component that obligated approximately 17 percent of DHS R&D funds in 2017, or $176 million, did not participate in the IPT process in 2018, as required. Nonparticipation poses a risk to R&D coordination efforts across DHS, including R&D project information not being shared among components. Furthermore, ensuring that all required components participate in the IPT process would help S&T maintain visibility of R&D projects in order to fulfill its statutory role of coordinating R&D, and mitigate the risk of potential duplication of effort.

S&T, in its coordination role for DHS, uses disparate information sources to identify and track R&D project information and faces challenges to track progress and other information for ongoing R&D projects. For example, R&D project information is stored in multiple information sources leading to difficulty in integrating complete R&D project information and resulting in reporting that is not comprehensive. By developing a mechanism to address these challenges, S&T can further improve its efforts to report and analyze R&D project information, and have improved visibility on R&D efforts across DHS. GAO also identified challenges in collecting information related to R&D performance. Among other things, DHS is not well positioned to integrate the results and share lessons learned because limited R&D customer feedback information is collected and analyzed. Of the seven DHS components with R&D budget authority, two reported having formal customer feedback mechanisms. As a result, DHS is unable to more fully understand its customers' perceptions and experience which would allow DHS to better assess the performance of its R&D efforts.

GAO is making four recommendations, with which DHS concurred, including that DHS: 1) ensure all components participate in the IPT process, 2) develop a mechanism that aligns R&D project tracking sources, and 3) collect feedback from R&D customers.

Conducting research and development (R&D) on technologies for  detecting, preventing, and mitigating terrorist and natural threats is vital to  enhancing the security of the nation. Since it began operations in 2003,  the Department of Homeland Security (DHS) has obligated billions of  dollars researching and developing technologies used to support a wide  range of missions, including securing the border, detecting nuclear  devices, and screening airline passengers and baggage for explosives,  among others. Within DHS, the Science and Technology directorate  (S&T) is responsible for coordinating and integrating R&D activities  across the department while DHS components may also conduct R&D to  support their respective missions.

Since 2003, questions have been raised regarding S&T’s ability to  oversee and coordinate R&D activities across the department as well as  to demonstrate the value of these investments over time. In 2012, we  found that DHS efforts to coordinate R&D across the department were  fragmented and overlapping, which increased the risk of unnecessary  duplication. We also reported that DHS did not know how much all of its  components invested in R&D, making it difficult to oversee these efforts  across the department. We recommended, among other things, that the  Secretary of Homeland Security develop and implement policies to  include elements such as a definition of R&D that provides reasonable  assurance that reliable accounting and reporting of R&D resources and  activities are achieved. In response to our recommendation, DHS issued  a memorandum in April 2014 that included a definition for R&D.  Furthermore, DHS implemented a common appropriations structure to  accurately identify the resources it is expending on R&D projects.

You asked us to evaluate DHS’s efforts to coordinate and monitor R&D  activities across the department. This report addresses the following  questions: (1) how much has DHS obligated for R&D from fiscal years  2010 through 2017 and what types of R&D does DHS conduct; (2) to  what extent does S&T coordinate R&D across DHS; and (3) how, if at all,  does DHS track and identify R&D efforts and assess R&D performance  department-wide.

To determine how much DHS has obligated for R&D, we analyzed federal  budget data to identify R&D-related obligations for fiscal years 2010  through 2017. We selected this time frame to analyze multiple years of  obligation trend data, and 2017 was the most recent year of available  data at the time of our review. To identify the R&D obligations data, we  searched the Office of Management and Budget’s (OMB) MAX database  for DHS accounts with the term “research” in the account title and  extracted total obligations for those accounts from the database for fiscal  years 2010 through 2017. We assessed the reliability of the data by  reviewing OMB guidance and procedures that govern OMB MAX data  inputs. We determined that the data were sufficiently reliable for reporting  general trends. However, because we previously found that some DHS  R&D-related obligation data were not properly identified, and because  R&D-related obligations could potentially be reported in other accounts  that may not contain the term “research,” we report the data as a  minimum of the amount that DHS could have obligated for fiscal years  2010 through 2017. We also interviewed DHS officials, including those from the DHS Office of the Chief Financial Officer (OCFO) and the S&T  Finance and Budget Division to obtain and corroborate information about  R&D-related obligations, budget and expenditure tracking tools and  reports. To identify what types of R&D DHS conducts, we reviewed DHS  documents including congressional budget justifications and fact sheets.  To learn more about the types of R&D activities that DHS components  conduct, we also interviewed officials from each of the 10 components  that are involved in the Integrated Product Teams (IPT), which is DHS’s  primary R&D coordination mechanism. Based on IPT charter documents  and S&T IPT guidance, we identified the components that are to  participate in the IPT process. These components were: the Countering  Weapons of Mass Destruction (CWMD) Office, the Federal Emergency  Management Agency, the Cybersecurity and Infrastructure Security  Agency, the U.S. Secret Service, the Office of Intelligence and Analysis,  the Transportation Security Administration (TSA), the U.S. Citizenship  and Immigration Services, the U.S. Coast Guard, the U.S. Customs and  Border Protection (CBP) and the U.S. Immigration and Customs  Enforcement.

To determine the extent to which S&T coordinates R&D across DHS, we  analyzed DHS guidance documents, such as management directives,  instructions, and memoranda, which assign R&D-related roles and  responsibilities and govern certain R&D coordination practices. The  guidance documents included a memorandum establishing the IPT  process and how the IPT process should coordinate R&D efforts across  DHS. We also interviewed DHS officials with roles and responsibilities  related to R&D to obtain their perspectives on R&D coordination and the  relevant guidance documents, including officials who manage IPT  operations. To further understand the IPT process, we conducted  interviews with officials from the 10 DHS components that participate in  the IPT process to obtain their perspectives on the benefits and  challenges of the process. Furthermore, we analyzed IPT-related  documentation including charter documents, IPT meeting agendas and  minutes, and IPT guidance documents to determine how the process  facilitates R&D coordination efforts. We also interviewed DHS officials  responsible for department-wide requirements identification.

To examine the extent to which DHS tracks and identifies R&D efforts, we  interviewed S&T officials to determine what tracking mechanisms are  used. We also reviewed existing lists of DHS R&D projects, such as  DHS’s Report of Coordinated R&D, DHS’s responses to congressional  inquiries regarding R&D project lists, outputs from electronic tracking  systems, such as S&T’s Project Tracker Database, and R&D projects  listed in congressional budget justification documents, among other  things. We also interviewed S&T officials involved in developing the R&D  project information sources to determine how the lists are compiled, how  information is entered into the sources, and what the benefits and  challenges are for each source of R&D project information that we  identified. In addition, we interviewed officials from DHS’s OCFO and  reviewed departmental financial management policies to learn about how  the recently-implemented common appropriations structure has affected  the transparency of R&D efforts across DHS.

To determine the extent to which DHS components with R&D budget  authority collected and reported performance information to assess their  R&D efforts, we reviewed applicable laws governing performance  reporting in the federal government, including the Government  Performance and Results Act of 1993 (GPRA), as updated and expanded  by the GPRA Modernization Act of 2010 (GPRAMA), and guidance for  implementing these laws. We also reviewed leading practices, identified  in our prior work, used by organizations related to R&D performance  indicators. In addition, we reviewed leading practices for project,  program and portfolio management and reviewed DHS documents  related to its processes for collecting and reporting its performance  information. The documents reviewed included policies and guidance  regarding how these processes are to operate, the information to be  collected, as well as documents used to communicate performance  information, including DHS’s Annual Performance Report, as well as  DHS’s congressional budget justification documents. To illustrate how  DHS uses one method to track R&D progress, we analyzed selected  milestones in the congressional budget justification documents for 7 of  DHS’s approximately 132 reported R&D projects. To conduct our  analysis, we identified two milestones each from the selection of seven  R&D projects that S&T officials consider to be high priority. We selected  the two milestones from the most recent year for each project; if there  were more than two milestones, we randomly selected two. The  milestones, selected from the fiscal year 2018 budget documents, were  assessed against DHS guidance that its components are to utilize to  develop the milestones. Two analysts independently reviewed the  milestones and resolved any disagreements in their assessments. To  further understand the information used to assess R&D performance, we  interviewed officials from the seven IPT-participating DHS components  with R&D budget authority to learn how they collect and report  performance information. We also examined guidance to components for  collecting and reporting performance information, such as performance  goals and milestones, and customer feedback mechanisms.

We also interviewed officials from various DHS offices that are involved in  managing, collecting or using R&D performance information from across  the department. Specifically, we interviewed officials from DHS’s Office of  Program Analysis and Evaluation, Office of Policy, and OCFO. DHS’s  Office of Program Analysis and Evaluation coordinates performance  management across DHS components. DHS’s Office of Policy develops  strategies, operational plans, and leads the development of operational  and resource allocation guidance for the department, among other things.  Further, we reviewed performance information and documents related to  DHS’s process for assessing and reporting its annual performance goals.  This included performance information reported in DHS’s Annual  Performance Report for fiscal years 2016 through 2018 and in  congressional budget justification documents for fiscal year 2018 for the  DHS components that conduct R&D. To conduct our analysis, we  compared performance information in the DHS annual performance  report and congressional budget justification documents to GPRA  requirements and OMB guidelines for agency performance information.

We conducted this performance audit from October 2017 through January  2019 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

The Homeland Security Act of 2002, as amended, designates the Under  Secretary for Science and Technology as responsible for coordinating all  R&D activities of DHS. The Act also provides that nothing in it precludes  other department components from carrying out R&D activities as long as  the activities are coordinated through S&T. As of September 2018,  seven DHS components have budget authority to conduct R&D  activities—S&T, the Coast Guard, the CWMD, the Secret Service, the  Cybersecurity and Infrastructure Security Agency, TSA, and the Office of  the Chief Information Officer within the Office of the Undersecretary for  Management. Figure 1 provides an organizational overview of DHS  components and offices that are involved in the R&D process as of  December 2018.

S&T reorganized its structure in September 2018 and currently has three  technical divisions responsible for managing R&D programs related to  improving border, immigration, and maritime security; supporting first  responders; and countering physical and cybersecurity threats among  others, as shown in figure 2.

Most of S&T’s R&D portfolio consists of applied and developmental R&D,  which can be transitioned to use within 3 years, as opposed to longer- term basic research. In addition to conducting projects for its DHS  customers, S&T conducts research for other federal agencies and first  responders. S&T is also responsible for conducting basic and applied  research, and collaborates with other government agencies, academia,  the private sector, and others.

Overview of Past R&D Management Challenges

Questions have been raised about S&T’s ability to demonstrate the  impact of its investments—in terms of value, tangible products, and  advances toward the homeland security mission. Accordingly, for  example, House appropriations committee report language has directed  S&T to demonstrate how its R&D efforts are timely, with results relatively  well defined and to make investment decisions based on clear and  sensible priorities. In 2016, Congress passed the National Defense  Authorization Act of 2017 (NDAA) which requires DHS to report annually  to Congress on the department’s R&D projects including details such as  the project name, the component carrying out the project, associated  funding levels, and expected objectives and milestones for each project,  among other items.

Since DHS began operations in 2003, we have made multiple  recommendations designed to improve DHS efforts to manage and  oversee R&D efforts, as described later in this report. In September 2012,  we reported that DHS did not have a department-wide policy defining  R&D or guidance directing its components how to report R&D activities.  As a result, DHS did not know its total annual investment in R&D, which  limited the department’s ability to oversee components’ R&D efforts and  align them with agency-wide R&D goals and priorities. We also reported  that DHS’s R&D efforts were fragmented and overlapping, which  increased the risk of unnecessary duplication. We recommended that  DHS develop policies and guidance for defining, reporting and  coordinating R&D activities across the department, and that DHS  establish a mechanism to track R&D projects. As of August 2017, DHS  had implemented these recommendations by, among other things,  issuing guidance defining research and development activities and  establishing Integrated Product Teams (IPT) as a primary mechanism for  coordinating R&D. These actions and others are described in more detail  later in this report.

Figure 3 provides a summary of key events related to S&T since its  inception.

After the consolidation in 2002 of 22 agencies into a single department,  DHS had, until recently, different appropriation structures and budget  management practices based on agencies’ funding structures prior to  DHS consolidation. In 2018, we found that, with over 70 different  appropriations and over 100 formal program, project, or activity accounts,  DHS operated for over a decade with significant budget disparities and  inconsistencies across its components. The lack of uniformity hindered  visibility, inhibited comparisons between programs, and complicated  spending decisions, including for R&D-related programs. For example, in  2012 we reported that S&T, the Domestic Nuclear Detection Office, and  Coast Guard were the only three DHS components with budget authority  to conduct R&D. However, in 2012, we identified an additional $255  million in R&D obligations by other DHS components at that time. Further,  we found in 2012 that the Domestic Nuclear Detection Office did not  report certain R&D budget data to OMB, and R&D budget accounts  included a mix of R&D and non-R&D spending, which further complicated  DHS’s ability to identify its total investment in R&D activities.

Role of the Integrated Product Teams in R&D

Within DHS, IPTs – established in 2015 – are to identify and prioritize  technological capability gaps, and identify current or future R&D efforts or  other solutions to close the gap, among other things. Specifically, the  IPT process consists of three activities: 1) Identifying R&D activities in  progress, funded, planned, or recently completed; 2) Prioritizing  technological capability gaps and corresponding R&D efforts to address  those gaps; and 3) Validating and reporting the gaps.

The DHS IPT effort is led by S&T, but the individual IPTs are composed  of senior-level officials from across DHS. IPT members prioritize R&D  gaps based on departmentwide needs and requirements, and align  current and planned R&D efforts to the identified gaps. In prioritizing and  evaluating the capability gaps, four pre-defined criteria and rating scales  are used and are discussed below:

Strategic alignment: assesses the R&D gaps alignment with DHS- level and component-level strategic priorities.

Impact: assesses if addressing the R&D gap would result in  enhanced risk or threat reduction capability, among other things.

Feasibility: assesses the feasibility of addressing the R&D gap, given  its technical complexity. Considerations include feasibility related to  technology, time, and transition.

R&D needs: assesses whether the R&D gap would provide a critical  R&D solution in an otherwise unaddressed area.

The IPT’s role in coordinating R&D is discussed later in this report.

Overview of DHS’s Processes for Assessing R&D Performance

GPRA, as updated and expanded by GPRAMA, requires agencies to  establish annual performance goals with target levels of performance  against which to measure progress towards those goals. In addition,  GPRA requires executive agencies to prepare an Annual Performance  Report on program performance for the previous fiscal year. DHS has  developed goals and targets to assess and communicate R&D  performance. As shown in figure 4, DHS’s performance assessment  process also includes identifying performance gaps and implementing  corrective actions to address unmet performance goals.

DHS uses strategic and management performance goals and measures  to assess and communicate on the performance of its R&D efforts. In  addition, DHS uses milestones to track and communicate progress of its  R&D project activities.

Milestones: A milestone is a scheduled event signifying the  completion of a major deliverable or a phase of work. Milestones can  help agencies demonstrate that they have clear and fully developed  strategies and are tracking progress to accomplish their goals.  Milestones are often used as the basis of an alternative form of  performance goal. Milestones related to DHS R&D efforts are  reported to Congress and publicly available through the DHS  congressional budget justification.

Strategic Goals: A type of performance goal used to reflect  achievement of missions that are publicly reported in the DHS Annual  Performance Report. As part of DHS’s Annual Performance Report,  these goals are subject to GPRA and GPRAMA requirements.

Management Goals: A type of performance goal used to gauge  program results and tie to resource requests that are reported to  Congress and publicly available through the DHS congressional  budget justification along with the strategic goals.

As we previously reported in 1997, experts in research measurement  have tried for years to develop indicators that would provide a measure of  the results of R&D. However, the very nature of the innovative process  makes measuring the performance of science-related projects difficult.  For example, a wide range of factors determine if and when a particular  R&D project will result in commercial or other benefits. It can also take  many years for a research project to achieve results.

DHS Has Obligated More Than $1 Billion per Fiscal Year from 2010 through 2017 for Conducting R&D That Aligns with Department Mission Areas across Seven DHS Components

DHS is required to report department-wide R&D-related funding to OMB  on an annual basis. DHS uses several mechanisms to report the R&D- related funding, including budget authority (the legal authorization to  obligate funds), obligations (binding agreements to make a payment for  services), and outlays (payments to liquidate obligations representing  amount expended). Further, OMB requires agencies to submit data on  R&D programs as part of their annual budget submissions on investments  for basic research, applied research, development, R&D facilities  construction, and major equipment for R&D using OMB’s definition of  R&D.

Based on our analysis of OMB’s federal obligations data, we identified  R&D-related obligations data for DHS components for fiscal years 2010  through 2017. Figure 5 depicts the R&D related obligations that were  reported for fiscal years 2010 through 2017, which, on average, were  about $1.3 billion annually or more than $10 billion overall for that time frame. Additionally, S&T obligated nearly 80 percent of all DHS R&D  funds for that time period.

Types of R&D Activities that S&T Conducts

S&T may conduct or fund R&D activities on its own or jointly with other  entities. In addition to S&T, six other DHS components currently have  budget authority to conduct R&D—the Coast Guard, CWMD, TSA, Secret  Service, Cybersecurity and Infrastructure Security Agency, and the  Undersecretary for Management. In August 2018, S&T reported that there  were at least 132 ongoing R&D projects across the Department. Some  R&D projects aim to produce a specific prototype or piece of technology  for an end user, while others might be for developing IT systems,  conducting specific training, or providing written reports, or knowledge  products.

According to S&T officials, S&T generally leads or funds R&D projects by  providing technology and knowledge products for four homeland security  areas:

Disaster resilience. Improving community resilience to natural  disasters through technology and tools that support planning,  decision-making and mitigation efforts;

Critical incidents. Improving technological capabilities during all  stages of critical incident response;

Border security. Improving the nation’s ability to detect, interdict and  prosecute illegal activity across air, land and sea.

Cybersecurity. Developing technologies, tools and techniques to  defend, mitigate, and secure current and future systems, networks  and critical infrastructures against cyberattacks.

Figure 6 illustrates the types of R&D projects that are either led or funded  by S&T for each category. For more in-depth examples and descriptions  of S&T projects, please see appendix I.

In its efforts to determine how to best support the DHS components and  first responders, S&T seeks first to identify the end user’s needs by  discussing operational challenges with components and first responders;  then develop prototypes or leverage existing technologies to find  solutions; and finally to test and evaluate potential solutions to ensure that  they meet the end user’s needs and ultimately deploy solutions to the  field.

The other six DHS components with R&D budget authority typically lead  and fund R&D projects tailored to support their specific operational  requirements and respective missions. Examples of R&D projects  conducted by DHS components other than S&T are listed below in table  1.

S&T Implemented R&D Coordination Mechanism and Directives to Strengthen Collaboration Across DHS, but Challenges Remain in Ensuring Participation

DHS Established a Process to Coordinate R&D Efforts

DHS established its IPT process in August 2015 as the central  mechanism to coordinate R&D efforts across the department, in  accordance with recommendations we made in 2012. The IPT process  works to identify DHS technological capability gaps and coordinate R&D  to close the gaps across DHS mission areas.

The IPTs consist of senior representatives from operational components.  As of October 2018, IPTs are organized according to the department’s  identified missions and include the following sub-IPTs, as shown in table  2.

Each IPT has an establishing charter document, which formally identifies  the IPT component members and responsibilities and lists the  corresponding sub-IPTs. IPTs and sub-IPTs are to meet multiple times  throughout the year to support the process of identifying and prioritizing  R&D capability gaps and R&D efforts. For example, the charter for the  “Secure Borders” IPT states that they anticipate meeting at least 2 or 3  times per year, or more frequently to support the annual program  planning and budgeting process.

Overall, components reported that the IPT process enhanced  collaboration and improved visibility into R&D efforts across DHS.  Officials from all 10 of the DHS components we interviewed reported the  IPT process has been helpful in various ways, including identifying  capability gaps, prioritizing and closing the gaps, and providing  transparency and insight into other components’ R&D efforts. For  example, CBP officials reported that, through the IPT process, they were  able to identify R&D projects that the Coast Guard had been pursuing  related to maritime security. The R&D projects that Coast Guard was  pursuing were also of interest to CBP, and therefore CBP worked through  the IPT process to prevent duplicative work and combine some of those  efforts. In another example, TSA officials reported that they collaborated  with the Secret Service to test explosive screening technologies, and that  the IPT process facilitated their ability to collaborate and share  information about the screening technologies.

In addition to enhancing collaboration, component officials provided their  perspectives on how the IPT process prioritizes technology capability  gaps that components have identified. For example, TSA officials  reported that the gap identification and prioritization process works well,  but that funding R&D activities to close the gaps is more challenging  because it is influenced heavily by competing budget priorities, emerging  threats, and other DHS senior leadership priorities. TSA officials further  reported that departmental resource constraints limit the number of  identified capability gaps that can be addressed. However, officials from  CBP reported that several R&D projects were successfully implemented  after CBP had worked with S&T to identify a capability gap and transition  a solution to close the gap, such as certain upgrades needed on CBP  trucks. S&T officials stated that they have also taken steps to integrate  with the department’s Joint Requirements Council and utilize component  requirement executives who work with component agencies to provide a  basis for requirements and aid the components with the means to track  the progress and disposition of each capability gap on a regular basis.

DHS Issued R&D Coordination Directives but Faces Challenges in Ensuring IPT Participation

In 2012, we found that, among other things, DHS had not developed a  policy defining who was responsible for coordinating R&D within the  department and what processes should be used to coordinate it. As a  result, components did not consistently coordinate with S&T on what R&D  was planned or underway, leading to increased risk of unnecessary  duplication of R&D efforts. We recommended that DHS develop and  implement policies and guidance for overseeing R&D that included,  among other things, a description of the department’s process and roles  and responsibilities for overseeing and coordinating R&D investments.  DHS concurred with our recommendation, and, in response, the  Secretary for Homeland Security delegated the authority to coordinate  and integrate the department’s R&D, testing, evaluation efforts to the  Under Secretary for Science and Technology in 2014. In 2015 and  2016, DHS issued two guidance documents regarding the establishment  and progress of the IPT process. These documents specified how DHS,  through the IPT process, is to implement processes and mechanisms to  coordinate department-wide R&D efforts. Additionally, in January 2017,  DHS issued an R&D directive and associated instruction to formalize  R&D reporting and coordination among components, as shown in figure  7.

The 2017 directive and associated instruction identify the roles and  responsibilities, including IPT participation requirements, for key entities  involved in R&D across DHS. However, the directive and instruction do  not specifically address steps to be taken if components do not adhere to  the requirements. For example, the January 2017 DHS instruction states  that “to effectively coordinate DHS R&D activities, DHS components are  required to follow the DHS IPT process.” However, officials from CWMD  stated that they do not participate in the S&T-led IPT sessions because  they have their own internal process for identifying and prioritizing  capability gaps. S&T officials stated that CWMD’s predecessor  organization, the Domestic Nuclear Detection Office, participated in the  IPT process until DHS initiated a reorganization of its weapons of mass  destruction programs (resulting in the current CWMD).

Current non-participation by CWMD, which has the second-largest R&D  budget within DHS and obligated approximately 17 percent of DHS R&D  funds, or $176 million in fiscal year 2017, poses risk of R&D project  information not being shared among components. In August 2018, we  reported that DHS’s chemical defense programs and activities were  fragmented and not well coordinated across the department, including  R&D activities. We recommended that CWMD develop a strategy and  implementation plan to help DHS integrate and coordinate its chemical  defense programs and activities, among other things. Additionally, in its  2014-2018 Strategic Plan, DHS states that, to anticipate key threats, DHS  should, among other things, prioritize R&D activities related to chemical,  biological, radiological, and nuclear terrorism. Given these factors,  CWMD’s participation in the IPT process is important to ensure that all  R&D efforts are fully coordinated thereby mitigating the risk of potential  duplication of other DHS R&D efforts.

S&T officials recognize that some components might not be complying  fully with the departmental directives and associated guidance documents  which require participating in the IPT process – the key R&D coordination  mechanism within DHS. S&T officials stated that, despite these  challenges, they have strong collaborative relationships with the  components, and the existing collaboration mechanisms, such as the IPT  process, continue to mature and facilitate R&D-related information  sharing. However, DHS guidance documents require that components  participate in the IPT process. By ensuring that all required components  participate in the IPT process, DHS can help S&T maintain visibility of  R&D projects in order to fulfill its statutory role of coordinating R&D.

S&T Has Taken Steps to Identify and Track DHS’s R&D Project Information, But Efforts Are Disjointed or Do Not Align with DHS Guidance

Since 2012, S&T has taken steps to identify and track information related  to ongoing R&D projects across DHS, and in 2017, DHS developed a  common appropriations structure that standardized R&D budgeting  processes across the department. However, S&T’s efforts to identify and  track R&D project information have limitations and can result in  information that is not comprehensive. We also identified challenges in  collecting information related to the achievement of R&D milestones.

DHS Developed the Common Appropriations Structure to Better Streamline and Standardize the Budgeting Process across Components

In 2017, DHS developed a common appropriations structure that allowed  it to calculate and monitor its expenses, including R&D expenses, across  the department. Officials from DHS’s OCFO reported that, prior to the  new structure, some components categorized their R&D expenses as  other types of expenses, such as “salaries and expenses.” These  categorizations made it difficult to account for R&D expenses outside of  an individual component’s budget management division. Furthermore,  OCFO officials reported that components previously utilized inconsistent  R&D definitions, which often led to discrepancies in how components  would report R&D activities.

In our April 2018 report, we found that DHS had operated for over a  decade with significant budget disparities and inconsistencies across its  components. We found that the lack of uniformity hindered visibility,  inhibited comparisons between programs, and complicated spending  decisions. According to DHS OCFO officials, the introduction of the  common appropriations structure, among other things, has helped  improve transparency within DHS and among the components so that  R&D can be more readily identified and tracked. DHS is also able to  compare R&D funding amounts throughout DHS more easily than in  previous years. In addition, of the seven components that have their own  R&D funding to report, five indicated that the new structure has improved  the department’s ability to identify and report R&D activities.

S&T’s Efforts to Maintain an Inventory of R&D Projects across DHS Have Resulted in Reporting That Is Not Comprehensive

We identified multiple sources of component R&D project information,  each posing its own challenges or limitations. As described below, these  challenges and limitations include difficulty in collecting and integrating  R&D project information, and reporting that is not comprehensive.

DHS’s response to the National Defense Authorization Act of 2017:  The NDAA, passed in December 2016, required DHS to provide a list of  ongoing R&D projects and accompanying milestone information by  January 2017, and annually thereafter, to specified congressional  committees. In December 2017, DHS officials reported that they had not  yet submitted the report, and anticipated that the response to the NDAA  requirement would be completed by January 2018. In August 2018, DHS  submitted its response to the committee, then 19 months late. S&T  officials stated that the reporting delays were due to challenges in  collecting and integrating the data. S&T officials also reported that it used  the components’ congressional budget justification documents as a  starting point to identify R&D projects to include in its report in response  to the NDAA requirement. However, additional details about the R&D  projects had to be collected via a “data call” process from the  components. S&T officials told us that it was a challenge to have  components report information about their R&D projects consistently and  systematically. Furthermore, S&T officials identified terminology-related  challenges in their R&D data call efforts, including making distinctions  between R&D projects, efforts, and activities. S&T officials also reported  that, in its current format, they would not be able to easily identify how  many projects were added to the NDAA list across years, or if a given  R&D project experienced a large increase or decrease in funding.

Annual Reports of Coordinated R&D: In response to a 2015 request  from the Secretary of Homeland Security that the IPTs identify R&D work  being performed across DHS, S&T issued a “Report of Coordinated R&D”  in 2016 and 2017. The content for the reports was developed through a  “data call” process, and the reports identified R&D activities and projects  across DHS. The reports – for 2016 and 2017 – contain tables of R&D  project names and the component leading the project, among other  things. However, during the course of our review, S&T officials reported  that these annual reports should not be considered authoritative lists of  R&D projects due to inconsistencies in the project information that  components reported which led to the reports not being comprehensive.  For example, when we asked about some significant variations in the  number of projects between 2016 and 2017, S&T officials told us that one  DHS component responded to the data call with a list of R&D activities  that included a “wish list” of R&D for their component, and not actual  ongoing R&D activities. DHS officials acknowledged that they do not have  a mechanism to ensure the comprehensiveness of information reported  by the components through the data call process. In addition, two  components did not respond to S&T’s request for R&D project information  for the 2017 Annual Report of Coordinated R&D.

Congressional Budget Justifications: In May 2018, in the absence of a  single, comprehensive list of R&D projects across DHS prior to the  issuance of its report in response to the NDAA, S&T officials referred us  to the R&D projects listed within the seven individual congressional  budget justifications for the components that currently have budget  authority to conduct R&D. Furthermore, as discussed earlier, S&T officials  used the congressional budget justifications as their starting point in  developing their response to the NDAA. However, S&T officials stated  that there may be differences between the projects listed in the NDAA  response and the projects listed in the congressional budget justifications.  For example, S&T stated that the report in response to the NDAA  includes all “ongoing” projects, regardless of the fiscal year in which they  received funds; while the congressional budget justifications include R&D  projects for which funding was requested for the given fiscal year. In other  words, S&T officials clarified, they included all R&D projects in their  response to the NDAA that had project activity, regardless of whether  funding was requested in a particular congressional budget justification.

S&T’s Project Tracker Database and the S&T Analytical Tracking  System: A 2014 House Appropriations Committee report noted that the  committee had repeatedly raised questions about S&T’s prioritization of  R&D projects and that, without the ability to easily review and compare  detailed information on all S&T projects and activities, the Under  Secretary for S&T could not effectively carry out S&T’s responsibilities.  Accordingly, the Committee directed S&T to develop a method or system  for tracking all S&T-funded R&D projects. A November 2016 DHS  Directive reiterates this requirement, specifying that the list of projects  should be updated on at least a quarterly basis throughout the duration of  an R&D project. S&T officials told us that, in response to the committee  report, they developed the Project Tracker Database, which was in use at  the time of our review, but was transitioned to a new system, the S&T  Analytical Tracking System, in September 2018. Neither the S&T  Analytical Tracking System nor its predecessor system, the Project  Tracker Database, is intended to comprehensively collect information on  R&D projects across the department, only for R&D projects managed  within S&T. Given the recent implementation of the S&T Analytical  Tracking System, it is too soon to tell whether it will improve and  streamline S&T’s efforts to collect and analyze R&D-related information  within the directorate.

In addition, S&T officials stated that none of the above R&D information  sources are suited to long-term trend analysis or data aggregation of  department-wide R&D project information, and that these sources are  disparate across DHS. S&T officials also acknowledged that better  aligning R&D project information sources is an important aspect of  improving how the department collects information DHS-wide.

Standards for Internal Control in the Federal Government call for  agencies to maintain quality information that is, among other things,  current, accurate, accessible, and provided on a timely basis.  Furthermore, the standards call for an agency’s management team to  process relevant data from reliable sources and utilize it to make informed  decisions. The disparate R&D project information sources that S&T  maintains, such as DHS’s response to the National Defense Authorization  Act of 2017 and the Annual Reports of Coordinated R&D discussed  above, and the manual data-call process it takes to update the sources  limits departmental access to current and reliable R&D project information. For example, an internal DHS web-based portal with pre- defined fields could provide component officials with a means for  reporting information more consistently and comprehensively. Without  complete and readily accessible R&D information, DHS may not have the  information it needs to make informed decisions about R&D investments,  such as which projects are to be prioritized. By developing a mechanism  to address challenges and limitations related to the collection, integration,  and comprehensiveness of R&D data across the department, S&T can  improve its visibility on R&D efforts across DHS in accordance with its  role as DHS’s coordinator of R&D efforts.

DHS Has Not Effectively Developed Some R&D Performance Management Information, Posing Challenges to Assess and Report on the Progress of R&D Projects

DHS components have processes in place to collect certain indicators of  R&D performance, but we found that these processes have limitations.  The methods used to assess and report performance and progress of  DHS R&D efforts we identified include:

Milestone information – used to assess and communicate progress to  Congress and agency decision makers on individual R&D projects

Strategic and Management Performance Goals – milestone and other  information is aggregated to provide summary information on R&D  performance by mission area

Customer feedback – information gathered by component officials on  R&D customer perspectives on the utility of ongoing or completed  projects  Below is our analysis of the three methods.

Milestones are often used as the basis of an alternative form of  performance goal. Performance goals specified in alternative form must  be described in a way that makes it possible to discern if progress is  being made toward the goal. Milestones related to DHS R&D efforts are  reported to Congress and publicly available through the DHS  congressional budget justification. A milestone is a scheduled event  signifying the completion of a major deliverable or a phase of work, and  can be described in a way that makes it possible to discern if progress is  being made toward a goal. Milestones can also help agencies  demonstrate that they have clear and fully developed strategies and are  tracking progress to accomplish their goals. In our analysis of 14  milestones for seven S&T high-priority R&D projects identified in fiscal  year 2018 DHS budget justification documents, we found that 3 of the 14  milestones fully adhered to DHS guidance for milestone descriptions.  DHS budget development guidance suggests DHS components, which  develop milestones for inclusion in congressional budget justification  documents, utilize leading practices provided in the guidance. The  leading practices state that successful milestones contain the following  characteristics:  1.  Specific - provide a clear understanding of expected results;  2.  Measurable - the result can be reported in quantitative or qualitative  3.  Results-Oriented/Relevant - milestone clearly links to results-oriented  activities such as strategy, budget, and/or program/project plans;  4.  Time-Bound - milestone specifies a beginning and end date for  As shown in table 3, we identified that more than half of the milestones (8  of 14) were not specific and 10 of 14 were not results-oriented. Eleven of  14 milestones we analyzed were measurable and time-bound. While our  analysis is not generalizable to all fiscal year 2018 R&D milestones, it  illustrates areas where the selected milestones do not fully incorporate  the DHS guidance.

Below is more detail on our assessment of the Specific and Results- Oriented guidance.

Specific. Of the 14 milestones we reviewed, eight did not contain specific  information that would allow reviewers to have a clear understanding of  the result expected in connection with the milestone. For example, one  milestone for a cyber-related R&D project states that “testbeds and pilots  would be conducted with at least one department or agency.” However,  the milestone is not specific enough to ascertain what types of testbeds or  pilots are being assessed and how the testbed effort would link to a  possible end result.

Results-Oriented and Relevant. Ten of the 14 milestones that we  reviewed did not clearly link the milestone back to results-oriented  activities, such as strategy, budget, or project/program plans. For  example, one milestone for a first responder program stated the following:  “Transition, commercialize, or make available through open source  platforms at least three technologies (e.g., Analyses, models, technology  prototypes and/or knowledge prototypes).” It is unclear which  technologies would be transitioned or how these technologies would be  transitioned and made available.

According to DHS OCFO officials, DHS congressional budget  justifications, which include milestones, serve to provide explanation and  detail for why DHS believes Congress should support the department’s  R&D projects. DHS components are instructed by DHS’s budget office to  routinely submit their congressional budget justifications for internal DHS  review, which is a process and mechanism that results in the supporting  justifications for R&D funding requests. DHS OCFO officials also stated  that they are not aware of a singular reason for why milestones do not  consistently incorporate DHS’s guidance and stated that they have also  identified instances in which milestones do not align with the guidance. As  our analysis indicates, S&T’s milestones could better incorporate  milestone criteria included in DHS’s budget preparation guidance. Without  milestone information that more closely aligns with DHS guidance,  Congress and DHS decision-makers may not be able to fully assess  whether R&D projects are meeting specific goals within assigned time  frames or identify what adjustments, if any, may be needed to facilitate  the achievement of project goals and the R&D mission overall.

Strategic and Management Performance Goals

DHS has developed 12 performance goals to assess and report on its  R&D efforts, DHS is required to identify department-wide goals in its  strategic plan and annual performance report. For fiscal years 2016  through 2018, DHS’s Annual Performance Report included two strategic  performance goals related to S&T’s R&D efforts. DHS’s congressional  budget justification includes the two strategic performance goals as well  as 10 related management performance goals. For a detailed listing of  the 12 performance goals, see table 4.

Seven of the 10 management performance goals were for S&T R&D  efforts and the remaining three were for Domestic Nuclear Detection  Office’s R&D efforts, which cover the components that account for 96  percent of DHS’ fiscal year 2017 R&D obligations. DHS has  performance goals for mission programs that produce operational results  that link directly to the DHS Strategic Plan, according to officials from the  OCFO’s Program Analysis and Evaluation division. DHS also uses  milestones to track the progress of the other components’ R&D efforts.

DHS components that conduct R&D use various methods to collect and  analyze customer feedback to assess their R&D efforts, as shown in table  5. However, DHS is not well positioned to integrate the results because  limited customer feedback information is collected and analyzed.

Six DHS components that have R&D-related responsibilities evaluate  customers’ needs and improve customer satisfaction by listening to  customers’ feedback about the quality of deliverables they receive—both  good and bad— and making changes necessary to enhance that  deliverable. Specifically, officials from S&T, the Coast Guard, CWMD,  TSA, the Cybersecurity and Infrastructure Security Agency, and the  Secret Service stated they have varying methods in place for gathering  customer feedback regarding the progress and the results of R&D  activities and deliverables. Below is a summary of these components’  efforts to consider customer feedback.

S&T. S&T’s project management guide outlines a process for  ensuring customer requirements are being adequately met using a  customer survey that can be modified and provided to the customer to  complete at each major milestone. In addition, proceedings (e.g.,  minutes) from regularly scheduled meetings with customer and end  user groups may be used to gather information regarding value and  operational impact in lieu of a survey. The S&T survey asks  customers to rate their overall satisfaction with S&T products and  services, along with specific aspects of support, such as providing  products in time to meet needs and effectively keeping customers  informed. However, out of the 97 R&D activities that S&T reported in  fiscal year 2017 and the 110 activities in fiscal year 2016, S&T  collected one customer survey form.

Coast Guard. The Coast Guard also has a process in place for  surveying and interviewing its customers following the completion of  an R&D project and officials reported using this information for future  R&D planning. The Coast Guard’s survey instrument seeks feedback  on: customer satisfaction, timeliness, utility, and communications,  among other things. The customer service survey is distributed for  feedback on deliverables. At least 6 months after an R&D project is  completed, Coast Guard also conducts an in-person interview with  project sponsors to collect project transition performance success and  feedback information. The surveys that Coast Guard uses to obtain  feedback elicit a relatively low number of responses from customers,  significantly limiting their usefulness in soliciting feedback data.  Specifically, the response rates for fiscal years 2013-2017 were 16%,  17%, 27%, 13%, and 17%, respectively. Experts on customer  satisfaction measurement have stated that although survey response  rates are never 100 percent, an organization should strive to get its  rate as close as possible to that number. They suggest that ideally,  organizations can obtain response rates of over 70 percent.

CWMD. CWMD does not have a formal mechanism, such as standard  processes and procedures, for collecting and analyzing customer  feedback. However, CWMD officials stated that certain informal  mechanisms are used to collect customer feedback. For example,  CWMD officials reported that the CWMD Office of Policy, Plans,  Analysis, and Requirements Directorate communicate with customers  and gather customer needs and requirements. In addition, as part of  these informal mechanisms, internal and external reviews feedback  may be obtained from eventual end users of the R&D technology such  as operators from CBP, the U.S. Coast Guard, and the TSA,  according to CWMD officials.

Cybersecurity and Infrastructure Security Agency. The directorate  does not have a formal mechanism for collecting and analyzing  customer feedback. However, periodic control gates are used to  gather customer feedback, according to directorate officials. The input  received during these reviews is used to make corrective actions and  manage R&D efforts as necessary. For example, according to  directorate officials, they conduct a comprehensive review of R&D  coordination efforts annually to determine what was effective and  what can be improved.

TSA. TSA does not have a formal mechanism for collecting and  analyzing customer feedback. However, according to TSA officials,  informal feedback may be obtained through review of the weekly  reports and meetings regarding recent developments and project  milestones. In addition, feedback may be obtained during quarterly  program management reviews, third party project development, and  certification testing.

Secret Service. The Secret Service does not have a formal  mechanism for collecting and analyzing customer feedback. However,  according to Secret Service officials, informal feedback may be  obtained in conjunction with other related internal review activities,  including program management reviews.

To formalize and improve customer feedback processes for R&D efforts,  the National Academy of Sciences has stated that feedback from both  R&D failures and successes may be communicated to stakeholders and  used to modify future investments. Research on leading practices in the  area of customer satisfaction suggests that multiple approaches are  needed to effectively listen to customers about their perceptions of quality  service and needs. The research also points to a need for centrally  integrating all customer feedback so that managers can achieve a better  understanding of customers’ perceptions and needs. Also, we have  previously reported that leading organizations combine quantitative and  qualitative listening tools to obtain customer feedback and then centrally  integrate the data in one location. Such approaches include the  following:

Customer satisfaction surveys. We previously reported that most  major organizations use tools such as surveys to periodically capture  customers’ overall perceptions about their organization and to  measure satisfaction with specific transactions soon after they  occur. These surveys can be administered through the mail, by  telephone, in person, or electronically.

Benchmark surveys. Benchmark surveys gather perceptions of  performance from the entire market. These surveys usually gather  customer perceptions of performance about top competitors in an  industry. This allows the company to examine its customer-perceived  strengths and weaknesses in the overall marketplace. While  continuous improvement may be a result of this listening tool, the real  value, according to the research in this area, comes from  breakthrough thinking to gain a sustainable advantage.

Focus groups. Organizations use focus groups to get better  information from customers than survey results provide. In these  groups, customers are probed about why they answered survey  questions the way they did.

Customer interviews. Conducting interviews with customers can  provide a way to get very detailed information about their specific  needs and problems. Like focus groups, this tool is used by leading  customer service organizations to probe survey respondents as to  why they answered survey questions a certain way.

The National Academy of Sciences have stated that evaluating the  relevance and impact of R&D is a key stage of the R&D process and that  measuring the impact of R&D activities requires looking to the end users  and stakeholders for an evaluation of the impact of a research program,  such as through polling or systematic outreach. In addition, Standards  for Internal Control in the Federal Government calls for entities to  determine an oversight structure to fulfill responsibilities that are set forth  by feedback from key stakeholders, among other things. As a result of the  limited customer feedback information that is collected and analyzed,  DHS is unable to more fully understand its customers’ perceptions and  experience to allow it to assess the benefits and performance of its R&D  efforts. Moving forward, standard processes and procedures for collecting  and analyzing R&D customer feedback would help in assessing R&D  efforts.

Since 2010, DHS has obligated more than $10 billion dollars on R&D to  develop technologies to support DHS’s efforts to prevent, mitigate, and  recover from terrorist and natural threats. S&T officials indicated that they  have strong collaborative relationships with components; however, it is  important that required components fully participate in the IPT process in  order for S&T to maintain visibility of R&D projects and successfully fulfill  its statutory role of coordinating R&D and to help reduce the risk of  potential duplication of R&D efforts across the department. Furthermore,  S&T faces challenges and limitations related to the collection, integration,  and comprehensiveness of information on R&D projects. Without a  mechanism that aligns information sources and results in comprehensive  and accurate data, among other things, DHS may not have the  information it needs to make informed decisions about R&D investments.  S&T also does not fully leverage existing guidance when developing  milestones for R&D efforts. Without milestone information that more fully  aligns with DHS criteria, Congress and DHS decision-makers may not  have a full understanding of R&D progress and challenges. Finally,  standard processes and procedures for collecting and analyzing R&D  customer feedback would help to assess its R&D efforts.

We are making the following four recommendations to the Deputy  Secretary of the Department of Homeland Security:  The Deputy Secretary of the Department of Homeland Security should  ensure that all components adhere to IPT participation requirements, in  accordance with the DHS directives. (Recommendation 1)

The Deputy Secretary of the Department of Homeland Security should  develop a mechanism that aligns processes and information sources for  collecting R&D project data from DHS components to ensure that the  information can be collected, integrated and result in a comprehensive  accounting of R&D projects DHS-wide. (Recommendation 2)

The Deputy Secretary of the Department of Homeland Security should  direct OCFO program officials to ensure that S&T take steps to more fully  incorporate leading practices, such as those included in DHS’s budget  preparation guidance, into R&D milestones. (Recommendation 3)

The Deputy Secretary of the Department of Homeland Security should  develop standard processes and procedures for collecting and analyzing  customer feedback, applicable to components conducting R&D, for  improving the usefulness of existing customer feedback mechanisms to  assess R&D efforts and for implementing such mechanisms where  absent. (Recommendation 4)

We provided a draft of this report to DHS for review and comment. DHS  provided written comments which are reproduced in appendix II. In its  comments, DHS concurred with our recommendations and described  actions planned to address them. S&T, OCFO, CBP, the Cybersecurity  and Infrastructure Security Agency, and CWMD also provided technical  comments, which we incorporated as appropriate.

With regard to our first recommendation, that the Deputy Secretary of the  Department of Homeland Security should ensure that all components  adhere to IPT participation requirements, in accordance with DHS  directives, DHS stated that S&T’s Office of Science & Engineering will  revise the relevant DHS directive to require participation in the IPT  process by all components. DHS estimated that this effort would be  completed by December 31, 2019. This action, if fully implemented,  should address the intent of the recommendation.

With regard to our second recommendation, that the Deputy Secretary of  the Department of Homeland Security should develop a mechanism for  collecting R&D project data in order to complete a comprehensive  accounting of R&D projects DHS-wide, DHS stated that S&T’s Office of  Science & Engineering will revise the relevant DHS directive to include  requirements for data collection on all R&D projects across DHS to  ensure alignment of the appropriate data elements and existing guidance.  DHS estimated that this effort would be completed by December 31,  2019. This action, if fully implemented, should address the intent of the  recommendation.

With regard to our third recommendation, that the Deputy Secretary of the  Department of Homeland Security should direct OCFO program officials  to ensure that S&T take steps to more fully incorporate leading practices,  such as those included in DHS’s budget preparation guidance, into R&D  milestones, DHS stated that the OCFO will continue to work with S&T to  incorporate the leading practices and that the OCFO will validate all S&T  annual budget submissions and provide S&T feedback, as appropriate.  DHS estimated that this effort would be completed by April 30, 2020. This  action, if fully implemented, should address the intent of the  recommendation.

With regard to our fourth recommendation, that the Deputy Secretary of  the Department of Homeland Security should standardize processes for  collecting and analyzing customer feedback to aid in assessing R&D  efforts, DHS stated that S&T’s Office of Science & Engineering will revise  the relevant DHS directive to incorporate customer feedback procedures  into the IPT process for the recipients of R&D programs. DHS estimated  that this effort would be completed by December 31, 2019. This action, if  fully implemented, should address the intent of the recommendation.

We are sending copies of this report to the appropriate congressional  committees and the Secretary of Homeland Security. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff members have any questions about this report, please  contact William Russell at (202) 512-8777 or russellw@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. Key contributors to this report are  listed in appendix III.

Appendix I: Overview of the Science and Technology Directorate’s Research and Development Projects

The following appendix provides a general overview of the Department of  Homeland Security’s (DHS) Science and Technology Directorate (S&T)  research and development (R&D) projects and programs that support the  following homeland security mission areas: (1) border security; (2)  disaster resilience; (3) critical incidents, and (4) cybersecurity. The  examples provided below are illustrative and therefore not intended to  provide a comprehensive list of DHS R&D programs or projects.

S&T and the U.S. Coast Guard conducted drone demonstrations and  tests at two sites in Mississippi to test unmanned aerial systems  before they are deployed to the field.

The Technical Assessment of Counter Unmanned Aerial Systems  Technologies in Cities is designed to help public safety and industry  officials identify potential methods for countering nefarious uses of  small unmanned aerial systems.

The Wall System Design Support Tool Independent Verification and  Validation seeks to strengthen the U.S. Border Patrol’s decision  analysis model used to identify the areas of the border where a wall  would be most beneficial.

The Border Research in Instrumented Construction Project is  designed to identify cameras, sensors and other technology that can  be applied on or near a smart wall via ground, surface/air, subsurface  and water to enhance border security and agent safety.

The Apex Border Situational Awareness program aims to help U.S.

Customs and Border Protection access more data sources, develop  decision support tools and share information with partner law  enforcement agencies to improve situational awareness.

The Integrated Maritime Domain Enterprise is a platform that seeks to  bridge disparate data systems to make it easier for DHS components  to share information and collaborate.

The Adaptive Sensor Analytics Project aims to provide automated  data analytics to process satellite imagery, identify patterns of  nefarious activity and alert DHS officials.

Ground-Based Technologies Program seeks to improve the ability to  detect illegal activity at the border through stronger situational  awareness, automated detection and alerts, target classification and  tools to promote agent safety

Air-Based Technologies Program is designed to identify, test and  evaluate unmanned and manned aircraft platforms and sensors for  law enforcement, search and rescue, and disaster response in both  land and maritime environments.

The Expert Tracker training program aims to help U.S. Border Patrol  agents improve their ability to track movement in rough terrain along  the nation’s borders.

Port of Entry Based Technology Program seeks to improve illicit cargo  detection and legitimate cargo throughput by upgrading legacy  scanning systems and linking them to new analysis and information  sharing tools that may make the most of personnel resources.

The Port of Entry People Screening Program aims to identify,  evaluate and implement combinations of process and technology  improvements that facilitate the movement of people through the  nation’s air, land and sea ports of entry.

Autopsy is an open-source digital forensics platform that seeks to help  law enforcement determine how electronic devices were used in a  crime and recover evidence.

Voice Forensics aims to help identify individuals who make hoax  rescue calls to the U.S. Coast Guard, which may make it easier to find  and prosecute suspects.

Child Exploitation Image Analytics seeks to reduce the amount of time  it takes to identify and rescue children from exploitation, as well as  identify perpetrators, through automated face recognition algorithms  and forensic tools.

The Tunnel Detection and Surveillance Program is designed to help  border officials detect and locate clandestine tunnels, as well as  gather forensic data to support investigation and prosecution of drug  smuggling activities.

The Port of Entry Forensics and Investigations Program aims to help  combat transnational crime and investigate child exploitation and  human trafficking through open source data and forensic analysis of  material collected from suspicious packages and cargo.

DHS S&T seeks to help improve community resilience to natural disasters  through technology and tools that support planning, decision making and  mitigation efforts.

The Canada-U.S. Enhanced Resiliency Experiment series aims to use  real-world exercises to demonstrate that seamless communication is  possible between responders on either side of the northern border  during a large-scale emergency.

DHS S&T and the Central United States Earthquake Consortium are  developing a suite of decision support tools designed to help  emergency managers analyze data used when planning, managing,  coordinating and communicating during natural disasters

The Mutual Aid Resource Planner is a prototype application designed  to help jurisdictions develop more accurate resource plans by  incorporating custom data on geospatial hazards, risk assessments  and potential mutual aid partners.

The National Mutual Aid Technology Exercise seeks to test existing  mutual aid systems to improve users’ ability to exchange information  between systems in real time and develop technical guidance for  future use.

The Coastal Resilience Center of Excellence aims to conduct targeted  research and education to address key challenges facing coastal  communities in the United States, including storm surge modeling,  pre-disaster planning, communicating risk and more.

The Flood Apex Program is designed to help identify and develop  technology that can reduce flood-related fatalities and property loss,  increase community resilience and improve flood preparation,  response and recovery.

The Internet of Things Low Cost Flood Inundation Sensors project  seeks to develop and test sensor technology that can provide real- time updates on rising water levels.

The Kentucky Dam Safety project aims to create technology and  processes to better monitor dams and alert communities of potential  danger, reducing loss of life and property.

The Advanced CIRCulation modeling tool seeks to accurately predict  coastal flooding threats to help emergency managers better  coordinate evacuation and response.

The Hurricane Evacuation -eXtended platform is a decision support  tool for emergency managers designed to organize and stage  resources for hurricane response.

The Simulation-Based Decision Support System for Water  Infrastructural Safety Lite™ tool is designed to quickly model the  effects of potential dam breaks, helping officials develop accurate  emergency response plans and anticipate evacuation needs.

The Tunnel Plug is an inflatable device that aims to seal off subway  tunnels to prevent water from flowing into the system, minimizing  damage to critical transportation systems.

The Linking the Oil and Gas Industry to Improve Cybersecurity project  seeks to facilitate cooperative research, development, testing and  evaluation procedures to improve cybersecurity in petroleum industry  digital control systems

The Homeowner Flood Insurance Roundtable aims to help reduce  future uninsured flood losses by identifying decision support and  research and development needs.

The Automated National Structures Inventory project is seeking to  build a comprehensive list of private and commercial property at risk  for flood damage, which may help promote proper insurance and  more effective flood protection efforts.

The Smart Cities IoT innovation project is designed to help first  responders improve their situational awareness through advances in  autonomous drone navigation, intelligent building sensors and body- worn interoperability platforms.

The Wireless Emergency Alerts Research, Development, Testing and  Evaluation program aims to inform changes to the Federal  Communications Commission’s alerting system, including increased  character length and adding URLs, pictures, videos and geo-targeting  capabilities.

The System Assessment and Validation for Emergency Responders  program seeks to evaluate available responder technology on  affordability, usability, and other criteria to help agencies understand  which equipment will best fit their needs.

The Urban Operational Experimentation program is designed to let  responders test new technologies in real-world settings, and may help  provide developers with direct feedback on how their products can  better meet operational needs.

The Enhanced Dynamic Geo-Social Environment training platform is a  free virtual tool that aims to allow responders to practice responding to  an active shooter incident, whether within a single agency or with  multiple jurisdictions and disciplines.

The Surface Transportation Explosives Threat Detection program is  aiming to develop screening technology that can identify potential  threats on people and in their bags without physically interacting with  them.

The Explosives Detection Canine program is designed to help  detection canine teams identify new explosive compounds through  non-hazardous training aids and increase their proficiency through  realistic self-assessment and training events across the country.

The Datacasting project aims to help responders send encrypted  video, data files, and other critical information through existing public  broadcast television signals, which helps prevent other  communication channels from being overwhelmed.

The Next-Generation Incident Command System, a web-based  platform, seeks to allow responders to share data and request  assistance in real-time, and also allows officials to observe and make  critical decisions during evolving situations to better support  preparation, response, and recovery.

The Android Team Awareness Kit, a free app, is designed to help  responders visually track team members and assets in real time  during an incident, as well as share encrypted data across  jurisdictions, disciplines, and components.

The Assistant for Understanding Data through Reasoning, Extraction  and Synthesis platform aims to help responders overcome information  overload by providing actionable insight based on up-to-the-minute  sensor data.

The First Responder Electronic Jamming Exercise seeks to identify  mitigation tactics against intentional or accidental communications  jamming, which responders were able to practice implementing in  realistic scenarios.

The Telephony Denial of Service program is designed to help improve  911 emergency call centers’ ability to defend against attacks through  cyber security technologies that can analyze incoming calls and may  help determine potential threats in real time.

The Finding Individuals for Disaster and Emergency Response is  designed to detect human heartbeats under up to 30 feet of rubble,  which may help responders more effectively target rescue efforts.

The Rapid DNA technology can complete a DNA test within 90  minutes or less from the field, which seeks to help officials identify  victims and inform family members in a timely manner.

The Forensic Video Exploitation and Analysis tool aims to help  responders quickly analyze video to identify potential suspects by  allowing users to tag a person to a left-behind item and reconstruct  that individual’s path across multiple camera views.

The Cyber Risk Economics program seeks to fund applied R&D,  knowledge products by gathering stakeholders across government,  industry and academia to discuss cyber risk economics capability  gaps and needs. Through these stakeholder discussions, along with  scholarly cybersecurity economics research literature reviews and  authoritative U.S. federal government documents, DHS S&T  developed the newly released Cyber Risk Economics Capability Gaps  Research Strategy which aims to consider business, legal, technical  and behavior factors impacting cyber risk.

Appendix II: Comments from the Department of Homeland Security

Appendix III: GAO Contact and Staff Acknowledgements

William Russell, (202) 512-8777 or russellw@gao.gov.

In addition to the contact named above, Ben Atwater (Assistant Director),  Melissa Hargy (Analyst-in-Charge), Nanette Barton and Gary M.  Malavenda made key contributions to this report. In addition, key support  was provided by Chris P. Currie, Dominick Dale, Michele Fejfar, Richard  Hung, Benjamin Licht, John Mingus, Janet Temko-Blinder, and Sarah  Veale.

The Department of Homeland Security’s Science and Technology Directorate is required to coordinate research and development (R&D) across the department, which helps to prevent duplicative efforts.

We found that the department's R&D project information is stored in various disparate sources (e.g., reports, data systems, etc.)—which makes it difficult to identify and track. By developing a method to align these sources, the directorate can better compile and analyze R&D project information.