Disaster Assistance: Opportunities to Enhance Implementation of the Redesigned Public Assistance Grant Program

FEMA, an agency of the Department of Homeland Security (DHS), has obligated more than $36 billion in PA grants to state, local, and tribal governments to help communities recover and rebuild after major disasters since 2009. Further, costs are rising with disasters, such as Hurricanes Harvey, Irma, and Maria in 2017. FEMA recently redesigned how the PA program delivers assistance to state and local grantees to improve operations and address past challenges identified by GAO and others. FEMA tested the new delivery model in selected disasters and announced implementation in September 2017.

GAO was asked to assess the redesigned PA program. This report examines, among other things, the extent to which FEMA's new delivery model addresses (1) past workforce management challenges and assesses future workforce needs; and (2) past information sharing challenges and key IT management controls. GAO reviewed FEMA policy, strategy, and implementation documents; interviewed FEMA and state officials, PA program applicants, and other stakeholders; and observed implementation of the new model at one test location following Hurricane Matthew in 2016.

The Federal Emergency Management Agency (FEMA) redesigned the Public Assistance (PA) grant program delivery model to address past challenges in workforce management, but has not fully assessed future workforce staffing needs. GAO and others have previously identified challenges related to shortages in experienced and trained FEMA PA staff and high turnover among these staff. These challenges often led to applicants receiving inconsistent guidance and to PA project delays. As part of its new model, FEMA is creating consolidated resource centers to standardize and centralize PA staff responsible for managing grant applications, and new specialized positions, such as hazard mitigation liaisons, program delivery managers, and site inspectors, to ensure more consistent guidance to applicants. However, FEMA has not assessed the workforce needed to fully implement the new model, such as the number of staff needed to fill certain new positions, or to achieve staffing goals for supporting hazard mitigation on PA projects. Fully assessing workforce needs will help to ensure that FEMA has the people and the skills needed to fully implement the new PA model and help to avoid the long-standing workforce challenges the program encountered in the past.

FEMA designed a new PA information and case management system—called the FEMA Applicant Case Tracker (FAC-Trax)—to address past information sharing challenges, such as difficulties in sharing grant documentation among FEMA, state, and local officials and tracking the status of PA projects, but additional actions could better ensure effective implementation. Both FEMA and state officials involved in testing of the new model stated that the new information system allows them to better manage and track PA applications and documentation, which could lead to greater transparency and efficiencies in the program. Further, GAO found that this new system fully addresses two of four key information technology (IT) management controls—project planning and risk management—that are necessary to ensure systems work effectively and meet user needs. However, GAO found that FEMA has not fully addressed the other two controls—requirements development and systems testing and integration. By better analyzing progress on high-level user requirements, for example, FEMA will have greater assurance that FAC-Trax will meet user needs and achieve the goals of the new delivery model.

GAO is making five recommendations, including that FEMA assess the workforce needed for the new delivery model and improve key IT management controls for its new information sharing and case management system, FAC-Trax. DHS concurred with all recommendations.

In August 2017, Hurricane Harvey caused catastrophic and  unprecedented flooding across southeast Texas and into Louisiana.  While federal agencies support the evacuation, rescue, and relief to  hundreds of thousands of residents, projections of damages exceed $50  billion. In the weeks that followed, Hurricanes Irma and Maria brought  winds and storm surge, which caused damage across the U.S. Virgin  Islands, Puerto Rico, and the southeast United States, adding to the  projected disaster costs. These events are emblematic of a trend of rising  costs from natural disasters. In both 2016 and 2017, 15 separate U.S.  disasters resulted in losses exceeding $1 billion each. Each year, the  federal government obligates billions of dollars to programs and activities  that provide assistance to state and local governments, tribes, certain  nonprofit organizations, and individuals that have suffered injury or  damages from major disaster or emergency incidents, such as  hurricanes, tornados, or fires. We reported in 2016 that the federal  government obligated at least $277.6 billion in disaster assistance during  fiscal years 2005 through 2014 and have recognized the rise in the  number—and the increase in severity—of disasters as a key source of  federal fiscal exposure.

The Federal Emergency Management Agency (FEMA), a component of  the Department of Homeland Security (DHS), leads the federal effort to  mitigate, respond to, and recover from disasters, both natural and  manmade. FEMA’s Public Assistance (PA) grant program provides  financial assistance to state, tribal, territorial, and local governments for  debris removal; emergency protective measures; and the repair,  replacement, or restoration of disaster-damaged, publicly owned facilities.

For example, flooding across northern and central California in early  February 2017 resulted in substantial property and infrastructure damage,  including severe damage to the Oroville Dam spillway. Shortly thereafter,  the governor of California requested federal assistance, and the President  subsequently issued an emergency declaration that made funding  available for repairs to the spillway through the PA program. PA is  FEMA’s largest disaster assistance grant program and from fiscal years  2009 through 2016, the agency obligated more than $36 billion in grants  for such projects.

PA is a complex and multistep grant program administered through a  partnership between FEMA and the state grantee, which provides funding  to local officials who are the subrecipients of a PA grant award. Thus, it  entails an extensive paperwork and review process between FEMA and  grantee officials based on specific eligibility rules that outline the types of  damage that can be reimbursed by the federal government and steps that  federal, state, and local governments must take in order to document  eligibility. Due in part to the complexity of the process, we identified a  number of past challenges in various aspects of the program. Specifically,  we assessed the PA program in 2008 during the Gulf Coast states’  recovery from Hurricanes Katrina and Rita. We identified a number of  challenges related to FEMA’s management of its PA workforce, including  lack of training and high turnover among PA staff, which led to project  delays and other problems. We also identified challenges related to  information sharing between FEMA and PA grant applicants, which led to  problems in tracking the status of projects, among other things. Moreover,  in 2015 we identified challenges in effectively incorporating mitigation into  PA projects and grant guidance during the recovery from Hurricane  Sandy in the northeastern United States. To address these various  challenges, we have made a number of recommendations, and FEMA  has taken or is taking various actions to address them, as we discuss  later in this report.

In recent years, FEMA has taken steps to redesign the PA program to  address past challenges and make the program easier for FEMA and  grantee officials to manage. As part of this effort, FEMA redesigned  processes for developing, reviewing, and approving grant applications.  The agency is also developing new PA staff positions; implementing a  centralized and standardized grant processing approach; and developing  a new information system to better maintain and share grant  documentation. FEMA officials also report taking steps to better  incorporate hazard mitigation during the PA grant process. Taken  together, these efforts represent FEMA’s “new delivery model” for  managing project development in the PA program, which we refer to as  the preaward process. FEMA has been testing the new delivery model at  certain disaster locations since 2015, in preparation for implementing it  nationwide for all new disasters.

In light of past challenges and recent changes to the PA program  preaward process, you asked us to assess FEMA’s progress in  addressing these challenges and implementing the new delivery model.  Specifically, this report addresses the extent to which FEMA designed the  new delivery model to address previously identified challenges related to  1.  managing its PA program workforce and the extent that FEMA  assessed the workforce needed to fully implement the new delivery  model;  2.  information sharing between FEMA and grant applicants and the  extent that FEMA’s new information sharing system satisfies key  information technology (IT) management controls; and  3.  incorporating hazard mitigation into the PA grant process.

To address all three objectives, we reviewed prior reports from GAO, the  DHS Office of the Inspector General (OIG), and FEMA’s own internal  assessments identifying past PA program management challenges  related to workforce management, information sharing, and hazard  mitigation. We compared information from these reports to information  and documentation of FEMA’s efforts to implement the new PA delivery  model to determine the extent that the new model is being designed to  address these challenges. We also reviewed program documentation,  such as after-action reports that FEMA developed after testing the new  model in disaster locations. We conducted 62 semistructured interviews  with FEMA officials at headquarters and in two selected regions out of the  10 FEMA regional offices, including interviews with members of FEMA’s  PA workforce at the centralized processing center in Denton, Texas, and  the Joint Field Office (JFO) in Savannah, Georgia, for Hurricane Matthew  recovery efforts in the state. We identified relevant officials based on job  title, responsibilities, and referrals from other officials. We asked  questions to identify their role in the implementation of the new delivery  model, their experiences with implementation, and their assessment of  the benefits and challenges of the model as it relates to our objectives  and scope. We also conducted 11 semistructured interviews with selected  state officials responsible for PA operations, to include those with and  without new delivery model experience. Lastly, we conducted three semistructured interviews with randomly selected applicants who applied  for PA funding after the Hurricane Matthew disaster in Georgia.  Although their responses are not generalizable across all applicants, they  helped inform our understanding of the experiences of recipients using  the new delivery model for the first time.

To address our first objective, we reviewed FEMA documentation,  including the 2014 PA Program Realignment report, the PA Cadre  Training Plan, and after-action reports from different tests of the new  delivery model implementation to identify key workforce management  changes under the new model. We also reviewed position descriptions,  job aids, and training materials developed by FEMA to support new  workforce roles and responsibilities under the new model. As part of the  interviews previously described, we interviewed officials managing  workforce activities under the new delivery model, including a PA  program coordinator for workforce. We compared FEMA’s workforce  management efforts under the new model to leading practices we have  identified for business process reengineering and strategic workforce  management.

To address our second objective, we reviewed documents related to  FEMA’s acquisition and development of the new information system for  the PA program. We interviewed program staff, stakeholders, and officials  from FEMA’s Recovery Technology Programs Division that are  responsible for developing the new system, to understand changes to  information sharing under the new model. We then compared FEMA’s  development efforts to key IT management control practices in the areas  of risk management, requirements development, project planning, and  systems testing and integration to determine the extent to which FEMA’s  efforts met these practices. We drew these key control practices from the  Software Engineering Institute’s Capability Maturity Model® Integration for  Acquisition and Development, the Project Management Institute’s Guide  to the Project Management Body of Knowledge (PMBOK® Guide), and  the Institute of Electrical and Electronics Engineers’ (IEEE) Standard for  Software and System Test Documentation.

To address our third objective, we reviewed documentation such as  FEMA PA policies and operations guidance related to incorporating  hazard mitigation into PA projects, as well as process guides, job aids,  and training materials developed under the new delivery model, to better  understand how mitigation is addressed. In addition, we interviewed PA  program officials; officials from FEMA’s Federal Insurance and Mitigation  Administration (FIMA), who assist PA officials to incorporate mitigation  into the recovery process; and state officials involved with the second and  third test implementations of the new model in Oregon and Georgia. We  also reviewed after-action reports from the second and third test  implementations of the new model and new delivery model performance  measures in the PA New Delivery Model Assessment Plan. We then  compared the information derived from the documentation and interviews  to the goals and objectives identified in FEMA strategic plans, the  National Disaster Recovery Framework, the National Mitigation  Framework, and Standards for Internal Control in the Federal  Government, to determine the extent that FEMA’s new delivery model  incorporates hazard mitigation planning into the PA process and includes  goals aligned with broader FEMA mitigation goals. Also, we obtained  data from FEMA’s existing PA information system—the Emergency  Management Mission Integrated Environment (EMMIE)—on the number  of PA projects that incorporated hazard mitigation in the third test  implementation of the new delivery model in Georgia. We interviewed PA  program officials, reviewed OIG reviews of the EMMIE system, and  reviewed related documentation and determined this EMMIE data was  reliable for describing the rate of hazard mitigation in PA projects.

We conducted this performance audit from July 2016 to November 2017  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Major disaster declarations can trigger a variety of federal response and  recovery programs for government and nongovernmental entities,  households, and individuals. FEMA’s Office of Response and Recovery  manages the PA grant program, providing funds to states, territorial  governments, local government agencies, Indian tribes, authorized tribal  organizations, and certain private nonprofit organizations in response to  presidentially declared disaster declarations to repair damaged public  infrastructure such as roads, schools, and bridges. Figure 1 shows the  total amount of PA funds obligated by county from January 2009 through  February 2017 for federal disaster declarations.

To implement the PA program, FEMA’s staff includes a mix of temporary,  reservist, and permanent employees under two authorities, the Stafford  Act and Title 5. Reservists make up the largest share of the PA  workforce, which consisted of 1,852 employees––1,041 reservists, 634  full-time equivalents, and 177 temporary Cadre of On-Call  Response/Recovery Employees––as of June 2017, according to PA  officials. Figure 2 summarizes the key characteristics for each type of  employee.

After a disaster, FEMA sends PA program staff to the affected area to  work with state and local officials to assess the damage prior to a disaster  declaration. FEMA officials establish a temporary Joint Field Office (JFO)  to house staff who will manage response and recovery functions after a  declared disaster (including operations, emergency response and support  teams, planning, administration, finance, and logistics). Once the  President has declared a disaster, PA staff work with grant applicants to  help them document damages, identify eligible costs and work, and  prepare requests for PA grant funds by developing project proposals.  These proposals may include proposals for hazard mitigation if the  hazard mitigation work is related to the repair of damaged facilities,  referred to as permanent work projects. Immediate emergency measures,  such as debris removal, are not eligible for hazard mitigation. Officials  then review and obtain approval of the projects prior to FEMA obligating  funds to state grantees. Figure 3 describes the process used to develop,  review, and obligate PA projects.

In addition to rebuilding and restoring infrastructure to its predisaster  state, the PA program can be used to fund hazard mitigation measures  that will reduce future risk to the infrastructure in conjunction with the  repair of disaster-damaged facilities. There is no preset limit to the  amount of PA funds a community may receive; however, PA hazard  mitigation measures must be determined to be cost effective. Some  examples of hazard mitigation measures that FEMA has predetermined to  be cost effective, if they meet certain requirements, include  installing shut-off valves on underground pipelines so that damaged  sections can be isolated during or following a disaster;  securing a roof using straps, clips, or other anchoring systems in  locations subject to high winds; and  installing shutters on windows or replacing glass with impact-resistant  material.

Applicants can also propose mitigation measures that are separate from  the damaged portions of a facility, such as constructing floodwalls around  damaged facilities to avoid future flooding. FEMA evaluates these  proposals, considering how the proposed measure protects damaged  portions of a facility and whether the measure is reasonable based on the  extent of the damage, and determines eligibility on a case-by-case  basis.

FEMA’s Federal Insurance and Mitigation Administration (FIMA) deploys  a cadre of mitigation staff to help coordinate and implement hazard  mitigation activities during disaster recovery, including PA hazard  mitigation. A primary task of these staff is to identify and assess  opportunities to incorporate hazard mitigation into PA projects. Generally,  if an applicant seeks to incorporate hazard mitigation measures into a PA  project, FIMA’s hazard mitigation staff develop a hazard mitigation  proposal.

Previous Challenges and Recommendations Related to the PA Program

We, the DHS OIG, and others have reported past challenges with  FEMA’s management of the PA program related to workforce  management, information sharing, and hazard mitigation. For example,  we reported in 2008 that the PA program had a shortage of experienced  and knowledgeable staff, relied on temporary rotating staff, and provided  limited training to their workforce, which impaired PA program delivery  and delayed recovery efforts after Hurricanes Katrina and Rita. We  found that staff turnover, coupled with information sharing challenges,  delayed projects when applicants had to provide the same information  each time FEMA assigned new staff and that poorly trained staff provided  incomplete and inaccurate information during their initial meetings with  applicants or made inaccurate eligibility determinations, which also  caused processing delays. We recommended that FEMA strengthen  continuity among staff involved in administering the PA program by  developing protocols to improve information and document sharing  among FEMA staff. In response, in 2013 FEMA instituted a PA  Consistency Initiative, which included hiring new managers for FEMA  regional offices, stakeholder training on PA program administration, and  using a newly developed internal website to allow staff to post and share  information to address continuity and knowledge sharing concerns during  disaster operations. FEMA also developed the Public Assistance Program  Delivery Transition Standard Operating Procedure to facilitate the transfer  of responsibility for PA program activities during cases of staff turnover  during recovery operations. Despite FEMA’s efforts to implement our  recommendations, the DHS-OIG, in 2016, found continuing challenges  after Hurricane Sandy with workforce levels, skills, and performance of  reservists, who make up the majority of the PA workforce.

Regarding information sharing, in 2008, we also identified difficulties  sharing documents among federal, state, and local participants in the PA  process and difficulties tracking the status of projects. We recommended  that FEMA improve information sharing within the PA process by  identifying and disseminating practices that facilitate more effective  communication among federal, state, and local entities. In response,  FEMA proceeded with the implementation of a grant tracking and  management system, called EMMIE, which was used previously in 2007.  However, in subsequent years we found weaknesses in how FEMA  developed the system and the DHS-OIG found that information sharing  problems similar to the ones identified in our 2008 report persisted.

Regarding hazard mitigation, we reported in 2015 that state and local  officials experienced challenges in using PA hazard mitigation during the  Hurricane Sandy recovery efforts because PA officials did not consistently  prioritize hazard mitigation, and in some cases discouraged mitigation  projects during the PA grant application process, among other  challenges. We recommended that FEMA assess the challenges state  and local officials reported, including the extent to which they can be  addressed, and implement corrective actions, as needed. In response to  our recommendation, FEMA developed a corrective action plan that  included actions and milestones for reviewing, updating, and  implementing PA hazard mitigation policy. FEMA also identified the PA  new delivery model as a solution for some of the challenges state and  local officials reported. Previously, the OIG also reported that PA program  officials did not consistently identify eligible PA hazard mitigation projects,  and that PA officials did not prioritize the identification of PA hazard  mitigation opportunities at the onset of recovery efforts after the 2005 Gulf  Coast hurricanes. See appendix I for a summary of findings and the  status of our past recommendations on challenges with workforce  management, information sharing, and hazard mitigation related to the  PA program since our last review in December 2008.

FEMA’s own internal reviews and outreach efforts have also identified  similar challenges. For example, at FEMA’s request the Homeland  Security Studies and Analysis Institute assessed the effectiveness and  efficiency of the PA program in 2011. The institute’s report outlined 3  key findings and 23 recommendations relating to the PA preaward  process. For example, the report found that FEMA could enhance training  programs to develop a skilled and experienced workforce; utilize  technology and employ web-based tools to support centralized  processing, transparency, and efficient decision making; and identify and  address potential special considerations, such as hazard mitigation  proposals, as early as possible in the preaward process to improve  consistency. In 2014, PA program officials analyzed the PA grant process  and used input from agency staff and officials involved in various aspects  of the program to identify potential improvements. The resulting Public  Assistance Program Realignment report found that challenges in  workforce management, information sharing, and hazard mitigation  continued, and included recommendations for improvement. For example,  the report concluded that a shortage of qualified staff, high turnover,  unclear organizational responsibilities, and inconsistent training were  long-standing and continuing challenges that impaired the PA pre-award  process. In addition, from January 2015 to April 2015, FEMA conducted  extensive outreach with more than 260 stakeholders across FEMA  headquarters, all 10 regions, 43 states, and 4 tribal nations to discuss  challenges in the PA program and opportunities for improvement. For  example, stakeholders identified challenges with ineffective information  collection during the preaward process and suggested identifying special  considerations, such as hazard mitigation, earlier in the PA process as an  idea for improvement. In response, FEMA began redesigning the PA  preaward process to operationalize the results of its 2014 report and  address areas for improvement identified through its outreach efforts.

FEMA awarded a contract for program support to help PA officials  implement a redesigned PA program in 2015. This included a new  process to develop and review grant applications, and obligate PA funds  to states affected by disasters; new positions, such as a new program  delivery manager who is the single point of contact throughout the grant  application process; a new Consolidated Resource Center (CRC) to  support field operations by supplementing project development,  validation, and review of proposed PA project applications; and a new  information system to maintain and share PA grant application  documents. As part of the new process, PA program officials identified  the need to ensure that staff emphasize special considerations, such as  hazard mitigation, earlier in the process. Taken together, these efforts  represent FEMA’s “new delivery model” for awarding PA program grants.  Enhancements in the PA program under the new delivery model are  presented in figure 4.

Regarding the new delivery model process, FEMA introduced several  changes to enhance outreach to applicants during the “exploratory call”— the first contact between FEMA and local officials—and during the first in- person meeting, called the “recovery scoping meeting.” FEMA also  revised decision points during the process, when program officials can  request more information from applicants, and applicants can review and  approve the completion of project development steps. FEMA also  incorporated special considerations, such as hazard mitigation, earlier in  the new process during the exploratory calls and recovery scoping  meetings. The changes and enhancements to the PA grant award  process in the new delivery model are presented in figure 5.

The new process divides proposed PA projects based on complexity and  type of work into three categories—100 percent completed, standard, and  specialized—that PA staff manage to expedite review or assign skilled  staff to technical projects as needed. If the applicant has already  completed work following a disaster, such as debris removal, it is  considered “100 percent completed” and JFO staff collect the necessary  documents and provide the information to the CRC staff who complete  the development of project applications, validate the information, and  complete all necessary reviews. Projects that require repairs and further  assistance from PA program staff at the JFO include “standard” and  “specialized” projects, which include a site inspection to document  damages, before the JFO staff provide the information to the CRC.  Further, PA program officials assign PA staff based on their skills and  experience to standard projects, which are less technically complex to  develop, and specialized projects, which are more technically complex  and costly. We discuss the new workforce positions FEMA developed for  JFOs and CRCs, the new information system FEMA developed to  maintain and share PA grant documents with applicants, and FEMA’s  efforts to incorporate hazard mitigation into PA projects later in this report.

Testing the New Delivery Model Prior to Full Implementation

Since 2015, FEMA has invested almost $9 million to redesign the PA  program through the reengineering and implementation of the new  delivery model, including about $4.7 million for contract support for  implementation, and $4 million for acquisition of the new information  system. FEMA tested the new delivery model in a series of selected  disasters, using a continuous process improvement approach to assess  and improve the process, workforce changes, and information system  requirements, prior to implementing the new model for all future  disasters. For example, FEMA first tested the new process in Iowa in  July 2015 and, in February 2016, PA program officials expanded their test  to include all of the new staff positions. In October 2016, PA program  officials added the new information system to achieve a comprehensive  implementation of all of the elements of the new delivery model for the  agency’s response to Hurricane Matthew in Georgia, two additional  disasters in Georgia in January 2017, and in Missouri, North Dakota,  Wyoming, Vermont, and two disasters in New Hampshire from June  through August 2017. The timeline for PA’s implementation of the new  delivery model is shown in figure 6.

According to program officials, FEMA planned to implement the new  model for all future disasters beginning in January 2018. However,  historic disaster activity during the 2017 hurricane season accelerated full  implementation. As a result, on September 12, 2017, FEMA officials  announced that, unless officials determined it would be infeasible in an  individual disaster, the program would use the new delivery model in all  future disasters.

FEMA Designed the New PA Delivery Model to Address Workforce Management Challenges, but Efforts to Support Full Implementation Could Be Enhanced

PA’s New Delivery Model Was Designed to Respond to Previously Identified Workforce Challenges

According to FEMA’s 2014 PA Program Realignment report and other  program documents, PA officials designed the new delivery model to  respond to persistent workforce management challenges related to  identifying the required number of staff and needed skills and training,  among other things, to improve the efficiency and effectiveness of the PA  preaward process. To address these challenges, PA program officials  centralized much of the responsibility for processing PA projects in the  CRCs, created additional new positions with specialized roles and  responsibilities in JFOs, and established training and mentoring programs  to help build the new staffs’ skills.

In 2016, PA program officials centralized some of the project activities  that otherwise were being carried out at individual JFOs at FEMA’s first  new CRC in Denton, Texas. Officials did so by establishing 18 new  positions, many of which directly correlated with positions that FEMA  deployed to individual JFOs in the legacy PA delivery model. According to  PA officials, centralizing positions will improve standardization in project  processing, and result in a higher quality work product. As part of the new  delivery model, PA program officials created a new hazard mitigation  liaison position for PA program staff at the CRC that did not previously  exist at individual JFOs. The other new positions that PA program officials  either created or centralized at the CRC included two specialized  positions responsible for costing and validating PA projects. Previously,  the PA project specialist deployed to the JFO would complete these tasks  and others; however, the consistency of project development varied  across the regions and disasters. In contrast, CRC staff are full-time  employees who receive training to specialize in completing standardized  project development steps for PA projects from multiple disasters on an  ongoing basis.

Program officials anticipate that centralizing new specialized staff at the  CRCs will also reduce PA administrative costs and staffing levels at the  JFOs. For example, staff at the CRCs, such as the new hazard mitigation  liaisons and insurance and costing specialists, could support project  development for multiple disasters and regions simultaneously, whereas  PA previously needed to deploy staff to each JFO to fulfill these roles. In  addition, once JFOs operating under the new model send projects to the  CRCs for processing and review, FEMA can more rapidly close its JFOs,  reducing associated administrative costs. For example, following  Hurricane Matthew, FEMA credited the new delivery model, in part, with  its ability to close the JFO in Georgia sooner than several other JFOs in  neighboring states not involved in the implementation of the new delivery  model.

PA program officials created new positions with more specialized roles  and responsibilities to help PA staff at JFOs provide more consistency in  the project development process and guidance to applicants. Program  officials split the broad responsibilities previously managed at the JFOs  by PA crew leaders and project specialists, into two new, specialized  positions—the program delivery manager and site inspector. The program  delivery manager serves as the applicant’s single point-of-contact  throughout the preaward process, manages communication with the  applicant, and oversees document collection. All three PA grant  applicants we spoke to following Hurricane Matthew in Georgia greatly  appreciated the knowledge and assistance provided by their program  delivery managers. Site inspectors are responsible for conducting the site  inspection to document all disaster-related damages; determining the  applicant’s plans for recovery, coordinating with other specialists, and  verifying the information collected with the applicant. Officials expect  deployed staff at JFOs can complete the fieldwork faster and provide  greater continuity of service to applicants. Further, program officials  believe that specializing roles will enable them to provide more targeted  training, and improve employee satisfaction.

Site inspection, hazard mitigation, and  environmental and historic preservation  specialists, along with a new Public  Assistance program mentor, conduct a site  inspection with the applicant to document  damages to a historic cemetery in Savannah,  Georgia, following Hurricane Matthew in  2016.

PA program officials designed new training and mentoring programs for  the new positions at the CRCs and JFOs and used a continuous  feedback process to update and improve the training, position guides,  and task books throughout the implementation of the new delivery model,  according to PA officials. According to a June 2017 update of the PA  Cadre Training Plan, training for the new model has five major focuses:  required training and skills for position qualification; on-site refresher  training; mentor training; regional-based state, local, tribal, and territorial  training; and training on the new information system. Specifically, officials  developed six new training courses, and identified which are required for  each position under the new delivery model. For example, a program  delivery manager at the JFO is required to complete both the program  delivery manager and site inspector specialist courses. As of June  2017, PA program officials had provided at least one new model training  course to 93 percent of their cadre (including program delivery manager  training to 366 individuals and site inspector training to 1,172 individuals)  and planned to provide 28 additional courses through September 2017 to  the PA cadre. According to regional and CRC officials, the training  courses and mentoring from experienced staff helped maximize new  staff’s capabilities in the new process.

PA Officials Planned Additional Training to Address Issues Identified during Implementation

Throughout the third implementation of the new delivery model, JFO and  CRC staff, as well as regional PA staff, stakeholders, and applicants,  identified staff skills and training as a key area that needed more attention  for full implementation of the new delivery model. Our work and FEMA’s  after-action reports from the third test in Georgia identified problems with  site inspector skills, which affected the timeliness and accuracy of  projects. Specialists and managers at the CRC noted that poorly trained  site inspectors did not consistently provide the necessary information  from the field, which resulted in delays for the CRC staff to process  projects, and after-action reports also identified challenges with site  inspector skills. According to a PA applicant in Georgia, the inconsistency  of skills and experience of their site inspector resulted in the need to  conduct a “do-over” site inspection on one of the applicant’s projects,  causing delays. PA staff and state officials attribute much of the site  inspectors’ skill gaps to their lack of training and experience in the  program. According to PA Region officials, providing timely training will be  a resource-intensive challenge for implementing the new delivery model  for all future disasters. For example, it can be difficult to train reservists  before FEMA deploys them to disasters, and many of the program’s  experienced reservists have retired or resigned, resulting in few mentors  for the program and a high need to provide training to inexperienced and  newly hired staff.

PA officials and stakeholders also emphasized the need for FEMA to  provide additional training for state and local officials to build capacity and  support the goals of the new delivery model. For example, according to  JFO officials at the third implementation, the new delivery model  increases responsibilities for applicants, who will require more applicant  training than FEMA currently provides. According to state officials,  applicant capabilities vary, and FEMA should provide training to state and  local officials on the new delivery model and the information system  before a disaster. Skill gaps among applicants could result in inconsistent  implementation of the new process, according to PA staff and  stakeholders, and PA staff said that training was important to prevent  applicants from reverting back to the legacy PA grant application process.

To support full implementation of the new delivery model for all disasters,  PA program officials have updated training courses for PA staff and  applicants, and planned additional training to address these challenges  and other lessons learned through the test implementation. For example,  PA officials told us they updated the site inspector training program in  May 2017 and scheduled a new site inspector training session in August  2017 to include more hands-on training to help address the skill gaps  identified for site inspectors. PA officials created a new training course for  FEMA’s regional offices, in part to enable regional PA staff to provide new  delivery model training to state and local officials. PA officials also  planned to develop a self-paced, online course for state and local officials  by the end of 2017.

Opportunities Exist to Enhance Workforce Assessment in the New Delivery Model

PA officials have not fully assessed the workforce needed for JFO field  operations, CRC staff, or FIMA’s hazard mitigation staff to support  implementation of the new delivery model for all future disasters. PA  program officials developed an initial assessment of the total number of  staff needed in the field and the CRCs in 2016 to estimate cost savings  associated with consolidating and specializing positions at the CRCs and  deploying fewer staff to JFOs. However, the assessment did not identify  the number of staff required to fill specific positions, including program  delivery managers and hazard mitigation specialists, needed to support  the new delivery model for full implementation. In reviewing the test  implementations of the new delivery model, we found that inadequate  staffing levels at the JFOs and CRCs, and with FIMA’s hazard mitigation  staff, affected staffs’ ability to achieve the goals of the new delivery  model.

Staff levels at the JFO. We identified challenges with having the right  number of program delivery managers and site inspection specialists  to achieve program goals for customer satisfaction, efficiency, and  quality in test implementations of the new delivery model. For  example, in the second test implementation of the new delivery model  in Oregon in 2016, PA did not deploy enough program delivery  managers to the disaster, which resulted in unmanageable caseloads  for program delivery managers, according to state and PA officials.  PA program officials assigned program delivery managers an average  caseload of 12 PA applicants, which was more than they could  effectively manage, according to PA staff, and program officials aim  for a caseload of 8 to 10 applicants. According to state officials, local  officials reported they did not always receive the support they needed  from program delivery managers who managed caseloads consisting  of dozens of projects at multiple sites for each applicant during the  Oregon implementation. As a result of overwhelmed program delivery  managers, local officials faced challenges understanding their  responsibilities, such as recognizing when they needed to provide  information for the project development to proceed, according to state  officials. PA staff involved with the third test implementation in  Georgia in 2016 and 2017 said there were not enough site inspectors  or program delivery managers to fully manage the workload at the  JFO. Because of the specialization of roles, projects could not move  forward when there were not enough staff to execute the next step in  the process. For example, PA staff at the JFO said program delivery  managers completed recovery scoping meetings rapidly, but faced a  bottleneck in scheduling site inspections because there were more  applicants awaiting site inspections than could be fulfilled by the  number of site inspection specialists available.

Staff levels at the CRC. Staff at the CRC reported challenges with  staffing levels during the Oregon and Georgia test implementations,  and expressed concerns about when PA officials will staff the CRCs  to support full implementation of the new model for all disasters.  During the Oregon test implementation, a CRC specialist said there  were not enough technical specialists to manage the workload and, as  a result, PA program officials had to redeploy site inspectors from  their JFO field operations to the CRC to complete costing estimates.  During the third test in Georgia, quality assurance specialists said that  their workload resulted in added stress trying to complete the work in  time while adhering to quality standards. According to CRC specialists  in Denton, Texas, PA officials had not determined required staff levels  for full implementation, but agreed that workload was too high and  program officials needed to determine the appropriate staff levels for  each CRC to support full implementation. PA officials were still  evaluating CRC processing times and workload management from the  Oregon and Georgia test implementations to determine staffing  needs, according to PA officials. Further, PA program officials plan to  establish a second CRC in Winchester, Virginia, before the end of  2017, but have not determined the number of additional permanent  full-time staff needed to support the CRCs for full implementation of  the new delivery model.

Staff levels for the hazard mitigation specialists. PA officials have  not identified the number of hazard mitigation specialists in FIMA’s  hazard mitigation cadre needed for full implementation of the new  delivery model. According to JFO staff, current hazard mitigation staff  levels are insufficient to provide the desired in-person participation of  hazard mitigation staff on all recovery scoping meetings to share  information on hazard mitigation with applicants and help them  identify potential mitigation opportunities. A PA program official said  officials missed opportunities to pursue hazard mitigation during the  test implementation after Hurricane Matthew in Georgia due to lack of  hazard mitigation specialists. In addition, for the test implementation in  Oregon, there were not enough hazard mitigation specialists to cover  all site inspections and implement their new delivery model  responsibilities, according to FEMA’s after-action reports. The  absence of hazard mitigation specialists in the early stages of PA  project development may cause delays in officials’ identifying hazard  mitigation opportunities, according to a FIMA official. PA program  officials said they did not work with FIMA to determine the appropriate  levels of hazard mitigation staff under the new delivery model  because they were refining the new process, but as of June 2017  were working with FIMA to do so.

One of the key implementation activities in our Business Process  Reengineering Assessment Guide includes addressing workforce  management issues. Specifically, this includes identifying how many and  which employees will be affected by the position changes and  retraining. Further, our prior work has found that high-performing  organizations identify their current and future workforce needs—including  the appropriate number and deployment of staff across the organization— and address workforce gaps, to improve the contribution of critical skills  and competencies needed for mission success.

According to a PA program official, their initial workforce assessment was  not comprehensive because they were still collecting data required to  make informed decisions. PA officials agreed that updating their  workforce assessments prior to full implementation could be helpful, and  acknowledged that program officials needed to be more proactive  applying the lessons learned as they pivot from testing to full  implementation of the new delivery model in 2018. FEMA also conducts a  standard agency wide workforce structure review every 2 to 3 years,  which helps officials determine the appropriate disaster workforce levels.  As of June 2017, PA officials were working with other offices within FEMA  to expedite the agency-wide assessment of the PA and FIMA hazard  mitigation cadres, but did not know when they would complete the  assessment. PA officials also acknowledged that they faced an  aggressive schedule to complete various planned activities for workforce  management, training, and other efforts, in support of full implementation,  and that they may not be able to complete all efforts as thoroughly as  they would like in order to expedite the transition of the PA program to the  new delivery model.

The gaps in PA workforce assessment in the JFOs, CRCs, and for  FIMA’s hazard mitigation cadre present a risk that PA program managers  will not have a sufficient workforce to support the goals of the new  delivery model. In addition, the timing and implementation of the hiring  and training activities for new PA program staff could take multiple  months, and program officials will need to know what staff levels are  necessary for full implementation of the new delivery model to inform  resource decisions for the program in coordination with other agency  offices. According to PA program officials, workforce assessment efforts  have been delayed as a result of disaster response and recovery efforts  related to Hurricanes Harvey, Irma, and Maria. Completing a workforce  assessment will help program officials identify gaps in their workforce and  skills, which could help PA program officials minimize the effects of long- standing workforce staffing and training challenges on the PA program  delivery and inform full implementation for all disasters.

FEMA Designed the New PA Information System to Resolve Past Challenges, but Opportunities Exist to Fully Implement Key Management Controls

FEMA’s New PA Information System Is Designed to Resolve Long-Standing Information Sharing Challenges

costs. For example, EMMIE does not collect information on all of the  preaward activities that are part of the PA grant application process. As a  result, PA program officials said they, and applicants, must use ad hoc  reports and personal tracking documents to manage and monitor the  progress of grant applications. PA officials added that EMMIE is not user- friendly and applicants often struggle to access the system. In response  to these ongoing challenges, PA program officials developed FAC-Trax— a separate information system from EMMIE—with new capabilities  designed to improve transparency, efficiency, and management of the PA  program.

Specifically, FAC-Trax allows FEMA staff (PA Grants Manager) and  applicants (PA Grants Portal), to review, manage, and track current PA  project status and documentation. For example, applicants can use FAC- Trax to submit requests for public assistance, upload required project  documentation, approve grant application items, and send and receive  notifications on grant progress and activities. In addition, the FAC-Trax  system includes standardized forms, as well as required fields and tasks  that PA program staff and applicants must complete before moving on to  the next steps in the PA preaward process. According to PA officials,  these capabilities increase transparency, encourage greater applicant  involvement, and enhance collaboration and communication between  FEMA and grant applicants, to improve efficiency in processing and  awarding grant applications and enhance the quality of project  development. Further, PA officials said that FAC-Trax could reduce  challenges associated with staff turnover during the project development  process because the system stores and maintains applicant information  and project documentation, making it easier for transitioning staff to assist  an applicant. They also said they use FAC-Trax to gather and analyze  data that supports management of the PA process, including measuring  the timeliness of the grant application process. For example, during the  test implementation of the new delivery model in Georgia following  Hurricane Matthew, officials were able to document that, on average,  program delivery managers took 5 days to conduct the exploratory call  and 14 days to hold the recovery scoping meeting with applicants, and  CRC officials took 33 days to develop and review grant proposals.  Managers use this data to assess staffing needs and identify bottlenecks  in the PA process, according to PA officials.

Opportunities Exist to More Fully Implement Two of Four Key IT Management Controls for FEMA’s New PA Information System

FAC-Trax is critical to the new PA delivery model and will be a primary  means of sharing grant application documents, tracking ongoing PA  projects, and ensuring that FEMA staff and applicants follow PA grant  policies and procedures. Given the importance of developing and testing  this new information sharing system, we evaluated its development  against four key IT management controls—(1) project planning; (2) risk  management; (3) requirements development; and (4) systems testing and  integration. When implemented effectively, these controls provide  assurance that IT systems will be delivered within cost and schedule and  meet the capabilities needed by its users. We found that FEMA’s  development of FAC-Trax fully satisfied best practices for project planning  and risk management, but additional steps are needed to fully satisfy the  areas of requirements development and systems testing and integration,  as discussed below. See appendix II for the full assessment of each IT  management control.

PA program officials fully satisfied all five practices in the project planning  control area, according to our assessment. Key project planning practices  are (1) establishing and maintaining the program’s acquisition strategy,  (2) developing and maintaining the overall project plan and obtaining  commitment from relevant stakeholders, (3) developing and maintaining  the program’s cost estimate, (4) establishing and maintaining the  program’s schedule estimate, and (5) identifying the necessary  knowledge and skills needed to carry out the program. To address the  first and second practices, program officials established detailed plans  that describe the acquisition strategy and objectives, the program’s  scope, and its framework for using an Agile software development  approach, among other key actions. Agile is a method of software  development that utilizes an iterative process and constantly improves  software based on user needs and feedback. Program officials also  developed a plan detailing the program’s approach to deploy and  maintain FAC-Trax and established stakeholder groups and an integrated  product team to support and oversee the development of FAC-Trax. To  address the third and fourth practices, they developed and maintained a  master schedule of all implementation tasks and milestones through  project completion, and developed a life-cycle cost estimate of over $19  million. Additionally, FAC-Trax’s acquisition performance baseline  describes the system’s minimum acceptable and desired baselines for  performance, schedule, and cost. Lastly, in regards to the fifth practice,  program officials identified the knowledge and skills needed to carry out  the program in the FAC-Trax Request for Proposal and FAC-Trax  Capability Development Plan.

PA program officials fully satisfied all four practices in the risk  management control area, according to our assessment. Key risk  management practices are (1) identifying risks, threats, and vulnerabilities  that could negatively affect work efforts, (2) evaluating and categorizing  each identified risk using defined risk categories and parameters, (3)  developing risk mitigation plans for selected risks, and (4) monitoring the  status of each risk periodically and implementing the risk mitigation plan  as appropriate. To address the first and second practices, program  officials identified key risks that could negatively affect FAC-Trax in a “risk  register”—an online site used to track risks, issues, and mitigating  actions. As of May 2017, officials had identified 13 risks in the risk  register—four open and nine closed—and evaluated and categorized the  identified risks based on the probability of occurrence and scope,  schedule, and cost impacts. For example, program officials reported that  two of its open risks have a “medium” risk rating—meaning the risk has  the potential to slightly affect project cost, schedule, or performance. To  address the third and fourth practices, program officials developed and  documented risk mitigation plans for all identified risks. For example, program officials planned to mitigate the risk of limited engagement of  subject matter experts by identifying and engaging with appropriate  experts through workshops, and monitoring the capability development  process to identify any issues that may cause project delays. In addition,  PA program officials documented the responsible officials, reevaluation  date, and risk status, among other things, for each risk in the register, and  reviewed and updated risks during weekly and monthly program reviews  with stakeholders throughout FEMA.

PA program officials fully satisfied four out of five practices in the  requirements development control area, according to our assessment.  Key requirements development practices are (1) eliciting stakeholder  needs, expectations, and constraints, and transforming them into  prioritized customer requirements; (2) developing and reviewing  operational concepts and scenarios to refine and discover requirements;  (3) analyzing requirements to ensure that they are complete, feasible, and  verifiable; (4) analyzing requirements to balance stakeholder needs and  constraints; and (5) testing and validating the system as it is being  developed. To address the first and second practices, program officials  developed a requirements management plan outlining how officials  capture, assess, and plan for FAC-Trax enhancements, and established a  change control process to review, prioritize, and verify user requests for  changes to the system and feedback. As of May 2017, the PA program  office received 734 change requests related to FAC-Trax, of which  program officials completed 420 changes and planned to address an  additional 277 entries. Program officials also developed a functional  requirements document outlining the high-level requirements for FAC- Trax and detailed operational concepts and scenarios for each phase of  the preaward process in the system’s concept of operations. To address  the fourth practice, program officials created a standard template to  analyze and document the user needs and acceptance criteria for  planned system capabilities in March 2017. In addition, PA program  officials identified risks and dependencies for recommended changes to  FAC-Trax, and balanced the cost and priority of system enhancements as  part of the change control process. Lastly, regarding the fifth practice,  program officials tested and evaluated FAC-Trax during development,  which included validating system enhancements through user acceptance  testing.

However, program officials did not fully address the third practice— analyzing requirements to ensure they are complete, feasible, and  verifiable—because they did not ensure detailed user requirements were  necessary and sufficient by tracking them back to higher-level  requirements. For example, although program officials reviewed change  requests for completeness and followed up with users to verify  requirements, officials did not track system enhancements, made in  response to detailed user requirements (e.g., allowing users to search PA  projects by project number), back to the high-level requirements (e.g.,  storing data and information provided by the applicant) identified in the  FAC-Trax functional requirements document and performance work  statement. Officials did not track system enhancements back to high-level  requirements because they did not have a complete understanding of  basic user needs and system requirements at the beginning of the FAC- Trax effort, according to the PA program manager. A PA official also said  the change control process was a way to identify the basic capabilities  FAC-Trax needed to have and that tracking enhancements back to high- level requirements could have made the change control process more  difficult to manage, and reduced user participation if, for example, users  needed to understand how their change requests related to high-level  requirements. However, program officials could have tracked  enhancements back to high-level requirements themselves using the  change control process without putting any additional burden on users.  Despite not having a complete understanding of user needs and system  requirements at the beginning of the FAC-Trax effort, analyzing whether  users’ change requests satisfy higher-level requirements identified in key  design and planning documents would have provided officials with a basis  for more detailed and precise requirements throughout project  development and helped them better manage the project, according to IT  management controls. Further, according to the PMBOK® Guide,  tracking or measuring system capabilities against approved requirements  is a key process for managing a project’s scope, measuring project  completion, and ensuring the project meets user needs and  expectations.

Program officials acknowledged the importance of tracking system  enhancements back to documented system requirements. Ensuring that  FAC-Trax meets user needs and expectations is especially important  because the information system is key to the success of the new delivery  model, according to PA officials. By analyzing progress made on  documented, high-level requirements, a step that reflects a key IT  management control for requirements development, the PA program will  have greater assurance that FAC-Trax will provide functionality that  meets user needs and expectations.

PA program officials did not fully satisfy either of the two practices in the  systems testing and integration control area, according to our  assessment. Key systems testing and integration practices are (1)  developing test plans and test cases, which include a description of the  overall approach for system testing, the set of tasks necessary to prepare  for and perform testing, the roles and responsibilities for individuals or  groups responsible for testing, and criteria to determine whether the  system has passed or failed testing; and (2) developing a systems  integration plan to identify all systems to be integrated, describe how  integration problems are to be documented and resolved, define roles  and responsibilities of all relevant participants, and establish a sequence  and schedule for every integration step.

In regards to the first practice, PA program officials and the FAC-Trax  contractor established a test plan that identifies the method and strategy  to perform testing, including the necessary tasks, such as responding to  user feedback and testing errors, and incorporating necessary resolutions  into future work, testing parameters, and the roles and responsibilities of  the individuals responsible for testing. However, program officials have  not developed system testing criteria to evaluate FAC-Trax, which would  align with the practice described above of using criteria to determine  whether the system has passed or failed testing. A key feature of Agile  software development is the “definition of done”—a set of clear,  comprehensive, and objective criteria, that the government should use to  evaluate software after each iteration of development. PA program  officials said they did not establish a definition of done because officials  initially managing the FAC-Trax effort lacked familiarity with system  development in the Agile environment. Officials acknowledged the  importance of establishing a definition of done and said they are planning  to develop one, but have not identified how or when to incorporate it into  the development process. According to the TechFAR—the government’s  handbook for procuring digital services using Agile processes—the  government and vendor should establish this definition after contract  award at the beginning of each cycle of software development. By  establishing criteria, such as a definition of done, to evaluate the  system—a step that reflects a key IT management control for system  testing and is an effective practice for applying Agile to software  development—the PA program will have greater assurance that FAC- Trax is usable and responsive to specified requirements.

In regards to the second practice, PA program officials developed a  systems integration plan in June 2017 that identified the potential for  integration of FAC-Trax with four FEMA systems, including EMMIE. In  addition, program officials included a description of how staff should  document integration problems and the resolution of problems in FAC- Trax development and test plans. However, the systems integration plan  does not define roles and responsibilities of all participants for system  integration activities or establish a sequence and schedule for every  integration step for the four FEMA systems. PA officials said that system  integration planning for FAC-Trax is in the early stages, but  acknowledged the importance of these elements of system integration  planning. Officials plan to define roles and responsibilities of all  participants for system integration activities and develop the sequence  and schedule for every integration step as they add new systems to the  FAC-Trax development plan and obtain funding needed for their  integration. Nonetheless, FEMA has used FAC-Trax for selected PA  disasters since October 2016 and plans to use FAC-Trax for all future  disasters. According to IT management controls, agencies should  establish the systems integration plan early in the project and revise it to  reflect evolving and emerging user needs. By ensuring that the FAC- Trax systems integration plan defines the roles and responsibilities of  relevant participants for all integration relationships and establishes a  sequence and schedule for every integration step, the PA program will  have greater assurance that FAC-Trax functions properly with other  systems and meets user needs.

FEMA’s New PA Model Enhances Hazard Mitigation Staff Participation, but Opportunities Exist to Further Promote Mitigation

Changes under the New Model Include Enhanced Participation of Hazard Mitigation Staff

FEMA’s new delivery model enhances participation of hazard mitigation  staff with the goal of identifying opportunities for mitigation earlier in the  PA preaward process, according to PA officials. Two key changes related  to hazard mitigation under the new model include (1) an emphasis on  engaging with hazard mitigation specialists at the JFO earlier in the PA  process and involving them in specific PA preaward activities and (2) the  establishment of the PA program’s hazard mitigation liaison at the CRC.  For example, position guides direct program delivery managers to  coordinate with FIMA’s hazard mitigation specialists prior to recovery  scoping meetings, and site inspectors to coordinate with hazard mitigation  specialists prior to site inspections to discuss a PA grant applicant’s  damages and any potential mitigation opportunities. PA program officials  also developed guidance for conducting the exploratory call and the  recovery scoping meeting with applicants, which include questions for PA  staff to ask on the applicant’s interest in or plans for incorporating hazard  mitigation into potential projects. In addition, a new hazard mitigation  liaison at the CRC is responsible for reviewing PA projects for hazard  mitigation opportunities and serving as a mitigation subject matter expert  for the PA program.

According to data provided by FEMA, PA grant applicants incorporated  hazard mitigation into approximately 18 percent of permanent work  projects for all disasters nationwide from 2012 to 2015. During test  implementation of the new delivery model, state, PA, and FIMA officials  all reported an increase in the number of hazard mitigation activities on  PA permanent work projects. For example, state officials who participated  in the second new model test in Oregon said that effective communication  and coordination between PA and hazard mitigation staff resulted in  applicants incorporating hazard mitigation into over 60 percent of  permanent work projects. Furthermore, PA officials reported an increase  in hazard mitigation during the third test implementation of the new model  in Georgia following Hurricane Matthew, where approximately 16 percent  of permanent work projects included mitigation, as of June 2017. This  represents an increase compared to the PA program’s estimate for the  proportion of projects that incorporate hazard mitigation among previous  PA hurricane disasters in Georgia, which was about 3 percent, according  to PA officials. While PA officials are trying to increase hazard mitigation  through the new delivery model, not all disasters present the same  number of opportunities to incorporate hazard mitigation. First, the PA  program only incorporates hazard mitigation measures for permanent  work projects, such as repairs to roads, bridges, and buildings. For  example, as of June 2017, approximately 60 percent of the projects  FEMA funded in Georgia for the third test implementation after Hurricane  Matthew were for emergency work, which is not eligible for hazard  mitigation measures. Second, the PA program only funds mitigation  measures that officials determine to be cost-effective. In addition, we  have previously reported on other factors that affect whether applicants  incorporate hazard mitigation into PA projects, such as their capacity to  manage and ability to fund hazard mitigation projects.

Opportunities to Better Promote Hazard Mitigation under the New Model

Hazard Mitigation Planning and Prioritization

National Planning for Hazard  Mitigation  In our 2015 report on disaster resilience  following Hurricane Sandy, we noted that  disaster affected areas have different threats  and vulnerabilities, and local stakeholders  make the ultimate determination whether or  not to incorporate hazard mitigation into a  project. Further, without a strategic approach  to making disaster resilience investments, the  federal government and its nonfederal  partners may be unable to fully capitalize on  opportunities for mitigation on the greatest  known threats and hazards. We  recommended that the Mitigation Framework  Leadership Group develop an investment  strategy to help ensure that federal funds  expended to enhance disaster resilience  achieve the goal of reducing the nation’s fiscal  exposure because of climate change and the  rise in the number of federal major disaster  declarations as effectively and efficiently as  possible. In response, the Federal Emergency  Management Agency (FEMA) plans to issue a  final National Mitigation Investment Strategy  in 2018. The goals of this strategy include  increasing the effectiveness of investments in  reducing disaster losses and increasing  resilience, and improving coordination of  disaster risk management among federal,  state, local, tribal, territorial, and private  entities.

Although the new model establishes hazard mitigation activities for PA  and FIMA staff in the preaward process, it does not standardize and  prioritize hazard mitigation planning at JFOs in the way FEMA has done  with prior PA program policy. Specifically, FEMA’s 2007 PA program  policy standardized planning for hazard mitigation across PA recovery  efforts by stating that agency and state officials should issue a  memorandum of understanding (MOU) early in the disaster, outlining how  PA hazard mitigation will be addressed for the disaster, including what  mitigation measures will be emphasized, and identifying applicable codes  and standards, and any potential integration with other mitigation grant  programs. However, PA program officials did not include guidance that  standardizes planning for hazard mitigation, such as encouraging the use  of an MOU, in FEMA’s 2010 PA program policy, the most recent update  to the Public Assistance Program and Policy Guide in April 2017, or the  New Delivery Model Operations Manual. As a result, FIMA officials said  FEMA and state officials do not consistently issue MOUs that outline how  FEMA and the state plan to promote PA hazard mitigation during the  recovery effort, explaining that the use of the MOU is based on the  preferences and priorities of the FEMA officials involved. When not  issuing an MOU, FIMA hazard mitigation staff and PA officials at the JFO  meet to determine the extent which hazard mitigation staff interact directly  with applicants regarding PA hazard mitigation during the recovery  process, according to a FIMA official.

Having a consistent approach to planning for and prioritizing hazard  mitigation across all disasters is important for FEMA, given that FEMA  experienced challenges consistently prioritizing and integrating hazard  mitigation across PA recovery efforts, according to GAO and others. For  example, in our 2015 report on resilience in the Hurricane Sandy  recovery, we found that state and local officials experienced challenges  maximizing disaster resilience in the recovery effort because PA officials  did not consistently prioritize hazard mitigation during project  development.

According to FEMA’s National Mitigation Framework, planning is vital for  mitigation efforts during disaster recovery, and federal, state, and local  officials should establish procedures that emphasize a coordinated  delivery of mitigation activities and capitalize on opportunities to reduce  future disaster losses. Similarly, the Recovery Federal Interagency  Operational Plan, which supports FEMA’s National Disaster Recovery  Framework, identifies planning as a key task for identifying mitigation  opportunities and integrating risk reduction considerations into decisions  and investments during the recovery process. FIMA officials agreed that  including the development of a formal plan, such as the historical 2007  PA program policy regarding the use of MOUs, for PA hazard mitigation  in operations guidance would help program officials plan for and prioritize  hazard mitigation. They noted that FIMA’s hazard mitigation field  operations guide includes procedures for implementing proposed MOUs  to achieve mitigation goals. PA program officials said that, in light of  changes to the PA process under the new model and subsequent  updates to program policies, the MOU policy from the 2007 PA program  policy was outdated. But officials agreed that planning for and prioritizing  hazard mitigation at the operational level is important and said they were  examining additional ways to incorporate these activities early in the PA  process. As FEMA continues to implement the new model, establishing  procedures to standardize hazard mitigation planning for each disaster,  as it did through prior policy, could improve the prioritization of hazard  mitigation in PA recovery efforts and increase the effectiveness of  mitigation for reducing disaster losses and increasing resilience.

New Delivery Model Performance Objectives and Measures Could Better Align with FEMA’s Strategic Goal for Hazard Mitigation

PA program officials developed performance objectives and measures for  hazard mitigation in the new delivery model, but could add measures to  better align performance assessment for the PA program with FEMA’s  broader strategic goals for hazard mitigation. In its strategic plan for  2014–2018, FEMA established an agency-wide goal to increase the  percentage of FEMA-funded disaster projects, such as those under the  PA program, that provide mitigation above local, state, and federal  building code requirements by 5 percentage points by the end of fiscal  year 2018. For example, local building codes may require measures for  new construction to mitigate against future damage. To align with FEMA’s  strategic goal, PA officials would also need to measure the number of PA  projects that included mitigation measures that bring any repaired  infrastructure to a level above applicable building codes. However,  under the new model, FEMA officials developed performance objectives  (and associated measures) to increase the number of projects that  include hazard mitigation by 5 percent, and increase the total dollars  spent on hazard mitigation by 2 percent. While these measures could  help to incentivize mitigation, they are not tied to building codes and do  not include specific information that FEMA could use to continually  assess the PA program’s contributions to meeting the agency’s strategic  goal.

According to Standards for Internal Control in the Federal Government,  agency management should design control activities, such as  establishing and reviewing performance measures, to achieve the agency’s objectives. In addition, our work on leading public sector  organizations has found that such organizations assess the extent to  which their programs and activities contribute to meeting their mission  and desired outcomes, and strive to establish clear hierarchies of  performance goals and measures. A clear connection between  performance measures and program offices helps to both reinforce  accountability and ensure that, in their day-to-day activities, managers  keep in mind the outcomes their organization is striving to achieve.  FEMA’s ability to evaluate and report on PA hazard mitigation data is  constrained, but officials are addressing this challenge through the  development of data reporting and analytics capabilities for the PA  program’s new information system, according to PA officials. PA program  officials developed measures they could use to evaluate the new model  during test implementation and compare new model performance to the  legacy PA process, and agreed that aligning PA program hazard  mitigation goals with FEMA’s agency-wide strategic goals would be  helpful. As FEMA continues to develop and implement the new model,  developing performance measures and objectives to better inform and  support the agency’s broader strategic goals could help to ensure that  FEMA capitalizes on hazard mitigation opportunities in PA recovery  efforts.

FEMA’s Public Assistance grant program is a complicated, multi-billion  dollar program that is critical to helping state and local communities  rebuild and recover after a major disaster. In recent years, FEMA has  undertaken a major reengineering effort to make the PA preaward  process simpler and more efficient for applicants and to address  challenges encountered during recovery from past disasters. FEMA’s new  delivery model represents a significant opportunity to strengthen the PA  program and address these past challenges, and growing pains are to be  expected when implementing any large reengineering effort. Further,  FEMA officials work to implement these changes while supporting  response and recovery following disasters, including the catastrophic  flooding from Hurricane Harvey in August 2017 and widespread damages  from Hurricanes Irma and Maria in September 2017. As such, it is critical  that feedback obtained and lessons learned while testing the new model  inform decisions and actions as FEMA proceeds with full implementation for all disasters, including the complex recovery efforts in the states and  territories affected by Hurricanes Harvey, Irma, and Maria.

FEMA has redesigned the PA delivery model to address various  challenges related to workforce management, information sharing with  state and local grantees, and incorporating hazard mitigation into PA  projects. FEMA has developed new workforce processes, training, and  positions to address past challenges, but completing a workforce  assessment that identifies the number of staff needed will inform  workforce management and resource allocation decisions to help FEMA  ensure a more successful implementation. This is particularly important  as FEMA is using the new model for the long-term recovery from the  2017 hurricanes, and FEMA faces capacity challenges as its workforce is  stretched thin. Further, FEMA’s new FAC-Trax information sharing  system provides FEMA and state and local applicants and grantees with  better capabilities to address past challenges in managing and tracking  PA projects. In developing FAC-Trax, FEMA implemented many of the  key IT management controls that can help ensure that new IT systems  are implemented effectively. However, additional steps are needed to fully  satisfy the areas of requirements development and systems testing and  integration. Finally, FEMA has taken some actions to better promote  hazard mitigation as part of its new PA model. However, more consistent  planning for hazard mitigation following a PA disaster and developing  specific performance measures and objectives that better align with and  support the agency’s broader strategic goals related to hazard mitigation  could help to ensure that mitigation is incorporated into recovery efforts,  which presents an opportunity to encourage disaster resilience and  reduce federal fiscal exposure from recurring catastrophic natural  disasters.

We are making the following five recommendations to FEMA’s Assistant  Administrator for Recovery:

The FEMA Assistant Administrator for Recovery should complete a  workforce staffing assessment that identifies the appropriate number  of staff needed at joint field offices, Consolidated Resource Centers,  and in FIMA’s hazard mitigation cadre to implement the new delivery  model nationwide. (Recommendation 1)

The FEMA Assistant Administrator for Recovery should establish  controls for tracking FAC-Trax capabilities to the system’s functional  and operational requirements to more fully satisfy requirements  development controls and ensure that the new information system  provides capabilities that meets users’ needs and expectations.  (Recommendation 2)

The FEMA Assistant Administrator for Recovery should establish  system testing criteria, such as a “definition of done,” to assess FAC- Trax as it is developed; define the roles and responsibilities of all  participants; and develop the sequence and schedule for integration  of other systems with FAC-Trax to more fully satisfy systems testing  and integration controls. (Recommendation 3)

The FEMA Assistant Administrator for Recovery, in coordination with  the Associate Administrator of the Federal Insurance and Mitigation  Administration, should implement procedures to standardize planning  for addressing PA hazard mitigation at the joint field offices, for  example, by requiring FEMA and state officials to develop a  memorandum of understanding outlining how they will prioritize and  address hazard mitigation following a disaster as it did through prior  policy. (Recommendation 4)

The FEMA Assistant Administrator for Recovery, in coordination with  the Associate Administrator of the Federal Insurance and Mitigation  Administration, should develop performance measures and  associated objectives for the new delivery model to better align with  FEMA’s strategic goal for hazard mitigation in the recovery process.  (Recommendation 5)

We provided a draft of this report to DHS and FEMA for review and  comment. DHS provided written comments, which are reproduced in  appendix III. In its comments, DHS concurred with our recommendations  and described actions planned to address them. FEMA also provided  technical comments, which we incorporated as appropriate.

With regard to our first recommendation, that FEMA complete a workforce  staffing assessment that identifies the number of staff needed at joint field  offices, Consolidated Resource Centers, and FIMA’s hazard mitigation  cadre, DHS stated that PA, in coordination with the Field Operations  Directorate and FIMA, will continue to refine and evaluate staffing needs  and update the cadre force structures under the new delivery model. DHS  estimated that this effort would be completed by June 28, 2019. This  action, if fully implemented, should address the intent of the  recommendation.

With regard to our second recommendation, that FEMA establish controls  for tracking FAC-Trax capabilities to ensure the new information system  meets users’ needs, DHS stated that Recovery program managers will  update the FAC-Trax Requirements Management Plan and the FAC-Trax  Release Plan to ensure the tracking and traceability of FAC-Trax  functional and operational requirements. DHS estimated that this effort  would be completed by January 31, 2018. This action, if fully  implemented, should address the intent of the recommendation.

With regard to our third recommendation, that FEMA establish systems  testing criteria to assess the development of FAC-Trax; and define the  roles and responsibilities, and sequence and schedule for system  integration, DHS stated that Recovery program managers will update the  FAC-Trax System Integration Plan to include integration with the  Deployment Tracking System, Enterprise Data Warehouse, Preliminary  Damage Assessment interface, and State Grants Management system  interface. DHS estimated that this effort would be completed by June 29,  2018. This action, if fully implemented, should address the intent of the  recommendation.

With regard to our fourth recommendation, that FEMA implement  procedures to standardize planning for addressing PA hazard mitigation  at the JFO, DHS stated that PA will update current process documents or  develop new documents to better incorporate mitigation into the  operational planning phase of the new delivery model. DHS estimated  that this effort would be completed by July 31, 2018. This action, if fully  implemented, should address the intent of the recommendation.

With regard to our fifth recommendation, that PA coordinate with FIMA to  develop performance measures and associated objectives for the new  delivery model that better align with FEMA’s strategic goals for hazard  mitigation in the recovery process, DHS stated that PA will reconvene the  PA-Mitigation working group to develop and refine PA related hazard  mitigation performance measures. DHS estimated that this effort would  be completed by June 29, 2018. This action, if fully implemented, should  address the intent of the recommendation.

We are sending copies of this report to the Secretary of Homeland  Security and interested congressional committees.

If you or your staff have any questions about this report, please contact  me at (404) 679-1875 or CurrieC@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix IV.

Appendix I: Selected Prior Work Related to Federal Emergency Management Agency’s (FEMA) Public Assistance (PA) Program

Appendix II: Assessment of Information Technology Management Controls for the FEMA Applicant Case Tracker (FAC-Trax)

Appendix II: Assessment of Information  Technology Management Controls for the  FEMA Applicant Case Tracker (FAC-Trax)

Table 2 shows details on the Federal Emergency Management Agency  (FEMA) Public Assistance (PA) program office’s implementation of key  practices across four information technology (IT) management control  areas for its new information system, the FEMA Applicant Case Tracker  (FAC-Trax). PA developed FAC-Trax as a web-based project tracking  and case management system to supplement the Emergency  Management Mission Integrated Environment (EMMIE) and help resolve  long-standing information sharing challenges. To determine the extent to  which the FAC-Trax program office implemented IT management  controls, we reviewed documentation from the FAC-Trax program and  compared it to key management best practices, including the Software  Engineering Institute’s Capability Maturity Model® Integration for  Acquisition and Development, the Project Management Institute’s Guide  to the Project Management Body of Knowledge (PMBOK® Guide), and  the Institute of Electrical and Electronics Engineers’ Standard for  Software and System Test Documentation. We assessed the program as  having fully implemented a practice if the agency provided evidence that it  fully addressed the practice; partially implemented if the agency provided  evidence that it addressed some, but not all, portions of the practice; and  not implemented if the agency did not provide any evidence that it  addressed the practice.

Table 2. Public Assistance (PA) Program Office’s Implementation of Key Information Technology Management Controls for  FAC-Trax  PA program officials developed an acquisition plan for FAC-Trax identifying the  capabilities the system is intended to deliver, the acquisition approach, and acquisition  objectives. Additionally, program officials developed a capability development plan  outlining a strategy for the program to obtain approval to acquire FAC-Trax. Lastly,  program officials developed a systems engineering plan describing the program’s  scope and its framework for using an Agile development approach, as well as a  deployment, support, and maintenance plan for FAC-Trax.

PA program officials developed an acquisition program baseline detailing FAC-Trax’s  cost parameters and a life-cycle cost estimate for the system. As of May 2017, the life- cycle cost estimate for FAC-Trax through fiscal year (FY) 2023 is approximately $19.3  million. PA program officials updated the life-cycle cost estimate for FYs 2016 and  2017 after price negotiations with the FAC-Trax contractor, and will continue to update  the estimate as annual budgets are approved, according to the Integrated Logistic  Support Plan. The contracting officer’s representative for FAC-Trax performs a cost  review at the end of each month, according to program officials. Furthermore, the  contractor’s weekly status report includes information on the number of hours worked  and the percent of contract value spent. Program officials also review program costs  with Office of Response and Recovery, PA, Office of the Chief Information Officer  (OCIO), and other program office stakeholders during a weekly program review.

PA program officials developed an acquisition program baseline detailing FAC-Trax’s  schedule parameters, as well as an integrated master schedule for the system. The  integrated master schedule identifies tasks, major milestones, and task dependencies.  The PA program manager reviews and updates the integrated master schedule on a  weekly basis. Program officials also review FAC-Trax’s schedule with Office of  Response and Recovery, PA, OCIO, and other program office stakeholders during a  weekly program review.

PA program officials identified the knowledge and skills needed to carry out the  program in FAC-Trax contract documentation and the capability development plan.  Specifically, program officials included an attachment to the FAC-Trax contract listing  the required labor categories and corresponding functional position descriptions.  Program officials also described the role, position type, minimum grade, and minimum  certification for required personnel resources for the acquisition, development, and  implementation of FAC-Trax.

PA program officials developed, reviewed, and maintained project planning documents  and obtained commitment from relevant stakeholders. For example, program officials  reviewed and updated the integrated master schedule and costs on a weekly and  monthly basis, respectively. Further, program officials reviewed the status of project  elements, such as the schedule, quality and technical issues, stakeholders, staffing,  cost, and risks, with Office of Response and Recovery, PA, OCIO, and other program  office stakeholders during a weekly program review. PA program officials also  established tactical, functional, and stakeholder groups, as well as an Integrated  Product Team to support and oversee the development of FAC-Trax.

FEMA’s Recovery Technology Programs Division (RTPD) has a division-level risk  management plan that serves as guidance for all Recovery systems, including FAC- Trax. Program officials identified key risks that could negatively affect FAC-Trax work  efforts in RTPD’s “risk register”—an online site used to track risks, issues, and  mitigating actions for the division and each program office. Program officials also  identified five technical, cost, and schedule risks in the FAC-Trax acquisition plan.  Program officials included one of these risks in the risk register, while the remaining  four were managed outside of the register. As of May 2017, program officials had  identified 13 risks in its risk register—four open and nine closed. The four open risks  were (1) limited subject matter expert engagement during requirements development,  (2) vacancies in program management office support positions, (3) unresolved service  level agreement support and funding issues, and (4) the loss of the authority to  operate due to a Trusted Internet Connection that is not compliant with Department of  Homeland Security security policy.

Program officials evaluated and categorized the identified risks based on the  probability of occurrence and scope, schedule, and cost impacts. These four points of  measurement are used to calculate an overall risk score. The risk score helps program  officials determine a risk’s risk rating—low, medium, or high. For example, program  officials reported that two of its open risks have a “medium” risk rating—meaning the  risk has the potential to slightly impact project cost, schedule, or performance. In  addition, program officials detailed the risk category, probability, and impact for the five  risks identified in the FAC-Trax acquisition plan.

Program officials developed risk mitigation and contingency plans for each risk in the  risk register. For example, program officials planned to mitigate the open risk  concerning subject matter expert engagement, by identifying and engaging with  appropriate subject matter experts through requirements development workshops  scheduled in advance of the sprint they are to support, and monitoring the  development of user stories to identify any issues that may cause delays. In addition,  program officials described the risk management plan and responsible officials for the  five risks identified in the FAC-Trax acquisition plan.

PA program officials review and update program risks during a monthly program  meeting. Program officials also review program risks with Office of Response and  Recovery, PA, OCIO, and other program office stakeholders during a weekly program  review. Furthermore, the FAC-Trax contractor provides a weekly status update which  includes a section on identified risks. Program officials established re-evaluation dates  and recorded updates, including any actions taken, for each risk in the risk register. In  addition, program officials were able to provide updates on the four risks identified in  the FAC-Trax acquisition plan and managed outside of the register. According to PA  officials, these risks were addressed and closed by the approval of program planning  documents, such as the mission needs statement, concept of operations, and  operational requirements document, following the solutions engineering review, which  demonstrates the readiness of the program to proceed with the procurement, in  September 2016.

Program officials established a requirements management plan outlining how it  captures, assesses, and plans for FAC-Trax enhancements, and established a change  control process to review, prioritize, and verify user requests for changes to the system  and feedback. As of May 2017, the PA program office received 734 change requests  related to FAC-Trax, of which program officials completed 420 changes and planned  to address an additional 277 entries. PA program officials also facilitated workshops to  gather requirements for specific user groups and obtained additional requirements for  FAC-Trax through customer feedback on a temporary technology tool— an Access  database referred to as the Public Assistance Recovery Information System—used to  support an early stage of the new model implementation. Further, program officials  developed a functional requirements document outlining the high-level functional and  operational requirements for FAC-Trax.

PA program officials developed a concept of operations for FAC-Trax detailing  operating concepts and scenarios for each phase of the PA preaward process.  Program officials also detailed the workflow, phases, business functions, and data  inputs and outputs for the re-engineered PA process in FAC-Trax’s functional  requirements document. In March 2017, program officials developed a standard  template to describe the process, tasks, and data inputs and outputs for specific  system capabilities.

As part of the change control process, PA program officials meet three times a week to  discuss and prioritize change requests. Specifically, program officials review  submissions to the change control form to ensure completeness, validate impacts and  root cause, and research details for incoming requests. PA program officials also  follow up with users to understand and verify requirements. In March 2017, program  officials developed a standard template to capture acceptance criteria for specific  requirements. However, PA program officials do not track system enhancements back  to the high-level requirements identified in FAC-Trax’s operational and functional  requirements documentation and performance work statement.

PA program officials identified system requirements and constraints in the FAC-Trax  concept of operations and functional and operational requirements documents.  Further, through its change control process, program officials collect suggestions,  issues, and feedback on FAC-Trax and system enhancements from stakeholders,  identify risks for change requests, and balance prioritized requirements and estimated  level of efforts with projected costs prior to each sprint. In March 2017, program  officials developed a standard template to analyze and document the urgency and  need for specific requirements.

PA program officials and the FAC-Trax contractor established a testing and evaluation  plan for the system, developed acceptance criteria for user stories, and obtained  feedback from users during and after testing. The testing process concludes with user  acceptance testing (UAT). If a change request fails during UAT or a new requirement  is discovered during development, the PA program will capture the failed request or  new requirement in the product backlog for implementation in a future product release.

Key practices  Systems testing and integration  Developing test plans and test cases  PA program officials and the FAC-Trax contractor tested and evaluated the system  during development. The FAC-Trax test plan identifies the method and strategy to  perform the testing, including the necessary tasks, testing parameters, and the roles  and responsibilities of the individuals responsible for testing. However, program  officials did not develop system testing criteria to evaluate FAC-Trax. A key feature of  Agile software development is the “definition of done”—a set of clear, comprehensive,  and objective criteria, that the government should use to evaluate software after each  iteration of development.

PA program officials developed a systems integration plan in June 2017 that identifies  potential integration of FAC-Trax and four FEMA systems, including the Emergency  Management Mission Integrated Environment. Specifically, the plan includes data  requirements and standards; descriptions of the four systems FEMA plans to integrate  with FAC-Trax and the proposed relationship for each connection; and security and  access management requirements. In addition, program officials included a description  of how integration problems are to be documented and resolved in FAC-Trax  development and test plans. However, the systems integration plan does not define  roles and responsibilities of all participants for system integration activities or establish  a sequence and schedule for every integration step for the four FEMA systems. ● Fully implemented: The agency provided evidence that it fully addressed this practice.  ◐ Partially implemented: The agency provided evidence that it addressed some, but not all, portions of this practice.  ◌ Not implemented: The agency did not provide any evidence that it addressed this practice.

Appendix IV: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Chris Keisling (Assistant  Director), Amanda R. Parker (Analyst-in-Charge), Mathew Bader, Allison  Bawden, Anthony Bova, Eric Hauswirth, Susan Hsu, Rianna Jansen,  Justin Jaynes, Tracey King, Matthew T. Lowney, Heidi Nielson, Claire  Peachey, Brenda Rabinowitz, Ryan Siegel, Martin Skorczynski, Niti  Tandon, Walter K. Vance, James T. Williams, and Eric Winter made key  contributions to this report.