DOD Health Care: Defense Health Agency Should Improve Tracking of Serious Adverse Medical Events and Monitoring of Required Follow-up

Adverse medical events are unintended incidents that may harm a patient. Serious adverse medical events, called sentinel events, have specific follow-up requirements. The National Defense Authorization Act for Fiscal Year 2017 (NDAA 2017) requires DHA to assume the military services' administrative responsibilities, such as adverse medical event reporting, for all MTFs beginning October 1, 2018.

The NDAA 2017 included a provision for GAO to examine the reporting and resolving of adverse medical events in the military health system. Among other objectives, this report reviews (1) the extent to which sentinel events and RCA reports are tracked and DHA ensures it has received complete information, and (2) the extent to which DHA ensures it has received MOS reports. GAO examined relevant policies; analyzed the most current available data on sentinel events from 2013 through 2016; and interviewed officials with DHA, the military services, and four MTFs selected for variety in military service, size, and geographic location.

GAO found that the process for tracking the most serious adverse medical events, called sentinel events, and their root cause analysis (RCA) reports are fragmented, impeding the Defense Health Agency's (DHA) ability to ensure that it has received complete information. Unlike other adverse medical events, sentinel events—which may result in severe harm or death—have additional reporting requirements that must be met within specified time frames. For example, military treatment facility (MTF) officials must develop RCA reports, which identify causal factors and corrective actions for sentinel events. However, because the database that DHA uses to collect information on adverse medical events does not currently have the capability to track this information, the military services (Army, Navy, and Air Force) and DHA each maintain their own tracking records for sentinel events and RCA reports. Due to these fragmented tracking efforts, DHA reconciles its information on sentinel events and RCA reports through monthly emails to the military services—a time-consuming, inefficient process. DHA officials emphasized that this process relies on the military services' cooperation because DHA does not currently have the authority to compel their responses. Moreover, despite DHA's reconciliation efforts, GAO identified discrepancies and missing information in DHA's tracking record. As a result, DHA lacks critical information about why a sentinel event may have occurred and what actions, if any, MTFs should take to prevent similar incidents in the future. Recently, DHA replaced its previous system of emails with a new tracker tool that can be accessed on the military health system website. However, the new tracker does not allow the military services to make edits, and as a result, any corrections or additional information must be submitted to DHA via email, which may perpetuate previous inefficiencies.

GAO found that DHA cannot ensure that it is receiving all reports on the implementation of corrective actions identified in RCA reports as required by a March 2015 memo. DHA officials stated that MTFs could meet this requirement by submitting copies of their measures of success (MOS) reports, which may be required by the Joint Commission, a hospital accrediting organization. As of September 2017, DHA had received 27 MOS reports for the 319 sentinel events that were reported in 2016. However, DHA does not know how many reports it is missing because MOS reports are not required for every sentinel event, and DHA did not began reconciling its information for these reports until January 2018, when it implemented its new tracker tool. Furthermore, GAO found that the new tracker tool documents the aggregate number of MOS reports received and does not indicate whether individual sentinel events have an MOS report, impeding DHA's ability to identify which reports are missing. This issue is compounded by the fact that the military services either track MOS reports in different ways or not at all, and military service officials said that DHA's requirement for MOS report submission is not clear. DHA officials stated that they expect to clarify this requirement in their update to the patient safety policy. Because it is unable to ensure it has received all reports on the implementation of corrective actions, DHA could be missing important information that could be used to help inform broader, system-wide patient safety improvement efforts.

GAO recommends that the Assistant Secretary of Defense (Health Affairs) ensure DHA (1) improve tracking of sentinel events and RCA reports, and (2) clarify its requirements for submitting reports on the implementation of corrective actions and consistently track and reconcile individual reports. DOD agreed with these recommendations.

One of the key patient safety responsibilities within the Department of  Defense’s (DOD) Military Health System (MHS) is the reporting and  tracking of adverse medical events—unintended health care incidents,  such as administering incorrect medication or treatment to a patient, that  may or may not result in harm. DOD’s Assistant Secretary of Defense for  Health Affairs sets policies—including patient safety policies—for the  MHS, and the Defense Health Agency (DHA) oversees the  implementation of these policies. The military services (Army, Navy, and  Air Force) and the National Capital Region (NCR) currently manage their  own hospitals and clinics, referred to as military treatment facilities (MTF).  In doing so, they develop policies that must align with DOD’s policies,  including policies for reporting adverse medical events that occur in their  facilities. However, the National Defense Authorization Act for Fiscal  Year 2017 (NDAA 2017) requires the transfer of administrative control of  MTFs, including the reporting of adverse medical events, to DHA,  beginning October 1, 2018.

Information gleaned from the reporting and tracking of adverse medical  events provides the military services, NCR, and DHA with critical  information that can be used to make improvements in the way health  care is delivered across the MHS. MTF officials are required to report all  adverse medical events in DHA’s Joint Patient Safety Reporting (JPSR)  system, which captures information on the factors—such as medication or  equipment—that may have contributed to the event. The most serious  type of adverse medical event is called a sentinel event, which can result  in unexpected death or serious physical or psychological harm to the  patient. For every sentinel event, MTF officials are required to prepare a  root cause analysis (RCA) report, which is intended to identify the factors  that caused or contributed to the sentinel event, as well as corrective  actions needed to prevent future incidents. For most sentinel events, an  additional report—a Measures of Success (MOS) report— may be  required by the Joint Commission, a hospital accrediting organization that  reviews sentinel events and assigns follow-up activities to accredited  MTFs for these events. This MOS report includes the determination of  whether the implementation of identified corrective actions was  successful.

In 2014, news articles highlighted concerns about medical errors and  lapses in patient safety at MTFs, including staff’s reluctance to report  these errors. In August 2014, DOD released a review of the MHS that  addressed patient safety, among other issues. While DOD’s review  found that the culture of safety within the system was comparable to that  found in the civilian sector, the report made a number of  recommendations that included the clarification of policy on the definitions  for sentinel events to reduce variation in interpretation. In December  2017, the DOD Inspector General released a follow-up report focused on  patient safety, which found that DOD had established courses of action to  resolve all relevant patient safety findings identified in the 2014 review.

The NDAA 2017 included a provision for us to examine several issues  related to DOD’s delivery of health care, such as reporting and resolving  adverse medical events. In this report, we examine  1.  the extent to which the military services’ and NCR’s policies align with  DOD’s policies for reporting adverse medical events;  2.  the extent to which the military services, NCR, and DHA track sentinel  events and RCA reports, and the extent to which DHA ensures it has  received complete information;  3.  the extent to which DHA ensures it has received MOS reports; and  4.  how DHA uses information about adverse medical events to make  improvements in its health care system.

To examine the extent to which the military services and NCR have  policies that align with DHA’s policies for reporting adverse medical  events, we reviewed and compared the military services’ and NCR’s  policies with DHA’s policies. We also reviewed relevant documents  related to these policies, including updated memorandums. In addition,  we interviewed officials from the military services’ medical commands and  NCR’s medical directorate to determine how they manage patient safety  efforts pertaining to reporting adverse medical events throughout their  MTFs and how the future transfer of this authority to DHA may affect  these efforts. We also conducted site visits to a non-generalizable  sample of four MTFs—Dwight. D. Eisenhower Army Medical Center (Fort  Gordon, Ga.), Robert E. Bush Naval Hospital (Twentynine Palms, Calif.),  Mike O’Callaghan Federal Medical Center (Nellis AFB, Nev.), and Walter  Reed National Military Medical Center (Bethesda, Md.)—to better  understand how the military services and NCR report adverse medical  events in their facilities. Our site selection criteria included one MTF from  each of the military services and NCR with a range of hospital sizes,  geographic diversity, and numbers of reported adverse medical events.  We also interviewed DHA officials, including officials from DHA’s Patient  Safety Program and the Patient Safety Analysis Center, and reviewed  relevant documentation about their patient safety policies, their plans to  update these policies, and how the future transfer of MTF administrative  responsibilities to DHA will impact the reporting of adverse medical  events.

To determine the extent to which the military services, NCR, and DHA  track sentinel events and RCA reports, we reviewed relevant military  service, NCR, and DHA policies to identify requirements for sentinel  events and RCA reports. We interviewed officials from the military  services, NCR, and DHA about their tracking processes for sentinel  events and RCA reports. We also interviewed officials during our site  visits to the four MTFs about how they transmit information about sentinel  events and their required RCA reports to their respective military service  or NCR. We evaluated our findings against the GAO Fragmentation,  Overlap, and Duplication: An Evaluation and Management Guide to  identify any potential fragmentation.

To determine how DHA ensures it has complete information, we reviewed  relevant DHA policies, interviewed DHA officials, and obtained documents  outlining DHA’s monitoring process. We obtained the most current data  available on reported sentinel events and RCA reports from 2013 through  2016 from tracking records maintained by each of the military services,  NCR, and DHA, and we compared the data to assess DHA’s monitoring  efforts. We interviewed military service, NCR, and DHA officials about  how these data are collected and documented, as well as the steps taken  to ensure that the data are complete, and on this basis we determined  that these data were sufficiently reliable for the purposes of our audit  objective.

To determine the extent to which DHA ensures that it has received MOS  reports, we reviewed relevant military service, NCR, and DHA policies to  identify requirements for completing and tracking these reports. We also  reviewed documentation on requirements for MOS reports from the Joint  Commission. We interviewed officials from the four MTFs we visited  about how they transmit MOS reports to their respective military services  or NCR. To determine how DHA monitors this information, we obtained  data from DHA’s tracking record on total MOS reports for reported  sentinel events from 2016, the first full year that such reports were  required to be submitted to DHA. We interviewed military service, NCR,  and DHA officials about how these data are collected and documented as  well as steps to ensure that these data are complete, and we determined  that these data were sufficiently reliable for the purposes of our audit  objective. We also assessed efforts to monitor MOS reports against  federal standards for internal controls.

To determine how DHA uses information about adverse medical events to  make improvements in its health care system, we reviewed  documentation on DHA’s patient safety initiatives, relevant DHA  publications, such as the Patient Safety Annual report, and the MHS  website. We also interviewed military service, NCR, and MTF officials  about the patient safety initiatives implemented by DHA, including how  these initiatives are communicated to them and how the information is  used to make improvements in their facilities.

We conducted this performance audit from February 2017 to April 2018 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

DHA requires the military services and NCR to categorize adverse  medical events by severity, using seven categories defined by the Agency  for Healthcare Research and Quality (AHRQ), ranging from unsafe  condition to death. (See table 1.)

MTF personnel must enter all adverse medical events in DHA’s JPSR  system, which was implemented in June 2011 in response to a statutory  mandate for the MHS to establish a patient care error reporting and  management system. The JPSR system is intended to provide ways to  facilitate the self-reporting, collection, and aggregation of adverse medical  event data across the MHS. The system includes prompts for information  about factors that may have contributed to the event, such as medication  or equipment, as well as the assignment of a severity category.

From 2013 through 2016, the total number of reported adverse medical  events in the JPSR system increased from over 76,000 to about 108,000.  When analyzing adverse medical events, DHA groups the data into three  categories—near miss, no harm, and harm. The highest increase was in  the near miss category (about 36,000 to 56,000) while the other two  categories increased to a lesser extent. According to an internal DHA  publication, a higher increase in near miss events alongside a decrease  in harm and no harm events is considered a positive trend because it  shows that more potential adverse medical events are being detected  before they reach the patient. (See fig. 1.)

The most severe types of adverse events are called sentinel events. In  March 2015, DOD issued a memo that revised its previous definition of a  sentinel event, which was an unexpected occurrence involving death or  serious physical or psychological injury or risk. The revised definition  states that a sentinel event is a patient safety event (not primarily related  to the natural course of the patient’s illness or underlying condition) that  results in death, permanent harm, or severe temporary harm. The  revised definition also added a list of events outlined by the Joint  Commission and the National Quality Forum that go beyond those that  result in unexpected death or serious physical or psychological harm to  the patient. (See app. I for the revised definition of sentinel events.)

From 2013 through 2016, DHA’s data showed an increase in the total  number of reported sentinel events—both medical and dental—from 121  to 319. Medical sentinel events approximately doubled from 101 to 206,  while dental sentinel events increased more than fivefold from 20 to 113.  (See fig. 2.) The sharp increase in events in 2015 may have been  influenced by DHA’s revised definition of sentinel events as well as the  Army’s inclusion of dental events that meet sentinel event criteria. A  DHA internal publication also noted that a culture shift in patient safety  reporting could have contributed to this increase.

As with all adverse medical events, MTF personnel must enter sentinel  events into the JPSR system; however, sentinel events have additional  reporting requirements that must be met within specified time frames. For  example, DHA policy requires MTF officials to report sentinel events to  their respective military service or NCR within 24 hours after they become  aware of the event. (See fig. 3, step 1.) MTFs also must report to and  comply with sentinel event reporting requirements established by the  Joint Commission. These requirements include the development and  submission of an RCA report for each sentinel event to identify the causal  and contributory factors associated with the event as well as the  corrective actions needed to prevent future incidents. The military  services and NCR submit copies of their RCA reports to DHA, which rates  the corrective actions included in each RCA report as stronger,  intermediate, or weaker based on an estimation of their effectiveness.  (See fig. 3, step 2.) DHA uses commercial process improvement software  called TapRooT to assist with the development of RCA reports, and DHA  requires all MTFs to use a methodology for its RCA reports that is  currently supported by this software.

Additionally, once the Joint Commission approves an RCA report and its  associated corrective action plan, it may require the preparation of an  MOS report that assesses the corrective actions 4 months after an RCA  report is submitted to determine whether the implementation of corrective  actions and outcome measures was successful. Unlike RCA reports,  these reports are only required for selected sentinel events as determined  by the Joint Commission. DOD’s March 2015 memo that revised the  definition of sentinel events contained an additional requirement for the  military services and NCR to submit copies of reports on the  implementation of corrective actions to DHA. (See fig. 3, step 3.) DHA  officials told us that MTFs could submit their MOS reports to meet this  requirement. For this report we use the term MOS report when referring  to this requirement.

Transition of MTF Administrative Responsibilities to DHA

Responsibility for the delivery of care in the MHS is shared among the  Office of the Assistant Secretary of Defense (Health Affairs), DHA, the  military service medical commands, and NCR’s medical directorate.  MTFs are currently under the direction and control of the Army Medical  Command, the Navy Bureau of Medicine and Surgery, and the Air Force  Major Commands. MTFs within the NCR are under the direction and  control of the NCR medical directorate, which reports to DHA. (See fig. 4.)

The NDAA 2017 included a provision that requires the Director of DHA to  be responsible for the administration of every MTF beginning October 1,  2018. This responsibility includes budgetary matters, patient safety  activities, information technology, and health care administration and  management, among other things. As part of the patient safety activities,  DHA officials will assume responsibility for adverse medical event  reporting. As required, DHA submitted initial plans to Congress in both  March and June 2017 about how it plans to implement its new  responsibilities. In September 2017, we reported that DHA’s plans  summarize its new roles and responsibilities at a high level and that a  significant amount of work remained to complete the implementation  plan. On March 30, 2018, DOD submitted an additional implementation  plan and stated that its final implementation plan will be completed by  June 30, 2018.

The Military Services’ and NCR’s Adverse Medical Event Policies Do Not Consistently Align with DOD’s Policies, but Transition to DHA’s Policies Is Planned

Policies established by the military services and NCR for reporting  adverse medical events are developed to implement DOD’s policies— which tend to be broad—and may include additional requirements specific  to their branch of military service. However, we found that aspects of  these policies do not consistently align with DOD’s policies, including the  definitions for adverse medical events and sentinel events, as well as  requirements for entering events into the JPSR system. (See table 2.)

Definition of adverse medical event. The Navy uses DOD’s  definition of an adverse medical event—which includes events that  may or may not result in harm to the patient. However, the Army, Air  Force, and NCR defined this term more narrowly, to include only an  event that causes actual harm to the patient. While the difference in  these definitions could potentially result in the underreporting of  events, officials from all four of the MTFs we visited told us that the  discrepancy does not have much of an impact because the individuals  who report these events—MTF personnel—are unlikely to be aware of  the difference and likely follow the broader DOD definition.

Policy on entering events in the JPSR system. Only NCR’s policy  states that adverse medical events should be entered into the JPSR  system. However, Army, Navy, and Air Force officials as well as  officials from one MTF we spoke with stated that they record all  adverse medical events in the JPSR system even though their  policies do not require it.

Policy on reviewing adverse medical events. NCR and Air Force  policies, which align with DOD’s policy, require a review of an adverse  event that is based on whether there is harm to the patient. In  contrast, Army and Navy policies do not require that an adverse  medical event be reviewed on the basis of whether there is harm to  the patient, but they do require the event to be reviewed for the level  of severity and probability of recurrence. However, Navy officials  told us that reviewing an event for severity includes an assessment of  harm to the patient even though this is not clearly stated in their  policy. Additionally, all of the MTF officials we interviewed said that  the JPSR system requires them to review an adverse medical event  on the basis of whether there is harm to the patient and to assign a  harm scale category.

Memorandum that revised the definition of a sentinel event. Only  the Army’s draft policy aligned with DOD’s March 2015 revised  definition of sentinel events. However, MTF officials from the other  military services and NCR told us that even though the revised  definition was not in their policies, they were aware of the memo and  were using this definition.

Memorandum that requires the military services and NCR to  submit copies of their reports on the implementation of  corrective actions to DHA. The Army’s draft policy that aligned with  DOD’s revised definition of sentinel events also included a section  requiring the submission of these reports to DHA. The policies of the  other military services and NCR do not include this requirement.  However, officials from the other military services we interviewed told  us that they are aware of this requirement and are submitting MOS  reports to meet this requirement. NCR officials told us that they are  aware of this requirement but have not begun submitting these  reports.

In March 2017, DOD’s senior military medical leadership published  operating principles to guide the implementation of specific MHS  requirements outlined in the NDAA 2017. One of the operating principles  to guide the transition of MTF administrative responsibilities to DHA  requires DHA to create all health care policies for the direct care system  (the MTFs) to ensure greater consistency and eliminate duplicative  governance. As a result, the military services and NCR will no longer be  establishing their own policies. According to DHA officials, the transition  for DHA to be the single policy writer for MTFs will take time, and policies  issued by the military services and NCR will remain in place until they are  superseded by revised DHA policies. DHA officials are in the process of  updating the department’s patient safety policy through the Patient Safety  Improvement Collaborative, a working group that includes patient safety  representatives from all of the military services, NCR, and DHA.  However, as of January 2018, DHA officials were uncertain as to when  this effort would be complete.

Fragmented Process for Tracking Sentinel Events and RCA Reports Impedes DHA’s Ability to Ensure It Has Received Complete Information

Process Used by the Military Services, NCR, and DHA to Track Sentinel Events and RCA Reports Is Fragmented

We found that the process used by the military services, NCR, and DHA  to track sentinel events is fragmented. (See fig. 5.) Similar to all other  types of adverse events, DHA requires that sentinel events be recorded in  the JPSR system. However, DHA officials told us there are additional  follow-up reports and associated deadlines for sentinel events that go  beyond the JPSR system’s current tracking capabilities, and as a result,  officials from each of the military services and NCR told us they track  sentinel events in their own tracking record outside of the JPSR system.  Officials told us the military services and NCR receive reports about  sentinel events from their MTFs via email, which are then entered in their  respective internal tracking records and reported to DHA via email. DHA  then enters and tracks the sentinel events in its own internal tracking  record. DHA officials told us that they do not believe that all sentinel  events are being entered in the JPSR system, and that the JPSR system  does not currently have the capability to pull sentinel event data for  tracking purposes. As a result, the same sentinel events are entered  and tracked in two separate tracking records—DHA’s tracking record and  the tracking records maintained by the military services or NCR.

In a similarly fragmented process, MTFs email RCA reports—a  requirement for sentinel events—separately to their respective military  services or NCR, which then emails them to DHA. Although DHA  requires MTFs to use a methodology currently supported by the TapRooT  system to complete their RCA reports, DHA officials told us the TapRooT  software is not compatible with most MTFs’ computer systems, and as a  result, MTFs do not share RCA reports through this system. Instead, they  told us MTFs use the methodology from the TapRooT system to prepare  the RCA report as a standalone document. Officials told us MTFs then  email the RCA reports to their military service or NCR, which notates the  RCAs in their respective internal tracking record. The military services  and NCR email the RCA reports to DHA, which notates the reports in its  own internal tracking record.

Fragmented Tracking Impedes DHA’s Ability to Ensure That It Has Complete Information on Sentinel Events and RCA Reports

Because the process used by the military services, NCR, and DHA to  track sentinel events and RCA reports is fragmented, DHA officials told us  they must rely on their reconciliation process to ensure they have  complete information. Specifically, on a monthly basis, DHA officials email  separate spreadsheets of DHA’s sentinel event records to each of the  military services and NCR requesting confirmation of reported sentinel  events and the status of overdue RCA reports, among other information.  DHA officials acknowledged that their reconciliation process is inefficient  and told us that their full-time employees and contractors spend an  average of 80 hours per month working on it. Additionally, officials told us  that sometimes information about sentinel events and RCA reports is lost  or not effectively communicated due to complexities related to routing the  email submissions and to turnover in the contract staff who track and  reconcile this information. The cooperation of the military services and  NCR is key to this process because officials told us that DHA currently  has no authority to compel a response from these entities, although this  may change with the transition of MTF administrative responsibilities to  DHA. DHA officials told us they sometimes do not receive a response to  their emails, and in these cases, DHA assumes concurrence.

In an effort to improve the reconciliation process and compliance with  RCA report submission requirements, DHA officials told us that they  developed a new tool called the Comprehensive Analysis Progress  Tracker for all three military services and NCR. DHA officials told us this  tracker shows the full cycle of each sentinel event, including which RCAs  are overdue, and is available on the MHS internal website. DHA officials  told us that this tracker, launched in October 2017, replaced the previous  system of separate monthly reconciliation emails with individual  spreadsheets for each military service and NCR. In January 2018, DHA  officials told us they began using this tracker at monthly Patient Safety  Improvement Collaborative meetings and will use it during monthly check-ins with the military services and NCR to discuss delayed or missing  items. However, the military services and NCR cannot directly edit the  Comprehensive Analysis Progress Tracker. As a result, DHA officials told  us that the military services and NCR will continue to use email to submit  their sentinel events and RCA reports as well as any corrections or  additional information needed for the tracker, which may perpetuate  previous inefficiencies.

Despite Reconciliation Efforts, DHA Does Not Have Complete Information on Sentinel Events and RCA Reports

Despite DHA’s efforts to reconcile its information on sentinel events and  RCA reports, we identified discrepancies and missing information in its  tracking record.

We found that the sentinel events in all of the military service and NCR  tracking records matched DHA’s tracking record except for those of the  Navy. Specifically, DHA had a record of 19 sentinel events that the Navy  did not have for 2013 through 2016. DHA officials were not sure of the  reason for the discrepancy between their tracking record and the Navy’s,  but told us that sometimes sentinel events are reported to DHA and later  determined to not be reportable, and DHA is not given the updated status  of the event. Navy officials told us that although they initially reported  these 19 events as sentinel, the Joint Commission informed the Navy that  it did not consider these events to be sentinel after reviewing the Navy’s  submission. Navy officials told us that they determined these events also  did not meet other sentinel event criteria per DHA’s revised definition,  which goes beyond the definition used by the Joint Commission. Further,  Navy officials told us they informed DHA that these events had been  deemed non-sentinel by the Joint Commission, and DHA’s tracking  record subsequently noted this. However, DHA did not remove the events  from its tracking record.

We found discrepancies in the number of RCA reports when comparing  DHA’s internal tracking record to the military services’ and NCR’s internal  tracking records. In some instances, we found that DHA had more RCA  reports in its tracking record than the military services or NCR for reported  sentinel events, and in other instances, DHA had fewer RCA reports in its  tracking record than the military services or NCR:

DHA had more RCA reports in its internal tracker than in the Army’s  internal tracker for 2015 (2 more) and 2016 (1 more).

DHA had fewer RCA reports than the Air Force in 2013 (3 less), 2014  (2 less), 2015 (13 less), and 2016 (1 less).

Additionally, DHA had fewer RCA reports for reported sentinel events  for NCR in 2015 (1 less) and 2016 (18 less).

Officials with the military services and NCR told us they did not know why  there were differences between their tracking records and those of DHA.  However, Army and NCR officials offered potential reasons for these  differences.

Army officials told us that they may have fewer RCA reports than DHA  because they recently transitioned their sentinel event and RCA  tracking record from a spreadsheet format to a database, and some  reports may not have been copied into the database.

NCR officials told us their tracking record may not match DHA’s  tracking record because an MTF may submit only one RCA report to  DHA that covers multiple similar sentinel events, so DHA may have  fewer reports documented in its internal tracking record.

For some reported sentinel events, we found that the required RCA  reports had not been recorded in any tracking record for the Army, NCR,  or DHA. (See table 3.)

Army and NCR officials told us that they did not know why they did not  have a record of an RCA report for every sentinel event in their internal  tracking record. However, these officials explained that there are a  number of potential reasons that RCA reports could be missing, including  insufficient MTF staff to carry out these activities, and MTF officials’  confusion about the revised definition of a sentinel event.

DHA officials told us that they did not know the reasons for the  discrepancies between the tracking records for the military services,  NCR, and DHA or for the missing RCA reports. Specifically, DHA officials  did not know whether these reports were completed but not submitted to  DHA or were not completed at all. They told us that they rely on the  cooperation of the military services and NCR to submit these reports and  cannot enforce the requirement, although this may change with the  transition of MTF administrative responsibilities to DHA. Because of these  discrepancies and missing RCA reports, DHA lacks critical information  about why a sentinel event may have occurred and what actions, if any,  MTFs should take to prevent similar incidents in the future.

We have previously reported that when fragmentation or overlap exists,  there may be opportunities to increase efficiency. In particular, our prior  work identified management approaches that may improve efficiency and  effectiveness, including implementing process improvement methods and  technology improvements. As MTF patient safety responsibilities are  transitioned to DHA, the fragmented tracking process may hamper DHA’s  ability to efficiently and effectively monitor sentinel events and RCA  reports, potentially leading to missed opportunities for systemic  improvements.

DHA’s Efforts to Ensure It Receives MOS Reports Are Limited and Impeded by Inconsistent Report Tracking and Unclear Requirements about Report Submission

DHA’s Efforts to Ensure It Receives MOS Reports Are Limited and Impeded by Inconsistent Report Tracking

As of September 2017, DHA had received 27 MOS reports for the 319  sentinel events that were reported in 2016. However, DHA does not know  how many reports it is missing because its efforts to reconcile information  for these reports have been limited. Prior to January 2018, DHA did not  include MOS reports as part of its reconciliation process for sentinel  events and RCA reports. However, in January 2018, DHA officials told us  they added MOS reports to their new monthly reconciliation process using  the Comprehensive Analysis Progress Tracker. While this tracker  displays the total number of MOS reports DHA has received, it does not  display whether individual reported sentinel events have an associated  MOS report. Without this information, DHA may be unable to identify  which MOS reports are missing. DHA officials told us that they may revise  the Comprehensive Analysis Progress Tracker to follow up on MOS  reports associated with specific sentinel events in the future.

DHA’s efforts to identify which MOS reports are missing are further  impeded by the military services’ and NCR’s inconsistent tracking efforts.  Specifically, the military services and NCR have been tracking the  submission of their MOS reports in different ways or not at all.

Army officials had told us that the completion of MOS reports was  noted in their internal tracking record for sentinel events and RCAs.  Army officials subsequently told us that as of January 2018, they  began tracking whether MOS reports were submitted to DHA in the  notes section of their internal tracking record.

Navy officials told us they indicated the due date of the MOS report  and the date of its submission to DHA in their internal tracking record  for sentinel events and RCA reports.

Air Force officials told us they indicated in their internal tracking record  for sentinel events and RCA reports the date that the MOS report was  sent to DHA. However, they told us the Air Force’s process for  tracking and submitting MOS reports to DHA has been inconsistent,  and they plan to revise it in the future.

NCR officials told us they did not track the completion of MOS reports  or their submission to DHA.

Because of these issues, DHA may not be able to fully reconcile its  information for individual MOS reports or identify the reports it is missing,  impeding its ability to obtain complete information on the effectiveness of  MTFs’ corrective action plans. This is inconsistent with federal internal  control standards, which require management to identify and respond to  risks to achieve its objectives, and for management to use quality  information to achieve its objectives.

DOD’s Requirement to Submit Reports on the Implementation of Corrective Actions Is Unclear

The requirement in DOD’s memo to submit reports on the implementation  of corrective actions is unclear, which may also impact DHA’s ability to  ensure that it is receiving these reports for all sentinel events. DHA  officials told us that MTFs could meet this requirement by submitting  copies of their MOS reports. According to the Joint Commission’s  guidance, the Joint Commission assigns MOS reports on an ad hoc  basis, depending on the sentinel event, RCA report, and corrective  actions, and as a result, an MOS report is not necessarily required for  each sentinel event.

DHA officials told us that they intended to obtain a report on the  implementation of corrective actions for every sentinel event, and they  believed that an MOS report was required and thus would be reported for  every sentinel event, similar to RCAs. However, DHA officials told us that  they learned from the military services and NCR at the January 2018  Patient Safety Improvement Collaborative meeting that an MOS report  was not required for every sentinel event and that DHA’s requirement for  submitting reports on the implementation of corrective actions was  unclear. Specifically, DHA officials told us the military services and NCR  told DHA that the 2015 memo did not state when the reports on the  implementation of corrective actions are required by DHA. For example,  the memo did not state whether DHA requires this report for a reported  sentinel event and RCA when the Joint Commission does not.

DHA’s unclear requirement is inconsistent with internal control standards,  which require management to review policies for continued relevance and  effectiveness in achieving the entity’s objectives. Under the current  policy, DHA cannot be sure it is receiving all reports on the  implementation of corrective actions—such as MOS reports—as it  intended, and therefore, it may be missing important information on the  effectiveness of MTFs’ implementation of their corrective actions that  could be used to help inform broader system-wide improvements. DHA  officials told us that they expect to clarify this requirement in DHA’s  update to its patient safety policy.

DHA Uses Information about Adverse Medical Events to Inform System-wide Patient Safety Improvement Initiatives

We found that DHA has introduced several system-wide patient safety  improvement initiatives informed by data on adverse medical events from  the JPSR system and data on sentinel events from DHA’s tracking  database, including the following:  DHA’s Partnership for Improvement. In January 2015, DHA  established an MHS-wide information technology system called the  Partnership for Improvement. The Partnership for Improvement collects  data from MTFs and assesses MTF performance on approximately 38  health care measures that were established by a committee of MHS  officials and designed to improve readiness, population health, and  quality of care as well as control costs. Three of these measures focus on  patient safety—central line-associated bloodstream infection, unintended  retained foreign object, and wrong site surgery. To track these  measures, DHA officials told us that they created an associated  performance dashboard, including acceptable ranges for each measure,  to provide visibility into MHS, military service-, and NCR-level  performance. The dashboard is available to all MHS users on the system  website and allows MTF leaders and staff to review MTF-level  performance data. DHA officials conduct quarterly system-wide  performance assessments on these measures. DHA officials told us they  use the data on this dashboard to determine what is improving and where  to make changes. Officials from each of the military services, NCR, and  each of the MTFs we visited told us they are aware of the Partnership for  Improvement and its associated dashboard and that they review the data  to assess their performance.

Publications on Patient Safety. DHA produces several types of  publications using adverse medical event and sentinel event data that  officials told us are generally distributed to MTFs through the military  services and NCR, including the following.

Patient Safety Data Snapshot. This monthly publication contains an  overview of adverse medical event and sentinel event data, trends  across the MHS, and short descriptions of sentinel events that have  been reported in the system in the same month. Additionally, this  publication may include reports of medical product deficiencies, or  materials that have been determined to be or are suspected of being  harmful, defective, deteriorated, or unsatisfactory because of  malfunction or design.

Annual patient safety report. This yearly publication provides a  retrospective status update on MHS patient safety initiatives and in- depth adverse event and sentinel event trend analysis, system-wide  and by military service. Content includes trends in adverse events  reported in JPSR, sentinel events, and RCAs, including information on  weaker, intermediate, and stronger corrective actions. This report also  describes progress on Partnership for Improvement measures  system-wide and by military service and NCR, the culture of patient  safety, and collaboration across DHA, the military services, and NCR.  The report also details online resources for MHS officials.

Focused review. According to officials, focused review publications  are produced three times a year, and the topics are related to adverse  medical events and associated follow-up data provided to DHA as  determined by data and performance trends. For example, in  September 2016, the publication included an explanation of the basic  components of an RCA, including their associated corrective actions  and factors DHA considers when determining if they are stronger,  intermediate, or weaker. This publication included 2013 through 2016  system-wide data, such as the number of RCAs submitted, the most  common root cause categories, and the proportion of RCAs with  stronger, weaker, or no corrective actions. The publication also  included an example of a decrease in occurrences of wrong-site  surgery accompanied by an improvement in RCAs with stronger  corrective actions, common pitfalls in conducting high-quality RCAs,  and recommendations to conduct better RCAs.

Patient safety alerts. DHA uses these publications to inform the  MHS about immediate hazards, and officials told us they produce  these publications on an as-needed basis. For example, a July 2016  report was focused on unintended retained foreign objects during  surgery, specifically, pieces of gloves. The publication described  recent occurrences of retained pieces of gloves, glove selection best  practices, tips for preventing unintended retention, and corrective  actions when retention occurs.

Global Trigger Tool. The Global Trigger Tool is a new tool for collecting  adverse medical event data by selecting a sample of medical charts that  was implemented MHS-wide as of September 2017. Unlike traditional  methods to detect adverse events, the Global Trigger Tool does not focus  on voluntary reporting and tracking of adverse medical events. Instead, a  team of three reviewers managed by DHA uses the tool methodology to  retrospectively examine a random selection of patient medical charts at a  facility over time to identify “triggers” (or clues) that may lead to an  adverse medical event. The 53 triggers include events such as a patient  fall or readmission to the emergency department within 48 hours of  treatment. If a trigger is discovered, the medical chart is further reviewed  to determine if an adverse event occurred. After the Global Trigger Tool  review is complete, the contractor is able to provide facility leaders with  rates of harmful adverse events per 1,000 patient days and per 100  admissions. Results from the tool are intended to aid MTFs in  understanding the true frequency of harm events and in identifying  systemic issues that contribute to patient safety events. All inpatient  MTFs across the MHS will use the tool, and implementation began in  2017. The Global Trigger Tool has just begun to provide data to the  MTFs, and DHA officials told us that 6 to 12 months of data is  recommended before the tool can be used to make improvements.

Sentinel Event and Root Cause Analysis (SERCA) tool. In October  2017, DHA released a dashboard called the SERCA tool, which DHA  officials told us will allow all MTF patient safety leaders to share lessons  learned in the course of sentinel event follow-up in real time. The SERCA  tool displays sentinel event and RCA data from DHA’s internal tracking  record reported by the military services and NCR. It is intended to provide  quick, online access to sentinel event trends MHS-wide and at the military  service, NCR, and MTF levels. The SERCA tool is also intended to  facilitate sharing of lessons learned and best practices based on sentinel  events and RCAs in a single platform. DHA officials told us that  individuals with access to the system will be able to see a breakdown of  corrective actions submitted by other MTFs for a particular type of  sentinel event and whether these corrective actions were rated as  stronger, intermediate, or weaker by DHA. DHA officials told us that for  now, they will allow the military services and NCR to determine who has  access to the system. Officials from two military services and NCR told us  that they have access to this tool and are responsible for granting access  to their MTFs. One MTF we visited told us they have access to this tool.  However, it is too early to evaluate how the SERCA tool will be used to  make improvements.

Each year, thousands of adverse medical events are reported at MTFs.  Tracking and conducting follow-up on these events is crucial for officials  to learn from and prevent these events in the future. As DHA assumes  administrative responsibility for all MTFs, its role in ensuring that sentinel  events—the most serious type of adverse medical events—are reported  and tracked and that required follow-up is conducted will become  increasingly critical. However, the current fragmented and inconsistent  tracking process across the military services and NCR has impeded the  efficiency of DHA’s efforts to ensure DHA has complete information about  sentinel events and RCA reports. Furthermore, DHA cannot ensure that it  is receiving all reports on the implementation of corrective actions, such  as MOS reports, and does not know how many reports it is missing for a  number of reasons, including those related to policy, tracking, and  reconciliation efforts. Collectively, all of these information gaps impair  DHA’s ability to fully understand the types of sentinel events that are  occurring in its MTFs, the corrective actions that have been implemented,  and whether these actions have been effective. This information is  essential to prevent adverse medical events from occurring in the future  and to ensure that the care provided by MTFs is safe and effective.

We are making the following two recommendations to the Assistant  Secretary of Defense (Health Affairs):

Ensure DHA improves as appropriate the systems and processes  used by the military services, NCR, and DHA to track sentinel events  and RCA reports and require the military services and NCR to  communicate with DHA the reasons RCA reports are not completed  for reported sentinel events. (Recommendation 1)

Ensure DHA clarifies its requirement that reports on the  implementation of corrective actions, such as MOS reports, should be  completed and submitted to DHA, and to work with the military  services and NCR to develop a standard system to help DHA  consistently track and reconcile information about individual reports.  (Recommendation 2)

DOD provided written comments on a draft of this report, including  technical comments, which we incorporated as appropriate. In its written  comments, which are reprinted in appendix II, DOD concurred with both  of our recommendations. In response to our first recommendation, DOD  acknowledged that its current tracking efforts for sentinel events and  RCAs are fragmented, inefficient, and unreliable. DOD stated that in the  future, it envisions a single system to track and monitor sentinel events,  RCAs, and corrective action implementation plan reports.  A single  system would eliminate the fragmentation associated with tracking these  reports and the need for a cumbersome reconciliation process, potentially  improving the completeness and reliability of DHA’s patient safety data as  well as its ability to identify and implement system-wide improvements.

In response to our second recommendation, DOD stated that it will clarify  the difference between an MOS report, which may be required by the  Joint Commission, and a corrective action implementation plan report,  which will always be required by DOD for reported sentinel events. DOD  explained that when an MOS report is required by the Joint Commission,  this report will satisfy DOD’s requirement.  However, when the Joint  Commission does not require an MOS report for a sentinel event, DOD  will require a corrective action implementation plan report.  DOD stated  that it expects the revised policy to be signed in late summer 2018 and in  effect by October 1, 2018—the date that DHA is to assume responsibility  for the administration of all MTFs.

We are sending copies of this report to the Secretary of Defense and  appropriate congressional committees. In addition, the report will be  available at no charge on GAO’s website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact  me at (202) 512-7114 or at draperd@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs can be found on the  last page of this report. Other major contributors to this report are listed in  appendix III.

Appendix I: Department of Defense’s (DOD) Revised Definition of a Sentinel Event

In a March 2015, the Assistant Secretary of Defense for Health Affairs  issued a memorandum about improving the sentinel event and root cause  analysis (RCA) reporting processes. This memorandum also revised  DOD’s definition of sentinel events, which previously stated that a sentinel  event is an unexpected occurrence involving death or serious physical or  psychological injury or risk. The revised sentinel event definition is a  patient safety event (not primarily related to the natural course of the  patient’s illness or underlying condition) that reaches a patient and results  in death, permanent harm, or severe temporary harm. This revised  definition also includes additional types of events outlined by the Joint  Commission and the National Quality Forum. (See table 4.)

DOD described the following sentinel events that are outlined by the Joint  Commission:

Suicide of any patient receiving care, treatment, and services in a  staffed around-the clock care setting or within 72 hours of discharge,  including from the hospital’s emergency department.

Unanticipated death of a full-term infant or discharge of an infant to  the wrong family.

Abduction of any patient receiving care, treatment, and services.

Any elopement (unauthorized departure) of a patient from a staffed  around-the-clock care setting (including the emergency department),  leading to death, permanent harm, or severe temporary harm to the  patient.

Destruction of red blood cells transfusion reaction involving  administration of blood or blood products that have major blood group  incompatibilities.

Rape, assault (leading to death, permanent harm, or severe  temporary harm), or homicide of any patient receiving care, treatment,  and services while on site at the hospital.

Invasive procedure, including surgery, on the wrong patient, at the  wrong site, or that is the wrong (unintended) procedure.

Unintended retention of a foreign object in a patient after an invasive  procedure.

Severe neonatal excess of bilirubin (bilirubin >30 milligrams/deciliter).

Prolonged fluoroscopy with cumulative dose >1,500 rads to a single  field or any delivery of radiotherapy to the wrong body region or >25  percent above the planned radiotherapy dose.

Fire, flame, or unanticipated smoke, heat, or flashes occurring during  an episode of patient care.

Any maternal death or severe maternal or morbidity occurring during  or after birth (24 hours).

Appendix II: Comments from the Department of Defense

In addition to those named above, key contributors to this report were:  Bonnie Anderson, Assistant Director; Danielle Bernstein, Analyst-in- charge; Jacquelyn Hamilton; Elizabeth T. Morrison; Vikki Porter; and  Helen Sauer.

Medical errors, system vulnerabilities, or process failures can permanently or temporarily harm—or kill—patients. In military health facilities, it's critical to track and analyze these "sentinel events" to help prevent them in the future.

The military services and the Defense Health Agency both track such events. Agency officials told us they spend 80 hours a month reconciling their records with the military services, but in some instances, the records don't match. The agency may be missing opportunities to make military health care safer. We recommended improving the tracking of sentinel events and follow-up requirements.