Comparative Effectiveness Research: Activities Funded by the Patient-Centered Outcomes Research Trust Fund

In 2010, the Patient Protection and Affordable Care Act (PPACA) authorized the establishment of PCORI to carry out CER and improve its quality and relevance. PPACA also established new requirements for HHS to, among other things, disseminate findings from federally funded CER, including findings published by PCORI; and coordinate with relevant federal health programs to build data capacity for this research. To fund CER activities, PPACA established the Trust Fund from which PCORI and HHS are expected to receive an estimated $4.0 billion from fiscal years 2010 through 2019.

PPACA included a provision for GAO to review PCORI's and HHS's use of the Trust Fund. This report examines (1) PCORI's use of the Trust Fund for CER activities, including the dissemination and use of research findings; and (2) HHS's use of the Trust Fund for these activities.

GAO examined PCORI and HHS documents and data related to use of the Trust Fund, such as commitment, obligation, and expenditure data; PCORI's audited financial statements; and descriptions of CER activities. GAO also interviewed PCORI and HHS officials responsible for planning and carrying out CER activities and interviewed officials from stakeholder organizations representing potential users of CER, including public and private payer organizations, provider organizations, and patient organizations. PCORI and HHS provided technical comments, which GAO incorporated as appropriate.

The Patient-Centered Outcomes Research Institute (PCORI) made about $2 billion in commitments for awards in fiscal years 2010 through 2017. PCORI is a federally funded, nonprofit corporation established to carry out and improve comparative clinical effectiveness research (CER), which evaluates and compares the health outcomes and the clinical effectiveness, risks, and benefits of two or more medical treatments, services, or items. PCORI provides funding through award commitments from the Patient Centered Outcomes Research Trust Fund (Trust Fund) and may pay these awards over multiple years. Of the $2 billion PCORI committed as of the end of fiscal year 2017, about $1.6 billion (or 79 percent of its commitments) is for research awards, and $325 million (or 16 percent) is for building the capacity to use existing health data for research. Through fiscal year 2017, commitments for dissemination and implementation awards—intended to share CER findings with potential users of this research—were limited because most PCORI-funded research was still underway. PCORI projects to commit an additional $721 million for awards in fiscal years 2018 through 2021. In addition to awards, PCORI spent $310 million on program and administrative support services in fiscal years 2010 through 2017 and projects to spend an additional $206 million for these services through fiscal year 2024.

From fiscal years 2011 through 2017, the Department of Health and Human Services (HHS) obligated about $448 million from the Trust Fund. Of this amount, HHS obligated about $260 million (or 58 percent of all obligations) to the dissemination and implementation of CER findings. As most PCORI-funded CER had not yet been completed due to the time needed to conduct this research, HHS efforts focused instead on the dissemination and implementation of CER funded by other federal entities. Additionally, HHS obligated funds for efforts to train researchers on conducting CER, build data capacity, and on administrative activities. HHS projects to obligate an additional $120 million for these activities in fiscal years 2018 through 2020.

Clinical research is often conducted to evaluate the effectiveness of a  specific treatment for a disease or condition; however, research is less  often conducted to compare the relative effectiveness of two or more  available treatment options. Comparative clinical effectiveness research  (CER) evaluates and compares the health outcomes and the clinical  effectiveness, risks, and benefits of two or more medical treatments,  services, or items. For example, a recent CER study found that the life  expectancy of women over age 50 with breast cancer in one breast, but  who do not carry the breast cancer gene mutation, is similar whether they  have one or both breasts removed. Another recent study compared the  effectiveness and safety of treatments for childhood anxiety disorders and  found that a combination of medications and therapy is likely more  effective than either treatment alone. According to the National  Academies of Medicine, CER can help clinicians, patients, payers, and  others make informed decisions to improve health care.

In 2010, the Patient Protection and Affordable Care Act (PPACA)  authorized the establishment of the Patient-Centered Outcomes  Research Institute (PCORI) as a federally funded, nonprofit corporation to  improve the quality and relevance of CER. In addition, PPACA established new requirements for the Department of Health and Human  Services (HHS) to, among other things, broadly disseminate findings from  federally funded CER, including findings published by PCORI; train  researchers on CER methodological approaches; and coordinate relevant  federal health programs to build data capacity for this research. To fund  PCORI’s and HHS’s CER activities, PPACA established the Patient- Centered Outcomes Research Trust Fund (Trust Fund) through which  PCORI and HHS expect to receive an estimated total of about $4 billion  from fiscal years 2010 through 2019.

PPACA included a provision for us to report on the use of the Trust Fund  for CER activities. In this report, we examine:  1.  PCORI’s use of the Trust Fund for CER activities, including the  dissemination and use of research findings, and  2.  HHS’s use of the Trust Fund for these activities.

To examine PCORI’s and HHS’s use of the Trust Fund for CER activities,  including the dissemination and use of research findings, we reviewed  PCORI award commitment and expenditure data and HHS data on  obligated funds. We also reviewed descriptions of PCORI’s and HHS’s  CER activities, relevant legislation, and previous GAO reports.  Furthermore, we conducted interviews with PCORI and HHS officials  responsible for planning and carrying out CER activities. Within HHS, we  interviewed officials from the Agency for Healthcare Research and Quality  (AHRQ), the Office of the Assistant Secretary for Planning and Evaluation  (ASPE), and the National Institutes of Health (NIH). We also interviewed  officials coordinating PCORI’s PCORnet initiative, the National Patient- Centered Clinical Research Network, to build data capacity for research.  We reviewed funding from fiscal year 2010, the first year funding was  made available, through 2017, the most recent data available at the time  of our analysis. To the extent available, we also reviewed the amount of  funding projected beyond fiscal year 2017. Finally, to gather the views of  potential users of CER findings, including public and private payers,  providers, and patients, we interviewed officials from several large  organizations that broadly represent each group of potential users.

To assess the reliability of PCORI commitment and expenditure data and  HHS obligation data, we collected information from PCORI and HHS  officials regarding the reliability of the data, including the accuracy of data  entry and the systems that contain the data. We also reviewed PCORI’s  annual audited financial statements. On the basis of these steps, we  determined the data were sufficiently reliable for the purposes of our  reporting objectives.

We conducted this performance audit from May 2017 to March 2018 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform our work to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

In 2010 PPACA authorized the establishment of PCORI to improve CER  quality and relevance. PPACA also established requirements for HHS to,  among other things, disseminate findings from federally funded CER,  including findings published by PCORI, and coordinate with relevant  federal health programs to build data capacity for research. PPACA  established a Trust Fund to fund these CER activities by PCORI and HHS  through fiscal year 2019.

PPACA authorized the establishment of PCORI as a federally funded,  nonprofit corporation aimed at advancing the quality and relevance of  evidence through research to help patients, clinicians, purchasers, and  policy-makers to make informed health care decisions. PCORI is  required to identify research priorities, establish a research project  agenda, fund research consistent with its research agenda, and  disseminate research findings, among other responsibilities. In 2015 we  reported that PCORI had conducted activities consistent with its  legislative requirements. For example, we reported that since its  inception in 2010, PCORI established and implemented priorities for  funding CER and related activities, developed plans to disseminate  funded research and track its utilization, and took steps to make its  research more centered on outcomes prioritized by patients. Further,  PCORI developed PCORnet as a distributed research network initiative  that enables electronic health-related data from multiple sources to be  available for research.

HHS CER-related Activities Required by PPACA

PPACA requires HHS to perform several requirements related to CER,  which it has implemented through AHRQ and ASPE. Specifically, AHRQ  is required to disseminate and support the incorporation of CER funded  by PCORI and other federal entities, as well as to foster capacity for  conducting CER by supporting training in the methods used to conduct  such research. ASPE, in turn, is required to build data capacity for  conducting CER. In 2015, we reported that AHRQ had taken some steps  to disseminate research findings, but had not taken other actions to help it  fully address its dissemination requirements. Furthermore, we reported  that ASPE coordinated among various agencies to fund projects intended  to build data capacity for research, but that its approach lacked key  elements—such as defined objectives, milestones, and time frames—that  are necessary to ensure effectiveness. In our 2015 report, we made five  recommendations to HHS to direct AHRQ and ASPE to address these  issues, as appropriate. HHS concurred with these recommendations and  specified actions it would take to address them. Four of the  recommendations have since been implemented.

PPACA established the Trust Fund through which PCORI and HHS  receive funds for CER activities. The law provides that for fiscal years  2010 through 2019, the Trust Fund will receive appropriations from the  general fund of the Treasury, transfers from the Medicare trust funds, and  fees collected by the Department of the Treasury (Treasury) from private  insurance and self-insured health plans. Eighty percent of the amounts in  the Trust Fund must be made available to PCORI in fiscal years 2011  through 2019, and Treasury must transfer the remaining 20 percent to the  Secretary of HHS in each of those years. Under current law,  appropriations and transfers to the Trust Fund will end in fiscal year 2019.  The law also provides that no amounts shall be available for expenditure  from the Trust Fund after September 30, 2019, and specifies that any  amounts remaining in the Trust Fund after that time will be transferred to  the general fund of the Treasury. (See fig. 1 for an overview of transfers  to the Trust Fund and distribution of funds to PCORI and HHS).

PPACA limits the use of CER in certain ways; for example, the law  prohibits PCORI from developing or using a dollars-per-quality adjusted  life-year to establish what type of health care is cost effective or  recommended, and prohibits the Secretary of HHS from using such  measures as a threshold to determine coverage, reimbursement, or  incentive programs under Medicare. HHS may use CER findings to help  inform Medicare coverage decisions, but PPACA does not allow Medicare  coverage to be denied solely on the basis of CER findings.

PCORI Committed Funds Primarily to Research and Data Capacity Efforts; Awards for Dissemination and Implementation of Findings Were Limited as Most Research Was Still Underway

In fiscal years 2010 through 2017, PCORI committed about $1.6 billion  (or 79 percent of its total award commitments of $2.0 billion) to awards for  conducting CER and $325 million (or 16 percent) to awards for building  data capacity for research. In addition, PCORI committed $93 million for  engagement and workforce awards to involve stakeholders in the  research process and expand the research workforce, and committed $12  million for awards to disseminate and implement its research findings.  Awards for the dissemination and implementation of its research findings  were limited as of the end of fiscal year 2017, as most of this research  was still underway. (See table 1 for PCORI’s award commitments for  fiscal years 2010 through 2017.)

By the end of fiscal year 2024, PCORI projects to spend a total of almost  $3.3 billion, which reflects its projected Trust Fund revenue through fiscal  year 2019 plus interest income. This total amount encompasses the  commitments PCORI has made for awards through fiscal year 2017, as  well as $514 million in projected additional research award commitments  to be made by the end of fiscal year 2019 and $207 million for other  award commitments to be made by the end of fiscal year 2021. In  addition to awards, the total includes PCORI’s expenditures for program  and administrative support services in fiscal years 2010 through 2017, as  well as projected expenditures for these services through fiscal year  2024. (See fig. 2 for PCORI’s actual and projected commitments and  expenditures and see app. I for an overview of PCORI’s awards.)

The following information provides details on PCORI’s awards related to  research, building data capacity, engagement and workforce activities,  and the dissemination and implementation of its research findings.

PCORI committed $1.6 billion, or 79 percent of its total award  commitments, for research in fiscal years 2010 through 2017. In fiscal  years 2018 and 2019, PCORI projects to commit an additional $514  million for research awards. PCORI research awards have increasingly  focused on conditions that impose a substantial health or financial burden  on patients and the healthcare system. (See table 2 for information on the  health conditions that received the highest research award funding.)

Similar to certain types of CER that may take many years, the entire  research award process for PCORI-funded CER may span multiple years  from the funding announcement to the dissemination of completed  research. Specifically, the process PCORI established can take as  many as 6 years, which includes requesting and reviewing proposals,  awarding contracts, recruiting participants or obtaining data, conducting  and reviewing research, and disseminating findings and typically involves  awards that span multiple years. For example, PCORI estimates that  the typical timeframe for announcing funding and selecting applications to  receive research awards can take 8 to 11 months as PCORI brings  scientists, patients, payers, and other stakeholders together to prioritize  proposals based on the impact of the condition, potential to improve  health, technical merit, patient-centeredness, and engagement. (See fig.  3.)

Most of PCORI’s research projects, awarded through fiscal year 2017,  were still underway. Only 53 of its 543 research projects had been  completed as of the end of fiscal year 2017—in part because PCORI’s  research award process typically takes 2 to 6.5 years to complete, and  because almost two-thirds of the funds committed for research projects  were awarded in fiscal years 2015 through 2017. While most PCORI- funded research is underway, a larger number of research studies are  projected to be completed by of the end of each year from 2018 to 2022,  with all of the remaining studies to be completed by 2024. (See fig. 4.)  PCORI officials told us that the institute attempts to manage its funds to  ensure that its research awards are funded and managed through  completion, including peer review and the distribution of research  findings, in recognition of the time needed to conduct this research as  well as the uncertainty regarding the total amount of funding available.

Officials from all but one of the stakeholder organizations we  interviewed—public and private payers, health care providers, and patient  advocacy organizations that represented potential users of CER— generally supported PCORI’s research award priorities. Most of the  stakeholders we interviewed stressed the importance of research  conducted by unbiased organizations, such as the federally funded  research funded by PCORI and HHS. In addition, most stakeholders also  told us that PCORI’s efforts to engage patients in the research process  have changed the way research is conducted for the better, such as  prioritizing research outcomes that are most meaningful to patients.  However, officials from an organization representing payers (and from an  individual health plan) told us that PCORI’s priorities did not fully align  with their needs, such as their needs for CER on certain high-cost  conditions, medications or treatments.

PCORI committed the second largest portion of award funding—$325  million through fiscal year 2017—for awards to build data capacity for  research through the development of PCORnet. PCORI officials told us  that the institute supported the development of the PCORnet initiative in  order to use existing medical records and claims data and to transform  much of that data into a common data model to be used for clinical  research, until such time when such data will have been standardized in  electronic health records so that they can easily be used for research. As  of December 2017, PCORnet included 36 partner networks agreeing to  link their electronic claims and health data. PCORI officials told us that  this distributed data network already comprises a nationally  representative sample of approximately 128 million individuals whose  data can be used in randomized clinical trials, large observational studies,  and other research. In fiscal years 2018 and 2019, PCORI projects to  commit an additional $70 million for these awards to continue building this  data capacity.

PCORnet research is managed through its Coordinating Center, which  oversees the translation of certain categories of the partner networks’  data into the common data model and forges agreements with each of the  partners to share results of queries using their data with researchers.

This research process generally starts when a researcher requests to  query data on a specific population, after which PCORnet may approve  the request and invite network partners to participate. Participating  network partners then run queries on their data following established  parameters and submit the results to a secure portal that the researcher  can access in order to analyze the results for research. (See fig. 5.)  PCORI officials told us that there were 32 research projects using  PCORnet that received funding through PCORI’s research award process  as of the end of December 2017, as well as 45 research projects funded  by other parties, including federal agencies and private industry.

Further, as part of its building data capacity awards, in fiscal year 2017  PCORI committed $25 million to the People-Centered Research  Foundation, a nonprofit foundation formed in March 2017 to support the  network partners and other entities conducting research using PCORnet.  This funding was provided to support this foundation’s development of a  business plan, as well as its governance structure, to ensure the  continuity of the PCORnet network partnership efforts after PCORI  funding for PCORnet ends. PCORI has indicated it may provide additional  funding to the foundation, provided that the foundation and the networks  make progress toward self-sustainability.

Officials from most stakeholder organizations we interviewed generally  agreed that PCORnet offers value by improving the data available to  conduct CER. Officials from two organizations told us that PCORnet has  made it possible to use network partners’ aggregated data to make  conducting research more efficient than in the past.

Through fiscal year 2017, PCORI also committed $93 million for  engagement and workforce awards. For example, PCORI committed a  total of $63 million for engagement awards, intended to involve a variety  of stakeholders in the research process and to improve the methodology  for carrying out CER. Engagement awards include “Eugene Washington  Engagement Awards” that are intended to bring patients, caregivers,  clinicians, and other healthcare stakeholders into the research process  and to disseminate study results. In addition, “Pipeline to Proposal  Awards” are intended to bring together stakeholders with strong interests  in a specific health issue to develop research proposals to address their  needs. Officials from the two patient advocacy organizations we  interviewed told us that PCORI’s engagement awards have helped to  support patient involvement in the research process. For example, one  official noted that, while it has not been easy to find patients willing to  participate, these awards have been important to train and support  patients in the research process.

PCORI also committed $30 million to workforce training awards for  clinicians and researchers. For example, one of PCORI’s career  development programs, conducted in partnership with AHRQ, is designed  to train clinician and research scientists to conduct patient-centered  outcomes research and to actively engage stakeholders in efforts to  improve the quality and safety of care.

Dissemination and implementation awards for PCORI-funded research  findings thus far have been limited as most of the research was still  underway, but, according to PCORI officials, awards for this work will  substantially increase as research is completed. Specifically, through  fiscal year 2017, PCORI committed a total of $12 million for awards to  disseminate and implement PCORI-funded research by helping  researchers and other stakeholders to publicize findings and by  supporting patients and providers to utilize findings. PCORI projects to  commit an additional $91 million for these awards in fiscal years 2018  through 2021.

Dissemination and implementation awards are intended to encourage  PCORI awardees that have completed research and their patient and  stakeholder partners to pursue strategic activities to disseminate and  implement their findings. For example, PCORI awarded about $0.4 million  to increase awareness and promote the use of research findings on using  technology to deliver virtual care home visits for those with Parkinson’s  disease. According to PCORI, these funds will be used to train  neurologists and other health professionals to provide virtual care for  patients in their homes. In addition, as part of its efforts to summarize  research findings, PCORI also awarded funds to the American Institutes  for Research to establish a Translation Center that develops two  summaries of each of PCORI’s research findings: a public abstract for  general audiences that is also translated into Spanish and a professional  abstract for clinicians.

In addition to awards, PCORI has fostered the dissemination and  implementation of its research findings in other ways, including through its  website, publications, and roundtable briefings. For example, according to  PCORI, it posts research findings on its website within 90 days of  receiving final peer-reviewed research results so that patients and  providers have access to the information to make healthcare decisions. In  addition, according to PCORI, it pays journals’ open access fees to allow  free public access to selected research and plans to support research  awardees to place accepted journal manuscripts in the PubMed Central  database. PCORI also facilitates roundtable briefings that bring together  clinicians, patients, and others with interests in recent findings in order to  build support for immediate use of the findings. PCORI also coordinates  its dissemination efforts with AHRQ.

PCORI considers the implementation of its research methods and  findings to be an integral part of its dissemination efforts and a  culmination of its work and so has begun efforts to track implementation,  such as the number of its findings published in peer-reviewed journals,  and the use of its findings in clinical care. For example, PCORI officials  told us that that there were 891 publications in peer-reviewed journals  that resulted from studies fully or partially funded by PCORI through  October 2017. According to PCORI, two PCORI-funded studies on  prostate cancer, one study on oral versus intravenous antibiotics for  certain children, and one study on self-monitoring of blood glucose were  included in medical resource software that is used by nearly 90 percent of  academic medical centers in the United States.

Most of the stakeholder officials we interviewed noted the importance of  disseminating research findings quickly and in ways that are readily  available and understandable to both experts and the general public to  raise awareness about the findings. While officials representing two  payers noted limitations to the usefulness of PCORI’s research findings  because they do not take treatment costs into account, most stakeholder  officials noted the importance of the PCORI-funded research underway  and looked forward to utilizing the research findings once they become  available. In particular, officials representing provider and patient  advocacy organizations told us that they were interested in ensuring that  the most important research findings would be quickly implemented by  patients and clinicians.

HHS Obligated Funds Primarily for the Dissemination and Implementation of Research

Between fiscal years 2011 and 2017, HHS’s AHRQ obligated about $260  million (or 58 percent of HHS’s $448 million in total obligations) for the  dissemination and implementation of CER findings. According to AHRQ  officials, because most PCORI-funded research had not been completed  by the end of fiscal year 2017, these efforts were primarily focused on the  dissemination and implementation of research funded by other entities,  including NIH and the Centers for Disease Control and Prevention (CDC).  Additionally, AHRQ obligated $94 million for efforts to train researchers  on conducting CER, and ASPE obligated $85 million for efforts to build  data capacity. AHRQ and ASPE have obligated a total of $9 million for  administrative activities during those years. Table 3 provides an overview  of HHS’s obligations in each fiscal year.

AHRQ and ASPE plan to obligate an additional $120 million for  dissemination and implementation, training, building data capacity, and  administrative activities during fiscal years 2018 through 2020. They  expect to have $245 million available to fund ongoing and future CER  activities, based on expected transfers from the Trust Fund in fiscal years  2018 and 2019. (See fig. 6.)

The following information provides details on HHS-funded projects related  to dissemination and implementation, training on conducting CER, and  building data capacity.

During fiscal years 2011 through 2017, AHRQ obligated a total of $260  million for CER dissemination and implementation initiatives and plans to  obligate an additional $93 million for these initiatives in fiscal years 2018  through 2020. According to officials, AHRQ plans to fund additional  dissemination and implementation initiatives in fiscal years 2018 and  2019 but had not finalized those plans as of January 2018. (See app. II  for an overview of all of AHRQ’s dissemination and implementation  initiatives.) AHRQ’s dissemination and implementation initiatives  comprise efforts to synthesize CER findings, translate and communicate  research findings to potential users, and implement them:

Synthesis of CER findings: According to AHRQ officials, AHRQ’s  Evidence-Based Practice Centers developed 48 systematic reviews of  CER findings based on completed research. As of the end of fiscal  year 2017, 40 of these reviews had been published, while 8 were still  in progress. Officials told us that these systematic reviews have likely  not included PCORI-funded research, as most of that research had  not been completed by the end of fiscal year 2017.

Translation and communication of CER findings: AHRQ funded  initiatives, which—according to the agency—are aimed at making  CER findings accessible and understandable to health care  professionals, patients, and others. For example, AHRQ developed a  “Library of Patient-Centered Outcomes Research Resources” website  with links to CER databases maintained by other entities including  NIH and PCORI. Another example is AHRQ’s “John M. Eisenberg  Center for Clinical Decisions and Communications Science,” which  translates research findings into information that can be used by  consumers, health care providers, and policymakers.

Implementation of CER findings: AHRQ funded four key initiatives to  implement CER findings. According to AHRQ officials, one of the four  initiatives includes PCORI-funded research, while the other three  have thus far focused on implementing existing CER funded by other  entities:

The “Dissemination and Implementation Initiative” was designed  to disseminate and implement government-funded CER findings— including PCORI-funded findings—relevant to physicians,  healthcare providers, patients, and others. This initiative consists  of a multi-step approach for identifying several areas of CER each  year that—according to AHRQ officials—have the greatest  potential for impact and are feasible to implement. (See figure 7  for an overview of this process.) According to AHRQ officials, as  of December 2017, 37 findings have been nominated for  consideration under AHRQ’s Dissemination and Implementation  Initiative, including 5 findings nominated by PCORI. According to  these officials, 1 of the findings PCORI has nominated is under  consideration for implementation. Two were rejected—1 because  of insufficient impact and the other because of challenges in  implementation feasibility. (Two are still under review.)

The “Evidence Now” initiative disseminates CER evidence directly  to primary care practices and supports them in implementing  clinical and organizational evidence in practice through regional  cooperatives.

The “Comparative Health System Performance Initiative”  established three centers of excellence and a coordinating center  to identify, classify, track, and compare health systems. AHRQ’s  goal is to understand the factors that affect health systems’ use of  CER and to identify best practices in disseminating and using  CER.

The “Clinical Decision Support (CDS) Initiative” is designed to use  CDS to promote the timely incorporation of CER findings into  clinical practice.

Some of AHRQ’s dissemination and implementation initiatives—such as  “Evidence Now” and “CDS Initiative”—include an evaluation component,  as described in app. II. According to AHRQ officials, as of January 2018  results from these evaluations were not yet available.

Between fiscal years 2011 and 2017, AHRQ obligated a total of $94  million for awards supporting training in the methods used to conduct  CER. AHRQ plans to obligate an additional $14 million for training on  conducting CER by fiscal year 2020. AHRQ has funded eight categories  of awards for individual researchers or research institutions. For example,  AHRQ’s “Infrastructure Development Program in Patient-Centered  Outcomes Research” award supports institutions in the development of  their capacity to conduct and implement CER. Its “Institutional Mentored  Career Development Award Program in Patient-Centered Outcomes  Research” award supports the development of researchers in academic  and applied settings. (See app. III for an overview of these awards.)  Starting in fiscal year 2018, AHRQ plans to fund an additional training  award category in conjunction with PCORI. AHRQ developed a plan to  evaluate its training activities and, according to AHRQ officials, the  evaluation is expected to be funded in fiscal year 2018.

Between fiscal years 2012 and 2017, ASPE obligated a total of $85  million for 30 projects designed to build data capacity for conducting CER  and plans to obligate an additional $6 million to existing projects and 1  new project through fiscal year 2019. Officials told us that ASPE plans to  fund additional projects to build data capacity in fiscal years 2018 and  2019, based on HHS leaders’ priorities, but had not finalized those plans  as of January 2018. ASPE manages these projects, which are largely  carried out by other HHS agencies through interagency agreements and  are intended to develop and maintain a comprehensive, interoperable  data network to collect, link, and analyze data on outcomes and  effectiveness from multiple sources for CER. (See app. IV for an  overview of these activities.)

In response to a recommendation in our 2015 report on HHS’s CER  activities, ASPE implemented a monitoring system to track progress  toward its milestones and deliverables for these projects. ASPE also  contracted to evaluate its projects to build data capacity for CER. The  evaluation, completed in December 2017, found that ASPE made  progress managing these projects towards the core functionalities  outlined in its strategic framework. However, among other things, the  evaluation found that additional efforts are needed to explore how to  enhance data privacy and security, ensure data quality, and  operationalize related standards. According to ASPE officials, the  evaluation will inform the development and implementation of future  ASPE projects to build data capacity for conducting CER.

We provided a draft of this report to PCORI and HHS for review and  comment. PCORI and HHS provided technical comments, which we  incorporated as appropriate.

We are sending copies of this report to the appropriate congressional  committees, the Executive Director of PCORI, the Secretary of Health  and Human Services, the Director of AHRQ, the Assistant Secretary for ASPE, and other interested parties. In addition, the report will be available  at no charge on GAO’s website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact  me at (202) 512-7114 or dickenj@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made major contributions to this report  are listed in app. V.

Appendix I: Patient-Centered Outcomes Research Institute (PCORI) Award Commitments Made During Fiscal Years 2010 through 2017

Appendix I: Patient-Centered Outcomes  Research Institute (PCORI) Award  Commitments Made During Fiscal Years 2010  through 2017 Award category description   These awards generally fund research studies in priority areas for conditions that impose a  substantial burden on patients and the healthcare system.  Information about individual research awards can be found at:  https://www.pcori.org/research-results?f%255B0%255D=field_project_type%3A298&f%5B0%5D=fiel d_project_type%3A298#search-results.

These awards fund infrastructure projects to build data capacity through the development of  PCORnet and support for clinical and patient-powered data research networks.  Information about individual building data capacity awards can be found at:  https://www.pcori.org/research-results?f%5B0%5D=field_project_type%3A441#search-results.

Engagement awards fund projects to improve the methodology for carrying out research by involving  patients, caregivers, clinicians, and other healthcare stakeholders into the research process.  Workforce training awards provide accredited continuing education opportunities, in coordination with  the Agency for Healthcare Research and Quality, for researchers and clinicians.  Information about individual engagement awards and workforce awards can be found at:  https://www.pcori.org/research-results?f%5B0%5D=field_project_type%3A299#search-results and  https://www.pcori.org/research-results/2017/k12-institutional-mentored-career-development-program.

Dissemination and implementation awards are intended to help researchers and other stakeholders  to publicize findings, and support the utilization of findings for patients and providers.  Information about individual dissemination and implementation awards can be found at:  https://www.pcori.org/research-results?f%5B0%5D=field_project_type%3A308#search-results.

Appendix II: Agency for Healthcare Research and Quality’s (AHRQ) Dissemination and Implementation Initiatives, as of September 30, 2017

Appendix II: Agency for Healthcare Research  and Quality’s (AHRQ) Dissemination and  Implementation Initiatives, as of September 30,  2017 Description  Disseminates evidence to primary care practices and supports them in implementing clinical  and organizational evidence in practice through regional cooperatives. AHRQ awarded a  separate grant to establish an independent, external evaluation to study improvements in  the delivery of the ABCs.

Develops three Centers of Excellence on “Comparative Health System Performance in  Accelerating PCOR Dissemination.” According to AHRQ, the Centers of Excellence will  identify and classify characteristics of health care systems over 5 years. They will also  identify ways to assess the quality and cost of such systems, including their use of PCOR,  understand the characteristics of high performing systems, and identify what system  characteristics are associated with more rapid adoption and diffusion of PCOR- recommended practices throughout a system.

Evaluates and synthesizes research findings to aid decision-making for patients, providers,  and payers, among others.

Translates PCOR findings into tools, such as research summaries and decision aids,  designed to help patients and consumers, clinicians, and policymakers make informed and  evidence-based health care decisions.

Expands an existing initiative or creates a new initiative that supports multi-site, multi-region,  multi-stakeholder dissemination and implementation of evidence.

Develops and tests methods for translating and disseminating PCOR findings to hard-to- reach audiences, including patients with low health literacy, disadvantaged populations,  isolated clinicians and policy makers, and other decision makers who may not have had the  benefit of more traditional translation and dissemination efforts.

Integrates PCOR into clinical practice using various methods shown to improve the uptake  of scientific evidence in clinical decision making. Grantees were asked to consider both  educational theory and the relevance of “new media” as they designed their programs.

Searches for emerging interventions, prioritizes those most likely to have a large impact in  the near future, and disseminates the information to the public. According to AHRQ, the  Horizon Scanning System screened more than 22,000 potential intervention leads and  tracked over 2,300 intervention topics.

Provides targeted audiences—such as providers and payers—with an accessible tool for  obtaining objective, detailed information on evidence-based clinical practice guidelines to  further their dissemination, implementation, and use.

Promotes collaboration, reduces redundancy, and improves transparency in patient  registries.

Description  Created informational tools to support the dissemination and implementation of PCOR  findings, including best practices and new knowledge about the use of electronic health  record data for research and quality improvement.

Promotes the timely incorporation of PCOR findings into clinical practice—which  encompasses a variety of tools to enhance clinical decision-making.

Collaborated with 176 national organizations to disseminate materials for the Effective  Health Care Program.

Multi-media campaign to educate health care consumers about the value of reviewing  medical evidence when weighing treatment options.

Educating the Educators  Conducted and disseminated research to develop a process for shared decision making that  includes exploring and comparing the benefits, harms, and risks of each option through  meaningful dialogue about what matters most to the patient.

Collects patient-generated health data, integrates patient-generated health data with PCOR  evidence, and disseminates PCOR findings using mobile health technology.

Identifies ways to reduce health care differences across diverse populations with a particular  focus on minority populations in under-resourced healthcare settings.

Established five regional offices, responsible for developing and cultivating dissemination  partnerships within each region.

This repository houses study data extracted from primary research publications during the  course of conducting systematic reviews. It is designed to increase the transparency of  comparative effectiveness reviews, improve the ability to update systematic reviews,  improve the quality of abstracted data, and enhance the efficiency and reduce the costs of  conducting reviews.

Gathers input from patients on a complex topic related to the implementation of evidence- based health-care decision making.

Increases the relevance of AHRQ systematic reviews for patients, clinicians, and  policymakers by examining and addressing challenging topic areas that may affect the  credibility and utility of the review for end users and that are areas of inconsistency or  variation among AHRQ systematic reviews.

Conducted three projects to improve the development of registries, a major activity of  AHRQ’s Effective Health Care Program.

Description  Provides online continuing education materials that inform physicians and other health care  providers about PCOR from the Effective Health Care Program.

Worked with health professional student associations to evaluate students’ understanding of  the importance and clinical applicability of PCOR and shared decision-making to their  practice and evaluated students’ educational needs and preferences related to integrating  PCOR findings into their training curricula.

Created a decision-modeling methods center that reviewed the existing research and  guidance published on modeling methods with input from a multidisciplinary group of  experts.

Provides for maintenance and updating of existing data resources to conduct future CER  through a grant competition. The grants fund three to four 1-year pilot projects aimed at  enabling a future, larger competition to enhance the data infrastructure and move the  resources to self-sustaining models.

Disseminates CER findings published by the Patient-Centered Outcomes Research Institute  (PCORI) and other government entities to providers, patients, payers, and others. This  initiative consists of a seven-step approach for identifying research findings that have the  greatest potential for implementation.

Provided an understanding of how AHRQ could effectively disseminate and promote PCOR  findings and tools in the development and maintenance of clinical decision support systems.  The project included a market analysis and an assessment of potential stakeholders and  audiences, including vendors of health information technology focused on clinical decision  support. Information gathered from this project directly informed the concept for the PCOR  clinical decision support initiative that was launched in 2016.

Promoted PCOR through public service announcements nationwide.

Created a new page on AHRQ’s website that highlights the agency’s own resources, as well  as directs researchers, health professionals, patients, caregivers, and families to additional  databases that collect information on CER. These databases provide summaries of findings  from a wide range of CER findings and research that is in progress.

Appendix III: Agency for Healthcare Research and Quality’s (AHRQ) Training Awards, as of September 30, 2017

Appendix III: Agency for Healthcare Research  and Quality’s (AHRQ) Training Awards, as of  September 30, 2017 Description  Funds a 5-year, renewable effort to support the development of PCOR capacity among  institutions that have basic health services research capacity but need to develop capacity to  conduct and implement PCOR. The program would potentially include institutions located in  geographic areas that lack capacity, and institutions that serve predominantly minority  populations.

Supports the development of researchers in academic and applied settings. The program  combines didactic and experiential opportunities, focusing on the generation, adoption, and  spread of new scientific evidence. The goal is to improve population-specific health  outcomes by developing and disseminating evidence-based information to patients,  clinicians, and other decision-makers, responding to their expressed needs, about which  interventions are most effective for which patients under specific circumstances.

Provides basic, advanced, and experiential training on the methods to conduct PCOR,  particularly prospective observational research, registries, and clinical trials. The program  was open to researchers employed in both the public and private sectors, particularly those  who serve minorities, economically or medically disadvantaged populations.

Facilitates the transition of postdoctoral candidates from mentored to independent research  positions, accelerating research independence for PCOR researchers.

Provides support for intensive, research career development for individual investigators in  academic or applied settings, leading to research independence in the field of PCOR and  the generation and translation of new scientific evidence and analytic tools.

Provides career development awards for established investigators to further develop their  research expertise in PCOR methodologies. This concept seeks to accelerate the  development of the research workforce capable of conducting PCOR.

Provides 2-year fellowships for training in PCOR. A focus for these fellowships is recruitment  of trainees from diverse disciplines, including social and behavioral sciences, business, and  engineering. The expected output of these fellowships is trained PCOR researchers.

Establishes an expert panel, comprised of 7 to 10 leaders in the fields of learning healthcare  system, health services research, and PCOR, to assess the current state of health services  research and PCOR training and recommend ways to improve core  competencies/curriculum to meet the needs of the health system. Develops a report  summarizing the panel’s recommendations concerning current deficiencies and  recommendations regarding skills and competencies needed to meet the challenges.

Appendix IV: Office of the Assistant Secretary for Planning and Evaluation’s (ASPE) Projects to Build Data Capacity, as of September 30, 2017

Total  obligations  (dollars in  millions)

Developed technical standards for how health care providers, researchers, and the  public health community access and extract data from electronic health records to  conduct Patient-Centered Outcomes Research (PCOR).

Identified and developed the functional and technical specifications necessary to  enable electronic health record systems to retrieve, display, and fill a structured  form or template and store and submit the completed form to an external repository.

Provided researchers with access to the Centers for Medicare & Medicaid Services’  Chronic Conditions Warehouse, which contains Medicare and Medicaid beneficiary,  claims, and assessment data, and supported infrastructure enhancements to  conduct CER.

Longitudinal follow-up of certain cancer patients to assess vital statistics, disease  recurrence, disease progression, and additional treatment types. Treatment data  submitted each year to the Centers for Disease Control and Prevention and  provided to researchers through the National Center for Health Statistics Research  Data Center.

Included clinical encounters for all patients and all conditions seen at the  community health centers from 2006 to 2013 in the Community Health Applied  Research Network Registry data warehouse, a research network comprising  18 community health centers. A de-identified analytic file and associated data  codebook were developed to support the use of analytic files by researchers  outside of the network.  Established a process for investigators to access the data warehouse through  the development of a data access plan.

Maintained the infrastructure for PCOR and for quality improvement in the  safety net.

Developed common data elements and standards for CER .The results were the  initial entries into the National Institutes of Health’s National Library of Medicine  common data element repository.

Developed a conceptual framework and environmental scan; produced policy  documents ranging from patient-initiated data, through research data on care  processes, transitions and coordination, to researcher access to claims data; and  developed the ‘HHS Strategic Roadmap for Building Data Capacity for Clinical  Comparative Effectiveness Research.’   The overall CER Inventory project was to design and implement a system for the  categorization and cataloguing of CER activities through a web-based tool. Due to  the rapidly evolving technologies supporting web-based search engines, and the  improved methods for identification of more recent CER, the development of the  CER Inventory (as a web-based search engine using a retrospective algorithm) was  determined to have been superseded by existing search engine tools available.

Description  Designed and conducted an independent evaluation of the ASPE portfolio to  systematically assess progress related to the strategic framework functionalities.

Conducted CER analyses on the beta release of the Multi-Payer Claims Database  and evaluated beta testers’ experiences requesting and using data from the MPCD  for research. Results of the beta test found that the project was successful in  achieving the key objectives of building a pilot database.

Planned for development and implementation of the Centers for Medicare &  Medicaid Services’ Blue Button—a service that allows patients to access their own  health information in electronic form.

Linkage of data on fact, cause, and manner of death from the National Death Index  to several federal population-based health data platforms in order to demonstrate  the feasibility of such linkage, enable PCOR on patterns and correlates of mortality  via the resulting linked data; and to facilitate collaboration between federal partners  regarding strengthening the infrastructure and methods for linking healthcare data  to mortality outcomes and using such linked data for PCOR.

Improve the infrastructure to support timely and complete mortality data collection  through more timely delivery of state death records to the National Death Index  database and by linking National Death Index database records with nationally  collected hospital datasets to obtain a more complete picture of patient care.

Identify the best patient attributes to address the challenge of linking patients’ data  across research, clinical, and claims data sets in order to support the PCOR data  infrastructure that enables standardization and sharing of patient data across  organizations.

Create a coordinated registry network for women’s health technologies that will  collect patient reported outcomes and employ structured data capture from  electronic health records for data collection and exchange.

Build data infrastructure for conducting PCOR using data from routine clinical  settings. The sources of these data may include, but are not limited to, insurance  billing claims, electronic health records, and patient registries. This project intends  to harmonize several existing common data models, potentially including PCORnet  and other networks.

Develop technical tools for collecting and integrating patient-reported outcome  assessments into electronic health records or other health information technology  products.

Create an interface that enables CMS beneficiaries to connect their  MyMedicare.gov data to applications and services they trust, including research  platforms related to research studies in which the beneficiary may be interested in  participating.

Provide technical assistance to the Trust Fund awardees in informatics and assist  ASPE in setting up additional oversight processes and procedures to monitor  progress.

Description  Develop a privacy and security data infrastructure blueprint, legal analysis, and  ethical framework to address legal and privacy and security related policy issues  that affect the use of data for various types of PCOR.

Convene clinical topic-specific working groups to discuss the data definitions  currently in use and how these definitions can be harmonized to promote common  definitions for outcome measures across systems. These common definitions are to  be made publicly available to PCOR researchers and analysts.

Develop a natural language processing service that will be accessible and publicly  available to researchers on the Public Health Community Platform – a cooperative  platform for sharing interoperable technologies to address public health priority  areas aimed at improving population health outcomes and health equity (e.g.,  tobacco use).

Leverage the Sync for Science and Blue Button application programming interface  programs to enable Medicare beneficiaries to donate their medical claims data for  scientific research studies.

Develop and test the capability to conduct timely and secure distributed regression  analysis in distributed data networks. Additionally, explore the feasibility of creating  virtual linkage capabilities to utilize data from multiple data sources and data for one  specific patient with information at different institutions.

Create the infrastructure for collecting data from patients through a mobile device  application, allowing patient-generated data to be linked with a single data partner  that participates in the Food and Drug Administration’s Sentinel distributed network.  The project will develop and pilot a mobile application to capture data from pregnant  women who volunteer to participate.

Develop a policy framework for the use of patient-generated data in research and  care delivery that addresses data collection tools, data donation policies, regulatory  gaps, combining data with medical record data, and interoperability of data across  health information systems and devices.

Create and implement a metadata standard data capture and querying system for  data quality and characteristics, data source and institutional characteristics, and  “fitness for use.”

Cross-Network Directory Service  Create an interoperable service that allows data partners to participate in multiple  data research networks, query across the networks, and share analytic capabilities  and knowledge across networks. The project will be piloted across two existing  networks: Food and Drug Administration’s Sentinel and PCORnet.

Generate tools and data standards that could be deployed in other CER studies by  leveraging the infrastructure of an existing research study called the ADAPTABLE  trial (Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long Term  Effectiveness). This trial is the first major randomized comparative effectiveness  trial to be conducted by PCORnet.

Description  Create a flexible, extensible, and computable mechanism for rolling data into  clinically relevant equivalence groups that enable more efficient processing  aggregation of laboratory data and other data from diverse health information  technology systems. The primary focus of this work will be on laboratory tests.

Total  obligations  (dollars in  millions)

Create a single point data capture approach from the electronic health record to  electronic data capture systems using the Retrieve Form for Data Capture  standard. Stakeholders will be provided with a tool to seamlessly integrate  electronic health record and electronic data capture systems.

In addition to the projects listed, ASPE plans to obligate $2.0 million for one new project starting in  fiscal year 2018.

In addition to the contact named above, Karin Wallestad, Assistant  Director; Michael Zose, Analyst-in-Charge; Kye Briesath; Laurie Pachter;  Vikki Porter, and Jennifer Whitworth made key contributions to this report.

The Patient Protection and Affordable Care Act established a fund to support research comparing the effectiveness of available treatments. This research can help patients, doctors, and others make decisions to improve health care.

We found that the Patient-Centered Outcomes Research Institute and Department of Health and Human Services expect to receive about $4 billion from the fund by the end of fiscal year 2019.

The money has primarily funded research and helped facilitate getting findings to the patients, doctors, and others who can use them. Most of the funded research will be completed in the coming years.


Waste Management: DOD Needs to Fully Assess the Health Risks of Burn Pits

Burn pits help base commanders manage waste generated by U.S. forces overseas, but they also produce harmful emissions that military and other health professionals believe may result in chronic health effects for those exposed.

This statement provides information on the extent to which DOD has assessed any health risks of burn pit use.

This statement is based on a GAO report issued in September 2016 (GAO-16-781). The report was conducted in response to section 313 of the Carl Levin and Howard P. “Buck” McKeon National Defense Authorization Act for Fiscal Year 2015. Specifically, GAO assessed the methodology DOD used in conducting a review of the compliance of the military departments and combatant commands with DOD instructions governing the use of burn pits in contingency operations and the adequacy of a DOD report for the defense committees. GAO also obtained updates from DOD on actions taken to assess health risks from burn pits since September 2016.

GAO reported in September 2016 that the effects from exposing individuals to burn pit emissions were not well understood, and the Department of Defense (DOD) had not fully assessed the health risks associated with the use of burn pits. Burn pits—shallow excavations or surface features with berms used to conduct open-air burning—were often chosen as a method of waste disposal during recent contingency operations in the U.S. Central Command (CENTCOM) area of responsibility, which extends from the Middle East to Central Asia and includes Iraq and Afghanistan. According to DOD Instruction 6055.01, DOD Safety and Occupational Health (SOH) Program , DOD should apply risk-management strategies to eliminate occupational injury or illness and loss of mission capability or resources. The instruction also requires all DOD components to establish procedures to ensure that risk-acceptance decisions were documented, archived, and reevaluated on a recurring basis. Furthermore, DOD Instruction 6055.05, Occupational and Environmental Health (OEH), requires that hazards be identified and risk evaluated as early as possible, including the consideration of exposure patterns, duration, and rates.

While DOD has guidance that applies to burn pit emissions among other health hazards, DOD had not fully assessed the health risks of use of burn pits, according to DOD officials.

According to DOD officials, DOD's ability to assess these risks was limited by a lack of adequate information on (1) the levels of exposure to burn pit emissions and (2) the health impacts these exposures had on individuals. With respect to information on exposure levels, DOD had not collected data from emissions or monitored exposures from burn pits as required by its own guidance. Given the potential use of burn pits near installations and during future contingency operations, establishing processes to monitor burn pit emissions for unacceptable exposures would better position DOD and combatant commanders to collect data that could help assess exposure to risks.

GAO recommended that the Secretary of Defense (1) take steps to ensure CENTCOM and other geographic combatant commands, as appropriate, establish processes to consistently monitor burn pit emissions for unacceptable exposures; and (2) in coordination with the Secretary of Veterans Affairs, specifically examine the relationship between direct, individual, burn pit exposure and potential long-term health-related issues. DOD concurred with the first recommendation and partially concurred with the second. In a May 2018 status update regarding these recommendations, DOD outlined a series of steps it had implemented as well as steps that it intends to implement. The department believes these efforts will further enhance its ability to better monitor burn-pit emissions and examine the relationship between direct, individual, burn pit exposure and potential long-term health related issues. GAO believes the steps DOD is taking are appropriate.

GAO made two recommendations focused on improving monitoring of burn pit emissions and examining any associated health effects related to burn pit exposure. DOD concurred with one recommendation and partially concurred with the other. GAO continues to believe the recommendations are valid.

I am pleased to submit this statement on our September 2016 report  covering the Department of Defense’s use of burn pits. Since the  initiation of military operations in Afghanistan in 2001 and Iraq in 2003,  the Department of Defense (DOD) has employed several methods to  dispose of the waste that U.S. forces have generated in both countries. In  general, the methods employed have been left to the discretion of base  commanders and include the use of incinerators, landfills, and open-air  burn pits on or near military bases. According to DOD officials, when  making these decisions base commanders may take into consideration a  number of factors, including the local security situation, the number of  personnel on the installation, and the amount and type of waste  generated by those personnel. As one of the options available, burn pits  help base commanders manage waste, but they also produce smoke and  harmful emissions that military and other health professionals believe  may result in acute and chronic health effects for those exposed to the  emissions.

My statement today focuses on the extent to which DOD has assessed  any health risks of burn pit use. This statement is based on our  September 2016 report. That work was conducted in response to section  313 of the Carl Levin and Howard P. “Buck” McKeon National Defense  Authorization Act for Fiscal Year 2015 (NDAA for Fiscal Year 2015).  Specifically, we assessed the methodology DOD used in conducting a  review of the compliance of the military departments and combatant  commands with DOD Instruction 4715.19, Use of Open-Air Burn Pits in  Contingency Operations, and the adequacy of the subsequent report  DOD sent to the defense committees containing the results of its review.

To evaluate the extent to which DOD has assessed any health effects of  burn pit use, we reviewed relevant health assessments on the effects of  burn pits, including a 2011 report by the Institute of Medicine that was  contracted by the Department of Veterans Affairs, as well as prior related  reports by GAO and the Special Inspector General for Afghanistan  Reconstruction. We also interviewed officials from U.S. Central Command  (CENTCOM), U.S. Army Central Command, U.S. Air Force Central  Command, Department of Veterans Affairs, and Institute of Medicine to  discuss any effects of exposures to burn pit emissions, among other  things. Additionally, we obtained an update from DOD in May 2018 on  actions taken regarding our findings and recommendations from our  September 2016 report.

We conducted the work on which this statement is based in accordance  with generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on  our audit objectives.

Burn pits—shallow excavations or surface features with berms used to  conduct open-air burning—were often chosen as a method of waste  disposal during recent contingency operations in the CENTCOM area of  responsibility, which extends from the Middle East to Central Asia and  includes Iraq and Afghanistan. In 2010, we reported that there were 251  active burns pits in Afghanistan and 22 in Iraq. However, in 2016, we  reported that the use of burn pits in the CENTCOM area of responsibility  had declined since that time. As of June 2016, DOD officials told us that  there were no military-operated burn pits in Afghanistan and only one in  Iraq. According to DOD officials, the decline in the number of burn pits  from 2010 to 2016 could be attributed to such factors as (1) using  contractors for waste disposal and (2) increased use of waste  management alternatives such as landfills and incinerators. However,  DOD officials acknowledged that burn pits were being used to dispose of  waste in other locations that are not military-operated. Specifically, these  officials noted instances in which local contractors had been contracted to  haul away waste and subsequently disposed of the waste in a burn pit  located in close proximity to the installation. In such instances, officials  stated that they requested that the contractors relocate the burn pit.  According to a DOD official, as of May 2018 there are two active burn pits  in the CENTCOM area of responsibility.

Although burn pits help base commanders to manage waste, they also  produce smoke and emissions that military and other health professionals  believe may result in acute and chronic health effects for those exposed.  We previously reported that some veterans returning from the Iraq and  Afghanistan conflicts have reported pulmonary and respiratory ailments,  among other health concerns, that they attributed to burn pit emissions.  Numerous veterans have also filed lawsuits against a DOD contractor  alleging that the contractor mismanaged burn pit operations at several  installations in both Iraq and Afghanistan, resulting in exposure to harmful  smoke that caused these adverse health effects. We also previously  reported on the difficulty of establishing a correlation between  occupational and environmental exposures and health issues. For  example, in 2012 we reported that establishing causation between an  exposure and an adverse health condition can be difficult for several  reasons, including that for many environmental exposures, there is a  latency period—the time period between initial exposure to a contaminant  and the date on which an adverse health condition is diagnosed. When  there is a long latency period between an environmental exposure and an  adverse health condition, choosing between multiple causes of exposure  may be difficult. In addition, in 2015 we reported that the Army had  recently published a study that evaluated associations between  deployment to Iraq and Kuwait and the development of respiratory  conditions post-deployment. However, the study was unable to identify a  causal link between exposures to burn pits and respiratory conditions.

DOD Had Not Fully Assessed the Health Risks of Burn Pits

In our 2016 report, we found that the effects from exposing individuals to  burn pit emissions were not well understood, and DOD had not fully  assessed these health risks. Under DOD Instruction 6055.01, DOD  Safety and Occupational Health (SOH) Program, it is DOD policy to apply  risk-management strategies to eliminate occupational injury or illness and  loss of mission capability or resources. DOD Instruction 6055.01 also  instructs all DOD components to establish procedures to ensure that risk- acceptance decisions were documented, archived, and reevaluated on a  recurring basis. Furthermore, DOD Instruction 6055.05, Occupational  and Environmental Health (OEH), requires that hazards be identified and  risk evaluated as early as possible, including the consideration of  exposure patterns, duration, and rates. Notwithstanding this guidance,  which applies to burn pit emissions among other health hazards, DOD  had not fully assessed the health risks of use of burn pits according to  DOD officials.

According to DOD officials, DOD’s ability to assess these risks was  limited by a lack of adequate information on (1) the levels of exposure to  burn pit emissions and (2) the health impacts these exposures had on  individuals. With respect to information on exposure levels, DOD had not  collected data from emissions or monitored exposures from burn pits as  required by its own guidance. DOD Instruction 4715.19 requires that  plans for the use of open-air burn pits include ensuring the area was  monitored by qualified force health protection personnel for unacceptable  exposures, and CENTCOM Regulation 200-2, CENTCOM Contingency  Environmental Standards, requires steps to be taken to sample or monitor  burn pit emissions. However, DOD officials stated that there were no  processes in place to specifically monitor burn pit emissions for the  purposes of correlating potential exposures. They attributed this to a lack  of singular exposure to the burn pit emissions, or emissions from any  other individual item; instead, monitoring was done for the totality of air  pollutants from all sources at the point of population exposure. As we  reported in September 2016, given the potential use of burn pits near  installations and their potential use in future contingency operations,  establishing processes to monitor burn pit emissions for unacceptable  exposures would better position DOD and combatant commanders to  collect data that could help assess exposure to risks.

In the absence of the collection of data to examine the effects of burn pit  exposure on servicemembers, the Department of Veterans Affairs in 2014  created the airborne hazards and open-air burn pit registry, which  allows eligible individuals to self-report exposures to airborne hazards  (such as smoke from burn pits, oil-well fires, or pollution during  deployment), as well as other exposures and health concerns. The  registry helps to monitor health conditions affecting veterans and  servicemembers, and to collect data that would assist in improving  programs to help those with deployment exposure concerns.

With respect to the information on the health effects from exposure to  burn pit emissions, DOD officials stated that there were short-term effects  from being exposed to toxins from the burning of waste, such as eye  irritation and burning, coughing and throat irritation, breathing difficulties,  and skin itching and rashes. However, the officials also stated that DOD  did not have enough data to confirm whether direct exposure to burn pits  caused long-term health issues. Although DOD and the Department of  Veterans Affairs had commissioned studies to enhance their  understanding of airborne hazards, including burn pit emissions, the then- current lack of data on emissions specific to burn pits limited DOD’s ability  to fully assess potential health impacts on servicemembers and other  base personnel, such as contractors.

For example, in a 2011 study that was contracted by the Department of  Veterans Affairs, the Institute of Medicine stated that it was unable to  determine whether long-term health effects are likely to result from burn  pit exposure due to inadequate evidence of an association. While the  study did not determine a linkage to long-term health effects, because of  the lack of data, it did not discredit the relationship either. Rather, it  outlined a methodology of how to collect the necessary data to determine  the effects of the exposure. Specifically, the 2011 study outlined the  feasibility and design issues for an epidemiologic study—that is, a study  of the distribution and determinants of diseases and injuries in human  populations—of veterans exposed to burn pit emissions. Further, the  2011 study reported that there were a variety of methods for collecting  exposure information, but the most desirable was to measure exposures  quantitatively at the individual level. Individual exposure measurements  could be obtained through personal monitoring data or biomonitoring.  However, if individual monitoring data were not available, and they rarely  are, individual exposure data might also be estimated from modeling of  exposures, self-reported surveys, interviews, job exposure matrixes, and  environmental monitoring. Further, to determine the incidence of chronic  disease, the study stated that servicemembers must be tracked from their  time of deployment, over many years.

While the Institute of Medicine outlined a methodology of how to conduct  an epidemiologic study, DOD had not taken steps to conduct this type of  research study, specifically one that focused on the direct, individual  exposure to burn pit emissions and the possible long-term health effects  of such exposure. Instead, some officials commented that there were no  long-term health effects linked to the exposures of burn pits because the  2011 study did not acknowledge any. Conversely, Veterans Affairs  officials stated that a study aimed at establishing health effect linkages  could be enabled by the data in its airborne hazards and open-air burn pit  registry, which collects self-reported information on servicemembers’  deployment location and exposure.

In response to a mandate contained in section 201 of Public Law 112- 260, the Department of Veterans Affairs entered into an agreement with  the National Academies of Sciences, Engineering, and Medicine to  convene a committee to provide recommendations on collecting,  maintaining, and monitoring information through the registry. The  committee assessed the effectiveness of the Department of Veterans  Affairs’ information gathering efforts and provided recommendations for  addressing the future medical needs of the affected groups. The study  was conducted in two phases. Phase 1 was a review of the data  collection methods and outcomes, as well as an analysis of the self- reported veteran experience data gathered in the registry. Phase 2 was  focused on the assessment of the effectiveness of the actions taken by  the Department of Veterans Affairs and DOD and provided  recommendations for improving the methods enacted. The committee  released its final report in February 2017. As we reported in September  2016, considering the results of this review as well as the methodology of  the 2011 Institute of Medicine study as part of an examination of the  relationship between direct, individual exposure to burn pit emissions and  long-term health effects could better position DOD to fully assess those  health risks.

In our September 2016 report we recommended that the Secretary of  Defense direct the Under Secretary of Defense for Acquisition,  Technology, and Logistics to:  take steps to ensure CENTCOM and other geographic combatant  commands, as appropriate, establish processes to consistently  monitor burn pit emissions for unacceptable exposures; and  in coordination with the Secretary of Veterans Affairs, specifically  examine the relationship between direct, individual, burn pit exposure  and potential long-term health-related issues. As part of that  examination, consider the results of the National Academies of  Sciences, Engineering, and Medicine’s report on the Department of  Veteran Affairs registry and the methodology outlined in the 2011  Institute of Medicine study that suggests the need to evaluate the  health status of service members from their time of deployment over  many years to determine their incidence of chronic disease, with  particular attention to the collection of data at the individual level,  including the means by which that data is obtained.

DOD concurred with the first recommendation, stating that the  department will ensure that geographic combatant commands establish  and employ processes to consistently monitor burn pit emissions for  unacceptable exposures at the point of exposure and if necessary at  individual sources. In a May 2018 status update regarding this  recommendation, DOD stated that it will be updating applicable  department policy and procedures, its tactics techniques and procedures  manual, and guidance for sampling and analysis plans to improve  monitoring of burn pit emissions and other airborne hazard emissions.  Specifically, DOD stated it will update DOD Instruction 6490.03,  Deployment Health; that the update will provide revised procedures on  deployment health activities required before, during, and after  deployments, including Occupational and Environmental Health Site  Assessments; and that it estimates this will be completed by the 4th  quarter of fiscal year 2018. In addition, the department stated it will  update its Occupational and Environmental Health Site Assessments  tactics, techniques, and procedures manual and update guidance for  sampling and analysis plans and that the updates will provide revised  tactics, techniques, and procedures that will improve the quality of health  risk assessment. The department expects this to be completed by the 1st  quarter of fiscal year 2019. GAO believes that upon completion of these  actions, DOD will have met the intent of this recommendation.

With respect to our recommendation to sponsor research, in coordination  with the Secretary of Veterans Affairs, to specifically examine the  relationship between burn pit exposure and potential health-related  issues, DOD partially concurred, stating that a considerable volume of  research studies had already been completed, were ongoing, or were  planned in collaboration with the Department of Veterans Affairs and  other research entities to improve the understanding of burn pit and other  ambient exposures to potential long-term health outcomes and that the  studies, where applicable, consider and incorporate the methodology  outlined in the 2011 Institute of Medicine study. In a May 2018 status  update regarding this recommendation, the department stated that DOD  and the Department of Veterans Affairs continue to collaborate with each  other and other entities on research activities that address burn pit and  other airborne exposures, and potential long-term health outcomes.  Specifically, the department cited a DOD/Veterans Affairs Airborne  Hazards Symposium held in May 2017; an update to the Veterans  Affairs/DOD Deployment Health Working Group "Airborne Hazards Joint  Action Plan" to be completed by the 3rd quarter of fiscal year 2018; and  the completion of research to examine airborne hazard exposures and  potential health-related issues. GAO believes that to the extent that  continued studies consider and incorporate the methodology outlined the  2011 Institute of Medicine study, where appropriate, DOD will have met  the intent of this recommendation.

Chairman Dunn, Ranking Member Brownley, and Members of the  Subcommittee, this concludes my statement for the record.

If you or your staff have any questions about this statement, please  contact Cary Russell, Director, Defense Capabilities and Management, at  202-512-5431 or russellc@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this statement. GAO staff who made key contributions to this statement  include Guy LoFaro (Assistant Director), Lorraine Ettaro, Shahrzad Nikoo,  Jennifer Spence, and Matthew Young.

This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.