VA Health Care: Improvements Needed in Suicide Prevention Media Outreach Campaign Oversight and Evaluation

Veterans suffer a disproportionately higher rate of suicide than the civilian population. VA has estimated that an average of 20 veterans die by suicide per day, and in 2018, VA identified suicide prevention as its highest clinical priority. VHA's suicide prevention media outreach campaign—its collective suicide prevention outreach activities—helps raise awareness among veterans and others in the community about suicide prevention resources. VHA has contracted with an outside vendor to develop suicide prevention media outreach content.

GAO was asked to examine VHA's suicide prevention media outreach campaign. This report examines the extent to which VHA (1) conducts activities for its suicide prevention media outreach campaign, and (2) evaluates the effectiveness of its campaign. GAO reviewed relevant VHA documents and data on the amount, type, and cost of suicide prevention outreach activities since fiscal year 2013. GAO also reviewed VHA's contract for developing suicide prevention outreach content and interviewed VA and VHA officials.

The Department of Veterans Affairs' (VA) Veterans Health Administration (VHA) conducts national suicide prevention media outreach on various platforms to raise awareness about VHA's suicide prevention resources. The primary focus of this campaign since 2010 has been to raise awareness of the Veterans Crisis Line (VCL), VHA's national hotline established in 2007 to provide support to veterans in emotional crisis. GAO found that VHA's suicide prevention media outreach activities declined in recent years due to leadership turnover and reorganization. For example, the amount of suicide prevention content developed by VHA's contractor for social media decreased in fiscal years 2017 and the first 10 months of 2018 after increasing in each of the 4 prior years.

VHA officials reported not having leadership available for a period of time to make decisions about the suicide prevention media outreach campaign. GAO found that VHA did not assign key leadership responsibilities or establish clear lines of reporting, and as a result, its ability to oversee the outreach campaign was hindered. Consequently, VHA may not be maximizing its reach with suicide prevention media content to veterans, especially those who are at-risk.

VHA evaluates the effectiveness of its suicide prevention media outreach campaign by collecting data on metrics, such as the number of people that visit the VCL website. However, VHA has not established targets for the majority of these metrics. Officials said they have not established targets because, apart from one industry-wide target they use, they lack meaningful targets for evaluating the campaign. However, VHA could use information about how its metrics performed in the past to develop reasonable and meaningful targets for future performance. Without established targets for its metrics, VHA is missing an opportunity to better evaluate the effectiveness of its suicide prevention media outreach campaign.

VHA should (1) establish an approach to oversee its suicide prevention media outreach campaign that includes clear delineation of roles and responsibilities, and (2) establish targets for its metrics to improve evaluation efforts. VA concurred with GAO's recommendations and described steps it will take to implement them.

Suicide is a public health problem facing all populations, particularly  veterans. After military service, veterans suffer a disproportionately higher  rate of suicide than the civilian population. According to the Department of  Veterans Affairs (VA), veterans accounted for 14.3 percent of all deaths  by suicide among United States adults in 2015, despite constituting only  8.3 percent of the adult population. VA reported in June 2018 that an  average of 20 veterans died by suicide each day in 2015, and that an  average of 6 of these 20 veterans (30 percent) were recent users of VA’s  Veterans Health Administration’s (VHA) health care services. Suicide is  not unique to veterans; the Centers for Disease Control and Prevention  (CDC) reports that suicide was the tenth leading cause of death in the  United States in 2015. However, according to VA, the rate of suicide is  higher for veterans than non-veterans (see figure 1).

VHA conducts national outreach to veterans and others in the community  to raise awareness about suicide prevention and services available  through VA, as one of several VHA initiatives to help reduce suicides in  the veteran community. Since 2010, VHA has awarded multiple contracts  to the same vendor to develop content for suicide prevention media  outreach. Questions have been raised about the effectiveness of VHA’s  collective suicide prevention media outreach activities, also referred to as  its suicide prevention media outreach campaign. You asked us to review  VHA’s suicide prevention media outreach activities and related oversight.  In this report, we examine:  1.  the extent to which VHA conducts activities for its suicide prevention  media outreach campaign; and  2.  the extent to which VHA evaluates the effectiveness of its suicide  prevention media outreach campaign.

To examine the extent to which VHA conducts activities for its suicide  prevention media outreach campaign, we reviewed VHA’s contract to  develop suicide prevention outreach content from fiscal years 2013  through 2016 and its contract to develop suicide prevention and mental  health outreach content in fiscal years 2017 and 2018. VHA’s first such  contract began in 2010; however, the Federal Acquisition Regulation only  requires federal agencies to maintain contract documentation for a period  of 6 years after the last payment. Therefore, we did not include the initial  contract or associated outreach in our review due to the lack of available  documentation. We also requested and reviewed data on the amount and  type of outreach content that VHA’s contractor developed from fiscal year  2013 through fiscal year 2018. We interviewed VHA officials responsible  for implementation and oversight of the contract about the data. We also  obtained written responses from the contractor about the steps it took to  compile and ensure the accuracy of the data. On the basis of these  actions, we found these data to be sufficiently reliable for the purpose of  our reporting objective.

We also reviewed documentation and interviewed VHA officials who  worked on the campaign about VHA’s suicide prevention media outreach  campaign, outreach activities, and VHA leadership officials’ involvement  in the campaign, and plans moving forward. We compared VHA’s  outreach campaign and related activities with federal internal control  standards related to control environment and monitoring. We also  interviewed relevant other VHA and VA officials who played a role in the  suicide prevention media outreach campaign, including officials in VA’s  Office of Acquisitions, Logistics, and Construction and VHA’s Office of  Communications.

To examine the extent to which VHA evaluates the effectiveness of its  suicide prevention media outreach campaign, we reviewed  documentation of VHA’s evaluation efforts, including monthly monitoring  reports, to identify any performance metrics VHA monitors and any  targets established for those metrics. We also reviewed the 2016 contract  to identify any requirements for the contractor to monitor and report on  the effectiveness of the campaign. Additionally, we interviewed VHA  officials responsible for monitoring and evaluating the campaign and  reviewed supporting documentation. Specifically, we asked officials about  the extent to which they had evaluated the campaign’s effectiveness  since fiscal year 2013, their evaluation processes and tools, and any  plans for completing future evaluations. We then compared VHA’s  evaluation efforts to criteria we developed in previous bodies of work for  designing program evaluations, as well as to federal internal control  standards for monitoring.

We conducted this performance audit from December 2017 to November  2018 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

VA has undertaken a number of initiatives to help prevent veteran suicide,  including identifying suicide prevention as VA’s highest clinical priority in  its strategic plan for fiscal years 2018 through 2024 (see fig. 2).

VA uses CDC’s research on risk factors and prevention techniques to  inform its approach to suicide prevention in the veteran community. There  is no single determining cause for suicide; instead, suicide occurs in  response to biological, psychological, interpersonal, environmental, and  societal influences, according to the CDC. Specifically, suicide is  associated with risk factors that exist at the individual level (such as a  history of mental illness or substance abuse, or stressful life events, such  as divorce or the death of a loved one), community level (such as barriers  to health care), or societal level (such as the way suicide is portrayed in  the media and stigma associated with seeking help for mental illness).  According to VA, veterans may possess risk factors related to their  military service, such as a service-related injury or a difficult transition to  civilian life. CDC reports that protective factors—influences that help  protect against the risk for suicide—include effective coping and problem- solving skills, strong and supportive relationships with friends and family,  availability of health care, and connectedness to social institutions such  as school and community.

VA’s 2018 National Strategy for Suicide Prevention identifies four focus  areas: (1) healthy and empowered veterans, families, and communities;  (2) clinical and community preventative services; (3) treatment and  support services; and (4) surveillance, research, and evaluation.  Collectively, these four areas encompass 14 goals for preventing veteran  suicide, one of which is implementing communication designed to prevent  veteran suicide by changing knowledge, attitude, and behaviors.

VHA’s suicide prevention media outreach campaign is just one of its  initiatives intended to reduce veteran suicide. For example, in 2007, VHA  established the Veteran’s Crisis Line (VCL), a national toll-free hotline  that supports veterans in emotional crisis. Veterans, as well as their  family and friends, can access the VCL by calling a national toll-free  number—1-800-273-8255—and pressing “1” to be connected with a VCL  responder, regardless of whether these veterans receive health care  through VHA. VHA added the option to communicate with VCL  responders via online chat in 2009, followed by text messaging in 2011.  Another VHA suicide prevention initiative is the Recovery Engagement and Coordination for Health – Veterans Enhanced Treatment initiative, or  REACH VET. Established in 2016, REACH VET uses predictive modeling  to analyze existing data from veterans’ health records to identify veterans  at increased risk for adverse outcomes, such as suicide, hospitalization,  or illness.

VHA’s Suicide Prevention Media Outreach Campaign

Suicide prevention officials within VHA’s Office of Mental Health and  Suicide Prevention (OMHSP) are responsible for implementing the  suicide prevention media outreach campaign. Since 2010, VHA has used  a contractor to develop suicide prevention media outreach content and  monitor its effectiveness. In September 2016, VHA awarded a new  contract to the same contractor to provide both suicide prevention and  mental health media outreach. Under the 2016 contract, the suicide  prevention and mental health outreach campaigns remain separate and  are overseen by separate suicide prevention and mental health officials,  both within OMHSP. VHA officials told us that beginning in fiscal year  2019, VHA will separate the contract for suicide prevention and mental  health media outreach. Specifically, VHA will utilize an existing agreement  with a different contractor for suicide prevention media outreach while the  existing contractor will continue to provide mental health media  outreach.

According to VHA, the purpose of its suicide prevention media outreach  campaign is to raise awareness among veterans, their families and  friends, and the general public about VHA resources that are available to  veterans who may be at risk for suicide. The primary focus of the  outreach campaign since 2010 has been to raise awareness of the  services available through the VCL.

VHA’s suicide prevention media outreach falls into two main categories:  unpaid and paid. Unpaid media outreach content is typically displayed on  platforms owned by VA or VHA, or is disseminated by external  organizations or individuals that share VHA suicide prevention content at  no cost, as discussed below (see fig. 3).

Social media. VA and VHA each maintain national social media  accounts on platforms such as Facebook, Twitter, and Instagram,  and post content, including suicide prevention content developed  by VHA’s contractor. VHA also works with other federal  agencies, non-governmental organizations, and individuals that  post its suicide prevention content periodically.

Public service announcements (PSA). VHA’s contractor  typically develops two PSAs per year, which various local and  national media networks display at no cost to VHA.

Website. VHA’s contractor maintains the content displayed on the  VCL website (veteranscrisisline.net), including much of the  content it develops for other platforms, such as PSAs and social  media content. Visitors to the website can both view the content  on the website and share it on their own platforms.

Paid digital media. An example of paid digital media includes  online keyword searches, in which VHA pays a search engine a  fee for its website to appear as a top result in response to selected  keywords, such as “veterans crisis line” or “veteran suicide.” Paid  digital media also includes social media posts for which VHA pays  a fee to display its content to a widespread audience, such as  users with a military affiliation.

Paid “out-of-home” media: “Out-of-home” refers to the locations  where this type of content is typically displayed. Examples include  billboards, bus and transit advertisements, and local and national  radio commercials.

VHA recognizes September as Suicide Prevention Month each year.  During this month, VHA establishes a theme and increases its outreach  activities, including a combination of both paid and unpaid media  outreach. According to VHA, it typically incorporates additional outreach  techniques during this month, such as enlisting the support of celebrities  or hosting live chat sessions on social media platforms, including  Facebook and Twitter.

VHA’s Suicide Prevention Media Outreach Activities Declined in Recent Years Due to Leadership Turnover and Reorganization VHA’s Suicide Prevention Media Outreach Activities Declined in Fiscal Years 2017 and 2018

VHA’s suicide prevention media outreach activities declined in fiscal  years 2017 and 2018 compared to earlier years of the campaign. We  identified declines in social media postings, PSAs, paid media, and  suicide prevention month activities, as discussed below.

Social media. The amount of social media content developed by  VHA’s contractor decreased in 2017 and 2018, after increasing in  each of the prior four years. Specifically, VHA reported that its  contractor developed 339 pieces of social media content in fiscal  year 2016, compared with 159 in fiscal year 2017, and 47 during  the first 10 months of fiscal year 2018 (see fig. 5.).

PSAs. VHA’s contractor is required to develop two suicide  prevention PSAs in each fiscal year. VHA officials said that the  development of the two PSAs was delayed in fiscal year 2018.  Specifically, as of August 2018, VHA reported that one PSA was  completed, but had not yet aired, and another PSA was in  development. As a result of this delay, VHA had not aired a  suicide prevention PSA on television or radio in over a year; this is  the first time there has been a gap of more than a month since  June 2012.

Paid media. VHA had a total budget of $17.7 million for its suicide  prevention and mental health media outreach for fiscal year 2018,  of which $6.2 million was obligated for suicide prevention paid  media. As of September 2018, VHA said it had spent $57,000 of  its $6.2 million paid media budget. VHA officials estimated that  they would spend a total of $1.5 million on suicide prevention paid  media for fiscal year 2018 and indicated that the remaining funds  would be de-obligated from the contract at the end of the fiscal  year and not used for suicide prevention media outreach. VHA  officials indicated that the reason they did not spend the remaining  funds on suicide prevention paid media in fiscal year 2018 was  that the approval of the paid media plan was delayed due to  changes in leadership and organizational realignment of the  suicide prevention program. As a result, VHA officials said they  limited the paid media outreach in fiscal year 2018 to activities that  were already in place, including 25 keyword search  advertisements, and 20 billboards and 8 radio advertisements in  selected cities across the United States.

In prior fiscal years, VHA conducted a variety of digital and out-of- home suicide prevention paid media. For example, in fiscal year  2015, with a suicide prevention paid media budget of more than  $4 million, VHA reported that it ran 58 advertisements on Google,  Bing, and Facebook, and ran 30 billboards, 180 bus  advertisements, more than 19,000 radio advertisements, 252 print  advertisements, and 39 movie theatre placements in selected  cities across the United States. VHA ran similar types of paid  media in fiscal years 2013, 2014, and 2016 with variation in  quantities based on the approved budget in each of these years.  In fiscal year 2017, VHA had a budget of approximately $1.7  million to spend on paid media for both the suicide prevention and  mental health outreach campaigns. However, VHA spent less than  10 percent of the funds (approximately $136,000) to run paid  advertisements on Google and Bing for suicide prevention in fiscal  year 2017; the remainder was spent on mental health outreach.

Suicide Prevention Month. VHA documentation indicated that  Suicide Prevention Month 2017 was a limited effort. VHA officials  said that this was because they did not begin preparing early  enough. In May 2018, VHA officials indicated that they were  similarly behind schedule for planning Suicide Prevention Month  2018, though they told us in August 2018 that they had caught up.

VHA Leadership Turnover and Reorganization Resulted in the Decline in Suicide Prevention Media Outreach Activities

VHA officials told us that the decrease in suicide prevention media  outreach activities was due to leadership turnover and reorganization  since 2017. For example, VHA officials said the National Director for  Suicide Prevention position was vacant from July 2017 through April  2018. VHA filled the role temporarily with a 6-month detail from another  agency from October 2017 through March 2018 and then hired this  individual as the permanent director on April 30, 2018. VHA officials that  worked on the campaign told us they did not have leadership available to  make decisions about the suicide prevention campaign during this time.  For example, VHA officials said they did not have a kick-off meeting  between VHA leadership and VHA’s contractor at the beginning of fiscal  year 2018—a requirement of the contract—because there was no  leadership available to participate in this meeting.

The officials also reported that suicide prevention leadership was not  available for weekly meetings to discuss suicide prevention outreach  activities, even after the suicide prevention program obtained an acting  director on detail from another agency. VHA staff said that at that time,  they focused their suicide prevention media outreach efforts on areas that  did not require leadership input, such as updating the VCL website. The  absence of leadership available to provide direction and make decisions  on the suicide prevention media outreach campaign is inconsistent with  federal internal control standards for control environment, which require  agencies to assign responsibilities to achieve its objectives. If a key role  is vacant, management needs to determine by whom and how those  responsibilities will be fulfilled in order to meet its objectives.

Officials that worked on the campaign told us they shifted their focus  away from the suicide prevention media outreach campaign toward the  mental health outreach campaign due to reorganization of the offices  responsible for suicide prevention activities in 2017. Specifically, under  the new organization, and in the absence of suicide prevention program  leadership, the officials began reporting directly to mental health program  leadership and became more focused on the mental health outreach  aspects of the contract. Following the reorganization, officials that  worked on the campaign did not have a clear line of reporting to the  suicide prevention program. This is also inconsistent with federal internal  control standards for control environment, which require agencies to  establish an organizational structure and assign responsibilities, such as  establishing lines of reporting necessary information to management.

VHA officials told us that one of the highest priorities for the suicide  prevention program since the beginning of fiscal year 2018 was to  establish a national strategy for preventing veteran suicides. The national  strategy, issued in June 2018, includes suicide prevention outreach as  one of the strategy’s 14 goals. The national strategy also emphasizes  VHA’s plans to shift to a public health approach to suicide prevention  outreach. The public health approach focuses less on raising awareness  of the VCL and more on reaching veterans before the point of crisis.  VHA officials told us they have been trying to shift to a public health  approach since 2016. Some of the campaign themes and messages have  reflected this shift; for example, the “Be There” campaign theme that was  adopted in fiscal year 2016—and has remained the theme since— emphasizes the message that everyone has a role in helping veterans in  crisis feel less alone and connecting them to resources. However, VHA  officials told us in May 2018 that they were just beginning to  conceptualize what the suicide prevention outreach campaign should look  like moving forward. Leadership officials also said that while they were  developing the national strategy, they delegated the responsibility for  implementing the suicide prevention outreach campaign to other officials  working on the campaign.

The decline in VHA’s suicide prevention media outreach activities over  the past 2 fiscal years is inconsistent with VA’s strategic goals, which  identify suicide prevention as the agency’s top clinical priority for fiscal  years 2018 through 2024. Further, VHA has continued to obligate  millions of dollars to its suicide prevention media outreach campaign each  year. Since fiscal year 2017, VHA has obligated $24.6 million to the  contract for media outreach related to both suicide prevention and mental  health.

By not assigning key leadership responsibilities and clear lines of  reporting, VHA’s ability to oversee the suicide prevention media outreach  activities was hindered and these outreach activities decreased. As a  result, VHA may not have exposed as many people in the community,  such as veterans at risk for suicide, or their families and friends, to its  suicide prevention outreach content. Additionally, without establishing  responsibility and clear lines of reporting, VHA lacks assurance that it will  have continuous oversight of its suicide prevention media outreach  activities in the event of additional turnover and reorganization in the  future, particularly as VHA begins implementing the suicide prevention  media outreach campaign under its new agreement that begins in fiscal  year 2019.

VHA Monitors Metrics for Its Suicide Prevention Media Outreach Campaign, but Has Not Established Targets against Which to Evaluate the Campaign’s Effectiveness VHA Monitors Metrics for Its Suicide Prevention Media Outreach Campaign

VHA works with its contractor to create and monitor metrics to help gauge  the effectiveness of its suicide prevention media outreach campaign in  raising awareness among veterans and others about VHA services, such  as the VCL. The metrics primarily focus on the number of individuals who  were exposed to or interacted with VHA’s suicide prevention content  across various forms of outreach, including social media, PSAs, and  websites. According to VHA, the metrics are intended to help VHA  ensure that its media outreach activities achieve intended results, such as  increasing awareness and use of the resources identified on the VCL  website. Examples of metrics monitored by VHA and its contractor  include those related to (1) social media, such as the number of times a  piece of outreach content is displayed on social media; (2) PSAs, such as  the total number of markets and television stations airing a PSA; and (3)  the VCL website, such as the total traffic to the website, as well as the  average amount of time spent on a page and average number of pages  viewed per visit.

VHA’s contractor is required to monitor the metrics and report results on a  monthly basis. Specifically, the contractor provides monthly monitoring  reports to VHA that summarize how outreach is performing, such as the  number of visits to the VCL website that were driven from paid media  sources. Officials noted these reports are key sources of information for  VHA on the results of its outreach. VHA officials also told us they  informally discuss certain metrics during weekly meetings with VHA’s  contractor. In addition, VHA works with its contractor to conduct a more  in-depth analysis of outreach efforts during and after Suicide Prevention  Month each year.

VHA Lacks Metric Targets to Evaluate the Effectiveness of Its Suicide Prevention Media Outreach Campaign

VHA has not established targets for the majority of the metrics it uses to  help gauge the effectiveness of its suicide prevention media outreach  campaign. As a result, VHA does not have the information it needs to fully  evaluate the campaign’s effectiveness in raising awareness of VHA’s  suicide prevention resources among veterans, including the VCL. For  example, we found that VHA’s contractor’s monitoring reports—a  summary of key metrics that VHA uses to routinely monitor information  regarding the campaign—generally focused on outreach “highlights” and  positive results. The reports did not set expectations based on past  outreach or targets for new outreach, and lacked more comprehensive  information on whether outreach performed against these expectations.  For example:

A monitoring report from 2018 showed that during one month,  there were 21,000 social media mentions of keywords specific to  VA suicide prevention, such as “VCL” or “veteran suicide,” across  social media platforms. These mentions earned 120 million  impressions; however, there was no indication of the number of  keyword mentions or impressions that VHA expected based on its  media outreach activities. In addition, the report did not indicate  the proportion of mentions that VHA believed were specifically  driven by its outreach activities, and there also was no indication  of whether these mentions were positive or negative, or what  actions to take based on this information.

Another monitoring report from January 2017 showed that paid  advertising drove 39 percent of overall website traffic during one  month, while unpaid sources drove the remaining 61 percent.  However, there was no information indicating the amounts of paid  advertising that VHA conducted during this monitoring period, and  whether this amount of website traffic from paid advertising met  expectations.

VHA’s 2016 Suicide Prevention Month summary report showed  that there were 194,536 visits to the VCL website, roughly an 8  percent increase from the Suicide Prevention Month in 2015.  However, the report did not indicate whether this increase from  the prior year met expectations, or a different result was expected.

VHA officials told us that they have not established targets for most of the  suicide prevention media outreach campaign because they lack  meaningful targets for the metrics to help evaluate the campaign. VHA  officials said that the only target they have established is for each PSA to  rank in the top 10 percent of the Nielsen ratings because this is the only  meaningful target available that is accepted industry-wide. VHA officials  stated that using any other targets would be arbitrary. For the remaining  metrics, VHA officials told us they assess the outcomes of their campaign  by comparing data from year to year, and examining any changes in the  outcomes over time. However, VHA could set targets that capture the  number of people who viewed or interacted with its outreach content,  similar to its Nielsen target set for television viewership. Doing so would  help VHA evaluate whether the campaign has been effective in raising  awareness of VHA’s suicide prevention resources. Further, creating  targets for these additional metrics need not be arbitrary, because VHA  could use information about how its metrics performed in the past to  develop reasonable and meaningful targets for future performance. VHA  could also adjust the targets over time to reflect changes in its metrics or  approach to the campaign, such as changes to its paid media budget  each year.

Federal internal control standards for monitoring require agencies to  assess the quality of its performance by evaluating the results of  activities. Agencies can then use these evaluations to determine the  effectiveness of its programs or need for any corrective actions. Further,  VA’s June 2018 National Strategy for Preventing Veteran Suicide also  emphasizes the importance of the agency evaluating the effectiveness of  its outreach. The absence of established targets leaves VHA without a  framework to effectively evaluate its campaign. Our prior work has shown  that establishing targets allows agencies to track their progress toward  specific goals. In particular, we have developed several key attributes of  performance goals and measures including, when appropriate, the  development of quantifiable, numerical targets for performance goals and  measures. Such targets can facilitate future evaluations of whether  overall goals and objectives were achieved by allowing for comparisons  between projected performance and actual results. Further, establishing  targets for its outreach metrics will enable VHA officials to determine  whether outreach performed as expected and raised awareness of VHA  resources such as the VCL, including identifying outreach efforts that  worked particularly well and those that did not. In doing so, VHA officials  will have the opportunity to make better informed decisions in their suicide  prevention media outreach campaign to support VA’s overall goal of  reducing veteran suicides.

VA has stated that preventing veteran suicide is its top clinical priority; yet  VHA’s lack of leadership attention to its suicide prevention media  outreach campaign in recent years has resulted in less outreach to  veterans. While VHA identifies the campaign as its primary method of  raising suicide prevention awareness, it has not established an effective  oversight approach to ensure outreach continuity. This became  particularly evident during a recent period of turnover and reorganization  in the office responsible for the suicide prevention outreach campaign.  Moving forward, VHA has an opportunity to improve its oversight to  ensure that its outreach content reaches veterans and others in the  community to raise awareness of VHA’s suicide prevention services,  particularly as VHA begins working with a new contractor beginning in  fiscal year 2019.

VHA is responsible for evaluating the effectiveness of its suicide  prevention media outreach campaign in raising awareness about VHA  services that are available to veterans who may be at risk for suicide. To  do so, VHA collects and monitors data on campaign metrics to help  gauge the effectiveness of its suicide prevention media outreach  campaign in raising such awareness, but has not established targets for  the majority of these metrics because officials reported that there are no  meaningful, industry-wide targets for them. We disagree with VHA’s  assertion that other targets would not be meaningful; VHA collects data  on its metrics that it can use to develop reasonable and meaningful  targets for future performance. In the absence of established targets,  VHA cannot evaluate the effectiveness of the campaign, and make  informed decisions about which activities should be continued to support  VA’s overall goal of reducing veteran suicides.

We are making the following two recommendations to VA:  1.  The Under Secretary for Health should establish an approach for  overseeing its suicide prevention media outreach efforts that includes  clear delineation of roles and responsibilities for those in leadership  and contract oversight roles, including during periods of staff turnover  or program changes. (Recommendation 1)  2.  The Under Secretary for Health should require officials within the  Office of Suicide Prevention and Mental Health to establish targets for  the metrics the office uses to evaluate the effectiveness of its suicide  prevention media outreach campaign. (Recommendation 2)

We provided a draft of this report to VA for review and comment. In its  written comments, summarized below and reprinted in Appendix I, VA  concurred with our recommendations. VA described ongoing and planned  actions and provided a timeline for addressing our recommendations. VA  also provided technical comments, which we incorporated as appropriate.

In response to our first recommendation, to establish an oversight  approach that includes delineation of roles and responsibilities, VA  acknowledged that organizational transitions and realignments within  OMHSP contributed to unclear roles and responsibilities in 2017 and  2018. VA said that OMHSP has made organizational improvements,  including hiring a permanent Director for Suicide Prevention and  establishing a new organizational structure. In its comments, VA  requested closure of the first recommendation based on these actions.

However, to fully implement this recommendation, VA will need to provide  evidence that it has established an oversight approach for the suicide  prevention media outreach campaign. This would include providing  information about the roles and responsibilities, as well as reporting  requirements, for contract and leadership officials involved in the suicide  prevention media outreach campaign under the new organizational  structure and the new contract. VA will also need to demonstrate that it  has a plan in place to ensure continued oversight of the suicide  prevention media campaign in the event of staff turnover or program  changes.

In response to our second recommendation, to establish targets against  which to evaluate suicide prevention metrics, VA said it has plans to work  with communications experts to develop metrics, targets, and an  evaluation strategy to improve its evaluation of its suicide prevention  program efforts, including outreach. VA expects to complete these actions  by April 2019.

As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the appropriate  congressional committees and the Secretary of Veterans Affairs. In  addition, the report will be available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-7114 or at DraperD@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix I.

Appendix I: Comments from the Department of Veterans Affairs

Appendix II: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Marcia A. Mann (Assistant  Director), Kaitlin McConnell (Analyst-in-Charge), Kaitlin Asaly, and Jane  Eyre made key contributions to this report. Also contributing were Jennie  Apter, Emily Bippus, Valerie Caracelli, Lisa Gardner, Jacquelyn Hamilton,  Teague Lyons, Vikki Porter, and Eden Savino.

Preventing veteran suicide is one of the Department of Veterans Affairs' highest priorities. VA reports that an average of 20 veterans die by suicide each day. Since 2010, VA has conducted national outreach to raise awareness about suicide prevention resources for veterans.

However, we found that VA's outreach activities dropped off in 2017 and 2018, and the office responsible for these activities lacked consistent leadership. We also found that VA did not have clear goals for evaluating the effectiveness of its outreach activities.

We recommended VA improve its oversight and evaluation of its suicide prevention outreach activities.


Defense Acquisitions: Information on Workforce, Organizational Structure, and Budgeting for Selected Programs

In 2018, DOD estimated that its 82 major defense acquisition programs would cost over $1.69 trillion in total to acquire. DOD relies on program offices—composed of civilian, military and contractor support personnel—to manage and oversee these technically complex programs.

GAO was asked to review factors affecting DOD's personnel needs for its acquisition programs and how DOD budgets for the costs associated with these personnel.

This report describes (1) factors affecting the workforce size, composition, and mix of contractor and government personnel, as well as organizational structure for selected programs; and (2) how personnel costs associated with those selected programs are included in DOD's budget justification documents.

GAO reviewed DOD acquisition, workforce, and financial management policies and regulations and identified a non-generalizable sample of 11 major defense acquisition programs, including programs from each military department that were recently approved to enter into system development. GAO requested information from each of these programs to identify the number and type of personnel supporting the program, reviewed program documentation, and interviewed program officials. GAO also reviewed DOD's budget justification documents for fiscal years 2018 and 2019.

The workforce size, composition, and mix, as well as the organizational structure of the 11 Department of Defense (DOD) major defense acquisition programs GAO reviewed were influenced by several interrelated factors. These factors include the government's role in developing and integrating key technologies, the availability of government personnel to provide the skills needed, and whether the program was managed as part of a portfolio of related programs or as a stand-alone program. For example, programs that assumed more responsibility for developing and integrating key technologies generally had a larger workforce, which was primarily composed of engineering and technical personnel. Program officials GAO met with stated that they generally prefer to use government personnel, but use contractor support when the number of government personnel allocated to the program is not sufficient to meet their needs, the technical skills are not available or are limited within the government, or to fulfill short-term tasks that are too brief to justify hiring government personnel. GAO also found that DOD structured the 11 programs to allow them to leverage available personnel with the necessary skills. Two programs were structured as standalone programs because they were new, high priority, and complex. The other nine programs were managed as a part of a portfolio of related programs. For example, the Air Force's F-15 program office manages a number of programs that add capabilities to the existing system.

DOD's Financial Management Regulation, which governs the formulation and presentation of DOD's budget request, gives DOD flexibility in how it submits program personnel costs. Consequently, the personnel costs for the 11 programs GAO reviewed were not separately and distinctly identified from other costs. For example, costs for civilian and military personnel are often centrally funded through appropriations categories that support many DOD activities and do not provide information on specific program personnel costs. GAO also found that costs for contractor support are often combined with other costs in individual program budget exhibits.

Each year the Department of Defense (DOD) spends billions of dollars to  develop, produce, and field its weapon systems. In 2018, DOD estimated  that its 82 major defense acquisition programs would cost over $1.69  trillion to develop and procure. These programs range from adding new  capabilities to older platforms, like the B-2 bomber, to developing new  weapon systems, such as the Columbia Class Ballistic Missile  Submarine. DOD establishes program offices to manage and oversee the  development, production, and sustainment of these technically complex  acquisition programs. To support its programs, DOD relies on a workforce  consisting of military and civilian personnel, such as program managers,  contracting officers, and engineers. These government personnel are also  supported by contractor personnel and other organizations.

You asked us to review issues related to how military departments meet  program personnel needs and how DOD budgets for the costs associated  with these personnel. This report describes (1) factors affecting the  workforce size, composition, and mix, as well as organizational structure  for selected major defense acquisition programs and (2) how personnel  costs associated with those selected programs are included in DOD’s  budget justification documents. For the purposes of this report, we refer to  the group of government and contractor support personnel dedicated to  and responsible for managing and supporting system acquisitions as a  “program,” rather than a program office.

To address both objectives, we used data collected as part of our annual  assessment of DOD weapon programs to identify major defense  acquisition programs that were at similar phases in DOD’s acquisition  process. DOD Instruction 5000.02, the overarching policy governing the  operation of the defense acquisition system, requires that programs  entering the system development phase of the acquisition process  provide an estimate, among other things, of their personnel needs. We  selected 11 programs, representing each military department, that were  most recently approved to enter system development as of March 2018.  The 11 programs we selected follow:

F-15 Eagle Passive Active Warning and Survivability System  (EPAWSS)

B-2 Defensive Management System Modernization (DMS-M)

Military Global Positioning System User Equipment (MGUE)

Next Generation Jammer Mid-Band (NGJ Mid-Band)

Joint Precision Approach and Landing System (JPALS)

John Lewis Class Fleet Replenishment Oiler (T-AO)

The results from these 11 case studies are not generalizable to all 82  major defense acquisition programs, but we used them to better  understand and illustrate the factors that influence a program’s size,  composition, and mix, as well as organizational structure.

We used a standardized data collection instrument to obtain program  personnel data on three workforce characteristics for each of the 11  programs:

Size—the overall number of full-time equivalent personnel, including  military, civilian, contractor support, and Federally Funded Research  and Development Center (FFRDC) personnel supporting the  program.

Composition—the acquisition functions performed by personnel  supporting the program and the resulting skill mix. We grouped these  functions into four categories: (1) program management; (2)  engineering and technical; (3) logistics; and (4) contracting, business,  and support.

Mix—the type of personnel supporting the program, including military,  civilian, contractor and FFRDC personnel. For the purposes of this  report, “contractor” refers to the support contractors who provide  technical and administrative services to DOD’s major defense  acquisition programs rather than prime contractors that develop and  produce those weapon systems or products.

For additional information on these programs, see GAO-18-360SP.

DOD officials stated that program personnel often change over time;  therefore, the associated data we collected are not static. The data  reported to us by the 11 selected programs provided a snapshot—as  reported between December 2017 and June 2018—of the personnel  supporting the program. To assess the consistency and reliability of these  data, we interviewed officials from each of the 11 programs to determine  how the data were derived and compared the data with available program  documentation that addressed workforce requirements, such as  acquisition strategies, cost analysis documents, and organizational  charts. Based on these steps, we determined that the data were  sufficiently reliable to identify and compare the programs’ workforce size,  composition, and mix.

To describe the organizational structure of the selected programs, we  reviewed some of the documentation required to proceed to the system  development phase of the defense acquisition process. We also reviewed  organizational charts and interviewed officials representing each of the  selected programs. To identify the factors affecting workforce size,  composition, and mix of the selected programs, we obtained and  reviewed program documentation, including acquisition strategies. We  interviewed officials from each of the selected programs as well as the  following DOD organizations that were involved in the policy and process  of manpower planning for acquisition programs:

Office of the Under Secretary of Defense for Personnel and

Office of the Assistant Secretary of the Army for Acquisition, Logistics,

Army Program Executive Office Ground Combat Systems;

Army Program Executive Office Intelligence, Electronic Warfare and

Army Program Executive Office Missiles and Space;

Air Force Space and Missile Systems Center;

Based on the discussions with program officials and review of acquisition- related documents, we identified the overarching factors that affected the  workforce size, composition, and mix, as well as the organizational  structure of the selected programs.

To identify how personnel costs for the 11 selected programs are  included in DOD’s budget justification documents, we reviewed the DOD  Financial Management Regulation that governs DOD input to the  President’s annual budget request, among other things. We reviewed  relevant DOD portions of the fiscal years 2018 and 2019 President’s  Budget and the specific budget exhibits for each of the 11 programs. We  also interviewed officials from Office of the Under Secretary of Defense  (Comptroller), the military departments, and Program Executive Office  (PEO), and program officials responsible for developing the budget  request for the 11 selected programs to obtain information on how military  departments budget for their personnel.

We conducted this performance audit from November 2017 to March  2019 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

DOD acquires new weapons for its warfighters through a management  process known as the defense acquisition process. This process has  multiple phases, including: (1) technology maturation and risk reduction,  (2) engineering and manufacturing development, and (3) production and  deployment. In this report we refer to these three phases as concept development, system development, and production. Programs typically  complete a series of milestone reviews and other key decision points that  authorize entry into a new acquisition phase.

DOD Instruction 5000.02 delegates responsibility for developing and  procuring weapon systems to the military departments and other defense  agencies. This policy does not specify a standard organizational  structure—or program structure—to manage acquisition programs, but  rather states that programs are to be tailored as much as possible to the  characteristics of the product being acquired, and to the totality of  circumstances associated with the program including operational urgency  and risk factors. In addition, DOD’s guidance for managing its workforce  states that the approach should be flexible, adaptive to program changes,  and responsive to new management strategies.

DOD decides how many personnel and how much program funding to  request for each military department through the Planning, Programming,  Budgeting, and Execution (PPBE) process. DOD programming policy  requires the military departments and defense agencies to develop a  program objective memorandum that identifies and prioritizes  requirements and total funding needs for the current budget year and 4  additional years into the future. As a part of this process, the  departments also estimate the personnel requirements and program  funding needed to execute their mission, including support for the  commands and PEOs that are responsible for managing acquisition  programs. The results of the PPBE process, including proposed funding  levels for programs, are captured in the President’s annual budget  request to Congress. For example, in its budget request, DOD identifies  and requests the total number of civilian full-time equivalent personnel,  among other things. Congress then authorizes and appropriates the  funding to pay for civilian personnel for each military department. When  budgeting for contracted services, DOD estimates the cost of the tasks to  be performed but not the number of individuals that may perform those  tasks. The military departments, commands and PEOs then distribute  approved funding (which, in part, is used to pay for civilian personnel and  contractor support) to the various organizations including the programs  that are responsible for managing and supporting defense acquisitions.

Each military department has a different approach to developing its  budget request, and program budgets may be spread across multiple  types of appropriations that are organized into various categories based  on their purpose such as research, development, testing and evaluation,  or procurement. Similarly, the military departments fund their personnel  through several different types of appropriations, including (1) operation  and maintenance; (2) military personnel; and (3) research, development,  test, and evaluation. Requests for funding are included in different  documents and often presented in multiple volumes that can be hundreds  of pages long. DOD’s Financial Management Regulation provides  instructions for the formulation and presentation of the budget request to  Congress, including general categories of costs that might be included in  program specific budgets. In addition, the regulation requires DOD  components to include specific budget exhibits for certain acquisition  programs to provide more insight into those programs’ funding needs.

Several Factors Affect the Workforce Size, Composition, and Mix, As Well As the Organizational Structure of Selected DOD Acquisition Programs

Several interrelated factors influenced the workforce size, composition,  and mix, as well as the organizational structure of the 11 major defense  acquisition programs we reviewed. We found the following:

Program workforce size and composition were influenced by the  degree to which the program assumed responsibility for technical  development and integration, as well as the program’s stage within  the acquisition life cycle.

Program workforce mix varied depending on the use of contractor  personnel, which was based on the workload requirements and the  availability of government personnel to provide the skills needed.

Programs were generally structured as either stand-alone—new, high  priority, complex weapon system platforms with dedicated  personnel—or as part of a portfolio of related programs to share  personnel across programs.

Factors Affecting Selected Programs’ Workforce Size and Composition

The number and composition of personnel that supported the selected  major defense acquisition programs varied considerably. As shown in  figure 1, the total number of personnel supporting the 11 selected  programs ranged from 30 to 397, and the composition of those personnel  varied based on the needs of the program.

While program officials cited a number of factors that influenced the  selected programs’ workforce size and composition, including department  priority and complexity, we identified two overarching factors—(1) the  level of program responsibility for technical development and integration,  and (2) the stage of the acquisition life cycle.

First, we found programs that assume more responsibility for technical  development and integration have more personnel—primarily those that  perform engineering as well as test and evaluation functions. The two  largest of the selected programs we reviewed, the Navy’s Next- Generation Jammer Mid-Band (NGJ Mid-Band) and Columbia Class  Ballistic Missile Submarine (Columbia), assumed significant responsibility  for system development and integration, activities a prime contractor  often undertook for the other programs we reviewed. For example,

NGJ Mid-Band officials explained that the program is responsible for  overseeing software integration and other efforts directly. In this case,  in addition to personnel assigned to the program office, the Navy  relies on personnel from other organizations such as the Naval Air  Warfare Center Aircraft Division instead of a prime contractor to  develop the software needed to operate the system, conduct system  testing, and manage integration into the platform.

Similarly, the Columbia program maintains responsibility for many  aspects of development and integration of the submarine including  most hull, mechanical, and electrical components. As a result, about  two-thirds of the 309 personnel supporting the program are  performing engineering and technical tasks.

In contrast, two programs with fewer personnel, the Air Force’s B-2  Defensive Management System Modernization program (DMS-M) and  Navy’s John Lewis Class Fleet Replenishment Oiler (T-AO), assigned  significant responsibility for development and integration to their  respective prime contractors. The Defensive Management System  Modernization program reported to us that it has a total of 11 engineering  and technical personnel, and T-AO reported that it has 35 engineering  and technical personnel.

Secondly, we found that program workforce size and composition  changed in response to the amount and nature of the work programs  perform at different stages of the acquisition life cycle. For example,  officials from our selected programs stated they generally planned to  increase in size as they progressed from concept development to system  development and also planned to concurrently increase the proportion of  engineering and technical personnel. Program officials stated that as the  program progresses into the logistics support stage, the number of  personnel supporting the program generally decreases as programs  release some personnel to other assignments while retaining enough  personnel to manage the logistics support stage. Figure 2 shows how the  size and composition of Army’s Joint-Air-to-Ground Missile (JAGM)  program changed from concept development into production.

A program’s total development and procurement cost was not necessarily  related to the number of personnel supporting the program for the 11  programs we reviewed. All 11 selected programs are classified as major  defense acquisition programs and ranged in total acquisitions cost from  $1.5 billion to $103.2 billion. Our analysis, shown in table 1 below,  indicates that total cost did not significantly influence the number of  personnel supporting these programs.

Selected Programs Used Contractor Support to Help Meet Workload Requirements

All 11 selected programs used contractors to help meet workload  requirements, but the level of contractor support varied from  approximately 5 percent to 72 percent of total program personnel, as  shown in figure 3.

Program officials told us that while they generally try to use civilian or  military personnel to meet workload requirements, they use contractor  support when the number of government personnel allocated to the  program is not sufficient to meet their needs, the technical skills are not  available or are limited within the government, or to fulfill short-term tasks  that are too brief to justify hiring government personnel.

Program officials stated the extent to which their programs use  contractor support often depends on the number civilians allocated to  the program by the command or PEO. In the case of the three  selected programs with the fewest personnel, the officials stated that  the number of personnel authorizations allocated to the program by  their respective command or PEO did not meet their estimated  workload requirements. For example, the B-2 Defensive Management  System Modernization program estimated it needed 82 personnel in  fiscal year 2018, but was only allocated 13 personnel. As a result,  program officials stated that they used program funds to pay for  contractor support personnel to partially offset the government civilian  staffing shortfalls. Officials at the Air Force Life Cycle Management  Center, the organization that allocated personnel to the B-2 program  office, told us that civilian personnel are allocated based on the risk  associated with each program.

Program officials told us that contractor support personnel are used to  augment civilian and military personnel by providing skills or technical  expertise that are limited or not available in the government. We found  that over two-thirds of the contractors that supported the 11 selected  programs we reviewed were performing engineering and technical  functions. For example, the John Lewis Class Fleet Replenishment  Oiler (T-AO) is a commercially-derived ship design. As such, program  officials stated that the required engineering expertise resides in the  commercial sector, which resulted in contracted engineers comprising  about 77 percent of the program’s total engineering personnel.

Program officials also stated that it is more effective to use contractor  support personnel to perform tasks that are relatively short in duration  than to go through the lengthy process of hiring government  personnel. Contracting for support allows the program to grow and  shrink to meet personnel requirements as they change. For example,  Joint Air-to-Ground Missile program officials stated they contracted for  support to execute tasks that are not recurring, such as developing  the required documents to get approval to start production.

Among the 11 programs we reviewed, the Air Force’s Military Global  Positioning System User Equipment (MGUE) program has a unique  workforce mix. Twenty-four percent of MGUE’s program personnel were  military, and MGUE was the only one of the 11 selected programs that  had FFRDC personnel. Program officials stated that the challenge of  obtaining civilian personnel with the required technical skills in a high  cost-of-living area around Los Angeles, California required the program to  rely more heavily on military personnel and contractors to support the  program. Program officials stated this is in part because it is easier to  assign military personnel in high cost-of-living areas than it is to hire  civilian personnel. In addition, programs in the Air Force’s Space and  Missile Systems Center often rely on FFRDC personnel from Aerospace  Corporation, which is located in the Los Angeles area and provides  technical expertise that is specific to space systems. Program officials  from the other 10 programs we reviewed reported that they did not have  FFRDC personnel.

Military Departments Structured Selected Acquisition Programs to Leverage Available Personnel with the Necessary Skills

While differences existed in the organizational structure of the 11  programs we reviewed, we identified factors that affected which of the two  common approaches the military departments used to leverage available  personnel with the necessary skills:

New, high priority, complex weapon system platforms that require a  significant amount of development and integration, such as the Navy’s  Columbia and the Army’s Armored Multi-Purpose Vehicle, are  structured as distinct standalone program offices with dedicated  program personnel.

Nine of the 11 selected programs were managed in a portfolio-based  program structure which included multiple related acquisition  programs. For these portfolio-based programs, personnel were  shared across the related programs to help meet fluctuating workload  requirements and maximize personnel resources.

Figure 4 compares the structure of a standalone program to the structure  of a portfolio-based program with multiple acquisition programs managed  under it. The figure also illustrates how the Air Force’s MGUE program  was situated within the Air Force’s Global Positioning Systems portfolio of  programs.

In both types of organizational structures illustrated above, the PEO and  the program office have personnel that oversee and support the  programs. These personnel may be dedicated to one program or may  split time between multiple portfolio-based programs. For example, the Air  Force PEO for Space has more than 5,000 military, civilian, and  contractor personnel and is responsible for managing 41 programs, the  responsibility for which is distributed among multiple program offices. One  of these program offices, the Global Positioning Systems program office,  has 628 personnel. This program office is responsible for overseeing and  supplementing the staff of several programs, including the Military Global  Positioning System User Equipment Program, which has about 70  personnel.

According to PEO and program officials, acquisition programs may be  managed within portfolios for several different reasons:

Programs are part of the same weapon system platform. The B-2  Defensive Management System Modernization program and the F-15  Eagle Passive Active Warning Survivability System program are  examples of upgrades to existing systems on mature aircraft and are  managed within a portfolio of programs within the B-2 and F-15  system program offices, respectively.

Programs have interrelated technologies. The Air Force’s MGUE  program is managed within the GPS program office, which also  manages other GPS satellite and ground system programs.

Programs have related acquisition strategies. The Navy’s John  Lewis Class Fleet Replenishment Oiler (T-AO) program is managed  within a portfolio of commercially designed and developed ships. This  program is managed within a program office that oversees  approximately 85 types of commercially derived auxiliary ships, boats,  service craft, and special mission ships.

Regardless of how the acquisition program is structured, other DOD  organizations also provide personnel to support a program’s workload  requirements. There are various specialized DOD organizations that  support programs and provide specific acquisition functions or skill sets,  such as contracting, cost estimating, and engineering. For the 11 selected  programs we reviewed, these organizations supported multiple programs  and were either structured (1) within the PEO that was responsible for the  programs we reviewed or (2) external to the PEO. These external support  organizations include contracting commands, warfare centers, and  engineering organizations that are intended to provide the program  specialized technical expertise from across the military department.  Program officials stated that these organizations may share personnel  with a program on a full or part-time basis, and the shared personnel may  or may not be co-located with the program. Figure 5 is a notional  representation of the way that programs are supported by different  organizations.

The major defense acquisition programs we reviewed used different  approaches to organizing and leveraging support organizations. For  example:

The Navy programs we reviewed relied on naval warfare centers to  provide the engineering expertise necessary to design, build,  maintain, and repair the Navy’s aircraft, ships, and submarines. For  example, the Navy’s NGJ Mid-Band relies heavily on warfare centers,  including the Naval Air Warfare Center Weapons Division and the  Naval Air Warfare Center Aircraft Division, to support the program.  We found that about 60 percent of the total number of personnel  supporting the program office were from these organizations.

The Army programs we reviewed relied on support organizations such  as the Army Contracting Command for contracting functions, the  Aviation and Missile Research Development and Engineering Center  for engineering expertise, and others to provide life cycle  management support.

The Air Force programs we reviewed relied on support organizations  established within their command. For example, Air Force’s Life Cycle  Management Center has organizations dedicated to supporting all of  its programs. These organizations provide support, such as  contracting and cost estimating expertise, to programs managed  under the Air Force’s Life Cycle Management Center. Personnel  within these organizations are not staffed to one particular program,  but share their time among many of the programs the Center is  responsible for managing.

Personnel Costs for Selected DOD Acquisition Programs Are Included in Multiple Parts of the Budget Justification Documents and Are Not Specifically Identified for Individual Programs

The personnel costs for each major defense acquisition program we  reviewed are included in different parts of the President’s annual budget  request, including budget justification documents, but are not always  clearly identifiable due to different approaches used to report such costs.  The DOD Financial Management Regulation gives the military  departments flexibility in how they submit program personnel costs. For  example, it suggests the use of “typical” personnel cost categories for  research, development, test, and evaluation programs to include in their  individual program budget exhibits, but it also allows the departments to  use the personnel cost categories they deem to be the most appropriate  when formulating the budget request. In reviewing DOD’s budget  requests for fiscal years 2018 and 2019 associated with the 11 selected  programs, we found that personnel costs are budgeted for in two main  wayscentrally by the military department, or by an individual  programdepending on whether the requests are for military, civilian, or  contractor support services. Personnel costs that are program-funded are  included in individual program budget justification requests, whereas  personnel costs that are centrally funded by the military departments are  aggregated into one or more line items in the military department’s  specific appropriation request. Table 2 shows how each military  department funds military and civilian personnel and contractor support  services for major defense acquisition programs.

Each military department centrally budgets for military personnel through  its respective Military Personnel appropriation requests, which aggregate  personnel funding. These requests include funding for pay, travel, and  other personnel-related costs. As these costs are combined and not  associated with a specific program, we could not determine the costs of  the military personnel supporting the 11 selected programs by reviewing  DOD’s budget justification documentation. In contrast, support contractor  costs were included in each program’s individual budget request.

The military departments also centrally budget for some civilian  personnel, but there are differences between the departments regarding  which appropriations categories they use to request these funds.  Regardless of the appropriation, we found that the budget requests do not  identify civilian personnel costs by specific program; therefore, we could  not determine the costs of the centrally funded civilian personnel  supporting the 11 programs we selected. For example, in fiscal year  2019, the Air Force requested funding for the civilian personnel  supporting its acquisition programs in development through the Research,  Development, Test, and Evaluation appropriation. It grouped the costs  into eight categories that represent various missions such as Cyber,  Network, and Business Systems; Global Battle Management; and Nuclear  Systems. The Air Force budget request indicates the total amount of  funds requested, but does not identify the estimated number of personnel  that these funds will support. Figure 6 illustrates how the Air Force  requested funds for its civilian acquisition workforce in fiscal year 2019.

The Navy and Army request funds for civilian personnel primarily through  their respective operation and maintenance appropriations. This  appropriation is used to fund a wide range of costs necessary to manage,  operate and maintain worldwide facilities and military operations. These  operation and maintenance budgets are divided into numerous categories  related to various missions, functions, or activities. For example, the  Navy’s Operation and Maintenance budget requests funding for civilian  personnel in several categories, such as “Ship Operational Support and  Training” and “Administration.” The Army Operation and Maintenance  budget requests funding for civilian acquisition personnel in one  combined category labeled as “Other Service Support.”

Apart from the portions of the budget described above, certain DOD  programs have specific budget exhibits that identify its funding  requirements. In reviewing the exhibits for the 11 selected programs, we  found that individual program requests include personnel costs that are  not funded centrally such as contractor support services costs, but these  costs are generally not specifically identified as personnel costs. For  example, according to program officials, the Air Force’s B-2 Defensive  Management Modernization program requested funds in its exhibit  accompanying the fiscal year 2019 Research Development, Test, and  Evaluation budget request labeled “PMA,” which stands for Program  Management Administration. According to program officials, PMA  includes costs for contractor support services, government travel, and  other costs but does not include civilian personnel costs (see figure 7).

In reviewing and discussing the budget exhibits for the 11 selected  programs with program officials, we found that personnel costs, including  civilian, contractor, and FFRDC, were generally spread across multiple  budget request lines that were associated with various tasks but were not  specifically identified as personnel costs. These include the following:

Development Test & Evaluation  For example, the Navy’s Joint Precision Approach and Landing System’s  fiscal year 2019 Research Development, Test and Evaluation budget  exhibit included personnel costs across seven lines that represented  various efforts including ship integration, test and evaluation, systems  engineering, and program management support, as shown in figure 8.

Of the 11 program’s fiscal year 2019 budgets we reviewed, one identified  personnel costs on a single line, and the remaining 10 programs included  personnel costs in two or more budget lines.

We provided a draft of this report to DOD for comment. DOD provided  technical comments that we incorporated into this report as appropriate.

We are sending copies of this report to the appropriate congressional  committees; the Acting Secretary of Defense and the Secretaries of the  Army, Navy, and Air Force, as well as the Under Secretary of Defense for  Personnel and Readiness. In addition, the report is available at no charge  on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or dinapolit@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix II.

Appendix I: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Justin Jaynes (Assistant  Director); Bradley Terry (Analyst-in-Charge); Matthew T. Crosby;  Stephanie Gustafson; Heather B. Miller; Karen Richey; Miranda Riemer;  Robin Wilson; and Chris Zakroff made significant contributions to this  review.

DOD relies on a mix of government personnel and contractors to carry out its multi-billion dollar major acquisition programs for new ships, planes, and more.

We reviewed, among other things, the number and type of personnel working at 11 major programs. DOD does not specify one way for structuring programs and they varied widely. Programs:

Generally had more personnel, especially in engineering, if they were developing key technologies for their projects

Generally preferred to use government personnel, but sometimes used contractors to meet workload needs or fulfill short-term tasks


Prescription Opioids: Medicare Should Expand Oversight Efforts to Reduce the Risk of Harm

Misuse of prescription opioids can lead to overdose and death. In 2016, over 14 million Medicare Part D beneficiaries received opioid prescriptions, and spending for opioids was almost $4.1 billion. GAO and others have reported on inappropriate activities and risks associated with these prescriptions.

This statement is based on GAO's October 2017 report (GAO-18-15) and discusses (1) CMS oversight of beneficiaries who receive opioid prescriptions under Part D, and (2) CMS oversight of providers who prescribe opioids to Medicare Part D beneficiaries. For the October 2017 report, GAO reviewed CMS opioid utilization and prescriber data, CMS guidance for plan sponsors, and CMS's strategy to prevent opioid misuse. GAO also interviewed CMS officials, the six largest Part D plan sponsors, and 12 national associations selected to represent insurance plans, pharmacy benefit managers, physicians, patients, and regulatory and law enforcement authorities.

The Centers for Medicare & Medicaid Services (CMS), within the Department of Health and Human Services (HHS), provides guidance on the monitoring of Medicare beneficiaries who receive opioid prescriptions to plan sponsors—private organizations that implement the Medicare drug benefit, Part D—but lacks information on most beneficiaries at risk of harm from opioid use.

CMS provides guidance to plan sponsors on how they should monitor opioid overutilization among Medicare Part D beneficiaries, and requires them to implement drug utilization review systems that use criteria similar to CMS's. CMS's criteria focused on beneficiaries who do all the following: (1) receive prescriptions of high doses of opioids, (2) receive prescriptions from four or more providers, and (3) fill prescriptions at four or more pharmacies. According to CMS, this approach focused actions on beneficiaries the agency determined to have the highest risk of harm.

CMS's criteria, including recent revisions, do not provide sufficient information about the larger population of potentially at-risk beneficiaries. CMS estimates that while 33,223 beneficiaries would have met the revised criteria in 2015, 727,016 would have received high doses of opioids regardless of the number of providers or pharmacies. In 2016, CMS began to collect information on some of these beneficiaries using a higher dosage threshold for opioid use. This approach misses some who could be at risk of harm, based on Centers for Disease Control and Prevention guidelines. As a result, CMS is limited in its ability to assess progress toward meeting the broader goals of its Opioid Misuse Strategy for the Medicare and Medicaid programs, which includes activities to reduce the risk of harm to beneficiaries from opioid use.

CMS oversees the prescribing of drugs at high risk of abuse through a variety of projects, but does not analyze data specifically on opioids. According to CMS officials, CMS and plan sponsors identify providers who prescribe large amounts of drugs with a high risk of abuse, and those suspected of fraud or abuse may be referred to law enforcement. However, GAO found that CMS does not identify providers who may be inappropriately prescribing large amounts of opioids separately from other drugs, and does not require plan sponsors to report actions they take when they identify such providers. As a result, CMS is lacking information that it could use to assess how opioid prescribing patterns are changing over time, and whether its efforts to reduce harm are effective.

In the October 2017 report, GAO made three recommendations that CMS (1) gather information on the full number of at-risk beneficiaries receiving high doses of opioids, (2) identify providers who prescribe high amounts of opioids, and (3) require plan sponsors to report to CMS on actions related to providers who inappropriately prescribe opioids. HHS concurred with the first two recommendations, but not with the third. GAO continues to believe the recommendation is valid, as discussed in the report and in this statement.

I am pleased to be here to discuss our recently released report on  oversight of opioid prescribing in the Medicare program. Misuse of  prescription opioids, which are used to treat both acute and chronic pain,  has become a serious public health problem for the U.S. population,  including Medicare beneficiaries. The Centers for Disease Control and  Prevention (CDC) reported that from 1999 to 2013 the rate of drug  poisoning deaths from prescription opioids nearly quadrupled from 1.4 to  5.1 per 100,000 people. In addition, the Department of Health and  Human Services (HHS) Office of Inspector General (HHS-OIG) reported  that 14.4 million people (about one-third) who participate in Medicare Part  D received at least one prescription for opioids in 2016, and that Part D  spending for opioids in 2016 was almost $4.1 billion. GAO and the HHS- OIG have previously reported on inappropriate activities that can be  associated with such prescriptions, including “doctor shopping” to receive  multiple opioid prescriptions from different providers; the diversion of  prescription drugs for uses other than what was intended; and  questionable prescribing practices by providers, including those in  Medicare.

In March 2015, HHS announced plans to make addressing opioid abuse  a high priority through two broad goals: (1) decreasing opioid overdoses  and overall overdose deaths, and (2) decreasing the prevalence of opioid  use disorder. In 2016, CDC issued guidelines with recommendations for  prescribing opioids in outpatient settings for chronic pain.  The guidelines  recommended that providers use caution when prescribing opioids at any  dose, carefully reassess evidence of individual benefits and risks when  increasing opioid dosage to 50 mg morphine-equivalent dose (MED) per  day or more, and either avoid or carefully justify dosage at 90 mg MED or  more. CDC guidelines also noted that providers should use additional  caution in prescribing opioids to patients aged 65 and older, because the  drugs can accumulate in the body to toxic levels. Further, in January  2017, the Centers for Medicare & Medicaid Services (CMS), the HHS  agency that administers Medicare, issued its Opioid Misuse Strategy for  the Medicaid and Medicare programs, including Medicare Part D. The  strategy includes the agency’s plans to address concerns about  beneficiary use of opioids and the prescribing of opioids by providers.

My remarks today discuss the findings and recommendations from our  report on CMS efforts to oversee prescription opioids. Accordingly, this  testimony focuses on (1) how CMS oversees beneficiaries who receive  opioid prescriptions under Medicare Part D, and (2) how CMS oversees  providers who prescribe opioids to Medicare Part D beneficiaries. For our  report, we reviewed CMS opioid utilization and prescriber data, CMS  guidance for plan sponsors, and CMS’s strategy to prevent opioid misuse.  We also interviewed officials from CMS, the six largest Part D plan  sponsors—private organizations, such as health insurance companies,  contracted by CMS to provide outpatient drug benefit plans to Medicare  beneficiaries—and 12 national associations selected to represent  insurance plans, pharmacy benefit managers, physicians, patients, and  regulatory and law enforcement agencies. More detailed information on  our objectives, scope, and methodology for that work can be found in the  issued report. We conducted the work on which this statement is based in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

CMS Delegates Monitoring of Beneficiaries who Receive Opioid Prescriptions to Plan Sponsors, but Does Not Have Sufficient Information on Those Most at Risk for Harm

CMS Delegates Monitoring of Individual Beneficiaries’ Opioid Prescriptions to Plan Sponsors

Our October 2017 report found that CMS provides guidance to Medicare  Part D plan sponsors on how the plan sponsors should monitor opioid  overutilization problems among Part D beneficiaries. The agency includes  this guidance in its annual letters to plan sponsors, known as call letters;  it also provided a supplemental memo to plan sponsors in 2012. Among  other things, these guidance documents instructed plan sponsors to  implement a retrospective drug utilization review (DUR) system to monitor  beneficiary utilization starting in 2013. As part of the DUR systems,  CMS requires plan sponsors to have methods to identify beneficiaries  who are potentially overusing specific drugs or groups of drugs, including  opioids.

Also in 2013, CMS created the Overutilization Monitoring System (OMS),  which outlines criteria to identify beneficiaries with high-risk use of opioids  and to oversee sponsors’ compliance with CMS’s opioid overutilization  policy. Plan sponsors may use the OMS criteria for their DUR systems,  but they have some flexibility to develop their own targeting criteria within  CMS guidance. At the time of our review, the OMS considered  beneficiaries to be at a high risk of opioid overuse when they met all three  of the following criteria:  1.  received a total daily MED greater than 120 mg for 90 consecutive  2.  received opioid prescriptions from four or more providers in the  previous 12 months, and  3.  received opioids from four or more pharmacies in the previous 12  months.

The criteria excluded beneficiaries with a cancer diagnosis and those in  hospice care, for whom higher doses of opioids may be appropriate.

Through the OMS, CMS generates quarterly reports that list beneficiaries  who meet all of the criteria and who are identified as high-risk, and then  distributes the reports to the plan sponsors. Plan sponsors are expected  to review the list of identified beneficiaries, determine appropriate action,  and then respond to CMS with information on their actions within 30 days.  According to CMS officials, the agency also expects that plan sponsors  will share any information with CMS on beneficiaries that they identify  through their own DUR systems. We found that some actions plan  sponsors may take include

Case management. Case management may include an attempt to  improve coordination issues, and often involves provider outreach,  whereby the plan sponsor will contact the providers associated with  the beneficiary to let them know that the beneficiary is receiving high  levels of opioids and may be at risk of harm.

Beneficiary-specific point-of-sale (POS) edits. Beneficiary-specific  POS edits are restrictions that limit these beneficiaries to certain  opioids and amounts. Pharmacists receive a message when a  beneficiary attempts to fill a prescription that exceeds the limit in place  for that beneficiary.

Formulary-level POS edits. These edits alert providers who may not  have been aware that their patients are receiving high levels of  opioids from other doctors.

Referrals for investigation. According to the six plan sponsors we  interviewed, the referrals can be made to CMS’s National Benefit  Integrity Medicare Drug Integrity Contractor (NBI MEDIC), which is  responsible for identifying and investigating potential Part D fraud,  waste, and abuse, or to the plan sponsor’s own internal investigative  unit, if they have one. After investigating a particular case, they may  refer the case to the HHS-OIG or a law enforcement agency,  according to CMS, NBI MEDIC, and one plan sponsor.

Based on CMS’s use of the OMS and the actions taken by plan sponsors,  CMS reported a 61 percent decrease from calendar years 2011 through  2016 in the number of beneficiaries meeting the OMS criteria of high  risk—from 29,404 to 11,594 beneficiaries—which agency officials  consider an indication of success toward its goal of decreasing opioid use  disorder.

In addition, we found that CMS relies on separate patient safety  measures developed and maintained by the Pharmacy Quality Alliance to  assess how well Part D plan sponsors are monitoring beneficiaries and  taking appropriate actions. In 2016, CMS started tracking plan sponsors’  performance on three patient safety measures that are directly related to  opioids. The three measures are similar to the OMS criteria in that they  identify beneficiaries with high dosages of opioids (120 mg MED),  beneficiaries that use opioids from multiple providers and pharmacies,  and beneficiaries that do both. However, one difference between these  approaches is that the patient safety measures separately identify  beneficiaries who fulfill each criterion individually.

CMS Does Not Have Sufficient Information on Most Beneficiaries Potentially at Risk for Harm

Our October 2017 report also found that while CMS tracks the total  number of beneficiaries who meet all three OMS criteria as part of its  opioid overutilization oversight across the Part D program, it does not  have comparable information on most beneficiaries who receive high  doses of opioids—regardless of the number of providers and pharmacies  used—and who therefore may be at risk for harm, according to CDC  guidelines. These guidelines note that long-term use of high doses of  opioids—those above a MED of 90 mg per day—are associated with  significant risk of harm and should be avoided if possible.

Based on the CDC guidelines, outreach to Part D plan sponsors, and  CMS analyses of Part D data, CMS has revised its current OMS criteria to  include more at-risk beneficiaries beginning in 2018. The new OMS  criteria define a high user as having an average daily MED greater than  90 mg for any duration, and who receives opioids from four or more  providers and four or more pharmacies, or from six or more providers  regardless of the number of pharmacies, for the prior 6 months. Based  on 2015 data, CMS found that 33,223 beneficiaries would have met these  revised criteria. While the revised criteria will help identify beneficiaries  who CMS determined are at the highest risk of opioid misuse and  therefore may need case management by plan sponsors, OMS will not  provide information on the total number of Part D beneficiaries who may  also be at risk of harm. In developing the revised criteria, CMS conducted  a one-time analysis that estimated there were 727,016 beneficiaries with  an average MED of 90 mg or more, for any length of time during a 6  month measurement period in 2015, regardless of the number of  providers or pharmacies used. These beneficiaries may be at risk of harm  from opioids, according to CDC guidelines, and therefore tracking the  total number of these beneficiaries over time could help CMS to  determine whether it is making progress toward meeting the goals  specified in its Opioid Misuse Strategy to reduce the risk of opioid use  disorders, overdoses, inappropriate prescribing, and drug diversion.  However, CMS officials told us that the agency does not keep track of the  total number of these beneficiaries, and does not have plans to do so as  part of OMS. (See fig. 1.)

We also found that in 2016, CMS began to gather information from its  patient safety measures on the number of beneficiaries who use more  than 120 mg MED of opioids for 90 days or longer, regardless of the  number of providers and pharmacies. The patient safety measures  identified 285,119 such beneficiaries—counted as member-years—in  2016. However, this information does not include all at-risk  beneficiaries, because the threshold is more lenient than indicated in  CDC guidelines and CMS’s new OMS criteria. Because neither the OMS  criteria nor the patient safety measures include all beneficiaries potentially  at risk of harm from high opioid doses, we recommended that CMS  should gather information over time on the total number of beneficiaries  who receive high opioid morphine equivalent doses regardless of the  number of pharmacies or providers, as part of assessing progress over  time in reaching the agency’s goals related to reducing opioid use. HHS  concurred with our recommendation.

CMS Oversees Providers through its Contractor and Plan Sponsors, but Efforts Do Not Specifically Monitor Opioid Prescriptions

Our October 2017 report found that CMS oversees providers who  prescribe opioids to Medicare Part D beneficiaries through its contractor,  NBI MEDIC, and the Part D plan sponsors.

NBI MEDIC’s data analyses to identify outlier providers. CMS requires  NBI MEDIC to identify providers who prescribe high amounts of  Schedule II drugs, which include but are not limited to opioids. Using  prescription drug data, NBI MEDIC conducts a peer comparison of  providers’ prescribing practices to identify outlier providers—the  highest prescribers of Schedule II drugs. NBI MEDIC reports the  results to CMS.

NBI MEDIC’s other projects. NBI MEDIC gathers and analyzes data  on Medicare Part C and Part D, including projects using the Predictive  Learning Analytics Tracking Outcome (PLATO) system. According  to NBI MEDIC officials, these PLATO projects seek to identify  potential fraud by examining data on provider behaviors.

NBI MEDIC’s investigations to identify fraud, waste, and abuse. NBI  MEDIC officials conduct investigations to assist CMS in identifying  cases of potential fraud, waste, and abuse among providers for  Medicare Part C and Part D. The investigations are prompted by  complaints from plan sponsors; suspected fraud, waste, or abuse  reported to NBI MEDIC’s call center; NBI MEDIC’s analysis of outlier  providers; or from one of its other data analysis projects.

NBI MEDIC’s referrals. After identifying providers engaged in potential  fraudulent overprescribing, NBI MEDIC officials said they may refer  cases to law enforcement agencies or the HHS-OIG for further  investigation and potential prosecution.

Plan sponsors’ monitoring of providers. CMS requires all plan  sponsors to adopt and implement an effective compliance program,  which must include measures to prevent, detect, and correct Part C or  Part D program noncompliance, as well as fraud, waste, and abuse.  CMS’s guidance focuses broadly on prescription drugs, and does not  specifically address opioids.

Our report concluded that although these efforts provide valuable  information, CMS lacks all the information necessary to adequately  oversee opioid prescribing. CMS’s oversight actions focus broadly on  Schedule II drugs rather than specifically on opioids. For example, NBI  MEDIC’s analyses to identify outlier providers do not indicate the extent to  which they may be overprescribing opioids specifically. According to CMS  officials, they direct NBI MEDIC to focus on Schedule II drugs, because  these drugs have a high potential for abuse, whether they are opioids or  other drugs. However, without specifically identifying opioids in these  analyses—or an alternate source of data—CMS lacks data on providers  who prescribe high amounts of opioids, and therefore cannot assess  progress toward meeting its goals related to reducing opioid use, which  would be consistent with federal internal control standards. Federal  internal control standards require agencies to conduct monitoring  activities and to use quality information to achieve objectives and address  risks. As a result, we recommended that CMS require NBI MEDIC to  gather separate data on providers who prescribe high amounts of opioids.  This would allow CMS to better identify those providers who are  inappropriately and potentially fraudulently overprescribing opioids. HHS  agreed, and noted that it intends to work with NBI MEDIC to identify  trends in outlier prescribers of opioids.

Our report also found that CMS also lacks key information necessary for  oversight of opioid prescribing, because it does not require plan sponsors  to report to NBI MEDIC or CMS cases of fraud, waste, and abuse; cases  of overprescribing; or any actions taken against providers. Plan  sponsors collect information on cases of fraud, waste, and abuse, and  can choose to report this information to NBI MEDIC or CMS. While CMS  receives information from plan sponsors who voluntarily report their  actions, it does not know the full extent to which plan sponsors have  identified providers who prescribe high amounts of opioids, or the full  extent to which sponsors have taken action to reduce overprescribing.  We concluded that without this information, it is difficult for CMS to assess  progress in this area, which would be consistent with federal internal  control standards. In our report, we recommended that CMS require plan  sponsors to report on investigations and other actions taken related to  providers who prescribe high amounts of opioids. HHS did not concur  with this recommendation. HHS noted that plan sponsors have the  responsibility to detect and prevent fraud, waste, and abuse, and that  CMS reviews cases when it conducts audits. HHS also stated that it  seeks to balance requirements on plan sponsors when considering new  regulatory requirements. However, without complete reporting—such as  reporting from all plan sponsors on the actions they take to reduce  overprescribing—we believe that CMS is missing key information that  could help assess progress in this area. Due to the importance of this  information for achieving the agency’s goals, we continue to believe that  CMS should require plan sponsors to report on the actions they take to  reduce overprescribing.  - - - - -  In conclusion, a large number of Medicare Part D beneficiaries use  potentially harmful levels of prescription opioids, and reducing the  inappropriate prescribing of these drugs is a key part of CMS’s strategy to  decrease the risk of opioid use disorder, overdoses, and deaths. Despite  working to identify and decrease egregious opioid use behavior—such as  doctor shopping—among Medicare Part D beneficiaries, CMS lacks the  necessary information to effectively determine the full number of  beneficiaries at risk of harm, as well as other information that could help  CMS assess whether its efforts to reduce opioid overprescribing are  effective. It is important that health care providers help patients to receive  appropriate pain treatment, including opioids, based on the consideration  of benefits and risks. Access to information on the risks that Medicare  patients face from inappropriate or poorly monitored prescriptions, as well  as information on providers who may be inappropriately prescribing  opioids, could help CMS as it works to improve care.

Chairman Jenkins, Ranking Member Lewis, and Members of the  Subcommittee, this concludes my prepared statement. I would be  pleased to respond to any questions that you may have at this time.

If you or your staff members have any questions concerning this  testimony, please contact me at (202) 512-7114 or CurdaE@gao.gov.  Contact points for our Office of Congressional Relations and Public  Affairs may be found on the last page of this statement. Other individuals  who made key contributions to this testimony include Will Simerl  (Assistant Director), Carolyn Feis Korman (Analyst-in-Charge), Amy  Andresen, Drew Long, Samantha Pawlak, Vikki Porter, and Emily Wilson.

This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.

What can Medicare do to help address the nation's opioid crisis?

We've previously found problems such as "doctor shopping" and questionable prescribing practices in Medicare's prescription drug benefits. With 14.4 million people receiving at least one opioid prescription through Medicare in 2016, reducing opioid overprescribing could make a difference.

We looked at Medicare's efforts to reduce inappropriate opioid prescribing, and we revisit our analysis in this testimony. We suggested ways to help Medicare collect information on doctor investigations and at-risk beneficiaries that could help it determine whether its efforts are working.