VA Health Care: Improved Guidance and Oversight Needed for the Patient Advocacy Program

VHA has designated patient advocates at each VAMC to receive and document feedback from veterans or their representatives, including requests for information, compliments, and complaints. In recent years, the importance of a strong patient advocacy program has taken on new significance given concerns with VHA's ability to provide veterans timely access to health care, among other issues.

The Comprehensive Addiction and Recovery Act of 2016 included a provision for GAO to review VHA's patient advocacy program. This report examines the extent to which VHA has (1) provided guidance on the governance of the program; (2) provided guidance on staffing the program; (3) assessed the training needs of patient advocates and monitored training completion; and (4) monitored patient advocacy program data-entry practices and reviewed program data. GAO reviewed VHA and VAMC documents, including summaries of program data. GAO interviewed VHA officials about the program, as well as officials from a non-generalizable selection of eight VAMCs and five VISNs selected based on the volume of veteran complaints and other factors. GAO also compared VHA policies and practices to federal internal control standards.

The Veterans Health Administration (VHA) provided limited guidance to Department of Veterans Affairs (VA) medical centers (VAMC) on the governance of its patient advocacy program and its guidance, a program handbook, has been outdated since 2010. VAMCs are still expected to follow the outdated handbook, which does not provide needed details on governance, such as specifying the VAMC department to which patient advocates should report. Officials from most of the VAMCs that GAO reviewed noted that the VAMC department to which patient advocates report can have a direct effect on the ability of staff to resolve veterans' complaints. The lack of updated and complete guidance may impede the patient advocacy program from meeting its expectations, to receive and address complaints from veterans in a convenient and timely manner.

VHA also has provided limited guidance to VAMCs on staffing the patient advocacy program. VHA's handbook states that every VAMC should have at least one patient advocate and appropriate support staff; however, it did not provide guidance on how to determine the number and type of staff needed. Officials at all but one of the eight VAMCs in GAO's review stated that their patient advocacy program staff had more work to do than they could accomplish. This limited guidance on staffing could impede VAMCs' efforts to ensure that they have the appropriate number and type of staff to address veterans' complaints in a timely manner.

Further, VHA has recommended training for patient advocates, but it has not developed an approach to routinely assess their training needs or monitored training completion. VHA officials stated that they relied on VAMC and Veterans Integrated Service Network (VISN) staff to conduct these activities. However, GAO found that for the eight VAMCs in its review, the training needs of patient advocates were not routinely assessed, and training completion was not always monitored. Without conducting these activities, VHA increases its risk that staff may not be adequately trained to advocate on behalf of veterans.

Finally, VHA has not monitored patient advocacy program data-entry practices or reviewed the data to assess program performance. VHA officials stated that they relied on VISN and VAMC officials to ensure that all complaints were consistently entered into VHA's Patient Advocate Tracking System (PATS). However, GAO identified inconsistencies in the extent to which VAMC officials did so. VHA's lack of monitoring may pose a risk that not all complaints are entered into this tracking system—a goal of the program. Additionally, VHA officials stated they did not systemically review data in the system to assess program performance and identify potential system-wide improvements because VHA considered this the responsibility of VAMCs. As a result, VHA officials may miss opportunities to improve veterans' experiences.

VHA is beginning to address many of these governance, staffing, training, and data issues, including directing a workgroup to provide recommendations by spring of 2018. However, because the recommendations will be advisory, and because program deadlines have slipped in the past, the nature and timing of the actions needed to resolve these issues remain unclear.

GAO is making 6 recommendations to improve guidance for and oversight of the patient advocacy program, focusing on governance, staffing, training, and PATS data entry and assessment. VA concurred with GAO's recommendations.

The Veterans Health Administration (VHA), within the Department of  Veterans Affairs (VA), operates one of the nation’s largest health care  systems. VA has faced a growing demand by veterans for its health care  services due, in part, to (1) servicemembers returning from the United  States’ military operations in Afghanistan and Iraq and (2) the growing  needs of an aging veteran population—trends that are expected to  continue. The total number of veteran enrollees in VA’s health care  system rose from 7.9 million to almost 9 million from fiscal year (FY) 2006  through FY 2016. As the number of veterans using VA health care  services increases, it is important to ensure that feedback about their care  is addressed in a convenient and timely manner. To that end, VHA has  designated patient advocates at each VA medical center (VAMC) to  receive and document feedback from veterans or their representatives,  including requests for information, compliments, and complaints.

Although VHA’s patient advocacy program has been in place since 1990,  it has taken on new significance in recent years given concerns about  veterans’ ability to receive timely and quality care, among other issues.  Our work, along with that of the VA Office of Inspector General (OIG) and  others, has cited longstanding concerns about VA’s oversight of its health  care system, including ambiguous policies and inconsistent processes.  These concerns contributed to the addition of veterans’ health care to  GAO’s High-Risk List in 2015, and its continued inclusion in the 2017  update. In 2017, the VA OIG raised concerns about the effectiveness of  VHA’s patient advocacy program, including a lack of monitoring of  program data, such as feedback documented by advocates, to identify  trends across VAMCs.

Until recently, VHA’s patient advocacy program was overseen by the  Office of Patient Centered Care and Cultural Transformation  (OPCC&CT). However, the Comprehensive Addiction and Recovery Act  of 2016 (CARA), included a provision for VHA to establish an Office of  Patient Advocacy (OPA) by July 2017 to take on oversight responsibilities  for the program, such as ensuring that patient advocates at VAMCs  receive training.

CARA also included a provision for us to review VHA’s patient advocacy  program. This report examines the extent to which VHA has  1.  provided guidance to VAMCs on the governance of the program;  2.  provided guidance to VAMCs on staffing the program;  3.  assessed the training needs of patient advocates and monitored  4.  monitored patient advocacy program data-entry practices and  reviewed program data.

For all four objectives, we interviewed officials involved in the patient  advocacy program from eight VAMCs and their five associated Veterans  Integrated Service Networks (VISN), regional networks of care. We  selected six of our eight VAMCs for variation in (1) the number of  complaints received and entered in VHA’s Patient Advocate Tracking  System (PATS)—an electronic system used to describe and track the  resolution of veterans’ feedback across VAMCs—in FY 2016, (2) facility  complexity level, (3) geographic location, and (4) the type of staff VAMCs  used to administer the program. We selected the remaining two VAMCs  based on one’s involvement in piloting a new approach to recording  patient advocacy program data, and one’s recent changes to the structure  of its program. We also ensured that at least three of the VAMCs we  selected received assessments from OPCC&CT on how they had  implemented the patient advocacy program to gain perspectives on the  office’s involvement with VAMCs. See table 1 for a list of the eight  VAMCs we selected and their associated VISNs. Perspectives obtained  from the eight VAMCs and five VISNs in our review cannot be  generalized.

We also interviewed VHA officials, including those who had overseen the  program when it was under OPCC&CT and who transitioned to OPA once  the office was established in 2017. In addition, we interviewed officials  from five veterans service organizations (VSO) to obtain their  perspectives on the patient advocacy program: American Legion,  Disabled American Veterans, Iraq and Afghanistan Veterans of America,  Veterans of Foreign Wars, and Vietnam Veterans of America.

To examine the extent to which VHA has provided guidance to VAMCs on  the governance of the patient advocacy program, we reviewed VHA’s  handbook for the program to determine, among other things, whether it  specified the VAMC department to which patient advocates should report  and whether it identifies responsibilities for VHA staff overseeing the  program. We also reviewed documentation of VHA’s planned efforts  related to improving the governance of the program, such as a draft  directive for the program. We evaluated the information we reviewed in  VHA’s handbook against federal internal control standards.

To examine the extent to which VHA has provided guidance to VAMCs on  staffing the patient advocacy program, we reviewed VHA’s handbook for  the program. Specifically, we reviewed the handbook to determine the  extent to which it provided guidance on how VAMCs should determine the  appropriate number and type of staff needed to administer the program.  We also reviewed documentation of VHA’s planned efforts related to  staffing the program identified in a workgroup charter. We evaluated  VHA’s efforts to provide guidance to VAMCs on staffing against key  principles for effective strategic workforce planning and federal internal  control standards.

To examine the extent to which VHA has assessed the training needs of  patient advocates and monitored training completion, we reviewed  training materials VHA provided to VAMCs, such as a list of  recommended training for patient advocates. We also reviewed  documentation of VHA’s planned efforts related to assessing the training  needs of patient advocates identified in a workgroup charter. We  evaluated the extent to which VHA has monitored training of patient  advocates against a guide for assessing strategic training and  development efforts and federal internal control standards.

To examine the extent to which VHA has monitored data-entry practices  and reviewed data from the patient advocacy program, we reviewed  VHA’s handbook for the program and summaries of data from PATS.  We also reviewed documentation of VHA’s planned efforts related to  PATS data-entry practices and reviewing program data identified in a  workgroup charter. We evaluated the extent to which VHA has monitored  data-entry practices and reviewed PATS data against a guide for  assessing the reliability of computer-processed data and federal internal  control standards.

We conducted this performance audit from February 2017 to April 2018 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

VHA’s patient advocacy program is intended to provide veterans with a  means to provide feedback about health care services they receive at  VAMCs. VHA sets forth minimum expectations for VAMCs’ administration  of the program, including that veterans must have easy access to a  patient advocate and must have their complaints addressed in a  convenient and timely manner.

Administration of the Patient Advocacy Program

The patient advocacy program is administered at the VAMC level. Each  of VA’s 170 VAMCs is responsible for making at least one patient  advocate available to respond to veterans’ feedback, and for ensuring  that feedback is recorded in PATS. VAMCs may designate other staff to  assist patient advocates in responding to feedback, such as lead patient  advocates and service-level advocates. Service-level advocates, such as  nurses or administrative staff, are designated at some VAMCs to respond  to veterans’ feedback before involving a patient advocate. All VAMC staff  that have a designated role in the administration of the patient advocacy  program are referred to as patient advocacy program staff. In addition to  designating program staff, VAMCs may use a variety of methods to make  veterans aware of the patient advocacy program, such as displaying  signage on site and including information about the program on their  websites. (See app. I for more information on the methods selected  VAMCs used to make veterans aware of the program.)

Patient advocacy program staff enter veterans’ feedback in PATS using a  report of contact (ROC) and assign one or more issue codes that  generally describe the nature of the feedback, such as coordination of  care. (See app. II for additional information on entering veterans’  feedback into PATS.) Each piece of feedback shared is categorized as  either a request for information, compliment, or complaint. VHA’s  handbook for the program specifies certain goals for data collection and  resolution—specifically, that all complaints should be entered in PATS to enable a comprehensive  understanding of veterans’ issues and concerns to, in turn, identify  potential system-wide improvements; and  responses should occur no later than 7 days after the complaint is  made.

With this guidance, patient advocacy program staff use a variety of  approaches for entering veterans’ feedback in PATS and closing it in the  system once addressed. For example, when VAMCs have designated  service-level advocates, the process for entering and closing feedback in  PATS is generally different than the approach used by VAMCs that have  only patient advocates. (See fig. 1.)

Patient advocacy program staff at each VAMC are assisted by a VISN- level coordinator who acts as a liaison between the VAMCs and VHA and  is responsible for ensuring consistency in PATS data collection within the  VISN. The VISN director is responsible for designating the coordinator  and ensuring that each VAMC within the VISN has at least one patient  advocate.

Oversight of the Patient Advocacy Program

The VHA office responsible for overseeing the patient advocacy program  changed as a result of CARA. From January 2011 to July 2017, the  program was overseen by OPCC&CT under VHA’s Deputy Under  Secretary for Health for Operations & Management. CARA included a  provision for VHA to establish OPA to begin overseeing the program and  specified that this office would report directly to the Under Secretary for  Health, a higher-level office within VHA. Although OPCC&CT is no  longer responsible for overseeing the program, it is to continue to play an  advisory role to OPA during the initial phases of its work, according to  OPCC&CT officials.

Many of OPA’s oversight responsibilities are specified in CARA including  ensuring that patient advocates advocate on behalf of veterans, manage  PATS, and identify trends in the data to determine whether there are  opportunities for improving veterans’ health care. Also, OPA’s director is  required to ensure that patient advocates receive relevant, consistent  training across VAMCs. When establishing the office in July 2017, VHA  officials wrote a memo indicating that OPA’s primary objectives were to  implement a standardized policy for the patient advocacy program and to  resolve any system-wide issues, such as concerns about care across  VAMCs identified through veterans’ feedback. In addition, in August 2017,  OPA began soliciting feedback from VAMCs on various aspects of the  patient advocacy program to identify improvement priorities and best  practices. By September 2017, OPA had identified an acting program  director, established a workgroup (called the National Strategic  Workgroup) to develop recommendations related to program  administration, and finalized a charter that identifies workgroup  deliverables.

VHA Has Provided Limited, Outdated Guidance to VAMCs on the Governance of the Patient Advocacy Program

VHA has provided limited guidance to VAMCs on the governance of the  patient advocacy program. Specifically, VHA provided limited guidance on  how to meet the program’s expectations that veterans have easy access  to a patient advocate who will hear their complaints and address them in  a timely manner. While VHA’s handbook for the program provides general  information on the responsibilities of patient advocacy program staff, it  does not specify the VAMC department to which patient advocates should  report to help ensure VAMCs meet these expectations. According to VHA  officials, the lack of specific guidance was intentional and due in part to  VHA officials’ view that leadership at each VAMC is in the best position to  understand the needs of veterans at their facilities, and therefore should  have flexibility to make decisions about governance in response to those  needs.

In addition to providing limited guidance to VAMCs, VHA’s patient  advocacy program handbook is out of date and does not incorporate  recent agency-wide changes, such as those made in response to VHA  Strategic Plan FY 2013 – 2018 which identifies the goal of providing  proactive, patient-driven health care. The handbook for the program  was issued in 2005, expired in 2010, and as of January 2018, no updates  had been released. In the absence of an updated document, VAMCs are  still expected to follow the outdated handbook. However, the handbook  does not identify the responsibilities of the current VHA office responsible  for overseeing the program. Instead, it identifies the responsibilities of the  VHA office that oversaw the program before OPCC&CT began  overseeing the program in 2011.

In recent years, OPCC&CT reviewed the implementation of the patient  advocacy program at some VAMCs and provided specific  recommendations on how to change program governance to better reflect  a more proactive patient advocacy program model. However, the  recommendations from these reviews were provided only to some VAMCs; guidance that could be applicable to all VAMCs was not added  to the handbook. OPCC&CT officials stated that they did not update the  handbook because they decided to instead spend time trying to  understand recent feedback they received from VAMC officials and  ensure that any updates would reflect system-wide shifts as a result of  VHA’s strategic plan.

OPCC&CT’s limited and outdated guidance to VAMCs on the governance  of the patient advocacy program is inconsistent with federal internal  control standards for the control environment, which require agencies to  establish an organizational structure, assign responsibility, and delegate  authority to achieve agency objectives—key aspects of governance. To  do so, an agency may develop an organizational structure that assigns  responsibilities to discrete units and defines reporting lines at all levels of  the organization. Without providing specific, timely guidance to VAMCs on  the governance of the patient advocacy program, the program is at risk of  not meeting its minimum expectations.

In light of the limited and outdated guidance on the governance of the  program, patient advocacy program staff at most of our selected VAMCs  noted that the VAMC department to which patient advocates report can  have a direct effect on the ability of staff to resolve veterans’ complaints.  For example, patient advocates at one VAMC said because of the  program’s position within the organization, they did not have the authority  to ensure that VAMC officials external to the patient advocacy program,  such as physicians, quickly engaged in responding to veterans’  complaints. In these cases, a patient advocate would contact the  physician to resolve a complaint, but may not have received a response  until the matter was brought to the attention of the physician’s  supervisor—a reporting line that is outside of the patient advocacy  program at this VAMC. Officials from several of our selected VAMCs and  VSOs noted that the position of the patient advocacy program within  VAMCs may not give patient advocates the authority to require VAMC  staff to respond to veterans’ complaints. They added that conflict-of- interest concerns could arise when a veteran has a complaint about a  VAMC for which the patient advocate works. (See app. III for additional  information on the governance of the patient advocacy program at  selected VAMCs.)

In VA’s written comments on a draft of this report, which are reproduced  in Appendix IV, VA stated that it issued its new directive for the patient  advocacy program that had been in development as we were conducting  our review. While the updated directive specifies that a VAMC’s lead  patient advocate should report to the facility director, it does not specify  the VAMC department to which other patient advocacy program staff,  including patient advocates who are not designated as lead patient  advocates and service-level advocates, should report. In addition, OPA’s  National Strategic Workgroup recently submitted recommendations to  OPA on the governance of the patient advocacy program. OPA officials  stated that they plan to prioritize the recommendations and elicit feedback  from VISN directors on how to operationalize the recommendations.  However, it is unclear whether OPA will provide additional guidance  related to the governance of the program based on these  recommendations, such as guidance on the VAMC department to which  all types of patient advocacy program staff should report. Until actions to  address the weaknesses we found are completed, guidance on the  governance of the program will continue to be lacking.

VHA Has Provided Limited Guidance to VAMCs on Staffing the Patient Advocacy Program

VHA has provided limited guidance to VAMCs on the number and type of  patient advocacy program staff needed to ensure that complaints from  veterans are addressed in a convenient and timely manner. According to  VHA’s existing handbook for the program, every VAMC should have at  least one patient advocate and appropriate administrative, technical, and  clerical support should be provided to allow for efficient performance of  the responsibilities of program staff. OPCC&CT did not provide guidance  on how VAMCs should determine the appropriate number of  administrative, technical, and clerical staff or type of patient advocacy  program staff, such as lead patient advocates and service-level  advocates. According to officials, this was because no assessment was  conducted to identify what staff resources would be needed to meet the  expectations of the program. In the absence of such an assessment,  OPCC&CT instead relied on each VAMC to determine what resources  would be needed based on the facility’s size and services provided.  However, VHA’s handbook for the program does not provide instruction  for VAMC or VISN officials on how to determine the number and type of  staff needed for the program. OPCC&CT officials added that budget  constraints can also affect a VAMC’s ability to hire the appropriate staff  for the program. (See app. III for additional information on the number  and type of patient advocacy program staff at selected VAMCs.)

Officials at all but one of the selected VAMCs stated that program staff at  their VAMCs had more work to do than they could handle. For example,  VAMC officials cited backlogs in work, such as calls from veterans not  being answered, messages not being responded to, voicemail boxes  being full, and not all veterans’ feedback being entered into PATS.  Officials from one VAMC we spoke with in July 2017 stated that due to  workload demands and not enough patient advocacy program staff at  their VAMC, they had roughly 300 unanswered phone calls at that time  from veterans who want to provide feedback to a patient advocate.  Officials from several VSOs we spoke with stated that there is not enough  patient advocate staff, adding that veterans reported that their calls to  patient advocates were not answered, they were unable to reach an  advocate, or their calls were not responded to in a timely manner.

The lack of staffing guidance is inconsistent with GAO’s Key Principles for  Effective Strategic Workforce Planning, which states that workforce  planning is essential to addressing an organization’s critical need to align  its human capital program with its current and emerging mission and  programmatic goals. Further, federal internal control standards require  agencies to design control activities to achieve objectives, a key aspect of  effectively staffing a program. Such control activities may include  effectively managing the agency’s workforce, such as by continually  assessing the knowledge, skills, and abilities of the workforce to achieve  organizational goals.

The lack of guidance on staffing may impede VAMCs’ efforts to ensure  that they have the appropriate number and type of staff to administer the  patient advocacy program. The resulting misalignment of staff resources  could have negatively affected VAMCs’ ability to achieve the program’s  objectives, including addressing veterans’ complaints in a timely manner.  For example, if there are not a sufficient number of patient advocates to  respond to veterans’ phone calls in a timely manner, VAMCs may not be  able to ensure that patient advocates can respond to veterans’ complaints  within 7 days, as called for by VHA’s handbook for the program.

According to VHA officials, OPA analyzed feedback from VAMCs on the  factors that should be considered in developing national guidelines for  staffing, such as facility size and complexity level, and directed its  National Strategic Workgroup to develop recommendations for  determining the extent to which VAMCs have utilized various patient  advocacy program staff, such as service-level advocates, by the spring of  2018. However, OPA expects that these efforts will result in  recommendations for consideration, and it is unclear what steps, if any,  will be taken based on the recommendations. Until actions to address the  weaknesses we found are completed, the lack of guidance for VAMCs on  determining the appropriate number and types of staff will put the patient  advocacy program at risk of being unable to address veterans’ complaints  in a convenient and timely manner.

VHA Has Recommended Training for Patient Advocates, but Has Not Developed an Approach to Routinely Assess Their Training Needs or Monitored Training Completion

VHA Has Developed a Recommended Training List for Patient Advocates, but Has Not Developed an Approach to Assess Their Training Needs on a Routine Basis

VHA has recently developed a list of recommended training for patient  advocates. In the spring of 2017, OPCC&CT officials updated a  recommended training list for patient advocates developed before 2011  when OPCC&CT began overseeing the patient advocacy program. The  training list covers a wide variety of topics, including how to enter and  examine trends in PATS data, as well as key responsibilities of patient  advocates outlined in VHA’s handbook for the program. OPCC&CT  officials stated that they would like to make the trainings required, but  have not pursued this because of the lengthy process within VHA to  designate required training for a specific group of staff. To update the list  in 2017, OPCC&CT convened a workgroup (which included several  patient advocates) to determine whether the old training list was  sufficient, and the workgroup shared its suggested updates with VISN- level coordinators for distribution to VAMCs in April 2017.

We found that OPCC&CT has not developed an approach to routinely  assess the training needs of patient advocates. Rather, OPCC&CT  officials stated that they relied on VAMC and VISN staff to conduct these  assessments. However, VHA’s handbook for the program does not  specify that VAMC or VISN officials are responsible for conducting routine  assessments of patient advocates’ training needs. None of our selected  VAMCs routinely conducted assessments of the training needs of patient  advocates, such as assessing whether advocates were adequately  trained to carry out their responsibilities. Officials from two VAMCs said  they used ad hoc approaches to assess training needs. For example, one  patient advocate supervisor stated that training is offered on an “as  needed” basis in patient advocate meetings when a training need is  identified.

The lack of an approach for routinely assessing the training needs of  patient advocates is inconsistent with federal standards for internal  control related to control activities. Under these standards relating to  human capital, management ensures that training is aimed at developing  and retaining employee knowledge, skills, and abilities to meet changing  organizational needs. Management should also continually assess the  knowledge, skills, and ability needs of a program so that the program is  able to obtain a workforce that has the required knowledge, skills, and  abilities to achieve organizational goals.

Without an approach for routinely assessing the training needs of patient  advocates, VHA may not be able to clearly identify gaps in the knowledge  and skills of these staff over time, which, in turn, could put the program at  risk of not meeting its goals. For example, if there is a gap in  understanding among patient advocates that all complaints should be  entered into PATS, addressing veterans’ complaints may be delayed, if  addressed at all, and opportunities to analyze complaint data for the  purpose of identifying system-wide improvements may be missed.

According to VHA officials, OPA analyzed feedback from VAMCs on the  training needs of patient advocates, including how to correctly enter data  into PATS, and directed its National Strategic Workgroup to develop  recommendations for assessing the training needs of patient advocates  by the spring of 2018. OPA expects that these efforts will result in  recommendations for OPA to consider, but it is unclear what steps, if any,  will be taken based on the recommendations. Until actions to address the  weaknesses we found are completed, the lack of routine assessments of  training needs will continue to put the program at risk of staff not having  the requisite skills and knowledge to carry out their duties.

VHA Has Not Monitored Training Completion for Patient Advocates

VHA has not monitored the completion of training for patient advocates.  Specifically, OPCC&CT officials said that they did not monitor the extent  to which patient advocates completed the recommended training  distributed in April 2017. Instead, these officials relied on patient advocate  supervisors to monitor training completion. However, VHA’s handbook for  the program does not specify that patient advocate supervisors are  responsible for monitoring the completion of training for patient  advocates.

Half of patient advocate supervisors at our selected VAMCs did not track  the completion of patient advocacy training. Patient advocate supervisors  said that they are able to track the completion of general VA employee  training through VA’s Talent Management System. However, most  training specific to patient advocacy were generally not included in this  system during the period of our review. Officials from our selected  VAMCs who did track patient advocacy training used various methods to  record completion, such as keeping attendance lists for the training  provided.

Taking steps to monitor training completion would be consistent with  GAO’s Guide for Assessing Strategic Training and Development Efforts in  the Federal Government which identifies components of the training and  development process, including having agencies collect and monitor data  corresponding to establishing training objectives. Monitoring training  completion would also be consistent with federal standards for internal  control related to control activities. Under these standards relating to  human capital, management ensures that training is aimed at developing  and retaining employee knowledge, skills, and abilities to meet changing  organizational needs. Management also continually assesses the  knowledge, skills, and ability needs of a program so that the program is  able to obtain a workforce that has the required knowledge, skills, and  abilities to achieve organizational goals—key components for monitoring  training completion.

If patient advocates are not properly trained in how to use PATS to  document and resolve complaints, tracking the status of complaints may  be more difficult, which could increase the likelihood that they are not  addressed in a timely manner, if at all. Further, CARA specifies that the  director of OPA should ensure that patient advocates receive training  specific to patient advocacy.

According to VHA officials, OPA did not obtain information on whether  patient advocates completed recommended training and did not identify  an approach for monitoring training completion moving forward. Without  monitoring training completion, there is an increased risk that patient  advocates have not received the training they need to effectively fulfill  their responsibilities such as advocating on behalf of veterans and  consistently using PATS to document and resolve complaints.

VHA Has Not Monitored Patient Advocacy Data-Entry Practices or Reviewed Patient Advocacy Data to Assess Program Performance and Identify System-Wide Improvements

VHA Has Not Monitored Whether Complaints Were Always Entered into PATS and Issue Codes Assigned Consistently

VHA officials have not monitored PATS data-entry practices to ensure  complaints were always entered into PATS and issue codes were  assigned consistently to ROCs. OPCC&CT officials told us they did not  monitor the data-entry practices of patient advocacy program staff to  ensure that all complaints were entered into PATS, a key goal according  to VHA’s handbook for the program. Rather, they relied on VISN and  VAMC officials to ensure that program staff entered all complaints into  PATS. Officials from two of the five VISNs we interviewed stated that they  did not perform any audits or checks of the data entered into PATS by  patient program staff at VAMCs.

We also found inconsistencies in the extent to which VAMC officials  entered complaints into PATS, with complaints always entered into PATS  at one of our selected VAMCs, while at other VAMCs some complaints  were left unrecorded, according to officials. For example, at one VAMC,  officials stated that over a third of the complaints received were not  entered into PATS due to the competing workload demands of patient  advocates. Similarly, at another selected VAMC, almost a quarter of the  complaints received were not entered into PATS, according to patient  advocates there who explained that they primarily used a document  outside of PATS to record veterans’ feedback.

In addition, OPCC&CT officials told us they did not monitor whether  patient advocates used a consistent practice to assign issue codes to  veterans’ feedback recorded into PATS. Using a consistent data-entry  practice is important to ensure that PATS data can be compared across  VAMCs to better enable an accurate and comprehensive understanding  of veterans’ issues and concerns, a goal of the patient advocacy program.  OPCC&CT officials stated that they relied on VISN-level coordinators to  monitor coding practices because VHA’s handbook for the program states  that these coordinators should develop VISN-wide consistent approaches  for entering complaints into PATS. VISN-level coordinators from two  selected VISNs stated that they created a standard practice for assigning  issue codes within a particular VISN; however, the coding practices  differed between VISNs, making national level analysis difficult.

We also found inconsistencies in how VAMC officials coded specific  veterans’ feedback. For example, patient advocates did not use  consistent practices to code issues related to the Veterans Choice  Program (Choice Program), one of the most common types of issues  patient advocates told us they hear about from veterans. Officials from  one of our selected VAMCs said they code feedback related to the  Choice Program under a specific “request for information” issue code,  regardless of whether the feedback was a request for information,  compliment, or complaint. In contrast, officials at another VAMC stated  that they typically code feedback related to the Choice Program as a  complaint related to billing. (See app. II for additional information on data- entry practices at selected VAMCs.)

OPCC&CT’s lack of monitoring of PATS data-entry practices is  inconsistent with GAO’s Assessing the Reliability of Computer-Processed  Data which identifies the importance of consistent data-entry practices to  ensure that data are reasonably complete and accurate. Further, federal  standards for internal control related to information and communications  require agencies to use quality information, such as relevant data from  reliable sources, to achieve the agency’s objectives. Under internal  control standards for control activities, management also is to monitor  performance to achieve objectives. Without OPCC&CT monitoring data- entry practices, the patient advocacy program is at risk of not meeting its  goal that all complaints are entered into PATS and there is an increased  likelihood of VHA not having an accurate understanding of veterans’  complaints across VAMCs.

Moving forward, in fall 2017, OPA distributed meeting minutes to all VISN  and VAMC directors stating that all veterans’ feedback should be  consistently recorded in PATS. OPA officials also updated some of the  issue codes in PATS in fall 2017 and added a code specifically for  community care issues, such as issues related to the Choice Program. In  addition, OPA officials stated that they plan to promote the consistent  assignment of issue codes to veterans’ feedback through national  training, but have not specified when this training will occur or if OPA staff  will monitor patient advocates’ consistent assignment of issue codes or of  data-entry practices generally. Until these actions are completed,  however, the gaps in monitoring of PATS data-entry practices that we  identified will continue to exist, putting the program at risk of incomplete  or unreliable data that may not allow an accurate understanding of  veterans’ complaints, critical to making system-wide improvements.

VHA Has Not Systematically Reviewed PATS Data to Assess Program Performance and Identify Potential System- Wide Improvements

VHA officials have not systematically reviewed PATS data to assess  program performance and identify potential system-wide improvements,  goals of the patient advocacy program. Specifically, OPCC&CT officials  stated that they reviewed PATS data in response to inquiries, but did not  conduct systematic reviews of the data over time. For example, they did  not track VAMC performance on responding to complaints in a timely  manner or track the most common complaints across VAMCs to identify  potential opportunities for system-wide improvements.

OPCC&CT officials stated that they did not conduct systematic reviews of  PATS data because VISN and VAMC officials were primarily responsible  for these analyses. However, according to VHA’s handbook for the  patient advocacy program, VHA officials have a responsibility to examine  PATS data for trends across VAMCs and identify any areas for system- wide improvement. Officials stated that it was challenging to analyze  PATS information included in narrative text, such as descriptions of  veterans’ feedback.

Not reviewing PATS data is inconsistent with federal standards for  internal control for monitoring which require agencies to establish and  operate monitoring activities, such as assessing the quality of  performance over time, and evaluate the results. Further, not  conducting systematic assessments of PATS data made it difficult for  OPCC&CT to determine program performance, such as whether the  program was meeting its goal that all complaints are entered into PATS  and responded to within 7 days. Officials explained that VHA interprets  this goal to mean that complaints are closed in PATS within 7 days.  According to VA, between FY 2014 and FY 2017 there were more than  53,000 complaints per year open for greater than 7 days. If OPCC&CT  officials had conducted systematic reviews of PATS data, they may have  been able to identify that there were a significant number of complaints  open for longer than 7 days and consider what actions should be taken,  such as providing additional guidance to VAMCs on how to address  complaints in a timely manner.

Furthermore, without systematically reviewing PATS data across VAMCs  to identify potential system-wide improvements, OPCC&CT officials may  have been unaware of important care issues across VAMCs. For  example, patient advocates from several of our selected VAMCs stated  that opioid prescription issues are among the most common complaints  they received from veterans. If OPCC&CT officials were to have  systematically reviewed PATS data across VAMCs to determine the  prevalence of these types of complaints, they could have identified the  need to address them on a national level and consider system-wide  policies or guidance in response.

According to VHA officials, OPA is in the process of identifying the data it  needs to review on a routine basis, and directed its National Strategic  Workgroup to identify program data that could be reviewed to assess  program performance and identify potential system-wide improvements  by the spring of 2018. However, OPA expects that these efforts will result  in recommendations for OPA to consider, and it is unclear what steps, if  any, will be taken based on the recommendations. Until actions to  address the weaknesses we found are completed, the lack of a  systematic review of PATS data will persist, putting the program at  continued risk of missed opportunities for identifying and addressing  weaknesses across VAMCs.

As one of the largest health care delivery systems in the nation, it is  critically important for VHA to ensure that each veteran who receives  health care services has easy access to an advocate who listens to that  veteran’s feedback and responds in a timely manner. This is especially  important given concerns about veterans’ ability to receive timely and  quality care. However, VHA’s efforts to ensure that the patient advocacy  program is meeting its goals—to identify potential system-wide  improvements and respond to complaints within 7 days—have fallen  short. OPCC&CT did not provide sufficient oversight to the program in the  four key areas of governance, staffing, training, and data-entry practices,  which has left the program at risk for not meeting its goals.

VHA’s newly established OPA has initiated plans to improve the patient  advocacy program in these four areas; however, most of these plans  center around a workgroup that will make recommendations for OPA to  consider, and it is unclear what specific actions, if any, will be taken  based on these recommendations. Further, documentation for several of  OPA’s planned efforts has not been finalized. Unless specific actions to  address the weaknesses we identified are completed expeditiously, the  program is at risk of not meeting its goals, including addressing veterans’  complaints in a convenient and timely manner. Furthermore, without  addressing the weaknesses we identified, OPA misses opportunities to  review PATS data across VAMCs to identify potential system-wide issues  that, if addressed, could significantly improve the experience of veterans.  Such reviews are critical to ensuring that VHA is taking steps to both  meet its goal in its strategic plan to provide veterans with timely and  quality health care, and to address recent issues it has faced, such as  veterans’ ability to access care in a timely manner.

We are making the following six recommendations to the VHA  Undersecretary for Health: provide updated guidance to VAMCs on the governance of the patient  advocacy program, including clear definitions of reporting lines.  (Recommendation 1) assess and provide guidance to VAMCs on appropriately staffing the  patient advocacy program, including guidance on how to determine  the appropriate number and type of staff. (Recommendation 2) develop an approach to routinely assess the training needs of patient  advocates. (Recommendation 3) monitor the completion of training for patient advocates.  (Recommendation 4) monitor PATS data-entry practices to ensure all complaints are  entered into PATS and that veterans’ feedback is coded consistently.  (Recommendation 5)  systematically review PATS data to assess program performance and  identify potential system-wide improvements. (Recommendation 6)

We provided a draft of this report to VA for comment. In its written  comments, which are reproduced in Appendix IV, VA concurred with our  recommendations and noted that it recently issued the new directive for  patient advocacy that had been in development as we were conducting  our review. The directive supersedes the outdated handbook for the  patient advocacy program and describes certain aspects of program  governance, including certain reporting lines, roles, and responsibilities.  Accordingly, VA requested that we close our first recommendation related  to governance. We revised our report to reflect the issuance of the new  directive. However, we do not believe the directive fully implements our  recommendation. While the updated directive specifies that a VAMC’s  lead patient advocate should report to the facility director, it does not  specify the VAMC department to which other patient advocacy program  staff, including patient advocates who are not designated as lead patient  advocates and service-level advocates, should report. Until VA specifies  the reporting lines for these other patient advocacy program staff, our  recommendation will remain open. In addition, VA stated in its written  comments that OPA has efforts underway related to staffing, training, and  PATS data entry and assessment and provided estimated completion  dates for these efforts. We will monitor VA’s efforts to address our  recommendations. VA did not provide technical comments.

We are sending copies of this report to the appropriate congressional  committees, the Secretary of Veterans Affairs, the Under Secretary for  Health, and other interested parties. In addition, the report will be  available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-7114 or at draperd@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in Appendix V.

Appendix I: Awareness of the Patient Advocacy Program

Our eight selected Department of Veterans Affairs (VA) medical centers  (VAMC) use a variety of methods to make veterans or their  representatives aware of the patient advocacy program, including  providing brochures on the program, displaying signage, and providing  program information on the VAMC’s website. (See fig. 2 for examples of  patient advocacy program signage at some of the VAMCs we visited.)

Appendix II: Patient Advocate Tracking System (PATS) Data Entry and Management

Patient advocacy program staff, such as patient advocates or service- level advocates who are designated to respond to veterans’ feedback,  enter feedback from veterans or their representatives in the Veterans  Health Administration (VHA) Patient Advocate Tracking System (PATS)  by creating a report of contact (ROC). Each ROC includes basic  information regarding the individuals involved, a description of the  feedback provided by the veteran, and a description of the steps taken to  resolve the issue. Patient advocacy program staff assign one or more  issue codes that generally describe the nature of the feedback, such as  “coordination of care.” (See figures 3 and 4.)

In order to organize veterans’ feedback, VHA categorizes feedback as  either requests for information, compliments, or complaints. Within each  of these categories VHA defines specific issue codes for program staff to  select from based on the description of the veteran’s feedback. (See table  2.)

The Comprehensive Addiction and Recovery Act of 2016 (CARA),  includes a provision for every VAMC to display the purpose of the  program, along with the contact information of a patient advocate at the  facility, in as many prominent locations as deemed appropriate to be seen  by the largest percentage of veterans. In September 2016, VHA Central  Office sent a memo to Veterans Integrated Service Network (VISN)  directors explaining this requirement and an Office of Patient Centered  Care and Cultural Transformation (OPCC&CT) official obtained  confirmation from all VHA facilities that this requirement was met in  October 2016. Nevertheless, officials from two veterans service  organizations (VSO) we interviewed stated they often encounter veterans  who are not aware of the patient advocacy program.

According to VA, in fiscal year (FY) 2017, there were 268,114 veterans  associated with ROCs entered in PATS. VA also reported that, in the  same year, patient advocacy program staff entered 414,256 unique  reports of contact in PATS. According to VA, from the unique reports of  contact in PATS, program staff documented 473,564 issues, which  included (but were not limited to) 112,722 requests for information,  35,839 compliments, and 325,003 complaints. See table 3 for the top  five issues that patient advocacy program staff across VAMCs entered in  PATS for FY 2017. According to VA, in FY 2017, a total of 1,391 program  staff system-wide entered data in PATS. In the same year, according to  PATS, veterans, rather than family members or friends, most often  provided feedback to patient advocacy program staff.

Our eight selected VAMCs varied in the number of patient level  advocates and service-level advocates who had access to PATS,  whether veterans’ feedback was recorded outside of PATS, and which  issue code or codes were used to record feedback related to the  Veterans Choice Program. (See table 4.)

Examples of methods that patient advocates and service-level advocates  used at selected VAMCs to record veterans’ feedback outside of PATS  included call logs and tracking spreadsheets. VAMC officials indicated  that recording information outside of PATS helped them track their  responses to veterans’ feedback. Some of the information recorded  outside of PATS was additional information that is not required to be  entered into PATS, such as requests for information.

Appendix III: Approaches to the Governance and Staffing of the Patient Advocacy Program

The eight Department of Veterans Affairs (VA) medical centers (VAMC)  selected for our review used a variety of approaches to govern the patient  advocacy program, resulting in differences in the number of positions for  patient advocates and service-level advocates and the title of the  positions. Service-level advocates, such as nurses or administrative staff,  are designated at some VAMCs to respond to veterans’ feedback before  involving a patient advocate. (See table 5.)

Patient advocates reported to a variety of departments among our  selected VAMCs. At two of the VAMCs, patient advocates reported to the  customer or consumer relations department, while at three, patient  advocates reported to the quality management department. In addition,  the placement of the department that patient advocates reported to within  the VAMC differed. For example, the patient advocate supervisor at one  of the selected VAMCs said that patient advocates reported to the  customer service manager, who did not report directly to the VAMC  director. At another VAMC, the patient advocate reported directly to the  VAMC director.

In addition to the Veterans Health Administration (VHA) handbook for the  patient advocacy program, all eight of our selected VAMCs developed  their own policies for the administration of the program, and these policies  varied. For example, while almost all of the policies specified the  responsibilities with respect to the patient advocacy program of the  service chiefs—officials who oversee the administration and operation of  service lines such as primary care, these responsibilities varied. For  example, two of the policies required service chiefs to incorporate  veterans’ feedback into performance measures used for VAMC staff  external to the patient advocacy program, such as physicians, while the  other policies did not.

We also found variation between our selected VAMCs with respect to  whether they had written descriptions of the service-level advocates’  roles. Of the six VAMCs that designated service-level advocates, three  had written descriptions of their roles, while three did not. Further, among  the VAMCs that had a written description of the role of a service-level  advocate, the expectations for these advocates varied. For example, one  VAMC’s written description specified that service-level advocates are  expected to enter veterans’ feedback into PATS within 7 days of receiving  the feedback. The written descriptions at the other two VAMCs did not  specify this expectation.

Appendix IV: Comments from the Department of Veterans Affairs

Appendix V: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Hernán Bozzolo (Assistant  Director), Rebecca Rust Williamson (Analyst-in-Charge), Jennie F. Apter,  Q. Akbar Husain, and Emily Loriso made key contributions to this report.  Also contributing were Julie Flowers, Jacquelyn Hamilton, and Vikki  Porter.

The Veterans Health Administration has designated patient advocates at each VA medical center to receive and document feedback from veterans.

Amid concerns about veterans’ ability to receive timely and quality care, we reviewed the VHA patient advocacy program.

We found that VHA is beginning to address governance, staffing, training, and other issues, including directing a workgroup to provide recommendations by spring of 2018. However, the recommendations are only for consideration and deadlines have slipped in the past.

We recommended 6 actions to improve guidance for and oversight of the program.