Veterans' Disability Benefits: Better Measures Needed to Assess Regional Office Performance in Processing Claims

Each year, VBA processes more than 1 million disability compensation claims and provides about $65 billion in benefits to veterans. The Jeff Miller and Richard Blumenthal Veterans Health Care and Benefits Improvement Act of 2016 includes a provision for GAO to review VBA's regional offices to help VBA achieve more consistent performance in processing disability compensation claims.

This report examines (1) how VBA manages workload and performance for the disability compensation claims process, (2) how well VBA's timeliness and accuracy measures capture its regional offices' performance in processing these claims, and (3) how well selected regional offices communicate with VSOs and congressional caseworkers about these claims. GAO reviewed VBA policies and procedures; visited four regional offices selected to represent a range of performance scores and claims processing volume in fiscal year 2017; and interviewed VBA headquarters officials and management and staff from the selected regional offices. GAO also interviewed VSOs and congressional caseworkers—selected for House, Senate, and bipartisan representation—to learn more about their communication with VBA.

In 2016, the Veterans Benefits Administration (VBA) centralized distribution of the disability compensation claims workload through the National Work Queue, which prioritizes and distributes claims to regional offices based on their capacity; however, there are gaps in VBA's guidance for processing claims with errors. Under the National Work Queue, multiple regional offices can work on a single claim instead of the claim remaining at one office for the duration of processing (see figure). GAO found gaps in guidance about whether a claims processor should fix an error made by another regional office, or return the claim to that office to be corrected. The former could result in missed opportunities to train staff who made the error, while the latter could result in processing delays.

VBA primarily uses timeliness and accuracy measures to assess its regional offices' performance in processing disability compensation claims, but these measures do not adequately capture performance. The timeliness measure can be skewed because it is a snapshot of how long claims have been pending at an office on the last day of the month, and does not capture performance over a period of time. The accuracy measure is attributed to the office that finishes the claim, even though 88 percent of claims completed in fiscal year 2017 were processed at more than one office. VBA officials acknowledged that these measures are limited and said the agency is exploring alternatives, but VBA has no specific plan or time frame for determining and implementing new measures. Without measures to more accurately assess regional office performance, VBA may be limited in its ability to make efficient and effective decisions.

Veterans service organizations (VSO) and staff working for Members of Congress (congressional caseworkers) interviewed by GAO were generally satisfied with regional office communication regarding disability compensation claims. However, VBA's policy on whom VSOs should contact during different points in the process did not always align with what occurs at the offices we visited or with VSO needs. This could result in VSOs not receiving consistent and timely responses from VBA. Evaluating this policy could help VBA assist VSOs in better serving veterans. In addition, congressional caseworkers GAO interviewed identified ways that communication could be improved or that additional support could be provided, such as a list of contacts at all regional offices for claim inquiries. VBA officials GAO interviewed described an open-door policy through which they may receive feedback from caseworkers, but the agency does not formally solicit periodic feedback from them. Without such feedback, the agency may miss opportunities to identify and address caseworker communication needs that could help them better serve veterans.

GAO is making five recommendations to VBA to clarify guidance for correcting errors, develop and implement measures to better assess timeliness and accuracy at regional offices, and evaluate communication with VSOs and caseworkers. The Department of Veterans Affairs concurred with GAO's recommendations.

The Department of Veterans Affairs’ (VA) Veterans Benefits  Administration (VBA) provides roughly $65 billion in disability  compensation benefits to more than 4 million veterans each year. VBA  primarily relies on its 57 regional offices to determine veterans’ eligibility  for disability compensation and, since 2010, these offices have processed  an average of more than 1 million claims each year. Prior to 2016, a  veteran’s claim was generally processed by the veteran’s local regional  office. However, in 2016, VBA implemented the National Work Queue to  centralize the distribution of claims to regional offices. Now, a claim can  be processed by multiple regional offices, and claims are distributed  based on regional office capacity. In order to monitor and improve its  claims processing performance, VBA sets goals for both processing time  and accuracy in disability determinations, and tracks this performance  across regional offices.

VBA’s disability compensation claims process can be challenging for  veterans. Therefore, independent veterans service organizations (VSO)  and staff working for Members of Congress (congressional caseworkers)  support veterans in navigating the claims process. VBA requires regional  offices to communicate with these VSOs and congressional caseworkers  as they work on behalf of veterans.

The Jeff Miller and Richard Blumenthal Veterans Health Care and  Benefits Improvement Act of 2016 includes a provision for GAO to review  VBA’s regional offices to help VBA achieve more consistent performance  in processing disability compensation claims. This report addresses (1)  how VBA manages workload and performance for the disability  compensation claims process, (2) how well VBA’s timeliness and  accuracy measures capture its regional offices’ performance in  processing these claims, and (3) how well selected VBA regional offices  communicate with VSOs and congressional caseworkers about these  claims.

To address these objectives, we reviewed VBA headquarters policies and  procedures related to management practices, performance, and  communication. We also visited four regional offices—Columbia, SC;  Montgomery, AL; St. Paul, MN; and Wichita, KS—selected to represent a  range of VBA-reported performance measure scores and claims  processing volume in fiscal year 2017. We assessed the reliability of data  on performance scores and claims volume by reviewing relevant  documentation, interviewing VBA staff, and performing electronic testing  for obvious errors in accuracy and completeness, and concluded that  these data were sufficiently reliable for the purposes of selecting regional  offices to visit. We interviewed VBA headquarters and district office  officials, and various groups of management and staff from the selected  regional offices regarding VBA and regional office management, regional  office performance measurement, and regional office communication with  VSOs and congressional caseworkers.

Specifically, to address the first objective, we reviewed VBA headquarters  documentation related to management practices for disability  compensation claims. We also reviewed VBA headquarters and district  office reports on site visits to regional offices. In addition, we reviewed  regional office-level policy and procedure documents related to managing  workload and performance from the four selected regional offices. We  discussed with VBA officials the National Work Queue and how it affects  VBA’s management of workload and performance. We did not assess the  extent to which the use of the National Work Queue affected performance  outcomes, in part, because performance changes may be attributed to  several factors, such as VBA’s move from a paper- to electronic-based  claims process, mandatory and voluntary overtime for claims processors,  and an agency-wide focus on reducing the claims backlog. In addition,  we compared management practices for identifying and correcting claims  processing errors to federal internal control standards related to the  design of control activities.

To address the second objective, we reviewed VBA’s policy and  documentation on performance measurement, such as the Director’s  Performance Plans for fiscal years 2017 and 2018, agency budget and  performance reports, and VBA-reported regional office performance data  related to timeliness and accuracy of disability compensation claims for  fiscal year 2017. We discuss the reliability of the timeliness data and the  regional office-level accuracy scores later in this report. We assessed the  reliability of the accuracy data related to the frequency of specific errors  nationwide by interviewing VBA staff, performing electronic testing for obvious errors in accuracy and completeness, and reviewing our prior  analyses of these data. We concluded that these data on errors were  sufficiently reliable for the purposes of our report. We compared VBA’s  regional office performance measures to federal internal control  standards on using quality information to achieve agency objectives.

To address the third objective, we reviewed VBA policies on  communication with VSOs and congressional caseworkers. We also  reviewed VA Office of Inspector General reports on regional offices, and  VBA headquarters reports on site visits to regional offices. In addition, we  reviewed congressional testimony from national VSOs related to  communication with regional offices. We interviewed officials from VA’s  Office of Inspector General and Office of Congressional and Legislative  Affairs. We also interviewed national officials from three congressionally  chartered VSOs, that are also recognized by VA—The American Legion,  Disabled American Veterans, and Veterans of Foreign Wars—and were  selected because of their size and the scope of veterans’ issues they  represent. During our visits to the four selected regional offices, we  interviewed 23 local VSO staff, selected for local VSOs’ availability to  meet with us during our visits. During our visits, we also interviewed 22  congressional caseworkers working for 16 Members of Congress. The  caseworkers were selected to include House, Senate, and bipartisan  representation, and for the volume of cases the caseworkers were  involved in at the regional offices we visited. We compared the regional  offices’ communication with VSOs and congressional caseworkers to  VA’s policies and federal internal control standards on communication.

We conducted this performance audit from July 2017 through October  2018 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

VA pays monthly disability compensation to veterans with service- connected disabilities (i.e., injuries or diseases incurred or aggravated  while on active military duty) according to the severity of the disability.  VBA’s Compensation Service sets policy and oversees the delivery of  disability compensation. VBA’s Office of Performance Analysis and  Integrity analyzes performance information related to claims. VBA’s Office  of Field Operations provides operational oversight to district and regional  offices. The 57 regional offices are grouped into five district offices, which  manage the regional offices in their areas. VBA staff in the Veterans  Service Centers of the regional offices process disability compensation  claims. These claims processors include Veterans Service  Representatives who gather evidence needed to determine entitlement  and review the amount of the award and authorize payment, if any, and  Rating Veterans Service Representatives who decide entitlement and the  rating percentage. Veterans may claim more than one medical  condition, and VBA assigns a rating percentage for each claimed medical  condition, as well as for the claim overall.

As shown in figure 1, after a veteran submits a claim to VBA, a Veterans  Service Representative reviews the claim and helps the veteran gather  the relevant evidence needed to evaluate the claim. Such evidence  includes the veteran’s military service records, medical examinations, and  treatment records from Veterans Health Administration medical facilities  and private medical service providers. Also, if necessary to provide  support to substantiate the claim, VA will provide a medical examination for the veteran. Once VBA has gathered the supporting evidence, a  Rating Veterans Service Representative—who typically has more  experience at VBA than a Veterans Service Representative—evaluates  the claim and determines whether the veteran is eligible for benefits and,  if so, assigns a percentage rating. A Veterans Service Representative  then determines the amount of the award, if any, and drafts a decision  notice. A senior Veterans Service Representative then authorizes the  award and releases the decision notice to the veteran following a review  of both for accuracy.

In May 2016, VBA completed implementation of the National Work  Queue—an electronic workload management initiative that prioritizes and  distributes claims across regional offices. Previously, a veteran’s claim  was generally processed from start to finish (i.e., awarding of benefits or  notification of denial) by the veteran’s local regional office of jurisdiction,  and the regional office’s workload generally depended on how many  claims were filed by veterans within its area of jurisdiction. Now, a claim  can be processed by multiple regional offices, and claims are distributed  based on regional office capacity (see fig. 2).

National Trends in Disability Compensation Claims Processing

VBA establishes national targets and tracks performance for disability  compensation claims processing. Since fiscal year 2014, national claims  processing timeliness has improved substantially, and accuracy scores  have decreased slightly, as shown in table 1. VBA’s 12-month issue- based accuracy target for fiscal year 2017 was 96 percent and its target  for fiscal year 2018 was the same. From fiscal year 2014 to 2017, VBA’s  national accuracy estimate decreased from about 96 percent to about 94  percent. In addition, VBA’s target for backlog claims—defined by VBA  as those pending for more than 125 days—for fiscal year 2017 was no  more than 15 percent of claims inventory and its target for fiscal year  2018 was no more than 21 percent of claims. In fiscal year 2017, VBA’s  reported percentage of backlog claims was 23 percent, with a reduction  from 240,443 to 70,965 total reported backlog claims from fiscal years  2014 to 2017.

Regional Office Performance Measures for Disability Compensation Claims Processing

VBA’s Office of Performance Analysis and Integrity collects a variety of  data on timeliness and accuracy, including on VBA’s claims backlog, so  that VBA can monitor regional office performance. To improve timeliness  and accuracy, and reduce the claims backlog, VBA sets performance  standards for the directors of regional offices. In fiscal year 2018, regional  office performance was assessed using two primary metrics—timeliness  (Time-in-Queue) and accuracy (12-month issue-based accuracy). Since  1999, VBA has assessed the accuracy of disability compensation claims  decisions at the national and regional office level using its Systematic  Technical Accuracy Review (STAR). With this tool, VBA reviews a  stratified random sample of completed claims, and certified reviewers use  a checklist to assess specific aspects of each claim.

Veterans Service Organizations and Congressional Caseworkers

According to VA, as of October 2017, 31 congressionally chartered VSOs  were recognized by VA under federal statute to help veterans navigate  the claims process. VSOs commonly are private nonprofit groups that  advocate without fees on behalf of veterans. VSOs employ individuals,  called veterans service officers, whose offices often are located at a VBA  regional office. Through a power of attorney, VSOs can represent  veterans before VA, and assist them and their families with disability  compensation claims, among other things. VSO staff are trained to help  veterans understand and apply for any VA benefits to which they may be  entitled, including disability compensation. In addition to helping veterans  submit claims to VBA, VSOs are allowed to communicate with VBA on  behalf of the veteran throughout the life of the claim, and are given up to  48 hours to review the claim decision before it is finalized (after the Rating  phase in figure 1 above). VSOs can have access to VBA’s electronic  claims management system to view claims status and submit claims  documents.

According to a Congressional Research Service report, as of March 2016,  919 congressional caseworkers were working for constituents on a variety  of issue areas, including veterans’ disability compensation claims. Also  according to the report, congressional caseworkers cannot legally  represent veterans, but with a privacy release form from the veteran, VBA  may respond to a congressional inquiry. According to VA officials,  congressional caseworkers can then obtain certain claim-related  information from VA, such as the status of the veteran’s claim. VA’s  guidance on “special controlled correspondence” governs VBA’s  communication with congressional caseworkers, including required time  frames for responding to congressional inquiries. Congressional  caseworkers generally work out of Congressional Members’ state and  district offices.

VBA Manages Workload and Performance through Established Processes, but Guidance for Claims with Errors Has Gaps

VBA Allocates Claims Workload across Offices Based on Their Capacity, but Guidance for Processing Claims with Errors Has Gaps

The National Work Queue, which VBA uses to distribute disability  compensation claims, was designed to even out the differences in claims  workload across regional offices by having multiple offices complete parts  of a claim and allocating claims based on each office’s capacity. For  example, as shown in figure 3, in fiscal year 2017, about 88 percent of all  disability compensation claims were processed by more than one office,  and over 75 percent were processed by three or more offices. This  distribution method is intended to keep all offices working at their  capacity, regardless of the volume of claims filed by veterans in each  region. While VBA officials stated that they had initially planned to  continue to have a majority of claims processed at veterans’ local regional  offices, after implementation of the National Work Queue they determined  that the system operates more effectively if veteran location is a lower  priority factor for claims distribution. Thus, very few claims are processed  entirely at a veteran’s local regional office, unless the veteran has a  documented hardship that may necessitate expediting the claim or face- to-face interaction.

VBA officials added that the National Work Queue formula distributes  claims based on VBA priorities. For example, VBA prioritizes claims for  veterans with documented hardships (e.g., terminal illness, financial  hardship). In addition, the National Work Queue formula takes into  account the length of time since the claim was received and prioritizes  backlog claims—defined by VBA as claims that have been open for more  than 125 days.

Once the National Work Queue allocates claims to a regional office, the  office has some discretion in managing the distribution of claims to its  staff and managing the claims review process. For example, while VBA  determines how the claims workload is allocated across offices, regional  office managers decide which claims within the office’s queue to work  first, how to program the office’s queue for distributing claims to individual  claims processors’ electronic work queues, and whether any changes to  this distribution are needed throughout the day. Regional office managers  at each of the four offices we visited reported using VBA’s timeliness  goals and daily data on claims processing timeliness to prioritize claims.  Managers at the offices we visited also described additional strategies to  manage their work queue, including:

At two of the four offices we visited, managers said that they provide a  list of claims to claims processors to prioritize, such as those that are  older or have been in the office’s work queue for multiple days.

Managers at one office said that they manually alter individual claims  processors’ electronic work queues so that older claims are  processed first.

Managers at one office stated that because they instruct claims  processors to focus on meeting timeliness targets for the office, all  claims are worked within a few days; thus, they encourage their staff  to focus on meeting the office timeliness goals rather than requiring  them to work the claims in their queue in a specified order.

VBA officials acknowledged that regional office managers may have  different strategies for managing workload, but noted that all offices are  expected to respond to VA national priorities—such as decreasing the  claims backlog—while also meeting their individual office performance  goals.

While VBA officials noted that having discretion in workload management  can be beneficial, such discretion can also lead to inconsistent handling  of the claims workload. In particular, we found gaps in guidance for  managing deferrals—actions taken by claims processors in VBA’s  electronic claims management system when they identify claims errors  that occurred earlier in the claims process. The deferral process began  with the National Work Queue since claims were, for the first time,  routinely being processed by multiple regional offices. Through deferrals,  when claims processors identify errors in a claim, they can use the  National Work Queue to return the claim for correction to the office that  made the error. According to VBA data, in fiscal year 2017, VBA claims  processors deferred claims in 450,305 instances, which represented  almost 4 percent of the total disability claims processing work  completed.

While VBA officials said that claims processors who find errors are  generally expected to defer a claim, managers and claims processors at  the regional offices we visited had different perspectives regarding when  Veterans Service Representatives should do this. At all four of the  regional offices we visited, managers and claims processors said that  they generally would not defer a claim if the error could be corrected and  the claim moved forward. At one regional office, managers and claims  processors said that they would log a deferral in the electronic claims  management system, so the error would be tracked and the previous  claims processor could be notified and trained, but that they would also  correct the error themselves to move the claim forward.

VBA provides some guidance to Rating Veterans Service  Representatives regarding the circumstances in which they should defer  claims, but does not have corresponding guidance for Veterans Service  Representatives. However, according to our analysis of VBA data from  fiscal year 2017, more than 75 percent of deferrals are logged during the  Initial Development, Supplemental Development, Award, or Authorization  phases—when Veterans Service Representatives are typically processing  claims. Existing guidance for Veterans Service Representatives on  deferrals in the National Work Queue Playbook and other documents  focuses on the process for deferring a claim in the electronic claims  management system, rather than on situations that merit a deferral.  Specifically, VBA does not provide guidance on when Veterans Service  Representatives should defer a claim or consider other options, such as  correcting the error and moving the claim forward, with or without a  deferral. VBA officials stated that the policy regarding when to defer  claims is not prescriptive—and they do not plan to provide additional  guidance—because they want to allow regional offices the discretion to  decide what action is best for the veteran. However, federal internal  control standards state that agencies should design control activities to  achieve objectives and respond to risks. For example, a control activity  that is performed routinely and consistently generally is more precise than  one performed sporadically. As such, deferrals may not serve as an  effective control without being used consistently across VBA’s regional  offices.

VBA’s lack of guidance on when to defer claims may lead to delays for  veterans and missed opportunities to train individuals who make errors. In  some cases, differences in regional office practices for when to defer  claims may lead to situations in which claims that could move forward are  instead sent back to the previous office, causing unnecessary delays for  veterans. In addition, we heard from managers or claims processors at  three offices we visited that claims may not always be deferred for  legitimate reasons and that the ability to defer claims may create  incentives for employees to defer a claim based on an insignificant error if  they want to avoid working on a complex claim. In other cases, more  significant errors may end up being fixed at a regional office without  providing feedback to the office that made the mistake. While the practice  of fixing the error rather than deferring the claim may keep the claim  moving for the veteran, it also means that claims processors who make  errors may repeat the same mistakes in the future.

VBA Sets Regional Office Performance Goals and Individual Expectations and Has Developed Processes for Managing Performance

VBA sets regional office goals and individual claims processor  expectations that align with national efforts to increase timeliness and  accuracy of claims decisions. VBA holds regional offices accountable for  meeting performance goals through the Director’s Performance Plan. For  disability compensation claims in fiscal year 2018, VBA assessed regional  office performance using the Time-in-Queue and 12-month issue-based  accuracy measures. VBA has developed processes and tools for  communicating performance information to regional offices and for  identifying common errors. For example, VBA sets timeliness goals for  regional offices and generates daily claims processing timeliness data for  each office. At the regional offices we visited, we observed that VBA  displays these data on monitors so that managers and employees can  see how their office is performing on a daily basis. In addition, VBA has  created performance reporting tools that allow regional office managers,  claims processors, and various VBA workgroups to download regional  office performance information and analyze office performance issues at  their discretion.

At the regional offices we visited, quality review teams analyze claims  processing errors made by their employees, such as those identified in  STAR reviews and through the deferral process. Based on common  mistakes they identify, quality review staff at all four offices we visited said  that they incorporate topics related to the errors into training sessions, or  provide direct coaching to individual employees. VBA also conducted an  In-Process Review pilot from November 2017 through May 2018 at  selected regional offices. The pilot involved a quality review for two  phases of the claims process. The purpose of the pilot was for employees  to learn from and correct mistakes in a non-punitive setting while the  claim was being processed. VBA officials reported that VBA discontinued  the pilot in May 2018—prior to its scheduled completion date—because  the pilot was not demonstrating the anticipated benefit of reducing the  number of errors at pilot offices that resulted in deferrals.

VBA also develops practices at the national level for managing individual  employee performance and, in some cases, provides regional office  managers with discretion for implementing those practices. In support of  the regional office performance standards, VBA sets individual employee  performance standards in the following five areas: (1) quality of work; (2)  timeliness of corrective actions and responsiveness to workload  assignments; (3) production (i.e., the number of transactions, or tasks,  completed within the assessment period); (4) completion of training; and  (5) organizational support. The production standards include a goal for  the number of credits, or points, that employees are expected to earn  during each pay period for their work activities. According to VBA  officials, regional office managers are held accountable for providing  feedback to employees on a regular basis and addressing performance  deficiencies appropriately and in a timely manner. In addition, according  to VBA officials, VA’s policy allows regional office managers “broad  discretion” in determining when a performance deficiency exists.

Employee performance incentive programs, which provide monetary  awards to top performers in each regional office, are also managed at the  national level. However, within regional offices, some managers told us  that they also occasionally provide small incentives or celebrations to  show appreciation for staffs’ contributions.

VBA’s Timeliness and Accuracy Measures Do Not Adequately Reflect Regional Office Performance for Disability Compensation Claims Processing

Regional Office Timeliness Measure Does Not Capture Performance over a Period of Time

VBA uses Time-in-Queue—the average number of business days that  claims have been pending at a regional office—to measure overall  regional office timeliness for processing disability compensation claims.  Time-in-Queue is measured separately for each phase of the claims  process—Initial Development, Supplemental Development, Rating,  Award, and Authorization—and VBA has established timeliness goals for  each of these phases. VBA holds regional offices accountable for meeting  timeliness goals through the Director’s Performance Plan, which rates  offices as successful if they meet Time-in-Queue standards for each  phase of the claims process in 10 out of 12 months. For this purpose, the  measure is a snapshot on the last day of each month that shows how  long, on average, claims have been pending at each office; however, it  does not capture regional office performance over a period of time.  Consequently, Time-in-Queue can provide a skewed picture for a period  of time, depending on the work that is assigned to the office toward the  end of the month and the speed with which claims are processed during  that limited time period.

Moreover, according to VBA officials, the agency used Time-in-Queue  scores and additional factors—such as space considerations and training  capacity—to determine the amount of new resources to allocate to its  regional offices in May 2017, and the agency will continue to consider  such performance information when allocating resources in the future.

However, federal internal control standards state that agencies should  use quality information to achieve objectives. For example, an agency  should obtain data from reliable sources in a timely manner and based on  identified requirements, and reliable sources are those that provide data  that are reasonably free from error and bias and faithfully represent what  they purport to represent. In addition, our prior work has shown that  practices for improving the usefulness of performance data include using  new methods of measurement to address data limitations, such as Time- in-Queue only capturing performance as a snapshot on 1 day.

VBA officials acknowledged that the Time-in-Queue performance  measure does not reflect the complete timeliness of a regional office.  These officials said that the agency is exploring adding a Time-to-Exit- Queue measure that could capture regional office timeliness over a  period of time. For example, Time-to-Exit-Queue could measure the  timeliness of all claims processing work completed throughout the month  instead of work pending on the last day of the month. However, VBA has  not yet completed the development of the Time-to-Exit-Queue  performance measure. VBA has also not determined whether or when it  will replace or supplement Time-in-Queue with a new primary metric— Time-to-Exit-Queue or something else—to measure regional office  timeliness. Until VBA implements a new measure to more fully assess  regional offices’ timeliness, the agency will not have a complete picture of  regional office performance over time, which could impair decision- making related to regional office performance, such as decisions about  targeting resources to high- or low-performing offices.

A Regional Office’s Accuracy Score Does Not Always Reflect the Work Completed in That Office

VBA uses the STAR 12-month issue-based accuracy score to measure  regional office accuracy in processing disability compensation claims, but  this score could provide a misleading picture of an office’s performance.  VBA’s accuracy measure attributes the accuracy of sampled claims to the  regional office that finishes the claim even though, under the National  Work Queue, that office may not have done all of the work on the claim.  In fiscal year 2017, about 88 percent of all disability compensation claims were processed by more than one office, and about 43 percent were  processed by five or more offices, as shown earlier in figure 3. As a  result, the scores attributed to each office may not reflect the true  accuracy of the office’s work. In addition, any errors made by other offices  earlier in the claims process would not be reflected in those offices’  accuracy scores. Therefore, the current regional office accuracy measure  does not reflect the accuracy of each office’s work and may skew the  score negatively or positively. According to VBA officials, the agency uses  issue-based accuracy scores, among other things, to determine how to  allocate resources to regional offices. However, federal internal control  standards state that agencies should use quality information to achieve  objectives. For example, an agency obtains data from reliable sources in  a timely manner based on identified requirements, and reliable sources  provide data that are reasonably free from error and bias and faithfully  represent what they purport to represent. In addition, our prior work has  shown that practices for improving the usefulness of performance data  include using new methods of measurement to address data limitations.

VBA officials said that they recognize the limitations of the agency’s  regional office accuracy measure, but VBA officials also said it is  reasonable to hold the office that completes the claim accountable  because Veterans Service Representatives are responsible for checking  for errors in the claims process before completing the claim during the  Authorization phase. However, according to VBA officials, some areas on  VBA’s accuracy checklist—such as whether the claimed conditions were  correctly granted or denied, and whether the correct percentage  evaluation was assigned—are beyond the scope of the Veterans Service  Representatives’ review or qualifications. These tasks are completed by  Rating Veterans Service Representatives. In fiscal year 2017, these two  areas—whether the claimed conditions were correctly granted or denied,  and whether the correct percentage was assigned—accounted for an  estimated 28 percent of all errors nationwide. In addition, these two areas  ranged from an estimated low of about 13 percent (5 of 40) of all errors  attributed to one regional office to an estimated high of about 55 percent  (16 of 29) of all errors attributed to another regional office. In addition,  while VBA officials said that it is reasonable to hold the office that  completes the claim accountable for errors, officials also said that when  STAR errors are identified, only the regional offices that actually made the errors are told about them in order to improve staff performance. This  suggests that VBA does not view the Veterans Service Representative  who completes the claim as fully responsible for all errors in the claims  process.

According to VBA officials, the agency has been exploring the  development of a new accuracy measure that would enable it to assign  error scores to the offices that actually made the errors. For example,  VBA is considering using the STAR reviews to produce a claims phase- based score that would attribute the accuracy of individual phases of the  claims process to the offices completing those phases. However,  according to VBA officials, sampling by each phase of the claims process  would be more complicated than the current system of sampling by  regional office and would require additional staff. In addition, the agency  is also exploring leveraging its existing Individual Quality Reviews— currently used to assess the accuracy of individual staff’s work—to create  individual regional office accuracy scores. VBA officials added, however,  that there are challenges with converting these individual accuracy scores  to office scores, such as calculating scores by claims phase instead of by  employee position since an employee may conduct work in various  phases. VBA has not determined which alternate measure, if any, to use,  and does not have a timeline for addressing the challenges it has  identified with the alternate measures being considered, or for  implementing a new accuracy measure. Until VBA implements a new  measure to assess regional offices’ accuracy, it will not have an accurate  picture of individual regional offices’ performance, which could impair  decision-making, such as targeting resources to high- or low-performing  offices.

Stakeholders Were Generally Satisfied with Communication at Selected Regional Offices, but VBA’s Communication Policies Are Applied Inconsistently

Selected Veterans Service Organizations Were Generally Satisfied with Access to Regional Office Staff, but VBA’s Communication Policy and Practice Are Not Aligned

Despite being generally satisfied with regional office communication,  VSOs we spoke with also expressed some frustrations. VSOs we spoke  with at all four offices reported generally being able to contact someone to  answer their questions. Moreover, VBA staff we spoke with reported  being flexible in communicating with VSOs in the manner in which they  preferred. In addition, Compensation Service and Benefits Assistance  Service site visit reports found that VSOs are generally satisfied with  regional office communication. However, VSOs at all four offices we  visited expressed some frustrations with communication, but they varied  some by offices. Examples of communication issues included:

Diminished contact. VSOs noted that the National Work Queue  reduced personal relationships and collaboration between VSOs and  regional office staff since claims are no longer fully processed at the  local regional office, and therefore VSOs can no longer simply walk  across the office to discuss a claim.

Delayed responses. VSOs said there sometimes are delays in  receiving responses from regional offices, with staff taking different  lengths of time to respond to an inquiry, or not responding at all.  Sometimes, once VSOs receive a response, the claim is no longer  being processed at the regional office they contacted, so the response  is no longer useful.

Decreased notice of activity. VSOs said that with the advent of  electronic claims processing, they no longer receive paper copies of  disability ratings and other documents that VBA sends to the veteran.  VSOs have access to such information in VBA’s electronic claims  management system, but the system does not notify them when VBA  has sent documents to the veteran, such as requests for information  and evidence. VSOs said it is time-consuming for them to proactively  monitor a large number of veterans’ electronic claims files for new  documents.

VSOs may communicate with a regional office throughout the life of a  claim for various purposes and, according to VBA officials, regional  offices generally have discretion in establishing local policies for handling  VSO questions or inquiries. One exception to this local discretion is  during the 48-hour review period when VSOs can review a completed  disability rating before it is finalized. A November 2016 VBA policy states  that during the 48-hour review period, VSOs may contact a regional  office’s Change Management Agent. The policy also states that VSOs  should not contact the Change Management Agent for claim status  updates, evidence submission, or any other type of inquiry unrelated to a  rating decision discrepancy. According to VBA officials, the policy to  contact the Change Management Agent during the 48-hour review period  was intended to streamline the inquiry process for VSOs, provide  consistent responses to them, and minimize disruptions for claims  processors. The previous policy required VSOs to first contact the Rating  Veterans Service Representative before the Change Management Agent  during the 48-hour review period.

VSOs at three offices we visited reported contacting the Change  Management Agent for inquiries during the 48-hour review period, but  also reported contacting the Change Management Agent at other points  during the claims process. VSOs at all four offices we visited also  reported contacting other staff, such as claims processors or their  supervisors at their local regional offices, during the 48-hour review  period, unrelated to the Change Management Agent’s availability or a  particular type of claim, which VBA officials stated were reasons for which  VSOs might contact an alternate VBA official. Federal standards for  internal control state that an agency should externally communicate the  necessary quality information to achieve an entity’s objectives, for  example, communicating with external parties using established reporting  lines, and periodically evaluating its communication methods.

VBA officials told us that the November 2016 policy was intended to  address communication during the 48-hour review period, and that  regional office discretion for communication with VSOs outside of this  period was still in place, including contacting Change Management  Agents if regional offices determined this was best. However, regional  offices and VSOs do not consistently implement this policy. Moreover, the  policy states that VSOs are not to contact Change Management Agents  for claim status updates, evidence submission, or any other question  unrelated to a rating decision discrepancy. These types of inquiries  generally occur outside of the 48-hour review period, so this portion of the  policy conflicts with VBA officials’ description of regional office discretion  for communication with VSOs throughout the life of a claim.

Although VSO communication with Change Management Agents did not  always appear to match VBA’s policy for communication during or outside  of the 48-hour review period, VSOs we spoke with seemed to value  regional offices’ flexibility in communicating with them. However, it is  possible that the policy’s lack of clarity or inconsistent application could  contribute to communication frustrations for VSOs, and that changes to  either the policy or its enforcement could better serve VSOs and regional  office staff. Evaluating its regional office communication policy with VSOs  and ensuring that the policy is clear, that it aligns with regional offices’  practices, and that it effectively meets VSOs’ communication needs,  could help VBA ensure that it is providing timely and consistent  responses to VSOs on behalf of the veterans they represent, while  minimizing disruptions to regional office staff. Such alignment could be  achieved either by adjusting the communication policy or better enforcing  the existing policy.

Selected Congressional Caseworkers Were Satisfied with Communication with Regional Offices, but VBA’s Communication Was Not Always Timely or Accurate

Congressional caseworkers we spoke with at all four offices we visited  were satisfied with regional office communication regarding disability  compensation claims, though some regional office responses were not  timely or accurate, according to VA Inspector General reports. VBA has  congressional liaisons at each of its regional offices to answer inquiries  from congressional caseworkers. Caseworkers generally contact the VBA  liaison at their local regional office when they inquire about claims— whether the claims are being processed at the local regional office or  another regional office. Caseworkers may also contact the VBA liaison at  the office where the claim is being processed once they find out from VBA  where that is. According to regional office officials at the four offices we  visited, most congressional inquiries received at the regional offices are  by email or phone, although some are by regular mail; the congressional  inquiries are most often regarding the status of a veteran’s claim.

While caseworkers we spoke with were satisfied with their communication  with regional offices, VA’s Office of Inspector General found that in some  instances, VBA regional offices had not provided timely or accurate  responses to special controlled correspondence, which includes  congressional inquiries. According to VBA guidance on special  controlled correspondence in fiscal year 2017, VBA liaisons are to  respond to caseworkers’ inquiries within 5 business days with a full or  interim response, for example. During its inspections of regional offices  during fiscal year 2017, the Office of Inspector General found that some  offices had not provided interim responses within 5 business days and, in  a few cases, had provided inaccurate responses. At some offices, the  Office of Inspector General made recommendations for improving  regional offices’ responses to inquiries and, according to its reports,  regional offices planned and implemented changes, such as providing  additional training to staff and improving oversight of correspondence.

Caseworkers we spoke with at three offices we visited identified ways that  regional offices could improve communication with them or ways that  VBA could provide them with additional information or support. For  example, while caseworkers generally contact their local regional office  with inquiries, caseworkers at two offices said that a regularly updated  contact list of VBA liaisons at all VBA regional offices could be helpful so  that they can immediately contact another regional office if they learn that  a claim is being processed there, or if their local VBA liaison is unable to  provide sufficient specifics on a claim. Some of these caseworkers  suggested that the list could either be posted to a non-public website or  sent to VBA regional offices to distribute to local caseworkers. According  to VBA officials, the agency does maintain a list of regional office VBA  liaisons, and updates it quarterly. The list is provided upon request, both  electronically and in hard copy, and caseworkers frequently request the  list, according to VBA officials. However, the caseworkers we spoke with  at all four offices we visited were not aware of this list.

In September 2017, VBA developed an online toolkit for congressional  caseworkers to better assist them in serving their veteran  constituencies. The toolkit webpage provides a central location for  caseworkers to quickly locate information regarding available VA benefits  and services. For example, the toolkit provides a link to a description of  the disability compensation program and how to apply for benefits. VBA  officials reported that in September 2017, they provided information on  the toolkit to VA’s Office of Congressional and Legislative Affairs for  distribution to congressional staff. However, caseworkers and VBA  liaisons at all four offices we visited were not aware of this online toolkit,  and caseworkers we spoke with at two offices we visited said that it could  have been useful to them if they had been aware of it or if it had  additional elements, such as regional office expectations for caseworker  inquiries.

According to VBA officials, they have not received any feedback on the  toolkit beyond that initially provided by another VA office. This could be, in  part, because VBA does not have an outreach mechanism to actively  obtain perspectives from congressional caseworkers on their  communication with regional offices or their information or support needs,  or to determine whether the findings from the Office of Inspector General  are typical across regional offices. The Office of Inspector General  stopped performing its reviews of regional offices—including evaluations  of communication with congressional caseworkers—in fiscal year 2017 to  focus its efforts on VBA-wide audits, so this information is no longer  available to VBA. Federal standards for internal control state that an  agency should externally communicate the necessary quality information  to achieve an entity’s objectives, for example, selecting the appropriate  methods to communicate externally, and periodically evaluating its  methods of communication so that the agency has the appropriate tools  to communicate quality information outside the agency.

VBA officials reported an open-door policy in which caseworkers can  share concerns and requests as needed, and said that a formal outreach  mechanism is not necessary. Although caseworkers can approach  regional office staff with ideas for improvement, this informal mechanism  is not a consistent process and does not facilitate candid feedback, nor  does it include documentation of potential improvements and actions  taken. By creating an outreach mechanism to solicit periodic feedback  from congressional caseworkers, VBA could streamline the inquiry  process and enable them to provide more accurate and timely information  to veterans.

VBA’s National Work Queue has been in place for more than 2 years and  provides opportunities for a higher level of service to veterans. However,  with claims moving among regional offices, the individual performance of  regional offices remains critical to VBA’s success. For example, regional  offices’ inconsistent use of deferrals when claims processors identify  errors could unnecessarily delay the decision on a veteran’s claim or  prevent staff from receiving needed training. In addition, VBA has  developed several practices to assess performance at regional offices,  but some of this information could be of limited use if the agency  continues using its existing measures. Specifically, VBA’s two primary  performance measures for regional offices do not allow the agency to  adequately measure claims timeliness and accuracy. Finally,  communication with VSOs and congressional caseworkers could be  improved by clarifying the VSO communication policy and aligning it with  practice and VSO needs, and conducting caseworker outreach in order to  provide more consistent and timely information to VSOs and  caseworkers. Without these improvements, VSOs and caseworkers may  not be able to serve veterans in as timely a manner as possible.

We are making the following five recommendations to VBA:

The Under Secretary for Benefits should clarify how Veterans Service  Representatives should handle claims when they identify an error,  including when to defer a claim and when to correct the error on their  own. (Recommendation 1)

The Under Secretary for Benefits should develop and implement a  new regional office performance measure that allows it to better  assess each regional office’s timeliness over a period of time.  (Recommendation 2)

The Under Secretary for Benefits should develop and implement a  new regional office performance measure that allows it to better  measure the accuracy of each regional office’s work.  (Recommendation 3)

The Under Secretary for Benefits should evaluate its policy for  regional office communication with VSOs to ensure that it is clear, that  it aligns with practice, and that it meets the communication needs of  VSOs. (Recommendation 4)

The Under Secretary for Benefits should develop and implement a  mechanism to obtain periodic feedback from congressional  caseworkers on their communication with regional offices regarding  claims and needed information or support. (Recommendation 5)

We provided a draft of this report to the Department of Veterans Affairs  for review and comment. VA provided written comments, which are  reproduced in appendix I. VA concurred with all of our recommendations  and described VBA’s plans for taking action to address them. Regarding  Recommendation 1, VA stated that VBA is working to clarify guidance to  regional offices for handling claims when errors are identified. Regarding  Recommendations 2 and 3, VA stated that VBA is working to develop and  implement new performance measures for regional office timeliness and  accuracy. Regarding Recommendation 4, VA stated that VBA will review  and enhance its policy for communication with VSOs. Regarding  Recommendation 5, VA stated that VBA will review existing practices on  support for congressional caseworkers, and develop and implement  mechanisms to strengthen this support. VA also reported that regional  office managers have been directed to meet at least quarterly with  congressional caseworkers to gather feedback and resolve issues. If VBA  can demonstrate that it is consistently using such feedback mechanisms  across regional offices to identify and address caseworker concerns, this  will meet the intent of our recommendation.

We are sending copies of this report to the appropriate congressional  committees, the Secretary of Veterans Affairs, and other interested  parties. In addition, the report is available at no charge on the GAO  website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-7215 or curdae@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last   page of this report. GAO staff who made key contributions to this report  are listed in appendix II.

Appendix I: Comments from the Department of Veterans Affairs

Appendix II: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Nyree Ryder Tee (Assistant  Director), Rebecca Kuhlmann Taylor (Analyst-in-Charge), Justin  Gordinas, and Martin E. Scire made significant contributions to the report.  Also contributing to the report were James E. Bennett, Alex Galuten,  Benjamin T. Licht, Liam O’Laughlin, David Perkins, Almeta Spencer,  Walter K. Vance, and Kathleen van Gelder.