VA Disability Benefits: Opportunities Exist to Better Ensure Successful Appeals Reform

VA's disability compensation program pays cash benefits to veterans with disabilities connected to their military service. In recent years, the number of appeals of VA's benefit decisions has been rising. For decisions made on appeal in fiscal year 2017, veterans waited an average of 3 years for resolution by either VBA or the Board, and 7 years for resolution by the Board. The Veterans Appeals Improvement and Modernization Act of 2017 makes changes to VA's current (legacy) appeals process, giving veterans new options to have their claims further reviewed by VBA or appeal directly to the Board. The Act requires VA to submit to Congress and GAO a plan for implementing a new appeals process, and includes a provision for GAO to assess VA's plan.

This testimony focuses on the extent to which VA's plan: (1) addresses the required elements in the Act, and (2) reflects sound planning practices identified in prior GAO work. GAO's work entailed reviewing and assessing VA's appeals plan and related documents against sound planning practices, and soliciting VA's views on GAO's assessments.

The Department of Veterans Affairs' (VA) plan for implementing a new disability appeals process while attending to appeals in the current process addresses most, but not all, elements required by the Veterans Appeals Improvement and Modernization Act of 2017 (Act). VA's appeals plan addresses 17 of 22 required elements, partially addresses 4, and does not address 1. For example, not addressed is the required element to include the resources needed by the Veterans Benefits Administration (VBA) and the Board of Veterans' Appeals (Board) to implement the new appeals process and address legacy appeals under the current process. VA needs this information to certify, as specified under the Act, that it has sufficient resources to implement appeals reform and make timely appeals decisions under the new and legacy processes.

VA's appeals plan reflects certain sound planning practices, but it could benefit from including important details in several key planning areas:

Performance measurement : VA's plan reflects steps taken to track performance, but could articulate a more complete and balanced set of goals and measures for monitoring and assessing performance on a range of dimensions of success. Specifically, the plan reports that VA is developing a process to track timeliness of the new and legacy processes. However, contrary to sound planning practices, the plan does not include timeliness goals for all five appeals options available to veterans, does not include goals or measures for additional aspects of performance (such as accuracy or cost), and does not explain how VA will monitor or assess the new process compared to the legacy process. Unless VA clearly articulates a complete and balanced set of goals and measures, it could inadvertently incentivize staff to focus on certain aspects of appeals performance over others or fail to improve overall service to veterans.

Project management : VA's plan includes a master schedule for implementing the new appeals plan; however, this schedule falls short of sound practices because it does not include key planned activities—such as its pilot test of two of the five appeals options. In addition, the schedule does not reflect other sound practices for guiding implementation and establishing accountability—such as articulating interim goals and needed resources for, and interdependencies among, activities. Unless VA augments its master schedule to include all key activities and reflect sound practices, VA may be unable to provide reasonable assurance that it has the essential program management information needed for this complex and important effort.

Risk assessment : VA has taken steps to assess and mitigate some risks related to appeals reform by, for example, pilot testing two of the five appeals options through its Rapid Appeals Modernization Program (RAMP). However, as designed, RAMP does not include key features of a well-developed and documented pilot test. For example, VA has not articulated how it will assess RAMP before proceeding with full implementation. In addition, RAMP is not pilot testing three options and, as a result, VA will not have data on the extent to which veterans will appeal directly to the Board when given the option. Unless VA identifies and mitigates key risks associated with implementing a new process, VA is taking a chance that untested aspects will not perform as desired.

In its forthcoming report, GAO is considering recommending that VA: fully address all legally required elements in its appeals plan, articulate how it will monitor and assess the new appeals process as compared to the legacy process, augment its master schedule for implementation, and more fully address risk.

I am pleased to be here today to discuss our review of the Department of  Veterans Affairs’ (VA) plan for implementing a new disability appeals  process while processing pending appeals under the current, or legacy,  process. VA provides cash benefits to veterans for disabling conditions  incurred in or aggravated by military service, paying an estimated $67  billion to about 4.3 million veterans in fiscal year 2016. If veterans are  dissatisfied with VA’s initial decision they can appeal—first to the  Veterans Benefits Administration (VBA) and then, if not satisfied there, to  the Board of Veterans’ Appeals (Board), a separate agency within VA. In  recent years, the number of appeals of VA’s benefit decisions has been  rising. For appeals resolved in fiscal year 2017, veterans waited an  average of 3 years from the date they initiated their appeal to resolution  by either VBA or the Board—and a cumulative average of 7 years for  appeals resolved by the Board. Due in part to the challenges VA faces  managing large workloads and deciding disability claims and appeals in a  timely manner, GAO in 2003 designated VA disability compensation and  other federal disability programs as one of the government’s highest  management risks.

In a March 2017 report, we examined VA’s approaches to address  challenges it identified as contributing to lengthy appeals processing  times—including VA efforts to hire staff, propose reform legislation to  Congress in April 2016, and upgrade its information technology  systems—and the extent to which those approaches were consistent with  sound planning practices. We made five recommendations to improve  VA’s ability to implement its proposed appeals process reform while  addressing a growing appeals workload. VA agreed in principle with our  five recommendations, which remain open as of January 2018. We  recommended, in essence, that VA develop: (1) a detailed workforce  plan, (2) a complete schedule of information technology (IT) updates, (3)  better estimates of future workloads and timeliness, (4) a robust plan for  monitoring appeals reform, and (5) a strategy for assessing whether the  new process improves veterans’ experiences over the current process.  We also suggested that Congress require VA to pilot test appeals reform.

Enacted on August 23, 2017, the Veterans Appeals Improvement and  Modernization Act of 2017 (the Act) will make changes to VA’s appeals  process. Specifically, the Act replaces the current appeals process with a  process that gives veterans various options to have their claim reviewed  further by VBA or to bypass VBA and appeal directly to the Board. The  Act also requires VA to submit a comprehensive plan for implementing  the new appeals process and processing legacy appeals (appeals that  remain pending in the current process prior to fully implementing appeals  reform) to the appropriate committees of Congress and GAO. The Act  delineates the required elements of this plan, and required VA to submit  its plan within 90 days of enactment. VA submitted its plan on November  22, 2017. The Act also includes a provision for GAO to assess VA’s  appeals plan, including whether the plan comports with sound planning  practices and/or contains gaps.

My testimony today is based on work being conducted for a report that  GAO expects to issue pursuant to the Act. This statement focuses on the  extent to which VA’s appeals plan (1) addresses the required elements in  the Act; and (2) reflects sound planning practices identified in prior GAO  work.

To assess the extent to which VA’s plan addresses the required elements  in the Act, we identified the required elements for VA’s comprehensive  plan under section 3(a) and (b) of the Act; compared the required  elements against VA’s appeals plan and supplemental materials VA  provided at our request; and made a preliminary determination as to  whether VA’s plan addressed, partially addressed, or did not address  each element. We then shared the results of this review with VA officials,  and considered their comments in arriving at our assessment.

To address the extent to which VA’s plan reflects sound planning  practices, we compared the appeals plan and supplemental materials  against relevant sound planning practices and other criteria identified in  our prior work. Our analyses focused on the information and elements  VA presented in its appeals plan and supplemental materials provided by  VA, rather than auditing the underlying information. We conducted the  work for this statement in accordance with generally accepted  government auditing standards. Those standards require that we plan  and perform the audit to obtain sufficient, appropriate evidence to provide  a reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives.

VA’s Disability Compensation Claims Process

VA’s process for deciding veterans’ eligibility for disability compensation  begins when a veteran submits a claim to VA. The veteran submits his or  her claim to one of VBA’s 56 regional offices, where staff members assist  the veteran by gathering additional evidence, such as military and  medical records, that is needed to evaluate the claim. Based on this  evidence, VBA decides whether the veteran is entitled to compensation  and, if so, how much.

A veteran dissatisfied with the initial claim decision can generally appeal  within 1 year from the date of the notification letter VBA sends to the  veteran. Under the current appeals process (now referred to by VA as the  legacy process), an appeal begins with the veteran filing a Notice of  Disagreement. VBA then re-examines the case and generally issues a  Statement of the Case that represents its decision.

A veteran dissatisfied with VBA’s decision can file an appeal with the  Board. In filing that appeal, the veteran can indicate whether a Board  hearing is desired. Before the Board reviews the appeal, VBA prepares  the file and certifies it as ready for Board review. If the veteran requests a  hearing to present new evidence or arguments, the Board will hold a  hearing by videoconference or at a local VBA regional office. The Board’s  members, also known as Veterans Law Judges, review the evidence and  either issue a decision to grant or deny the veteran’s appeal or refer (or  remand) the appeal back to VBA for further work.

The 2017 Act made changes to VA’s legacy appeals process that will  generally take effect no earlier than February 2019, which is  approximately 18 months from the date of enactment. According to its  appeals plan, VA intends to implement the Act by replacing the current  appeals process with a process offering veterans who are dissatisfied  with VBA’s decision on their claim one of five options: two of those  options afford the veteran an opportunity for an additional review of VBA’s  decision within VBA, and the other three options afford them the  opportunity to bypass additional VBA review and appeal directly to the  Board.

Under the new appeals process, the two VBA options will be:  1.  Request higher-level review: The veteran asks VBA to review its  initial decision based on the same evidence but with a higher-level  official reviewing and issuing a new decision.  2.  File supplemental claim: The veteran provides additional  evidence and files a supplemental claim with VBA for a new  decision on the claim.

The three Board options will be:  3.  Request Board review of existing record: The veteran appeals  to the Board and asks it to review only the existing record without  a hearing.  4.  Request Board review of additional evidence, without a  hearing.  5.  Request Board review of additional evidence, with a hearing.

The Act also requires VA to submit to the appropriate committees of  Congress and GAO, within 90 days of the date of enactment, a  comprehensive plan for (1) processing appeals under the legacy process  until there are no more to process, (2) implementing the new appeals  process, (3) processing of claims under the new appeals process in a  timely manner, and (4) monitoring implementation of the new appeals  process. In addition to these four broad elements, the Act lists 18  elements required to be included in the plan that relate to, among other  things:  staffing, information technology (IT), and other resources required to  implement the plan; estimated timelines for hiring and training VA employees; and a description of risks associated with each element of the plan.

The Act also includes a provision for GAO to assess the plan within 90  days after VA submits it.

The Act also requires VA to provide progress reports to the appropriate  committees of Congress and GAO at least once every 90 days (starting  after VA submits its plan), until the date the Act’s legal changes to the  appeals process generally go into effect and then at least once every 180  days after this date for 7 years.

Rapid Appeals Modernization Program (RAMP)

The Act also authorized VA to carry out a program to test any  assumptions relied upon in developing its comprehensive plan and test  the feasibility and advisability of any facet of the new appeals process. In  its appeals plan, VA reported its decision to pilot test two of the five new  options by allowing veterans with pending appeals in the legacy process  (known as legacy appeals) to elect the VBA supplemental claim or the  higher-level review options beginning in November 2017. This program,  which VA refers to as RAMP, is intended to reduce legacy appeals by  providing veterans with a chance for early resolution of their claims within  VBA while the Board focuses on reducing its inventory of legacy appeals,  according to VA. Participation in RAMP is voluntary, but veterans must  withdraw their pending legacy appeal to participate, according to VA.  Veterans dissatisfied with their RAMP decisions must wait until VA fully  implements the new appeals process (in February 2019 at the earliest)  before pursuing an appeal with the Board under the new process,  according to VA officials.

VA’s Plan Addresses Most of the Act’s Required Elements for the New and Legacy Disability Appeals Processes

VA’s appeals plan addresses 17 of the Act’s 22 required elements,  partially addresses 4 related to monitoring implementation and workforce  planning, and does not address 1 element related to identifying total  resources. For example, VA’s appeals plan addresses the required  elements related to, among others, identifying legal authorities for hiring  and removing employees, estimating timelines for hiring and training  employees, and outlining the outreach VA expects to conduct. For the  elements in the Act that VA’s appeals plan partially addresses or does not  address, see table 1. For a detailed list of the 22 required elements in the  Act, see appendix I.

When we provided VA with our preliminary assessment, VA officials said  they disagreed with our assessment and that their appeals plan  addresses all 22 of the required elements. In general, they said that data  are not available, and VA cannot yet forecast the information required by  the Act until aspects of the new appeals process are tested or  implemented.

We continue to believe the information as presented in VA’s appeals plan  and supplemental materials addresses 17 of the required elements,  partially addresses 4, and does not address 1 element. Without complete  information on all 22 of the required elements, Congress does not have  the information it needs to fully conduct oversight of VA’s appeals plan  and the agency’s efforts to implement and administer the new process  while addressing legacy appeals. VA also is required to provide  information on resources, among other areas, before it can certify that the  agency is prepared to carry out timely processing of appeals under the  new and legacy appeals process. Further, as discussed below,  addressing required elements through a more comprehensive plan and  underlying analysis is consistent with sound planning practices and would  better position VA to implement the new appeals process while attending  to legacy appeals; for example, a plan that provides for carefully  monitoring the new and legacy appeals processes against balanced goals  and metrics, and clearly articulates resources, milestones and other  information needed for effective program management.

VA’s Appeals Plan Reflects Certain Sound Planning Practices, but Could Improve on Others

VA’s appeals plan reflects certain sound planning practices, such as  convening a working group on performance tracking; however, the plan  could benefit from including important details related to three key planning  areas:  1.  articulating a balanced set of goals and related measures to  monitor and assess the performance of the new appeals process,  in conjunction with the legacy process;  2.  developing a high-quality and reliable implementation schedule to  manage key steps and activities of the project; and  3.  assessing key risks in a comprehensive manner, including  respective mitigation strategies, and articulating clear criteria and  an assessment plan for RAMP, and more fully testing or analyzing  all appeal options.

VA’s Appeals Plan Indicates Steps to Assess Process Changes, but Should Also Include Goals and Measures to Provide Full Picture of Success

VA’s appeals plan reflects steps taken to track performance, but it could  improve its planning practices related to monitoring and assessing  performance on a range of key dimensions of success. Sound planning  practices suggest that agencies develop overall goals tied to meaningful  and balanced performance measures. These measures include a mix of  outcome, output, and efficiency measures to ensure that an  organization’s priorities—as well as government-wide priorities such as  quality, timeliness, and cost of service—are addressed.

VA’s appeals plan reports that the agency convened a working group to  design a process for tracking timeliness of both the legacy appeals and  appeals within the new process. In supporting documentation that we  requested, VA officials stated they are also determining the best way to  measure veterans’ satisfaction with the new appeals process. VA’s  appeals plan and supporting documentation also identify timeliness goals  for the two VBA-only options and one of the three Board options.  Nevertheless, its appeals plan does not articulate a set of goals and  measures that cover all aspects of its new appeals process, such as  accuracy of decisions and cost. The plan also does not provide details on the metrics the agency will develop, how it will assess if the new appeals  process is an improvement over the legacy appeals process, and how it  will monitor the allocation of resources between legacy and new appeals  claims. More specifically:

VA’s reported timeliness measures are incomplete: VA’s appeals  plan outlines timeliness goals for the two VBA options (average  processing time of 125 days) and for the Board option that does not  include new evidence or a hearing (average processing time of 365  days). However, VA’s plan does not establish timeliness goals for  the other two Board options: Board review of additional evidence  without a hearing and Board review of additional evidence with a  hearing. In commenting on our assessment, while VA officials  indicated they expect the new process to be more efficient than the  legacy process (and, therefore, more timely), data to inform goal  setting for all Board options will not be available until VA fully  implements these options. However, establishing timeliness goals for  all options would provide a more complete picture of VA’s vision for  the new appeals process, and help VA to develop concrete, objective,  and observable performance measures to show progress in achieving  that vision, as well as inform resource estimates.

VA’s reported measures lack adequate balance: Other than  including certain timeliness goals, VA’s appeals plan does not  articulate additional aspects of performance important for managing  appeals, such as accuracy of decisions, veteran satisfaction with the  process, or cost. We previously reported that VA officials said that  they wanted to also use veteran survey results, wait times, and  inventories as sources of information to measure progress under the  new appeals process. Further, VA’s fiscal year 2018 annual  performance plan includes an overall customer satisfaction score for  veterans’ benefits. However, these and other potential measures of  success are not specified in VA’s appeals plan for monitoring the new  appeals process as compared with legacy appeals. By not articulating  a set of comprehensive and balanced goals and measures in its  appeals plan, VA could be inadvertently creating skewed incentives  by focusing on one area of program performance to the detriment of  other areas (e.g., processing claims quickly but inaccurately).

In commenting on our assessment, VA officials recognized the need  to develop additional goals and measures and indicated, for example,  that they are developing and testing whether the existing quality  assurance goal—requiring 92 percent accuracy—is appropriate for  the new process. According to VA officials, once they have developed  these other goals and measures, VA will communicate this  information as part of the required progress reports to the appropriate  committees of Congress and GAO.

VA’s plan does not reflect how it will establish baseline data:  VA’s approach for evaluating the efficiency and effectiveness of the  implementation of the new appeals process falls short of sound  practices for using baseline data to assess performance. Our prior  work has demonstrated that by tracking and developing a  performance baseline for all measures, including those that  demonstrate the effectiveness of a program, agencies can better  evaluate progress made and whether or not goals are being  achieved. However, VA’s appeals plan did not provide important  details about what aspects of the new appeals process’ performance  will be compared to what aspects of the legacy process’ performance.

In particular, section 5 of the Act lists a number of metrics VA is  required to report periodically, including some that could be used as  baseline measures. For example, VA is required to periodically  publish on its website the average time that elapsed between the filing  of an initial claim and the final resolution of the claim, for legacy  appeals as well as appeals under the new system, which is consistent  with our prior recommendation. However, VA’s appeals plan does  not explain how or when the agency would collect and use these or  other data about the legacy and new processes’ performance—such  as accuracy, veteran satisfaction, and cost—to assess their relative  performance.

As we had previously reported, VA’s business case for reform in some  instances relied on unproven assumptions and limited analyses of its  legacy process to identify root causes of performance problems.  Specifically, VA determined that the open-ended nature of its legacy  appeals process, whereby a veteran can submit additional evidence  numerous times at any point during the VA appeals process, can  cause additional cycles of re-adjudication, a process VA refers to as  “churning.” According to VA, this re-adjudication can occur multiple  times and can add years to the time needed to reach a final decision  on an appeal. Without fully articulating a plan for collecting and using  baseline and trend data, VA cannot determine the extent to which the  new appeals process, which also allows for multiple appeal  opportunities, will achieve final resolution of veterans’ appeals sooner,  on average, than the legacy process.

In commenting on our assessment, VA indicated that it is working  toward capturing the metrics listed in section 5 of the Act. VA officials  also noted that reporting on the new appeals process will require IT  system functionality that currently does not exist, but stated that  efforts are underway to add this functionality.

VA’s plan does not explain how the agency will monitor  processing of legacy versus new appeals: In addition, VA’s  appeals plan does not fully articulate how the agency will monitor  whether resources are being appropriately devoted to both the new  and legacy appeals process and how it will track both sets of  workloads. An appeals plan that does not specifically articulate how  VA will manage the two processes in parallel exposes the agency to  risk that veterans with appeals in the legacy process may experience  significant delays or otherwise poor results relative to those in the new  appeals process or vice versa. In commenting on our assessment, VA  officials noted that VA was not required under section 3 of the Act to  provide a description of its plans to capture metrics listed in section 5.  Even if not required by the Act, developing an approach for carefully  monitoring the management of new and legacy appeals would help  VA track progress being made and achievement of goals.

Until VA establishes complete and balanced goals and measures,  identifies baseline data, and develops a plan for monitoring and  assessing both the new and legacy processes, VA runs the risk of  promoting skewed behaviors, or not fully understanding whether the new  process is an improvement or whether veterans with appeals in the  legacy process are experiencing poor results.

VA’s Appeals Plan Needs a Reliable Implementation Schedule to Manage the Project

VA’s appeals plan reflects certain aspects of sound planning practices  related to managing the implementation of process change; however,  other key components are not addressed. Sound planning practices for  implementing process change suggest establishing a transition team.  Consistent with such practices, VA’s appeals plan states that the agency  convened an agency-wide governance structure to coordinate  implementation of its new appeals process; it is comprised of senior-level  employees with authority to make necessary decisions to keep the project  on track. VA’s appeals plan also includes a copy of a master schedule. In  its plan, VA asserts that the master schedule reflects timelines, interim  goals and milestones, reporting requirements, and established deadlines,  and that it will be used to guide implementation. VA’s appeals plan also  reports that VA is consulting with project management professionals, who  are using the master schedule, among other tools, to monitor  implementation. In addition, VA made progress addressing some of the  issues we previously identified by developing steps and timetables for updating training in anticipation of implementing the new appeals  process.

However, VA’s master schedule for implementing reform is missing  elements of a high-quality and reliable implementation schedule for key  activities. We have previously reported that having a well-planned  schedule is a fundamental management tool. Generally recognized sound  practices from the Project Management Institute (PMI) and GAO call for  organizations to employ an integrated and reliable master schedule that  defines when work activities will occur, who will complete the work, how  long they will take, how they are related to one another, and the  constraints affecting the start and completion of work elements, as well as  whether resources will be available when they are needed. Such a  project management schedule not only provides a road map for  systematic project execution, but also provides the means by which to  gauge progress, identify and address potential problems, and promote  accountability.

The master schedule VA provided in its appeals plan should have  included other sound practices for project management related to a  reliable schedule. Specifically:

Key activities and their duration are not included: VA’s master  schedule does not capture the Rapid Appeals Modernization Program  (RAMP) activities, even though this pilot test is occurring at the same  time VA is preparing for full implementation of appeals options at VBA  and the Board. In addition, specific Board-related activities are  missing from the schedule, such as efforts to develop metrics, and the  schedule and other project plans we reviewed do not go beyond  February 2019. For example, the schedule does not indicate the  period of time when VA expects to no longer be processing legacy  appeals. When all key and necessary activities are not included, it  raises questions about whether all activities are scheduled in the  correct order, resources are properly allocated, or the estimated  completion dates are reliable. In addition, if the schedule does not  fully and accurately reflect VA’s efforts, it will not serve as an  appropriate basis for analysis and may result in unreliable completion  dates and delays.

Sequencing and linkages among activities are not identified: For  the high-level activities VA’s appeals plan identifies, VA’s master  schedule does not indicate whether there were linkages or  sequencing among them, which is not consistent with sound  scheduling practices. Linkages and sequencing would show, for  example, if any of these activities or sub-activities must finish prior to  the start of other activities, or the amount of time an activity could be  delayed before the delay affects VA’s estimated implementation  date. For example, VA cannot train new employees until after it hires  them.

The activities VA identifies also do not appear supported by lower- level project schedules. Specifically, when we requested  documentation to support VA’s high-level summary of activities and  milestones, VA officials did not provide intermediate or more detailed  schedules that reflected these practices. In particular, VA’s appeals  plan lacks a complete schedule for IT modifications that clearly  defines what is to be achieved and the time frames for achievement.  We previously recommended that VA develop a schedule for IT  updates that explicitly addresses when and how process reform will  be integrated into new systems and when these systems will be ready  to support the new appeals process at its onset. For example, VA’s  appeals plan references several required IT modifications that do not  appear in its master schedule. Schedules that are defined at too  high a level may disguise risk that is inherent in lower-level  activities.

Interim goals are not reflected: VA officials stated that they have  interim goals and milestones, though VA’s appeals plan and  supporting documentation generally do not include this information.  Sound planning and redesign practices suggest closely monitoring  implementation and developing project goals that include a mix of  intermediate goals to be met at various stages. VA’s appeals plan  does not include this information. We previously made a  recommendation that VA develop a more robust plan for closely  monitoring implementation of process reform, including metrics and  interim goals to help track progress, evaluate efficiency and  effectiveness, and identify trouble spots—all of which are consistent  with sound planning practices.

Resources are not assigned to all identified activities: The high- level summary schedule that VA provided us also lacks details  regarding the assignment of resources for all activities. Specifically,  while the plan identifies workgroups responsible for coordinating  elements in the plan, such as regulations, training, and outreach, the  schedule does not assign resources to the 40 listed activities. As  discussed previously, VA’s appeals plan also does not provide  information on the total resources required for this reform effort.  Assigning resources to the listed activities, as well as providing other  information, could provide a better indication of the estimated total  resources required to implement the new appeals process and  address legacy appeals.

In commenting on our assessment, VA officials stated that the agency is  developing lower-level project schedules for key activities—such as  RAMP and IT requirements—and will provide these schedules as part of  the required progress reports to the appropriate committees of Congress  and GAO. VA officials also noted that future updates will include  additional dependencies and risks, which VBA and the Board are still  developing. Until VA has a robust integrated master schedule, supported  by detailed project plans that adhere to sound practices, VA’s appeals  plan does not provide reasonable assurance that decision makers have  the essential program management information needed for this complex  and important effort.

VA’s Plan Addresses Some but Not All Key Risks Related to the New Appeals Process

VA’s appeals plan includes an assessment of risks involved in  implementing the new appeals system, but could more comprehensively  reflect key risks posed by such a significant reform effort. VA’s appeals  plan and supplementary materials include a “risk register” that describes  risks associated with many elements of its plan and the remaining level of  risk after its planned response to these risks. VA’s appeals plan also  states that senior leaders will receive regular updates of risks and  mitigation strategies. However, because VA has not yet articulated a  balanced set of performance goals and measures in its appeals plan, it is  hindered in its ability to identify and assess risks.

Federal internal control standards state, and our previous work at VA and  other agencies demonstrates, that establishing clear performance goals  and objectives is a necessary pre-condition to effectively assessing risk.  Having, for example, more complete timeliness goals, and goals and  measures reflecting other areas of performance, would allow VA to better  identify and target risks associated with managing two processes in  parallel, including the potential that veterans with appeals in the legacy  process may experience significant delays relative to those in the new  appeals process.

Importantly, VA is missing an opportunity to fully benefit from RAMP by  not testing and assessing other aspects of the new appeals process. The  Act authorizes VA to test the feasibility and advisability of any facet of the  new appeals process, and VA is taking a positive step to mitigate some  risks by testing the two review options available within VBA (review of a  claim by a higher-level official based on the same evidence and review of  a supplemental claim with additional evidence) through RAMP. In  November 2017, VA began RAMP by inviting 500 veterans whose  appeals have been pending the longest to participate. According to VA  officials, each month VA plans to continue offering RAMP to additional  eligible veterans with pending legacy appeals until January 2019—a  month before VA anticipates fully implementing the new appeals system.  However, as designed, RAMP does not include features that—consistent  with a well-developed and documented pilot test program—would provide  VA with an opportunity to evaluate fully the soundness of new processes  and practices on a smaller scale. Specifically:

VA’s plan does not clearly define success criteria for RAMP: VA’s  appeals plan states that the agency will collect certain data from  RAMP, such as the rate at which eligible veterans opt into the  process, timeliness of claims processing, and individual employee  productivity. VA also established an overall average processing time  goal of 125 days for the two VBA options; however, the plan and  supporting documentation do not clearly articulate whether RAMP  reviews are expected to meet this timeliness goal. The plan also did  not identify other success criteria for RAMP or the types of results  expected before fully implementing the new appeals process. For  example, VA’s plan does not articulate the expected number and type  of subsequent appeals to the Board that result from RAMP.

In commenting on this assessment, VA noted that its intent in  implementing RAMP was to collect data and test aspects of the new  process, and that RAMP was not an initiative in and of itself. However,  developing performance measures and data gathering procedures  and defining success criteria for a pilot test before proceeding to full  implementation are sound practices for process redesign and pilot  testing. In addition, because RAMP was not included in VA’s risk  assessment, we asked VA if it had identified any risks or mitigation  strategies specific to RAMP. In its supplemental materials, VA stated  that the greatest risk to RAMP is a low participation rate among  eligible veterans with legacy claims. VA also indicated that it would  need 10 percent of eligible veterans to opt into RAMP to yield  meaningful results. However, this threshold is not articulated in VA’s  appeals plan as an explicit success criterion or objective. According to  data provided by VA, as of January 22, 2018, 238 veterans opted in.  Of veterans with pending claims in RAMP, two-thirds chose the  higher-level review option. VA also reported that 47 RAMP decisions  have been made so far. As of yet, no appeals of RAMP decisions  have been filed.

VA’s plan does not articulate how it will assess RAMP before  proceeding with full implementation: Although VA’s appeals plan  describes a “close-out” phase in which VA intends to assess the  results of RAMP, it does not detail the conditions that would have to  be met (or not met) to trigger changes. For example, VA’s plan does  not explain when or how it might respond to low opt-in rates for  RAMP—other than stating it will increase outreach to eligible  veterans—or to unexpectedly high appeal rates to the Board resulting  from RAMP decisions. Sound redesign and change management  practices both suggest that pilot tests be rigorously monitored and  evaluated, and that further roll-out occur only after an agency’s  transition team takes any needed corrective action and determines  that the new process is achieving previously identified success  criteria. Without fully articulating its plan for deciding how and when to  roll out changes more broadly, it is not clear whether VA would be  prepared to fully implement a new appeals process that achieves its  aim of better serving veterans.

RAMP does not test all aspects of the new appeals process:  RAMP provides an opportunity to learn about experiences at VBA  under the new system, such as the rate at which eligible veterans  choose those options and the resources that will be required to  process their appeals. However, RAMP was not designed to test how  many veterans would choose to appeal directly to the Board and,  therefore, it will not provide comparable information on the Board  appeals options. Sound workforce planning practices suggest that  agencies identify the total resources needed to manage the risk of  implementing new processes and conduct scenario planning to  determine those needs.

In addition, although we previously recommended VA conduct  additional sensitivity analyses to inform projections of future appeals  inventories, VA’s appeals plan does not reflect VA’s use or intended  use of sensitivity analyses when projecting staffing needs for new  appeals options at the Board. In commenting on our assessment,  VA officials said they do not plan to conduct additional sensitivity  analyses to project future workloads until they have more information  from RAMP to inform their assumptions. As a result, VA will lack data  on scenarios in which veterans may overwhelmingly choose options  available at the Board over those at VBA when the appeals plan is  fully implemented. This presents a risk that VA’s early production  projections and initial resource allocations may not be properly  balanced between the Board and VBA. This, in turn, may result in an  unexpectedly large number of appeals pending with the Board, and  corresponding lengthy average wait and decision times for some, if  not all, Board options.

Having information on the number of veterans who are likely to appeal  to the Board is particularly critical, given that similar efforts to create  additional review options at VBA did not achieve their goals of  reducing the percentage of appeals that continue on to the Board. In  2001, VA established the Decision Review Officer (DRO) process—in  which senior staff have the authority to overturn an initial disability  claim decision without any new evidence—to resolve more appeals at  the regional level and avoid long waits at the Board. However, we  reported in 2011 that, although the DRO process helped some  veterans get additional benefits at the regional office level, it did not  accomplish the program’s primary goal of reducing the percentage of  appeals continuing on to the Board.

In responding to our assessment, VA officials reiterated their plans to  increase outreach in the event of low opt-in rates for RAMP and indicated  they recently began to send follow-up RAMP invitation letters. With  respect to assessing all appeal options, VA officials stated that, while no  legal bar prevents testing of the Board options, the Board is focused on  reducing its inventory of pending appeals while RAMP provides early  resolution of appeals within the new VBA-only options. Officials conceded  that this approach means they cannot collect data on the rate at which  veterans opt to appeal directly to the Board (e.g., bypassing additional  VBA review) until the new process is fully implemented. However, they  noted that they can collect some data on the rate at which veterans  whose appeals go through RAMP file subsequent appeals to the Board,  even though the Board will not begin processing those appeals until full  implementation.

By pursuing an approach that does not identify or mitigate significant risks  associated with implementing a new process, VA is taking a chance that  untested aspects will not perform as desired. The Act provides VA  authority to pilot aspects of the process and flexibility on the timing of  implementing the new process, which could allow some additional time  for VA to carefully measure performance under RAMP and determine  whether any corrective actions are necessary. If VA does not take full  advantage of this authority, it risks moving forward without knowing  whether the new appeals process improves experiences for veterans, and  potentially implementing a process that is more expensive or results in  longer wait times than originally anticipated.

In conclusion, in implementing appeals reform after the enactment of the  Veterans Appeals Improvement and Modernization Act of 2017, VA is  undertaking a complex endeavor that has the potential to affect the lives  of hundreds of thousands of veterans with service-connected disabilities.

Such an endeavor demands a commensurate level of planning to be  successful. While the Act required VA to submit its plan within 90 days of  enactment, VA had proposed and began to plan for appeals reform much  earlier, and had our March 2017 recommendations to guide its planning  efforts from a foundation of sound practices.

VA’s November 2017 appeals plan is a positive step forward. Certain  elements of the plan—such as establishing an agency-wide governance  structure to oversee implementation and testing aspects of reform prior to  full implementation—are notable gains since our March 2017 report. At  the same time, the plan partially addresses or does not address five of  the required elements called for by the Act, such as delineating the total  resources required by VBA and the Board to implement and administer  the new appeals process and address legacy appeals. The plan also is  not fully responsive to our past recommendations and does not reflect a  number of sound planning practices that are essential for gauging  progress, establishing accountability, and linking resources to results.

One such key practice is articulating a desired “end state”—a vision for  what successful implementation would look like for the new appeals  process as well as the wind-down of the legacy process, such as  accurate and timely processing of appeals while ensuring veteran  satisfaction. Without establishing a complete and balanced set of goals  and related performance measures to achieve this end state and  monitoring and assessing progress along the way, VA risks falling short of  its overarching objective—to improve timeliness of appeals decisions for  veterans overall. By not fully articulating how it plans to monitor workloads  and devote resources to both the new and legacy processes, VA runs the  risk of disadvantaging veterans with legacy appeals relative to those in  the new process, or vice versa.

Just as important is establishing a robust integrated master schedule— rather than a high-level timeline—that is built upon and clearly reflects  extensive detailed planning and includes all of the activities necessary to  execute the program and interdependencies between these activities.  Without such a road map, VA’s appeals plan does not provide reasonable  assurance that decision makers have the essential information needed to  manage this complex and important program.

We are encouraged that VA has taken some steps toward assessing  risks, including establishing a risk register and implementing RAMP to  collect information on the two VBA appeals options; however, unless VA  assesses risks against a balanced set of goals and measures, VA may  not be fully aware of risks that may impede successful implementation of  appeals reform. Further, although VA will undoubtedly learn from the  RAMP experience, it may not learn all that it should from its efforts  without (1) establishing clear criteria for what success looks like (or the  circumstances that would cause VA to consider making course  corrections) and (2) building in time to take stock of the lessons learned  before moving to full implementation.

VA’s plan places a lot of weight on RAMP to, among other efforts,  mitigate risk and generate estimates of the resources needed for  successful implementation after fiscal year 2018, even though RAMP  does not fully test options for appealing to the Board that will be available  to veterans after full implementation. Unless VA addresses key risks  associated with fully implementing appeals reform—by either testing or  conducting sensitivity analyses for all five appeals options, to better  understand potential workloads at the Board—VA runs the risk of fully  implementing the process without knowing if it is improving the process  for veterans.

In our forthcoming report, we anticipate making recommendations to  address these issues. Specifically, we are preliminarily considering  recommending that the Secretary of Veterans Affairs: address all of the required elements in the Act in VA’s appeals plan to  Congress—including delineating resources required for all VBA and  Board appeals options—using sensitivity analyses and RAMP results,  where appropriate and needed.  clearly articulate in VA’s appeals plan how VA will monitor and assess  the new appeals process compared to the legacy process, including  specifying a balanced set of goals and measures—such as timeliness  goals for all VBA appeals options and Board dockets, and measures  of accuracy, veteran satisfaction, and cost—and related baseline  data. augment the master schedule for VA’s appeals plan to reflect all  activities—such as RAMP and modifications to IT systems—as well  as assigned responsibilities, interdependencies, start and end dates  for key activities for each workgroup, and resources, to establish  accountability and reduce overall risk of implementation failures. ensure that the appeals plan more fully addresses risk associated with  appeals reform—for example, by assessing risks against a balanced  set of goals and measures, articulating success criteria and an  assessment plan for RAMP, and testing or conducting sensitivity  analyses of all appeal options—prior to fully implementing the new  appeals process.

Chairman Roe, Ranking Member Walz, and Members of the Committee,  this concludes my prepared statement. I would be pleased to respond to  any questions that you may have at this time.

For further information about this testimony, please contact Elizabeth  Curda at (202) 512-7215 or curdae@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this testimony. Other key contributors to this testimony  include Michele Grgich (Assistant Director), James Whitcomb (Analyst in  Charge), and Rachael Chamberlin. In addition, key support was provided  by Susan Aschoff, Mark Bird, David Chrisinger, Daniel Concepcion,  Clifton Douglas, Alex Galuten, Nisha Hazra, Melissa Jaynes, Benjamin  Licht, Patricia McClure, Sheila McCoy, Lorin Obler, Gloria Proa, Almeta  Spencer, James Sweetman, Walter Vance, and Greg Whitney.

Appendix I: Our Assessment of VA’s Appeals Plan Against Required Elements in the Act

To assess the extent to which VA’s appeals plan addresses the required  elements in the Veterans Appeals Improvement and Modernization Act of  2017 (the Act), we first identified and developed a checklist reflecting  each required element for VA’s appeals plan (including sub-parts) under  section 3(a) and (b) of the Act. To compare the required elements and  their sub-parts against VA’s appeals plan and supplemental materials  provided, we developed decision rules for determining whether the VA’s  appeals plan addressed, partially addressed, or did not address each  required element. Specifically, we concluded that VA’s plan addressed (or  partially addressed) a required element if the plan included information  related to all (or some) subparts of the requirement. We focused on the  plan as presented, rather than auditing the information VA relied on in  developing the plan. For example, the Act’s section 3(b)(10) required  VA’s plan to include a description of the modifications to the IT systems  that VBA and the Board require to carry out the new appeals system,  including cost estimates and a timeline for making the IT modifications.  We concluded that VA’s plan addressed all sub-parts of this element  because it provided a description of required IT modifications, a reference  to costs included in the Appeals Modernization IT budget, and a timeline.  However, our determination that VA addressed this element should not  be construed to necessarily mean that VA fully identified or described all  IT requirements, or provided complete estimated costs and timelines  associated with those requirements, or that the information in VA’s  appeals plan comported with sound planning practices. This type of  assessment was outside the scope of this objective. Table 2 summarizes  our assessment of VA’s appeals plan against the 22 required elements in  the Act.