Military Personnel: DOD Needs to Improve Dental Clinic Staffing Models and Evaluate Recruitment and Retention Programs

DOD has taken steps to modernize its Military Health System to ensure that it operates efficiently. For example, in September 2013, the Defense Health Agency was created, in part, to implement common clinical and business processes across the services. Essential to this effort are the services' ability to effectively staff their medical facilities, including the processes used for staffing dental clinics and the services' ability to recruit and retain military dentists.

Senate Report 115-125 included a provision for GAO to review the services' processes for determining requirements for dentists and its programs for recruiting and retaining military dentists, among other things. GAO assessed the extent to which the services (1) use validated dental clinic staffing models that also incorporate cross-service staffing standards, and (2) have recruited and retained military dentists and measured the effectiveness of their recruitment and retention programs. GAO assessed service dental clinic models, analyzed recruitment and retention data from fiscal year 2012 through 2016, and interviewed DOD and service officials.

The Army and the Air Force use validated staffing models for their respective dental clinics, and the Navy has developed a model that is under review. The Army and the Air Force's models are based on service-specific staffing standards. For example, the Army's model generally projects dental clinic staffing based on historical facility data and, according to officials, the Air Force model is largely a population-based model that requires one dentist for every 650 servicemembers. In contrast, in the absence of a validated model, officials stated that, the Navy uses a general ratio of one dentist for every 1,000 servicemembers to staff its dental clinics. The Navy has developed a model that is under review, and if approved, according to officials, will be subject to the Navy's validation processes.

While the services have developed and implemented cross-service staffing standards for 42 medical specialties, according to a key official involved in developing these standards, they currently do not plan to develop a similar set of standards for dental care. Cross-service staffing standards help the services standardize clinic staffing to address the common day-to-day health needs of patients. Service officials maintain that they must operate their respective dental clinics autonomously and in a manner that best supports their service-specific needs and unique command structures. However, as oral health requirements for servicemembers are standardized across the Department of Defense (DOD), it is unclear why dental care has been excluded from the staffing standardization effort—especially because the services have implemented cross-service staffing standards for 42 other medical specialties.

The Army, the Navy, and the Air Force meet their needs for military dentists by recruiting both dental students and fully qualified dentists. The services generally met their recruitment goals for dental students between fiscal years 2012 and 2016, but faced challenges recruiting and retaining fully qualified dentists during that period. For example, the Army missed its recruitment goals for fully qualified dentists in all 5 years, the Navy missed its goals in 2 out of 5 years, and the Air Force missed its goals in 3 out of 5 years. These challenges are most pronounced for certain specialties. For example, service data indicate that the Army and the Navy were unable to recruit any oral surgeons, while the Air Force recruited 50 percent of the oral surgeons it needed. Service officials cited various reasons for not meeting recruitment goals, including the availability of more lucrative careers in the private sector and quality of life concerns associated with military service.

The services rely on various programs, including scholarships and special pay and incentives, to attract and retain military dentists, and service officials stated that they monitor their programs by reviewing their goals, among other actions. However, GAO found that some services continue to provide incentive bonuses for positions that are overstaffed and have met or exceeded recruitment goals, but they do not know whether this is necessary because they have not evaluated the effectiveness of their programs. Without evaluating their programs, the services lack the information necessary to ensure that they are using recruitment and retention incentives effectively and efficiently for attracting and retaining dentists.

GAO recommends that each of the services develop cross-service staffing standards to be incorporated into their staffing models, and evaluate the effectiveness of their recruitment and retention programs. DOD did not provide comments on a draft of this report.

In recent years, the Department of Defense (DOD) has taken steps to  modernize its Military Health System to ensure that it operates as  efficiently as possible in the delivery of health services, while fully  supporting its mission and maintaining medical readiness. Key to this  modernization is ensuring that the Military Health System has the right  number of medical providers, including dentists, with the requisite skill set  and competencies to staff its Military Treatment Facilities (MTF). Military  dentists practice across all major dental specialties with the primary  mission of ensuring the dental readiness of all active duty  servicemembers.

According to a February 2015 U.S. Department of Health and Human  Services report, projected future nationwide shortages of dentists are  anticipated through 2025 as the demand for dental care in the United  States exceeds supply. Projected nationwide shortages of dentists  emphasize the importance for DOD to ensure its dental clinics are  adequately staffed and the services are able to recruit and retain certain  dental specialists. In 2010, we found that DOD’s processes for  determining MTF personnel requirements could be improved, and we  made recommendations aimed at enhanced collaboration across the  services. Specifically, we recommended that by identifying shared  medical capabilities, the services would benefit from the development of  cross-service standards for determining the amount of personnel needed  to meet the day-to-day health needs of the patient population. We also  made recommendations aimed at ensuring the validity and verifiability of  their MTF staffing models. DOD generally concurred with these  recommendations and took actions to implement them. These  recommendations and DOD’s actions are discussed in more detail in  subsequent sections of this report.

Senate Report 115-125, accompanying a bill for the National Defense  Authorization Act for Fiscal Year 2018, included a provision for us to  review the services’ processes for determining requirements for dentists  and DOD’s programs for recruiting and retaining military dentists, among  other things. This report assesses the extent to which the services (1)  use validated dental clinic staffing models that also incorporate cross- service staffing standards, (2) have recruited and retained military  dentists, and are measuring the effectiveness of their recruitment and  retention programs.

To address our first objective, we reviewed DOD, Army, Navy, and Air  Force policy and guidance for reviewing and validating workforce  models. We described the processes the services use for reviewing and  validating their dental clinic staffing models as it pertains to this guidance.  We interviewed officials with the Army Medical Command, the Navy  Bureau of Medicine and Surgery, and the Air Force Medical Service, as  well as officials from each service’s dental corps and other service  organizations who are responsible for the development, operation, and  implementation of these models. We reviewed the models and  interviewed service officials to determine similarities and differences  within the methodologies used in the models and processes, and  assessed the extent to which the services collaborated in the  development of their models and based their models on common  standards.

To address our second objective, we analyzed the extent to which the  services have met their recruitment goals for military dentists during fiscal  years 2012 through 2016, the most recent year for which data were  available at the time of our review. We assessed the reliability of these  data by obtaining and reviewing information from the services regarding  the processes and internal controls that are in place to ensure the  reliability and quality of the data. Based on our review, we believe these  data are sufficiently reliable for the purposes of this report. Additionally,  we used data from DOD’s Health Manpower and Personnel Data System  (HMPDS) and service-level data for fiscal years 2012 through 2016 to  assess the extent to which the Army, the Navy, and the Air Force (1) have  met authorized end-strengths for dentists, (2) are retaining dentists, and  (3) have gaps in dental specialties. We selected this timeframe to enable  us to evaluate trends over time, and fiscal year 2016 was the most recent  year of HMPDS and service-level data available at the time of our review.  We assessed the reliability of the HMPDS and service-level data by  collecting information and interviewing officials to determine how the  services and the Defense Manpower Data Center compile the data and  what internal controls are in place to ensure their reliability. We found  these data to be sufficiently reliable for the purposes of this report.

Additionally, we reviewed DOD’s special and incentive pay plan and other  DOD policies and guidance related to DOD recruitment and retention  programs. We interviewed officials and received written responses from  the services regarding their identified gaps, the programs they use to  address gaps in dental specialties, and any challenges they experience in  recruiting and retaining dentists. We also interviewed officials with the  Office of the Assistant Secretary of Defense for Health Affairs as well as  each service’s dental corps regarding the programs used and the  measures in place to evaluate the effectiveness of any such programs.  We reviewed our prior work on effective strategic workforce planning,  which state agencies should periodically measure their progress toward  meeting human capital goals and the extent that human capital activities  contributed to achieving programmatic goals and provide information for  effective oversight by identifying performance shortfalls and appropriate  corrective actions. Further, we reviewed Standards for Internal Control  in the Federal Government, including requirements for monitoring internal  control systems and evaluating results.

We conducted this performance audit from August 2017 to December  2018 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

The Military Health System is responsible for, among other things,  assuring the overall oral health of all uniformed DOD personnel. As part  of this health system, each service’s dental corps provides dental care for  its servicemembers. The Army, the Navy, and the Air Force Dental Corps  include approximately 3,000 active duty dentists and approximately 247  (200 in the United States) dental clinics to serve over 1.3 million  servicemembers.

Unlike their medical counterparts, the services’ dental corps rarely  provide beneficiary care, according to service officials. The primary role  of military dentists is to ensure the oral health and readiness of  servicemembers. Servicemembers’ oral health is evaluated using  standardized measures to determine the extent to which they are  deployable. Generally, servicemembers with identified urgent, emergent,  or unknown dental treatment needs are not considered to be worldwide  deployable until their oral health is adequately addressed.

Most military dentists enter service through the Armed Forces’ Health  Professions Scholarship Program (AFHPSP), a scholarship program  available to students enrolled in or accepted to dental school. Under the  services’ AFHPSP program, DOD pays for tuition, books, and fees, and  provides a monthly stipend. In return, the students incur an obligation to  serve 6 months of active duty service for each 6 months of benefits  received, with a 2-year minimum obligation. AFHPSP dental students  can pursue either a Doctor of Dental Surgery or Doctor of Dental  Medicine degree to become a general dentist.

In addition to the AFHPSP program, the services recruit fully qualified  licensed dentists. For example, individuals may become military dentists  through direct accessions, either by entering the service as a fully trained,  licensed dentist or through the Financial Assistance Program, which  provides stipends for dentists accepted or enrolled in a residency  program. For additional information on these and other recruitment  programs, see appendix I.

Regardless of the recruitment program, dentists may begin to practice  after obtaining a degree and completing licensure requirements. Military  dentists may pursue postgraduate training through a general dentistry  program, such as the Advanced Education in General Dentistry Program,  a general practice residency, or a specialty dental program offered  through the Uniformed Services University of the Health Sciences  Postgraduate Dental College. Postgraduate dental college includes  training and/or residency within a specific specialty and typically requires  between 1 to 6 years of additional training. While in a postgraduate dental  college program, participants incur an additional 6 months of active duty  service obligation for each 6 months in training, with a minimum of 2  years active duty service obligation. However, this obligation can be  served concurrently with obligations already incurred through AFHPSP if  incurred through sponsored postgraduate education in a military or  affiliated program. Figure 1 portrays the path to becoming a military  dentist and the active duty obligation incurred for AFHPSP dental  students.

Each service takes steps to validate whether the military dentist has the  appropriate professional qualifications and clinical abilities. Validation  includes ensuring the dentist is credentialed and privileged to practice.

See appendix II for more details on service processes for monitoring  qualification and performance of dentists.

Roles and Responsibilities for the Recruitment and Retention of Military Dentists

The Assistant Secretary of Defense for Health Affairs (ASD(HA)) serves  as the principal advisor for all DOD health policies and programs. The  ASD(HA) issues DOD instructions, publications, and memorandums that  implement policy approved by the Secretary of Defense or the Under  Secretary of Defense for Personnel and Readiness and governs the  management of DOD medical programs. The ASD(HA) also exercises  authority, direction, and control over the President of the Uniformed  Services University of the Health Sciences (USUHS). Further, ASD(HA)  sets the special and incentive pay amounts for all military dentists. The  ASD(HA) reports to the Under Secretary of Defense for Personnel and  Readiness, who in turn reports to the Secretary of Defense.

The Army, the Navy, and the Air Force medical commands and agencies  report through their service chiefs to their respective military department  secretaries and then to the Secretary of Defense. The Army, the Navy,  and the Air Force have the authority to recruit, train, and retain dentists.  Each military service has its own organizational structure and  responsibilities. See figure 2.

In September 2013, the Defense Health Agency was established to  support greater integration of clinical and business processes across the  Military Health System. The Defense Health Agency, among other things,  manages the execution of policies issued by the ASD(HA) and manages  and executes the Defense Health Program appropriation, which funds the  services medical departments. By no later than September 30, 2021, the  Director of the Defense Health Agency will assume responsibility for the  administration of each military treatment facility, to include budgetary  matters, information technology, and health care administration and  management, among other things. Although military treatment facilities include dental clinics, DOD initially intended to exclude dental care  (except oral and maxillofacial surgery), from the transfer to the Defense  Health Agency. However, as of September 2018, DOD stated it is  assessing the extent to which dental care will fall under the Defense  Health Agency’s administration.

GAO’s Prior Work on Military Treatment Facility Staffing Models and Tools

In July 2010, we found that the services’ collaborative planning efforts to  determine staffing of medical personnel working in fixed military treatment  facilities, including dentists, were limited, and that their staffing models  and tools had not been validated and verified in all cases as DOD policy  requires. Specifically, we found that some Army specialty modules  contained outdated assumptions, and that only a portion of the models  had been completely validated. We also found that the Navy did not have  a model, but instead employed a staffing tool that used current manning  as a baseline and adjusted its requirements based on emerging needs or  major changes to its mission. However, the Navy’s tool was not validated  or verified in accordance with DOD policy. Further, we found that the Air  Force may not know its true medical requirements because the model it  relied on also was not validated or verified.

We made several recommendations in our 2010 report, two of which were  aimed at improving staffing of MTFs. Specifically, we recommended that  the services identify common medical capabilities shared across military  treatment facilities and develop and implement cross-service medical  staffing standards for these capabilities as appropriate. We also  recommended that each service update or develop medical personnel  requirements determination tools as needed to ensure that they use  validated and verifiable processes.

The Army, the Navy, and the Air Force have implemented our  recommendation related to the development and implementation of validated and verifiable tools for developing medical personnel  requirements. Additionally, they identified and developed standardized  cross-service staffing standards for over 40 medical specialties and  incorporated them into their individual MTF staffing tools.

Two of Three Services Use Validated Dental Clinic Staffing Models, and None of the Models Incorporate Cross- Service Standards

The Army and the Air Force Use Validated Dental Clinic Staffing Models, and the Navy’s Proposed Model Is under Review

The Army and the Air Force have validated the dental clinic staffing  models that they use, and the Navy’s draft model is under review. In the  absence of a validated model, the Navy uses a general ratio to staff its  dental clinics. See table 1 for a description of each of the services’  methodology for staffing dental clinics.

The Army and the Air Force models, which were developed in  accordance with DOD guidance and service-specific requirements,  are subject to the following validation processes:

Army. Since 2011, the Army has used the Army Dental Clinic Model,  which, according to officials, is intended to determine the minimum  number of dentists necessary, by location, to ensure the medical  readiness of soldiers. Army staffing models are subject to validation  by the U.S. Army Manpower Analysis Agency, which validated the  Army’s Dental Clinic Model when it was developed in 2011.  According to an Army official, the model’s validation expired in 2014,  and was not re-validated until May 2018 due to limited resources.  Additionally, Army officials stated that the data used in the model are  updated on an annual basis and that the model is subject to  revalidation every 5 years.

Air Force. Since 2014, according to Air Force officials, the Air Force  has used its Dental Manpower Model to determine the minimum  number of dentists required, by clinic, to ensure the medical readiness  of servicemembers served by Air Force dental clinics. According to Air  Force officials, the Air Force Dental Manpower Model is subject to  review and validation that includes input from the Air Force Medical  Service; Surgeon General’s Manpower, Personnel, and Resources  office; Air Force Personnel Center; and consultants. Officials told us  the model is reviewed and validated annually and presented to the  Dental Operations Panel and Air Force’s medical service corporate  structure. The model was most recently validated in April 2018.

According to Navy Bureau of Medicine and Surgery (BUMED) officials,  the Navy does not yet have a model and therefore instead uses a general  ratio of one dentist for every 1,000 sailors as a baseline to initially  determine the staffing requirements of its dental clinics. This ratio is  adjusted based upon emerging needs or major changes to mission. In  2013, according to Navy officials, BUMED began developing a Dental  Services Model that could be used to determine dental clinic staffing  needs. In November 2016, BUMED internally released a draft report recommending that the dental corps approve and implement the Dental  Services Model as the staffing standard for dental clinics. According to a  Navy official, this report was provided to dental corps leadership for  review in July 2018 and they are expected to complete their review in  October 2018. According to BUMED officials, if the dental corps  leadership approves the model for use as an official staffing standard, the  model would be subject to official Navy validation processes which, in  accordance with DOD policy, would entail verification and validation  throughout the model’s lifecycle. Conversely, if the dental corps decides  to use the model as an informal staffing tool to supplement its current  processes, a BUMED official stated that it will be subject to an ad-hoc  internal review every 3 years that mirrors the Navy’s review of its  validation process.

The Services Have Not Developed Cross-Service Staffing Standards for Dental Care

Currently, the Army, the Navy and the Air Force each use different  service-specific standards and other factors to determine the number of  dentists needed at their respective dental clinics. As previously  discussed, the services have developed and are in the process of  implementing cross-service staffing standards—that is, a standardized  approach to staffing the common day-to-day health needs of the patient  population—for certain medical specialties. In response to DOD policy  and our 2010 recommendation, the services established a working group  to identify and develop common cross-service staffing standards, and in  2017, the tri-service working group established such standards for 42  different medical specialties. These standards are based on actual  workload data for common capabilities within selected medical specialties  and were incorporated into each service’s staffing tools to provide  consistent values for the minimum number of staff required to meet  patient needs. However, according to an official involved in the  development of the standards, the services have not collaborated to  develop a plan to establish a similar set of standards for dental care.

DOD guidance directs modeling and simulation management to develop  plans and procedures and to pursue common and cross-cutting modeling  tools and data across the services. Also, the ASD(HA) has supported  the effort to establish consistent workload drivers across services for  determining personnel requirements for MTFs.

According to a tri-service working group co-chair, they did not develop  cross-service staffing standards for dental care because at the time, the  quality of available data on dental procedure frequency and duration  varied across the services. The same official stated that these data have  been improved, but they still do not have plans to develop cross-service  staffing standards for dental care. Additionally, service officials  maintained that they must operate their respective dental clinics  autonomously and in a manner that best supports their service-specific  needs and unique command structures. Specifically, officials from each  service’s dental corps stated that their primary mission is focused on the  medical readiness of servicemembers and generally does not involve  beneficiary care. As such, they have not collaborated on staffing efforts  with the other services.

While we recognize that each service operates under a different  command structure, readiness requirements for oral health are  standardized across DOD, and all servicemembers are required to meet  the same level of oral health in order to be deployable. Additionally, since  DOD is currently assessing whether it will consolidate the services’ dental  corps staff under the Defense Health Agency’s administration, it  remains unclear to us why dental care has been excluded from cross- service efforts to develop a common set of standards for staffing military  dental clinics—especially because the services have developed common  staffing standards for42 other medical specialties.

The Services Generally Have Met Goals for Recruiting Dental Students, but Not for Fully Qualified Dentists and Do Not Know the Extent to Which Certain Programs Are Effective at Helping Recruit and Retain Dentists

The Army, the Navy, and the Air Force have generally met their  recruitment goals for dental students, but generally have not met their  recruitment goals for fully qualified dentists to address oral health needs  of the services. Overall, we found that the services maintained their  staffing levels for military dentists during fiscal years 2012 through 2016,  but experienced gaps within certain specialties. Further, the services  rely on various programs and special pays and incentives, to recruit and  retain military dentists, but they do not know the extent to which some of  these programs are effective at helping them to do so.

The Services Generally Met Recruitment Goals for Dental Students, but Faced Challenges Recruiting Fully Qualified Dentists

Our analysis of Army, Navy, and Air Force data found that in fiscal years  2012 through 2016, the services generally met their goals for dental  students recruited through the Armed Forces Health Professions  Scholarship Program (AFHPSP). From fiscal year 2012 through fiscal  year 2016, the Army met 94 percent of its goals, the Navy met 100  percent of its goals, and the Air Force met 97 percent of its goals. Figure  3 shows the AFHPSP recruitment goals and achievements, by service for  fiscal years 2012 through 2016.

To address their immediate need for dental providers, the services also  recruit fully qualified general dentists or specialists. However, the services  have experienced challenges meeting their recruitment goals for fully  qualified dentists. Figure 4 below shows the recruitment goals and  achievements for fully qualified dentists from fiscal years 2012 through  2016. As shown in the figure, the Army did not meet its recruitment goals  for 5 consecutive years, the Navy did not meet its goals for 2 of these 5  years, and the Air Force did not meets its goals for 3 of these 5 years.

While the services have experienced challenges in recruiting fully  qualified dentists, the challenges are most pronounced in certain  specialties. For example, based on our analysis of service data from fiscal  years 2012 through 2016, the Army and the Navy were unable to recruit  any oral surgeons and the Air Force recruited 50 percent of the oral  surgeons it needed. Service officials cited various reasons for not being  able to meet their recruitment goals for certain specialties, including the  availability of more lucrative careers in the private sector and quality of life  concerns associated with military service, such as frequent moves.  Additionally, Air Force officials stated that they are not always able to  offer accession bonuses consistently, which has caused challenges in  recruiting. Table 2 shows the recruitment goals and percentage  achieved for fully qualified dentists, by specialty, from fiscal years 2012  through 2016.

The Services Have Maintained Overall Staffing Levels for Military Dentists, but Have Experienced Challenges Retaining Certain Specialties

While the services maintained overall staffing levels for military dentists,  they have experienced some challenges retaining certain specialties.  Overall, military dentist end strengths—the actual number of dentists on  board at the end of the fiscal year—have generally met or exceeded  dental authorizations. Specifically, between fiscal years 2012 and 2016,  the Army’s dental authorizations were filled, on average, at about 109  percent, the Navy’s at about 101 percent, and the Air Force’s at about 97  percent. Further, DOD data show that average overall gain rates are  equal to average overall loss rates for the services’ dental corps in fiscal  years 2012 through 2015 at approximately 10 percent for the Army, 9  percent for the Navy, and 11 percent for the Air Force. Additionally,  according to our analysis of Army and Navy data, on average,  approximately 73 percent of Army dentists continue on active duty after 5  years of service, and approximately 63 percent of Navy dentists continue  on active duty after 5 years of service. According to Air Force officials, the  Air Force does not routinely track data on the number of dentists that  continue on active duty after 5 years of service.

Although the services have been able to maintain their overall numbers  for the total number of dentists in their respective dental corps, we found,  based on the data the services provided us, that each service  experienced gaps in certain dental specialties, including critically short  wartime specialties. For example, all of the services experienced gaps  in comprehensive dentistry from fiscal years 2012 through fiscal year  2016. In addition, for the same time period, all of the services  experienced gaps in prosthodontists and oral surgeons. Officials from all  three services cited family concerns, frequent moves, and competition  from the private sector as reasons why these and other dentists choose  to leave the military. Additionally, Army and Navy officials cited limited  training and education opportunities and limited scope of practice as  reasons why specialists such as oral surgeons leave the military.

The Services Monitor Their Recruitment and Retention Programs, but Do Not Know Whether The Programs Are Effective

The services rely on programs, such as AFHPSP, the Critical Wartime  Skills Accession Bonus (CWSAB), and special pay and incentives, to  attract and retain military dentists, but they do not know the extent to  which some of these programs are effective at helping them meet their  recruiting and retention goals. Our prior work on effective strategic  workforce planning principles concluded that agencies should periodically  measure their progress toward meeting human capital goals. These  principles state that measuring the extent that human capital activities  contribute to achieving programmatic goals provides information for  effective oversight by identifying performance shortfalls and appropriate  corrective actions. Further, according to these principles, agencies  should develop use of flexibilities and other human capital strategies that  can be implemented with the resources that can be reasonably expected  to be available, and should consider how these strategies can be aligned  to eliminate gaps. Additionally, Standards for Internal Control in the  Federal Government states that management should monitor internal  control systems, through ongoing monitoring and evaluations. According  to these standards, evaluations should be used to provide feedback on  the effectiveness of ongoing monitoring and should be used to help  design systems and determine effectiveness. The standards also provide  that management should determine the appropriate corrective actions to  address any identified deficiencies upon completing its evaluation.

According to Army, Navy, and Air Force officials, the services have taken  various actions to monitor their recruitment and retention programs. For  example, officials told us that they review recruitment goals,  achievements, and retention rates; conduct workforce planning and  modeling; and participate in recruitment and retention working groups.  Specifically, Army officials stated that they use forecasts from a 5-year  management plan to determine the Army’s recruiting mission and review  its continuation rates to assess retention of dentists. Navy officials told  us that they review annual recruitment goals and track whether they are  meeting those goals on a weekly basis. Air Force officials stated that they  participate in the Medical Accessions Working Group three times per year  to assess ongoing recruitment activities. While the services monitor their  progress toward recruitment and retention goals, they do not know the  extent to which the programs designed to help them meet their goals  affect their ability to recruit and retain dentists because they have not  evaluated their effectiveness.

For example, DOD Directive 1304.21 allows the services to use  accession bonuses to meet their personnel requirements and specifies  that bonuses are intended to influence personnel inventories in specific  situations in which less costly methods have proven inadequate or  impractical. The services have the discretion to issue up to $20,000 as an accession bonus under the AFHPSP—in addition to paying full tuition,  education expenses, and a monthly stipend. In fiscal years 2012  through 2016, the Army and the Navy offered the accession bonus and  generally met their recruitment goals—an achievement that Army officials  credit, in part, to their use of the incentive. Specifically, Army officials told  us that prior to using the bonus in 2009, they were not meeting their  recruitments goals. They also expressed concern that, if they were to  discontinue use of the bonus, they would not be able to meet their current  goals. Conversely, Air Force officials told us that they stopped offering the  bonus in 2012 because the number of AFHPSP applicants had exceeded  the number of AFHPSP positions; the Air Force has continued to meet its  recruiting goals without the use of the bonus. An Air Force official  acknowledged that not offering the bonus could result in their losing  potential applicants to the services that do offer the bonus, but Air Force  officials also recognized that money is not always a deciding factor for  those who choose to serve as a dentist in the military. The uncertainty  described by the Army and Air Force officials demonstrates their lack of  information about what factors motivate individuals to join the military.  Moreover, the differing use of the accession bonus by the two services  with similar outcomes indicates that the services do not know when it is  necessary to use the recruiting tool because they have not evaluated the  effectiveness of this program.

Another bonus the services can offer is the CWSAB, which ranges from  $150,000 for general dentists to $300,000 for comprehensive dentists,  endodontists, prosthodontists, and oral maxillofacial surgeons, to  individuals entering the military as a dentist in a critically short wartime  specialty. While a bonus can be offered to any dental specialty  designated as a critically short wartime specialty, data that we analyzed  indicate that the bonus may be disproportionately effective in recruiting for  these specialties. For example, from fiscal years 2012 through 2016, the  Navy used this type of bonus and was able to meet or exceeded its  recruitment goals for critically short wartime specialty general dentists and  staffed this specialty at between 108 and 122 percent. However, our  analysis of the Navy’s data also found that, even after offering this bonus,  the Navy was unable to recruit any oral surgeons during the same time  period. However, like with the accession bonus, service officials do not  know the extent to which the CWSAB bonus is an effective recruitment  incentive for some or all of the critically short wartime specialties because  they have not evaluated the effectiveness of this program.

In addition, the services offer special pay and incentives, which vary by  specialty, to qualified dentists. Special pay and incentives include  incentive pay, retention bonuses, and board certification pay. Each bonus  and incentive, except board certification pay, requires an additional  service obligation, thus creating a retention tool for the services. The  services and officials from the Office of the ASD(HA) participate in the  Health Professions Incentives Working Group to review recruitment and  retention special pay and incentives and recommends adjustment to  amounts offered as necessary. Service and officials from the Office of the  ASD(HA) told us that there are no ways to evaluate the effectiveness of  these incentives because they cannot account for the emotional or non- monetary decisions that contribute to whether servicemembers stay in the  military, and money is not always an effective incentive for getting people  to train in certain specialties or to continue their service. However, in our  recent review of DOD’s special pay and incentive programs in 2017, we  recommended that DOD take steps to improve the effectiveness of its  special pay and incentive programs. Additionally, in February 2018,  through our review of gaps in DOD’s physician specialties, we  recommended that the services develop targeted and coordinated  strategies to alleviate military physician gaps. An official from the Office  of the ASD(HA) stated that they have started discussing measures with  the services to evaluate the effectiveness of DOD’s medical and dental  recruitment and retention programs, including special pay and incentives.  Additionally, Office of ASD(HA) and service officials stated that they will  begin reviewing the dental special pays and incentives in fiscal year 2019.  Because these reviews are in the early stages, it is too soon to know how  effective they will be in evaluating pay and incentive programs.

Although service officials also told us that they believe their recruitment  and retention programs are effective because they have generally met  their overall recruiting and retention goals, until the services evaluate the  effectiveness of their recruitment and retention efforts, they will not have  the information to know which programs are the most efficient and cost- effective.

DOD continues to implement several major initiatives to support the  mission of its health system maintaining the medical readiness of  servicemembers while operating as efficiently as possible. The dental  corps plays a critical role in these efforts by ensuring the oral health and  dental readiness of all servicemembers. Ensuring dental readiness  requires, in part, that the services are able to staff dentists adequately  and consistently across DOD’s dental clinics. However, the Army, the  Navy, and the Air Force have not collaborated in their approaches to  staffing dental clinics, and have not developed cross-service staffing  standards for dental clinic staffing. As DOD progresses in its efforts to  implement efficiencies across its Medical Health System and assesses  the scope of medical care to be transferred to the Defense Health  Agency, it could be of benefit to the dental corps to develop cross-service  standards that could result in improvements to the consistency and  efficiency of dental clinic staffing.

In addition to ensuring the appropriate number of dentists at each clinic to  support the dental corps’ mission, recruiting and retaining fully qualified  dentists has been an ongoing challenge for the services. However, the  services have not evaluated whether their existing programs have been  effective at helping them recruit and retain dentists, and therefore do not  know whether they are effectively targeting their resources to address the  most significant recruitment and retention challenges.

We are making a total of six recommendations, including two to the Army,  two to the Navy, and two to the Air Force. Specifically:  The Secretary of the Army should ensure that the Surgeon General of the  Army Medical Command (1) collaborate with the Navy Bureau of  Medicine and Surgery and the Air Force Medical Service to develop a  common set of planning standards to be used to help determine dental  clinic staffing needs, and (2) incorporate these standards into the Army’s  dental corps staffing model. (Recommendation 1)

The Secretary of the Navy should ensure that the Surgeon General of the  Navy Bureau of Medicine and Surgery (1) collaborate with the Army  Medical Command and the Air Force Medical Service to develop and  implement a common set of planning standards to be used to help  determine dental clinic staffing needs, and (2) incorporate these  standards into the Navy’s dental corps staffing model. (Recommendation  2)

The Secretary of the Air Force should ensure that the Surgeon General of  the Air Force Medical Service (1) collaborate with the Army Medical  Command and the Navy Bureau of Medicine and Surgery to develop and  implement a common set of planning standards to be used to help  determine dental clinic staffing needs, and (2) incorporate these  standards into the Air Force’s dental corps staffing model.  (Recommendation 3)

The Secretary of the Army should ensure that the Surgeon General of the  Army Medical Command evaluates the effectiveness of its recruitment  and retention programs for military dentists, including the need for and  effectiveness of the recruitment and retention incentives currently offered.  (Recommendation 4)

The Secretary of the Navy should ensure that the Surgeon General of the  Navy Bureau of Medicine and Surgery evaluates the effectiveness of its  recruitment and retention programs for military dentists, including the  need for and effectiveness of the recruitment and retention incentives  currently offered. (Recommendation 5)

The Secretary of the Air Force should ensure that the Surgeon General of  the Air Force Medical Service evaluates the effectiveness of its  recruitment and retention programs for military dentists, including the  need for and effectiveness of the recruitment and retention incentives  currently offered. (Recommendation 6)

We provided a draft of this report to DOD for review and comment. DOD  did not provide comments. DOD did provide us with technical comments,  which we have incorporated, as appropriate.

We are sending copies of this report to the appropriate congressional  committees, the Secretary of Defense, the Under Secretary of Defense  for Personnel and Readiness, the Office of the Assistant Secretary of  Health Affairs, the Secretaries of the Army, the Navy, the Air Force, and  the President of the Uniformed Services University of the Health  Sciences. In addition, the report is available at no charge on the GAO  website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-3604 or FarrellB@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix III.

Appendix I: Military Dentist Accession Programs and Incentives

In addition to the Department of Defense’s (DOD) Armed Forces Health  Professions Scholarship Program, DOD uses several other programs and  incentives to recruit military dentists. Table 3 includes a selection of  DOD’s military dentist accession programs and incentives.

Appendix II: The Services’ Mechanisms to Monitor Qualifications and Performance of Military Dentists

DOD policy requires that all military dentists must be credentialed and  privileged to practice dentistry. Credentialing is the process of inspecting  and authenticating the documentation to ensure appropriate education,  training, licensure, and experience. Privileging is the corresponding  process that defines the scope and limits of practice for health care  professionals based on their relevant training and experience, current  competence, peer recommendations, and the capabilities of the facility  where they are practicing.

According to officials, the services have developed and implemented  processes to continuously monitor dentist performance in accordance  with DOD policy. According to officials, the services monitor military  dentist performance through On-Going Professional Practice Evaluations  (OPPE) and Focused Professional Practice Evaluations (FPPE). The  OPPE is a continuous evaluation of dentist performance that reviews six  dimensions of performance: (1) patient care, (2) medical knowledge, (3)  professionalism, (4) practice-based learning and improvement, (5)  interpersonal and communication skills, and (6) systems-based practice.  The FPPE is a process of periodic evaluation by the dental clinic of the  specific competence of a dentist performing procedures and  administering care. FPPEs are conducted during a dentist’s initial  appointment, when granting new privileges, or if a question arises about a  dentist’s ability to provide, safe, high quality patient care.

In addition to the performance monitoring required by DOD, according to  officials, the Army and the Air Force have instituted their own  mechanisms for monitoring the quality and performance of their dentists.

Army: According to officials, the Army monitors dental quality through  its quarterly Continuous Quality Management Program. This program  includes the review of data related to records audits, infection control,  radiation protection, utilization management, implant reports, drug  utilization reports, patient safety events, and risk management.  According to officials, these reviews are intended to identify and  address any errors or trends in dental care.

Air Force: According to officials, annually, Air Force dentists must  document that they have reviewed and will follow the Air Force  Medical Service Dental Clinical Practice Guidelines. According to  officials, this ensures that all dentists are following the same standard  of care for dental treatment. Additionally, according to officials, Air  Force dentists participate in a peer review process known as Clinical  Performance Assessment and Improvement. According to officials, in  this process, a licensed peer dentist preferably of the same specialty  reviews the dentist’s practice and procedures. Further, according to  officials, depending on the nature of issues found during the review,  corrective actions—ranging from refresher courses to a loss of license  and credentials—may be taken.

Appendix III: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Kimberly Mayo, Assistant  Director; Nicole Collier; Alexandra Gonzalez; Amie Lesser; Tida Barakat  Reveley; Rachel Stoiko; John Van Schaik; Lillian Yob; and Elisa Yoshiara  made key contributions to this report.

Defense Health Care: Additional Assessments Needed to Better Ensure  an Efficient Total Workforce. GAO-18-102, Washington, D.C.: Nov. 27,  2018.

Defense Health Care: DOD Should Demonstrate How Its Plan to Transfer  the Administration of Military Treatment Facilities Will Improve Efficiency.  GAO-19-53, Washington, D.C.: Oct. 30, 2018.

Defense Health Care: Expanded Use of Quality Measures Could  Enhance Oversight of Provider Performance. GAO-18-574, Washington,  D.C.: Sept. 17, 2018.

Military Personnel: Additional Actions Needed to Address Gaps in Military  Physician Specialties. GAO-18-77. Washington, D.C.: Feb. 28, 2018.

Defense Health Reform: Steps Taken to Plan the Transfer of the  Administration of the Military Treatment Facilities to the Defense Health  Agency, but Work Remains to Finalize the Plan, GAO-17-791R.  Washington, D.C.: Sept. 29, 2017.

Military Compensation: Additional Actions Are Needed to Better Manage  Special and Incentive Pay Programs. GAO-17-39. Washington, D.C.:  Feb. 3, 2017.

Defense Health Care Reform: DOD Needs Further Analysis of the Size,  Readiness, and Efficiency of the Medical Force. GAO-16-820.  Washington, D.C.: Sept. 21, 2016.

Defense Health Care: Additional Information Needed about Mental Health  Provider Staffing Needs. GAO-15-184. Washington, D.C.: Jan. 30, 2015.

Human Capital: Additional Steps Needed to Help Determine the Right  Size and Composition of DOD’s Total Workforce. GAO-13-470.  Washington, D.C.: May 29, 2013  Defense Health Care: Actions Needed to Help Ensure Full Compliance  and Complete Documentation for Physician Credentialing and Privileging.  GAO-12-31. Washington, D.C.: Dec. 15, 2011.

Military Cash Incentives: DOD Should Coordinate and Monitor Its Efforts  to Achieve Cost-Effective Bonuses and Special Pays. GAO-11-631.  Washington, D.C.: June 21, 2011.

Military Personnel: Enhanced Collaboration and Process Improvements  Needed for Determining Military Treatment Facility Medical Personnel  Requirements. GAO-10-696. Washington, D.C.: Jul. 29, 2010.

Military Personnel: Status of Accession, Retention, and End Strength for  Military Medical Officers and Preliminary Observations Regarding  Accession and Retention Challenges. GAO-09-469R. Washington, D.C.:  Apr. 16, 2009.

Military Personnel: Better Debt Management Procedures and Resolution  of Stipend Recoupment Issues Are Needed for Improved Collection of  Medical Education Debts, GAO-08-612R. Washington, D.C.: Apr. 1, 2008.

Primary Care Professionals: Recent Supply Trends, Projections, and  Valuation of Services. GAO-08-472T. Washington, D.C.: Feb. 12, 2008.

Military Physicians: DOD’s Medical School and Scholarship Program.  GAO/HEHS-95-244. Washington, D.C.: Sept. 29, 1995.

Defense Health Care: Military Physicians’ Views on Military Medicine.  GAO/HRD-90-1. Washington, D.C.: Mar. 22, 1990.

The Army, Navy, and Air Force have implemented common standards for how they determine necessary staffing levels at their health clinics for a number of medical specialties—but haven’t done so for their dental clinics. As a result, DOD dental clinics may not be staffed as efficiently as possible to provide dental care across the military services.

The military services have also struggled to recruit and retain certain types of dental specialists, such as oral surgeons. We recommended that the services develop a common set of dental staffing standards and a process to evaluate recruitment and retention efforts.


Veterans Affairs Research: Actions Needed to Help Better Identify Agency Inventions

VA manages a $1.9 billion research program that has produced numerous healthcare inventions, such as the pacemaker. In 2000, VA created a program to help transfer VA inventions to the private sector so that they can be commercialized and used by veterans and the public, while VA retains ownership and collects royalties. Many of VA's 3,000 researchers also hold positions at universities, which take the lead in commercializing inventions developed by these researchers. Researchers and universities are required to disclose such inventions to VA, and universities are to report on commercialization activities according to their agreements with VA.

GAO was asked to examine VA's ability to ensure its ownership of inventions made with VA resources. This report examines, among other things, the extent to which VA has taken steps to ensure that (1) researchers disclose inventions and (2) universities report on commercialization activities for joint inventions. GAO reviewed laws; policies; a nongeneralizable sample of university agreements based on backlogs of disclosures, among other factors; and interviews with officials and researchers from VA medical centers and their affiliated universities.

The Department of Veterans Affairs (VA) has taken steps to educate agency researchers about its requirements to disclose inventions to VA, but officials reported that researchers have not consistently done so. VA policy requires researchers to disclose inventions to both VA and the university they work for even when they do not use VA resources. GAO found, through discussions with VA officials and researchers, that several factors contribute to researchers not consistently disclosing their inventions, including that VA researchers may have:

disclosed inventions to their university, assuming the university would then disclose them to VA;

not been familiar with VA's invention disclosure process, because they may not have frequently developed inventions; or

thought that invention disclosure was unnecessary when they did not use VA resources to develop their invention.

In 2017, VA staff visited universities and VA medical centers 26 times to meet with researchers about invention disclosure. Also, VA created an online training course to educate researchers on the need to disclose inventions, but the training is not mandatory, and about 4 percent of researchers took it. Without mandatory training to communicate invention disclosure requirements—consistent with federal internal control standards for internally communicating quality information—VA researchers may not be fully informed about those requirements, which can result in lost technology transfer opportunities and royalties for VA.

VA has improved communication with universities but has not ensured that they are consistently reporting information on commercialization activities for joint inventions. VA reported that about three-quarters of VA's 79 university partners did not submit the annual reports required by VA in 2017. GAO reviewed a nongeneralizable sample of agreements VA has with universities and found that reporting requirements about timing and content of reports were unclear. Without providing a standardized method that clearly guides universities in fulfilling VA's reporting requirements, consistent with federal standards for internal control, VA cannot ensure that it has adequate information to account for its licenses and royalties.

GAO recommends that VA (1) make training about invention disclosure mandatory and (2) provide universities with a standardized method for annual reporting. VA concurred with GAO’s recommendations.

For more than 90 years, the Department of Veterans Affairs (VA) has  managed a research program that has led to healthcare breakthroughs  that improve the lives of veterans and the public. According to VA, the  department has been instrumental in medical advancements such as  therapies for tuberculosis following World War II, the implantable cardiac  pacemaker, concepts for computerized axial tomography scans  (commonly referred to as CAT scans), the first successful liver  transplants, and the first powered ankle-foot prosthesis.

In 2000, VA created a technology transfer program to help transfer the  use of inventions to the private sector so that the inventions could be  commercialized and used by veterans and the public at large, while VA  would retain ownership and collect royalties for its inventions. Today VA  oversees a research program that spends $1.9 billion annually and  employs more than 3,000 researchers focused on health issues such as  cancer, diabetes, traumatic brain injury, and other conditions affecting  veterans. In fiscal year 2016, VA patents from such research resulted in  45 licenses providing the agency about $316,000 in royalties, according  to VA officials.

VA’s research program is unique in that the majority of VA’s researchers  are dually appointed—they work at VA and at another research institution, such as a university or medical college. VA’s dually appointed  researchers’ inventions are subject to agreements between VA and the  university and are considered joint inventions. In most cases, the  affiliated university is responsible for the bulk of technology transfer  activities, including filing patents, finding licensees to further develop the  technology, and collecting and distributing royalties. The process, from  disclosure of the invention to VA to commercialization of the invention,  requires coordination between researchers and administrators at VA  medical centers and staff at VA’s technology transfer office, as well as  between VA’s technology transfer office and the affiliated universities.

Congress has raised questions about the extent to which VA is  consistently asserting ownership of inventions developed with its  resources. You asked us to examine the VA technology transfer program  to ensure VA was not missing opportunities to obtain rights to inventions  developed using VA resources. This report examines the extent to which  VA has taken steps to ensure that (1) researchers and universities  disclose inventions and (2) universities have reported information to VA  about commercialization activities for joint inventions.

For both objectives, we reviewed documents provided by VA, affiliated  universities, and a VA advisory committee. These included documents on  the history of VA’s technology transfer program; technology transfer  policies and procedures; and documents outlining challenges faced in  implementing the program and recent efforts to address these challenges.  We also examined relevant statutes and regulations. We compared VA’s  procedures, policies, and internal and external communications to federal  internal control standards for control activities and information and  communication. We also interviewed officials from the VA technology  transfer office and the VA Office of the General Counsel; officials selected  through a nonprobability sample of six VA medical centers and their  affiliated universities; and two of the three members of a subcommittee of  the National Research Advisory Council that submitted a report to VA on  the department’s technology transfer program in fiscal year 2017. We  conducted site visits at five of the six VA medical centers—Atlanta,  Georgia; Baltimore, Maryland; Palo Alto and San Francisco, California;  and Salt Lake City, Utah—for interviews with researchers who were  inventors and with the Associate Chiefs of Staff for Research at each  medical center, to whom we refer as the research administrators. We also  met with officials of each of the universities affiliated with the respective  medical centers—Emory University; the University of Maryland at  Baltimore; Stanford University; the University of California, particularly the  University of California at San Francisco; and the University of Utah. In  addition, we conducted telephone interviews with researchers and the  research administrator at the sixth VA medical center in Cleveland, Ohio,  and officials of the university affiliated with it, Case Western Reserve  University. We selected the medical centers and their respective  university affiliates based on high levels of license activity or backlogs of  disclosures, among other factors. The results of these interviews are not  generalizable to all VA medical centers and their affiliated universities but  provide illustrative examples.

The VA technology transfer program is affiliated with 79 universities and  has agreements in place with all but three of these. VA began negotiating  these, which we refer to as original agreements, soon after the creation of  the VA technology transfer program in 2000. Beginning in fiscal year  2015, VA initiated a new type of agreement establishing new provisions  for VA, which we will refer to as new agreements. We reviewed original  agreements for five of the six universities in our sample. One of the  universities in our sample, Emory University, did not have an agreement  with VA as of January 2018. We also reviewed a nonprobability sample  of an additional three original agreements with other universities based on  their licensing and commercialization activities, for a total of eight original  agreements. In addition to the original agreements, we also reviewed the  first eight new agreements that VA negotiated with universities. In total,  we reviewed 16 agreements—the 8 original agreements and 8 new ones  to determine whether they contained requirements for disclosure and  reporting to VA.

For objective two, we requested and reviewed annual reports from VA for  the eight universities whose original technology transfer agreements we  reviewed. The annual reports VA provided were those the eight  universities submitted to VA in 2016 and 2017, covering technology  transfer activities, for a total of 12 reports. Not every university submitted  an annual report to VA for each year; instead, at least one university  provided an annual report that was late from a prior year, going back as  far as 2011. We reviewed at least one annual report from each of the  eight universities.

We conducted this performance audit from October 2016 to April 2018, in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Just after World War II, VA developed affiliations with medical schools to  improve acute care and physical and mental rehabilitation for veterans.  As part of the relationship, VA medical centers have contributed to the  education of medical students and residents. Besides medical students  and residents, other dual appointees—clinicians and researchers—spend  either a full 40-hour week or a fraction of the work week at VA and other  time at the affiliated university.

On January 23, 1950, Executive Order 10,096 established that the  government shall obtain the entire right, title, and interest in and to all  inventions made by government employees during working hours; with a  contribution by the government of facilities, equipment, materials, funds,  or information, or time and services of other government employees on  official duty; or which bear a direct relation to or are made as a  consequence of the employee’s duties. Since the early 1980s, the  federal government has taken several actions related to technology  transfer from federal laboratories.

Technology transfer is the process of transferring scientific findings from  one organization to another for the purpose of further development and  commercialization. In this regard, federal agencies are authorized to issue  licenses to outside entities granting rights to make, use, or sell  government owned inventions. One of the first technology transfer laws,  the Stevenson-Wydler Technology Innovation Act of 1980, articulated the  need for a strong national policy supporting domestic technology transfer.  The law requires federal laboratories to establish an office of research  and technology applications and devote budget and personnel resources  to promoting technology cooperation and the transfer of technologies to  private industry and state and local governments. In addition, the act  requires federal agencies that operate or direct federal laboratories to  report information on technology transfer performance annually to the  Office of Management and Budget, as part of their annual budget  submission. Copies of those reports should be transmitted to the  Secretary of Commerce who must submit a summary report to Congress  and the President.

For many years after the Stevenson-Wydler Technology Innovation Act of  1980, VA waived ownership rights to inventions generated by its  researchers, leaving the responsibilities for patenting, marketing, and  licensing with the inventor and the VA medical center’s university partner.  As a result of this practice, according to former VA officials, some VA  research was not commercialized because VA did not have a technology  transfer program or other means to promote commercialization.

In 2000, VA created the VA Technology Transfer Program to facilitate the  commercialization of VA inventions to benefit veterans and the American  public. VA developed technology transfer agreements with universities to  help facilitate technology transfer. Under the terms of the agreements, the  universities can take the lead on patenting and commercialization, and  VA can retain joint ownership of inventions. Among other things, the  original agreements gave the universities the right of first refusal to apply  for and manage patents, market the technologies, negotiate licenses, and  collect royalties to be shared with VA.

As of November 2017, the VA Technology Transfer Office, located in  Washington, D.C., employed five technology transfer specialists  responsible for all technology transfer activities for VA’s solely owned  inventions. These inventions may come from more than 3,000 VA  researchers at over 100 VA medical centers, as well as from VA  employees at other VA locations. In addition, the technology transfer  specialists are responsible for coordinating with universities on inventions  made by dually appointed researchers. According to VA officials, VA  relies on affiliated universities for most of the technology transfer efforts  connected with such inventions, since the universities have their own  offices with expertise in technology transfer and are usually willing to take  the lead.

Under a Veterans Health Administration 2002 policy on invention  disclosures, which was revised in January 2017, VA employees who  invent something are directed to disclose those inventions to VA using a  disclosure form and complete a certification form to certify whether VA  resources were used. VA employees are to disclose inventions to VA  even if they were not created with VA resources. Affiliated universities  may also require dually appointed researchers to disclose inventions to  the university. Under agreements between the universities and VA,  universities are required to disclose a dually appointed researcher’s  invention to VA, as an additional assurance to aid VA in capturing  relevant inventions. Similarly, VA is to notify the university when a dually  appointed researcher’s invention comes to its attention.

According to VA policy, researchers’ supervisors or research  administrators at VA medical centers are to review the disclosure forms  and send them to the VA Technology Transfer Office. The office  evaluates the information and provides a recommendation to VA’s  General Counsel on whether VA should assert ownership. If General  Counsel’s review finds that VA should assert ownership, the General  Counsel notifies the VA researcher’s and the VA medical center’s  research and development office of the determination. The Technology  Transfer Office then notifies the researcher’s university about VA’s  ownership of the invention. At this point the department expects the  university to include VA as an owner during the patenting process,  according to VA officials. Figure 1 shows VA’s process for determining  ownership of inventions created by dually appointed researchers,  according to VA policies.

If the university takes the lead on an invention of a dually appointed  researcher, original VA agreements require universities to provide annual  reports to update VA on commercialization activities, such as progress in  licensing inventions or collecting royalties from licensees. While less  commonly used, alternative processes for commercialization are shown in  appendix I.

We and others have identified a number of challenges associated with  technology transfer from federal research facilities. For example, we  found that technology transfer is often not a priority for laboratory  managers; researchers may not understand the potential commercial  applicability of their innovations; or the technologies are often not  developed enough for use in market-ready products and may require  investment of additional time and money to develop. We also have  reported that pharmaceutical inventions in particular may take a relatively  long time to develop. For example, the entire discovery, development,  and review process of a new drug can take up to 15 years.

VA Has Taken Steps to Educate Researchers and Universities about Requirements but Could Enhance Researchers’ Training

VA Has Taken Steps to Educate Researchers but Reported that Researchers Have Not Always Disclosed Their Inventions

Although VA has taken steps to educate researchers about disclosure of  inventions, VA officials reported that the researchers have not  consistently disclosed inventions to the department because they did not  always fully understand VA’s disclosure policy. Officials from VA’s  technology transfer office told us on multiple occasions that they believed  researchers did not consistently disclose inventions. For example, in  December 2016, VA officials said that once the technology transfer office  began sending researchers e-mail notices about the need to disclose  inventions, the number of disclosures increased, which they said  suggested underreporting had been occurring. In March 2017, the  officials told us that many of the inventions from more than 50  researchers during a 5-year period at one university had not been  disclosed until VA checked with the university and discovered the error.  By November 2017, VA technology transfer officials thought disclosure  had improved throughout VA, but they were still not able to describe the  extent of the problem.

The researchers we interviewed at the six medical centers in our sample  generally believed that they had properly disclosed inventions. However,  according to VA officials, a university official, and two VA researchers,  there could be several reasons that contributed to researchers not  consistently disclosing their inventions to VA, including the following:

Researchers may have disclosed inventions to their university,  assuming the university would disclose them to VA on their behalf.

Researchers may have disclosed their inventions to the university  because it was more convenient than disclosing to VA, as the  university’s technology transfer officials were more accessible to  answer questions.

Researchers were not familiar with VA’s invention disclosure process  because the process was not routine to them.

Researchers may have believed they did not use VA resources and  did not realize they were still required to disclose to VA.

VA research administrators may not always have reminded  researchers of the need to disclose inventions, as they did not  consider this requirement a priority.

VA made efforts since fiscal year 2016 to inform researchers about its  disclosure policy. For example, according to VA officials, the department  has increased its in-person communication with VA researchers. In the  first 8 months of fiscal year 2017, VA staff made 26 visits to universities  and VA medical centers to meet with researchers to encourage the  disclosure of inventions. However, VA officials said participation rates  among researchers at these voluntary meetings were low in some cases.  At one medical center, only the research administrator and one other  researcher attended the meeting, according to the administrator.

In addition, VA established an online training program in 2017 covering  the invention disclosure process, but the training is not mandatory. VA  provided us with a report from October 2017 indicating that out of over  3,000 eligible researchers, 130 had taken the training (about 4 percent).  One VA research administrator said that mandatory training would be  helpful. Under federal internal control standards, management is to  internally communicate the necessary quality information to achieve the  entity’s objectives, such as by communicating that information down and  across reporting lines to enable personnel to perform key roles in  achieving those objectives. Given that VA has not made the meetings or  online training on disclosure policy mandatory, its importance may not be  clear to all researchers. Also, because researchers do not make  discoveries every year, and the process is not routine, taking such  training once may not be sufficient to educate users. Without requiring  researchers to take online training on the invention disclosure process  annually, researchers may not be fully informed about the requirement to  disclose inventions, which can result in lost technology transfer  opportunities and lost royalties for VA if the inventions are not disclosed.

VA Has Taken Steps to Make Universities Aware of VA Researchers and Disclosure Requirements

Based on our interviews with VA and university officials in our sample,  since fiscal year 2016, the department took steps to make universities  aware of VA researchers and their disclosure requirements in an effort to  improve university disclosures to VA. We reviewed 16 agreements  between VA and affiliated universities, including the five universities with  agreements in our sample, and all of the universities agreed to disclose  joint inventions to the department. However, VA officials we interviewed  said that universities may not always disclose all inventions to VA.  Although they said they could not identify the extent of the problem, the  officials highlighted one university in our sample that had not disclosed  inventions to VA for at least 5 years. This university did not disclose  inventions to VA, as agreed, until prompted by VA’s technology transfer  office late in fiscal year 2016. Responsible university officials said they  had assumed the dually appointed researchers were disclosing the  inventions to VA. According to VA officials, when the VA technology  transfer office received a report from the university in fiscal year 2017 that  covered 5 years of disclosures, VA learned it had not received 80 percent  of the disclosures from that university for that period. VA officials said  they had not contacted the university sooner because their technology  transfer office had been understaffed until early in calendar year 2016. VA  officials from the technology transfer office had not identified a similar  problem of this magnitude with the other universities, including those in  our sample.

According to our interviews with VA and university officials, some of VA’s  university partners may not have been aware of which researchers were  also VA employees because the universities’ lists of VA researchers were  not current and universities generally relied on the researchers to disclose  whether they were VA employees. Furthermore, in some cases, the  university disclosure forms did not specifically ask whether the researcher  also worked at VA. For example, two of the six forms used by universities  in our sample did not specifically ask the researcher to indicate whether  they were VA employees. Upon recognizing some shortcomings in  universities’ disclosures to VA, the department provided current lists of  VA researchers at affiliated VA medical centers to their respective  universities in fiscal year 2017, and VA technology transfer officials said  they intend to provide such updated lists to the universities semi-annually.  VA officials said that universities may not be using these lists, but they will  not know until time has elapsed. VA technology transfer officials said their  site visits to VA medical centers—they conducted 26 visits in fiscal year  2017—along with other communications with their counterparts at the  universities should help the disclosure process.

VA Increased Communication with Universities about Reporting Commercialization Activities but Has Not Ensured that Such Activities Are Consistently Reported

VA has increased communication with universities since 2016 to help  ensure that universities report information about commercialization  activities for joint inventions, but universities’ reporting remained  inconsistent as of January 2018, according to VA. Under the original  agreements, such as the ones in our sample of eight agreements,  universities have the exclusive right to license and commercialize joint  inventions. VA’s awareness of the commercialization of such inventions  depends on universities providing this information through annual reports,  as required by the agreements. However, according to VA officials, prior  to 2011, only about 20 percent of the 79 universities with which VA has  agreements submitted annual reports. According to VA officials, VA made  an effort to increase annual reporting, and by 2013 it was up to 80  percent. The officials said, however, that the percentage of universities  submitting annual reports dropped again after losing staff in the  technology transfer office—the office retained only three staff in  subsequent years until fiscal year 2017 when there were 11 staff,  including 5 technology transfer specialists.

In addition, VA officials we interviewed said that there was some  confusion among universities regarding when they needed to submit  annual reports. For example, they said that some universities may not  have understood whether they needed to provide annual reports during  years when there was no new patenting or licensing activity. The officials  said that this was at least part of the reason some universities did not  submit annual reports. VA officials told us that they expect universities to  provide annual reports even when there is no new patenting or licensing  activity, and in fiscal year 2016 technology transfer officials e-mailed  universities to clarify this expectation. The officials also said that in  October 2016 they sent a letter to each of the 79 universities with which  the department has agreements to remind universities to submit the  required annual reports. Further, as stated earlier, VA staff made 26 visits  to VA medical centers and universities in the first 8 months of fiscal year  2017 to discuss reporting and disclosure requirements. However, VA  reported that 24 percent of the 79 affiliated universities provided annual  reports in fiscal year 2017 even after VA’s outreach. Because they did not  always receive annual reports, VA officials said they were often not aware  of a joint license until the university sent VA the first royalty check for a  joint invention. VA officials said they plan to conduct audits to check the  accuracy of university information.

Beginning in fiscal year 2015, VA began creating new agreements with  universities to give VA enhanced responsibility in licensing and  commercialization of joint inventions. By the end of fiscal year 2017, VA  had new agreements in place with 11 of the 79 universities. Based on our  review of 8 of the new agreements, VA will now, for the first time, have  the option to take the lead in licensing joint inventions. For inventions for  which VA does not take the lead role, under the new agreements, it will  have the right to review and provide input on all joint licenses. This new  provision improves VA’s awareness of any joint licenses created in the  future. However, because original agreements did not include this  provision, VA will still need to rely on accurate and updated annual  reports from universities for information on licenses negotiated under  those agreements.

In addition, the new agreements do not improve or clarify language from  the original agreements about what details need to be included in the  annual reports. According to our analysis, these eight new agreements,  similar to the original eight agreements we reviewed, do not contain  details on the specific information and format in which to present the  annual report. For example, both the original and four of the new  agreements we reviewed require universities to provide an annual  report, but four other new agreements state that the universities will  provide annual reports upon request. The original agreements as well as  all eight of the new agreements indicate that reports should include the  status of all patent prosecution, commercial development, and licensing  activity on joint inventions but do not explain whether an annual report is  needed if there has been no commercialization activity. As noted above,  VA officials said universities were confused about whether they were  required to report to VA if they had no new activity in a given period;  however, VA officials told us they still need reports in these situations.

Furthermore, based on our analysis of 12 annual reports from eight  universities, the format and content of the reports has been inconsistent.  Four universities submitted reports in a spreadsheet format; two  universities submitted reports in portable document format (PDF); one  university submitted a report in a Word format; and one submitted five  different documents, including both PDF and spreadsheet. In addition to  differences in format, the annual reports differed considerably in the  content they provided—the more detailed annual reports included patent  application numbers, patent expenses, the status of patent applications,  and information about whether the patent had been licensed. In contrast,  the less detailed annual reports did not provide any of this information on  patents for the joint inventions. Moreover, one university only included  active license agreements in its annual report, while other universities  also included license agreements that were terminated. VA officials we  interviewed agreed that the reports are not very detailed or standardized  but said they would like to eventually standardize the annual report format  and content so they can use the reports to track and audit joint inventions  The differences in annual reports exist because VA has not provided the  universities with a standardized method for reporting, including the format  that should be used for the annual reports and the content to include in  them. Under federal standards for internal control, management should  design control activities to achieve objectives and respond to risks. Such  control activities include providing a standardized method that guides  universities in fulfilling VA’s reporting requirements to ensure the  objectives of the program are being achieved. Without providing a  standardized method that clearly guides universities in fulfilling VA’s  reporting requirements for these annual reports, including their format and  content, the department will not be able to ensure detailed and  standardized annual reports that include details about licenses and  royalties. VA officials said that they were working on a template for  universities to use in reporting on commercialization activities for joint  inventions. However it is not clear whether the template would inform  universities of VA’s requirements to submit an annual report even if they  had no new commercialization activity in a given period.

VA manages a research program unique within the federal government in  that most of its researchers are dually appointed to universities, and their  inventions are jointly owned by VA and the universities, which typically  take the lead on commercialization activities. While VA has taken steps to  educate researchers about requirements for researchers and universities  to disclose inventions to VA, VA officials reported that researchers have  not consistently done so, because they did not always fully understand  the policy. Given that VA has not made its online training on disclosure  policy mandatory, the policy’s importance may not be clear to  researchers. Also, because researchers do not make discoveries every  year, and the process of disclosure is not routine, taking such training  once may not be sufficient. Without requiring researchers to take online  training on the invention disclosure process annually, researchers may  not be fully informed about the requirement to disclose inventions, which  can result in lost technology transfer opportunities as well as lost royalties  for VA if the inventions are not disclosed.

VA has also taken steps to improve communication with universities to  increase reporting of commercialization activities, but said that such  reporting by universities is inconsistent, and VA may not have adequate  information to account for all of its licenses and royalties. Without  providing a standardized method that clearly guides universities in  fulfilling VA’s reporting requirements for these annual reports, including  their format and content, the department will not be able to ensure  detailed and standardized annual reports.

We are making the following two recommendations to VA:  The Under Secretary of Health should make VA’s online training on  invention disclosure mandatory for researchers and require that it be  completed annually. (Recommendation 1)

The Under Secretary of Health should provide a standardized method  that guides universities in fulfilling VA’s reporting requirements for these  annual reports, including their format and content. (Recommendation 2)

We provided a draft of this report to the Department of Veterans Affairs  for review and comment. In written comments reproduced in appendix II,  VA agreed with our recommendations. Specifically, for our first  recommendation, VA said it will develop a plan to ensure its researchers  complete online technology transfer training on invention disclosure  annually. Furthermore, the plan will contain contingencies for those who  do not meet the requirements. The department expects to issue a training  requirement, train staff, and also demonstrate training is done by  September 2019. In addition, for our second recommendation, VA said it  will develop a standardized method that guides universities in fulfilling  VA’s reporting requirements for the university technology transfer annual  reports. VA has a target completion date of December 2018. VA also  provided technical comments, which we incorporated as appropriate.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At this time, we will send copies of this report to the  appropriate congressional committees, the Secretary of Veterans Affairs,  and other interested parties. In addition, the report is available at no  charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please  contact me at (202) 512-3841 or neumannj@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. GAO staff who made key contributions to  the report are listed in appendix III.

Appendix I: Department of Veterans Affairs Commercialization Process, Revenue, and Total Joint Patents with Universities

The process for commercializing a Department of Veterans Affairs (VA)  invention can take several avenues. Generally a university takes the lead  on inventions of dual appointees who work for VA and a university, and  VA researchers who are not dually appointed rely on VA to patent and  license their inventions. Also, VA can take the lead on joint inventions, for  example, if the university is not interested in ownership. (see fig. 2).

VA received about $316,000 in royalties from 45 licenses for its  inventions in fiscal year 2016 (see table 1). VA has U.S. and foreign  patents. From calendar years 2000 through November 2017, the U.S.  Patent and Trademark Office has granted VA 82 patents for which VA is  the sole assignee, according to VA officials. Also, table 2 shows by  university the breakdown of the 206 patents for which VA shares  ownership with an affiliate.

Appendix II: Comments from the Department of Veterans Affairs

Appendix III: GAO Contacts and Staff Acknowledgments

John Neumann, (202) 512-3841 or neumannj@gao.gov.

In addition to the contact named above, Rob Marek (Assistant Director),  Daniel Semick (Analyst in Charge), Ivelisse Aviles, Navaiyoti Barkakati,  Kevin Bray, Ellen Fried, Matthew Hunter, Cynthia Saunders, Dan C.  Royer, Ardith Spence, and Kiki Theodoropoulos made key contributions  to this report.

Did you know that the Department of Veterans Affairs employed researchers who helped develop the pacemaker in 1960? Since 2000, VA research has resulted in VA-owned patents. VA can grant licenses to private sector firms to make its medical technologies commercially available while VA collects the royalties.

To ensure this happens, VA requires its researchers and the universities that employ them to disclose their inventions.

We found inconsistent disclosures from researchers and low turnout in VA's training on disclosures. We also found that VA’s reporting requirements for universities were unclear—and recommended ways to address these issues.