Transitioning Veterans: Coast Guard Needs to Improve Data Quality and Monitoring of Its Transition Assistance Program

Thousands of Coast Guard servicemembers have left the military and transitioned into civilian life, and some of these new veterans may face significant challenges, such as finding and maintaining employment. To help them prepare, federal law mandated that DHS provide separating Coast Guard servicemembers with counseling, employment assistance, and information on veterans' benefits through TAP. GAO was asked to examine TAP implementation.

This review analyzes (1) the reliability of TAP data on participation levels for Coast Guard servicemembers and the factors that affect participation, and (2) the Coast Guard's performance measures and monitoring efforts related to TAP. GAO interviewed Coast Guard headquarters staff; surveyed 12 Coast Guard installations that conduct TAP (100 percent response rate); collected and reviewed participation data for reliability; and interviewed TAP managers from three installations selected for size and location, and 25 Coast Guard servicemembers at one location. (For a companion report on TAP implementation for separating and retiring servicemembers in other military services, see GAO-18-23 .)

The United States Coast Guard (Coast Guard), which is overseen by the Department of Homeland Security (DHS), lacks complete or reliable data on participation in the Transition Assistance Program (TAP), designed to assist servicemembers returning to civilian life. According to senior Coast Guard officials, a major reason why data are not reliable is the lack of an up-to-date Commandant Instruction that specifies when to record TAP participation data. Consequently, the data are updated on an ad-hoc basis and may not be timely or complete, according to officials. Federal internal control standards call for management to use quality information to achieve the entity's objectives. Until the Coast Guard issues an up-to-date Commandant Instruction that establishes policies and procedures to improve the reliability and completeness of TAP data, it will lack quality information to gauge the extent to which it is meeting TAP participation requirements in the VOW to Hire Heroes Act of 2011.

According to GAO's survey of Coast Guard installations, various factors affected participation, such as servicemembers serving at geographically remote locations or separating from the Coast Guard rapidly. TAP officials and Coast Guard servicemembers GAO interviewed said commanders and direct supervisors sometimes pulled servicemembers out of TAP class or postponed participation because of mission priorities. TAP managers also said they rely on delivering TAP online because many Coast Guard servicemembers are stationed remotely.

The Coast Guard cannot effectively measure performance to ensure key TAP requirements are met because it lacks reliable data and does not monitor compliance with several TAP requirements. Further, the Coast Guard has not established a formal performance goal against which it can measure progress, although federal internal control standards stipulate that management should consider external requirements—such as the laws with which the entity is required to comply—to clearly define objectives in specific and measurable terms. Establishing a goal could help the Coast Guard define expected performance. In addition, the Coast Guard does not monitor TAP requirements regarding the timeliness of servicemembers' TAP participation or their access to additional 2-day classes. Consequently, it cannot know whether servicemembers are starting TAP early enough to complete the program or those who elected to attend additional 2-day classes were able to do so before separation or retirement, as required by the Act. Finally, the Coast Guard lacks an up-to-date Commandant Instruction that establishes the roles and responsibilities of Coast Guard staff in implementing TAP. Federal internal control standards stipulate that management should assign responsibility and delegate authority to key roles throughout the entity. Issuing an up-to-date Commandant Instruction that defines roles and responsibilities would clarify who is ultimately responsible for ensuring Coast Guard servicemembers attend TAP, thereby facilitating accountability.

GAO is making seven recommendations, including that the Coast Guard issue a new Commandant Instruction establishing data collection policies, set TAP performance goals, monitor timeliness and access, and define roles and responsibilities. DHS agreed with all of GAO's recommendations.

Over the past several years, thousands of Coast Guard servicemembers  have left the military with more soon to follow. Some of these new  veterans may face significant challenges as they transition to civilian life,  such as finding and maintaining employment. To help them, one section  of the VOW to Hire Heroes Act of 2011 (VOW Act) generally mandates  the Department of Homeland Security (DHS) to assure that all separating  Coast Guard servicemembers participate in the Transition Assistance  Program (TAP) to receive counseling, employment assistance, and  information on federal veteran benefits, among other supports. TAP is  designed to help all servicemembers, including those from the Coast  Guard, successfully prepare for civilian life.

Concurrently with the VOW Act, the prior administration initiated a  redesign of TAP which, among other changes, created a standardized  curriculum sometimes referred to as Transition GPS (Goals, Plans,  Success). The redesign was led by an interagency task force, which  included DHS. The task force developed an extended, core curriculum  for TAP that covered employment, veterans’ benefits, and financial  planning, among other topics. The redesigned TAP also included  additional 2-day classes for Coast Guard servicemembers interested in  pursuing higher education, career technical training, or entrepreneurship.

In 2014, GAO issued an initial report on TAP implementation. You asked  us to conduct a follow-up review of TAP. We developed two reports  based on our audit work—a previously issued report that focuses on the  Department of Defense’s (DOD) TAP implementation, and this report  that focuses on the Coast Guard’s implementation of TAP. This report  examines:  1.  What is known about the reliability of Transition Assistance Program  data on participation levels and the factors that affect Coast Guard  servicemembers’ participation?  2.  To what extent does the Coast Guard measure TAP performance and  monitor key areas of Transition Assistance Program implementation?

To address these questions, we surveyed 12 Coast Guard installations  with on-site TAP managers and achieved a 100 percent response rate.  We reviewed Coast Guard data on TAP participation for fiscal years 2012  to 2017 and interviewed Coast Guard officials in headquarters  responsible for overseeing TAP implementation for the Coast Guard. We  interviewed 3 of the Coast Guard’s 13 TAP managers—one in person and  two by phone—selected for diversity in location of the regions they  oversee. We visited one Coast Guard installation where, in addition to  interviewing the on-site TAP manager, we interviewed additional TAP  staff as well as 25 Coast Guard servicemembers (including both officers  and enlisted personnel) who had participated in the program.

We assessed the reliability of the Coast Guard’s TAP participation data  and, based on interviews with agency officials knowledgeable about the  data, determined these data were not sufficiently reliable due to  limitations with the Coast Guard’s data collection system, which does not  have sufficient controls to ensure data are complete and accurate. We  discuss the issues with data reliability in our report. We also reviewed  relevant federal laws, regulations, policies, documents, and publications.  Information in this report is current as of the date GAO received formal  agency comments from DHS. For more information on our methodology,  see appendix I.

We conducted this performance audit from February 2016 to April 2018 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Coast Guard Organizational Structure for TAP

Coast Guard staffing for the TAP program reflects the organizational  structure of its Health, Safety, and Work-Life Directorate, which oversees  TAP policy. The Coast Guard’s TAP managers are assigned to 13  installations where Health, Safety, and Work-Life offices are located.  One or two TAP managers are assigned to each of the Coast Guard’s  nine districts, which often span multiple states and territories, and these  TAP managers oversee operations both for the installation where they  work and for units stationed throughout the region (see fig. 1). For  example, the TAP manager assigned to Coast Guard Base Cleveland  oversees TAP implementation both for that installation and for Coast  Guard units serving in Coast Guard District 9—a region that  encompasses portions of eight states and the Great Lakes area. The  program manager in Coast Guard Headquarters manages Coast Guard’s  Transition Assistance Program. The Coast Guard protects and defends  over 100,000 miles of U.S. coastline and inland waterways, and  consequently, TAP-eligible Coast Guard servicemembers sometimes  work in small, widely dispersed units assigned to remote locations,  including on Coast Guard vessels. One aspect of the Coast Guard’s  mission—a first responder for maritime search and rescue in United  States waters—can require Coast Guard servicemembers to respond to  emergency situations at a moment’s notice.

The Coast Guard, which is overseen by DHS, not DOD, generally  oversees TAP implementation for its servicemembers. Federal law  requires DOD and DHS to require eligible servicemembers under their  respective command to participate in TAP, with some exceptions. In  response to this statutory requirement, DOD has promulgated regulations  and developed issuances which require that servicemembers complete  the component parts of the TAP program, and that commanding officers  ensure that servicemembers under their command complete these parts,  with some exceptions. In contrast, according to Coast Guard officials,  Coast Guard has not promulgated any regulations to implement TAP.  Further, Coast Guard issued its most recent Commandant Instruction in  2003, approximately 8 years prior to TAP redesign in 2011. However,  Coast Guard issued policy guidance in 2014 that made some limited  updates to the Commandant Instruction. Coast Guard officials also said  the Coast Guard plans to issue a new TAP Commandant Instruction in  May 2018.

Under the redesigned TAP, Coast Guard servicemembers—like their  DOD counterparts—begin TAP by attending pre-separation or transition  counseling where they are briefed on TAP requirements and available  transition resources. Pre-separation or transition counseling can be  delivered by TAP managers, uniformed career counselors, or online (see  fig. 2). Coast Guard servicemembers are able to participate in TAP  either through the Coast Guard or at a DOD installation, if space is  available.

During or at the end of pre-separation or transition counseling,  participants register for and attend TAP courses. The core curriculum  includes three required courses—the Department of Labor (DOL)  Employment Workshop, unless exempt, and Department of Veterans  Affairs (VA) Benefits Briefings I and II—and other courses that focus on  aspects such as translating military skills and experiences into  credentialing for civilian jobs and preparing a financial plan. Participants  may also elect to attend additional 2-day classes either at a Coast Guard  or DOD installation or online through the Joint Knowledge Online  platform, according to agency officials. These additional 2-day classes  include Accessing Higher Education, Career Technical Training, and  Entrepreneurship. Federal law requires the Coast Guard to permit  servicemembers who elect to take these additional 2-day classes to  receive them.

Federal law establishes a time frame within which servicemembers with  anticipated separation or retirement dates should begin the program.  According to federal law, retirees with anticipated separation dates are  expected to begin TAP as soon as possible during the 24-month period  preceding that date, but not later than 90 days before separation.  Similarly, servicemembers with anticipated separation dates who are not  retiring are expected to begin as soon as possible during the 12-month  period preceding that date, but not later than 90 days before separation.  Servicemembers who learn that they will separate or retire from the  military fewer than 90 days before their anticipated separation or  retirement date are expected to begin TAP as soon as possible within  their remaining period of service.

As we previously reported, officials from multiple federal agencies  collaborate to deliver and assess TAP. The TAP interagency  governance structure includes senior officials from DOD, VA, DOL, DHS,  the Department of Education, the U.S. Office of Personnel Management,  and the Small Business Administration (SBA), who participate in TAP  Senior Steering Group meetings at least every month and TAP Executive  Council meetings each quarter. Further, officials tasked to particular  interagency working groups focus on specific elements of TAP (e.g.,  curriculum or performance measures), meet more frequently (typically at  least once a month), and generally communicate weekly, according to  agency officials. The TAP program manager for the Coast Guard told us  that he participates in several of the working groups.

One such working group is the performance management working group  that oversees the interagency TAP evaluation plan, which includes  monitoring performance measures related to TAP requirements,  indicators of post-program outcomes, and formal evaluations sponsored  by interagency partners. While DOD tracks TAP-specific performance  measures, other interagency partners track indicators of how well  veterans fare after leaving military service. For example, DOD tracks  performance measures prior to servicemembers’ separation, such as TAP  participation and credential attainment rates, while other agencies track  post-separation indicators, such as unemployment rates among veterans  ages 18 to 24. The performance management working group also  reviews the formal evaluation efforts led by individual agencies and  provides feedback to help shape their efforts in accordance with the TAP  Evaluation Plan.

Coast Guard Lacks Reliable Data and Cites Several Factors that Affect Participation Coast Guard Lacks Reliable Data on Servicemembers’ Participation in TAP

The Coast Guard does not have complete or reliable data on participation  levels in TAP. According to Coast Guard officials, a major reason why  the data are not reliable is that the Coast Guard lacks an up-to-date  Commandant Instruction that specifies when to record TAP participation  data. Consequently, the data are updated on an ad-hoc basis,  according to agency officials, and may not be timely or complete. For  example, one TAP manager said she updates the list of TAP participants  for her installation only once every few months because of her other  duties. According to federal internal control standards, management  should use quality information—including current and timely information— to achieve the entity’s objectives and to communicate quality information  to external parties. Given the lack of timely and complete data, we  determined the Coast Guard’s TAP data were not sufficiently reliable for an analysis of participation in TAP classes. Because it lacks policies and  procedures governing reliable data collection, including when data should  be entered and by whom, the Coast Guard cannot determine to what  extent its servicemembers attend TAP, although federal law mandates  that DHS ensure all TAP-eligible servicemembers participate in the  program.

In addition, the data collection system currently used to track TAP  participation is not sufficient to ensure reliable data. For example,  according to Coast Guard staff, TAP staff enter TAP participation data  into a shared spreadsheet that all TAP managers can edit. Specifically,  staff record the names of servicemembers they identify as TAP-eligible  and whether these individuals completed required portions of TAP. Coast  Guard officials said they are in the process of adopting a new data  system, in October 2018, to more reliably track TAP participation and that  they expect to fully adopt this system–DOD’s TAP-IT Enterprise  System—after a new Commandant Instruction is finalized, in May 2018.  In November 2016, DOD launched the new system to collect TAP-related  data for servicemembers in the Army, Navy, Air Force, and Marine  Corps. In addition to standardizing data collection and improving data  completeness and accuracy, the TAP-IT Enterprise System is expected to  track information related to the time frames of servicemembers’  participation. According to a senior DOD official, the military services will  not be able to use the system to generate unit-level or installation-level  reports until October 2018.

Serving at a Remote Installation and Rapid Separations Hindered TAP Participation, As Did Limited Staff Capacity and Competing Priorities

According to our survey, the most common factor affecting TAP  participation, cited at 11 of the 12 Coast Guard installations we surveyed,  pertained to servicemembers assigned to geographically remote  locations. The next three most commonly cited factors–each cited by 7 of  the 12 installations surveyed—relate to the timing of TAP participation:  rapid separation from the military, not being sufficiently aware of the need  to attend TAP, and starting the transition process too late to attend. (See  fig. 3.) Headquarters-based TAP officials identified additional factors that  may affect servicemember participation, such as separating from the  Coast Guard Reserves or retiring with no plans to work after leaving the  military. However, the Coast Guard lacks participation data to verify  whether participation rates for these groups are in fact lower than for  other Coast Guard servicemembers.

Coast Guard installations we surveyed did not indicate that unit  commanders or direct supervisors affected participation in TAP’s required  courses or additional 2-day classes. However, Coast Guard  servicemembers and TAP officials we spoke with said unit commanders  or direct supervisors sometimes prevented participation. All three TAP  managers we spoke with (of 12 nationwide) told us that while  commanders generally allowed servicemembers to register for TAP  courses, they occasionally required them to return to their duties before  completing the courses. We observed this during a TAP class at a Coast  Guard installation we visited when a servicemember’s commander  ordered her to return to the unit during TAP training and she missed a  briefing she wanted to attend. Two of three TAP managers we  interviewed also said commanders sometimes required servicemembers  under their command to wait to take TAP classes until close to their  separation date because of mission priorities.

Two of three TAP managers interviewed said that commanders in the  Coast Guard face unique challenges in ensuring TAP participation. They  said commanders in all branches of the military must balance competing  demands, including their primary mission and the training needs of the  personnel they oversee. They said it can be particularly difficult for Coast  Guard commanders to juggle these priorities because Coast Guard  servicemembers are sometimes assigned to very small units or called to  return to duty for emergency situations during scheduled TAP classes.  One TAP manager said that a commander in a remote location had  collaborated with her to provide a classroom-based TAP class for  transitioning Coast Guard servicemembers within the commander’s unit,  but rescue efforts occurred during the class which resulted in most of  those servicemembers returning to their vessel to respond to the  emergency. In addition, all three TAP managers we spoke with said there  are limited resources for holding TAP in a classroom setting.  Consequently, classroom-based TAP may not be offered frequently in  remote locations, making rescheduling difficult. One TAP manager said  that her installation typically offers three or four TAP classes a year and  because classes are so infrequent, servicemembers are encouraged to  start TAP as soon as possible prior to separation.

Coast Guard staff we interviewed said that juggling competing priorities  affected the Coast Guard’s ability to implement TAP. Both the frontline  and headquarters staff who oversee TAP implementation said they  oversee at least three other programs in addition to TAP at their  installation and throughout their regions, including the Coast Guard’s  relocation and spousal employment programs.

Coast Guard Relies on Online Delivery of TAP for Several Categories of Servicemembers

The Coast Guard relies on online delivery of TAP information and classes  for servicemembers who are rapidly separating and assigned to remote  and geographically dispersed units, according to our survey results and  several Coast Guard staff we interviewed. For example, all 12  installations we surveyed cited servicemembers facing rapid separations  as a reason for accessing TAP training online, and 11 cited  servicemembers being remotely stationed as a reason. Coast Guard staff  also said online TAP was used for servicemembers interested in  attending additional 2-day classes.

The three TAP managers we interviewed also identified several reasons  why installations had to rely on online TAP classes. For example, one  manager corroborated our survey results, saying that many Coast Guard  servicemembers worked in small units assigned to remote and  geographically dispersed locations, making it difficult to convene a  sufficient number of transitioning Coast Guard servicemembers to meet  minimum class size requirements. In addition, all three managers said  they used the online version of TAP for remotely stationed Coast Guard  servicemembers because the Coast Guard lacked the resources for them  to attend classes in person. Although they preferred that  servicemembers participate in live, classroom-based TAP classes, all of  the managers acknowledged that the online version of TAP played an  integral role in ensuring that more servicemembers could participate in  the program. However, two of them noted that while classroom delivery of  TAP classes provided an interactive learning environment that allowed  participants to ask questions and learn from their peers, online  participants generally clicked quickly through the slides and had difficulty  understanding the information being presented. Two managers told us  that they regularly used the online version to deliver parts of the TAP  curriculum. For example, one TAP manager said she required  participants to complete the crosswalk of military and civilian occupations  class online before attending required classes in person. Two managers  noted that additional 2-day classes were available online, and one noted  that some servicemembers attended these classes in a classroom setting  either on a Coast Guard base or a DOD installation. Finally, all three TAP  managers said that many participants in online TAP classes would benefit  from participating in a real-time virtual version of TAP led by live  facilitators. Two managers told us that having a remote facilitator  delivering TAP in real time would give participants more opportunity to  ask questions and better understand and absorb class content.

Feedback About TAP Was Generally Positive

Despite these challenges, TAP managers and separating Coast Guard  servicemembers we interviewed provided generally positive feedback  about the TAP program. All of the 25 Coast Guard servicemembers we   spoke with said that the information they received during the courses was  useful and they liked the instructors. One Coast Guard servicemember  praised the classroom courses for being interactive, and several Coast  Guard servicemembers said they wanted the opportunity to retake TAP  before or shortly after they separated from the Coast Guard. However,  many said the volume of information presented in a short period of time  could be overwhelming and was like “trying to drink from a firehose.”

Coast Guard Cannot Effectively Measure Performance or Monitor Implementation to Ensure Key TAP Requirements Are Met

Coast Guard Has Not Set a Formal Performance Goal for TAP Participation and Cannot Effectively Measure Program Performance Because it Lacks Reliable Data

The Coast Guard has not set a formal performance goal for TAP  participation, according to a Coast Guard official, and as previously  discussed, does not have complete, reliable data. Without reliable  information, the Coast Guard cannot effectively monitor TAP  implementation or measure program performance. DHS is mandated to  ensure that all TAP- eligible servicemembers of the Coast Guard  participate in TAP before leaving military service. However, without  effective monitoring of program participation, the Coast Guard cannot  know to what extent its servicemembers receive the required training they  need to prepare for civilian life. According to federal internal control  standards, management should consider external requirements—such as  the laws with which the entity is required to comply—to clearly define  objectives in specific and measurable terms. In addition, establishing  goals can help agencies define expected performance and articulate  results. A Coast Guard official said the Coast Guard’s long-term goal is  for full compliance with TAP requirements, but in the interim, the Coast  Guard uses DOD’s 85 percent VOW compliance goal as an informal  benchmark against which to gauge the Coast Guard’s TAP  performance. However, the Coast Guard has not communicated a  specific, measurable goal to TAP staff implementing the program, or to  Coast Guard commanders who oversee separating and retiring Coast  Guard servicemembers, according to a Coast Guard official. Establishing  and communicating a formal goal could help the Coast Guard define  expected performance. The official also told us that, like DOD, the Coast  Guard tracks the elements of TAP mandated under the VOW Act— transition or pre-separation counseling, VA Benefits I and II, and the DOL  Employment Workshop.

Coast Guard Does Not Monitor Compliance with Additional TAP Requirements

The Coast Guard does not monitor the (1) timeliness of participation in  TAP, and (2) access to additional 2-day classes. A Coast Guard official  said the Coast Guard does not currently monitor TAP beyond tracking  whether separating servicemembers participate in the required courses,  and currently lacks the capacity to undertake additional monitoring efforts.  However, he said additional monitoring would be possible once the Coast  Guard completed the move to the DOD TAP-IT Enterprise data system.

According to a Coast Guard official, the Coast Guard does not currently  monitor the timeliness of TAP participation although federal law  prescribes time frames for servicemembers to begin TAP participation.  Generally, separating servicemembers who are not retiring are to begin  TAP participation no later than 90 days before their separation date.  Without a systematic method for monitoring timeliness, the Coast Guard  cannot know whether its servicemembers begin the program on time or  account for the timeliness of TAP participation. As a result, the Coast  Guard cannot know whether its servicemembers are starting TAP early  enough to complete the training they need to adequately prepare for their  transition to civilian life.

The Coast Guard does not track which of its servicemembers participate  in the additional 2-day classes, according to a Coast Guard official we  interviewed, even though federal law requires that DHS ensure those who  elect to participate are able to receive the training. By not tracking which Coast Guard servicemembers participate in 2-day classes or requiring  transition staff to document when servicemembers ask to attend, the  Coast Guard cannot determine the extent to which servicemembers who  wished to attend these courses were able to do so, as required by law.

Roles and Responsibilities Are Not Clearly Defined

Coast Guard commanders and TAP managers do not have clearly  defined roles and responsibilities in implementing TAP because of the  lack of an up-to-date Commandant Instruction, according to TAP staff we  interviewed. As previously discussed, the Coast Guard’s last  Commandant Instruction on TAP was issued in 2003, approximately 8  years prior to TAP’s redesign. According to federal internal control  standards, to achieve the entity’s objectives, management should assign  responsibility and delegate authority to key roles throughout the entity.  Without an up-to-date Commandant Instruction, TAP managers and  commanders may be unclear on who is ultimately responsible for  ensuring servicemembers attend TAP. Moreover, two TAP managers also  told us that an up-to-date Commandant Instruction might lead some  commanders to place higher priority on ensuring TAP participation. Coast  Guard officials said the Coast Guard was in the process of revising the  TAP Commandant Instruction and anticipated issuing the new instruction  in May 2018.

Coast Guard Does Not Share Participation or Performance Data with Commanders or TAP Interagency Partners, Limiting Monitoring and Evaluation

The Coast Guard lacks the ability to share data with commanders, limiting  its ability to monitor TAP participation and ensure servicemembers attend  the program. According to a Coast Guard official, the Coast Guard’s  current data collection system also cannot generate installation or unit- level participation rates to share with commanders who oversee  transitioning and retiring servicemembers. Federal internal control  standards state that management should share quality information  throughout an organization to enable personnel to perform key roles, and  we have previously reported that by regularly sharing useful performance  information with leaders at multiple levels of an organization, agencies  can help leaders make informed decisions. Without this information,  individual unit commanders or the commanders’ supervisors cannot  determine whether Coast Guard servicemembers under their command  completed TAP or identify whether there is a need for corrective actions  to ensure they do so.

As we mention earlier in this report, the Coast Guard plans to adopt  DOD’s TAP-IT Enterprise System, which according to officials, could help  the Coast Guard ensure eligible servicemembers participate in the  program. According to a Coast Guard official, once the system is fully  implemented by the Coast Guard, commanders will be required to verify  and document whether Coast Guard servicemembers under their  command completed TAP, potentially making commanders more vested  in the process. We previously reported that a senior DOD official said that  the TAP-IT Enterprise System may be able to generate unit- and  installation-level reports for the four DOD-led military services by October  2018, and a Coast Guard official said he would work with DOD to identify  whether this capability could also extend to the Coast Guard. Once data  reliability improves, sharing installation and unit-level TAP performance  information with Coast Guard commanders could support monitoring  efforts.

The performance measures tracked by the TAP interagency working  group do not reflect TAP implementation broadly across all five military  services, according to a Coast Guard official we interviewed. The Coast  Guard does not currently share TAP data it collects with DOD or other  members of the interagency performance working group. While the  benefits of interagency data sharing cannot be realized without the Coast  Guard first improving the quality and completeness of its TAP data, we  have identified leading practices for interagency collaboration, including  that members of interagency working groups identify and share relevant  agency performance data. Moreover, federal internal control standards  call for management to communicate quality information to external  parties. Because the Coast Guard does not share TAP data, the  performance measures tracked by the interagency group do not reflect  Coast Guard servicemembers’ experiences and thus do not provide a  complete picture of TAP implementation across the five military services.  More specifically for the Coast Guard, without such data sharing, future  TAP evaluations may not be able to assess the effectiveness of TAP  delivery, hindering the Coast Guard’s ability to make program  adjustments to better prepare its servicemembers to successfully  transition to life after military service. Coast Guard officials said migrating  to DOD’s TAP-IT Enterprise System will facilitate information sharing with  interagency partners and that improving data completeness and reliability  is a top priority for 2018.

Given the sacrifices servicemembers have made to serve their country, it  is imperative they are afforded every chance to adequately prepare for  civilian life before leaving military service. In order to make a successful  transition, servicemembers need to be well-positioned to get a job or  make an informed decision about whether to pursue additional education  or start a small business. As such, the Transition Assistance Program  (TAP) serves a critically important function—to give servicemembers the  tools and information they need to successfully transition to life outside  the military. Federal law requires that the Coast Guard ensure all eligible  servicemembers participate in the program, but thousands of Coast  Guard servicemembers may have transitioned without the support  provided by TAP. Reliably tracking participation has proven to be a  challenge for the Coast Guard, in part because it lacks a current  Commandant Instruction that defines the roles and responsibilities of staff  responsible for implementing TAP and ensuring complete and reliable  data are collected. In preparing to issue an updated Commandant  Instruction, the Coast Guard has taken a positive step toward addressing  the limitations of its current TAP data, and will be better positioned to  ensure compliance with VOW Act requirements using reliable data.

In addition to collecting reliable data, the Coast Guard could further  demonstrate its commitment to meeting TAP requirements by  establishing formal performance goals that measure the extent to which  Coast Guard servicemembers participate in TAP. By establishing interim  performance goals, the agency would be able to show its progress  towards achieving full compliance. Moreover, communicating  performance goals to unit and installation commanders could enhance  accountability and might spur progress toward meeting federal program  requirements.

By expanding its monitoring efforts beyond tracking participation in TAP’s  required classes, the Coast Guard could enhance its ability to ensure  other TAP requirements are met and that its servicemembers are able to  access additional transition resources. Monitoring the timeliness of  participation would help ensure Coast Guard servicemembers have  adequate time to complete TAP before leaving the military. Further, by  monitoring requests to participate in additional 2-day classes and 2-day  class attendance, the Coast Guard would be in a better position to identify  whether servicemembers who wish to attend the classes are able to do  so, to determine whether more classes are needed, and to communicate  this information to the interagency partners responsible for delivering  these classes.

Commanders can also play a key role in bolstering TAP participation.  Having an up-to-date written Commandant Instruction that explicitly  describes commanders’ roles and responsibilities could enhance  commanders’ ability to ensure TAP’s proper implementation and  compliance with VOW Act requirements. Moreover, once data quality  improves, providing commanders a mechanism to readily determine  whether servicemembers under their command have completed TAP  could help them monitor the program to ensure that all TAP-eligible  servicemembers receive the resources they need to successfully  transition to civilian life.

Finally, once more reliable data on Coast Guard servicemember  participation are available, sharing this information with interagency  partners could improve TAP implementation on a broader scale. Sharing  reliable data, such as participation figures for the Coast Guard, would  give TAP interagency partners a more complete picture of implementation  across all five military services. Sharing such information would also  enhance the interagency group’s ability to evaluate how well TAP serves  the entire population of servicemembers. Improving the reliability of the  Coast Guard’s TAP data will be essential for the benefits of data sharing  to be realized.

To ensure that all eligible Coast Guard servicemembers are provided the  opportunity to complete the Transition Assistance Program (TAP), we  recommend the Commandant of the Coast Guard take the following  seven actions:  Issue an updated Commandant Instruction that establishes policies  and procedures to improve the reliability and completeness of TAP  data by including when and by whom data should be recorded and  updated. (Recommendation 1)

Establish a formal performance goal with a measurable target for  participation rates in VOW Act-mandated portions of TAP.  (Recommendation 2)

Monitor the extent to which Coast Guard servicemembers participate  in TAP within prescribed time frames. (Recommendation 3)

Monitor the extent to which Coast Guard servicemembers who elect  to participate in additional 2-day classes are afforded the opportunity  to attend. (Recommendation 4)

Issue an updated Commandant Instruction that defines the roles and  responsibilities of the personnel who administer the program and  ensure servicemembers’ participation. (Recommendation 5)

Once reliable data are available by installation or unit, enable unit  commanders and the higher-level commanders to whom they report  to access TAP performance information specifically for the units they  oversee so that they can monitor compliance with all TAP  requirements. (Recommendation 6)

Once reliable data are available, share TAP information with DOD and  other interagency partners, such as data on participation in required  TAP courses and additional 2-day classes. (Recommendation 7)

We provided a draft of this report to the Departments of Homeland  Security, Defense, Education, Labor, and Veterans Affairs, the Office of  Personnel Management, and the Small Business Administration for their  review and comment. The formal written response of the Department of  Homeland Security (DHS) is reproduced in appendix II. In addition, DHS  provided technical comments from Coast Guard officials that we  incorporated into the report as appropriate. The other agencies did not  provide any comments.

In its written comments, DHS agreed with all seven of our  recommendations.

If you or your staff have any questions about this report, please contact  me at (202) 512-7215 or brownbarnesc@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix III.

Appendix I: Objective, Scope, and Methodology

This report examines (1) what is known about the reliability of Transition  Assistance Program (TAP) data on participation levels and the factors  that affect Coast Guard servicemembers’ participation, and (2) the extent  to which the Coast Guard measures TAP performance and monitors key  areas of TAP implementation. To address these questions, we surveyed  Coast Guard installations with full-time TAP operations; reviewed Coast  Guard data on TAP participation for fiscal years 2012 to 2017; visited one  Coast Guard installation and interviewed TAP managers from two  additional Coast Guard installations selected for diversity in location,  among other reasons; and interviewed Coast Guard officials responsible  for overseeing TAP implementation for the Coast Guard. We also  reviewed relevant federal laws, regulations, policies, documents, and  publications. Information in this report is current as of the date GAO  received formal agency comments from DHS.

Our survey of Coast Guard installations with full-time TAP operations  asked about how TAP was being implemented. The survey included  questions about the accessibility of TAP components, challenges Coast  Guard servicemembers faced in attending the components, and the level  of commander support for participation. Our survey targeted front-line  TAP managers, who could draw on the expertise of TAP course  facilitators, transition counselors, career counselors, and other key TAP  staff as necessary.

After drafting the survey questions, we pretested them with a TAP  manager to ensure (1) the questions were clear and unambiguous, (2)  terminology was used correctly, (3) the survey did not place an undue  burden on agency officials, (4) the information could feasibly be obtained,  and (5) the survey was comprehensive and unbiased. We revised the  content and format of the survey based on the feedback we received.

We initially sent the survey to TAP managers at all 13 Coast Guard  installations at which TAP staff were located. We removed one installation  when we later found that the TAP manager position was vacant and  revised the total to 12 Coast Guard installations. The survey was  accessible online from October 31, 2016, through January 18, 2017,  through a secure server that recipients were able to access using unique  usernames and passwords. We sent an email announcement to TAP staff  at all 13 Coast Guard installations at which TAP staff are located on  October 24, 2016. We sent a second email on October 31, 2016 to notify  participants the survey was available online, and provided their unique  passwords and usernames. We sent two follow-up e-mails (November 14,  2016 and November 28, 2016) to those who had not responded. Finally,  we contacted all remaining nonrespondents by telephone starting  December 5, 2016. The survey was available online until we reached a  100 percent response rate.

Interviews with Coast Guard Installation TAP Staff and Servicemembers

To increase our understanding into how TAP was being implemented at  installations and supplement our survey findings, we visited one Coast  Guard installation and interviewed TAP managers from two additional  installations. We selected the installations based on several factors,  including the size of the installation, proximity to Department of Defense  (DOD) installations, and diverse locations in the United States. (See table  1.) At Coast Guard Base Elizabeth City in North Carolina, the installation  we visited, we interviewed the TAP manager, uniformed career  counselors, and senior installation leadership. During our interviews with  TAP managers at all three installations, we asked about the extent to  which Coast Guard servicemembers participate in TAP’s required and  additional 2-day classes, including whether the servicemembers attended  classes online or in a classroom setting, challenges to ensuring Coast  Guard servicemembers participate in TAP, and the extent to which they  monitor Coast Guard servicemembers’ participation in TAP. At Coast  Guard Base Elizabeth City, we also interviewed 25 Coast Guard  servicemembers (both officers and enlisted personnel) to get their  perspective on how well TAP worked and any challenges they had  participating. To help guide the interviews with the Coast Guard  servicemembers, we asked them to complete a short questionnaire that  asked about their experiences with the TAP program.

We also interviewed TAP staff at Coast Guard headquarters to learn  about TAP policy, monitoring efforts, and performance measures for the  service overall. For example, we asked what policies and procedures  guide installations’ TAP implementation; what performance measures the  Coast Guard uses to monitor TAP; how performance results are reported  and shared with different levels of Coast Guard leadership; and to what  extent the Coast Guard uses results from TAP participant satisfaction  assessments. We also asked whether the Coast Guard plans to shift to  DOD’s new TAP-IT Enterprise System and how using the new system  could affect its monitoring efforts in the future. In evaluating the Coast  Guard’s performance measures, we focused on measures related to  servicemembers’ transition experiences before leaving the military. We  did not gather information on post-program evaluations and outcomes  because they were determined to be outside the scope of this review.

We reviewed DHS data on TAP participation for fiscal years 2012 to  2017. To assess the reliability of the Coast Guard’s TAP participation  data, we interviewed agency officials knowledgeable about the data. We  determined these data were not sufficiently reliable due to limitations with  the Coast Guard’s data collection system. Specifically, the system lacks  adequate controls to ensure TAP data are complete and accurate.

Appendix III: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Meeta Engle (Assistant Director),  Amy MacDonald (Analyst-in-Charge), James Bennett, Holly Dye, David  Forgosh, Ying Long, Jonathan McMurray, Jean McSween, Andrew  Sherrill, Benjamin Sinoff, and Timothy Young, made significant  contributions to this report.

Also contributing to this report were Susan Aschoff, Jessie Battle,  Ramona Burton, Melinda Cordero, Elizabeth Curda, Dawn Hoff, Ben  Licht, Serena Lo, Sheila McCoy, Almeta Spencer, Christopher Schmitt,  James Whitcomb, and Jill Yost.

Transitioning Veterans: DOD Needs to Improve Performance Reporting  and Monitoring for the Transition Assistance Program, GAO-18-23.  Washington, D.C.: November, 8, 2017.

Transitioning Veterans: Improvements Needed in DOD’s Performance  Reporting and Monitoring of the Transition Assistance Program,  GAO-18-225T. Washington, D.C.: November 8, 2017.

Department of Defense: Transition Assistance Program (TAP) for Military  Personnel, GAO-16-302R. Washington, D.C.: December 17, 2015.

Veterans’ Employment: Need for Further Workshops Should Be  Considered before Making Decisions on Their Future, GAO-15-518.  Washington, D.C.: July 16, 2015.

Military and Veteran Support: DOD and VA Programs That Address the  Effects of Combat and Transition to Civilian Life, GAO-15-24.  Washington, D.C.: November 7, 2014.

Veterans Affairs: Better Understanding Needed to Enhance Services to  Veterans Readjusting to Civilian Life, GAO-14-676. Washington, D.C.:  September 10, 2014.

Transitioning Veterans: Improved Oversight Needed to Enhance  Implementation of Transition Assistance Program, GAO-14-144.  Washington, D.C.: March 5, 2014.

Military and Veterans’ Benefits: Enhanced Services Could Improve  Transition Assistance for Reserves and National Guard, GAO-05-544.  Washington, D.C.: May 20, 2005.

Military and Veterans’ Benefits: Observations on the Transition  Assistance Program, GAO-02-914T (July 18, 2002).