Railroad Retirement Board: Additional Controls and Oversight of Financial Interchange Transfers Needed

RRB collects payroll taxes and administers retirement, disability, and Medicare benefits for rail workers and their families. A financial interchange exists between RRB, SSA, and HHS in order to put the trust funds for these benefits in the same financial position as if Social Security covered rail workers. RRB generally transfers to the Social Security and Hospital Insurance trust funds the taxes that would be collected from rail workers and employers, while SSA provides RRB the benefits that would otherwise be paid directly to rail workers. GAO was asked to review the financial interchange calculation process.

This report examines (1) the steps taken to calculate financial interchange amounts, (2) factors that could account for trends in transfers over time, and (3) the extent to which RRB, SSA, and HHS provide oversight to ensure calculations are accurate. GAO reviewed agency policies, procedures, and regulations; observed RRB staff calculating four cases selected for beneficiary type; reviewed data on payment and beneficiary trends; and interviewed agency officials.

Established in 1937, the Railroad Retirement Board (RRB) administers retirement and disability benefits for rail workers and their families. A financial interchange between RRB and the Social Security Administration (SSA) was created in 1951, which as GAO previously reported, helped finance RRB benefits as they increased over time to keep pace with growing Social Security benefits to individuals. Through its financial interchange calculation, RRB takes steps each year to estimate the amount of funds that would have flowed in and out of Social Security's trust funds if rail beneficiaries were covered by Social Security instead of RRB. Five key steps go into the annual calculation:

RRB is credited for (1) the estimated amount of benefits it would have paid to beneficiaries under SSA rules, (2) administrative costs, and (3) interest accrued on the financial interchange amount.

SSA is credited for the revenues it would have received from rail workers if they paid into Social Security; specifically, (4) payroll taxes and (5) income taxes paid on benefits received.

The determined net amounts are transferred between the agencies, which since 1958 have been from SSA to RRB each year. RRB received $4.1 billion in fiscal year 2016, almost one-third of the $12.4 billion in retirement and disability benefits it paid that year. The financial interchange was expanded to Medicare in 1965 to facilitate funding of Medicare benefits to rail workers; RRB transfers Medicare payroll taxes collected, income taxes paid on benefits received, and interest, minus administrative costs to the Department of Health and Human Services (HHS).

A high ratio of beneficiaries to active railroad workers primarily explains the net transfers from Social Security's trust funds to RRB each year since 1958. Rail employment has fallen steadily since World War II, and the number of beneficiaries has exceeded the number of workers since 1961. RRB had 2.7 beneficiaries for every worker in 2015. As a result, RRB has paid out more in benefits than it has collected in payroll taxes and projects this to continue for the foreseeable future.

RRB takes a number of steps each year to ensure the accuracy of its calculations, such as checking that the sample of cases used to estimate benefit payments is complete, reviewing the work of new employees, and using electronic alerts to help prevent staff from entering incorrect information into its computer system. SSA and HHS also conduct high-level reviews of the calculation results to identify any significant changes from one year to the next. However, RRB's process includes manual data entry and its electronic edit checks cannot flag entries that are incorrect but plausible, which could lead to calculation errors. RRB also has limited documentation of its calculation process, and does not have formal policies on how staff should address some potential calculation errors and on how supervisors should review staff work. This is contrary to internal control standards for having quality data and documenting procedures. In terms of SSA and HHS, they do not currently review case-level calculations made by RRB, and cannot reasonably ensure that work used to determine the transfers they made and received is correct.

GAO makes eight recommendations, including that RRB create formal policies and improve documentation of its processes, work with SSA to obtain data electronically, and that SSA and HHS increase their oversight. RRB and SSA agreed, while HHS did not, asserting that statute limits its authority; however, HHS continues to review this issue. HHS should seek this authority if it determines it necessary.

Established in 1937, the Railroad Retirement Board (RRB) administers  retirement and disability benefits for rail workers and their families. After  Medicare was enacted in 1965, RRB began administering these benefits  for rail employees and their families. For most other workers in the United  States, the Social Security Administration (SSA) administers retirement  and disability benefits and the Department of Health and Human Services  (HHS) administers Medicare benefits.

We previously reported that the 1951 amendments to the Railroad  Retirement Act of 1937 substantially increased railroad retirement  benefits to bring them in line with benefit increases granted to individuals  under Social Security, and that a financial interchange was created  between the agencies in 1951 to help pay for these increases. The  financial interchange was expanded in 1965 to include HHS. The financial  interchange is intended to put the Old-Age and Survivors (OASI),  Disability Insurance (DI)—which finance retirement and disability benefits  respectively—and Hospital Insurance trust funds in the same position that  they would have been had railroad employment been covered under the  Social Security Act and the Federal Insurance Contributions Act.  Through this financial interchange, RRB typically receives sizable  transfers each year from the OASI and DI trust funds and transfers funds  to the Hospital Insurance Trust Fund. In fiscal year 2016, RRB reported  that it received about $4.1 billion in transfers from the OASI and DI trust  funds—about one-third of the $12.4 billion it paid in retirement and  disability benefits.

You asked us to review the financial interchange calculation process. This  report examines:  1.  What steps does RRB take to calculate financial interchange  amounts?  2.  What factors, if any, account for the trends in transfers between RRB,  SSA, and HHS through the financial interchange?  3.  To what extent do the agencies provide oversight to ensure that  financial interchange transfer amounts are accurate?

To address the first objective we reviewed RRB guidance and  documentation to determine the methodology RRB uses for its financial  interchange calculations, as well as assessments of the soundness of the  methodology. We also observed RRB staff while they conducted  calculations on four non-generalizable four cases—selected to provide  variation in terms of the type of benefits being received—to determine the  steps taken by staff.

To address the second objective, we analyzed data on the amounts of  financial interchange transfers from calendar year 1951 to 2015, the  number of RRB beneficiaries from fiscal year 1937 to 2016, and the  number of rail workers from calendar year 1937 to 2016, the most recent  available data. We assessed the reliability of these data by reviewing  RRB documentation regarding internal controls and interviewing  knowledgeable officials. We determined that the data on the number of  beneficiaries and workers were sufficiently reliable for our purposes. We  determined that the data we report on the amounts of financial  interchange transfers were sufficiently reliable for our purposes of  discussing trends in transfers over time.

To address the third objective, we reviewed agency policies, procedures,  and documentation. We also observed RRB staff demonstrate, on the  four illustrative cases described above, the safeguards to prevent errors  that are built into the system they use to compute financial interchange  amounts. We compared these steps taken by the agencies to criteria for  using quality data and documenting agency procedures in standards for  internal controls in the federal government.

For all three objectives, we reviewed relevant federal laws and  regulations, and interviewed staff at the three agencies to provide  additional context.

We conducted this performance audit from January 2017 to April 2018 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

The rail industry was one of the first to pioneer private pensions for its  employees in the late 19th century, and by the 1930s, these pensions  were more developed than in most other industries. However, according  to RRB, these private rail pensions had serious defects that were  magnified by the effects of the Great Depression. For instance, RRB  noted that the plans were generally inadequately financed and that  employers could terminate the plans at will. In prior work, we noted that  the Railroad Retirement Act of 1937 was enacted at the urging of rail  labor and established the national railroad retirement system  administered by RRB. The program was to be solely supported by  employees and employers of the rail industry through payroll taxes.

According to RRB, this system was created separately from Social  Security for several reasons. For instance, RRB notes that Social  Security—created in 1935—would not begin payments for several years  or credit workers for work prior to 1937, while the deteriorating state of  private rail pensions called for immediate retirement payments based on  prior service.

We previously reported that the 1951 amendments to the Railroad  Retirement Act of 1937 substantially increased railroad retirement  benefits to bring them in line with benefit increases granted to individuals  under Social Security, and that a financial interchange was created  between the agencies in 1951 to help pay for these increases. RRB  annually computes the amounts that SSA would have collected in taxes  from rail workers and their employers, and what SSA would have paid in  benefits if rail workers had been covered under Social Security, with the  net difference transferred between the agencies. The amounts computed  under the financial interchange do not necessarily represent the actual  RRB benefits paid to rail workers and their beneficiaries. RRB determined  that it was due a net transfer from SSA each year since 1958.

Financial interchange transfers make up a significant portion of the  financing for RRB’s retirement, disability, and survivors benefits. In fiscal  year 2016, RRB paid about $12.4 billion in these benefits and collected  $5.9 billion in payroll taxes from rail employees and employers. RRB  reported that the remainder of its funding for these benefits came from the  financial interchange ($4.1 billion), transfers from the National Railroad  Retirement Investment Trust ($1.4 billion), income taxes collected on  RRB benefits ($758 million), and other funding sources, such as  appropriations. The interchange also serves as a vehicle to fund  Medicare Part A (Hospital Insurance) benefits for rail workers.

The benefits provided by RRB consist of a core-level of benefits that are  similar to those available to most workers covered under Social Security,  including Medicare. Rail workers also receive a second level of retirement  benefits that approximate payments from private pension plans (see table  1).

For non-rail workers, Social Security and Medicare benefits are paid from  their respective trust funds:

Retirement benefits are paid from SSA’s OASI Trust Fund;

Disability benefits are paid from SSA’s DI Trust Fund; and

Medicare Part A benefits are paid from the Hospital Insurance Trust  Fund.

RRB Calculates Financial Interchange Amounts by Approximating Key Flows In and Out of SSA and HHS Trust Funds

The financial interchange is intended to place Social Security’s OASI and  DI Trust Funds and HHS’s Hospital Insurance Trust Fund on the same  financial footing as if rail workers and beneficiaries were covered under  Social Security instead of by RRB. Regarding Social Security, RRB is  credited for what it paid beneficiaries, administrative costs involved with  paying benefits, and interest for the time between the determination of the  interchange amount and its actual transfer. SSA is credited for the  amount of payroll and income taxes it would have collected from rail  workers and for income taxes that would have been paid by RRB  beneficiaries on Social Security equivalent benefits. The net of the five  amounts is the amount that is transferred (see fig. 1). A net transfer from  SSA to RRB means that rail workers would have been a net draw on  SSA’s trust funds if covered under Social Security. RRB calculates the  financial interchange amount each year, which is done on a retrospective  basis, i.e., the amount is determined for the previous fiscal year. By law,  the agencies must complete their determination by June of each year.

In keeping with the purpose of keeping the OASI and DI trust funds in the  same place as if rail workers were covered under Social Security, RRB  determines the retirement and disability benefits that rail workers and  dependents would have received if they were covered under Social  Security. Specifically, RRB uses railroad earnings data provided by  employers to replicate SSA’s benefits calculations. Although the basic  retirement and disability benefits that SSA and RRB pay to their  beneficiaries are based on the same formulas, there are several eligibility  differences between the two programs. For instance, a rail worker may  receive unreduced retirement benefits at age 60 after 30 years of work,  whereas the earliest most workers covered under Social Security can  begin receiving retirement benefits is at age 62. According to RRB  officials, even though a 60-year-old railroad worker may be receiving RRB  retirement benefits, RRB would not receive credit through the interchange  for that individual. Once that individual turns 62, RRB determines the  amount of reduced Social Security retirement benefits for which he or she  would have been eligible, given the person’s earnings history and Social  Security’s benefits rules. According to RRB officials, the agency  receives a credit through the interchange for this amount even though the  individual is receiving full RRB retirement benefits.

To account for these potential differences, RRB officials said that the  agency must make calculations for individual RRB cases. Additionally,  RRB officials said that in light of the number of RRB cases—nearly  400,000—it is not practical to make these calculations annually for each  case. Instead, RRB uses SSA rules to calculate benefits for a subset of  RRB cases in which the worker’s Social Security number ends in 30,  which approximates a 1-percent sample. The sample size was about  4,000 for fiscal year 2016. Once RRB completes its benefit calculation for  each of those cases, it aggregates the result and produces an estimated  amount for its entire population of cases (see fig. 2).

RRB reported in its annual financial interchange determination report that  it was credited $7.2 billion dollars in fiscal year 2016 for the estimated  amount beneficiaries would have been paid under Social Security.

These expenses represent those that SSA would have incurred to  administer benefits had rail workers been covered under Social Security  (as opposed to the actual amount RRB spent to administer its programs).  These expenses, which SSA would have funded out of its trust funds,  include the cost to enroll individuals in its programs and maintain its  benefit rolls. RRB calculates the amount of administrative expenses  based on unit-cost data provided by SSA. RRB reported that it was  credited about $22 million in administrative costs for fiscal year 2016.

SSA credits RRB for interest that accrues on the annual financial  interchange transfer from the period in time for which it is calculated (the  end of the fiscal year on September 30) until the amount is transferred to  RRB in June of each year. The interest rates are equal to those SSA  earns on its trust funds. RRB reported that it was credited about $163  million in interest for fiscal year 2016.

This amount represents the payroll taxes rail employees and employers  would have paid into Social Security’s trust funds had workers been  covered under Social Security. SSA and RRB generally levy payroll taxes  on earnings at the same rate, and RRB officials told us they use payroll  data from employers to determine this amount. RRB reported that it  credited SSA $2.4 billion for fiscal year 2016.

Some RRB beneficiaries pay income taxes on the benefits they receive,  and that tax revenue is credited to SSA’s trust funds through the financial  interchange. To put the OASI and DI trust funds in the same place as if  rail workers were covered under Social Security, RRB credits SSA for the  amount of income tax railroad beneficiaries paid on Social Security  equivalent benefits. RRB computes this amount using tax data from the  Department of the Treasury, and credited about $296 million to SSA for  fiscal year 2016.

RRB also may adjust calculations on transfers from prior years; for  instance, if new income was reported for individuals or if benefit  overpayments are discovered for individuals in the sample.

The process for determining the financial interchange transfer with HHS— which helps finance Medicare benefits for rail workers—has fewer  components than for retirement and disability benefits. Generally, RRB  determines the Medicare payroll taxes and income taxes paid by rail  workers and transfers this amount, less administrative expenses, to HHS  (see fig. 3). RRB estimates how much it collects in Medicare payroll taxes  by using payroll data provided by employers for workers whose Social  Security numbers end in 30. RRB credited HHS for about $637 million  for fiscal year 2016.

Overall, the procedures we observed, and which RRB explained and  demonstrated, for calculating the financial interchange are consistent with  the methodology agreed to by RRB, SSA, and HHS. An annual  determination report produced by the three agencies documents this  methodology. Additionally, several audits conducted for the RRB Office of  Inspector General determined that the methodology is appropriate for  achieving the purpose of the financial interchange. Specifically, the audits  concluded that the sample used in calculating benefits was representative  of RRB’s population of beneficiaries, the formulas used to project the  results of the sample on the entire population of beneficiaries were  consistent with RRB’s design, and that assumptions made by RRB when  carrying out calculations were reasonable.

High Ratio of Beneficiaries to Rail Workers Has Resulted in Transfers From SSA to RRB Each Year Since 1958

SSA has made a net transfer to RRB through the financial interchange  each year since 1958. The cumulative net transfer from the Social  Security trust funds to RRB through 2015 was approximately $266 billion  in 2016 dollars. Of this amount, transfers related to retirement and  survivor benefits comprised about $256 billion and disability benefits  accounted for about $10 billion. This trend in transfers is primarily  caused by RRB benefit payments exceeding payroll taxes collected as  calculated by the interchange, which has been the case each year of the  financial interchange, resulting in a net amount owed to RRB from SSA  each year (see fig. 4).

Based on the data RRB reported, the continuing flow of funds to RRB  from SSA has largely been driven by a steadily shrinking number of active  workers in the rail industry paying payroll taxes in support of a larger  population of beneficiaries. According to RRB data, the number of  workers in the rail industry peaked at the end of World War II, when there  were almost 1.7 million workers. Since then, this number declined steadily  to about 231,000 in 2016. Additionally, the number of beneficiaries has  exceeded the number of active workers since 1961. According to RRB  data, there was about 1 beneficiary for every 10 workers in 1938; the ratio  had increased to 3 beneficiaries for every 10 rail workers in 1951, when  the financial interchange was created. By 2016, there were 28  beneficiaries for every 10 workers. Furthermore, RRB officials noted  that another factor causing increased fund transfers from SSA to RRB  was a series of successive amendments to the Social Security Act which  raised benefits immediately while deferring tax increases to pay for the  increased benefits. As a result of these two factors, the payroll taxes paid  by rail workers have not been sufficient to pay for all of the benefits paid  by RRB. Hence, the financial interchange has consistently transferred  money from SSA to RRB (see fig 5).

According to SSA actuarial estimates, the flow of funds to RRB from SSA  is projected to continue. Social Security’s 2017 trustees report projects  that the amount of transfers to RRB will continue to grow though at least  2026. Moreover, RRB’s most recent actuarial valuation report estimates  that under three employment assumptions—optimistic, moderate, and  pessimistic—the number of beneficiaries will continue to exceed the  number of rail workers through at least 2088.

RRB has collected payroll taxes for HHS since 1966. From 1966 through  2016, RRB reported that it transferred a total of $30 billion in 2016 dollars  through the financial interchange to the Hospital Insurance Trust Fund  (see fig. 6).

RRB Takes Measures to Oversee the Financial Interchange Calculation, but Shortcomings Increase the Risk of Errors

RRB Takes Oversight Steps, but Manual Data Entry and Systems Limitations May Prevent RRB from Detecting Mistakes

RRB takes a number of steps to ensure that the financial interchange  amount is accurately calculated each year. For example:

Sample verification: To make sure that the financial interchange  sample is up to date, RRB staff told us that they query their  beneficiary database at the beginning and end of the annual financial  interchange calculation to ensure that all beneficiaries who should be  part of its sample—those with a Social Security number ending in  30—are included. Those included in the sample can change from year  to year, for instance, when new beneficiaries join the retirement rolls  or when beneficiaries die.

Supervisory review: RRB officials told us that the work of a new  employee who calculates the financial interchange is reviewed by  another employee until the new employee is determined to be  proficient.

Error checks: Electronic error checks built into the system RRB uses  to calculate the financial interchange help prevent mistakes by  flagging erroneous values. These checks alert employees in real time  that an incorrect value may have been entered (for example, a benefit  amount that exceeds what beneficiaries can receive). Officials also  told us that they run similar checks in batches throughout the year to  sweep for any potential errors that were not addressed by employees.  They noted that they will work with staff to address all potential errors  before the financial interchange calculation is finalized. However,  RRB’s error checks do not cover all potential erroneous values.

High-level review: RRB officials told us that the Chief of Benefit and  Employment Analysis and his staff review the results of the  interchange calculations and determine if the end result is reasonable  compared to projections made earlier in the year, based on actual  payroll and beneficiary data.

Despite these steps, limitations in RRB’s error checks and its reliance on  manual data entry are potential sources of mistakes in financial  interchange calculations. The process RRB staff follow in computing  benefit amounts for the financial interchange involves manual data entry  of earnings data and SSA-equivalent benefits. RRB’s error checks will  help identify values that are impossible—such as a benefit amount that  exceeds the maximum a beneficiary can receive—but not values that are  incorrect but still within the range of possibility. RRB staff demonstrated  this scenario for us and acknowledged this as a limitation in their internal  controls.

Any data entry errors have the potential to result in larger errors in the  financial interchange determination. The benefits portion of the financial  interchange determination is based on a sample of all cases. Should any  errors occur in the sample, they will be magnified when RRB inflates the  estimate to arrive at an amount for the entire population of beneficiaries.  Additionally, RRB’s process could result in incorrect transfers for years.  The sample is chosen in the same way each year—individuals with Social  Security numbers ending in 30—so the same cases will remain part of the  sample until the individuals leave the rolls. RRB officials told us that they  generally only have to do a full set of calculations for new cases or cases  in which additional income is detected that affects benefit amounts. RRB  officials estimated that about 20 percent of cases in the financial  interchange sample each year require a full calculation. For the remainder  of cases in the interchange sample, officials said that no annual  recomputation is needed. Instead, the previous year’s results are  adjusted according to any cost of living increase. If a data entry error is  made in one of these cases, RRB may not discover it until the individual  leaves the rolls or dies, at which point RRB staff told us they recalculate  the individual’s benefit amount.

Data sharing between RRB and SSA could reduce the potential for data  entry errors, but the two agencies have not recently pursued this option.  RRB officials told us that prior to 2008 they used computer code to  automatically save data from SSA databases into spreadsheets, where  the data could be used for calculating the financial interchange. However,  SSA instructed RRB to stop using this method in 2008 because of  security concerns about saving this information outside of SSA systems.  RRB officials added that this constraint prevents them from developing a  more efficient method of data collection that would improve the accuracy  and timeliness of benefit calculations for the financial interchange.  However, RRB officials said that they have not formally approached SSA  in the last several years to discuss potential alternatives for gaining  greater access to data. SSA officials said that RRB should follow SSA’s  procedures for requesting a data exchange if RRB wishes to revisit this  topic. Federal internal control standards state that agencies should use  quality information to achieve their objectives. By taking additional steps  to obtain data from SSA electronically, RRB can better position itself to  ensure that data entered into its systems are correct and that its  calculations are free of errors.

Limited Documentation and Formal Policies Increase the Risk for Errors in Key Aspects of the Financial Interchange Process

RRB has limited documentation and does not have formal policies to  guide several key aspects of the financial interchange calculation. While  we did not identify any actual errors in its calculations, these  shortcomings in its controls increase the risk of calculations being carried  out inconsistently or incorrectly.

Limited Documentation of the Financial Interchange Process

The broad steps that RRB takes to determine the amounts of the financial  interchange are documented in an annual determination report produced  by RRB. They include, for example, the factors used to calculate  administrative costs, discussion of adjustments made to calculations from  prior years, and descriptions of the formulas used to project the results of  RRB’s benefit sample to the population of railroad beneficiaries. However,  the agency does not have clear documentation of the detailed steps used  by staff to calculate the interchange amounts. A 2010 audit of the  financial interchange process conducted for the RRB Office of Inspector  General found that documentation of the financial interchange process  was insufficient for a knowledgeable third party to replicate without verbal  explanation from RRB staff. In response, RRB officials told us that they  produced some documentation such as charts showing the workflows for  different portions of the process, such as for calculating benefits, payroll  taxes, and financial projection—and instructions for staff in RRB’s Bureau  of the Actuary for high-level review of the formulas and entries for the final  calculation results.

However, the documentation did not provide enough detail about the  steps staff must take when conducting financial interchange calculations  so the process can be followed without additional explanation. For  instance, the documentation did not discuss the process by which staff  obtain earnings data and enter it into SSA’s benefit calculator, manually  enter the results into RRB’s system, or the different alerts that notify staff  of potential mistakes and how staff deal with them. Federal internal  control standards state that effective documentation provides a means to  retain organizational knowledge and mitigate the risk of having knowledge  limited to a few personnel, as well as a means to communicate that  knowledge as needed to external parities, such as auditors. Written  documentation with specific steps for carrying out the financial  interchange calculation and using its data system would help RRB ensure  that its staff and others could carry out and replicate its process  consistently.

Limited Documentation of RRB’s Computer System

RRB does not have current or complete documentation related to the  computer system it uses to compute the financial interchange.  Specifically, RRB officials said that they do not have current  documentation such as a manual or data dictionary that would provide  information on the data elements in the system, their definitions,  descriptions, and range of potential values. They said a data dictionary is  not necessary because data are contained in a format in which rows and  columns are labeled according to fields and years. However, such  labeling does not include documentation, for example, about whether  values entered in those fields are allowable. Federal internal control  standards state that effective documentation is needed to retain  knowledge and prevent knowledge from being limited to a few staff. Even  if the data system is relatively uncomplicated, without such  documentation, it is difficult for RRB staff and others to fully understand  all elements in the system, and it could complicate efforts to make any  changes in the future or bring new staff up to speed on the system.

No Written Documentation on Procedures for Overriding Potential Errors

RRB does not have written procedures for how to address instances in  which staff do not correct potential errors flagged by its computer system.  As noted earlier, RRB’s system for calculating the financial interchange  will alert staff to potential data entry errors. RRB officials said this system  has the ability to allow staff to override the alert in some cases, generally  in complex cases, such as when RRB benefits are offset by other public  pensions. In these cases, the system does not distinguish between an  actual error and instances in which additional work and review are  needed because of complex benefit calculations. Staff can override the  alert in these cases where there is no actual error, but officials noted that  a report of potential errors that is generated by the system would still  include these cases, which may be referred back to staff for clarification  or correction. If implemented correctly, these procedures could help  staff take appropriate action on these complex cases. However, current  procedures are not formally documented and officials said they have not  considered producing written procedures because they believe the  process for addressing alerts is clear. Federal internal control standards  indicate that effective documentation assists in management’s design of  internal controls and can mitigate the risk that knowledge is limited to a  few staff. RRB’s lack of written procedures can make it difficult for staff or  reviewers to know if procedures are carried out consistently—such as  whether staff appropriately override an error alert—and can create  challenges if there is staff turnover. It is important to ensure that all  potential errors are addressed correctly given that mistakes in the  financial interchange sample can be multiplied when estimating benefit  payments for the universe of RRB beneficiaries.

According to RRB officials, new employees will have their calculations  reviewed until the employees are deemed to be proficient, and  calculations by any staff member are subject to review and periodically  reviewed for accuracy. Federal internal control standards call for  documenting agency procedures. However, RRB does not have a  minimum or maximum time established for which it will review the work of  new staff, and does not have an overall policy for reviewing staff  members’ work after they have been deemed proficient. Officials told us  they had not considered setting a policy regarding supervisory review.  They added that individualized, on-the-job training is more appropriate for  new staff than a formalized process. In the case of current employees,  any potential errors would be identified when the case is terminated, at  which time all cases are reviewed and recomputed. Additionally, officials  said that a formal policy would not increase the number of cases  reviewed and potentially constrain their ability to correct new errors as  they occur. Nonetheless, without formal policies on supervisory review,  RRB cannot reasonably ensure that the work performed by staff is  adequately or consistently reviewed for quality.

SSA and HHS Do Not Review the Results of Case-Level Calculations

SSA and HHS provide some oversight of the financial interchange  process, but do not review case-level calculations. Both agencies  approve the results of the financial interchange calculations, but officials  from SSA and HHS told us that their oversight is limited to high-level  reviews of RRB’s calculations to determine whether results significantly  vary from previous years. For instance, staff from SSA’s Office of the  Chief Actuary told us that they examine RRB’s payments and revenues  against SSA’s benefits paid and payroll taxes collected to determine if  there are large or inexplicable changes from year to year, in which case  they will ask RRB for additional information to understand the changes.  Additionally, RRB officials told us that formulas used in their spreadsheets  to calculate the results of the interchange have been reviewed by SSA  actuaries.

While these actions could help identify larger errors, the agencies will not  be able to detect whether errors are made on complex, case-level  calculations or if SSA rules are being correctly followed. In response to  prior errors in financial interchange calculations, RRB officials told us that  SSA reviewed case-level calculations from the 1990s until 2002. SSA  officials told us that they have not reviewed cases since then because of  resource constraints. A 2009 SSA Office of the Inspector General report  recommended that the agency consider increasing its oversight of the  process, such as setting a schedule for review of individual cases given  the importance of reviews in verifying transfers. However, SSA has not  taken action on this recommendation. HHS officials told us that the  financial interchange is one of a number of relatively small funding  streams and the agency has never had cause to suspect mistakes and  has never examined case-level calculations. Federal internal control  standards state that agencies should establish and operate monitoring  activities to evaluate the results of activities. Without monitoring how  calculations are made, SSA cannot reasonably ensure that the transfers it  makes or receives with RRB are accurate. In commenting on a draft of  this report, HHS raised questions about whether it has the authority to  review case-level calculations, but noted in follow-up communication that  this issue is currently undergoing legal review at HHS. As a result, HHS  officials told us that they would not be able to provide additional  clarification at this time. We continue to believe that HHS would be better  positioned to ensure that transfers it makes and receives are calculated  correctly if it reviews case-level calculations.

The financial interchange provides RRB with a significant portion of its  funding, and trends in the number of beneficiaries and workers suggest  this will continue to be the case in the future. RRB developed a process to  calculate the financial interchange amount, and the accuracy of the  calculations depends in large part on correct data being manually entered  into RRB’s computer system. However, RRB’s current controls do not  address some potential sources of error. Having the ability to  electronically obtain data from SSA could help reduce the risk posed by  data entry errors.

Further, RRB has limited written documentation for carrying out aspects  of the financial interchange calculation, such as how its computer system  is structured, how to address instances when staff override error alerts,  and how staff work is reviewed. Without such documentation, RRB puts  itself at risk of staff carrying out actions inconsistently, losing operational  knowledge when staff leave or retire, and complicating oversight of its  operations.

Lastly, SSA and HHS increase the risk of errors by not performing case- level reviews of financial interchange calculations. This is especially true  for the SSA portion of the interchange, which involves complex  calculations performed according to SSA rules. In its role as the  administrator of the OASI and DI programs, SSA is best positioned to  determine if its rules are properly being applied to financial interchange  calculations. The large sums SSA transfers through the interchange— over $4 billion annually—warrant additional oversight to ensure that  transfer amounts are correct.

We are making a total of eight recommendations, including five to RRB  (The Board), two to the Commissioner of SSA, and one to the Secretary  of HHS.

The Board should work with SSA to explore options for obtaining data  electronically and limiting the reliance of the financial interchange  process on manual data entry. (Recommendation 1)

The Board should produce written documentation on the financial  interchange process such that a knowledgeable third party could carry  out and replicate its process consistently without further explanation.  (Recommendation 2)

The Board should produce written documentation of its computer  system and its structure, such as a manual for the computer system,  and data dictionary to provide information on the data elements in the  system, their definitions, descriptions, and range of potential values.  (Recommendation 3)

The Board should produce written documentation of its procedures for  instances when staff override error alerts generated by its computer  system. (Recommendation 4)

The Board should produce formal policies on how the work of staff  performing the financial interchange is reviewed. (Recommendation 5)

The Commissioner of SSA should work with RRB to explore options  for electronically sharing data and limiting the reliance of the financial  interchange process on manual data entry. (Recommendation 6)

The Commissioner of SSA should take additional steps to provide  oversight of financial interchange calculations at the individual-case  level. This could include periodically reviewing a subset of these  cases. (Recommendation 7)

The Secretary of HHS should, consistent with its existing statutory  authority, take additional steps to provide oversight of financial  interchange calculations at the individual-case level. If the Secretary  concludes that there are limitations in its authority in this area, the  Secretary should seek to obtain the necessary additional authority.  (Recommendation 8)

We provided a draft of this report to RRB, SSA, and HHS for review and  comment. In written comments, both RRB and SSA agreed with the  recommendations. RRB noted that it will devote the resources needed to  improve the written documentation of its procedures and computer  system. RRB and SSA also provided technical comments which we  incorporated as appropriate. Copies of their written comments are  reproduced in appendixes I and II.

In written comments, which are reproduced in appendix III, HHS  disagreed with the recommendation that it take additional steps to provide  oversight of financial interchange calculations at the individual-case level.  HHS noted that while in theory it may be a good idea to incorporate such  review into the process, it is limited by statute in its ability to oversee how  RRB calculates transfers between HHS and RRB. HHS went on to  describe a section of the Social Security Act that they noted “pertains  more to Supplemental Medical Insurance trust fund draws for  administrative costs.” Notably, with respect to HHS, our report does not  involve that trust fund, but rather addresses the Hospital Insurance Trust  Fund. Although HHS’s comments did not clarify why it believes that this  section of law would limit its authority with respect to the Hospital  Insurance Trust Fund, it nevertheless asserted that it does apply in this  scenario. We reached out to HHS to seek clarification of its comments.   For example, we inquired about the applicability of a separate provision of  law that would appear to establish a role for HHS to work with RRB to  determine financial interchange amounts. Ultimately, HHS did not  provide the clarification we sought, instead indicating via email that this  recommendation is currently undergoing legal review and that HHS is  unable to provide a response to our questions at this time. HHS further  stated that it will continue to work on this issue to provide GAO with  updates in the future. In light of the uncertainty surrounding HHS’s  authority in this area and the fact that HHS declined to respond to our  requests for clarification of its legal authority, we have modified our  recommendation to reflect the fact that HHS may need to seek additional  statutory authority to implement our recommendation, should HHS  determine it to be necessary.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to the  appropriate congressional committees, the Railroad Retirement Board,  the Commissioner of the Social Security Administration, and the  Secretary of the Department of Health and Human Services. In addition,  the report will be available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions, please contact me at (202) 512- 7215 or curdae@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  GAO staff who made significant contributions to this report are listed in  Appendix IV.

Appendix I: Comments from the Railroad Retirement Board

Appendix II: Comments from the Social Security Administration

Appendix III: Comments from the Department of Health and Human Services

Appendix IV: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Mark Glickman (Assistant  Director), Daniel R. Concepcion (Analyst-in-Charge), and Randy DeLeon  made key contributions to this report. Additional contributors include  David Ballard, Carl Barden, William Boutboul, James Cosgrove,  Alexander Galuten, Jennifer Gregory, Sheila McCoy, Jean McSween,  Mimi Nguyen, Joseph Silvestri, Almeta Spencer, and Kate van Gelder.


Military Personnel: DOD Needs to Improve Funding Process for Morale, Welfare, and Recreation Programs

DOD's MWR programs provide servicemembers and their families with three categories of programs: Category A (e.g., fitness and libraries), Category B (e.g., camping and performing arts), and Category C (e.g., golf). DOD oversees the percentage of appropriated funding allocated to MWR programs by category and measures the military services' compliance with established funding targets. DOD set the targets at 85 percent for Category A and 65 percent for Category B. DOD did not set a target for Category C since this category has the ability to generate revenue from user fees.

House Report 115-200 accompanying a bill for the National Defense Authorization Act for Fiscal Year 2018 includes a provision for GAO to review DOD' s MWR programs. GAO assessed the extent to which (1) the services have met DOD's established funding targets for each category of MWR programs and DOD has comprehensively evaluated the relevance of its targets, and (2) DOD has oversight structures and performance measures that include measurable goals, including those for cost-effectiveness, by which to review MWR programs. GAO analyzed MWR program information for fiscal years 2012-2017 and compared DOD's MWR policy with guidance for using measures and evaluating goals.

The Department of Defense (DOD) established funding targets for two categories of Morale, Welfare, and Recreation (MWR) programs—Category A, which promotes the physical and mental well-being of servicemembers, and Category B, which funds community support systems for servicemembers and their families. These targets are intended to ensure that the military services adequately fund these programs with appropriated funds instead of requiring servicemembers and their families to pay fees out of pocket to cover program costs. However, GAO found the following:

In fiscal years 2012-2017, the military services generally met the DOD-set target to provide 85 percent of appropriated funding for Category A programs but not the 65-percent target for Category B programs. Service officials said they are taking steps to meet the Category B target, such as by restoring targeted levels of appropriated funding support in future budget planning. Data GAO reviewed indicate that these steps are helping the services get closer to meeting the target for Category B.

DOD has not comprehensively evaluated the targets, established more than 20 years ago, to ensure that they are appropriate. DOD officials said they agree that it is time to evaluate the relevancy of the targets as the current operating environment is fundamentally different than when the targets were established 2 decades ago. Further, DOD officials said that they are unsure of the process or methodology used to originally develop the targets because they have no documentation supporting these decisions. Until DOD comprehensively evaluates the appropriateness of the targets and, based on its evaluation, documents any changes made, it cannot be certain that the targets reflect the current operating environment and do not pose undue financial burden on servicemembers.

DOD established oversight structures and performance measures for MWR programs, but has not established measurable goals to assess the cost-effectiveness of the 55 activities that make up MWR programs. DOD's MWR policy identifies six broad performance measure categories for the program. DOD officials responsible for developing MWR program goals acknowledged that DOD's MWR policy does not include measurable goals for assessing the cost-effectiveness of program activities, and do not currently have plans to make any changes to the goals. Service officials told GAO that they collect and use various types of information within the categories to assess specific activities. While both the categories established by DOD and the service-specific efforts provide useful context about the status of individual MWR activities, they do not replace the need for measurable goals that can be used to assess whether the programs are operating cost-effectively. The services are in the early stages of developing more specific performance measures, but it is too early to determine whether these efforts will result in measurable goals that can be used to assess cost-effectiveness. Until DOD develops performance measures that include measurable goals, it cannot ensure that MWR programs meet servicemember needs in a cost-effective manner.

GAO recommends that DOD evaluate the funding targets and document any changes needed and develop measurable goals for MWR programs' performance measures. DOD concurred with the recommendations.

The Department of Defense’s (DOD) Morale, Welfare, and Recreation  (MWR) programs are a multibillion dollar effort to provide  servicemembers and their families with a wide range of benefits designed  to support military missions and readiness, both in times of war and  peace. In fiscal year 2017, DOD spent approximately $3.9 billion on MWR  programs administered by the Army, the Navy, the Marine Corps, and the  Air Force. DOD’s three categories of MWR programs are mission- sustaining programs promoting the physical and mental well-being of  servicemembers (Category A), community support system programs for  servicemembers and their families (Category B), and recreational  activities for servicemembers and their families that are revenue- generating (Category C). The Office of the Under Secretary of Defense  for Personnel and Readiness (USD(P&R)) oversees the percentage of  appropriated funding provided to MWR programs by category and  measures the services’ compliance with established funding “targets.”  These targets set the minimum level of appropriated funding by category  as a percentage of total expenses compared with funding received from  nonappropriated funding sources, such as fees collected from revenue- generating activities.

House Report 115-200 accompanying a bill for the National Defense  Authorization Act for Fiscal Year 2018 includes a provision for us to  review DOD’s MWR programs. Specifically, the committee expressed  concern that the services have not complied with established DOD policy  to ensure appropriate financial support of family, welfare, and recreational  programs for the past several years, particularly for programs within  Category B, which relies on both appropriated and nonappropriated  funding. For this report, we assessed the extent to which (1) the services  have met DOD’s established funding targets for each category of MWR  programs and DOD has comprehensively evaluated the relevance of its  targets, and (2) DOD has oversight structures and performance measures  that include measurable goals, including those for cost-effectiveness, by  which to review MWR programs.

For our first objective, we assessed the extent to which the Army, the  Navy, the Marine Corps, and the Air Force met DOD’s established  funding targets in fiscal year 2012 through fiscal year 2017. We met with  officials from USD(P&R) and the services to discuss MWR program  policies and procedures and any challenges they have meeting funding  targets as well as any actions they plan to take to meet the targets. We  also reviewed memorandums the services developed showing their plans  to meet the funding targets when they missed them. Based on responses  to data reliability questionnaires from USD(P&R) and the services, we  determined that the data we obtained on DOD’s MWR programs are  sufficiently reliable for the purpose of reporting program information— including costs from fiscal years 2012 through 2017, which are the most  current data available.

For our second objective, we reviewed DOD Instruction 1015.10, Military  Morale, Welfare, and Recreation (MWR) Programs, which specifies  DOD’s oversight roles and responsibilities for the MWR programs, the  goals for those programs, and the information the services should collect  on the programs. We compared this instruction with the Standards for  Internal Control in the Federal Government that states, among other  things, that managers should establish activities to monitor performance  measures. We also compared DOD’s instruction with DOD’s Financial  Management Regulation and with guidance we have identified in our  prior work on performance measurement. This work provides guidance  for using performance measures and evaluating whether goals were met  and for assessing cost-effectiveness. We met with officials from  USD(P&R) and the services to discuss oversight of the program and  processes they have in place for meeting program goals and measuring  cost-effectiveness.

We conducted this performance audit from August 2017 to August 2018  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

DOD’s MWR Program Categories and Funding Sources

DOD Instruction 1015.10, Military Morale, Welfare, and Recreation  (MWR) Programs, establishes policy, assigns responsibilities, and  prescribes procedures for operating and managing programs for military  MWR programs. Specifically, the policy states that the services are to  establish MWR programs in order to maintain individual, family, and  mission readiness and that these programs are an integral part of the  military and its benefits package. The Office of USD(P&R) oversees  DOD’s MWR programs, develops policy, and oversees MWR programs’  funding. DOD’s instruction specifies the purpose of, the funding sources  for, and the activities within each of MWR’s three designated program  categories—all of which are summarized below in table 1. For a complete  listing of the activities by program category, see appendix I.

Each service supports MWR programs with a mix of appropriated and  nonappropriated funding. According to officials, the services allocate  appropriated funding amounts for MWR purposes, which primarily  supports Category A and B programs. Nonappropriated funding is  government money from sources other than amounts appropriated by  Congress and may be generated in a number of ways to support MWR  programs. For example, bowling programs, marinas, and golf programs  generate nonappropriated funding revenue through participation fees for  recreational activities paid by servicemembers and their families. Services  must use any nonappropriated funding generated from or associated with  MWR programs within their MWR programs.

According to DOD Instruction 1015.10, the MWR programs are divided  into three distinct categories, two of which also have specific funding  targets. According to DOD’s 2016 report to Congress on appropriated  funding support for MWR programs, the funding targets are intended to  ensure that the services adequately fund MWR programs instead of  requiring the servicemembers and their families to pay out of their own  pockets for costs that should be borne by appropriated funding. While  DOD Instruction 1015.10 establishes minimum funding targets for MWR  Category A and B programs, it directs that the basic funding target,  regardless of program category, is to use appropriated funding for 100  percent of costs for which they were authorized. While DOD’s Instruction  allows the services to use appropriated funding for 100 percent of  authorized costs, according to service officials this is generally not  possible given budget constraints. Therefore, for MWR Category A  mission sustaining programs, the DOD instruction establishes the funding  target—stating that DOD is to use appropriated funding amounts for a  minimum of 85 percent of total expenditures. For the MWR Category B  community support system programs, the DOD instruction establishes the  funding target as DOD’s use of appropriated funding amounts for a  minimum of 65 percent of total expenditures. For the MWR Category C  recreational activities for servicemembers and their families, appropriated  funding support should generally be limited because this category has the  highest capability of generating nonappropriated funding revenues.

Budget, Funding, and Accounting Processes for MWR Programs

The services have annual budget processes for MWR programs that vary  based on whether appropriated or nonappropriated funding is being used.  For MWR programs supported by appropriated funding, according to  officials, the services submit and validate program requirements through  DOD’s Planning, Program, Budgeting, and Execution process. DOD and  service guidelines for certain MWR programs as well as annual service- issued budget guidance provide input for determining MWR programs’  requirements. Service officials from the Army, the Marine Corps, and the  Air Force also stated that they determine program requirements using  input from installations and service components, while service officials  from the Navy stated that they use a budget model along with  performance measures and budget guidance to determine program  requirements. The requirements are then submitted to higher level  components within the services for review, adjustment, and approval.  Once the services validate the requirements, they are provided to the  Office of the Secretary of Defense for inclusion in the President’s Budget.  Figure 1 provides an overview of the general process the services use to  budget for appropriated funding support of MWR activities.

Budget processes and authorities for nonappropriated funding, or  program-generated revenue, vary by service. Specifically, the services  maintain nonappropriated funding budgets and budget approvals at  different levels within the service organization. For example, officials  stated that Marine Corps and Air Force installations maintain and manage  nonappropriated funding generated at their locations while Army and  Navy installations submit nonappropriated funding and budgets to a  higher level of command, Installation Directorates for the Army and  Regions for the Navy, as well as the service headquarters component.  The services plan for and manage their nonappropriated funding budgets  based on a number of factors, including revenue generated; projected  revenues; and the amount, if any, of appropriated funding available.  Figure 2 provides an overview of the general process the services use to  approve and manage nonappropriated funding generated within the  service.

Each service uses processes to provide funds for the implementation of  its MWR programs. Service officials stated that during program execution  the services execute their programs and make adjustments to their  budgets based on funding authorized from appropriated funding and  nonappropriated funding sources. Commanders have authority over  budget implementation and the guidelines and parameters for  commanders vary by service. For example, according to Army officials,  during the fiscal year Army commanders can change MWR program  budgets and have some flexibility to move funding to other non-MWR  command priorities. Installations report to the services actual  expenditures and income generated, which are included in the services’  annual reports. Figure 3 provides an overview of the general process the  services use to provide funding for MWR programs.

Each service uses accounting processes for its MWR programs.  According to service officials, accounting is handled differently at each  service depending on the service’s organizational structure. According to  service officials, the Navy and the Marine Corps centrally manage their  MWR accounting processes at their service headquarters; the Army  manages its accounting process at its headquarters and at the Defense  Financial and Accounting Services Nonappropriated Financial Services;  and the Air Force manages its accounting process at its Secretariat and  at the service components. According to service officials, program  managers at the service headquarters and activity level are able to review  financial data, such as expenditures and revenues, for MWR programs on  a recurring basis. DOD’s Instruction 1015.10 states that the services  should identify appropriated and nonappropriated funding accounts in  annual budgets, and the services have designated codes to categorize  expenditures. Service officials stated they use the codes to report  annually to USD(P&R) on MWR programs’ expenditures for both  appropriated and nonappropriated funding.

The Services Did Not Consistently Meet One of the Two Appropriated Funding Targets and Are Taking Steps to Address This, but DOD Has Not Comprehensively Evaluated the Targets to Ensure They Are Appropriate

The services generally met the funding target for fiscal years 2012  through 2017 for MWR Category A mission-sustaining programs, but did  not consistently meet the target for Category B programs that provide  community support systems to servicemembers and their families during  the same time period. Service officials said they are taking steps to meet  the Category B target, such as restoring targeted levels of appropriated  funding support in future budget planning. Data indicate that the services  are getting closer to meeting the target. However, DOD has not  comprehensively evaluated the funding targets, which were established  more than 20 years ago, to ensure they currently are appropriate.

The Services Generally Met the Funding Target for MWR Category A Mission- Sustaining Programs

For MWR Category A mission-sustaining programs, the services  generally met the 85-percent target for appropriated funding support.  Specifically, the Navy and the Air Force consistently met or exceeded the  85-percent funding target in fiscal years 2012 through 2017, and the Army  met or exceeded the target every year except for fiscal year 2012 when it  reported that 84 percent of its Category A programs were supported with  appropriated funds. The Marine Corps exceeded the minimum funding  target for Category A programs in fiscal years 2012 through 2017, but  consistently fell below the target with appropriated funding support  ranging from 77 percent to 84 percent from fiscal years 2013 through  2016. Table 2 provides additional detail on the extent to which each  service met the 85-percent funding target for MWR Category A mission- sustaining programs in fiscal years 2012 through 2017.

The Services Did Not Consistently Meet the Funding Target for MWR Category B Community Support Programs, but Are Taking Steps to Meet the Target in the Future

For MWR Category B community support programs, the services missed  the 65-percent target for appropriated funding support with increasing  frequency from fiscal years 2012 through 2017. Service officials stated  that constrained budgets and competing priorities have made it difficult to  allocate the appropriated funding needed to support their programs.  However, service officials said they are taking steps to meet the Category  B funding target in the future. Specifically, we found that the services  collectively missed the funding target over 60 percent of the time from  fiscal years 2012 through 2017. All four services missed the funding  target in fiscal years 2015 and 2016 with appropriated fund support  ranging from 55 to 63 percent. Most recently, in fiscal year 2017 the Army  met the 65-percent funding target, but the Navy, the Marine Corps, and  the Air Force fell below the 65-percent funding target with appropriated  funding support ranging from 60 percent to 62 percent. Although the Air  Force did not meet the 65-percent target for fiscal years 2012–2017 citing  resource issues, Air Force leadership has increased appropriated funding  for the MWR programs each year to help get closer to meeting the  Category B funding target. Air Force officials said they plan to continue to  increase funding each year so they can meet the target in the future.  Table 3 provides additional detail on the extent to which each service met  the 65-percent funding target for MWR Category B community support  programs in fiscal years 2012 through 2017.

The USD(P&R) monitors the services’ compliance in meeting the targets.  When a funding target is missed, USD(P&R) officials said a memorandum  is sent to the services that asks for a detailed plan on how they will  achieve the required level of appropriated funding support for the missed  target in the future, and these officials said that each service has provided  such a plan when they fell below the 65-percent funding target. In  instances when a service does not respond to the initial request for a  remediation plan, USD(P&R) officials said a second memorandum is sent  notifying the service that they missed the funding target and that they  need to submit a plan detailing how they intend to come into compliance.  For example, in fiscal year 2015 the Army did not meet the 65-percent  funding target for Category B programs. In June 2016, the Assistant  Secretary of Defense for Manpower and Reserve Affairs sent the Army a  memorandum asking it to submit a plan on how it would meet the target.  After not receiving a response, the Assistant Secretary of Defense for  Manpower and Reserve Affairs sent the Army a second memorandum in  September 2016 that noted the missed target and reiterated the need to  submit a plan for achieving compliance with designated funding targets.  Following the second memorandum, the Army issued a memorandum in  December 2016 stating it would fully fund Category A and B programs to  the required targets in fiscal year 2017. Following these communications,  in February 2018, the Army sent USD(P&R) its fiscal year 2017 program  and metric report showing that it had successfully met the Category A and  B funding targets as planned.

Service officials said they are taking steps to meet the Category B target,  and data from fiscal years 2015 through 2017 indicate that the services  are getting closer to meeting it. However, in the prior years when the  services have not met appropriated funding targets for Category B  programs, officials said that the services have relied on nonappropriated  funding as supplemental support to help ensure that such programs  continue to operate. Specifically, according to USD(P&R) officials, the  services have used nonappropriated funding—that is, revenue generated  largely through user fees incurred by servicemembers and their families— to cover MWR program costs for which appropriated funding was  authorized. However, the use of nonappropriated funds to cover shortfalls  in appropriated funding support for MWR programs has been a long- standing issue about which Congress has previously expressed concern.  Specifically, in House Report 104-563, which accompanied H.R. 3230, a  bill for the National Defense Authorization Act for Fiscal Year 1997, the  House Committee on National Security established the annual DOD  Category A and B MWR programs reporting requirement to Congress,  after receiving testimony from the services’ MWR managers and noting a  disparity in the degree of appropriated funding support afforded these  programs particularly in the area of Category A and B programs. While  the committee recognized that shortfalls in appropriated funding support  for MWR programs requires the use of nonappropriated funding to meet  requirements, it also stated that the use of nonappropriated funding  resources—soldier, sailor, airman, and Marine money—to subsidize  appropriated funding activities should be minimized.

While the Army met the Category B funding target for fiscal year 2017,  the Navy, the Marine Corps, and the Air Force have each submitted plans  and briefed USD(P&R) on how they plan to meet the target in the future.  Navy officials said that they acknowledged the Navy’s challenges with  meeting the Category B funding target and, as a result, began assessing  their Category B programs to eliminate those that had limited use,  consolidate some where possible, and implement operational efficiencies.  Marine Corps officials indicated that the Marine Corps is committed to  preserving valuable MWR programs and restoring appropriate levels of  appropriated funding support in future budget planning. Specifically, the  Marine Corps plans to readdress appropriated funding levels in the  budget planning process in 2019. However, Marine Corps officials noted  they may continue to have challenges meeting the 65-percent funding  target in fiscal year 2018. Air Force officials said they will continue to  advocate for retaining established MWR program funding in the budget  process. Air Force officials said that for fiscal years 2014 through 2017,  Air Force leadership has increased appropriated funding for the MWR  programs each year to help get the Air Force closer to meeting the  Category B funding target.

DOD Has Not Comprehensively Evaluated the Funding Targets to Ensure They Are Appropriate

DOD has not comprehensively evaluated the funding targets for Category  A and B programs, which were instituted more than 20 years ago, to  ensure they are appropriate. Standards for Internal Control in the Federal  Government recommends that management periodically review policies  and procedures for continued relevance and effectiveness in achieving an  entity’s objectives. According to USD(P&R)officials, a limited evaluation  took place prior to 1995 that resulted in the Category A funding target in  DOD’s instruction being changed from 100 percent to 85 percent.  USD(P&R) officials said that the Category A appropriated funding target  was changed because some of the activities within the category have  expenses, such as for the food and beverage elements, that are able to  generate revenue and thus not authorized to use appropriated funds.

USD(P&R) officials stated that since that time there have been no further  evaluations of the Category A or Category B targets and agree that it is  time to evaluate the current relevance of the targets. Specifically they  noted the considerable changes to the budgeting and funding  environment that have taken place in the more than 20 years since the  Category A funding target was modified. In addition, officials told us they  also agree that it is time to evaluate the relevance of the Category B  funding target, which has never been modified. Specifically, officials said  that the services’ extended engagement in overseas conflicts and  constrained budgets have resulted in an operating environment that is  substantially different from the peacetime setting in which the targets  were first established.

Moreover, Standards for Internal Control in the Federal Government  requires management to document internal controls to meet operational  needs. Documentation of controls, including changes to controls, is  evidence that controls are identified, capable of being communicated to  those responsible for their performance, and capable of being monitored  and evaluated by an entity. Documentation also provides a means to  retain organizational knowledge and mitigate the risk of having that  knowledge limited to a few personnel, as well as a means to  communicate that knowledge as needed to external parties, such as  external auditors. As previously stated, officials stated that the Category A  funding target was updated sometime prior to 1995; however, officials did not have any specific documentation related to this change. Furthermore,  USD(P&R) officials said the targets were developed so long ago that  there is a general lack of information on the funding targets’ origins and  that they are not sure of the process or methodology that was used to  develop them.

The amount of time that has passed since Category A’s target was  modified, recent challenges in meeting the Category B target, and the  general lack of information on the funding targets’ origins raise concerns  about the appropriateness and continued relevance and effectiveness of  the targets in achieving MWR programs objectives. Until DOD  comprehensively evaluates the appropriateness of current targets for  Category A and B programs and, based on its evaluation, documents any  changes it makes to its funding targets, DOD cannot be certain that the  targets reflect the current operating environment and do not pose undue  financial burden on the servicemembers.

DOD Has Established an Oversight Structure and Performance Measures for MWR Programs but Has Not Developed Measurable Goals for Determining Whether MWR Programs Are Cost-Effective

DOD Has Established a Structure to Provide Oversight of MWR Programs

DOD has established a structure that specifies roles, responsibilities, and  procedures for overseeing MWR programs. Specifically, DOD Instruction  1015.10 assigns roles and responsibilities for oversight of MWR programs  to the USD(P&R), the Secretaries of the military departments, and the  Chiefs of the military services (i.e., the Chiefs of Staff for the Army and  the Air Force, the Chief of Naval Operations, and the Commandant of the  Marine Corps). In addition, the services’ respective policies assign roles  and responsibilities for MWR program oversight to the commander level.  Table 4 summarizes the general oversight roles and responsibilities for  DOD’s MWR programs.

The first level of oversight responsibility for MWR programs is assigned to  the USD(P&R). Specifically, responsibilities include the development of  department-level policies, program goals, performance measures, funding  targets, and the oversight of appropriated and nonappropriated funding  and expenditures for all MWR programs. To help ensure consistent  quality, USD(P&R) monitors the services’ compliance in meeting  minimum MWR funding targets and performance measures. As  previously discussed, if a service misses a funding target, USD(P&R)  officials said they ask that service to submit a remediation plan that  summarizes its intent to meet the target in the future, as USD(P&R) did in  fiscal year 2015 when several services missed appropriated funding  targets for Category A and B activities.

The second level of oversight is assigned to the Secretaries of the military  departments who are responsible for designating a central point of  contact within their respective service to facilitate MWR programs policy  compliance, coordinating with USD(P&R), and establishing funding  priorities and strategy for MWR programs. For example, service officials  we met with from the military departments said they have designated their  respective Assistant Secretary Offices for Manpower and Reserve Affairs  as the central point of contact for the services’ MWR programs.

The third level of oversight is assigned to the Chiefs of the military  services who are responsible for the development of overall goals and  uniform quality measures, which could include performance measures, for  MWR programs consistent with the performance measures set by DOD in  its instruction. For example, the Commander, Navy Installations  Command has developed uniform quality measures for the Navy MWR  Fitness program based on items such as customer satisfaction, usage  rates, and equipment maintenance, among other things. According to  officials, these quality measures provide a common tool to measure  customer satisfaction and the quality of each installation’s MWR Physical  Fitness program. Additionally, these Chiefs are also responsible for  helping to ensure MWR programs are resourced with appropriated and  nonappropriated funding according to financial categories and for  identifying their respective appropriated and nonappropriated accounts in  annual budgets to meet DOD funding goals. Service Chiefs are also  responsible for ensuring that military installations operate customer-driven  MWR programs that are determined locally by market analysis.

Lastly, the services’ respective policies assign roles and responsibilities  for MWR program oversight to the commander level. Additionally,  according to service officials, commanders assist with preparing an  annual briefing for USD(P&R) on their MWR programs, which includes  initiatives, challenges, program trends, and financial information. For  example, in fiscal year 2017, each of the services reported on new  initiatives to support MWR programs for servicemembers and their  families, some of which are highlighted in table 5.

DOD and the Services Have Performance Measures to Assess MWR Programs but These Measures Lack Measurable Goals for Determining Cost- Effectiveness

DOD Instruction 1015.10 identifies six broad categories of performance  measures that the services use to assess their respective MWR  programs. However, these measures do not include measurable goals,  which are needed to assess the cost-effectiveness of the 55 activities that  currently make up the MWR programs. Specifically, DOD identifies six  broad performance measure categories in its instruction and, according to  service officials, the services collect and use various types of information  within these categories to periodically assess and adjust these activities,  as appropriate. Table 6 summarizes the types of information that DOD  requires the services to collect across the six categories established in its  instruction.

In addition to the information that is to be collected across these six broad  categories, DOD established separate, more specific performance  measures for 2 of the 55 activities—namely, for Physical Fitness and for  Library Programs and Information Services. For the Physical Fitness  activity, the services are required to submit annual reports to DOD on  their compliance with meeting more specific performance measures in a  variety of areas such as administrative operations, staff qualifications,  facility equipment, and child play areas. Similarly, DOD requires the  services to report on a variety of areas related to the Library Programs  and Information Services activity, such as library operation plans,  customer programs and service, and technology infrastructure. Unlike the  broad measures contained in DOD’s Instruction, the specific performance  measures DOD established for the Physical Fitness and Library  Programs and Information Services activities tell the services exactly  what information to collect and report in each performance measure  category instead of the services having to develop specific measures on  their own.

In an effort to better evaluate MWR programs, the services also have  efforts underway that include the following to develop specific  performance measures for their programs beyond the broad performance  measures contained in DOD Instruction 1015.10.

Army. Army officials told us that they partnered with the Army Public  Health Center to build evidence-based MWR programs. Based on this  review, the Army found that Army MWR Community Recreation and  Fitness programs have not been formally evaluated as directed by  DOD Instruction 1015.10 requirements to measure and assess  programs. Additionally, the Army found that, while the Army Office of  the Assistant Chief of Staff for Installation Management provides  program oversight, it does not possess the capability to conduct  program evaluations. According to the results of the Army Public  Health Center report issued in June 2017, the Army initiated a three- phase approach for evaluating its MWR programs. The report showed  that assessing the evaluability of the Army MWR programs is phase  one. According to the Army, these evaluations will enable the Army to  validate program outcomes and better position itself to compete for  scarce resources. The report also showed that many of the 13 Army  MWR programs selected for review do not have direct links between  activities and the priority outcomes with behavioral, social, and  physical health, and that they do not have sufficient outcomes data  that have been consistently collected. Army officials said that phase  two will include the development of formal evaluation plans for  selected evaluable MWR programs. Lastly, Army officials said that  phase three will be the execution of the evaluation for two selected  MWR programs, which is on target to be completed by December  2018. While Army officials are learning how to evaluate programs  through this partnership with the Army Public Health Center, they said  that they have also learned that these endeavors are costly. Officials  said that a very modest program evaluation requires approximately  $300,000 to $500,000. Army officials also stated that program  evaluation requires support and participation by those organizations  and people that deliver the programs. Furthermore, according to Army  officials, resource reductions at the operational level (garrisons) are  increasingly restrictive, preventing them from collecting critical  information to support this multiphase effort.

Navy. Navy officials said that they use the MWR Enterprise Modeling  System, which is based on performance measures that have been  developed and routinely reviewed and updated by headquarters,  regional, and installation program managers. The MWR Enterprise  Modeling System is used as the baseline for the annual MWR  performance data call that measures actual program performance  against performance standards. Navy officials said that the  performance measures provide the business strategy and guidance to  ensure efficient, effective and market-driven delivery of programs and  services.

Marine Corps. Marine Corps officials said they collaborated with the  RAND Corporation to provide an analytically rigorous assessment  framework to evaluate program performance. The RAND Corporation  provided draft measures of performance. Marine Corps officials said  that the RAND Corporation also provided a user guide that outlines an  evaluation methodology and ensures consistent and standard  application. Marine Corps officials said that they are reviewing the  draft measures to determine appropriate data collection and have  drafted an implementation plan. Specifically, Marine Corps officials  said that they plan to brief Marine Corps installations in June 2018 on  the performance measures they plan to collect data from, which will  begin in fall 2018.

Air Force. Air Force officials said that they are building off the work  that the RAND Corporation undertook for the Marine Corps and have  also started collaborating with the RAND Corporation. The objective of  the Air Force study is to develop an evidence-based evaluation  framework for MWR programs that identifies immediate and mid-term  outcomes that contribute to airman and family readiness and  resilience. Specifically, the goal is to provide the Air Force with logic  models and performance measures that are tied to each of the  programs and services in the MWR portfolio. Air Force officials said  they expect to finish this study by June 2018. However, the officials  noted that implementing the performance measures will be a  challenge since these types of MWR programs are difficult to measure  and hard to capture data for.

While both the broad and specific measures established by DOD and the  services can provide useful context about the status of individual MWR  activities, they do not contain measurable goals that service officials could  use to compare program results with costs to determine whether an  individual activity is cost-effectively operating. Because the services’  efforts to develop specific performance measures are in early stages of  development it is too early to determine whether these efforts will result in  measurable goals that can be used to assess the cost-effectiveness of  the MWR programs.

DOD’s Financial Management Regulation specifies that performance  measurement should include program accomplishments in terms of  outputs and how those outputs effectively meet intended agency mission  goals. Further, cost itself can be a performance metric, but should also be  combined with an effectiveness measure, such as the percentage of a  goal achieved at a level of expected performance, to ensure that the  resulting output is cost effective. Additionally, through our prior work on  performance measurement, we have reported that performance goals  and measures should align with an agency’s goals and mission. However,  in reviewing DOD Instruction 1015.10, we found no mention of any goals,  mission, objectives, or purpose for the MWR programs. There is one  section entitled “policy” in the instruction that included items that resemble  goals. Specifically, the instruction stated that MWR programs:  1.  are an integral part of the military and benefits package;  2.  build healthy families and communities and provide consistently high- quality support services that are commonly furnished by other  employers or by state and local governments to their employees and  citizens;  3.  encourage positive individual values and aid in recruitment and  retention of personnel; and  4.  promote esprit de corps and provide for the physical, cultural, and  social needs; general well-being; quality of life; and hometown  community support of servicemembers and their families.

USD(P&R) officials who have responsibility for developing MWR program  goals acknowledged that these policy items function as strategic goals  but were not clearly identified as such in the instruction and also  acknowledged that the instruction does not include measurable goals for  assessing cost-effectiveness. In addition, USD(P&R) officials said that  they are starting a review of DOD Instruction 1015.10 and did not know  yet whether they would make any changes to the goals or expand the  reporting requirement to include all 55 activities. Until DOD develops  performance measures that include measurable goals, DOD officials and  other decision makers, such as Members of Congress, may find it difficult  to determine whether the MWR programs and the activities that make up  the MWR programs are meeting servicemember needs in a cost-effective  manner.

DOD’s multibillion dollar MWR programs provide a wide range of benefits  for servicemembers and their families that ultimately help support military  missions and readiness, both in times of war and peace. DOD has  established funding targets for providing appropriated funding support for  Category A and B MWR programs. However, the funding targets have not  been comprehensively evaluated in the last 20 years to determine their  current relevance. Until DOD comprehensively evaluates the  appropriateness of current funding targets and documents any changes  made to the targets, DOD’s funding targets may not reflect the current  operating environment, and may be posing an undue burden on the  servicemembers. DOD has also not developed performance measures  with measureable goals that would allow it to assess the cost- effectiveness of its MWR programs. Without performance measures that  include such measurable goals, it will be difficult for DOD and Congress  to determine whether the individual activities and overall MWR programs  are meeting desired outcomes in a cost-effective manner.

We are making the following two recommendations to DOD.

We recommend that the Secretary of Defense ensure that the USD(P&R),  in consultation with the Secretaries of the military departments,  comprehensively evaluate the funding targets for Category A and B MWR  programs and document any changes made to the targets and the  methodology used. (Recommendation 1)

We recommend that the Secretary of Defense ensure that the USD(P&R),  in consultation with the Secretaries of the military departments, develop  measurable goals for its MWR programs’ performance measures to  determine the programs’ cost-effectiveness. (Recommendation 2)

We provided a draft of this report to DOD for review and comment. In its  comments, DOD concurred with our recommendations and noted actions  that it is taking. DOD’s comments are reprinted in their entirety in  appendix II. DOD also provided technical comments, which we  incorporated into the report as appropriate.

We are sending copies of this report to the appropriate congressional  committees; the Secretary of Defense; the Secretaries of the Army, the  Navy, and the Air Force; the Commandant of the Marine Corps; and the  Under Secretary of Defense for Personnel and Readiness. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff members who made key contributions to  this report are listed in appendix III.

Appendix I: Department of Defense’s Morale, Welfare, and Recreation Program Categories

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Kimberly A. Mayo, Assistant  Director; Rebekah Boone; Mae Frances Jones; Felicia Lopez; Stephanie  Moriarty; Cynthia Saunders; John W. Van Schaik; Paul Seely; Carter  Stevens; and Roger Stoltz made key contributions to this report.