F-35 Aircraft Sustainment: DOD Needs to Address Challenges Affecting Readiness and Cost Transparency

The F-35 aircraft represents the future of tactical aviation for the U.S. military, and is DOD's most expensive weapon system, with sustainment costs alone estimated at more than $1 trillion over a 60-year life cycle. As the F-35 program approaches full-rate production, DOD is working to deliver an affordable sustainment strategy that is able to meet the needs of the military services. This strategy is being tested as DOD stands up military depots, trains personnel, and supports its first operational squadrons—with plans to establish multi-year, performance-based contracts by 2020.

The National Defense Authorization Act for fiscal year 2017 includes a provision for GAO to review the F-35 program's sustainment support structure. This report assesses (1) the status of DOD's efforts to sustain the F-35 fleet and any challenges it has faced; (2) the extent to which DOD is positioned to enter into multi-year, performance-based F-35 sustainment contracts; and (3) the progress, if any, DOD has made toward reducing F-35 sustainment costs and the extent to which costs are transparent. GAO reviewed DOD and contractor documentation, analyzed data, and interviewed relevant officials.

The Department of Defense (DOD) is sustaining over 250 F-35 aircraft (F-35) and plans to triple the fleet by the end of 2021, but is facing sustainment challenges that are affecting warfighter readiness (see table). These challenges are largely the result of sustainment plans that do not fully include key requirements or aligned (timely and sufficient) funding. DOD is taking steps to address some challenges, but without more comprehensive plans and aligned funding, DOD risks being unable to fully leverage the F-35's capabilities and sustain a rapidly expanding fleet.

DOD's plan to enter into multi-year, performance-based F-35 sustainment contracts with the prime contractor has the potential to produce costs savings and other benefits, but DOD may not be well positioned to enter into such contracts by 2020. To date, DOD has not yet achieved its desired aircraft performance under pilot (i.e., trial) performance-based agreements with the prime contractor. In addition, the level of performance DOD has contracted for is generally below what the services desire (see figure 2 for Marine Corps example). Also, the three performance metrics DOD is using to incentivize the contractor under these pilot agreements may not be the appropriate metrics to achieve desired outcomes, in part because they are not fully reflective of processes for which the contractor has control. This can make it difficult for DOD to hold the contractor accountable. Further, due to system immaturity, DOD does not have full information on F-35 sustainment costs and technical characteristics such as reliability and maintainability, which could hinder its ability to effectively negotiate performance-based contracts with the contractor by 2020. Without examining whether it has the appropriate metrics to incentivize the contractor or a full understanding of the actual costs and technical characteristics of the aircraft before entering into multi-year, performance-based contracts, DOD risks overpaying the contractor for sustainment support that does not meet warfighter requirements.

DOD has taken actions to reduce F-35 sustainment costs, but estimated life cycle costs have increased and are not fully transparent to the military services (see figure 3). Specifically, the services do not fully understand how the costs they are being charged by the program office are linked to the capabilities they are receiving, citing unexplained cost increases and difficulty in tracking their requirements to contracts. For example, the Marine Corps received an initial funding requirement for fiscal year 2017 sustainment of $293 million, which then increased to $364 million in the execution year. This lack of transparency is due in part to insufficient communication between the program office and the services, and it puts the services in a difficult position as they consider critical trade-offs that may make F-35 sustainment more affordable. Without improving communication with the services about the costs they are being charged, the services may not be able to effectively budget for long-term sustainment.

GAO recommends that DOD revise sustainment plans, re-examine metrics and ensure that it has sufficient knowledge of costs and technical characteristics before entering into performance-based contracts, and improve communication with the services about sustainment costs. DOD concurred with these recommendations.

The F-35 Lightning II aircraft (F-35) represents the future of tactical  aviation for the Department of Defense (DOD), and is intended to replace  a variety of legacy fighter aircraft in the Air Force, Navy, and Marine  Corps. The F-35 is also DOD’s most ambitious and costly weapon system  in history, with sustainment costs alone for the three U.S. military services  estimated at $1.12 trillion over a 60-year life cycle. These sustainment  costs provide for requirements such as a supply chain for the delivery of  spare parts, technical data to perform maintenance, engineering support,  and other functions critical to support the fleet. The sustainment contract  signed in fiscal year 2017 to support such requirements amounted to  almost $1.1 billion. Because DOD has pursued a highly concurrent  acquisition strategy for the F-35 with significant overlap among  development, testing, and manufacturing activities, it must stretch its  resources to meet the needs of continued system development and  production while at the same time sustaining the more than 250 aircraft it  has already fielded. As the F-35 program approaches planned milestones  such as operational testing in 2018 and full-rate production in 2019, DOD  is working to deliver a sustainment strategy that will be both affordable  and able to meet the needs of the military services, its international  partners, and the nearly 3,200 F-35 aircraft expected to be fielded  globally over the life cycle of the program. This strategy is now being  tested as DOD stands up military depots, trains personnel, and supports  the first three operational F-35 squadrons, with an eye toward  establishing multi-year, performance-based contracts with the prime  contractor and rapidly expanding its fleet.

In 2014, we reported that annual F-35 operating and support costs were  estimated to be considerably higher than the combined annual costs of  several legacy aircraft, and that while DOD had begun some cost-savings  efforts and established sustainment affordability targets, DOD did not use  the military services’ budgets to set these targets. In 2016, we found that  DOD faced risks that could affect the F-35’s Autonomic Logistics  Information System (ALIS)—a complex system supporting operations,  mission planning, supply-chain management, maintenance, and other  processes. We recommended that DOD develop affordability constraints  linked to the military services’ budgets, develop a plan to address ALIS  risks, and improve its sustainment cost estimates, among other things.  The department generally concurred with our recommendations, and has  taken some actions in response. See the Related GAO Products page at  the end of this report for a list of our previous F-35 products.

The National Defense Authorization Act for Fiscal Year 2017 includes a  provision for us to review the sustainment support structure of the F-35  program. This report assesses (1) the status of DOD’s efforts to sustain  the F-35 fleet and any challenges it has faced; (2) the extent to which  DOD is positioned to enter into multi-year performance-based F-35  sustainment contracts; and (3) the progress, if any, DOD has made  toward reducing F-35 sustainment costs, and the extent to which costs  are transparent to the military services.

In support of these objectives, we gathered various data related to F-35  sustainment, such as supply chain and repair data and aircraft  performance data. To determine the reliability of these data, we collected  information on how the data were collected, managed, and used through  a questionnaire and interviews with relevant DOD officials and the prime  contractor. In our assessment, we identified some limitations in the way  that certain data are collected and reported, such as data related to  aircraft performance, aircraft that are not mission-capable due to supply  issues, and parts cannibalization rates that could potentially result in  inaccuracies. However, these data come from the program’s data  systems of record, and are the same data used by the program office and  prime contractor to monitor the health of the supply chain and assess  aircraft performance against contract requirements and program  objectives. As such, they are the best source of data available to provide  information on the progress and challenges within the program. We  determined that the data presented in our findings were sufficiently  reliable for how we reported them. Specifically, the parts cannibalization  rates that we reported are consistent with the trends observed across  other key data elements within the program, and with the testimonial  evidence provided to us by the units with whom we met during our review,  and they are sufficiently reliable to report as a data trend relative to  program objectives. All other performance data presented in our report  are sufficiently reliable to present as specific data points, in order to  describe the status of sustainment requirements and measured aircraft  performance across key metrics as reported by the prime contractor and  DOD.

To assess the status of DOD’s efforts to sustain the F-35 fleet and any  challenges it has faced, we reviewed DOD and contractor plans,  briefings, and schedules to determine the current status of key  requirements and decision points necessary to establish F-35  sustainment capabilities and compared actual data about F-35 repair and  supply chain capabilities with DOD’s objectives for these capabilities to  identify areas of challenge for the program. Specifically, we obtained data  on aircraft that were not mission-capable due to supply issues from  January 2017 through August 7, 2017, and on average repair times as of  May 2017, in order to provide the most recently available information  about the health of the supply chain. As discussed above, we determined  that these data were sufficiently reliable to present as specific data points.  In addition, we identified key acquisition program management practices  that can improve program outcomes if implemented, and we assessed  DOD’s sustainment planning efforts against these criteria.

To assess the extent to which DOD is positioned to enter into multi-year  performance-based F-35 sustainment contracts, we reviewed  documentation related to DOD’s pilot—or trial— performance-based  agreements for F-35 sustainment, reviewed performance management  guidance and processes, and interviewed officials to determine how  performance data were being collected and assessed. We also obtained  aircraft performance data from the Sustainment Performance  Management System for the 2016 pilot performance-based agreement  (March 2016 – December 2016) and the 2017 pilot performance-based  agreement (March 2017–June 2017) to the extent available at the time  we completed our audit work. These time periods are the only time  periods for which the program office has assessed contractor  performance under these pilot arrangements. As discussed above, we  determined that these data were sufficiently reliable to present as specific  data points. In addition, we reviewed aircraft maturity, reliability, and  maintainability data, reviewed documentation related to cost visibility  issues, and spoke to relevant officials to determine DOD’s level of  understanding of the costs and technical characteristics that will affect  future sustainment support. Further, we reviewed DOD guidance and best  practices to identify attributes of ideal performance metrics and effective  performance-based agreements, and we used those attributes as a basis  for determining whether DOD has the appropriate metrics to achieve  desired outcomes, and the necessary information to effectively negotiate  multi-year, fixed-price, performance-based contracts with the prime  contractor by 2020, as planned.

To assess the progress, if any, DOD has made toward reducing F-35  sustainment costs and the extent to which costs are transparent to the  military services, we reviewed documentation related to cost-reduction  efforts, sustainment contracts, F-35 cost-sharing rules, and budget  documentation from both the program office and the military services. We  also reviewed F-35 Joint Program Office sustainment cost estimates from  fiscal year 2012 through fiscal year 2016 in order to identify changes  made to the estimate since the program’s sustainment cost baseline was   established in 2012. The fiscal year 2016 sustainment cost estimate is the  most current cost estimate conducted by the program office. In addition,  we interviewed relevant officials from the program office and military  services to discuss how the program office informs the military services of  F-35 sustainment costs, and the degree to which the services understand  these costs and the sustainment capabilities provided. We compared this  information with program guidance, and with key operating principles for  programs that involve multiple government customers identified in our  prior work, in order to assess the transparency of F-35 sustainment costs  for the military services. Appendix I describes our objectives, scope, and  methodology in greater detail, including more information about our data  reliability assessment and a complete list of the organizations with whom  we met during this review.

We conducted this performance audit from October 2016 to October 2017  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

The F-35 Lightning II program, also known as the Joint Strike Fighter  program, is a joint, multinational acquisition intended to develop and field  a family of next-generation strike fighter aircraft for the U.S. Air Force,  Navy, and Marine Corps (hereinafter referred to as the services), eight  international partners, and foreign military sales customers. There are  three F-35 variants and each will be a multi-role, stealthy strike aircraft  replacement for or complement to legacy fighter aircraft, as seen in   figure 1.

DOD initiated the F-35 program in October 2001, and it is nearing the end  of system development and preparing for operational testing. DOD has  also been concurrently fielding and operating a growing fleet of aircraft as  part of low-rate initial production. As of August 2017, 253 aircraft have  been fielded and are flying from nine locations in the United States and  three international locations. The Marine Corps and Air Force declared  initial operational capability in 2015 and 2016, respectively, and the Navy  is scheduled to declare initial operational capability in 2018. In 2019,  DOD plans to begin full-rate production of the aircraft. See figure 2 for a  timeline of major events and anticipated fleet growth in the F-35 program.

By full-rate production, DOD would generally be required to establish  adequate sustainment and support systems for the F-35. Per DOD  guidance for weapon system acquisitions, these sustainment and support  systems should be defined in a support concept that is incorporated into a  sustainment strategy. For the F-35, this concept should comprise the  necessary plans to conduct operations, maintenance, and sustainment  throughout the system’s life cycle, with the F-35 Life Cycle Sustainment  Plan serving as the principal document governing F-35 sustainment.  According to F-35 operational requirements, this concept must provide  warfighting and peacetime capability with the lowest cost of ownership,  and all variants must be able to deploy rapidly, sustain high mission  reliability, and sustain a high sortie-generation rate.

Sustainment for the F-35 aircraft is a large and complex undertaking with  many stakeholders. The F-35 Joint Program Office is responsible for  managing and overseeing the support functions required to field and  maintain the readiness and operational capability of the F-35 aircraft  across the enterprise. The F-35 program currently relies heavily on  contractors to provide sustainment support and has two product support  integrators. As the product support integrator for the aircraft system,  Lockheed Martin is charged with integrating sustainment support for the  system, including that for the F-35 supply chain, depot maintenance, and  pilot and maintainer training, as well as providing engineering and  technical support. Currently, DOD is contracting for sustainment support  with Lockheed Martin largely through annual contracts, and according to  F-35 program officials, plans to transition to 5-year, fixed-price,  performance-based sustainment contracts in 2020.

DOD has established a Hybrid Product Support Integrator organization— a collaboration of government and contractor organizations tasked with  managing product support to meet the F-35 strategy and performance  outcomes. This organization was initially established in 2016 as a part of  the F-35 Joint Program Office, and is expected to be fully implemented by  2019. According to program officials, the establishment of the Hybrid  Product Support Integrator is an acknowledgement that DOD needs to  take a more significant role in providing sustainment support for the F-35.  In addition, the U.S. Air Force, Navy, and Marine Corps have each  established an F-35 integration office or cell focused on how the services  will operate and afford the F-35, among other things.

DOD is planning to meet the sustainment requirements of its F-35  customers by providing a common, global support solution. As part of this  common solution, participants share critical aspects of sustainment  support, some of which are discussed below, and which are in various  stages of implementation to support the growing fleet.

Depot maintenance: The F-35 sustainment strategy has a two-level  maintenance concept, consisting of organizational-level maintenance  performed by squadron-level personnel, and depot-level maintenance.  Depot-level maintenance includes structural repair, software  upgrades, engine system overhaul and repair, component repair, and  other activities that require specialized skills, facilities, or tooling to  conduct the repairs. DOD is establishing modification and repair  capabilities at six military service depots in the United States and  additional repair facilities overseas.

Supply chain: All F-35 customers, including the U.S. military services  and international partners, share a global pool of spare parts, which is  managed by Lockheed Martin. According to program officials, these  pooled assets are unique to the F-35 and include consumable and  repairable spare parts for the airframe, support equipment, pilot flight  equipment, and training devices. The services and international  partners can also purchase packages of spare parts that are tailored  to their individual deployment and shipboard operational  requirements.

Training: Currently, the F-35 program is conducting pilot and  maintainer training at Eglin Air Force Base, Luke Air Force Base,  Marine Corps Air Station Beaufort, and Naval Air Station Lemoore.  The F-35 program’s training system includes pilot and maintenance  training devices and courseware that are tailored to multiple variants  and services.

Infrastructure: F-35 customers are responsible for setting up their  own F-35 facilities—hangars, training facilities, and depots, among  other things—and the program office works with them in a supporting  role. Sustainment infrastructure requirements to support the F-35 are  defined in a series of facility requirement documents that are updated  and provided to all customers annually.

F-35 Costs and Technical Characteristics

Many of the costs of F-35 sustainment—also known as operating and  support costs—are allocated across the military services and international  partners based upon a number of factors, including the number of aircraft  that each customer owns and their operational requirements. Such  operating and support costs consist of sustainment costs incurred from  the initial system deployment through the end-of-system operations, and  they include all costs of operating, maintaining, and supporting a fielded  system. The Office of the Director for Cost Assessment and Program  Evaluation develops independent cost estimates for F-35 operating and  support costs, which are reported in DOD’s annual F-35 Selected  Acquisition Report as the official operating and support cost estimates for  the program. Additionally, the program office develops an annual estimate  for the operating and support costs of maintaining and supporting the F- 35 over its 60-year life cycle, which can differ from the estimate  conducted by the Office of the Director for Cost Assessment and Program  Evaluation, due in part to differences in assumptions between the two  estimates. Additionally, there are numerous factors that will affect life- cycle operating and support costs for the F-35, including aspects of the F- 35 program that are still maturing. These include the following:

Reliability and maintainability: Reliability and maintainability data  measure aircraft performance to determine how often the aircraft  experiences failures and how much time it takes to repair those  failures. These data are monitored through a series of metrics that  measure the intended performance of the aircraft in meeting its  requirements as it progresses toward maturity at a cumulative  200,000 flight hours, with at least 75,000 flight hours each for the F- 35A and F-35B, and 50,000 flight hours for the F-35C. Reliability and  maintainability drive sortie-generation rates and the size of the  logistics footprint for the F-35, as well as inform program operating  and support costs, which are tied to the performance of the system at  maturity.

Technical data: Technical data for weapon systems include the  details necessary to ensure the adequacy of performance, as well as  instruction, maintenance, and other actions needed to support  weapon systems. Technical data constitute an important part of a  weapon system program, such as the F-35. Identifying technical data  needs, costs, and ownership are essential for DOD to effectively  consider and maximize competition for future product support of F-35  sustainment.

DOD Is Currently Sustaining More Than 250 F-35 Aircraft, but Insufficient Planning Has Led to Significant Challenges That Pose Risk to Its Growing Fleet

DOD Is Sustaining More Than 250 F-35 Aircraft, but Faces Significant Challenges That Are Affecting Readiness

DOD has currently fielded and is sustaining more than 250 F-35 aircraft,  and the number is expected to triple by the end of 2021 and keep growing  as the program moves into full-rate production. DOD has also supported  significant F-35 milestones such as the initial operational capability  declarations of the Marine Corps and Air Force in 2015 and 2016,  respectively, and the transfer of an operational squadron to Japan in early  2017. As a fifth generation aircraft, the F-35 is intended to improve  situational awareness through sensor fusion and will enhance the ability  of legacy aircraft to conduct various missions while flying together with  it. The F-35 was also designed with increased stealth capabilities, the  capacity to carry weapons internally instead of externally to reduce drag  and enable stealth, and advanced sensor systems. In particular, the  aircraft is designed to execute missions in high-threat areas, requiring  fewer support assets and possessing a greater survivability rate as  compared with fourth generation aircraft such as the Air Force’s F-16s  and the Navy’s and Marine Corps’ F/A-18s. Squadron officials at multiple  F-35 locations that we visited expressed enthusiasm for the unique  capabilities of the aircraft, such as the increased situational awareness  that the F-35 provides pilots relative to legacy aircraft and the relative  ease with which pilots are able to learn how to employ its tactical  capabilities. They also noted improvement in the performance of the  aircraft as it has been continuously developed.

However, DOD is facing several key sustainment challenges that pose  risks to its ability to meet current and future warfighter readiness  requirements, and these could limit the ability of the military services to  fully leverage the capabilities of the aircraft. Table 1 summarizes these  challenges, which are largely attributable to insufficient planning, as  discussed in more detail below.

Repair capacity: DOD does not have enough capacity to repair F-35  aircraft parts because the establishment of repair capabilities at the  military depots is 6 years behind schedule. There are many different  components of the F-35 aircraft that DOD plans to repair at the six military  depots within the United States, as documented in an F-35 Depot  Implementation Plan. Repair capabilities at the military depots were  originally planned to be completed by 2016, but program officials told us  that some capabilities have now been delayed until 2022. Program  officials in part attributed these delays to the military services not  providing enough funding for depot requirements; however, service  officials told us that the program office did not clearly identify some depot  requirements in a timely manner necessary for the services to fund those  requirements.

In addition, DOD did not plan for and fund the stocks of material needed  to repair parts at the depots—referred to as “lay-in material.” Program  officials said that they had incorrectly assumed that lay-in material would  be included as part of the contracts for establishing repair capabilities at  the military depots. As a result, DOD has had to fund and negotiate  additional contracts with the prime contractor for the lay-in material.  Currently, moreover, due in part to the late identification of requirements  and funding, the lay-in material to support repairs for more than a dozen  different aircraft components is not expected to be delivered to the depots  until months—or in some cases, years—after the technical capabilities to  conduct the repairs have been established. As seen in figure 3, for certain  F-35 parts, these delays have resulted in repair times that are significantly  longer than those the program had projected, leading to repair backlogs.  According to prime contractor officials, because of these capacity  shortfalls, DOD is currently relying on the original equipment  manufacturers to repair parts, but the capacity of these manufacturers is  already strained by requirements to produce the parts needed to support  aircraft production.

Program officials said that establishing the depot repair capabilities is now  the F-35 Joint Program Office Product Support Manager’s top priority.  As such, the program is working to implement several different initiatives  to accelerate the development of repair capabilities, including trying to  better align lay-in material requirements with the activation of repair  capabilities, prioritizing the establishment of certain repair capabilities to  align with the readiness requirements of the fleet, and looking at options  to decrease the amount of time that it takes to establish repair capabilities  for each component line. However, program officials said that plans are  still preliminary, and that they are unsure how much funding will be  available to implement these initiatives.

Spare parts: DOD is experiencing shortages of spare parts in the F-35  supply chain, resulting in lower than expected readiness. From January  through August 7, 2017, the prime contractor reported that the average  percentage of time that F-35 aircraft were unable to fly because they were  awaiting parts was about 22 percent—more than double that of DOD’s  objective of 10 percent, as seen in figure 4.

According to program office and contractor officials, the shortages of  spare parts are due in part to the delays in the establishment of depot  repair capabilities, incomplete plans and funding that did not account for  the long lead time for parts, insufficient amounts of service funding, and  poor reliability of certain parts. For instance, 19 percent of F-35 parts  have a lead time of more than 2 years. The 2 to 3 years that it takes to  procure these parts includes both a lengthy period for contracting and a  period for the production of the parts once contracts have been  established. However, program office and military service officials told us  that the timing of prior service funding authorizations and contract awards  did not account for this long lead time to procure parts, resulting in parts  that were late to meet the military services’ operational needs. According  to DOD officials, the parts within the F-35 global pool of spare parts are  unique to the F-35 system and generally cannot be obtained from other  sources.

The program office and prime contractor have identified steps needed to  increase the availability of spare parts to prevent these challenges from  worsening as the number of aircraft in the fleet grows, such as improving  the production and repair capacity of suppliers and aligning the timing of  the military services’ funding authorizations with the required lead time for  parts. However, according to DOD documentation, planned funding and  contract awards for fiscal years 2018 and 2019 are still forecasted to be  later than needed to meet demand for new parts, and the program’s  ability to accelerate this timeline is uncertain. Thus, parts shortages are  expected to continue for several years and may worsen if DOD and the  contractor are not able to fully implement these actions.

DOD Has Not Fully Defined Future Technical Data Needs, and Some Technical Data Are Immature

DOD has not fully defined all of the technical data it needs from the prime  contractor to maximize the potential for future competition of contracts  among providers for sustainment requirements, nor does it know the  associated costs of these data. In 2014, we recommended that the  program office develop a long-term Intellectual Property Strategy to  include the identification of all critical technical data needs and their  associated costs. As of September 2017, the program has taken some  steps to develop an Intellectual Property Strategy, but it has not identified  all critical needs and their associated costs. Program officials said that  they are currently working with the prime contractor to develop a list of  technical data requirements. Program officials said that once this effort is  complete, DOD will be in position to begin prioritizing and negotiating for  specific data rights that the program needs to facilitate its sustainment  plans. Officials acknowledged, however, that there is risk associated with  efforts to obtain required technical data rights for F-35 sustainment to  promote increased competition because the contractors may not be  willing to provide these rights, or the costs may be too high. They also  told us that the program office deals with such risks on a case-by-case  basis, and that if a data right needed by the program office to implement  the sustainment strategy cannot be obtained, then plans will have to be  adjusted accordingly. Program officials said that, in some cases, they will  likely have to make legal claims against the prime contractor’s technical  data rights assertions, based on government funding of such products.

Moreover, the technical data needed to repair F-35 aircraft, such as  maintenance instructions, are still not fully developed. According to  contractor officials, the contractor and DOD have developed and verified  more than 84 percent of the unit-level technical data needed to address  known maintenance requirements, such as instructions for how to replace  specific parts on the aircraft. However, according to program and  contractor officials, the technical data needed for maintainers to  troubleshoot issues with the aircraft are lagging behind planned  development. Such data are intended to help maintainers when the  source of a maintenance issue is unclear, by providing guidance on the  actions needed to isolate the most likely problems. In the absence of  troubleshooting instructions, maintainers sometimes incorrectly identify  what needs to be fixed on the aircraft. For instance, officials from one  squadron said that the troubleshooting data are sometimes insufficient to  pinpoint the issue with the aircraft, which can lead the maintainer to  remove a component, order a new part from the contractor, and  subsequently find that the new part does not fix the issue—a scenario  that is both inefficient and costly. According to program and contractor  officials, the immaturity of technical data for troubleshooting maintenance  issues could be contributing to the high rate of parts that the F-35  squadrons are sending to the depots for repair that do not actually need  to be repaired, resulting in inefficiencies at the depots. For example,  officials at one depot we visited said that 68 percent of the parts they  receive from F-35 squadrons do not need to be repaired and that the  process for testing such parts usually takes nearly 10 hours to complete,  which is both inefficient and can add to repair backlogs.

DOD Has Not Developed a Plan for Intermediate-level Maintenance Capabilities

The Navy and Marine Corps require intermediate-level maintenance  capabilities for shipboard deployments because it is more difficult and  time-consuming to obtain spare parts, or to send parts to the depots for  repair, when onboard a ship. DOD has been conducting analyses to  support the requirement and has recently identified the initial  intermediate-level repair capabilities that it plans to implement, including  select avionics, support equipment, and hydraulic repairs. These  decisions will trigger other requirements and related costs that must be  planned for—such as for personnel, technical data, support equipment,  and updates to policies governing the maintenance of spare parts— before the capability can be implemented. For example, program officials  told us that once determinations are made about intermediate-level  maintenance, the program will have to develop a plan that specifies what  technical data rights are needed, and when, to facilitate intermediate-level  maintenance, and will then have to negotiate with the contractor to obtain  those technical data rights. In August 2017, the program office identified  new funding requirements for DOD to implement initial intermediate-level  maintenance capabilities for fiscal years 2019 through 2023. However,  these requirements are not currently funded in DOD’s budget, leaving a  projected shortfall of $267 million over this time period.

Because a funded plan for intermediate-level maintenance is not yet in  place, the Marine Corps will not have the desired level of intermediate- level maintenance capabilities for its initial shipboard deployments  planned for 2018. Accordingly, it will be highly reliant on the currently  challenged F-35 supply chain and depot repair capabilities for support,  and will likely experience degraded readiness. In addition, without such a  plan, it is unclear whether such capabilities will be available to support the  Navy’s first planned F-35 shipboard deployments in 2021.

DOD Faces Delays in Required ALIS Development, and Its Development Plan Is Not Fully Funded

Central to F-35 sustainment is the Autonomic Logistics Information  System (ALIS)—a complex system supporting operations, mission  planning, supply-chain management, maintenance, and other processes.  However, ALIS is in continuous development, with planned updates that  support required sustainment capabilities for years to come. For example,  future versions of ALIS are intended to improve data collection and  reporting, and to provide capabilities to support intermediate-level  maintenance. Historically, ALIS has experienced delays. For instance, an  ALIS version that was initially planned to be completed for testing in 2010,  is now being tested in 2017. In 2016 we found that DOD did not have a  plan to ensure that ALIS was fully functional as key program milestones  approached, and we recommended that DOD develop a plan to prioritize  and address ALIS risks. Since that time, the program office has  implemented this recommendation through the development of an ALIS  Technical Roadmap to plan for these requirements. However, emerging  requirements, such as to address cyber security vulnerabilities and  system obsolescence, will likely lead to changes in the Roadmap that  could further delay the date when these sustainment capabilities are  provided. Furthermore, the requirements and associated timelines for  ALIS development that are identified in this plan may not be realistic  because the requirements are not fully funded in upcoming service  budgets, resulting in additional risks to the program’s plan.

DOD’s Sustainment Plans Do Not Fully Include Key Requirements, Associated Timelines, and Aligned Funding, but Some Initial Steps Are Being Taken

As discussed above, DOD’s challenges are due in large part to  sustainment plans that do not fully include key requirements, associated  timelines, and aligned (that is, timely and sufficient) funding to support  those requirements. F-35 program stakeholders have long recognized the  program’s need for more comprehensive and detailed planning  documents to identify the key activities and decision points necessary to  establish sustainment capabilities and guide the F-35 sustainment  strategy. For instance, in 2009 an Independent Logistics Assessment  team recommended, among other things, that DOD develop a program- wide integrated master schedule that includes key governmental activities  and tasks necessary to establish F-35 logistics capabilities required  through full-rate production, but the program did not develop such a tool.  In 2014 the program office identified the need to establish a road map  with clear decision points to prepare the F-35 enterprise for long-term  sustainment. Finally, in December 2016 the Under Secretary of Defense  for Acquisition, Technology, and Logistics directed the program office to  submit an integrated master schedule for the deployment of global F-35  sustainment capabilities by January 2017, which is not yet completed.

Program officials said that they are now developing an integrated master  schedule, and that this schedule will incorporate major sustainment  milestones required to implement the program’s sustainment strategy.  DOD is also updating sustainment strategy documents, including the F-35  Life Cycle Sustainment Plan and Acquisition Strategy, to include an  Intellectual Property Strategy. However, the timeframes for completion of  these documents are uncertain, in part due to ongoing DOD efforts to  refine its follow-on modernization plans for the F-35, which will affect the sustainment plans. Thus, the scope and the degree to which these  updates will address the challenges that DOD is facing are unclear. For  instance, an Office of the Secretary of Defense official charged with  reviewing these plans said that there is still significant work to be done by  the military services and the program office to identify and align  sustainment requirements with funding in order to support the fiscal year  2019 budget process, which will ultimately be necessary to inform these  plans. Military service headquarters officials told us that, as customers of  the program, they need to better understand from the program office  when sustainment capabilities—such as military depots—will be  established, and when associated funding is needed to support that  schedule. In August 2017, the program office identified some specific  funding requirements for the military services, beyond what they have  already budgeted for F-35 sustainment, which are needed to address  some of the sustainment challenges discussed above—including spare  parts shortages, gaps in depot lay-in material, and ALIS development.  While this is a positive step by the program office, it also demonstrates  that DOD faces a funding shortage of approximately $1.5 billion between  fiscal years 2018 and 2023 for F-35 sustainment, as well as significant  readiness risks associated with this lack of alignment between  requirements and funding.

The different elements of F-35 sustainment support are highly integrated,  and challenges or delays in one area can significantly affect outcomes in  other areas. For example, the delays in established repair capacity at the  depots constitute one of the reasons why the program has an insufficient  supply of spare parts. Procurement decisions can also significantly affect  sustainment outcomes. The Air Force and Marine Corps are considering  an acceleration of their purchases of F-35 aircraft, thus creating more  demand on the already strained sustainment enterprise, for which DOD  has not always provided timely funding (for example, funding for spare  parts).

Our prior work on acquisition program management has identified a  number of key program management practices that can improve program  outcomes if implemented, such as clearly establishing well-defined  requirements, developing realistic cost estimates and schedules, and  securing stable funding that matches resources to requirements. As  DOD prepares for the growth of the fleet and attempts to address existing  sustainment challenges, its effort to develop an integrated master  schedule is a positive step. Such a schedule, if comprehensive and  realistic, could be a critical tool to guide the revision of DOD’s  sustainment plans to better ensure that the plans that form the basis of its  strategy are sufficient to meet warfighter requirements. Ultimately,  however, without plans that include all key requirements and decision  points with aligned funding, the F-35 program will likely face continual  challenges in providing timely sustainment support to the warfighter, and  may have difficulties in fully implementing its F-35 sustainment strategy in  time to meet the needs of a growing fleet. Further, as the services  consider accelerating their purchases of F-35 aircraft, DOD risks  purchasing aircraft that the program and the services are not ready to  sustain.

DOD Is Testing Agreements with the Contractor but May Not Be Well Positioned to Enter into Multi-year, Performance-based Sustainment Contracts by 2020

DOD Is Testing Performance-based Agreements to Incentivize the Prime Contractor

DOD is conducting pilot—or trial—performance-based agreements with  the prime contractor as a part of its annual cost-reimbursable sustainment  contracts, in order to test metrics and performance-management  processes. According to F-35 program officials, DOD plans to transition  to multi-year, fixed-price, performance-based contracts in fiscal year  2020. Performance-based logistics is a support strategy that emphasizes  performance in contracts, rather than delivery of goods and services, and  payment is related to the degree to which performance meets contracted  standards. In 2012, the Under Secretary of Defense for Acquisition,  Technology, and Logistics directed an increased use of performance- based logistics agreements, stating that such agreements can yield  significant cost and performance benefits if effectively implemented.

DOD has developed a series of performance objectives to provide insight  into the level of sustainment support that the prime contractor is providing  to the military services. From these objectives, DOD has selected three  system-level metrics, listed below, to incentivize the contractor under the  pilot performance-based agreements:

Air Vehicle Availability (AVA): measures the percentage of total  time during which aircraft are safe to fly, available for use, and able to  perform at least one tasked mission;

Full Mission Capable (FMC): measures the percentage of time  during which aircraft are fully capable of accomplishing all tasked  missions;

Mission Effectiveness (ME): measures the extent to which the F-35  components and mission systems affected the successful completion  of each assigned mission.

In these pilot agreements, DOD and the contractor together negotiated  minimum and objective targets against which the performance of the  aircraft—and the support provided by the contractor to enable that  performance—is measured. For fiscal years 2016 and 2017, these  agreements were 1-year, cost-reimbursable contracts with potential incentives for the contractor based on assessed performance of the  aircraft across the three system-level metrics. According to F-35  program officials and documentation we reviewed, DOD plans to  establish a 2-year contract for fiscal years 2018 through 2019, with select  elements that are performance-based, in preparation to transition to a 5- year, fixed-price, performance-based contract for the 2020—2024 time  period. Program officials said that this 5-year contract is planned to  include 2 base years and 3 pre-negotiated option years.

DOD Has Not Achieved Most of Its Performance Targets for the Pilot Agreements and May Not Be Using the Appropriate Metrics to Achieve Desired Outcomes

DOD Has Not Achieved Most of Its Performance Targets

DOD did not achieve most of the performance targets that it set for the  pilot performance-based agreements for the 2016 sustainment contract.  Subsequently, DOD negotiated lower targets for some metrics in the 2017  sustainment contract. As of June 2017, DOD was meeting several of the  minimum targets established in the 2017 sustainment contract, but none  of the objective targets. According to program and contractor officials, the  failure to meet these targets is largely due to the sustainment challenges  that we discussed previously in this report. For example, the limited  availability of spare parts within the F-35 supply chain is contributing to  lower than expected AVA and FMC rates. Figure 5 below shows the  actual fleet performance results for the 2016 and 2017 (through June  2017) pilot performance-based agreements. The 2016 pilot performance- based agreement began in March 2016 and spanned a 10-month period,  through December 2016. The 2017 agreement began in March 2017, and  program officials said that it is expected to continue through February  2018.

Furthermore, the performance targets established in the sustainment  contracts for the pilot performance-based agreements are lower than the  desired aircraft performance targets that the services have identified for  their aircraft. As part of the pilot performance-based agreements, each of  the military services has established individual agreements with the  program office that identify their respective required levels of minimum  and objective aircraft performance for their units, across key metrics.  Program officials said that while they try to meet the services’  performance requirements when negotiating the contracts, the  agreements with the services are not binding. The performance targets  that have been negotiated on the sustainment contracts are generally  lower than those required by the services. For instance, the Marine Corps  established a minimum performance target for non-deployed units of 60  percent FMC aircraft for 2017, but the minimum target established in the  contract for that same metric was 14 percent. Similarly, the Air Force  identified a minimum performance target for non-deployed units of 65  percent AVA, but the minimum target established in the contract for that  same metric was 52 percent. Program officials said that the costs of  meeting the services’ performance requirements would be too high given  the current supply chain challenges across the fleet. Figure 6 shows the  differences between the performance targets required by the Marine  Corps and those that DOD was able to negotiate under the pilot  performance-based agreement in 2017.

DOD may not be using the appropriate metrics under the pilot  performance-based agreements to achieve desired outcomes. DOD  guidance states that optimal performance-based contracts use objective,  measurable, and manageable metrics that accurately assess the support  provider’s performance against the delivery of targeted warfighter  outcomes. It also defines ideal metrics as those that are, among other  things: (1) reflective of processes over which the contractor has control,  and (2) able to motivate desired behavior. We found the following:

The contractor does not have full control over the performance  outcomes for which it is paid: The system-level metrics that the  prime contractor is being assessed against are not fully reflective of  processes over which the contractor has control, because actions that  the F-35 squadrons take when maintaining or operating the aircraft  affect the metric outcomes being measured. For example, a  contractor official at one site that we visited cited an instance when a  military service maintainer towed an aircraft into a hangar and broke a  surface panel, resulting in the aircraft not being able to fly for 60 days  because there was no surface panel replacement available in the  supply chain. Thus, the contractor could be held accountable for a  lack of performance that the customer created. Conversely, to keep  aircraft flying, military service maintainers have taken actions that  mask contractor failures to provide support—for example,  cannibalizing parts from other aircraft at rates significantly higher than  DOD intends, based on data provided by the prime contractor and  shown in figure 7. Because the contractor does not fully control the  outcomes for which it is being assessed, prime contractor and military  service officials said that contentious negotiations have occurred at  times about how to assign responsibility for performance. This  ultimately makes it difficult for DOD to hold the contractor  accountable. Further, one of the three system-level metrics—Mission  Effectiveness—is assessed by pilots subjectively after each flight.  Some pilots and service officials whom we spoke to said that different  pilots may make differing determinations about the effectiveness of  the mission, which could affect the measured performance outcomes.

DOD has established performance review groups to review and reconcile  data in instances where the contractor does not believe that it should be  held responsible for certain metric outcomes, but this process requires  both DOD and the contractor to make subjective determinations about the  root causes of particular performance failures in order to determine  whether the contractor or the military services are to blame. Figure 8  shows how this reconciliation process can result in adjustments to the  measured performance data when assessing the level of support  provided by the contractor. Under the pilot performance-based  agreements, the reconciled data points serve as the basis for calculating  contractor incentive fees.

Additionally, DOD is working to implement agreements that define lower- level metrics for which the military services will be held responsible, such  as defining how long it should take for maintainers to conduct  maintenance, but these agreements have not yet been fully implemented.  Ultimately, service officials told us that the complexity of these  adjudication efforts indicates that DOD may not be holding the contractor  accountable for the appropriate metrics.

Current metrics may not motivate the desired behaviors from all  stakeholders. The current metrics may not consistently motivate the  necessary behaviors from all stakeholders to either achieve desired  warfighter outcomes or meet the current metrics on contract. For  example, DOD has established AVA as its primary metric, and it  provides greater incentive fees to the contractor for meeting the AVA  targets as compared with the other two metrics. However, Marine  Corps and Navy officials told us that FMC aircraft are more important  for operational deployments, as they represent aircraft that are ready  for war. DOD’s performance-based logistics guidance states that it is  important to exercise caution when selecting a combination of metrics,  to ensure that they do not create undesirable conflicts. The  achievement of the AVA and FMC metrics may at times be in conflict  with one another. For instance, according to contractor and program  officials, an aircraft is still considered to be available if its low  observable—or stealth—systems are not working, but for it to be  considered a fully-mission capable aircraft, a military service would  have to ground the aircraft for several days to repair the low  observable system. Contractor officials have also expressed concern  that the metrics they are being paid for may not be as important to the  services as other factors—such as achievement of flying hours or the  ability to train pilots—and that this could affect whether the services  will take all necessary actions to meet the targets for which the  contractor is paid. Officials from a training unit we visited said that  they were focused on training pilots, not on achieving the metric  targets identified in the contract. This unit was able to exceed its  required flight hours to support pilot training in April 2017, even  though the performance of its aircraft fell well below desired Marine  Corps performance levels for AVA and FMC.

Program office and contractor officials noted that pilot performance-based  agreements were put in place to gather lessons learned and ensure that  DOD has the appropriate metrics before entering into 5-year, fixed-price  contracts. However, contractor officials said that the performance review  process does not include a step to review how the metrics are driving  behaviors or to determine whether DOD has the appropriate metrics in  place, and they suggested that a more robust effort to consider lessons  learned from the pilot agreements is needed. Service officials have  suggested that incentivizing simpler metrics that focus on individual  aspects of F-35 sustainment for which the contractor has more control— such as supply chain responsiveness or depot-level repair—instead of  system-level performance metrics may be more appropriate. Without  reexamining the metrics to ensure that they are objectively measurable,  reflective of processes that the contractor can control, and able to  motivate desired behaviors, DOD may not be well positioned to  accurately assess contractor performance or achieve optimal outcomes  across future performance-based sustainment contracts that will likely  cost tens of billions of dollars.

DOD Does Not Yet Have Full Information on F-35 Sustainment Costs or Technical Aircraft Characteristics

DOD does not yet have full information on F-35 sustainment costs or  technical characteristics such as reliability and maintainability, and this  could pose risks to its ability to effectively negotiate 5-year, fixed-price  performance-based contracts with the prime contractor by 2020. Although  DOD has fielded more than 250 aircraft, the aircraft system remains  immature. DOD has established a target for system maturity of 200,000  total flight hours, with minimum flight hours for each variant. DOD  reached 100,000 total F-35 flight hours in July 2017, and it does not  expect to reach its maturity targets for all variants until fiscal year 2024.

Specifically, we found that DOD does not have full visibility into the actual  costs for some key sustainment requirements that are considered cost- drivers within the program, such as the actual costs of parts and repairs.  Given the immaturity of the system, DOD has relied on projected parts  reliability and pricing to formulate cost estimates, but officials said that  actual costs are needed to improve both their confidence in the estimates  and their understanding of how cost is related to performance. There is  potential for the actual costs of sustainment requirements to change  significantly from initial projections. For instance, the costs of initial spare  parts over the life cycle increased by $447 million in the program’s  estimate from the 2014 estimate to the 2015 estimate, due largely to  increases in unit prices from those initially projected. According to  program officials, their understanding of actual costs is limited in part  because of the immaturity of the system. Program officials said that they  are taking steps to obtain more actual cost information as the aircraft  matures, and to determine how much repairs should cost, in order to  better position themselves for contract negotiations. However, in addition  to system immaturity, program officials said that they are experiencing  challenges in obtaining important details about existing cost data needed  to inform their cost models from the contractor, such as the costs of the  individual parts and repairs that the contractor purchases from its  suppliers.

Further, we found that there are a number of technical aspects of the  aircraft that are immature or uncertain. While the F-35 is meeting  expectations for some measurements of reliability and maintainability,  other measurements are still lagging behind operational requirements.  For example, aircraft are experiencing failures that result in the loss of a  capability to perform a mission-essential function at more than twice the  rate expected across all variants. Mean repair times for critical  components that fail are also more than twice as long as the operational  requirements dictate. Additionally, the significant software releases  required to complete F-35 system development—referred to as Block  3F—are planned to be tested and released in 2017. However in April  2017 we reported that the program’s schedule for completion of Block 3F  and associated testing would likely be delayed due in part to software  issues and system instability. Additionally, as of June 2017, the DOD  Office of the Director for Operational Test and Evaluation predicted that  required initial operational test and evaluation for Block 3F would likely  not begin until late 2018 or early 2019. According to operational testing  officials, such software releases can lead to different reliability and  maintainability issues than were previously known, as the aircraft  becomes capable of flying at higher speeds and altitudes. According to  these officials, there would be inherent risk in signing a fixed-price,  performance-based contract before the reliability and maintainability data  for Block 3F are more fully known, as those data will influence how much  aircraft performance should cost at maturity.

DOD guidance states that in order for performance-based arrangements  to be effective, the government must clearly understand program  requirements, costs, and technical characteristics; and that systems  should achieve a level of maturity and design stability. Program officials  said they believe that DOD can gain sufficient knowledge of the costs and  technical characteristics of the aircraft prior to 2020, and that they will  seek to write options into the multi-year, performance-based contract if  there are still risks that need to be mitigated. However, program officials  said that the program office has not established criteria addressing the  extent of the cost and technical data that it will require prior to entering  into the planned agreements. While the program still has a few years until  that date, program officials said that the process to develop this contract  is expected to begin in late 2017. In April 2017 we reported on the risks of  moving forward with additional F-35 program development before DOD  has a full understanding of the aircraft’s baseline Block 3F capabilities,  specifically citing difficulties in presenting a sound business case for  soliciting contractor proposals without such knowledge. The program  office could face similar challenges preparing for a fixed-price,  performance-based sustainment contract amid existing uncertainty.

Without a full understanding of F-35 costs and technical characteristics at  maturity, DOD may not be well positioned to accurately determine how  much fleet performance should cost over a 5-year, fixed-price,  performance-based contract, and thus may be at risk of overpaying the  contractor while not receiving the expected level of sustainment support.

DOD Has Taken Some Actions Aimed at Reducing F-35 Sustainment Costs, but These Costs Continue to Rise and Are Not Fully Transparent to the Military Services

DOD Has Undertaken Some Initiatives to Reduce Rising F-35 Sustainment Costs but Has Not Established Affordability Constraints Based on the Military Services’ Budgets

DOD has taken some actions to try to reduce estimated sustainment  costs for F-35 operating and support, which, according to the program  office’s fiscal year 2016 cost estimate, are projected to cost $1.06 trillion  in then-year dollars (see figure 9 below). For example, the program  office has established a Cost War Room to identify and implement cost- reduction initiatives with the goal of reducing the program office’s 2012  operating and support cost estimate by 30 percent by 2022. These  initiatives include updating assumptions about fuel usage, among others.  According to program documentation, such efforts are projected to result  in a cost avoidance of $60.7 billion. The program office also has an effort  targeted at improving reliability and maintainability of F-35 components.  As of May 2017, the program office had completed 38 improvement projects that are expected to result in $1.7 billion in operating and support  cost avoidance. However, at the same time, the projected operating and  support costs estimated by the program office have increased from fiscal  year 2012 to fiscal year 2016, due to an increase in projected flying  hours, an extension of the aircraft’s life cycle from 56 to 60 years, and  refinements to the cost models, among other factors. Figure 9 shows the  increase to the program office’s life cycle operating and support cost  estimate since fiscal year 2012.

In addition, DOD has not established affordability constraints for the F-35  program that are linked to the military services’ budgets, as we  recommended in September 2014. In our prior work, we found that the  program’s affordability targets may not be reflective of what the services  can actually afford because it did not use the military services’ budgets to  establish the targets. At that time, the annual F-35 operating and support  costs were estimated to be considerably higher than the combined annual  costs of several legacy aircraft, and according to DOD officials, the  sustainment strategy was not affordable. We recommended that DOD  establish affordability targets linked to the services’ budgets, because  without such targets DOD cannot be sure whether the cost savings they  are pursuing will lead to an affordable sustainment strategy. The  department concurred with this recommendation but has not taken  specific action on it at the program level. We made this a priority  recommendation for DOD in July 2017. The Senate Armed Services  Committee also directed DOD to provide it with a plan for improving the  transparency and affordability of the F-35 sustainment strategy, to include  identifying affordability constraints linked to, and informed by, the military  services’ budgets. The Marine Corps has recently taken steps to  develop budget-based affordability targets for their portion of F-35  sustainment costs. The Marine Corps identified the need to reduce  steady-state sustainment costs per aircraft by at least 20 percent through  cost modeling efforts and budget analysis, and Marine Corps officials said  they believe that such a reduction would make the program affordable for  the Marine Corps. Marine Corps officials stated that to achieve such  reductions, they are exploring options to reduce costs—such as  transitioning maintenance tasks from depots to operational units, and  revising sustainment support personnel requirements—in coordination  with the program office and prime contractor. The program office could  use this service target to inform the establishment of program-level  affordability constraints.

As previously discussed, the program is experiencing sustainment  challenges due in part to some requirements not being fully funded, and  this could present a continued risk going forward if sustainment for the F- 35 is not affordable within the services’ budgets. Program officials also  told us that if the services cannot fully fund sustainment requirements,  DOD will have to prioritize funding and defer requirements to later years.  However, given the F-35’s global sustainment strategy of providing  support across the military services and the international partners through  shared pools of funding, a single customer that cannot fully fund  requirements may affect the ability of DOD and the contractor to provide  adequate sustainment support across the global F-35 fleet.

Actual F-35 Sustainment Costs Are Not Fully Transparent to the Military Services

F-35 actual sustainment costs that are being charged by the program  office to the military services, as well as the capabilities associated with  those costs, are not fully transparent to the services. In addition to  estimating projected costs for F-35 sustainment over the aircraft’s life  cycle as described above, the program office also calculates the actual F- 35 sustainment costs that will be charged to the military services on an  annual basis. To determine these actual sustainment costs, the military  services first submit their F-35 sustainment capability requirements to the  program office for approval. The program office approves requirements  as a basis for its annual life-cycle operating and support cost estimate,  which is used to provide each of the military services with an estimate for  their respective portion of F-35 sustainment costs to support the services’  budget planning process. The program office then negotiates with the  prime contractor the level of support the contractor will provide to meet  service sustainment requirements. It is at this point that the program  office informs the services of the actual costs that they will be charged for  contracted sustainment. According to program officials, the contracted  level of support may not include all the requirements initially submitted by  the military services for a given contract period, and the associated costs  of the contract services may not align with initial estimates given to the  military services, because support is negotiated between the program  office and the prime contractor.

Air Force, Navy, and Marine Corps officials told us that they do not fully  understand how the actual costs that they are charged by the program  office for F-35 sustainment are clearly linked to the capabilities that they  are receiving. They cited issues related to unexplained cost increases,  difficulty in tracking their requirements to the contracts, and concerns  about how to track their dollars to shared pools of sustainment assets, as  discussed in detail below.

Unexplained or unexpected growth in actual sustainment costs:  Service headquarters officials cited concerns about unexplained or  unexpected growth in sustainment costs, particularly between the cost  estimate that they were quoted for budget planning purposes and  what they are actually charged by the program office in the budget  execution year. For example, according to program documentation,  the Marine Corps was initially given a funding requirement for fiscal  year 2017 sustainment support of $293 million, which then increased  to $364 million in the execution year, largely due to increases in  contractor personnel costs. Marine Corps officials said that the  reasons behind this growth in personnel costs were not clearly  substantiated for the Marine Corps by the program office. In order to  afford these increased costs for sustainment support, Marine Corps  officials said that the Marine Corps had to reduce its planned flying  hours. In another instance, documents provided by the Navy show  that the program office increased the cost of the Navy’s and Marine  Corps’ combined spare parts requirements for fiscal year 2017 from  an original estimate of $261 million to $402 million over the course of  the execution year. In addition, service officials told us that they  sometimes become aware of the growth in sustainment costs late in  the services’ budgeting process, making it difficult for them to find  additional funding for such changes.

Tracking requirements to negotiated contract services and  costs: Officials from two of the services told us that they have had  difficulty in tracking their respective services’ requirements to the  costs being charged by the program and the capabilities that are  negotiated on the contract. For instance, Air Force officials stated that  the Air Force specified a desired performance level for AVA of 65  percent to the program office as a minimum target for its squadrons,  but ultimately the program office contracted for a target of 52 percent.  Air Force officials said they were not aware of this change until after  the contract was negotiated. Similarly, Navy officials also told us that  the program office does not notify the Navy of changes from the  estimated costs to the actual contract costs or the requirements that  are included during negotiations for sustainment contracts, even when  the requirements differ from what the Navy intended. As a result,  officials said that the services often have limited visibility into the  support that the contractor will provide along with the actual costs for  which the services are responsible, until after the contract is signed.

Shared pools of F-35 sustainment assets: These transparency  concerns are complicated by the fact that the services are paying into  shared pools for F-35 sustainment, and the costs they are being  charged for some requirements—such as for spare parts—cannot be  directly tracked to an item that the services own or support that is  specifically provided to an individual service. Service officials said that  the funds they have contributed to the shared sustainment support  have not resulted in the expected sustainment support. Specifically,  Air Force officials questioned why key performance points in the  program—such as depot repair capabilities and supply availability— are lagging by several years in some instances, and said that they  need better accounting from the program office on how the money the  Air Force has contributed to the program has been spent, and why  those funds have not resulted in improved performance. Furthermore,  Air Force officials raised questions about whether all program  participants are paying for their required shares of F-35 sustainment  costs, and said that they have not been able to obtain such  information from the program office.

This lack of transparency is due in part to insufficient communication  between the program office and the services, particularly as requirements  and costs change. Program officials have acknowledged that the program  office has not always provided the services with the level of detail and  clarity around costs that the services would like, but said that recently the  program has been more focused on communicating with the military  services. Program officials also told us that the services are free to  contact the program office should they have any concerns regarding F-35  sustainment costs and how they are shared. However, given the  consistent concerns expressed to us across the services, it appears that  this level of dialogue has not been adequate to facilitate the services’  understanding of sustainment costs. Two of the services have requested  organizations external to the F-35 program to conduct reviews of the  program to better understand their respective portions of F-35  sustainment costs and, in some cases, identify potential opportunities for  cost savings. While these studies will likely provide valuable information  to the services and the program office, they also add costs to an already  expensive weapon system. For example, according to program officials,  the contract for the study requested by the Marine Corps has cost the  program office at least $2.7 million. Further, reliance on one-time studies  by external organizations to help program participants understand their F- 35 sustainment costs and associated capabilities is not a practical  substitute for the effective communication needed in a program of this  magnitude.

F-35 program guidance has emphasized the need to ensure that costs  are transparent to stakeholders. Further, our prior work examining  programs with multiple governmental customers found that when  customers understand how costs and underlying assumptions are  determined, they can better anticipate potential changes to those  assumptions, identify their effects on costs, and incorporate that  information into their budget plans. Without better communication on the  relationship between the costs and the associated capabilities delivered  for F-35 sustainment support, the military services may not be able to  appropriately plan for sustainment costs over the life cycle of the F-35 or  to make affordability trade-offs between requirements, as they try to  prioritize funding within their budgets.

DOD’s F-35 program is at a critical juncture. With aircraft development  nearing completion within the next few years, DOD must now shift its  attention and resources to sustaining the growing F-35 fleet. While  production accelerates, DOD’s reactive approach to planning for and  funding the capabilities needed to sustain the F-35 has resulted in  significant readiness challenges—including multi-year delays in  establishing repair capabilities and spare parts shortages. There is little  doubt that the F-35 brings unique capabilities to the U.S. military, but  without revising sustainment plans to include the key requirements and  decision points needed to fully implement the F-35 sustainment strategy,  and without aligned funding plans to meet those requirements, DOD is at  risk of being unable to leverage the capabilities of the aircraft it has  recently purchased. Furthermore, until it improves its plans, DOD faces a  larger uncertainty as to whether it can successfully sustain a rapidly  expanding fleet.

DOD’s plan to enter into multi-year, performance-based contracts with the  prime contractor has the potential to produce cost savings and other  benefits. However, important lessons are emerging from its pilot  agreements with the contractor that are intended to inform the upcoming  multi-year contract negotiations. To date, DOD has not achieved the  desired aircraft performance under the pilot agreements, but it continues  to move quickly toward negotiating longer-term contracts—which are  likely to cost tens of billions of dollars—by 2020. Without examining  whether it has the appropriate metrics to incentivize the contractor or a  sufficient understanding of the actual costs and technical characteristics  of the aircraft before entering into multi-year, performance-based  contracts, DOD could find itself overpaying for sustainment support that is  not sufficient to meet warfighter requirements.

Finally, on a broader level, DOD’s projected costs to sustain the F-35 fleet  over its life cycle have risen since 2012 despite the department’s  concerted efforts to reduce costs. Already the most expensive weapon  system in DOD’s history, these rising costs are particularly concerning  because the military services do not fully understand what they are  paying for. This puts them in a precarious position as they consider  critical trade-offs that might make F-35 sustainment more affordable.  Without improving communication with the services to help them better  understand how the sustainment costs they are being charged relate to  the capabilities that they receive, the services may not be able to  effectively budget for the F-35 over the long term.

We are making the following four recommendations to DOD.

The Under Secretary of Defense for Acquisition, Technology, and  Logistics, in coordination with the F-35 Program Executive Officer,  should revise sustainment plans to ensure that they include the key  requirements and decision points needed to fully implement the F-35  sustainment strategy and aligned funding plans to meet those  requirements. (Recommendation 1)

The Under Secretary of Defense for Acquisition, Technology, and  Logistics, in coordination with the F-35 Program Executive Officer,  should re-examine the metrics that it will use to hold the contractor  accountable under the fixed-price, performance-based contracts to  ensure that such metrics are objectively measurable, are fully  reflective of processes over which the contractor has control, and  drive desired behaviors by all stakeholders. (Recommendation 2)

The Under Secretary of Defense for Acquisition, Technology, and  Logistics, in coordination with the F-35 Program Executive Officer,  should, prior to entering into multi-year, fixed-price, performance- based contracts, ensure that DOD has sufficient knowledge of the  actual costs of sustainment and technical characteristics of the aircraft  after baseline development is complete and the system reaches  maturity. (Recommendation 3)

The Under Secretary of Defense for Acquisition, Technology, and  Logistics, in coordination with the F-35 Program Executive Officer,  should take steps to improve communication with the services and  provide more information about how the F-35 sustainment costs they  are being charged relate to the capabilities received.  (Recommendation 4)

We provided a draft of this report to DOD for review and comment. In its  written comments, reproduced in appendix II, DOD concurred with our  recommendations and identified actions that it would take in response.

We are sending copies of this report to appropriate congressional  committees; the Secretary of Defense; the Under Secretary of Defense  for Acquisition, Technology, and Logistics; the F-35 Program Executive  Officer; the Secretaries of the Air Force and Navy; and the Commandant  of the Marine Corps. In addition, the report is available at no charge on  the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-5431 or russellc@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. Staff members making key contributions to this report  are listed in appendix III.

For each of our objectives, we reviewed relevant sustainment plans,  guidance, and program documentation, and collected information by  interviewing officials from the Office of the Assistant Secretary of Defense  (Logistics and Materiel Readiness), the F-35 Joint Program Office, the  U.S. Air Force, the U.S. Navy, the U.S. Marine Corps, and the prime  contractor, Lockheed Martin. To interview officials and observe F-35  operations, maintenance, and training, we conducted visits to two F-35  operational locations—Hill Air Force Base, Utah, and Marine Corps Air  Station Iwakuni, Japan; two F-35 training locations—Eglin Air Force Base,  Florida, which also includes a Navy F-35 training squadron, and Marine  Corps Air Station Beaufort, South Carolina; and two F-35 maintenance  depots—Ogden Air Logistics Complex, Utah, and Fleet Readiness Center  Southeast, Florida. A full listing of organizations with whom we met is  provided later in this appendix.

We also gathered various data related to F-35 sustainment, such as  supply chain and repair data and aircraft performance data. To determine  the reliability of these data, we collected information on how the data  were collected, managed, and used through a questionnaire and  interviews with cognizant Department of Defense (DOD) officials and the  prime contractor. In our assessment, we identified some limitations in the  way that certain data are collected and reported, such as data related to  aircraft performance (Air Vehicle Availability, Full Mission Capable, and  Mission Effectiveness metrics), data related to aircraft that are not  mission capable due to supply issues, and parts cannibalization rates that  could potentially result in inaccuracies. However, these data come from  the program’s data systems of record, and are the same data used by the  program office and prime contractor to monitor the health of the supply  chain and assess aircraft performance against contract requirements and  program objectives. As such, they are the best source of data available to  provide information on the progress and challenges within the program.  We determined that these data presented in our findings are sufficiently  reliable for the way in which we report them. Specifically, the parts  cannibalization rates are consistent with the trends observed across other  key data elements within the program, and with the testimonial evidence  provided to us by the units with whom we met during our review, and are  sufficiently reliable to report as a data trend relative to program  objectives. All other performance data presented in our report are  sufficiently reliable to present as specific data points, in order to describe  the status of sustainment requirements and measured aircraft  performance across key metrics as reported by the prime contractor and  DOD.

To assess the status of DOD’s efforts to sustain the F-35 fleet and any  challenges it has faced, we reviewed DOD and contractor plans,  briefings, and schedules to determine the current status of key  requirements and decision points necessary to establish F-35  sustainment capabilities, such as depot and other maintenance  capabilities, the supply chain, technical data, and development of key  software systems, among other things, and spoke with cognizant officials  about these issues. We also compared actual data obtained about F-35  repair and supply chain capabilities with DOD’s objectives for these  capabilities to identify areas of challenge for the program. Specifically, we  obtained data on aircraft that were not mission capable due to supply  issues from January 2017 through August 7, 2017 and average repair  times as of May 2017, in order to provide the most recently available  information about the health of the supply chain. As discussed above, we  determined that these data are sufficiently reliable to present as specific  data points. In addition, we identified key acquisition program  management practices that can improve program outcomes if  implemented—such as clearly establishing well-defined requirements,  developing realistic cost estimates and schedules, and securing stable  funding that matches resources to requirements—and assessed DOD’s  sustainment planning efforts against these criteria.

To assess the extent to which DOD is positioned to enter into multi-year  performance-based F-35 sustainment contracts, we reviewed  documentation related to DOD’s pilot—or trial— performance-based  agreements for F-35 sustainment, such as sustainment contracts,  readiness data provided by the military services, metric taxonomies, and  agreements between the program office and the military services that  identify the services’ desired performance targets. We also obtained  aircraft performance data from the Sustainment Performance  Management System for the 2016 pilot performance-based agreement  (March 2016 – December 2016) and the 2017 pilot performance-based  agreement (March 2017 – June 2017) to the extent available at the time  we completed our audit work. As discussed above, we determined that  these data are sufficiently reliable to present as specific data points.  These time periods are the only time periods for which the program office  has assessed contractor performance under these pilot arrangements.  We also reviewed performance-management guidance and processes  and interviewed officials to determine how performance data are being  collected and assessed. In addition, we reviewed aircraft maturity,  reliability, and maintainability data, and documentation related to cost- visibility issues, and we spoke with cognizant officials about these issues  to determine DOD’s level of understanding of the costs and technical  characteristics that will affect future sustainment support. In addition, we  obtained cannibalization data from March 2016 to March 2017 in order to  review and report recent trends in cannibalization rates over a time in  which the program has introduced a significant amount of aircraft to the  fleet. As discussed above, we determined that these data are sufficiently  reliable to present as trend data relative to the program objective. Further,  we reviewed DOD guidance and best practices related to performance- based agreements to identify attributes of ideal performance metrics and  effective performance-based agreements. We then compared these  attributes with the information described above to determine whether  DOD has the appropriate metrics to achieve desired outcomes and the  necessary information to effectively negotiate multi-year, fixed-price,  performance-based contracts with the prime contractor by 2020, as  planned.

To assess the progress, if any, DOD has made toward reducing F-35  sustainment costs, and the extent to which costs are transparent to the  military services, we reviewed F-35 Joint Program Office sustainment- cost estimates from fiscal year 2012 to fiscal year 2016 in order to identify  changes to the estimate since the program’s sustainment cost baseline  was established in 2012; documentation related to program office and  service cost-reduction efforts; sustainment contracts; F-35 cost-sharing  rules; and budget documentation from both the program office and the  military services. The fiscal year 2016 sustainment-cost estimate is the  most current cost estimate conducted by the program office. In addition,  we interviewed cognizant officials from the F-35 Joint Program Office and  military services to discuss how the program office informs the military  services of F-35 sustainment costs, and the degree to which the services  understand these costs and the sustainment capabilities provided for  those costs. We compared this information with program guidance and  with key operating principles for programs that involve multiple  governmental customers identified in our prior work in order to assess  the transparency of F-35 sustainment costs for the military services.

We conducted this performance audit from October 2016 to October 2017  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

We met with officials from the following Department of Defense (DOD)  and contractor organizations during our review. We selected these  organizations based on their oversight, planning, and execution roles in  support of F-35 sustainment and operations.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Alissa Czyz (Assistant Director),  Vincent Buquicchio, Kasea Hamar, Jeff Hubbard, Amie Lesser, Sean  Manzano, Carol Petersen, Clarice Ransom, Michael Silver, Maria  Staunton, Cheryl Weissman, and Delia Zee made key contributions to this  report.

F-35 Joint Strike Fighter: DOD’s Proposed Follow-on Modernization  Acquisition Strategy Reflects an Incremental Approach Although Plans  Are Not Yet Finalized. GAO-17-690R. Washington, D.C.: August 8, 2017.

F-35 Joint Strike Fighter: DOD Needs to Complete Developmental  Testing Before Making Significant New Investments. GAO-17-351.  Washington, D.C.: April 24, 2017.

F-35 Joint Strike Fighter: Continued Oversight Needed as Program Plans  to Begin Development of New Capabilities. GAO-16-390. Washington,  D.C.: April 14, 2016.

F-35 Sustainment: DOD Needs a Plan to Address Risks Related to Its  Central Logistics System. GAO-16-439. Washington, D.C.: April 14, 2016.

F-35 Joint Strike Fighter: Preliminary Observations on Program Progress.  GAO-16-489T. Washington, D.C.: March 23, 2016.

F-35 Joint Strike Fighter: Assessment Needed to Address Affordability  Challenges. GAO-15-364. Washington, D.C.: April 14, 2015.

F-35 Sustainment: Need for Affordable Strategy, Greater Attention to  Risks, and Improved Cost Estimates. GAO-14-778. Washington, D.C.:  September 23, 2014.

F-35 Joint Strike Fighter: Slower Than Expected Progress in Software  Testing May Limit Initial Warfighting Capabilities. GAO-14-468T.  Washington, D.C.: March 26, 2014.

F-35 Joint Strike Fighter: Problems Completing Software Testing May  Hinder Delivery of Expected Warfighting Capabilities. GAO-14-322.  Washington, D.C.: March 24, 2014.

F-35 Joint Strike Fighter: Restructuring Has Improved the Program, but  Affordability Challenges and Other Risks Remain. GAO-13-690T.  Washington, D.C.: June 19, 2013.

F-35 Joint Strike Fighter: Program Has Improved in Some Areas, but  Affordability Challenges and Other Risks Remain. GAO-13-500T.  Washington, D.C.: April 17, 2013.

F-35 Joint Strike Fighter: Current Outlook Is Improved, but Long-Term  Affordability Is a Major Concern. GAO-13-309. Washington, D.C.: March  11, 2013.

Joint Strike Fighter: DOD Actions Needed to Further Enhance  Restructuring and Address Affordability Risks. GAO-12-437. Washington,  D.C.: June 14, 2012.

Joint Strike Fighter: Restructuring Added Resources and Reduced Risk,  but Concurrency Is Still a Major Concern. GAO-12-525T. Washington,  D.C.: March 20, 2012.

Joint Strike Fighter: Implications of Program Restructuring and Other  Recent Developments on Key Aspects of DOD’s Prior Alternate Engine  Analyses. GAO-11-903R. Washington, D.C.: September 14, 2011.

Joint Strike Fighter: Restructuring Places Program on Firmer Footing, but  Progress Is Still Lagging. GAO-11-677T. Washington, D.C.: May 19,  2011.

Joint Strike Fighter: Restructuring Places Program on Firmer Footing, but  Progress Still Lags. GAO-11-325. Washington, D.C.: April 7, 2011.

Joint Strike Fighter: Restructuring Should Improve Outcomes, but  Progress Is Still Lagging Overall. GAO-11-450T. Washington, D.C.:  March 15, 2011.