Defense Business Systems: DOD Needs to Continue Improving Guidance and Plans for Effectively Managing Investments

DOD spends billions of dollars each year on systems that support its key business areas, such as personnel and logistics. For fiscal year 2018, DOD reported that these business system investments are expected to cost about $8.7 billion. The NDAA for Fiscal Year 2016 requires DOD to perform activities aimed at ensuring that business system investments are managed efficiently and effectively, to include taking steps to limit their complexity and cost.

The NDAA also includes a provision for GAO to report every 2 years on the extent to which DOD is complying with the act's provisions on business systems. For this report, GAO assessed, among other things, the department's guidance for managing defense business system investments and its business and IT enterprise architectures (i.e., descriptions of DOD's current and future business and IT environments and plans for transitioning to future environments). To do so, GAO compared the department's system certification guidance and architectures to the act's requirements. GAO also interviewed cognizant DOD officials.

The Department of Defense (DOD) has made progress in complying with most legislative provisions for managing its defense business systems, but additional actions are needed. For example, the National Defense Authorization Act (NDAA) for Fiscal Year 2016 required DOD and the military departments to issue guidance to address five requirements for reviewing and certifying the department's business systems. While DOD has issued guidance addressing all of these requirements, as of February 2018, the military departments had shown mixed progress.

● Fully addressed: The department provided evidence that it fully addressed this requirement.

◐ Partially addressed: The department provided evidence that it addressed some, but not all, portions of this requirement.

◌ Not addressed: The department did not provide any evidence that it addressed this requirement.

Source: GAO analysis of Department of Defense documentation. | GAO-18-130

The military departments' officials described plans to address the gaps in their guidance; however, none had defined when planned actions are to be completed. Without guidance that addresses all five requirements, the military departments risk developing systems that, among other things, are overly complex and costly to maintain.

DOD has efforts underway to improve its business enterprise architecture, but its information technology (IT) architecture is not complete. Specifically, DOD's business architecture includes content called for by the act. However, efforts to improve this architecture to enable the department to better achieve outcomes described by the act, such as routinely producing reliable business and financial information for management, continue to be in progress. In addition, DOD is updating its IT enterprise architecture, which describes, among other things, the department's computing infrastructure. However, the architecture lacks a road map for improving the department's IT and computing infrastructure for each of the major business processes. Moreover, the business and IT enterprise architectures have yet to be integrated, and DOD has not established a time frame for when it intends to do so. As a result, DOD lacks assurance that its IT infrastructure will support the department's business priorities and related business strategies.

GAO is making six recommendations, including that DOD and the military departments establish time frames for, and issue, required guidance; and that DOD develop a complete IT architecture and integrate its business and IT architectures. DOD concurred with three and partially concurred with three recommendations. GAO continues to believe all of the recommendations are warranted as discussed in this report.

The Department of Defense (DOD) spends billions of dollars each year to  acquire modernized systems that are fundamental to achieving its  business transformation goals, including systems that address key areas  such as personnel, financial management, health care, and logistics.  However, while DOD’s capacity for modernizing its business systems has  improved over time, significant challenges remain. We have designated  the department’s business systems modernization program as high risk  since 1995.

In addition, we have reported that significant potential exists for identifying  and avoiding costs associated with duplicative functionality across the  department’s portfolio of defense business systems. Since 1995, we  have made a series of recommendations aimed at strengthening the  department’s institutional approach to business systems modernization  and reducing the risks associated with its key investments. Congress  included provisions consistent with our recommendations in the  department’s annual authorizing legislation for fiscal year 2005, and in  subsequent amendments. Most recently, the National Defense  Authorization Act for Fiscal Year 2016 (NDAA), which was enacted in  November 2015, included provisions that revised U.S. Code Title 10,  Section 2222. These provisions required DOD to perform certain activities  aimed at ensuring that its business system investments are managed  efficiently and effectively.

The NDAA included provisions for GAO to assess, biennially, the  department’s compliance with the provisions in the act. Accordingly, our  objective for this review was to assess the extent to which actions taken  by DOD comply with the act’s provisions to:  1.  establish guidance for effectively managing its defense business  2.  develop and maintain a defense business enterprise architecture and  information technology (IT) enterprise architecture, in accordance with  relevant laws and Office of Management and Budget (OMB) policies  and guidance;  3.  use the Defense Business Council to provide advice to the Secretary  on developing the business enterprise architecture, reengineering the  department’s business processes, developing and deploying  business systems, and developing requirements for business  systems; and  4.  ensure that covered business systems are reviewed and certified in  accordance with the act.

To assess the extent to which actions taken by DOD comply with the act’s  provisions to establish guidance for effectively managing its defense  business system investments, we obtained and reviewed the  department’s guidance for managing business system investments. We  then compared the guidance with the act’s requirements that the  guidance include six specific elements related to managing business  system investments.

We also assessed DOD’s guidance for managing business system  investments relative to the act’s certification requirements. Specifically,  we compared the DOD Deputy Chief Management Officer’s (DCMO)  April 2017 certification guidance with the act’s certification requirements.  In addition, we compared the guidance established by the Departments of  the Air Force, the Army, and the Navy for certifying their business  systems with the act’s certification requirements.

To assess the extent to which actions taken by DOD comply with the act’s  provisions to develop and maintain a defense business enterprise  architecture in accordance with relevant laws and OMB policies and  guidance, we reviewed the department’s business enterprise architecture,  version 10, which was released in February 2013. We also reviewed data  in the department’s Integrated Business Framework—Data Alignment  Portal system and DOD IT Portfolio Repository—which the department  defined as part of its business enterprise architecture. We compared the  architecture with the act’s requirements and OMB policy and guidance  for the architecture’s content. We also reviewed the department’s January  2017 plan for improving the architecture and changes in the plan since it  was issued.

To assess the extent to which actions taken by DOD comply with the act’s  provisions regarding DOD’s IT enterprise architecture, we first determined  the extent to which the IT architecture addressed relevant laws and OMB policies and guidance. In particular, we reviewed the department’s  information enterprise architecture, which the department considers its IT  enterprise architecture. The information enterprise architecture was  released in August 2012. We compared its content with the act’s  requirements and OMB policy and guidance for enterprise architectures  to determine if it includes a plan for improving the IT and computing  infrastructure for each of the department’s major business processes, as  required by the act.

We also assessed the extent to which the department has integrated its  business and IT enterprise architectures. To do this, we obtained and  reviewed documentation about the architectures, including the  department’s business enterprise architecture compliance guide and  DOD Directive 8000.01: Management of the Department of Defense  Information Enterprise. Further, we reviewed business and IT  architecture working group meeting documentation that discussed plans  to update each architecture.

To assess the extent to which actions taken by DOD comply with the act’s  provisions to use the Defense Business Council to provide advice to the  Secretary on developing the business enterprise architecture,  reengineering the department’s business processes, developing and  deploying business systems, and developing requirements for business  systems, we reviewed Defense Business Council meeting documentation,  including meeting summaries and presentations, to identify instances in  which the Council discussed the topics. We also discussed with officials  in the Office of the DCMO how the Council provides advice on these  topics to the Secretary of Defense.

To assess the extent to which actions taken by DOD comply with the act’s  provisions to ensure that covered business systems are reviewed and  certified in accordance with the act, we reviewed a nongeneralizable  sample of business systems from DOD’s two categories of covered  defense business systems that require certification. To select the  sample, we considered Category I systems, which were systems that  were expected to have a total amount of budget authority of more than  $250 million over the period of the current future-years defense  program, and Category II systems, which were systems that were  expected to have a total amount of budget authority of between $50  million and $250 million over the period of the future-years defense  program.

We further categorized the Category II systems into four groups—those  owned by the Air Force, the Army, Navy, and the remaining DOD  components. We selected one system with the highest expected cost  over the course of the department’s future-years defense program from  each group. This resulted in our selection of five systems: one Category I  system, one Category II system from each military department, and one  Category II system from the remaining DOD components.

We reviewed, respectively, DOD’s Healthcare Management System  Modernization Program; Air Force’s Maintenance, Repair and Overhaul  initiative; Army’s Reserve Component Automation System; Navy’s  Electronic Procurement System; and the Defense Logistics Agency’s  Defense Agencies Initiative Increment 2. We compared documentation of  the certifications for fiscal year 2017 of the five systems with the  requirements in the act.

We supplemented our analyses with interviews of cognizant DOD  officials. Specifically, we interviewed officials in the department’s Office of  the DCMO, Office of the Under Secretary of Defense for Acquisition,  Technology, and Logistics, and Office of the Chief Information Officer  (CIO). We also interviewed officials in the Offices of the Chief  Management Officers of the Departments of the Air Force, Army, and  Navy, as well as other cognizant military department officials. Additional  details on our objectives, scope, and methodology are contained in  appendix I.

We conducted this performance audit from January 2017 to March 2018,  in accordance with generally accepted government auditing standards.

Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objective. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objective.

DOD is the largest U.S. federal department and one of the most complex  organizations in the world. In support of its military operations, the  department manages many interdependent business functions, including  logistics management, procurement, health care management, and  financial management. DOD relies extensively on IT to support its  business functions.

According to its IT investment data, the department has 2,097 business  system investments. The department’s fiscal year 2018 IT budget  request states that DOD plans to spend about $8.7 billion in fiscal year  2018 on its defense business systems.

The IT budget organizes investments by mission areas. The four mission  areas are enterprise information environment, business, warfighting, and  defense intelligence. Figure 1 shows the amount of DOD’s requested  fiscal year 2018 IT budget that the department plans to spend on each  mission area.

The department further organizes its IT budget by segments. For  example, the business mission area includes segments such as financial  management, health, and human resource management. Figure 2  shows the department’s projected fiscal year 2018 spending for each  segment in the business mission area.

GAO designated the department’s business systems modernization  efforts as high risk in1995 and has continued to do so in the years since.  DOD currently bears responsibility, in whole or in part, for half of the  programs (17 of 34 programs) across the federal government that we  have designated as high risk. Seven of these areas are specific to the  department, and 10 other high-risk areas are shared with other federal  agencies.

Collectively, these high-risk areas are linked to the department’s ability to  perform its overall mission and affect the readiness and capabilities of  U.S. military forces. DOD’s business systems modernization is one of the  department’s specific high-risk areas and is essential for addressing  many of the department’s other high-risk areas. For example, modernized  business systems are integral to the department’s efforts to address its  financial and supply chain high-risk areas.

Since 2005, we have issued 11 reports in response to mandates directing  GAO to assess DOD’s actions to respond to business system  modernization provisions contained in Section 2222 of Title 10, United  States Code. These reports contained 23 recommendations to help  strengthen the department’s management of its business systems. As of  September 2017, the department had implemented 13 of the  recommendations and 2 had been closed as not implemented. The other  8 recommendations remain open. The 11 reports are listed in appendix II.

The NDAA for Fiscal Year 2016 Included Provisions for Managing Defense Business Systems

The NDAA for Fiscal Year 2016 included provisions requiring DOD to  perform certain activities aimed at ensuring that its business system  investments are managed efficiently and effectively. Specifically, the act  established requirements for the department related to issuing policy and  guidance for managing defense business systems; developing and  maintaining a defense business enterprise architecture; establishing a  Defense Business Council to provide advice to the Secretary on  managing defense business systems; and obtaining approvals before  systems proceed into development (or if no development is required, into  production or fielding) and related annual reviews.

According to the Joint Explanatory Statement accompanying the NDAA  for Fiscal Year 2016, the act revised Section 2222 of Title 10, United  States Code, to streamline requirements and clarify the responsibilities of  senior officials related to acquiring and managing business systems.  Key revisions pertain to:

Covered defense business systems. The code previously defined a  covered defense business system as a system having a total cost of  over $1 million over the period of the future-years defense program.

As revised, the code now defines a covered defense business system  as a system that is expected to have a total amount of budget  authority over the period of the current future-years defense program  of over $50 million.

Priority defense business systems. The act established a new category of system, called a priority defense business system. This is a system that is (1) expected to have a total amount of budget authority of over $250 million over the period of the current future- years defense program, or (2) designated by the DCMO as a priority defense business system based on specific program analyses of factors including complexity, scope, and technical risk, and after notification to Congress of such designation.

Thresholds and officials responsible for review and certification of defense business systems. The code previously stated that the DCMO had responsibility for reviewing and certifying all defense business system investments over $1 million over the future-years defense program. The revised code states that, unless otherwise assigned by the Secretary of Defense, military department Chief Management Officers (CMO) are to have approval authority for their covered defense business system investments below $250 million over the future-years defense program. The DCMO is to have approval authority for defense business systems owned by DOD components other than the military departments, systems that will support the business process of more than one military department or other component, and priority defense business systems.

Certification requirements. The code previously required that a defense business system program be reviewed and certified, at least annually, on the basis of its compliance with the business enterprise architecture and appropriate business process reengineering. In addition to these requirements, the revised code requires that the business system program be reviewed and certified on the basis of having valid, achievable requirements and a viable plan for implementing the requirements; having an acquisition strategy designed to eliminate or reduce the need to tailor commercial off-the- shelf systems; and being in compliance with the department’s  auditability requirements.

Key Roles and Responsibilities for Managing Defense Business Systems

DOD Instruction 5000.75: Business Systems Requirements and  Acquisition assigns roles and responsibilities for managing defense  business system investments. Table 1 identifies the key entities and  their responsibilities for managing defense business system investments.

DOD Has Made Progress in Complying with Legislative Provisions for Managing Defense Business Systems, but More Remains to Be Done

DOD has taken steps to address provisions of the NDAA for Fiscal Year  2016 related to defense business system investments. Specifically, as  called for in the act, the department has established guidance that  addresses most legislative requirements for managing its defense  business systems; however, the military departments are still developing  guidance to fully address certification requirements for their systems.

Further, DOD has developed a business enterprise architecture and is in  the process of updating the architecture to improve its content. The  department also has a plan to improve the usefulness of the business  enterprise architecture; however, the department has not delivered the  plan’s intended capabilities. In addition, the department is in the process  of updating its IT enterprise architecture; however, it does not have a plan  for improving the department’s IT and computing infrastructure for each of  the major business processes. Further, the department has not yet  demonstrated that the business enterprise architecture and the IT  enterprise architecture are integrated.

The department fully addressed the act’s requirement related to defense  business system oversight. Specifically, the department’s governance  board, called the Defense Business Council, addressed legislative  provisions to provide advice to the Secretary of Defense.

Lastly, DOD and the military departments did not apply new legislative  requirements when certifying business systems for fiscal year 2017.  Instead, the DOD DCMO certified the systems in our sample in  accordance with the previous fiscal year’s (fiscal year 2016) certification  requirements.

DOD Issued Guidance Addressing Most Legislative Requirements for Managing Business Systems

The NDAA for Fiscal Year 2016 required the Secretary of Defense to  issue guidance by December 31, 2016 to provide for the coordination of,  and decision making for, the planning, programming, and control of  investments in covered defense business systems. The act required this  guidance to address six elements:

Policy to ensure DOD business processes are continuously reviewed  and revised to implement the most streamlined and efficient business  processes practicable and eliminate or reduce the need to tailor  commercial off-the-shelf systems to meet or incorporate requirements  or interfaces that are unique to the department.

A process to establish requirements for covered defense business  systems.

Mechanisms for planning and controlling investments in covered  defense business systems, including a process for the collection and  review of programming and budgeting information for covered  defense business systems.

Policy requiring the periodic review of covered defense business  systems that have been fully deployed, by portfolio, to ensure that  investments in such portfolios are appropriate.

Policy to ensure full consideration of sustainability and technological  refreshment requirements, and the appropriate use of open  architectures.

Policy to ensure that best acquisition and systems engineering  practices are used in the procurement and deployment of commercial  systems, modified commercial systems, and defense-unique systems  to meet DOD missions.

Of these six elements called for by the act, the department has issued  guidance that fully addresses four elements and partially addresses two  elements. Table 2 summarizes our assessment of DOD’s guidance  relative to the act’s requirements.

DOD fully addressed the element requiring policy to ensure that the  business processes of the department are continuously reviewed and  revised. For example, DOD Instruction 5000.75 requires the functional  sponsor of a defense business system to engage in continuous process  improvement throughout all phases of the business capability acquisition  cycle. The department also fully addressed the element to provide a  process for establishing requirements for covered defense business  systems with DOD Instruction 5000.75, which introduces the business  capability acquisition cycle for business system requirements and  acquisition.

In addition, DOD fully addressed the element to provide mechanisms for  planning and controlling investments in covered defense business  systems. Specifically, the department’s Financial Management  Regulation; Directive 7045.14 on its planning, programming, budgeting,  and execution process; and the April 2017 Defense Business System  Investment Management Guidance provide such mechanisms. For  example, the April 2017 investment management guidance includes a  process, called the integrated business framework, which the department  is to follow for selecting, managing, and evaluating the results of  investments in defense business systems. In addition, the directive  assigns the DOD CIO responsibility for participating in the department’s  annual resource allocation process and for advising the Secretary and  Deputy Secretary of the Defense on IT resource allocations and  investment decisions.

Further, DOD fully addressed the requirement for a policy requiring the  periodic review of covered business systems that have been fully  deployed, by portfolio, to ensure that investments in such portfolios are  appropriate. Specifically, the department’s April 2017 Defense Business  System Investment Management Guidance requires the department to  annually review an organization’s plan for managing its portfolio of  defense business systems over the period of the current future-years  defense program (e.g., Army’s plan for its financial management systems)  to ensure, among other things, that the portfolio is aligned with applicable  functional strategies (e.g., DOD’s strategy for its financial management  functional area).

DOD partially addressed the element requiring policy to ensure full  consideration of sustainability and technological refreshment  requirements, and the appropriate use of open architectures. Specifically,  the department established policy requiring consideration of open  architectures, but it has not established policy requiring consideration of  sustainability and technological refreshment requirements. The Office of  the DCMO stated that future guidance is expected to provide a policy to  ensure full consideration of sustainability and technological refreshment  requirements. However, the department could not provide a time frame  for when the guidance will be developed and issued. Without a policy  requiring full consideration of sustainability and technological refreshment  requirements for its defense business system investments, the  department may not be able to ensure that it has a full understanding of  the costs associated with these requirements. As a result, the department  may not be able to effectively manage spending on these systems.

DOD has also partially addressed the element requiring policy to ensure  that best acquisition and systems engineering practices are used in the  procurement and deployment of commercial, modified-commercial, and  defense-unique systems. Specifically, the department has established  policy requiring the acquisition of business systems to be aligned with  commercial best practices and to minimize the need for customization of  commercial products to the maximum extent possible. On the other hand,  the department has not established policy to ensure the use of best  systems engineering practices.

With regard to this finding, officials in the Office of the DCMO asserted  that DOD Instruction 5000.75 addresses the requirement. However, while  the instruction requires the system acquisition strategy to include a  description of how the program plans to leverage systems engineering, it  does not require the use of best systems engineering practices. Without a  policy requiring the use of best systems engineering practices in the  procurement and deployment of commercial, modified, and defense- unique systems, the department may be limited in its ability to effectively  balance meeting system cost and performance objectives.

DOD Issued Guidance that Addresses New Certification Requirements, and the Military Departments Have Made Mixed Progress in Issuing Supporting Guidance

In addition to guidance for addressing the aforementioned legislative  requirements for business systems management, the NDAA for Fiscal  Year 2016 requires the Secretary to direct the DCMO and the CMO of  each of the military departments to issue and maintain supporting  guidance, as appropriate and within their respective areas of  responsibility. In this regard, one of the key areas for which the DCMO  and military department CMOs are to provide supporting guidance is the  review and certification of defense business systems in accordance with  specific requirements.

Specifically, the act requires that, for any fiscal year in which funds are  expended for development or sustainment pursuant to a covered defense  business system program, the appropriate approval official is to review  the system to determine if the system: has been, or is being, reengineered to be as streamlined and efficient  as practicable, and whether the implementation of the system will  maximize the elimination of unique software requirements and unique  interfaces;  is in compliance with the business enterprise architecture or will be in  compliance as a result of planned modifications; has valid, achievable requirements, and a viable plan for  implementing those requirements (including, as appropriate, market  research, business process reengineering, and prototyping activities); has an acquisition strategy designed to eliminate or reduce the need  to tailor commercial off-the-shelf systems to meet unique  requirements, incorporate unique requirements, or incorporate unique  interfaces to the maximum extent practicable; and  is in compliance with the department’s auditability requirements.

The act and DOD Instruction 5000.75 define the systems that the DOD  DCMO is responsible for certifying and the systems that military  department CMOs are responsible for certifying.

Consistent with the act, in April 2017, the DCMO issued guidance for  certifying officials that addresses the certification requirements. Table 3  provides our rating and assessment of the DCMO’s guidance for  implementing defense business system certification requirements.

By establishing guidance requiring that defense business systems be  certified on the basis of the legislative requirements, the department is  better positioned to ensure that a covered system does not proceed into  development (or, if no development is required, into production or fielding)  without the appropriate due diligence. Further, the department has taken  steps which should help ensure that funds are limited to systems in  development or sustainment that meet these requirements.

Air Force and Navy Guidance Partially Addresses Certification Requirements; Army Has Not Yet Issued Guidance

The military departments have made mixed progress in developing  supporting guidance to assist in making certification decisions regarding  systems within their respective areas of responsibility. More specifically,  the Air Force has issued supporting guidance that addresses three of the  act’s five certification requirements, but does not address the remaining  two requirements. Navy has issued guidance that addresses two of the  certification requirements, partially addresses one requirement, and does  not address two requirements. The Army has not yet issued guidance  on any of the five certification requirements.

Table 4 provides an overview of our assessment of the Air Force’s,  Navy’s, and Army’s guidance relative to the NDAA for Fiscal Year 2016  certification requirements. Each department’s efforts are further  discussed following the table.

Air Force. In April 2017, the Department of the Air Force issued guidance for certifying business systems for fiscal year 2018. The guidance addresses the requirements that a system be certified on the basis of sufficient business process reengineering, business enterprise architecture compliance, and valid requirements and a viable plan to implement them. Specifically, the guidance states that Air Force core defense business systems are required to comply with the business process reengineering guidance prescribed in the DCMO’s February 2015 Defense Business Systems Investment Management Process Guidance and for systems to assert compliance with the architecture through DCMO’s Integrated Business Framework—Data Alignment Portal. In addition, the guidance states that the department must follow DOD Instruction 5000.75, which requires that certifying officials determine that business requirements are valid and capability efforts have feasible implementation plans.

However, the Air Force guidance does not address the remaining two certification requirements. Officials in the office of the Air Force DCMO acknowledged that the Air Force’s business system certification guidance does not address determining how the acquisition strategy is designed to eliminate or reduce the need to  tailor commercial off-the-shelf systems to meet unique requirements,  incorporate unique requirements, or incorporate unique interfaces to  the maximum extent practicable or is in compliance with DOD’s  auditability requirements.

In May 2017, Air Force DCMO officials stated that the department was  in the process of developing guidance. However, as of December  2017, the Air Force had not described specific plans to update its  business system certification guidance.

Navy. The Department of the Navy issued guidance in May 2016.

This guidance addresses the requirements that a system be certified  on the basis of sufficient business process reengineering and  business enterprise architecture compliance. In this regard, the  guidance provides guidelines for documenting business process  reengineering and requires verification that business process  reengineering is complete. The guidance also specifies that defense  business systems are to map alignment with the business enterprise  architecture in DCMO’s Integrated Business Framework–Data  Alignment Portal.

Navy’s guidance partially addresses the certification requirement for  determining if a defense business system has valid requirements and  a viable plan to implement them. Specifically, the guidance includes  information on validating requirements; however, it does not include  information on determining if a system has a viable plan to implement  the requirements.

In addition, Navy’s guidance does not address the remaining two  certification requirements, which are to determine that the covered  defense business system has an acquisition strategy that eliminates  or reduces the need to tailor commercial-off-the-shelf systems, and  that the system is in compliance with DOD’s auditability requirements.  In August 2017, officials in the Office of the Under Secretary of the  Navy (Management) stated that the office was in the process of  updating its May 2016 Defense Business System Investment  Certification Manual. The officials stated that the goal is to issue  interim investment certification guidance by May 2018. As of  November 2017, however, Navy had not established a plan for when  it expects to publish finalized certification guidance.

Army. The Department of the Army has not issued guidance that  addresses any of the act’s certification requirements. The Army  issued a template that was to be used to develop fiscal year 2018  portfolio review submissions. However, the template does not address  any of the certification requirements.

Officials in the Army’s Office of Business Transformation explained  that the Army used DOD DCMO’s 2014 guidance to certify its  business systems for fiscal year 2017. In May 2017, they stated that  the Army was in the process of developing guidance to implement  DOD’s new instruction. In November 2017, an official in the Army’s  Office of Business Transformation stated that the office was in the  process of completing the guidance and aimed to provide it to the  Deputy Under Secretary’s office for signature in January 2018.  However, the department has not committed to a specific time frame  for when the new guidance is expected to be issued.

Without guidance for the certification authority to determine that defense  business systems have addressed each of the act’s certification  requirements, the Air Force, Navy, and Army risk allowing systems to  proceed into development or production that do not meet these  requirements. In particular, the military departments risk wasting funds on  developing and maintaining systems that do not have valid requirements  and a viable plan to implement the requirements, introduce unnecessary  complexity, or that do not adequately support the Department of  Defense’s efforts to meet its auditability requirements.

DOD Has Efforts Underway to Improve Its Business Enterprise Architecture, but Its IT Architecture Is Not Complete

According to the NDAA for Fiscal Year 2016, DOD is to develop and  maintain a defense business enterprise architecture to guide the  development of integrated business processes within the department. In  addition, the act states that the business architecture must be consistent  with the policies and procedures established by the Director of the Office  of Management and Budget. Among other things, OMB policy calls for  agencies to develop an enterprise architecture that describes the current  architecture, target architecture, and a transition plan to get to the target  architecture. The act also calls for the business architecture to contain  specific content, including policies, procedures, business data standards,  business information requirements, and business performance measures  that are to apply uniformly throughout the department.

DOD has developed a business enterprise architecture that is intended to  help guide the development of its defense business systems. The  department issued version 10 of the business architecture, which is  currently being used to support system certification decisions, in February  2013. The business architecture and related documentation include  content describing aspects of the current architecture, target architecture,  and a transition plan to get to the target architecture. In addition, the  business architecture includes content that addresses the act’s  requirements. Table 5 provides examples of required content in DOD’s  business enterprise architecture.

Nevertheless, some content included in version 10 of the business  architecture is outdated and incomplete. For example, version 10 of the  business architecture’s repository of laws, regulations, and policies was  last updated in February 2013, and officials in the Office of the DOD CIO  and Office of the DCMO confirmed that they are not current. Further, the  department’s March 2017 business architecture compliance guidance  stated that not all relevant business data standards are identified in the  business architecture. In addition, based on our review, information about  performance measures documented in the architecture is incomplete. For  example, target values for performance measures associated with  acquisition and logistics initiatives are not identified.

According to officials in the Office of the DCMO, the department is  working to update the business architecture. Specifically, the department  has developed version 11 of the business architecture to, in part, replace  outdated architecture content. According to the officials, version 11 of the  architecture is currently available online, but version 10 remains the  official version of the business enterprise architecture used for system  certification decisions. The officials stated that the department continues  to add content to version 11, and they expect that it will be used as the  basis of system certification decisions for fiscal year 2019.

In addition, DOD has ongoing work to address a key recommendation we  made in July 2015 associated with improving the usefulness of its  business architecture. In particular, we reported that the majority of  military department portfolio managers that we surveyed believed that  the business architecture had not been effective in meeting intended  outcomes. For example, only 25 percent of the survey respondents  reported that the business architecture effectively enabled DOD to  routinely produce timely, accurate, and reliable business and financial  information for management purposes. In addition, only 38 percent  reported that the business architecture effectively guided the  implementation of interoperable defense business systems. As a result,  we reported that the architecture had produced limited value and  recommended that the department use the results of our survey to  determine additional actions that can improve the department’s  management of its business enterprise architecture activities.

In response to our recommendation, DOD identified opportunities to  address our survey findings and developed a plan for improving its ability  to achieve architecture-related outcomes. DOD’s business enterprise  architecture improvement plan was signed by the Assistant DCMO in  January 2017. However, the department has not yet demonstrated that it  has delivered the capabilities described by the plan; thus, we will continue  to monitor DOD’s progress to fully address this recommendation.

DOD Has Taken Steps to Develop Its IT Enterprise Architecture, but Does Not Have a Plan That Provides a Road Map for Improving the Department’s IT and Computing Infrastructure

In addition to the business enterprise architecture, according to the act,  the DOD CIO is to develop an IT enterprise architecture. This architecture  is to describe a plan for improving the IT and computing infrastructure of  the department, including for each of the major business processes.

Officials in the Office of the DOD CIO stated that the department  considers its information enterprise architecture to be its IT enterprise  architecture. The DOD CIO approved version 2.0 of its information  enterprise architecture in August 2012. According to DOD documentation,  this architecture describes the department’s current information enterprise  (i.e., information resources, assets, and processes used to share  information across the department and with its mission partners) and  includes a vision for the target information enterprise; documents required  capabilities, and the activities, rules, and services needed to provide  them; and includes information for applying and complying with the  architecture.

Nevertheless, while the architecture includes content describing the  department’s current and target information enterprise, which is  consistent with OMB guidance, it does not include a transition plan that  provides a road map for improving the department’s IT and computing  infrastructure. Related to this finding, DCMO officials did not agree with  our assessment concerning the department’s IT enterprise architecture  transition plan. In this regard, officials in the Office of the DCMO stated  that the department’s DOD IT Portfolio Repository includes information for  managing efforts to improve IT and computing infrastructure at the  system level. According to the repository’s data dictionary, this  information can include system life cycle start and end dates, as well as  information that supports planning for a target environment.

However, documentation describing DOD’s information enterprise  architecture does not identify the DOD IT Portfolio Repository as being  part of the architecture. Moreover, it does not include a plan for improving  the department’s IT and computing infrastructure for each of the major  business processes. Officials in the Office of the CIO acknowledged that  the architecture does not include such plans. According to the officials,  the department is currently developing version 3.0 of its information  enterprise architecture (i.e., its IT enterprise architecture). The officials  stated that the department does not currently intend for the architecture to  include a plan for improving the department’s IT and computing  infrastructure that addresses each of the major business processes. They  added, however, that there is an effort to ensure that functional areas,  such as human resources management, are included. DCMO officials  stated that the department has not defined how the DOD IT enterprise  architecture needs to be segmented for each major business process  because the infrastructure requirements seem to be similar for each of  the processes.

Without an architecture that includes a plan for improving its IT and  computing infrastructure, including for each of the major business  processes, DOD risks not ensuring that stakeholders across the  department have a consistent understanding of the steps needed to  achieve the department’s future vision, agency priorities, potential  dependencies among investments, and emerging and available  technological opportunities.

DOD Has Not Demonstrated That Its Business and IT Architectures Are Integrated

According to the act, the DOD business enterprise architecture is to be  integrated into the DOD IT enterprise architecture. The department’s  business architecture compliance guide also recognizes that the business  architecture is to be integrated with the IT enterprise architecture.

However, the department has not demonstrated that it has integrated the  business enterprise architecture into the information enterprise  architecture. Specifically, the department did not provide documentation  associated with either architecture that describes how the two are, or are  to be, integrated. The business enterprise architecture compliance guide  states that DOD Directive 8000.01 implements the requirement that the  two architectures are to be integrated. However, the directive does not  address how they are, or are to be, integrated.

Officials in the Offices of the CIO and the DCMO described steps they  were taking to coordinate the development of the next versions of the  information enterprise architecture (i.e., IT enterprise architecture) and  business enterprise architecture. However, these steps were not sufficient  to help ensure integration of the two architectures. Specifically, in June  2017, officials in the Office of the DOD CIO stated they were participating  in the development of the next version of the business architecture and  that the DOD CIO is represented on the Business Enterprise Architecture  Configuration Control Board. Officials in the Office of the DCMO  confirmed that DOD CIO officials participate on the board. However,  officials from the Office of the DCMO said that, until it met in June 2017  the board had not met since 2014. Moreover, documentation of the June  2017 meeting, and a subsequent November 2017 meeting, did not  indicate that the board members had discussed integration of the  department’s business and information enterprise architectures.

In addition, officials in the Office of the DCMO reported that the office has  not actively participated in the information enterprise architecture working  group. Further, our review of meeting minutes from this working group did  not identify participation by officials in the Office of the DCMO, or that  integration of the architectures was discussed. The Office of the DCMO  described other mechanisms for its sharing of information about  architectures with the Office of the DOD CIO. For example, the Office of  the DCMO stated that it participates with DOD CIO bodies governing  version 3.0 development. Nevertheless, the Office of the DCMO  reiterated that technical integration of the architectures has not been  designed.

Until DOD ensures that its business architecture is integrated into its IT  enterprise architecture, the department may not be able to ensure that its  business strategies capitalize on technologies and that its IT  infrastructure will support DOD’s business priorities and related business  strategies.

The Defense Business Council Addressed Legislative Provisions to Provide Advice on Defense Business Systems to the Secretary of Defense

The NDAA for Fiscal Year 2016 requires the Secretary to establish a  Defense Business Council, chaired by the DCMO and the DOD CIO, to  provide advice to the Secretary on: developing the business enterprise architecture,  reengineering the department’s business processes, developing and deploying business systems, and developing requirements for business systems.

DOD established the department’s Defense Business Council in October  2012, prior to the act. According to its current charter, dated December  2014, the Council is co-chaired by the DCMO and the DOD CIO. In  addition, the Council is to serve as the principal governance body for  vetting issues related to managing and improving defense business  operations. Among other things, it serves as the investment review board  for defense business system investments.

The Defense Business Council charter also states that the Council was  established as a principal supporting tier of governance to the Deputy’s  Management Action Group. The Deputy’s Management Action Group was  established by an October 2011 memorandum issued by the Deputy  Secretary of Defense. According to information published on DCMO’s  website, the group was established to be the primary civilian-military  management forum that supports the Secretary of Defense, and is to  address top department issues that have resource, management, and  broad strategic and/or policy implications. The group’s primary mission  is to produce advice for the Deputy Secretary of Defense in a  collaborative environment and to ensure that the group’s execution aligns  with the Secretary of Defense’s priorities. According to the Office of the  DCMO, the Defense Business Council determines whether or not to  elevate a topic to the Deputy’s Management Action Group to address on  behalf of the Secretary.

Based on our review of meeting documentation for 27 meetings that the  Defense Business Council held between January 2016 and August 2017,  the Council discussed the four topics on which the NDAA for Fiscal Year  2016 requires it to provide advice to the Secretary. According to the  Office of the DCMO, during the discussions of these topics, the Council  did not identify any issues related to the topics that needed to be elevated  to the Deputy’s Management Action Group. Table 6 identifies the number  of meetings in which the Council discussed each topic during this time  period.

By ensuring that the required business system topics are discussed  during Defense Business Council meetings, the department should be  positioned to raise issues to the Deputy’s Management Action Group, and  ultimately, to advise the Secretary of Defense on matters associated with  these topics.

DOD Certified Selected Business Systems for Fiscal Year 2017 on the Basis of Earlier Certification Requirements

The NDAA for Fiscal Year 2016 requires that, for any fiscal year in which  funds are expended for development or sustainment pursuant to a  covered defense business system program, the Secretary of Defense is  to ensure that a covered business system not proceed into development  (or, if no development is required, into production or fielding) unless the  appropriate approval official reviews the system to determine if the  system meets five key requirements, as previously discussed in this  report. In addition, the act requires that the appropriate approval official  certify, certify with conditions, or decline to certify that the system satisfies  these five requirements.

The department issued DOD Instruction 5000.75, which established  business system categories and assigned certifying officials, consistent  with the act. Table 7 describes the business system categories and the  assigned certifying officials, as defined in DOD Instruction 5000.75.

The DOD DCMO certified the five systems in our sample (which included  the military departments’ systems) for fiscal year 2017. However, these  certifications were issued in accordance with the previous fiscal year’s  (fiscal year 2016) certification requirements. Those requirements had  stipulated that a defense business system program was to be reviewed  and certified on the basis of the system’s compliance with the business  enterprise architecture and appropriate business process reengineering,  rather than on the basis of having met all five requirements identified in  the NDAA for Fiscal Year 2016.

Specifically, DCMO certified the systems on the basis of determining that  the systems were in compliance with the business enterprise architecture  and had been sufficiently reengineered. However, none of the systems  were certified on the basis of a determination that they had valid,  achievable requirements and a viable plan for implementing them; had an  acquisition strategy to reduce or eliminate the need to tailor commercial  off-the-shelf systems; or were in compliance with the department’s  auditability requirements.

Officials in the Offices of the DOD DCMO, the Air Force DCMO, the  Under Secretary of the Navy (Management), and Army Business  Transformation told us that the systems were not certified relative to three  of the requirements because the department did not issue guidance to  reflect changes made by the NDAA for Fiscal Year 2016 in time for the  fiscal year 2017 certification process. Prior to the NDAA for Fiscal Year  2016, relevant legislation and DOD guidance only called for annual  determinations to be made regarding whether a system complied with the  business enterprise architecture and whether appropriate business  process reengineering had been conducted.

In January 2016, the DCMO issued a memorandum stating that the  department planned to issue new guidance and policy to implement the  new legislation by the end of February 2016. However, the department  did not issue additional guidance addressing the new certification  requirements until April 2017. The system certifications, which were  required by the act to be completed before systems could spend fiscal  year 2017 funds, occurred in August and September 2016.

In explaining the delay in issuing new guidance on the certification  requirements, officials in the Office of the DCMO stated that the statutory  deadline for issuing guidance was December 31, 2016. They added that,  given this statutory deadline, and the start of fiscal year 2017 on October  1, 2016, it was their determination that Congress did not intend for the  NDAA for Fiscal Year 2016’s certification requirements to be fully  implemented before fiscal year 2017 started. DCMO officials stated that  they intend for the department to use the certification requirements  established by the NDAA for Fiscal Year 2016 for future system  certifications.

While it was reasonable for the department to use the earlier guidance for  its fiscal year 2017 certifications, given that the new guidance had not yet  been issued, it will be important going forward that the department  certifies business systems on the basis of the certification requirements  established in the NDAA for Fiscal Year 2016 and its related guidance  addressing these requirements. Certifying systems on the basis of the  act’s requirements should help ensure that funds are not wasted on  developing and maintaining systems that do not have valid requirements  and a viable plan to implement the requirements, that introduce  unnecessary complexity, or that impede the Department of Defense’s  efforts to meet its auditability requirements.

Since the NDAA for Fiscal Year 2016 was signed in November 2015,  DOD has issued guidance that addresses most provisions of the NDAA  for Fiscal Year 2016 related to managing defense business system  investments. However, the department has not established policies  requiring consideration of sustainability and technology requirements and  the use of best systems engineering practices in the procurement and  deployment of its systems. Having these policies would better enable the  department to ensure it is efficiently and effectively procuring and  deploying its business systems.

In addition, the Air Force, the Army, and Navy have made mixed progress  in issuing guidance to assist in making certification decisions regarding  systems within their respective areas of responsibility. Specifically, the Air  Force and Navy issued guidance on the certification of business systems  that does not fully address new certification requirements, while the Army  has not issued any updated guidance for its certifications. As a result, the  Air Force, Navy, and Army risk wasting funds on developing and  maintaining systems that do not have valid requirements and a viable  plan to implement the requirements, introduce unnecessary complexity,  or do not adequately support the Department of Defense’s efforts to meet  its auditability requirements.

Also, DOD has developed an IT architecture, but this architecture does  not address the act’s requirement that it include a plan for improving the  department’s IT and computing infrastructure, including for each business  process. In addition, DOD’s plans for updating its IT architecture do not  address how the department intends to integrate its business and IT  architectures, as called for by the act. As a result, DOD risks not having a  consistent understanding of what is needed to achieve the department’s  future vision, agency priorities, potential dependencies among  investments, and emerging and available technological opportunities.

We are making six recommendations, including three to the Secretary of  Defense and one to each of the Secretaries of the Air Force, the Navy  and the Army:  The Secretary of Defense should define a specific time frame for  finalizing, and ensure the issuance of (1) policy requiring full  consideration of sustainability and technological refreshment  requirements for its defense business system investments; and (2) policy  requiring that best systems engineering practices are used in the  procurement and deployment of commercial systems, modified  commercial systems, and defense-unique systems to meet DOD  missions. (Recommendation 1)

The Secretary of the Air Force should define a specific time frame for  finalizing, and ensure the issuance of guidance for certifying the  department’s business systems on the basis of (1) having an acquisition  strategy designed to eliminate or reduce the need to tailor commercial off- the-shelf systems to meet unique requirements, incorporate unique  requirements, or incorporate unique interfaces to the maximum extent  practicable; and (2) being in compliance with DOD’s auditability  requirements. (Recommendation 2)

The Secretary of the Navy should define a specific time frame for  finalizing, and ensure the issuance of guidance for certifying the  department’s business systems on the basis of (1) having a viable plan to  implement the system’s requirements; (2) having an acquisition strategy  designed to eliminate or reduce the need to tailor commercial off-the-shelf  systems to meet unique requirements, incorporate unique requirements,  or incorporate unique interfaces to the maximum extent practicable; and  (3) being in compliance with DOD’s auditability requirements.  (Recommendation 3)

The Secretary of the Army should define a specific time frame for  finalizing, and ensure the issuance of guidance for certifying the  department’s business systems on the basis of (1) being reengineered to  be as streamlined and efficient as practicable, and determining that  implementation of the system will maximize the elimination of unique  software requirements and unique interfaces; (2) being in compliance with  the business enterprise architecture; (3) having valid, achievable  requirements and a viable plan to implement the requirements; (4) having  an acquisition strategy designed to eliminate or reduce the need to tailor  commercial off-the-shelf systems to meet unique requirements,  incorporate unique requirements, or incorporate unique interfaces to the  maximum extent practicable; and (5) being in compliance with DOD’s  auditability requirements. (Recommendation 4)

The Secretary of Defense should ensure that the DOD CIO develops an  IT enterprise architecture which includes a transition plan that provides a  road map for improving the department’s IT and computing infrastructure,  including for each of its business processes. (Recommendation 5)

The Secretary of Defense should ensure that the DOD CIO and Chief  Management Officer work together to define a specific time frame for  when the department plans to integrate its business and IT architectures  and ensure that the architectures are integrated. (Recommendation 6)

DOD provided written comments on a draft of this report, which are  reprinted in appendix III. In the comments, the department stated that it  concurred with three of the recommendations and partially concurred with  three of the recommendations. DOD also provided evidence that it has  fully addressed one of the recommendations. In addition, DOD provided  technical comments that we incorporated in the report, as appropriate.

DOD stated that it concurred with our first recommendation, which called  for it to define a specific time frame for finalizing, and ensure the issuance  of, policies that fully address provisions in the NDAA for Fiscal Year 2016.  Furthermore, the department stated that it had complied with the  recommendation. Specifically, the department stated that it had published  its defense business systems investment management guidance in April  2017. This guidance identifies DOD’s Financial Management Regulation,  Volume 2B, Chapter 18 “Information Technology” and supporting IT  budget policy and guidance as well as DOD Instruction 5000.75 and  supporting acquisition policy and guidance. The department stated that  the Financial Management Regulation specifically addresses the  requirement for sustainability and technological refreshment requirements  for its defense business system investments.

While DOD reported taking this action, we do not agree that the  department has complied with our recommendation. In reviewing the  department’s guidance, we found that none of the cited management  documents includes a policy requiring consideration of sustainability and  technological refreshment requirements for DOD’s defense business  systems. Further, none of these documents includes a policy requiring  that best systems engineering practices be used in the procurement and  deployment of commercial, modified-commercial, and defense unique  systems.

Without a policy requiring full consideration of sustainability and  technological refreshment requirements for its defense business system  investments, the department may not be able to ensure that it has a full  understanding of the costs associated with these requirements. Further,  without a policy requiring the use of best systems engineering practices in  systems procurement and deployment, the department may be limited in  its ability to effectively balance meeting system cost and performance  objectives. Accordingly, we continue to believe that our recommendation  is valid.

The department concurred with our second recommendation, that the  Secretary of the Air Force define a specific time frame for finalizing, and  ensure the issuance of, guidance that fully addresses certification  requirements, in accordance with the NDAA for Fiscal Year 2016.  Moreover, the department stated that the Air Force has complied with the  recommendation. Specifically, DOD stated that Air Force Manual 63-144  details the consideration of using existing commercial solutions without  modification or tailoring.

However, while the manual provides a foundation on which the Air Force  can build, it is not sufficient to fully address our recommendation because  it does not include guidance on certifying business systems on the basis  of having an acquisition strategy that eliminates or reduces the need to  tailor commercial-off-the-shelf systems. In addition, the department did  not demonstrate that the Air Force has issued guidance for certifying  business systems on the basis of being in compliance with DOD’s  auditability requirements. Rather, the Air Force stated that it has pending  guidance that addresses the acquisition strategy and auditability  requirements. We plan to evaluate the guidance to determine the extent  to which it addresses our recommendation after it is issued.

The department partially agreed with our third recommendation, that the  Secretary of the Navy define a specific time frame for finalizing, and  ensure the issuance of, guidance that fully addresses certification  requirements. Specifically, DOD stated that Navy agreed to issue  guidance. Subsequently, on March 8, 2018, Navy issued its updated  guidance.

However, Navy disagreed with the recommendation, as written, and  suggested that GAO revise the recommendation to state that “The  Secretary of the Navy should ensure guidance is issued according to   established timeline for certifying the department’s business systems. . .”  According to Navy, this change would support alignment with the timeline  for certifying the department’s business systems driven by the Chief  Management Officer investment review timeline.

Based on our analysis, we found the guidance that Navy issued to be  consistent with our recommendation. Thus, we plan to close the  recommendation as fully implemented. We have also annotated this  report, where appropriate, to explain that the Navy issued guidance while  the draft of this report was at the department for comment. On the other  hand, we did not revise the wording of our recommendation, as we  believe it appropriately reflected the importance of Navy taking action to  ensure the issuance of its guidance.

The department stated that it concurred with our fourth recommendation,  which called for the Secretary of the Army to define a specific time frame  for finalizing, and ensure the issuance of, guidance for certifying the  department’s business systems on the basis of the certification  requirements. Furthermore, on March 23, 2018, the Army issued its  guidance. However, because of the timing of this report relative to when  the Army provided its guidance to us (on March 27, 2018), we have not  yet completed an assessment of the guidance.  We have annotated this  report, where appropriate, to reflect the Army’s action on our  recommendation.

The department stated that it partially concurred with our fifth  recommendation. This recommendation called for the DOD CIO to  develop an IT enterprise architecture which includes a transition plan that  provides a road map for improving the department’s IT and computing  infrastructure, including for each of its business processes. Toward this  end, the department agreed that the DOD CIO should develop an  architecture that enables improving the department’s IT and computing  infrastructure for each of its business processes. However, the  department also stated that the recommendation is not needed because  the goal is already being accomplished by a set of processes,  organizations, protocols, and architecture data.

For example, the department described processes and relationships  between the Office of the DOD CIO and the Office of the Chief  Management Officer and the boards that support the department’s  business and IT enterprise architectures. In particular, the department  stated that information enterprise architecture data relevant to the  business enterprise are accessed via the DOD Information Enterprise  Architecture Data Selection Wizard and imported into the business  enterprise architecture. The department further stated that, if the business  capability acquisition cycle process indicates a need to improve the IT or  computing infrastructure, the Office of the Chief Management Officer has  a protocol to initiate a proposal to change the information enterprise  architecture.

We agree that the department’s processes, organizations, protocols, and  architecture data are keys to successful IT management. However, during  the course of our audit, we found that documentation describing DOD’s IT  architecture did not include a plan for improving the department’s IT and  computing infrastructure for each of the major business processes.  Moreover, officials in the Office of the CIO acknowledged that the  architecture did not include such a plan.

Without a transition plan that provides a road map for improving the  department’s IT and computing infrastructure, including for each of its  business processes, it will be difficult for the department to rely on its  personnel to timely and proactively manage and direct modernization  efforts of such a magnitude as DOD’s systems modernization efforts.  Further, without such a plan, DOD risks not being able to ensure that  stakeholders across the department have a consistent understanding of  the steps needed to achieve the department’s future vision, agency  priorities, potential dependencies among investments, and emerging and  available technological opportunities. Thus, we maintain that the  department should fully implement our recommendation.

The department stated that it partially concurred with our sixth  recommendation, that the DOD CIO and DCMO work together to define a  specific time frame for when the department plans to integrate its  business and IT architectures. In particular, the department stated that it  agrees that the DOD CIO and Chief Management Officer should work  together to establish a time frame and ensure coordination and  consistency of the IT and business architectures. However, the  department disagreed with the use and intent of the term “integrate,” as  stated in the recommendation, although it did not explain the reason for  this disagreement. Instead, it proposed that we change our  recommendation to read “The GAO recommends the Secretary of  Defense ensure the DoD CIO and CMO work together to define a specific  timeline for coordinating its business and IT architectures to achieve  better enterprise alignment among the architectures.”

We agree that it is important to achieve coordination and consistency  between the business and IT architectures. However, the department did  not provide documentation associated with either architecture that  describes how the two are, or are to be, integrated, as called for by the  NDAA for Fiscal Year 2016 and DOD guidance.

Integrating the architectures would help ensure that business strategies  better capitalize on existing and planned technologies and that IT  solutions and infrastructure support business priorities and related  business strategies. Thus, we continue to believe that our  recommendation is valid. However, we have updated the  recommendation to state that the DOD CIO and the Chief Management  Officer should work together. We made this change because, effective  February 1, 2018, the Secretary of Defense eliminated the DCMO  position and expanded the role of the Chief Management Officer, in  accordance with the National Defense Authorization Act for Fiscal Year  2018.

We are sending copies of this report to appropriate congressional  committees; the Secretary of Defense; the Secretaries of the Army, Navy,  and Air Force; and the Director of the Office of Management and Budget.  In addition, the report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff members have any questions about this report, please  contact me at (202) 512-4456 or harriscc@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. GAO staff who made contributions to this  report are listed in appendix IV.

Appendix I: Objective, Scope, and Methodology

Our objective was to determine the actions taken by the Department of  Defense (DOD) to comply with provisions included in the National  Defense Authorization Act for Fiscal Year 2016 (NDAA). These  provisions require DOD to perform certain activities aimed at ensuring  that its business system investments are managed efficiently and  effectively. Specifically, we determined to what extent DOD has  1.  established guidance for effectively managing its defense business  2.  developed and maintained a defense business enterprise architecture  and information technology (IT) enterprise architecture, in accordance  with relevant laws and Office of Management and Budget (OMB)  policies and guidance;  3.  used the Defense Business Council to provide advice to the Secretary  on developing the business enterprise architecture, reengineering the  department’s business processes, developing and deploying business  systems, and developing requirements for business systems; and  4.  ensured that covered business systems are reviewed and certified in  accordance with the act.

To address the extent to which DOD has established guidance for  effectively managing defense business system investments, we obtained  and analyzed the department’s guidance, as well as the guidance  established by the Departments of the Air Force, Army, and Navy, for  managing defense business systems relative to the act’s requirements.  Specifically, the NDAA for Fiscal Year 2016 required the Secretary of  Defense to issue guidance, by December 31, 2016, to provide for the  coordination of and decision making for the planning, programming, and  control of investments in covered defense business systems. The act  required this guidance to include the following six elements:

Policy to ensure DOD business processes are continuously reviewed  and revised to implement the most streamlined and efficient business  processes practicable and eliminate or reduce the need to tailor  commercial off-the-shelf systems to meet or incorporate requirements  or interfaces that are unique to the department.

Process to establish requirements for covered defense business  systems.

Mechanisms for planning and controlling investments in covered  defense business systems, including a process for the collection and  review of programming and budgeting information for covered  defense business systems.

Policy requiring the periodic review of covered defense business  systems that have been fully deployed, by portfolio, to ensure that  investments in such portfolios are appropriate.

Policy to ensure full consideration of sustainability and technological  refreshment requirements, and the appropriate use of open  architectures.

Policy to ensure that best acquisition and systems engineering  practices are used in the procurement and deployment of commercial  systems, modified commercial systems, and defense-unique systems  to meet DOD missions.

We assessed the February 2017 DOD Instruction 5000.75, Business  Systems Requirements and Acquisitions, and April 2017 defense  business system investment management guidance, which the  department issued to address the act’s requirements. In addition, we  assessed the department’s Financial Management Regulation and  directive on its planning, programming, budgeting, and execution  process, which the department stated also address the act’s provisions.

We also assessed DOD’s guidance for managing business system  investments relative to the act’s business system certification  requirements. The act requires that the Secretary of Defense ensure that  a covered defense business system not proceed into development (or, if  no development is required, into production or fielding) unless the  appropriate approval official determines that the system meets five  requirements. The act further requires for any fiscal year in which funds  are expended for development or sustainment pursuant to a covered  defense business system program, the appropriate approval official to  review the system to determine if the system: has been, or is being, reengineered to be as streamlined and efficient  as practicable, and whether the implementation of the system will  maximize the elimination of unique software requirements and unique  interfaces;  is in compliance with the business enterprise architecture or will be in  compliance as a result of planned modifications; has valid, achievable requirements, and a viable plan for  implementing those requirements (including, as appropriate, market  research, business process reengineering, and prototyping activities); has an acquisition strategy designed to eliminate or reduce the need  to tailor commercial off-the-shelf systems to meet unique  requirements, incorporate unique requirements, or incorporate unique  interfaces to the maximum extent practicable; and  is in compliance with the department’s auditability requirements.

We compared Office of the Deputy Chief Management Office (DCMO)  certification guidance with the act’s certification requirements. In addition,  we compared the guidance established by the Departments of the Air  Force, the Army, and the Navy for certifying their business systems with  the act’s certification requirements.

We also interviewed cognizant officials responsible for managing defense  business system investments at DOD, including the military departments.  Specifically, we interviewed officials in the Office of the DCMO, the Office  of the Under Secretary of Defense for Acquisition, Technology, and  Logistics, the Office of the Chief Information Officer (CIO), and the Offices  of the CMOs in the Departments of the Air Force, Army, and Navy.

To determine the extent to which DOD has developed and maintained a  defense business enterprise architecture and IT enterprise architecture, in  accordance with relevant laws and OMB policy and guidance, we  assessed the business enterprise architecture against the relevant laws  and OMB policy and guidance; the IT enterprise architecture against the  relevant laws and OMB policy and guidance; and the department’s efforts  to integrate its business and IT architectures against the act’s  requirement.

To determine the extent to which the department has developed and  maintained a business enterprise architecture in accordance with relevant  laws and OMB policy and guidance, we reviewed version 10 of its  business enterprise architecture, which was released in February 2013,  and related information relative to the act’s requirements; U.S. Code,  Title 44, Section 3601, which defines an enterprise architecture; and  OMB policy and guidance. We also reviewed version 11 of the  architecture to determine the extent to which it differed from version 10.

Further, we reviewed the department’s business enterprise architecture  improvement plan, which it developed in response to a recommendation  we made in July 2015. Specifically, we recommended that the  department use the results of our portfolio manager survey to determine  additional actions that could improve the department’s management of its  enterprise architecture activities. In response to our recommendation, the  department developed and approved a plan in January 2017. We  assessed the extent to which the department had delivered the planned  capabilities relative to the plan. We also reviewed the extent to which the  delivery dates of the three planned capabilities and associated tasks  changed over time relative to the plan.

To assess the extent to which the department developed and maintained  an IT enterprise architecture in accordance with relevant laws and OMB  policy and guidance, we reviewed content from the department’s IT  enterprise architecture and compared it with requirements from the act,  U.S. Code, Title 44, Section 3601, and OMB policy and guidance.

Specifically, we reviewed version 2.0 of the department’s information  enterprise architecture, which was released in August 2012, relative to  the act’s requirement for the DOD CIO to develop an IT enterprise  architecture that is to describe a plan for improving the IT and computing  infrastructure of the department, including for each of the major business  processes. We reviewed volumes I and II of the information enterprise  architecture and the four enterprise-wide reference architectures to  determine if the architecture described a plan for improving the IT and  computing infrastructure of the department, as called for by the act. We  also reviewed whether the architecture included content that described  the current and the target environments, and a transition plan to get from  the current to the target environment, consistent with OMB policy and  guidance.

To determine the extent to which the department has integrated its  business and IT architectures, as required by the act, we reviewed DOD  Directive 8000.01, Management of the Department of Defense  Information Enterprise. We also reviewed meeting documentation from  the information enterprise architecture working group responsible for the  development of an updated architecture. In addition, we reviewed  meeting documentation from the Business Enterprise Architecture  Configuration Control Board to identify any discussions among CIO and  DCMO officials regarding integration of the two architectures, as well as  the level of participation by both parties.

Finally, we interviewed officials in the Office of the DCMO and the Office  of the CIO about efforts to develop and maintain a business enterprise  architecture, develop an IT enterprise architecture, and integrate the  business and IT architectures.

To determine the extent to which the department has used the Defense  Business Council to provide advice to the Secretary of Defense on  developing the business enterprise architecture, reengineering the  department’s business processes, developing and deploying business  systems, and developing requirements for business systems, in  accordance with the act, we analyzed the department’s December 2014  Defense Business Council Charter and April 2017 defense business  systems investment management guidance. We compared information in  the charter and guidance to the requirement that the Secretary establish  the Defense Business Council to advise the Secretary on the required  defense business system topics.

In addition, we obtained and analyzed meeting summaries and briefings  for 27 Defense Business Council meetings that took place from January  2016 through August 2017. Specifically, we assessed the frequency  with which the meetings held during this time period addressed the  required topics. We chose this time period because 2016 was the first  calendar year following the enactment of the NDAA for Fiscal Year 2016.  Further, we chose August 2017 as our end date because it was the last  month’s data that we could reasonably expect to obtain and review within  our reporting time frame. We also interviewed officials in the Offices of the  DCMO and CIO about the Defense Business Council and the Deputy’s  Management Action Group, which is the governance entity to which the  Council reports.

To determine the extent to which DOD has ensured that covered  business systems are reviewed and certified in accordance with the act,  we reviewed a nongeneralizable sample of business systems from DOD’s  two categories of covered defense business systems that require  certification. To select the sample, we considered Category I systems,  which were systems that were expected to have a total amount of budget  authority of more than $250 million over the period of the current future- years defense program, and Category II systems, which were systems  that were expected to have a total amount of budget authority of between  $50 million and $250 million over the period of the future-years defense  program.

We further categorized the Category II systems into four groups—those  owned by the Air Force, the Army, Navy, and the remaining DOD  components. We selected one system with the highest expected cost  over the course of the department’s future-years defense program from  each group. This resulted in our selection of five systems: one Category I  system, one Category II system from each military department, and one  Category II system from the remaining DOD components.

We reviewed, respectively, DOD’s Healthcare Management System  Modernization Program; Air Force’s Maintenance, Repair and Overhaul  initiative; Army’s Reserve Component Automation System; Navy’s  Electronic Procurement System; and the Defense Logistics Agency’s  Defense Agencies Initiative Increment 2. We determined that the number  of systems we selected was sufficient for our evaluation.

For each system, we assessed the extent to which it had been certified  on the basis of the five certification requirements in the act. Specifically,  we evaluated investment decision memos and certification assertions to  determine if each system had been certified according to the act’s  requirements, which include ensuring that the system had been, or was  being, reengineered to be as streamlined and efficient as practicable, and  the implementation of the system would maximize the elimination of  unique software requirements and unique interfaces; was in compliance  with the business enterprise architecture or would be in compliance as a  result of planned modifications; had valid, achievable requirements, and a  viable plan for implementing those requirements; had an acquisition  strategy designed to eliminate or reduce the need to tailor commercial off- the-shelf systems to meet unique requirements, incorporate unique  requirements, or incorporate unique interfaces to the maximum extent  practicable; and was in compliance with the department’s auditability  requirements. We did not determine whether the certification assertions  were valid. For example, we did not evaluate business process  reengineering activities to determine if they were sufficient. We also  interviewed DOD DCMO and military department officials about the  certification of these systems.

To determine the reliability of the business system cost data used to  select the systems, we reviewed system documentation for the three  systems DOD uses to store data, which include the Defense Information  Technology Investment Portal, the DOD Information Technology Portfolio  Repository, and the Select and Native Programming-Information  Technology system. In this regard, we requested and reviewed  department responses to questions about the systems and about how the  department ensures the quality and reliability of the data. In addition, we  requested and reviewed documentation related to the systems (e.g., data  dictionaries, system instructions, and user training manuals) and  reviewed the data for obvious issues, including missing or questionable  values. We also reviewed available reports on the quality of the  inventories (e.g., inspector general reports). We found the data to be  sufficiently reliable for our purpose of selecting systems for evaluation.

We conducted this performance audit from January 2017 to March 2018  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix II: Prior GAO Reports on Department of Defense Business System Modernization

Since 2005, we have issued 11 reports assessing DOD’s actions to  respond to business system modernization provisions contained in U.S.  Code, Title 10, Section 2222. The reports are listed below.

DOD Business Systems Modernization: Additional Action Needed to  Achieve Intended Outcomes, GAO-15-627 (Washington, D.C.: July 16,  2015).

Defense Business Systems: Further Refinements Needed to Guide the  Investment Management Process, GAO-14-486 (Washington, D.C. May  12, 2014).

DOD Business Systems Modernization: Further Actions Needed to  Address Challenges and Improve Accountability, GAO-13-557  (Washington, D.C.: May 17, 2013).

DOD Business Systems Modernization: Governance Mechanisms for  Implementing Management Controls Need to Be Improved, GAO-12-685  (Washington, D.C.: June 1, 2012).

Department of Defense: Further Actions Needed to Institutionalize Key  Business System Modernization Management Controls, GAO-11-684  (Washington, D.C.: June 29, 2011).

Business Systems Modernization: Scope and Content of DOD’s  Congressional Report and Executive Oversight of Investments Need to  Improve, GAO-10-663 (Washington, D.C.: May 24, 2010).

DOD Business Systems Modernization: Recent Slowdown in  Institutionalizing Key Management Controls Needs to Be Addressed,  GAO-09-586 (Washington, D.C.: May 18, 2009).

DOD Business Systems Modernization: Progress in Establishing  Corporate Management Controls Needs to Be Replicated Within Military  Departments, GAO-08-705 (Washington, D.C.: May 15, 2008).

DOD Business Systems Modernization: Progress Continues to Be Made  in Establishing Corporate Management Controls, but Further Steps Are  Needed, GAO-07-733 (Washington, D.C.: May 14, 2007).

Business Systems Modernization: DOD Continues to Improve Institutional  Approach, but Further Steps Needed, GAO-06-658 (Washington, D.C.:  May 15, 2006).

DOD Business Systems Modernization: Important Progress Made in  Establishing Foundational Architecture Products and Investment  Management Practices, but Much Work Remains, GAO-06-219  (Washington, D.C.: November 23, 2005).

Appendix III: Comments from the Department of Defense

Appendix IV: GAO Contact and Staff Acknowledgments

In addition to the contact above, individuals making contributions to this  report include Michael Holland (Assistant Director), Cheryl Dottermusch  (Analyst in Charge), John Bailey, Chris Businsky, Camille Chaires, Nancy  Glover, James Houtz, Anh Le, Tyler Mountjoy, Monica Perez-Nelson,  Priscilla Smith, and Adam Vodraska.