Women-Owned Small Business Program: Actions Needed to Address Ongoing Oversight Issues

In 2000, Congress authorized the WOSB program, allowing contracting officers to set aside procurements to women-owned small businesses in industries in which they are substantially underrepresented. To be eligible to participate in the WOSB program, firms have the option to self-certify or be certified by a third-party certifier. However, the 2015 NDAA changed the WOSB program by (1) authorizing SBA to implement sole-source authority, (2) eliminating the option for firms to self-certify as being eligible for the program and (3) allowing SBA to implement a new certification process.

GAO was asked to review the WOSB program. This report discusses (1) the extent to which SBA has addressed the 2015 NDAA changes, (2) SBA's efforts to address previously identified deficiencies, and (3) use of the WOSB program. GAO reviewed relevant laws, regulations, and program documents; analyzed federal contracting data from April 2011 through June 2018; and interviewed SBA officials, officials from contracting agencies selected to obtain a range of experience with the WOSB program, and three of the four private third-party certifiers.

The Small Business Administration (SBA) has implemented one of the three changes to the Women-Owned Small Business (WOSB) program authorized in the National Defense Authorization Act of 2015 (2015 NDAA). Specifically, in September 2015 SBA published a final rule to implement sole-source authority, effective October 2015. As of February 2019, SBA had not eliminated the option for program participants to self-certify that they are eligible to participate, as required by 2015 NDAA. SBA officials stated that this requirement would be addressed as part of the new certification process for the WOSB program, which they expect to implement by January 1, 2020.

SBA has not addressed WOSB program oversight deficiencies identified in GAO's 2014 review (GAO-15-54). For example, GAO previously recommended that SBA establish procedures to assess the performance of four third-party certifiers—private entities approved by SBA to certify the eligibility of WOSB firms. While SBA conducted a compliance review of the certifiers in 2016, it has no plans to regularly monitor them. By not improving its oversight of the WOSB program, SBA is limiting its ability to ensure third-party certifiers are following program requirements. In addition, the implementation of sole-source authority in light of these continued oversight deficiencies can increase program risk. Consequently, GAO maintains that its prior recommendations should be addressed. In addition, similar to previous findings from SBA's Office of Inspector General, GAO found that about 3.5 percent of contracts using a WOSB set-aside were awarded for ineligible goods or services from April 2011 through June 2018. SBA does not review contracting data that could identify this problem and inform SBA which agencies making awards may need targeted outreach or training. As a result, SBA cannot provide reasonable assurance that WOSB program requirements are being met and that the program is meeting its goals.

While federal contract obligations to all women-owned small businesses and WOSB program set-asides have increased since fiscal year 2012, WOSB program set-asides remain a small percentage (see figure).

GAO recommends that SBA develop a process for periodically reviewing the extent to which WOSB program set-asides are awarded for ineligible goods or services and use the results to address identified issues, such as through targeted outreach or training on the WOSB program. SBA agreed with the recommendation.

Women-owned businesses play an increasingly significant role in the U.S.  economy. According to the most recent U.S. Census Survey of Business  Owners, the number of women-owned businesses grew by almost 27  percent from 2007 through 2012. However, the federal goal of awarding  5 percent of procurements government-wide to women-owned small  businesses has only been met once since fiscal year 2013. In 2000,  Congress authorized the Women-Owned Small Business Federal  Contracting Program (WOSB program) to, among other things, reserve  contracts for qualified women-owned small businesses in industries  where these businesses are substantially underrepresented. On October  7, 2010, the Small Business Administration (SBA) issued a final rule to  implement the WOSB program, and the program began operating in  2011.

SBA allows businesses to establish program eligibility through self- certification or certification by third-party certifiers. In 2014, we reviewed  the WOSB program and found a number of deficiencies in SBA’s  oversight of the four SBA-approved third-party certifiers and in SBA’s  eligibility examination processes. In addition, in 2015 and 2018 the SBA  Office of Inspector General (OIG) reviewed the WOSB program and also  found several oversight deficiencies.

In December 2014, the National Defense Authorization Act of 2015 (2015  NDAA) amended the WOSB program by, among other things, granting  contracting officers the authority to award sole-source contracts to eligible  women-owned small businesses (WOSB) and economically  disadvantaged women-owned small businesses (EDWOSB). The act  also removed firms’ ability to self-certify as eligible for the WOSB program  and allowed SBA to certify the eligibility of program participants as an  additional alternative to third-party certification.

You requested that we evaluate SBA’s WOSB program, including any  actions SBA has taken in response to the 2015 NDAA and to previously  identified deficiencies, as well as usage of the program. This report  examines (1) the extent to which SBA has implemented changes to the  WOSB program made by the 2015 NDAA; (2) the extent to which SBA  has implemented changes to address previously identified oversight  deficiencies; and (3) changes in WOSB program use since 2011 and  stakeholder views on its use, including since the 2015 implementation of  sole-source authority.

To address the first objective, we reviewed the 2015 NDAA, proposed  regulations, and SBA documentation, and we interviewed SBA officials.  To respond to the second and third objectives, we interviewed SBA  officials, three of the WOSB program’s four private third-party certifiers  (referred to in the report as “third-party certifiers”), three selected federal  agencies (and three agency components within two of the agencies), and  a total of eight selected contracting offices within the six selected  agencies or components. Using data from the Federal Procurement Data  System-Next Generation (FPDS-NG), we judgmentally selected the three  federal agencies and three components (for a total of six selected  agencies and components) because their WOSB dollar obligations  (including competed and sole-source) were among the largest or because  we had interviewed them for our prior work. Specifically, we selected the  following six agencies or agency components: the Department of  Homeland Security (DHS) and, within DHS, the Coast Guard; the  Department of Defense (DOD) and, within DOD, the U.S. Army and U.S.  Navy; and the General Services Administration (GSA). Within the  components and GSA, we judgmentally selected eight contracting offices  (two each from Coast Guard, U.S. Army, U.S. Navy, and GSA) based on  whether they had a relatively large amount of obligations and had used  multiple types of set-asides (competed or sole-source) to WOSBs or  EDWOSBs.

To address the second objective, we reviewed SBA policies and  procedures for the WOSB program and SBA oversight activities, such as  SBA compliance reviews of the four third-party certifiers. We analyzed  FPDS-NG data on obligations for WOSB program set-aside contracts and  the goods and services for which they were awarded—represented by  North American Industry Classification System (NAICS) codes—against  NAICS codes approved for WOSB program eligibility to determine if these  contracts were made in eligible industries. We compared SBA’s activities  against federal internal control standards, GAO’s framework for managing  fraud risk, and our prior recommendations to SBA. We also reviewed  relevant SBA OIG reports and interviewed SBA OIG officials.

To address the third objective, we analyzed FPDS-NG data from April  2011 through June 2018 (the most recent data available during the period  of our analysis) and identified trends in total obligation amounts of  competed and sole-source contracts awarded under the WOSB program  (including both WOSB and EDWOSB contracts). To determine the  relative usage of the WOSB program, we compared data on obligations  for set-asides under the WOSB program with federal contract obligations  for WOSB-program-eligible goods and services to all woman-owned small  businesses. We assessed the reliability of FPDS-NG data by reviewing  available documentation and prior GAO data reliability assessments and  by electronically testing for missing data, outliers, and inconsistent  coding. We found the data to be reliable for the purposes of reporting on  trends in the WOSB program and the usage of sole-source authority  under the program. Appendix I provides more detail on our scope and  methodology.

We conducted this performance audit from October 2017 to March 2019  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Federal agencies conduct a variety of procurements that are reserved for  small business participation through small business set-asides. The set- asides can be for small businesses in general, or they can be specific to  small businesses that meet additional eligibility requirements in the  Service-Disabled Veteran-Owned Small Business (SDVOSB), Historically  Underutilized Business Zone (HUBZone), 8(a) Business Development  (8(a)), and WOSB programs.

The WOSB program enables federal contracting officers to identify and  establish a sheltered market, or set-aside, for competition among WOSBs  and EDWOSBs in certain industries. To determine the industries eligible  under the WOSB program, SBA is required to conduct a study to  determine which NAICS codes are eligible under the program and to  report on such studies every 5 years. WOSBs can receive set-asides in  industries in which SBA has determined that women-owned small  businesses are substantially underrepresented. EDWOSBs can receive  set-asides in WOSB-eligible industries as well as in an additional set of  industries in which SBA has determined that women-owned small  businesses are underrepresented but not substantially so. As of  February 2019, there were a total of 113 four-digit NAICS codes  (representing NAICS industry groups) eligible under the WOSB  program—92 eligible NAICS codes for WOSBs and 21 for EDWOSBs.

Additionally, businesses must be at least 51 percent owned and  controlled by one or more women who are U.S. citizens to participate in  the WOSB program. The owner must provide documents demonstrating  that the business meets program requirements, including a document in  which the owner attests to the business’s status as a WOSB or  EDWOSB. EDWOSBs are WOSBs that are controlled by one or more  women who are citizens and who are economically disadvantaged in  accordance with SBA regulations. According to SBA, as of early October  2018, there were 13,224 WOSBs and 4,488 EDWOSBs registered in  SBA’s online certification database.

SBA’s Office of Government Contracting administers the WOSB program  by promulgating regulations, conducting eligibility examinations of  businesses that receive contracts under a WOSB or EDWOSB set-aside,  deciding protests related to eligibility for a WOSB set-aside, conducting  studies to determine eligible industries, and working with other federal  agencies in assisting WOSBs and EDWOSBs. According to SBA officials,  the Office of Government Contracting also works at the regional and local  levels with SBA’s Small Business Development Centers and district  offices, and with other organizations (such as Procurement Technical  Assistance Centers), to help WOSBs and EDWOSBs obtain contracts  with federal agencies. The services SBA coordinates include training,  counseling, mentoring, facilitating access to information about federal  contracting opportunities, and business financing. According to SBA, as  of October 2018, there were two full-time staff within the Office of  Government Contracting whose primary responsibility was the WOSB  program.

Initially, the program’s statutory authority allowed WOSBs to be self- certified by the business owner or certified by an approved third-party  national certifying entity as eligible for the program. Self-certification is  free, but some third-party certification options require businesses to pay a  fee. Each certification process requires businesses to provide signed  representations attesting to their WOSB or EDWOSB eligibility.  Businesses must provide documents supporting their status before  submitting an offer to perform the requirements of a WOSB set-aside  contract. In August 2016, SBA launched certify.sba.gov, which is an  online portal that allows firms to upload required documents and track  their submission and also enables contracting officers to review firms’  eligibility documentation. According to the Federal Acquisition  Regulation (FAR), contracting officers are required to verify that all  required documentation is present in the online portal when selecting a  business for an award. In addition, businesses must register and attest to  being a WOSB in the System for Award Management, the primary  database of vendors doing business with the federal government.

In 2011, SBA approved four organizations to act as third-party certifiers:

NWBOC (previously known as the National Women Business Owners

Women’s Business Enterprise National Council.

These organizations have been the WOSB program’s third-party certifiers  since 2011. According to SBA data, the Women’s Business Enterprise  National Council was the most active third-party certifier in fiscal year  2017—performing 2,638 WOSB certification examinations. The U.S.

Women’s Chamber of Commerce, NWBOC, and El Paso Hispanic  Chamber of Commerce—completed 644, 105, and 12 certifications,  respectively.

As discussed previously, in 2014 we reviewed the WOSB program and  found a number of deficiencies in SBA’s oversight of the four SBA- approved third-party certifiers and in SBA’s eligibility examination  processes and we made related recommendations for SBA. In addition,  in 2015 and 2018 the SBA OIG reviewed the WOSB program and also  found oversight deficiencies, including evidence of WOSB contracts set  aside for ineligible firms. In both reports, the SBA OIG also made  recommendations for SBA. Further, in July 2015, we issued GAO’s  fraud risk framework, which provides a comprehensive set of key  components and leading practices that serve as a guide for agency  managers to use when developing efforts to combat fraud in a strategic,  risk-based way. In July 2016, the Office of Management and Budget  issued guidelines requiring executive agencies to create controls to  identify and respond to fraud risks. These guidelines also affirm that  managers should adhere to the leading practices identified in GAO’s  fraud risk framework.

SBA Has Implemented One of the Three Changes Made by the 2015 NDAA

As of February 2019, SBA had implemented one of the three changes  that the 2015 NDAA made to the WOSB program—sole-source authority.  The two other changes—authorizing SBA to implement its own  certification process for WOSBs and requiring SBA to eliminate the  WOSB self-certification option—have not been implemented. The 2015  NDAA did not require a specific time frame for SBA to update its  regulations. SBA officials have stated that they will not eliminate self- certification until the new certification process for the WOSB program is in  place, which they expect to be completed by January 1, 2020.

Sole-Source Authority Has Been Implemented

In September 2015, SBA published a final rule to implement sole-source  authority for the WOSB program (effective October 2015). Among other  things, the rule authorized contracting officers to award a contract to a  WOSB or EDWOSB without competition, provided that the contracting  officer’s market research cannot identify two or more WOSBs or  EDWOSBs in eligible industries that can perform the requirements of the  contract at a fair and reasonable price. In the final rule, SBA explained  that it promulgated the sole-source rule before the WOSB certification  requirements for two reasons. First, the sole-source rule could be  accomplished by simply incorporating the statutory language into the  regulations, whereas the WOSB certification requirements would instead  require a prolonged rulemaking process. Second, SBA said that  addressing all three regulatory changes at the same time would delay the  implementation of sole-source authority. SBA described the sole-source  mechanism as an additional tool for federal agencies to ensure that  women-owned small businesses have an equal opportunity to participate  in federal contracting and to ensure consistency among SBA’s  socioeconomic small business procurement programs.

According to SBA, most of the 495 comments submitted about the sole- source rule supported the agency’s decision to implement the authority  quickly. However, the SBA OIG’s June 2018 audit report cautioned that  allowing sole-source contracting authority while firms can still self-certify  exposes the WOSB program to unnecessary risk of fraud and abuse, and  the report recommended that SBA implement a new certification process  for the WOSB program per the 2015 NDAA. In addition, our previous  report identified risks of program participation by ineligible firms  associated with deficiencies in SBA’s oversight structure. As we discuss  in detail later, SBA has still not addressed these risks, which may be  exacerbated by the implementation of sole-source authority without  addressing the other changes made by the 2015 NDAA, including  eliminating the self-certification option.

A New WOSB Program Certification Process Has Not Been Implemented

As of February 2019, SBA had not published a proposed rule for public  comment to establish a new certification process for the WOSB program.  Previously, in October 2017, an SBA official stated that SBA was about  1–2 months away from publishing a proposed rule. However, in June  2018, SBA officials stated that a cost analysis would be necessary before  the draft could be sent to the Office of Management and Budget for  review. Certain stages of the rulemaking process have mandated time  periods, such as the required interagency review process for certain  rules. In June 2017, we reported that SBA officials said that an increase  in the number of statutorily mandated rules in recent years had  contributed to delays in the agency’s ability to promulgate rules in a more  timely fashion. As of February 2019, SBA had not provided  documentation or time frames for issuing a proposed rule or completing  the rulemaking process. However, in response to the SBA OIG  recommendation that SBA implement the new certification process, SBA  stated that it would fulfill the recommendation (meaning implement a new  certification process) by January 1, 2020.

In December 2015, SBA published an advance notice of proposed  rulemaking to solicit public comments to assist the agency with drafting a  proposed rule to implement a new WOSB certification program. In the  notice, SBA stated that it intends to address the 2015 NDAA changes,  including eliminating the self-certification option, through drafting  regulations to implement a new certification process. Previously, in its  September 2015 final rule implementing sole-source authority, SBA  stated that there was no evidence that Congress intended that the  existing WOSB program, including self-certification, be halted before  establishing the infrastructure and new regulations for a new certification  program. The advance notice requested comments on various topics,  such as how well the current certification processes were working, which  of the certification options were feasible and should be pursued, whether  there should be a grace period for self-certified WOSB firms to complete  the new certification process, and what documentation should be  required.

Three third-party certifiers submitted comments in response to the  advance notice of proposed rulemaking, and none supported the option of  SBA acting as a WOSB certifier. One third-party certifier commented  that such an arrangement is a conflict of interest given that SBA is also  responsible for oversight of the WOSB program, and two certifiers  commented that SBA lacked the required resources. The three third-party  certifiers also asserted in their comments that no other federal agency  should be allowed to become an authorized WOSB certifier, with one  commenting that federal agencies should instead focus on providing  contracting opportunities for women-owned businesses. All three certifiers  also proposed ways to improve the current system of third-party  certification—for example, by strengthening oversight of certifiers or  expanding their number. The three certifiers also suggested that SBA  move to a process that better leverages existing programs with  certification requirements similar to those of the WOSB program, such as  the 8(a) program. In the advance notice, SBA asked for comments on  alternative certification options, such as SBA acting as a certifier or  limiting WOSB program certifications to the 8(a) program and otherwise  relying on state or third-party certifiers. Further, in June 2018, SBA  officials told us that they were evaluating the potential costs of a new  certification program as part of their development of the new certification  rule.

SBA Has Not Fully Addressed Deficiencies in Oversight and Program Implementation

SBA Has Not Implemented Procedures to Regularly Monitor and Assess the Performance of Third- Party Certifiers

SBA has not fully addressed deficiencies in its oversight of third-party  certifiers that we identified in our October 2014 report. We reported that  SBA did not have formal policies for reviewing the performance of its four  approved third-party certifiers, including their compliance with their  agreements with SBA. Further, we found that SBA had not developed  formal policies and procedures for, among other things, reviewing the  monthly reports that certifiers submit to SBA. As a result, we  recommended that SBA establish comprehensive procedures to monitor  and assess the performance of the third-party certifiers in accordance  with their agreements with SBA and program regulations. While SBA has  taken some steps to address the recommendation, as of February 2019 it  remained open.

In response to our October 2014 recommendation, in 2016 SBA  conducted compliance reviews of the four SBA-approved third-party  certifiers. According to SBA, the purpose of the compliance reviews was  to ensure the certifiers’ compliance with regulations, their signed third- party certifier certification form (or agreement) with SBA, and other  program requirements. The compliance reviews included an assessment  of the third-party certifiers’ internal certification procedures and  processes, an examination of a sample of applications from businesses  that the certifiers deemed eligible and ineligible for certification, and an  interview with management staff.

SBA officials said that SBA’s review team did not identify significant  deficiencies in any of the four certifiers’ processes and found that all were  generally complying with their agreements. However, one compliance  review report described “grave concerns” that a third-party certifier had  arbitrarily established eligibility requirements that did not align with WOSB  program regulations and used them to decline firms’ applications. SBA  noted in the report that if the third-party certifier failed to correct this  practice SBA could terminate the agreement. As directed by SBA, the  third-party certifier submitted a letter to SBA outlining actions it had taken  to address this issue, among others. The final compliance review  reports for the other third-party certifiers also recommended areas for  improvement, including providing staff with additional training on how to  conduct eligibility examinations and reviewing certification files to ensure  they contain complete documentation. In addition, two of the three  compliance review reports with recommendations (including the  compliance review report for the certifier discussed above) required the  certifier to provide a written response within 30 days outlining plans to  correct the areas. SBA officials said that they reviewed the written  responses and determined that no further action was required.

In January 2017, SBA’s Office of Government Contracting updated its  written Standard Operating Procedures (SOP) to include policies and  procedures for the WOSB program, in part to address our October 2014  recommendation. The 2017 SOP discusses what a third-party-certifier  compliance review entails, how often the reviews are to be conducted, and how findings are to be reported. The 2017 SOP notes that SBA may  initiate a compliance review “at any time and as frequently as the agency  determines is necessary.” In September 2018, SBA officials told us that  they were again updating the SOP, in part to address deficiencies we  identified in our prior work and during this review. However, as of  February 2019, SBA had not provided an updated SOP.

In addition, in April 2018, SBA finalized a WOSB Program Desk Guide  that, according to SBA, is designed to provide program staff with detailed  guidance for conducting oversight procedures, including compliance  reviews of third-party certifiers. For example, the Desk Guide discusses  how staff should prepare for a compliance review of a third-party certifier,  review certification documents, and prepare a final report. However, the  Desk Guide does not describe specific activities designed to oversee  third-party certifiers on an ongoing basis. In November 2017, SBA  officials told us that they planned to conduct additional compliance  reviews of the third-party certifiers. However, in June 2018, officials said  there were no plans to conduct further compliance reviews until the final  rule implementing the new certification process was completed. Further,  SBA officials said that the 2016 certifier compliance reviews did not result  in significant deficiencies. However, as noted previously, one of the  compliance review reports described a potential violation of the third-party  certifier’s agreement with SBA.

Per written agreements with SBA, third-party certifiers are required to  submit monthly reports that include  the number of WOSB and EDWOSB applications received, approved,  and denied;  identifying information for each certified business, such as the  business name;  concerns about fraud, waste, and abuse; and a description of any changes to the procedures the organizations  used to certify businesses as WOSBs or EDWOSBs.

In our October 2014 report, we noted that SBA had not followed up on  issues raised in the monthly reports and had not developed written  procedures for reviewing them. At that time, SBA officials said that they  were unaware of the issues identified in the certifiers’ reports and that the  agency was developing procedures for reviewing the monthly reports but  could not estimate a completion date.

In our interviews for this report, SBA officials stated that SBA still does not  use the third-party certifiers’ monthly reports to regularly monitor the  program. Specifically, SBA does not review the reports to identify any  trends in certification deficiencies that could inform program oversight.  Officials said the reports generally do not contain information that SBA  considers helpful for overseeing the WOSB program, although staff  sometimes use the reports to obtain firms’ contact information. SBA  officials also said that staff very rarely receive information about  potentially fraudulent WOSB firms from the third-party certifiers—maybe  three firms per year—and that this information is generally received via  email and not as part of the monthly reports. SBA officials said that when  they receive information about potentially fraudulent firms, WOSB  program staff conduct an examination to determine the firm’s eligibility  and report the results back to the certifier. However, a third-party certifier  told us it has regularly reported firms it suspected of submitting potentially  fraudulent applications in its monthly reports and that SBA has not  followed up with them. In addition, two third-party certifiers said that if  SBA is not cross-checking the list of firms included in their monthly  reports, a firm deemed ineligible by one certifier may submit an  application to another certifier and obtain approval.

The three third-party certifiers we spoke with said that SBA generally had  not communicated with them about their implementation of the program  since the 2016 compliance reviews. However, SBA officials noted that  three of the four third-party certifiers attended an SBA roundtable in  March 2017 to discuss comments on the proposed rulemaking. In  addition, SBA officials said that the third-party certifiers may contact them  with questions about implementing the WOSB program, but SBA  generally does not reach out to them.

Although SBA has taken steps to enhance its written policies and  procedures for oversight of third-party certifiers, it does not have plans to  conduct further compliance reviews of the certifiers and does not intend to  review certifiers’ monthly reports on a regular basis. SBA officials said  that third-party certifier oversight procedures would be updated, if  necessary, after certification options have been clarified in the final  WOSB certification rule. However, ongoing oversight activities, such as  regular compliance reviews, could help SBA better understand the steps  certifiers have taken in response to previous compliance review findings  and whether those steps have been effective. In addition, leading fraud  risk management practices include identifying specific tools, methods,  and sources for gathering information about fraud risks, including data on  fraud schemes and trends from monitoring and detection activities, as  well as involving relevant stakeholders in the risk assessment process.  Without procedures to regularly monitor and oversee third-party certifiers,  SBA cannot provide reasonable assurance that certifiers are complying  with program requirements and cannot improve its efforts to identify  ineligible firms or potential fraud. Further, it is unclear when SBA’s final  rule will be implemented. As a result, we maintain that our previous  recommendation should be addressed—that is, that the Administrator of  SBA should establish and implement comprehensive procedures to  monitor and assess the performance of certifiers in accordance with the  requirements of the third-party certifier agreement and program  regulations.

SBA Has Not Implemented Procedures to Improve Its Eligibility Examinations of WOSB Program Participants

SBA also has not fully addressed deficiencies found in our 2014 review  related specifically to eligibility examinations. We found that SBA lacked  formalized guidance for its eligibility examination processes and that the  examinations continued to identify high rates of potentially ineligible  businesses. As a result, we recommended that SBA enhance its  examination of businesses that register for the WOSB program to ensure  that only eligible businesses obtain WOSB set-asides. Specifically, we  suggested that SBA consider (1) completing the development of  procedures to conduct annual eligibility examinations and implementing  such procedures; (2) analyzing examination results and individual  businesses found to be ineligible to better understand the cause of the  high rate of ineligibility in annual reviews and determine what actions are  needed to address the causes, and (3) implementing ongoing reviews of  a sample of all businesses that have represented their eligibility to  participate in the program.

SBA has taken some steps to implement our recommendation—such as  by completing its 2017 SOP and its Desk Guide, both of which include  written policies and procedures for WOSB program eligibility  examinations. The 2017 SOP includes a brief description of what  activities are entailed in the examinations, the staff responsible for  conducting them, and how firms are selected. In addition, as noted  previously, SBA officials told us in September 2018 that a forthcoming  update to the SOP would address deficiencies we identified regarding  WOSB eligibility examinations. However, as of February 2019, SBA had  not provided an updated SOP. The Desk Guide contains more detailed  information on eligibility examinations. It notes that a sample of firms is to  be examined annually and it provides selection criteria, which can include  whether the agency has received information challenging the firm’s  eligibility for the program. The Desk Guide also provides specific  instructions on how to determine whether a firm meets the WOSB  program’s ownership, control, and financial requirements and what  documentation should be consulted or requested.

SBA does not collect reliable information on the results of its annual  eligibility examinations. According to SBA officials, SBA has conducted  eligibility examinations of a sample of businesses that received WOSB  program set-aside contracts each year since fiscal year 2012. However,  SBA officials told us that the results of annual eligibility examinations— such as the number of businesses found eligible or ineligible—are  generally not documented. As a result, we obtained conflicting data from  SBA on the number of examinations completed and the percentage of  businesses found to be ineligible in fiscal years 2012 through 2018. For  example, based on previous information provided by SBA, we reported in  October 2014 that in fiscal year 2012, 113 eligibility examinations were  conducted and 42 percent of businesses were found to be ineligible for  the WOSB program. However, during this review, we received information  from SBA that 78 eligibility examinations were conducted and 37 percent  of businesses were found ineligible in fiscal year 2012. We found similar  disparities when we compared fiscal year 2016 data provided by SBA for  this report with a performance memorandum summarizing that fiscal  year’s statistics. Regardless of the disparity between the data sources,  the rate of ineligible businesses has remained significant. For example,  according to documentation SBA provided during this review, in fiscal  year 2017, SBA found that about 40 percent of the businesses in its  sample were not eligible.

In addition, SBA continues to have no mechanism for evaluating  examination results in aggregate to inform the WOSB program. In 2014,  we reported that SBA officials told us that most businesses that were  deemed ineligible did not understand the documentation requirements for  establishing eligibility. However, we also reported that SBA officials could  not explain how they knew a lack of understanding was the cause of  ineligibility among businesses and had not made efforts to confirm that  this was the cause. In June 2018, SBA officials told us they did not  analyze the annual examinations in aggregate for common eligibility  issues because the examination results are unique to each WOSB firm.  They noted that this was not necessary as WOSB program staff are  familiar with common eligibility issues through the annual eligibility  examinations. As we noted in 2014, by not analyzing aggregate  examination results, the agency is missing opportunities to obtain  meaningful insights into the program, such as the reasons many  businesses are deemed ineligible.

Also, SBA still conducts eligibility examinations only of firms that have  already received a WOSB award. In 2014, we concluded that this  sampling practice restricts SBA’s ability to identify potentially ineligible  businesses prior to a contract award. Similarly, during this review, SBA  officials said that while some aspects of the sample characteristics have  changed since 2012, the samples still generally consist only of firms that  have been awarded a WOSB set-aside. In addition, officials said that the  sample size of the eligibility examinations has varied over time and is  largely based on the workload of WOSB program staff. Restricting the  samples in this way limits SBA’s ability to better understand the eligibility  of businesses before they apply for and are awarded contracts, as well as  its ability to detect and prevent potential fraud.

SBA officials said that their other means of reducing participation by  ineligible firms and mitigating potential fraud is through WOSB or  EDWOSB status protests—that is, allegations that a business receiving  an award does not meet program eligibility requirements. A federal  contractor can file a status protest against any firm receiving an award  that represents itself as a WOSB in the System for Award Management  for grounds that include failure to provide all required supporting  documentation. The penalties for misrepresenting a firm’s status, per  regulation, include debarment or suspension. However, one third-party  certifier expressed in its comments to the advance notice of proposed  rulemaking on certification that status protests alone are not a viable  option for protecting the integrity of the WOSB program. The certifier  questioned how a firm could have sufficient information about a  competitor firm to raise questions about its eligibility. According to SBA  officials, 11 status protests were filed under the WOSB program in fiscal  year 2018. Of these, four firms were deemed ineligible for the WOSB  program, four were deemed eligible, and three status protests were  dismissed. In fiscal year 2017, 9 status protests were filed; of these, three  firms were found ineligible, two were found eligible, and four status  protests were dismissed.

We recognize that SBA has made some effort to address our previous  recommendation by documenting procedures for conducting annual  eligibility examinations of WOSB firms. However, leading fraud risk  management practices state that federal program managers should  design control activities that focus on fraud prevention over detection and  response, to the extent possible. Without maintaining reliable  information on the results of eligibility examinations, developing  procedures for analyzing results, and expanding the sample of  businesses to be examined to include those that did not receive  contracts, SBA limits the value of its eligibility examinations and its ability  to reduce ineligibility among businesses registered to participate in the  WOSB program. These deficiencies also limit SBA’s ability to identify  potential fraud risks and develop any additional control activities needed  to address these risks. As a result, the program may continue to be  exposed to the risk of ineligible businesses receiving set-aside contracts.  In addition, in light of these continued oversight deficiencies, the  implementation of sole-source authority without addressing the other  changes made by the 2015 NDAA could increase program risk. For these  reasons, we maintain that our previous recommendation that SBA  enhance its WOSB eligibility examination procedures should be  addressed.

SBA Has Not Addressed Previously Identified Issues with WOSB Set- Asides Awarded Under Ineligible Industry Codes

In 2015 and 2018, the SBA OIG reported instances in which WOSB set- asides were awarded using NAICS codes that were not eligible under the  WOSB program, and our analysis indicates that this problem persists. In  2015, the SBA OIG reported on its analysis of a sample of 34 WOSB set- aside awards and found that 10 awards were set aside using an ineligible  NAICS code. The SBA OIG concluded that this may have been due to  contracting officers’ uncertainty about NAICS code requirements under  the program and recommended that SBA provide additional, updated  training and outreach to federal agencies’ contracting officers on the  program’s NAICS code requirements. In response, SBA updated WOSB  program training and outreach documents in March 2016 to include  information about the program’s NAICS code requirements.

In 2018, the SBA OIG issued another report evaluating the WOSB  program, with a focus on the use of the program’s sole-source contract  authority. Here, the SBA OIG identified additional instances of contracting  officers using inaccurate NAICS codes to set aside WOSB contracts.  Specifically, the SBA OIG reviewed a sample of 56 awards and found that  4 were awarded under ineligible NAICS codes. The report included two  recommendations for SBA aimed at preventing and correcting improper  NAICS code data in FPDS-NG: (1) conduct quarterly reviews of FPDS- NG data to ensure contracting officers used the appropriate NAICS codes  and (2) in coordination with the Office of Federal Procurement Policy and  GSA, strengthen controls in FPDS-NG to prevent contracting officers from  using ineligible NAICS codes.

SBA disagreed with both of these recommendations. In its response to  the first recommendation, SBA stated that it is not responsible for the  oversight of other agencies’ contracting officers and therefore is not in a  position to implement the corrective actions. With respect to the second  recommendation, SBA stated that adding such controls to FPDS-NG  would further complicate the WOSB program and increase contracting  officers’ reluctance to use it. SBA also stated its preference for focusing  its efforts on ensuring that contracting officers select the appropriate  NAICS code at the beginning of the award process.

In our review, we also found several issues with WOSB program set- asides being awarded under ineligible NAICS codes. Our analysis of  FPDS-NG data on all obligations to WOSB program set-asides from the  third quarter of fiscal year 2011 through the third quarter of fiscal year  2018 found the following:

3.5 percent (or about $76 million) of WOSB program obligations were  awarded under NAICS codes that were never eligible for the WOSB  program;

10.5 percent (or about $232 million) of WOSB program obligations  made under an EDWOSB NAICS code went to women-owned  businesses that were not eligible to receive awards in EDWOSB- eligible industries; and

17 of the 47 federal agencies that obligated dollars to WOSB program  set-asides during the period used inaccurate NAICS codes in at least  5 percent of their WOSB set-asides (representing about $25 million).

According to SBA officials we spoke with during this review, WOSB  program set-asides may be awarded under ineligible NAICS codes  because of human error when contracting officers are inputting data in  FPDS-NG or because a small business contract was misclassified as a  WOSB program set-aside. They characterized the extent of the issue as  “small” relative to the size of the FPDS-NG database and said that such  issues do not affect the program’s purpose. Rather than review FPDS-NG  data that are inputted after the contract is awarded, SBA officials said that  they have discussed options for working with GSA to add controls  defining eligible NAICS codes for WOSB program set-aside opportunities  on FedBizOpps.gov—the website that contracting officers use to post  announcements about available federal contracting opportunities.  Adding controls to this system, officials said, would help contracting  officers realize as they are writing the contract requirements that they  should not set aside contracts under the WOSB program without  reviewing the proper NAICS codes. However, SBA officials said that the  feasibility of this option was still being discussed and that the issue was  not a high priority. For these reasons, according to officials, SBA’s  updated oversight procedures described in the 2017 SOP and the Desk  Guide do not include a process for reviewing WOSB program set-aside  data in FPDS-NG to determine whether they were awarded under the  appropriate NAICS codes.

Further, as of November 2018, the WOSB program did not have targeted  outreach or training that focused on specific agencies’ use of NAICS  codes. As noted previously, in March 2016, SBA updated its WOSB  program training materials to address NAICS code requirements in  response to a 2015 SBA OIG recommendation. In fiscal year 2018, SBA  conducted three WOSB program training sessions for federal contracting  officers, including (1) a virtual learning session, (2) a session conducted  during WOSB Industry Day at the Department of Housing and Urban  Development, and (3) a session conducted during a Department of  Defense Small Business Training Conference. However, with the  exception of the virtual learning session, these training sessions were  requested by the agencies. SBA officials did not identify any targeted  outreach or training provided to specific agencies to improve  understanding of WOSB NAICS code requirements (or other issues  related to the WOSB program).

Congress authorized SBA to develop a contract set-aside program  specifically for WOSBs and EDWOSBs to address the  underrepresentation of such businesses in specific industries. In addition,  federal standards for internal control state that management should  design control activities to achieve objectives and respond to risks and to  establish and operate monitoring activities to monitor and evaluate the  results. Because SBA does not review whether contracts are being  awarded under the appropriate NAICS codes, it cannot provide  reasonable assurance that WOSB program requirements are being met  or identify agencies that may require targeted outreach or additional  training on eligible NAICS codes. As a result, WOSB contracts may  continue to be awarded to groups other than those intended, which can  undermine the goals of and confidence in the program.

Federal Contracts to WOSB Set-Asides Remain Relatively Small, and Stakeholders Discussed Various Aspects of Program Use

The Percentage of Obligations to Women- Owned Small Businesses under the WOSB Program Increased Slightly since 2012

Federal dollars obligated for contracts to all women-owned small  businesses increased from $18.2 billion in fiscal year 2012 to $21.4 billion  in fiscal year 2017. These figures include contracts for any type of good  or service awarded under the WOSB program, under other federal  programs, or through full and open competition. Contracts awarded to all  women-owned small businesses within WOSB-program-eligible industries  also increased during this period—from about $15 billion to $18.8 billion,  as shown in figure 1. However, obligations under the WOSB program  represented only a small share of this increase. In fiscal year 2012,  WOSB program contract obligations were 0.5 percent of contract  obligations to all women-owned small businesses for WOSB-program- eligible goods or services (about $73.5 million), and in fiscal year 2017  this percentage had grown to 3.8 percent (about $713.3 million) (see fig.  1).

From fiscal years 2012 through 2017, 98 percent of total dollars obligated  for contracts to all women-owned small businesses in WOSB-program- eligible industries were not awarded under the WOSB program. Instead,  these contracts were awarded without a set-aside or under other, longer- established socioeconomic contracting programs, such as HUBZone, the  SDVOSB, and 8(a). For example, during this period, dollars obligated to  contracts awarded to women-owned small businesses without a set-aside  represented about 34 percent of dollars obligated for contracts to all  women-owned small businesses in these industries (see fig. 2).

As shown in table 1, six federal agencies—DOD, DHS, Department of  Commerce, Department of Agriculture, Department of Health and Human  Services, and GSA—collectively accounted for nearly 83 percent of the  obligations awarded under the WOSB program from the third quarter of  fiscal year 2011 through the third quarter of fiscal year 2018, with DOD  accounting for about 49 percent of the total.

Contracting officers’ use of sole-source authority was relatively limited,  representing about 12 percent of WOSB program obligations from  January 2016 through June 2018. In fiscal year 2017—the only full fiscal  year for which we have data on sole-source authority—about $77 million  were obligated using sole-source authority. The share of sole-source  awards as a percentage of total WOSB program set-asides also varied  considerably by quarter—from as low as 5 percent in the third quarter of  2016 to as high as 21 percent in the first quarter of 2017 (see fig. 3).

Stakeholders Discussed Various Issues Related to WOSB Program Usage

We spoke with 14 stakeholder groups to obtain their views on usage of  the WOSB program. These groups consisted of staff within three federal  agencies (DHS, DOD, and GSA), eight contracting offices within these  agencies, and three third-party certifiers. Issues stakeholders discussed  included the impact of sole-source authority and program-specific NAICS  codes on program usage. Stakeholders also noted the potential effect of  other program requirements on contracting officers’ willingness to use the  program, and some suggested that SBA provide additional guidance and  training to contracting officers.

Sole-source authority. Participants in 12 of the 14 stakeholder groups  commented on the effect of sole-source authority on WOSB program  usage. Staff from 4 of the 12 stakeholder groups—including three  contracting offices—said that sole-source authority generally had no  effect on the use of the WOSB program. One of these stakeholders  believed contracting officers seldom use the authority because they lack  an understanding of how and when to use it; therefore, in this  stakeholder’s opinion, use of the WOSB program has not generally  changed since the authority was implemented. However, staff from two  contracting offices and one third-party certifier said that sole-source  authority was a positive addition because, for example, it can significantly  reduce the lead time before a contracting officer can offer a contract  award to a firm. Staff from one of these two contracting offices stated that  the award process can take between 60 to 90 days using sole-source  authority, compared to 6 to 12 months using a competitive WOSB  program set-aside. These staff also said that negotiating the terms of a  sole-source contract is easier, from a contracting officer’s perspective,  because they can communicate directly with the firm. As discussed  previously, SBA officials we interviewed said that adding sole-source  authority to the WOSB program made the program more consistent with  other existing socioeconomic set-aside programs, such as 8(a) and  HUBZone.

The remaining five stakeholder groups that discussed the effects of  WOSB sole-source authority described difficulties with implementing it.  Specifically, representatives from DHS, DOD, and one third-party certifier  said that executing sole-source authority under the WOSB program is  difficult for contracting officers because rules for sole-source authority  under WOSB are different from those under other SBA programs, such as  8(a) and HUBZone. For example, the FAR’s requirement that contracting  officers justify, in writing, why they do not expect other WOSBs or  EDWOSBs to submit offers on a contract is stricter under the WOSB  program than it is for the 8(a) program. Further, staff from one contracting  office noted that justifications for WOSB set-asides must then be  published on a federal website. In contrast, contracting officers generally  do not need to prepare and publish a justification under the 8(a)  program. According to staff from another contracting office, it may be  difficult to find more than one firm qualified to do the work under some  WOSB-eligible NAICS codes, but contracting officers would still have to  conduct market research and explain why they do not expect additional  offers in order to set the contract aside for a WOSB.

Program-specific NAICS codes. Participants in 13 of the 14 stakeholder  groups we interviewed commented on the requirement that WOSB  program set-asides be awarded within certain industries, represented by  NAICS codes. For example, two third-party certifiers we interviewed  recommended that the NAICS codes be expanded or eliminated to  provide greater opportunities for WOSBs to win contracts under the  program. Another third-party certifier said that some of its members focus  their businesses’ marketing efforts on industries specific to the WOSB  program to help them compete for such contracts.

Representatives from GSA and DHS made comments about limitations  with respect to the WOSB program’s NAICS code requirement. Staff we  interviewed from three contracting offices made similar statements,  adding that the NAICS codes limit opportunities to award a contract to a  WOSB or EDWOSB because they are sufficient in some industry areas  but not others. All five of these stakeholder groups suggested that NAICS  codes be removed from the program’s requirements to increase  opportunities for WOSBs.

Conversely, staff from five other contracting offices we interviewed  generally expressed positive views about the WOSB program’s NAICS  code requirements and stated that eligible codes line up well with the  services for which they generally contract. Finally, SBA officials noted that  there are no plans to reassess the NAICS codes until about 2020.  However, SBA officials also stated that the NAICS code requirements  complicate the WOSB program and add confusion for contracting officers  who use program, as compared to other socioeconomic programs that do  not have such requirements, such as HUBZone or 8(a).

Requirement to verify eligibility documentation. Staff from 7 of the 14  stakeholder groups we interviewed discussed the requirement for the  contracting officer to review program eligibility documentation and how  this requirement affects their decision to use the program. For example,  staff from one contracting office said that using the 8(a) or HUBZone  programs is easier because 8(a) and HUBZone applicants are already  certified by SBA; therefore, the additional step to verify documentation for  eligibility is not needed. GSA officials noted that eliminating the need for  contracting officers to take additional steps to review eligibility  documentation for WOSB-program set-asides—in addition to checking  the System for Award Management—could create more opportunities for  WOSBs by reducing burden on contracting officers. However, staff from  two contracting offices said it is not more difficult to award contracts under  the WOSB program versus other socioeconomic programs.

WOSB program guidance. Staff from 13 of the14 stakeholder groups we  interviewed discussed guidance available to contracting officers under the  WOSB program. Most generally said that the program requirements  outlined in the FAR are fairly detailed and help contracting officers  implement the program. According to SBA officials, SBA provides training  on WOSB program requirements to contracting officers in federal  agencies by request, through outreach events, and through an annual  webinar. SBA officials also said that the training materials include all the  regulatory issues that contracting officers must address.

However, representatives from two third-party certifiers described  feedback received from their members about the need to provide  additional training and guidance for contracting officers to better  understand and implement the WOSB program. Staff from two  contracting offices also expressed the need for SBA to provide additional  training and guidance. Staff from one of these contracting offices said that  the last time they received training on the WOSB program was in 2011,  when the program was first implemented. Staff in the other contracting  office added that the most recent version of a WOSB compliance guide  they could locate online was at least 6 years old. SBA officials estimated  that the WOSB compliance guide was removed from their public website  in March 2016 because it was difficult to keep the document current and  officials did not want to risk publishing a guide that was out-of-date. SBA  officials also said that there are no plans to issue an updated guide as the  FAR is sufficient.

The stakeholder groups also identified positive aspects of the WOSB  program. Specifically, staff from seven stakeholder groups believed that  the program provided greater opportunities for women-owned small  businesses to obtain contracts in industries in which they are  underrepresented. In addition, staff from three stakeholder groups  mentioned that SBA-led initiatives, such as the Small Business  Procurement Advisory Council and SBA’s co-sponsorship of the  ChallengeHER program, help improve collaboration between federal  agencies and the small business community and overall government  contracting opportunities for women-owned small businesses.

The WOSB program aims to enhance federal contracting opportunities for  women-owned small businesses. However, weaknesses in SBA’s  management of the program continue to hinder its effectiveness. As of  February 2019, SBA had not fully implemented comprehensive  procedures to monitor the performance of the WOSB program’s third- party certifiers and had not taken steps to provide reasonable assurance  that only eligible businesses obtain WOSB set-aside contracts, as  recommended in our 2014 report. Without ongoing monitoring and  reviews of third-party certifier reports, SBA cannot ensure that the  certifiers are fulfilling the requirements of their agreements with SBA, and  it is missing opportunities to gain information that could help improve the  program’s processes. Further, limitations in SBA’s procedures for  conducting, documenting, and analyzing eligibility examinations inhibit its  ability to better understand the eligibility of businesses before they apply  for and potentially receive contracts, which exposes the program to  unnecessary risk of fraud. In addition, given that SBA does not expect to  finish implementing the changes in the 2015 NDAA until January 1, 2020,  these continued oversight deficiencies increase program risk. As a result,  we maintain that our previous recommendations should be addressed.

In addition, SBA has not addressed deficiencies that the SBA OIG  identified previously—and that we also identified during this review— related to WOSB set-asides being awarded under ineligible industry  codes. Although SBA has updated its training and outreach materials for  the WOSB program to address NAICS code requirements, it has not  developed plans to review FPDS-NG data or provide targeted outreach or  training to agencies that may be using ineligible codes. As a result, SBA  is not aware of the extent to which individual agencies are following  program requirements and which agencies may require targeted outreach  or additional training. Reviewing FPDS-NG data would allow SBA to  identify those agencies (and contracting offices within them) that could  benefit from such training. Without taking these additional steps, SBA  cannot provide reasonable assurance that WOSB program requirements  are being met.

The SBA Administrator or her designee should (1) develop a process for  periodically reviewing FPDS-NG data to determine the extent to which  agencies are awarding WOSB program set-asides under ineligible NAICS  codes and (2) take steps to address any issues identified, such as  providing targeted outreach or training to agencies making awards under  ineligible codes. (Recommendation 1)

We provided a draft of this report to DHS, DOD, GSA, and SBA for review  and comment.  DHS, DOD, and GSA indicated that they did not have  comments. SBA provided a written response, reproduced in appendix II,  in which it agreed with our recommendation. SBA stated that it will  implement a process to review WOSB program data extracted from  FPDS-NG and certified by each agency. Specifically, through the  government-wide Small Business Procurement Advisory Council, SBA  plans to provide quarterly presentations to contracting agencies’ staff that  would include training and an analysis and review of the data. The  response also reiterated that SBA has contacted GSA to implement a  system change to FedBizOpps.gov that would prevent contracting officers  from entering an invalid NAICS code for a WOSB program set-aside.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time we will send copies of this report to appropriate  congressional committees and members, the Acting Secretary of DOD,  the Secretary of DHS, the Administrator of GSA, the Administrator of  SBA, and other interested parties. This report will also be available at no  charge on our website at http://www.gao.gov.

If you or your staff have any questions concerning this report, please  contact me at (202) 512-8678 or shearw@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made major contributions to  this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

This report examines (1) the extent to which the Small Business  Administration (SBA) has implemented changes to the Women-Owned  Small Business Program (WOSB program) made by the 2015 National  Defense Authorization Act (2015 NDAA); (2) the extent to which SBA has  implemented changes to address previously identified oversight  deficiencies; and (3) changes in WOSB program use since 2011 and  stakeholder views on its use, including since the 2015 implementation of  sole-source authority.

To describe the extent to which SBA has implemented changes to the  WOSB program made by the 2015 NDAA, we reviewed relevant  legislation, including the 2015 NDAA; related proposed regulations; and  SBA documentation. We reviewed comment letters on the advance notice  of proposed rulemaking for the new WOSB program certification process  from three of the four SBA-approved third-party certifiers: the El Paso  Hispanic Chamber of Commerce, the U.S. Women’s Chamber of  Commerce, and the Women’s Business Enterprise National Council. To  ensure the accuracy of our characterization of the comment letters, one  staff member independently summarized the third-party certifiers’  comments on the advance notice, and a second staff member then  reviewed the results. We also interviewed SBA officials, including officials  from SBA’s Office of Government Contracting and Business  Development.

To respond to the second and third objectives, we conducted interviews  on SBA’s implementation and oversight of the WOSB program and its use  with SBA officials, three of the WOSB program’s four third-party certifiers,  three selected agencies (and three agency components within two of the  agencies), and a total of eight selected contracting offices within six  selected agencies or components. Using data from the Federal  Procurement Data System-Next Generation (FPDS-NG), we judgmentally  selected the three federal agencies and three components (for a total of  six federal agencies and components) because their WOSB program  dollar obligations (including competed and sole-source) were among the  largest or because we had interviewed them for our prior work.  Specifically, we selected the following six agencies or agency  components: the Department of Homeland Security (DHS) and, within  DHS, the Coast Guard; the Department of Defense (DOD) and, within  DOD, the U.S. Army and U.S. Navy; and the General Services  Administration (GSA). Within the components and GSA, we judgmentally  selected eight contracting offices (two each from Coast Guard, U.S.  Army, U.S. Navy, and GSA) based on whether they had a relatively large  amount of obligations and had used multiple types of WOSB program set- asides (competed or sole-source) to WOSBs or economically  disadvantaged women-owned small businesses (EDWOSB).

To address our second objective, we reviewed the findings and  recommendations in our October 2014 report and in audit reports issued  by the SBA Office of Inspector General (OIG) in May 2015 and June  2018. We also reviewed SBA documentation on the WOSB program,  including SBA’s 2017 Standard Operating Procedures and 2018 WOSB  Program Desk Guide, results from 2016 compliance reviews of the four  third-party certifiers, and SBA eligibility examinations from fiscal years  2012 through 2018. In addition, we analyzed FPDS-NG data on contract  obligations to WOSB program set-asides from the third quarter of fiscal  year 2011 through the third quarter of fiscal year 2018 to determine  whether set-asides were made using eligible program-specific North  American Industry Classification System (NAICS) codes. To conduct this  analysis, we compared contract obligations in FPDS-NG with the NAICS  codes eligible under the WOSB program at the time of the award for the  time frame under review. The WOSB program’s eligible NAICS codes  have changed three times since the program was implemented in 2011,  but the eligible industries have changed once. SBA commissioned the  RAND Corporation to conduct the first study to assist SBA in determining  eligible NAICS codes under the WOSB program. Based on the results of  the RAND study, SBA identified 45 four-digit WOSB NAICS codes and 38  four-digit EDWOSB NAICS codes, for a total of 83 four-digit NAICS  codes. WOSB and EDWOSB NAICS codes are different and do not  overlap. In December 2015, the Department of Commerce issued the  next study, which increased the total NAICS codes under the program to  113 four-digit codes, with 92 WOSB NAICS codes and 21 EDWOSB  NAICS codes (which became effective March 2016). Often, there is a  time lag between the effective date of NAICS codes and when they are  entered in FPDS-NG. Therefore, we did not classify a contract as having  an ineligible NAICS code if the code eventually became eligible under the  WOSB program. We also excluded actions in FPDS-NG coded other than  as a small business. These actions represented a small amount of  contract obligations—approximately $125,000. We compared SBA  information on its oversight activities and responses to previously  identified deficiencies, federal internal control standards, and GAO’s fraud  risk framework.

We assessed the reliability of FPDS-NG data by considering their known  strengths and weaknesses, based on our past work and through  electronic testing for missing data, outliers, and inconsistent coding in the  data elements we used for our analysis. We also reviewed FPDS-NG  documentation, including the FPDS-NG data dictionary, FPDS-NG data  validation rules, FPDS-NG user manual, prior GAO reliability  assessments, and relevant SBA OIG audit reports. Based on these steps,  we concluded that the data were sufficiently reliable for the purposes of  reporting on trends in the WOSB program and the use of sole-source  authority under the program.

To describe how participation in the WOSB program has changed since  2011, including since the 2015 implementation of sole-source authority,  we analyzed FPDS-NG data from the third quarter of fiscal year 2011  through the third quarter of fiscal year 2018. We identified any trends in  WOSB program participation using total obligation dollars set aside for  competitive and sole-source contracts awarded to WOSBs and  EDWOSBs under the program. We also compared data on obligations for  set-asides under the WOSB program with federal contract obligations for  WOSB-program-eligible goods and services to all women-owned small  businesses, including those made under different set-aside programs or  with no set-asides, to determine the relative usage of the WOSB program.  In our analysis, we excluded from WOSB program set-aside data actions  in FPDS-NG coded other than as a small business (representing  approximately $125,000) or coded under ineligible NAICS codes that  were never eligible under the WOSB program (representing  approximately $76.3 million).

To describe stakeholder views on WOSB program use, we conducted  semistructured interviews to gather responses from 14 stakeholder  groups. These groups consisted of staff within three federal agencies  (DHS, DOD, and GSA), eight contracting offices within these agencies,  and three third-party certifiers (selection criteria described above). One  person summarized the results of the interviews, and another person  reviewed the summary of the interviews to ensure an accurate depiction  of the comments. In addition, a third person then reviewed the  summarized results.

We conducted this performance audit from October 2017 to March 2019  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix II: Comments from the U.S. Small Business Administration

Appendix III: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Allison Abrams (Assistant  Director), Tiffani Humble (Analyst-in-Charge), Pamela Davidson,  Jonathan Harmatz, Julia Kennon, Jennifer Schwartz, Rebecca Shea,  Jena Sinkfield, Tyler Spunaugle, and Tatiana Winger made key  contributions to this report.

The Women-Owned Small Business program aims to give these businesses an edge when competing for federal contracts in industries where women have traditionally been underrepresented.

Each year, the Small Business Administration examines a sample of businesses that received contracts under the program to determine whether they were eligible to participate. In fiscal year 2017, SBA found about 40 percent of the businesses in its sample weren't eligible. Moreover, SBA doesn't use the examination results to improve its oversight.

We made a recommendation and reiterated previous ones to address this and other persistent problems we found.