Identity Theft: Improved Collaboration Could Increase Success of IRS Initiatives to Prevent Refund Fraud

IRS estimates that fraudsters attempted at least $14.5 billion in IDT tax refund fraud in tax year 2015. Since 2015, GAO's High-Risk List has included IRS's efforts to address IDT refund fraud. Starting with its March 2015 Security Summit, IRS has partnered with state tax administrators and tax preparation companies, among others, on initiatives aimed at better preventing and detecting IDT refund fraud.

GAO was asked to examine IRS's efforts to collaborate with these partners. This report, among other things, (1) describes actions taken to implement the ISAC and RRT, (2) evaluates the extent to which the ISAC pilot aligns with leading practices for pilot design, and (3) identifies actions, if any, that IRS could take to improve the ISAC pilot.

GAO reviewed planning and other documents on the initiatives. It interviewed IRS and state officials and industry and trade organization representatives, among others involved in the ISAC and RRT. GAO also conducted four non-generalizable focus groups with state and industry partners.

The Internal Revenue Service (IRS) launched an Identity Theft Tax Refund Fraud Information Sharing and Analysis Center (ISAC) pilot for the 2017 filing season. It aims to allow IRS, states, and tax preparation industry partners to quickly share information on identity theft (IDT) refund fraud. The ISAC pilot includes two components: an online platform run by IRS to communicate data on suspected fraud, and an ISAC Partnership, a collaborative organization comprised of IRS, states, and industry, which is intended to be the governance structure. As of November 2017, the ISAC had 48 members: 31 states (including full members and those receiving alerts only), 14 tax preparation companies, and 3 financial institutions. In addition, IRS is using a Rapid Response Team (RRT) in partnership with states and industry members to coordinate responses to IDT refund fraud incidents that pose a significant threat within 24 to 72 hours of being discovered. IRS deployed the RRT for six incidents in 2016 and once in 2017.

GAO found that the ISAC pilot aligns with key aspects of all five leading practices for effective pilot design GAO previously identified, but none fully. For example, IRS has worked to incorporate stakeholder input, but its message about the ISAC's benefits has not fully reached states. Further, IRS does not have criteria for assessing whether the pilot's objectives have been met. Without this assessment and better alignment with leading practices, IRS, its partners, and Congress will have difficulty determining the effectiveness of the pilot and whether to implement it more broadly.

IRS has taken actions to improve the ISAC pilot, but the ISAC Partnership does not have an outreach plan. While the ISAC Senior Executive Board limited industry participation to partners who participated in its Security Summit, the ISAC has obtained support from trade organizations. However, officials from almost all states represented in our focus groups noted that they either had not used, or were unfamiliar with, the ISAC-specific resources. While the ISAC Board has taken steps to engage stakeholders, the ISAC Partnership does not have an outreach plan to increase membership and improve states' and industry partners' understanding of the ISAC's benefits. Without such a plan, less effective collaboration is likely among stakeholders and opportunities to prevent IDT refund fraud may be missed.

GAO recommends IRS ensure (1) the ISAC better aligns with leading practices for effective pilot design, and (2) the ISAC Partnership develops an outreach plan to expand membership and improve understanding of the ISAC's benefits. IRS and the ISAC Board state and industry co-chairs agreed with the recommendations.

Identity theft (IDT) tax refund fraud is an evolving and costly problem that  causes hardship for legitimate taxpayers who are victims of the crime and  demands an increasing amount of the Internal Revenue Service’s (IRS)  resources. IDT refund fraud occurs when a refund-seeking fraudster  obtains an individual’s Social Security number, date of birth, or other  personally identifiable information (PII), and uses it to file a fraudulent tax  return seeking a refund. This crime burdens honest taxpayers because  authenticating their identities is likely to delay the processing of their  returns and refunds. IRS estimates that at least $14.5 billion in IDT tax  refund fraud was attempted in tax year 2015, of which it prevented at  least $12.3 billion (85 percent). Of the amount attempted, IRS estimated it  paid at least $2.2 billion (15 percent).

Given current and emerging risks, in 2015 we added IRS’s efforts to  address IDT refund fraud to our high-risk area for enforcement of tax  laws. This is part of our broader body of work on IDT refund fraud since  2014. We previously reported that IRS had undertaken substantial  research efforts to combat this problem, such as estimating the cost of  IDT refund fraud. The efforts also include examining the size of the  problem and evaluating whether IRS’s methods for authenticating  taxpayers provide reasonable assurance that the authentication  determination is accurate. Our work and associated recommendations  have helped IRS continue to adapt as it confronts new and evolving  schemes.

To further address IDT refund fraud, IRS held a Security Summit in March  2015, with representatives such as state tax administrators and industry  partners, including tax preparation and software firms and financial  institutions. Following the Summit, seven working groups were created to  combat IDT refund fraud in different ways. These working groups— composed of Security Summit partners—have collaborated on various  initiatives aimed at better detecting and preventing IDT refund fraud.

You asked us to examine IRS’s efforts to collaborate with state and  industry partners to combat IDT refund fraud. This report (1) describes  actions Security Summit partners are taking to implement two initiatives,  an Information Sharing and Analysis Center (ISAC) and a Rapid  Response Team (RRT); (2) evaluates the extent to which the ISAC pilot  aligns with leading practices for pilot design; and (3) identifies actions, if  any, that IRS could take to improve the ISAC pilot.

We selected the ISAC and RRT from among those initiatives identified in  the June 2016 IRS Commissioner’s Security Summit Update Report as  the focus of our review because of their importance, the potential for a  major effect on IDT refund fraud, and the timeline for planned actions.

To address each objective, we reviewed documents from IRS, the ISAC  Senior Executive Board (Board), the ISAC working group, and the  Information Sharing working group. These documents included meeting  minutes, planning documents, the biweekly ISAC dashboard, and weekly  ISAC updates from IRS’s contractor. We observed a training session  IRS’s contractor conducted for new ISAC members, and we received a  demonstration from the contractor of the ISAC online platform  capabilities. In addition, we conducted semistructured interviews with  IRS, state, and industry co-leads of the ISAC and Information Sharing  working groups; ISAC Board co-chairs; the outreach and metrics ISAC  Board subgroups; and trade organizations including the Federation of Tax  Administrators and American Coalition of Taxpayer Rights.

To further address all objectives, we conducted four focus groups in  March and April 2017—two sessions with states and two sessions with  industry partners. We randomly selected states from among those with an  official who participated in the ISAC or Information Sharing working  groups and from among those that had not been involved in either  working group. We selected industry partners from among those involved  in those working groups. We excluded from our sample states or industry  partners with whom we previously conducted—or planned to conduct—a  separate semistructured interview. We asked similar questions of each  focus group with some variation between state and industry groups. We  transcribed the focus group sessions and analyzed the data to identify  common themes and patterns. We used these sessions to provide  illustrative examples of state and industry perceptions of the  accomplishments of, and challenges to implementing, the ISAC and RRT.

The responses are non-generalizable and do not reflect opinions of all  states or industry partners.

To evaluate the extent to which the ISAC aligns with the five leading  practices for pilot design, we reviewed our prior work and compared IRS  actions against these practices and criteria. Our April 2016 report  describes the criteria we developed for evaluating pilot design and the  methodology we used to do so. Further, we compared IRS actions to  the internal control standard for having a plan to meet its objective. See  appendix I for additional details on the objectives, scope, and  methodology.

We conducted this performance audit from August 2016 to November  2017 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

Viewed broadly, IDT refund fraud is comprised of two crimes: (1) stealing  or compromising PII and (2) using stolen (or otherwise compromised) PII  to file a fraudulent tax return and collect a fraudulent refund. Figure 1  presents an example of how fraudsters may use stolen PII and other  information, real or fictitious (e.g., sources and amounts of income), to  complete and file a fraudulent tax return and receive a refund. In this  example, a taxpayer may alert IRS of IDT refund fraud. Alternatively, IRS  can detect IDT refund fraud through its automated filters that search for  specific characteristics as well as through other reviews of taxpayer  returns.

Information Sharing and Analysis Centers

In May 1998, Presidential Decision Directive 63 introduced and  promulgated the concept of ISACs, which help critical infrastructure  owners and operators protect facilities, personnel, and customers from  cyber and physical security threats and other hazards. ISACs typically  collect, analyze, and disseminate actionable threat information to their  members and provide members with tools to mitigate risks and enhance  resiliency. ISACs have been used in other sectors such as energy,  financial services, and surface transportation to facilitate coordination  between public and private entities. We have reported that ISACs have  developed diverse management structures and operations to meet the  requirements of their respective critical infrastructure sectors. Likewise,  we also have assessed federal support to fusion centers, information  sharing platforms between the government and the private sector that  prevent and respond to criminal and terrorist activity.

ISAC characteristics differ across various sectors; however, we have  reported common challenges—including information sharing—that need  to be addressed for an ISAC to be successful. Barriers to information  sharing may stem from practical considerations because the benefits of  sharing information are often difficult to discern, while the risks and costs  of sharing are direct and foreseeable. As a result, we have noted that it is  important to lower the practical risks of sharing information through both technical means and policies, and to develop internal systems that are  capable of supporting operational requirements without interfering with  core operations.

IRS’s Information Sharing and Analysis  Center Mission   The mission is to provide a secure platform  via a sustainable public/private partnership to  facilitate information sharing, consistent with  applicable law, and analytics necessary to  detect, prevent, and deter activities related to  stolen identity refund fraud.

IRS’s ISAC—the Identity Theft Tax Refund Fraud-Information Sharing  and Analysis Center— is intended to improve collaboration and  information sharing among IRS, states, and industry partners and began  as a pilot in January 2017. (See sidebar.) Two entities operate under the  ISAC umbrella. One entity is the ISAC Partnership, a collaborative  organization run jointly by IRS, states, and industry partners. The other  entity is the ISAC online platform, which is controlled by IRS and includes  an early warning alarm system that allows states and industry partners to  share information related to IDT refund fraud and schemes more quickly  to better defend against fraud.

Outside of the ISAC, four other efforts have supported information sharing  about potential IDT refund fraud for years.

Suspicious Filer Exchange: The Federation of Tax Administrators  (FTA) operates an online platform for states to share information— including record-level data—among themselves about suspected  fraud.

Industry Leads Program: This IRS-operated program requires tax  preparation companies to perform post-filing analysis and provide, on  a recurring and timely basis, information to IRS on IDT refund fraud  patterns and indices as a condition of electronically filing returns. IRS  then provides this information to states, which are to use the  information to bolster their fraud detection and prevention efforts.

External Leads Program: This IRS-operated program involves third  parties such as banks or other financial institutions providing  information to IRS about questionable refunds. If the questionable  refund is confirmed as fraudulent, IRS requests that the financial  institution return the refund.

Opt-In Program: IRS operates this program that allows financial  institutions to electronically reject suspicious refunds and return them  to IRS and indicate why the institution is rejecting the refunds.

The RRT, which began in the 2016 filing season, coordinates responses  to IDT refund fraud incidents that IRS, states, or industry partners believe  pose a significant and immediate threat to taxpayers or the tax system.  The Information Sharing work group is responsible for managing the RRT  and is led by one representative each from IRS, states, and industry. The  main component of the RRT process is a call among relevant IRS, state,  and industry partners to coordinate a response to the incident. IRS’s goal  is to convene the call within 24 to 72 hours after an incident is discovered.  The RRT process describes the next steps for the first 3 days after an  incident is identified.

The RRT process differs depending on whether the incident is reported  by IRS, a state, or an industry partner, based on the laws governing  information sharing discussed later in this report. For example, if a state  identifies an incident, the RRT process indicates that the state should  share that information—including Social Security numbers as  appropriate—with IRS and other states on the next business day and with  industry in the next 2 to 3 days. If IRS or an industry partner identifies an  incident, the RRT process indicates that IRS or the industry partner  should share relevant information in the next 2 to 3 days.

In the 2016 filing season, the RRT was deployed for six incidents. For  example, as we reported in January 2017, IRS announced in February  2016 that cybercriminals had stolen more than 100,000 e-file Personal  Identification Numbers (PIN) from an online tool. Stolen e-file PINs  could be used to file fraudulent federal tax returns.

IRS Has Taken Significant Actions to Facilitate Information Sharing through the ISAC and RRT

IRS implemented the ISAC in 2017 to facilitate information sharing among  IRS, state, and industry partners—subject to disclosure prohibitions—by  launching an online platform, establishing a governance structure, and  recruiting members. IRS and state officials and industry representatives  attributed increased trust and improved relationships to IRS’s efforts in  recent years. Additionally, IRS coordinated with state and industry  partners to establish the RRT in 2016, which has been initiated once thus  far in 2017.

IRS Actions to Implement the ISAC Include Launching the Online Platform and Recruiting Members

The ISAC online platform provides two capabilities—alerts and record- level data—which facilitate information sharing.

Alerts: This capability consists of alerts on potential IDT refund fraud  that have been identified by IRS, states, or an industry partner and  shared on the ISAC online platform. Alerts are available to all states  and Security Summit partners who sign a terms of use agreement.  Alerts include detailed information about identified schemes,  indicators of suspicious activity, and types of accounts targeted,  among other things. Alerts may also include anecdotal evidence from  ISAC members who have already been targeted by this scheme.

Record-level data and analysis: This capability consists of several  tools to facilitate IDT refund fraud prevention and detection, including  a secure data transfer tool that members can use to input IDT refund  fraud data and record-level data. Record-level data may include PII or  other details about suspected fraud. States and industry partners  share record-level data with the ISAC. However, according to IRS  officials, IRS does not due to legal restrictions. This part of the ISAC  also contains, among other things, analytic reports which identify, for  example, Internet Protocol (IP) addresses associated with potential  fraud. This space is only accessible to full ISAC members.

Information that is shared and available to be reviewed by various ISAC  stakeholders is controlled by disclosure laws within the Internal Revenue  Code. According to IRS officials, IRS does not contribute Federal Tax  Information to the ISAC because those data are protected from disclosure  under section 6103 of the Internal Revenue Code, which generally  prohibits IRS from disclosing tax returns or return information. Similarly,  IRS does not control or have ownership of any record-level data on the  ISAC. Instead, IRS receives record-level data directly from states and  industry partners through other channels such as the External Leads  Program. IRS can, however, still contribute alerts that do not include  record-level data.

Moreover, unless exempted, section 7216 of the Internal Revenue Code  prohibits disclosure or use of taxpayer information by preparers of returns  and imposes criminal penalties on knowing or reckless disclosure.  Disclosure of information from one preparer to another preparer or  disclosure to federal, state, or local officials to inform them of activities  that may constitute a crime is permitted by Department of the Treasury  (Treasury) regulation. As seen in figure 2, tax preparation companies— covered under section 7216 and referred to as industry 7216—have full  access to all of the information provided to the ISAC. However, financial  institutions—not covered under section 7216 and referred to as industry  non-7216—are not able to view record-level data submitted by, or  comingled with data from, tax preparation companies. Three of the 17  industry members of the ISAC are financial institutions—non-7216  entities—and therefore have this more limited view.

IRS contracted with a company to facilitate information sharing among  partners. The contractor developed and manages the online platform  and also analyzes data on IDT fraud, which it makes available to IRS’s  ISAC members. In addition, IRS developed a governance structure for the  ISAC. Figure 3 shows these and other key events.

Three of IRS’s goals for the ISAC when it launched in 2017 were to (1)  launch the online platform, (2) establish the governance structure, and (3)  recruit new members.

In terms of its first goal, as noted, the online platform became operational  January 23, 2017. IRS’s contractor provided ISAC members with training  on how to use the online platform and how to use the data visualization  tools. (See figure 4.) The data visualization tools include charts and  figures with data on trends in refund fraud. The tools are available to  members of the ISAC with the exception of financial institutions that  cannot view data visualization tools compiled with tax preparation  company data (as noted in figure 2 earlier).

The ISAC also established a community of practice (COP) that brings  together fraud analysts from IRS, states, and industry partners to share  leading practices. The intent is to encourage dialogue among staff  involved in implementing fraud prevention strategies. In our focus groups,  an industry official said that the COP has been a positive experience for  industry, but most state officials said they were not familiar with the COP.

In terms of establishing a governance structure, the ISAC Partnership is  governed by the ISAC Senior Executive Board (Board) that consists of 15  members, with 5 representatives each from IRS, states, and industry. The  Board is principally responsible for crafting mission or vision statements  for the ISAC Partnership, recommending ISAC operating procedures,;  and nominating new ISAC Platform participants and recommending the  removal of such participants, among other responsibilities. An IRS  executive official must approve any recommendation by the Board that  affects the online platform. The partnership also includes three  subgroups: metrics, outreach, and governance.

IRS also made progress on its goal of recruiting new participants. As of  November 2017, the ISAC had 24 full state members, 7 alerts-only state  members, 14 tax preparation company members, and 3 financial  institution members. An additional 7 states have membership pending. In  total, 38 states are members (either full members or those receiving only  alerts) or have membership pending. Goals moving into the 2018 filing  season include increasing the participation of current members, exploring  additional analytical capabilities, and establishing and refining  performance metrics.

Partners Attributed Improved Collaboration to the Security Summit and ISAC

In our focus groups, industry representatives said that they see ISAC  collaboration as critical to managing IDT threats. The ISAC is intended to  go beyond other efforts, most notably in that it brings IRS, states, and  industry together in equal partnership and allows for communication  among all stakeholders. IRS reports over 1.8 million leads submitted to  the ISAC from 14 partners. However, the number of leads does not reflect  their quality. Industry representatives we spoke with in our focus groups  said that they would like feedback from IRS on the usefulness of industry  leads so that they can adjust their fraud filters and provide more accurate  leads.

These comments about the usefulness and quality of industry leads are  consistent with what our prior work has found on the value of external  leads. Specifically, in 2014, we recommended that IRS take the following  actions on its External Leads Program:  1.  provide aggregated information on both the success of external leads  in identifying suspicious returns, and also emerging trends (pursuant  to section 6103 restrictions), and  2.  develop a set of metrics to track external leads by the submitting third  party.

IRS has taken steps to address these recommendations, including  developing timeliness metrics for managing leads and holding six  feedback sessions with financial institutions participating in the External  Leads Program. As of November 2017, we are following up with industry  members to determine if they consider the feedback accurate, timely, and  actionable. Without such feedback, the more than 600 external parties  participating in the External Leads Program do not know if the leads they  provide to IRS are useful and they may not be able to assess their  success in identifying IDT refund fraud or improve their detection tools.

In the focus groups, both state officials and industry representatives said  the relationship among IRS, states, and industry has improved as a result  of increased collaboration over the last several years. As of November  2017, the ISAC had 48 members. Further, IRS officials said they think  trust and the relationship between all parties has and is continuing to  improve. Likewise, in the focus groups, industry officials cited benefits of  improved coordination from the Security Summit. For example, one  industry representative cited IRS’s pushing out communications faster  because of the Security Summit, while another noted that participation in  the summit has made IRS officials more accessible.

However, in focus groups, a few state officials noted that because IRS is  compartmentalized, they have found their interactions with IRS to be  inconsistent. For example, these state officials reported some IRS units  are more responsive than others and that information sometimes is not  shared among IRS units.

IRS Established the RRT in 2016 and Initiated the RRT Process Once in the 2017 Filing Season

As part of establishing the RRT, IRS outlined the responsibilities of IRS,  states, and industry to respond to significant IDT refund fraud incidents.  As noted earlier in this report, the RRT was activated six times in 2016.  IRS initiated the RRT once in the 2017 filing season for a data breach  related to the Department of Education.

In March 2017, IRS and the Department of Education responded to  security concerns and removed access on https://www.fafsa.gov and  https://www.StudentLoans.gov to IRS’s Data Retrieval Tool—the online  process through which student financial aid applicants obtain their  family’s tax information. IRS suspects that fraudsters used personal  information obtained elsewhere to access the Data Retrieval Tool in an  attempt to access tax information, particularly adjusted gross income.

As of April 6, 2017, IRS reported that fewer than 8,000 fraudulent returns  from this incident had been filed, processed, and issued refunds, but IRS  estimated that about 100,000 taxpayers may have been affected. The  Data Retrieval Tool was taken offline while IRS and the Department of  Education made updates and will not be available for completing  applications for the current school year (2017-2018). As of November  2017, taxpayers could use the Data Retrieval Tool for completing financial  aid applications for the next school year (2018-2019). While IRS initiated  the RRT for this incident, an industry official said that the information  provided in the press release was more detailed than what was previously  provided to industry partners via the RRT.

The RRT is administered separately from the ISAC. According to IRS  officials, they intend to eventually integrate components of the RRT into  the ISAC to further streamline information sharing. Specifically, IRS  envisions the ISAC serving as the primary mechanism for states and  industry partners to report and escalate IDT refund fraud incidents by  facilitating communication among participants. IRS does not have a  timeline for this integration.

The ISAC Pilot Partially Aligns with Leading Practices for Pilot Design, but IRS Does Not Have a Plan to Improve Alignment

In 2016, we identified five leading practices for designing a well- developed and documented pilot program: (1): ensuring stakeholder  communication, (2) establishing objectives, (3) ensuring scalability, (4)  having an assessment methodology, and (5) developing a data-analysis  plan. These practices enhance the quality, credibility, and usefulness of  evaluations and help ensure that time and resources are used effectively.  Each leading practice shares common elements but serves a unique  purpose and builds on the other. For example, four of the five leading  practices recommend either establishing criteria for assessing whether  the pilot’s objectives have been met or developing a data plan necessary  for effectively evaluating the pilot.

While the ISAC pilot is in nascent stages, IRS has taken steps that  partially align with key aspects of all five leading practices. (See figure 5.)

Ensure appropriate two-way stakeholder communication: In 2016,  we reported that it is critical that agencies identify who the relevant  stakeholders are and communicate early and often to address their  concerns and convey the initiative’s overarching benefits.

IRS’s efforts mostly aligned with this practice because IRS included  stakeholder input during the design, implementation, and preliminary  stages of the data-gathering and assessment phases of the pilot. IRS,  through the ISAC working group and the Board, communicated with  stakeholders before, during, and after forming the ISAC. Such  communication helped ensure that stakeholders were engaged and that  their views were understood and incorporated. For example, in 2016,  IRS’s contractor conducted a preliminary assessment and interviews to  compile and present stakeholder views and aspirations for the ISAC. This  process included meeting with state officials and industry partners about  ISAC preferences, suggestions, concerns, and risks. According to the IRS  ISAC Executive Official, ahead of the ISAC launch, IRS established  several mechanisms to ensure ongoing stakeholder input, including  coordinating with both state and industry trade organizations, including  the FTA and the American Coalition of Taxpayer Rights, to gain their  endorsement.

IRS and its contractor also solicited feedback at conferences, such as  FTA’s annual conferences. During a 3-day fraud simulation exercise  hosted by IRS’s contractor, participants discussed partner actions, needs,  and processes to inform the ISAC’s development. Additionally, IRS  conducted a stakeholder analysis which documented stakeholders’  engagement in the ISAC Partnership. This is intended to inform the  development of the ISAC communications plan. Finally, the ISAC’s  Partnership governance structure, which includes representatives from  states and industry, helps facilitate communication among stakeholders.

Despite these efforts, IRS’s message about the ISAC’s benefits has not  fully reached states. In our focus groups, a few state officials reported  they are unclear about the benefit of the ISAC. To help improve  communication, the Board invited relevant trade organizations to  participate in its July Board meetings. IRS officials reported that the  message about the benefits of the ISAC may not have initially reached  states because it took time to build trust among state and industry  partners. FTA confirmed that states may not have understood the benefits  of working with IRS and industry partners and were wary of joining the  ISAC. Further, IRS officials said that some trade organizations that  endorsed the ISAC had differing views about the organization of the  ISAC—such as who should be invited to participate—which made it  challenging for IRS to effectively garner support. A few states reported in  our focus groups that FTA’s endorsement was important to their decision  to join the ISAC. Until IRS further communicates the ISAC’s benefit to  current and potential stakeholders, IRS and the ISAC Board may face  challenges in reaching their goal of increasing robust participation in the  ISAC. We discuss how IRS can improve its outreach to state and industry  partners later in this report.

Establish well-defined, appropriate, clear, and measureable  objectives: In our 2016 report, we found that well-formulated objectives  help ensure that appropriate evaluation data can be collected from the  outset of the pilot so that data are available for measuring performance  against clear goals and standards. Broad objectives should be translated  into specific researchable questions that articulate what will be  assessed. Additionally, we have reported that agencies should establish  measurable goals for determining when the pilot progresses from one  stage to the next to improve their ability to evaluate the success of the  pilot.

IRS’s efforts mostly aligned with this leading practice. For example the  ISAC’s charter sets forth objectives, which include (1) exchanging  information among participants, (2) providing a forum for real-time  responses to fraud schemes, and (3) promoting strategies to detect and  prevent fraud. In February 2017, the Board established the metrics  subgroup to assess the performance of the ISAC and develop metrics.  The Board noted that metrics are essential for showing the value added  by the ISAC compared to other efforts. The ISAC Roadmap, a planning  document that outlines three developmental phases over 4 years, shows  that IRS and the Board have considered an implementation plan, as well  as how the online platform might evolve in the areas of program  operations, infrastructure, analytics, and partner engagement.  Additionally, IRS’s contractor anticipated and developed risk mitigation  strategies to handle scenarios that might arise before, during, and after  the ISAC’s launch and interfere with reaching the pilot’s objectives.  Finally, ahead of ISAC’s launch, the contractor refined key operational  attributes to help define ISAC’s full desired capabilities.

However, IRS has not translated its objectives into specific, researchable  questions that articulate what will be assessed. For example, one of the  ISAC’s objectives is to facilitate the exchange of information among  members. While IRS closely monitors members’ use of the ISAC, IRS  does not have performance goals, such as desired participation levels, or  a plan to assess progress towards those goals, such as members’ usage  of ISAC data and tools. These are needed to ensure that appropriate  evaluation data are collected during the pilot. Furthermore, IRS does not  have measurable goals to determine when the pilot should progress to full  implementation.

In the early stages of a new program or initiative within a program,  evaluation questions tend to focus on program process—on how well  authorized activities are carried out and reach intended recipients. We  have previously reported that common evaluation questions include the  following:  Is the program being delivered as intended to the targeted recipients?

Have any feasibility or management problems emerged?

What progress has been made in implementing changes or new  provisions?

According to IRS officials, the ISAC pilot is still in early stages; they did  not know what to expect the first year but knew they wanted to focus on  building trust and, therefore, did not set goals for participation. However,  we have previously reported that without well-defined, appropriate, clear,  and measurable objectives, it will be difficult to ensure appropriate  evaluation data are collected and available to measure performance  against the objectives and goals. In short, it will be difficult for IRS to  know whether it achieved its objectives. Without knowing this, IRS will  have difficulty justifying investing additional resources.

Ensure scalability of pilot design: The purpose of a pilot is generally to  inform a decision on whether and how to implement a new approach in a  broader context. Identifying criteria or standards for identifying lessons  about the pilot will help inform an agency’s decisions about scalability and  when to integrate pilot activities into overall efforts. We previously  reported that the criteria and standards should be observable and  measureable events, actions, or characteristics that provide evidence that  the pilot objectives have been met.

IRS’s efforts in designing the ISAC partially aligned with this leading  practice. First, IRS identified and integrated lessons learned into its pilot.  For example, ahead of ISAC’s launch, IRS’s contractor identified potential  capabilities of the ISAC based on lessons learned from four ISACs from  other industries and a 2-day collaborative session in summer 2015. In  February 2017, 1 month after the ISAC’s launch, the Board established  the metrics subgroup to develop evaluation criteria to determine the  extent to which the pilot objectives have been met. According to ISAC  Board officials, the metrics subgroup is developing and testing metrics  that the ISAC Board expects to use beginning in the 2018 filing season.  The metrics are designed to measure participation in the ISAC,  contribution of data or information to the ISAC, and the effectiveness of  the data or information provided.

IRS also took steps to improve the ISAC pilot design, which will help it  scale the pilot in the future. For example, in May 2017, IRS’s contractor  presented lessons learned from the 2017 filing season, including what  was accomplished, what should be changed in future filing seasons, and  areas for future attention to consider how well the lessons learned can be  applied when the pilot is scaled up. The contractor’s presentation also  outlined recommendations from a May 2017 independent assessment of  the ISAC, including the current status of each recommendation and  actions needed to implement them. In addition, during the July 2017 ISAC  Board meeting, IRS’s contractor discussed lessons learned, and the IRS  ISAC Executive Official discussed takeaways thus far from standing up  the ISAC. Finally, IRS took steps to establish criteria for assessing the  pilot’s performance, but these steps are primarily related to participation,  access, and data contribution requirements.

IRS does not have criteria that would inform decisions about the ISAC’s  scalability, including when it is appropriate to include more state and  industry members, how to identify additional members, or how to expand  the functionalities of the online platform. For example, IRS has yet to  articulate the criteria to determine the appropriate time frame for the ISAC  to remain in the pilot stage and does not have a plan to decide how and  when the ISAC will move from the pilot stage into full implementation.  However, IRS officials have said that the ISAC will likely continue in pilot  phase through the 2018 filing season.

According to IRS officials, IRS had prioritized other activities and is now  turning its attention to plans for scaling the pilot. Without measurable  evaluation criteria that provide evidence that the ISAC pilot objectives  have been met, the Board will have difficulty assessing the ISAC’s  performance and making decisions about scalability.

Clearly articulate an assessment methodology: In 2016, we reported  that key features of an assessment methodology include a strategy for  comparing the pilot’s implementation and results with other efforts; a clear  plan that details the type and source of the data necessary to evaluate  the pilot; and methods for data collection, including the timing and  frequency.

While IRS’s efforts minimally aligned with this leading practice, it has  taken some steps to clearly articulate its assessment methodology. For  example, according to the IRS ISAC Executive Official, IRS plans to  evaluate the extent to which the revenue protected by the ISAC pilot  compares to existing fraud detection and prevention efforts, including the  External Leads Program. To help accomplish this, IRS took preliminary  steps to collect and track metrics related to ISAC’s performance and  compare ISAC’s efforts against other mechanisms to combat fraud. For  example, IRS’s contractor collects and disseminates program metrics and  ISAC analytics weekly, including the total number of members, leads,  alerts, and Internet Protocol (IP) addresses. This is intended to help  assess progress in expanding the ISAC and identifying fraud. In addition,  the metrics subgroup started comparing ISAC leads against information  collected from the states as part of its effort to assess ISAC data quality.

However, IRS has not completed an assessment methodology and data  gathering strategy that outlines the type and source of data necessary to  evaluate the pilot to assess the progress in achieving each of the ISAC’s  objectives, including whether the ISAC successfully facilitates the  exchange of information and helps detect and prevent fraud. IRS also  does not have a strategy for comparing the pilot’s implementation and  results with other efforts. For example, while IRS officials expect to  determine federal revenue protected by the ISAC and compare that to  other efforts, IRS has not formalized this plan and IRS officials do not  expect to start until at least October 2017, when the needed data become  available. Additionally, according to IRS’s ISAC Executive Official, state  and industry partners—who are important stakeholders in the ISAC—may  not be able to track dollars protected through the ISAC. As a result, IRS  may only know the federal dollars protected, while the amount protected  at the state level may remain unknown. This makes it more difficult to  communicate the potential benefits to states. Furthermore, the ISAC  could be collecting additional data to better meet its objectives. While  quantifying federal dollars protected is a key indicator of the ISAC’s  success, that metric alone will not demonstrate the ISAC’s benefit and  effectiveness.

Without a documented strategy to compare the ISAC pilot to other efforts  and a methodology that details the type and source of data necessary to  evaluate the pilot—beyond the federal dollars protected by the ISAC that  would otherwise have been undetected—IRS may find it difficult to  assess the effectiveness of the pilot, identify areas for improvement, and  demonstrate its capabilities compared with other efforts.

Develop a data-analysis plan: In conjunction with a clearly articulated  assessment methodology, a detailed data-analysis plan identifies who will  analyze the data as well as when and how data will be analyzed to  assess the pilot’s performance and draw conclusions about how to  improve procedures moving forward. As we previously reported, the  results will show the successes and challenges of the pilot, and in turn,  how the pilot can be incorporated into broader efforts.

While IRS’s efforts minimally aligned with this leading practice, it has  taken some steps to measure performance at the activity level. For  example, IRS worked with its contractor to regularly track and report  engagement metrics; user statistics; and analytics on alerts, leads, and  device IP addresses, which at times are categorized and aggregated.  (See figure 4 earlier in this report for an example of the ISAC data  visualization tool with illustrative data.) IRS’s contractor also surveyed  ISAC members to better gauge user experience with alerts and what  participants found to be most valuable on the online platform. In response  to other recommendations to develop metrics for measuring ISAC’s  performance and success, the contractor’s May 2017 ISAC evaluation  outlined actions, including beginning to track recommended metrics and  exploring means of quantifying the benefit.

However, IRS has not formalized the plan to determine the amount of  revenue protected nor has it developed a detailed data-analysis plan to  determine how the ISAC pilot’s performance will be tracked. The ISAC’s  metrics subgroup reported that it is working to develop preliminary  performance metrics to benchmark the ISAC pilot’s progress. It  acknowledged that metrics and a detailed analysis plan are essential to  demonstrate the ISAC’s benefit. The subgroup reported it is in the  process of developing them. Without a detailed data analysis and  evaluation plan that identifies data sources and criteria, IRS cannot fully  determine or demonstrate the pilot’s performance and challenges. As a  result, IRS, its partners, and Congress will have difficulty determining the  ISAC’s effectiveness and whether IRS should expand the pilot.

IRS officials said they are still learning about the five leading practices for  pilot design, and as noted, the ISAC at least partially aligns with each  one. According to internal control standards in the federal government, an  agency should formulate plans to achieve its objectives in order to meet  them. Without such a plan to inform decisions about the ISAC’s benefits  and performance, IRS, its partners, and Congress will have difficulty  determining the effectiveness of the pilot and whether to proceed with full  implementation.

The ISAC Board Should Develop an Outreach Plan to Improve the Pilot

IRS took actions to improve the ISAC pilot, including waiving the  requirement for states to contribute data. However, IRS does not have an  outreach plan to increase membership or inform states about the ISAC’s  benefits.

IRS Waived the Data Contribution Requirement for 2017 and Improved Collaboration with Endorsing Organizations

IRS officials determined that requiring participating states to contribute  data on suspected fraud may be a potential barrier and limit participation  in the ISAC. Therefore, IRS waived the data contribution requirement for  the first year and one state subsequently contributed data to the ISAC in  the 2017 filing season. However, as of October 2017, 5 states had  contributed data and 8 states had submitted 29 alerts. In our focus  groups, officials from a few states reported they were concerned about  the data contribution requirement and were unsure if they had the  resources to contribute such data and did not fully understand the terms  of the data contribution requirement. IRS officials attribute the low data  contribution this year to it taking time to build trust among partners. The  ISAC Board sought to reframe the discussion about data contribution and,  in July 2017, changed the language to describe data contribution as a  data/information opportunity.

Endorsing organizations are another potential tool to increase  participation in the ISAC. Five trade organizations—American Coalition of  Taxpayer Rights, Council for Electronic Revenue Communication  Advancement, Computer and Communications Industry Association, the  Free File Alliance, and FTA—are supporting the ISAC Partnership as  endorsing organizations. According to IRS, endorsing organizations  provide additional support for the ISAC concept and are uniquely  positioned to serve as links between the ISAC and the sectors they  represent. While they are not ISAC members and therefore cannot  access the online platform, their role is important to build connections  between stakeholders. However, according to FTA officials, IRS did not  effectively leverage FTA to communicate the benefits to states during the  first year of the pilot, but IRS and the ISAC Board have since taken  important steps to improve collaboration.

FTA endorsed the ISAC in February 2017 and, in our focus groups, both  state and industry officials said the endorsement was important for  securing more widespread state participation. According to FTA, IRS did  not incorporate its feedback about the probable response from states to  the ISAC, which FTA officials believe may have resulted in a lower-than- expected rate of participation by states in the early months of the ISAC.  According to IRS officials, IRS attempted to work with endorsing  organizations while standing up the ISAC online platform and received  comments from FTA and an industry trade organization that reflected  different interests and priorities. According to IRS officials, IRS attempted  to find a middle ground. More recently, the Board attempted to better  engage endorsing organizations by including them in a July 2017 meeting  about planning the next steps for the ISAC.

Taxpayer Data Safeguards Determine Access to Information Shared in the ISAC

IRS, states, and industry partners have all faced data safeguarding  challenges to participating in the ISAC. For example, IRS is unable to  share taxpayer or record-level data in the ISAC due to the section 6103  safeguards discussed earlier in this report. In a June 2017 report to  Congress, the Electronic Tax Administration Advisory Committee  (ETAAC) recommended IRS identify, analyze, and mitigate barriers that  preclude IRS from sharing information in the ISAC. IRS officials said  that IRS not sharing information in the ISAC limits the full benefit of the  ISAC. While the ISAC is designed to be a three-pronged collaboration  between IRS, states, and industry, because IRS does not view or  contribute record-level data, such data only flows between states and  industry. This limits the full value of the ISAC. Further, it may be  challenging for the ISAC partnership to meet a key goal of increasing  participation among state and industry members if a key stakeholder in  the partnership is unable to fully participate. IRS officials said the agency  is considering options to allow it to participate more fully in the ISAC.  Specifically, IRS included a request for a legislative change to section  6103 in a report to Treasury. This request is an important step to enable  the ISAC to be an effective information sharing and collaboration tool.

Likewise, some states faced legal hurdles to joining the ISAC. According  to FTA, while it outlined potential concerns about those hurdles in a  memo to state legal counsels, it expected those would be manageable for  states.

Furthermore, some industry partners face difficulties in accessing the  ISAC’s online platform. As previously mentioned and shown in figure 2,  tax preparation companies—covered under section 7216 and referred to  as 7216 industry partners—have full access to all of the information  provided to the ISAC. However, financial institutions—not covered under  section 7216 and referred to as non-7216 industry partners—have limited  access to information in the ISAC. According to IRS officials, IRS is  considering a request from financial institutions to amend regulations  under section 7216 to allow them greater access to the ISAC.

ISAC Partnership Has Not Developed an Outreach Plan to Improve State and Industry Partners’ Participation

In the 2017 filing season, contribution levels from IRS, states, and  industry partners varied significantly. While IRS invited states and  Security Summit partners to participate, other stakeholders—such as  industry partners that are not members of the Security Summit—have not  been included. While IRS has taken steps to reach out to state and  industry partners, IRS and the ISAC Partnership have opportunities to  more fully engage stakeholders.

One challenge to state participation is that there has been a disconnect,  at times, between the ISAC Board’s and states’ perceptions of how the  ISAC can be used to prevent and detect fraud. For example, IRS views  the ISAC as the key tool for information sharing between IRS, states, and  industry partners in the future. However, officials from all states  represented in our focus groups noted that they either had not used, or  were unfamiliar with, the ISAC-specific resources—such as the data  visualization tools shown previously in figure 4. These are intended to  help users identify IDT refund fraud trends more broadly. Moreover,  officials from a few states reported IRS already sends more data on  suspected fraud through other channels than they can effectively process  with their current resources.

IRS is working to quantify the benefits of the ISAC, which could help  enhance states’ understanding. The ISAC Board is working with IRS’s  research organization to quantify the refund fraud averted and federal  dollars protected by analyzing Treasury receipts. According to IRS, it is  working with ISAC state members to communicate the value of the ISAC  to their leadership and share key activities, as appropriate, to enable their  continued involvement. IRS and the ISAC Board also took several steps  to inform states and members of industry—both members of the ISAC  and non-members—about the benefits of the ISAC. For example, IRS’s  contractor provided training to users of the ISAC to demonstrate the  platform’s functionality and tools. In addition, IRS officials presented  information about the ISAC at conferences with tax industry partners.

Relatedly, ETAAC recently recommended that IRS encourage greater  participation in the ISAC by stakeholders involved in tax administration.

In addition to inviting states to join the ISAC, IRS invited industry partners  who were members of the Security Summit to join. Security Summit  industry partners account for the majority of tax returns IRS accepts using  a paid preparer or tax software. The ISAC Board limited industry  participation in the ISAC Partnership to Security Summit partners  because it was concerned about securely authenticating new members  and scaling up the size of the pilot to accommodate additional  participants.

Furthermore, although three ISAC members are non-7216 financial  institutions, IRS does not consider banks or credit unions—both of which  cash refund checks—to be fully represented in the ISAC. IRS officials  said they were focused on engaging tax preparation companies and  building trust among existing stakeholders. In June 2017, ETAAC  recommended that IRS should address expanding the participation of  financial institutions in the ISAC, as well as in other efforts.

Although the ISAC Partnership does not have an outreach plan, such a  plan could, for example, address how to expand ISAC membership or the  disconnect between the benefits identified by the ISAC Board and how  states perceive the ISAC can be used to prevent and detect fraud in their  states. According to IRS officials, the ISAC Partnership has not  developed a plan yet because it has been focused on other priorities.

Project management standards state that when an entity is planning a  project—that is, a temporary endeavor to create a unique product,  service, or result—it is important to define relevant activities and  determine the scope, sequence, and schedule of those activities, among  other things. In addition, federal Standards for Internal Control in the  Federal Government state that federal agencies should establish plans to  help ensure goals and objectives—such as increasing participation in the  ISAC—can be met. Additionally, internal control standards state that  documentation of agency decisions and activities is important because it  provides a means to retain organizational knowledge, mitigate the risk of  having that knowledge limited to a few personnel, and communicate that  knowledge to external parties, as appropriate. Furthermore, we have  reported that without developing a user outreach plan, an agency risks  being unable to provide services to its users where they need them  most. For the ISAC, this could mean less effective collaboration among  stakeholders or missed opportunities to prevent IDT refund fraud.

IRS has taken important steps to improve its ability to respond to the  ongoing challenge of IDT refund fraud. Among these efforts, the ISAC  and RRT show promise for increasing information sharing and  collaboration among IRS, states, and industry to help detect and prevent  IDT refund fraud and coordinate responses to fraud incidents. The ISAC  pilot goes beyond existing fraud information sharing efforts and has  strengthened collaboration among stakeholders. While IRS has taken  actions that partially align with key aspects of five leading practices for  effective pilot design, its actions do not fully align with any of the  practices. Further, IRS has not developed criteria for assessing whether  the pilot’s objectives have been met. Without this assessment and better  alignment with leading practices for pilot design, IRS, its partners, and  Congress will have difficulty determining the effectiveness of the pilot and  whether and when to proceed with full-scale implementation.

The benefit of the ISAC can only fully be realized when there is robust  participation among stakeholders. However, officials from all states  represented in our focus groups noted that they either had not used, or  were unfamiliar with, the ISAC-specific resources. Part of the issue is that  IRS has not effectively communicated the benefits of the ISAC to states,  so they can better understand how the ISAC will help them combat IDT  refund fraud. Developing an outreach plan to broaden membership to  additional states, non-Security Summit members of industry, and financial  institutions would further promote stakeholders collaborating and sharing  fraud information.

We are making the following two recommendations to IRS:  The Acting Commissioner of Internal Revenue should ensure that the  Information Sharing and Analysis Center (ISAC) pilot better aligns with  leading practices for effective pilot design. This should include establishing criteria for assessing whether the pilot’s objectives have  been met before making decisions about its scalability and whether,  how, and when to when to proceed to full implementation; and developing a data analysis plan that identifies data sources and  criteria necessary for effectively evaluating the pilot.  (Recommendation 1)

The Acting Commissioner of Internal Revenue should ensure that the  ISAC Partnership develops an outreach plan to expand membership and  improve states’ and industry partners’ understanding of the ISAC’s  benefits. (Recommendation 2)

We provided a draft of this report to IRS and the co-chairs of the ISAC  Board for comment. In written comments reproduced in Appendix II, IRS  agreed with both recommendations. IRS reported it will be finalizing an  Identity Theft Tax Refund Fraud Pilot Management Plan to help it better  align the ISAC pilot with leading practices for pilot design. Additionally,  IRS reported it will work with the ISAC Board to ensure that the Board  develops an outreach plan to expand membership and improve states'  and industry partners' understanding of the ISAC's benefits.

In an email dated October 27, 2017, the ISAC Board state and industry  co-chairs also agreed with both recommendations and provided technical  comments which were incorporated, as appropriate.

We are sending copies of this report to the Chairmen and Ranking  Members of other Senate and House committees and subcommittees that  have appropriation, authorization, and oversight responsibilities for IRS.  We are also sending copies to the Acting Commissioner of Internal  Revenue, the Secretary of the Treasury, and other interested parties. In  addition, the report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-9110 or lucasjudyj@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The objectives of this engagement were to (1) describe actions Security  Summit partners are taking to implement an Information Sharing and  Analysis Center (ISAC) and a Rapid Response Team (RRT); (2) evaluate  the extent to which the ISAC pilot design aligns with leading practices;  and (3) identify actions, if any, that the Internal Revenue Service (IRS)  could take to improve the ISAC pilot.

We selected the ISAC and RRT from among those initiatives identified in  the June 2016 IRS Commissioner’s Security Summit Update Report as  the focus of our review because of their importance, the potential for a  major effect on IDT refund fraud, and the timeline for planned actions.  Although the External Leads Process and the Industry Leads Process are  discussed in this report, we did not select them for in-depth review.

To address all objectives, we reviewed IRS, ISAC Senior Executive Board  (Board), ISAC working group, and Information Sharing working group  documents. These included meeting minutes, planning documents, the  biweekly ISAC dashboard, and IRS’s contractor’s weekly ISAC updates.  We also observed a training session IRS’s contractor conducted for new  ISAC members and we received a demonstration of the ISAC online  platform capabilities, including the visualization tools. (See figure 4.) In  addition, we conducted semistructured interviews with IRS, state, and  industry co-leads of the ISAC and the Information Sharing working  groups; ISAC Board co-chairs; the outreach and metrics ISAC Board  subgroups; and trade organizations including the Federation of Tax  Administrators and American Coalition of Taxpayer Rights.

To further address all objectives, we conducted four focus groups in  March and April 2017—two sessions with states and two sessions with  industry partners:  1.  Five representatives from members of industry that were involved in  the ISAC or RRT.  2.  Seven representatives from members of industry that were involved in  the ISAC or RRT.  3.  Six officials from states randomly selected from among those with an  official who participated in the ISAC or Information Sharing working  groups.  4.  Five officials from states randomly selected from among those that  had not been involved in either working group.

We excluded from our focus group sample those states or industry  partners with whom we previously conducted—or planned to conduct— a  separate semistructured interview.

We asked similar questions for each focus group with some variation  between state and industry groups. We recorded and transcribed the  focus group sessions for review. We analyzed the focus group transcripts  to identify common themes, patterns, and comments. We used these  focus group discussions to provide illustrative examples of state and  industry perceptions of the benefits and challenges to implementing the  ISAC and RRT. However, the responses are non-generalizable and do  not reflect opinions of all states or industry partners. Because of concerns  about identifying which state and industry partners have been involved in  these fraud prevention efforts, we are not identifying the focus group  participants or the state officials and industry representatives that we  interviewed.

To evaluate the extent to which the ISAC aligns with the five leading  practices for pilot design, we reviewed our prior work and compared IRS  actions against these practices and criteria. Our April 2016 report  describes the criteria we developed for evaluating pilot design and the  methodology we used to do so. For this work, we evaluated each  subcomponent of the leading practices to determine if it met fully, mostly,  partially, or not at all with the criteria. Each of those assessments was  subsequently verified by another individual.

To identify actions, if any, that IRS could take to improve the ISAC pilot,  we assessed IRS and the ISAC Board’s efforts to implement the ISAC  pilot using internal control standards and performance management  standards.

We conducted this performance audit from August 2016 to November  2017 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

Appendix II: Comments from the Internal Revenue Service

Appendix III: GAO Contact and Staff Acknowledgments

In addition to the individual named above, the following staff made key  contributions to this report: Joanna Stamatiades, Assistant Director;  Melissa King, Analyst-in-Charge; Parul Aggarwal; Amy Bowser; Ann  Czapiewski; Robert Gebhart; Layla Moughari; and Cynthia Saunders.

The IRS estimates that would-be thieves used false identities to try to steal $14.5 billion in refunds in the 2015 tax year.

To address this high risk area, the IRS launched a pilot initiative in 2017 to help it, states, and tax industry members quickly share information. So far, 31 states and 17 industry groups are participating.

We reviewed this effort and assessed how well it aligned with 5 leading practices for pilot design. We recommended that the IRS and the initiative's Executive Board (1) take steps to improve the pilot's implementation, and (2) develop a plan to expand membership to more states and tax industry members.