Elections: Observations on Voting Equipment Use and Replacement

Much of the voting equipment acquired with federal funds after the enactment of the Help America Vote Act in 2002 may now be reaching the end of its life span, and some states and local election jurisdictions—which number about 10,300 and generally have responsibility for conducting federal elections—have or are considering whether to replace their equipment. GAO was asked to examine voting equipment use and replacement.

This report addresses (1) the types of voting equipment jurisdictions used for the 2016 general election and their perspectives on the equipment; (2) factors considered when deciding whether to replace equipment and replacement approaches in selected jurisdictions; and (3) stakeholder perspectives on how federal voting system guidelines affect replacing and developing equipment.

GAO surveyed officials from a nationwide generalizable sample of 800 local jurisdictions (68 percent weighted response rate) and all 50 states and the District of Columbia (46 responded) to obtain information on voting equipment use and replacement. GAO also interviewed officials from (1) five jurisdictions, selected based on population size and type of voting equipment used, among other things, to illustrate equipment replacement approaches; and (2) seven voting system vendors, selected based on prevalence of jurisdictions' use of equipment, type of equipment manufactured, and systems certified, to obtain views on federal voting system guidelines. These interviews are not generalizable, but provide insights into jurisdictions' and vendors' experiences.

Local election jurisdictions primarily used optical scan and direct recording electronic (DRE), also known as touch screen, equipment during the 2016 general election and were generally satisfied with voting equipment performance. Specifically, on the basis of GAO's nationwide generalizable survey of local election jurisdictions, GAO estimated that jurisdictions with 63 percent (from 54 to 72 percent) of the population nationwide used optical or digital scan equipment as their predominant voting equipment during the election, while jurisdictions with 32 percent (from 23 to 41 percent) of the population nationwide used DREs. In addition, the survey results indicated that accurate vote counting and efficiency of operation were top benefits experienced by jurisdictions for both types of equipment, and storage and transportation costs were a top challenge. Further, GAO estimated that jurisdictions with 93 percent (from 88 to 96 percent) of the population nationwide did not experience equipment errors or malfunctions on a very or somewhat common basis and jurisdictions with 96 percent (from 94 to 98 percent) of the population were very or generally satisfied with the performance of their equipment during the 2016 general election.

GAO identified four key factors that jurisdictions and states consider when deciding whether to replace voting equipment—(1) need for equipment to meet federal, state, and local voting system standards and requirements; (2) cost to acquire new equipment and availability of funding; (3) ability to maintain equipment and receive timely vendor support; and (4) overall performance and features of equipment. When replacing equipment, the five jurisdictions GAO selected for interviews used varying approaches based on their specific needs and resources. For example, Los Angeles County, California, which has a large and diverse electorate, is self-designing its own voting equipment and, according to officials, has incorporated a user-centered approach that prioritizes the needs and expectations of its voters. Lafayette County, Florida, which has a small population, joined a consortium of other small counties to help obtain funding and pool purchasing power to replace its equipment.

The state election officials we surveyed and the seven selected voting system vendors we interviewed, among other stakeholders, had varying perspectives on how the current voluntary federal voting system guidelines affected the replacement and development of voting equipment. These guidelines can be used to test and certify equipment to verify that it meets baseline functionality, accessibility, and security requirements. The stakeholders we surveyed or interviewed generally indicated that the guidelines and their associated testing processes provide helpful guidance for equipment developers, cost savings for states that do not have to duplicate federal testing, and assurance that certified equipment meets certain requirements. However, some of these stakeholders stated that aspects of the guidelines could discourage the development of innovative equipment and limit the choices of voting equipment on the market. The Election Assistance Commission (EAC), which is responsible for developing the federal guidelines, is updating them with stakeholder input and plans to issue a new version in late summer 2018.

GAO incorporated technical comments provided by the EAC and election officials from the selected local jurisdictions and their respective states as appropriate.

The voting equipment that is used to cast and count the ballots of millions  of voters nationwide is essential to our nation’s electoral process.  Challenges experienced during the 2000 presidential election with the  effectiveness and accuracy of some voting equipment for casting and  counting votes raised questions about existing voting equipment and  highlighted the need to replace aging equipment. To help address some  of the issues identified in the 2000 election, the Help America Vote Act  (HAVA) was enacted in 2002 and authorized over $3 billion in federal  funding over several fiscal years to assist state and local governments in  making improvements in election administration, such as replacing aging  voting equipment. Further, to help promote effective state and local  administration of federal elections, HAVA established the Election  Assistance Commission (EAC) as an independent federal commission  and, among other things, directed the Commission to develop voluntary  voting system guidelines against which voting equipment can be tested  and certified. According to HAVA, participation in the EAC testing and  certification program is optional but states may, by law or practice, require  some participation in this program, such as by formally adopting the  voluntary guidelines and making these guidelines mandatory in their  jurisdictions or requiring equipment to be tested by a federally accredited  laboratory. If vendors choose to have their voting equipment tested and  certified against the voluntary guidelines, their equipment must meet the  guidelines’ requirements in order to receive federal certification.

After the enactment of HAVA and the subsequent distribution of federal  funds to replace voting systems, many local election jurisdictions and  states acquired new voting equipment. Many states also incorporated the  use of the EAC’s voluntary voting system guidelines or its testing and  certification program into their own state-level requirements for approving  the use of equipment. However, studies have reported that much of the  voting equipment that was procured by state and local election  administrators with federal funds more than 10 years ago is now at or  approaching the end of its designed service life. Some state and local  election officials have noted that the use of aging equipment can  potentially affect how efficiently and accurately elections are carried out  and can require administrators to devote increasingly more resources and  effort to keep the equipment operational. Some states and local election  jurisdictions are considering whether they need to replace their voting  equipment and others have recently replaced their equipment or are in  the process of doing so.

The process for replacing voting equipment exists within an administrative  and regulatory framework in which the authority to regulate and carry out  elections is shared by federal, state, and local officials. For example,  states are responsible for administering elections; however, the local  election jurisdictions within each state are largely responsible for  managing, planning, and conducting elections, with about 10,300 local  election jurisdictions nationwide performing these duties. With respect to  voting equipment, this decentralization of the responsibility for  administering elections has led to the use of a diverse variety of  equipment, as well as different processes and approaches for carrying  out the responsibilities related to the selection, funding, implementation,  and maintenance of the equipment.

Since 2001, GAO has issued a number of reports on various aspects of  the election process describing the types of voting equipment used in  federal elections, how the performance of the equipment is measured,  and the federal voting system certification process, among other issues.  Given the potential challenges that can result from the use of aging voting  equipment, you asked us to obtain and examine information about the  voting equipment being used across the country, plans by states and  local election officials to replace voting equipment, and the EAC’s efforts  to update the voluntary voting system guidelines, among other things.  This report addresses the following questions:  1.  What types of voting equipment did local election jurisdictions use for  the 2016 general election, and what are jurisdiction perspectives on  equipment use and performance?  2.  What factors are considered when deciding whether to replace voting  equipment and what approaches have selected jurisdictions taken to  replace their equipment?  3.  What are selected stakeholders’ perspectives on how federal voting  system guidelines affect the replacement and development of voting  equipment, and what actions has the EAC taken to update the  guidelines?

To address our first objective, we conducted a web-based survey of  officials from a stratified random sample of 800 local election jurisdictions  nationwide. In total, we received 564 completed questionnaires for a  weighted response rate of 68 percent. In stratifying our nationwide  sample, we used a two-level stratified sampling method in which the  sample units, or jurisdictions, were broken out into rural and non-rural  strata. We surveyed the officials about the types of voting equipment  they used, various characteristics of the equipment used, their  perspectives on the benefits and challenges they experienced while using  the equipment, and how satisfied they were with its performance during  the election. Unless noted otherwise, the point estimates we report are  national-level point estimates representing the experiences, views, and  opinions of all local election jurisdictions nationwide with populations  greater than 2,500. We also provide some point estimates for jurisdiction  population subgroups, such as large jurisdictions (greater than 100,000  persons), medium jurisdictions (25,001 to 100,000 persons), and small  jurisdictions (2,501 to 25,000 persons), and jurisdictions that used a  particular type of voting equipment, in cases where statistically significant  differences exist between the subgroups that may be of interest. The  jurisdictions we surveyed were selected with probability proportionate to  population size, so rather than expressing the point estimates in terms of  the percentage of jurisdictions nationwide that had a specified  characteristic, we express the point estimates for the survey responses in  terms of the percentage of the population nationwide that resides within  jurisdictions that had a specified characteristic. Similarly, in instances  where we report point estimates for jurisdiction subgroups, we express  the point estimate in terms of the percentage of the population that  resides within jurisdictions of that respective subgroup that had a  specified characteristic.

To address our second objective, we used our local election jurisdiction  survey described above to obtain information from jurisdictions about the  factors they consider when determining whether to replace their voting  equipment. In addition to the local election jurisdiction survey, we also  conducted a web-based survey of the state-level election offices in the 50  states and the District of Columbia about issues pertaining to the states’  roles in selecting and acquiring voting equipment, including the factors  considered when determining whether to replace voting equipment. We  obtained responses from 46 of these offices, while 5 did not respond (a  90 percent response rate). For additional perspectives and context on the  factors considered when replacing voting equipment, we also reviewed  reports and studies about voting equipment and elections and interviewed  nine selected election subject matter experts, including representatives  from nongovernmental research and other organizations involved in the  field of election administration and voting equipment. We selected these  subject matter experts based on our review of reports and studies related  to voting equipment and their expertise and work in this area. Further, we  interviewed election officials from five local jurisdictions—Los Angeles  County, California; Travis County, Texas; Anne Arundel County,  Maryland; Lafayette County, Florida; and Beaver County, Utah—that  replaced their voting equipment between 2012 and 2016 or plan to  replace their equipment in time for the 2020 general election to learn  about the approaches and practices they used and obtain their  perspectives on the replacement process. We selected these jurisdictions  to obtain variation in, to the extent possible, population size, type of voting  equipment replaced and selected, state involvement in selecting and  funding voting equipment, and particular practices used to replace  equipment (e.g., self-designing equipment, leasing equipment), among  other factors. For each jurisdiction, we interviewed—on site or by  phone—local election officials, state election officials in the jurisdiction’s  state, and individuals who have served as poll workers at the jurisdiction’s  polling locations if applicable. While these five jurisdictions are not  representative of all local election jurisdictions nationwide that replaced or  plan to replace their voting equipment, they provide examples of various  approaches for replacing voting equipment and perspectives on key  issues related to replacing equipment. We corroborated various  information we obtained through these interviews by reviewing relevant  state statutes and documentation that these jurisdictions provided to us,  such as postelection reports, voting system studies, expenditure  summaries, and solicitations for vendor proposals to provide voting  equipment and services.

To address our third objective, we used responses to our survey of state  election officials and interviews with seven selected voting system  vendors and the nine selected subject matter experts mentioned above to  obtain perspectives on how federal voting system guidelines and their  associated testing and certification processes affect the replacement and  development of voting equipment. We selected the seven vendors based  on the prevalence of jurisdictions’ use of their equipment, type of voting  equipment manufactured, and systems certified, among other criteria.  The perspectives of the seven voting system vendors and nine subject  matter experts are not generalizable but provide examples of views on  the federal guidelines and their associated testing and certification  processes from a range of stakeholders. We also reviewed EAC and  National Institute of Standards and Technology (NIST) documents on  actions taken to update the guidelines and interviewed officials from the  EAC and NIST and the seven voting system vendors about their  involvement in and perspectives on these actions. See appendix I for  additional information on our scope and methodology.

We conducted this performance audit from June 2016 to April 2018 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

In the United States, election authority is shared by federal, state, and  local officials, and election administration is highly decentralized and  varies among state and local jurisdictions. Congressional authority to  regulate elections derives from various constitutional sources, depending  upon the type of election. Federal election laws have been enacted that  include provisions pertaining to voter registration, protecting the voting  rights of certain minority groups, and other areas of the elections process.  States regulate various election activities, including some requirements  related to these federal laws, but generally delegate election  administration responsibilities to local jurisdictions.

Congress has passed legislation in major functional areas of the voting  process. For example, HAVA includes a number of provisions related to  voting equipment and other election administration activities, including, for  instance, requiring at least one voting system equipped for persons with  disabilities at each polling place in federal elections. After HAVA was  enacted, Congress appropriated more than $3 billion for the EAC to  distribute to states to make election administration improvements, such  as the replacement of punch card and mechanical lever voting  equipment.

In addition to HAVA, federal laws have been enacted in other areas of the  voting process. For example, the Voting Rights Act of 1965, as amended,  contains, among other requirements, provisions designed to protect the  voting rights of U.S. citizens of certain ethnic groups whose command of  the English language may be limited. In accordance with the act,  covered states and jurisdictions must provide written materials—such as  ballots or registration forms—in the language of certain “language  minority groups” in addition to English, as well as other assistance, such  as bilingual poll workers.

State and Local Roles and Responsibilities

The responsibility for the administration of elections resides at the state  and local levels. States regulate various election activities, such as  absentee and early voting requirements and Election Day procedures, but  generally delegate election administration responsibilities to local  jurisdictions. Some states have mandated statewide election  administration guidelines and procedures that foster uniformity in the  ways local jurisdictions conduct elections, including the types of voting  equipment used. Other states have guidelines that generally permit local  election jurisdictions considerable autonomy and discretion in the way  they run elections. Although some states bear some election costs,  including those associated with voting equipment, local jurisdictions  generally pay for most aspects of election administration. Unless states  require otherwise, local jurisdictions generally have discretion over  activities such as training election officials and, in most states, over the  selection and purchase of voting technology. Among other things, local  election officials register eligible voters; design ballots; educate voters on  how to use voting technology; provide information on the candidates and  ballot measures; arrange for polling places; recruit, train, organize, and  mobilize poll workers; prepare and test voting equipment for use; and  count ballots.

States have established alternatives for voters to cast a ballot other than  at the polls on Election Day, including absentee voting and early voting.  All states and the District of Columbia have provisions allowing voters to  cast their ballots before Election Day by voting absentee, with variations  on who may vote absentee, whether the voter needs to provide an  excuse for requesting an absentee ballot, and the time frames for  applying for and submitting absentee ballots. Some states also permit  registered voters to apply for an absentee ballot on a permanent basis so  that those voters automatically receive an absentee ballot in the mail prior  to every election without providing an excuse or reason for voting  absentee. In addition to absentee voting, some states allow early in- person voting. In general, early voting allows voters from any precinct in  the jurisdiction to cast their vote in person without providing an excuse,  before Election Day either at one specific location or at one of several  locations. Further, three states and a number of local election jurisdictions  in other states conduct vote-by-mail elections, wherein ballots are  automatically sent to every eligible voter.

For in-person voting on Election Day, election authorities subdivide local  election jurisdictions into precincts. Voters generally cast their ballots at  the polling places for the precincts to which they are assigned by election  authorities. In addition, some states provide jurisdictions the discretion to  allow voters to cast their ballots at vote centers, which are polling places  at which any registered voter in the local election jurisdiction may vote on  Election Day, regardless of the precinct in which the voter resides.

Within the polling place, poll workers check in voters and determine their  eligibility to vote by verifying their registration using voter lists or poll  books—a list of individuals eligible to vote within the voting precinct or  local jurisdiction. After checking the voters in, poll workers direct them to  a voting booth to mark their electronic or paper ballots, and then voters  submit the ballots for counting. The manner in which votes are cast and  counted can vary depending on the voting method and technology  employed by the jurisdiction.

Following the close of the polls on Election Day, election officials and poll  workers complete steps such as securing equipment and ballots,  transferring paper ballots or electronic records of vote counts to a central  location for counting, and determining the outcome of the election. Votes  counted include those cast on Election Day, absentee ballots, early votes  (where applicable), and valid provisional ballots. While preliminary  results are available usually by the evening of Election Day, the certified  results are generally not available until a later date.

EAC Voluntary Voting System Guidelines and Testing and Certification Program

Overview of Voluntary Guidelines and Testing and Certification Program

The EAC has responsibility for developing the voluntary voting system  guidelines and overseeing the testing and certification of voting systems  based on these guidelines. The EAC works in conjunction with NIST and  the Technical Guidelines Development Committee (TGDC) to develop the  voluntary guidelines. According to the EAC, these guidelines are a set  of specifications and requirements against which voting systems,  including hardware and software, can be tested to receive a certification  from the EAC. According to NIST, the guidelines are intended to ensure  that federal testing provides assurance to state and local election officials  that the voting systems meet a defined set of requirements. The EAC  testing and certification program verifies that voting systems comply with  basic functionality, accessibility, and security capabilities established by  the voluntary guidelines. Typically, voting system vendors submit their  systems to the EAC for testing and certification and the systems are  evaluated by EAC-accredited voting system test laboratories against the  guidelines. These laboratories make recommendations regarding  certification to the EAC. According to the EAC, an EAC-certified voting  system means that the voting system has been tested by a federally  accredited test laboratory and complies with the guidelines.

Establishment of Federal Voting System Guidelines and Updates

According to the EAC, prior to its establishment and the creation of its  voluntary voting system guidelines, the first set of federal voluntary Voting  System Standards were adopted in 1990 by the Federal Election  Commission. The National Association of State Election Directors  voluntarily assumed the role of accrediting voting system test laboratories  and certifying voting systems to the federal standards. In 2002, the  Federal Election Commission adopted a new version of the federal  standards.

After the EAC’s creation, in 2005, the EAC developed and adopted the  third iteration of federal standards, in accordance with HAVA, and the  standards were renamed the Voluntary Voting System Guidelines  (VVSG). This third iteration of federal voting system guidelines was  referred to as the 2005 VVSG or VVSG 1.0, as it is called today.  According to the EAC, VVSG 1.0 increased security requirements for  voting systems and were intended to expand access, including  opportunities to vote privately and independently, for individuals with  disabilities. In 2006, the National Association of State Election Directors  terminated its voting system testing program and subsequently, in 2007,  the EAC launched its own testing and certification program. In March  2015, a fourth iteration of the voluntary guidelines was adopted by the  EAC, referred to as VVSG 1.1. According to the EAC, VVSG 1.1 clarified  the guidelines to improve testability by testing laboratories, among other  updates, and focused on areas that could be improved without requiring  significant changes to the testing and certification process. In January  2016, the EAC adopted an implementation plan for VVSG 1.1 whereby all  new voting systems being tested for certification would be required to be  tested against the VVSG 1.1 beginning on July 6, 2017. As of  November 2017, no voting systems have been certified using VVSG 1.1.

The EAC, NIST, and TGDC are in the process of developing the next  iteration of the voluntary guidelines (known as VVSG 2.0), and these  guidelines are expected to be issued in late summer 2018. Typically, a  lag exists between when guidelines are issued and when they are used  for testing and certification. EAC officials stated that it has generally taken  about 18 months before the guidelines are ready for use for testing voting  systems. This is due in part to the need for the voting system test  laboratories to be reaccredited to test to the new voluntary guidelines by  the EAC. According to EAC officials, after the guidelines are approved  for use, it typically takes 2 to 4 years before voting system vendors can  develop voting systems that are ready for testing and certification.

States’ Participation in the EAC Testing and Certification Program

Participation in the EAC testing and certification program is voluntary.  Each state determines its own standards for voting systems in statute or  administrative regulation, which can be based on the voluntary guidelines  established by the EAC. Specifically, most states require some level of  participation in the EAC testing and certification program as mandated by  their state laws or regulations. As of December 2017, 13 states require  federal certification of their voting systems, 24 states and the District of  Columbia require testing by a federally accredited laboratory or require  testing to federal voting system standards, and 13 states have no federal  requirements. Some states have their own voting system standards and  conduct their own testing and certification to these standards, either in  addition to or as an alternative to the federal voluntary guidelines.  Vendors that want to supply their voting systems to local jurisdictions and  states must comply with state requirements. See appendix II for federal  certification and testing requirements by state, including the associated  statutes and regulations we reviewed.

Local Election Jurisdictions Primarily Used Two Types of Voting Equipment, Monitored Such Equipment, and Were Generally Satisfied with Equipment Performance

Local Election Jurisdictions Primarily Used Optical/Digital Scan and Direct Recording Electronic Equipment during the 2016 General Election

According to our analysis of the predominant type of equipment used to  process the largest number of ballots during the 2016 general election,  jurisdictions using optical/digital scan equipment represented the largest  estimated share of the population nationwide, followed by jurisdictions  using direct recording electronic (DRE) equipment. Specifically, on the  basis of our local election jurisdiction survey, we estimate that  jurisdictions with about 63 percent of the population nationwide used  optical/digital scan equipment as their predominant voting equipment  during the election, while jurisdictions with an estimated 32 percent of the  population nationwide used DREs. Jurisdictions with less than 1 percent  of the population nationwide used paper hand-counted ballots. See  figure 1.

Within the optical/digital scan equipment category, the most widely used  model of optical/digital scan equipment was the precinct count  optical/digital scan, with jurisdictions having an estimated 46 percent of  the population nationwide using it as their predominant voting  equipment. Figure 2 shows the predominant types of voting equipment  that were used by jurisdictions during the 2016 general election, broken  out by model of equipment used.

While many jurisdictions predominantly used one type of voting  equipment, some reported using multiple types. Jurisdictions may choose  to use more than one type of equipment as a means to process different  types of ballots such as absentee or provisional or to provide accessibility  options for voters with disabilities. Overall, we estimate that jurisdictions  with about 59 percent of the population nationwide used only one type of  equipment during the 2016 general election, while jurisdictions with about  37 percent of the population nationwide used multiple types of equipment  during the election. Jurisdictions that used two types of equipment are  estimated to have about 30 percent of the population nationwide, while  those that used more than two types of voting equipment had  approximately 6 percent of the population nationwide. See figure 3 for  the types of voting equipment used.

Local Election Jurisdictions Monitored Equipment Performance in Various Ways

According to results from our survey of local election jurisdictions,  jurisdictions monitored the performance of their voting equipment during  the 2016 general election through a variety of methods, such as  equipment testing, performance measurement and tracking of  malfunctions, and postelection audits and recounts. Such monitoring can  provide information to jurisdictions about how their equipment is  functioning and help ensure the accuracy of the outcomes of elections  and address any identified issues or problems.

Results from our survey of local election jurisdictions indicate that the  extent to which jurisdictions tested their voting equipment varied by test  type. Key types of voting equipment testing include acceptance testing,  logic and accuracy testing, and parallel testing. Acceptance testing  verifies that new equipment or any equipment that has been outside  election administrators’ control (e.g., for repair) conforms to the purchase  agreements and is identical to equipment that was tested and certified by  state or federal testing organizations. According to our local jurisdiction  survey results, jurisdictions with an estimated 49 percent of the population  nationwide conduct acceptance testing of their equipment. Logic and  accuracy (also known as functional or readiness) testing is performed in  advance of an election to determine whether voting equipment will  function properly, such as displaying the correct ballot, collecting votes,  and tabulating results. Parallel testing is performed on Election Day by  running test votes cast with known results, then comparing the actual and  expected results. Of these two types of testing, according to our local  jurisdiction survey results, logic and accuracy testing was the most widely  performed type of testing as jurisdictions with 99 percent of the population  nationwide conducted such testing for the 2016 general election.  Jurisdictions with an estimated 37 percent of the population nationwide  conducted parallel testing.

Performance Measures and Reported Errors and Malfunctions

According to our local jurisdiction survey results, jurisdictions monitored  the performance of their predominant voting equipment during the 2016  general election using a variety of measures. Accuracy of the equipment  in counting votes was tracked, measured, or assessed by jurisdictions  having an estimated 87 percent of the population nationwide. Another widely monitored aspect of voting equipment performance was the  accuracy of the equipment in recording voter selections before counting— jurisdictions with 78 percent of the population nationwide tracked,  measured, or assessed that aspect. Overvotes and undervotes were  also widely used measures, with jurisdictions having about 63 and 64  percent of the population nationwide, respectively, tracking, measuring, or  assessing those measures.

According to the results of our local jurisdiction survey, most jurisdictions  did not experience extensive or widespread errors or malfunctions with  their equipment during the 2016 general election. We estimate that  jurisdictions with 93 percent of the population did not experience  equipment errors or malfunctions on a “somewhat” or “very” common  basis during the election. Of those that did experience equipment errors  or malfunctions of some type on a “somewhat” or “very” common basis,  the error or malfunction most frequently encountered was jams or  misfeeds. We estimate that this error or malfunction was experienced on  a “very common” basis by jurisdictions with about 1 percent of the  population nationwide and on a “somewhat common” basis by  jurisdictions with about 3 percent of the population nationwide. The next  most frequent error or malfunction experienced as a “very” or “somewhat”  common occurrence was that equipment response was sluggish or  slower than acceptable, which was experienced by jurisdictions with an  estimated 3 percent of the population nationwide.

State and local election officials also determined how their voting  equipment performed and verified election results by conducting  postelection audits and recounts. According to 35 out of 46 respondents  to our state survey, the state election agency or local election jurisdictions  in their states conducted postelection audits or targeted recounts of  results from the 2016 general election. On the basis of our local  jurisdiction survey, we estimate that jurisdictions with approximately 45  percent of the population nationwide conducted postelection audits or  targeted recounts. Among jurisdictions of different size, large  jurisdictions had a higher estimated share of their population within  jurisdictions that conducted postelection audits or recounts than did  medium or small jurisdictions. Specifically, jurisdictions with 82 percent of  the population within large jurisdictions conducted postelection audits or  recounts. In contrast, an estimated 55 percent and 37 percent of the  population within medium and small jurisdictions, respectively, was  represented by jurisdictions that conducted postelection audits or  recounts.

Local Election Jurisdictions Experienced Various Benefits and Challenges with Voting Equipment and Were Generally Satisfied or Very Satisfied with Equipment Performance Benefits and Challenges of Predominant Equipment Used

According to the results of our local election jurisdiction survey,  jurisdictions using the two main types of voting equipment (DRE or  optical/digital scan) experienced mostly similar benefits as a result of  using their respective type of predominant equipment. Table 1 shows  the top benefits experienced by jurisdictions according to the type of  predominant voting equipment used.

In addition to the benefits mentioned above, jurisdictions experienced  other benefits associated with using their respective type of predominant  voting equipment. For example, jurisdictions that had an estimated half or  more of the population within jurisdictions using each of the different  types of voting equipment also experienced the following benefits from  using their equipment:  Jurisdictions predominantly using DREs: accessibility for individuals  with disabilities or impairments, timely election night reporting, ease of  presenting lengthy ballots in a clear and understandable way,  protection and preservation of votes cast against potential non- cybersecurity related threats, and customer support and problem  resolution assistance from vendor.

Jurisdictions predominantly using optical/digital scan equipment:  timely election night reporting, ease of troubleshooting or resolving  equipment malfunctions during Election Day, preventing or alerting  voters of any overvotes or undervotes before ballot is cast, ability to  facilitate a postelection audit, security of equipment against outside  electronic hacking or intrusion, and ease of conducting routine  maintenance.

Jurisdictions also experienced challenges while using their predominant  voting equipment, although to a lesser extent overall than they  experienced benefits. Table 2 shows the top challenges experienced by  jurisdictions according to the type of predominant voting equipment used.

The next most frequently experienced challenges by jurisdictions were  the following (estimates with the values for the 95 percent confidence  intervals are shown in parentheses):  Jurisdictions predominantly using DREs: cost to maintain voting  equipment (an estimated 12 percent; 6, 19); cost to operate voting  equipment (8 percent; 3, 14); and ease of conducting routine  maintenance (7 percent; 2, 14).

Jurisdictions predominantly using optical/digital scan equipment: cost  to operate voting equipment (an estimated 11 percent; 7, 15);  preventing or alerting voters of any overvotes or undervotes before  ballot is cast (9 percent; 2, 23), and ease of connectivity with other  election administration systems (e.g., voter registration, election night  reporting) (9 percent; 2, 23).

Satisfaction with Predominant Voting Equipment

On the basis of our local election jurisdiction survey, we estimate that  jurisdictions with approximately 96 percent of the population nationwide  were very satisfied or generally satisfied with the performance of their  predominant voting equipment during the 2016 general election.  Specifically, we estimate that jurisdictions with approximately 70 percent  of the population nationwide were very satisfied with their voting  equipment’s performance and 26 percent were generally satisfied (see  fig. 4). Jurisdictions with about 2 percent of the population nationwide  were generally dissatisfied or very dissatisfied with the performance of  their predominant voting equipment.

When comparing satisfaction with the performance of their predominant  voting equipment used in the 2016 general election against the  performance of their predominant equipment used in the 2012 general  election, we estimate that jurisdictions with 67 percent of the population  nationwide were just as satisfied with their equipment’s performance in  2016 as in 2012, while 16 percent reported they were more satisfied (see  fig. 5). Among jurisdictions that used different predominant types of  equipment, jurisdictions that predominantly used optical/digital scan  equipment that were more satisfied with their equipment’s performance in  2016 had a larger estimated share of their population (20 percent)  compared to jurisdictions that predominantly used DRE equipment (4  percent).

Local Election Jurisdictions and States Consider Multiple Factors and Selected Jurisdictions Have Varying Approaches When Replacing Voting Equipment

Local Election Jurisdictions and States Consider Multiple Factors When Deciding Whether to Replace Voting Equipment

On the basis of our review of literature and studies, interviews with  election subject matter experts, and analysis of our local election  jurisdiction and state surveys, we identified four key factors and related  issue areas within them that jurisdictions and states consider when  deciding whether to replace voting equipment. After considering the  factors, jurisdictions may decide to replace their equipment or continue  using their existing equipment. The four key factors we identified are: (1)  the need for voting equipment to meet federal, state, and local voting  system standards and requirements; (2) the cost to acquire new  equipment and availability of funding; (3) the ability to maintain equipment  and receive timely vendor support; and (4) the overall performance and  features of voting equipment. In our local election jurisdiction and state  surveys, we asked election officials to rate issue areas related to each of  these factors as to how important they were when determining whether to  replace voting equipment and then rank the issue areas in terms of which  were “most important” in making the determination. Analysis of the  results of our surveys indicates that the 24 issue areas within the four  factors vary in their relative importance to jurisdictions and states when  determining whether to replace voting equipment.

Need for Voting Equipment to Meet Federal, State, and Local Voting System Standards and Requirements

The need for voting equipment to meet applicable federal, state, and local  voting system standards and requirements is a factor considered by local  election jurisdictions and states when determining whether to replace  equipment. At the federal level, HAVA generally requires that voting  equipment be accessible to individuals with disabilities. As discussed  earlier, HAVA also established the EAC which developed and maintains  the voluntary guidelines that voting equipment can be tested against to  receive federal certification. In turn, many states have established  requirements that voting equipment be federally certified or meet some or  all of the standards established by the federal guidelines. According to  election subject matter experts we spoke with, in addition to federal  requirements and standards, some states have imposed additional  requirements that voting equipment must meet or satisfy such as having  the capability to present all ballot issues and candidates on one page or  presenting ballots in multiple languages, for example.

We identified four issue areas related to this factor. Figure 6 shows the  importance local jurisdictions and state election officials attributed to the  various issue areas within this factor when determining whether to  replace voting equipment. For example, the need for equipment to meet  state and local requirements and standards was considered “very  important” by jurisdictions with 87 percent of the population nationwide  and as one of the three “most important” issue areas overall by  jurisdictions with 36 percent of the population nationwide. Among the  states, this issue area was considered as “very important” by 18 out of the  25 states that indicated having a role in determining whether to replace  voting equipment and as one of the three “most important” issue areas  overall by 7 out of the 25 states.

According to election subject matter experts we spoke with, the costs to  acquire new equipment and the availability of funding to pay those costs  is a key factor that jurisdictions and states consider when determining  whether to replace voting equipment. Acquiring new voting equipment  involves a variety of costs and expenses. For example, in addition to the  cost of the equipment itself, there can be other associated costs, such as  training for poll workers and elections staff on the new equipment and  voter outreach and education about the change in equipment, that may  be incurred as existing equipment is replaced. These related acquisition  and transition costs and expenses are incurred by the jurisdictions and  states, which in turn must obtain or allocate resources to cover those  costs.

We identified four issue areas related to this factor. Figure 7 shows the  importance local jurisdictions and state election officials attributed to  these issue areas when determining whether to replace voting equipment.  For example, the availability of state and local funds was considered “very  important” by jurisdictions with 62 percent of the population nationwide  and as one of the three “most important” issue areas overall by  jurisdictions with 18 percent of the population nationwide. Among the  states, this issue area was considered as “very important” by 20 out of the  25 states that indicated having a role in determining whether to replace  voting equipment and as one of the three “most important” issue areas  overall by 9 out of the 25 states.

Given the importance of funding for the acquisition of new voting  equipment and the assistance federal HAVA grants have previously  provided, we asked states and jurisdictions additional questions in our  surveys about their funding practices and the extent to which they have  HAVA grant funds remaining to acquire voting equipment. The results  from our surveys provided the following additional information about these  issues:

Use of local and state funding sources for acquisition of new voting  equipment: On the basis of our local election jurisdiction survey, we  estimate that, among various potential funding sources, jurisdictions  with 79 percent of the population nationwide obtain funds to acquire  new voting equipment through local general funds or budgets as a  direct appropriation. Additionally, we estimate that jurisdictions with  43 percent of the population nationwide use state financial assistance  or cost sharing as a source of funds for new equipment. According  to the results from our state survey, states have different levels of  involvement in providing funds for the acquisition of voting equipment.  Over half (24) of the 46 states that responded to our survey indicated  that they do not provide any financial assistance or cost sharing to  local jurisdictions for equipment acquisition, while 11 indicated that  they cover all acquisition costs. Eight states indicated that their state  provides some financial assistance or cost sharing with local  jurisdictions for equipment acquisition, while 2 states indicated a  different type of involvement in funding the acquisition of voting  equipment, such as covering only the costs of acquiring accessible  voting equipment.

Availability of HAVA funds: On the basis of our local jurisdiction  survey, we estimate that jurisdictions with 10 percent of the population  nationwide had HAVA funds remaining to apply toward the acquisition  of new voting equipment, with jurisdictions representing 6 percent of  the population only having enough HAVA funds to acquire a portion of  the equipment needed. Additionally, we estimate that jurisdictions  with 42 percent of the population nationwide had no HAVA funds  remaining while jurisdictions with 46 percent of the population did not  know whether they had any HAVA funds remaining.

Impact of lack of HAVA funds: Among jurisdictions that did not have  any HAVA funds remaining or only enough to buy a portion of the  equipment needed, jurisdictions with an estimated 36 percent of the  population indicated that the lack of HAVA funds had affected their  decisions regarding the replacement of voting equipment. Further,  jurisdictions with an estimated 57 percent of the population in this  subgroup (of jurisdictions that indicated that the lack of HAVA funds  affected their replacement decisions) delayed the replacement of  voting equipment while jurisdictions with 25 percent of the population  in this subgroup were not able to acquire the equipment that would  best meet their needs.

Ability to Maintain Equipment and Receive Timely Vendor Support

The ability of local election jurisdictions and states to maintain voting  equipment and receive timely vendor support is a factor considered when  determining whether to replace equipment, particularly as the equipment  ages. Election subject matter experts we spoke with noted the importance  of access to replacement parts for existing voting equipment as  something jurisdictions and states may consider when determining  whether to replace equipment. Without adequate access to replacement  parts and technical service, either from vendors or supplied by in-house  expertise, it can be difficult for jurisdictions and states to maintain their  current equipment at a satisfactory level.

We identified five issue areas related to this factor. Figure 8 shows the  importance local jurisdictions and state election officials attributed to  these issue areas when determining whether to replace voting equipment.  For example, the sufficiency of vendor support and problem resolution  was considered “very important” by jurisdictions with 81 percent of the  population nationwide and as one of the three “most important” issue  areas overall by jurisdictions with 7 percent of the population  nationwide. Among the states, this issue area was considered as “very  important” by 15 out of the 25 states that indicated having a role in  determining whether to replace voting equipment but no state considered  it as one of the three “most important” issue areas overall.

The overall performance and features, both of the existing voting  equipment and of potential replacement equipment, is also a factor  considered by local election jurisdictions and states when determining  whether to replace voting equipment. For example, jurisdictions and  states may consider the age of their current equipment and how well it is  performing, as well as how its performance compares to that of new  equipment available for acquisition. In addition, according to elections  literature we reviewed and election subject matter experts we spoke with,  jurisdictions and states may also take into account specific features new  voting equipment can provide that might better meet their needs. The  desired features may vary from jurisdiction to jurisdiction depending on  specific needs and circumstances, but such features may include an  enhanced ability to process a high volume of absentee ballots, capability  to present ballots in multiple languages, or ease for poll workers to set up  and for voters to use, for example.

We identified 11 issue areas related to this factor. Figure 9 shows the  importance local jurisdictions and state election officials attributed to  these issue areas when determining whether to replace voting equipment.  For example, the overall performance of the voting equipment was  considered “very important” by jurisdictions with 83 percent of the  population nationwide and as one of the three “most important” issue  areas overall by jurisdictions with 20 percent of the population  nationwide. Among the states, this issue area was considered as “very  important” by 18 out of the 25 states that indicated having a role in  determining whether to replace voting equipment while 4 out of the 25  states considered it as one of the three “most important” issue areas  overall.

Given the potential challenges local election officials have identified with  using aging or outdated equipment, in our local election jurisdiction  survey we asked jurisdictions when they first used their predominant  voting equipment. Based on their responses, we estimate that  jurisdictions with over half of the population nationwide used predominant  voting equipment in the 2016 general election that was first deployed  between 2002 and 2006 (see fig. 10) Jurisdictions with the next largest  estimated share of the population (28 percent) used equipment that was  first deployed between 2012 and 2016.

Approaches to Replacing Voting Equipment Varied across Selected Jurisdictions

The five local election jurisdictions we selected to include in our review  either replaced their voting equipment between 2012 and 2016 or plan to  replace their equipment in time for the 2020 general election. We  selected these jurisdictions to obtain variation in, to the extent possible,  population of jurisdiction, type of voting equipment replaced and selected,  and state involvement in selecting and funding voting equipment  replacement, among other factors. Table 3 summarizes information  related to voting equipment replacement across the five selected  jurisdictions.

These jurisdictions illustrate varying approaches that localities have used  or are using to replace their voting equipment based on their specific  needs, circumstances, and resources. For example,

Los Angeles County, California. The county has a large and diverse  electorate and is in the process of self-designing its own voting  system, which is expected to consist of ballot marking devices that  produce paper ballots to be tallied on central count digital scanners.  County officials stated that the current design concept for the new  equipment is intended to provide greater flexibility in administering  elections, provide a more user-friendly and accessible voting  experience, enhance accuracy and auditability, and could potentially  lower costs for system upgrades if developed as planned. For  example, according to officials, the ballot marking device is intended  to provide the ease of use of a touch screen interface, which would  incorporate features such as scrolling and tapping that are familiar to  voters who use mobile devices, and will include a headset, tactile  keypad, and other devices for voters with disabilities. It would also  allow the county to have ballots with multiple formats and a large  number of races.

The county’s process for developing and deploying its new voting  equipment began in 2009 and has five phases—(1) public opinion and  stakeholder baseline research, (2) establishment of voting system  guiding principles, (3) system design and engineering, (4)  manufacturing and certification, and (5) phased implementation.  According to officials, the county has taken a user-centered approach  to the design of the new voting equipment that prioritizes the specific  needs and expectations of the voters. The county is currently in the  manufacturing and certification phase and reported that about $19  million has been expended to develop the new voting equipment as of  December 31, 2017. County officials told us they plan to retain  ownership of the intellectual property rights of the new voting  equipment so that the system remains publicly owned and not  proprietary like traditional vendor equipment. The county plans to pilot  the new equipment in some early voting locations in 2019 and fully roll  it out in 2020.

Travis County, Texas. The county began its efforts to design its own  voting equipment based in part on findings and recommendations  from an election study group it convened in 2009. In 2012, it  developed a concept for a DRE with a voter-verified paper audit trail  that centered on system security, auditability, and the use of  commercial off-the-shelf technology. In September 2017, the county  announced that it had decided to no longer pursue building the voting  equipment because the proposals it received from vendors and other  organizations for developing key components of the equipment were  not sufficient to build a complete voting system, among other reasons.  According to county officials, the county plans to acquire either DREs  or ballot marking devices with precinct count digital scanners from a  voting system vendor with the goal that whatever equipment it  acquires incorporates some of the key features it had intended for its  self-designed equipment. For example, officials stated that the new  equipment must produce printed paper records that can be tallied and  connected with electronic voting records through an automated  process and allow for third party verification of results and better  postelection audits. They noted that they are prepared to work with  vendors to customize existing equipment to meet the county’s  requirements if needed. County officials estimate that the new  equipment will cost about $16 million and stated that acquisition will  be funded through local bonds. The county issued a request for  proposals for the equipment in November 2017 and plans to have it in  place for the 2020 election.

Anne Arundel County, Maryland. In 2016, the county replaced its  DREs with a system in which voters manually mark paper ballots and  insert them into precinct count digital scanners which then count  them. Maryland requires the use of uniform voting equipment in  polling places statewide and the state and counties each pay 50  percent of the costs of acquiring equipment. In 2007, Maryland  enacted a law that prohibited the use of a voting system unless the  State Board of Elections (SBE) determined that the system provides a  voter-verifiable paper record, thereby requiring the state’s DREs to be  replaced. According to Maryland SBE officials, state law specifically  required the purchase of precinct count scanners so the board did not  consider other types of voting equipment.

The SBE issued a request for proposals for the new voting equipment  in July 2014 and four vendors responded. The board formed an  evaluation committee to analyze the technical and financial details of  the proposals, and according to officials, the committee hosted a  public demonstration to collect feedback on the equipment under  consideration and worked with the University of Baltimore to perform  usability and accessibility testing on the equipment. The SBE decided  to lease rather than purchase the equipment for a number of  reasons. For example, officials said that leasing provided increased  flexibility to update or replace equipment more frequently and had  lower upfront costs. According to SBE officials, the current payment to  the vendor for leasing the digital scan equipment statewide is  approximately $1.1 million per quarter. SBE and Anne Arundel County  officials stated that deployment of the new equipment in the 2016  general election went smoothly with no significant challenges. The  state contracted with a third party vendor to conduct a postelection  audit of the 2016 general election by using independent software to  tally all digital ballot images. The audit confirmed the accuracy of the  election results. According to SBE officials, the new equipment’s  ability to capture and store digital images of the ballots made this type  of audit possible. Anne Arundel County officials stated that the ability  to conduct such an audit is one of the main benefits of the new  equipment.

Lafayette County, Florida. Lafayette County has a small population  and, in 2016, replaced its precinct count optical scan equipment with  precinct count digital scan equipment. The county formed a  consortium with 11 other counties in the state to help acquire its new  equipment. According to the county’s Supervisor of Elections,  having the consortium approach state officials as a group helped  secure HAVA funds to help the counties purchase the voting  equipment. In addition, he stated that being a part of the consortium  helped the counties negotiate a lower price for their equipment than  what they could have obtained individually because they pooled their  purchases and acquired a higher volume of machines. According to  the Supervisor of Elections, the consortium decided to purchase  precinct count digital scanners from the same vendor the counties had  used before because county staff were familiar with the vendor and  equipment, among other reasons. He stated that the total cost to  purchase Lafayette County’s new voting equipment was about  $70,000.

The Supervisor of Elections said that the digital scanners have  features that were an improvement over the county’s previous optical  scan equipment. For example, he told us that the new scanners have  more robust security features, such as locking panels, seals, and a  requirement for a passcode to access the system. He also noted  that the scanners digitally capture and store ballot images. The  Supervisor of Elections and the two poll workers we interviewed  stated that deployment of the new voting equipment went smoothly  and the county did not experience any challenges because the new  and previous equipment are both precinct count scanning systems.  According to the Supervisor of Elections, a postelection audit that was  conducted, in which the county manually tallied ballots from a  randomly selected race and precinct, found that the results were  accurate.

Beaver County, Utah. Beaver County has a small population and  previously used DREs with a voter-verified paper audit trail. In 2014,  Beaver County began conducting vote-by-mail elections and replaced  its DREs with central count digital scan equipment to support this  change. County officials said that, in 2014, they verbally requested  proposals for the new equipment from their current vendor and an  elections services company that the county had employed in 2012 to  provide training, systems testing, and other support for elections.  According to the Deputy Clerk, the county requested proposals from  these two entities because county officials were familiar with them and  were not aware of other vendors that might submit proposals. Officials  stated that the county received a proposal from the elections services  company, and selected the company because it was the only bid  received and the equipment the company sold met the county’s needs  and was federally certified. The county reported that the cost to  purchase the equipment was about $46,000. Officials said that they  are very satisfied with the performance of the new voting equipment.  They noted that conducting vote-by-mail elections and using central  count scanners allow them to administer elections from one location  on Election Day, which requires less time and resources than having  to manage multiple polling places. Officials also stated that the new  digital scanners are able to count a high volume of ballots in a short  period of time. According to officials, the county conducted two  postelection audits for the 2016 general election—one required by the  state and another that the county initiated. They reported that both  audits validated the election results.

See appendix V for additional details about voting equipment replacement  in our five selected jurisdictions, including the factors that influenced their  decisions to replace voting equipment; selection, acquisition, and  implementation of their equipment; and perspectives on the process.

Stakeholders Have Varying Views on How the Voting System Guidelines Affect Equipment Replacement and Development, and the EAC is Updating the Guidelines with Stakeholder Input

Stakeholders Provided Varying Perspectives on How the Current Voluntary Guidelines and Testing and Certification Processes Affect Replacing and Developing Voting Equipment

On the basis of our survey of state election officials and interviews with  officials from selected voting system vendors and subject matter  experts—representatives from nongovernmental research and other  organizations involved in the field of election administration—we found  that these stakeholders have varying perspectives on how the current  Voluntary Voting System Guidelines (VVSG 1.0 and VVSG 1.1) and their  associated testing and certification processes facilitated or posed  challenges to the replacement and development of voting equipment.  The states we surveyed and the other selected stakeholders we  interviewed primarily had experience with VVSG 1.0. As discussed  earlier, the VVSG 1.1 were issued in March 2015, but due to the time it  generally takes to implement updates to new guidelines, including  developing testing programs, among other things, no systems had been  certified under this version of the guidelines as of November 2017. One  vendor’s system underwent partial testing using VVSG 1.1 but the vendor  withdrew the system before the testing was completed.

Perspectives on How the Voluntary Guidelines Facilitate Replacing and Developing Voting Equipment

States and selected vendors and subject matter experts provided varying  perspectives on how aspects of the current voluntary voting system  guidelines and their associated testing and certification processes  facilitate the replacement and development of voting equipment.  Generally, stakeholders indicated that the guidelines and processes  provide assurance that new equipment meets certain requirements,  provide guidance for equipment developers, provide a model for state  standards, and provide cost savings for states that do not have to  duplicate federal testing. For example, 15 of the 26 state survey  respondents said the guidelines provide assurance that new voting  equipment meets baseline requirements related to security, functionality,  usability, accessibility, and privacy. One of these 15 state respondents  noted that if the EAC certified voting equipment against the federal  guidelines, he believes it meets the highest election standards and also  meets requirements set by his state. Another of these 15 state  respondents noted that voting equipment that has been tested using the  federal guidelines and certified by the EAC will have a higher level of  reliability than equipment that has not met these guidelines or been  certified by the EAC.

Subject matter experts from one nongovernmental organization noted that  states that establish their own voting system standards often use the  federal guidelines as a base to help develop their standards because the  federal guidelines have comprehensive requirements and are well vetted.  Experts from another nongovernmental organization said that the  guidelines establish a standard for voting equipment features and  performance, which may help small jurisdictions that want to acquire new  voting equipment but may not have the expertise to independently  evaluate the equipment. Further, officials from most of the vendors we  interviewed agreed that the federal standards serve as effective baseline  requirements. For example, officials from five of the seven vendors we  interviewed said that when they are developing voting systems, the  federal guidelines help them define the baseline standards that their  systems should meet, and five of the nine subject matter experts said the  federal guidelines provide baseline requirements.

Further, 4 of the 26 state survey respondents indicated that the current  voluntary guidelines help reduce the costs and resources needed for  states to test and approve new voting equipment. For example, one of the  4 state respondents reported that states do not have to rely on their own  voting system testing laboratories for all aspects of the testing and  certification of new voting equipment to meet state requirements because  most of the testing and certification relevant to state requirements has  already been done by EAC-accredited testing laboratories and the EAC.  The official noted that this allows the states to do less testing, which could  save them money.

Perspectives on How the Voluntary Guidelines Pose Challenges to Replacing and Developing Voting Equipment

The states we surveyed and selected vendors and subject matter experts  we interviewed also reported that aspects of the current voluntary voting  system guidelines and their associated testing and certification processes  could pose challenges to the replacement and development of voting  equipment in a number of ways. Specifically, some stakeholders indicated  that aspects of the guidelines and processes could discourage innovation  in equipment development, could limit the choices of voting equipment on  the market because the testing and certification processes take too long,  and could be costly for states and vendors. For example, officials  representing three of the seven vendors we interviewed said the current  federal guidelines may discourage innovation for new voting equipment  because they are too specific or overly prescriptive. Officials from one of  these three vendors said the current guidelines require a specific oval  size on the ballots, prescribing how tall and wide the oval should be.  Instead of such requirements, the officials said they would like the  guidelines to be more performance-based and state, for example, that  voters should be able to successfully mark a ballot a specified percentage  of the time. Further, officials from another vendor said that the current  guidelines are generally written for the purpose of testing and certifying  end-to-end voting systems rather than system components such as ballot  marking devices, which are generally developed by smaller vendors. As a  result, according to this vendor, smaller vendors may face challenges  getting new technology certified and into the market. EAC officials stated  that they recognize that the current guidelines should be more flexible  because specificity may limit innovation and they believe the updates to  the VVSG 2.0 should help address this issue.

In addition, some stakeholders said they believed that the voluntary  guidelines and associated testing and certification processes take too  long, and thus limit the choices of voting equipment on the market and  make it difficult to make improvements to existing equipment. For  example, officials from 8 of the 27 state survey respondents and three  subject matter experts said the guidelines and their respective processes  limit the number of voting systems that are available for acquisition. Three  of the 8 states and three subject matter experts said, in their view, the  EAC testing and certification process takes too long. In addition,  according to one subject matter expert, if a jurisdiction wants to make  changes to its existing voting equipment, such as incorporating new  software, it can be a difficult and lengthy process to certify the modified  equipment, and in some cases the entire system must be recertified.  Also, an official from one vendor said that the federal certification  processes are complicated, onerous, and time-consuming and they  discourage vendors from making modifications to their voting systems  even though the modifications might improve the systems. EAC officials  said they have heard from stakeholders that the certification process  takes too long but stated that this perception was more accurate in the  years immediately following the EAC’s issuance of the VVSG 1.0 in 2005.  They said that if voting equipment has been modified and is ready for  testing and there are no significant problems encountered during the  testing, certifying modifications should take a few weeks to a few months  to complete and full system testing and certification of new systems  should take about 6 to 9 months.

Further, officials from 4 of the 27 states that responded to our survey said  the EAC testing and certification process can be costly. One state  election official said that the cost of certification may discourage vendors  from developing new systems and pursuing EAC certification for their  systems, which could limit their ability to sell or supply their systems to  state and local election jurisdictions. In addition, this state election official  noted that costly federal certification of voting systems has limited the  voting equipment choices for election officials. Further, officials from one  vendor said that they submitted a new voting system for EAC testing and  certification and spent over $12 million before they learned that there  were significant issues with getting their system certified. According to  EAC officials, this was an uncommon occurrence that resulted from the  vendor submitting a system that needed additional work and was not  ready for certification. The vendor decided to withdraw its system from the  testing and certification process.

The EAC Is Updating the Voluntary Voting System Guidelines with Stakeholder Input and Plans to Issue the New Version in 2018

Shortly after the adoption of VVSG 1.1 in March 2015, the EAC, in  conjunction with NIST and the TGDC, began work to develop the next  iteration of the guidelines, VVSG 2.0, and anticipates issuing the new  version in late summer 2018. The EAC, NIST, and the TGDC have taken  actions to develop VVSG 2.0 that may address some of the issues with  the earlier iterations of the guidelines that were raised by stakeholders.  For example, they have established goals to guide the VVSG 2.0 development process, established working groups to inform the guidelines, and developed VVSG 2.0 high-level principles and guidelines.

Establishment of Voluntary Voting System Guidelines Development Goals and Working Groups

According to the EAC and NIST, in August 2014, the Future VVSG  Working Group, which consisted of officials from state and local election  offices, technical experts in such areas as security and disability, and  voting system vendors, among others, began work which culminated in  the creation of 12 goals to guide the development efforts for the voluntary  guidelines. One goal, for example, states that the guidelines’  requirements should be performance based and technology neutral. The  goal statement further elaborates that the guidelines should be free from  detailed descriptions of any technology, and that the guidelines should be  functional in nature so that they can more easily be redefined as  technology changes. Another development goal states that the voluntary  guidelines and its testing and certification processes should not impose  unanticipated cost burdens onto organizations. These goals are designed  to address some of the issues with the current voluntary guidelines  identified by the stakeholders we interviewed as posing challenges to the  replacement and development of voting systems, such as discouraging  innovation because they are too specific and discouraging vendors and  other voting system developers from pursuing EAC certification for their  systems because the process is potentially costly.

After the 12 goals for the voluntary guidelines were developed, the EAC  and NIST established a new process for developing the next guidelines  that is intended to allow for broader and more transparent stakeholder  involvement than prior guidelines’ development efforts. This new process  brings stakeholders together through a working group structure to  develop the guidelines. According to the EAC, the previous process did  not fully allow for stakeholder input or effectively leverage stakeholder  expertise in developing the guidelines because comments on the  guidelines were solicited from the Standards Board and external  stakeholders after most of the work had been done.

In 2015, the EAC and NIST established seven working groups to obtain  feedback and input from stakeholders early in the voluntary guidelines  development process. According to the EAC and NIST, the four  constituency and three election cycle working groups were created as a  public/private partnership to inform the development of the guidelines and  are composed of state and local election officials, representatives from  the federal and private sectors, members of standards bodies, EAC  committee members, academic researchers, and other interested  parties.

The working groups are led by EAC and NIST staff, and have more than  600 participants across the seven groups. EAC and NIST officials stated  that they have informed election officials and other stakeholders about  opportunities to participate on these working groups to share their ideas.  The four constituency working groups represent areas related to human  factors (accessibility and usability), cybersecurity, interoperability, and  testing and are charged with developing guidance or other deliverables  related to these four areas. For example, one objective for the human  factors working group is to identify gaps or issues with current  accessibility and usability requirements for voting. The election cycle  working groups—focused on pre-election, election, and postelection  activities—develop process models related to election activities. For  example, an objective for the election working group is to identify the  necessary functionality of election systems needed to administer early  voting and Election Day activities. The work by these seven working  groups will help inform the development of the voluntary guidelines’  requirements. Table 4 shows the seven working groups and their  respective responsibilities.

Some of the stakeholders we interviewed participate in these working  groups. For example, officials from six of the seven voting system  vendors we contacted said they have a representative on one or more of  the constituency working groups. Generally, these six vendors said the  working groups are a positive feature of the voluntary guidelines’  development process. For example, officials from one vendor said they  have been encouraged by the amount of collaboration on the working  groups, and officials from another vendor said it is beneficial that vendors  are part of the working groups because they bring experience and  expertise with designing and developing various types of voting systems.

Development of the VVSG 2.0 High-Level Principles and Guidelines

In August 2017, the TGDC adopted high-level principles and supporting  guidelines for the VVSG 2.0. These principles and guidelines are  intended to provide system design goals and broad descriptions of the  functions that make up a voting system, in contrast to the VVSG 1.1  which focused more on device- or system-specific requirements. The  VVSG 2.0 will be supplemented by requirements consisting of technical  details voting system vendors can use to design devices that meet the  new guidelines. The supplemental requirements will also detail test  assertions for how the accredited test laboratories will validate that a  system complies with the requirements. One of the VVSG 2.0 principles,  for example, is that ballots and vote selections should be presented in a  clear, understandable way so that they can be marked, verified, and cast  by all voters. The corresponding guidelines for this principle focus on  ballots being perceivable, operable, and understandable. For example,  the guideline for perceivable ballots notes that default voting system  settings for displaying ballots should work for the widest range of voters  and allow voters to adjust settings and preferences to meet their needs.

Another VVSG 2.0 principle is that the voting system should be designed  to support interoperability, including having voting devices that can  interface with each other. The corresponding guidelines for this principle  include using standard data formats and commercial off-the-shelf devices  if they meet applicable requirements. According to NIST officials, one  goal of the interoperability working group is to develop guidance that will  enable election equipment and interfacing software to interoperate more  easily and “speak the same language.” NIST officials stated that this goal  is intended to allow vendors to build and certify system components  instead of a full voting system. These principles are designed to help  address some of the issues reported by stakeholders, such as the impact  of prescriptive requirements for ballot designs on vendor innovation and  the challenges encountered with component certification under the  current voluntary guidelines.

Further, officials from the EAC told us that one key change with the VVSG  2.0 is that the EAC commissioners no longer have to approve changes to  the supplemental requirements and test assertions, which will instead be  vetted by the EAC’s Board of Advisors and Standards Board. EAC  officials noted that this allows for greater flexibility to make improvements  to the requirements and testing process, including making changes in  response to technological advancements. Additionally, depending on the  situation, the new voluntary guidelines are intended to allow for more  streamlined testing and certification processes. For example, EAC  officials said that under the new guidelines, if there are modifications that  have been made to a voting system that has already been certified, the  changes can be tested without having the entire voting system go back  through the testing and certification process.

According to EAC officials, the next steps in the VVSG 2.0 development  process are to share the high-level principles and guidelines with the  EAC’s Board of Advisors and Standards Board for further vetting, provide  the public the opportunity to comment on them, and provide them to the  EAC commissioners for approval. Specifically, before final adoption of the  guidelines, both boards are to review and submit comments and  recommendations regarding the guidelines to the commissioners. EAC  officials anticipate that the EAC boards will likely review and pass  resolutions in support of the principles and guidelines in April 2018.  Following the board reviews, there will be a 90-day period for public  comment on the VVSG 2.0, as required by HAVA. The EAC hopes that  the time it typically takes to respond to public comments will be shorter  than for prior voluntary guidelines, due to the extensive feedback and  comments received and considered by the working groups during the  development phase. EAC officials anticipate that the EAC commissioners  will vote on the VVSG 2.0 principles and guidelines in August or  September 2018, and the VVSG 2.0 will be issued after they are  approved. According to EAC and NIST officials, the working groups have  begun developing the supplemental requirements for the new guidelines.  They said that the requirements are expected to be drafted by the  summer of 2018 and test assertions for most voting systems are  expected to be developed by the summer of 2019.

EAC officials noted that it will likely take 12 to 24 months after the EAC  commissioners approve the new guidelines before they are ready for use.  EAC officials plan to submit to the EAC commissioners a range of  recommended dates to consider for implementation. They added that in  developing these dates, including when vendors will be required to test  new equipment against the updated guidelines, they must consider  various factors such as the time voting equipment vendors will need to  build their new equipment to VVSG 2.0, and reaccreditation of voting  system test laboratories to ensure they can test to VVSG 2.0. Because of  the lag between when the guidelines will be issued and when they will be  used for testing and certification, EAC officials stated that it is unlikely that  systems will be certified in time to be ready for use in the 2020 election.  However, these officials noted that they are available to meet with  vendors that would like to start developing equipment based on the new  guidelines.

We provided a draft of this report to the EAC, NIST, and election offices in  the five local election jurisdictions that we selected and their respective  states for review and comment. The EAC, two jurisdictions, and two  states provided technical comments, which we incorporated in the report  as appropriate. NIST, three jurisdictions, and three states indicated that  they had no comments in e-mails received from March 1 through March  23, 2018.

We are sending copies of this report to the EAC, NIST, election offices in  the five selected local jurisdictions and their respective states that  participated in our research, appropriate congressional committees and  members, and other interested parties. In addition, this report is available  at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions, please contact Rebecca Gambler  at (202) 512-8777 or gamblerr@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made significant contributions to this report  are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

This report addresses the following questions:  1.  What types of voting equipment did local election jurisdictions use for  the 2016 general election, and what are jurisdiction perspectives on  equipment use and performance?  2.  What factors are considered when deciding whether to replace voting  equipment and what approaches have selected jurisdictions taken to  replace their equipment?  3.  What are selected stakeholders’ perspectives on how federal voting  system guidelines affect the replacement and development of voting  equipment, and what actions has the Election Assistance Commission  (EAC) taken to update the guidelines?

For our first objective, we conducted a web-based survey of officials from  a stratified random sample of 800 local election jurisdictions nationwide to  obtain information from the jurisdictions on the voting equipment used  during the 2016 general election and perspectives on equipment use and  performance. In total, we received 564 completed questionnaires for a  weighted response rate of 68 percent. We surveyed the officials about  the types of voting equipment they used, various characteristics of the  equipment used, their perspectives on the benefits and challenges they  experienced while using the equipment, and how satisfied they were with  its performance during the election.

Overall, there are 10,340 local election jurisdictions nationwide that are  responsible for conducting elections. States can be divided into two  groups according to how they delegate election responsibilities to the  local election jurisdictions. One group is composed of 41 states that  delegate election responsibilities primarily to counties. We also included  the District of Columbia in this group of states. However, even within this  group there are some exceptions to how election responsibilities are  delegated. For example, there are no counties in Alaska, so the state  groups all of its Boroughs and Census Areas into four election regions;  and 6 states—Illinois, Maryland, Missouri, Nevada, New York, and  Virginia—delegate responsibilities to some cities independently from  counties. The group of 41 states and the District of Columbia contains  about one-fourth of the local election jurisdictions nationwide. The other  group is composed of 9 states that delegate election responsibilities to  subcounty governmental units, known by the U.S. Census Bureau as  Minor Civil Divisions (MCD). This group of states contains about three- fourths of the local election jurisdictions nationwide. The categorization of  the 50 states and the District of Columbia by how election responsibilities  are organized is as follows (states in bold delegate election  responsibilities to some cities independently from counties):

County-level states: Alabama, Alaska (four election regions), Arizona,  Arkansas, California, Colorado, Delaware, the District of Columbia,  Florida, Georgia, Hawaii, Idaho, Illinois, Indiana, Iowa, Kansas,  Kentucky, Louisiana, Maryland, Mississippi, Missouri, Montana,  Nebraska, Nevada, New Jersey, New Mexico, New York, North  Carolina, North Dakota, Ohio, Oklahoma, Oregon, Pennsylvania,  South Carolina, South Dakota, Tennessee, Texas, Utah, Virginia,  Washington, West Virginia, and Wyoming

MCD–level states: Connecticut, Maine, Massachusetts, Michigan,  Minnesota, New Hampshire, Rhode Island, Vermont, and Wisconsin  While 27 percent of election jurisdictions nationwide are in states that  delegate election responsibilities primarily to counties, according to the  2010 Census, 89 percent of the U.S. population lived in these states. The  U.S. population distribution between the two state groups is shown in  table 5.

The sampling unit for our survey was the geographically distinct local  election jurisdiction at the county, city, or MCD level of local government  (or, in Alaska, the election region). We constructed our nationwide sample  frame of all local election jurisdictions using 2010 decennial Census data  and information on local jurisdictions from state election office websites.  Census population data were available for all counties, county  equivalents, and MCDs.

To obtain a representative sample that included a mix of both rural and  non-rural jurisdictions, we used a two-level stratified sampling method in  which the sample units, or jurisdictions, were broken out into rural and  non-rural strata. To do this, we used the U.S. Department of Agriculture’s  Economic Research Service’s Rural-Urban Continuum Code (RUCC)  system which classifies counties into a nine-category continuum based on  their characteristics and location relative to metropolitan areas. The  RUCC continuum coding scheme is shown in table 6.

To assign a continuum code to each local election jurisdiction, we  matched the RUCC county code to each county in the population frame.  Cities that are independent local election jurisdictions and spread  geographically across one or more counties received the lowest  numbered code among the counties which contain them (i.e., most  urban). For independent cities that administer their own elections but are  contained geographically within a single county, the city received the  code assigned to the county. Where necessary, the parent state’s 2010  decennial Census report was checked to make sure all counties that  included part of the independent city were identified. MCDs in New  England and the Midwest received the code of the parent county that  contained them. For our sampling purposes, the rural stratum was  defined as all local election jurisdictions with an RUCC code of 7, 8, or 9.  The non-rural stratum was defined as all local election jurisdictions with a  code of 1, 2, 3, 4, 5, or 6. Of the 10,340 local election jurisdictions  nationwide, 70 percent were classified as non-rural while 30 percent were  classified as rural.

We selected a two-level stratified sample of 800 local election  jurisdictions. Using the RUCC codes, we allocated 600 sampling units, or  jurisdictions, to the non-rural stratum and 200 to the rural stratum. To  obtain a sample that also reflected the population distribution across  jurisdictions nationwide, we used the population of the local election  jurisdiction as the measure of unit size and selected the sample units  within each stratum with probability proportionate to population of the  local election jurisdiction, without replacement. We used jurisdiction  population size, rather than the number of eligible or registered voters,  because these Census data were readily available for all counties and  MCDs nationwide. Because the sample was selected with probability  proportionate to population size, any jurisdiction (county or MCD) with  more than about 225,000 people was selected with certainty. Table 7  shows the breakout of jurisdictions by population size, the total population  within each size grouping, and the number of jurisdictions sampled.

After selecting the units to be included in our survey sample, we obtained  contact information for the chief election official within the jurisdictions  selected. To do this, we first collected contact information for local  election jurisdictions from state election office websites and other publicly  available sources. We then called the jurisdiction offices directly to  confirm the accuracy of the information and the appropriate official and e- mail address to which the survey URL and the respondent’s login  information for the questionnaire should be sent. We launched our web- based local election jurisdiction survey on March 27, 2017, and made it  available to respondents to complete online through July 14, 2017. Log in  information to the survey was e-mailed to the chief election official of each  sampled jurisdiction. Between April 4, 2017, and July 10, 2017, we  conducted follow-up with nonrespondents by phone and e-mail. During  this follow-up, we learned that some MCDs in Minnesota contract with  their respective counties to carry out election administration  responsibilities, including those concerning the use of voting equipment.  In these cases, we reassigned and sent the questionnaire for the  particular MCD to the appropriate county election official for completion.  Finally, we adjusted the sampling weights to compensate for  nonresponse using weighting classes within each stratum that were  based upon population size of the jurisdictions.

All sample surveys are subject to sampling error—that is, the extent to  which the survey results differ from what would have been obtained if the  whole population had been observed. Because we followed a probability  procedure based on random selections, our sample is only one of a large  number of samples that we might have drawn. As each sample could  have provided different estimates, we express our confidence in the  precision of our particular sample’s results as a 95 percent confidence  interval. This is the interval that would contain the actual population value  for 95 percent of the samples we could have drawn. As a result, we are  95 percent confident that each of the confidence intervals based on our  web-based survey includes the true values in the sample population.

In addition to the reported sampling errors, the practical difficulties of  conducting any survey may introduce other types of errors, commonly  referred to as nonsampling errors. For example, differences in how a  particular question is interpreted, the sources of information available to  respondents, or the types of people who do not respond can introduce  unwanted variability into the survey results. We took numerous steps in  questionnaire development, data collection, and the editing and analysis  of the survey data to minimize nonsampling errors. For example, to inform  the development of our questionnaire, we reviewed existing reports and  studies about voting equipment and elections, such as those by various  national public policy research organizations and professional  associations of state and local officials involved in election administration,  as well as previous GAO surveys and work related to this issue area. In  addition, we interviewed election subject matter experts and  representatives from organizations in the field of election administration  and voting equipment to obtain their views and perspectives on potential  issues and subject areas to consider covering in our questionnaire. We  also pretested the draft questionnaire by telephone with officials in 4 local  election jurisdictions (3 counties and 1 MCD) of various sizes in 4 states  and had the draft questionnaire reviewed by two election experts. We  used these pretests and reviews to further refine our questions, develop  new questions, clarify any ambiguous portions of the questionnaire, and  identify any potentially biased questions, and made revisions, as  necessary. Further, during our analysis of the responses, we found that  due to a higher level of nonresponse by very small jurisdictions of 2,500  persons or less, some national-level estimates that included responses  from jurisdictions of all sizes had wider than desired confidence intervals.  To improve the precision of these national-level estimates, we  subsequently excluded the very small jurisdictions of 2,500 persons or  less from our analysis. Computer analyses were conducted to identify any  inconsistencies in response patterns or other indications of questionnaire  response errors. All computer syntax was peer reviewed and verified by  separate programmers to ensure that the syntax had been written and  executed correctly.

For our second objective, we used our local election jurisdiction survey as  described above to obtain information from jurisdictions about the factors  they consider when determining whether to replace their voting  equipment. In addition to the local election jurisdiction survey, we also  conducted a web-based survey of the state-level election offices in the 50  states and the District of Columbia about issues pertaining to the states’  role in selecting and acquiring voting equipment, including the factors  considered when determining whether to replace voting equipment. In  total, we obtained 46 responses (a 90 percent response rate). We took  the same steps to develop the state questionnaire as we did in  developing the local election jurisdiction questionnaire described above.  We conducted pretests of our draft state questionnaire by telephone with  election officials of 4 states with varying election system characteristics  such as type of voting equipment used, population size, use of federal  voting equipment certification processes, and age of equipment, among  other characteristics. We also had the draft questionnaire reviewed by  two election experts. We used these pretests and reviews to help further  refine our questions, develop new questions, clarify any ambiguous  portions of the survey, and identify any potentially biased questions, and  made revisions, as necessary.

Prior to fielding our state survey, we contacted the secretaries of state or  other responsible state-level officials, as well as officials from the District  of Columbia, to confirm the contact information for the director of  elections or comparable official for their respective state. We launched  our web-based state survey on April 6, 2017, and made it available to  respondents to complete online through May 19, 2017. Log-in information  to the survey was e-mailed to directors of elections or comparable  officials. Between April 12, 2017, and May 16, 2017, we conducted follow- up with nonrespondents by phone and e-mail. The total number of  responses to individual questions may be fewer than 46, depending upon  how many respondents were eligible or chose to respond to a particular  question. For example, survey respondents who indicated that their state  did not have a role in determining whether to replace voting equipment  were directed to skip all subsequent questions related to the factors  considered when determining whether to replace equipment.

Because this survey was not a sample survey, there are no sampling  errors. However, the practical difficulties of conducting any survey may  introduce nonsampling errors. For example, differences in how a  particular question is interpreted, the sources of information available to  respondents, or the types of people who do not respond can introduce  unwanted variability into the survey results. We included steps in both the  data collection and data analysis stages for the purpose of minimizing  such nonsampling errors. For example, we examined the survey results  and performed computer analyses to identify inconsistencies and other  indications of error. Where these occurred, survey respondents were  contacted to provide clarification and the response was modified to reflect  the revised information. A second, independent analyst checked the  accuracy of all computer analyses. The scope of this work did not include  verifying states’ survey responses with local election officials.

For additional perspectives and context on the factors considered by  jurisdictions and states when replacing voting equipment, we also used  our reviews of existing reports and studies about voting equipment and  elections and interviews with election subject matter experts, including  representatives from nongovernmental research and other organizations  involved in the field of election administration and voting equipment. For  our review of existing reports and studies, we reviewed literature covering  the period from 2005 through 2017 including general news, trade and  industry articles, association and nonprofit publications, and government  reports related to voting system technology, specifically on the  replacement and development of voting systems and voting system  standards or guidelines. For our interviews, we identified and selected  nine subject matter experts based on our review of reports and studies on  voting equipment, their expertise and work in this area, and  recommendations from these and other researchers. These subject  matter experts represented the following organizations: (1) Brennan  Center for Justice, (2) National Conference of State Legislatures, (3)  National Association of Secretaries of State, (4) National Association of  Counties, (5) National Association of State Election Directors, (6) Verified  Voting, (7) Kennesaw State University Center for Election Systems, (8)  Center for Election Innovation and Research, and (9) Election Data  Services, Inc. The information we obtained from these experts cannot be  generalized; however, these experts provided additional perspectives and  information on the factors considered by jurisdictions and states when  replacing voting equipment.

In addition, we interviewed election officials from five local jurisdictions— Los Angeles County, California; Travis County, Texas; Anne Arundel  County, Maryland; Lafayette County, Florida; and Beaver County, Utah— that replaced their voting equipment between 2012 and 2016 or plan to  replace their equipment in time for the 2020 general election to learn  about the approaches and practices they used and obtain their  perspectives on the replacement process. We selected these jurisdictions  to reflect variation in, to the extent possible, population of jurisdiction, type  of voting equipment replaced and selected, state involvement in selecting  and funding voting equipment, and particular practices used to replace  equipment (e.g., self-designing equipment, leasing equipment), among  other factors. For each jurisdiction, we interviewed—on site or by  phone—local election officials, state election officials in the jurisdiction’s  state, and individuals who have served as poll workers at the jurisdiction’s  polling locations if applicable. While these five jurisdictions are not  representative of all local election jurisdictions nationwide that replaced or  plan to replace their voting equipment, they provide examples of various  approaches for replacing voting equipment and perspectives on key  issues with replacing equipment. We corroborated various information we  obtained through these interviews by reviewing relevant state statutes  and documentation that these jurisdictions provided to us, such as  postelection reports, voting system studies, expenditure summaries, and  solicitations for vendor proposals to provide voting equipment and  services.

To address objective 3, we used responses to our survey of state election  officials and interviews with seven selected voting system vendors, the  nine selected subject matter experts mentioned above, and officials from  the EAC and National Institute of Standards and Technology (NIST) to  obtain perspectives on how federal voting system guidelines and their  associated testing and certification processes affect the replacement and  development of voting equipment. We obtained perspectives on the most  recent federal voluntary voting system guidelines (Voluntary Voting  System Guidelines, versions 1.0 and 1.1) because they are currently  being used to federally test and certify voting systems. We selected the  seven voting system vendors based on the prevalence of jurisdictions’  use of their equipment, and to obtain variation in the type of voting system  manufactured, such as optical scanners and direct recording electronic  voting equipment, and whether systems were federally certified, under  test to be certified, or not certified. We also wanted to include a  company that plans to enter the voting system market and potentially  submit its product for federal certification. Based on these criteria, we  selected the following voting equipment vendors—Dominion Voting  Systems, DFM Associates, Election Systems and Software, Everyone  Counts, Hart InterCivic, Open Source Election Technology Institute, and  Unisyn Voting Solutions.

To determine the actions taken or planned by the EAC to update the  federal voluntary voting system guidelines, we reviewed EAC and NIST  documents and interviewed officials from the EAC and NIST about these  actions. We also interviewed the seven selected voting system vendors  about their involvement, if any, in updating the guidelines and their  perspectives on these actions.

The perspectives of the seven voting system vendors and nine subject  matter experts are not generalizable but provide examples of views on  the federal guidelines and their associated testing and certification  processes from a range of stakeholders.

We conducted this performance audit from June 2016 to April 2018 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix II: Categories of State Requirements for Federal Certification and Testing of Voting Systems

We reviewed state statutes and regulations as of December 2017  regarding the testing and certification of voting systems to describe the  extent to which state laws and regulations reference federal voting  system certification or testing standards and the extent to which states  require the use of these standards. As shown in table 8 below, we  grouped the state laws into three categories for the purposes of this  report: (1) requires full federal certification; (2) requires testing by a  federally accredited laboratory and/or testing to federal voting system  standards; and (3) no federal requirements. Category 2 includes states  that use some aspect of the federal testing and certification program but  do not require full certification. A number of states in this category require  both testing by a federally accredited laboratory and testing to federal  standards, but we included in this category states that had either  requirement in state law or regulation. Category 3 includes some states  that utilize the federal certification or testing standards to some extent but  that do not require certification or testing to meet federal standards by law  or regulation. We then sent our categorization to state officials in the 50  states and the District of Columbia and incorporated changes that we  received from those officials.

Appendix III: Results of GAO’s Survey of Local Election Jurisdictions on Voting Equipment

To determine the types of voting equipment local election jurisdictions  used for the 2016 general election, jurisdiction perspectives on equipment  use and performance, and the factors jurisdictions consider when  deciding whether to replace voting equipment, we conducted a web- based survey of officials from a stratified random sample of 800 local  election jurisdictions nationwide. In total, we received 564 completed  questionnaires for a weighted response rate of 68 percent. The  questions we asked in our survey are shown below. Our survey was  composed of closed- and open-ended questions. In this appendix, we  include all survey questions and results of responses to the closed-ended  questions; we do not provide information on responses provided to open- ended questions.

The tables below represent the estimated percentages of the jurisdictions’  responses to the closed-ended questions. The estimates we report are  rounded to the nearest percentage point and are national-level point  estimates representing the experiences, views, and opinions of all local  election jurisdictions nationwide with populations greater than 2,500.  Because our estimates are from a generalizable sample, we express our  confidence in the precision of our particular estimates as 95 percent  confidence intervals which are also provided in the tables. As the  jurisdictions we surveyed were selected with probability proportionate to  population size, rather than expressing the point estimates in terms of the  percentage of jurisdictions nationwide that had a specified characteristic,  we express the point estimates for the survey responses in terms of the  percentage of the population nationwide that resides within jurisdictions  that had a specified characteristic. For a more detailed discussion of our  survey methodology, see appendix I.

Question 1 (open-ended question): What is the name, title, telephone  number, and e-mail address of the primary person completing this  questionnaire so that we may contact someone if we need to clarify any  responses?

Use of Commercial Off-the-Shelf (COTS) Components

The Election Assistance Commission’s (EAC) Voluntary Voting System  Guidelines, Version 1.1, defines commercial off-the-shelf (COTS)  products as software, firmware, devices, or components that are used in  the United States by many different people or organizations for many  different applications other than certified voting systems and are  incorporated into the voting system with no manufacturer- or application- specific modification. Examples of COTS components include hardware  that can be purchased commercially (e.g., tablet devices, scanners,  printers, memory cards or chips, etc.) and integrated as part of voting  equipment. The next series of questions asks about your jurisdiction’s  integration of COTS components into voting equipment that was acquired  from a vendor or self-designed by your jurisdiction. For the purpose of  questions 30-36 (the next 7 questions), the term “voting equipment” refers  only to the equipment your jurisdiction used to cast and count votes.

Question 58 (open-ended question): If you have any additional comments  concerning any of the topics covered in this questionnaire, please use the  space below.

Appendix IV: Results of GAO’s Survey of States on Voting Equipment

To obtain information on the types of voting equipment used in the 2016  general election and the factors states consider when deciding whether to  replace voting equipment, we conducted a web-based survey of state- level election offices in the 50 states and the District of Columbia. The  questions we asked in our survey of state election offices are shown  below. Our survey was composed of closed- and open-ended questions.  In this appendix, we include all survey questions and results of responses  to the closed-ended questions; we do not provide information on  responses provided to open-ended questions that required manually  entered text responses. The tables below represent the frequencies of  state responses to the questions. We received surveys from 46 states (a  90 percent response rate), while 5 states did not respond. However, the  total number of responses to individual questions may be fewer than 46,  depending upon how many states were eligible or chose to respond to a  particular question. For a more detailed discussion of our survey  methodology, see appendix I.

Question 46 (open-ended question): If you have any additional comments  concerning any of the topics covered in this questionnaire, please use the  space below.

Appendix V: Approaches to Voting Equipment Replacement in Selected Local Election Jurisdictions

The five local election jurisdictions we selected to include in our review— Los Angeles County, California; Travis County, Texas; Anne Arundel  County, Maryland; Lafayette County, Florida; and Beaver County, Utah— used varying approaches in replacing their voting equipment. Election  officials in these jurisdictions and in their respective state election offices  provided a range of perspectives on their experiences and the  replacement process.

Los Angeles County is the most populous local election jurisdiction in the  nation. It currently uses hand-marked paper ballots that are tallied using  central count optical scan equipment, which has been in place since  2003. Prior to 2003 and dating back to 1968, these same ballots were  used for its punch card voting system. The county is in the process of  self-designing its own voting system, which is expected to consist of  electronic ballot marking devices (BMDs) that produce paper ballots to be  tallied on central count digital scanners, and plans to fully implement it in  2020.

Key Factors That Influenced the County’s Decision to Replace Its Voting Equipment

According to county officials, the overall performance and features of the  county’s voting equipment and the need for the equipment to meet  potential state and local requirements were among the key factors that  influenced the county’s decision to begin the process of replacing its  optical scan system. County election officials stated that while the  county’s current voting equipment is reliable, accurate, and familiar to  voters, the design and the age of the equipment do not offer the technical  and functional flexibility necessary to continue to accommodate potential  state regulatory changes and the growing and increasingly diverse county  electorate. For example, officials stated that the current equipment may  not be able to effectively accommodate state mandates that may require  changes to ballot formats or length. Specifically, officials said that state  legislation enacted in 2015 requires many cities within Los Angeles  County to consolidate their elections with the county’s by 2022, and as a  result, the number of races and measures on the ballot may exceed the  12-page capacity that the current equipment can accommodate. They  also noted that the technical limitations of the equipment present  challenges to providing voters with greater voting options, such as early  voting or the use of vote centers on Election Day, and features that  enhance accessibility and ease of use.

Planned New Voting Equipment and In-Person Voting Process

The county has developed a design concept and specifications for its new  voting equipment and is in the process of soliciting and selecting vendors  to manufacture it. It has acquired several functional prototypes of the  current design for the new equipment and has outlined the planned in- person voting process using this equipment, as shown in figure 11.  According to county officials, the equipment specifications and in-person  voting process have not been finalized and continue to be refined.

County officials stated that the current design concept for the new  equipment is intended to provide greater flexibility in administering  elections, provide a more user-friendly and accessible voting experience,  enhance accuracy and auditability, and could potentially lower costs for  system upgrades if developed as planned:

Greater flexibility for administering elections. According to county  election officials, the new equipment is designed to provide more  flexibility for administering elections and to respond to changing  legislative provisions on conducting elections. For example, the  California Voter’s Choice Act, which was enacted in September 2016,  generally authorizes Los Angeles County to conduct vote center  elections beginning in 2020 if certain conditions are met. Officials  stated that the proposed new equipment is expected to facilitate the  use of vote centers because it would have the capability to  electronically retrieve a voter’s ballot regardless of the precinct in  which the voter is registered. They also noted that the BMD would  allow the county to have ballots with multiple formats and a large  number of races.

A more user-friendly and accessible voting experience. County  election officials stated that the BMD is intended to provide the ease  of use of a touch screen interface, which would incorporate features  such as scrolling and tapping that are familiar to voters who use  mobile devices. The BMD would also allow voters to select from  English or the 11 other languages the county plans to support and is  designed to include accessibility devices, such as a headset and  tactile keypad for voters with vision impairments and other  disabilities. Voters would be able to make their selections and cast  their paper ballot without having to handle the ballot. Officials stated  that these features are expected to allow voters with special needs to  use the same equipment as all other voters and cast their votes  independently and privately. The county’s proposed design also  includes an interactive sample ballot which voters can access from  their computers or mobile devices to pre-mark their vote selections,  convert to a Quick Response (QR) code, and then scan into the voting  equipment to populate their ballots. Officials stated that this feature  may help reduce lines by decreasing the time it takes for voters to  mark their ballots once they reach the BMD.

Enhance accuracy and auditability. The new voting equipment is  designed to record vote selections on paper in human readable text.  County officials stated that this is expected to more clearly capture  voter intent than manually marked ballots, reduce the time and  resources needed by county staff to interpret voters’ intent, and  increase the accuracy of election results and public trust in the voting  process. Officials stated that the new equipment is also expected to  improve the county’s auditing capabilities. For example, the digital  scanner is designed to allow the county to efficiently audit the results  of individual races and measures, including conducting risk-limiting  audits in which a specified number of ballots cast for a particular race  are reviewed to confirm the election result for that race. According to  officials, the county’s current equipment tallies ballots by precinct and  does not keep an electronic record of the specific votes cast on  individual ballots. As such, it provides the capability of auditing the  results by precinct but not individual races at the ballot level.

Easier and less costly upgrades. According to county officials, the  design of the voting equipment is intended to be modular so that key  components can be replaced individually. Officials stated that this is  intended to allow the county to more easily update equipment and  incorporate technological advances because it will be able to swap  out components if more affordable, better technology becomes  available on the market. Officials said that the cost of replacing  equipment parts is expected to be lower than with traditional voting  systems.

Process for Developing the New Voting Equipment

Los Angeles County’s Voting Systems Assessment Project (VSAP) was  established by the Registrar-Recorder/County Clerk in 2009 to help guide  the development and acquisition of the county’s new voting equipment.  According to county election officials, the VSAP has taken a user- centered approach to the design of the new voting equipment that  prioritizes the specific needs and expectations of the voters and  incorporates the requirements of county election administrators. Officials  also stated that they sought to have a transparent design process that  included voter input and participation to help promote public confidence in  the new voting equipment. The project has five phases—(1) public  opinion and stakeholder baseline research, (2) establishment of voting  system guiding principles, (3) system design and engineering, (4)  manufacturing and certification, and (5) phased implementation. The  county is currently in the manufacturing and certification phase. Officials  reported that about $19 million has been expended to develop the new  voting equipment as of December 31, 2017. Officials also stated that after  the new system is certified, an additional $49 million in state funds from  the Voting Modernization Bond Act of 2002 will be available to the  county. Table 108 describes the VSAP phases, their associated expenditures and funding sources, and examples of key actions taken or  planned in each phase.

County officials told us they plan to retain ownership of the intellectual  property rights of the new voting equipment so that the system remains  publicly owned and not proprietary like traditional vendor equipment. The  county also plans to use an open source technology framework wherein  the source code for the system software is available for review and use  by other election jurisdictions and entities by license. According to  county election officials, this will allow other jurisdictions to, for example,  have similar systems manufactured for their use. Officials stated that  having the county own the system design on behalf of the public and  using an open source software model are expected to provide greater  flexibility for any jurisdictions using the software to cost-effectively make  modifications to the equipment and adapt it to their varying needs and  requirements. For example, jurisdictions would no longer be limited to  relying on a single manufacturer if they would like to make an  enhancement to the equipment or replace parts.

Officials noted that there is currently no licensing model or institutional  framework in use for a publicly owned elections system. However, they  stated that open source technology solutions in other industries have  been successfully implemented and administered, and the county’s new  system software could potentially be licensed and administered in a  similar manner. In addition, county officials stated that they have outlined  a clear business plan in the Request for Proposal (RFP) and during  various information sessions with vendors which officials believe will help  incentivize them to participate in building the system without potentially  owning the equipment or its intellectual property rights. Specifically,  officials noted that vendors would primarily receive revenue from the  services they would provide, such as building the equipment and software  platform and providing ongoing maintenance and support, rather than  from selling the equipment itself.

County officials stated that implementing the new voting equipment and  moving to vote center elections in 2020 are changes to administering  elections for the county that will require a substantial educational and  informational effort. Officials noted that they have involved numerous  stakeholders throughout the VSAP process to help effectively prepare for  these changes and plan to allocate resources to educate voters and train  poll workers. Some of these efforts are already underway. For example,  the county has posted information and videos on the planned new voting  equipment and process on the VSAP website and has been using the  BMD prototype for public demonstrations and internal training on the new  voting process.

Travis County currently uses direct recording electronic (DRE) equipment  without a voter-verified paper audit trail (VVPAT), which has been in place  since 2001. The county also has conducted vote center elections since  2011. Starting in 2009, the county took steps to design and build its own  equipment, including developing a concept for a DRE with a VVPAT that  centered on system security and auditability. In September 2017, the  county decided to no longer pursue building the voting equipment and  plans to purchase equipment from a vendor. The county plans to have the  new equipment in place for the 2020 election.

According to county officials, the overall performance and features of the  county’s voting equipment was the primary reason for deciding to begin  the process of replacing its DREs. In 2009, the Travis County Clerk  convened an Election Study Group to assess the county’s current  equipment and make recommendations for future equipment. This group  was composed of 45 members representing election officials and  workers, advocacy organizations, voters with disabilities, computer  security experts, academics, and other segments of the community.  According to the report that the group issued, most members expressed  confidence in the way Travis County conducted elections and in the  accuracy of its current equipment. However, they also expressed  concerns over the equipment’s age and the lack of a paper trail, which  they said decreased voter trust in the system and increased the risk of  election equipment tampering. The group noted that the Travis County  Clerk’s Office’s use of safeguards and security and testing procedures  beyond those required by law helped minimize the risk of tampering. The  report recommended that the county move toward using equipment that  offers an electronic count and paper record as soon as an alternative that  met the county’s requirements became available.

Selection and Acquisition of New Voting Equipment

Development of Self-Designed Voting Equipment

The Election Study Group outlined 19 key requirements that Travis  County’s new equipment should meet. The requirements included, for  example, producing a paper voting record that can be verified by the voter  and be used to independently, transparently, and efficiently reconcile an  electronic tally in an audit or recount; allowing voters with special needs  to vote using the same equipment as other voters; enabling early voting  and the use of vote centers; and having reasonable purchase,  operational, and system upgrade costs. The group found that no  equipment on the market in 2009 met the needs of the county and, as a  result, the county began exploring options to design its own equipment.  Officials stated that this effort was also intended to provide an alternative  to the current vendor model that could reduce maintenance costs and  annual licensing fees that are incurred with proprietary systems.

In 2012, the county Clerk convened a group of election administrators,  usability experts, and academic experts in computer science and  statistics, and through a series of discussion sessions, developed the  concept for the county’s new system, which they named STAR (Secure,  Transparent, Auditable, and Reliable) Vote. STAR-Vote was designed to  be centered around a DRE that produces verifiable and auditable paper  records. At the polling place, voters would make their selections on a  DRE device with a commercial off-the-shelf (COTS) tablet, which would  also be equipped with an auditory interface for visually impaired voters  and other features to assist individuals with special needs. The voters’  selections would be encrypted and stored on the internally networked  DRE devices, and voters would also receive a printed paper record with  their choices. After reviewing the paper record and confirming their  selections, voters would feed the paper record into a ballot box scanner to  cast their vote. Once the polls closed, the devices storing the votes  would be transported to receiving stations, where voting data are  transmitted for electronic tabulation. The paper records would be  available for audit or recount purposes.

In addition, county officials stated that the equipment’s proposed  encryption technology was designed to potentially allow for the following  features without revealing any individual’s vote:

Voters would receive a receipt that was attached to their paper  records at the polling place and could go online after Election Day and  use a code on the receipt to verify that their ballots had been cast and  counted.

Third parties, such as the League of Women Voters or political  parties, could access encrypted voting data to verify that the results  the county had reported matched vote totals they had independently  derived from the data.

The county could conduct risk-limiting audits to verify the consistency  between the electronic and printed vote records and test the accuracy  of the reported election outcomes. Audits could be conducted on  individual ballots or races if needed.

In June 2015, the county issued a Request for Information for STAR-Vote  to solicit input on the design, development, implementation, and  maintenance of the equipment. Based on information gathered from the  request, it issued an RFP in October 2016 to solicit proposals from voting  system vendors and others for the development and implementation of  key components of the equipment for in-person voting. The county also  issued a Statement of Intent for the equipment to inform interested parties  of the county’s planned approach for the long-term management and  support of STAR-Vote. According to these documents, the county  planned to own the intellectual property rights for the equipment and  provide open source software for its system to the elections community  under a licensing agreement, which would allow other jurisdictions to use  similar equipment. The Statement of Intent described the formation of a  nonprofit organization to manage and support STAR-Vote and sought $25  million in funding from interested parties to complete the development of  the open source software components, support the organization’s  operating budget for the first 5 years, and provide a cash reserve. The  county planned to use these funding commitments and local budget  appropriations to develop, build, and deploy the equipment.

In September 2017, the county announced that it had decided to no  longer pursue developing and building STAR-Vote. The county stated that  it received 12 proposals in response to the RFP but they were not  sufficient to build a complete voting system. According to county officials,  none of the proposals included the election management system for the  equipment that would handle ballot definition and the tallying of results,  among other related tasks. In addition, officials stated that they received  limited responses to their solicitation for financial commitments in the  Statement of Intent and thus lacked the necessary funding to develop and  build the equipment. Officials noted that the open source software  platform they had envisioned was seen by voting equipment vendors as a  low-revenue business model in the current elections marketplace. They  added that potential participants in a STAR-Vote entity may not have had  a clear concept of how its business model might work, which they said  was perhaps due to the county’s more limited focus on this aspect when  they were initially designing the system. Given these obstacles and the  age of the county’s current equipment, the county decided that it needed  to move toward acquiring more immediately deliverable voting equipment  through a voting system vendor.

Selection and Acquisition of New Voting Equipment from a Vendor

The county has incorporated some of the features of STAR-Vote into its  requirements for new voting equipment. According to county officials, the  county plans to acquire either DREs or ballot marking devices with  precinct count digital scanners because, in their view, they are accurate  (e.g., prevent voter errors, such as overvotes or stray marks on the ballot,  and minimize questions about voter intent), allow individuals with  disabilities to vote on the same equipment as other voters, support vote  center elections, and offer fast reporting of election results. The county  also plans to require that its next voting equipment have the following  features:

A voter-verified, paper list of choices for recount purposes. County  officials stated that the equipment must produce printed paper records  that can be tallied and connected with electronic voting records  through an automated process. This electronic connectivity would  allow paper-ballot recounts to be conducted on individual races.

Security features that include support for third party verification of  results and better postelection audits. According to county officials,  the equipment they acquire must allow for third parties to  independently verify reported election results and must support risk- limiting audits.

Officials stated that they believe there is or will be equipment on the  market in the near future that could support these features. They noted  that they are also prepared to work with vendors to customize existing  equipment to meet the county’s requirements if needed, acknowledging  that such additions may increase expenses or require additional time to  recertify parts of the voting system. County officials estimate that the new  equipment will cost about $16 million and stated that acquisition will be  funded through local bonds.

County officials said they would like to have the new equipment in place  for the 2020 election, which would require them to start deploying it no  later than May 2019. The county issued an RFP for the system in  November 2017, and officials stated that they plan to assemble a group of  stakeholders similar to those who participated in the 2009 Election Study  Group, as well as the individuals who designed STAR-Vote, to help  evaluate the proposals received. Officials noted that their current  equipment is functioning and robust, but that the new equipment must be  deployed before the current equipment begins to degrade. In addition,  they stated that the May 2019 implementation date is the latest possible  date in order to allow sufficient time to educate voters and train county  staff and election judges on the new equipment before using it in the 2020  election.

Anne Arundel County had used DREs without a VVPAT since 2004 and  replaced its equipment in 2016 with a system in which voters manually  mark paper ballots and insert them into precinct count digital scanners  which then count them. Maryland requires the use of uniform voting  equipment in polling places statewide and the state and counties each  pay 50 percent of the costs of acquiring equipment. In our state survey,  Maryland officials reported that the state determines when voting  equipment is to be acquired and selects the type and model of voting  equipment that local jurisdictions use.

Key Factors That Influenced Maryland’s Decision to Replace Its Voting Equipment

According to the Maryland State Board of Elections (SBE) and Anne  Arundel County Board of Elections officials, the need for voting equipment  to meet state requirements, the overall performance and features of the  equipment, and the ability to maintain the equipment were among the key  factors that influenced the state’s decision to replace its equipment.

Specifically, in 2007, Maryland enacted a law that prohibited the use of a  voting system unless the SBE determined that the system provides a  voter-verifiable paper record, thereby requiring the state’s DREs to be  replaced. SBE officials said that the passage of the new law was driven  primarily by a push from voting advocates to move to new equipment that  used paper ballots and provided a verifiable paper trail. Although the law  was enacted in 2007, state funding for the new equipment was not  available until 2014 due to budgetary constraints.

While the change in state law was the main reason for replacing its voting  equipment, both SBE and Anne Arundel County officials noted that the  state’s previous DRE equipment was nearing the end of its life cycle and  various problems had begun to occur more frequently. For example, SBE  officials said that nonresponsive touch screens and battery unit failures  became more common with the equipment used in the state. In addition,  Anne Arundel County officials stated that while their equipment generally  performed satisfactorily, some of the touch screens had begun to degrade  and develop calibration issues, which resulted in the appearance of  incorrectly recording voters’ selections. In addition, county officials said  that the equipment could no longer support certain software or security  updates, and replacement parts were challenging to acquire.

According to SBE officials, state law specifically required the purchase of  precinct count scanners so the board did not consider other types of  voting equipment. The SBE issued an RFP in July 2014 and four voting  system vendors submitted proposals. The SBE formed an evaluation  committee to analyze the technical and financial details of the proposals.  According to SBE officials, the committee’s members included a state  official with expertise on voting systems, a county election director, a  county technical specialist, and election experts and researchers, among  others. Anne Arundel County election officials stated that the SBE also  established various subcommittees to solicit input from county officials as  the state made its selection. They said that relevant local elections staff  members were involved in the selection process and that in their view, the  process had worked well.

According to SBE officials, in addition to assessing the vendors’  proposals, the evaluation committee worked with the University of  Baltimore to perform usability and accessibility testing on the equipment  under consideration. The committee also hosted a public demonstration  to collect feedback on such areas as ease of use and confidence that  votes were accurately cast. Officials stated that after conducting its  assessment of the equipment, the committee presented its findings to the  SBE, and in October 2014, the board selected the voting equipment to be  acquired based on the committee’s recommendation.

Maryland requires equipment to be certified by the EAC and the SBE  before use in the state. The selected equipment had been certified by the  EAC in July 2014 and was certified by the SBE in December 2014. As  part of the certification process, the SBE tested the equipment to ensure  that it met requirements in the Maryland elections code, including  simulating primary and general elections using ballots typically used by  jurisdictions in the state, and reviewed the findings from the public  demonstration and usability testing performed during the selection  process.

The SBE decided to lease rather than purchase the equipment for a  number of reasons. Specifically, SBE officials said that leasing provided  increased flexibility to update or replace equipment more frequently and  had lower upfront costs. In addition, the state did not want to buy new  equipment until the implementation of updated federal guidelines. Under  the current contract to lease the digital scan equipment, payments are  made to the vendor on a quarterly basis. According to SBE officials, the  current payment to the vendor for leasing the digital scan equipment  statewide is approximately $1.1 million per quarter.

SBE officials said that the process to acquire new equipment is inherently  challenging, but in their view, the process generally went well. Knowing  what type of equipment the state needed to acquire simplified the process  and reduced the number of proposals that officials needed to review.  Nevertheless, they noted that the process took more of their time and  resources than they had anticipated, which presented challenges  because the state was holding elections during the same time period it  was selecting and acquiring the equipment. However, the SBE met its  goal of implementing the new equipment by 2016.

SBE and Anne Arundel County officials stated that deployment of the new  equipment in the 2016 general election went smoothly with no significant  challenges. The officials said they took a number of steps to help ensure  a successful rollout. For example, SBE officials said that they established  a strong project management team and hired contractors to assist with  tracking progress toward key deadlines; drafting policies, procedures, and  training manuals; and testing equipment and sending it to the counties.  Anne Arundel County officials said that they hired about 40 temporary  staff to assist with deploying the new equipment and other tasks during  the general election. In addition, they stated that the county conducted  extensive election judge training and held mock elections using the new  equipment. The officials noted that with the new paper-based system, the  county needed to recruit and train more election judges compared to past  elections to hand out ballots, show voters how to operate the equipment,  and handle provisional voting. The two election judges we interviewed  stated that the training they received was very comprehensive and  effectively prepared them for Election Day.

Both SBE and Anne Arundel County officials stated that additional voter  education efforts would have been beneficial. According to SBE officials,  the SBE had developed plans for a statewide multimedia effort to educate  voters on the new equipment but did not receive funding to implement it.  A scaled down effort was carried out instead, which included  demonstrating voting equipment at meetings and fairs around the state,  producing local media news stories, and posting a video on the SBE’s  website on how to use the new equipment. SBE and Anne Arundel  County officials stated that the more limited voter education efforts might  have contributed to longer lines on Election Day in some polling places  because many voters were unfamiliar with the equipment and some had  questions or needed assistance with using it. However, these officials  noted that voter wait times were not a widespread or significant issue  during the general election. The two election judges we interviewed  stated that some voters needed help inserting their ballots into the  scanner, but observed that voters generally appeared to find the new  equipment easy to use. They also noted that some voters commented  that paper ballots provided them with reassurance with regards to the  security of their vote.

SBE and Anne Arundel County officials said that the equipment itself  performed satisfactorily in the 2016 general election with only minor  problems. For example, state officials said that the scanners jammed  occasionally, but this was easily resolved by elections personnel. In  addition, most polling locations in the state were allocated only one  scanner, so some jurisdictions with two-page ballots, such as Anne  Arundel County, experienced lines because of the length of time it took  for voters to scan their ballots. Anne Arundel County officials plan to  analyze voter registration data to help determine the number of scanners  needed at each polling place and share the information with the SBE to  help inform allocations for future elections. More generally, SBE officials  noted that the new system has less equipment to manage—about 2,600  digital scan units compared to the approximately 18,000 DRE units used  statewide in prior elections—so there is less pre-election testing and  postelection maintenance that has to be done, saving time and labor for  the state and counties.

The state contracted with a third party vendor to conduct a postelection  audit of the 2016 general election by using independent software to tally  all digital ballot images. The audit confirmed the accuracy of the election  results. According to SBE officials, the new equipment’s ability to  capture and store digital images of the ballots made this type of audit  possible. Anne Arundel County officials stated that the ability to conduct  such an audit is one of the main benefits of the new equipment.

Lafayette County has a small population and, in 2016, replaced its  precinct count optical scan equipment with precinct count digital scan  equipment. The county formed a consortium with other counties in the  state to help acquire its new equipment.

According to the county’s Supervisor of Elections, the cost to acquire new  equipment and availability of funding and the need to meet state  requirements were among the key factors that influenced the county’s  decision to replace its voting equipment. He stated that Lafayette  County’s optical scanners were approximately 15 years old but were  generally in good condition and performed satisfactorily in prior elections.  County officials had planned to replace the county’s aging voting  equipment by 2018 or 2020, but decided to replace it in 2016 because of  the opportunity to join a consortium of counties that formed to acquire  new equipment, which the Supervisor stated helped secure funding for  and lower the costs of purchasing the equipment.

In addition, the Supervisor of Elections said that, to comply with state law,  the county needed to acquire a paper ballot system with a BMD to  replace the DRE it had used for voters with disabilities. Specifically, as of  July 2008, Florida law required all voting in the state to be done using  mark-sense paper ballots, which are generally counted using optical or  digital scanners, except for voting by individuals with disabilities. Current  state law requires jurisdictions to use these paper ballots for accessible  voting by 2020. As such, according to the Supervisor of Elections, part of  the impetus for acquiring new voting equipment was to replace the  county’s DRE to meet the 2020 deadline in the law.

The Supervisor of Elections stated that Lafayette County is a small county  and does not have much purchasing power. He said that Lafayette  County and other small counties in the state formed a consortium to lobby  the state for assistance and to leverage their collective purchasing  power. The 12-county consortium was established in a 2015 meeting  that was attended by county election officials, the Florida Deputy  Secretary of State, and the vendor that supplied the counties’ previous  voting system. According to the Lafayette County Supervisor of  Elections, the consortium decided to purchase precinct count digital  scanners from the same vendor the counties had used before because  county staff were familiar with the vendor and equipment, and the cost for  the equipment was lower than similar equipment from another vendor that  some counties in the consortium had considered. In addition, the  Supervisor of Elections stated that the digital scanners have features that  were an improvement over the county’s previous optical scan equipment.  For example, he stated that the new scanners have more robust security  features, such as locking panels, seals, and a requirement for a passcode  to access the system. He also noted that the scanners have touch  screens that flip up and are back-lit, which are easier for voters and poll  workers to read and more clearly identify overvotes. Further, he stated  the scanners digitally capture and store ballot images. The two Lafayette  County poll workers we interviewed confirmed that the new equipment  more clearly identified overvotes for them and for voters than did the  previous equipment.

According to the county’s Supervisor of Elections, having the consortium  approach state officials as a group helped secure HAVA funds to help the  counties purchase the voting equipment. In addition, he stated that being  a part of the consortium helped the counties negotiate a lower price for  their equipment than what they could have obtained individually because  they pooled their purchases and acquired a higher volume of machines.  While the consortium negotiated as a unit, each county has an individual  contract with the vendor. The Supervisor of Elections stated that the total  cost to purchase Lafayette County’s new voting equipment—which  included seven digital scanners, seven BMDs for voters with disabilities,  and various system components—was about $70,000. The equipment  was acquired primarily with HAVA funds, although he noted that the  county allocated about $12,000 in local funds to purchase three additional  BMDs. A memorandum of agreement for funding and purchasing the  equipment was signed by Lafayette County and the state in November  2015 and, according to the Supervisor of Elections, the equipment was  acquired in late 2015 and first used in the March 2016 primary election.

The Supervisor of Elections and the two poll workers we interviewed  stated that deployment of the new voting equipment went smoothly and  the county did not experience any challenges because the new and  previous equipment are both precinct count scanning systems. The  Supervisor noted that the voting process remained the same for the voter,  so extensive voter education efforts were not needed. He stated that  Lafayette County did not experience any equipment malfunctions during  the November 2016 general election, and a postelection audit that was  conducted, in which the county manually tallied ballots from a randomly  selected race and precinct, found that the results were accurate.

Beaver County has a small population and previously used DREs with a  VVPAT. In 2014, Beaver County began conducting vote-by-mail elections  and replaced its DREs with central count digital scan equipment to  support this change.

According to Beaver County officials, the overall performance and  features of the equipment and the ability to maintain the equipment were  among the key factors in their decision to replace the county’s equipment.  Officials stated that the county had been using DREs since 2005 and that  by 2013, they had come to the conclusion that the equipment was not  very efficient or user-friendly for administering elections. For example, the  Deputy Clerk stated that it was time consuming to both set up the  equipment and tally the votes, which required collecting and uploading  the memory component from each of the DREs. She also noted that the  operating software for the equipment’s election management system had  become out-of-date and did not have a user-friendly interface. According  to the Deputy Clerk, this made it difficult for staff to navigate without  detailed training, which was time consuming and costly. In addition,  county election officials said that they were unsure about future  maintenance and system upgrade costs and decided it would be more  cost-effective to spend funds on purchasing new voting equipment rather  than on upgrades to equipment with which they were not very satisfied.

In 2013, the county decided to begin conducting vote-by-mail elections  the following year and to acquire new equipment to support this change.  According to county officials, this decision was due to the performance of  their DREs and a desire to reduce costs and increase the efficiency of  administering elections, among other reasons. Officials said that because  the county was moving to vote-by-mail elections and DREs would no  longer be needed for each precinct, the county would instead acquire  central count scanners designed to count the mail-in ballots it would  receive at the county elections office.

According to Beaver County officials, the main individuals involved in the  process to select and acquire the county’s new voting system included  the current Beaver County Clerk, Deputy Clerk, a county information  technology official, and the previous county clerk, among others. When  the county started the process in 2013, the state had not initiated any  efforts to help local jurisdictions acquire new equipment. As such, both  Utah and Beaver County election officials said that the state was aware of  the county’s decision to replace its equipment but was not involved in the  selection and acquisition process.

County officials stated that they wanted to acquire central count scanners  to support conducting vote-by-mail elections and a BMD for in-person  voting at the elections office for individuals with disabilities. Officials said  that, in 2014, they verbally requested proposals from their current vendor  and an elections services company that the county had employed in 2012  to provide training, systems testing, and other support for elections.  According to the Deputy Clerk, the county requested proposals from  these two entities because county officials were familiar with them and  were not aware of other vendors that might submit proposals. Officials  said that the county received a proposal from the elections services  company, and selected the company because it was the only bid received  and the equipment the company sold met the county’s needs and was  federally certified. They stated that one of the challenges they  experienced as a small county looking to purchase equipment was that  vendors were not actively marketing to them. In addition, the Deputy  Clerk noted that she had limited elections and information technology  experience when the county started the selection process. However, she  said that the election services company was familiar with Utah’s elections  code and federal voting system requirements, helped negotiate with the  vendor to acquire the new equipment, and educated county staff on the  equipment.

Beaver County reported that the cost to purchase the equipment—two  central count digital scanners, a BMD, and associated system  components—was about $46,000. Local funds were used to purchase the  scanners and HAVA funds were used to purchase the BMD. According to  Beaver County officials, county commissioners approved the procurement  of the equipment in spring 2014 and it was first used in the June 2014  primary elections.

Beaver County officials stated that they deployed the new equipment in  2014 because it was more manageable to conduct such a transition  during a non-presidential election year. They noted that they needed to  educate the public about both voting by mail and the new voting  equipment. Officials stated that the county used local newspaper ads,  social media posts, and direct mailings to provide information on these  changes. Officials also posted information on the county’s website and  allowed people to observe logic and accuracy testing of the equipment.  They noted that educating the public on the new voting method and  equipment in smaller elections during 2014 and 2015 helped voters  become more comfortable with what to expect for the presidential election  in 2016.

County officials said that they are very satisfied with the performance of  the new voting equipment. They noted that conducting vote-by-mail  elections and using central count scanners allow them to administer  elections from one location on Election Day, which requires less time and  resources than having to manage multiple polling places. Officials also  stated that the new digital scanners are able to count a high volume of  ballots in a short period of time. They said that, for the November 2016  general election, the vote tallying was completed within an hour of the  polls closing, which allowed the county to report results quickly. However,  one challenge they experienced was that the new equipment’s data  format for election night reporting of results to the state was not  compatible with the state’s reporting system. To address this issue,  county officials reformatted the data to produce a report that could be  uploaded into the state’s system, but cautioned that this may not be  feasible for larger jurisdictions.

According to officials, the county conducted two postelection audits for  the 2016 general election—one required by the state and another that the  county initiated. For the state audit, the county hand counted 1 percent of  total ballots from a randomized list. In addition, the county conducted its  own audit by running all ballots on its other digital scanner to compare  results. According to officials, both audits validated the election results.

Appendix VI: GAO Contact and Acknowledgments

In addition to the contact named above, Tom Jessor (Assistant Director),  David Alexander, Carl Barden, Chuck Bausell, Brett Fallavollita, Sally  Gilley, Christopher Hatscher, Eric Hauswirth, Richard Hung, Jill Lacey,  Serena Lo, Jan Montgomery, Heidi Nielson, Shannin O’Neill, Claire  Peachey, Jeff Tessin, and Johanna Wong made significant contributions  to this report.

We gratefully acknowledge the substantial time and cooperation of the  state and local election officials, and stakeholders and experts whom we  interviewed.