The Independent View: When System For Verifying Integrity Of Voting Software Lacks Integrity

When the System for Verifying the Integrity of Voting Software Lacks Integrity, there is a Problem, and uncovered in the Hash-Verification Process for certain Voting Machines, made by Election Systems and Software (ES&S), highlight Issues with the Trustworthiness of Voting Software and Voting Mmachine Vendors.

In September 2020, just weeks before Voters went to the Polls, in one of the Nation’s most Critical and Contentious Presidential Elections, State Officials in Texas, learned of a disturbing Problem with Election Software used widely across Texas and the Country: a Component of Software provided by ES&S, the Top Voting Machine maker in the Country, didn’t work the way it was supposed to work.

The Component wasn’t involved in Tabulating Votes, instead it was a Software Tool provided by ES&S, to help Officials Verify that the Voting Software Installed on Election Equipment was the Version of ES&S Software Certified by a Federal Lab, and that it hadn’t been Altered by the Vendor or anyone else, since Certification.

But Texas Officials learned that the Tool, known as a Hash-Verification Tool, would indicate that ES&S Software Matched the Certified Version of Code even when No Match had been Performed. This meant Election Officials had been relying on an Integrity Check that had Questionable Integrity.

When Voters or Security Experts express Concern that Elections can be Hacked, Officials often cite the Hash-Verification Process as One reason to Trust Election Results. Hash Verification involves running Software through an Algorithm to produce a Cryptographic Value, or Hash, of the Code. The Hash, a string of Letters and Numbers, serves as a Fingerprint of the Program. If the Software is Altered and then Run through the same Hashing Algorithm again, the Hash that’s Created won’t Match the Original Hash.

But Brian Mechler, an Engineering Scientist at Applied Research Laboratories, at the University of Texas at Austin, Discovered while Testing ES&S Software for the Texas Secretary of State’s Office last year, that the Company’s Hash Verification Tool didn’t always Work Correctly.

Letting the Vendor conduct Verification Checks of its Own Software was the Equivalent of the Fox Guarding the Hen House, One Voting Systems Examiner, for the State said. This wasn’t the only Problem. ES&S’s own Employees were sometimes Conducting Hash Checks instead of Election Officials, a Conflict of Interest that Undermined a Process meant to provide Officials with an Independent Integrity Check of Software installed on their Machines.

Texas officials were Furious with the Findings, and in Emails exchanged among themselves and with ES&S, obtained by Election Integrity Activist, Jennifer Cohn, through a Public Records Request, they expressed Alarm that the Vendor had been Operating without Oversight.

Letting the Vndor conduct Verification Checks of its Own Software was the Equivalent of the Fox guarding the Hen House, One Voting Systems Examiner for the State said. “If the hash validation process is performed by the same vendor technician who performed the [software] installation,” the Secretary of State’s Office wrote in an Email to ES&S, “then that validation process loses one of its major purposes, which is to keep the vendor honest and ensure that the vendor has complied with the certification requirements imposed by the state.”

The Conflict of Interest was even more concerning, because a number of Hash Checks conducted on ES&S Software Failed Verification, but ES&S Withheld this Information from the Secretary of State’s Office because it deemed the Failures to be Insignificant and Not True Failures.

Jacob Stauffer, Vice President of Operations for Coherent Cyber, who Conducted Voting-Machine Security Assessments for California’s Secretary of State for a Decade, said Vendors often do Hash Checks for Election Officials because the Process can be Daunting to Manage. “[N]ot to be saying anything bad about election officials, but ES&S hash verification process may be over the head of some of these election officials, and it’s not intuitive whatsoever,” he said.

An ES&S Spokeswoman said that in the wake of the Texas Findings, it’s “streamlining” the Hash Validation Process to make it more Intuitive for Local Jurisdictions to Perform. “The new process will reduce the number of steps which must be performed to help ensure the process is completed accurately,” Katina Granger wrote in an Email.

Susan Greenhalgh, Senior Advisor on Election Security to Free Speech For People, an Election Integrity Organization, says the Issues in Texas, illustrate how the Legitimacy of Election Integrity Checks, can dissolve quickly when Inspected closely. “Though it has its limitations, hash verification is an important tool for checking voting system software versions. Unfortunately, because of the fatal flaws in [ES&S’s] hash verification script… ES&S hash verification has little to no meaning,” she said.

The Script that conducts the Hash Check is an Open-Source Application called “diff”; the Script that Reports the Results is written by ES&S. The Problem Mechler found was that the ES&S Reporting Script would indicate that Hashes Matched even when there weren’t Two Hashes being Compared. Mechler Discovered this by Accident One day when he Forgot to Upload the Trusted EAC Hash to the System doing the Hash Check.

Although the “diff” Script produced an Error Message indicating it couldn’t find the EAC Hash to do a Check, the ES&S Script reported that the Hash Check was Successful, even though No Verification was Done. It was the same with every ES&S Voting Machine Model Mechler Tested, the Company’s DS200 Optical-Scan System, One of the most Popular Voting Systems used across the Country, the DS450 and DS850 Optical-Scan Machines and the Company’s Line of Ballot-Marking Systems: the ExpressTouch, ExpressVote, and ExpressVote XL.

In a Report he wrote for the Secretary of State’s Office, Mechler criticized ES&S for producing a Poorly Written Verification Script. “[The script] should have performed explicit checks on the existence of the two files being compared; failing loudly if either does not exist,” he wrote. “[T]his bug … indicates that ES&S has not developed their hash verification process with sufficient care, quality assurance, and concern for usability.”

But it wasn’t just ES&S that Failed to Verify the Script’s Accuracy. Vendors are Required to include their Verification Method or Tool with their Voting Software when they Submit the Software to Federal Labs for Testing and Certification. But there’s No Indication that the Labs Verify that the Vendor Verification Methods and Tools work; they simply Check that the Vendor submitted a Tool “[T]his bug … indicates that ES&S has not developed their hash verification process with sufficient care, quality assurance, and concern for usability.”

The Government Accountability Office (GAO) recognized this Problem more than a Decade Ago, in a Report it published in 2008. The GAO advised the EAC to create a Repository of Certified Voting Machine Software, establish Procedures for Conducting Hash Checks of that Software, and create a Protocol for Testing Vendor Hash-Check Tools and making sure they work. Four years later, however, the GAO noted that the EAC had Ignored their Advice and had No Plans to develop Standards for Hash-Verification Tools or a Testing Protocol to Verify that they work properly. This meant, the GAO wrote, that State and Local Jurisdictions would lack “the means to effectively and efficiently verify that voting systems used in federal elections are [using] the same [software] as those certified by EAC.”

“The GAO predicted, almost ten years ago, that without reliable testing of the hash verifications by the EAC, states would not have [the reliable] tools they needed to verify voting system software,” Greenhalgh says. “And that’s exactly what happened.” She believes that a Failed Verification Tool should Disqualify a Voting System from being Certified.

An ES&S Contract with Collin County, Texas, which Stipulates that the Company’s Employees must Conduct “acceptance testing” of their Own Software and Equipment, or else Officials will Void their Machine Warranties. Acceptance Testing occurs when a State or County receives New Voting Equipment or Software, or Updates to Existing Software. Officials check that All Components are present and working properly. And in Texas, the Acceptance Testing also includes a Hash Check of the New Software.

Given that the Collin County Contract stipulates that ES&S does the Acceptance Testing of its own Equipment, instead of Election Officials, this suggests that the Company does the Hash Check of the Software as well. But Granger Insists that ES&S Technicians Stop-Short of Conducting the Hash Check. In any case, Greenhalgh and others say that even giving ES&S Contractual Authority to do Acceptance Testing of its Own Hardware and Software is a Bad Practice that provides Opportunity for a Rogue insider to Subvert the Systems and Cover their tracks.

Texas Oofficials contacted the EAC to weigh-in on the matter. The EAC in turn contacted the Voting Machine Testing Labs to get their Advice. After Reviewing that a Outdated Bitmap File, caused the False Hash Problem, the Labs agreed with ES&S that the Mismatched Hashes weren’t a Concern. Then with the Presidential Election, just weeks away, the Labs, the EAC, and Texas State Officials, all agreed that, with the Correct Bitmap File, Jurisdictions could use the ES&S Software.