Theranos’ weirdest sin: Sloppy record keeping

Give Theranos some credit. After months of waging a counter PR attack to blunt negatives stories, the company has shifted gears and appears to be trying to be more transparent.

For instance, in announcing the latest sanctions by federal regulators last week, Theranos also opted to release a fully unredacted version of the letter it received from the U.S. Centers for Medicare & Medicaid Services. (Read it here.)

“While it was obviously difficult to hear the outcome of the CMS findings, we will work non-stop to resolve the issues identified,” the company wrote on its corporate blog. “In an effort to be open with the public and express our continued commitment to realizing the highest standards of quality and compliance, we are sharing the letter from CMS.”

Great. Having read through the CMS letter over the weekend, and some of the previously disclosed public documents published by the Wall Street Journal, I will confess that there is quite a bit of the scientific process and jargon that escapes me.

But one thing that did strike me in the recent CMS letter was what sounds like Theranos’ completely chaotic records keeping regarding its products, testing, and training. Indeed, in some cases the CMS wasn’t necessarily rejecting Theranos’ arguments, but rather was saying that the records provided were simply impossible to understand.

For instance, on page two of the letter, there is a section called “Flash Drives.”

In response to concerns raised by CMS, Theranos delivered five password-protected flash drives with relevant records to bolster its case. There was just one problem, according to the CMS letter: “The information on the flash drives was difficult to evaluate and access.”

It seems that data on any given patient was scattered across different flash drives, requiring investigators to manually comb through each document for a patient to see if it matched any other patient records on other flash drives. This made it impossible, the CMS said, to determine whether Theranos had provided the complete patient records the agency had requested.

The agency hit another records-keeping bump in the case of documents related to “a revised quality assessment mechanism.” The agency said Theranos “submitted no documentation to indicate that the revised quality assessment mechanism has been effectuated.” Other training documents were incomplete or inconsistent, making it impossible for regulators to determine whether proper training had been given, the CMS said.

In another section, the CMS notes that Theranos failed to submit certain types of documents, such as those that might “indicate that any patient results were evaluated based on the requirement in the procedure, especially at the borderline abnormal high level.” In another area, a response Theranos gave regarding a testing procedure misidentified which of the CMS complaints it was meant to be addressing. “We note that the procedure refers to incorrect sections,” the CMS wrote.

While the CMS figured out which were the right ones, it said Theranos was still missing some critical documentation to support its claims. “However, there is no evidence submitted to support that patients were not impacted, nor was there documentation showing that the investigation included a review of patient results as required by the revised procedure,” the CMS wrote.

Underlying all of this confusion was that fact that Theranos was having to go back and correct many of its original patient reports, which the company acknowledged had incorrect data. But again, the CMS said it appeared that Theranos had only provided a portion of the corrected reports.

Finally, the CMS dinged Theranos over its quality control process. Again, while the company provided a statement by an employee that quality control happened, the company did not provide any documentation for certain time periods that such a program existed, and if so, how it worked.

The CMS wrote: “Again, however, the laboratory did not provide any documented evidence that a QC procedure existed or was in use prior to May 15, 2014. We also note that the attestation states that the employee’s duties included running QC, which is contrary to the information given to the surveyor at the time of the onsite survey, but did not indicate if he/she had responsibility for evaluating QC prior to releasing patient test results.”

As a result of this record-keeping breakdown, CMS said: “The fraction of patient results truly impacted, and the nature and magnitude of any effect, are unknown.”

At yet another point, Theranos said it has stopped using a particular machine at a certain point. But documents it submitted indicated that it had continued to use the machine.

“However, documentation given to the surveyor by the laboratory at the time of the onsite survey indicated that the laboratory did, in fact, use the BCS XP for patient testing after September 17, 2015,” the CMS wrote. “The laboratory has not provided sufficient evidence to rebut the information contained in that documentation. Also, these contradictory statements in the submissions call into question the reliability of the information contained in the submissions.” (emphasis mine)

The letter goes on in that vein regarding missing or inaccurate or incomplete documents. Given the nature of its work in the realm of science, one would have thought that documenting every step of every process would have been a central part of Theranos’ operations.

And given that the company hired one of the nation’s most reputable law firms — Boies, Schiller & Flexner — to defend it, one would also assume that every submission would have been thoroughly reviewed and polished to help accelerate any review process.

Instead, company founder Elizabeth Holmes is facing a two-year ban. And the company is faced with the daunting task of still responding to the CMS accusations while rebooting its lab and keeping the faith of its employees and investors.