Too small to fail: How HealthCare.gov should have been built

This is a guest post by Practice Fusion cofounder Matthew Douglass.

In the coverage surrounding the failed rollout of the insurance marketplaces on HealthCare.Gov, the same number keeps popping up: 500 million. It’s the amount of money some are estimating has been spent on the marketplaces so far: $500 million. It’s also the number of lines of code that have been written to create the site.

Understandably, neither of these figures may mean much to the average person. To put it in context, Facebook was launched and successfully operated for six years before using up $500 million in funding. 500 million lines of code is ten times the amount used in the development of Windows Vista, a far more complex project, which required only 50 million lines of code.

Related: Read our full report on the challenges of relying on a “tech surge” to make improvements to HealthCare.gov.

Whatever the final cost turns out to be, the marketplace system was built with too much complexity and at too great an expense. To make matters worse, it was built and paid for before any real testing was done at scale. A recent disclosure revealed that the first real wave of testing occurred only a few days before the national rollout date. The testing, which was to verify the effect of ten thousand simultaneous registrations, failed after a few hundred visitors. A product like this never should have been released with such a paltry capacity.

Bringing ‘lean startup’ methodologies to government

None of these missteps would have occurred if the contractors had taken a gradual, agile approach. Agile is the modern gold standard for software development, through which programs are built in stages instead of all-at-once — a technique known as “waterfall.”

At my own company, Practice Fusion, we release new product updates for our physician and patient customers about every two weeks. These releases are tested vigorously to ensure that the code is of high quality and works seamlessly when bundled with the rest of the product.  In other words, we make small changes that we know will work the way it should and provide a good experience for our users.

Of course, you can never completely guarantee that a release will be error-free. With smaller releases, however, we can immediately identify a problem when it arises — whether it’s a user experience issue, an underlying system problem, or anything else that doesn’t allow the technology to operate as it should.

Keep it simple! Excessive code increases risk

Officials at HealthCare.gov have been spending weeks trying to figure out if the problem was due to an unexpected spike in volume, account registration bugs, or problems with the one of the many connections the website has to make to external databases (containing insurance options for each geographical region, among other information).

The contractors’ excessive code just makes matters worse — imagine untangling that ball of computer cords next to your desk if it had ten times the number of wires. Such unnecessary complexity increases the risk of error.

A number of options could have been taken to gradually roll out HealthCare.gov. The site could have been released earlier with only the marketing website and eligibility tools to help visitors check their status before registering.  This kind of modular development also allows users to learn how to use the technology in stages. It also could have been launched in waves to certain sections of the country to ensure that the website could scale effectively and reduce the complexity of connections to external databases to begin with.

The irony of this failed rollout of HealthCare.gov — the hallmark of President Obama’s signature domestic achievement — is that the Obama campaign performed such a masterful job of employing technology in its own campaign. I personally worked as a tech volunteer on this campaign, as a member of the agile engineering organization that generated impressive resources for the campaign’s advertising spend and voter outreach. We were running circles around the Romney campaign — who themselves failed to release in an agile fashion and had issues with its poll-monitoring system Orca on the very day of its rollout, Election Day.

The glaring need for procurement reform

But just because agile is the gold standard of effective software development doesn’t mean everyone has caught up with the times.

The contractors that the government uses are often those who are best able to navigate the convoluted procurement process. This procurement process is intended to protect taxpayer dollars, with many levels of validation and a complex vetting process. However, the process has seemingly had the opposite effect of wasting taxpayer dollars by isolating those companies who are good at government procurement, not development.

We need more open competition among bidding vendors to keep this from happening again — to ensure that we actually get the “best and brightest”, and not the most bloated and expensive. The White House’s Chief Technology Officer Todd Park has already begun working, through the Presidential Innovation Fellow program, to revamp the procurement process and make it easier for small businesses to bid on contracts from the government. Taking this a step further, Park should require contractors to outline how they would develop products in agile phases and clearly demonstrate their progress to encourage iteration and avoid ballooning costs and delays.

There are many reasons to remain optimistic about the health marketplaces — the sheer outpouring of traffic, for one, shows how much demand there is for getting improved, more affordable access to health insurance via the marketplaces. But to succeed, this effort doesn’t need more waste and complexity—it needs smaller, smarter, more agile teams. It needs more Silicon Valley and less Washington, DC — or, ideally, a Washington, DC that has learned the hard-earned lessons gained by technologists over the last 15 years of innovation in Silicon Valley.

As Practice Fusion’s technology co-founder, Matt created the SaaS technology framework that enables rapid development of the electronic health record’s nationwide platform, now used daily by over 100,000 medical professionals. He has spoken on healthcare technology at SDForum, Health 2.0, MIT, Stanford, and Microsoft events.