Cloud devs: We could have saved buggy HealthCare.gov

VentureBeat’s Eric Blattberg contributed reporting to this story.

Millions of Americans flooded the new health insurance marketplaces when they opened for business earlier this month.

What we didn’t expect were the glitches, bugs, error messages, and a website that looks like it’s from 10 years ago. Almost two weeks later, people are still struggling to log in to the Obama administration’s online insurance marketplace, HealthCare.gov, let alone shop for insurance.

The marketplace is intended to serve those who don’t have a state-run option.

The promise of the Affordable Care Act is that every U.S. citizen who lacks health care should have the opportunity to shop for it under terms of guaranteed coverage, with no rate inflation for preexisting medical conditions. However, both federal and state-run exchanges experienced technical difficulties, which prevented Americans from enrolling and buying insurance.

The technical problems with HealthCare.gov were particularly profound.

The feds pointed to server overload, supposedly caused by too many people rushing to the website. Reporters attempting to assign blame to one of the two-dozen IT contractors behind HealthCare.gov have found problems at every turn. HealthCare.gov failed to meet expectations and deliver a quality user experience due to series of missteps — financial, technical and managerial.

The government is keeping mum about the organizations contracted to build the site — but legacy giants like Oracle, Quality Software Solutions, Booz Allen, and CGI Federal were all assigned to perform different functions.

When too many cooks are in the kitchen, everyone is at fault — and no one is.

Proponents of the cloud and open source technologies say these technical bugs could have been avoided by the use of modern technologies.

The problem, they say is that the federal exchange was supported by legacy systems. So legacy, in fact, that Henry Chao, the chief digital architect for the project, feared that Healthcare.gov would feel like a “third world experience.”

“Facebook wasn’t built in a day”

Frank Febbraro, chief technology officer for Phase 2 Technology, which has built systems for the White House, Homeland Security, and other government agencies, believes that Healthcare.gov’s problems are temporary. The government is leaning heavily on its contractors to ensure that Americans can enroll for insurance before the mid-December deadline to sign up for coverage that begins Jan. 1.

Todd Park, the President’s chief technology advisor, said that the problem has proven more complex than adding more servers. To add extra computing power, developers will move part of the site from using virtual machine technology to dedicated servers. Febbraro reminded the press in a recent interview that “Facebook wasn’t built in a day.”

John Engates, chief technology officer of cloud company Rackspace, isn’t buying it.

“Healthcare.gov is no Facebook; nor will it ever be,” said Engates. “The potential user base is only in the millions, [and] they probably know the exact number of potential users based on who doesn’t have insurance.” (It’s worth noting, though, that in the site’s first few days of operation, it very likely drew large numbers of people who weren’t there to sign up for insurance — government workers, Obamacare opponents, press, insurance reps, health care providers, and members of the general public looking for answers about what the Affordable Care Act will actually look like.)

With the government embarrassed by the failure of Healthcare.gov and many of the state-run exchanges, startups are taking the opportunity to elucidate how technology has evolved in the past decade.

We saw many of these modern, cloud-based technologies in use by software developers during the Obama re-election campaign. As Slate writer and software engineer David Auerbach points out, these brilliant young programmers did not build Heathcare.gov, one of the most pivotal websites of our time. To get close, they would have had to work for a company mired in bureaucracy and procurement regulations.

Indeed, building a functioning website like Healthcare.gov isn’t an insurmountable feat. It’s the contractor procurement process that’s so desperately in need of reform.

“The type of companies we see doing IT consulting work for the government are not the nimble, forward-thinking technologies that we see in Silicon Valley,” said Dan Scholnick, cloud computing investor at Trinity Ventures. Outside experts have questioned whether HealthCare.gov was run on cloud-based technology, which is equipped to adjust to spikes in traffic. For instance, Amazon, eBay, and other e-commerce sites leverage cloud-based scalable technologies and prepare for huge traffic spikes the week before Christmas.

State vs. federal exchanges

The state exchanges appear to be more nimble and less balky than the site run by the federal government.

California’s state exchange, called Covered California, was briefly shut down over the first weekend, but is now up and running. It’s not yet clear how many people have bought insurance in California on the exchange since its debut, but we should know more in due course.

Chini Krishnan, chief executive of venture-funded startup GetInsured.com, signed a deal with the Centers for Medicare & Medicaid Services, the government agency responsible for issuing the contracts. GetInsured’s software, which competes with sites like eHealthInsurance.com, is now integrated into Covered California’s web portal.

Krishnan has become an outspoken proponent of cloud services. GetInsured’s technology is cloud-based and delivers one aspect of the exchange: a service for small businesses to provide their employees with access to private insurance. GetInsured is partnered with Accenture, which is the government’s primary contractor in California.

“I think the real story here is that this modern software-as-a-service approach works,” said Krishnan by phone.

California was among the first states to commit to setting up its own private health exchange, so it wasn’t a last minute scramble. The same story is true for New Mexico, another state that incorporated aspects of GetInsured’s software into its exchange.

States like California and New Mexico were also able to test their more modern environments to ensure that the exchanges could cope with traffic spikes.

“Because of cloud computing, you have the ability to simulate as many users as you want,” said Tom Lounibos, president and chief executive of SOASTA, a cloud testing service. On the subject of the federal exchange, Lounibos added, “There really wasn’t any excuse that the testing wasn’t done.”

Will the government embrace the cloud?

The government’s pace of adopting new technology has been glacial. For even the most minor tech projects, approval processes are slow, systems are typically on-premise, and much of the monitoring and upkeep is outsourced to contractors.

However, experts believe that government agencies will have no choice but to turn to modern technologies and the cloud. In the past, the cloud has been perceived by government officials as less secure than on-premise technologies, but recent advancements in security protocols are diffusing these concerns.

“The government will begin to start working with firms that are specialized in the cloud and health IT,” Krishnan predicts.

This prediction is already ringing true. In recent months, Microsoft Azure announced that it will offer a cloud platform specifically tailored to the government. Amazon Web Services won a $600 million contract over IBM to build a private cloud service inside the CIA’s data centers.

Krishnan told me that other states are now considering cloud-based services like GetInsured, rather than relying on the sluggish federal exchange. Mississippi is in talks with GetInsured, so it can get aspects of its service running fairly quickly. The decision whether to run a state exchange or rely on the federal government caused some intense rifts among Mississippi’s political leaders. In the end, Mississippi was one of the states that refused to create and run a state-based exchange.

Loubinos refers to the failures of the health exchanges as a “kind of a cautionary tale for tech in general.” Many of the issues have now been fixed, but the government faces a more difficult task ahead to alleviate the damage to the public’s confidence.