Xbox 360 defects: an inside history of Microsoft’s video game console woes

When his fourth Xbox 360 video game console died in April, Chris Szarek wasn’t surprised.

The Chicopee, Mass. gamer was accustomed to the hardware failures that became known throughout the Internet as RROD, or the “red rings of death” which flash when the console becomes inoperable.

A 40-year-old photographer, Szarek was a hardcore Microsoft fan who spent more than $1,000 on his games. But each time one of his Xbox 360 consoles failed, he had to spend time convincing Microsoft’s tech support that they should send him a new console. Each time he got a refurbished console as a replacement (a machine that had been returned to a repair center in Texas, fixed as much as possible, and then shipped back out). When he complained on the Internet and to the media about the shoddy product and poor customer service, people branded him a cry baby and wrote him off as a statistical anomaly. But by the spring of 2008, Szarek was vindicated. There were at least a million or two other people like him.

Szarek’s fourth machine lasted almost two years, experiencing the same short life that many other Xbox 360s suffered. Microsoft replaced these machines for free under the warranty that it announced on July 5, 2007, for defective Xbox 360s exhibiting what it more politely called the “three flashing red lights.” That warranty program cost Microsoft up to $1.15 billion, but the loss of face and loyalty among gamers in the fierce console war with Nintendo and Sony has been immeasurable. Szarek, who became a spokesman for dispossessed defective Xbox 360 owners, played a part in making Microsoft acknowledge its console quality problem.

This is the unauthorized tale of how Microsoft lost its chance to become the leader in the biggest market it has attacked beyond its twin monopolies in Office and Windows software. Rival game console maker Nintendo out-thought the larger players Microsoft and Sony by designing the Wii game console with a clever, intuitive game controller. Even so, Microsoft could have captured more gamers during this product generation, yet the RROD problem held it back. The Xbox 360’s defect problem will go down as one of the worst snafus in consumer electronics history.

Its own worst enemy

Microsoft knew it had flawed machines, but it did not delay its launch because it believed the quality problems would subside over time. With each new machine, the company figured it would ride the “learning curve,” or continuously improve its production. Even though Microsoft’s leaders knew their quality wasn’t top notch, they did not ensure that resources were in place to handle returns and quickly debug bad consoles. There were plenty of warning signs, but the company chose to ignore them. The different parts of the business weren’t aligned.

It reminds me of the German war machine just before World War I, as chronicled by Barbara Tuchman in the classic history book, “The Guns of August.” The German generals were intent on keeping their trains on time; but the leaders overlooked their chances for stopping the war altogether. The Schlieffen plan called for them to strike first. Once the Russians and French mobilized, the Germans had to move into action. They marched off blindly into tragedy.

Likewise, Microsoft’s strategy depended on beating its rivals to market. It couldn’t afford to stop and delay the launch in order to solve its quality problems, or so upper management believed. What Microsoft’s leaders didn’t realize was that getting to market first with a flawed machine would only win them a battle; and it risked the loss of the war.

“They got enamored with the idea of the Microsoft army rolling everything out at the same time,” said one knowledgeable source who asked not to be identified.

The quality problem negated much of the advantage of going first, and it has delayed the company’s plan to aggressively market the console and slash its prices. (Microsoft disputes this point; it cut the price of all three versions of its Xbox consoles by $50 to $79 on Wednesday. And the company believes it will sell more boxes than Sony will. But prices ought to be lower still during this stage of the console life cycle). That has stopped the company from reaching the broader market of consumers that Nintendo has won over. It has lowered its ambitions, hoping instead just to get a clear edge on third-placed Sony. The future profits that the company once hoped for are now likely to wind up in Nintendo’s pockets.

Microsoft’s top game executive, Robbie Bach, president of the Entertainment & Devices group, said at a dinner in July that Microsoft’s own research shows that gamers have largely forgiven the company for defective Xbox 360s. Microsoft has still sold more Xbox 360 consoles than Sony to date. But there is no doubt that the company has lost considerable good will among gamers. Before Microsoft offered free replacements, connsumers grumbled that they had to turn to forums, such as those on Ars Technica, to vent and to find solutions to problems that the company didn’t openly discuss. And for a couple of months now, Sony’s PlayStation 3 has been outselling the Xbox 360 in the U.S. for the first time.

“Fundamentally, their thinking shows that they are a software company at heart,” said one veteran manufacturing executive. “They put something out and figure they can fix it with the next patch or come up with a bug fix.”

The terrifying part of the story is that this kind of problem — where technology fails and no one knows what to do about it — can happen to any company.

About this story

I asked Microsoft to confirm or deny 35 different facts contained in this story. Instead, I received a formal statement from a Microsoft spokesperson, saying the company had already acknowledged an “unacceptable number of repairs” to Xbox 360 consoles and responded to the hardware failures with a free replacement program. The statement also said, “This topic has already been covered extensively in the media. This new story repeats old information, and contains rumors and innuendo from anonymous sources, attempting to create a new sensational angle, and is highly irresponsible.”

I don’t think this story is sensational. I have tried to verify the facts over several years. I view this story as the last chapter for my book on the making of the Xbox 360, “The Xbox 360 Uncloaked: The Real Story Behind Microsoft’s Next-Generation Video Game Console.”

The facts revealed themselves slowly, emerging from the day-to-day stories that I wrote about the game business. Some people might consider this post mortem to be ancient history. But the reverberations are still playing out today. They help explain why Microsoft isn’t being aggressive with its price cuts and why gamers aren’t getting bargains on hardware as they did the last generation. While I talked to many people for this story, few were willing to let me use their names. As you will see, not every source is anonymous, and we have included the viewpoint of Microsoft executives from past interviews.

The details are interesting because they offer a deeper look into how the console business runs than is otherwise available. Microsoft, for instance, still hasn’t perfected its Xbox 360 manufacturing process. In the absence of a precise chronology from Microsoft, some anonymous sources have tried to describe what happened. But the history of the decision making and inside story of what happened on the RROD has never been told, until now. (Please check out our GamesBeat 09 conference on March 24).