The zCloud revisited: lessons from Zynga’s public-private cloud experience

Recently, we were proud to publicly unveil the private cloud computing infrastructure that we use to scale our social games — infrastructure that we internally (and affectionately) call zCloud. zCloud leverages both our internal infrastructure components and our public cloud partner, Amazon Web Services. This hybrid cloud, using private and public clouds in unison, allows us to scale our social games efficiently and effectively for our millions of daily players.

The first rollout of zCloud was focused on building an Infrastructure-as-a-Service (IaaS) model designed specifically for social games in terms of availability, network connectivity, server processing power and storage throughput. And as we built and deployed zCloud, we challenged ourselves to drive innovation past the concept of a hybrid cloud and ask what other services zCloud could provide to our social games. We wanted to understand how we could make Zynga’s games faster, more reliable and easier to scale.

Before we could answer this question, we needed to understand the infrastructure requirements of each of our social games. Zynga is a particularly challenging environment to find exact computing requirements since each of our games is unique. Each one uses a variety of programming languages, storage mechanisms and social mechanics. On top of that, games continuously evolve, and we constantly explore the boundaries of new technologies to make our games more social.

To begin, our teams spent time building instrumentation for each layer of our application stack and understanding the existence of potential bottlenecks. We did this instrumentation both on zCloud and on AWS – this step was vitally important to help us understand our games’ operating environments. In some cases, we found servers that were constrained by the multi-core CPU performance, in others we found that our networking throughput or storage I/O was a limiting factor or that we had over-provisioned memory on some servers. In a private cloud, you need to measure how your applications perform and build matching infrastructure.

Because zCloud is our own private cloud that we can manipulate and control, we were able to optimize our infrastructure to quickly alleviate nearly all of these potential bottlenecks. Public clouds, while exceptionally good at providing a wealth of services for a great number of customers and computing needs, often leave little flexibility for infrastructure optimization.  We found that we had to modify how our games operated for their environment.

The flexibility that zCloud provides for our games is another key to our ability to scale. When we conceived of zCloud, instead of thinking about building infrastructure for specific games, we thought about the need for every one of our games to operate on any server in any geography – or on every server in all geographies. It was essential that we understood the implication of this flexibility on networking, load balancing, storage, caching, security, application messaging, and so forth.

As the next step in building zCloud, we looked to build software and tools that allow us to operate and scale faster. These new automation tools allow us to provision services in minutes that used to take days to deliver and validate. We can now consolidate and correlate thousands of alerts into a single notification, associating specific processes for people to perform. We also built a reporting dashboard that shows both real-time and historical information about the performance of each game on zCloud.

It’s this next set of zCloud services that has made our games perform faster, scale better and be more reliable. Our keys were instrumenting our application well, building the right infrastructure to match the application needs and thinking about the flexibility that a private cloud allows for our games.

As unique as Zynga’s infrastructure needs may be, these are tenets that all businesses may want to consider when exploring cloud deployments.

Allan Leinwand is the chief technology officer for infrastructure engineering at Zynga. He is responsible for all aspects of technology infrastructured used in the delivery of Zynga’s social games, from data centers to cloud computing. He was previously a venture partner at Panorama Capital and founded Vyatta in 2005. He spoke with Dean Takahashi, lead writer for GamesBeat at VentureBeat, at our CloudBeat conference this week.