Otoy’s cloud tools can render stunningly realistic video (exclusive)

Filmmakers and game creators would love to create animations so realistic that they look like they were produced inside a Star Trek Holodeck, the virtual simulator from the sci-fi TV and film series. In the Holodeck, you couldn’t tell whether what you were seeing was real or was just a computer animation.

We’re not there yet. But the folks at Otoy — including some of the best computer film animators and graphics experts in the world — believe they’ve taken the first step toward the Holodeck. And it’s all part of a plan to democratize game and film development so just about anybody can create outstanding animations.

A big vision for cloud films and games

Otoy’s team has created cloud-based animation tools that can create hyper-realistic imagery like the pictures in this story, which are stills from Otoy animations. And they will put this technology in the hands of people who could otherwise never afford such special effects. These technologies can be used to create imagery that accurately creates lighting in an animated scene without requiring a huge amount of supercomputing power or programming know-how.

Otoy uses a combination of server technology and a user’s own computer to calculate 3D graphics scenes that look spectacular. The Los Angeles-based company believes that future game consoles will use this kind of web-based technology to deliver outstanding game experiences to users. Otoy won’t make these consoles itself, but its potential partners could do so and license Otoy’s technology.

“Our view is that next-generation cloud and console games, where there is a graphics chip on the client, will go down this path,” said Jules Urbach, chief executive of Otoy, in an interview. “We want to make this available to a broad range of artists and developers.”

For example, indie filmmakers and game creators will have access to cloud rendering technology that was only available to big-budget filmmakers before. Sometimes, even the big-budget movies don’t use these lighting technologies because it takes too long to render. Urbach said the scene in this link, for instance, is so hard to do that most big-budget films skip this effect, known as caustic lensing (where light intensifies on the wood after bending through the glass).

Otoy has built a cloud-based rendering technology that uses powerful graphics chips inside servers in a data center to create highly realistic images. Those images are built with a graphics technology known as “ray tracing,” where a 3D scene is calculated by sending out virtual “rays” from a single point of view. If the rays collide into an object, they bounce back and reveal the limits of how far a ray of light can penetrate into the scene.

Even more complex than ray tracing is path tracing, where rays are used to create every pixel in a scene. Otoy’s tools can be used to render extremely realistic “path tracing” scenes for modern video games or films.

The product rollout

Otoy will roll out four different tools this year and next for game developers and filmmakers. For consumers, the result will be some eye-popping special effects and animations delivered at a low cost.

“The developer tools are part of a broader platform we are building,” Urbach said.

The tools include Octane Render for film animators (here’s an example of a student project). Octane Render has been in the works for 18 months with 10,000 beta test users and it will be live by the end of the year. The technology includes the first “spectrally correct graphics processing unit path tracer,” said Urbach. That means it can be used to animations where it feels like you’re looking at photos. Octane Render will likely be available for under $300.

Meanwhile, Otoy will also launch Brigade, the first path tracer for rendering games which can create imagery in real time but not quite at the level of quality needed for films. Developed by Bikker in 2010, Brigade will enable game developers to make much more realistic games with scenes where it feels like you’re walking around inside a photo where the scenery and characters look real, Urbach said. If there are flaws in the image because your graphics chip in your PC can’t keep up with the demands of the image, then Otoy’s cloud technology will detect that and send down the data that is needed for the image at a high speed.

“If there is noise in the path tracing, we can patch that with the streaming technology,” Urbach said.

The Otoy SDK will also include a WebCL compiler, a portable super-fast OpenCl-based cloud service for Intel, Nvidia, ARM and Advanced Micro Devices hardware. The WebCL software can run cool imagery in a web browser without the need for a plug-in. That means that users won’t have to download huge files just to be able to view path-traced films or games on the web.

Building a brain trust

Otoy enlisted some top talent in graphics technology to create the tools. Rendering experts Terrence Vergauwen and Jacco Bikker worked with Otoy chief scientist Rob Wyatt and noted animator Paul Debevec. Wyatt hinted at the technology when he spoke at our GamesBeat 2011 conference, where he noted that high-end graphics will be viewable on any kind of device, not just high-end game consoles.

“You write games for this platform and play on any device, anywhere,” Wyatt told the GamesBeat 2011 audience. “We can do Pixar-quality games in real time. We could play anything from Tetris to Uncharted. As long as it can decode a stream, we can play it on that device.”

If Otoy’s technology works, then the game consoles may never see another generation of new hardware, Wyatt said. You could buy a smart, connected TV with a game controller and use that to play streamed games on the TV, without a console. Console makers can adapt to the cloud or become obsolete, he said.

Otoy also plans to include its LightStage capture software, which allows developers to capture someone’s face and create an accurate computer model of it, in the Otoy SDK. That technology was used to create film effects in films like Avatar, Tron Legacy and the Curious Case of Benjamin Button. LightStage was started in 2008 by Debevec and Tim Hawkins as a subsidiary of Otoy. The LightStage team won an Academy Award for LightStage’s use in Benjamin Button, Avatar and Spider-man 2 and 3. Now LightStage is used in almost every film with computer-generated actors. Otoy will include LightStage in its SDK next year, which will enable indie filmmakers to get access to the same technology via off-the-shelf equipment. Over time, Otoy wants to make it simple to include LightStage effects in any game or film, Urbach said.

Otoy has 25 employees. Urbach declined to comment on whether the company has raised money.

The road to deployment

Once all of these technologies are out in the market and become commonplace in games and movies, then Otoy will be on its way to developing a true Star Trek Holodeck. Debevec has already begun research on what a display (link to PDF) for such a Holodeck would be like. The approach is doable, Urbach said, but it’s not necessarily cost-effective now. At some point in the future, it could be.

To make these technologies widely accessible to developers and consumers, Otoy will first have to roll out its own network of servers in the North American region. Urbach says that will happen later this year and that Urbach has a large partner to help it do that.

Both Octane Render, which Vergauwen has spearheaded, and Brigade have been in development for more than two years. Both are deeply integrated into Otoy’s streaming technology and are part of Otoy’s software development kit. Otoy will sell the Octane Render product as a tool for developers once it is ready.

There are still some challenges Otoy has to overcome. It has to complete the developer tools, launch a network of servers across North America, and find ways for larger numbers of users to use that network. Once it does that, more and more developers will be attracted to using the Otoy tools. If those developers make full use of Otoy’s cloud service, it will be a success and Otoy will be on its way. Cloud-based game consoles could emerge that use the Otoy technology at some point in the future, bringing the Otoy technology to the masses of game players. But Otoy will have to accomplish all that in the face of competition from OnLive, Gaikai Electronic Arts, Valve, and GameStop.

OnLive in particular has a considerable head start in signing up consumers for its cloud-gaming service. While Otoy uses the cloud in a similar way to OnLive, Otoy isn’t planning to directly challenge OnLive in a subscriber-based game streaming business, because Otoy’s customers will be game developers, not consumers. But it’s still possible that Otoy’s partners could challenge OnLive.

Where Otoy has an advantage is in the software it has created. It has tapped some of the best graphics experts to create software tools that can render realistic scenes without a lot of computing power. The software taps the ability of graphics chips to do non-graphics computations through software such as Nvidia’s CUDA programming environment or OpenCL.

What the tools can do

The cloud renderer can handle tasks that once took days to do, such as the rendering equation created by Microsoft researcher James Kajiya in the 1980s. What once took days to render can now be done in real-time on a single 2009-era graphics chip. One of the technologies that the cloud technology can handle is ray tracing.

The Otoy Brigade solution running on a single graphics chip can process a billion rays per second. Each additional graphics chip adds another 1 billion rays per second of rendering power for calculating scenes. The content creators can render their physically correct ray-traced scenes on either the client or the server, or both in tandem.

If a user’s system has a powerful graphics chip, then Otoy’s 3D scene player, the ORBX codec, can create the scene as needed. But if the scene is too sophisticated, then the server kicks in and compensates for the weakness of the user’s computer. If the client machine can’t download a 3D scene fast enough to play its smoothly, then the server does all the work and sends a two-dimensional video displaying the scene to the user’s machine. Thus, even users with relatively weak hardware can view sophisticated cloud-rendered scenes and therefore view cool animated films or play games with outstanding graphics.

Octane and Brigade are unique compared to other path tracers because of their speed and quality while rendering in real-time. Both can handle path tracing at 60 frames per second on a single graphics chip, as shown in this video. Wyatt, who worked on the original Xbox and the PlayStation 3 as well as lots of games created by Insomniac, is the architect of the Otoy render technology. Otoy is adapting several other rendering solutions that are appropriate for other markets and other uses of path tracing.

Check out a couple more cloud-rendered images below. They look like photos, but they are all computer-generated animations built with the Otoy technology. Among the artists whose work is displayed here is Cornel Swoboda.

This house was rendered via Otoy’s Octane Render beta tool.

This fake woodpecker was created with the Octane Render tool.