How JavaScript will lead the way to open video

Editor’s note: This story is part of our Microsoft-sponsored series on cutting-edge innovation. Shay David is the vice president of business and community development at Kaltura, a company offering video tools for publishers.

The “open Web”, a vision for the future of the Internet that is participatory, collaborative, and free from vendor lock-in is finally coming to fruition. Following Mozilla Firefox’s successful wedge of open Web standards into the browser platform, today we see every browser vendor and every web-enabled device gravitating towards supporting a vendor-neutral platform for rich media web experiences.

Like many contentious agreements, the devil is in the details — and there are a lot of details. This article will highlight how the industry is transitioning away from targeting a single vendor (Adobe Flash) for rich media web experiences to instead targeting multiple rich media web browsers. In this environment, middle-layer solutions will bridge the small differences between implementations.

Businesses looking to take advantage of the promise of a single platform to deliver rich web experiences should also be looking towards middle layer solutions to help bridge the small gaps in rich media implementations.

The open Web has traditionally faced significant challenges when it comes to the handling of rich media, including video, audio and multi-media. Rich media on the web has been a battle ground for proprietary solutions for years. Adobe Flash has won the current battle and is now the de facto standard for rich media on the desktop web. But it’s not supported by the iPhone or the iPad. [Editor’s note: In fact, Apple chief executive Steve Jobs published a long essay this week criticizing Flash.] The new specifications for rich-media handling that are part of HTML5, the latest version of the basic language of the Web, are a great leap forward in addressing these challenges. They specify a vendor neutral, device neutral way of including rich media in webpages, just like images are handled today.

However, HTML5 requires a transition period. Until there is wide industry adoption and universal agreement on all the details of the specification, systems will be required to support existing media playback systems in cases where HTML5 is not supported. In the next few years, we will likely see open source JavaScript middle layers drawing from both Flash and HTML5 to build robust solutions for rich media web experiences. This robustness will include handling everything from analytics to playlists. During this transition period, developers can use JavaScript middle layers to bridge the current gaps between Flash and HTML5, in order to provide a unified user experience regardless of the browser and the underlying technologies that are used.

Some background about HTML5 and the future of rich-media on the Web

HTML5 aims to reduce the need for proprietary plugin-based rich Internet applications. The promise of HTML5 is that developers can write rich-media applications once, and run it on any browser, on any device, from PCs through tablets and smartphones to set-top boxes and IP-enabled televisions.

The HTML5 working group, WHATWG, is trusted with the mission of upgrading the web to support rich web experiences without dependence on proprietary technologies. True to its mission, the group has been operating in a very open manner. While key working group members are employed by major vendors, the standard creation process is open and encourages large-scale participation from web developers and browser engineers.

With all major browser vendors providing feedback or working implementations as the spec evolves, there is room for innovation while addressing a wide range of stakeholder concerns. This is no small task when vendors like Microsoft, Apple, Google, and Adobe compete for dominance in an area of rich-media, which they all perceive to be key for the future of the Web. With Microsoft’s announcement a few weeks ago that HTML5 will be supported in the next release of Internet Explorer, the standard got a big boost. But while the standard sets a high bar for interoperability, debates about the underlying technologies put the whole endeavor in jeopardy.

The most heated debate revolves around the codec agnostic nature of the HTML5 video tag. Many open web advocates promoted the idea of having a baseline, royalty-free codec supported by all the browsers be part of the specification. However when trying to get convince all the stakeholders, the standard was not able to specify a standard codec, leaving codec selection open to vendors. As a result, Apple’s Safari and Microsoft IE9 support the h.264 codec, Google Chrome supports h.264 & OGG, and Firefox exclusively supports OGG/Theora. The situation is further complicated by multiple delivery options. For example, an Apple platform will support Apple http adaptive streaming while the Google Chrome browser can only decode normal http h.264 assets, and Microsoft is likely to support Silverlight Smoothstream as a delivery option.

In other words, although all the browser vendors are participating in the HTML5 standard around video, it does not nessesaraly mean the same video with the same web page will work on all browsers.

For example, imagine a visitor who comes to your site with IE6 and Flash, or with iPad and Safari. Delivering a high quality web video experience becomes almost impossible once again, despite the standard, as the codec and delivery options require once again browser specific and device specific behavior. Moreover, as we have seen with other standards, there are likely to be many idiosyncrasies among other HTML5 feature implementations, or simply incomplete support among browsers. This will make it even more difficult to take advantage of the new features without leaving a sizable percentage of web visitors literally in the dark.

In this context, it is clear to see why the proprietary platforms like Adobe’s Flash will probably be around for quite some time.

We have been here before

If you rewind the Web a few years back, you may remember the nightmarish undertaking it was to develop any rich web experience that supported all the browsers of the day. This was not only because the browsers were not mature but also because we lacked quality JavaScript libraries that could handle browser detection and act appropriately. The latest development models were outdated.

In early 2006, jQuery hit the scene and changed the way web applications were developed and deployed. jQuery supported the emergence of new paradigms that fit what web developers actually needed. The JavaScript layer with open source development methodology enabled developers to share workarounds and chain together shortcuts of common tasks. This enabled developers to quickly build innovative rich web experiences without having to worry so much about the increasingly complicated set of underling web platforms.

Today it’s hard to imagine building a web application without using a JavaScript framework. jQuery alone is used by nearly 30 percent of the top 10,000 web sites.

In much the same way, we anticipate JavaScript libraries to emerge as bridges between the underling complexities of delivering a robust HTML5 feature set. Web developers will not have to worry about technicalities such as how their video will be played back or how their cross domain request will work. Rather, they will just issue the high level call and the underling JavaScript library will automatically map to HTML5 or Flash accordingly, allowing web developers to focus on the innovative aspects of their application, not its “plumbing”. The browser platforms are rapidly evolving to include full parity with desktop applications, including features such as 3D graphics and offline storage. As more web applications begin to make use of these feature sets, workarounds and solutions for older browsers or specific platforms and devices will become even more critical.

HTML5video.org and Kaltura’s JS library

Kaltura has developed one such library to handle HTML5 video, and has created an industry resource to get developers involved. The library was developed in partnership with the Wikimedia Foundation to handle video on Wikipedia, where only patent unencumbered free formats are supported. Kaltura’s HTML5 media library uses a “fallback” method to automatically provide the viewer with the optimal viewing experience by detecting the browser and supported formats on the backend, and displaying the right content in the right format and right container while maintaining a single look & feel and feature set. For example, if you want to deliver an h.264 video to a Firefox client, the Kaltura library would automatically switch to a Flash player in a manner that is completely transparent to the developers’ custom interface components.

To learn more, to experience HTML5 video in action, to give feedback, to get involved and to become part of changing the way video is done on the Web, go to www.HTML5Video.org.

[Thanks to Michael Dale for help with an early draft of this post.]

The community responds

— Charles McCathieNevile, Chief Standards Officer, Opera

JavaScript libraries have become a critical tool for web developers, and they have come a long way since Macromedia produced probably the first really widely-used basic effect libraries in the last century. In particular they are helpful in dealing with cross-browser testing. Some demos do not recognize that Opera on my Mac understands HTML5 video and the free Ogg format, and want to give me a Flash version – they would obviously benefit from a well-written and well-maintained library. Likewise, libraries are important in reducing coding errors; copying in an entire maintained library is easier than remembering to write all the right pieces of code yourself.

But video does need to be coded in different versions for the time being — while more or less automatable, that’s still a piece of work you have to get right before the scripts can do all their magic. And the libraries need to be built well and used properly.

While these libraries are an important part of the solution, and for many authors will be indispensable until we get agreement on a free standard for video format, we need to remember that reaching that particular nirvana actually has practical benefit for the world — less need for script libraries means less processing, less battery used, less bandwidth, fewer programming errors (one of the original problems that the libraries already help to minimize) and a Web that works better.

— Caleb Elston, Vice President of Products, Justin.tv

It’s exciting to see the enthusiasm and support that browser vendors and technology companies have thrown behind HTML 5 video, but there are still issues to be resolved before we make a big move to adopt it. Our goal is to make live video part of the everyday web experience, and that means being wherever users are and creating a great experience — HTML 5 isn’t the way to do that, at least not yet.

First, supporting two new video codecs in H.264 and Theroa for a massive video library, like Justin.tv’s, is extremely painful; we hope to see unification on that front. Second, only a small percentage of our users even have the ability to view HTML5 video, and as a startup we will continue to focus on serving as large an audience as possible to grow our business. Finally, Justin.tv users embed their live and past broadcasts all over the web, and we have yet to find an easy way to make embeds usable on sites that limit Javascript use, like WordPress.com and Google’s Blogger.

While HTML5 is technically exciting, we only care about what our users actually experience, and today we do not see much user benefit to HTML5 video in desktop browsers.

Don’t miss MobileBeat 2010, VentureBeat’s conference on the future of mobile. The theme: “The year of the superphone and who will profit.” Now expanded to two days, MobileBeat 2010 will take place on July 12-13 at The Palace Hotel in San Francisco. Early-bird pricing is available until May 15. For complete conference details, or to apply for the MobileBeat Startup Competition, click here.