FriendFeed working on a new RSS supplement to speed up data retrieval

It looks like the rapid fire site updates are about to start again for the social content conversation site FriendFeed. Just a few days after the launch of its new “beta” area, FriendFeed is finalizing a new technology that could help pull content into the site at a much faster rate.

The technology, called Simple Update Protocol (SUP) will process updates from the various services that FriendFeed imports faster than it currently does using traditional Really Simple Syndication (RSS) feeds, FriendFeed co-founder Paul Buchheit told Tech Confidential.

But it’s not meant to be an RSS replacement, as the Tech Confidential story seems to indicate. Instead, it is “a simple addition to RSS,” Buchheit told us today.

Though he plans to talk about it more next week, the basic gist is that SUP will allow FriendFeed to find new content within minutes or even seconds of it being updated elsewhere on the web. Compare this to FriendFeed’s current method for most services which checks RSS feeds every half hour to hour for updates.

You may notice that updates from the micro-messaging site Twitter seem to come in the fastest to FriendFeed. This is because Twitter gives FriendFeed access to its Extensible Messaging and Presence Protocol (XMPP) data. This can be thought of as a “fire hose” of sorts for data, it sends out all the service’s data as it comes in. An RSS feed, on the other hand, would be more like a timed sprinkler. It sends out flurries of information but only after certain intervals of time have passed. SUP hopes to remove much of timing restriction on that sprinkler.

SUP is being developed by Buchheit alongside another FriendFeed engineer, Gary Burd (the man who was brought onto the team after making the popular FriendFeed application Mail2FF). When it launches, SUP will likely remain in the new beta area for a little while as it’s still experimental, Buchheit explained to Tech Confidential. He also mentioned other “real-time features” being added to the site down the road.

Update: After a flurry of interest in the topic, Buchheit decided to post about SUP a little earlier than he intended. He outlines SUP as follows:

SUP (Simple Update Protocol) is a simple and compact “ping feed” that web services can produce in order to alert the consumers of their feeds when a feed has been updated. This reduces update latency and improves efficiency by eliminating the need for frequent polling.

Benefits include:

• Simple to implement. Most sites can add support with only few lines of code if their database already stores timestamps.
• Works over HTTP, so it’s very easy to publish and consume.
• Cacheable. A SUP feed can be generated by a cron job and served from a static text file or from memcached.
• Compact. Updates can be about 21 bytes each. (8 bytes with gzip encoding)
• Does not expose usernames or secret feed urls (such as Google Reader Shared Items feeds)

SUP is designed to be especially easy for feed publishers to create. It’s not ideal for small feed consumers because they will only be interested in a tiny fraction of the updates. However, intermediate services such as Gnip or others could easily consume a SUP feed and convert it into a subscribe/push model using XMPP or HTTP callbacks.

Sites wishing to produce a SUP feed must do two things:

• Add a special <link> tag to their SUP enabled Atom or RSS feeds. This <link> tag includes the feed’s SUP-ID and the URL of the appropriate SUP feed.
• Generate a SUP feed which lists the SUP-IDs of all recently updated feeds.

Feed consumers can add SUP support by:

• Storing the SUP-IDs of the Atom/RSS feeds they consume.
• Watching for those SUP-IDs in their associated SUP feeds.

Buchheit notes that SUP will not eliminate the need for regular RSS polling, as SUP has the potential to miss updates if a server is down or if the service it’s pinging has a malfunction at the moment it is pinged.

More information can be found on the Google Code page for SUP.

Find me on FriendFeed here, along with fellow VentureBeat writiers Eric Eldon, Dean Takahashi, Anthony Ha, Chris Morrison and Dan Kaplan.