Gesture recognition technology for games poised for breakthrough

update
In ten years, the gaming instruments we use today will look ridiculously old-fashioned.

The Nintendo Wii is big because it lets you move things on your screen with gestures of your hand, using a motion-sensitive paddle. But that’s nothing compared with what’s coming: controlling the screen with nothing in your hand at all. One day, very soon, you’ll be able to control an avatar or character on a screen with a mere gesture of your hands or body.

A range of companies are working on this dream, but few seem to be succeeding. They’re often hyped and fail to live up to their promise. We wrote about Softkinetic a year ago (it has a very cool demo), but the company hasn’t released any news since last year, and its CEO has since moved on. At that time, Softkinetic demonstrated the product to us, but in a very controlled lighting environment, and they wouldn’t let us personally try it out. From what we saw, though, we were impressed.

Softkinetic’s product required infrared cameras, mounted on your screen, to track your gestures. The infrared tracked the depth of your movement (distance from screen), by calculating how much time it took for the signal pulses to bounce off the body part and return to the camera. Also, it turns out, there are only so many ways a body, head and shoulders can rotate (joints constrain your movements), and so Softkinetic’s algorithmic software and visual sensors provided all the data necessary to supplement the depth input. We’re not sure why Softkinetic has apparently languished and not hit the market yet.

Another company Xtr3D, tracks your movements from a camera, but without infrared. The technology, while apparently less expensive, doesn’t seem fast enough to respond to gestures, at least when we saw it, and so isn’t ready for prime-time either.

Next, there’s the quiet Oblong Industries, a secretive Los Angeles-based “gesture recognition” startup that has just raised around $8.8 million in a first round of funding led by Foundry Group. Brad Feld and Ryan McIntyre of Foundry have taken board seats on the company, but we don’t know whether this is a gesture technology for games or something more focused, like an iPhone or some other PDA device [Update: AlarmClock has some basic information on the company]

Next, there’s ZCam, a product from 3DV, an Israeli company that offers a technology similar to Softkinetic’s. But the company says it won’t hit the market for the first time until the second half of next year. Like Softkinetic, 3DV is showing off demos that impress, but which are still controlled by the company. Dean Takahashi writes up a review of the product here.

3DV says it will ship its camera and a game that works with it for less than $100, and will put you in the game so you can fly an airplane with your hand gestures, and then use finger gestures to go higher, bank, or begin shooting (lift your thumb and press down on an imaginary joystick). Very much like Softkinetic, the camera measures the distance of the object (your hand and body) from the camera on a real-time basis, calculating it for each pixel in the frame. It uses infrared and image sensors too, which depend on regular light. That lighting is what has limited the effectiveness of the technology so far, because varied lighting can create distortion.

Unlike Softkinetic, 3DV apparently has some high-profile venture capital backing. It is 62 percent owned by Israeli company Elron — at least it was a year ago. That’s when the company announced it had signed an agreement with Silicon Valley venture firm Kleiner Perkins and Pitango for a $15 million investment. But at the time, Elron said the investment was contingent on certain conditions being met. The firms are noted as investors on the company’s web site, so we assume the investment was made (Update: Kleiner Perkins has since confirmed an investment, but not an amount).

Finally, there’s GestureTek, of Sunnyvale, which received a strategic investment Spanish telecom giant Telefonica, and earlier funding from NTT DoCoMo, which we reported just a couple of days ago. See demo video below. There’s also a demo of its mobile application here.