Facebook demos Prisma-like augmented reality app for live video

Facebook is working on new camera technology that overlays famous artists’ styles onto your live videos. This feature was demonstrated on stage at the WSJD Live conference, but it’s not something that will be released in the foreseeable future.

Chris Cox, Facebook’s chief product officer, said the company predicts that in the next five years, 70 percent of internet traffic will be video. This is expected to be true across all of Facebook’s properties, whether for messaging — moving from voice to video calls — on Instagram, or in other applications. He revealed that Facebook has been looking at a “perennial product problem for phones, which is how do you make it easy for someone to capture and create a video that’s nice using a mobile phone? This is something that most of us aren’t super comfortable doing; it’s really hard to create something you’re really proud of.”

One solution that Facebook is looking at involves the use of image-style transfer, which came out of a white paper published in Germany this year. The technology lets you transfer the style of an artist, say, Monet or Rembrandt, onto your image by using what’s called Convolutional Neural Networks and computer vision. Think of it like Prisma.

What Facebook is exploring is not applying filters after the fact, but rather using live filters for in-the-moment experiences. Cox explained that real-time implementation was the hardest challenge the company faced, as the program functions at 24 frames per second. “It’s taking something that’s a known technology, but it was getting it to be fast on a phone and to be able to be done at a low-enough latency, without dropping frames, stuttering, or…blurring.”

The feature is just a prototype right now. However, the company is shifting investment toward the camera because it’s a creative tool you’ll be able to use for Instagram, Messenger, WhatsApp, and many other applications that Facebook could roll out in the future.

“Whether an intimate moment or a broadcast, this is the thing that will take the technology to the next level,” Cox claimed.