On-device AI emerging as the true evolution of artificial intelligence solutions

Presented by Qualcomm Technologies, Inc.

---

Mobile is fast becoming the largest AI platform in the world, as we plunge full-speed-ahead into an era of trillions of connected devices, from automobiles to HD cameras, smartphones, wearables, and other IoT devices. With new capabilities for existing solutions, on-device AI makes them all smarter and faster, as well as making brand-new innovations possible. That could mean smarter assistants, safer cars, enhanced security, leaps in robotics, an evolution in health care solutions, and more.

Machine learning and data processing in the cloud won’t go away, but on-device AI delivers personalized experiences with some tremendous benefits, including hugely improved performance, especially for those AI use cases that can’t afford even a microsecond lag: think auto safety. On-device AI boosts privacy and security, protecting sensitive data like voice ID and face scans that could be compromised in the cloud. And when your AI power is in your hand or at your fingertips, reliability is no longer an issue of network availability or bandwidth.

Better products, new capabilities

“We’re seeing things today that people have always seen in movies and dreamed of doing at home become ordinary everyday use cases for users on their smartphones,” says Jeff Gehlhaar, Vice President, Technology and Head of AI Software Platforms at Qualcomm Technologies, Inc.

That includes always-on capabilities, for instance, smartphone assistant features like voice wake-up, always-on noise suppression, language understanding, disambiguation of circumstance, or ability to hear and understand you at varying distances from your device’s speaker. It also powers on-demand, high-performance smartphone capabilities such as instantaneous language translation and more.

For example, very good cameras have been featured in phones for a long time. But once you apply AI to a camera in a smartphone you can enable very demanding computer vision applications, giving photographers and videographers the ability to shoot soft focus — the bokeh effect — on the fly at 30 frames per second and the ability to track subjects. These were things that used to require post-processing in the cloud, or weren’t possible, because enough data wasn’t collected when you shot the video to do it effectively.

All this can have a direct impact on the industry. The increased value of the experience for a user increases the certainty that they’ll upgrade their device to a flagship model in order to snag those capabilities.

“When you finally see it manifest in a product and you see a user post a video, or you see a picture on Twitter from an ordinary user who got a new phone and it does this remarkable thing, we understand all the work that went into that,” he says. “But to bring that experience to a user in a way that appears seamless and magical is really exciting.”

There’s also a big explosion in the interest and application of “voice UI,” which encapsulates virtually everything from keyword detection and waking a device up or starting to interact with a device, all the way through to the end solution, whether it’s understanding multiple languages or interactive dialogue.

Voice AI devices have moved beyond simple question-and-answer transactions, where you’re required to wake your device over and over again for each query. The AI understands the context of your previous query, and applies it to the context of your current query, making the interaction a continuous-feeling conversation.

“Again, by having this AI on-device, you have this very natural interaction with it,” he says. “You start to forget that it’s a device. It starts to be an agent of sorts on your behalf.”

In automotive, Gehlhaar imagines essentially eliminating vehicle accidents in the next ten to 15 years as a result of enabling these AI systems. In the shorter term, he envisions on-device AI just making the experience of being in a vehicle a lot more natural, with things like voice interaction, better traffic routing and so on — all steps in enabling a better mobility experience.

In other areas, like robotics, on-device AI enables companies to provide solutions to their customers that they have aspired to do, but couldn’t due to either cost or power limitations. The idea that a home vacuum cleaner would have a computer vision system that can map your house and understand the difference between a person and a dog, and at a price point where almost any consumer can buy it (not knowing why it works better, just that it does) — that’s a lot of value for that industry, he explains.

That’s true all across the board, whether it’s automotive or industrial IOT or home security. These companies are able to offer solutions to their customers that weren’t possible at power, performance, and price points even a year ago.

“The people who are moving aggressively are capturing a big part of the market,” says Gehlhaar. “We see players who are very innovative and they’re pushing hard. Those players are going to likely come out ahead.”

The cloud, the 5G era, and the wireless edge

Right now, 4G, WiFi, and Bluetooth each work hand-in-hand with on-device AI, but with 5G on the rise, consumers will see higher data rates, further reduced latency, cost-per-bit reduction, higher system capacity — and massive device connectivity. There will be an even more seamless interaction between cloud and device, and it’s set to enable a wide range of applications across industries.

“This gives you layers of possibilities, and it really puts the right functionality in the right places, in a way where everything’s connected over really fast links,” Gehlhaar explains, “with as much processing as you can bring to bear in the local situation on the device itself.”

For example, with 4K videos, you’ll be able to add effects immediately, and rely on immediate neural processing for specific capabilities, but then move it to social media with the snap of a finger, where it can be indexed, tagged, run through face identification, archiving, and more.

The AI-at-the-edge business advantage

Demand and appetite for these capabilities is tremendous among leading companies across industries, from smartphone manufacturers to robotics and automotive OEMS, all of which are seeing the potential of on-device AI to make their solutions better, Gehlhaar says.

“Part of our job as a provider of efficient AI processing, principally, is to look at how we help our customers take their very powerful neural networks and bring them on device,” he explains. “Sometimes that involves compressing them or optimizing them or quantizing them or a combination of those things. We have research teams working on algorithms that help our customers do that, and we’re rapidly bringing those into our commercial products.”

Qualcomm Technologies creates tools designed to save customers time and effort in optimizing performance of applications with trained neural networks on devices with Qualcomm® Snapdragon™ mobile platforms. The most recent mobile platform, Snapdragon 855, has dedicated Qualcomm® Hexagon™ Tensor Accelerator hardware targeted at increasing the performance and power efficiency of running neural networks on Snapdragon platforms, and the Qualcomm® AI Engine is currently in the fourth generation.

Qualcomm Technologies is also working with industry partners, like Google with TensorFlow, and Facebook with PyTorch, to help shape the industry overall in terms of on-device AI acceleration and power efficiency.

“We want to make sure that we’re not just influencing the industry,” he says, “but also taking the input from our partners in the industry and then shaping our future product portfolio in a way that will continue to advance our on-device AI solutions.”

The company focuses on future-looking research initiatives, which affects its product road maps, because it can take several years in some cases to go from an idea to a product, or a platform that can support a particular idea.

“It puts us in a position where I feel like we have a very firm understanding of the use cases our customers are trying to solve,” he says. “We have a pretty good idea about where we think they will end up needing to be, and we have research people working on that. We are at the core of on-device AI.”

Qualcomm Snapdragon, Qualcomm AI Engine and Qualcomm Hexagon are products of Qualcomm Technologies, Inc. and/or its subsidiaries.

---

Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. Content produced by our editorial team is never influenced by advertisers or sponsors in any way. For more information, contact sales@venturebeat.com.