This new mobile app uses big data to analyze your face

Do you know what your true skin tone and undertone are? What about how your skin has changed over time? Beautiful Me is a new mobile app that applies big data and a deep neural network to make your skin profile only several taps away. This app was created by ModiFace, the company behind the Sephora 3D Augmented Reality mirror.

Beautiful Me can auto-download up to 500 of your Facebook photos, and it chooses 100 high-quality photos out of them for analyzing. Based on these 100 photos, the app picks up skin pixels from eyes and lips.

Above: The Beautiful Me app.

Image Credit: Beautiful Me

Its four-layered neural network starts here. The first layer determines which skin pixels provide valid information and tosses the other pixels away. The neural network is then trained with the valid information to detect the correct skin tone, exact shade, and other skin parameters.

“We’ve trained the neutral network on photos, and it’s a training that keeps going,” Parham Aarabi, ModiFace’s founder and chief executive, said in an interview with VentureBeat. “Every time people use the application, it keeps training the neural network. And the idea is [that] over time it could become better and better in presenting the correct skin tone.”

Aarabi is also an electrical and computer engineering professor at the University of Toronto. He’s currently on leave.

Neural networks weren’t the first technology ModiFace came up with for this app. The company first tried simple signal processing, which wasn’t very good. Then it tried a basic machine-learning algorithm, which worked better.

“But I heard quite a lot from the research community about deep neural networks and how they could come up with a much better estimate than other machine-learning algorithms,” Aarabi said. “So we tried deep neural networks as a replacement, and we found that solution is quite interesting, that [it] is one of the best results we have retained.”

The use of Facebook photos is also strategic. People tag their photos all the time. These tags help the app identify who to analyze out of all the people in the photos. Facebook photos also provide date information, which enables the app to come up with a trending graph to show users how their skin has changed over time.

So how will the company make money off this app? Product recommendation is the answer. Based on the information collected, the app will “pull together a combination of curated skin, anti-aging, and cosmetic products that best match the user’s profile.”

However, one problem is that people’s skin profile doesn’t change much over a short period of time. The app’s open rate might not be very high.

Deep learning has become really hot in the artificial intelligence field recently. Baidu just hired Google’s Andrew Ng to work on its own brain. Netflix is running graphic processing units (GPUs) in the Amazon Web Services public cloud for its deep learning research.

Small startups are also popping up in this field. Nervana specializes in building hardware for deep learning. Skymind makes money by providing commercial support for open-source deep-learning software. Ersatz Labs provides a cloud service and a hardware-software package for deep learning.

Now, ModiFace tells you deep learning could be useful for vanity purposes, too.