Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders. Subscribe Now
Celebrity-detecting AI? Why not? Giphy, one of the world’s largest online GIF-sharing platforms, today open-sourced a machine learning model — the Giphy Celebrity Detector — that’s capable of recognizing over 2,300 famous faces in GIFs with 98 percent accuracy. Both the model and supporting code are available on GitHub.
The company says the goal was to build a deep learning model — i.e., layers of mathematical functions that mimic biological neurons — capable of annotating its most popular content: celebrity GIFs. Toward that end, Giphy claims the Celebrity Detector achieves performance on par or better than “similar fee-based models,” like Amazon’s Rekognition.
“Between our existing GIF library and the celebrity-based content we generate internally, we knew we’d have abundant training data from which to pull,” Giphy director of R&D Nick Hasty wrote in a blog post. “Having total control of the model lets us update it as needed to maintain Giphy’s unique position at the cutting-edge of popular culture.”
To compile a training dataset, Giphy’s research and development team extracted celebrity names from the top 50,000 searches across all of its platforms, including Giphy’s web platform, mobile apps, and integrations with Facebook, Twitter, and Slack. They supplemented the resulting corpus of 2,300 celebrity names and images with data from the web — mostly for celebs who appeared less frequently in the catalog. And to minimize false positives for less-famous celebrities, they used a separate machine learning model which grouped images by similar facial features.
AI Scaling Hits Its Limits
Power caps, rising token costs, and inference delays are reshaping enterprise AI. Join our exclusive salon to discover how top teams are:
- Turning energy into a strategic advantage
- Architecting efficient inference for real throughput gains
- Unlocking competitive ROI with sustainable AI systems
Secure your spot to stay ahead: https://bit.ly/4mwGngO
“Clean datasets had smooth distributions, whereas noisy datasets were more unevenly distributed and clumpy,” Giphy explained. “[C]lumps in our noisy datasets that contained the most images tended to have similar distributions as our clean datasets, indicating those groupings were positive images of the celeb and could be safely used in training. This process helped us to de-noise and improve the accuracy of our model considerably.”
The Giphy Celebrity Detector consists of two parts: a face detector and face recognizer. The first attempts to pick out faces across all frames of a target GIF using MTCNN, a pretrained AI model, and the second — a deep convolutional neural network trained on the aforementioned celebrity dataset — spits out celebrity predictions together with feature vectors (numeric representation of facial features). A third and final postprocessing algorithm clusters faces by their vector representations, and computes an aggregate prediction for all faces within a given cluster yielding one or more celebrity names.
To validate the Giphy Celebrity Detector’s accuracy, the team used the open source labeled faces in the Labeled Faces in the Wild test and a crowdsourced labeled dataset containing nearly 1,000 popular Giphy celebrities. On the former, it scored 96.8 percent accuracy, and on the latter, it achieved 98 percent precision.
That’s mighty impressive, but it’s worth noting that the Celebrity Detector falls short in other respects. It can’t recognize as many faces as Clarifai’s Celebrity model (over 10,000), and there’s no API to speak of — getting it up and running requires a bit of legwork. But Giphy is committed to refining it, for its part.
“Over the next few months we’ll be providing more details on this project … including a technical deep-dive and an overview of how we tested the model for different types of bias,” wrote Hasty. “Until then, we encourage you to download and play with the model and let us know if you come up with any cool use cases or extend the model’s capabilities for your own needs.”
