Pinterest now lets you find visually similar things inside of pins

Pinterest is enhancing its search engine to better help users discover new and interesting content.

The company announced that, starting on Monday, it’ll launch a new visual search tool which allows you to take any existing pin, zoom into any part of the associated image, and then get results for visually similar pins. This offering will be available to all of the more than 100 million monthly active users globally across iOS, Android, and the Web.

[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":1836124,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"social,","session":"A"}']

“Sometimes you spot something you really love on Pinterest, but you don’t know how to find it in real life, or what it’s even called,” Kevin Jing, an engineering manager at Pinterest, wrote in a blog post. “There’s that perfect lamp hiding in a Pin of someone’s living room, or maybe a random street style shot with the exact shoes you’re looking for.”

When you spot something in a pin that you’d like to know more about, tap on the search tool that’ll appear in the corner of the image. Then select the specific part of the pin you’re interested in (kind of like cropping the image), and Pinterest’s deep learning algorithm will scan that selection and pull up associated pins that it thinks match. The company says that you’ll be able to further filter the results by topic.

This new visual search tool builds upon services that Pinterest has debuted over the past couple of years. Two years ago, the company debuted Related Pins, which showed you results based on pins you’ve liked or things that you’ve saved on boards.That capability was updated earlier this year when Pinterest enhanced its search with deep learning and announced plans to utilize object recognition (!), through its acquisition of VisualGraph.

Another key component of this visual search tool is Guided Search, which launched in 2014. This is a tool that provides you with pins based on what you’ve typed into your search — even if you didn’t know exactly what you’re looking for. Kevin Ma, software engineer on Pinterest’s Discovery team, stated previously, “Since its launch, Guided Search has become an important driver of Pinterest search traffic and Pinner engagements.”

Pinterest said that it took four engineers a few months to develop the core functionality of this visual search system. The company collaborated with the Berkeley Vision and Learning Center to apply deep learning to analyze images across its “annotated dataset of billions of pins.” As a result of this project, the team built a distributed index and search system with open source tools to enable Pinterest’s back end to scan the massive numbers of images within Pinterest’s database.

The collaboration with the Berkeley Vision and Learning Center isn’t surprising, as the two have a pre-existing partnership. In fact, the center’s director, Professor Trevor Darrell, is an advisor to the company, and Pinterest has hired two of the center’s alumni.

With the initial application of deep learning, you can expect some errors, but Pinterest said that as you post to the site, the technology will adapt, making visual discovery that much better.

[aditude-amp id="medium1" targeting='{"env":"staging","page_type":"article","post_id":1836124,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"social,","session":"A"}']

This visual search capability will likely mesh well with Pinterest for both users and brand advertisers. For the former, it’ll become easier to find things by just pointing and clicking (in a manner of speaking). Brands, on the other hand, may see an opportunity to apply a Promoted Pin, so if you’re looking for a dining room table from IKEA, or perhaps a suit from Tom Hardy, or even that Omega watch from the new James Bond movie Spectre, brands can have their pins show up in your feed as being related. Pinterest hasn’t disclosed how this will work, but it’s something we will likely see in future.

Pinterest certainly isn’t the only one deploying image recognition to improve search — Google, Facebook, and Snapchat (to name a few) have also been utilizing the technology — but the recent development is significant for the company. Billing itself as a visual search and discovery engine, Pinterest wants to become the new Google, giving users the ability to just browse and click to find what they’re interested in. And if a picture is worth a thousand words, these pins will definitely say a lot.