Bing image search on the mobile web takes a page from Pinterest

When you’re on a mobile device, the image search feature in Microsoft’s Bing search engine on the web now lets you highlight a specific portion of an image and find images that are similar to that particular part. To access the feature, you simply tap the magnifying glass in the top left corner of the image. This functionality came first to Bing’s native iOS app earlier this year and arrived on Bing’s mobile site in May, a Microsoft spokesperson told VentureBeat in an email.

But before Bing got this nifty feature, it was available on Pinterest, that app for pinning photos and other content to boards that you can share. In September 2014, Pinterest started experimenting with “visually similar results,” before rolling the feature out to all users on web and mobile in November.

[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":1999528,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"big-data,bots,business,dev,mobile,","session":"D"}']

It wouldn’t be fair to say that Bing is trying to catch up to Pinterest in the domain of visual search, though, because Bing actually launched visual search using the camera on mobile devices before Pinterest introduced that feature.

The general point is that technology companies are continuing to develop more efficient ways for people to find exactly what they’re looking for.

There are no armies of part-time employees sitting around and picking out recommendations for images that might be relevant to end users — it’s all happening automatically, using a type of artificial intelligence called deep learning, which involves training artificial neural networks on lots of data, such as images, and having them make inferences about new data. Microsoft was exploring deep learning in the context of Bing image search as far back as 2013. Two months after Microsoft published a blog post documenting that work, Pinterest announced its acquisition of VisualGraph, whose team later built the visually similar search into Pinterest.

Google, which has done extensive research in image recognition and deep learning, has not yet deployed visually similar search on mobile devices, although it has taken steps in that direction — last month it introduced object recognition in the Google Now on Tap feature of Android, and earlier this week that feature became capable of reading QR codes and barcodes. (Bing’s native Android and iOS apps can read those, too.) Reverse image search has long been available from Google — and Bing — but it doesn’t let you select a specific part of a given image and ask Google to find images that are similar to that part.

And image search from both Google and Bing have previously taken inspiration from Pinterest’s carousel of keyword suggestions to refine searches.

Microsoft did not immediately respond to a request for comment. Pinterest had no comment.