Pinterest launches Lens visual search in beta on Android and iOS in the U.S.

At a media event at its San Francisco headquarters today, Pinterest announced the beta of its Lens feature for snapping a picture with the camera on your mobile device and then find themes and pins that are associated with those themes. The feature is being rolled out over the next few weeks on Android and iOS in the U.S.

Pinterest first introduced Lens last year, but it hasn’t been available to people to everyone until now. And when Pinterest first introduced it, the idea was to identify objects that you capture with your camera, but engineers have modified it since then so that the results include exact or close matches but also, importantly, larger scenes that include a specific object. These search results can provide context about the environment in which objects appear.

Above: An example of Pinterest’s Lens feature.

Image Credit: Pinterest

Just as Pinterest has previously used “pinning” to describe its functionality, now Pinterest cofounder and chief product officer Evan Sharp is talking about using “lens” as a verb. For example, he said, if he sees something he really likes, such as a pair of red shoes, “now I can Lens them and find similar styles,” he said.

This type of visual search goes beyond what Google offers inside the Now on Tap feature for Android, and what Samsung will include with the Galaxy S8 flagship smartphone, because Pinterest’s search results are different, and also, it extracts multiple characteristics from each image. By virtue of its content collection, Pinterest’s implementation is somewhere in between the types of things that you would get from e-commerce providers (read: very structure product search results) and social networks like Facebook and Twitter (read: lower-value user-generated content).

Smaller companies, like CamFind, that offer visual search that takes queries from the camera on a mobile device, but they lack the content that these bigger companies already have.

From a user interface perspective, Google Image Search and Microsoft’s Bing Image Search have both implemented the tab-style category search that appeared earlier in Pinterest’s app. The themes that pop up when you run a Lens search are derived from text input from users, as opposed to predicted computer-generated labels.

Pinterest is also introducing an Instant Ideas feature that puts a circle in the bottom right corner of each pin, and when you tap the circle for a pin, the display will be filled up with more pins.

There’s also a new “Shop the Look” feature that builds on the object-detection capabilities that Pinterest released last year. If a person in a pinned photo is wearing multiple items of clothing, you can tap on the dots to uncover pins of some of the items in the pin. Additionally there’s a new How to Wear It feature that, in Sharp’s words, “lets people see other ways to to style the product.” Shop the Look is going live on Android, iOS, and the web. The thing about Shop the Look is that it when you tap on an item in a pin, you’ll be shown buyable pins, as well as rich pins. That could help Pinterest bring in more revenue over time.

Visual discovery isn’t a new theme at Pinterest; in fact, it might well be the biggest high-level thing the startup offers. Pinterest is just coming up with more and more ways to take advantage of the visual information that it currently has. And there is a lot — more than 100 billion pins now.

A blog post from Sharp has more detail.