Microsoft’s latest experimental Android app lets you easily copy and share text onscreen

Microsoft’s Garage division, which regularly pumps out experimental apps, today released a new Android app that could be useful to many people. It examines the raw text that’s displayed on the screen and makes it super simple to copy whichever regions of text you’re interested in and then do things with it — like share it in another app, email it, or make a Wunderlist task out of it.

This app, which goes by the name Clip Layer, might seem like a low-level utility — it’s just a clipboard, you can say. But it’s more sophisticated. It relies on Android’s Assist application programming interface (API) to detect visible text and then take action on it.

[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":2118495,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"bots,business,","session":"B"}']

Google’s own Google App for Android does this, in order to deliver Google Now On Tap, a feature that takes a look at what’s onscreen and then shows you cards about entities that Google recognizes, along with deep links to related content in apps.

Microsoft has even tapped the Assist API for its Cortana virtual assistant app on Android. But with that app, when you use the gesture to trigger the assistant — holding down on the home button — the Cortana homescreen pops up and gives you a shortcut to making a voice query.

With Clip Layer the implementation is different, and so are the results. After holding down the home button you’ll see very thin squares appear around text onscreen, and when you tap inside those squares, the contents will darken. You can check to see what the app has picked up by hitting the T in a green circle in the top right corner of the display. If you want, you can tap on other regions. Once you’ve got what you want, you can hit “copy,” “task,” “email,” and “share” buttons at the bottom of the screen to take action on the text that the app has grabbed.

The app doesn’t work perfectly in the sense that it can simply not detect some text that’s onscreen.

But still, this app could have value because sometimes text is simply difficult to copy and then paste into another app, and with Clip Layer that type of thing is easier.

The person behind the app is Steve Won, who is a senior designer on the Microsoft Office team. Before Clip Layer, he worked on another Microsoft Garage app, the Hub Keyboard third-party virtual keyboard app, which has a button that lets you quickly copy text to the clipboard, among other things.

“There’s been a lot of growth with smartphones and ecosystems, as there are more apps that come into stores, but the same problem keeps persisting: less apps able to talk with each other, as they’re designed to work independent of each other,” Won is quoted as saying in a blog post today from Athima Chansanchai, a writer at the Microsoft News Center. “But while the keyboard is focused on input and only shows up when the app allows input, Clip Layer has more coverage and works on any screen. Hub Keyboard was focused [on] composing, and this one is focused on sharing and re-using what already exists.”