LiveScribe lets the paper do the talking, and thinking

Updated

LiveScribe, an Oakland, Calif. company, releases tomorrow one of the more wondrous communication features we’ve seen lately.

It has developed technology that lets you take handwritten notes from a lecture or interview with a high-tech pen. The pen will remember everything that was being said when you took the notes.

So if you’re looking for a portion of the conversation you didn’t quite understand or remember, you simply tap your paper notes, and the spoken interview will play back for you.

The technology is all stored in the pen, which has an audio recorder, but which also has sophisticated visual sensors that tracks tiny dots on the paper you’re using so that it remembers which spot in the lecture matches which notes you took.

More profoundly, the company’s software lets you build applications on the paper. So you can email your article directly from your notes, or even post an article to your blog from your notes! Without logging on to the computer.

LiveScribe requires special paper – the tiny dots spaced out at particular unique intervals — but otherwise the paper looks quite normal.

This is remarkable stuff, best understood by these animations: Here (about paper recording), here (about blogging) and here (about how the pen translates other languages for you). It unveils tomorrow at the D5: All Things Digital conference.

I got a personal demo of the technology from Jim Marggraff, chief executive of the company, and I was immediately sold. As someone who takes copious notes and would like to find the vocal version instantly, this is quite the journalist’s nirvana. Moreover, it lets you hook up to a computer, so that you can see the spoken version of your notes unfolding on your computer screen (it uses translation technology, to translate the voice into text). It shows you where you are in the lecture, and the parts that are still to come — in a different shade of color.

This is not snake oil. Marggraff is a leading visionary in the area of new ways of learning. His breakthrough was at the well-known children’s education company Leapfrog, where he helped build its paper-based popular multimedia products — books with paper that spoke — and which became the top-selling toys in the U.S. Leapfrog sold the technology in 60 million books, reaping $1 billion in five years. Marggraff later moved on to a Swedish company called Anoto, and continued his experimentation, but left last year, to build out what is effectively a mobile computer in a pen. His goal: To take the technology to adults.

In his demo for us, Marggraff showed how to create little applications on a page, letting us write on the paper, and then tap on them to do arithmetic. For example, tap on the figures “2”, “+”, and “2” and “=” and then a speaker in the pen answers with “4”.

To be clear, it wasn’t completely bug-free. There were moments where we had to pause or press again in order for the pen to register what we were doing.

We weren’t able to test filing a blog post to our blog from the piece of paper, but Marggraff promises to let us do that soon.

That can work because the paper application sends a signal through the pen to an Internet connection — which allows you to file the post. The pen will be docked with the computer, for the first release. However, in a second release, the pen will carry WiFi, so you won’t need to have your computer with you.

A life-size version of the pen is seen below:

More on the technology: LiveScribe licensed the dot paper technology from Anoto, when that company decided to pass on investing in Marggraff’s ambitious portable computer-pen idea (Anoto has been focused on its own financial turnaround).

The micro-dots in the paper are part of a unique grid system developed by Anoto. The camera in the smartpen sees these dots and tracks the pen’s location to enable digital indexing of content. Dot paper may also be printed on certified home or business printers.

The Anoto technology licensing allowed Marggraff to start building LiveScribe beginning last year. He set out to raise $22 million in venture capital from investors. VantagePoint Venture Partners, a Silicon Valley firm, is a backer, and Marggraff is looking to close the round.

The note-taking feature is called “Paper Replay.” The pen records the conversation and digitizes the handwriting, automatically synching the ink and audio. By later tapping the ink (twice, just like you click twice on a mouse), the smartpen replays the conversation from the exact moment the note was written. Notes and audio can also be uploaded to a PC where they can be replayed, saved and searched. The search tool is pretty impressive — you can search through a lecture for a particular word.