Imogen Heap’s sensor-packed gloves will change digital music making forever (interview)

Digital music is made out of square and sine waves. These sounds don’t have bodies. They don’t have a character, but according to musical mastermind Imogen Heap, wearables can bring them to life.

Heap believes she’s found a way to make digital music making a bit more human. And all it takes is a pair of sensor-packed, Wi-Fi-connected gloves.

After years of development with a team of interactive artists, sensor technologists, and musicians, we have an inside look at her plans — a first step in Heap’s mission to move from buttons and faders to natural gestures.

Harrison Weber:   What do gestures and wearables have to do with music?

Imogen Heap:   In my mind, when I think of sounds — or even when I’m talking to you — I’m gesticulating the entire time. Especially with music, I find myself trying to explain things with gestures.

For a lot of people, it’s very natural. If you were to say to someone “show me a high sound,” they’d probably say “oh, it’s somewhere up here.” These innate, intuitive gestures are somehow built within us and can be applied to music and software.

Harrison Weber:   What’s the story behind your “Mi.Mu Glove?” Can you share where the idea came from?

Imogen Heap: I never thought to actually use gloves as a controller until I saw this girl, Elena (Elly) Jessop, at the MIT media lab. She was developing a glove called the VAMP glove. She showed it to me; the minute she sang, she clinched her middle finger to her thumb and took a tiny nano-sample of her voice and synthesized it (turning it into a pure note). Then, she did a simple motion and created a vibrato.

That simple motion of capturing something with your own hand — I always wished I could have something to stretch my voice and send it over here, or over there. It’s totally natural. It’s obvious to use, but it’s not easy to make.

I asked Elly if we could work on this together, but she was tied up. I tried to find something available off the shelf, but all of the software and the hardware out there was really limiting. And so there was only one thing to do: I had to make my own.

We began initial stages using fiber optic gloves, but we were quickly limited; we had no gyroscope, accelerometer, magnetometer, or any positional data — we only had bend sensors. We thought about using conductive thread, but that’s a bit further ahead in the future.

Harrison Weber:   Hardware aside, how do your gloves work?

Imogen Heap:   In a way, our algorithms and software are the real meat of the project. It was bluetooth, and now we’ve gone for Wi-Fi. The gloves come up as an Internet device on your computer. It pops up, they’re really easy to calibrate, and you can make your own gestures. You’ve got up, down, left, right, twist, yaw, pitch, backwards, forwards, up, down, and accelerometer peaks for drums.

Harrison Weber:   I know you want to offer these gloves to the public, but what about the music tools that are already available?

Imogen Heap:   There are loads of incredible stuff for iPads, but again you’re stuck because you have to use your iPad. You’re coming closer to the machine, when the machine should come to you.

Things go in stages, and I don’t think we’re ever going to be stuck behind the screen in reality. Someday we’ll wear our tech like you’d wear makeup, or wear a jacket.

Harrison Weber:   Is this a return to a more human way of making music? Like composing a digital orchestra?

Imogen Heap:   It’s about wanting to bring life to instruments that don’t have a body. They’re for things that don’t have a character. There are lots of sounds that people make out of nothing — they’re sine waves and square waves. For all of those things, gestural interaction can really bring them to life.

Harrison Weber:   What inspired you to explore gestural music making?

Imogen Heap:   It’s always frustrating to see a band you absolutely love, and they’re really getting into it — but they’re head-banging over their laptop. You’re like: yeah, OK, I can see you head-banging, but I don’t know what you’re doing. Maybe it’s not that important for everyone else, but for me, I’d love to know what it is that he’s actually tweaking there and what he’s getting excited about.”

How can they bring themselves out from behind the desk and into the audience’s arms?

Harrison Weber:   What’s your experience with this?

Imogen Heap:   When I do my live shows, I like to share the little bits of technology I have on stage. I like to explain my looper or the synths and stuff I have on stage.

When I would have my little Korg micro controller — it’s a little controller which has eight faders on it, a load of buttons, and a keyboard. I might use that to bring in vocal effects or bring in different instruments. I try my best to make things as seamless as possible instead of having to go into Ableton (music production software) and pressing “solo.”

I tried my best to not let the technology make me stop and start my performance. More and more, I was trying to have lots of different sensors on the stage. That was my way of not staying behind one desk.

But, it was just always frustrating. I would always have something to do something. I would have a controller that I had to press in order to do something, which stopped me from doing something else. It’s not very interesting to just watch someone press a button.

Harrison Weber:   How did you originally become involved in music tech? What was your timeline from using loops to your gloves?

Imogen Heap:   It was a slow process. I didn’t really think about looping until I saw this guy when I was 19, Mitch Gerber; he’s a double bass player in Switzerland. He had these delay units that were made by Echoplex. That was the first time I had a go at a looping machine. I loved it. I could suddenly record something and be more than one person at a time.

I tried to find better ways to bring the fun things I’d do in my studio on stage.

When you’re on a keyboard, you have to use one hand to play something and one hand for manipulation. You’re limited, because one hand is taken up by doing something very boring (turning up the volume or panning something), and we’ve got all these other ways that we can move around.

Harrison Weber:   And your first public prototype is available on Kickstarter?

Imogen Heap:   I don’t think we’ll make our Kickstarter goal. If we make it, brilliant, but if we don’t, we have a plan. If we don’t, we’re going to make 10 or 15 pairs. People will still have to pay through the nose for them because they’ve very expensive. Maybe by December, if we have those core collaborators, we can develop something which is much more affordable for people. For now, we just want to get on with it. We’ve got loads of amazing software and hardware. I’m going to keep developing it with the team.

We didn’t want to find venture capital investors. The company’s not for profit. We reinvest everything into the gloves.

We believe there’s a need for them, and we’re going to do our best to try to make that happen.

You can back Imogen Heap’s Kickstarter project here.