GM’s advanced tech chief talks gamification and self-driving cars

General Motors is the second-largest car maker in the world, and it’s been in business for 107 years. The company has relied on new technology for much of that time, but it hasn’t always looked like a technological leader as it’s been slow to adapt to such advances as hybrid electric vehicles and electric cars. It’s the job of Frankie James, managing director of GM’s advanced technology in Silicon Valley, to change that impression.

The emergence of new technologies, from fancy dashboards to self-driving vehicles, has underscored the need for GM to tap the technology world for transportation solutions. As part of its effort to become a more modern company, the company invested $500 million in ride-sharing company Lyft.

James appeared at the recent DICE Summit game conference as part of a panel that focused on the intersection of games, technology, and cars. We talked further while driving around Las Vegas in a GM SUV — the 2016 GMC Yukon Denali — which had some cool tech features, like a pad for charging your smartphone. Our conversation covered what General Motors can learn from gamification and game design, as well as the latest in self-driving cars, virtual reality racing, navigation, and the future of traffic.

Here’s an edited transcript of our interview.

VB: Were there any points you wanted to drive home on the panel? Anything you felt strongest about?

James: In terms of the gamification aspect, I got across the point I wanted to. We dabbled in it, and we’ve looked at some of it based on what we’ve learned from EV drivers, but there’s so much more we could potentially be doing. There are so many more things that we could be helping drivers learn. We need to figure out the right ways to get people excited about doing the right things.

I have a human computer interface (HCI) background. It’s not about giving people a whole lot of training manuals and saying, “This is how you have to do it.” You have to steer behavior toward what you want.

VB: Do the cars have more capability than their drivers make use of? Is that one of the points you’re exploring with gamification?

James: There are definitely a lot more systems in people’s cars than they use. People probably learn about four or five functions that they use all the time and then there’s a whole lot of technology they don’t use. But I’m referring more to the fact that a lot of people, when they’re driving today, they’re distracted. They’re playing with their phones or doing a bunch of other things that aren’t driving, that aren’t the best driving behaviors. It would be great if there were ways to help people be better drivers.

You don’t want to do that by being naggy or preachy. You want to do it in a way that’s going to make people excited about paying more attention or whatever things you want to teach them to do.

VB: You do think there are things you can learn from game design?

James: Absolutely. You’re talking about people who know how to get users to do certain things. Gamers are good at that. They understand that you provide a certain reward structure. You make the experience really immersive. How do you get people into that flow? Games are excellent at getting people into a flow.

VB: I covered a lot of gamification on the enterprise side, as people started talking about it a while ago. It hasn’t quite lived up to the very large market expectations. I’m not sure where it stands right now — how much it works or where it’s really succeeded, and whether or not people are taking that learning and applying it elsewhere. It’s hard to tell.

James: Potentially a lot of it has to do with what people are trying to do in gamification. A lot of what’s happened has been the leaderboard style of thing. Richard made an excellent point, that leaderboards only matter if you care about the people who are on that leaderboard with you. Unless you’re in the top 10, you want to know how you’re doing compared to your people – your friends and family and so on.

On the other hand, how much fun are leaderboards? They’re fun for a little while, but it’s not sustainable fun. People don’t play World of Warcraft to be top of the leaderboard forever. They do it for other reasons.

VB: Are they more extrinsic than intrinsic, the ways of motivating people? From what I hear, intrinsic motivators should be stronger.

James: You want people to be excited about a particular activity and learn things along the way, as opposed to getting preachy about it, like I said.

VB: A reward program in your car: “You used your signal every time you were supposed to.”

James: It seems just a little preachy. Do you get a sticker for it? [laughs] Or we send a report card to your parents. We’ve had teen driver programs before, but this is the first one that gives a readout where, ideally, the parent or caregiver or whomever can talk to the teen about how they’re driving. I guess the reward there is getting to have the keys next time.

You get another human in the loop, another person who cares about what’s happening and is going to act on it. Richard also mentioned the insurance companies. If you’re getting a drop in your insurance rate because you do certain behaviors, then heck yeah, you’re gonna do that. Well, some of us won’t, because the type of driving the insurance companies look for isn’t our style.

VB: The other part of games — you don’t want the driver to play too many games. But the back seat, it seems like there’s potential for entertaining people.

James: That’s another thing we talk about. What different options can we provide to the rear seat passengers? How can we leverage the fact that you’re in the car? Maybe you don’t just want to be playing the games you play in your living room. Is there something we can do that’s connected to the driving experience? There are ways you could make use of the vehicle data on a particular trip, for example, to make a game for the folks in the back seat, who are kids a lot of the time.

VB: Has anyone come up with anything specific yet?

James: We’ve looked at some interesting window technology. I don’t think we’re doing anything with it yet — it’s more of a science project — but drawing on your windows, things like that. It came out of our research office in Israel. They’re looking at having information up on the window, a sort of augmented reality thing where you can look at different things along the way.

You might have it circle a particular hotel or a particular feature of the landscape and give you some information about what you see. The example they had in their video, the kids in the back said, “Hey, can we stop? We want to see this cool thing it’s pointing out.” There are plenty of things we can do in the back seat.

We’re trying to make sure we provide drivers with functionality they want without distracting them from the driving task. We have one stream of work around how you develop the HMI appropriately and give them information at the right time while they’re driving. On the other hand, everybody’s working on autonomous driving. When you get fully autonomous, all this work on the distraction part suddenly becomes a lot less important. There’s nobody to distract because nobody’s driving. You open up tons of possibilities. Along that route, though, as we add more and more automation, we have to worry about keeping the driver engaged enough even when they’re not “driving,” so they can take over if they need to.

VB: That seems like a hard handoff.

James: Exactly. My fear is that if you have that handoff, a lot of these situations where you’re asking a driver to take over are in situations where they haven’t been driving, first of all, and it’s going to be a difficult driving task. If you haven’t been driving, and you may not have been driving in the last week or so because your car’s been able to handle it and all of a sudden you’ve gotta take over in an emergency, forget just trying to get my attention and get me to start driving. The lack of practice is going to factor in as well. The path to full autonomy is not a step function. It’s definitely a path. It’s going to be interesting to walk down, for sure.

VB: I listened to Nvidia’s call today. They said the supercomputer in a car got a lot of attention at CES. They talked about something like 70 different self-driving car projects. That’s a lot of things in the works.

James: It’s interesting, too, that the guys from Nvidia were talking a lot about simulation. That’s very important. He’s absolutely right. There are a million corner cases that come up when you’re doing autonomous driving and you have to make a decision. The more your system has had a chance to practice those, the better it’s going to be at making a right decision. The best way to practice it is through simulation.

We’re at rush hour, so I wasn’t sure what we were going to get, but this is good for showing off the adaptive cruise control. It really does show the kind of stop-and-go traffic it can deal with. You don’t want to have to deal with that yourself. You’re maybe not paying attention 100 percent of the time because you’re just trying to get home, trying to not be grumpy about the traffic and let the car take over some of it.

VB: Is this new for the model year?

James: It started in 15 and a half. Part of the reason for showing this is because next year, we’ll have what we call Supercruise as an option. It takes this, and it builds on hands-free driving. It’ll steer on the highway. It’s not for use in all driving conditions — you need to be paying attention — but it was designed for hands-free and feet-free driving. There’s a camera that will look at you and check how often you’re paying attention. If you’re not, it’ll let you know you need to retake control. But if you’re looking up every so often, that’s going to keep it satisfied. It has a full map underneath it. It knows where it’s going. It has five-foot GPS to know where the vehicle is. It’s been in development for a while, but that’ll be the first year you can buy it.

VB: Is there any kind of schedule for self-driving, whenever that might happen?

James: We’re moving up the road to full autonomy — 2017 CT6 is going to have the Supercruise. One of the things we’re doing is campus-based testing back in Warren, Michigan with autonomous vehicles. There will be a small fleet of Volts, and you’ll be able to call one up on your phone. It’ll pick you up at whatever building and drive you to the next building you need to get to. That’s going to work autonomously. It doesn’t have a driver. A coordinator will be behind the wheel in that level of testing. Then we have other tests planned that are a bit more ambitious in scope afterward. But we’re only talking about the campus testing right now. We’ll learn a lot through that.

That program starts a bit ahead of Supercruise. Then we’ll have Supercruise. Finally, something new will be out in 2017, which is more of a safety feature. That’s vehicle-to-vehicle communication, dedicated short-range communication (DSRC). The 2017 CTS will have that. That’ll be a big part of smart cities. The five finalists for the smart city project were unveiled in March. In all the cities that GM backed for that grant, we asked for DSRC. This will only work if you get it in a lot of vehicles. If it’s in a broad range of vehicles, you can cut down up to 80 percent of non-impaired traffic accidents.

When everyone has VTV, if someone starts to cut you off like that, the other cars around you will move out of the way to give you room. It’s one of these things where, can you get 100 percent penetration? That’s hard to do with any automotive technology. But the more you have, the better the other cars can respond to a potential problem. It’s a parallel technology to autonomous, but it’s primarily a safety technology. It’s the third launch we’ll have in the next year around this stuff.

VB: Have you guys looked at VR (Virtual Reality) at all?

James: Absolutely. One of the folks on my team is here because he’s looking at different VR technologies. We’re exploring all the ways in which VR can be relevant for automotive. There are obvious things around, as I mentioned, replacing simulators, but also other applications we can potentially foresee as well.

A lot of people at CES showed things around the buying experience or choosing a car. You look at some of the games that are out right now, like Forza and Gran Turismo. The quality of the rendering on those cars is amazing. You could look at something in VR and get a really good idea of what it’s like inside that car. How much is that going to make you want it? It’ll be huge. We think there’s potentially a lot of applications for VR.

VB: I’ve seen demos of racing in VR. I played Sony’s Drive Club game with the PlayStation VR. When you crash, it’s like any game with a controller, where you have to back up and straighten out. But you can kind of go around in circles when you’re doing that. Going around in circles in VR will definitely make you nauseous.

James: I’ve played a couple of VR racing games as well. There’s one in Vegas they had for a while, the NASCAR experience. Any time you crash, that makes me so sick. It’s terrible.

VB: They just need to reset it, right? Just pick you up and put you back down on the straightaway.

James: Exactly. There are things that can be tweaked. I was reading about a new head-mounted VR display with a stationary bike to go with it. There are all these different racing games that go with that, and apparently you can get nauseous with that as well. Part of it is the speed. We’re very good at fixing the problems at a walking speed, but I’m not sure that the tech has caught up quite yet for experiences beyond walking speed. But it’ll get there, for sure.

We’re watching the systems get better and better at the high end. My simulator sickness doesn’t come up as often in these VR environments. And on the low end, you can get a pretty darn good head-mounted display for about $30. You know the technology’s going to continuously improve. The quality will get better. That we have things so affordable already is pretty amazing.

VB: Where is the brain trust, between Silicon Valley and Detroit, on self-driving cars?

James: There’s quite a lot of work happening in Milford in particular, in southeast Michigan. The Milford proving ground is where we’re doing a lot of tests on self-driving. A lot more is going on in Warren. Our focus for self-driving is in Michigan.

That said, I’m a Silicon Valley person, so I have my own biases. So much of autonomous is really about software. I feel like we need to leverage everything that’s going on in the Valley. There are so many good people, and there are a lot of different startups and other folks who are working on autonomous. We need to pay attention to that. My group is focused on working with startups and working with technologies coming from outside of GM. We’re trying to make those connections and see how we can bring the two together so that everybody can work on the same goals.

I keep looking for that golden age when the people who don’t want to be driving — the people who are driving now, but who are distracted all the time — are using autonomous driving. The people left driving will be the people who want to be driving. That’s going to be a much better road to drive on. There’s nothing like being on a race track, because everyone wants to be driving. It feels great.

You have a lot of people who buy cars based on the infotainment system and how easily they can connect their iPhone. That’s the only thing in the car they care about. For people like that, autonomous is probably going to make them very happy. But for a lot of us, there are times when autonomous will be just fine to serve the purpose of the trip. We can save our mental cycles to devote to more fun topics.

VB: Are there any major events going on around self-driving cars?

James: There are different meetups for autonomous technology. There’s Auto Tech Console. A lot of stuff is going on in Silicon Valley. Part of our value add to GM and the industry is we don’t solely focus on car-related events. We’re trying to find stuff that our competitors haven’t found yet.

We do that by going to more events, going to all the different meetups that are focused on IOT or some other thing where we see a potential connection. But it’s not an obvious connection, not something where someone has already said, “We’re doing this for automotive.” We like to look at companies coming out of other verticals, look at the potential automotive use case and see if they can make it happen.

A good example is this technology right here, the Powermat wireless charging pad. We identified that as a potentially useful technology when Powermat first came on the scene. Could that be something people would like to have in their car, to avoid having wires sticking out of their console? Now, first thing we had to figure out was whether Powermat would follow a major standard. Turns out that they would support both. That’s step one of the process. By that time we had already engaged people in our productivity groups and our HMI teams. We said, “What else do we need to know?”

We put together a prototype, and we discovered that it’s great, except that your phone gets really hot. The phone will shut off. In a consumer experience, or in Starbucks, that doesn’t happen. In a vehicle it happens. So we sent it back to our research team and they figured out a way to provide ventilation on the back. Once that was working, we said, “Great, we have a wireless charging pad for anyone who gets in the car.”

Took it a while, but that’s a technology that came out of our office and ended up in production. That’s what I was talking about, where we go from a parallel or related industry and then look at what needs to happen to transition into automotive.

VB: What’s the next big technological advance that we’ll see go into the mainstream? Are self-parking cars going to be the majority of models soon?

James: That’s a great use case. I like the one where it can actually drive away from where you are and find its own parking spot. A lot of people are looking at, “Can I get out of my car and have the car park itself into this really narrow space?” Those sorts of things are potentially very interesting.

VB: Do you have a different insight into the subject from the tech companies like Google or Apple?

James: We come at it from a different perspective. In some ways that can be a bit of a hindrance, and in some ways it can be a help. We know the problems that we’ve gone through. We know the standards we have for our brands in terms of quality and reliability and durability.

Tech companies, newer companies, they don’t think about the limitations because they haven’t had to face them before. That gives them, in some sense, the freedom to try new things, but at some point they’ll have to think about those limitations. It’s good to have the different perspectives.

VB: I’m finding that sometimes GPS is not quite accurate enough, still. At San Jose airport, the Lyft and Uber drivers still can’t find you in places where they’re allowed to pick up passengers. They go to the exit doors when they’re supposed to go to an island instead, and it’s so close, GPS can’t distinguish.

James: We’ve gotten Supercruise down to five feet. The average lane is 12 feet. It’ll keep you in the right lane.

VB: Is LIDAR sensing going to be a real thing for self-driving cars?

James: Yeah, we’re making use of LIDAR. We have a range of sensors that we fuse together to get the information we need for autonomous driving. LIDAR is something we continue to look at. There are other technologies that we’re looking at as well.

In my mind, I consider two things. The sensors need to improve, and we need to be aware of what sensors are available and what are the best at a given time. When you get to the software side, in some sense you need to black-box that. Whatever sensor is telling me where I am, this is the degree of precision I need. This is the information I need. Then, as the technology advances, you can potentially swap it out. That’s the way it will work, I think.

VB: I talk to the chip guys at times, working with gallium nitride. They’re getting their chips into the LIDAR systems, making it more accurate and cheaper. It seems like some of those could eventually drive demand for gallium nitride instead of silicon chips.

James: Like I say, I think about black-boxing that while the sensors continue to improve. Whatever kind of unobtainium they’re made of, if they give us the accuracy we need for positioning, that’s great. We always have to worry about cost and packaging. That’s the other thing we look at – how far is a technology advanced down the road map in terms of getting to a size we need, a level of reliability and durability we need? And how much does it cost? New technology is always expensive, and autonomous requires a lot of technology at once. It’ll be an expensive feature to implement, the first few times.

The sensor fusion I was talking about, we’re running in the loop now with real-time testing. It’s still bench testing, still simulator-based, but the first loop is ready now. There are two sides, the east side and west side, and the west-side group is basically mapped and set. I looked at it the other day, and it’s pretty cool. It’s crude at this point, but it’ll get better. It’s cool just to see the wheel turning itself.

VB: Now someone just needs to do something about Bay Area traffic.

James: Everyone talks about how, once you get to autonomous vehicles, you can have these pelotons of faster-moving vehicles. But just where we’re going with sharing is going to help quite a bit, as well. I’ve heard people tell me that businesses in San Francisco, instead of giving out company cars, just give you an unlimited Uber pass. It’s easier for them in terms of parking, and it’s also easier for city traffic.

I’d like to see our public transit systems upgraded, as well. I went into the city on Super Bowl weekend, and we took the train out of Sunnyvale. By the time we got to Menlo Park, the train was so full they couldn’t let anybody on. We need more trains, more options, better infrastructure. But we have the rudiments there.

VB: They’re talking about building another trans-bay tunnel to accommodate BART (Bay Area Rapid Transit) traffic growth. I have no idea how much that will cost.

James: What do we have for direct transportation to San Francisco airport? You can take Caltrain or the BART. Why don’t we start connecting some more major hubs and go from there? I’m from Boston originally. I’m used to really good public transportation. In a major city, you should have ways to get around without a car. It was a shock moving to San Francisco.

VB: At CES, I saw one of the smart steering wheels for teens. It registered whether you had your hands on the wheel in the right places. If you didn’t, it wonders whether you’re texting.

James: Like I was saying, our Malibu has a teen driving system in it that analyzes a bunch of different aspects of driving behavior. The parent or guardian or whoever decides whether you get the keys looks at that report and decides whether they need to have a conversation about how you’ve been doing. All that feedback — it gets back to the gamification aspect, as well. If you know that your mom’s looking at how you drive, you may decide to drive differently.

Even little things like making sure you can’t turn up the radio until your seatbelt’s fastened, those are just smart things. And then, once you get in the habit, you’ll never notice it again. If you’re always in the habit of wearing your seatbelt, you won’t notice that the radio doesn’t respond to your input. Obviously, it’ll do what you want, because you did the right thing.

VB: Do you see people using MHL cables very much?

James: I don’t think we’ve seen much from them as far as vehicle use. We talked with MHL right when they came out and decided not to go that route, because we didn’t see a lot of consumer usage. Like I say, that’s one of the things my group does. We hit those trends when we’re early, and we decide if we think it’ll be impactful enough that we need to worry about it. Sometimes it’s not.