Above: Jade Raymond (right) answers questions at MIGS 15 session moderated by Jason Della Rocca. Her colleague Amy Hennig is in the center.
This section includes questions from the audience.
GamesBeat: Jade, how do you deal with questions of cost versus creativity? Sometimes people ask for more time, but that’s going to be very expensive. As a manager, how do you deal with that dilemma?
Raymond: Cost is a very real thing that needs to be managed, whether you’re an indie or a big company. We love making games, but ultimately we do need to pay bills.
These days making games is pretty expensive. You have to start with the right product. You can’t start with a budget and then cut your game to fit that. You have to start by thinking about what will be an amazing product for players. That’s been something I’ve been happy to see EA focus on with “player first.” If you focus on that you’ll see a good result.
When you’re looking around at teams that have done well and successful games that have come out of small studios, that’s a common thread. Watch your costs and be conscious of them, but as long as you’re making decisions based on what’s best for the player, then you should find success.
Della Rocca: Part of it is having a clear vision of what you’re making. What ends up happening — we see this on the indie side — is that if you don’t have a clear sense of what your game is about or a vision of what you’re trying to deliver, an extra cost is just an extra cost. As opposed to, “This will cost extra money but it serves the vision. It’s justifiable.”
Hennig: Part of it is the crucible of having a team you have to answer to, or management you have to answer to. As frustrating as it can be at moments, it’s the grit that’s going to make the pearl. You have to put limits on yourself and challenge yourself and assure yourself that it’s okay to debate things and challenge yourself.
Raymond: Going back to creativity, where are you putting your emphasis? To make a 90+ game, to make the best game possible for a player, it doesn’t necessarily mean everything has to be an A+. Sometimes the best experience could result from pushing to innovate in certain areas, but not all the areas. It’s making choices and recognizing that our goal isn’t maxing all the sliders to the top.
Question: You mentioned how crucial playtesting can be, and you emphasized in-house testing. What about online testing, in a situation where you can’t gather testers in person? How would you draw metrics from them and engage them?
Above: Scene from Star Wars: The Force Awakens.
Hennig: Betas are great because you get a big sample set. The data coming out of that can be very important to see what mechanics people are using, what they aren’t getting, where they get stuck. Unfortunately you do lose out on that observation aspect, which is huge, the anecdotal evidence.
Interviews are semi-useful anyway, because you have to interpret what people mean and not take it literally sometimes. They may not be articulating what’s really bothering them.
Raymond: Different types of testing are good for different phases of development. Also, it depends on the product. With smaller indie companies and early access on Steam, there’s different approaches for bringing bigger testing to a smaller product, as opposed to the type of game we’re working on, which has a different focus. You have to pick the right format for your playtesting given the game you’re working on and the phase it’s in.
Della Rocca: There’s no excuse not to fully instrument your game, regardless of the platform or business model. The teams we’ve worked with, everyone’s put in analytics for tracking player behavior. It’s not just for mobile free-to-play stuff. Every game we do, we’re able to instrument and have data to analyze player behavior.
Question: You talked about reinventing the wheel when you make a new game. When you’re building a sequel, though, what’s the process for re-creating the formula that made the first game a success? When might you not use that formula again?
Hennig: You want to see if you can do it better, right? When we made Uncharted 2, we actually threw the whole engine away, pretty much, and rewrote it. Which, in hindsight, was pretty crazy. It obviously improved on the formula quite a bit. For developers there’s usually a ton of stuff you wanted to do the first time and couldn’t get to, so you’re kind of champing at the bit to get to the sequel and try all that.
Some of those changes can be misguided. You think it works, and with some testing it still works, but when it gets to a wide audience it doesn’t work as well. Sometimes you wind back. But I always describe what we do as moving the ball down the field. We’re experimenting with every game we make. It’s not like we’ve got it nailed and done and perfect. It’s an evolution. We see how people respond, as an industry, and then we adjust accordingly.
Question: Have you ever run into the issue where every playtester playing your game has a different problem, and you’re trying to fix every single thing and getting bogged down in details?
Raymond: You have to avoid that. There is a tendency — often you have to remind people that these are specific players. Sometimes it’s a bit random who you wind up using, compared to the larger population. One person’s opinion, especially if it’s about a very particular thing, might not reflect what you need to fix for players in general to be happy. We hired a playtest analyst at Ubisoft Toronto who was great. He helped put in place a methodology to make sure that the teams weren’t paying too much attention to outliers.
Hennig: There are themes that come up. And then you do have to interpret feedback and not just take it completely literally. It’s almost like reading forums. If you do trudge through that, you’re probably going to be able to see certain themes coming back again and again, versus one or two people grousing about a particular thing. You have to let that go. But everything is saying something similar, you have to find out what’s bothering.