Twitch already broadcasts and records millions of hours of gameplay video every month. Now that it’s built into Xbox One and opened its software-development kit to any developer, the amount of video it records is only going to grow exponentially.
That much video could cause some problems.
Video is difficult to sift through because visual and audio data are pretty incompatible with the way that humans search for content. A gamer looking for a clip of someone using a specific weapon on a specific level in Call of Duty: Black Ops II is gonna have trouble finding something that granular. Add a couple million extra hours of footage a day, and that task becomes virtually impossible.
The problem with video
“One of the big problems with broadcasting is that you sit down and stream for an hour and a half, and to Twitch, that is just a big video file,” Twitch director of business development Brooke Van Dusen told GamesBeat. “And we know nothing about that video file except maybe what game it is. Other than that, we really have no information about it.”
After a video like that airs live, no one really wants to watch it. Van Dusen likens it to the Academy Awards. No one wants to sit down and watch a full rerun of that broadcast. At the most, viewers wants very specific highlights.
Twitch broadcasts share that same issue. Each stream has highlights buried under hours of downtime that isn’t interesting after the fact. Those highlights are nearly impossible to discover without a lot of effort from humans time-stamping and editing together SportsCenter-like highlight reels.
Above: Twitch interface in Company of Heroes 2.
That’s where Twitch’s new technology is going to change things. The company plans to make video more useful by enabling the collection of metadata, which is background information that details what is going on in a video. Think of it like the data the identifies a song’s name and artist on your iPod. But in this case, it’s way more powerful.
“We’re working with game publishers to not just send us the video and audio from a title but also the game data,” said Van Dusen. “Information like the numbers of kills and deaths, which map the player is on, what gun they are using, and what level they are on.”
This is all automatically powered through Twitch’s SDK if a developer chooses to use it. Games like Call of Duty: Black Ops II, Planetside 2, and Company of Heroes 2 already use Twitch’s code to power native livestreaming, and they can use this new metadata functionality as well.
“What that means for players is that after they are done broadcasting, instead of having this 90-minute chunk of video, they will automatically get a notification after a match that suggests some moments that we think are interesting,” said Van Dusen. “Players can then share them with their friends. We do this by using that game data to identify points in the game that we think would be interesting.”
Changing how video discovery works for everyone
Of course Twitch isn’t going to keep this tool to itself. It’s going to give players full control over what kind of highlights they see after the game. A Call of Duty player could toggle the options so that he only sees moments where he dies, allowing him to quickly try to see why it happened. A League of Legends player could flip through the data to quickly find all of their quadruple kills.
“Players can find these places in the video that enable them to find something that they want to see rather than having to search manually through an hour-and-a-half block,” said the director.
Twitch also expects its highlight suggestions to get smarter over time. The company can see how many times players actually share one of the suggested highlights. If only a handful of players actually take the time to send a clip of a double kill to their social networks, then Twitch can stop suggesting that to most people.
Developers can also use this video data to leverage their communities to help promote the game.
“Studios can see what videos are popular, and this is a very powerful tool on their end,” said Van Dusen. “For example, developers could promote a weekly ‘best of’ video where they search for a team that was 20 kills behind and came back to win.”
Another example is for a free-to-play game where a team is using a new premium weapon to beat up on another team. Developers can instantly find videos where a team is using a certain item and wins by a certain amount. They can then promote that video to show how a real team is using that purchasable content to dominate the competition.
Before Twitch devised this technology, it was incredibly difficult for anyone to find a video with that level of precision. Millions of videos might feature a certain weapon, but that information only shows up in search results if someone manually tags it.
“Now, we have all this great information,” said Van Dusen. “Once this is widely implemented, all video on Twitch becomes perfectly cataloged.”
It’s probably important to note, in light of something like the government’s spying program PRISM, that “metadata” isn’t as scary as the word makes it sound. It is simply data about data. None of the information Twitch is collecting here is associated with an individual. It is tied to the videos and just helps Twitch figure out what is happening within the footage.
How the Twitch community will make it even better
Twitch expects this open data to really take off when its dedicated community starts playing around with it. While the company will work on offering highlights to games that use its SDK, it will enable its broadcasters and viewers to take this functionality to the next level.
“There’s so much potential here to do cool stuff that Twitch couldn’t even hope to do it all by itself,” said Van Dusen. “That’s why we want to make this available to our community.”
Van Dusen used an example where fans of a racing game could find all videos of players racing on a specific track. Gamers can organize those videos by best time, which would effectively create a video leaderboard that anyone from the community could easily manage.
The search tools can get even more granular than that. Twitch can find all the videos of players on that track using a specific car or one kind of upgrade. It’s only limited by the data that the developers decide to surface.
“It’s much more work than Twitch could do alone,” said Van Dusen. “But by making this available to the community, there’s really no limit to what we could see thanks to this project.”
Now, it’s just up to developers to feed Twitch that data. The company is working with studios to make that happen. Once it’s getting enough information, that’s when it plans to go public with these tools.