We don’t know exactly what gaming is going to look like when it comes to Microsoft’s take on augmented reality, but it sure looks like the company could bring us the next generation of full-motion video (FMV) games with HoloLens.
The company has released a new video that shows how it captures and translates real people and objects into 3D characters that work in the HoloLens head-mounted display. To create these “holograms,” Microsoft points more than 100 cameras at actors and then runs that video data through a number of algorithms to build the three-dimensional models. It’s a complicated process, as you’ll see, but the end result creates actors that appear to exist in the world around you. And once you see it in action, it’s not a huge leap to imagine game developers using similar tech to build games around recording video like this — just like studios used to back in the early days of CD gaming with FMV.
[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":1775363,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"games,","session":"C"}']FMV games were slightly interactive movies that featured real people acting as characters and talking to you through your television. Yes, it is as dumb as it sounds, but that also kinda made it incredible. You should check out clips from Command & Conquer: Red Alert 3 to see Oscar-award winning actor J.K. Simmons give you a few pointers about your real-time strategy game.
Filming actors for games is something that has fallen way out of favor, but maybe it could make a come back if they were also holograms.
Check out Microsoft’s HoloLens video-recording tech below to see what we mean:
The methodology for creating this content is way more complicated than pointing a camera at something. After capturing the original footage, it transforms the data into a point-map data that provides a rough outline of a 3D body. In the video, Microsoft points out that this provides more than 2.7 million points of data per frame.
Microsoft’s software then turns the point map into a “water-tight mesh that gives the 3D characters a bubbly skin that eliminates holes.
Next, Microsoft determines important features — like faces and hands — to preserve details. This enables the company to build a final wireframe mesh that they can build a texture onto.
This is an intricate technique, but it’s also one that produces a video file that you can easily stream over the Internet or a mobile network. Microsoft notes that the clips in its video are streaming at about 12 megabits per second. This means that you could build content like this for HoloLens that people will watch on demand through sites similar to YouTube.