Watch Portal come to life thanks to Microsoft’s Kinect

Valve’s Portal video game has spawned thousands of videos that appear to bring the first-person shooter’s holographic windows to real life. This is different.

Creative software designer Roland Smeenk built a system that leverages Microsoft’s Kinect V2 sensor to re-create Portal’s portals in real time. He uses one Kinect to record a 3D view of the world in real-time and another to track the head of the user to see where they are looking.

This isn’t just a bunch of cool video editing. Smeenk is reconstructing the camera view as if the user is looking through a virtual window. Taking it a step further, he uses both Kinects for 3D world reconstruction and head-tracking to re-create a two-way portal effect with his laptop and a connected projector:

The projector shows the larger portal, and the laptop shows the smaller one. To ensure proper head-tracking, the setup opens a portal as soon as Kinect detects someone.

Smeenk notes this has a size limitation to consider: smaller portals hide more of the other side of the scene and provide for larger head movements because the field of view is limited. There is also a distance limitation (about 7 meters when using USB 3.0 extension cables) because latency is being minimized by only transferring the recorded bodyframes through a network connection (as opposed to transferring the whole recorded 3D scene).

This is quite the technical achievement, given that Kinect wasn’t built for this type of thing. That said, Microsoft’s HoloLens was.

Smeenk envisions how technologies like Kinect and HoloLens could transform human communications:

Looking through a window is a familiar experience for almost everyone. Imagine being in a Skype conversation and having the capability to move your head to see who your caller is looking at or talking to (when it’s not you). A holographic window has the power to give people the feeling of being in the same space and allows for interesting new interactions.

In other words, one day you could put on Microsoft’s HoloLens and see holographic windows into not just virtual worlds but other parts of the real world as events happen there — and in real time.