What Happens When You Combine Kinect And Oculus Rift?

Matt Davidson, the R&D Engineer behind the latest #1 game on the Aussie app store, League of War, recently had the question put to him of which technology excites him most. His answer? The Oculus Rift combined with Kinect.

I’ve been chatting to Davidson about League of War, and will post more of that later, but this answer in an early interview intrigued me. Davidson’s been involved in many technologies over the years, mostly relating to visual effects, including making advances in rigid body simulation and computational fluid dynamics.

His technical knowledge is intimidatingly deep -- while a student at AIE, he chose to make his own engine for a game project (the game was the project, not the engine) - and ended up rewriting it the night before it was due. Talking to him is a bit like reading John Carmack’s tweets. One can distill truths from the torrent of terminology, but the thought train slows down for no one.

So I asked him what excites him most, keeping in mind I’ll probably understand 10% of what he’s saying. His answer:

Quite a lot of people are starting to develop tools and experiences using Oculus Rift. I am intrigued by that platform, and have done some work with it. At the moment, a lot of my personal blue sky research is in augmented reality, using dense track, so every point tracking with point clouds, from Kinect-like devices. Taking those two pieces of tech, an infrared camera, or a type of light camera, and integrating it with a VR platform like the Oculus Rift, to map and augment environments, is something I’m really interested in pursuing.

I think being able to track a space, map it, resolve its mesh, and use it for compositing against a plate, along with re-projecting it onto the Oculus, would be a really fun experiment. You wouldn’t need the treadmill because it’s not an actual virtual space, it’s a reprojected, augmented virtual space, so you wouldn’t need the track wheel.

Davidson says he’s concerned about the latency in these devices, but has hope for them in the future. It’d be pretty interesting to see a game solve the problem of you possibly running into walls by projecting a virtual barrier, such as a cliff face, to make you avoid continuing in that direction. The many unintended uses of Kinect continue!