Project Tango Hands-On: Computer Vision Is So Much Cooler Than You Think

Project Tango Hands-On: Computer Vision Is So Much Cooler Than You Think

Google’s Project Tango tablet can see in 3D, but I didn’t really understand why I, human person, would actually care about the technology beyond an abstract appreciation of the ideas behind it. Until I strapped one to my face, with an Oculus Rift-like head mask.

Until I strapped the mask on, I never believed that Project Tango could be more than just a concept. At best, I thought, it would end up being a gimmick, sold to sucker consumers who didn’t know how to shop. I was so wrong.

Announced earlier this year, Project Tango wants “to give mobile devices a human scale understanding of space and motion.” It uses a motion-sensing camera, a 3D depth camera (basically a really sophisticated Kinect), as well as a regular visual camera. All of the data is pulled in and processed in real-time by two computer vision processors.

The tablet version of the hardware uses Nvidia’s Tegra K. In the words of one Google employee, “it’s basically the most powerful tablet in the world right now.” And the tech works very well! During Lee’s presentation today, he says that the team has achieved just 1 per cent drift when using the tablets to map complicated spaces by walking around in them.

But it goes beyond just developer edition dreams. Project Tango’s technical lead Johnny Lee announced that in addition to the dev-focused tablets, ATAP is working with LG to make a consumer version for 2015. For normals. Seriously.

Why would a regular person ever want to use a Tango device, even if it can see like a human and fit in your pocket? What does that mean in practical terms? How does it transcend gimmick status? Today I got my hands on a number of the apps and tools Google’s been working on to explore that very question.

The one that made me giggle (joyful, not mocking) the most was an augmented reality headset built by Durious. The company already makes a Dive headset that lets you insert a phone so you can use it for VR experiences like an Oculus Rift. At Google I/O, though, Durious showed off a prototype developed especially for Project Tango, as well as a little demonstration app, that overlays an aquarium in front of your face. Fishes flying around right in front of Google’s booth personnel. You can even move in and out of the water by kneeling and standing up. It takes all of the 3D gaming and immersive experience potential of the Oculus Rift but steps it up because it can see the world in front of you as well. Sure, the application is a little silly, but it shows you the potential of what’s possible with Google Tango used as a head mounted display.

It’s also an interesting expansion on the other VR concept Google is showing off at I/O. The simple concept uses a very basic cardboard design to turn your smartphone into a VR headset — so it’s basically like a Dive made out of cardboard instead of Dive.

Gaming is one of the main development focuses for Project Tango team, and they have gone to lengths to build out a few different gaming experiences as well as partnering with studios like Epic Games. At I/O, Google showed off a few sample games, including a zombie killer in which you aim by moving the tablet around. Killing zombies is fun! But the most compelling experiences were some relatively empty maps you might find in a 3D shooter, which use Project Tango’s 3D sensing capabilities. When you take a step forward in real life, the tablet notices you’ve moved and moves you inside the map you’re looking at. As with the augmented reality experience above, it adds a new dimension to the the 360 VR you get from Oculus or Cardboard.

cThen there are more utilitarian applications that let you map the real world in front of you. In one very simple app, you walk to the four corners of a room, and Google Tango rends a very precise SketchUp outline of the space.

In another, designed by AutoCAD, you can precisely measure and model a room and its contents so that it’s easier to do a little interior designing. No more going back and forth from the furniture store to measure something over and over. No more guess how much carpeting you need to buy.

More than anything else, I was impressed at just how well the technology works. Some of the real-time rendering in front of you can be a little a choppy and buggy, but when it comes to the actual measurements that the technology is making, they’re ultra-precise. Project Tango works, and it’s got real-world potential. It just needs some regular people polish now.

Over the last month or so I’ve spoken with several of the partner companies that actually built the technology that underlies Project Tango about what exactly Google hoped to do — and what it should do with Project Tango going forward. According to Farshid Sabet, SVP of Movidius, a company that makes one of the computer vision processors in the Tango smartphone from earlier this year, Google really wants to illustrate that the technology is more than just a professional tool or a wonky concept. And to make it more than tool for professionals, the software has to work in a dead simple way, because we’re not going to put up with the hiccups the way a developer or a member of ATAP might.

Today, I got my first taste of what Google’s been cooking up, and I can’t wait to try to whole thing. And what might be most amazing is that it won’t be long before we all get to.