The most futuristic projects at Google come out of their advanced technologies and projects group, or ATAP. And today at Google I/O, the ATAP team unveiled their vision of tomorrow, where your body, behaviour and clothing will be the new interfaces that control your gadgets — and keep them secure, too.
Project Soli: A Radar Chip That Sees Your Hand Gestures
Over the past 10 months, ATAP has developed a chip the size of your thumbnail that serves as a radar detector. It’s used to sense small, flickering movements of your hands so that just rubbing your thumb and forefinger together allows you to scroll or select items on a screen. During the ATAP session, we saw interaction designer Ivan Poupyrev show off the tiny chip, and then demonstrate how it picked up small finger movements and translated them into scrolling, selecting, and twiddling a knob.
The technology evolved after the ATAP team realised that the wanted “hands to be user interfaces” because fingers are — as Poupyrev put it “easy to use and very ergonomic.” To pick up hand gestures, though, they had to throw away conventional wisdom about what kinds of sensors to use — they would have to pick up movement, and be able sense through a lot of materials. Radar was the answer.
Working with hardware manufacturers, the group miniaturized the kinds of radio frequency sensors used in everything from submarines to satellites, and made it small enough that it can fit into a gadget.
ATAP radio engineer Jaime Lien showed the crowd how the chip uses everything from doppler to spectrogram readings to see hand motion, size, and velocity — eventually using machine learning to resolve those motions into recognisable commands.
One can easily see how Soli would work well with something like Google’s augmented reality project Magic Leap, providing users with a 3D immersive experience that only requires tiny finger gestures to control.
The group is working on a board and a software API, which they are planning to release to developers and makers later this year. Look for it!
Project Jacquard: Your Clothing Is A Sensor
Poupyrev also introduced us to Project Jacquard (here’s our hands-on experience with Jacquard), which is a system for weaving electronically conductive thread into yarn. The result is a garment that is essentially a giant touchpad. Just tap your sleeve, and make a phone call.
Or brush your fingers over a sofa pillow to activate the lights. We’ve tested their touch-sensitive fabric ourselves, and it does indeed work just like a mousepad or touchscreen, capturing everything from taps and swipes to two-finger gestures like pinches.
With your fingers and clothes serving as an interface, the Jacquard team hopes that people of the future will be able to control their devices without constantly poking at their phones — or, as one team member put it, you can check your phone while also “looking your dinner date in the eye.”
Jacquard is ready to scale up production and has partnered with Levis to develop a line of digitally-enabled clothing. Look out for a crowdsourced project from Levis, that will encourage makers and crafters to contribute their ideas for clothing that incorporates Jacquard technology.
Body Authentication and Vault: Passwords Suck
ATAP captain Regina Dugan, former director of DARPA, gave us an introduction to ATAP’s suite of security tools and futuristic research. First she explained that ATAP is working on a way of eliminating passwords by creating a system that authenticates your identity by analysing your everyday behaviours — from how you walk and type, to how you talk and where you usually go. Imagine a phone that tracks you every behaviour, learns what’s typical, and then grants you permission to certain apps (like your bank account) based on whether it “recognises” you. The idea is that if an adversary grabs your phone, her behaviour won’t match and she won’t be able to get into your sensitive data.
This project flowed nicely from what’s coming out of Soli and Jacquard. Your body is becoming part of your devices, and this can make things more secure, as well as easier. (And possibly scarily dystopian.)
Then security legend Mudge took the stage to introduce Vault, a “digital mobile safe” that takes the form of an ordinary SIM card. Basically, he said, “it’s a separate ultrasecure minicomputer” where you can keep your sensitive data, but also use the onboard crypto tools to engage in secure communication like chat.
We already have SIM cards controlled by our cellular providers and trusted hardware elements controlled by vendors. “Where’s the security element that you have complete control over?” Mudge asked. “We made one.” Mudge added that when you insert the Vault into your device, it looks like a regular SIM with a read and write directory — so no specialised apps or knowledge are required.
Vault runs an ARM processor, has an NFC, and is stuffed with a suite of crypto services including a hardware random number generator and 4 GB of isolated storage. Everything in it, including the processor, is open hardware. Mudge thinks it’s ideal for anything from phones to other connected devices.
There’s a developer kit for Vault available now on github. This is the ideal tool for putting your security back in your own hands.
Taken together, Soli, Jacquard, Vault and ATAP’s more speculative authentication tools all suggest that Google imagines a future where our machines are seamlessly integrated into our bodies and habits. This can only help build up an ecosystem of connected devices that look exactly like the objects in our world now: jackets, pillows, SIM cards. But when our bodies become input and authentication devices, there are potential hazards too. The more our devices merge with us, the greater the potential betrayal when they get buggy or become vulnerable to attack.