Currently, Google home devices require you to activate the assistant by saying ‘Hey Google’ or ‘OK Google’. But soon, you will be able to just… stare at your device and the Google Assistant will start listening to you.
This creepy new feature is called ‘Look and Talk’ and it’s first rolling out to Google’s Nest Hub Max.
Look and Talk
It was announced during Google I/O this morning and touted by the search giant as “making it much easier for users to initiate a conversation with a Google Assistant”.
From a user perspective, there’s not much more to it: Look and Talk allows you to communicate with the assistant by looking and talking. Google reckons this is a lot more natural, because IRL when you want to strike up a conversation with someone, you simply look at them and start talking.
“Achieving this was no easy task, we needed to build an assistant that could distinguish things like intentional eye contact and simply a passing glance,” Google Assistant product manager Jaclyn Konzelmann said during a press briefing.
“This required six machine learning models, to process over 100 signals from proximity to heigh orientation and gaze direction, all in real time.”
If you’re thinking, What if I accidentally glance at my Google device while I’m having a very private conversation? Well, in addition to the constant reassurance that our devices aren’t listening to us, Google reckons Look and Talk is wrapped in the Nest Hub Max’s “camera sensing safeguards”.
Look and Talk uses face match and voice match to recognise that it’s you. It also takes into consideration your proximity, head orientation and gaze direction. Google says the video transaction is processed completely on-device (which means it isn’t shared with Google or any third parties), and the feature is also opt-in (you have to opt-in to each part, whch might seem tedious, but privacy, people).
Look and Talk starts rolling out on Android this week, with iOS to follow ‘soon’.
Quick Phrases
Already available on Pixel phones, ‘Quick Phrases’ is also heading to Google’s smart home devices.
If you aren’t familiar, this feature allows you to skip that whole ‘Hey Google’ thing for common daily tasks. The idea behind Quick Phrases is straightforward: allowing people to respond to common situations with short voice commands while skipping wake words entirely.
You will be able to choose which Quick Phrases you want available from a list Google provides, including setting timers, turning lights on and asking for the weather. It will also be able to handle the different ways people ask for the weather, for example (it’s also voice-matched to each person in your household).
Powerful speech and language models
From early 2023, the Google Assistant will be able to start having more ‘natural’ conversations with you, with the ability to understand the nuances of your voice (such as if you pause mid-sentence) and not chime in with what it thinks you were searching for. Konzelmann said this mimics more ‘human’ conversations.
Answering questions on Look and Talk, Konzelmann was asked if the idea was to eliminate the need for ‘Hey Google’ altogether. She said no, rather it’s to expand the ways in which you can start talking to the assistant that mirror real-life.
“I really like to think of ‘Hey Google’ as being similar to the assistant’s name … similar to how all of us have names and they’re very useful, sometimes people call my name before they want to talk to me, but other times they don’t. They might simply look towards me, they might tap me on the shoulder, or if it’s just the two of us sitting in a room, they might simply ask their question,” she said.
So what you’re saying is next my Google Nest Hub Max will reach out and tap me on the shoulder if it wants my attention?