Among updates about new features coming to Android P, a new direction for Android Wear, and improved image recognition in Google Lens, the thing that really stole the show at Google I/O 2018 was Duplex, the automated phone service that can make a dinner reservation or an appointment at a hair salon without you ever needing to actually dial a number.
Screenshot: Google
After all, Duplex’s Google Assistant-powered voice sounded so natural with its perfectly timed umms and ahhs, while also handling the over-the-phone dance of booking a meal complete with time, date and party size with an impressive level of ease and fluidity.
But at the same time, Duplex also scared the hell out of people. There were so many unanswered questions: Did the person on the other end of the line know that they were talking to a robot? What happens when Duplex gets things wrong? And more importantly, what does Duplex mean for the future of phone calls as a whole?
That brings us to now, where less than two months after I/O 2018, Google is going to unleash Duplex into the wild over the next few weeks to a limited number of users and businesses to get some real world feedback.
Meanwhile, I’ve just come back from a Thai restaurant on Manhattan’s Upper East Side, where Google hosted a demo that shed some more light on its impressive (if a bit intimidating) robo-calling service, and even gave me a chance to try Duplex out for myself.
So what’s the first thing I did? Attempt to break it, of course.
And even though I successfully flummoxed Duplex while posing as a restaurant host as it tried to make a reservation, I couldn’t help but be impressed at how good Google’s voice assistant was. Unfortunately, we weren’t allowed to record audio or video at the event, but hopefully my first-hand account will be just as good.
Back at Google I/O, what CEO Sundar Pichai didn’t show was all the ways Google designed Duplex to be an asset to both businesses and individuals before ever making a call. From a user standpoint, Duplex is really just one small tool Google can use to streamline our lives.
Currently, if you’re trying to book a restaurant using the Google Assistant and there’s a suitable reservation available via OpenTable, your phone will just use the app instead, there’s no need to get Duplex involved.
However, if the restaurant doesn’t support online booking, something that Google says is true of around 60 per cent of all small businesses, that’s when you might want some extra help. Enter Google Duplex.
Screenshot: Google
After making a request to book a table and providing the relevant information, the only thing a person will ever see on their phone is a message from the Google Assistant saying that it’s going contact the restaurant, and that it will get back to you in 15 minutes or so.
As for businesses, Duplex will be an opt-in service, which should prevent Google from clogging up the phone with unwanted calls.
When Duplex actually gets around booking an appointment, the first thing it tells the person on the other end is: “Hi! I’m calling to make a reservation. I’m Google’s automated booking service, so I’ll record the call.”
So as good as its voice sounds, that disclaimer means from the start, there should be no confusion regarding what’s going on, what Duplex is, and why it s calling. This is huge because it sets limits on what people expect to take place during the call, while also calming people who might understandably be concerned about being scammed, tricked or otherwise taken advantage of.
Additionally, Duplex’s ability to record calls also plays an important benefit when it comes the service’s first major function: Checking holiday hours.
That’s because if just one person uses Duplex to inquire about holiday hours, Duplex can use that info to update Google Search as a whole. Now, the next time someones wants to know if a shop is open on Australia Day, Google doesn’t even have to trouble the business with a call, because it already knows.
And once Duplex has a chance to test its skills checking holiday hours, booking reservations and making hair appointments are next.
At Google’s demo event, conversations between Duplex and mix of presenters and attendees played out almost exactly like they did on the stage back at Google. At first, Google reps played it straight, posing as the host of the restaurant while Duplex made reservations using info provided by crowd.
But then, Google turned Duplex over to a handful of journalists, including me, and gave them a chance to act out a scene themselves, with each person adding a twist to the regular procedure of making a restaurant.
When one person asked if they were talking to a person or a computer, Duplex responded by identifying itself as an automated assistant. When asked for an email address instead of a phone number, Duplex said it wasn’t authorised to provide that info. And when a person posing as a host asked if anyone in the party had any allergies, Duplex simply responded that it didn’t know.
But regardless of the curveballs the four people who demoed the system ahead of me lobbed at Duplex, each call ended with a successful reservation. Not too shabby, though naturally, I took that as my cue to find a way to break Google’s system.
My plan was simple enough. After Duplex asked about a reservation for 6PM, I simply responded by saying, “Sorry, the restaurant is booked for a private event at the time from 6 to 10.” Nothing outlandish and a relatively common enough occurrence that many New Yorkers have encountered from time to time.
However, being the accommodating digital assistant that is, Duplex didn’t give up, and instead asked about moving the reservation up to 5:30.
Presented with the same situation, a person might have intuited that booking a table 30 minutes before a private event probably wouldn’t fly, but it never hurts to ask, right? So I responded by saying if the party would be able to eat and leave within 45 minutes to an hour after arriving, I could still book the reservation.
Look, I know that isn’t exactly a polite thing to do, but it’s a situation that’s happened to me multiple times, especially when I’ve fancied eating at anywhere even remotely trendy, which in some areas of Brooklyn, is basically every other restaurant.
Unfortunately for Duplex though, it was that request that caused it to stumble, and after trying multiple times to rephrase the question and suggest a 5:30 reservation, Google’s robo assistant gave up.
However, the Duplex experience wasn’t quite over, because almost immediately after the system realised it was at an impasse, the AI voice was replaced by an actual human operator, who stepped in to confirm the details and finish booking the reservation.
Now some might call that a failure, but to me, having a backup system is an important piece of forethought, as it shows Google knows that even as slick as Duplex often seems, it still needs some help every once now and then.
And in a strange way, the demo at the Thai restaurant echoes the success rates Google has seen so far, with Duplex able to complete four out of five calls completely unassisted, while the latter 20 per cent still needs a bit of human intervention.
As for the future, Duplex is in a strange spot, because while Google is bringing its robo-calling assistant to a small number of users and businesses this winter, adding new calling abilities isn’t as simple as it might seem. That’s because even though AI and machine learning play a big part in Duplex’s abilities, Google can’t just flip a switch and add additional skills.
To get Duplex just to this point, Google had to manually analyse and annotate hundreds if not thousands of calls manually in order to create decision trees that Duplex could break down and understand.
Then Google had to provide assistance during the learning processes every time Duplex made a mistake, not only to make sure the task was performed correctly, but to give the system guidance on what it should have done better.
It’s only in the last stage where Duplex is able to largely function on its own.
Currently, Google says it isn’t shooting for a specific success rate, its more focused on making sure Duplex delivers a tangible value to both businesses and users, and in Duplex’s early days, that makes a lot of sense. Only then, will Google look to expand Duplex’s skills to other areas.
But at some point, those success rates are going to have to be sorted. Because even if Duplex is released only in the US, that’s potentially 50 to 100 million devices that could use Duplex to make calls. With that kind of volume, even a 99 per cent success rate would leave Google with up to 500,000 calls it would need to intervene on, a figure much too large to have humans waiting around to handle.
However, for a project that was only revealed to the public less than two months ago, Duplex has shown some incredible results. And after getting a chance to test it for myself, while I’m still not sure that AI won’t cause the eventual downfall of the human race, I’m relatively confident Duplex won’t be the little snippet of AI that starts the war.