Impressively human-sounding ‘Google Duplex’ A.I. demo could be the future of customer, B2B conversations
By onAnnouncements | Business Practices | Insurance | Market Trends | Repair Operations | Technology
The next customer calling your auto body shop for an appointment might be an artificial intelligence — and you might not even know it.
Google earlier this month revealed Google Duplex, in which a user tells Google Assistant to schedule an appointment or reservation for it in a situation where this process couldn’t be handled online.
Google CEO Sundar Pichai told the Google I/O conference that 60 percent of U.S. small businesses don’t offer online booking for appointments.
“We think A.I. can help with this problem,” Pichai said. He even mentioned the automotive aftermarket (booking an oil change) as an example of the kind of calls Duplex could make on behalf of a consumer.
He then went on to reveal an impressive demonstration of incredibly lifelike A.I. female and male voices — Duplex even throws in an “um” and an “I gotcha,” for example — booking a woman’s haircut and a restaurant reservation for four.
In it, Google Assistant took a simple command like “Make me a haircut appointment on Tuesday morning anytime between 10 and 12” and handled the entire booking itself, responding to the natural flow of the conversation. (See the demo here starting at about the 35:00 mark.)
This isn’t the usual robocall or chatbot you recognize after a couple of seconds and start hollering “Representative!” in frustration. This thing sounds like a real human.
The news outlet Axios has questioned the nature of the conversations in the demo, but Pinchai said the first involved a “real salon” and the restaurant conversation was a “real call” as well. A Google AI blog post even shows two Google Duplex staffers eating at a restaurant table booked by the A.I. software.
The Google Duplex technology hasn’t actually begun testing yet with real people; Google says it will be integrated into Google Assistant for such research starting this summer, which means shops might actually begin to take inquiries from the robots.
“Google Duplex’s conversations sound natural thanks to advances in understanding, interacting, timing, and speaking,” Google Duplex principal engineer
“At the core of Duplex is a recurrent neural network (RNN) designed to cope with these challenges, built using TensorFlow Extended (TFX). To obtain its high precision, we trained Duplex’s RNN on a corpus of anonymized phone conversation data. The network uses the output of Google’s automatic speech recognition (ASR) technology, as well as features from the audio, the history of the conversation, the parameters of the conversation (e.g. the desired service for an appointment, or the current time of day) and more. We trained our understanding model separately for each task, but leveraged the shared corpus across tasks. Finally, we used hyperparameter optimization from TFX to further improve the model. …
“The system also sounds more natural thanks to the incorporation of speech disfluencies (e.g. ‘hmm’s and ‘uh’s). These are added when combining widely differing sound units in the concatenative TTS or adding synthetic waits, which allows the system to signal in a natural way that it is still processing. (This is what people often do when they are gathering their thoughts.) In user studies, we found that conversations using these disfluencies sound more familiar and natural.”
“Also, it’s important for latency to match people’s expectations. For example, after people say something simple, e.g., “hello?”, they expect an instant response, and are more sensitive to latency. When we detect that low latency is required, we use faster, low-confidence models (e.g. speech recognition or endpointing). In extreme cases, we don’t even wait for our RNN, and instead use faster approximations (usually coupled with more hesitant responses, as a person would do if they didn’t fully understand their counterpart). This allows us to have less than 100ms of response latency in these situations. Interestingly, in some situations, we found it was actually helpful to introduce more latency to make the conversation feel more natural — for example, when replying to a really complex sentence.”
Even though the Google technology could eventually offer an alternative to consumers who prefer digital scheduling over making a call in person, shops who don’t offer online scheduling might wish to consider adding it anyway.
A 2015 Cox Automotive study found 14 percent of dealership service visits were scheduled online, 76 percent of those would pick that option again, and 45 percent of the 86 percent not scheduling online didn’t know that option existed — suggesting the numbers preferring online appointments might really be higher.
Fixed Ops Journal in October 2017 described a dealership service center which one day had a third of its appointments scheduled digitally. Health care marketing firm PatientPop in 2016 claimed a survey of an unspecified number of consumers found 41.5 percent of them would prefer to schedule doctor visits online, and 34.3 percent of online scheduling occurred after-hours.
One interesting concern about the system, besides the obvious fear of being used for scams or just mischief like using bots to book all of a restaurant’s tables automatically, surrounds how well the party on the other end of the line knows they’re talking to a robot. Google employees explained to Bloomberg how the company might handle the issue in this May 10 article:
Scott Huffman, an executive on Google’s Assistant team, said the response to Duplex was mixed. Some people were blown away by the technical demos, while others were concerned about the implications. Huffman said he understands the concerns. Although he doesn’t endorse one proposed solution to the creepy factor: Giving it an obviously robotic voice when it calls. “People will probably hang up,” he said.
In an interview on Wednesday, Huffman suggested the machine could say something like, “I’m the Google assistant and I’m calling for a client.” More experiments are planned for this summer, he noted.
Another Google employee working on the assistant seemed to disagree. “We don’t want to pretend to be a human,” designer Ryan Germick said when discussing the digital assistant at a developer session earlier on Wednesday.
“The Google Duplex technology is built to sound natural, to make the conversation experience comfortable,”It’s important to us that users and businesses have a good experience with this service, and transparency is a key part of that. We want to be clear about the intent of the call so businesses understand the context. We’ll be experimenting with the right approach over the coming months.”
One could see Google or some competitor offering the inverse of this product for businesses as well for basic conversations — confirming one’s hours, taking a reservation, or even telling a user about variables like gluten-free entrees or the chef’s special — making the interaction more natural than scripted robotic answers.
This raises the question if someday customers will be talking to a soothing A.I. to report a crash, refer a shop and (through predictive analytics) receive an initial estimate. Besides the basic Q&A and scheduling discussed above, that seems like the most likely Google Duplex conversation in the industry. It’s possible an insurer might use an A.I. to handle conversations with shops and or consumers regarding supplements, but any instance where the carrier — technically, the set of instructions given to the software — disputes the shop’s charge would probably just lead the A.I. to defer to a human given the nuances of the topic.
“The Google Duplex system is capable of carrying out sophisticated conversations and it completes the majority of its tasks fully autonomously, without human involvement,” Leviathan and Matias wrote (emphasis theirs). “The system has a self-monitoring capability, which allows it to recognize the tasks it cannot complete autonomously (e.g., scheduling an unusually complex appointment). In these cases, it signals to a human operator, who can complete the task.”
But set aside those fancy insurtech issues for now. Another intriguing application for Google Duplex or a competitor in the near term might be to reduce the inefficiency in mundane administrative communications between members of the collision repair ecosystem in situations where “call and fax” is the only option. Such inquiries seem like the kind of basic conversations Google hopes to automate between consumers and businesses — so why not allow them in instances where the consumer is another business?
For example, could an insurer or OEM ask Google Assistant to “See if Shop X has any openings this week?” and save a live phone call in instances where the carrier/OEM’s software isn’t integrated with the shop’s scheduling system? Could a shop lacking Collision Link ask the A.I. to “Check if Batmobile Dealership X has price-matching on an OEM 2012 Tumbler radiator?” Or could a repairer without automated inventory management request Duplex to “Order me 5 more rolls of tape from ABC Jobber Inc.”
While likely not as seamless as a true system-system integration, the “caller” could eliminate some administrative inefficiency in these instances should Google or a competitor go B2B with it.
More information:
“Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone”
Yaniv Leviathan and Yossi Matias, Google AI blog, May 8, 2018
Google Developers YouTube channel, May 8, 2018
“Google Grapples With ‘Horrifying’ Reaction to Uncanny AI Tech”
Bloomberg, May 10, 2018
Featured image: The next customer calling your auto body shop for an appointment might be an artificial intelligence — and you might not even know it. (gremlin/iStock)