Google Discusses the Future of its Assistant
- Paul Thurrott
- Sep 08, 2017
-
39
Google quietly hosted a developer show in Europe this week, and one of the best things I’ve seen so far is an exciting update about the future of Google Assistant.
That developer show, called Google Developer Days Europe, was held in Poland on Tuesday and Wednesday, and you can find some videos from the keynotes and sessions on YouTube. Among them is the Day 2 Keynote, in which the firm discusses some of its plans for the improving Google Assistant across devices. (I was tipped off to this information by Android Police.) It’s worth watching.
“Google Assistant is one of the core implementations of an AI-first world,” Google senior engineering director Behshad Bezhadi said during his talk. “We feel that we [Bezhadi leads a team of over 100 engineers building Google Assistant] are part of this next revolution.”
After a short discussion of why AI is such a revolution, Bezhadi argues that AI, in some ways, is making smartphones truly smart, by adding conversational, situational, and intra-device capabilities.
“Conversation is really the core of what we think about when we think about Google Assistant because we think that conversation is the most-used interface human beings have used,” he says. “If we solve the problem of conversational understanding, we can expand it to many use cases.”
Bezhadi then showed some demos of where Google Assistant is at today, using a Pixel smartphone. None of this will be particularly surprising to anyone who has interacted with Google Assistant or any other digital personal assistant: You can have short, meaningless conversations, find out about the weather, discover local attractions, and so on. Google Assistant can also be used to learn about much more specific things: He asked about rides at a particular amusement park, and then asked about the height restrictions at a particular ride at that park.
More intriguing, of course, are the Google Assistant features he demoed that are not yet live. Some of these will launch in the next few months, he said, while others will arrive next year.
“We’re trying to understand more complex natural language,” he said, launching into the following question: “What is the name of the movie where Tom Cruise acts in it and he plays pool, and while he plays pool he dances?” Google Assistant responds, correctly, with some information about the movie “The Color of Money.”
That’s impressive, but Bezhadi also shows how Google Assistant can be enhanced by connecting to other services, either from Google or from third parties.
“Be my Vietnamese translator,” he says to Google Assistant, which says it understands and will translate all of his following sentences to Vietnamese. All he has to do to stop is say, “Stop translation.” This, too, is an impressive demo, and it’s not hard to imagine how this functionality would transform international travel.
Bezhadi goes on to control Google Maps’ Street View (“I would like to be on the top of Eiffel Tower now, can you please bring me there?” as well as a third-party service—“Talk to WebMD”—where he asks about the side-effects of aspirin. He asks implicit questions—“How is my team doing?”—and Assistant uses context about his profile, preferences, and query history (and presumably the time of the year) to answer about the right team.
That latter capability is particularly impressive when you consider that you can teach Assistant about yourself and it will get smarter as you do. This capability is already present in a basic form now, but it will get more capable in an update coming next year by adding natural language learning.
“When the weather is more than 25 degrees [Celsius], I can swim in the lake of Zurich,” he tells Google Assistant. Then, he asks whether he can swim in the lake of Zurich this weekend. “No, you can’t,” Google Assistant replies. “The temperature is less than 25 degrees.”
On a related note, the way that Google Assistant handles context is also going to get more sophisticated. Consider contextual voice recognition: He asks about a particular mountain range. And then he asks about the height of one of the mountains in that range by name. Because the mountain range discussion had happened, Google Assistant understood the context of the next question.
“Contextual voice recognition is going to play a big role,” he said. “Because people always try to continue talking about things which are related, location specific, or contextually relevant.”
In another example, Bezhadi asks about “pictures of Thomas,” but because there is no context, it shows him pictures of Thomas the Tank Engine. So he asks about a sports team roster, and is shown a list of players. So he then asks about Thomas again, and this time he is shown pictures of a player on that team who is named Thomas.
Context is also important for having a continuing conversation. He asks about where the Empire State Building is, and gets an answer. Then asks, “I want to see pictures.” And he is shown pictures of that building. And he asks “how tall is it?” he is given the answer for the Empire State Building. He goes on to ask other questions: “Who built it?” “When?” and “What are the Italian restaurants around there?” and “Call the first one.” It’s a conversation.
Context also works visually (or soon will): Through the integration of Google Lens for visual input, Google Assistant will be able to perform some impressive feats. For example, he points Lens at an apple and then asks, “How many calories?” and Google Assistant tells him how many calories are in an apple. He also did a demo with money in which Assistant converted it to a different currency.
Google is also working on improving voice recognition in noisy environments. With music blaring and the audience roaring, Bezhadi asked questions through his phone, and the Assistant answered correctly. (This one is obviously hard to describe.)
Overall, this was an impressive set of demos. And anyone who bets against Google in this area isn’t seeing the big picture. There’s no question whether Google Assistant wins this war. The only question is which assistant ends up in second place.
Tagged with
Conversation 39 comments
-
Stooks
<blockquote><a href="#174969"><em>In reply to Darmok N Jalad:</em></a></blockquote><p>Google is one serious government legislation away from huge earning stumble. With something like 89% of their revenue coming from ads, which come from information gathered from their free apps it could be a company killer if some serious privacy legislation comes down.</p><p><br></p><p>The public is growing increasingly more concerned about privacy. I think we are going to see some serious regulation in the US that will impact Google and others about what is collected and stored. </p>
-
Stooks
<blockquote><a href="#174971"><em>In reply to redstar92:</em></a></blockquote><p>Agreed. If I wanted so much Google coverage, 9to5 Google is a much better choice.</p>
-
Stooks
<p>Paul you need a "Google" tab at the top of you site. Only question is, does it come before or after "Windows 10" ??</p>
-
m_p_w_84
<p>they must have more than 100 people</p><p>working on this </p>
-
Bats
<p>AI is going to be super important and from what I see here, in addition to what I am personally experiencing Google has the upper hand thanks to the Google Assistant. I have dictated Google Docs documents with a lot of success. It just works! However, Google's Voice Recognition works great for me in the car. LOL. I can simply text without typing, while I am driving. That's just one of the great features with Android. You can do anything with it, without even touching your phone.</p><p><br></p><p>Without question the Google is the best, by far, Digital Assistant around. I don't know about Siri, but she is no match for Google Assistant. As for Cortana….forget about it. It's a little better than Siri, but completely stupid. I have Google Home and Cortana (via PC) open all the time and I trigger with hotwords. The Google Assistant always gets my question and tells me the answer and it's usually about weather, facts about actors (because I watch tv), and other things. Cortana always hears me wrong. Like a couple weeks ago, I asked who the actor was who Rhaegar Targarian on Game of Thrones. Google Assistant gave me the name of the actor and some little story about him. I asked Cortana and she gave me the website for Raymour and Flanigan furniture store.</p>