Google to Reimagine Android with AI

During its Google I/O keynote today, Google announced that it was undertaking a multi-year effort to reimagine Android with Gemini AI at the core.

“We’re at a once-in-a-generation moment where the latest advancements in AI are reinventing what phones can do,” Google president Sameer Samat said. “With Google AI at the core of Android’s operating system, the billions of people who use Android can now interact with their devices in entirely new ways.”

Google started this effort by bringing its Gemini Nano AI model to its Pixel 8 series phones and then the Samsung Galaxy S24 series. And by releasing a Gemini mobile app that can replace the in-box Google Assistant. Since then, it’s added new AI capabilities to all its most popular Android apps, alongside new AI features like Circle to Search. And today at IO, it announced the next steps.

First up is Circle to Search, which now supports physics, math, and word problem-solving capabilities for students. This feature will be expanded further this year to help solve more complex problems involving symbolic formulas, diagrams, graphs, and more. And while it’s available today on over 100 million devices, Google will add more devices in 2024 with the goal of more than doubling its availability by the end of the year.

While Gemini is installed like any other app on Android, it’s really an integrated experience, and it will simply be included in future versions of the system. In a coming update, you’ll be able to display a Gemini overlay on top of any app so you can use it in more integrated ways. “For example, you can drag and drop generated images into Gmail, Google Messages, and other places,” Samat notes, “or tap ‘Ask this video’ to find specific information in a YouTube video.” Those paying for Gemini Advanced will also get unique options for summarizing and querying large PDF files.

Gemini Nano—the on-device version of Google’s AI model—will soon evolve into a new Gemini Nano with Multimodality model that will appear first on select Pixel devices later this year. This version of Gemini won’t be limited to processing text, but will also support vision and audio inputs, including spoken languages.

Google is also bringing Gemini Nano’s multimodal capabilities to TalkBack, the Android accessibility service, to help users with blindness or low vision get better image descriptions. Since Gemini Nano is on-device, these descriptions happen quickly and even work when there’s no network connection, Google says.

Gemini will also help transform Android’s already impressive anti-scam and anti-spam tools. Google showed off a feature that provides real-time alerts during scam phone calls that will help protect customers from identity attacks and other scams. As with Talkback, these improvements will leverage on-device Gemini Nano capabilities.

There’s a lot more, of course, and Google says it will have more announcements related to Android 15 and the Android ecosystem tomorrow. Stay tuned.