Google Turns Off Some AI Overviews Following Reports of “Dangerous” Medical Information

Google has quietly tweaked its AI Overviews in search results to remove some AI-generated summaries that provided inaccurate medical information. An investigation from The Guardian revealed that in some cases, Google’s AI overviews provided health advice that experts described as “dangerous” and “alarming.”

AI Overviews are designed to provide ChatGPT-like answers at the top of search results, and Google is still in the process of rolling them out to more markets. However, the company publicly acknowledged that its AI responses “may include mistakes.”

As The Guardian discovered, asking Google “what is the normal range for liver blood tests” may result in an AI Overview with misleading information that doesn’t take into account sex, age, ethnicity, and other criteria.

What Google’s AI Overviews said was normal may vary drastically from what was actually considered normal, experts said. The summaries could lead to seriously ill patients wrongly thinking they had a normal test result, and not bother to attend follow-up healthcare meetings.

You may recall that when Google began rolling out AI Overviews in the US nearly two years ago, the company had to remove many inaccurate and embarrassing responses. In some examples, the AI-generated answers told users to put glue on pizza and eat rocks.

Google remains the most dominant search engine on the market, and with that great power comes great responsibility. Following The Guardian’s investigation, the company removed AI Overviews for the search terms “what is the normal range for liver blood tests” and “what is the normal range for liver function tests”.

“We do not comment on individual removals within Search. In cases where AI Overviews miss some context, we work to make broad improvements, and we also take action under our policies where appropriate,” a company spokesperson also told The Guardian.

The news of Google providing inaccurate health-related AI responses comes right after OpenAI launched ChatGPT Health, which is designed to provide personalized health and wellness advice based on users’ medical records. The new experience will allow users to securely connect their medical records and wellness apps, though the company made it clear that ChatGPT Health can only help users “navigate medical care, not replace it.”