Google improves AI Overviews following initial rollout challenges

Google is making some changes to its AI Overviews, after the artificial intelligence-driven search feature gave what the company calls “odd and erroneous” responses to people’s online searches.

AI Overviews were introduced last month at Google’s annual I/O developer conference. Now, when people use Google Search to find information on certain topics, a box of AI-generated text appears at the top of the search results, annotated with links to external websites. Traditional search results appear below the AI Overviews, marking a major shift in how Google presents information.

According to a blog post from Google VP Liz Reid, AI Overviews results are generated using the company’s large language model (LLM), Gemini, and are designed for instances when someone wants to “get both a quick overview of a topic and links to learn more.”

Google technology expert Alex Joseph told ABC Audio that AI Overviews is able to field more complex questions than a traditional Google Search.

“With an AI Overview what [Google] can really do is synthesize a lot of information and get you the answer that you’re looking for very quickly,” said Joseph.

Instead of presenting users with pages of links to comb though, Joseph said, AI Overviews streamlines the process by summarizing information and providing users with a concise answer.

“They’ll have less friction, they won’t have to click through to a number of different websites, which can often be quite a bit of a bugbear if you just want some information very quickly,” notes Chris Stokel-Walker, technology journalist and author of the book “How AI Ate the World: A Brief History of Artificial Intelligence – and Its Long Future.”

However, Stokel-Walker said the new feature makes it harder for people using Google Search to verify the accuracy of the information they’re reading.

“We’ve kind of got used to over the last two decades of Google Search dominance to the results that we get to a search term being largely right,” he told ABC Audio. “Suddenly, if you get rid of that, as Google is proposing, and actually just shove an answer straight into the search results page that its created via generative AI, you have no real way of identifying and kind of analyzing that information to see whether it’s true or not.”

There are other concerns about the new feature as well. For one, generative AI technology, both from Google and elsewhere, has faced criticism for “hallucinating” – that is, generating information that’s unreliable and inaccurate.

For example, in the few weeks since AI Overviews have been available to the public, people using Google Search have been advised to eat at least one small rock per day, and it told one user that a good way to get cheese to stick to pizza is to mix glue into the tomato sauce – both of which are, of course, very bad ideas. It also said that Andrew Jackson, the seventh U.S. president who died in 1845, graduated from college in 2005.

Stokel-Walker said any benefit from AI Overviews ultimately comes down to a tradeoff between convenience and cost. “You no longer have kind of click through five or six different pages and maybe several pages of search results to find the right answer, but also it does mean that either the answer might be wrong or it might not be the one that you actually want to get,” he said.

“We’ve always been very clear about the limitations of LLMs, that there will be occasionally hallucinations,” Google’s Alex Joseph said, adding that’s why AI Overviews also cites the websites it uses to generate its answers.

“It’s part of the reason that we present all of the information for you holistically,” said Joseph. “These are quick shortcuts to help you get some information to you quickly, but they’re followed along with areas where you can go, double check, verify.”

Joseph also said not all queries are best served by an AI Overviews: “We only show them on queries where we have a high confidence that it’s going to be helpful and actually enhance the experience.”

In the wake of the unusual responses some social media users have reported, Google announced it had made “more than a dozen technical improvements” to AI Overviews. According to Liz Reid’s blog post, they include limiting the inclusion of user-generated content, as well as satirical or humorous webpages, in the data used to craft AI Overviews. Reid said Google also “launched additional triggering refinements to enhance our quality protections” regarding health content, and that it “aim[s] to not show AI Overviews for hard news topics, where freshness and factuality are important.”

The blog post also notes that “AI Overviews generally don’t ‘hallucinate’ or make things up in the ways that other LLM products might,” and that the incorrect answers are the result of “misinterpreting queries, misinterpreting a nuance of language on the web, or not having a lot of great information available.”

Aside from the accuracy concerns, Stokel-Walker said Google prioritizing AI Overviews over traditional search results could affect revenue and reshape how business is done on the web.

“Websites produce content; they try and make it attractive to Google. Google will show them in its search results. And as a result, people click through to their website, they then see ads off the back of that, and the publisher makes the money that allows them to put new content onto websites,” Stokel-Walker explained.

By replacing the top of the Google Search results page with AI-generated content, however, Stokel-Walker said websites could see fewer visitors – and therefore less ad revenue.

It’s an ironic situation, according to Stokel-Walker. That’s because Gemini, the LLM Google uses to create its AI Overviews, relies on the websites it’s now appearing ahead of.

“These websites do still need to exist, and they do need to have a way of making income, because otherwise, there’s nothing for those AI-generated search results to be based on,” Stokel-Walker pointed out.

In a statement to ABC News, Google said its testing has shown that the opposite is actually happening: that the links included in AI Overviews get more clicks than if the page had appeared as it typically does in search results. Google also said it will “continue to focus” on sending valuable traffic to publishers and creators.

Regardless of how the concerns about AI Overviews ultimately shake out, it’s just one of a slew of features the company has planned for its line of technology products.

“I think that doing something like this as quickly as Google is doing it is concerning,” said technology journalist C. Scott Brown of the website Android Authority.

Google has announced plans for additional features similar to AI Overviews, which will aim to answer questions about specific webpages or YouTube videos. Brown says those features will hit the market against the backdrop of increasing competition.

“And the reason it’s doing that is because it feels like it has to. It has to keep up with companies – especially like OpenAI, for example – that are creating generative AI technologies that are threatening Google’s core business, which is delivering information to people through Google search, and thus delivering them advertisements that allow Google to make billions and billions of dollars,” said Brown.

“With Google seeing these things as a threat, it can’t just rest. It can’t figure out how to do this cleanly, and do it right,” Brown added. “It just has to do it.”

Google has been at the forefront of artificial intelligence (AI) development for years, but even tech giants like Google can face challenges when rolling out new AI features. In recent years, Google has made significant improvements to its AI Overviews following some initial rollout challenges.

One of the key challenges that Google faced with its AI Overviews was ensuring that the technology was accurate and reliable. In the early stages of the rollout, users reported instances where the AI Overviews provided inaccurate information or failed to understand context. This led to frustration among users and raised concerns about the effectiveness of Google’s AI technology.

To address these challenges, Google invested heavily in improving the accuracy and reliability of its AI Overviews. The company implemented advanced machine learning algorithms that were able to better understand context and provide more accurate information. Google also increased its training data sets to ensure that the AI Overviews were able to learn from a wider range of examples and improve over time.

Another challenge that Google faced with its AI Overviews was ensuring that the technology was able to adapt to different languages and dialects. In the early stages of the rollout, users in non-English speaking countries reported issues with the AI Overviews not understanding their language or dialect. This limited the usefulness of the technology for a large portion of Google’s user base.

To address this challenge, Google implemented language-specific training models that were able to better understand the nuances of different languages and dialects. This allowed the AI Overviews to provide more accurate information and insights to users in non-English speaking countries. Google also worked closely with linguists and language experts to ensure that the AI Overviews were able to accurately interpret and respond to different languages.

Overall, Google’s improvements to its AI Overviews have been well-received by users and have helped to solidify Google’s position as a leader in AI technology. By addressing the initial rollout challenges and investing in improving the accuracy, reliability, and adaptability of its AI Overviews, Google has been able to provide users with a more seamless and effective AI experience. As AI technology continues to evolve, Google’s ongoing commitment to innovation and improvement will be crucial in ensuring that its AI Overviews remain at the cutting edge of artificial intelligence development.