Google is upgrading its visual search app, Lens, with the ability to answer real-time questions about your surroundings using AI-powered video analysis. This new feature, available for English-speaking Android and iOS users through Google’s Search Labs program, allows users to capture videos with Lens and ask questions about objects in the footage. The AI then provides relevant answers from Google’s AI Overviews, which summarizes information from across the web.
Lou Wang, director of product management for Lens, highlighted how the feature uses Google’s “customized” Gemini AI model to interpret video content and respond to user inquiries. “Let’s say you want to learn more about some interesting fish,” Wang explained. “[Lens will] produce an overview that explains why they’re swimming in a circle, along with more resources and helpful information.”
While recording a video, users can ask questions, and Lens will analyze specific frames that are most relevant to the inquiry. Although the system is asynchronous, meaning you can’t interact in real-time like with chat-based AI, it marks a significant advancement in visual search technology.
This update puts Google ahead of Meta and OpenAI, both of which have teased similar AI video analysis capabilities. Meta’s Ray-Ban AR glasses and OpenAI’s Advanced Voice Mode are set to introduce real-time video search, but Google’s new Lens feature is already available.
In addition to video, Lens is expanding its image-based search capabilities and adding new e-commerce functionality. When Lens recognizes a product, it will now display details like price, reviews, deals, and stock information. This feature is initially limited to select shopping categories such as electronics and beauty products.
Google has also begun integrating ads into AI Overviews, its AI-generated summaries for certain search queries. The company is expanding its AI-driven search results pages in the U.S., part of its broader strategy to maintain dominance in search as competitors like OpenAI’s ChatGPT and Perplexity gain traction.
Despite the promise of AI Overviews, the feature has been controversial. Critics have pointed out that some of the information it surfaces is unreliable, and Google has made efforts to improve its accuracy, especially regarding current events and health topics.
Still, Google is betting on AI Overviews to boost user engagement, particularly among its key demographic of 18- to 24-year-olds. Now, with the addition of ads, the company is taking steps to monetize the feature. As VP of user experience for Google Search Rhiannon Bell put it, “We’re doing everything that we can to bring the right content to our users.”
