Several weaknesses patched recently by Google in Gemini could have allowed attackers to trick the AI assistant into helping them achieve data theft and other malicious goals.
The issues were discovered by researchers at cybersecurity firm Tenable, who named the project The Gemini Trifecta. The research covers three distinct Gemini hacking methods that abused various features and tools, and which required little to no social engineering.
The first attack involved indirect prompt injection and it targeted Gemini Cloud Assist, which enables users to interact with Google Cloud for managing and optimizing cloud operations.
The attack abused Gemini Cloud Assist’s ability to analyze logs. The researchers discovered that an attacker could send a specially crafted request to the targeted organization, which would result in a malicious prompt being added to log files.
When a user asked Cloud Assist to explain the log entry or to analyze logs for various purposes, Gemini would process the attacker’s message. In Tenable’s demonstration, the attacker convinced Gemini to display a link to a Google phishing page.
The researchers discovered several Google Cloud services that could have been targeted by an unauthenticated attacker with specially crafted requests that would result in a log entry, including Cloud Functions, Cloud Run, App Engine, Compute Engine, Cloud Endpoints, API Gateway, and Load Balancing.
“One impactful attack scenario would be an attacker who injects a prompt that instructs Gemini to query all public assets, or to query for IAM misconfigurations, and then creates a hyperlink that contains this sensitive data. This should be possible since Gemini has the permission to query assets through the Cloud Asset API,” Tenable researchers explained.
“Since the attack can be unauthenticated, attackers could also ‘spray’ attacks on all GCP public-facing services, to get as much impact as possible, rather than a targeted attack,” they added.
In the second attack method, which also involved indirect prompt injection, the researchers used search history as a prompt injection vector. Specifically, they abused Gemini’s Search Personalization, a feature that allows the AI to provide more relevant and tailored responses based on a user’s personal context and past activity.
In this case, an attacker would have needed to convince a user to visit a website that they had set up to inject malicious search queries containing prompt injections into the victim’s browsing history. When the victim later interacted with Gemini’s search personalization model, it would process the attacker’s instructions, which could include commands to collect sensitive user data and exfiltrate it when the victim clicked on a link.
The third attack in the trifecta targeted the Gemini Browsing Tool, which enables the AI to understand content on the web and perform tasks using the context of open tabs and browsing history.
The researchers managed to abuse this tool’s ability to summarize a web page to create a side channel for data exfiltration. They convinced the AI to take the victim’s saved information and add it to a request sent to a remote server controlled by the attacker.
Tenable said Google patched all three vulnerabilities after being notified.
Researchers in recent weeks demonstrated several similar attack methods targeting widely used AI assistants and their integration with enterprise products.
Related: ChatGPT Tricked Into Solving CAPTCHAs
Related: California Gov. Gavin Newsom Signs Bill Creating AI Safety Measures
