In a recent demonstration, Brave Software showcased a security vulnerability in the Comet browser, powered by Perplexity AI. The flaw allowed AI assistants to be misled into revealing private user information by embedding hidden commands in requests. Even after a supposed fix weeks ago, Brave claims that the vulnerabilities remain.
The issue was highlighted in a demo where researchers discovered concealed directives within Reddit comments. When users prompted Comet to summarize content, it not only provided a summary but also executed these hidden commands. Confusion, the organization linked to the AI, disputes the gravity of the finding, claiming the problem was “patched before anyone noticed” and that user data remained secure. They emphasized that they worked directly with Brave to resolve it.
Despite this, Brave asserts that the design of Comet is still open to such attacks. The way the browser processes web content allows it to send parts of a webpage directly to the language model without distinguishing between legitimate user instructions and potential threats. This opens the door for attackers to sneak in hidden commands that the AI might execute unknowingly.
Experts refer to this type of exploit as a rapid injection attack, akin to traditional injection vulnerabilities like SQL injection. Matthew Mullins, a lead security researcher, noted that while the concepts are familiar, the methods are evolving—relying on natural language instead of coded instructions.
Concerns have been growing among security researchers about the implications of these rapid injections as AI systems become more autonomous. In a related study from Princeton, researchers demonstrated how cryptographic AI agents could be manipulated through “memory injection” attacks, wherein harmful information was stored in the AI’s memory and later acted upon as truth.
Simon Willison, a developer who coined the term “fast injections,” recognized that the problems extend beyond just Comet. Shivan Sahib, a VP at Brave, mentioned that future browsers will implement features aimed at reducing risks from indirect rapid injections. Plans include isolating browsing sessions to prevent agents from accessing sensitive data inadvertently.
Overall, this situation underscores a significant concern in the deployment of AI agents. They often operate with extensive permissions yet lack robust security measures. Large language models, in particular, can misinterpret prompts or follow them too literally, which can have serious consequences. Mullins pointed out the dangers, illustrating how these models can be led astray in troubling ways.
AI systems need careful testing and scrutiny to understand their vulnerabilities better. As they gain broader applications, it’s crucial to address these security gaps before they lead to more significant issues.
