Prompt Injection and AI Agents by Toju Duke, a Top AI Expert, Ex-Google

So we now have AI agents all over the place. There's a big, um, excitement and rise in adoption of agentic AI. But in the midst of all of this, they can easily be fooled. As we well know, that LLMs can easily be fooled, and they are artificial intelligence, not human intelligence, and even humans can easily be fooled. And there's a new method called prompt injection, which is one of the cybersecurity attacks that people can make on AI agents. And you don't have to be so tech savvy to be able to do this. And I'm not teaching you how to, to hack an AI agent. The whole point of this is to show the vulnerability of AI agents where they, they're able to leak sensitive information and sensitive data out. And it's such a simple trick. All they just need to do is put a line of command, um, on a website. And if this AI agent has an instruction to carry out some certain tasks for instance, go build a website for me for a marketing event, the malicious actor can just put a line of code on that website that tells the AI agent to release the bank statements from the email of the person who sent it, something along those lines, and the AI agent will do it. It wouldn't recognize that line of command separately from the line of command from the developer that gave it the initial instructions, the person that gave it the initial instructions. They are trained and built to follow commands. So wherever the command comes from, as long as there's a line of code that they're able to crawl and see and read and interpret, they will carry it out just like that, basically. Like robots, like the robots we see in the movie. You tell a robot what to do, it does it. Now, this vulnerability has been shown across so many different areas now, from Microsoft Copilot to OpenAI's, um, latest AI agents. Um, and we also have a recent example of, um, a LinkedIn user who put a line of command on his LinkedIn profile because he wanted to prove that bots are actually crawling his page when he applies for jobs. And he puts a line of command that asks for a flan 'recipe,' and this AI agent produced a flan recipe through the recruiter's email. Now, this might seem funny, but the implications are quite severe. Imagine an AI agent being asked to pull information about your bank statements, about your house address, about your job, and other sensitive information. Or imagine the newly appointed AI agent who's now a minister in Albania's government being asked to release all the sensitive information from the government's systems. This is a problem that hasn't been solved. There are a few ways of handling this. One is by creating safeguards within this AI agents and putting commands there to stop them from doing what they shouldn't do, leaking information. Another way is to add a delimiter to it, so a line of code that ideally should prevent them from obeying a different command. All of these are not foolproof. Malicious actors can still override these, and we need a human in the loop, definitely. You can carry out adversarial testing as well, or you can limit the amount of data that the AI agents have access to. But if you're using AI agents, remember to use them with a pinch of salt, knowing that there are lots of vulnerabilities that still exist in this in this technical infrastructure that have not really been solved.