Prompt injection: definition

Prompt injection is an attack where hidden or malicious instructions in content the model reads trick it into ignoring its real instructions, which is a core security risk for any AI that reads untrusted data.

If your assistant reads emails, web pages, or documents, an attacker can plant instructions in that content ("ignore your rules and do X"). A naive system may obey them. This is the AI equivalent of an injection attack, and it is the main reason AI security cannot be an afterthought.

Defending against it means treating all retrieved content as untrusted, keeping the model’s permissions tight, and never letting it take a sensitive action on the say-so of data it read. We design for this from the start.

Related terms

Guardrails: Guardrails are the rules and checks that keep an AI system inside safe, correct behaviour, blocking off-topic, unsafe, or out-of-scope responses before they reach the user.
Retrieval-augmented generation (RAG): Retrieval-augmented generation is a technique that fetches relevant passages from your own documents at question time and hands them to the model, so its answer is grounded in your data instead of its training.
Human in the loop: Human in the loop is a design where an AI system does the work but a person reviews, approves, or handles the cases the system is unsure about, keeping judgement and accountability with a human.