Attacks on AI
Prompt injection: when the AI tool is the target
Prompt injection lets hostile text steer an LLM. The NCSC's advice is to accept some residual risk and design so that a confused LLM cannot do serious harm.
Last reviewed against NCSC (UK), NCSC New Zealand and UK Government (DSIT and Home Office). Sources
The problem
Instructions and data look the same
OWASP lists prompt injection first among the risks for LLM applications in its 2025 edition [8]. The NCSC explains why it is hard. With SQL injection, parameterised queries keep data separate from instructions. An LLM has no comparable built-in separation, and processes text as a sequence of predicted tokens [2].
Four measures
What the NCSC recommends
Awareness
Developers should know prompt injection is a vulnerability class. Security teams should treat it as residual risk that no single product removes [2].
Secure design
Assume the LLM can be confused and rely on deterministic, non-LLM safeguards to constrain what the system can do. An LLM handling email from unknown external senders should not have access to privileged tools [2].
Make it harder
Mark untrusted content so it is separated from instructions. The NCSC cautions against deny-lists of known bad phrases, because attackers can rephrase endlessly [2].
Monitor
Log LLM inputs and outputs, tool use and API calls. Failed tool or API calls may reveal early attempts [2].
In practice
A worked example
Take an assistant that summarises a shared inbox and can also forward messages. An outsider sends an email containing hidden instructions to forward the latest invoices elsewhere. The assistant cannot reliably tell the instruction from the content it was asked to summarise. The NCSC's answer is not a cleverer filter but a design that removes the damage: the assistant should not hold the ability to forward messages while it reads mail from unknown senders [2].
Beyond injection
Other risks on the list
Use securing your own AI tools for a checklist.
Also on the OWASP list
Sensitive information disclosure, supply chain, data and model poisoning, excessive agency and unbounded consumption all appear in the 2025 top ten for LLM applications [8].
Keep reading
Read next
What they are
A plain definition of AI cyber attacks with the three types that matter to UK organisations, each tied to an official source or a documented case.
Read the guide
Are they new?
The NCSC judges AI mostly enhances existing attack methods to 2027. What that means for how UK organisations prioritise their defences.
Read the guide
Your AI tools
A sourced checklist for securing the AI tools your organisation uses: privileges, untrusted content, logging, vendors and the OWASP LLM risks.
Read the guideHow CyPro can help
Need a hand with any of this?
Patching is one job. Keeping a vulnerable system safe until it is patched, and proving it, is several more. CyPro's consultants, testers and SOC analysts can take any of them off your plate.
Vulnerability scanning
Find every internet-facing system you run, including the forgotten ones, and see which are still on a vulnerable build.
See the scanning service
Hardening and segmentation
Our consultants put the controls in place that limit the damage when something is missed: firewall rules, network isolation and tighter admin access.
More on cypro.co.uk
24/7 security monitoring
Our SOC analysts watch your systems day and night for the signs of an attack in progress: unexpected processes, new outbound connections, logins that do not fit.
More on cypro.co.uk
Penetration testing
A focused test of your systems and the controls around them, so you know they hold before an attacker finds out for you.
More on cypro.co.uk
Incident response
If you may already be compromised, our responders contain it, preserve the evidence, find out what was touched and get you running again safely.
More on cypro.co.uk
Fractional CISO for the board
A senior security lead who advises the board on what to fix first, what to accept and what to tell customers, and owns the decision record.
More on cypro.co.uk
Know your exposure
Not sure you are protected?
AI makes it quicker to find and use weak points, so it helps to find them first. A penetration test shows which of your internet-facing systems, staff processes and AI tools an attacker could actually use.