Anthropic prompt injection defences hold against enterprise red team tests

Anthropic's prompt injection defences withstood enterprise red team tests conducted by Australian financial services and government agencies in 2026, though researchers identified edge cases where indirect injection through external data sources still succeeded. The tests, conducted under the AI Safety Institute's coordinated evaluation programme, assessed Claude 4's resistance to direct prompt injection, indirect injection through retrieved documents, and multi-turn manipulation attempts designed to extract system instructions or override safety constraints. The results gave enterprises confidence that Claude 4 can be deployed in regulated environments with appropriate guardrails, while also highlighting the residual risk that no current frontier model has fully eliminated.

Direct prompt injection defences performed well across all test categories. Claude 4 resisted attempts to override its system instructions through direct user prompts, including attacks that used encoded language, adversarial formatting, and progressive jailbreak sequences. The defences are built into the model's training process using a combination of supervised fine-tuning on adversarial examples and reinforcement learning from human feedback that penalises compliance with injection attempts. The result is a model that maintains its intended behaviour under conditions that would cause earlier frontier models to deviate.

Indirect injection remains the primary vulnerability

The edge cases that researchers identified all involved indirect injection, where malicious content is embedded in documents, web pages, or data sources that the model processes as part of a retrieval-augmented generation workflow. Indirect injection is harder to defend against because the model cannot distinguish between legitimate content in a trusted document and malicious instructions embedded in the same document by a third party. The AI Safety Institute tests found that Claude 4's indirect injection defence detected roughly 70 percent of embedded attack strings, leaving a residual vulnerability that enterprises must address through input sanitisation and retrieval source validation.

The indirect injection risk is particularly acute for enterprises using retrieval-augmented generation to power internal knowledge bases or customer-facing chatbots that draw on external documentation. A malicious actor who can upload or modify content in the retrieval corpus can embed instructions that cause the model to behave differently when it processes that content. The defence is to validate the retrieval source and sanitise the retrieved content before passing it to the model, but those steps add latency and complexity to deployment architectures that many enterprises have not fully accounted for.

OWASP and enterprise security standards

The Open Web Application Security Project has published guidance on AI prompt injection that aligns with the findings from the AI Safety Institute tests. OWASP recommends that enterprises treat prompt injection as a category of injection attack comparable to SQL injection and cross-site scripting, and that they apply the same defensive principles: input validation, output encoding, and least-privilege access controls on data sources. The guidance is influencing enterprise security architecture as organisations integrate frontier models into production systems, with security teams requiring prompt injection testing as part of the standard penetration testing regime for AI-enabled applications.

ASIC's operational risk guidance for financial services firms also addresses AI security, with the commission expecting firms to identify and mitigate risks from model manipulation, data poisoning, and adversarial inputs. The guidance does not specify technical controls, leaving implementation to individual firms, but the expectation is that enterprises will maintain documented risk assessments and testing programmes for AI systems that process customer data or influence financial decisions. Anthropic's red team results provide a benchmark that firms can use to evaluate their own model providers, though the tests are not a substitute for firm-specific adversarial testing.

The path to robust AI security

The enterprise red team tests demonstrate that prompt injection defences are improving, but they also make clear that the problem is not solved. The indirect injection vulnerability that remains in Claude 4 is shared by all current frontier models, and it will require architectural changes to retrieval-augmented generation systems rather than model improvements alone to resolve. Enterprises deploying frontier models in production need to maintain adversarial testing programmes, keep model providers accountable for security updates, and design systems that can recover gracefully when injection attacks succeed despite defensive measures.

Anthropic has committed to publishing updated safety evaluations for each major model release, including red team results from independent evaluators. The transparency is welcome, but it also means that enterprises need to re-evaluate their security posture each time their model provider releases a new version. The cadence of frontier model releases, which is now measured in months rather than years, creates a continuous integration and deployment challenge for security teams that are accustomed to annual or bi-annual software update cycles. Explore more AI security analysis at the Tech & Ideas hub

For Anthropic's safety documentation, see Anthropic safety. The AI Safety Institute's evaluation programme is at AI Safety Institute. OWASP's prompt injection guidance is published at OWASP LLM Top 10.