Jailbreaking and red teaming
Published jailbreak research, safety benchmarks and defensive evaluation. Credited links and summaries only; no attack payloads.
- Prompt Injection (4)
- Agent And Tool Security (2)
- Threat Frameworks And Standards (3)
- Security Tooling (3)
- Safety And Alignment Evals (126)
- Jailbreak research (2)
140 resources · page 5 of 5
UK education generative-AI product safety standards
Creator not supplied; see original source.
UK Department for Education generative AI product safety standards for AI tools used in education.
Design Patterns for Securing LLM Agents against Prompt Injections
Summary of a paper's defensive agent architectures.
A Mathematical Framework for Transformer Circuits
Creator not supplied; see original source.
Anthropic Transformer Circuits paper A Mathematical Framework for Transformer Circuits (2021).
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
Creator not supplied; see original source.
arXiv paper (2024) introducing AgentDojo, an environment for evaluating prompt injection attacks and defences on tool agents.
AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
Creator not supplied; see original source.
arXiv paper (2024) introducing AgentHarm, a benchmark measuring harmful task completion by LLM agents.
Amnesty: Xenophobic Machines (Dutch childcare benefits)
Creator not supplied; see original source.
Amnesty International report Xenophobic Machines on discriminatory algorithmic profiling in the Dutch childcare benefits scandal.
Defeating Prompt Injections by Design (CaMeL)
Creator not supplied; see original source.
arXiv paper (2025) by Google DeepMind introducing CaMeL, a capability-based design that defeats prompt injections.
Embrace The Red (Johann Rehberger)
Creator not supplied; see original source.
Embrace The Red, Johann Rehberger's blog documenting prompt-injection and data-exfiltration exploits in real AI agents and IDEs.
Emergent Misalignment
Creator not supplied; see original source.
arXiv paper (2025) "Emergent Misalignment", showing narrow fine-tuning on insecure code produces broadly misaligned behaviour.
On the Biology of a Large Language Model
Creator not supplied; see original source.
Anthropic Transformer Circuits paper On the Biology of a Large Language Model: attribution graphs on Claude 3.5 Haiku.
The Instruction Hierarchy
Creator not supplied; see original source.
arXiv paper (2024) by OpenAI introducing the instruction hierarchy to train models to prioritise privileged instructions.
Universal adversarial attacks (GCG)
Creator not supplied; see original source.
arXiv paper (2023) introducing GCG, universal transferable adversarial suffixes that jailbreak aligned LLMs.