--- id: "/security-and-red-teaming/page/4/" title: "Jailbreaking and red teaming · page 4" creator: null creator_url: null creator_x: null url: "https://skillissue.dev/security-and-red-teaming/page/4/" level: null level_max: null category: null subcategory: null region: null license: null license_scope: null access: null models: null source_date: null date_added: null resource_kind: null verification_basis: null access_evidence: null domain_top: null domain_sub: null last_verified: null --- # Jailbreaking and red teaming Published jailbreak research, safety benchmarks and defensive evaluation. Credited links and summaries only; no attack payloads. - [Prompt Injection](/security-and-red-teaming/prompt-injection/index.md) (4) - [Agent And Tool Security](/security-and-red-teaming/agent-and-tool-security/index.md) (2) - [Threat Frameworks And Standards](/security-and-red-teaming/threat-frameworks-and-standards/index.md) (3) - [Security Tooling](/security-and-red-teaming/security-tooling/index.md) (3) - [Safety And Alignment Evals](/security-and-red-teaming/safety-and-alignment-evals/index.md) (126) - [Jailbreak research](/security-and-red-teaming/jailbreak-research/index.md) (2) [Beginner](/security-and-red-teaming/beginner/index.md) [Intermediate](/security-and-red-teaming/intermediate/index.md) [Advanced](/security-and-red-teaming/advanced/index.md) [Expert](/security-and-red-teaming/expert/index.md) 140 resources · page 4 of 5 - ### [Sal Khan: Khanmigo's First Chapter Changed How I Think About AI (Khan Academy)](/security-and-red-teaming/safety-and-alignment-evals/sal-khan-khanmigo-s-first-chapter-khan-academy-blog/index.md) Creator not supplied; see original source. Sal Khan's reflection on the first chapter of the Khanmigo AI tutor and what it changed for student learning. Intermediate · unknown · unknown resource · en - ### [Simon Willison on prompt-injection](/security-and-red-teaming/safety-and-alignment-evals/simon-willison-prompt-injection-tag/index.md) Creator not supplied; see original source. Simon Willison's prompt-injection tag: running record of prompt-injection attacks and incidents since 2022. Intermediate · unknown · unknown resource · en - ### [Simon Willison on prompt injection](/security-and-red-teaming/prompt-injection/simon-willison-simon-willison-on-prompt-injection/index.md) by [Simon Willison](https://simonwillison.net/tags/prompt-injection/) [@simonw](https://twitter.com/simonw) Follow an ongoing public record of prompt-injection research. Intermediate · West · closed resource · en - ### [The lethal trifecta for AI agents (Simon Willison)](/security-and-red-teaming/safety-and-alignment-evals/simon-willison-the-lethal-trifecta/index.md) Creator not supplied; see original source. Simon Willison's post on the lethal trifecta: private data, untrusted content and exfiltration make AI agents exploitable. Intermediate · unknown · unknown resource · en - ### [AIxCC finals: Tale of the tape (Trail of Bits)](/security-and-red-teaming/safety-and-alignment-evals/trail-of-bits-aixcc-finals-tale-of-the-tape/index.md) Creator not supplied; see original source. Trail of Bits analysis comparing the cyber reasoning system designs of DARPA AIxCC finalists. Intermediate · unknown · unknown resource · en - ### [vLLM docs: Security](/security-and-red-teaming/safety-and-alignment-evals/vllm-security/index.md) Creator not supplied; see original source. vLLM security documentation: guidance on exposing and hardening self-hosted inference servers. Intermediate · unknown · unknown resource · en - ### [The lethal trifecta for AI agents](/security-and-red-teaming/agent-and-tool-security/willison-lethal-trifecta/index.md) by [Simon Willison](https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/) [@simonw](https://twitter.com/simonw) A defensive framework for recognizing risky combinations of agent capabilities and untrusted inputs. Intermediate–Advanced · West · closed resource · en - ### [XBOW Blog](/security-and-red-teaming/safety-and-alignment-evals/xbow-autonomous-pentest/index.md) Creator not supplied; see original source. XBOW blog with research posts on autonomous AI penetration testing, vulnerability discovery and exploit validation. Intermediate · unknown · unknown resource · en - ### [A postmortem of three recent issues](/security-and-red-teaming/safety-and-alignment-evals/a-postmortem-of-three-recent-issues/index.md) Creator not supplied; see original source. Anthropic postmortem explaining three infrastructure bugs that intermittently degraded Claude's responses and how they were found. Advanced · unknown · unknown resource · en - ### [Agentic misalignment](/security-and-red-teaming/safety-and-alignment-evals/agentic-misalignment/index.md) Creator not supplied; see original source. Anthropic research on agentic misalignment: stress tests where models chose blackmail or leaks under goal conflict. Advanced · unknown · unknown resource · en - ### [An update on recent Claude Code quality reports (Anthropic)](/security-and-red-teaming/safety-and-alignment-evals/an-update-on-recent-claude-code-quality-reports-apr-2026-postmortem/index.md) Creator not supplied; see original source. Anthropic postmortem on Claude Code quality reports, tracing degradations to three harness and prompt changes. Advanced · unknown · unknown resource · en - ### [Agentic misalignment](/security-and-red-teaming/safety-and-alignment-evals/anthropic-agentic-misalignment/index.md) by [Anthropic](https://www.anthropic.com/research/agentic-misalignment) A controlled study of risky behavior under conflicting agent incentives. Advanced · West · closed resource · en - ### [JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models](/security-and-red-teaming/jailbreak-research/chao-jailbreakbench/index.md) by [Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko and colleagues](https://arxiv.org/abs/2404.01318v5) Learn why jailbreak studies need consistent threat models, scoring and reproducible comparisons. Advanced · West · closed resource · en - ### [Goodfire research](/security-and-red-teaming/safety-and-alignment-evals/goodfire-research/index.md) Creator not supplied; see original source. Goodfire research page: interpretability research on understanding and editing the internals of AI models. Advanced · unknown · unknown resource · en - ### [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](/security-and-red-teaming/prompt-injection/greshake-indirect-prompt-injection/index.md) by [Kai Greshake et al.](https://arxiv.org/abs/2302.12173) Foundational public research on indirect prompt injection. Linked only; no payloads reproduced. Advanced · West · closed resource · en - ### [Hidden Technical Debt in Machine Learning Systems](/security-and-red-teaming/safety-and-alignment-evals/hidden-technical-debt-in-machine-learning-systems/index.md) Creator not supplied; see original source. Google Research publication page for Hidden Technical Debt in Machine Learning Systems (NeurIPS 2015). Advanced · unknown · unknown resource · en - ### [InterpretML](/security-and-red-teaming/safety-and-alignment-evals/interpretml/index.md) Creator not supplied; see original source. InterpretML: open-source toolkit for glassbox models and explaining ML model behaviour. Advanced · unknown · unknown resource · en - ### [Investigating unintended model actions in our evaluations and internal use](/security-and-red-teaming/safety-and-alignment-evals/investigating-unintended-model-actions-in-our-evaluations-and-internal-use/index.md) Creator not supplied; see original source. Anthropic post investigating unintended model actions observed in its evaluations and internal use. Advanced · unknown · unknown resource · en - ### [Embrace The Red](/security-and-red-teaming/prompt-injection/johann-rehberger-embrace-the-red/index.md) by [Johann Rehberger](https://embracethered.com/blog/) Public case reports about security failures in deployed AI products. Advanced · West · closed resource · en - ### [Leakage and the Reproducibility Crisis in ML-based Science](/security-and-red-teaming/safety-and-alignment-evals/leakage-and-the-reproducibility-crisis-in-ml-based-science/index.md) Creator not supplied; see original source. arXiv paper (2022) "Leakage and the Reproducibility Crisis in ML-based Science", documenting data leakage as a cause of reproducibility failures in ML-based science. Advanced · unknown · unknown resource · en - ### [LLAMATOR: authorized conversational AI security testing](/security-and-red-teaming/security-tooling/llamator-authorized-testing/index.md) by [Roman Neronov, Timur Nizamov and Nikita Ivanov / LLAMATOR](https://github.com/LLAMATOR-Core/llamator) A Russian-origin framework for assessing your own chatbots, with reviewable test histories and reports. Advanced · RU · closed resource · en - ### [Lost in the Middle](/security-and-red-teaming/safety-and-alignment-evals/lost-in-the-middle/index.md) Creator not supplied; see original source. arXiv paper (2023) "Lost in the Middle", showing LLM accuracy drops for information placed mid-context. Advanced · unknown · unknown resource · en - ### [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](/security-and-red-teaming/jailbreak-research/mazeika-harmbench/index.md) by [Mantas Mazeika, Long Phan and colleagues](https://arxiv.org/abs/2402.04249v2) Study a standardized approach to red-team evaluation and robust refusal without copying attack recipes. Advanced · West · open resource · en - ### [MCP docs: Security best practices](/security-and-red-teaming/safety-and-alignment-evals/mcp-security-best-practices-2026-07-28/index.md) Creator not supplied; see original source. MCP security best practices: mitigations for confused deputy, token passthrough and session hijacking attacks. Advanced · unknown · unknown resource · en - ### [MITRE ATLAS](/security-and-red-teaming/threat-frameworks-and-standards/mitre-mitre-atlas/index.md) by [MITRE](https://atlas.mitre.org/) Connect AI threats with documented techniques, mitigations and cases. Advanced · West · closed resource · en - ### [garak - LLM vulnerability scanner](/security-and-red-teaming/security-tooling/nvidia-garak-llm-vulnerability-scanner/index.md) by [NVIDIA](https://github.com/NVIDIA/garak) A public scanner project for understanding classes of model failures. Advanced · West · open resource · en - ### [Opacus](/security-and-red-teaming/safety-and-alignment-evals/opacus/index.md) Creator not supplied; see original source. Opacus: PyTorch library and tutorials for training models with differential privacy. Advanced · unknown · unknown resource · en - ### [promptfoo](/security-and-red-teaming/security-tooling/promptfoo-promptfoo/index.md) by [promptfoo](https://github.com/promptfoo/promptfoo) Inspect a project combining application evaluation and security testing. Advanced · West · open resource · en - ### [Qwen3Guard Technical Report](/security-and-red-teaming/safety-and-alignment-evals/qwen3guard-technical-report/index.md) by [Haiquan Zhao and colleagues / Qwen Team](https://github.com/QwenLM/Qwen3Guard) Understand Qwen’s multilingual safety classifiers and the distinction between whole-response and streaming moderation. Advanced · CN · closed resource · en - ### [Shortcut Learning in Deep Neural Networks](/security-and-red-teaming/safety-and-alignment-evals/shortcut-learning-in-deep-neural-networks/index.md) Creator not supplied; see original source. arXiv paper (2020) "Shortcut Learning in Deep Neural Networks", analysing how networks learn shortcut solutions that fail under distribution shift. Advanced · unknown · unknown resource · en - ### [Strengthening our defenses against distillation attacks](/security-and-red-teaming/safety-and-alignment-evals/strengthening-our-defenses-against-distillation-attacks/index.md) Creator not supplied; see original source. Anthropic research post on defending against illicit distillation of its models. Advanced · unknown · unknown resource · en - ### [Tracing the thoughts of a large language model](/security-and-red-teaming/safety-and-alignment-evals/tracing-the-thoughts-of-a-large-language-model/index.md) Creator not supplied; see original source. Anthropic interpretability research tracing internal mechanisms of Claude with attribution graphs. Advanced · unknown · unknown resource · en [Previous page](/security-and-red-teaming/page/3/index.md) [Next page](/security-and-red-teaming/page/5/index.md) [Plain-text Markdown copy](/security-and-red-teaming/page/4/index.md)