Safety, security & interpretability for intermediate learners
90 resources · page 3 of 3
HiddenLayer research
by HiddenLayer
HiddenLayer research page publishing findings on AI vulnerabilities, adversarial attacks and model security threats.
How we built Claude Code auto mode
by Anthropic
Anthropic engineering post on how Claude Code auto mode decides when to skip permission prompts safely.
Interpretable Machine Learning
Christoph Molnar's Interpretable Machine Learning book: free online guide to explainability methods and their limits.
Artificial Intelligence (AI): Guidance for Judicial Office Holders
by Courts and Tribunals Judiciary
UK Judiciary guidance (PDF, October 2025) for judicial office holders on AI use cases, risks and verification duties.
Lakera blog
by Lakera
Lakera blog on AI security, prompt injection and LLM attacks, with research posts and product news.
Many-shot jailbreaking
by Anthropic
Anthropic research on many-shot jailbreaking, a long-context attack using many faux dialogue examples.
Statement on CVE-2025-48757 (Matt Palmer)
by Matt Palmer
Matt Palmer's statement on CVE-2025-48757: missing row-level security in Lovable-generated apps exposed user data.
MCP blog: Tool annotations
MCP blog post on tool annotations such as read-only and destructive hints, and how far clients should trust them.
MCP docs: Local server security
MCP documentation on hardening local stdio MCP servers.
MITRE ATLAS
by MITRE
MITRE ATLAS: knowledge base and ATT&CK-style matrix of adversary tactics and techniques against AI systems.
NCSC/CISA Guidelines for secure AI system development
by UK National Cyber Security Centre (NCSC)
UK NCSC and partner guidelines for secure AI system development across design, development, deployment and operation.
How To Become A Mechanistic Interpretability Researcher
by Neel Nanda
Neel Nanda's AI Alignment Forum guide on how to become a mechanistic interpretability researcher, which replaced his earlier getting-started page.
NIST AI 100-2 Adversarial ML taxonomy
by NIST
NIST AI 100-2 report: taxonomy and terminology of adversarial machine learning attacks and mitigations.
OWASP Top 10 for Agentic Applications 2026
by OWASP Gen AI Security Project
OWASP Top 10 for Agentic Applications 2026: list of security risks specific to AI agents.
Patronus AI
by Patronus AI
Patronus AI blog with posts on LLM evaluation, benchmarks and AI reliability research.
From Naptime to Big Sleep: Using Large Language Models To Catch Vulnerabilities In Real-World Code
Google Project Zero post on Big Sleep, an LLM agent that found an exploitable SQLite vulnerability in real code.
promptfoo docs: OWASP LLM Top 10 red teaming
by promptfoo
Promptfoo documentation mapping its red-teaming plugins to the OWASP LLM Top 10 risks.
Red Teaming LMs to Reduce Harms (Ganguli et al.)
by Deep Ganguli et al. (Anthropic)
arXiv paper (2022) by Anthropic on red teaming language models: methods, scaling behaviour and a released attack dataset.
Redwood Research
Redwood Research page listing its research on AI control and keeping AI systems aligned with developer intent.
Retraction Watch: papers with evidence of ChatGPT writing
Retraction Watch running list of published papers and peer reviews containing evidence of ChatGPT-written text.
Sal Khan: Khanmigo's First Chapter Changed How I Think About AI (Khan Academy)
by Khan Academy
Sal Khan's reflection on the first chapter of the Khanmigo AI tutor and what it changed for student learning.
Simon Willison on prompt-injection
Simon Willison's prompt-injection tag: running record of prompt-injection attacks and incidents since 2022.
The lethal trifecta for AI agents (Simon Willison)
Simon Willison's post on the lethal trifecta: private data, untrusted content and exfiltration make AI agents exploitable.
AIxCC finals: Tale of the tape (Trail of Bits)
Trail of Bits analysis comparing the cyber reasoning system designs of DARPA AIxCC finalists.
vLLM docs: Security
by vLLM
vLLM security documentation: guidance on exposing and hardening self-hosted inference servers.
XBOW Blog
by XBOW
XBOW blog with research posts on autonomous AI penetration testing, vulnerability discovery and exploit validation.