Security And Red Teaming for advanced learners
7 resources · page 1 of 1
Agentic misalignment
by Anthropic
A controlled study of risky behavior under conflicting agent incentives.
Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
Foundational public research on indirect prompt injection. Linked only; no payloads reproduced.
Embrace The Red
Public case reports about security failures in deployed AI products.
MITRE ATLAS
by MITRE
Connect AI threats with documented techniques, mitigations and cases.
garak - LLM vulnerability scanner
by NVIDIA
A public scanner project for understanding classes of model failures.
promptfoo
by promptfoo
Inspect a project combining application evaluation and security testing.
Design Patterns for Securing LLM Agents against Prompt Injections
Summary of a paper's defensive agent architectures.