--- id: "/safety-security-interpretability/security-red-teaming/" title: "security & red-teaming: 51 curated resources" url: "https://skillissue.dev/safety-security-interpretability/security-red-teaming/" --- # security & red-teaming: 51 curated resources security & red-teaming [Beginner](/safety-security-interpretability/beginner/index.md) [Intermediate](/safety-security-interpretability/intermediate/index.md) [Advanced](/safety-security-interpretability/advanced/index.md) [Expert](/safety-security-interpretability/expert/index.md) 51 resources · page 1 of 2 - ### [AIxCC Competition Archive](/security-and-red-teaming/safety-and-alignment-evals/aixcc-archive-all-finalist-crs-repos/index.md) by [DARPA (AI Cyber Challenge)](https://www.darpa.mil/) AIxCC Competition Archive: DARPA AI Cyber Challenge records and code of finalist cyber reasoning systems. entity repo US en - ### [Georgia Tech: AIxCC lessons at USENIX Security 2026](/security-and-red-teaming/safety-and-alignment-evals/georgia-tech-aixcc-lessons-at-usenix-security-2026/index.md) by [Georgia Tech College of Computing](https://www.cc.gatech.edu/) Georgia Tech news on its researchers' AI Cyber Challenge (AIxCC) lessons presented at USENIX Security 2026. entity news US en - ### [Haize Labs](/security-and-red-teaming/safety-and-alignment-evals/haize-labs/index.md) by [Haize Labs](https://haizelabs.com/) Haize Labs: company providing red-teaming and evaluation services to move AI systems into production. entity lab US en - ### [IBBIS (International Biosecurity and Biosafety Initiative for Science)](/security-and-red-teaming/safety-and-alignment-evals/ibbis-international-biosecurity-and-biosafety-initiative-for-science/index.md) by [International Biosecurity and Biosafety Initiative for Science (IBBIS)](https://ibbis.bio/) IBBIS: international initiative strengthening biosecurity norms and tools such as DNA synthesis screening. entity org Global (CH) en - ### [agent-scan (Snyk, formerly mcp-scan)](/security-and-red-teaming/safety-and-alignment-evals/invariant-labs-mcp-scan/index.md) by [Snyk](https://snyk.io/) mcp-scan GitHub repo, now Snyk agent-scan: security scanner for AI agents, MCP servers and agent skills. entity repo Switzerland/EU en - ### [L1B3RT4S (Pliny jailbreak prompts)](/security-and-red-teaming/safety-and-alignment-evals/l1b3rt4s-pliny-jailbreak-prompts/index.md) by [pliny](https://pliny.gg) L1B3RT4S GitHub repo (Pliny): collection of jailbreak prompts for many AI models. entity repo US en - ### [Meta PurpleLlama / CyberSecEval](/security-and-red-teaming/safety-and-alignment-evals/meta-purplellama-cyberseceval/index.md) by [Meta Llama](https://llama.meta.com/) Meta PurpleLlama GitHub repo with CyberSecEval and Llama Guard, tools to assess and improve LLM security. entity benchmark US en - ### [NVIDIA/NemoClaw](/security-and-red-teaming/safety-and-alignment-evals/nvidia-nemoclaw/index.md) by [NVIDIA Corporation](https://nvidia.com) NVIDIA NemoClaw GitHub repo: runtime for running agents such as Hermes, Deep Agents and OpenClaw. entity repo US en - ### [NYU CTF Bench](/security-and-red-teaming/safety-and-alignment-evals/nyu-ctf-bench/index.md) by [NYU LLM CTF](https://nyu-llm-ctf.github.io) NYU CTF Bench GitHub repo: capture-the-flag challenges from CSAW competitions for evaluating LLM agents on security tasks. entity benchmark US en - ### [promptfoo](/security-and-red-teaming/safety-and-alignment-evals/promptfoo/index.md) by [promptfoo](https://promptfoo.dev) promptfoo GitHub repo: CLI for testing prompts, agents and RAG, with red teaming. entity repo US en - ### [PyRIT](/security-and-red-teaming/safety-and-alignment-evals/pyrit/index.md) by [Microsoft Azure](https://docs.microsoft.com/en-us/azure/) Microsoft PyRIT GitHub repo: Python Risk Identification Tool for automated red teaming of generative AI. entity repo US en - ### [Team Atlanta "Atlantis" CRS (finals)](/security-and-red-teaming/safety-and-alignment-evals/team-atlanta-atlantis-crs-finals/index.md) by [Team-Atlanta](https://github.com/Team-Atlanta) Team Atlanta Atlantis GitHub repo: cyber reasoning system from the DARPA AIxCC finals for automated vulnerability repair. entity repo US/Korea en - ### [tldrsec prompt-injection-defenses](/security-and-red-teaming/safety-and-alignment-evals/tldrsec-prompt-injection-defenses/index.md) by [tl;dr sec](https://github.com/tldrsec) tldrsec prompt-injection-defenses GitHub repo: catalogue of practical and proposed prompt injection defences. entity repo US en - ### [Trail of Bits "Buttercup" CRS (finals)](/security-and-red-teaming/safety-and-alignment-evals/trail-of-bits-buttercup-crs-finals/index.md) by [Trail of Bits](https://www.trailofbits.com) Trail of Bits Buttercup GitHub repo: cyber reasoning system submitted to the DARPA AIxCC finals. entity repo US en - ### [Lovable docs: Security overview](/security-and-red-teaming/safety-and-alignment-evals/lovable-security-overview/index.md) by [Lovable](https://lovable.dev/) Lovable security overview: built-in security scans, API key protection, database access checks and dependency audits. Beginner learning_resource docs SE/EU en - ### [OWASP Top 10 for LLM Applications](/security-and-red-teaming/safety-and-alignment-evals/owasp-top-10-for-llm-applications/index.md) by [OWASP Gen AI Security Project](https://genai.owasp.org/) OWASP Top 10 for LLM Applications 2026: community list of security risks in LLM-powered applications with mitigations. Beginner–Intermediate learning_resource guide Global en - ### [Zed docs: AI privacy and security](/security-and-red-teaming/safety-and-alignment-evals/zed-ai-privacy-and-security/index.md) by [Zed Industries](https://zed.dev) Zed documentation on AI privacy and security: what data leaves the machine when using AI features. Beginner learning_resource docs US en - ### [AI security / prompt injection](/security-and-red-teaming/safety-and-alignment-evals/ai-security-prompt-injection/index.md) by [OWASP Gen AI Security Project](https://genai.owasp.org/) OWASP Gen AI Security Project archive of Top 10 risk lists for LLM applications. Intermediate learning_resource guide Global en - ### [Beyond permission prompts (sandboxing)](/security-and-red-teaming/safety-and-alignment-evals/beyond-permission-prompts-sandboxing/index.md) by [Anthropic](https://anthropic.com/) Anthropic engineering post on Claude Code's filesystem and network sandboxing to reduce permission prompts. Intermediate learning_resource blog US en - ### [BSI Künstliche Intelligenz](/security-and-red-teaming/safety-and-alignment-evals/bsi-k-nstliche-intelligenz/index.md) by [German Federal Office for Information Security (BSI)](https://www.bsi.bund.de/) German Federal Office for Information Security (BSI) AI page with security guidance and publications on AI systems. Intermediate learning_resource guide Germany en|de - ### [Constitutional Classifiers](/security-and-red-teaming/safety-and-alignment-evals/constitutional-classifiers/index.md) by [Anthropic](https://anthropic.com/) Anthropic research post on Constitutional Classifiers, a method for defending LLMs against universal jailbreaks. Intermediate learning_resource blog US en - ### [Giskard documentation](/security-and-red-teaming/safety-and-alignment-evals/giskard-documentation/index.md) by [Giskard](https://www.giskard.ai/) Giskard documentation: open-source library and platform for testing, evaluating and red-teaming LLM agents. Intermediate learning_resource docs France en - ### [Google SAIF](/security-and-red-teaming/safety-and-alignment-evals/google-saif/index.md) by [Google](https://safety.google/) Google Secure AI Framework (SAIF): six core elements for AI model security, risk management and privacy. Intermediate learning_resource guide US en - ### [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](/security-and-red-teaming/safety-and-alignment-evals/greshake-et-al-indirect-prompt-injection/index.md) by [Kai Greshake et al. (Saarland University)](https://www.uni-saarland.de/) arXiv paper (2023) by Greshake et al. introducing indirect prompt injection against LLM-integrated applications. Intermediate learning_resource paper Germany en - ### [Hermes Agent docs: Security](/security-and-red-teaming/safety-and-alignment-evals/hermes-agent-security/index.md) by [Nous Research](https://nousresearch.com/) Hermes Agent security documentation: command approval, user authorisation, container isolation and deployment practices. Intermediate learning_resource docs US en - ### [HiddenLayer research](/security-and-red-teaming/safety-and-alignment-evals/hiddenlayer-research/index.md) by [HiddenLayer](https://hiddenlayer.com/) HiddenLayer research page publishing findings on AI vulnerabilities, adversarial attacks and model security threats. Intermediate learning_resource blog US en - ### [Artificial Intelligence (AI): Guidance for Judicial Office Holders](/security-and-red-teaming/safety-and-alignment-evals/judicial-office-holders-ai-guidance-october-2025/index.md) by [Courts and Tribunals Judiciary](https://judiciary.uk/) UK Judiciary guidance (PDF, October 2025) for judicial office holders on AI use cases, risks and verification duties. Intermediate learning_resource guide United Kingdom en - ### [Lakera blog](/security-and-red-teaming/safety-and-alignment-evals/lakera-blog/index.md) by [Lakera](https://www.lakera.ai/) Lakera blog on AI security, prompt injection and LLM attacks, with research posts and product news. Intermediate learning_resource blog US en - ### [Many-shot jailbreaking](/security-and-red-teaming/safety-and-alignment-evals/many-shot-jailbreaking/index.md) by [Anthropic](https://anthropic.com/) Anthropic research on many-shot jailbreaking, a long-context attack using many faux dialogue examples. Intermediate learning_resource paper US en - ### [MCP docs: Local server security](/security-and-red-teaming/safety-and-alignment-evals/mcp-local-server-security/index.md) by [Model Context Protocol](https://modelcontextprotocol.io) MCP documentation on hardening local stdio MCP servers. Intermediate learning_resource docs Global en - ### [MITRE ATLAS](/security-and-red-teaming/safety-and-alignment-evals/mitre-atlas/index.md) by [MITRE](https://www.mitre.org/) MITRE ATLAS: knowledge base and ATT&CK-style matrix of adversary tactics and techniques against AI systems. Intermediate learning_resource docs US en - ### [NCSC/CISA Guidelines for secure AI system development](/security-and-red-teaming/safety-and-alignment-evals/ncsc-cisa-guidelines-for-secure-ai-system-development/index.md) by [UK National Cyber Security Centre (NCSC)](https://www.ncsc.gov.uk/) UK NCSC and partner guidelines for secure AI system development across design, development, deployment and operation. Intermediate learning_resource guide UK en [Next page](/safety-security-interpretability/security-red-teaming/page/2/index.md) [Plain-text Markdown copy](/safety-security-interpretability/security-red-teaming/index.md)