--- id: "/safety-security-interpretability/intermediate/page/2/" title: "Safety, security & interpretability for intermediate learners · page 2" url: "https://skillissue.dev/safety-security-interpretability/intermediate/page/2/" --- # Safety, security & interpretability for intermediate learners [Beginner](/safety-security-interpretability/beginner/index.md) [Intermediate](/safety-security-interpretability/intermediate/index.md) [Advanced](/safety-security-interpretability/advanced/index.md) [Expert](/safety-security-interpretability/expert/index.md) 90 resources · page 2 of 3 - ### [NVIDIA/NemoClaw](/security-and-red-teaming/safety-and-alignment-evals/nvidia-nemoclaw/index.md) by [NVIDIA Corporation](https://nvidia.com) NVIDIA NemoClaw GitHub repo: runtime for running agents such as Hermes, Deep Agents and OpenClaw. entity repo US en - ### [NYU CTF Bench](/security-and-red-teaming/safety-and-alignment-evals/nyu-ctf-bench/index.md) by [NYU LLM CTF](https://nyu-llm-ctf.github.io) NYU CTF Bench GitHub repo: capture-the-flag challenges from CSAW competitions for evaluating LLM agents on security tasks. entity benchmark US en - ### [OpenFHE](/security-and-red-teaming/safety-and-alignment-evals/openfhe/index.md) by [OpenFHE](https://openfhe.org/) OpenFHE GitHub repo: open-source library for fully homomorphic encryption. entity repo International en - ### [promptfoo](/security-and-red-teaming/safety-and-alignment-evals/promptfoo/index.md) by [promptfoo](https://promptfoo.dev) promptfoo GitHub repo: CLI for testing prompts, agents and RAG, with red teaming. entity repo US en - ### [PyRIT](/security-and-red-teaming/safety-and-alignment-evals/pyrit/index.md) by [Microsoft Azure](https://docs.microsoft.com/en-us/azure/) Microsoft PyRIT GitHub repo: Python Risk Identification Tool for automated red teaming of generative AI. entity repo US en - ### [PySyft](/security-and-red-teaming/safety-and-alignment-evals/pysyft/index.md) by [OpenMined](https://openmined.org) OpenMined PySyft GitHub repo: data science on data that stays on the owner's server, for privacy-preserving access. entity repo Global en - ### [SAELens](/security-and-red-teaming/safety-and-alignment-evals/saelens/index.md) by [Decode Research](https://decoderesearch.org) SAELens GitHub repo: training and analysing sparse autoencoders on language models. entity repo Global en - ### [Safe Superintelligence Inc.](/security-and-red-teaming/safety-and-alignment-evals/safe-superintelligence-inc/index.md) by [Safe Superintelligence Inc.](https://ssi.inc/) Safe Superintelligence Inc. (SSI): company focused solely on building safe superintelligence; site has a short mission statement. entity org US/Israel en - ### [STAT: IBM Watson recommended "unsafe and incorrect" cancer treatments](/security-and-red-teaming/safety-and-alignment-evals/stat-ibm-watson-recommended-unsafe-and-incorrect-cancer-treatments/index.md) by [STAT](https://statnews.com/) STAT report on internal IBM documents showing Watson for Oncology gave unsafe and incorrect treatment advice. entity news US en - ### [Stephenson Harwood: "Law and error"](/security-and-red-teaming/safety-and-alignment-evals/stephenson-harwood-law-and-error/index.md) by [Stephenson Harwood](https://stephensonharwood.com/) Stephenson Harwood article on court scrutiny of AI tools after fictitious citations in legal filings. entity standard-policy UK en - ### [Team Atlanta "Atlantis" CRS (finals)](/security-and-red-teaming/safety-and-alignment-evals/team-atlanta-atlantis-crs-finals/index.md) by [Team-Atlanta](https://github.com/Team-Atlanta) Team Atlanta Atlantis GitHub repo: cyber reasoning system from the DARPA AIxCC finals for automated vulnerability repair. entity repo US/Korea en - ### [The Register: Figma AI accused of plagiarising Apple](/security-and-red-teaming/safety-and-alignment-evals/the-register-figma-ai-accused-of-plagiarising-apple/index.md) by [The Register](https://www.theregister.com/) The Register report on Figma pulling its Make Designs AI feature after it copied Apple's Weather app. entity news UK en - ### [TILOS MacroPlacement](/security-and-red-teaming/safety-and-alignment-evals/tilos-macroplacement/index.md) by [TILOS-AI-Institute](https://github.com/TILOS-AI-Institute) TILOS MacroPlacement GitHub repo: benchmarks, evaluators and an open reproduction of Google's Circuit Training for chip macro placement. entity company US en - ### [tldrsec prompt-injection-defenses](/security-and-red-teaming/safety-and-alignment-evals/tldrsec-prompt-injection-defenses/index.md) by [tl;dr sec](https://github.com/tldrsec) tldrsec prompt-injection-defenses GitHub repo: catalogue of practical and proposed prompt injection defences. entity repo US en - ### [Trail of Bits "Buttercup" CRS (finals)](/security-and-red-teaming/safety-and-alignment-evals/trail-of-bits-buttercup-crs-finals/index.md) by [Trail of Bits](https://www.trailofbits.com) Trail of Bits Buttercup GitHub repo: cyber reasoning system submitted to the DARPA AIxCC finals. entity repo US en - ### [TransformerLens](/security-and-red-teaming/safety-and-alignment-evals/transformerlens/index.md) by [TransformerLensOrg](https://github.com/TransformerLensOrg) TransformerLens GitHub repo: library for mechanistic interpretability of GPT-style language models. entity repo US en - ### [Transluce](/security-and-red-teaming/safety-and-alignment-evals/transluce/index.md) by [Transluce](https://transluce.org/) Transluce: nonprofit lab building open tools for AI oversight and interpretability. entity lab US en - ### [Waymo Safety Impact hub](/security-and-red-teaming/safety-and-alignment-evals/waymo-safety-impact-hub/index.md) by [Waymo](https://waymo.com/) Waymo Safety Impact hub: company-published crash-rate comparisons between rider-only driving miles and human benchmarks. entity dataset US en - ### [Anthropic Transparency Hub](/security-and-red-teaming/safety-and-alignment-evals/anthropic-transparency-hub/index.md) by [Anthropic](https://anthropic.com/) Anthropic Transparency Hub: system cards, model reports and voluntary commitments. Beginner–Expert learning_resource paper US en - ### [OpenAI Deployment Safety Hub](/security-and-red-teaming/safety-and-alignment-evals/openai-deployment-safety-hub-system-cards/index.md) by [OpenAI](https://openai.com/) OpenAI Deployment Safety Hub: system cards with safety evaluations and safeguards for deployed OpenAI models. Beginner–Expert learning_resource paper US en - ### [OWASP Top 10 for LLM Applications](/security-and-red-teaming/safety-and-alignment-evals/owasp-top-10-for-llm-applications/index.md) by [OWASP Gen AI Security Project](https://genai.owasp.org/) OWASP Top 10 for LLM Applications 2026: community list of security risks in LLM-powered applications with mitigations. Beginner–Intermediate learning_resource guide Global en - ### [Final report of the Royal Commission into the Robodebt Scheme (Prime Minister of Australia)](/security-and-red-teaming/safety-and-alignment-evals/royal-commission-into-the-robodebt-scheme-final-report-pm-media-release/index.md) by [Prime Minister of Australia](https://www.pm.gov.au/) Australian Prime Minister's media release on the final report of the Royal Commission into the Robodebt automated debt scheme. Beginner–Expert learning_resource guide Australia en - ### [AI Safety Governance Framework v1.0 (TC260) on ETO AGORA](/security-and-red-teaming/safety-and-alignment-evals/ai-safety-governance-framework-v1-0-tc260-agora-record/index.md) by [Emerging Technology Observatory (CSET, Georgetown University)](https://eto.tech/) ETO AGORA record of China's AI Safety Governance Framework v1.0 published by TC260 in 2024. Intermediate learning_resource guide US (record of CN doc) en - ### [AI security / prompt injection](/security-and-red-teaming/safety-and-alignment-evals/ai-security-prompt-injection/index.md) by [OWASP Gen AI Security Project](https://genai.owasp.org/) OWASP Gen AI Security Project archive of Top 10 risk lists for LLM applications. Intermediate learning_resource guide Global en - ### [Apollo Research: Science](/security-and-red-teaming/safety-and-alignment-evals/apollo-research/index.md) by [Apollo Research](https://apolloresearch.ai/) Apollo Research science page listing its research and pre-deployment evaluations on AI scheming and mitigations. Intermediate–Advanced learning_resource docs UK en - ### [Beyond permission prompts (sandboxing)](/security-and-red-teaming/safety-and-alignment-evals/beyond-permission-prompts-sandboxing/index.md) by [Anthropic](https://anthropic.com/) Anthropic engineering post on Claude Code's filesystem and network sandboxing to reduce permission prompts. Intermediate learning_resource blog US en - ### [BSI Künstliche Intelligenz](/security-and-red-teaming/safety-and-alignment-evals/bsi-k-nstliche-intelligenz/index.md) by [German Federal Office for Information Security (BSI)](https://www.bsi.bund.de/) German Federal Office for Information Security (BSI) AI page with security guidance and publications on AI systems. Intermediate learning_resource guide Germany en|de - ### [Constitutional Classifiers](/security-and-red-teaming/safety-and-alignment-evals/constitutional-classifiers/index.md) by [Anthropic](https://anthropic.com/) Anthropic research post on Constitutional Classifiers, a method for defending LLMs against universal jailbreaks. Intermediate learning_resource blog US en - ### [Giskard documentation](/security-and-red-teaming/safety-and-alignment-evals/giskard-documentation/index.md) by [Giskard](https://www.giskard.ai/) Giskard documentation: open-source library and platform for testing, evaluating and red-teaming LLM agents. Intermediate learning_resource docs France en - ### [Google SAIF](/security-and-red-teaming/safety-and-alignment-evals/google-saif/index.md) by [Google](https://safety.google/) Google Secure AI Framework (SAIF): six core elements for AI model security, risk management and privacy. Intermediate learning_resource guide US en - ### [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](/security-and-red-teaming/safety-and-alignment-evals/greshake-et-al-indirect-prompt-injection/index.md) by [Kai Greshake et al. (Saarland University)](https://www.uni-saarland.de/) arXiv paper (2023) by Greshake et al. introducing indirect prompt injection against LLM-integrated applications. Intermediate learning_resource paper Germany en - ### [Hermes Agent docs: Security](/security-and-red-teaming/safety-and-alignment-evals/hermes-agent-security/index.md) by [Nous Research](https://nousresearch.com/) Hermes Agent security documentation: command approval, user authorisation, container isolation and deployment practices. Intermediate learning_resource docs US en [Previous page](/safety-security-interpretability/intermediate/index.md) [Next page](/safety-security-interpretability/intermediate/page/3/index.md) [Plain-text Markdown copy](/safety-security-interpretability/intermediate/page/2/index.md)