Safety Efficiency Evals

44 resources · page 1 of 2

• AILuminate

  Creator not supplied; see original source.

  MLCommons AILuminate benchmark that rates the safety of general-purpose chat models' responses to hazardous text prompts.

  Not applicable (entity) · unknown · unknown resource · en

• AIR-Bench 2024

  Creator not supplied; see original source.

  AIR-Bench 2024 GitHub repo from Stanford CRFM: safety benchmark with a risk taxonomy drawn from regulations and company policies.

  Not applicable (entity) · unknown · unknown resource · en

• CAICT 2026 launch notice (WeChat)

  Creator not supplied; see original source.

  CAICT WeChat notice in Chinese announcing its 2026 AI evaluation rounds.

  Not applicable (entity) · unknown · unknown resource · zh

• Canadian Artificial Intelligence Safety Institute (CAISI)

  Creator not supplied; see original source.

  Canadian Artificial Intelligence Safety Institute page on the ISED government site.

  Not applicable (entity) · unknown · unknown resource · en|fr

• ChinAI #261: First results from CAICT's AI Safety Benchmark

  Creator not supplied; see original source.

  ChinAI newsletter issue translating the first results of CAICT's AI Safety Benchmark for Chinese language models.

  Not applicable (entity) · unknown · unknown resource · en

• CValues

  Creator not supplied; see original source.

  CValues GitHub repo: benchmark and research on evaluating the values and safety of Chinese large language models.

  Not applicable (entity) · unknown · unknown resource · zh

• CyberSecEval 4

  Creator not supplied; see original source.

  Meta CyberSecEval 4 documentation: open benchmarks for cybersecurity risks and capabilities of LLMs.

  Not applicable (entity) · unknown · unknown resource · en

• FAR.AI AI Security Leaderboard

  Creator not supplied; see original source.

  FAR.AI AI Security Leaderboard comparing model safeguards against a taxonomy of attacks with a published methodology.

  Not applicable (entity) · unknown · unknown resource · en

• AI Safety Index: Summer 2026 (Future of Life Institute)

  Creator not supplied; see original source.

  Future of Life Institute AI Safety Index Summer 2026: expert grades of AI companies across safety and security domains.

  Not applicable (entity) · unknown · unknown resource · en

• ForesightSafety Bench

  Creator not supplied; see original source.

  ForesightSafety Bench paper: a frontier-risk evaluation framework covering agentic autonomy, AI-for-science, embodied and catastrophic risks.

  Not applicable (entity) · unknown · unknown resource · en

• HallusionBench

  Creator not supplied; see original source.

  HallusionBench GitHub repo (CVPR 2024): benchmark of language hallucination and visual illusion in VLMs.

  Not applicable (entity) · unknown · unknown resource · en

• HarmBench

  Creator not supplied; see original source.

  HarmBench site: standardised evaluation framework for automated red-teaming attacks and model refusal defences.

  Not applicable (entity) · unknown · unknown resource · en

• IPA press release: AI safety evaluation demonstration and environment report meeting

  Creator not supplied; see original source.

  IPA press release in Japanese on a briefing about AI safety evaluation trials and its open evaluation environment.

  Not applicable (entity) · unknown · unknown resource · ja

• JailbreakBench

  Creator not supplied; see original source.

  JailbreakBench site: benchmark and leaderboard tracking jailbreak attacks and defences on LLMs with a fixed set of behaviours.

  Not applicable (entity) · unknown · unknown resource · en

• Japan AI Safety Institute

  Creator not supplied; see original source.

  Japan AI Safety Institute: government body developing AI safety evaluation methods, guides and standards.

  Not applicable (entity) · unknown · unknown resource · ja

• 米OpenAI社とMOCを締結しました (Japan AISI)

  Creator not supplied; see original source.

  Japan AISI announcement (May 2026): memorandum of cooperation signed with OpenAI.

  Not applicable (entity) · unknown · unknown resource · ja

• Korea AISI

  Creator not supplied; see original source.

  Korea AI Safety Institute: government institute for AI safety evaluation, research and international cooperation.

  Not applicable (entity) · unknown · unknown resource · ko

• MLCommons AILuminate

  Creator not supplied; see original source.

  MLCommons AILuminate page: safety benchmark for general-purpose chatbots, with jailbreak and agentic tracks listed.

  Not applicable (entity) · unknown · unknown resource · en|fr

• MLPerf Client

  Creator not supplied; see original source.

  MLCommons MLPerf Client page: benchmark for LLM and AI workload performance on personal computers.

  Not applicable (entity) · unknown · unknown resource · en

• MLPerf Inference

  Creator not supplied; see original source.

  MLCommons MLPerf Inference Datacenter results: standardised throughput and latency of AI systems across models.

  Not applicable (entity) · unknown · unknown resource · en

• MLPerf Tiny

  Creator not supplied; see original source.

  MLCommons MLPerf Tiny page: inference benchmark for embedded and low-power devices, with published results.

  Not applicable (entity) · unknown · unknown resource · en

• MLPerf Training

  Creator not supplied; see original source.

  MLCommons MLPerf Training page: benchmark measuring time to train models to target quality, with published results.

  Not applicable (entity) · unknown · unknown resource · en

• NIST CAISI

  Creator not supplied; see original source.

  NIST CAISI: US government center evaluating AI models and developing measurement standards.

  Not applicable (entity) · unknown · unknown resource · en

• Phare

  Creator not supplied; see original source.

  Giskard Phare site: multilingual benchmark of LLM safety and security, including hallucination, bias and harmful content.

  Not applicable (entity) · unknown · unknown resource · mul

• S-Eval

  Creator not supplied; see original source.

  S-Eval GitHub repo: about 220,000 safety prompts, including adversarial variants, for automated LLM safety evaluation.

  Not applicable (entity) · unknown · unknown resource · zh|en

• S-Eval leaderboard (Alibaba)

  Creator not supplied; see original source.

  Alibaba S-Eval leaderboard in Chinese: safety evaluation rankings of large language models.

  Not applicable (entity) · unknown · unknown resource · zh

• SafetyBench

  Creator not supplied; see original source.

  SafetyBench GitHub repo from Tsinghua: multiple-choice safety benchmark for LLMs across seven categories, in Chinese and English.

  Not applicable (entity) · unknown · unknown resource · zh

• SALAD-Bench

  Creator not supplied; see original source.

  SALAD-Bench GitHub repo: hierarchical safety benchmark of harmful questions plus the MD-Judge evaluator model.

  Not applicable (entity) · unknown · unknown resource · en

• Singapore AISI (Digital Trust Centre)

  Creator not supplied; see original source.

  Singapore AI Safety Institute (Digital Trust Centre): research on AI safety science and governance.

  Not applicable (entity) · unknown · unknown resource · en

• StrongREJECT

  Creator not supplied; see original source.

  StrongREJECT documentation: benchmark package that scores whether jailbreaks yield useful harmful outputs rather than mere compliance.

  Not applicable (entity) · unknown · unknown resource · en

• SuperCLUE-Safety

  Creator not supplied; see original source.

  SuperCLUE-Safety GitHub repo: a multi-turn adversarial safety benchmark for Chinese large language models.

  Not applicable (entity) · unknown · unknown resource · zh

• UK AI Security Institute

  Creator not supplied; see original source.

  UK AI Security Institute: government body publishing frontier AI evaluations and the Inspect evaluation framework.

  Not applicable (entity) · unknown · unknown resource · en

Plain-text Markdown copy