safety & efficiency evals: 44 curated resources
safety & efficiency evals
44 resources · page 1 of 2
AILuminate
by MLCommons
MLCommons AILuminate benchmark that rates the safety of general-purpose chat models' responses to hazardous text prompts.
AIR-Bench 2024
by Stanford Center for Research on Foundation Models
AIR-Bench 2024 GitHub repo from Stanford CRFM: safety benchmark with a risk taxonomy drawn from regulations and company policies.
CAICT 2026 launch notice (WeChat)
by China Academy of Information and Communications Technology (CAICT)
CAICT WeChat notice in Chinese announcing its 2026 AI evaluation rounds.
Canadian Artificial Intelligence Safety Institute (CAISI)
by Government of Canada (ISED)
Canadian Artificial Intelligence Safety Institute page on the ISED government site.
ChinAI #261: First results from CAICT's AI Safety Benchmark
ChinAI newsletter issue translating the first results of CAICT's AI Safety Benchmark for Chinese language models.
CValues
by X-PLUG
CValues GitHub repo: benchmark and research on evaluating the values and safety of Chinese large language models.
CyberSecEval 4
by Meta Llama
Meta CyberSecEval 4 documentation: open benchmarks for cybersecurity risks and capabilities of LLMs.
FAR.AI AI Security Leaderboard
by FAR.AI
FAR.AI AI Security Leaderboard comparing model safeguards against a taxonomy of attacks with a published methodology.
AI Safety Index: Summer 2026 (Future of Life Institute)
Future of Life Institute AI Safety Index Summer 2026: expert grades of AI companies across safety and security domains.
ForesightSafety Bench
ForesightSafety Bench paper: a frontier-risk evaluation framework covering agentic autonomy, AI-for-science, embodied and catastrophic risks.
HallusionBench
by Tianyi Lab
HallusionBench GitHub repo (CVPR 2024): benchmark of language hallucination and visual illusion in VLMs.
HarmBench
by Center for AI Safety (HarmBench)
HarmBench site: standardised evaluation framework for automated red-teaming attacks and model refusal defences.
IPA press release: AI safety evaluation demonstration and environment report meeting
by Information-technology Promotion Agency, Japan (IPA)
IPA press release in Japanese on a briefing about AI safety evaluation trials and its open evaluation environment.
JailbreakBench
JailbreakBench site: benchmark and leaderboard tracking jailbreak attacks and defences on LLMs with a fixed set of behaviours.
Japan AI Safety Institute
Japan AI Safety Institute: government body developing AI safety evaluation methods, guides and standards.
米OpenAI社とMOCを締結しました (Japan AISI)
Japan AISI announcement (May 2026): memorandum of cooperation signed with OpenAI.
Korea AISI
by Korea AISI
Korea AI Safety Institute: government institute for AI safety evaluation, research and international cooperation.
MLCommons AILuminate
by MLCommons
MLCommons AILuminate page: safety benchmark for general-purpose chatbots, with jailbreak and agentic tracks listed.
MLPerf Client
by MLCommons
MLCommons MLPerf Client page: benchmark for LLM and AI workload performance on personal computers.
MLPerf Inference
by MLCommons
MLCommons MLPerf Inference Datacenter results: standardised throughput and latency of AI systems across models.
MLPerf Tiny
by MLCommons
MLCommons MLPerf Tiny page: inference benchmark for embedded and low-power devices, with published results.
MLPerf Training
by MLCommons
MLCommons MLPerf Training page: benchmark measuring time to train models to target quality, with published results.
NIST CAISI
by NIST
NIST CAISI: US government center evaluating AI models and developing measurement standards.
Phare
by Giskard
Giskard Phare site: multilingual benchmark of LLM safety and security, including hallucination, bias and harmful content.
S-Eval
by IS2Lab
S-Eval GitHub repo: about 220,000 safety prompts, including adversarial variants, for automated LLM safety evaluation.
S-Eval leaderboard (Alibaba)
Alibaba S-Eval leaderboard in Chinese: safety evaluation rankings of large language models.
SafetyBench
by thu-coai
SafetyBench GitHub repo from Tsinghua: multiple-choice safety benchmark for LLMs across seven categories, in Chinese and English.
SALAD-Bench
SALAD-Bench GitHub repo: hierarchical safety benchmark of harmful questions plus the MD-Judge evaluator model.
Singapore AISI (Digital Trust Centre)
by Singapore AI Safety Institute
Singapore AI Safety Institute (Digital Trust Centre): research on AI safety science and governance.
StrongREJECT
by StrongREJECT team (UC Berkeley)
StrongREJECT documentation: benchmark package that scores whether jailbreaks yield useful harmful outputs rather than mere compliance.
SuperCLUE-Safety
SuperCLUE-Safety GitHub repo: a multi-turn adversarial safety benchmark for Chinese large language models.
UK AI Security Institute
UK AI Security Institute: government body publishing frontier AI evaluations and the Inspect evaluation framework.