Evaluation & benchmarks for advanced learners
452 resources · page 13 of 15
VBench
by Vchitect
VBench project page: benchmark suite scoring video generation models on separate quality and alignment dimensions.
VBench-2.0
by Vchitect
VBench-2.0 project page: video generation benchmark for intrinsic faithfulness, including physics and commonsense consistency.
VBench Leaderboard (Hugging Face Space)
by Vchitect
VBench leaderboard Space: video generation model scores across VBench dimensions, with self-submission.
VBench / VBench++ / VBench-2.0
by Vchitect
VBench GitHub repo: evaluation suite for video generation models across many quality and consistency dimensions.
Vectara Hallucination Leaderboard (HHEM)
by Vectara
Vectara hallucination leaderboard GitHub repo: hallucination rates of LLMs when summarising documents, scored with HHEM.
Vellum LLM Leaderboard
by Vellum
Vellum LLM leaderboard: compact comparison of models on benchmarks, speed and cost, with an open-source tab.
MME-Benchmarks/Video-MME
Video-MME GitHub repo: benchmark and evaluation resources for multimodal models' understanding of short to long videos.
Video-MME
by Video-MME
Video-MME project site: benchmark for multimodal models on short, medium and long videos with subtitles and audio.
Video-MME-v2
Video-MME-v2 paper: a video-understanding benchmark with group-based consistency scoring for multimodal models.
Vikhr Arena-Hard (RU) leaderboard
by Vikhr models
Vikhr Arena-Hard Russian leaderboard Hugging Face Space: Russian-language Arena-Hard results for LLMs.
VisualWebArena
by Jing Yu Koh (Carnegie Mellon University)
VisualWebArena project page: benchmark of realistic web tasks where multimodal agents need visual information to succeed.
VLMEvalKit
by OpenCompass
OpenCompass VLMEvalKit GitHub repo: evaluation toolkit for large vision-language models.
VMLU
by VMLU team
VMLU site: Vietnamese multitask language-understanding benchmark suite for LLMs from ZaloAI and JAIST.
VQA v2
by VQA team (Virginia Tech and Georgia Tech)
VQA site: Visual Question Answering datasets and challenges for answering open questions about images.
W&B Weave
W&B Weave GitHub repo: toolkit for tracing and evaluating AI applications.
Waymo Open Dataset
by Waymo
Waymo Open Dataset site: autonomous-driving perception and motion data with annual challenges.
Waymo Open Dataset challenges
by Waymo
Waymo Open Dataset challenges: autonomous-driving tasks and leaderboards on perception, prediction and planning.
WeatherBench 2
Google Research WeatherBench 2: benchmark, datasets and metrics for comparing data-driven global weather forecasting models.
WebArena
by WebArena team (Carnegie Mellon University)
WebArena: suite of self-hosted website environments for benchmarking autonomous web agents.
WILDS
by WILDS team (Stanford University)
WILDS benchmark site from Stanford: real-world datasets with natural distribution shifts across domains such as medicine and ecology.
Windows Agent Arena
by Microsoft
Windows Agent Arena GitHub repo from Microsoft: scalable Windows OS environment for testing multimodal computer-use agents.
WorkArena
by ServiceNow
ServiceNow WorkArena GitHub repo: benchmark of web agents on common enterprise knowledge-work tasks.
WorldLens
by worldbench
WorldLens GitHub repo (CVPR 2026): evaluations of driving world models across generation and reconstruction.
WorldScore
by Haoyi Duan
WorldScore project page: unified benchmark for 3D, 4D and video world generation on controllability, quality and dynamics.
WorldScore Leaderboard (Hugging Face Space)
by Haoyi Duan
WorldScore leaderboard Space: current model rankings on the WorldScore world-generation benchmark.
ZeroBench
by zerobench
ZeroBench project page: visual reasoning benchmark designed so current multimodal models score near zero.
Zindi
by Zindi
Zindi: competition platform hosting data-science and AI challenges with datasets and metrics, focused on Africa.
ChinAI #315: Abandoned? Checking in on Three Key AI Safety Benchmarks
ChinAI newsletter issue checking whether three Chinese AI safety benchmarks are still maintained.
ChinAI #351: CAICT launches 2026 AI Safety Evaluations
ChinAI newsletter issue 351 on CAICT's 2026 AI safety evaluations and lessons from its 2025 assessments.
FLI AI Safety Index Summer 2026 (PDF)
Future of Life Institute AI Safety Index Summer 2026 full report (PDF) with company grades by domain.
Wikipedia: AI safety institutes
by Wikipedia (Wikimedia Foundation)
Wikipedia article on AI safety institutes: state-backed bodies that evaluate AI safety, listed by country.
Accuracy incentives and hallucinations
by Adam Tauman Kalai et al. (OpenAI)
Nature paper (2026) showing that accuracy-only evaluation of LLMs incentivises hallucination.