modality evals (image/video/speech): 87 curated resources
modality evals (image/video/speech)
87 resources · page 3 of 3
SongEval
by ASLP-lab
SongEval GitHub repo: song aesthetic evaluation toolkit and dataset.
T2I-CompBench
by Karine_H
T2I-CompBench GitHub repo: benchmark for compositional text-to-image generation such as attribute binding and spatial relations.
T3Bench
by T3Bench team
T3Bench site: benchmark for text-to-3D generation measuring quality and alignment with the prompt.
TTS Arena V2 (Hugging Face Space)
by TTS AGI
TTS Arena V2 Space: community arena voting between anonymous text-to-speech models.
UK AISI Inspect Evals
by Department of Business, Energy and Industrial Strategy
UK AISI Inspect Evals: registry of evaluations implemented in Inspect, including cybersecurity, safety and agent benchmarks.
Vals AI MMMU Pro Leaderboard
by Vals AI
Vals AI MMMU-Pro page: archived multimodal reasoning results and methodology; no longer run on new models.
VBench
by Vchitect
VBench project page: benchmark suite scoring video generation models on separate quality and alignment dimensions.
VBench-2.0
by Vchitect
VBench-2.0 project page: video generation benchmark for intrinsic faithfulness, including physics and commonsense consistency.
VBench Leaderboard (Hugging Face Space)
by Vchitect
VBench leaderboard Space: video generation model scores across VBench dimensions, with self-submission.
VBench / VBench++ / VBench-2.0
by Vchitect
VBench GitHub repo: evaluation suite for video generation models across many quality and consistency dimensions.
MME-Benchmarks/Video-MME
Video-MME GitHub repo: benchmark and evaluation resources for multimodal models' understanding of short to long videos.
Video-MME
by Video-MME
Video-MME project site: benchmark for multimodal models on short, medium and long videos with subtitles and audio.
Video-MME-v2
Video-MME-v2 paper: a video-understanding benchmark with group-based consistency scoring for multimodal models.
VLMEvalKit
by OpenCompass
OpenCompass VLMEvalKit GitHub repo: evaluation toolkit for large vision-language models.
WorldLens
by worldbench
WorldLens GitHub repo (CVPR 2026): evaluations of driving world models across generation and reconstruction.
ZeroBench
by zerobench
ZeroBench project page: visual reasoning benchmark designed so current multimodal models score near zero.
FlagEvalMM
arXiv paper (2025) introducing FlagEvalMM, BAAI's framework for evaluating multimodal models.
MME
by Chaoyou Fu et al. (Nanjing University)
arXiv paper (2023) introducing MME, a benchmark for perception and cognition in multimodal LLMs.
SUPERB
by Shu-wen Yang et al. (National Taiwan University)
arXiv paper (2021) "SUPERB: Speech processing Universal PERformance Benchmark", introducing SUPERB, a benchmark of speech representations across many speech tasks.
3D Arena paper
arXiv paper (2025) introducing 3D Arena, an open pairwise-preference platform for evaluating generative 3D models.
MMMU-Pro paper
by Xiang Yue et al. (Carnegie Mellon University)
arXiv HTML version of the MMMU-Pro paper (2024), a harder multimodal multi-discipline benchmark.
Open ASR Leaderboard paper
by Vaibhav Srivastav et al. (Hugging Face)
arXiv PDF (2025) of the Open ASR Leaderboard paper: evaluation method, datasets and results for speech recognition models.
VBench-2.0 paper
by Dian Zheng et al. (Nanyang Technological University and Shanghai AI Laboratory)
arXiv HTML version of the VBench-2.0 paper (2025) evaluating video generation on physics, commonsense and faithfulness.