modality evals (image/video/speech): 87 curated resources
modality evals (image/video/speech)
87 resources · page 1 of 3
3D Arena (HF Space)
by 3D Arena
3D Arena Space: voting arena and leaderboard for AI-generated 3D models.
3DGen-Arena (3DTopia)
by 3DTopia
3DTopia 3DGen-Arena Space: arena comparing open-source text- and image-to-3D generation models.
ActivityNet
by ActivityNet
ActivityNet GitHub repo: tools and evaluation code for the ActivityNet human activity benchmark.
Artificial Analysis ASR leaderboard
Artificial Analysis leaderboard for non-streaming speech-to-text, comparing word error rate, processing speed and price per model.
Artificial Analysis Image Arena
Artificial Analysis Image Arena: blind-vote rankings of text-to-image and image editing models.
Artificial Analysis Image Editing Leaderboard
Artificial Analysis image editing leaderboard: blind-vote rankings of image editing models with speed and price.
Artificial Analysis speech arena
Artificial Analysis Speech Arena: blind human votes between paired text-to-speech samples to rank voice models.
Artificial Analysis Text to Speech Leaderboard
Artificial Analysis text-to-speech leaderboard: Elo rankings of speech models from blind pairwise votes.
Artificial Analysis Speech to Text Leaderboard
Artificial Analysis speech-to-text leaderboard ranking ASR models by word error rate, speed and price.
Artificial Analysis text-to-image arena
Artificial Analysis text-to-image arena where users vote between anonymous image outputs to rank models.
Artificial Analysis Text-to-Image Leaderboard
Artificial Analysis text-to-image leaderboard: blind-vote rankings of image models with generation speed and price.
Artificial Analysis TTS leaderboard
Artificial Analysis text-to-speech leaderboard comparing voice quality preference, speed and price of speech generation models.
Artificial Analysis video arena
Artificial Analysis Video Arena: blind-vote rankings of AI video generation models.
AudioBench
by AudioLLMs
AudioBench GitHub repo: benchmark toolkit for evaluating audio large language models across speech and audio tasks.
Audiobox Aesthetics
Meta audiobox-aesthetics GitHub repo: automatic quality assessment models for speech, music and sound.
BLINK
by Xingyu Fu
BLINK project page: benchmark of visual perception tasks that multimodal LLMs find hard, such as depth and correspondence.
ChartQA
ChartQA GitHub repo: benchmark for question answering over charts.
CIFAR-10 and CIFAR-100
by Alex Krizhevsky (University of Toronto)
Creator-hosted page for the CIFAR-10 and CIFAR-100 image classification datasets, with download links and format details.
DocVQA
by DocVQA team (CVC, Universitat Autònoma de Barcelona)
DocVQA site: document visual question answering challenges and datasets built from real document images.
DPG-Bench (TencentQQGYLab/ELLA)
TencentQQGYLab ELLA GitHub repo: LLM-equipped diffusion models and the DPG-Bench dense prompt benchmark.
DreamBench++
DreamBench++ project page: a human-aligned benchmark for personalised, subject-driven image generation.
EditBench
Google Research page for Imagen Editor and EditBench, a benchmark for text-guided image inpainting with human evaluation.
EgoSchema
by egoschema
EgoSchema site: long-form egocentric video question answering for testing long-range video understanding.
encodechka
by David Dale
encodechka GitHub repo: benchmark of Russian sentence encoders across tasks, with a results table.
EvalCrafter
by EvalCrafter
EvalCrafter project page: a framework for evaluating text-to-video models with automatic metrics and human ratings.
GEdit-Bench (StepFun Step1X-Edit)
by stepfun-ai
StepFun Step1X-Edit GitHub repo: open image-editing model whose release includes GEdit-Bench, a benchmark of real-world editing requests.
GenAI-Arena (TIGER Lab)
by TIGER-Lab
TIGER Lab GenAI-Arena Space: academic arena voting between generative image, editing and video models.
GenEval
by Dhruba Ghosh
GenEval GitHub repo: object-focused framework for evaluating text-to-image alignment.
GQA
GQA dataset site: real-image visual question answering with scene graphs for testing compositional reasoning.
HEAR
HEAR benchmark site: holistic evaluation of audio representations across speech, music and environmental sound tasks.
HEIM
Stanford CRFM HEIM: holistic evaluation of text-to-image models on quality, alignment, bias and other dimensions.
ImageReward
by Z.ai
ImageReward GitHub repo (NeurIPS 2023): human preference reward model and data for text-to-image evaluation.