Evaluation & benchmarks for expert learners
443 resources · page 2 of 14
Artificial Analysis Image Arena
Artificial Analysis Image Arena: blind-vote rankings of text-to-image and image editing models.
Artificial Analysis Image Editing Leaderboard
Artificial Analysis image editing leaderboard: blind-vote rankings of image editing models with speed and price.
Artificial Analysis large open-weight cohort
Artificial Analysis listing of open-weight models above 150B parameters with intelligence, speed and price measurements.
Artificial Analysis LLM leaderboard
Artificial Analysis LLM leaderboard comparing over 250 models on intelligence index, price, output speed and latency.
Artificial Analysis medium open-weight cohort
Artificial Analysis listing of open-weight models from 40B to 150B parameters with intelligence, speed and price measurements.
Artificial Analysis small open-weight cohort
Artificial Analysis listing of open-weight models from 4B to 40B parameters with intelligence, speed and price measurements.
Artificial Analysis speech arena
Artificial Analysis Speech Arena: blind human votes between paired text-to-speech samples to rank voice models.
Artificial Analysis Text to Speech Leaderboard
Artificial Analysis text-to-speech leaderboard: Elo rankings of speech models from blind pairwise votes.
Artificial Analysis Speech to Text Leaderboard
Artificial Analysis speech-to-text leaderboard ranking ASR models by word error rate, speed and price.
Artificial Analysis text-to-image arena
Artificial Analysis text-to-image arena where users vote between anonymous image outputs to rank models.
Artificial Analysis Text-to-Image Leaderboard
Artificial Analysis text-to-image leaderboard: blind-vote rankings of image models with generation speed and price.
Artificial Analysis tiny open-weight cohort
Artificial Analysis listing of open-weight models with 4B parameters or fewer, with intelligence, speed and price measurements.
Artificial Analysis TTS leaderboard
Artificial Analysis text-to-speech leaderboard comparing voice quality preference, speed and price of speech generation models.
Artificial Analysis video arena
Artificial Analysis Video Arena: blind-vote rankings of AI video generation models.
AssistantBench
AssistantBench project page: realistic, time-consuming web tasks for evaluating whether web agents can complete them.
AudioBench
by AudioLLMs
AudioBench GitHub repo: benchmark toolkit for evaluating audio large language models across speech and audio tasks.
Audiobox Aesthetics
Meta audiobox-aesthetics GitHub repo: automatic quality assessment models for speech, music and sound.
augment-swebench-agent
by Augment Code
Augment augment-swebench-agent GitHub repo: open implementation of Augment's SWE-bench Verified agent.
awesome-japanese-llm
by llm-jp
LLM-jp awesome-japanese-llm page: overview of public Japanese LLMs and Japanese evaluation benchmarks.
BEHAVIOR
by Stanford Vision and Learning Lab (BEHAVIOR)
BEHAVIOR project site from Stanford: a simulation benchmark of 1,000 everyday household activities for embodied AI agents.
BEHAVIOR Challenge (BEHAVIOR-1K)
by Stanford Vision and Learning Lab (BEHAVIOR)
BEHAVIOR Challenge page: a competition on full-length household tasks in the BEHAVIOR-1K simulator, with rules and results.
BEIR
by BEIR Cellar
BEIR GitHub repo: heterogeneous information-retrieval benchmark of many datasets for zero-shot evaluation of retrievers.
sierra-research/tau-bench
Sierra τ-bench GitHub repo: code and data for evaluating tool-using agents in customer service domains.
Bench2Drive
Thinklab-SJTU Bench2Drive GitHub repo (NeurIPS 2024): closed-loop benchmark for end-to-end autonomous driving.
BenchLM.ai
by BenchLM
BenchLM.ai: aggregator listing AI models across many benchmarks with scores, pricing and context windows.
BenCzechMark
BenCzechMark Hugging Face Space: Czech LLM benchmark leaderboard accepting model result submissions.
Berkeley Function-Calling Leaderboard
Berkeley Function Calling Leaderboard (BFCL V4): model accuracy on function and tool calling, including agentic tasks.
BHASA
by AI Singapore
BHASA GitHub repo from AI Singapore: linguistic and cultural evaluation suite for LLMs in Southeast Asian languages.
BIG-bench
by Google
BIG-bench GitHub repo from Google: collaborative benchmark of over 200 tasks probing language model capabilities.
BigCodeBench
BigCodeBench leaderboard: practical programming tasks that require calling many libraries, for evaluating code LLMs.
BioASQ
by BioASQ
BioASQ site: recurring challenges on biomedical semantic indexing and question answering, with datasets and results.
BIRD
by bird_sql
BIRD benchmark site: large-scale text-to-SQL evaluation over real databases with dirty values and a public leaderboard.