Evaluation

Measure model capabilities and application behavior with explicit criteria.

• Benchmarks And Leaderboards (512)
• Eval Harnesses (3)
• Application Evals (2)
• Llm As Judge (1)
• Agent Evals (1)

519 resources · page 2 of 17

• Artificial Analysis Image Arena

  Creator not supplied; see original source.

  Artificial Analysis Image Arena: blind-vote rankings of text-to-image and image editing models.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis Image Editing Leaderboard

  Creator not supplied; see original source.

  Artificial Analysis image editing leaderboard: blind-vote rankings of image editing models with speed and price.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis large open-weight cohort

  Creator not supplied; see original source.

  Artificial Analysis listing of open-weight models above 150B parameters with intelligence, speed and price measurements.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis LLM leaderboard

  Creator not supplied; see original source.

  Artificial Analysis LLM leaderboard comparing over 250 models on intelligence index, price, output speed and latency.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis medium open-weight cohort

  Creator not supplied; see original source.

  Artificial Analysis listing of open-weight models from 40B to 150B parameters with intelligence, speed and price measurements.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis small open-weight cohort

  Creator not supplied; see original source.

  Artificial Analysis listing of open-weight models from 4B to 40B parameters with intelligence, speed and price measurements.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis speech arena

  Creator not supplied; see original source.

  Artificial Analysis Speech Arena: blind human votes between paired text-to-speech samples to rank voice models.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis Text to Speech Leaderboard

  Creator not supplied; see original source.

  Artificial Analysis text-to-speech leaderboard: Elo rankings of speech models from blind pairwise votes.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis Speech to Text Leaderboard

  Creator not supplied; see original source.

  Artificial Analysis speech-to-text leaderboard ranking ASR models by word error rate, speed and price.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis text-to-image arena

  Creator not supplied; see original source.

  Artificial Analysis text-to-image arena where users vote between anonymous image outputs to rank models.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis Text-to-Image Leaderboard

  Creator not supplied; see original source.

  Artificial Analysis text-to-image leaderboard: blind-vote rankings of image models with generation speed and price.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis tiny open-weight cohort

  Creator not supplied; see original source.

  Artificial Analysis listing of open-weight models with 4B parameters or fewer, with intelligence, speed and price measurements.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis TTS leaderboard

  Creator not supplied; see original source.

  Artificial Analysis text-to-speech leaderboard comparing voice quality preference, speed and price of speech generation models.

  Not applicable (entity) · unknown · unknown resource · en

• Artificial Analysis video arena

  Creator not supplied; see original source.

  Artificial Analysis Video Arena: blind-vote rankings of AI video generation models.

  Not applicable (entity) · unknown · unknown resource · und

• AssistantBench

  Creator not supplied; see original source.

  AssistantBench project page: realistic, time-consuming web tasks for evaluating whether web agents can complete them.

  Not applicable (entity) · unknown · unknown resource · en

• AudioBench

  Creator not supplied; see original source.

  AudioBench GitHub repo: benchmark toolkit for evaluating audio large language models across speech and audio tasks.

  Not applicable (entity) · unknown · unknown resource · en

• Audiobox Aesthetics

  Creator not supplied; see original source.

  Meta audiobox-aesthetics GitHub repo: automatic quality assessment models for speech, music and sound.

  Not applicable (entity) · unknown · unknown resource · en

• augment-swebench-agent

  Creator not supplied; see original source.

  Augment augment-swebench-agent GitHub repo: open implementation of Augment's SWE-bench Verified agent.

  Not applicable (entity) · unknown · unknown resource · en

• awesome-japanese-llm

  Creator not supplied; see original source.

  LLM-jp awesome-japanese-llm page: overview of public Japanese LLMs and Japanese evaluation benchmarks.

  Not applicable (entity) · unknown · unknown resource · ja

• BEHAVIOR

  Creator not supplied; see original source.

  BEHAVIOR project site from Stanford: a simulation benchmark of 1,000 everyday household activities for embodied AI agents.

  Not applicable (entity) · unknown · unknown resource · en

• BEHAVIOR Challenge (BEHAVIOR-1K)

  Creator not supplied; see original source.

  BEHAVIOR Challenge page: a competition on full-length household tasks in the BEHAVIOR-1K simulator, with rules and results.

  Not applicable (entity) · unknown · unknown resource · en

• BEIR

  Creator not supplied; see original source.

  BEIR GitHub repo: heterogeneous information-retrieval benchmark of many datasets for zero-shot evaluation of retrievers.

  Not applicable (entity) · unknown · unknown resource · en

• sierra-research/tau-bench

  Creator not supplied; see original source.

  Sierra τ-bench GitHub repo: code and data for evaluating tool-using agents in customer service domains.

  Not applicable (entity) · unknown · unknown resource · en

• Bench2Drive

  Creator not supplied; see original source.

  Thinklab-SJTU Bench2Drive GitHub repo (NeurIPS 2024): closed-loop benchmark for end-to-end autonomous driving.

  Not applicable (entity) · unknown · unknown resource · en

• BenchLM.ai

  Creator not supplied; see original source.

  BenchLM.ai: aggregator listing AI models across many benchmarks with scores, pricing and context windows.

  Not applicable (entity) · unknown · unknown resource · en

• BenCzechMark

  Creator not supplied; see original source.

  BenCzechMark Hugging Face Space: Czech LLM benchmark leaderboard accepting model result submissions.

  Not applicable (entity) · unknown · unknown resource · cs

• Berkeley Function-Calling Leaderboard

  Creator not supplied; see original source.

  Berkeley Function Calling Leaderboard (BFCL V4): model accuracy on function and tool calling, including agentic tasks.

  Not applicable (entity) · unknown · unknown resource · en

• BHASA

  Creator not supplied; see original source.

  BHASA GitHub repo from AI Singapore: linguistic and cultural evaluation suite for LLMs in Southeast Asian languages.

  Not applicable (entity) · unknown · unknown resource · en

• BIG-bench

  Creator not supplied; see original source.

  BIG-bench GitHub repo from Google: collaborative benchmark of over 200 tasks probing language model capabilities.

  Not applicable (entity) · unknown · unknown resource · en

• BigCodeBench

  Creator not supplied; see original source.

  BigCodeBench leaderboard: practical programming tasks that require calling many libraries, for evaluating code LLMs.

  Not applicable (entity) · unknown · unknown resource · en

• BioASQ

  Creator not supplied; see original source.

  BioASQ site: recurring challenges on biomedical semantic indexing and question answering, with datasets and results.

  Not applicable (entity) · unknown · unknown resource · en

• BIRD

  Creator not supplied; see original source.

  BIRD benchmark site: large-scale text-to-SQL evaluation over real databases with dirty values and a public leaderboard.

  Not applicable (entity) · unknown · unknown resource · en

Plain-text Markdown copy