Evaluation

Measure model capabilities and application behavior with explicit criteria.

• Benchmarks And Leaderboards (512)
• Eval Harnesses (3)
• Application Evals (2)
• Llm As Judge (1)
• Agent Evals (1)

519 resources · page 8 of 17

• MERA leaderboard (RU)

  Creator not supplied; see original source.

  MERA leaderboard in Russian: current results for Text, Industrial, Code and Multi tracks with CSV and JSON export.

  Not applicable (entity) · unknown · unknown resource · ru

• MERA

  Creator not supplied; see original source.

  MERA English site: open Russian-language benchmark for foundation models with Text, Reason, Code and Industrial tracks.

  Not applicable (entity) · unknown · unknown resource · ru

• MERA Multimodal

  Creator not supplied; see original source.

  MERA Multimodal GitHub repo: open benchmark for evaluating multimodal models in Russian, with task definitions and code.

  Not applicable (entity) · unknown · unknown resource · ru

• Meta-World

  Creator not supplied; see original source.

  Meta-World site: 50 simulated robot manipulation tasks for benchmarking meta-reinforcement and multi-task learning.

  Not applicable (entity) · unknown · unknown resource · en

• METR

  Creator not supplied; see original source.

  METR: research nonprofit publishing evaluations of frontier AI models' autonomous capabilities and risks.

  Not applicable (entity) · unknown · unknown resource · en

• METR Time Horizons

  Creator not supplied; see original source.

  METR time horizons page: measured length of tasks frontier models complete autonomously, tracked over time.

  Not applicable (entity) · unknown · unknown resource · en

• MILU

  Creator not supplied; see original source.

  AI4Bharat MILU dataset: multiple-choice questions across subjects in Indian languages for evaluating India-specific knowledge.

  Not applicable (entity) · unknown · unknown resource · en|mul

• Minari

  Creator not supplied; see original source.

  Minari documentation: Python library and dataset standard for offline reinforcement learning research.

  Not applicable (entity) · unknown · unknown resource · en

• MineRL

  Creator not supplied; see original source.

  MineRL GitHub repo: Minecraft environments and human demonstrations for sample-efficient RL.

  Not applicable (entity) · unknown · unknown resource · en

• mini-SWE-agent

  Creator not supplied; see original source.

  mini-SWE-agent GitHub repo: a roughly 100-line reference agent for solving GitHub issues, used as a simple SWE-bench baseline.

  Not applicable (entity) · unknown · unknown resource · en

• MIZAN Persian LLM Leaderboard

  Creator not supplied; see original source.

  MIZAN Persian LLM leaderboard: Persian benchmarks including MT-Bench, IFEval and PerMMLU.

  Not applicable (entity) · unknown · unknown resource · fa

• MLCommons AILuminate

  Creator not supplied; see original source.

  MLCommons AILuminate page: safety benchmark for general-purpose chatbots, with jailbreak and agentic tracks listed.

  Not applicable (entity) · unknown · unknown resource · en|fr

• MLE-bench

  Creator not supplied; see original source.

  MLE-bench paper from OpenAI: 75 Kaggle competitions used to evaluate AI agents on machine-learning engineering work.

  Not applicable (entity) · unknown · unknown resource · en

• MLPerf Client

  Creator not supplied; see original source.

  MLCommons MLPerf Client page: benchmark for LLM and AI workload performance on personal computers.

  Not applicable (entity) · unknown · unknown resource · en

• MLPerf Inference

  Creator not supplied; see original source.

  MLCommons MLPerf Inference Datacenter results: standardised throughput and latency of AI systems across models.

  Not applicable (entity) · unknown · unknown resource · en

• MLPerf Tiny

  Creator not supplied; see original source.

  MLCommons MLPerf Tiny page: inference benchmark for embedded and low-power devices, with published results.

  Not applicable (entity) · unknown · unknown resource · en

• MLPerf Training

  Creator not supplied; see original source.

  MLCommons MLPerf Training page: benchmark measuring time to train models to target quality, with published results.

  Not applicable (entity) · unknown · unknown resource · en

• MMAU

  Creator not supplied; see original source.

  MMAU project page: audio understanding and reasoning benchmark over speech, music and environmental sounds.

  Not applicable (entity) · unknown · unknown resource · en

• MMAU-Pro

  Creator not supplied; see original source.

  MMAU-Pro GitHub repo: benchmark for audio understanding and reasoning in audio-language models.

  Not applicable (entity) · unknown · unknown resource · en

• MMBench

  Creator not supplied; see original source.

  OpenCompass MMBench GitHub repo: multimodal evaluation benchmark in multiple languages.

  Not applicable (entity) · unknown · unknown resource · zh

• MMLU

  Creator not supplied; see original source.

  MMLU GitHub repo: the original Measuring Massive Multitask Language Understanding test with 57 subjects and evaluation code.

  Not applicable (entity) · unknown · unknown resource · en

• MMLU-Pro

  Creator not supplied; see original source.

  MMLU-Pro GitHub repo: code and data for a harder, ten-option version of MMLU for evaluating language models.

  Not applicable (entity) · unknown · unknown resource · en

• MMLU-ProX

  Creator not supplied; see original source.

  MMLU-ProX dataset: MMLU-Pro translated into many languages for cross-lingual reasoning evaluation.

  Not applicable (entity) · unknown · unknown resource · und

• MMMU

  Creator not supplied; see original source.

  MMMU project site: college-level multimodal reasoning benchmark across disciplines, including MMMU-Pro and leaderboards.

  Not applicable (entity) · unknown · unknown resource · en

• MOSES

  Creator not supplied; see original source.

  MOSES GitHub repo: benchmarking platform with metrics and baselines for molecular generation models.

  Not applicable (entity) · unknown · unknown resource · en

• MTEB Leaderboard

  Creator not supplied; see original source.

  MTEB leaderboard Space: embedding model scores across many retrieval, clustering and classification tasks.

  Not applicable (entity) · unknown · unknown resource · mul

• embeddings-benchmark/mteb

  Creator not supplied; see original source.

  MTEB GitHub repo: Massive Text Embedding Benchmark code for evaluating embedding models across tasks and languages.

  Not applicable (entity) · unknown · unknown resource · mul

• MuChoMusic

  Creator not supplied; see original source.

  MuChoMusic GitHub repo: benchmark for music understanding in audio language models.

  Not applicable (entity) · unknown · unknown resource · en

• Music Arena

  Creator not supplied; see original source.

  Music Arena: arena where users vote between anonymous text-to-music generations to rank models.

  Not applicable (entity) · unknown · unknown resource · en

• MusTBench

  Creator not supplied; see original source.

  Sony MusTBench GitHub repo: benchmark for temporal grounding of music in large audio-language models.

  Not applicable (entity) · unknown · unknown resource · en

• MVBench (OpenGVLab/Ask-Anything)

  Creator not supplied; see original source.

  OpenGVLab Ask-Anything GitHub repo: VideoChat models and the MVBench video understanding benchmark.

  Not applicable (entity) · unknown · unknown resource · en

• NAVSIM (v2)

  Creator not supplied; see original source.

  NAVSIM GitHub repo: pseudo-simulation benchmark for evaluating end-to-end autonomous-driving planners.

  Not applicable (entity) · unknown · unknown resource · en

Plain-text Markdown copy