Evaluation & benchmarks for intermediate learners
454 resources · page 8 of 15
MERA leaderboard (RU)
MERA leaderboard in Russian: current results for Text, Industrial, Code and Multi tracks with CSV and JSON export.
MERA
MERA English site: open Russian-language benchmark for foundation models with Text, Reason, Code and Industrial tracks.
MERA Multimodal
MERA Multimodal GitHub repo: open benchmark for evaluating multimodal models in Russian, with task definitions and code.
Meta-World
by meta-world
Meta-World site: 50 simulated robot manipulation tasks for benchmarking meta-reinforcement and multi-task learning.
METR
by METR
METR: research nonprofit publishing evaluations of frontier AI models' autonomous capabilities and risks.
METR Time Horizons
by METR
METR time horizons page: measured length of tasks frontier models complete autonomously, tracked over time.
MILU
by AI4Bharat
AI4Bharat MILU dataset: multiple-choice questions across subjects in Indian languages for evaluating India-specific knowledge.
Minari
Minari documentation: Python library and dataset standard for offline reinforcement learning research.
MineRL
by MineRL Labs
MineRL GitHub repo: Minecraft environments and human demonstrations for sample-efficient RL.
mini-SWE-agent
by SWE-agent
mini-SWE-agent GitHub repo: a roughly 100-line reference agent for solving GitHub issues, used as a simple SWE-bench baseline.
MIZAN Persian LLM Leaderboard
by MCINext
MIZAN Persian LLM leaderboard: Persian benchmarks including MT-Bench, IFEval and PerMMLU.
MLCommons AILuminate
by MLCommons
MLCommons AILuminate page: safety benchmark for general-purpose chatbots, with jailbreak and agentic tracks listed.
MLE-bench
by Jun Shern Chan et al. (OpenAI)
MLE-bench paper from OpenAI: 75 Kaggle competitions used to evaluate AI agents on machine-learning engineering work.
MLPerf Client
by MLCommons
MLCommons MLPerf Client page: benchmark for LLM and AI workload performance on personal computers.
MLPerf Inference
by MLCommons
MLCommons MLPerf Inference Datacenter results: standardised throughput and latency of AI systems across models.
MLPerf Tiny
by MLCommons
MLCommons MLPerf Tiny page: inference benchmark for embedded and low-power devices, with published results.
MLPerf Training
by MLCommons
MLCommons MLPerf Training page: benchmark measuring time to train models to target quality, with published results.
MMAU
by mmaubench
MMAU project page: audio understanding and reasoning benchmark over speech, music and environmental sounds.
MMAU-Pro
by Sonal Kumar
MMAU-Pro GitHub repo: benchmark for audio understanding and reasoning in audio-language models.
MMBench
by OpenCompass
OpenCompass MMBench GitHub repo: multimodal evaluation benchmark in multiple languages.
MMLU
MMLU GitHub repo: the original Measuring Massive Multitask Language Understanding test with 57 subjects and evaluation code.
MMLU-Pro
by TIGER Lab
MMLU-Pro GitHub repo: code and data for a harder, ten-option version of MMLU for evaluating language models.
MMLU-ProX
by LiLab
MMLU-ProX dataset: MMLU-Pro translated into many languages for cross-lingual reasoning evaluation.
MMMU
by MMMU
MMMU project site: college-level multimodal reasoning benchmark across disciplines, including MMMU-Pro and leaderboards.
MOSES
by MOSES
MOSES GitHub repo: benchmarking platform with metrics and baselines for molecular generation models.
MTEB Leaderboard
by Massive Text Embedding Benchmark
MTEB leaderboard Space: embedding model scores across many retrieval, clustering and classification tasks.
embeddings-benchmark/mteb
by Massive Text Embedding Benchmark
MTEB GitHub repo: Massive Text Embedding Benchmark code for evaluating embedding models across tasks and languages.
MuChoMusic
by MuLaB
MuChoMusic GitHub repo: benchmark for music understanding in audio language models.
Music Arena
by Music Arena team (Carnegie Mellon University)
Music Arena: arena where users vote between anonymous text-to-music generations to rank models.
MusTBench
by Sony
Sony MusTBench GitHub repo: benchmark for temporal grounding of music in large audio-language models.
MVBench (OpenGVLab/Ask-Anything)
by OpenGVLab
OpenGVLab Ask-Anything GitHub repo: VideoChat models and the MVBench video understanding benchmark.
NAVSIM (v2)
NAVSIM GitHub repo: pseudo-simulation benchmark for evaluating end-to-end autonomous-driving planners.