--- id: "/evaluation-benchmarks/page/16/" title: "Evaluation & benchmarks · page 16" url: "https://skillissue.dev/evaluation-benchmarks/page/16/" --- # Evaluation & benchmarks Evaluation & benchmarks - [agent & coding evals](/evaluation-benchmarks/agent-coding-evals/index.md) (85) - [autonomous driving](/evaluation-benchmarks/autonomous-driving/index.md) (1) - [general leaderboards](/evaluation-benchmarks/general-leaderboards/index.md) (248) - [modality evals (image/video/speech)](/evaluation-benchmarks/modality-evals-image-video-speech/index.md) (87) - [physics & simulation](/evaluation-benchmarks/physics-simulation/index.md) (1) - [safety & efficiency evals](/evaluation-benchmarks/safety-efficiency-evals/index.md) (44) - [safety evals](/evaluation-benchmarks/safety-evals/index.md) (1) - [science & domain evals](/evaluation-benchmarks/science-domain-evals/index.md) (45) - [vision](/evaluation-benchmarks/vision/index.md) (1) [Beginner](/evaluation-benchmarks/beginner/index.md) [Intermediate](/evaluation-benchmarks/intermediate/index.md) [Advanced](/evaluation-benchmarks/advanced/index.md) [Expert](/evaluation-benchmarks/expert/index.md) 513 resources · page 16 of 17 - ### [METR experienced-developer productivity experiment](/evaluation/benchmarks-and-leaderboards/metr-experienced-developer-productivity-experiment/index.md) by [METR](https://metr.org/) METR study of how early-2025 AI tools affected experienced open-source developers' productivity. Advanced learning_resource paper US en - ### [MLflow documentation](/evaluation/benchmarks-and-leaderboards/mlflow-documentation/index.md) by [MLflow project (Databricks)](https://mlflow.org/) MLflow documentation: experiment tracking, model registry, GenAI tracing, evaluation and deployment. Advanced learning_resource docs US/global en - ### [MME](/evaluation/benchmarks-and-leaderboards/mme/index.md) by [Chaoyou Fu et al. (Nanjing University)](https://www.nju.edu.cn/) arXiv paper (2023) introducing MME, a benchmark for perception and cognition in multimodal LLMs. Advanced learning_resource paper China en - ### [Model Cards for Model Reporting](/evaluation/benchmarks-and-leaderboards/model-cards-for-model-reporting/index.md) by [Google Research](https://research.google/) Google Research publication page for Model Cards for Model Reporting, a standard for documenting trained models. Advanced learning_resource paper United States / international collaboration en - ### [MoleculeNet](/evaluation/benchmarks-and-leaderboards/moleculenet/index.md) by [Zhenqin Wu et al. (Stanford University)](https://www.stanford.edu) arXiv paper (2017) "MoleculeNet: A Benchmark for Molecular Machine Learning", introducing the MoleculeNet benchmark datasets for molecular property prediction. Advanced learning_resource paper US / International en - ### [Open Ko-LLM Leaderboard2 paper](/evaluation/benchmarks-and-leaderboards/open-ko-llm-leaderboard2-paper/index.md) by [Hyeonwoo Kim et al. (Upstage)](https://www.upstage.ai) arXiv paper (2024) describing Open Ko-LLM Leaderboard2 for evaluating Korean LLMs. Advanced–Expert learning_resource paper Korea ko|en - ### [Introducing TRAIL: A Benchmark for Agentic Evaluation (Patronus AI)](/evaluation/benchmarks-and-leaderboards/patronus-trail-benchmark-for-agentic-evaluation-2025-06-05/index.md) by [Patronus AI](https://www.patronus.ai/) Patronus AI post introducing TRAIL, a benchmark of annotated agent traces for detecting reasoning, planning and execution errors. Advanced learning_resource blog US en - ### [Quantifying infrastructure noise in agentic coding evals](/evaluation/benchmarks-and-leaderboards/quantifying-infrastructure-noise-in-agentic-coding-evals/index.md) by [Anthropic](https://anthropic.com/) Anthropic engineering post quantifying how infrastructure noise affects agentic coding evaluation scores. Advanced learning_resource paper US en - ### [Safetywashing](/evaluation/benchmarks-and-leaderboards/safetywashing/index.md) by [Richard Ren et al. (Center for AI Safety)](https://safe.ai/) arXiv paper (2024) "Safetywashing", testing whether AI safety benchmarks measure safety or general capability. Advanced learning_resource paper US / International en - ### [SEA-HELM paper](/evaluation/benchmarks-and-leaderboards/sea-helm-paper/index.md) by [Yosephine Susanto et al. (AI Singapore)](https://aisingapore.org/) arXiv paper (2025) introducing SEA-HELM, holistic evaluation of language models on Southeast Asian languages. Advanced–Expert learning_resource paper SE Asia en - ### [τ-Bench: Benchmarking AI agents for the real-world (Sierra)](/evaluation/benchmarks-and-leaderboards/sierra-bench-2024-06-20/index.md) by [Sierra](https://sierra.ai/) Sierra post introducing τ-bench, a benchmark of tool-using agents talking to simulated users, with the pass^k reliability metric. Advanced learning_resource blog US en - ### [SUPERB](/evaluation/benchmarks-and-leaderboards/superb/index.md) by [Shu-wen Yang et al. (National Taiwan University)](https://www.ntu.edu.tw/) arXiv paper (2021) "SUPERB: Speech processing Universal PERformance Benchmark", introducing SUPERB, a benchmark of speech representations across many speech tasks. Advanced learning_resource paper International en - ### [SWE-Bench Pro Verified](/evaluation/benchmarks-and-leaderboards/swe-bench-pro-verified-audit/index.md) by [Pujun Zheng et al. (Shanghai Artificial Intelligence Laboratory)](https://www.shlab.org.cn) arXiv paper (2026) introducing SWE-Bench Pro Verified, an audited software-engineering benchmark that removes leakage and flawed tasks. Advanced learning_resource paper China / International en - ### [The Leaderboard Illusion](/evaluation/benchmarks-and-leaderboards/the-leaderboard-illusion/index.md) by [Shivalika Singh et al. (Cohere)](https://cohere.com) arXiv paper (2025) "The Leaderboard Illusion", auditing how private testing and data access distort Chatbot Arena rankings. Advanced learning_resource paper International en - ### [Tutor CoPilot: a human-AI approach to real-time expertise](/evaluation/benchmarks-and-leaderboards/tutor-copilot-a-human-ai-approach-to-real-time-expertise/index.md) by [Rose E. Wang et al. (Stanford University)](https://www.stanford.edu) arXiv paper (2024) reporting a randomized field study of Tutor CoPilot, an AI assistant for human tutors. Advanced learning_resource paper United States en - ### [UNLP 2026 Ukrainian NLP workshop paper](/evaluation/benchmarks-and-leaderboards/unlp-2026-ukrainian-nlp-workshop-paper/index.md) by [Svitlana Galeshchuk et al.](https://aclanthology.org/people/svitlana-galeshchuk/unverified/) UNLP 2026 workshop paper (PDF) on Ukrainian NLP evaluation. Advanced–Expert learning_resource paper Ukraine uk|en - ### [3D Arena paper](/evaluation/benchmarks-and-leaderboards/3d-arena-paper/index.md) by [Dylan Ebert (Hugging Face)](https://huggingface.co/) arXiv paper (2025) introducing 3D Arena, an open pairwise-preference platform for evaluating generative 3D models. Expert learning_resource paper US/France (HF) en - ### [BEHAVIOR 2025 1st-place report](/evaluation/benchmarks-and-leaderboards/behavior-2025-1st-place-report/index.md) by [Ilia Larchenko, Gleb Zarin and Akash Karnatak](https://ilialarchenko.com/) arXiv PDF (2025) of the first-place report from the NeurIPS 2025 BEHAVIOR household robotics challenge. Expert learning_resource paper unstated en - ### [A framework for evaluating the chemical knowledge and reasoning abilities of large language models (ChemBench)](/evaluation/benchmarks-and-leaderboards/chembench-paper-nature-chemistry-2025/index.md) by [Adrian Mirza et al. (Helmholtz Institute Jena)](https://www.hi-jena.de) ChemBench paper (Nature Chemistry 2025, PMC full text): framework evaluating chemical knowledge and reasoning of LLMs against chemists. Expert learning_resource paper Germany en - ### [EvalScope τ-bench docs (ModelScope/Alibaba)](/evaluation/benchmarks-and-leaderboards/evalscope-bench-docs-modelscope-alibaba/index.md) by [ModelScope (Alibaba)](https://www.modelscope.cn/) EvalScope documentation on τ-bench: running the conversational agent benchmark with Alibaba's ModelScope evaluation framework. Expert learning_resource docs China en|zh - ### [FinanceBench: A New Benchmark for Financial Question Answering](/evaluation/benchmarks-and-leaderboards/financebench-paper-arxiv-2311-11944/index.md) by [Pranab Islam et al. (Stanford University)](https://www.stanford.edu) arXiv paper (2023) introducing FinanceBench, a benchmark for financial question answering over company filings. Expert learning_resource paper US en - ### [FrontierMath paper](/evaluation/benchmarks-and-leaderboards/frontiermath-paper/index.md) by [Elliot Glazer et al. (Rutgers, The State University of New Jersey)](https://www.rutgers.edu/) arXiv paper (2024) introducing FrontierMath, a benchmark of unpublished advanced mathematics problems. Expert learning_resource paper US en - ### [MASK paper](/evaluation/benchmarks-and-leaderboards/mask-paper/index.md) by [Richard Ren et al. (Center for AI Safety)](https://safe.ai/) arXiv PDF (2025) of the MASK benchmark paper on measuring honesty separately from accuracy in LLMs. Expert learning_resource paper US en - ### [Medical VLM contamination audit](/evaluation/benchmarks-and-leaderboards/medical-vlm-contamination-audit/index.md) by [Bruce Changlong Xu et al. (Stanford University)](https://www.stanford.edu) arXiv paper (2026) auditing pretraining contamination of public medical vision-language benchmarks. Expert learning_resource paper unstated en - ### [MMMU-Pro paper](/evaluation/benchmarks-and-leaderboards/mmmu-pro-paper/index.md) by [Xiang Yue et al. (Carnegie Mellon University)](https://www.cmu.edu/) arXiv HTML version of the MMMU-Pro paper (2024), a harder multimodal multi-discipline benchmark. Expert learning_resource paper US en - ### [Music Arena paper](/evaluation/benchmarks-and-leaderboards/music-arena-paper/index.md) by [Yonghyun Kim et al. (Carnegie Mellon University, LMArena and Sony AI)](https://www.cmu.edu/) arXiv paper (2025) introducing Music Arena, live human-preference evaluation for text-to-music models. Expert learning_resource paper US/Japan (CMU and Sony listed in search) en - ### [Open ASR Leaderboard paper](/evaluation/benchmarks-and-leaderboards/open-asr-leaderboard-paper/index.md) by [Vaibhav Srivastav et al. (Hugging Face)](https://huggingface.co/) arXiv PDF (2025) of the Open ASR Leaderboard paper: evaluation method, datasets and results for speech recognition models. Expert learning_resource paper France/UK en - ### [RoboArena paper page (Penn)](/evaluation/benchmarks-and-leaderboards/roboarena-paper-page-penn/index.md) by [University of Pennsylvania](https://www.upenn.edu/) RoboArena paper page (University of Pennsylvania): distributed real-world evaluation of generalist robot policies. Expert learning_resource paper US en - ### [Frontier AI Risk Management Framework in Practice (Shanghai AI Lab)](/evaluation/benchmarks-and-leaderboards/shanghai-ai-lab-frontier-ai-risk-management-framework-technical-report/index.md) by [Shanghai AI Lab (Shanghai AI Laboratory)](https://www.shlab.org.cn/) arXiv paper (2025) by Shanghai AI Laboratory applying its Frontier AI Risk Management Framework to analyse model risks. Expert learning_resource paper China [STATE] en - ### [TaxPraBen: Chinese real-world tax practice](/evaluation/benchmarks-and-leaderboards/taxpraben-chinese-real-world-tax-practice/index.md) by [Gang Hu et al.](https://aclanthology.org/people/gang-hu/) ACL 2026 paper introducing TaxPraBen, a benchmark of LLMs on Chinese real-world tax practice tasks. Expert learning_resource paper China en - ### [VBench-2.0 paper](/evaluation/benchmarks-and-leaderboards/vbench-2-0-paper/index.md) by [Dian Zheng et al. (Nanyang Technological University and Shanghai AI Laboratory)](https://www.ntu.edu.sg/) arXiv HTML version of the VBench-2.0 paper (2025) evaluating video generation on physics, commonsense and faithfulness. Expert learning_resource paper Singapore/China en - ### [When Rubrics Fail](/evaluation/benchmarks-and-leaderboards/when-rubrics-fail-medical-eval-hallucination/index.md) by [Griffin Farrow et al. (University of Oxford)](https://www.ox.ac.uk/) arXiv PDF (2026) "When Rubrics Fail", a critique of rubric-based evaluation in medical LLM benchmarks. Expert learning_resource paper unstated en [Previous page](/evaluation-benchmarks/page/15/index.md) [Next page](/evaluation-benchmarks/page/17/index.md) [Plain-text Markdown copy](/evaluation-benchmarks/page/16/index.md)