--- id: "/tags/evaluation-benchmarks/page/15/" title: "Evaluation Benchmarks · page 15" url: "https://skillissue.dev/tags/evaluation-benchmarks/page/15/" --- # Evaluation Benchmarks [Beginner](/levels/beginner/index.md) [Intermediate](/levels/intermediate/index.md) [Advanced](/levels/advanced/index.md) [Expert](/levels/expert/index.md) 513 resources · page 15 of 17 - ### [Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity](/evaluation/benchmarks-and-leaderboards/metr-developer-productivity-rct-2025/index.md) by [Joel Becker et al. (METR)](https://metr.org/) arXiv paper (2025) by METR reporting a randomized trial of early-2025 AI tools' effect on experienced open-source developers' speed. Intermediate learning_resource paper US en - ### [METR time-horizon paper](/evaluation/benchmarks-and-leaderboards/metr-time-horizon-paper/index.md) by [Thomas Kwa et al. (METR)](https://metr.org/) arXiv paper (2025) by METR measuring AI ability to complete long software tasks as a time horizon. Intermediate learning_resource paper US en - ### [The Open Arabic LLM Leaderboard 2](/evaluation/benchmarks-and-leaderboards/oall-v2-launch-blog/index.md) by [Hugging Face](https://huggingface.co/) Hugging Face post launching the Open Arabic LLM Leaderboard 2 with native Arabic benchmarks. Intermediate learning_resource blog MENA ar|en - ### [OECD AI Incidents Monitor](/evaluation/benchmarks-and-leaderboards/oecd-ai-incidents-monitor/index.md) by [OECD](https://www.oecd.org/) OECD AI Incidents Monitor: database of AI incidents and hazards compiled from news reports. Intermediate learning_resource docs International; OECD en - ### [promptfoo docs: Evaluate coding agents](/evaluation/benchmarks-and-leaderboards/promptfoo-evaluate-coding-agents/index.md) by [promptfoo](https://promptfoo.dev) Promptfoo guide to evaluating coding agents with its open-source eval harness. Intermediate learning_resource guide US en - ### [Sarvam-M multilingual language model (Sarvam AI)](/evaluation/benchmarks-and-leaderboards/sarvam-m-blog/index.md) by [Sarvam AI](https://sarvam.ai/) Sarvam AI technical post on Sarvam-M, a hybrid reasoning model for Indian languages, with post-training and inference details. Intermediate learning_resource blog India en - ### [SWE-bench paper](/evaluation/benchmarks-and-leaderboards/swe-bench-paper/index.md) by [Carlos E. Jimenez et al. (Princeton University)](https://www.princeton.edu/) arXiv paper (2023) introducing SWE-bench, evaluating language models on resolving real GitHub issues. Intermediate learning_resource paper US en - ### [The SWE-Bench Illusion](/evaluation/benchmarks-and-leaderboards/the-swe-bench-illusion/index.md) by [Shanchao Liang et al. (Purdue University)](https://www.purdue.edu/) arXiv paper (2025) "The SWE-Bench Illusion", testing whether LLMs memorise SWE-bench instead of reasoning. Intermediate learning_resource paper US (Microsoft) en - ### [W&B Weave documentation](/evaluation/benchmarks-and-leaderboards/weights-biases-weave/index.md) by [CoreWeave (Weights & Biases)](https://www.coreweave.com/) W&B Weave documentation, now hosted by CoreWeave: tracing, testing and evaluating LLM applications. Intermediate learning_resource docs US en - ### [What We've Learned From A Year of Building with LLMs](/evaluation/benchmarks-and-leaderboards/what-we-ve-learned-from-a-year-of-building-with-llms/index.md) by [Eugene Yan, Bryan Bischof, Charles Frye, Hamel Husain, Jason Liu and Shreya Shankar](https://applied-llms.org/) Applied LLMs guide by Yan, Bischof, Frye, Husain, Liu and Shankar: tactical, operational and strategic lessons from building LLM products. Intermediate learning_resource guide US en - ### [Wikipedia: AI safety institutes](/evaluation/benchmarks-and-leaderboards/wikipedia-ai-safety-institutes/index.md) by [Wikipedia (Wikimedia Foundation)](https://www.wikipedia.org/) Wikipedia article on AI safety institutes: state-backed bodies that evaluate AI safety, listed by country. Intermediate–Advanced learning_resource docs Global en - ### [Accuracy incentives and hallucinations](/evaluation/benchmarks-and-leaderboards/accuracy-incentives-and-hallucinations/index.md) by [Adam Tauman Kalai et al. (OpenAI)](https://openai.com/) Nature paper (2026) showing that accuracy-only evaluation of LLMs incentivises hallucination. Advanced learning_resource paper US en - ### [Agentic Benchmark Checklist (ABC)](/evaluation/benchmarks-and-leaderboards/agentic-benchmark-checklist-abc/index.md) by [Yuxuan Zhu et al.](https://orcid.org/0009-0001-6738-8930) arXiv paper (2025) proposing the Agentic Benchmark Checklist, guidelines for building valid agentic benchmarks. Advanced learning_resource paper US (UIUC et al.) en - ### [ARC Prize 2025 Technical Report](/evaluation/benchmarks-and-leaderboards/arc-prize-2025-technical-report/index.md) by [François Chollet et al. (ARC Prize Foundation)](https://arcprize.org/) arXiv paper (2026) ARC Prize 2025 technical report on ARC-AGI-2 results and winning approaches. Advanced learning_resource paper US en - ### [Arize Phoenix docs: Evaluating multi-agent systems](/evaluation/benchmarks-and-leaderboards/arize-phoenix-evaluating-multi-agent-systems/index.md) by [Arize AI](https://www.arize.com) Arize Phoenix documentation on evaluating multi-agent systems using traces of each agent's steps. Advanced learning_resource docs US en - ### [Benchmark-based assessment of Indian foundation models](/evaluation/benchmarks-and-leaderboards/benchmark-based-assessment-of-indian-foundation-models/index.md) by [Avinash Agarwal and Vridhi Jain](https://openalex.org/authors/A5123875505) arXiv paper (2026) comparing publicly benchmarked Indian foundation models across benchmarks. Advanced–Expert learning_resource paper India en - ### [C-Eval paper](/evaluation/benchmarks-and-leaderboards/c-eval-paper/index.md) by [Yuzhen Huang et al. (Shanghai Jiao Tong University)](https://www.sjtu.edu.cn) arXiv paper (2023) introducing C-Eval, a multi-level, multi-discipline Chinese evaluation suite for foundation models. Advanced–Expert learning_resource paper China en - ### [CLARIN-PT-LDB paper](/evaluation/benchmarks-and-leaderboards/clarin-pt-ldb-paper/index.md) by [João Silva et al. (University of Lisbon)](https://www.ulisboa.pt) arXiv paper (2026) introducing CLARIN-PT-LDB, an open LLM leaderboard for European Portuguese. Advanced–Expert learning_resource paper Portugal pt - ### [Coding-agent leaderboard resolution audit](/evaluation/benchmarks-and-leaderboards/coding-agent-leaderboard-resolution-audit/index.md) by [Fengshuo Liu et al. (Imperial College London)](https://www.imperial.ac.uk) arXiv paper (2026) arguing that coding agents have converged so the SWE-bench leaderboard can no longer separate them. Advanced learning_resource paper China / International en - ### [Cross-Context Verification: Hierarchical Detection of Benchmark Contamination](/evaluation/benchmarks-and-leaderboards/cross-context-verification-contamination-detection/index.md) by [Tae-Eun Song](https://openalex.org/authors/A5128671997) arXiv paper (2026) introducing Cross-Context Verification, a hierarchical method for detecting benchmark contamination. Advanced learning_resource paper unstated en - ### [DuetBench (Decagon)](/evaluation/benchmarks-and-leaderboards/decagon-duetbench/index.md) by [Decagon](https://decagon.ai/) Decagon post introducing DuetBench, a benchmark for self-improving customer-service agents. Advanced learning_resource blog US en - ### [Fairlearn](/evaluation/benchmarks-and-leaderboards/fairlearn/index.md) by [Fairlearn project](https://fairlearn.org/) Fairlearn: open-source toolkit and guides for assessing and mitigating fairness issues in ML models. Advanced learning_resource docs International en - ### [FlagEval Findings Report (LRMs)](/evaluation/benchmarks-and-leaderboards/flageval-findings-report-lrms/index.md) by [Bowen Qin et al. (BAAI)](https://www.baai.ac.cn/) arXiv paper (2025) FlagEval findings report: BAAI's preliminary evaluation of large reasoning models. Advanced–Expert learning_resource paper China en - ### [FlagEvalMM](/evaluation/benchmarks-and-leaderboards/flagevalmm/index.md) by [Zheqi He et al. (BAAI)](https://www.baai.ac.cn/) arXiv paper (2025) introducing FlagEvalMM, BAAI's framework for evaluating multimodal models. Advanced–Expert learning_resource paper China en - ### [HAE-RAE Evaluation Toolkit / HRET](/evaluation/benchmarks-and-leaderboards/hae-rae-evaluation-toolkit-hret/index.md) by [Hanwool Lee et al.](https://orcid.org/0000-0001-5565-5184) arXiv paper (2025) introducing the HAE-RAE Evaluation Toolkit (HRET), a unified framework for evaluating Korean LLMs. Advanced learning_resource paper South Korea en - ### [HLE paper](/evaluation/benchmarks-and-leaderboards/hle-paper/index.md) by [Long Phan et al. (Center for AI Safety and Scale AI)](https://safe.ai/) arXiv paper (2025) introducing Humanity's Last Exam, a benchmark of expert-written questions across academic fields. Advanced learning_resource paper US en - ### [IFEval](/evaluation/benchmarks-and-leaderboards/ifeval/index.md) by [Jeffrey Zhou et al. (Yale University)](https://www.yale.edu) arXiv paper (2023) introducing IFEval, instruction-following evaluation with automatically verifiable instructions. Advanced learning_resource paper US en - ### [Inspect India Evals](/evaluation/benchmarks-and-leaderboards/inspect-india-evals/index.md) by [Abhishek Kumar Singh et al.](https://openalex.org/authors/A5143884522) arXiv paper (2026) introducing Inspect India Evals, an open framework of Indian benchmarks with a fairness index. Advanced–Expert learning_resource paper India mul - ### [J-AISI annual report 2025](/evaluation/benchmarks-and-leaderboards/j-aisi-annual-report-2025/index.md) by [Japan AI Safety Institute](https://aisi.go.jp/) Japan AI Safety Institute 2025 annual activity report (PDF, Japanese). Advanced–Expert learning_resource paper Japan ja - ### [Korea AISI 5th AI Safety Forecast Report](/evaluation/benchmarks-and-leaderboards/korea-aisi-5th-ai-safety-forecast-report-2026-09-03/index.md) by [Korea AISI](https://aisi.re.kr/) Korea AI Safety Institute 5th AI Safety Forecast Report covering May–July 2026. Advanced–Expert learning_resource paper Korea ko - ### [La Leaderboard paper](/evaluation/benchmarks-and-leaderboards/la-leaderboard-paper/index.md) by [María Grandury et al. (Universidad Politécnica de Madrid)](https://www.upm.es) arXiv paper (2025) introducing La Leaderboard, an LLM leaderboard for Spanish varieties and languages of Spain and Latin America. Advanced–Expert learning_resource paper Spain/LatAm es|ca|eu|gl - ### [LabSafety Bench](/evaluation/benchmarks-and-leaderboards/labsafety-bench/index.md) by [Yujun Zhou et al. (University of Notre Dame)](https://www.nd.edu/) Nature Machine Intelligence paper introducing LabSafety Bench for LLM safety risks in scientific labs. Advanced learning_resource paper International en [Previous page](/tags/evaluation-benchmarks/page/14/index.md) [Next page](/tags/evaluation-benchmarks/page/16/index.md) [Plain-text Markdown copy](/tags/evaluation-benchmarks/page/15/index.md)