Evaluation & benchmarks for expert learners
443 resources · page 14 of 14
CLARIN-PT-LDB paper
by João Silva et al. (University of Lisbon)
arXiv paper (2026) introducing CLARIN-PT-LDB, an open LLM leaderboard for European Portuguese.
FlagEval Findings Report (LRMs)
arXiv paper (2025) FlagEval findings report: BAAI's preliminary evaluation of large reasoning models.
FlagEvalMM
arXiv paper (2025) introducing FlagEvalMM, BAAI's framework for evaluating multimodal models.
Inspect India Evals
by Abhishek Kumar Singh et al.
arXiv paper (2026) introducing Inspect India Evals, an open framework of Indian benchmarks with a fairness index.
J-AISI annual report 2025
Japan AI Safety Institute 2025 annual activity report (PDF, Japanese).
Korea AISI 5th AI Safety Forecast Report
by Korea AISI
Korea AI Safety Institute 5th AI Safety Forecast Report covering May–July 2026.
La Leaderboard paper
by María Grandury et al. (Universidad Politécnica de Madrid)
arXiv paper (2025) introducing La Leaderboard, an LLM leaderboard for Spanish varieties and languages of Spain and Latin America.
Open Ko-LLM Leaderboard2 paper
by Hyeonwoo Kim et al. (Upstage)
arXiv paper (2024) describing Open Ko-LLM Leaderboard2 for evaluating Korean LLMs.
SEA-HELM paper
by Yosephine Susanto et al. (AI Singapore)
arXiv paper (2025) introducing SEA-HELM, holistic evaluation of language models on Southeast Asian languages.
UNLP 2026 Ukrainian NLP workshop paper
UNLP 2026 workshop paper (PDF) on Ukrainian NLP evaluation.
3D Arena paper
arXiv paper (2025) introducing 3D Arena, an open pairwise-preference platform for evaluating generative 3D models.
BEHAVIOR 2025 1st-place report
by Ilia Larchenko, Gleb Zarin and Akash Karnatak
arXiv PDF (2025) of the first-place report from the NeurIPS 2025 BEHAVIOR household robotics challenge.
A framework for evaluating the chemical knowledge and reasoning abilities of large language models (ChemBench)
by Adrian Mirza et al. (Helmholtz Institute Jena)
ChemBench paper (Nature Chemistry 2025, PMC full text): framework evaluating chemical knowledge and reasoning of LLMs against chemists.
EvalScope τ-bench docs (ModelScope/Alibaba)
EvalScope documentation on τ-bench: running the conversational agent benchmark with Alibaba's ModelScope evaluation framework.
FinanceBench: A New Benchmark for Financial Question Answering
by Pranab Islam et al. (Stanford University)
arXiv paper (2023) introducing FinanceBench, a benchmark for financial question answering over company filings.
FrontierMath paper
by Elliot Glazer et al. (Rutgers, The State University of New Jersey)
arXiv paper (2024) introducing FrontierMath, a benchmark of unpublished advanced mathematics problems.
MASK paper
by Richard Ren et al. (Center for AI Safety)
arXiv PDF (2025) of the MASK benchmark paper on measuring honesty separately from accuracy in LLMs.
Medical VLM contamination audit
by Bruce Changlong Xu et al. (Stanford University)
arXiv paper (2026) auditing pretraining contamination of public medical vision-language benchmarks.
MMMU-Pro paper
by Xiang Yue et al. (Carnegie Mellon University)
arXiv HTML version of the MMMU-Pro paper (2024), a harder multimodal multi-discipline benchmark.
Music Arena paper
by Yonghyun Kim et al. (Carnegie Mellon University, LMArena and Sony AI)
arXiv paper (2025) introducing Music Arena, live human-preference evaluation for text-to-music models.
Open ASR Leaderboard paper
by Vaibhav Srivastav et al. (Hugging Face)
arXiv PDF (2025) of the Open ASR Leaderboard paper: evaluation method, datasets and results for speech recognition models.
RoboArena paper page (Penn)
RoboArena paper page (University of Pennsylvania): distributed real-world evaluation of generalist robot policies.
Frontier AI Risk Management Framework in Practice (Shanghai AI Lab)
by Shanghai AI Lab (Shanghai AI Laboratory)
arXiv paper (2025) by Shanghai AI Laboratory applying its Frontier AI Risk Management Framework to analyse model risks.
TaxPraBen: Chinese real-world tax practice
ACL 2026 paper introducing TaxPraBen, a benchmark of LLMs on Chinese real-world tax practice tasks.
VBench-2.0 paper
by Dian Zheng et al. (Nanyang Technological University and Shanghai AI Laboratory)
arXiv HTML version of the VBench-2.0 paper (2025) evaluating video generation on physics, commonsense and faithfulness.
When Rubrics Fail
by Griffin Farrow et al. (University of Oxford)
arXiv PDF (2026) "When Rubrics Fail", a critique of rubric-based evaluation in medical LLM benchmarks.
Who Validates the Validators? (Shankar et al.)
by Shreya Shankar et al. (UC Berkeley)
arXiv paper (2024) by Shankar et al. on aligning LLM-assisted evaluators with human preferences and criteria drift.