--- id: "/evaluation/page/16/" title: "Evaluation · page 16" creator: null creator_url: null creator_x: null url: "https://skillissue.dev/evaluation/page/16/" level: null level_max: null category: null subcategory: null region: null license: null license_scope: null access: null models: null source_date: null date_added: null resource_kind: null verification_basis: null access_evidence: null domain_top: null domain_sub: null last_verified: null --- # Evaluation Measure model capabilities and application behavior with explicit criteria. - [Benchmarks And Leaderboards](/evaluation/benchmarks-and-leaderboards/index.md) (512) - [Eval Harnesses](/evaluation/eval-harnesses/index.md) (3) - [Application Evals](/evaluation/application-evals/index.md) (2) - [Llm As Judge](/evaluation/llm-as-judge/index.md) (1) - [Agent Evals](/evaluation/agent-evals/index.md) (1) [Beginner](/evaluation/beginner/index.md) [Intermediate](/evaluation/intermediate/index.md) [Advanced](/evaluation/advanced/index.md) [Expert](/evaluation/expert/index.md) 519 resources · page 16 of 17 - ### [J-AISI annual report 2025](/evaluation/benchmarks-and-leaderboards/j-aisi-annual-report-2025/index.md) Creator not supplied; see original source. Japan AI Safety Institute 2025 annual activity report (PDF, Japanese). Advanced–Expert · unknown · unknown resource · ja - ### [Korea AISI 5th AI Safety Forecast Report](/evaluation/benchmarks-and-leaderboards/korea-aisi-5th-ai-safety-forecast-report-2026-09-03/index.md) Creator not supplied; see original source. Korea AI Safety Institute 5th AI Safety Forecast Report covering May–July 2026. Advanced–Expert · unknown · unknown resource · ko - ### [La Leaderboard paper](/evaluation/benchmarks-and-leaderboards/la-leaderboard-paper/index.md) Creator not supplied; see original source. arXiv paper (2025) introducing La Leaderboard, an LLM leaderboard for Spanish varieties and languages of Spain and Latin America. Advanced–Expert · unknown · unknown resource · es|ca|eu|gl - ### [LabSafety Bench](/evaluation/benchmarks-and-leaderboards/labsafety-bench/index.md) Creator not supplied; see original source. Nature Machine Intelligence paper introducing LabSafety Bench for LLM safety risks in scientific labs. Advanced · unknown · unknown resource · en - ### [METR experienced-developer productivity experiment](/evaluation/benchmarks-and-leaderboards/metr-experienced-developer-productivity-experiment/index.md) Creator not supplied; see original source. METR study of how early-2025 AI tools affected experienced open-source developers' productivity. Advanced · unknown · unknown resource · en - ### [MLflow documentation](/evaluation/benchmarks-and-leaderboards/mlflow-documentation/index.md) Creator not supplied; see original source. MLflow documentation: experiment tracking, model registry, GenAI tracing, evaluation and deployment. Advanced · unknown · unknown resource · en - ### [MME](/evaluation/benchmarks-and-leaderboards/mme/index.md) Creator not supplied; see original source. arXiv paper (2023) introducing MME, a benchmark for perception and cognition in multimodal LLMs. Advanced · unknown · unknown resource · en - ### [Model Cards for Model Reporting](/evaluation/benchmarks-and-leaderboards/model-cards-for-model-reporting/index.md) Creator not supplied; see original source. Google Research publication page for Model Cards for Model Reporting, a standard for documenting trained models. Advanced · unknown · unknown resource · en - ### [MoleculeNet](/evaluation/benchmarks-and-leaderboards/moleculenet/index.md) Creator not supplied; see original source. arXiv paper (2017) "MoleculeNet: A Benchmark for Molecular Machine Learning", introducing the MoleculeNet benchmark datasets for molecular property prediction. Advanced · unknown · unknown resource · en - ### [Open Ko-LLM Leaderboard2 paper](/evaluation/benchmarks-and-leaderboards/open-ko-llm-leaderboard2-paper/index.md) Creator not supplied; see original source. arXiv paper (2024) describing Open Ko-LLM Leaderboard2 for evaluating Korean LLMs. Advanced–Expert · unknown · unknown resource · ko|en - ### [simple-evals](/evaluation/eval-harnesses/openai-simple-evals/index.md) by [OpenAI](https://github.com/openai/simple-evals) Read reference evaluation code with an explicit maintenance caveat. Advanced stale · West · open resource · en - ### [Introducing TRAIL: A Benchmark for Agentic Evaluation (Patronus AI)](/evaluation/benchmarks-and-leaderboards/patronus-trail-benchmark-for-agentic-evaluation-2025-06-05/index.md) Creator not supplied; see original source. Patronus AI post introducing TRAIL, a benchmark of annotated agent traces for detecting reasoning, planning and execution errors. Advanced · unknown · unknown resource · en - ### [Quantifying infrastructure noise in agentic coding evals](/evaluation/benchmarks-and-leaderboards/quantifying-infrastructure-noise-in-agentic-coding-evals/index.md) Creator not supplied; see original source. Anthropic engineering post quantifying how infrastructure noise affects agentic coding evaluation scores. Advanced · unknown · unknown resource · en - ### [Safetywashing](/evaluation/benchmarks-and-leaderboards/safetywashing/index.md) Creator not supplied; see original source. arXiv paper (2024) "Safetywashing", testing whether AI safety benchmarks measure safety or general capability. Advanced · unknown · unknown resource · en - ### [SEA-HELM paper](/evaluation/benchmarks-and-leaderboards/sea-helm-paper/index.md) Creator not supplied; see original source. arXiv paper (2025) introducing SEA-HELM, holistic evaluation of language models on Southeast Asian languages. Advanced–Expert · unknown · unknown resource · en - ### [τ-Bench: Benchmarking AI agents for the real-world (Sierra)](/evaluation/benchmarks-and-leaderboards/sierra-bench-2024-06-20/index.md) Creator not supplied; see original source. Sierra post introducing τ-bench, a benchmark of tool-using agents talking to simulated users, with the pass^k reliability metric. Advanced · unknown · unknown resource · en - ### [SUPERB](/evaluation/benchmarks-and-leaderboards/superb/index.md) Creator not supplied; see original source. arXiv paper (2021) "SUPERB: Speech processing Universal PERformance Benchmark", introducing SUPERB, a benchmark of speech representations across many speech tasks. Advanced · unknown · unknown resource · en - ### [SWE-Bench Pro Verified](/evaluation/benchmarks-and-leaderboards/swe-bench-pro-verified-audit/index.md) Creator not supplied; see original source. arXiv paper (2026) introducing SWE-Bench Pro Verified, an audited software-engineering benchmark that removes leakage and flawed tasks. Advanced · unknown · unknown resource · en - ### [The Leaderboard Illusion](/evaluation/benchmarks-and-leaderboards/the-leaderboard-illusion/index.md) Creator not supplied; see original source. arXiv paper (2025) "The Leaderboard Illusion", auditing how private testing and data access distort Chatbot Arena rankings. Advanced · unknown · unknown resource · en - ### [Tutor CoPilot: a human-AI approach to real-time expertise](/evaluation/benchmarks-and-leaderboards/tutor-copilot-a-human-ai-approach-to-real-time-expertise/index.md) Creator not supplied; see original source. arXiv paper (2024) reporting a randomized field study of Tutor CoPilot, an AI assistant for human tutors. Advanced · unknown · unknown resource · en - ### [Inspect](/evaluation/eval-harnesses/uk-ai-security-institute-inspect/index.md) by [UK AI Security Institute and Meridian Labs](https://inspect.aisi.org.uk/) Compose evaluation datasets, agents, tools and scorers. Advanced · West · closed resource · en - ### [UNLP 2026 Ukrainian NLP workshop paper](/evaluation/benchmarks-and-leaderboards/unlp-2026-ukrainian-nlp-workshop-paper/index.md) Creator not supplied; see original source. UNLP 2026 workshop paper (PDF) on Ukrainian NLP evaluation. Advanced–Expert · unknown · unknown resource · uk|en - ### [3D Arena paper](/evaluation/benchmarks-and-leaderboards/3d-arena-paper/index.md) Creator not supplied; see original source. arXiv paper (2025) introducing 3D Arena, an open pairwise-preference platform for evaluating generative 3D models. Expert · unknown · unknown resource · en - ### [BEHAVIOR 2025 1st-place report](/evaluation/benchmarks-and-leaderboards/behavior-2025-1st-place-report/index.md) Creator not supplied; see original source. arXiv PDF (2025) of the first-place report from the NeurIPS 2025 BEHAVIOR household robotics challenge. Expert · unknown · unknown resource · en - ### [A framework for evaluating the chemical knowledge and reasoning abilities of large language models (ChemBench)](/evaluation/benchmarks-and-leaderboards/chembench-paper-nature-chemistry-2025/index.md) Creator not supplied; see original source. ChemBench paper (Nature Chemistry 2025, PMC full text): framework evaluating chemical knowledge and reasoning of LLMs against chemists. Expert · unknown · unknown resource · en - ### [EvalScope τ-bench docs (ModelScope/Alibaba)](/evaluation/benchmarks-and-leaderboards/evalscope-bench-docs-modelscope-alibaba/index.md) Creator not supplied; see original source. EvalScope documentation on τ-bench: running the conversational agent benchmark with Alibaba's ModelScope evaluation framework. Expert · unknown · unknown resource · en|zh - ### [FinanceBench: A New Benchmark for Financial Question Answering](/evaluation/benchmarks-and-leaderboards/financebench-paper-arxiv-2311-11944/index.md) Creator not supplied; see original source. arXiv paper (2023) introducing FinanceBench, a benchmark for financial question answering over company filings. Expert · unknown · unknown resource · en - ### [FrontierMath paper](/evaluation/benchmarks-and-leaderboards/frontiermath-paper/index.md) Creator not supplied; see original source. arXiv paper (2024) introducing FrontierMath, a benchmark of unpublished advanced mathematics problems. Expert · unknown · unknown resource · en - ### [MASK paper](/evaluation/benchmarks-and-leaderboards/mask-paper/index.md) Creator not supplied; see original source. arXiv PDF (2025) of the MASK benchmark paper on measuring honesty separately from accuracy in LLMs. Expert · unknown · unknown resource · en - ### [Medical VLM contamination audit](/evaluation/benchmarks-and-leaderboards/medical-vlm-contamination-audit/index.md) Creator not supplied; see original source. arXiv paper (2026) auditing pretraining contamination of public medical vision-language benchmarks. Expert · unknown · unknown resource · en - ### [MMMU-Pro paper](/evaluation/benchmarks-and-leaderboards/mmmu-pro-paper/index.md) Creator not supplied; see original source. arXiv HTML version of the MMMU-Pro paper (2024), a harder multimodal multi-discipline benchmark. Expert · unknown · unknown resource · en - ### [Music Arena paper](/evaluation/benchmarks-and-leaderboards/music-arena-paper/index.md) Creator not supplied; see original source. arXiv paper (2025) introducing Music Arena, live human-preference evaluation for text-to-music models. Expert · unknown · unknown resource · en [Previous page](/evaluation/page/15/index.md) [Next page](/evaluation/page/17/index.md) [Plain-text Markdown copy](/evaluation/page/16/index.md)