--- id: "/evaluation/page/15/" title: "Evaluation · page 15" creator: null creator_url: null creator_x: null url: "https://skillissue.dev/evaluation/page/15/" level: null level_max: null category: null subcategory: null region: null license: null license_scope: null access: null models: null source_date: null date_added: null resource_kind: null verification_basis: null access_evidence: null domain_top: null domain_sub: null last_verified: null --- # Evaluation Measure model capabilities and application behavior with explicit criteria. - [Benchmarks And Leaderboards](/evaluation/benchmarks-and-leaderboards/index.md) (512) - [Eval Harnesses](/evaluation/eval-harnesses/index.md) (3) - [Application Evals](/evaluation/application-evals/index.md) (2) - [Llm As Judge](/evaluation/llm-as-judge/index.md) (1) - [Agent Evals](/evaluation/agent-evals/index.md) (1) [Beginner](/evaluation/beginner/index.md) [Intermediate](/evaluation/intermediate/index.md) [Advanced](/evaluation/advanced/index.md) [Expert](/evaluation/expert/index.md) 519 resources · page 15 of 17 - ### [LiveBench paper](/evaluation/benchmarks-and-leaderboards/livebench-paper/index.md) Creator not supplied; see original source. arXiv paper (2024) introducing LiveBench, a contamination-limited LLM benchmark with regularly refreshed questions. Intermediate · unknown · unknown resource · en - ### [LiveCodeBench paper](/evaluation/benchmarks-and-leaderboards/livecodebench-paper/index.md) Creator not supplied; see original source. arXiv paper (2024) introducing LiveCodeBench, a contamination-free code benchmark built from recent contest problems. Intermediate · unknown · unknown resource · en - ### [Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity](/evaluation/benchmarks-and-leaderboards/metr-developer-productivity-rct-2025/index.md) Creator not supplied; see original source. arXiv paper (2025) by METR reporting a randomized trial of early-2025 AI tools' effect on experienced open-source developers' speed. Intermediate · unknown · unknown resource · en - ### [METR time-horizon paper](/evaluation/benchmarks-and-leaderboards/metr-time-horizon-paper/index.md) Creator not supplied; see original source. arXiv paper (2025) by METR measuring AI ability to complete long software tasks as a time horizon. Intermediate · unknown · unknown resource · en - ### [The Open Arabic LLM Leaderboard 2](/evaluation/benchmarks-and-leaderboards/oall-v2-launch-blog/index.md) Creator not supplied; see original source. Hugging Face post launching the Open Arabic LLM Leaderboard 2 with native Arabic benchmarks. Intermediate · unknown · unknown resource · ar|en - ### [OECD AI Incidents Monitor](/evaluation/benchmarks-and-leaderboards/oecd-ai-incidents-monitor/index.md) Creator not supplied; see original source. OECD AI Incidents Monitor: database of AI incidents and hazards compiled from news reports. Intermediate · unknown · unknown resource · en - ### [promptfoo docs: Evaluate coding agents](/evaluation/benchmarks-and-leaderboards/promptfoo-evaluate-coding-agents/index.md) Creator not supplied; see original source. Promptfoo guide to evaluating coding agents with its open-source eval harness. Intermediate · unknown · unknown resource · en - ### [Sarvam-M multilingual language model (Sarvam AI)](/evaluation/benchmarks-and-leaderboards/sarvam-m-blog/index.md) Creator not supplied; see original source. Sarvam AI technical post on Sarvam-M, a hybrid reasoning model for Indian languages, with post-training and inference details. Intermediate · unknown · unknown resource · en - ### [SWE-bench paper](/evaluation/benchmarks-and-leaderboards/swe-bench-paper/index.md) Creator not supplied; see original source. arXiv paper (2023) introducing SWE-bench, evaluating language models on resolving real GitHub issues. Intermediate · unknown · unknown resource · en - ### [SWE-bench Leaderboards](/evaluation/agent-evals/swe-bench-team-swe-bench-leaderboards/index.md) by [SWE-bench team (Princeton/Stanford)](https://www.swebench.com/) Understand coding-agent scores in terms of resolved repository tasks. Intermediate · West · closed resource · en - ### [The SWE-Bench Illusion](/evaluation/benchmarks-and-leaderboards/the-swe-bench-illusion/index.md) Creator not supplied; see original source. arXiv paper (2025) "The SWE-Bench Illusion", testing whether LLMs memorise SWE-bench instead of reasoning. Intermediate · unknown · unknown resource · en - ### [W&B Weave documentation](/evaluation/benchmarks-and-leaderboards/weights-biases-weave/index.md) Creator not supplied; see original source. W&B Weave documentation, now hosted by CoreWeave: tracing, testing and evaluating LLM applications. Intermediate · unknown · unknown resource · en - ### [What We've Learned From A Year of Building with LLMs](/evaluation/benchmarks-and-leaderboards/what-we-ve-learned-from-a-year-of-building-with-llms/index.md) Creator not supplied; see original source. Applied LLMs guide by Yan, Bischof, Frye, Husain, Liu and Shankar: tactical, operational and strategic lessons from building LLM products. Intermediate · unknown · unknown resource · en - ### [Wikipedia: AI safety institutes](/evaluation/benchmarks-and-leaderboards/wikipedia-ai-safety-institutes/index.md) Creator not supplied; see original source. Wikipedia article on AI safety institutes: state-backed bodies that evaluate AI safety, listed by country. Intermediate–Advanced · unknown · unknown resource · en - ### [Accuracy incentives and hallucinations](/evaluation/benchmarks-and-leaderboards/accuracy-incentives-and-hallucinations/index.md) Creator not supplied; see original source. Nature paper (2026) showing that accuracy-only evaluation of LLMs incentivises hallucination. Advanced · unknown · unknown resource · en - ### [Agentic Benchmark Checklist (ABC)](/evaluation/benchmarks-and-leaderboards/agentic-benchmark-checklist-abc/index.md) Creator not supplied; see original source. arXiv paper (2025) proposing the Agentic Benchmark Checklist, guidelines for building valid agentic benchmarks. Advanced · unknown · unknown resource · en - ### [ARC Prize 2025 Technical Report](/evaluation/benchmarks-and-leaderboards/arc-prize-2025-technical-report/index.md) Creator not supplied; see original source. arXiv paper (2026) ARC Prize 2025 technical report on ARC-AGI-2 results and winning approaches. Advanced · unknown · unknown resource · en - ### [Arize Phoenix docs: Evaluating multi-agent systems](/evaluation/benchmarks-and-leaderboards/arize-phoenix-evaluating-multi-agent-systems/index.md) Creator not supplied; see original source. Arize Phoenix documentation on evaluating multi-agent systems using traces of each agent's steps. Advanced · unknown · unknown resource · en - ### [Benchmark-based assessment of Indian foundation models](/evaluation/benchmarks-and-leaderboards/benchmark-based-assessment-of-indian-foundation-models/index.md) Creator not supplied; see original source. arXiv paper (2026) comparing publicly benchmarked Indian foundation models across benchmarks. Advanced–Expert · unknown · unknown resource · en - ### [C-Eval paper](/evaluation/benchmarks-and-leaderboards/c-eval-paper/index.md) Creator not supplied; see original source. arXiv paper (2023) introducing C-Eval, a multi-level, multi-discipline Chinese evaluation suite for foundation models. Advanced–Expert · unknown · unknown resource · en - ### [CLARIN-PT-LDB paper](/evaluation/benchmarks-and-leaderboards/clarin-pt-ldb-paper/index.md) Creator not supplied; see original source. arXiv paper (2026) introducing CLARIN-PT-LDB, an open LLM leaderboard for European Portuguese. Advanced–Expert · unknown · unknown resource · pt - ### [Coding-agent leaderboard resolution audit](/evaluation/benchmarks-and-leaderboards/coding-agent-leaderboard-resolution-audit/index.md) Creator not supplied; see original source. arXiv paper (2026) arguing that coding agents have converged so the SWE-bench leaderboard can no longer separate them. Advanced · unknown · unknown resource · en - ### [Cross-Context Verification: Hierarchical Detection of Benchmark Contamination](/evaluation/benchmarks-and-leaderboards/cross-context-verification-contamination-detection/index.md) Creator not supplied; see original source. arXiv paper (2026) introducing Cross-Context Verification, a hierarchical method for detecting benchmark contamination. Advanced · unknown · unknown resource · en - ### [DuetBench (Decagon)](/evaluation/benchmarks-and-leaderboards/decagon-duetbench/index.md) Creator not supplied; see original source. Decagon post introducing DuetBench, a benchmark for self-improving customer-service agents. Advanced · unknown · unknown resource · en - ### [Evaluating the Effectiveness of LLM-Evaluators (LLM-as-Judge)](/evaluation/llm-as-judge/eugene-yan-evaluating-the-effectiveness-of-llm-evaluators/index.md) by [Eugene Yan](https://eugeneyan.com/writing/llm-evaluators/) [@eugeneyan](https://twitter.com/eugeneyan) Compare model-based graders against the judgments they must approximate. Advanced · West · closed resource · en - ### [Fairlearn](/evaluation/benchmarks-and-leaderboards/fairlearn/index.md) Creator not supplied; see original source. Fairlearn: open-source toolkit and guides for assessing and mitigating fairness issues in ML models. Advanced · unknown · unknown resource · en - ### [FlagEval Findings Report (LRMs)](/evaluation/benchmarks-and-leaderboards/flageval-findings-report-lrms/index.md) Creator not supplied; see original source. arXiv paper (2025) FlagEval findings report: BAAI's preliminary evaluation of large reasoning models. Advanced–Expert · unknown · unknown resource · en - ### [FlagEvalMM](/evaluation/benchmarks-and-leaderboards/flagevalmm/index.md) Creator not supplied; see original source. arXiv paper (2025) introducing FlagEvalMM, BAAI's framework for evaluating multimodal models. Advanced–Expert · unknown · unknown resource · en - ### [HAE-RAE Evaluation Toolkit / HRET](/evaluation/benchmarks-and-leaderboards/hae-rae-evaluation-toolkit-hret/index.md) Creator not supplied; see original source. arXiv paper (2025) introducing the HAE-RAE Evaluation Toolkit (HRET), a unified framework for evaluating Korean LLMs. Advanced · unknown · unknown resource · en - ### [HLE paper](/evaluation/benchmarks-and-leaderboards/hle-paper/index.md) Creator not supplied; see original source. arXiv paper (2025) introducing Humanity's Last Exam, a benchmark of expert-written questions across academic fields. Advanced · unknown · unknown resource · en - ### [IFEval](/evaluation/benchmarks-and-leaderboards/ifeval/index.md) Creator not supplied; see original source. arXiv paper (2023) introducing IFEval, instruction-following evaluation with automatically verifiable instructions. Advanced · unknown · unknown resource · en - ### [Inspect India Evals](/evaluation/benchmarks-and-leaderboards/inspect-india-evals/index.md) Creator not supplied; see original source. arXiv paper (2026) introducing Inspect India Evals, an open framework of Indian benchmarks with a fairness index. Advanced–Expert · unknown · unknown resource · mul [Previous page](/evaluation/page/14/index.md) [Next page](/evaluation/page/16/index.md) [Plain-text Markdown copy](/evaluation/page/15/index.md)