--- id: "/evaluation/page/14/" title: "Evaluation · page 14" creator: null creator_url: null creator_x: null url: "https://skillissue.dev/evaluation/page/14/" level: null level_max: null category: null subcategory: null region: null license: null license_scope: null access: null models: null source_date: null date_added: null resource_kind: null verification_basis: null access_evidence: null domain_top: null domain_sub: null last_verified: null --- # Evaluation Measure model capabilities and application behavior with explicit criteria. - [Benchmarks And Leaderboards](/evaluation/benchmarks-and-leaderboards/index.md) (512) - [Eval Harnesses](/evaluation/eval-harnesses/index.md) (3) - [Application Evals](/evaluation/application-evals/index.md) (2) - [Llm As Judge](/evaluation/llm-as-judge/index.md) (1) - [Agent Evals](/evaluation/agent-evals/index.md) (1) [Beginner](/evaluation/beginner/index.md) [Intermediate](/evaluation/intermediate/index.md) [Advanced](/evaluation/advanced/index.md) [Expert](/evaluation/expert/index.md) 519 resources · page 14 of 17 - ### [Langfuse joins ClickHouse](/evaluation/benchmarks-and-leaderboards/langfuse-joining-clickhouse/index.md) Creator not supplied; see original source. Langfuse post announcing the company joined ClickHouse and describing plans for the open-source project. Beginner · unknown · unknown resource · en - ### [Latent Space: SWE-bench dead](/evaluation/benchmarks-and-leaderboards/latent-space-swe-bench-dead/index.md) Creator not supplied; see original source. Latent Space episode with OpenAI's Mia Glaese and Olivia Watkins on retiring SWE-bench Verified and next frontier evals. Beginner · unknown · unknown resource · en - ### [MMLU (classic)](/evaluation/benchmarks-and-leaderboards/mmlu-classic/index.md) Creator not supplied; see original source. arXiv paper (2020) "Measuring Massive Multitask Language Understanding", introducing MMLU, a 57-subject multitask knowledge benchmark. Beginner · unknown · unknown resource · en - ### [BEHAVIOR Challenge Charts the Way Forward for Domestic Robotics (Stanford HAI)](/evaluation/benchmarks-and-leaderboards/stanford-hai-behavior-challenge-article/index.md) Creator not supplied; see original source. Stanford HAI article on the BEHAVIOR Challenge, a household-robotics competition in simulation held at NeurIPS 2025. Beginner · unknown · unknown resource · en - ### [AI Agents That Matter](/evaluation/benchmarks-and-leaderboards/ai-agents-that-matter/index.md) Creator not supplied; see original source. arXiv paper (2024) "AI Agents That Matter", on cost-aware and reproducible evaluation of AI agents. Intermediate · unknown · unknown resource · en - ### [March 2026: Arena Updates](/evaluation/benchmarks-and-leaderboards/arena-march-2026-updates-blog/index.md) Creator not supplied; see original source. Arena blog post on March 2026 updates: new document and video-edit arenas and leaderboard changes. Intermediate · unknown · unknown resource · en - ### [Arize Phoenix docs: LLM-as-a-judge](/evaluation/benchmarks-and-leaderboards/arize-phoenix-llm-as-a-judge/index.md) Creator not supplied; see original source. Arize Phoenix documentation on LLM-as-a-judge evaluation: designing judge prompts and criteria. Intermediate · unknown · unknown resource · en - ### [Berget AI: AI in Swedish](/evaluation/benchmarks-and-leaderboards/berget-ai-ai-in-swedish/index.md) Creator not supplied; see original source. Berget AI post on choosing models for Swedish-language applications, pointing to EuroEval and Scandinavian embedding benchmarks. Intermediate · unknown · unknown resource · sv|en - ### [Braintrust documentation](/evaluation/benchmarks-and-leaderboards/braintrust-documentation/index.md) Creator not supplied; see original source. Braintrust documentation for running experiments, scoring outputs and collecting production feedback on AI apps. Intermediate · unknown · unknown resource · en - ### [Braintrust docs: Evaluate systematically](/evaluation/benchmarks-and-leaderboards/braintrust-evaluate/index.md) Creator not supplied; see original source. Braintrust documentation on systematic evaluation: datasets, scorers and regression detection for AI applications. Intermediate · unknown · unknown resource · en - ### [Can We Trust AI Benchmarks?](/evaluation/benchmarks-and-leaderboards/can-we-trust-ai-benchmarks/index.md) Creator not supplied; see original source. arXiv paper (2025) reviewing about 100 studies on the trustworthiness and limitations of AI benchmarks. Intermediate · unknown · unknown resource · en - ### [Chatbot Arena paper](/evaluation/benchmarks-and-leaderboards/chatbot-arena-paper/index.md) Creator not supplied; see original source. arXiv paper (2024) describing Chatbot Arena, its crowdsourced pairwise voting and Bradley-Terry ranking method. Intermediate · unknown · unknown resource · en - ### [How Some of China's Top AI Thinkers Built Their Own AI Safety Institute (Carnegie)](/evaluation/benchmarks-and-leaderboards/china-ai-safety-development-association-cnaisda-carnegie-analysis/index.md) Creator not supplied; see original source. Carnegie Endowment analysis of the China AI Safety and Development Association and its role in Chinese AI governance. Intermediate · unknown · unknown resource · en - ### [Demystifying evals for AI agents](/evaluation/benchmarks-and-leaderboards/demystifying-evals-for-ai-agents/index.md) Creator not supplied; see original source. Anthropic post on designing evals for AI agents, combining automated tests, graders and human review. Intermediate · unknown · unknown resource · en - ### [SimpleQA Verified: Benchmark review (Epoch AI)](/evaluation/benchmarks-and-leaderboards/epoch-benchmark-review-simpleqa-verified/index.md) Creator not supplied; see original source. Epoch AI review of the SimpleQA Verified benchmark, checking a sample of questions for correctness. Intermediate · unknown · unknown resource · en - ### [Evaluating the Effectiveness of LLM-Evaluators (Eugene Yan)](/evaluation/benchmarks-and-leaderboards/eugene-yan-llm-evaluators/index.md) Creator not supplied; see original source. Eugene Yan's review of LLM-evaluator (LLM-as-judge) research: use cases, techniques, alignment and critiques. Intermediate · unknown · unknown resource · en - ### [GAIA paper](/evaluation/benchmarks-and-leaderboards/gaia-paper/index.md) Creator not supplied; see original source. arXiv paper (2023) introducing GAIA, a benchmark of real-world questions for general AI assistants. Intermediate · unknown · unknown resource · en - ### [Announcing Agent Control (Galileo)](/evaluation/benchmarks-and-leaderboards/galileo-agent-control-open-source-control-plane/index.md) Creator not supplied; see original source. Galileo post announcing Agent Control, an open-source control plane for runtime guardrails on AI agents. Intermediate · unknown · unknown resource · en - ### [Launching Agent Leaderboard v2 (Galileo)](/evaluation/benchmarks-and-leaderboards/galileo-agent-leaderboard-v2/index.md) Creator not supplied; see original source. Galileo post introducing Agent Leaderboard v2, which evaluates LLMs as enterprise customer-support agents. Intermediate · unknown · unknown resource · en - ### [Best Practices to Navigate the Complexities of Evaluating AI Agents (Galileo)](/evaluation/benchmarks-and-leaderboards/galileo-best-practices-for-evaluating-ai-agents/index.md) Creator not supplied; see original source. Galileo guide to evaluating AI agents, covering accuracy metrics, action evaluation and production monitoring. Intermediate · unknown · unknown resource · en - ### [SberDevices blog on Habr](/evaluation/benchmarks-and-leaderboards/habr-sberdevices-blog/index.md) Creator not supplied; see original source. SberDevices company blog on Habr with Russian-language technical posts on GigaChat models and evaluation. Intermediate · unknown · unknown resource · ru - ### [AI Evals FAQ](/evaluation/application-evals/hamel-husain-ai-evals-faq/index.md) by [Hamel Husain and Shreya Shankar](https://hamel.dev/blog/posts/evals-faq/) Practical answers about error analysis, annotation and reliable evaluators. Intermediate · West · closed resource · en - ### [Your AI Product Needs Evals](/evaluation/application-evals/hamel-husain-your-ai-product-needs-evals/index.md) by [Hamel Husain](https://hamel.dev/blog/posts/evals/) Build a feedback loop from real failures, not general benchmark scores. Intermediate · West · closed resource · en - ### [Using LLM-as-a-Judge For Evaluation: A Complete Guide (Hamel Husain)](/evaluation/benchmarks-and-leaderboards/hamel-llm-as-a-judge-guide/index.md) Creator not supplied; see original source. Hamel Husain's guide to building an LLM judge and validating it against domain-expert labels. Intermediate · unknown · unknown resource · en - ### [HF blog: Hebrew leaderboard](/evaluation/benchmarks-and-leaderboards/hf-blog-hebrew-leaderboard/index.md) Creator not supplied; see original source. Hugging Face blog source file announcing the Hebrew LLM leaderboard and its evaluation tasks. Intermediate · unknown · unknown resource · en - ### [AI Evals: Everything You Need to Know (Hamel Husain)](/evaluation/benchmarks-and-leaderboards/husain-shankar-ai-evals-everything-you-need-to-know/index.md) Creator not supplied; see original source. Hamel Husain and Shreya Shankar's AI evals FAQ for engineers and PMs: testing AI systems, error analysis, RAG and agent evals. Intermediate · unknown · unknown resource · en - ### [Announcing Evals and Releases for Fin (Intercom)](/evaluation/benchmarks-and-leaderboards/intercom-announcing-evals-and-releases-for-fin-2026-08-13/index.md) Creator not supplied; see original source. Intercom post announcing Evals and Releases for its Fin support agent, for testing before, during and after launch. Intermediate · unknown · unknown resource · en - ### [Judging LLM-as-a-Judge (MT-Bench)](/evaluation/benchmarks-and-leaderboards/judging-llm-as-a-judge-mt-bench/index.md) Creator not supplied; see original source. arXiv paper (2023) on using strong LLMs as judges, with MT-Bench, Chatbot Arena and analysis of judge biases. Intermediate · unknown · unknown resource · en - ### [The Most Common Questions About Agent Evals, Answered (LangChain)](/evaluation/benchmarks-and-leaderboards/langchain-common-questions-about-agent-evals/index.md) Creator not supplied; see original source. LangChain post answering common questions on agent evals: offline vs online, evaluator types, LLM judges and multi-turn tests. Intermediate · unknown · unknown resource · en - ### [Langfuse blog: Evaluating AI agent skills](/evaluation/benchmarks-and-leaderboards/langfuse-evaluating-ai-agent-skills-2026-02-26/index.md) Creator not supplied; see original source. Langfuse blog post on how to evaluate AI agent skills (SKILL.md files) with traces and evals. Intermediate · unknown · unknown resource · en - ### [LangSmith docs: Evaluation concepts](/evaluation/benchmarks-and-leaderboards/langsmith-evaluation-concepts/index.md) Creator not supplied; see original source. LangSmith documentation source on evaluation concepts: datasets, evaluators, offline and online evals. Intermediate · unknown · unknown resource · en - ### [Legal Technology Insider: Vals report missing the market leaders](/evaluation/benchmarks-and-leaderboards/legal-technology-insider-vals-report-missing-the-market-leaders/index.md) Creator not supplied; see original source. Legal Technology Insider article on the Vals AI legal research benchmark report, noting which major vendors did not take part. Intermediate · unknown · unknown resource · en [Previous page](/evaluation/page/13/index.md) [Next page](/evaluation/page/15/index.md) [Plain-text Markdown copy](/evaluation/page/14/index.md)