--- id: "/tags/agent-coding-evals/page/3/" title: "Agent Coding Evals · page 3" url: "https://skillissue.dev/tags/agent-coding-evals/page/3/" --- # Agent Coding Evals [Beginner](/levels/beginner/index.md) [Intermediate](/levels/intermediate/index.md) [Advanced](/levels/advanced/index.md) [Expert](/levels/expert/index.md) 85 resources · page 3 of 3 - ### [Berget AI: AI in Swedish](/evaluation/benchmarks-and-leaderboards/berget-ai-ai-in-swedish/index.md) by [Berget AI](https://berget.ai/) Berget AI post on choosing models for Swedish-language applications, pointing to EuroEval and Scandinavian embedding benchmarks. Intermediate learning_resource blog Sweden sv|en - ### [Demystifying evals for AI agents](/evaluation/benchmarks-and-leaderboards/demystifying-evals-for-ai-agents/index.md) by [Anthropic](https://anthropic.com/) Anthropic post on designing evals for AI agents, combining automated tests, graders and human review. Intermediate learning_resource guide US en - ### [GAIA paper](/evaluation/benchmarks-and-leaderboards/gaia-paper/index.md) by [Grégoire Mialon et al. (Meta AI and Hugging Face)](https://ai.meta.com/) arXiv paper (2023) introducing GAIA, a benchmark of real-world questions for general AI assistants. Intermediate learning_resource paper France/US en - ### [Announcing Agent Control (Galileo)](/evaluation/benchmarks-and-leaderboards/galileo-agent-control-open-source-control-plane/index.md) by [Galileo](https://galileo.ai/) Galileo post announcing Agent Control, an open-source control plane for runtime guardrails on AI agents. Intermediate learning_resource blog US en - ### [Launching Agent Leaderboard v2 (Galileo)](/evaluation/benchmarks-and-leaderboards/galileo-agent-leaderboard-v2/index.md) by [Galileo](https://galileo.ai/) Galileo post introducing Agent Leaderboard v2, which evaluates LLMs as enterprise customer-support agents. Intermediate learning_resource blog US en - ### [Best Practices to Navigate the Complexities of Evaluating AI Agents (Galileo)](/evaluation/benchmarks-and-leaderboards/galileo-best-practices-for-evaluating-ai-agents/index.md) by [Galileo](https://galileo.ai/) Galileo guide to evaluating AI agents, covering accuracy metrics, action evaluation and production monitoring. Intermediate learning_resource blog US en - ### [The Most Common Questions About Agent Evals, Answered (LangChain)](/evaluation/benchmarks-and-leaderboards/langchain-common-questions-about-agent-evals/index.md) by [LangChain](https://www.langchain.com/) LangChain post answering common questions on agent evals: offline vs online, evaluator types, LLM judges and multi-turn tests. Intermediate learning_resource blog US en - ### [Langfuse blog: Evaluating AI agent skills](/evaluation/benchmarks-and-leaderboards/langfuse-evaluating-ai-agent-skills-2026-02-26/index.md) by [Langfuse](https://langfuse.com) Langfuse blog post on how to evaluate AI agent skills (SKILL.md files) with traces and evals. Intermediate learning_resource blog Germany/EU en - ### [LiveCodeBench paper](/evaluation/benchmarks-and-leaderboards/livecodebench-paper/index.md) by [Naman Jain et al. (Cornell University)](https://www.cornell.edu) arXiv paper (2024) introducing LiveCodeBench, a contamination-free code benchmark built from recent contest problems. Intermediate learning_resource paper US en - ### [METR time-horizon paper](/evaluation/benchmarks-and-leaderboards/metr-time-horizon-paper/index.md) by [Thomas Kwa et al. (METR)](https://metr.org/) arXiv paper (2025) by METR measuring AI ability to complete long software tasks as a time horizon. Intermediate learning_resource paper US en - ### [promptfoo docs: Evaluate coding agents](/evaluation/benchmarks-and-leaderboards/promptfoo-evaluate-coding-agents/index.md) by [promptfoo](https://promptfoo.dev) Promptfoo guide to evaluating coding agents with its open-source eval harness. Intermediate learning_resource guide US en - ### [SWE-bench paper](/evaluation/benchmarks-and-leaderboards/swe-bench-paper/index.md) by [Carlos E. Jimenez et al. (Princeton University)](https://www.princeton.edu/) arXiv paper (2023) introducing SWE-bench, evaluating language models on resolving real GitHub issues. Intermediate learning_resource paper US en - ### [The SWE-Bench Illusion](/evaluation/benchmarks-and-leaderboards/the-swe-bench-illusion/index.md) by [Shanchao Liang et al. (Purdue University)](https://www.purdue.edu/) arXiv paper (2025) "The SWE-Bench Illusion", testing whether LLMs memorise SWE-bench instead of reasoning. Intermediate learning_resource paper US (Microsoft) en - ### [Agentic Benchmark Checklist (ABC)](/evaluation/benchmarks-and-leaderboards/agentic-benchmark-checklist-abc/index.md) by [Yuxuan Zhu et al.](https://orcid.org/0009-0001-6738-8930) arXiv paper (2025) proposing the Agentic Benchmark Checklist, guidelines for building valid agentic benchmarks. Advanced learning_resource paper US (UIUC et al.) en - ### [Arize Phoenix docs: Evaluating multi-agent systems](/evaluation/benchmarks-and-leaderboards/arize-phoenix-evaluating-multi-agent-systems/index.md) by [Arize AI](https://www.arize.com) Arize Phoenix documentation on evaluating multi-agent systems using traces of each agent's steps. Advanced learning_resource docs US en - ### [Coding-agent leaderboard resolution audit](/evaluation/benchmarks-and-leaderboards/coding-agent-leaderboard-resolution-audit/index.md) by [Fengshuo Liu et al. (Imperial College London)](https://www.imperial.ac.uk) arXiv paper (2026) arguing that coding agents have converged so the SWE-bench leaderboard can no longer separate them. Advanced learning_resource paper China / International en - ### [HAE-RAE Evaluation Toolkit / HRET](/evaluation/benchmarks-and-leaderboards/hae-rae-evaluation-toolkit-hret/index.md) by [Hanwool Lee et al.](https://orcid.org/0000-0001-5565-5184) arXiv paper (2025) introducing the HAE-RAE Evaluation Toolkit (HRET), a unified framework for evaluating Korean LLMs. Advanced learning_resource paper South Korea en - ### [Introducing TRAIL: A Benchmark for Agentic Evaluation (Patronus AI)](/evaluation/benchmarks-and-leaderboards/patronus-trail-benchmark-for-agentic-evaluation-2025-06-05/index.md) by [Patronus AI](https://www.patronus.ai/) Patronus AI post introducing TRAIL, a benchmark of annotated agent traces for detecting reasoning, planning and execution errors. Advanced learning_resource blog US en - ### [Quantifying infrastructure noise in agentic coding evals](/evaluation/benchmarks-and-leaderboards/quantifying-infrastructure-noise-in-agentic-coding-evals/index.md) by [Anthropic](https://anthropic.com/) Anthropic engineering post quantifying how infrastructure noise affects agentic coding evaluation scores. Advanced learning_resource paper US en - ### [SWE-Bench Pro Verified](/evaluation/benchmarks-and-leaderboards/swe-bench-pro-verified-audit/index.md) by [Pujun Zheng et al. (Shanghai Artificial Intelligence Laboratory)](https://www.shlab.org.cn) arXiv paper (2026) introducing SWE-Bench Pro Verified, an audited software-engineering benchmark that removes leakage and flawed tasks. Advanced learning_resource paper China / International en - ### [EvalScope τ-bench docs (ModelScope/Alibaba)](/evaluation/benchmarks-and-leaderboards/evalscope-bench-docs-modelscope-alibaba/index.md) by [ModelScope (Alibaba)](https://www.modelscope.cn/) EvalScope documentation on τ-bench: running the conversational agent benchmark with Alibaba's ModelScope evaluation framework. Expert learning_resource docs China en|zh [Previous page](/tags/agent-coding-evals/page/2/index.md) [Plain-text Markdown copy](/tags/agent-coding-evals/page/3/index.md)