--- id: "/tags/agent-coding-evals/" title: "Agent Coding Evals" creator: null creator_url: null creator_x: null url: "https://skillissue.dev/tags/agent-coding-evals/" level: null level_max: null category: null subcategory: null region: null license: null license_scope: null access: null models: null source_date: null date_added: null resource_kind: null verification_basis: null access_evidence: null domain_top: null domain_sub: null last_verified: null --- # Agent Coding Evals [Beginner](/levels/beginner/index.md) [Intermediate](/levels/intermediate/index.md) [Advanced](/levels/advanced/index.md) [Expert](/levels/expert/index.md) 85 resources · page 1 of 3 - ### [AA-3D](/evaluation/benchmarks-and-leaderboards/aa-3d/index.md) Creator not supplied; see original source. Artificial Analysis AA-3D benchmark comparing how well AI models build interactive 3D experiences, judged by blind panel comparisons. Not applicable (entity) · unknown · unknown resource · en - ### [Artificial Analysis Intelligence Benchmarking methodology](/evaluation/benchmarks-and-leaderboards/aa-agentic-evals-gdpval-aa-aa-briefcase-automationbench-aa-gdp-pdf/index.md) Creator not supplied; see original source. Artificial Analysis methodology page describing the benchmarks, weights and scoring behind its Intelligence Index for LLMs. Not applicable (entity) · unknown · unknown resource · en - ### [AgentBench](/evaluation/benchmarks-and-leaderboards/agentbench/index.md) Creator not supplied; see original source. AgentBench GitHub repo (Tsinghua): benchmark evaluating LLMs as agents across multiple interactive environments. Not applicable (entity) · unknown · unknown resource · en - ### [AgentBoard](/evaluation/benchmarks-and-leaderboards/agentboard/index.md) Creator not supplied; see original source. HKUST AgentBoard GitHub repo: analytical evaluation board for multi-turn LLM agents with progress tracking. Not applicable (entity) · unknown · unknown resource · en - ### [ethz-spylab/agentdojo](/evaluation/benchmarks-and-leaderboards/agentdojo/index.md) Creator not supplied; see original source. ETH SPY Lab AgentDojo GitHub repo: environment evaluating prompt injection attacks and defences for LLM agents. Not applicable (entity) · unknown · unknown resource · en - ### [ai-safety-institute/AgentHarm](/evaluation/benchmarks-and-leaderboards/agentharm/index.md) Creator not supplied; see original source. UK AISI AgentHarm dataset on Hugging Face: harmful agentic tasks for measuring whether LLM agents comply with misuse requests. Not applicable (entity) · unknown · unknown resource · en - ### [AI Verify Foundation / Project Moonshot (Singapore)](/evaluation/benchmarks-and-leaderboards/ai-verify-foundation-project-moonshot-singapore/index.md) Creator not supplied; see original source. Singapore's AI Verify Foundation site, home of the AI Verify testing toolkit and Project Moonshot for LLM red-teaming and benchmarking. Not applicable (entity) · unknown · unknown resource · en - ### [Aider polyglot leaderboard](/evaluation/benchmarks-and-leaderboards/aider-polyglot-leaderboard/index.md) Creator not supplied; see original source. Aider LLM leaderboards: model scores on polyglot code-editing exercises with cost. Not applicable (entity) · unknown · unknown resource · en - ### [AndroidWorld](/evaluation/benchmarks-and-leaderboards/androidworld/index.md) Creator not supplied; see original source. Google Research AndroidWorld GitHub repo: environment and benchmark for autonomous agents on Android. Not applicable (entity) · unknown · unknown resource · en - ### [AppWorld](/evaluation/benchmarks-and-leaderboards/appworld/index.md) Creator not supplied; see original source. AppWorld benchmark: a simulated world of everyday apps and APIs for testing coding agents on stateful multi-app tasks. Not applicable (entity) · unknown · unknown resource · en - ### [Arena Raises $200M Series B at $3.1B Valuation](/evaluation/benchmarks-and-leaderboards/arena-alignment-index-series-b-post/index.md) Creator not supplied; see original source. Arena blog post on its Series B and the launch of its AI Alignment Index, measuring unsafe agent behaviour on real traces. Not applicable (entity) · unknown · unknown resource · en - ### [augment-swebench-agent](/evaluation/benchmarks-and-leaderboards/augment-swe-bench-agent/index.md) Creator not supplied; see original source. Augment augment-swebench-agent GitHub repo: open implementation of Augment's SWE-bench Verified agent. Not applicable (entity) · unknown · unknown resource · en - ### [sierra-research/tau-bench](/evaluation/benchmarks-and-leaderboards/bench/index.md) Creator not supplied; see original source. Sierra τ-bench GitHub repo: code and data for evaluating tool-using agents in customer service domains. Not applicable (entity) · unknown · unknown resource · en - ### [Berkeley Function-Calling Leaderboard](/evaluation/benchmarks-and-leaderboards/berkeley-function-calling-leaderboard/index.md) Creator not supplied; see original source. Berkeley Function Calling Leaderboard (BFCL V4): model accuracy on function and tool calling, including agentic tasks. Not applicable (entity) · unknown · unknown resource · en - ### [BigCodeBench](/evaluation/benchmarks-and-leaderboards/bigcodebench/index.md) Creator not supplied; see original source. BigCodeBench leaderboard: practical programming tasks that require calling many libraries, for evaluating code LLMs. Not applicable (entity) · unknown · unknown resource · en - ### [BioASQ](/evaluation/benchmarks-and-leaderboards/bioasq/index.md) Creator not supplied; see original source. BioASQ site: recurring challenges on biomedical semantic indexing and question answering, with datasets and results. Not applicable (entity) · unknown · unknown resource · en - ### [BIRD](/evaluation/benchmarks-and-leaderboards/bird/index.md) Creator not supplied; see original source. BIRD benchmark site: large-scale text-to-SQL evaluation over real databases with dirty values and a public leaderboard. Not applicable (entity) · unknown · unknown resource · en - ### [BrowserGym](/evaluation/benchmarks-and-leaderboards/browsergym/index.md) Creator not supplied; see original source. ServiceNow BrowserGym GitHub repo: Gym environment for evaluating web task automation agents. Not applicable (entity) · unknown · unknown resource · en - ### [CodeClash](/evaluation/benchmarks-and-leaderboards/codeclash/index.md) Creator not supplied; see original source. CodeClash benchmark: AI coding agents compete in multi-round tournaments to pursue open-ended goals in codebases. Not applicable (entity) · unknown · unknown resource · en - ### [CyberGym](/evaluation/benchmarks-and-leaderboards/cybergym/index.md) Creator not supplied; see original source. Berkeley CyberGym GitHub repo: evaluation of AI agents on real software vulnerabilities. Not applicable (entity) · unknown · unknown resource · en - ### [EvalPlus](/evaluation/benchmarks-and-leaderboards/evalplus/index.md) Creator not supplied; see original source. EvalPlus site: HumanEval+ and MBPP+ with extended test cases for more accurate evaluation of code LLMs. Not applicable (entity) · unknown · unknown resource · en - ### [GAIA](/evaluation/benchmarks-and-leaderboards/gaia/index.md) Creator not supplied; see original source. GAIA organisation page on Hugging Face: benchmark of real-world questions for general AI assistants needing reasoning, browsing and tools. Not applicable (entity) · unknown · unknown resource · en - ### [GAIA leaderboard](/evaluation/benchmarks-and-leaderboards/gaia-leaderboard/index.md) Creator not supplied; see original source. GAIA leaderboard Space: scores of AI assistants and agents on the GAIA real-world question benchmark. Not applicable (entity) · unknown · unknown resource · en - ### [HAL: Holistic Agent Leaderboard](/evaluation/benchmarks-and-leaderboards/hal-holistic-agent-leaderboard/index.md) Creator not supplied; see original source. Princeton HAL Holistic Agent Leaderboard: third-party, cost-aware evaluation of AI agents across benchmarks. Not applicable (entity) · unknown · unknown resource · en - ### [Harbor framework](/evaluation/benchmarks-and-leaderboards/harbor-framework/index.md) Creator not supplied; see original source. Harbor framework site: tooling for evaluating and optimising sandboxed agents, running datasets such as Terminal-Bench and SWE-bench Pro. Not applicable (entity) · unknown · unknown resource · en - ### [LiveCodeBench](/evaluation/benchmarks-and-leaderboards/livecodebench/index.md) Creator not supplied; see original source. LiveCodeBench leaderboard: code generation results on problems collected from recent programming contests. Not applicable (entity) · unknown · unknown resource · en - ### [MedQA](/evaluation/benchmarks-and-leaderboards/medqa/index.md) Creator not supplied; see original source. MedQA GitHub repo: data and baselines for open-domain medical exam question answering. Not applicable (entity) · unknown · unknown resource · en - ### [Melting Pot](/evaluation/benchmarks-and-leaderboards/melting-pot/index.md) Creator not supplied; see original source. DeepMind Melting Pot GitHub repo: test scenarios for generalisation in multi-agent reinforcement learning. Not applicable (entity) · unknown · unknown resource · en - ### [MERA Code](/evaluation/benchmarks-and-leaderboards/mera-code/index.md) Creator not supplied; see original source. MERA Code GitHub repo: a benchmark for evaluating code generation by LLMs on tasks described in Russian. Not applicable (entity) · unknown · unknown resource · ru - ### [MERA Code / SWE-MERA](/evaluation/benchmarks-and-leaderboards/mera-code-swe-mera/index.md) Creator not supplied; see original source. MERA Code and SWE-MERA: benchmarks of code generation and software-engineering agents for Russian-language tasks. Not applicable (entity) · unknown · unknown resource · ru - ### [METR](/evaluation/benchmarks-and-leaderboards/metr/index.md) Creator not supplied; see original source. METR: research nonprofit publishing evaluations of frontier AI models' autonomous capabilities and risks. Not applicable (entity) · unknown · unknown resource · en - ### [METR Time Horizons](/evaluation/benchmarks-and-leaderboards/metr-time-horizons/index.md) Creator not supplied; see original source. METR time horizons page: measured length of tasks frontier models complete autonomously, tracked over time. Not applicable (entity) · unknown · unknown resource · en [Next page](/tags/agent-coding-evals/page/2/index.md) [Plain-text Markdown copy](/tags/agent-coding-evals/index.md)