--- id: "/tags/agent-coding-evals/page/2/" title: "Agent Coding Evals · page 2" creator: null creator_url: null creator_x: null url: "https://skillissue.dev/tags/agent-coding-evals/page/2/" level: null level_max: null category: null subcategory: null region: null license: null license_scope: null access: null models: null source_date: null date_added: null resource_kind: null verification_basis: null access_evidence: null domain_top: null domain_sub: null last_verified: null --- # Agent Coding Evals [Beginner](/levels/beginner/index.md) [Intermediate](/levels/intermediate/index.md) [Advanced](/levels/advanced/index.md) [Expert](/levels/expert/index.md) 85 resources · page 2 of 3 - ### [MineRL](/evaluation/benchmarks-and-leaderboards/minerl/index.md) Creator not supplied; see original source. MineRL GitHub repo: Minecraft environments and human demonstrations for sample-efficient RL. Not applicable (entity) · unknown · unknown resource · en - ### [mini-SWE-agent](/evaluation/benchmarks-and-leaderboards/mini-swe-agent/index.md) Creator not supplied; see original source. mini-SWE-agent GitHub repo: a roughly 100-line reference agent for solving GitHub issues, used as a simple SWE-bench baseline. Not applicable (entity) · unknown · unknown resource · en - ### [MLE-bench](/evaluation/benchmarks-and-leaderboards/mle-bench/index.md) Creator not supplied; see original source. MLE-bench paper from OpenAI: 75 Kaggle competitions used to evaluate AI agents on machine-learning engineering work. Not applicable (entity) · unknown · unknown resource · en - ### [OpenRouter rankings](/evaluation/benchmarks-and-leaderboards/openrouter-rankings/index.md) Creator not supplied; see original source. OpenRouter rankings: models ranked by tokens processed through the OpenRouter API, by app and category. Not applicable (entity) · unknown · unknown resource · en - ### [Polish LLM benchmarks (CodeSOTA)](/evaluation/benchmarks-and-leaderboards/polish-llm-benchmarks-codesota/index.md) Creator not supplied; see original source. CodeSOTA page aggregating Polish LLM benchmark results for models such as Bielik, PLLuM, GPT, Claude and Gemini. Not applicable (entity) · unknown · unknown resource · pl - ### [ProgramBench](/evaluation/benchmarks-and-leaderboards/programbench/index.md) Creator not supplied; see original source. ProgramBench site: benchmark testing whether language models can rebuild complete programs from scratch. Not applicable (entity) · unknown · unknown resource · en - ### [Pydantic AI](/evaluation/benchmarks-and-leaderboards/pydantic-ai-repo-evals-durable-exec/index.md) Creator not supplied; see original source. Pydantic AI GitHub repo: type-safe Python agent framework with evals and durable execution. Not applicable (entity) · unknown · unknown resource · en - ### [RULER](/evaluation/benchmarks-and-leaderboards/ruler/index.md) Creator not supplied; see original source. NVIDIA RULER GitHub repo: synthetic benchmark measuring the real usable context length of long-context models. Not applicable (entity) · unknown · unknown resource · en - ### [SciCode](/evaluation/benchmarks-and-leaderboards/scicode/index.md) Creator not supplied; see original source. SciCode benchmark site: research-level scientific coding problems for evaluating language models. Not applicable (entity) · unknown · unknown resource · en - ### [ScienceAgentBench](/evaluation/benchmarks-and-leaderboards/scienceagentbench/index.md) Creator not supplied; see original source. ScienceAgentBench project page: data-driven scientific discovery tasks from published papers for evaluating language agents. Not applicable (entity) · unknown · unknown resource · en - ### [Spider 2.0](/evaluation/benchmarks-and-leaderboards/spider-2-0/index.md) Creator not supplied; see original source. Spider 2.0 site: 632 enterprise text-to-SQL workflow problems for evaluating language models and agents. Not applicable (entity) · unknown · unknown resource · en - ### [SWE-bench](/evaluation/benchmarks-and-leaderboards/swe-bench/index.md) Creator not supplied; see original source. SWE-bench leaderboards: coding-agent results on resolving real GitHub issues, across SWE-bench Verified, Multilingual and Multimodal. Not applicable (entity) · unknown · unknown resource · en - ### [SWE-bench Pro](/evaluation/benchmarks-and-leaderboards/swe-bench-pro/index.md) Creator not supplied; see original source. Scale AI SWE-bench Pro GitHub repo: benchmark of long-horizon software engineering tasks for agents. Not applicable (entity) · unknown · unknown resource · en - ### [SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?](/evaluation/benchmarks-and-leaderboards/swe-bench-pro-paper/index.md) Creator not supplied; see original source. SWE-Bench Pro paper: long-horizon software-engineering tasks, including a held-out commercial split, for evaluating coding agents. Not applicable (entity) · unknown · unknown resource · en - ### [SWE-Lancer](/evaluation/benchmarks-and-leaderboards/swe-lancer/index.md) Creator not supplied; see original source. OpenAI SWE-Lancer GitHub repo: dataset and code for real freelance software engineering tasks. Not applicable (entity) · unknown · unknown resource · en - ### [SWE-MERA](/evaluation/benchmarks-and-leaderboards/swe-mera/index.md) Creator not supplied; see original source. SWE-MERA page: continuously refreshed software-engineering tasks built as a contamination-resistant alternative to SWE-bench. Not applicable (entity) · unknown · unknown resource · en - ### [SWE-rebench](/evaluation/benchmarks-and-leaderboards/swe-rebench/index.md) Creator not supplied; see original source. SWE-rebench: continuously updated, decontaminated software-engineering benchmark with leaderboard. Not applicable (entity) · unknown · unknown resource · en - ### [τ-bench](/evaluation/benchmarks-and-leaderboards/tau-bench-family-tau-tau2-tau3-bench/index.md) Creator not supplied; see original source. τ-bench site: benchmarks of AI agents conversing with users, calling tools and following policies across domains. Not applicable (entity) · unknown · unknown resource · en - ### [tau2-bench](/evaluation/benchmarks-and-leaderboards/tau2-bench/index.md) Creator not supplied; see original source. tau2-bench GitHub repo (Sierra): benchmark of tool-using agents interacting with users under domain policies. Not applicable (entity) · unknown · unknown resource · en - ### [Terminal-Bench](/evaluation/benchmarks-and-leaderboards/terminal-bench/index.md) Creator not supplied; see original source. Terminal-Bench: benchmark of AI agents completing tasks in sandboxed terminal environments. Not applicable (entity) · unknown · unknown resource · en - ### [Terminal-Bench 2.0 (Snorkel)](/evaluation/benchmarks-and-leaderboards/terminal-bench-2-0-snorkel/index.md) Creator not supplied; see original source. Snorkel-hosted Terminal-Bench 2.0 leaderboard: 89 human-checked terminal tasks scored by resolution rate. Not applicable (entity) · unknown · unknown resource · en - ### [The Agent Company](/evaluation/benchmarks-and-leaderboards/the-agent-company/index.md) Creator not supplied; see original source. TheAgentCompany site: benchmark of simulated software-company tasks for evaluating LLM agents across business tools. Not applicable (entity) · unknown · unknown resource · en - ### [ToolBench](/evaluation/benchmarks-and-leaderboards/toolbench/index.md) Creator not supplied; see original source. OpenBMB ToolBench GitHub repo (ICLR 2024): platform and benchmark for training and evaluating LLM tool use. Not applicable (entity) · unknown · unknown resource · en - ### [Top 3D AI Arena](/evaluation/benchmarks-and-leaderboards/top-3d-ai-arena/index.md) Creator not supplied; see original source. Top 3D AI leaderboard ranking 3D generation tools by Elo from blind community votes. Not applicable (entity) · unknown · unknown resource · en - ### [VisualWebArena](/evaluation/benchmarks-and-leaderboards/visualwebarena/index.md) Creator not supplied; see original source. VisualWebArena project page: benchmark of realistic web tasks where multimodal agents need visual information to succeed. Not applicable (entity) · unknown · unknown resource · en - ### [VQA v2](/evaluation/benchmarks-and-leaderboards/vqa-v2/index.md) Creator not supplied; see original source. VQA site: Visual Question Answering datasets and challenges for answering open questions about images. Not applicable (entity) · unknown · unknown resource · en - ### [WebArena](/evaluation/benchmarks-and-leaderboards/webarena/index.md) Creator not supplied; see original source. WebArena: suite of self-hosted website environments for benchmarking autonomous web agents. Not applicable (entity) · unknown · unknown resource · en - ### [Windows Agent Arena](/evaluation/benchmarks-and-leaderboards/windows-agent-arena/index.md) Creator not supplied; see original source. Windows Agent Arena GitHub repo from Microsoft: scalable Windows OS environment for testing multimodal computer-use agents. Not applicable (entity) · unknown · unknown resource · en - ### [WorkArena](/evaluation/benchmarks-and-leaderboards/workarena/index.md) Creator not supplied; see original source. ServiceNow WorkArena GitHub repo: benchmark of web agents on common enterprise knowledge-work tasks. Not applicable (entity) · unknown · unknown resource · en - ### [What matters when evaluating AI agents for customer service (Intercom)](/evaluation/benchmarks-and-leaderboards/intercom-what-really-matters-when-evaluating-ai-agents-for-cs/index.md) Creator not supplied; see original source. Intercom post on criteria beyond performance for evaluating AI agents for customer service. Beginner · unknown · unknown resource · en - ### [Latent Space: SWE-bench dead](/evaluation/benchmarks-and-leaderboards/latent-space-swe-bench-dead/index.md) Creator not supplied; see original source. Latent Space episode with OpenAI's Mia Glaese and Olivia Watkins on retiring SWE-bench Verified and next frontier evals. Beginner · unknown · unknown resource · en - ### [AI Agents That Matter](/evaluation/benchmarks-and-leaderboards/ai-agents-that-matter/index.md) Creator not supplied; see original source. arXiv paper (2024) "AI Agents That Matter", on cost-aware and reproducible evaluation of AI agents. Intermediate · unknown · unknown resource · en [Previous page](/tags/agent-coding-evals/index.md) [Next page](/tags/agent-coding-evals/page/3/index.md) [Plain-text Markdown copy](/tags/agent-coding-evals/page/2/index.md)