Evaluation for advanced learners

455 resources · page 12 of 15

• SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?

  Creator not supplied; see original source.

  SWE-Bench Pro paper: long-horizon software-engineering tasks, including a held-out commercial split, for evaluating coding agents.

  Not applicable (entity) · unknown · unknown resource · en

• SWE-Lancer

  Creator not supplied; see original source.

  OpenAI SWE-Lancer GitHub repo: dataset and code for real freelance software engineering tasks.

  Not applicable (entity) · unknown · unknown resource · en

• SWE-MERA

  Creator not supplied; see original source.

  SWE-MERA page: continuously refreshed software-engineering tasks built as a contamination-resistant alternative to SWE-bench.

  Not applicable (entity) · unknown · unknown resource · en

• SWE-rebench

  Creator not supplied; see original source.

  SWE-rebench: continuously updated, decontaminated software-engineering benchmark with leaderboard.

  Not applicable (entity) · unknown · unknown resource · en

• T-Tech Ru Dialogue Benchmarks

  Creator not supplied; see original source.

  T-Tech collection on Hugging Face of benchmarks for evaluating Russian-language dialogue models.

  Not applicable (entity) · unknown · unknown resource · ru

• T2I-CompBench

  Creator not supplied; see original source.

  T2I-CompBench GitHub repo: benchmark for compositional text-to-image generation such as attribute binding and spatial relations.

  Not applicable (entity) · unknown · unknown resource · en

• T3Bench

  Creator not supplied; see original source.

  T3Bench site: benchmark for text-to-3D generation measuring quality and alignment with the prompt.

  Not applicable (entity) · unknown · unknown resource · en

• TabArena leaderboard

  Creator not supplied; see original source.

  TabArena leaderboard Hugging Face Space: living benchmark of machine-learning models on tabular datasets.

  Not applicable (entity) · unknown · unknown resource · en

• τ-bench

  Creator not supplied; see original source.

  τ-bench site: benchmarks of AI agents conversing with users, calling tools and following policies across domains.

  Not applicable (entity) · unknown · unknown resource · en

• tau2-bench

  Creator not supplied; see original source.

  tau2-bench GitHub repo (Sierra): benchmark of tool-using agents interacting with users under domain policies.

  Not applicable (entity) · unknown · unknown resource · en

• TaxCalcBench (Column Tax)

  Creator not supplied; see original source.

  Column Tax post introducing TaxCalcBench, a benchmark of tax-return calculation cases with an analysis of where models fail.

  Not applicable (entity) · unknown · unknown resource · en

• TC260 national cybersecurity standards committee

  Creator not supplied; see original source.

  TC260 official site: China's national cybersecurity standardisation committee, which issues AI security standards.

  Not applicable (entity) · unknown · unknown resource · zh

• Temporal Graph Benchmark

  Creator not supplied; see original source.

  Temporal Graph Benchmark documentation: datasets and tasks for evaluating machine learning on dynamic graphs.

  Not applicable (entity) · unknown · unknown resource · en

• Terminal-Bench

  Creator not supplied; see original source.

  Terminal-Bench: benchmark of AI agents completing tasks in sandboxed terminal environments.

  Not applicable (entity) · unknown · unknown resource · en

• Terminal-Bench 2.0 (Snorkel)

  Creator not supplied; see original source.

  Snorkel-hosted Terminal-Bench 2.0 leaderboard: 89 human-checked terminal tasks scored by resolution rate.

  Not applicable (entity) · unknown · unknown resource · en

• Thai LLM Leaderboard

  Creator not supplied; see original source.

  Thai LLM Leaderboard Hugging Face Space: evaluation results of models across Thai benchmarks.

  Not applicable (entity) · unknown · unknown resource · th

• The Agent Company

  Creator not supplied; see original source.

  TheAgentCompany site: benchmark of simulated software-company tasks for evaluating LLM agents across business tools.

  Not applicable (entity) · unknown · unknown resource · en

• Therapeutics Data Commons

  Creator not supplied; see original source.

  Therapeutics Data Commons: datasets, tasks and leaderboards for machine learning across drug discovery and development.

  Not applicable (entity) · unknown · unknown resource · en

• TII Arabic LLM benchmark index

  Creator not supplied; see original source.

  TII Arabic-LLM-Benchmarks GitHub repo: list of benchmarks for evaluating Arabic LLMs.

  Not applicable (entity) · unknown · unknown resource · en

• TMMLU+ (Taiwan)

  Creator not supplied; see original source.

  iKala TMMLU+ dataset: Traditional Chinese multiple-choice questions across 66 subjects with a Taiwan-specific context.

  Not applicable (entity) · unknown · unknown resource · zh

• ToolBench

  Creator not supplied; see original source.

  OpenBMB ToolBench GitHub repo (ICLR 2024): platform and benchmark for training and evaluating LLM tool use.

  Not applicable (entity) · unknown · unknown resource · en

• Top 3D AI Arena

  Creator not supplied; see original source.

  Top 3D AI leaderboard ranking 3D generation tools by Elo from blind community votes.

  Not applicable (entity) · unknown · unknown resource · en

• Российский энтузиаст создал бесплатный ИИ-бенчмарк (Tproger)

  Creator not supplied; see original source.

  Tproger report in Russian on LLM Arena, a free blind pairwise benchmark for Russian-language models.

  Not applicable (entity) · unknown · unknown resource · ru

• TruthfulQA

  Creator not supplied; see original source.

  TruthfulQA GitHub repo: benchmark testing whether language models repeat common human misconceptions, with data and evaluation code.

  Not applicable (entity) · unknown · unknown resource · en

• TTS Arena V2 (Hugging Face Space)

  Creator not supplied; see original source.

  TTS Arena V2 Space: community arena voting between anonymous text-to-speech models.

  Not applicable (entity) · unknown · unknown resource · en

• UK AI Security Institute

  Creator not supplied; see original source.

  UK AI Security Institute: government body publishing frontier AI evaluations and the Inspect evaluation framework.

  Not applicable (entity) · unknown · unknown resource · en

• UK AISI Inspect Evals

  Creator not supplied; see original source.

  UK AISI Inspect Evals: registry of evaluations implemented in Inspect, including cybersecurity, safety and agent benchmarks.

  Not applicable (entity) · unknown · unknown resource · en|mul

• Announcing Inspect Evals (UK AISI)

  Creator not supplied; see original source.

  UK AISI announcement of Inspect Evals, an open-source collection of LLM evaluations.

  Not applicable (entity) · unknown · unknown resource · en

• Ukrainian LLM Leaderboard (lang-uk)

  Creator not supplied; see original source.

  lang-uk Ukrainian LLM Leaderboard Hugging Face Space: dashboard of model results on Ukrainian tasks.

  Not applicable (entity) · unknown · unknown resource · uk

• Vals.ai

  Creator not supplied; see original source.

  Vals AI benchmarks: independently run leaderboards on coding, finance, legal and science tasks with evaluation costs.

  Not applicable (entity) · unknown · unknown resource · en

• Vals AI MMMU Pro Leaderboard

  Creator not supplied; see original source.

  Vals AI MMMU-Pro page: archived multimodal reasoning results and methodology; no longer run on new models.

  Not applicable (entity) · unknown · unknown resource · en

• Vals AI's Latest Benchmark Finds Legal and General AI Now Outperform Lawyers in Legal Research (LawNext)

  Creator not supplied; see original source.

  LawNext article on Vals AI's legal research benchmark comparing legal AI tools and general LLMs with a lawyer baseline.

  Not applicable (entity) · unknown · unknown resource · en

Plain-text Markdown copy