Benchmarks And Leaderboards: 512 curated resources

Curated material on benchmarks and leaderboards within evaluation.

512 resources · page 5 of 16

• ForesightSafety Bench

  Creator not supplied; see original source.

  ForesightSafety Bench paper: a frontier-risk evaluation framework covering agentic autonomy, AI-for-science, embodied and catastrophic risks.

  Not applicable (entity) · unknown · unknown resource · en

• FrontierMath

  Creator not supplied; see original source.

  Epoch AI FrontierMath: benchmark of unpublished expert-level mathematics problems in tiers, with model results.

  Not applicable (entity) · unknown · unknown resource · en

• GAIA

  Creator not supplied; see original source.

  GAIA organisation page on Hugging Face: benchmark of real-world questions for general AI assistants needing reasoning, browsing and tools.

  Not applicable (entity) · unknown · unknown resource · en

• GAIA leaderboard

  Creator not supplied; see original source.

  GAIA leaderboard Space: scores of AI assistants and agents on the GAIA real-world question benchmark.

  Not applicable (entity) · unknown · unknown resource · en

• GEdit-Bench (StepFun Step1X-Edit)

  Creator not supplied; see original source.

  StepFun Step1X-Edit GitHub repo: open image-editing model whose release includes GEdit-Bench, a benchmark of real-world editing requests.

  Not applicable (entity) · unknown · unknown resource · en

• GenAI-Arena (TIGER Lab)

  Creator not supplied; see original source.

  TIGER Lab GenAI-Arena Space: academic arena voting between generative image, editing and video models.

  Not applicable (entity) · unknown · unknown resource · en

• GenEval

  Creator not supplied; see original source.

  GenEval GitHub repo: object-focused framework for evaluating text-to-image alignment.

  Not applicable (entity) · unknown · unknown resource · en

• GEO-Bench

  Creator not supplied; see original source.

  GEO-Bench GitHub repo from ServiceNow: benchmark for foundation models on Earth-observation classification and segmentation.

  Not applicable (entity) · unknown · unknown resource · en

• Global-MMLU

  Creator not supplied; see original source.

  Cohere Labs Global-MMLU dataset: MMLU translated into many languages with labels marking culturally sensitive questions.

  Not applicable (entity) · unknown · unknown resource · mul

• GLUE

  Creator not supplied; see original source.

  GLUE benchmark site: collection of natural-language-understanding tasks with a leaderboard for comparing models.

  Not applicable (entity) · unknown · unknown resource · en

• GPQA

  Creator not supplied; see original source.

  GPQA GitHub repo: graduate-level, Google-proof science questions with baselines for evaluating expert reasoning.

  Not applicable (entity) · unknown · unknown resource · en

• GQA

  Creator not supplied; see original source.

  GQA dataset site: real-image visual question answering with scene graphs for testing compositional reasoning.

  Not applicable (entity) · unknown · unknown resource · en

• GSM8K

  Creator not supplied; see original source.

  GSM8K GitHub repo from OpenAI: grade-school math word problems with solutions and evaluation code; archived.

  Not applicable (entity) · unknown · unknown resource · en

• GuacaMol

  Creator not supplied; see original source.

  BenevolentAI GuacaMol GitHub repo: benchmarks for generative chemistry models.

  Not applicable (entity) · unknown · unknown resource · en

• Habitat Challenge

  Creator not supplied; see original source.

  Annual Habitat Challenge for embodied AI, with simulated navigation and object-rearrangement tasks, rules and leaderboards for agent evaluation.

  Not applicable (entity) · unknown · unknown resource · en

• HAE-RAE Bench

  Creator not supplied; see original source.

  HAE-RAE Bench GitHub repo: a Korean benchmark on vocabulary, history, general knowledge and reading comprehension.

  Not applicable (entity) · unknown · unknown resource · ko

• HAL: Holistic Agent Leaderboard

  Creator not supplied; see original source.

  Princeton HAL Holistic Agent Leaderboard: third-party, cost-aware evaluation of AI agents across benchmarks.

  Not applicable (entity) · unknown · unknown resource · en

• HallusionBench

  Creator not supplied; see original source.

  HallusionBench GitHub repo (CVPR 2024): benchmark of language hallucination and visual illusion in VLMs.

  Not applicable (entity) · unknown · unknown resource · en

• Harbor framework

  Creator not supplied; see original source.

  Harbor framework site: tooling for evaluating and optimising sandboxed agents, running datasets such as Terminal-Bench and SWE-bench Pro.

  Not applicable (entity) · unknown · unknown resource · en

• HarmBench

  Creator not supplied; see original source.

  HarmBench site: standardised evaluation framework for automated red-teaming attacks and model refusal defences.

  Not applicable (entity) · unknown · unknown resource · en

• HEAR

  Creator not supplied; see original source.

  HEAR benchmark site: holistic evaluation of audio representations across speech, music and environmental sound tasks.

  Not applicable (entity) · unknown · unknown resource · en

• Hebrew LLM Leaderboard

  Creator not supplied; see original source.

  Hebrew LLM Leaderboard Hugging Face Space: model results on Hebrew language benchmarks.

  Not applicable (entity) · unknown · unknown resource · he

• HEIM

  Creator not supplied; see original source.

  Stanford CRFM HEIM: holistic evaluation of text-to-image models on quality, alignment, bias and other dimensions.

  Not applicable (entity) · unknown · unknown resource · en

• HELM (Stanford CRFM)

  Creator not supplied; see original source.

  Stanford CRFM HELM: holistic evaluation of language models across many scenarios and metrics.

  Not applicable (entity) · unknown · unknown resource · en

• HELMET

  Creator not supplied; see original source.

  HELMET GitHub repo from Princeton: benchmark for long-context language models across seven application categories.

  Not applicable (entity) · unknown · unknown resource · en

• HF Find a Leaderboard

  Creator not supplied; see original source.

  Hugging Face Find a Leaderboard Space: searchable directory of leaderboards hosted on the Hub.

  Not applicable (entity) · unknown · unknown resource · und

• Hugging Face LightEval

  Creator not supplied; see original source.

  Hugging Face LightEval GitHub repo: toolkit for evaluating LLMs across many backends and tasks.

  Not applicable (entity) · unknown · unknown resource · en

• Humanity's Last Exam

  Creator not supplied; see original source.

  Humanity's Last Exam: dataset of 2,500 expert-written questions across academic fields, with paper and results.

  Not applicable (entity) · unknown · unknown resource · en

• Humanloop

  Creator not supplied; see original source.

  Humanloop site, now a notice that the team joined Anthropic and the platform was shut down with a migration guide.

  Not applicable (entity) · unknown · unknown resource · en

• IberBench

  Creator not supplied; see original source.

  UPV repository record for IberBench, a benchmark for evaluating LLMs on Iberian languages such as Spanish, Catalan and Basque.

  Not applicable (entity) · unknown · unknown resource · mul

• IFBench

  Creator not supplied; see original source.

  IFBench paper on generalising verifiable instruction following, with out-of-distribution output constraints for evaluating LLMs.

  Not applicable (entity) · unknown · unknown resource · en

• ImageNet

  Creator not supplied; see original source.

  ImageNet site: the large-scale image classification dataset behind the ImageNet challenge, with download access.

  Not applicable (entity) · unknown · unknown resource · en

Plain-text Markdown copy