Benchmarks And Leaderboards: 512 curated resources

Curated material on benchmarks and leaderboards within evaluation.

512 resources · page 6 of 16

• ImageReward

  Creator not supplied; see original source.

  ImageReward GitHub repo (NeurIPS 2023): human preference reward model and data for text-to-image evaluation.

  Not applicable (entity) · unknown · unknown resource · en

• ImgEdit

  Creator not supplied; see original source.

  ImgEdit GitHub repo from PKU: image-editing dataset and benchmark for evaluating instruction-based editing models.

  Not applicable (entity) · unknown · unknown resource · en

• INCLUDE

  Creator not supplied; see original source.

  Cohere Labs INCLUDE dataset: questions from local academic and professional exams in 44 languages for regional knowledge evaluation.

  Not applicable (entity) · unknown · unknown resource · mul

• IndicGLUE

  Creator not supplied; see original source.

  AI4Bharat IndicGLUE page: natural-language-understanding benchmark for Indian languages with tasks and datasets.

  Not applicable (entity) · unknown · unknown resource · mul

• IndoMMLU

  Creator not supplied; see original source.

  IndoMMLU dataset: questions from Indonesian school exams for evaluating language models on local knowledge.

  Not applicable (entity) · unknown · unknown resource · id

• IndoNLU

  Creator not supplied; see original source.

  IndoNLU GitHub repo: Indonesian natural-language-understanding benchmark with tasks, pretrained models and starter code.

  Not applicable (entity) · unknown · unknown resource · en

• InfiniteBench

  Creator not supplied; see original source.

  InfiniteBench GitHub repo: long-context tasks beyond 100K tokens for evaluating language models.

  Not applicable (entity) · unknown · unknown resource · en

• Inspect Evals

  Creator not supplied; see original source.

  UK AISI inspect_evals GitHub repo: collection of evaluations implemented for the Inspect AI framework.

  Not applicable (entity) · unknown · unknown resource · en

• Inspect (UK AISI)

  Creator not supplied; see original source.

  UK AI Security Institute's Inspect: open-source framework for LLM evaluations with tools, sandboxes, scorers and logs.

  Not applicable (entity) · unknown · unknown resource · en

• IPA press release: AI safety evaluation demonstration and environment report meeting

  Creator not supplied; see original source.

  IPA press release in Japanese on a briefing about AI safety evaluation trials and its open evaluation environment.

  Not applicable (entity) · unknown · unknown resource · ja

• IrokoBench

  Creator not supplied; see original source.

  Masakhane IrokoBench collection on Hugging Face: human-translated NLI, MMLU and math tasks in 16 African languages.

  Not applicable (entity) · unknown · unknown resource · en

• ISP RAS Trusted AI Research Center

  Creator not supplied; see original source.

  ISP RAS research center for trustworthy AI: unit of the Russian Academy of Sciences' Institute for System Programming.

  Not applicable (entity) · unknown · unknown resource · ru

• JailbreakBench

  Creator not supplied; see original source.

  JailbreakBench site: benchmark and leaderboard tracking jailbreak attacks and defences on LLMs with a fixed set of behaviours.

  Not applicable (entity) · unknown · unknown resource · en

• Japan AI Safety Institute

  Creator not supplied; see original source.

  Japan AI Safety Institute: government body developing AI safety evaluation methods, guides and standards.

  Not applicable (entity) · unknown · unknown resource · ja

• 米OpenAI社とMOCを締結しました (Japan AISI)

  Creator not supplied; see original source.

  Japan AISI announcement (May 2026): memorandum of cooperation signed with OpenAI.

  Not applicable (entity) · unknown · unknown resource · ja

• JGLUE

  Creator not supplied; see original source.

  JGLUE GitHub repo from Yahoo Japan: Japanese general language understanding benchmark built natively in Japanese.

  Not applicable (entity) · unknown · unknown resource · ja

• JudgeBench

  Creator not supplied; see original source.

  JudgeBench GitHub repo: benchmark for evaluating LLM-based judges on verifiable response-comparison tasks.

  Not applicable (entity) · unknown · unknown resource · en

• Kazakh LLM leaderboard

  Creator not supplied; see original source.

  Kaz LLM Leaderboard Hugging Face Space: Kazakh-language benchmark results with model submissions.

  Not applicable (entity) · unknown · unknown resource · kk

• KazMMLU

  Creator not supplied; see original source.

  KazMMLU paper: a knowledge benchmark of Kazakhstan-specific questions in Kazakh and Russian for evaluating language models.

  Not applicable (entity) · unknown · unknown resource · kk|ru

• KITTI Vision Benchmark Suite

  Creator not supplied; see original source.

  KITTI Vision Benchmark Suite: real driving data and leaderboards for stereo, optical flow, odometry and object detection.

  Not applicable (entity) · unknown · unknown resource · en

• KLUE

  Creator not supplied; see original source.

  KLUE GitHub repo: Korean natural-language-understanding benchmark with eight tasks and baseline code.

  Not applicable (entity) · unknown · unknown resource · ko

• KMMLU

  Creator not supplied; see original source.

  KMMLU paper: a Korean multitask knowledge benchmark written natively in Korean rather than translated from English MMLU.

  Not applicable (entity) · unknown · unknown resource · en

• KMMLU-Redux and KMMLU-Pro

  Creator not supplied; see original source.

  Paper introducing KMMLU-Redux, a corrected version of KMMLU, and KMMLU-Pro, built from Korean professional licensing exams.

  Not applicable (entity) · unknown · unknown resource · en

• KMMLU-Redux / KMMLU-Pro

  Creator not supplied; see original source.

  LG AI Research publication page for KMMLU-Redux and KMMLU-Pro, Korean benchmarks built from professional licensing exams.

  Not applicable (entity) · unknown · unknown resource · ko

• Korea AISI

  Creator not supplied; see original source.

  Korea AI Safety Institute: government institute for AI safety evaluation, research and international cooperation.

  Not applicable (entity) · unknown · unknown resource · ko

• La Leaderboard

  Creator not supplied; see original source.

  La Leaderboard Hugging Face Space: evaluation of LLMs on Spanish and other languages of Spain and Latin America.

  Not applicable (entity) · unknown · unknown resource · es|ca|eu|gl

• LAB-Bench

  Creator not supplied; see original source.

  LAB-Bench GitHub repo from FutureHouse: practical biology-research tasks such as literature, sequence and protocol questions.

  Not applicable (entity) · unknown · unknown resource · en

• langfuse/langfuse

  Creator not supplied; see original source.

  Langfuse GitHub repo: open-source LLM tracing, evaluation and observability platform.

  Not applicable (entity) · unknown · unknown resource · en

• lechmazur Confabulations

  Creator not supplied; see original source.

  lechmazur Confabulations GitHub repo: document-based benchmark measuring how often LLMs hallucinate answers in RAG settings.

  Not applicable (entity) · unknown · unknown resource · en

• LegalBench

  Creator not supplied; see original source.

  LegalBench site from Stanford: collaboratively built benchmark of legal-reasoning tasks for evaluating LLMs on US law.

  Not applicable (entity) · unknown · unknown resource · en

• LIBERO

  Creator not supplied; see original source.

  LIBERO project site: robot manipulation task suites in simulation for evaluating knowledge transfer in lifelong robot learning.

  Not applicable (entity) · unknown · unknown resource · en

• LIBERO-Plus

  Creator not supplied; see original source.

  LIBERO-Plus paper: a robustness analysis of vision-language-action models using systematically perturbed LIBERO manipulation tasks.

  Not applicable (entity) · unknown · unknown resource · en

Plain-text Markdown copy