Evaluation & benchmarks for beginner learners
420 resources · page 6 of 14
ImageNet
by ImageNet team (Stanford University and Princeton University)
ImageNet site: the large-scale image classification dataset behind the ImageNet challenge, with download access.
ImageReward
by Z.ai
ImageReward GitHub repo (NeurIPS 2023): human preference reward model and data for text-to-image evaluation.
ImgEdit
ImgEdit GitHub repo from PKU: image-editing dataset and benchmark for evaluating instruction-based editing models.
INCLUDE
by Cohere Labs
Cohere Labs INCLUDE dataset: questions from local academic and professional exams in 44 languages for regional knowledge evaluation.
IndicGLUE
AI4Bharat IndicGLUE page: natural-language-understanding benchmark for Indian languages with tasks and datasets.
IndoMMLU
by IndoLEM
IndoMMLU dataset: questions from Indonesian school exams for evaluating language models on local knowledge.
IndoNLU
by IndoNLP
IndoNLU GitHub repo: Indonesian natural-language-understanding benchmark with tasks, pretrained models and starter code.
InfiniteBench
by OpenBMB
InfiniteBench GitHub repo: long-context tasks beyond 100K tokens for evaluating language models.
Inspect Evals
by Department of Business, Energy and Industrial Strategy
UK AISI inspect_evals GitHub repo: collection of evaluations implemented for the Inspect AI framework.
Inspect (UK AISI)
UK AI Security Institute's Inspect: open-source framework for LLM evaluations with tools, sandboxes, scorers and logs.
IPA press release: AI safety evaluation demonstration and environment report meeting
by Information-technology Promotion Agency, Japan (IPA)
IPA press release in Japanese on a briefing about AI safety evaluation trials and its open evaluation environment.
IrokoBench
Masakhane IrokoBench collection on Hugging Face: human-translated NLI, MMLU and math tasks in 16 African languages.
ISP RAS Trusted AI Research Center
by Ivannikov Institute for System Programming (ISP RAS)
ISP RAS research center for trustworthy AI: unit of the Russian Academy of Sciences' Institute for System Programming.
JailbreakBench
JailbreakBench site: benchmark and leaderboard tracking jailbreak attacks and defences on LLMs with a fixed set of behaviours.
Japan AI Safety Institute
Japan AI Safety Institute: government body developing AI safety evaluation methods, guides and standards.
米OpenAI社とMOCを締結しました (Japan AISI)
Japan AISI announcement (May 2026): memorandum of cooperation signed with OpenAI.
JGLUE
by Yahoo! JAPAN
JGLUE GitHub repo from Yahoo Japan: Japanese general language understanding benchmark built natively in Japanese.
JudgeBench
by ScalerLab
JudgeBench GitHub repo: benchmark for evaluating LLM-based judges on verifiable response-comparison tasks.
Kazakh LLM leaderboard
Kaz LLM Leaderboard Hugging Face Space: Kazakh-language benchmark results with model submissions.
KazMMLU
KazMMLU paper: a knowledge benchmark of Kazakhstan-specific questions in Kazakh and Russian for evaluating language models.
KITTI Vision Benchmark Suite
by Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI)
KITTI Vision Benchmark Suite: real driving data and leaderboards for stereo, optical flow, odometry and object detection.
KLUE
KLUE GitHub repo: Korean natural-language-understanding benchmark with eight tasks and baseline code.
KMMLU
by Guijin Son et al. (Yonsei University)
KMMLU paper: a Korean multitask knowledge benchmark written natively in Korean rather than translated from English MMLU.
KMMLU-Redux and KMMLU-Pro
by Seokhee Hong et al. (LG AI Research)
Paper introducing KMMLU-Redux, a corrected version of KMMLU, and KMMLU-Pro, built from Korean professional licensing exams.
KMMLU-Redux / KMMLU-Pro
LG AI Research publication page for KMMLU-Redux and KMMLU-Pro, Korean benchmarks built from professional licensing exams.
Korea AISI
by Korea AISI
Korea AI Safety Institute: government institute for AI safety evaluation, research and international cooperation.
La Leaderboard
La Leaderboard Hugging Face Space: evaluation of LLMs on Spanish and other languages of Spain and Latin America.
LAB-Bench
by FutureHouse
LAB-Bench GitHub repo from FutureHouse: practical biology-research tasks such as literature, sequence and protocol questions.
langfuse/langfuse
by Langfuse
Langfuse GitHub repo: open-source LLM tracing, evaluation and observability platform.
lechmazur Confabulations
by lechmazur
lechmazur Confabulations GitHub repo: document-based benchmark measuring how often LLMs hallucinate answers in RAG settings.
LegalBench
by Hazy Research (Stanford University)
LegalBench site from Stanford: collaboratively built benchmark of legal-reasoning tasks for evaluating LLMs on US law.
LIBERO
LIBERO project site: robot manipulation task suites in simulation for evaluating knowledge transfer in lifelong robot learning.