Evaluation & benchmarks for expert learners
443 resources · page 4 of 14
CyberGym
by sunblaze-ucb
Berkeley CyberGym GitHub repo: evaluation of AI agents on real software vulnerabilities.
CyberSecEval 4
by Meta Llama
Meta CyberSecEval 4 documentation: open benchmarks for cybersecurity risks and capabilities of LLMs.
Cyberspace Administration of China (网信办)
by Cyberspace Administration of China
Cyberspace Administration of China official site: regulator publishing China's internet and AI rules.
D4RL
Farama D4RL GitHub repo: reference environments and datasets for offline reinforcement learning.
data.gouv.fr "IA culturelles"
by data.gouv.fr (French Government)
data.gouv.fr topic page for the French Ministry of Culture's IA culturelles challenge, linking related open datasets.
SuperCLUE benchmark results (DataLearner)
by DataLearner
DataLearner page mirroring SuperCLUE Chinese LLM leaderboard scores across releases.
DeepEval
by Confident AI
Confident AI DeepEval GitHub repo: framework for unit-testing and evaluating LLM outputs with metrics.
Lapa LLM developers have created a leaderboard comparing AI models (dev.ua)
by dev.ua
dev.ua report on a leaderboard ranking LLMs on Ukrainian language skill, by the Lapa LLM developers.
DocVQA
by DocVQA team (CVC, Universitat Autònoma de Barcelona)
DocVQA site: document visual question answering challenges and datasets built from real document images.
DPG-Bench (TencentQQGYLab/ELLA)
TencentQQGYLab ELLA GitHub repo: LLM-equipped diffusion models and the DPG-Bench dense prompt benchmark.
DREAM Challenges
by Sage Bionetworks (DREAM Challenges)
Sage Bionetworks page for DREAM Challenges: community biomedical challenges with held-out data and evaluation protocols.
DreamBench++
DreamBench++ project page: a human-aligned benchmark for personalised, subject-driven image generation.
EditBench
Google Research page for Imagen Editor and EditBench, a benchmark for text-guided image inpainting with human evaluation.
EgoSchema
by egoschema
EgoSchema site: long-form egocentric video question answering for testing long-range video understanding.
EleutherAI LM Evaluation Harness
by EleutherAI
EleutherAI lm-evaluation-harness GitHub repo: framework for few-shot evaluation of language models on many benchmarks.
encodechka
by David Dale
encodechka GitHub repo: benchmark of Russian sentence encoders across tasks, with a results table.
Epoch AI Benchmarking Hub
by Epoch AI
Epoch AI Benchmarking Hub: results of AI models on challenging benchmarks, run by Epoch and others, with methods.
HealthBench Professional: Benchmark review (Epoch AI)
by Epoch AI
Epoch AI review of HealthBench Professional, auditing sampled tasks for errors and assessing the benchmark's reliability.
EU AI Office
European Commission page on the EU AI Office, which supervises general-purpose AI models under the EU AI Act.
EuroEval/EuroEval
by EuroEval
EuroEval GitHub repo: a benchmarking framework for language models across European languages, with reproducible datasets.
EuroEval
by Alexandra Institute (EuroEval)
EuroEval: benchmark and leaderboards for language models across 30+ European languages.
EuroEval HF org
by EuroEval
EuroEval Hugging Face organisation hosting European language model evaluation datasets and results.
European LLM Leaderboard (Eurolingua)
Eurolingua European LLM Leaderboard Hugging Face Space: model results on benchmarks in European languages.
EvalCrafter
by EvalCrafter
EvalCrafter project page: a framework for evaluating text-to-video models with automatic metrics and human ratings.
EvalPlus
by evalplus
EvalPlus site: HumanEval+ and MBPP+ with extended test cases for more accurate evaluation of code LLMs.
EvalScope
by ModelScope
ModelScope EvalScope GitHub repo: framework for evaluating model quality and inference performance.
FAR.AI AI Security Leaderboard
by FAR.AI
FAR.AI AI Security Leaderboard comparing model safeguards against a taxonomy of attacks with a published methodology.
FinanceBench
by patronus-ai
FinanceBench GitHub repo from Patronus AI: open-book financial question answering over public company filings.
FinanceBench (Patronus AI)
by Patronus AI
Patronus AI announcement of FinanceBench, question answering over public financial filings to measure LLM failures.
FlagEval
by FlagOpen
BAAI FlagEval GitHub repo: large model evaluation system covering many models and tasks.
FlagEval (BAAI)
by Beijing Academy of Artificial Intelligence (BAAI)
FlagEval: Beijing Academy of Artificial Intelligence platform for evaluating large models.
Flames
by OpenAI45Lab
Flames GitHub repo: an adversarial Chinese benchmark of prompts on fairness, safety, morality, legality and data protection.