science & domain evals: 45 curated resources
science & domain evals
45 resources · page 1 of 2
ARC-AGI leaderboard
ARC Prize leaderboard: model scores on ARC-AGI abstraction benchmarks plotted against cost per task.
BEIR
by BEIR Cellar
BEIR GitHub repo: heterogeneous information-retrieval benchmark of many datasets for zero-shot evaluation of retrievers.
CAMEO
by CAMEO (SIB Swiss Institute of Bioinformatics)
CAMEO platform: weekly automated, blind benchmarking of protein structure prediction servers against newly released structures.
CASP Prediction Center
by Protein Structure Prediction Center (UC Davis)
CASP Prediction Center: blind community assessment of protein structure prediction methods.
CASP17
by Protein Structure Prediction Center (UC Davis)
CASP17 site: the 2026 round of the Critical Assessment of protein Structure Prediction, with targets and assessment results.
DREAM Challenges
by Sage Bionetworks (DREAM Challenges)
Sage Bionetworks page for DREAM Challenges: community biomedical challenges with held-out data and evaluation protocols.
HealthBench Professional: Benchmark review (Epoch AI)
by Epoch AI
Epoch AI review of HealthBench Professional, auditing sampled tasks for errors and assessing the benchmark's reliability.
FinanceBench
by patronus-ai
FinanceBench GitHub repo from Patronus AI: open-book financial question answering over public company filings.
FinanceBench (Patronus AI)
by Patronus AI
Patronus AI announcement of FinanceBench, question answering over public financial filings to measure LLM failures.
FrontierMath
by Epoch AI
Epoch AI FrontierMath: benchmark of unpublished expert-level mathematics problems in tiers, with model results.
GPQA
GPQA GitHub repo: graduate-level, Google-proof science questions with baselines for evaluating expert reasoning.
GSM8K
by OpenAI
GSM8K GitHub repo from OpenAI: grade-school math word problems with solutions and evaluation code; archived.
GuacaMol
by BenevolentAI
BenevolentAI GuacaMol GitHub repo: benchmarks for generative chemistry models.
LAB-Bench
by FutureHouse
LAB-Bench GitHub repo from FutureHouse: practical biology-research tasks such as literature, sequence and protocol questions.
LegalBench
by Hazy Research (Stanford University)
LegalBench site from Stanford: collaboratively built benchmark of legal-reasoning tasks for evaluating LLMs on US law.
LiveMedBench
by alphaXiv
LiveMedBench paper page: a continuously refreshed medical LLM benchmark built from real clinical cases with automated rubric scoring.
Matbench
Materials Project Matbench GitHub repo: benchmark tasks for materials property prediction.
Matbench Discovery
Matbench Discovery GitHub repo: evaluation framework for ML models in materials stability discovery.
MATH
MATH dataset GitHub repo: competition mathematics problems with step-by-step solutions for evaluating reasoning.
MathArena
by MathArena team (SRI Lab, ETH Zurich)
MathArena: evaluation of LLMs on newly released math competitions and arXiv-derived problems to avoid contamination.
MedBench
by Shanghai AI Laboratory (OpenCompass)
OpenCompass MedBench portal: Chinese medical benchmark platform for evaluating LLMs on clinical knowledge and tasks.
MedHELM
Stanford CRFM MedHELM: evaluation of language models on clinician-defined medical tasks beyond exam-style questions.
MedHELM (Stanford CRFM)
Latest MedHELM results page from Stanford CRFM, scoring language models on a taxonomy of real clinical tasks.
MedMNIST
by MedMNIST team (Shanghai Jiao Tong University)
MedMNIST site: standardised 2D and 3D biomedical image classification datasets for lightweight benchmarking.
MERA Industrial
MERA Industrial: benchmark of foundation models on industry-specific Russian-language tasks.
MOSES
by MOSES
MOSES GitHub repo: benchmarking platform with metrics and baselines for molecular generation models.
Open Catalyst Project
by Meta FAIR Chemistry and Carnegie Mellon University (Open Catalyst Project)
Open Catalyst Project site: atomistic datasets, models and challenges for AI-based catalyst discovery.
Open Graph Benchmark
by Open Graph Benchmark team (Stanford University)
Open Graph Benchmark site from Stanford: graph datasets, data loaders and evaluators for graph machine learning.
frontier-evals
by OpenAI
OpenAI frontier-evals GitHub repo: code for evaluations such as PaperBench and SWE-Lancer.
PhysioNet Challenges
by PhysioNet (MIT Laboratory for Computational Physiology)
PhysioNet Challenges page: annual open challenges on clinical physiological-signal analysis with scoring methods and data.
ProteinGym
by Marks Lab (Harvard Medical School)
ProteinGym: benchmark and leaderboard for predicting the effects of protein mutations.
Robot learning leaderboard (RoboArena)
by RoboArena team (Pranav Atreya, Karl Pertsch et al.)
Robot-learning.ml leaderboard site for evaluating robot learning policies.