Evaluation

Measure model capabilities and application behavior with explicit criteria.

• Benchmarks And Leaderboards (512)
• Eval Harnesses (3)
• Application Evals (2)
• Llm As Judge (1)
• Agent Evals (1)

519 resources · page 10 of 17

• OpenCompass HF leaderboard

  Creator not supplied; see original source.

  OpenCompass LLM Leaderboard Hugging Face Space: OpenCompass evaluation results for language models.

  Not applicable (entity) · unknown · unknown resource · zh|en

• OpenDriveLab Autonomous Grand Challenge 2025

  Creator not supplied; see original source.

  OpenDriveLab Autonomous Grand Challenge 2025: CVPR 2025 autonomous-driving and embodied AI tracks such as NAVSIM.

  Not applicable (entity) · unknown · unknown resource · en

• OpenDriveLab events

  Creator not supplied; see original source.

  OpenDriveLab events page listing its autonomous-driving and embodied AI challenges and workshops.

  Not applicable (entity) · unknown · unknown resource · en

• OpenML

  Creator not supplied; see original source.

  OpenML: open platform for sharing machine-learning datasets, tasks, algorithms and experiment runs.

  Not applicable (entity) · unknown · unknown resource · en

• OpenRouter rankings

  Creator not supplied; see original source.

  OpenRouter rankings: models ranked by tokens processed through the OpenRouter API, by app and category.

  Not applicable (entity) · unknown · unknown resource · en

• OSWorld 2.0

  Creator not supplied; see original source.

  OSWorld 2.0 site: benchmark of long-horizon real-world computer tasks for evaluating computer-use agents.

  Not applicable (entity) · unknown · unknown resource · en

• Polska AI przegrała z gigantami. Bielik i PLLuM wypadły słabo w testach (Bankier.pl)

  Creator not supplied; see original source.

  Bankier.pl article reporting a Polish LLM test of 20 tasks in which the Polish models Bielik and PLLuM were compared to global models.

  Not applicable (entity) · unknown · unknown resource · pl

• P3D-Bench

  Creator not supplied; see original source.

  P3D-Bench GitHub repo: benchmark of MLLMs on parametric 3D generation and structural reasoning.

  Not applicable (entity) · unknown · unknown resource · en

• PDEBench

  Creator not supplied; see original source.

  PDEBench GitHub repo: PDE datasets and baselines for scientific machine learning.

  Not applicable (entity) · unknown · unknown resource · en

• Phare

  Creator not supplied; see original source.

  Giskard Phare site: multilingual benchmark of LLM safety and security, including hallucination, bias and harmful content.

  Not applicable (entity) · unknown · unknown resource · mul

• PhysBench

  Creator not supplied; see original source.

  PhysBench project page: benchmark of physical-world questions testing vision-language models on objects, motion and interactions.

  Not applicable (entity) · unknown · unknown resource · en

• PhysioNet Challenges

  Creator not supplied; see original source.

  PhysioNet Challenges page: annual open challenges on clinical physiological-signal analysis with scoring methods and data.

  Not applicable (entity) · unknown · unknown resource · en

• Polish LLM benchmarks (CodeSOTA)

  Creator not supplied; see original source.

  CodeSOTA page aggregating Polish LLM benchmark results for models such as Bielik, PLLuM, GPT, Claude and Gemini.

  Not applicable (entity) · unknown · unknown resource · pl

• POPE

  Creator not supplied; see original source.

  POPE GitHub repo (EMNLP 2023): polling-based evaluation of object hallucination in vision-language models.

  Not applicable (entity) · unknown · unknown resource · en

• PORTULAN Portuguese LLM Leaderboard

  Creator not supplied; see original source.

  PORTULAN Portuguese LLM Leaderboard Hugging Face Space: live ranking of models on Portuguese tasks.

  Not applicable (entity) · unknown · unknown resource · pt

• Procgen Benchmark

  Creator not supplied; see original source.

  OpenAI Procgen GitHub repo: procedurally generated game environments for RL generalisation.

  Not applicable (entity) · unknown · unknown resource · en

• ProgramBench

  Creator not supplied; see original source.

  ProgramBench site: benchmark testing whether language models can rebuild complete programs from scratch.

  Not applicable (entity) · unknown · unknown resource · en

• ProteinGym

  Creator not supplied; see original source.

  ProteinGym: benchmark and leaderboard for predicting the effects of protein mutations.

  Not applicable (entity) · unknown · unknown resource · en

• Pydantic AI

  Creator not supplied; see original source.

  Pydantic AI GitHub repo: type-safe Python agent framework with evals and durable execution.

  Not applicable (entity) · unknown · unknown resource · en

• Rapidata Image-to-3D

  Creator not supplied; see original source.

  benchmark.ai Image-to-3D leaderboard from Rapidata ranking 3D generation models by pairwise human preference votes.

  Not applicable (entity) · unknown · unknown resource · en

• RewardBench

  Creator not supplied; see original source.

  RewardBench GitHub repo from Ai2: tool and dataset for evaluating reward models across preference categories.

  Not applicable (entity) · unknown · unknown resource · en

• RLBench

  Creator not supplied; see original source.

  RLBench GitHub repo: large-scale benchmark and learning environment for robot manipulation.

  Not applicable (entity) · unknown · unknown resource · en

• RoboArena

  Creator not supplied; see original source.

  RoboArena: real-world evaluation of generalist robot policies through blind pairwise trials on DROID robots.

  Not applicable (entity) · unknown · unknown resource · en

• RoboCasa

  Creator not supplied; see original source.

  RoboCasa site: large-scale kitchen simulation framework for training and evaluating household robot policies.

  Not applicable (entity) · unknown · unknown resource · en

• Robot learning leaderboard (RoboArena)

  Creator not supplied; see original source.

  Robot-learning.ml leaderboard site for evaluating robot learning policies.

  Not applicable (entity) · unknown · unknown resource · und

• RoboTwin

  Creator not supplied; see original source.

  RoboTwin 2.0 project page: dual-arm robot manipulation benchmark and simulated data generator.

  Not applicable (entity) · unknown · unknown resource · en

• RULER

  Creator not supplied; see original source.

  NVIDIA RULER GitHub repo: synthetic benchmark measuring the real usable context length of long-context models.

  Not applicable (entity) · unknown · unknown resource · en

• Russian AI Alliance (Альянс в сфере ИИ)

  Creator not supplied; see original source.

  Russian AI Alliance site, the industry association of Sber, Yandex, MTS and others that runs the MERA benchmark.

  Not applicable (entity) · unknown · unknown resource · ru

• Russian SuperGLUE

  Creator not supplied; see original source.

  Russian SuperGLUE GitHub repo: Russian general language understanding benchmark with tasks and baselines.

  Not applicable (entity) · unknown · unknown resource · ru

• S-Eval

  Creator not supplied; see original source.

  S-Eval GitHub repo: about 220,000 safety prompts, including adversarial variants, for automated LLM safety evaluation.

  Not applicable (entity) · unknown · unknown resource · zh|en

• S-Eval leaderboard (Alibaba)

  Creator not supplied; see original source.

  Alibaba S-Eval leaderboard in Chinese: safety evaluation rankings of large language models.

  Not applicable (entity) · unknown · unknown resource · zh

• SafetyBench

  Creator not supplied; see original source.

  SafetyBench GitHub repo from Tsinghua: multiple-choice safety benchmark for LLMs across seven categories, in Chinese and English.

  Not applicable (entity) · unknown · unknown resource · zh

Plain-text Markdown copy