Evaluation

Measure model capabilities and application behavior with explicit criteria.

• Benchmarks And Leaderboards (512)
• Eval Harnesses (3)
• Application Evals (2)
• Llm As Judge (1)
• Agent Evals (1)

519 resources · page 4 of 17

• CyberGym

  Creator not supplied; see original source.

  Berkeley CyberGym GitHub repo: evaluation of AI agents on real software vulnerabilities.

  Not applicable (entity) · unknown · unknown resource · en

• CyberSecEval 4

  Creator not supplied; see original source.

  Meta CyberSecEval 4 documentation: open benchmarks for cybersecurity risks and capabilities of LLMs.

  Not applicable (entity) · unknown · unknown resource · en

• Cyberspace Administration of China (网信办)

  Creator not supplied; see original source.

  Cyberspace Administration of China official site: regulator publishing China's internet and AI rules.

  Not applicable (entity) · unknown · unknown resource · zh

• D4RL

  Creator not supplied; see original source.

  Farama D4RL GitHub repo: reference environments and datasets for offline reinforcement learning.

  Not applicable (entity) · unknown · unknown resource · en

• data.gouv.fr "IA culturelles"

  Creator not supplied; see original source.

  data.gouv.fr topic page for the French Ministry of Culture's IA culturelles challenge, linking related open datasets.

  Not applicable (entity) · unknown · unknown resource · fr

• SuperCLUE benchmark results (DataLearner)

  Creator not supplied; see original source.

  DataLearner page mirroring SuperCLUE Chinese LLM leaderboard scores across releases.

  Not applicable (entity) · unknown · unknown resource · zh

• DeepEval

  Creator not supplied; see original source.

  Confident AI DeepEval GitHub repo: framework for unit-testing and evaluating LLM outputs with metrics.

  Not applicable (entity) · unknown · unknown resource · en

• Lapa LLM developers have created a leaderboard comparing AI models (dev.ua)

  Creator not supplied; see original source.

  dev.ua report on a leaderboard ranking LLMs on Ukrainian language skill, by the Lapa LLM developers.

  Not applicable (entity) · unknown · unknown resource · en|uk

• DocVQA

  Creator not supplied; see original source.

  DocVQA site: document visual question answering challenges and datasets built from real document images.

  Not applicable (entity) · unknown · unknown resource · en

• DPG-Bench (TencentQQGYLab/ELLA)

  Creator not supplied; see original source.

  TencentQQGYLab ELLA GitHub repo: LLM-equipped diffusion models and the DPG-Bench dense prompt benchmark.

  Not applicable (entity) · unknown · unknown resource · en

• DREAM Challenges

  Creator not supplied; see original source.

  Sage Bionetworks page for DREAM Challenges: community biomedical challenges with held-out data and evaluation protocols.

  Not applicable (entity) · unknown · unknown resource · en

• DreamBench++

  Creator not supplied; see original source.

  DreamBench++ project page: a human-aligned benchmark for personalised, subject-driven image generation.

  Not applicable (entity) · unknown · unknown resource · en

• EditBench

  Creator not supplied; see original source.

  Google Research page for Imagen Editor and EditBench, a benchmark for text-guided image inpainting with human evaluation.

  Not applicable (entity) · unknown · unknown resource · en

• EgoSchema

  Creator not supplied; see original source.

  EgoSchema site: long-form egocentric video question answering for testing long-range video understanding.

  Not applicable (entity) · unknown · unknown resource · en

• EleutherAI LM Evaluation Harness

  Creator not supplied; see original source.

  EleutherAI lm-evaluation-harness GitHub repo: framework for few-shot evaluation of language models on many benchmarks.

  Not applicable (entity) · unknown · unknown resource · en

• encodechka

  Creator not supplied; see original source.

  encodechka GitHub repo: benchmark of Russian sentence encoders across tasks, with a results table.

  Not applicable (entity) · unknown · unknown resource · en

• Epoch AI Benchmarking Hub

  Creator not supplied; see original source.

  Epoch AI Benchmarking Hub: results of AI models on challenging benchmarks, run by Epoch and others, with methods.

  Not applicable (entity) · unknown · unknown resource · en

• HealthBench Professional: Benchmark review (Epoch AI)

  Creator not supplied; see original source.

  Epoch AI review of HealthBench Professional, auditing sampled tasks for errors and assessing the benchmark's reliability.

  Not applicable (entity) · unknown · unknown resource · en

• EU AI Office

  Creator not supplied; see original source.

  European Commission page on the EU AI Office, which supervises general-purpose AI models under the EU AI Act.

  Not applicable (entity) · unknown · unknown resource · en

• EuroEval/EuroEval

  Creator not supplied; see original source.

  EuroEval GitHub repo: a benchmarking framework for language models across European languages, with reproducible datasets.

  Not applicable (entity) · unknown · unknown resource · en

• EuroEval

  Creator not supplied; see original source.

  EuroEval: benchmark and leaderboards for language models across 30+ European languages.

  Not applicable (entity) · unknown · unknown resource · mul

• EuroEval HF org

  Creator not supplied; see original source.

  EuroEval Hugging Face organisation hosting European language model evaluation datasets and results.

  Not applicable (entity) · unknown · unknown resource · mul

• European LLM Leaderboard (Eurolingua)

  Creator not supplied; see original source.

  Eurolingua European LLM Leaderboard Hugging Face Space: model results on benchmarks in European languages.

  Not applicable (entity) · unknown · unknown resource · eu|mul

• EvalCrafter

  Creator not supplied; see original source.

  EvalCrafter project page: a framework for evaluating text-to-video models with automatic metrics and human ratings.

  Not applicable (entity) · unknown · unknown resource · en

• EvalPlus

  Creator not supplied; see original source.

  EvalPlus site: HumanEval+ and MBPP+ with extended test cases for more accurate evaluation of code LLMs.

  Not applicable (entity) · unknown · unknown resource · en

• EvalScope

  Creator not supplied; see original source.

  ModelScope EvalScope GitHub repo: framework for evaluating model quality and inference performance.

  Not applicable (entity) · unknown · unknown resource · zh

• FAR.AI AI Security Leaderboard

  Creator not supplied; see original source.

  FAR.AI AI Security Leaderboard comparing model safeguards against a taxonomy of attacks with a published methodology.

  Not applicable (entity) · unknown · unknown resource · en

• FinanceBench

  Creator not supplied; see original source.

  FinanceBench GitHub repo from Patronus AI: open-book financial question answering over public company filings.

  Not applicable (entity) · unknown · unknown resource · en

• FinanceBench (Patronus AI)

  Creator not supplied; see original source.

  Patronus AI announcement of FinanceBench, question answering over public financial filings to measure LLM failures.

  Not applicable (entity) · unknown · unknown resource · en

• FlagEval

  Creator not supplied; see original source.

  BAAI FlagEval GitHub repo: large model evaluation system covering many models and tasks.

  Not applicable (entity) · unknown · unknown resource · zh

• FlagEval (BAAI)

  Creator not supplied; see original source.

  FlagEval: Beijing Academy of Artificial Intelligence platform for evaluating large models.

  Not applicable (entity) · unknown · unknown resource · en

• Flames

  Creator not supplied; see original source.

  Flames GitHub repo: an adversarial Chinese benchmark of prompts on fairness, safety, morality, legality and data protection.

  Not applicable (entity) · unknown · unknown resource · zh

Plain-text Markdown copy