Benchmarks And Leaderboards: 512 curated resources

Curated material on benchmarks and leaderboards within evaluation.

512 resources · page 9 of 16

• Nejumi / Japanese LLM Leaderboard

  Creator not supplied; see original source.

  W&B llm-leaderboard GitHub repo: code and methodology for the Nejumi Leaderboard, which evaluates LLMs on Japanese tasks.

  Not applicable (entity) · unknown · unknown resource · ja

• Nejumi LLM Leaderboard 4

  Creator not supplied; see original source.

  Nejumi LLM Leaderboard 4 report (Weights & Biases Japan, now on CoreWeave Forge): evaluation of LLMs on Japanese tasks.

  Not applicable (entity) · unknown · unknown resource · ja

• NIST CAISI

  Creator not supplied; see original source.

  NIST CAISI: US government center evaluating AI models and developing measurement standards.

  Not applicable (entity) · unknown · unknown resource · en

• NoLiMa

  Creator not supplied; see original source.

  NoLiMa paper: a long-context benchmark where the needle must be found without literal word overlap with the question.

  Not applicable (entity) · unknown · unknown resource · en

• nuScenes

  Creator not supplied; see original source.

  nuScenes site: autonomous-driving dataset with camera, lidar and radar data plus detection, tracking and prediction benchmarks.

  Not applicable (entity) · unknown · unknown resource · en

• nuScenes leaderboards

  Creator not supplied; see original source.

  nuScenes leaderboards: autonomous-driving benchmarks for detection, tracking, prediction and planning.

  Not applicable (entity) · unknown · unknown resource · en

• OALL Hugging Face organization

  Creator not supplied; see original source.

  Open Arabic LLM Leaderboard (OALL) Hugging Face organisation: evaluation of Arabic language models.

  Not applicable (entity) · unknown · unknown resource · ar

• ObjectNet

  Creator not supplied; see original source.

  ObjectNet site: real-world object recognition test set with controlled backgrounds, rotations and viewpoints.

  Not applicable (entity) · unknown · unknown resource · en

• OCRBench

  Creator not supplied; see original source.

  MultimodalOCR GitHub repo: OCRBench evaluation of text recognition in large multimodal models.

  Not applicable (entity) · unknown · unknown resource · en

• OCRBench v2 leaderboard

  Creator not supplied; see original source.

  OCRBench v2 leaderboard: scores of multimodal models on visual text localisation and reasoning tasks.

  Not applicable (entity) · unknown · unknown resource · en

• olmOCR-Bench

  Creator not supplied; see original source.

  olmOCR GitHub repo from Ai2: PDF linearisation toolkit that includes olmOCR-Bench, unit-test cases for OCR quality.

  Not applicable (entity) · unknown · unknown resource · en

• OmniDocBench

  Creator not supplied; see original source.

  OmniDocBench GitHub repo: benchmark for document parsing covering layout, tables, formulas and OCR across document types.

  Not applicable (entity) · unknown · unknown resource · zh

• Open ASR Leaderboard

  Creator not supplied; see original source.

  Hugging Face Open ASR Leaderboard: word error rate and speed of speech recognition models on public datasets.

  Not applicable (entity) · unknown · unknown resource · en|mul

• Open Catalyst Project

  Creator not supplied; see original source.

  Open Catalyst Project site: atomistic datasets, models and challenges for AI-based catalyst discovery.

  Not applicable (entity) · unknown · unknown resource · en

• Open Graph Benchmark

  Creator not supplied; see original source.

  Open Graph Benchmark site from Stanford: graph datasets, data loaders and evaluators for graph machine learning.

  Not applicable (entity) · unknown · unknown resource · en

• Open ITA LLM Leaderboard

  Creator not supplied; see original source.

  Open Ita LLM Leaderboard Hugging Face Space: filterable results of models on Italian benchmarks.

  Not applicable (entity) · unknown · unknown resource · it

• Open Ko-LLM Leaderboard

  Creator not supplied; see original source.

  Upstage Open Ko-LLM Leaderboard Space on Hugging Face for comparing language models on Korean benchmarks.

  Not applicable (entity) · unknown · unknown resource · ko

• Open LLM French Leaderboard

  Creator not supplied; see original source.

  Open LLM French Leaderboard Hugging Face Space: filterable results of models on French benchmarks.

  Not applicable (entity) · unknown · unknown resource · fr

• Open LLM Leaderboard

  Creator not supplied; see original source.

  Open LLM Leaderboard Space (archived): historical scores of open language models filterable by size.

  Not applicable (entity) · unknown · unknown resource · en

• Open LLM Turkish Leaderboard v0.2

  Creator not supplied; see original source.

  Open LLM Turkish Leaderboard v0.2 Hugging Face Space: filterable results of models on Turkish benchmarks.

  Not applicable (entity) · unknown · unknown resource · tr

• Open Persian LLM Leaderboard org readme

  Creator not supplied; see original source.

  Open Persian LLM Leaderboard organisation readme describing the Amirkabir University-led initiative.

  Not applicable (entity) · unknown · unknown resource · fa

• Open Persian LLM Leaderboard (PartAI)

  Creator not supplied; see original source.

  PartAI Open Persian LLM Leaderboard Hugging Face Space: comparison of models on Persian benchmarks.

  Not applicable (entity) · unknown · unknown resource · fa

• Open PL LLM Leaderboard (SpeakLeash)

  Creator not supplied; see original source.

  SpeakLeash Open PL LLM Leaderboard Hugging Face Space: detailed results of models on Polish tasks.

  Not applicable (entity) · unknown · unknown resource · pl

• Open Portuguese LLM Leaderboard

  Creator not supplied; see original source.

  Open Portuguese LLM Leaderboard Hugging Face Space (archived): past results of models on Portuguese benchmarks.

  Not applicable (entity) · unknown · unknown resource · pt

• Open VLM Leaderboard (OpenCompass)

  Creator not supplied; see original source.

  OpenCompass Open VLM Leaderboard Space: vision-language model results aggregated from VLMEvalKit benchmarks.

  Not applicable (entity) · unknown · unknown resource · en|zh

• OpenAI Evals

  Creator not supplied; see original source.

  OpenAI Evals GitHub repo: framework and registry for evaluating LLMs and LLM systems.

  Not applicable (entity) · unknown · unknown resource · en

• frontier-evals

  Creator not supplied; see original source.

  OpenAI frontier-evals GitHub repo: code for evaluations such as PaperBench and SWE-Lancer.

  Not applicable (entity) · unknown · unknown resource · en

• OpenAI MRCR

  Creator not supplied; see original source.

  OpenAI MRCR dataset: multi-round coreference resolution task for testing long-context retrieval, as reported in model cards.

  Not applicable (entity) · unknown · unknown resource · en

• OpenCompass 司南

  Creator not supplied; see original source.

  OpenCompass leaderboard in Chinese: scores and rankings of language and multimodal models across ability dimensions.

  Not applicable (entity) · unknown · unknown resource · en

• OpenCompass

  Creator not supplied; see original source.

  OpenCompass GitHub repo: LLM evaluation platform with configs for many datasets, inference, judging and leaderboards.

  Not applicable (entity) · unknown · unknown resource · zh

• OpenCompass CompassRank: Multimodal Leaderboard

  Creator not supplied; see original source.

  OpenCompass CompassRank multimodal leaderboard: scores of vision-language models across benchmarks.

  Not applicable (entity) · unknown · unknown resource · zh

• OpenCompass HF leaderboard

  Creator not supplied; see original source.

  OpenCompass LLM Leaderboard Hugging Face Space: OpenCompass evaluation results for language models.

  Not applicable (entity) · unknown · unknown resource · zh|en

Plain-text Markdown copy